2020-11-20T11:58:09.637 INFO:root:teuthology version: 1.0.0-357fd4a 2020-11-20T11:58:09.639 DEBUG:teuthology.run:Teuthology command: teuthology --lock --block --owner scheduled_pdonnell@teuthology --verbose --name pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi --archive /home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640336 --description fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} -- /tmp/teuthology-worker.4nttuzww.tmp 2020-11-20T11:58:09.682 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T11:58:09.757 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640336 branch: wip-pdonnell-testing-20201120.053228 description: fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client} email: pdonnell@redhat.com first_in_suite: false job_id: '5640336' kernel: client: branch: testing kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: wip-pdonnell-testing-20201120.053228 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 15m rados osd op timeout: 15m global: ms die on skipped message: false mds: debug mds: 20 debug ms: 1 mds bal frag: true mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20201120.053228 sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb owner: scheduled_pdonnell@teuthology priority: 95 repo: git://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb sleep_before_teardown: 0 suite: fs suite_branch: wip-pdonnell-testing-20201120.053228 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201120.053228/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell rpm: - python3-cephfs extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volume_client teuthology_branch: master timestamp: 2020-11-20_08:39:42 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.21172 2020-11-20T11:58:09.758 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201120.053228/qa; will attempt to use it 2020-11-20T11:58:09.759 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201120.053228/qa/tasks 2020-11-20T11:58:09.760 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-11-20T11:58:09.762 INFO:teuthology.task.internal:Checking packages... 2020-11-20T11:58:09.787 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'basic' and ceph hash 'b86e691bf191566922a8f08e2a5abf64fde8cfcb' 2020-11-20T11:58:09.787 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2020-11-20T11:58:09.788 INFO:teuthology.packaging:ref: None 2020-11-20T11:58:09.788 INFO:teuthology.packaging:tag: None 2020-11-20T11:58:09.789 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20201120.053228 2020-11-20T11:58:09.789 INFO:teuthology.packaging:sha1: b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T11:58:09.790 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=wip-pdonnell-testing-20201120.053228 2020-11-20T11:58:09.963 INFO:teuthology.task.internal:Found packages for ceph version 16.0.0-7397-gb86e691b-1bionic 2020-11-20T11:58:09.963 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-11-20T11:58:09.973 INFO:teuthology.task.internal:no buildpackages task found 2020-11-20T11:58:09.973 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-11-20T11:58:09.987 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-11-20T11:58:09.988 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T11:58:10.041 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-11-20T11:58:20.058 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-11-20T11:58:30.086 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T11:58:40.101 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T11:58:50.117 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T11:59:00.134 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T11:59:10.170 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T11:59:20.190 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T11:59:30.209 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T11:59:40.228 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T11:59:50.245 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-11-20T12:00:00.295 DEBUG:teuthology.lock.ops:lock_many request: {'locked_by': 'scheduled_pdonnell@teuthology', 'count': 2, 'machine_type': 'smithi', 'description': '/home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640336'} 2020-11-20T12:00:00.341 DEBUG:teuthology.lock.ops:locked ubuntu@smithi116.front.sepia.ceph.com, ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:00:00.444 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f32cd9e81e0>, signals=[15]) 2020-11-20T12:00:00.659 INFO:teuthology.provision.fog.smithi173:Scheduling deploy of ubuntu 18.04 2020-11-20T12:00:00.664 INFO:teuthology.provision.fog.smithi116:Scheduling deploy of ubuntu 18.04 2020-11-20T12:00:01.041 INFO:teuthology.orchestra.console:Power off smithi173 2020-11-20T12:00:01.042 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-20T12:00:01.058 INFO:teuthology.orchestra.console:Power off smithi116 2020-11-20T12:00:01.059 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2020-11-20T12:00:01.072 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:01.083 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:05.104 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:05.126 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:09.229 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:09.242 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:09.452 INFO:teuthology.orchestra.console:Power off for smithi116 completed 2020-11-20T12:00:09.553 INFO:teuthology.orchestra.console:Power on smithi116 2020-11-20T12:00:09.554 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-20T12:00:09.574 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:13.341 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:13.552 INFO:teuthology.orchestra.console:Power off for smithi173 completed 2020-11-20T12:00:13.596 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:13.653 INFO:teuthology.orchestra.console:Power on smithi173 2020-11-20T12:00:13.654 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2020-11-20T12:00:13.675 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:17.697 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:17.709 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi116.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:18.028 INFO:teuthology.orchestra.console:Power on for smithi116 completed 2020-11-20T12:00:18.129 INFO:teuthology.provision.fog.smithi116:Waiting for deploy to finish 2020-11-20T12:00:21.809 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi173.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2020-11-20T12:00:22.177 INFO:teuthology.orchestra.console:Power on for smithi173 completed 2020-11-20T12:00:22.278 INFO:teuthology.provision.fog.smithi173:Waiting for deploy to finish 2020-11-20T12:03:04.376 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:03:14.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:03:23.209 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:03:23.555 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:03:32.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:03:32.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:03:41.353 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:03:41.670 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:03:50.410 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:03:50.726 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:03:59.466 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:03:59.782 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:04:08.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:04:17.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:04:23.789 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:04:32.874 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:04:41.930 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:04:50.985 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:04:52.966 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2020-11-20T12:04:58.973 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:04:59.148 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-20T12:04:59.149 INFO:teuthology.orchestra.run.smithi116:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-20T12:04:59.528 INFO:teuthology.provision.fog.smithi116:Node is ready 2020-11-20T12:04:59.528 INFO:teuthology.orchestra.run.smithi116:> hostname 2020-11-20T12:04:59.577 INFO:teuthology.orchestra.run.smithi116.stdout:smithi116.front.sepia.ceph.com 2020-11-20T12:04:59.577 INFO:teuthology.orchestra.run.smithi116:> grep smithi116.front.sepia.ceph.com /etc/hosts 2020-11-20T12:04:59.625 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.15.116 smithi116.front.sepia.ceph.com smithi116 2020-11-20T12:04:59.626 INFO:teuthology.orchestra.run.smithi116:> sudo hostname smithi116 2020-11-20T12:04:59.715 INFO:teuthology.orchestra.run.smithi116:> sudo sed -i -e 's/smithi116.front.sepia.ceph.com/smithi116/g' /etc/hosts 2020-11-20T12:04:59.729 INFO:teuthology.orchestra.run.smithi116:> sudo sed -i -e 's/172.21.15.116/172.21.15.116/g' /etc/hosts 2020-11-20T12:04:59.786 INFO:teuthology.orchestra.run.smithi116:> sudo sed -i -e 's/smithi116.front.sepia.ceph.com/smithi116/g' /etc/hostname 2020-11-20T12:04:59.842 INFO:teuthology.orchestra.run.smithi116:> sudo hostname smithi116 2020-11-20T12:04:59.895 INFO:teuthology.provision.fog.smithi116:Deploy complete! 2020-11-20T12:04:59.896 DEBUG:teuthology.parallel:result is None 2020-11-20T12:05:23.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:05:24.013 INFO:teuthology.orchestra.run:Running command with timeout 600 2020-11-20T12:05:24.014 INFO:teuthology.orchestra.run.smithi173:> while [ ! -e '/.cephlab_net_configured' ]; do sleep 5; done 2020-11-20T12:05:24.443 INFO:teuthology.provision.fog.smithi173:Node is ready 2020-11-20T12:05:24.445 INFO:teuthology.orchestra.run.smithi173:> hostname 2020-11-20T12:05:24.523 INFO:teuthology.orchestra.run.smithi173.stdout:smithi173.front.sepia.ceph.com 2020-11-20T12:05:24.524 INFO:teuthology.orchestra.run.smithi173:> grep smithi173.front.sepia.ceph.com /etc/hosts 2020-11-20T12:05:24.586 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.15.173 smithi173.front.sepia.ceph.com smithi173 2020-11-20T12:05:24.587 INFO:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2020-11-20T12:05:24.670 INFO:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hosts 2020-11-20T12:05:24.740 INFO:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/172.21.15.173/172.21.15.173/g' /etc/hosts 2020-11-20T12:05:24.807 INFO:teuthology.orchestra.run.smithi173:> sudo sed -i -e 's/smithi173.front.sepia.ceph.com/smithi173/g' /etc/hostname 2020-11-20T12:05:24.877 INFO:teuthology.orchestra.run.smithi173:> sudo hostname smithi173 2020-11-20T12:05:24.949 INFO:teuthology.provision.fog.smithi173:Deploy complete! 2020-11-20T12:05:24.950 DEBUG:teuthology.parallel:result is None 2020-11-20T12:05:25.231 INFO:teuthology.lock.ops:Checking smithi116.front.sepia.ceph.com 2020-11-20T12:05:25.233 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-20T12:05:25.259 INFO:teuthology.lock.ops:Checking smithi173.front.sepia.ceph.com 2020-11-20T12:05:25.260 INFO:teuthology.lock.ops:New key found. Updating... 2020-11-20T12:05:25.285 INFO:teuthology.lock.ops:Updating [smithi116.front.sepia.ceph.com]: set os type and version on server 2020-11-20T12:05:25.285 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T12:05:25.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:05:25.439 INFO:teuthology.orchestra.run.smithi116:> true 2020-11-20T12:05:25.688 INFO:teuthology.orchestra.run.smithi116:> uname -m 2020-11-20T12:05:25.740 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2020-11-20T12:05:25.741 INFO:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2020-11-20T12:05:25.789 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="Ubuntu" 2020-11-20T12:05:25.789 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-20T12:05:25.789 INFO:teuthology.orchestra.run.smithi116.stdout:ID=ubuntu 2020-11-20T12:05:25.789 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE=debian 2020-11-20T12:05:25.790 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-20T12:05:25.790 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="18.04" 2020-11-20T12:05:25.790 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-20T12:05:25.790 INFO:teuthology.orchestra.run.smithi116.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-20T12:05:25.791 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-20T12:05:25.791 INFO:teuthology.orchestra.run.smithi116.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-20T12:05:25.791 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_CODENAME=bionic 2020-11-20T12:05:25.791 INFO:teuthology.orchestra.run.smithi116.stdout:UBUNTU_CODENAME=bionic 2020-11-20T12:05:25.792 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2020-11-20T12:05:25.810 INFO:teuthology.lock.ops:Updating [smithi173.front.sepia.ceph.com]: set os type and version on server 2020-11-20T12:05:25.811 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T12:05:25.812 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:05:25.939 INFO:teuthology.orchestra.run.smithi173:> true 2020-11-20T12:05:26.182 INFO:teuthology.orchestra.run.smithi173:> uname -m 2020-11-20T12:05:26.230 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2020-11-20T12:05:26.231 INFO:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2020-11-20T12:05:26.278 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="Ubuntu" 2020-11-20T12:05:26.278 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-20T12:05:26.278 INFO:teuthology.orchestra.run.smithi173.stdout:ID=ubuntu 2020-11-20T12:05:26.279 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE=debian 2020-11-20T12:05:26.279 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-20T12:05:26.279 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="18.04" 2020-11-20T12:05:26.279 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-20T12:05:26.279 INFO:teuthology.orchestra.run.smithi173.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-20T12:05:26.280 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-20T12:05:26.280 INFO:teuthology.orchestra.run.smithi173.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-20T12:05:26.280 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_CODENAME=bionic 2020-11-20T12:05:26.280 INFO:teuthology.orchestra.run.smithi173.stdout:UBUNTU_CODENAME=bionic 2020-11-20T12:05:26.281 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2020-11-20T12:05:26.299 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2020-11-20T12:05:26.334 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi116.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSQ6rwLDP6wHpBzj/BMCabw7CJZ5RrVUWL6TJg6UPwAIYJiyVyCYAAhQcdWkhAmr1NA9onXfbiMEAfgCTWdmNXvm1krr2mxps5atb0Ysoama3Au3nD06o+XNn2uOpbk4IIL+c9W7Rb4uithZkr3oVws9oDcOLokt+XznhRAvi0ql8nzgiVeakbObZyiHqdzQXU8PLq4A9i80rCeKOGAhpekkg3DcRRZjeKRgPkvOIIFLSyK3KdxxWnVd++kGVjZd3wlI0KmvGhTNES7sm7z115aQOgS0IxVivad0unAoDTYngOsWyAEioT2Mez7A4uVkx90vHm8z5Dwwvn22Dxmnzt smithi173.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRY9tyR9uY3qAegbul+z/sE02z16putaUgDL8Gea4ZwyArv4XkNs/CHTDbgCYxVoa7OhzxZwCAC47Gkrt6UOjTfjvznSBdSDqEhSwfAIWFaLkZ2K7Wx/TM2BZz6otRv5nYA9hPt0PeV8IDWk06mZqOIPPGr+sxtU+MW8jiByrvO/hQD0oknWXIxs+kdP8avuqxzg/KMyzhYC9D7sDDhOX0grZ43pbDy9tLC3gulOpvXQ3RvQwVsDdW1IxqEQxRcQidY8sLj6X6bffoP1Qb/1fRemuhGLMqBYxTYmlCIWQETj4XY/jhH6bfQzJ4XmveY62sj05hO/ZB/gy9vuJfqWSf 2020-11-20T12:05:26.335 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-11-20T12:05:26.388 INFO:teuthology.run_tasks:Running task internal.save_config... 2020-11-20T12:05:26.396 INFO:teuthology.task.internal:Saving configuration 2020-11-20T12:05:26.419 INFO:teuthology.run_tasks:Running task internal.check_lock... 2020-11-20T12:05:26.430 INFO:teuthology.task.internal.check_lock:Checking locks... 2020-11-20T12:05:26.448 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-20 12:00:00.326458', 'description': '/home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640336', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIN0IERFWEVdKpeDK6MO8v7zuG93ROCMpJhcGkVDaKJ8X', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi116.front.sepia.ceph.com'} 2020-11-20T12:05:26.463 DEBUG:teuthology.task.internal.check_lock:machine status is {'is_vm': False, 'locked': True, 'locked_since': '2020-11-20 12:00:00.326479', 'description': '/home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640336', 'locked_by': 'scheduled_pdonnell@teuthology', 'up': True, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAdxP3TPIz3Q55mKBrnrygmVyrMEmC5NrxjqkOIahGWH', 'os_version': '18.04', 'machine_type': 'smithi', 'vm_host': None, 'os_type': 'ubuntu', 'arch': 'x86_64', 'mac_address': None, 'name': 'smithi173.front.sepia.ceph.com'} 2020-11-20T12:05:26.464 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2020-11-20T12:05:26.485 INFO:teuthology.task.internal:roles: ubuntu@smithi116.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2020-11-20T12:05:26.486 INFO:teuthology.task.internal:roles: ubuntu@smithi173.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2020-11-20T12:05:26.486 INFO:teuthology.run_tasks:Running task console_log... 2020-11-20T12:05:26.552 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f32cda759d8>, signals=[15]) 2020-11-20T12:05:26.553 INFO:teuthology.run_tasks:Running task internal.connect... 2020-11-20T12:05:26.565 INFO:teuthology.task.internal:Opening connections... 2020-11-20T12:05:26.566 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:05:26.569 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:05:26.643 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:05:26.645 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:05:26.724 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2020-11-20T12:05:26.734 INFO:teuthology.orchestra.run.smithi116:> uname -m 2020-11-20T12:05:26.845 INFO:teuthology.orchestra.run.smithi116.stdout:x86_64 2020-11-20T12:05:26.846 INFO:teuthology.orchestra.run.smithi116:> cat /etc/os-release 2020-11-20T12:05:26.896 INFO:teuthology.orchestra.run.smithi116.stdout:NAME="Ubuntu" 2020-11-20T12:05:26.897 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-20T12:05:26.897 INFO:teuthology.orchestra.run.smithi116.stdout:ID=ubuntu 2020-11-20T12:05:26.897 INFO:teuthology.orchestra.run.smithi116.stdout:ID_LIKE=debian 2020-11-20T12:05:26.897 INFO:teuthology.orchestra.run.smithi116.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-20T12:05:26.898 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_ID="18.04" 2020-11-20T12:05:26.898 INFO:teuthology.orchestra.run.smithi116.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-20T12:05:26.898 INFO:teuthology.orchestra.run.smithi116.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-20T12:05:26.898 INFO:teuthology.orchestra.run.smithi116.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-20T12:05:26.898 INFO:teuthology.orchestra.run.smithi116.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-20T12:05:26.899 INFO:teuthology.orchestra.run.smithi116.stdout:VERSION_CODENAME=bionic 2020-11-20T12:05:26.899 INFO:teuthology.orchestra.run.smithi116.stdout:UBUNTU_CODENAME=bionic 2020-11-20T12:05:26.899 INFO:teuthology.lock.ops:Updating smithi116.front.sepia.ceph.com on lock server 2020-11-20T12:05:26.916 INFO:teuthology.orchestra.run.smithi173:> uname -m 2020-11-20T12:05:26.929 INFO:teuthology.orchestra.run.smithi173.stdout:x86_64 2020-11-20T12:05:26.929 INFO:teuthology.orchestra.run.smithi173:> cat /etc/os-release 2020-11-20T12:05:26.978 INFO:teuthology.orchestra.run.smithi173.stdout:NAME="Ubuntu" 2020-11-20T12:05:26.978 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION="18.04.5 LTS (Bionic Beaver)" 2020-11-20T12:05:26.978 INFO:teuthology.orchestra.run.smithi173.stdout:ID=ubuntu 2020-11-20T12:05:26.979 INFO:teuthology.orchestra.run.smithi173.stdout:ID_LIKE=debian 2020-11-20T12:05:26.979 INFO:teuthology.orchestra.run.smithi173.stdout:PRETTY_NAME="Ubuntu 18.04.5 LTS" 2020-11-20T12:05:26.979 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_ID="18.04" 2020-11-20T12:05:26.979 INFO:teuthology.orchestra.run.smithi173.stdout:HOME_URL="https://www.ubuntu.com/" 2020-11-20T12:05:26.979 INFO:teuthology.orchestra.run.smithi173.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2020-11-20T12:05:26.980 INFO:teuthology.orchestra.run.smithi173.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2020-11-20T12:05:26.980 INFO:teuthology.orchestra.run.smithi173.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2020-11-20T12:05:26.980 INFO:teuthology.orchestra.run.smithi173.stdout:VERSION_CODENAME=bionic 2020-11-20T12:05:26.980 INFO:teuthology.orchestra.run.smithi173.stdout:UBUNTU_CODENAME=bionic 2020-11-20T12:05:26.981 INFO:teuthology.lock.ops:Updating smithi173.front.sepia.ceph.com on lock server 2020-11-20T12:05:26.999 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2020-11-20T12:05:27.038 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2020-11-20T12:05:27.049 INFO:teuthology.task.internal:Checking for old test directory... 2020-11-20T12:05:27.050 INFO:teuthology.orchestra.run.smithi116:> test '!' -e /home/ubuntu/cephtest 2020-11-20T12:05:27.052 INFO:teuthology.orchestra.run.smithi173:> test '!' -e /home/ubuntu/cephtest 2020-11-20T12:05:27.058 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2020-11-20T12:05:27.068 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2020-11-20T12:05:27.068 INFO:teuthology.orchestra.run.smithi116:> test -z $(ls -A /var/lib/ceph) 2020-11-20T12:05:27.101 INFO:teuthology.orchestra.run.smithi173:> test -z $(ls -A /var/lib/ceph) 2020-11-20T12:05:27.110 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2020-11-20T12:05:27.191 INFO:teuthology.run_tasks:Running task kernel... 2020-11-20T12:05:27.223 DEBUG:teuthology.task.kernel:normalized config {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2020-11-20T12:05:27.223 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2020-11-20T12:05:27.224 INFO:teuthology.orchestra.run.smithi116:> sudo lsb_release -is 2020-11-20T12:05:27.278 INFO:teuthology.orchestra.run.smithi116.stdout:Ubuntu 2020-11-20T12:05:27.283 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-20T12:05:27.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=testing 2020-11-20T12:05:27.564 DEBUG:teuthology.task.kernel:sha1 for client.0 is ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-20T12:05:27.565 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9"... 2020-11-20T12:05:27.565 INFO:teuthology.orchestra.run.smithi116:> uname -r 2020-11-20T12:05:27.571 INFO:teuthology.orchestra.run.smithi116.stdout:4.15.0-123-generic 2020-11-20T12:05:27.572 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-123-generic vs ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-20T12:05:27.572 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2020-11-20T12:05:27.573 INFO:teuthology.orchestra.run.smithi173:> sudo lsb_release -is 2020-11-20T12:05:27.626 INFO:teuthology.orchestra.run.smithi173.stdout:Ubuntu 2020-11-20T12:05:27.631 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-20T12:05:27.631 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=testing 2020-11-20T12:05:27.929 DEBUG:teuthology.task.kernel:sha1 for client.1 is ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-20T12:05:27.930 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9"... 2020-11-20T12:05:27.930 INFO:teuthology.orchestra.run.smithi173:> uname -r 2020-11-20T12:05:27.936 INFO:teuthology.orchestra.run.smithi173.stdout:4.15.0-123-generic 2020-11-20T12:05:27.937 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-123-generic vs ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-20T12:05:27.937 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2020-11-20T12:05:27.938 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2020-11-20T12:05:27.938 INFO:teuthology.orchestra.run.smithi116:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2020-11-20T12:05:27.943 INFO:teuthology.orchestra.run.smithi116.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2020-11-20T12:05:28.045 INFO:teuthology.orchestra.run.smithi116.stdout:Initialized empty Git repository in /lib/firmware/updates/.git/ 2020-11-20T12:05:28.047 INFO:teuthology.orchestra.run.smithi116:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2020-11-20T12:05:28.121 INFO:teuthology.orchestra.run.smithi116:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2020-11-20T12:05:28.179 INFO:teuthology.orchestra.run.smithi116:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master 2020-11-20T12:05:47.308 INFO:teuthology.orchestra.run.smithi116.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2020-11-20T12:05:47.309 INFO:teuthology.orchestra.run.smithi116.stderr: * [new branch] main -> origin/main 2020-11-20T12:05:47.309 INFO:teuthology.orchestra.run.smithi116.stderr: * [new branch] master -> origin/master 2020-11-20T12:05:47.336 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190312 -> 20190312 2020-11-20T12:05:47.336 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190416 -> 20190416 2020-11-20T12:05:47.336 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190514 -> 20190514 2020-11-20T12:05:47.337 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190618 -> 20190618 2020-11-20T12:05:47.337 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190717 -> 20190717 2020-11-20T12:05:47.337 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190815 -> 20190815 2020-11-20T12:05:47.337 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20190923 -> 20190923 2020-11-20T12:05:47.338 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20191022 -> 20191022 2020-11-20T12:05:47.338 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20191215 -> 20191215 2020-11-20T12:05:47.338 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200122 -> 20200122 2020-11-20T12:05:47.339 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200316 -> 20200316 2020-11-20T12:05:47.339 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200421 -> 20200421 2020-11-20T12:05:47.340 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200519 -> 20200519 2020-11-20T12:05:47.340 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200619 -> 20200619 2020-11-20T12:05:47.340 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200721 -> 20200721 2020-11-20T12:05:47.340 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200817 -> 20200817 2020-11-20T12:05:47.340 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20200918 -> 20200918 2020-11-20T12:05:47.342 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20201022 -> 20201022 2020-11-20T12:05:47.344 INFO:teuthology.orchestra.run.smithi116.stderr: * [new tag] 20201118 -> 20201118 2020-11-20T12:05:49.476 INFO:teuthology.orchestra.run.smithi116.stderr:Checking out files: 53% (1256/2360) Checking out files: 54% (1275/2360) Checking out files: 55% (1298/2360) Checking out files: 56% (1322/2360) Checking out files: 57% (1346/2360) Checking out files: 58% (1369/2360) Checking out files: 59% (1393/2360) Checking out files: 60% (1416/2360) Checking out files: 61% (1440/2360) Checking out files: 62% (1464/2360) Checking out files: 63% (1487/2360) Checking out files: 64% (1511/2360) Checking out files: 65% (1534/2360) Checking out files: 66% (1558/2360) Checking out files: 67% (1582/2360) Checking out files: 68% (1605/2360) Checking out files: 69% (1629/2360) Checking out files: 70% (1652/2360) Checking out files: 71% (1676/2360) Checking out files: 72% (1700/2360) Checking out files: 73% (1723/2360) Checking out files: 74% (1747/2360) Checking out files: 75% (1770/2360) Checking out files: 76% (1794/2360) Checking out files: 77% (1818/2360) Checking out files: 78% (1841/2360) Checking out files: 79% (1865/2360) Checking out files: 80% (1888/2360) Checking out files: 81% (1912/2360) Checking out files: 81% (1913/2360) Checking out files: 82% (1936/2360) Checking out files: 83% (1959/2360) Checking out files: 84% (1983/2360) Checking out files: 85% (2006/2360) Checking out files: 86% (2030/2360) Checking out files: 87% (2054/2360) Checking out files: 88% (2077/2360) Checking out files: 89% (2101/2360) Checking out files: 90% (2124/2360) Checking out files: 91% (2148/2360) Checking out files: 92% (2172/2360) Checking out files: 93% (2195/2360) Checking out files: 94% (2219/2360) Checking out files: 95% (2242/2360) Checking out files: 96% (2266/2360) Checking out files: 97% (2290/2360) Checking out files: 98% (2313/2360) Checking out files: 99% (2337/2360) Checking out files: 100% (2360/2360) Checking out files: 100% (2360/2360), done. 2020-11-20T12:05:49.480 INFO:teuthology.orchestra.run.smithi116.stdout:HEAD is now at 54c797a amdgpu: add sienna cichlid firmware for 20.45 2020-11-20T12:05:49.490 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2020-11-20T12:05:49.491 INFO:teuthology.orchestra.run.smithi173:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2020-11-20T12:05:49.497 INFO:teuthology.orchestra.run.smithi173.stderr:bash: line 0: cd: /lib/firmware/updates: No such file or directory 2020-11-20T12:05:49.597 INFO:teuthology.orchestra.run.smithi173.stdout:Initialized empty Git repository in /lib/firmware/updates/.git/ 2020-11-20T12:05:49.599 INFO:teuthology.orchestra.run.smithi173:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2020-11-20T12:05:49.674 INFO:teuthology.orchestra.run.smithi173:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2020-11-20T12:05:49.738 INFO:teuthology.orchestra.run.smithi173:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/master 2020-11-20T12:06:08.826 INFO:teuthology.orchestra.run.smithi173.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2020-11-20T12:06:08.827 INFO:teuthology.orchestra.run.smithi173.stderr: * [new branch] main -> origin/main 2020-11-20T12:06:08.828 INFO:teuthology.orchestra.run.smithi173.stderr: * [new branch] master -> origin/master 2020-11-20T12:06:08.852 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190312 -> 20190312 2020-11-20T12:06:08.852 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190416 -> 20190416 2020-11-20T12:06:08.852 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190514 -> 20190514 2020-11-20T12:06:08.853 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190618 -> 20190618 2020-11-20T12:06:08.853 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190717 -> 20190717 2020-11-20T12:06:08.853 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190815 -> 20190815 2020-11-20T12:06:08.853 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20190923 -> 20190923 2020-11-20T12:06:08.853 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20191022 -> 20191022 2020-11-20T12:06:08.854 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20191215 -> 20191215 2020-11-20T12:06:08.854 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200122 -> 20200122 2020-11-20T12:06:08.855 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200316 -> 20200316 2020-11-20T12:06:08.855 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200421 -> 20200421 2020-11-20T12:06:08.856 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200519 -> 20200519 2020-11-20T12:06:08.856 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200619 -> 20200619 2020-11-20T12:06:08.856 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200721 -> 20200721 2020-11-20T12:06:08.856 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200817 -> 20200817 2020-11-20T12:06:08.856 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20200918 -> 20200918 2020-11-20T12:06:08.857 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20201022 -> 20201022 2020-11-20T12:06:08.857 INFO:teuthology.orchestra.run.smithi173.stderr: * [new tag] 20201118 -> 20201118 2020-11-20T12:06:10.974 INFO:teuthology.orchestra.run.smithi173.stderr:Checking out files: 53% (1260/2360) Checking out files: 54% (1275/2360) Checking out files: 55% (1298/2360) Checking out files: 56% (1322/2360) Checking out files: 57% (1346/2360) Checking out files: 58% (1369/2360) Checking out files: 59% (1393/2360) Checking out files: 60% (1416/2360) Checking out files: 61% (1440/2360) Checking out files: 62% (1464/2360) Checking out files: 63% (1487/2360) Checking out files: 64% (1511/2360) Checking out files: 65% (1534/2360) Checking out files: 66% (1558/2360) Checking out files: 67% (1582/2360) Checking out files: 68% (1605/2360) Checking out files: 69% (1629/2360) Checking out files: 70% (1652/2360) Checking out files: 71% (1676/2360) Checking out files: 72% (1700/2360) Checking out files: 73% (1723/2360) Checking out files: 74% (1747/2360) Checking out files: 75% (1770/2360) Checking out files: 76% (1794/2360) Checking out files: 77% (1818/2360) Checking out files: 78% (1841/2360) Checking out files: 79% (1865/2360) Checking out files: 80% (1888/2360) Checking out files: 81% (1912/2360) Checking out files: 82% (1936/2360) Checking out files: 83% (1959/2360) Checking out files: 84% (1983/2360) Checking out files: 85% (2006/2360) Checking out files: 86% (2030/2360) Checking out files: 86% (2044/2360) Checking out files: 87% (2054/2360) Checking out files: 88% (2077/2360) Checking out files: 89% (2101/2360) Checking out files: 90% (2124/2360) Checking out files: 91% (2148/2360) Checking out files: 92% (2172/2360) Checking out files: 93% (2195/2360) Checking out files: 94% (2219/2360) Checking out files: 95% (2242/2360) Checking out files: 96% (2266/2360) Checking out files: 97% (2290/2360) Checking out files: 98% (2313/2360) Checking out files: 99% (2337/2360) Checking out files: 100% (2360/2360) Checking out files: 100% (2360/2360), done. 2020-11-20T12:06:10.977 INFO:teuthology.orchestra.run.smithi173.stdout:HEAD is now at 54c797a amdgpu: add sienna cichlid firmware for 20.45 2020-11-20T12:06:10.986 INFO:teuthology.task.kernel:Downloading kernel ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 on client.0... 2020-11-20T12:06:10.987 INFO:teuthology.orchestra.run.smithi116:> sudo lsb_release -is 2020-11-20T12:06:11.041 INFO:teuthology.orchestra.run.smithi116.stdout:Ubuntu 2020-11-20T12:06:11.046 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-20T12:06:11.046 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-20T12:06:11.218 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-20T12:06:11.218 INFO:teuthology.orchestra.run.smithi116:> rm -f /tmp/linux-image.deb && echo linux-image-5.10.0-rc2-ceph-gac4f7b07f403_5.10.0-rc2-ceph-gac4f7b07f403-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/ --input-file=- 2020-11-20T12:06:11.221 INFO:teuthology.task.kernel:Downloading kernel ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 on client.1... 2020-11-20T12:06:11.221 INFO:teuthology.orchestra.run.smithi173:> sudo lsb_release -is 2020-11-20T12:06:11.267 INFO:teuthology.orchestra.run.smithi173.stdout:Ubuntu 2020-11-20T12:06:11.271 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-20T12:06:11.272 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 2020-11-20T12:06:11.444 INFO:teuthology.task.kernel:fetching, builder baseurl is https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-20T12:06:11.445 INFO:teuthology.orchestra.run.smithi173:> rm -f /tmp/linux-image.deb && echo linux-image-5.10.0-rc2-ceph-gac4f7b07f403_5.10.0-rc2-ceph-gac4f7b07f403-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/ --input-file=- 2020-11-20T12:06:11.447 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi116.front.sepia.ceph.com to complete... 2020-11-20T12:06:21.615 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20 12:06:21 URL:https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/linux-image-5.10.0-rc2-ceph-gac4f7b07f403_5.10.0-rc2-ceph-gac4f7b07f403-1_amd64.deb [33925808/33925808] -> "/tmp/linux-image.deb" [1] 2020-11-20T12:06:21.616 INFO:teuthology.orchestra.run.smithi173.stderr:FINISHED --2020-11-20 12:06:21-- 2020-11-20T12:06:21.616 INFO:teuthology.orchestra.run.smithi173.stderr:Total wall clock time: 10s 2020-11-20T12:06:21.616 INFO:teuthology.orchestra.run.smithi173.stderr:Downloaded: 1 files, 32M in 3.2s (10.0 MB/s) 2020-11-20T12:06:22.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20 12:06:22 URL:https://1.chacra.ceph.com/r/kernel/testing/ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9/ubuntu/bionic/flavors/default/pool/main/l/linux-5.10.0-rc2-ceph-gac4f7b07f403/linux-image-5.10.0-rc2-ceph-gac4f7b07f403_5.10.0-rc2-ceph-gac4f7b07f403-1_amd64.deb [33925808/33925808] -> "/tmp/linux-image.deb" [1] 2020-11-20T12:06:22.346 INFO:teuthology.orchestra.run.smithi116.stderr:FINISHED --2020-11-20 12:06:22-- 2020-11-20T12:06:22.346 INFO:teuthology.orchestra.run.smithi116.stderr:Total wall clock time: 11s 2020-11-20T12:06:22.347 INFO:teuthology.orchestra.run.smithi116.stderr:Downloaded: 1 files, 32M in 4.0s (8.19 MB/s) 2020-11-20T12:06:22.347 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi173.front.sepia.ceph.com to complete... 2020-11-20T12:06:22.348 INFO:teuthology.task.kernel:Installing kernel ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 on client.0... 2020-11-20T12:06:22.348 INFO:teuthology.orchestra.run.smithi116:> sudo dpkg -i /tmp/linux-image.deb 2020-11-20T12:06:27.303 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package linux-image-5.10.0-rc2-ceph-gac4f7b07f403. 2020-11-20T12:06:27.344 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... 102077 files and directories currently installed.) 2020-11-20T12:06:27.347 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack /tmp/linux-image.deb ... 2020-11-20T12:06:28.127 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking linux-image-5.10.0-rc2-ceph-gac4f7b07f403 (5.10.0-rc2-ceph-gac4f7b07f403-1) ... 2020-11-20T12:06:31.057 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up linux-image-5.10.0-rc2-ceph-gac4f7b07f403 (5.10.0-rc2-ceph-gac4f7b07f403-1) ... 2020-11-20T12:06:31.284 INFO:teuthology.orchestra.run.smithi116.stderr:update-initramfs: Generating /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:06:43.053 INFO:teuthology.orchestra.run.smithi116.stderr:Sourcing file `/etc/default/grub' 2020-11-20T12:06:43.077 INFO:teuthology.orchestra.run.smithi116.stderr:Generating grub configuration file ... 2020-11-20T12:06:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:06:43.439 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:06:43.755 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-20T12:06:43.788 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-20T12:06:45.432 INFO:teuthology.orchestra.run.smithi116.stderr:done 2020-11-20T12:06:45.581 INFO:teuthology.orchestra.run.smithi116:> dpkg-deb -c /tmp/linux-image.deb 2020-11-20T12:06:45.590 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./ 2020-11-20T12:06:45.590 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./boot/ 2020-11-20T12:06:45.590 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 5164573 2020-11-13 14:15 ./boot/System.map-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:06:45.651 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 153248 2020-11-13 14:15 ./boot/config-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:06:45.654 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9847776 2020-11-13 14:15 ./boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:06:45.798 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/ 2020-11-20T12:06:45.798 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/ 2020-11-20T12:06:45.798 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/postinst.d/ 2020-11-20T12:06:45.798 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/postrm.d/ 2020-11-20T12:06:45.799 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/preinst.d/ 2020-11-20T12:06:45.799 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/prerm.d/ 2020-11-20T12:06:45.799 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/ 2020-11-20T12:06:45.799 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/ 2020-11-20T12:06:45.799 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-20T12:06:45.800 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/ 2020-11-20T12:06:45.800 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/ 2020-11-20T12:06:45.800 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/ 2020-11-20T12:06:45.800 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/ 2020-11-20T12:06:45.800 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 488985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/aesni-intel.ko 2020-11-20T12:06:45.801 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/blowfish-x86_64.ko 2020-11-20T12:06:45.802 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2020-11-20T12:06:45.802 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2020-11-20T12:06:45.802 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 86201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-x86_64.ko 2020-11-20T12:06:45.802 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 98425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2020-11-20T12:06:45.803 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 143041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2020-11-20T12:06:45.803 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/crc32-pclmul.ko 2020-11-20T12:06:45.803 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2020-11-20T12:06:45.804 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2020-11-20T12:06:45.804 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2020-11-20T12:06:45.805 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/glue_helper.ko 2020-11-20T12:06:45.805 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2020-11-20T12:06:45.805 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-avx2.ko 2020-11-20T12:06:45.806 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2020-11-20T12:06:45.806 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha1-ssse3.ko 2020-11-20T12:06:45.807 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha256-ssse3.ko 2020-11-20T12:06:45.807 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha512-ssse3.ko 2020-11-20T12:06:45.808 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2020-11-20T12:06:45.808 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2020-11-20T12:06:45.809 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-x86_64.ko 2020-11-20T12:06:45.809 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/ 2020-11-20T12:06:45.810 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/ 2020-11-20T12:06:45.810 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/mce/ 2020-11-20T12:06:45.810 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2020-11-20T12:06:45.810 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpuid.ko 2020-11-20T12:06:45.810 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/msr.ko 2020-11-20T12:06:45.811 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/ 2020-11-20T12:06:45.811 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 247537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm-amd.ko 2020-11-20T12:06:45.813 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 826857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm-intel.ko 2020-11-20T12:06:45.822 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1834553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm.ko 2020-11-20T12:06:45.844 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/oprofile/ 2020-11-20T12:06:45.844 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 135385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/oprofile/oprofile.ko 2020-11-20T12:06:45.845 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ 2020-11-20T12:06:45.846 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/af_alg.ko 2020-11-20T12:06:45.846 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/algif_hash.ko 2020-11-20T12:06:45.846 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/algif_skcipher.ko 2020-11-20T12:06:45.848 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ansi_cprng.ko 2020-11-20T12:06:45.848 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/anubis.ko 2020-11-20T12:06:45.848 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/arc4.ko 2020-11-20T12:06:45.848 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/asymmetric_keys/ 2020-11-20T12:06:45.848 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2020-11-20T12:06:45.849 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/ 2020-11-20T12:06:45.849 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_memcpy.ko 2020-11-20T12:06:45.849 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_pq.ko 2020-11-20T12:06:45.850 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_raid6_recov.ko 2020-11-20T12:06:45.851 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_tx.ko 2020-11-20T12:06:45.851 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_xor.ko 2020-11-20T12:06:45.851 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/raid6test.ko 2020-11-20T12:06:45.851 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/authenc.ko 2020-11-20T12:06:45.852 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/authencesn.ko 2020-11-20T12:06:45.852 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blake2b_generic.ko 2020-11-20T12:06:45.852 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blowfish_common.ko 2020-11-20T12:06:45.852 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blowfish_generic.ko 2020-11-20T12:06:45.852 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/camellia_generic.ko 2020-11-20T12:06:45.853 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast5_generic.ko 2020-11-20T12:06:45.854 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast6_generic.ko 2020-11-20T12:06:45.854 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast_common.ko 2020-11-20T12:06:45.855 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ccm.ko 2020-11-20T12:06:45.855 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cmac.ko 2020-11-20T12:06:45.855 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crc32_generic.ko 2020-11-20T12:06:45.855 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cryptd.ko 2020-11-20T12:06:45.855 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_engine.ko 2020-11-20T12:06:45.856 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_simd.ko 2020-11-20T12:06:45.856 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_user.ko 2020-11-20T12:06:45.856 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cts.ko 2020-11-20T12:06:45.856 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/des_generic.ko 2020-11-20T12:06:45.856 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/dh_generic.ko 2020-11-20T12:06:45.857 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/echainiv.ko 2020-11-20T12:06:45.857 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/essiv.ko 2020-11-20T12:06:45.859 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/fcrypt.ko 2020-11-20T12:06:45.859 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/khazad.ko 2020-11-20T12:06:45.859 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lrw.ko 2020-11-20T12:06:45.859 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lz4.ko 2020-11-20T12:06:45.859 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lz4hc.ko 2020-11-20T12:06:45.860 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/md4.ko 2020-11-20T12:06:45.860 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/michael_mic.ko 2020-11-20T12:06:45.860 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/pcbc.ko 2020-11-20T12:06:45.860 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/pcrypt.ko 2020-11-20T12:06:45.860 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd128.ko 2020-11-20T12:06:45.861 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd160.ko 2020-11-20T12:06:45.861 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd256.ko 2020-11-20T12:06:45.861 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd320.ko 2020-11-20T12:06:45.861 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/salsa20_generic.ko 2020-11-20T12:06:45.861 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/seed.ko 2020-11-20T12:06:45.862 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/serpent_generic.ko 2020-11-20T12:06:45.862 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 170649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tcrypt.ko 2020-11-20T12:06:45.863 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tea.ko 2020-11-20T12:06:45.864 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tgr192.ko 2020-11-20T12:06:45.864 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/twofish_common.ko 2020-11-20T12:06:45.864 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/twofish_generic.ko 2020-11-20T12:06:45.864 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/vmac.ko 2020-11-20T12:06:45.864 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/wp512.ko 2020-11-20T12:06:45.865 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xcbc.ko 2020-11-20T12:06:45.865 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xor.ko 2020-11-20T12:06:45.865 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xts.ko 2020-11-20T12:06:45.865 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xxhash_generic.ko 2020-11-20T12:06:45.866 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ 2020-11-20T12:06:45.866 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/ 2020-11-20T12:06:45.866 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_extlog.ko 2020-11-20T12:06:45.866 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_ipmi.ko 2020-11-20T12:06:45.866 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_pad.ko 2020-11-20T12:06:45.867 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/apei/ 2020-11-20T12:06:45.868 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/apei/einj.ko 2020-11-20T12:06:45.868 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/ec_sys.ko 2020-11-20T12:06:45.868 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/sbs.ko 2020-11-20T12:06:45.868 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/sbshc.ko 2020-11-20T12:06:45.869 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 82265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/video.ko 2020-11-20T12:06:45.869 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ 2020-11-20T12:06:45.869 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/acard-ahci.ko 2020-11-20T12:06:45.869 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 84793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ahci.ko 2020-11-20T12:06:45.869 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ahci_platform.ko 2020-11-20T12:06:45.870 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 81825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/libahci.ko 2020-11-20T12:06:45.871 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/libahci_platform.ko 2020-11-20T12:06:45.872 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_acpi.ko 2020-11-20T12:06:45.872 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ali.ko 2020-11-20T12:06:45.872 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_amd.ko 2020-11-20T12:06:45.872 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_artop.ko 2020-11-20T12:06:45.872 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_atiixp.ko 2020-11-20T12:06:45.873 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_atp867x.ko 2020-11-20T12:06:45.873 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cmd640.ko 2020-11-20T12:06:45.873 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cmd64x.ko 2020-11-20T12:06:45.873 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cypress.ko 2020-11-20T12:06:45.873 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_efar.ko 2020-11-20T12:06:45.874 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt366.ko 2020-11-20T12:06:45.874 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt37x.ko 2020-11-20T12:06:45.874 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt3x2n.ko 2020-11-20T12:06:45.874 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt3x3.ko 2020-11-20T12:06:45.875 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_it8213.ko 2020-11-20T12:06:45.875 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_it821x.ko 2020-11-20T12:06:45.875 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_jmicron.ko 2020-11-20T12:06:45.877 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_legacy.ko 2020-11-20T12:06:45.878 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_marvell.ko 2020-11-20T12:06:45.878 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_mpiix.ko 2020-11-20T12:06:45.878 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_netcell.ko 2020-11-20T12:06:45.878 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ninja32.ko 2020-11-20T12:06:45.879 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ns87410.ko 2020-11-20T12:06:45.879 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ns87415.ko 2020-11-20T12:06:45.879 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_oldpiix.ko 2020-11-20T12:06:45.879 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_opti.ko 2020-11-20T12:06:45.879 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_optidma.ko 2020-11-20T12:06:45.880 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pcmcia.ko 2020-11-20T12:06:45.880 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pdc2027x.ko 2020-11-20T12:06:45.880 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pdc202xx_old.ko 2020-11-20T12:06:45.880 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_piccolo.ko 2020-11-20T12:06:45.880 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_platform.ko 2020-11-20T12:06:45.881 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_radisys.ko 2020-11-20T12:06:45.881 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_rdc.ko 2020-11-20T12:06:45.881 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_rz1000.ko 2020-11-20T12:06:45.881 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sch.ko 2020-11-20T12:06:45.881 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_serverworks.ko 2020-11-20T12:06:45.882 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sil680.ko 2020-11-20T12:06:45.882 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sl82c105.ko 2020-11-20T12:06:45.882 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_triflex.ko 2020-11-20T12:06:45.883 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_via.ko 2020-11-20T12:06:45.883 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pdc_adma.ko 2020-11-20T12:06:45.883 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_inic162x.ko 2020-11-20T12:06:45.883 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 76097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_mv.ko 2020-11-20T12:06:45.883 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_nv.ko 2020-11-20T12:06:45.885 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_promise.ko 2020-11-20T12:06:45.885 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_qstor.ko 2020-11-20T12:06:45.885 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sil.ko 2020-11-20T12:06:45.886 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sil24.ko 2020-11-20T12:06:45.886 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sis.ko 2020-11-20T12:06:45.886 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_svw.ko 2020-11-20T12:06:45.886 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sx4.ko 2020-11-20T12:06:45.886 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_uli.ko 2020-11-20T12:06:45.887 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_via.ko 2020-11-20T12:06:45.887 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_vsc.ko 2020-11-20T12:06:45.887 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/auxdisplay/ 2020-11-20T12:06:45.887 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/auxdisplay/ks0108.ko 2020-11-20T12:06:45.888 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/bcma/ 2020-11-20T12:06:45.888 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 123697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/bcma/bcma.ko 2020-11-20T12:06:45.888 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/ 2020-11-20T12:06:45.888 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/aoe/ 2020-11-20T12:06:45.888 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 115129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/aoe/aoe.ko 2020-11-20T12:06:45.889 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/cryptoloop.ko 2020-11-20T12:06:45.889 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/drbd/ 2020-11-20T12:06:45.889 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1050545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/drbd/drbd.ko 2020-11-20T12:06:45.898 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 176513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/floppy.ko 2020-11-20T12:06:45.900 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/mtip32xx/ 2020-11-20T12:06:45.900 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 107353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/mtip32xx/mtip32xx.ko 2020-11-20T12:06:45.901 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 112145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/nbd.ko 2020-11-20T12:06:45.903 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 92265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/null_blk.ko 2020-11-20T12:06:45.904 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 83089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/pktcdvd.ko 2020-11-20T12:06:45.905 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 260233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rbd.ko 2020-11-20T12:06:45.908 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rsxx/ 2020-11-20T12:06:45.908 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 113017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rsxx/rsxx.ko 2020-11-20T12:06:45.910 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 140897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/skd.ko 2020-11-20T12:06:45.911 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/sx8.ko 2020-11-20T12:06:45.912 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/umem.ko 2020-11-20T12:06:45.912 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/xen-blkback/ 2020-11-20T12:06:45.913 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 101497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/xen-blkback/xen-blkback.ko 2020-11-20T12:06:45.914 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/zram/ 2020-11-20T12:06:45.914 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/zram/zram.ko 2020-11-20T12:06:45.915 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ 2020-11-20T12:06:45.915 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/applicom.ko 2020-11-20T12:06:45.916 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hangcheck-timer.ko 2020-11-20T12:06:45.916 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/ 2020-11-20T12:06:45.917 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/amd-rng.ko 2020-11-20T12:06:45.917 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/intel-rng.ko 2020-11-20T12:06:45.917 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/timeriomem-rng.ko 2020-11-20T12:06:45.917 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/via-rng.ko 2020-11-20T12:06:45.918 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/virtio-rng.ko 2020-11-20T12:06:45.918 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ 2020-11-20T12:06:45.918 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_devintf.ko 2020-11-20T12:06:45.919 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 138969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2020-11-20T12:06:45.919 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2020-11-20T12:06:45.919 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 142913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_si.ko 2020-11-20T12:06:45.921 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_ssif.ko 2020-11-20T12:06:45.922 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2020-11-20T12:06:45.922 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/lp.ko 2020-11-20T12:06:45.923 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/mwave/ 2020-11-20T12:06:45.923 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 88665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/mwave/mwave.ko 2020-11-20T12:06:45.924 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/nvram.ko 2020-11-20T12:06:45.925 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/ 2020-11-20T12:06:45.925 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 102257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/cm4000_cs.ko 2020-11-20T12:06:45.925 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/cm4040_cs.ko 2020-11-20T12:06:45.926 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 86617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/synclink_cs.ko 2020-11-20T12:06:45.927 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ppdev.ko 2020-11-20T12:06:45.927 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/raw.ko 2020-11-20T12:06:45.928 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tlclk.ko 2020-11-20T12:06:45.928 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/ 2020-11-20T12:06:45.929 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_atmel.ko 2020-11-20T12:06:45.929 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2020-11-20T12:06:45.929 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2020-11-20T12:06:45.930 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2020-11-20T12:06:45.930 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_infineon.ko 2020-11-20T12:06:45.931 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_nsc.ko 2020-11-20T12:06:45.931 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/xen-tpmfront.ko 2020-11-20T12:06:45.931 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/ 2020-11-20T12:06:45.931 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/xillybus_core.ko 2020-11-20T12:06:45.933 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/xillybus_pcie.ko 2020-11-20T12:06:45.933 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/ 2020-11-20T12:06:45.933 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-palmas.ko 2020-11-20T12:06:45.934 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-s2mps11.ko 2020-11-20T12:06:45.934 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-si5351.ko 2020-11-20T12:06:45.934 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-twl6040.ko 2020-11-20T12:06:45.934 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-wm831x.ko 2020-11-20T12:06:45.935 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/ 2020-11-20T12:06:45.935 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2020-11-20T12:06:45.935 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/p4-clockmod.ko 2020-11-20T12:06:45.935 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/speedstep-lib.ko 2020-11-20T12:06:45.935 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ 2020-11-20T12:06:45.936 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ 2020-11-20T12:06:45.936 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ccp-crypto.ko 2020-11-20T12:06:45.937 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 203753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ccp.ko 2020-11-20T12:06:45.937 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/padlock-aes.ko 2020-11-20T12:06:45.937 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/padlock-sha.ko 2020-11-20T12:06:45.938 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/ 2020-11-20T12:06:45.938 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_common/ 2020-11-20T12:06:45.938 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 279065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_common/intel_qat.ko 2020-11-20T12:06:45.940 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_dh895xcc/ 2020-11-20T12:06:45.940 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko 2020-11-20T12:06:45.941 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/virtio/ 2020-11-20T12:06:45.942 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/virtio/virtio_crypto.ko 2020-11-20T12:06:45.942 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dca/ 2020-11-20T12:06:45.942 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dca/dca.ko 2020-11-20T12:06:45.942 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ 2020-11-20T12:06:45.943 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/dw/ 2020-11-20T12:06:45.943 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/dw/dw_dmac.ko 2020-11-20T12:06:45.944 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ioat/ 2020-11-20T12:06:45.944 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 124161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ioat/ioatdma.ko 2020-11-20T12:06:45.944 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/ 2020-11-20T12:06:45.944 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/amd64_edac_mod.ko 2020-11-20T12:06:45.945 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/e752x_edac.ko 2020-11-20T12:06:45.945 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/edac_mce_amd.ko 2020-11-20T12:06:45.946 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i3000_edac.ko 2020-11-20T12:06:45.946 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i3200_edac.ko 2020-11-20T12:06:45.946 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5000_edac.ko 2020-11-20T12:06:45.947 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5100_edac.ko 2020-11-20T12:06:45.948 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5400_edac.ko 2020-11-20T12:06:45.948 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i7300_edac.ko 2020-11-20T12:06:45.948 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i7core_edac.ko 2020-11-20T12:06:45.949 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i82975x_edac.ko 2020-11-20T12:06:45.950 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/ie31200_edac.ko 2020-11-20T12:06:45.950 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/sb_edac.ko 2020-11-20T12:06:45.950 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/x38_edac.ko 2020-11-20T12:06:45.950 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/ 2020-11-20T12:06:45.950 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-gpio.ko 2020-11-20T12:06:45.951 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max14577.ko 2020-11-20T12:06:45.951 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max77693.ko 2020-11-20T12:06:45.953 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max8997.ko 2020-11-20T12:06:45.953 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-palmas.ko 2020-11-20T12:06:45.954 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-rt8973a.ko 2020-11-20T12:06:45.954 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-sm5502.ko 2020-11-20T12:06:45.954 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/ 2020-11-20T12:06:45.954 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/dmi-sysfs.ko 2020-11-20T12:06:45.955 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/efi/ 2020-11-20T12:06:45.955 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/efi/efi-pstore.ko 2020-11-20T12:06:45.955 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/iscsi_ibft.ko 2020-11-20T12:06:45.955 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/ 2020-11-20T12:06:45.956 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-adp5520.ko 2020-11-20T12:06:45.956 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-adp5588.ko 2020-11-20T12:06:45.956 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-amd8111.ko 2020-11-20T12:06:45.956 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-arizona.ko 2020-11-20T12:06:45.957 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-da9052.ko 2020-11-20T12:06:45.957 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-da9055.ko 2020-11-20T12:06:45.957 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-dln2.ko 2020-11-20T12:06:45.957 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-f7188x.ko 2020-11-20T12:06:45.958 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-generic.ko 2020-11-20T12:06:45.958 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-ich.ko 2020-11-20T12:06:45.963 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-janz-ttl.ko 2020-11-20T12:06:45.963 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-kempld.ko 2020-11-20T12:06:45.963 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-lp3943.ko 2020-11-20T12:06:45.964 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max7300.ko 2020-11-20T12:06:45.964 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max730x.ko 2020-11-20T12:06:45.964 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max732x.ko 2020-11-20T12:06:45.964 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-ml-ioh.ko 2020-11-20T12:06:45.965 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-pca953x.ko 2020-11-20T12:06:45.965 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-pcf857x.ko 2020-11-20T12:06:45.965 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-rdc321x.ko 2020-11-20T12:06:45.965 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-sch.ko 2020-11-20T12:06:45.965 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-sch311x.ko 2020-11-20T12:06:45.966 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-tps65912.ko 2020-11-20T12:06:45.966 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-twl4030.ko 2020-11-20T12:06:45.966 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-twl6040.ko 2020-11-20T12:06:45.966 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-viperboard.ko 2020-11-20T12:06:45.967 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-vx855.ko 2020-11-20T12:06:45.967 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm831x.ko 2020-11-20T12:06:45.967 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm8350.ko 2020-11-20T12:06:45.967 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm8994.ko 2020-11-20T12:06:45.968 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/ 2020-11-20T12:06:45.968 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-a4tech.ko 2020-11-20T12:06:45.968 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-apple.ko 2020-11-20T12:06:45.968 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-appleir.ko 2020-11-20T12:06:45.969 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-aureal.ko 2020-11-20T12:06:45.969 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-axff.ko 2020-11-20T12:06:45.969 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-belkin.ko 2020-11-20T12:06:45.969 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cherry.ko 2020-11-20T12:06:45.969 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-chicony.ko 2020-11-20T12:06:45.970 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cp2112.ko 2020-11-20T12:06:45.970 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cypress.ko 2020-11-20T12:06:45.970 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-dr.ko 2020-11-20T12:06:45.970 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-elecom.ko 2020-11-20T12:06:45.971 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-elo.ko 2020-11-20T12:06:45.971 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-emsff.ko 2020-11-20T12:06:45.971 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ezkey.ko 2020-11-20T12:06:45.971 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gaff.ko 2020-11-20T12:06:45.971 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-generic.ko 2020-11-20T12:06:45.972 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gt683r.ko 2020-11-20T12:06:45.972 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gyration.ko 2020-11-20T12:06:45.972 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtek-kbd.ko 2020-11-20T12:06:45.972 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtek-mouse.ko 2020-11-20T12:06:45.973 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtekff.ko 2020-11-20T12:06:45.973 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-hyperv.ko 2020-11-20T12:06:45.973 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-icade.ko 2020-11-20T12:06:45.973 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-kensington.ko 2020-11-20T12:06:45.974 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-keytouch.ko 2020-11-20T12:06:45.974 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-kye.ko 2020-11-20T12:06:45.974 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lcpower.ko 2020-11-20T12:06:45.974 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-led.ko 2020-11-20T12:06:45.975 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lenovo.ko 2020-11-20T12:06:45.975 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lg-g15.ko 2020-11-20T12:06:45.975 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech-dj.ko 2020-11-20T12:06:45.975 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech-hidpp.ko 2020-11-20T12:06:45.979 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 93089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech.ko 2020-11-20T12:06:45.980 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-magicmouse.ko 2020-11-20T12:06:45.980 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-microsoft.ko 2020-11-20T12:06:45.980 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-monterey.ko 2020-11-20T12:06:45.980 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-multitouch.ko 2020-11-20T12:06:45.980 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ntrig.ko 2020-11-20T12:06:45.981 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ortek.ko 2020-11-20T12:06:45.981 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-penmount.ko 2020-11-20T12:06:45.981 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-petalynx.ko 2020-11-20T12:06:45.982 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 74209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-picolcd.ko 2020-11-20T12:06:45.982 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-pl.ko 2020-11-20T12:06:45.982 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-plantronics.ko 2020-11-20T12:06:45.982 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-primax.ko 2020-11-20T12:06:45.983 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-rmi.ko 2020-11-20T12:06:45.983 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-arvo.ko 2020-11-20T12:06:45.983 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-common.ko 2020-11-20T12:06:45.983 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-isku.ko 2020-11-20T12:06:45.984 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-kone.ko 2020-11-20T12:06:45.984 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-koneplus.ko 2020-11-20T12:06:45.984 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-konepure.ko 2020-11-20T12:06:45.984 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-kovaplus.ko 2020-11-20T12:06:45.984 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-lua.ko 2020-11-20T12:06:45.985 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-pyra.ko 2020-11-20T12:06:45.985 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-ryos.ko 2020-11-20T12:06:45.985 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-savu.ko 2020-11-20T12:06:45.985 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat.ko 2020-11-20T12:06:45.986 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-saitek.ko 2020-11-20T12:06:45.986 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-samsung.ko 2020-11-20T12:06:45.986 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sensor-hub.ko 2020-11-20T12:06:45.986 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sjoy.ko 2020-11-20T12:06:45.987 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sony.ko 2020-11-20T12:06:45.987 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-speedlink.ko 2020-11-20T12:06:45.987 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-steelseries.ko 2020-11-20T12:06:45.987 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sunplus.ko 2020-11-20T12:06:45.987 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-tivo.ko 2020-11-20T12:06:45.988 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-tmff.ko 2020-11-20T12:06:45.988 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-topseed.ko 2020-11-20T12:06:45.988 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-twinhan.ko 2020-11-20T12:06:45.988 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-uclogic.ko 2020-11-20T12:06:45.989 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-waltop.ko 2020-11-20T12:06:45.989 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 133729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-wiimote.ko 2020-11-20T12:06:45.989 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-xinmo.ko 2020-11-20T12:06:45.989 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-zpff.ko 2020-11-20T12:06:45.990 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-zydacron.ko 2020-11-20T12:06:45.990 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 284097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid.ko 2020-11-20T12:06:45.990 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/i2c-hid/ 2020-11-20T12:06:45.990 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/i2c-hid/i2c-hid.ko 2020-11-20T12:06:45.992 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/uhid.ko 2020-11-20T12:06:45.992 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/ 2020-11-20T12:06:45.993 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 125177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbhid.ko 2020-11-20T12:06:45.993 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbkbd.ko 2020-11-20T12:06:45.993 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbmouse.ko 2020-11-20T12:06:45.993 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 222905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/wacom.ko 2020-11-20T12:06:45.994 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/ 2020-11-20T12:06:45.994 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_balloon.ko 2020-11-20T12:06:45.994 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 79585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_utils.ko 2020-11-20T12:06:45.994 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 277809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_vmbus.ko 2020-11-20T12:06:45.995 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ 2020-11-20T12:06:45.995 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/abituguru.ko 2020-11-20T12:06:45.995 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/abituguru3.ko 2020-11-20T12:06:45.995 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/acpi_power_meter.ko 2020-11-20T12:06:45.995 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ad7414.ko 2020-11-20T12:06:45.996 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ad7418.ko 2020-11-20T12:06:45.996 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adc128d818.ko 2020-11-20T12:06:45.996 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1021.ko 2020-11-20T12:06:45.996 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1025.ko 2020-11-20T12:06:45.997 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 98513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1026.ko 2020-11-20T12:06:45.998 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1029.ko 2020-11-20T12:06:45.998 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1031.ko 2020-11-20T12:06:45.999 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm9240.ko 2020-11-20T12:06:45.999 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ads7828.ko 2020-11-20T12:06:45.999 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7410.ko 2020-11-20T12:06:45.999 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7411.ko 2020-11-20T12:06:46.000 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 75745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7462.ko 2020-11-20T12:06:46.000 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7470.ko 2020-11-20T12:06:46.000 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7475.ko 2020-11-20T12:06:46.000 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7x10.ko 2020-11-20T12:06:46.001 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/amc6821.ko 2020-11-20T12:06:46.001 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/applesmc.ko 2020-11-20T12:06:46.001 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asb100.ko 2020-11-20T12:06:46.001 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asc7621.ko 2020-11-20T12:06:46.003 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asus_atk0110.ko 2020-11-20T12:06:46.003 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/atxp1.ko 2020-11-20T12:06:46.003 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/coretemp.ko 2020-11-20T12:06:46.004 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/da9052-hwmon.ko 2020-11-20T12:06:46.004 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/da9055-hwmon.ko 2020-11-20T12:06:46.004 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/dell-smm-hwmon.ko 2020-11-20T12:06:46.004 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 87601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/dme1737.ko 2020-11-20T12:06:46.005 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ds1621.ko 2020-11-20T12:06:46.005 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ds620.ko 2020-11-20T12:06:46.005 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc1403.ko 2020-11-20T12:06:46.005 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc2103.ko 2020-11-20T12:06:46.005 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc6w201.ko 2020-11-20T12:06:46.006 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f71805f.ko 2020-11-20T12:06:46.006 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 94745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f71882fg.ko 2020-11-20T12:06:46.008 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f75375s.ko 2020-11-20T12:06:46.008 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/fam15h_power.ko 2020-11-20T12:06:46.008 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/fschmd.ko 2020-11-20T12:06:46.009 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/g760a.ko 2020-11-20T12:06:46.013 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/g762.ko 2020-11-20T12:06:46.013 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/gl518sm.ko 2020-11-20T12:06:46.013 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/gl520sm.ko 2020-11-20T12:06:46.013 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/hih6130.ko 2020-11-20T12:06:46.014 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/hwmon-vid.ko 2020-11-20T12:06:46.014 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/i5500_temp.ko 2020-11-20T12:06:46.014 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/i5k_amb.ko 2020-11-20T12:06:46.014 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ibmaem.ko 2020-11-20T12:06:46.015 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ibmpex.ko 2020-11-20T12:06:46.015 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ina209.ko 2020-11-20T12:06:46.015 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ina2xx.ko 2020-11-20T12:06:46.015 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 109417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/it87.ko 2020-11-20T12:06:46.019 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/jc42.ko 2020-11-20T12:06:46.020 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/k10temp.ko 2020-11-20T12:06:46.020 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/k8temp.ko 2020-11-20T12:06:46.020 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lineage-pem.ko 2020-11-20T12:06:46.020 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm63.ko 2020-11-20T12:06:46.020 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm73.ko 2020-11-20T12:06:46.021 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm75.ko 2020-11-20T12:06:46.021 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm77.ko 2020-11-20T12:06:46.021 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm78.ko 2020-11-20T12:06:46.021 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm80.ko 2020-11-20T12:06:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm83.ko 2020-11-20T12:06:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 74897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm85.ko 2020-11-20T12:06:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm87.ko 2020-11-20T12:06:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm90.ko 2020-11-20T12:06:46.022 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm92.ko 2020-11-20T12:06:46.023 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 104761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm93.ko 2020-11-20T12:06:46.023 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95234.ko 2020-11-20T12:06:46.023 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95241.ko 2020-11-20T12:06:46.023 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95245.ko 2020-11-20T12:06:46.024 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc2945.ko 2020-11-20T12:06:46.024 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4151.ko 2020-11-20T12:06:46.024 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4215.ko 2020-11-20T12:06:46.024 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4222.ko 2020-11-20T12:06:46.025 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4245.ko 2020-11-20T12:06:46.025 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4260.ko 2020-11-20T12:06:46.025 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4261.ko 2020-11-20T12:06:46.025 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max16065.ko 2020-11-20T12:06:46.026 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max1619.ko 2020-11-20T12:06:46.026 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max1668.ko 2020-11-20T12:06:46.026 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max197.ko 2020-11-20T12:06:46.026 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6639.ko 2020-11-20T12:06:46.026 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6642.ko 2020-11-20T12:06:46.027 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6650.ko 2020-11-20T12:06:46.027 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6697.ko 2020-11-20T12:06:46.027 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/mc13783-adc.ko 2020-11-20T12:06:46.027 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/mcp3021.ko 2020-11-20T12:06:46.027 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2020-11-20T12:06:46.028 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct6683.ko 2020-11-20T12:06:46.028 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 124641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct6775.ko 2020-11-20T12:06:46.032 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 58769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct7802.ko 2020-11-20T12:06:46.032 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ntc_thermistor.ko 2020-11-20T12:06:46.032 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 79793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pc87360.ko 2020-11-20T12:06:46.033 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pc87427.ko 2020-11-20T12:06:46.033 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pcf8591.ko 2020-11-20T12:06:46.033 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ 2020-11-20T12:06:46.033 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/adm1275.ko 2020-11-20T12:06:46.034 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/lm25066.ko 2020-11-20T12:06:46.034 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ltc2978.ko 2020-11-20T12:06:46.034 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max16064.ko 2020-11-20T12:06:46.034 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max34440.ko 2020-11-20T12:06:46.035 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max8688.ko 2020-11-20T12:06:46.035 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/pmbus.ko 2020-11-20T12:06:46.035 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2020-11-20T12:06:46.035 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/tps40422.ko 2020-11-20T12:06:46.035 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ucd9000.ko 2020-11-20T12:06:46.036 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ucd9200.ko 2020-11-20T12:06:46.036 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/zl6100.ko 2020-11-20T12:06:46.036 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/powr1220.ko 2020-11-20T12:06:46.036 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch5627.ko 2020-11-20T12:06:46.037 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch5636.ko 2020-11-20T12:06:46.037 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch56xx-common.ko 2020-11-20T12:06:46.037 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sht15.ko 2020-11-20T12:06:46.037 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sht21.ko 2020-11-20T12:06:46.038 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/shtc1.ko 2020-11-20T12:06:46.038 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sis5595.ko 2020-11-20T12:06:46.038 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smm665.ko 2020-11-20T12:06:46.038 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47b397.ko 2020-11-20T12:06:46.038 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47m1.ko 2020-11-20T12:06:46.039 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47m192.ko 2020-11-20T12:06:46.039 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/thmc50.ko 2020-11-20T12:06:46.039 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp102.ko 2020-11-20T12:06:46.039 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp103.ko 2020-11-20T12:06:46.040 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp401.ko 2020-11-20T12:06:46.040 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp421.ko 2020-11-20T12:06:46.040 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/via-cputemp.ko 2020-11-20T12:06:46.040 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/via686a.ko 2020-11-20T12:06:46.040 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/vt1211.ko 2020-11-20T12:06:46.041 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/vt8231.ko 2020-11-20T12:06:46.041 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83627ehf.ko 2020-11-20T12:06:46.041 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83627hf.ko 2020-11-20T12:06:46.041 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83781d.ko 2020-11-20T12:06:46.046 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83791d.ko 2020-11-20T12:06:46.047 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 71337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83792d.ko 2020-11-20T12:06:46.047 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 101689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83793.ko 2020-11-20T12:06:46.047 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 119921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83795.ko 2020-11-20T12:06:46.048 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83l785ts.ko 2020-11-20T12:06:46.048 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83l786ng.ko 2020-11-20T12:06:46.048 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/wm831x-hwmon.ko 2020-11-20T12:06:46.048 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/wm8350-hwmon.ko 2020-11-20T12:06:46.048 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/ 2020-11-20T12:06:46.049 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/ 2020-11-20T12:06:46.049 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2020-11-20T12:06:46.049 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2020-11-20T12:06:46.050 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/ 2020-11-20T12:06:46.050 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali1535.ko 2020-11-20T12:06:46.050 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali1563.ko 2020-11-20T12:06:46.050 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2020-11-20T12:06:46.051 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2020-11-20T12:06:46.051 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd756.ko 2020-11-20T12:06:46.051 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd8111.ko 2020-11-20T12:06:46.051 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2020-11-20T12:06:46.051 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2020-11-20T12:06:46.052 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-core.ko 2020-11-20T12:06:46.052 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2020-11-20T12:06:46.052 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2020-11-20T12:06:46.052 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2020-11-20T12:06:46.053 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-dln2.ko 2020-11-20T12:06:46.053 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-gpio.ko 2020-11-20T12:06:46.053 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 64897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-i801.ko 2020-11-20T12:06:46.053 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-isch.ko 2020-11-20T12:06:46.054 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ismt.ko 2020-11-20T12:06:46.054 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-kempld.ko 2020-11-20T12:06:46.054 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2020-11-20T12:06:46.054 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-nforce2.ko 2020-11-20T12:06:46.054 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ocores.ko 2020-11-20T12:06:46.055 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-parport.ko 2020-11-20T12:06:46.055 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2020-11-20T12:06:46.055 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-piix4.ko 2020-11-20T12:06:46.055 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2020-11-20T12:06:46.056 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-scmi.ko 2020-11-20T12:06:46.056 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-simtec.ko 2020-11-20T12:06:46.056 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis5595.ko 2020-11-20T12:06:46.056 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis630.ko 2020-11-20T12:06:46.056 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis96x.ko 2020-11-20T12:06:46.057 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2020-11-20T12:06:46.057 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2020-11-20T12:06:46.057 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-via.ko 2020-11-20T12:06:46.057 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-viapro.ko 2020-11-20T12:06:46.058 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-viperboard.ko 2020-11-20T12:06:46.058 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-xiic.ko 2020-11-20T12:06:46.058 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-mux.ko 2020-11-20T12:06:46.058 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-smbus.ko 2020-11-20T12:06:46.059 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-stub.ko 2020-11-20T12:06:46.059 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/ 2020-11-20T12:06:46.059 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2020-11-20T12:06:46.059 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2020-11-20T12:06:46.059 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2020-11-20T12:06:46.060 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ 2020-11-20T12:06:46.060 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ 2020-11-20T12:06:46.060 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 259017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_cm.ko 2020-11-20T12:06:46.061 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 848849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_core.ko 2020-11-20T12:06:46.061 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_umad.ko 2020-11-20T12:06:46.062 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 309825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_uverbs.ko 2020-11-20T12:06:46.065 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 102945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/iw_cm.ko 2020-11-20T12:06:46.066 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 263657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/rdma_cm.ko 2020-11-20T12:06:46.069 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/rdma_ucm.ko 2020-11-20T12:06:46.070 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ 2020-11-20T12:06:46.070 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/cxgb4/ 2020-11-20T12:06:46.070 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 512473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2020-11-20T12:06:46.077 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx4/ 2020-11-20T12:06:46.077 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 420329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2020-11-20T12:06:46.082 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx5/ 2020-11-20T12:06:46.083 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 721305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2020-11-20T12:06:46.092 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mthca/ 2020-11-20T12:06:46.092 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 287121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2020-11-20T12:06:46.096 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ocrdma/ 2020-11-20T12:06:46.096 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 162257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2020-11-20T12:06:46.098 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/usnic/ 2020-11-20T12:06:46.098 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 177449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2020-11-20T12:06:46.100 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ 2020-11-20T12:06:46.100 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ipoib/ 2020-11-20T12:06:46.101 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 272041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2020-11-20T12:06:46.104 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/iser/ 2020-11-20T12:06:46.104 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 105369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2020-11-20T12:06:46.105 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/isert/ 2020-11-20T12:06:46.105 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 114745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2020-11-20T12:06:46.106 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srp/ 2020-11-20T12:06:46.107 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 133665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2020-11-20T12:06:46.108 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srpt/ 2020-11-20T12:06:46.108 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 131153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2020-11-20T12:06:46.110 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/ 2020-11-20T12:06:46.110 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/evbug.ko 2020-11-20T12:06:46.110 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/ff-memless.ko 2020-11-20T12:06:46.111 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/ 2020-11-20T12:06:46.112 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/emu10k1-gp.ko 2020-11-20T12:06:46.112 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/fm801-gp.ko 2020-11-20T12:06:46.112 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/gameport.ko 2020-11-20T12:06:46.112 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/lightning.ko 2020-11-20T12:06:46.113 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/ns558.ko 2020-11-20T12:06:46.113 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/input-polldev.ko 2020-11-20T12:06:46.113 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/joydev.ko 2020-11-20T12:06:46.115 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/ 2020-11-20T12:06:46.116 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5520-keys.ko 2020-11-20T12:06:46.116 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5588-keys.ko 2020-11-20T12:06:46.116 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5589-keys.ko 2020-11-20T12:06:46.116 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2020-11-20T12:06:46.117 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/gpio_keys.ko 2020-11-20T12:06:46.117 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2020-11-20T12:06:46.117 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lkkbd.ko 2020-11-20T12:06:46.117 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lm8323.ko 2020-11-20T12:06:46.117 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lm8333.ko 2020-11-20T12:06:46.118 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/matrix_keypad.ko 2020-11-20T12:06:46.118 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/max7359_keypad.ko 2020-11-20T12:06:46.118 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/mcs_touchkey.ko 2020-11-20T12:06:46.119 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2020-11-20T12:06:46.119 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/newtonkbd.ko 2020-11-20T12:06:46.119 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/opencores-kbd.ko 2020-11-20T12:06:46.119 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/qt1070.ko 2020-11-20T12:06:46.120 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/qt2160.ko 2020-11-20T12:06:46.120 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/samsung-keypad.ko 2020-11-20T12:06:46.120 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/stowaway.ko 2020-11-20T12:06:46.120 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/sunkbd.ko 2020-11-20T12:06:46.122 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/tca6416-keypad.ko 2020-11-20T12:06:46.122 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/tca8418_keypad.ko 2020-11-20T12:06:46.122 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/twl4030_keypad.ko 2020-11-20T12:06:46.122 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/xtkbd.ko 2020-11-20T12:06:46.122 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/matrix-keymap.ko 2020-11-20T12:06:46.123 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/ 2020-11-20T12:06:46.123 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/appletouch.ko 2020-11-20T12:06:46.123 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/bcm5974.ko 2020-11-20T12:06:46.123 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 124297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/cyapatp.ko 2020-11-20T12:06:46.123 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 94273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/elan_i2c.ko 2020-11-20T12:06:46.124 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/gpio_mouse.ko 2020-11-20T12:06:46.124 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 337393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/psmouse.ko 2020-11-20T12:06:46.125 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/sermouse.ko 2020-11-20T12:06:46.125 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/synaptics_i2c.ko 2020-11-20T12:06:46.125 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/synaptics_usb.ko 2020-11-20T12:06:46.126 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/vsxxxaa.ko 2020-11-20T12:06:46.126 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/rmi4/ 2020-11-20T12:06:46.126 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 113449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/rmi4/rmi_core.ko 2020-11-20T12:06:46.127 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ 2020-11-20T12:06:46.127 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/altera_ps2.ko 2020-11-20T12:06:46.128 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/arc_ps2.ko 2020-11-20T12:06:46.128 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ct82c710.ko 2020-11-20T12:06:46.128 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/hyperv-keyboard.ko 2020-11-20T12:06:46.128 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/parkbd.ko 2020-11-20T12:06:46.128 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/pcips2.ko 2020-11-20T12:06:46.129 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ps2mult.ko 2020-11-20T12:06:46.131 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/serio_raw.ko 2020-11-20T12:06:46.131 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/serport.ko 2020-11-20T12:06:46.131 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/sparse-keymap.ko 2020-11-20T12:06:46.131 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/ 2020-11-20T12:06:46.132 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/amd/ 2020-11-20T12:06:46.132 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/amd/iommu_v2.ko 2020-11-20T12:06:46.132 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/ 2020-11-20T12:06:46.132 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-88pm860x.ko 2020-11-20T12:06:46.132 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-adp5520.ko 2020-11-20T12:06:46.133 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-bd2802.ko 2020-11-20T12:06:46.133 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-blinkm.ko 2020-11-20T12:06:46.133 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-clevo-mail.ko 2020-11-20T12:06:46.133 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-da903x.ko 2020-11-20T12:06:46.133 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-da9052.ko 2020-11-20T12:06:46.134 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-gpio.ko 2020-11-20T12:06:46.134 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3530.ko 2020-11-20T12:06:46.135 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3533.ko 2020-11-20T12:06:46.135 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm355x.ko 2020-11-20T12:06:46.135 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3642.ko 2020-11-20T12:06:46.135 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lp3944.ko 2020-11-20T12:06:46.135 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lp8788.ko 2020-11-20T12:06:46.137 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-max8997.ko 2020-11-20T12:06:46.138 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-mc13783.ko 2020-11-20T12:06:46.138 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-menf21bmc.ko 2020-11-20T12:06:46.138 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca9532.ko 2020-11-20T12:06:46.138 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca955x.ko 2020-11-20T12:06:46.139 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca963x.ko 2020-11-20T12:06:46.139 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-ss4200.ko 2020-11-20T12:06:46.139 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-tca6507.ko 2020-11-20T12:06:46.139 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-wm831x-status.ko 2020-11-20T12:06:46.140 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-wm8350.ko 2020-11-20T12:06:46.140 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ 2020-11-20T12:06:46.140 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-audio.ko 2020-11-20T12:06:46.140 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2020-11-20T12:06:46.140 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-camera.ko 2020-11-20T12:06:46.141 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2020-11-20T12:06:46.141 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2020-11-20T12:06:46.141 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2020-11-20T12:06:46.141 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2020-11-20T12:06:46.142 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-timer.ko 2020-11-20T12:06:46.142 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-transient.ko 2020-11-20T12:06:46.142 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/ 2020-11-20T12:06:46.142 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/bcache/ 2020-11-20T12:06:46.142 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 633113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/bcache/bcache.ko 2020-11-20T12:06:46.143 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-bio-prison.ko 2020-11-20T12:06:46.143 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-bufio.ko 2020-11-20T12:06:46.144 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-cache-smq.ko 2020-11-20T12:06:46.144 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 146689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-cache.ko 2020-11-20T12:06:46.145 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 98065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-crypt.ko 2020-11-20T12:06:46.147 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-delay.ko 2020-11-20T12:06:46.147 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-era.ko 2020-11-20T12:06:46.148 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-flakey.ko 2020-11-20T12:06:46.148 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-log-userspace.ko 2020-11-20T12:06:46.149 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-log.ko 2020-11-20T12:06:46.149 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-mirror.ko 2020-11-20T12:06:46.149 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 75609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-multipath.ko 2020-11-20T12:06:46.150 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-queue-length.ko 2020-11-20T12:06:46.150 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-raid.ko 2020-11-20T12:06:46.151 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-region-hash.ko 2020-11-20T12:06:46.151 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-round-robin.ko 2020-11-20T12:06:46.152 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-service-time.ko 2020-11-20T12:06:46.152 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 112913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-snapshot.ko 2020-11-20T12:06:46.153 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-switch.ko 2020-11-20T12:06:46.153 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 182441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-thin-pool.ko 2020-11-20T12:06:46.155 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-verity.ko 2020-11-20T12:06:46.156 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-zero.ko 2020-11-20T12:06:46.157 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/faulty.ko 2020-11-20T12:06:46.157 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/linear.ko 2020-11-20T12:06:46.157 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/multipath.ko 2020-11-20T12:06:46.158 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/persistent-data/ 2020-11-20T12:06:46.158 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 179553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2020-11-20T12:06:46.159 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid0.ko 2020-11-20T12:06:46.159 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 111753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid1.ko 2020-11-20T12:06:46.161 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 156737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid10.ko 2020-11-20T12:06:46.163 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 385993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid456.ko 2020-11-20T12:06:46.168 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/ 2020-11-20T12:06:46.169 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/ 2020-11-20T12:06:46.169 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 194137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptbase.ko 2020-11-20T12:06:46.171 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptctl.ko 2020-11-20T12:06:46.172 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptfc.ko 2020-11-20T12:06:46.172 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptlan.ko 2020-11-20T12:06:46.173 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 129617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptsas.ko 2020-11-20T12:06:46.174 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 90705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptscsih.ko 2020-11-20T12:06:46.175 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptspi.ko 2020-11-20T12:06:46.177 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ 2020-11-20T12:06:46.177 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm800.ko 2020-11-20T12:06:46.177 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm805.ko 2020-11-20T12:06:46.177 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm80x.ko 2020-11-20T12:06:46.178 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ab3100-otp.ko 2020-11-20T12:06:46.178 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/arizona-i2c.ko 2020-11-20T12:06:46.178 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/bcm590xx.ko 2020-11-20T12:06:46.178 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/cros_ec_dev.ko 2020-11-20T12:06:46.180 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/dln2.ko 2020-11-20T12:06:46.180 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/htc-pasic3.ko 2020-11-20T12:06:46.181 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/janz-cmodio.ko 2020-11-20T12:06:46.181 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/kempld-core.ko 2020-11-20T12:06:46.181 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lm3533-core.ko 2020-11-20T12:06:46.181 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lm3533-ctrlbank.ko 2020-11-20T12:06:46.181 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lp3943.ko 2020-11-20T12:06:46.182 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lpc_ich.ko 2020-11-20T12:06:46.182 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lpc_sch.ko 2020-11-20T12:06:46.182 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/max8907.ko 2020-11-20T12:06:46.182 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/mc13xxx-core.ko 2020-11-20T12:06:46.183 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/mc13xxx-i2c.ko 2020-11-20T12:06:46.183 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/menf21bmc.ko 2020-11-20T12:06:46.183 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633-adc.ko 2020-11-20T12:06:46.183 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633-gpio.ko 2020-11-20T12:06:46.183 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633.ko 2020-11-20T12:06:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/rdc321x-southbridge.ko 2020-11-20T12:06:46.185 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/retu-mfd.ko 2020-11-20T12:06:46.186 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 76449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/si476x-core.ko 2020-11-20T12:06:46.186 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/sm501.ko 2020-11-20T12:06:46.186 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ti_am335x_tscadc.ko 2020-11-20T12:06:46.186 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/tps65010.ko 2020-11-20T12:06:46.187 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/tps6507x.ko 2020-11-20T12:06:46.187 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/viperboard.ko 2020-11-20T12:06:46.187 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/vx855.ko 2020-11-20T12:06:46.187 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/wl1273-core.ko 2020-11-20T12:06:46.188 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/wm8994.ko 2020-11-20T12:06:46.188 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ 2020-11-20T12:06:46.188 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ad525x_dpot-i2c.ko 2020-11-20T12:06:46.188 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ad525x_dpot.ko 2020-11-20T12:06:46.188 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/altera-stapl/ 2020-11-20T12:06:46.189 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2020-11-20T12:06:46.189 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/apds9802als.ko 2020-11-20T12:06:46.189 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/apds990x.ko 2020-11-20T12:06:46.191 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/bh1770glc.ko 2020-11-20T12:06:46.191 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/ 2020-11-20T12:06:46.191 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2020-11-20T12:06:46.192 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/core.ko 2020-11-20T12:06:46.192 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/cb710/ 2020-11-20T12:06:46.192 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/cb710/cb710.ko 2020-11-20T12:06:46.192 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ds1682.ko 2020-11-20T12:06:46.193 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/dummy-irq.ko 2020-11-20T12:06:46.193 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/echo/ 2020-11-20T12:06:46.193 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/echo/echo.ko 2020-11-20T12:06:46.193 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/ 2020-11-20T12:06:46.194 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/at24.ko 2020-11-20T12:06:46.194 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/eeprom.ko 2020-11-20T12:06:46.194 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2020-11-20T12:06:46.194 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/max6875.ko 2020-11-20T12:06:46.194 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/enclosure.ko 2020-11-20T12:06:46.195 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/genwqe/ 2020-11-20T12:06:46.195 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 151177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/genwqe/genwqe_card.ko 2020-11-20T12:06:46.195 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/hmc6352.ko 2020-11-20T12:06:46.195 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/hpilo.ko 2020-11-20T12:06:46.195 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ibmasm/ 2020-11-20T12:06:46.196 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 71465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ibmasm/ibmasm.ko 2020-11-20T12:06:46.197 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ics932s401.ko 2020-11-20T12:06:46.197 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/isl29003.ko 2020-11-20T12:06:46.197 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/isl29020.ko 2020-11-20T12:06:46.198 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/ 2020-11-20T12:06:46.198 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2020-11-20T12:06:46.198 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2020-11-20T12:06:46.198 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/ 2020-11-20T12:06:46.198 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 121553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei-me.ko 2020-11-20T12:06:46.199 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei-txe.ko 2020-11-20T12:06:46.199 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 279201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei.ko 2020-11-20T12:06:46.200 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/phantom.ko 2020-11-20T12:06:46.200 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/pvpanic.ko 2020-11-20T12:06:46.200 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ti-st/ 2020-11-20T12:06:46.200 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 74905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ti-st/st_drv.ko 2020-11-20T12:06:46.201 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tifm_7xx1.ko 2020-11-20T12:06:46.202 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tifm_core.ko 2020-11-20T12:06:46.202 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tsl2550.ko 2020-11-20T12:06:46.202 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_balloon.ko 2020-11-20T12:06:46.202 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_vmci/ 2020-11-20T12:06:46.202 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 179385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2020-11-20T12:06:46.204 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ 2020-11-20T12:06:46.204 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/bonding/ 2020-11-20T12:06:46.204 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 433953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/bonding/bonding.ko 2020-11-20T12:06:46.209 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/ 2020-11-20T12:06:46.210 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/b53/ 2020-11-20T12:06:46.210 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 96929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/b53/b53_common.ko 2020-11-20T12:06:46.211 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/bcm-sf2.ko 2020-11-20T12:06:46.211 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6060.ko 2020-11-20T12:06:46.212 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6xxx/ 2020-11-20T12:06:46.212 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 346377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2020-11-20T12:06:46.215 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dummy.ko 2020-11-20T12:06:46.215 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/eql.ko 2020-11-20T12:06:46.216 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ 2020-11-20T12:06:46.216 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/ 2020-11-20T12:06:46.217 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2020-11-20T12:06:46.217 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2020-11-20T12:06:46.217 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 123353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c59x.ko 2020-11-20T12:06:46.218 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/typhoon.ko 2020-11-20T12:06:46.219 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/ 2020-11-20T12:06:46.219 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/8390.ko 2020-11-20T12:06:46.220 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2020-11-20T12:06:46.220 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2020-11-20T12:06:46.221 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 96881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2020-11-20T12:06:46.222 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/adaptec/ 2020-11-20T12:06:46.222 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/adaptec/starfire.ko 2020-11-20T12:06:46.223 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/agere/ 2020-11-20T12:06:46.223 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/agere/et131x.ko 2020-11-20T12:06:46.223 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/alteon/ 2020-11-20T12:06:46.224 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 58761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/alteon/acenic.ko 2020-11-20T12:06:46.224 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/altera/ 2020-11-20T12:06:46.224 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/altera/altera_tse.ko 2020-11-20T12:06:46.225 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/ 2020-11-20T12:06:46.225 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/amd8111e.ko 2020-11-20T12:06:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2020-11-20T12:06:46.226 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 90281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/pcnet32.ko 2020-11-20T12:06:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/ 2020-11-20T12:06:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/alx/ 2020-11-20T12:06:46.227 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2020-11-20T12:06:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1c/ 2020-11-20T12:06:46.228 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 98001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2020-11-20T12:06:46.229 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1e/ 2020-11-20T12:06:46.230 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 89785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2020-11-20T12:06:46.231 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/ 2020-11-20T12:06:46.231 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 86737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2020-11-20T12:06:46.232 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2020-11-20T12:06:46.232 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/ 2020-11-20T12:06:46.233 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/b44.ko 2020-11-20T12:06:46.233 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 167025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2020-11-20T12:06:46.236 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2020-11-20T12:06:46.236 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1528577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2020-11-20T12:06:46.257 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 130113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/cnic.ko 2020-11-20T12:06:46.259 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/genet/ 2020-11-20T12:06:46.260 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 129345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2020-11-20T12:06:46.261 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 314625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/tg3.ko 2020-11-20T12:06:46.266 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/ 2020-11-20T12:06:46.266 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/bna/ 2020-11-20T12:06:46.266 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 330209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2020-11-20T12:06:46.270 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/ 2020-11-20T12:06:46.271 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb/ 2020-11-20T12:06:46.271 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 187345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2020-11-20T12:06:46.272 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2020-11-20T12:06:46.273 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 349953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2020-11-20T12:06:46.277 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2020-11-20T12:06:46.278 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1039081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2020-11-20T12:06:46.291 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2020-11-20T12:06:46.291 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 165321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2020-11-20T12:06:46.292 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2020-11-20T12:06:46.293 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2020-11-20T12:06:46.293 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/ 2020-11-20T12:06:46.294 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/enic/ 2020-11-20T12:06:46.294 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 205121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2020-11-20T12:06:46.295 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/ 2020-11-20T12:06:46.296 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/ 2020-11-20T12:06:46.296 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2020-11-20T12:06:46.297 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 97801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/de4x5.ko 2020-11-20T12:06:46.298 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2020-11-20T12:06:46.298 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 147153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2020-11-20T12:06:46.300 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2020-11-20T12:06:46.301 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2020-11-20T12:06:46.301 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2020-11-20T12:06:46.302 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/ 2020-11-20T12:06:46.302 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/dl2k.ko 2020-11-20T12:06:46.303 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/sundance.ko 2020-11-20T12:06:46.303 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dnet.ko 2020-11-20T12:06:46.304 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ec_bhf.ko 2020-11-20T12:06:46.304 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/ 2020-11-20T12:06:46.304 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/benet/ 2020-11-20T12:06:46.305 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 287161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2020-11-20T12:06:46.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ethoc.ko 2020-11-20T12:06:46.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fealnx.ko 2020-11-20T12:06:46.309 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fujitsu/ 2020-11-20T12:06:46.309 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2020-11-20T12:06:46.310 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ 2020-11-20T12:06:46.310 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e100.ko 2020-11-20T12:06:46.311 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000/ 2020-11-20T12:06:46.311 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 293081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2020-11-20T12:06:46.314 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000e/ 2020-11-20T12:06:46.314 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 549313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2020-11-20T12:06:46.321 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/fm10k/ 2020-11-20T12:06:46.321 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 247433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2020-11-20T12:06:46.324 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/i40e/ 2020-11-20T12:06:46.325 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 892665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2020-11-20T12:06:46.336 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/iavf/ 2020-11-20T12:06:46.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 250857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2020-11-20T12:06:46.339 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igb/ 2020-11-20T12:06:46.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 509433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igb/igb.ko 2020-11-20T12:06:46.345 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igbvf/ 2020-11-20T12:06:46.346 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 90673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2020-11-20T12:06:46.347 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgb/ 2020-11-20T12:06:46.347 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 123905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgb/ixgb.ko 2020-11-20T12:06:46.348 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbe/ 2020-11-20T12:06:46.348 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 743497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2020-11-20T12:06:46.357 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbevf/ 2020-11-20T12:06:46.358 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 147065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2020-11-20T12:06:46.359 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 82449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/jme.ko 2020-11-20T12:06:46.360 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/ 2020-11-20T12:06:46.360 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2020-11-20T12:06:46.361 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 103057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/skge.ko 2020-11-20T12:06:46.362 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 130193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/sky2.ko 2020-11-20T12:06:46.363 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/ 2020-11-20T12:06:46.363 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/ 2020-11-20T12:06:46.364 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 739097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2020-11-20T12:06:46.373 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 273433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2020-11-20T12:06:46.377 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/ 2020-11-20T12:06:46.377 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2020-11-20T12:06:46.378 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 920849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2020-11-20T12:06:46.388 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ 2020-11-20T12:06:46.388 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ks8842.ko 2020-11-20T12:06:46.388 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ks8851_mll.ko 2020-11-20T12:06:46.389 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 94705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2020-11-20T12:06:46.390 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/ 2020-11-20T12:06:46.390 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/myri10ge/ 2020-11-20T12:06:46.391 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 112513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2020-11-20T12:06:46.391 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/ 2020-11-20T12:06:46.392 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 75729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2020-11-20T12:06:46.392 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2020-11-20T12:06:46.393 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/ 2020-11-20T12:06:46.393 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 156657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/s2io.ko 2020-11-20T12:06:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/vxge/ 2020-11-20T12:06:46.395 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 221913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/vxge/vxge.ko 2020-11-20T12:06:46.398 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/nvidia/ 2020-11-20T12:06:46.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 142057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2020-11-20T12:06:46.400 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/ 2020-11-20T12:06:46.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2020-11-20T12:06:46.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2020-11-20T12:06:46.401 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/ 2020-11-20T12:06:46.401 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/netxen/ 2020-11-20T12:06:46.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 242793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2020-11-20T12:06:46.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 88409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2020-11-20T12:06:46.405 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2020-11-20T12:06:46.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 706337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2020-11-20T12:06:46.414 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/rdc/ 2020-11-20T12:06:46.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/rdc/r6040.ko 2020-11-20T12:06:46.415 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/ 2020-11-20T12:06:46.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/8139cp.ko 2020-11-20T12:06:46.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 83753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/8139too.ko 2020-11-20T12:06:46.416 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/atp.ko 2020-11-20T12:06:46.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 173241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/r8169.ko 2020-11-20T12:06:46.419 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/ 2020-11-20T12:06:46.419 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/sxgbe/ 2020-11-20T12:06:46.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 104689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2020-11-20T12:06:46.420 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sfc/ 2020-11-20T12:06:46.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 824569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sfc/sfc.ko 2020-11-20T12:06:46.431 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/silan/ 2020-11-20T12:06:46.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/silan/sc92031.ko 2020-11-20T12:06:46.432 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/ 2020-11-20T12:06:46.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/sis190.ko 2020-11-20T12:06:46.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 64625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/sis900.ko 2020-11-20T12:06:46.433 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/ 2020-11-20T12:06:46.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/epic100.ko 2020-11-20T12:06:46.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2020-11-20T12:06:46.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 74569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2020-11-20T12:06:46.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2020-11-20T12:06:46.436 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/ 2020-11-20T12:06:46.436 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/ 2020-11-20T12:06:46.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2020-11-20T12:06:46.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2020-11-20T12:06:46.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2020-11-20T12:06:46.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 381753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2020-11-20T12:06:46.441 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/ 2020-11-20T12:06:46.441 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 106913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/cassini.ko 2020-11-20T12:06:46.442 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 176753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/niu.ko 2020-11-20T12:06:46.445 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/sungem.ko 2020-11-20T12:06:46.446 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/sunhme.ko 2020-11-20T12:06:46.447 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/tehuti/ 2020-11-20T12:06:46.447 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2020-11-20T12:06:46.447 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ti/ 2020-11-20T12:06:46.448 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ti/tlan.ko 2020-11-20T12:06:46.448 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/ 2020-11-20T12:06:46.448 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 82449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/via-rhine.ko 2020-11-20T12:06:46.449 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 86073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/via-velocity.ko 2020-11-20T12:06:46.450 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/ 2020-11-20T12:06:46.450 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/w5100.ko 2020-11-20T12:06:46.451 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/w5300.ko 2020-11-20T12:06:46.451 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/xircom/ 2020-11-20T12:06:46.451 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 58281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2020-11-20T12:06:46.452 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/ 2020-11-20T12:06:46.452 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/defxx.ko 2020-11-20T12:06:46.453 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/skfp/ 2020-11-20T12:06:46.453 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 236713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/skfp/skfp.ko 2020-11-20T12:06:46.455 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/geneve.ko 2020-11-20T12:06:46.456 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/hyperv/ 2020-11-20T12:06:46.456 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 227313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/hyperv/hv_netvsc.ko 2020-11-20T12:06:46.459 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ieee802154/ 2020-11-20T12:06:46.459 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ieee802154/fakelb.ko 2020-11-20T12:06:46.460 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ifb.ko 2020-11-20T12:06:46.460 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ipvlan/ 2020-11-20T12:06:46.461 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 84633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ipvlan/ipvlan.ko 2020-11-20T12:06:46.461 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/macvlan.ko 2020-11-20T12:06:46.462 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/macvtap.ko 2020-11-20T12:06:46.462 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/ 2020-11-20T12:06:46.462 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2020-11-20T12:06:46.463 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-bitbang.ko 2020-11-20T12:06:46.463 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-gpio.ko 2020-11-20T12:06:46.463 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio.ko 2020-11-20T12:06:46.463 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mii.ko 2020-11-20T12:06:46.463 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/netconsole.ko 2020-11-20T12:06:46.465 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/nlmon.ko 2020-11-20T12:06:46.466 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/pcs/ 2020-11-20T12:06:46.466 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/pcs/pcs-xpcs.ko 2020-11-20T12:06:46.466 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/ 2020-11-20T12:06:46.466 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/amd.ko 2020-11-20T12:06:46.467 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm-phy-lib.ko 2020-11-20T12:06:46.467 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm7xxx.ko 2020-11-20T12:06:46.467 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm87xx.ko 2020-11-20T12:06:46.467 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/broadcom.ko 2020-11-20T12:06:46.467 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/cicada.ko 2020-11-20T12:06:46.468 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/davicom.ko 2020-11-20T12:06:46.468 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/et1011c.ko 2020-11-20T12:06:46.468 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/icplus.ko 2020-11-20T12:06:46.468 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/lxt.ko 2020-11-20T12:06:46.469 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 64985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/marvell.ko 2020-11-20T12:06:46.469 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/micrel.ko 2020-11-20T12:06:46.469 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/national.ko 2020-11-20T12:06:46.469 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/phylink.ko 2020-11-20T12:06:46.471 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/qsemi.ko 2020-11-20T12:06:46.471 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/realtek.ko 2020-11-20T12:06:46.472 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/smsc.ko 2020-11-20T12:06:46.472 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/ste10Xp.ko 2020-11-20T12:06:46.472 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/vitesse.ko 2020-11-20T12:06:46.472 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/plip/ 2020-11-20T12:06:46.473 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/plip/plip.ko 2020-11-20T12:06:46.473 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ 2020-11-20T12:06:46.473 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/bsd_comp.ko 2020-11-20T12:06:46.473 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_async.ko 2020-11-20T12:06:46.473 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_deflate.ko 2020-11-20T12:06:46.474 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_mppe.ko 2020-11-20T12:06:46.474 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_synctty.ko 2020-11-20T12:06:46.474 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pppoe.ko 2020-11-20T12:06:46.474 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pppox.ko 2020-11-20T12:06:46.475 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pptp.ko 2020-11-20T12:06:46.475 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/sb1000.ko 2020-11-20T12:06:46.475 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/slip/ 2020-11-20T12:06:46.475 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/slip/slip.ko 2020-11-20T12:06:46.475 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/sungem_phy.ko 2020-11-20T12:06:46.476 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/tap.ko 2020-11-20T12:06:46.477 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/ 2020-11-20T12:06:46.477 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 102721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team.ko 2020-11-20T12:06:46.477 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_activebackup.ko 2020-11-20T12:06:46.478 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_broadcast.ko 2020-11-20T12:06:46.478 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_loadbalance.ko 2020-11-20T12:06:46.478 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_random.ko 2020-11-20T12:06:46.478 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_roundrobin.ko 2020-11-20T12:06:46.478 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/veth.ko 2020-11-20T12:06:46.479 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vmxnet3/ 2020-11-20T12:06:46.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 116905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vmxnet3/vmxnet3.ko 2020-11-20T12:06:46.479 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 171137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vxlan.ko 2020-11-20T12:06:46.480 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/xen-netback/ 2020-11-20T12:06:46.480 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 143721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/xen-netback/xen-netback.ko 2020-11-20T12:06:46.481 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ntb/ 2020-11-20T12:06:46.481 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ntb/ntb.ko 2020-11-20T12:06:46.481 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/ 2020-11-20T12:06:46.481 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/ 2020-11-20T12:06:46.482 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 209665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/nvme-core.ko 2020-11-20T12:06:46.483 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 92745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/nvme.ko 2020-11-20T12:06:46.484 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/ 2020-11-20T12:06:46.484 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 130825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport.ko 2020-11-20T12:06:46.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_ax88796.ko 2020-11-20T12:06:46.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_cs.ko 2020-11-20T12:06:46.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 84865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_pc.ko 2020-11-20T12:06:46.487 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_serial.ko 2020-11-20T12:06:46.487 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/ 2020-11-20T12:06:46.488 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/ 2020-11-20T12:06:46.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2020-11-20T12:06:46.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2020-11-20T12:06:46.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2020-11-20T12:06:46.489 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/pci-stub.ko 2020-11-20T12:06:46.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/xen-pcifront.ko 2020-11-20T12:06:46.490 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/ 2020-11-20T12:06:46.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/i82092.ko 2020-11-20T12:06:46.490 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 150113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia.ko 2020-11-20T12:06:46.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia_core.ko 2020-11-20T12:06:46.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2020-11-20T12:06:46.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pd6729.ko 2020-11-20T12:06:46.492 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 111129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/yenta_socket.ko 2020-11-20T12:06:46.493 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/ 2020-11-20T12:06:46.494 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/broadcom/ 2020-11-20T12:06:46.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2020-11-20T12:06:46.494 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/samsung/ 2020-11-20T12:06:46.494 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2020-11-20T12:06:46.495 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/ 2020-11-20T12:06:46.495 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/intel/ 2020-11-20T12:06:46.495 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2020-11-20T12:06:46.496 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/ 2020-11-20T12:06:46.496 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/ 2020-11-20T12:06:46.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2020-11-20T12:06:46.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec.ko 2020-11-20T12:06:46.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2020-11-20T12:06:46.497 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2020-11-20T12:06:46.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2020-11-20T12:06:46.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2020-11-20T12:06:46.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2020-11-20T12:06:46.498 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/ 2020-11-20T12:06:46.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/acer-wmi.ko 2020-11-20T12:06:46.500 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/acerhdf.ko 2020-11-20T12:06:46.500 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/alienware-wmi.ko 2020-11-20T12:06:46.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/apple-gmux.ko 2020-11-20T12:06:46.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-laptop.ko 2020-11-20T12:06:46.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-nb-wmi.ko 2020-11-20T12:06:46.501 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 71465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-wmi.ko 2020-11-20T12:06:46.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/classmate-laptop.ko 2020-11-20T12:06:46.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dcdbas.ko 2020-11-20T12:06:46.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell-smo8800.ko 2020-11-20T12:06:46.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell-wmi-aio.ko 2020-11-20T12:06:46.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell_rbu.ko 2020-11-20T12:06:46.504 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/eeepc-laptop.ko 2020-11-20T12:06:46.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/eeepc-wmi.ko 2020-11-20T12:06:46.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/fujitsu-laptop.ko 2020-11-20T12:06:46.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/fujitsu-tablet.ko 2020-11-20T12:06:46.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hdaps.ko 2020-11-20T12:06:46.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp-wireless.ko 2020-11-20T12:06:46.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp-wmi.ko 2020-11-20T12:06:46.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp_accel.ko 2020-11-20T12:06:46.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/ibm_rtl.ko 2020-11-20T12:06:46.506 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel-rst.ko 2020-11-20T12:06:46.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel-smartconnect.ko 2020-11-20T12:06:46.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel_ips.ko 2020-11-20T12:06:46.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel_menlow.ko 2020-11-20T12:06:46.507 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/msi-wmi.ko 2020-11-20T12:06:46.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/mxm-wmi.ko 2020-11-20T12:06:46.508 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/panasonic-laptop.ko 2020-11-20T12:06:46.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/samsung-laptop.ko 2020-11-20T12:06:46.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/samsung-q10.ko 2020-11-20T12:06:46.509 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 260993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/thinkpad_acpi.ko 2020-11-20T12:06:46.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/topstar-laptop.ko 2020-11-20T12:06:46.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2020-11-20T12:06:46.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/toshiba_haps.ko 2020-11-20T12:06:46.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/wmi-bmof.ko 2020-11-20T12:06:46.511 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/wmi.ko 2020-11-20T12:06:46.511 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/ 2020-11-20T12:06:46.512 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ 2020-11-20T12:06:46.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/88pm860x_battery.ko 2020-11-20T12:06:46.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/88pm860x_charger.ko 2020-11-20T12:06:46.512 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq2415x_charger.ko 2020-11-20T12:06:46.513 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq24190_charger.ko 2020-11-20T12:06:46.514 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq24735-charger.ko 2020-11-20T12:06:46.514 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/da9030_battery.ko 2020-11-20T12:06:46.515 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/da9052-battery.ko 2020-11-20T12:06:46.515 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2760_battery.ko 2020-11-20T12:06:46.515 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2780_battery.ko 2020-11-20T12:06:46.515 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2781_battery.ko 2020-11-20T12:06:46.515 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2782_battery.ko 2020-11-20T12:06:46.516 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/gpio-charger.ko 2020-11-20T12:06:46.516 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/isp1704_charger.ko 2020-11-20T12:06:46.516 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/lp8727_charger.ko 2020-11-20T12:06:46.516 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max14577_charger.ko 2020-11-20T12:06:46.518 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max17040_battery.ko 2020-11-20T12:06:46.518 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max17042_battery.ko 2020-11-20T12:06:46.519 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max8903_charger.ko 2020-11-20T12:06:46.519 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max8925_power.ko 2020-11-20T12:06:46.519 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/pcf50633-charger.ko 2020-11-20T12:06:46.519 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/pda_power.ko 2020-11-20T12:06:46.519 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/sbs-battery.ko 2020-11-20T12:06:46.520 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/smb347-charger.ko 2020-11-20T12:06:46.520 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/test_power.ko 2020-11-20T12:06:46.520 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/tps65090-charger.ko 2020-11-20T12:06:46.520 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm831x_backup.ko 2020-11-20T12:06:46.521 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm831x_power.ko 2020-11-20T12:06:46.521 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm8350_power.ko 2020-11-20T12:06:46.521 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/ 2020-11-20T12:06:46.521 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/intel_rapl_common.ko 2020-11-20T12:06:46.524 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/intel_rapl_msr.ko 2020-11-20T12:06:46.524 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/ 2020-11-20T12:06:46.524 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/ 2020-11-20T12:06:46.525 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps-gpio.ko 2020-11-20T12:06:46.525 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps-ldisc.ko 2020-11-20T12:06:46.525 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps_parport.ko 2020-11-20T12:06:46.525 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/pps_core.ko 2020-11-20T12:06:46.526 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ 2020-11-20T12:06:46.526 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ptp.ko 2020-11-20T12:06:46.526 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ptp_kvm.ko 2020-11-20T12:06:46.526 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/ 2020-11-20T12:06:46.526 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-88pm80x.ko 2020-11-20T12:06:46.527 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-88pm860x.ko 2020-11-20T12:06:46.527 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ab3100.ko 2020-11-20T12:06:46.527 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-bq32k.ko 2020-11-20T12:06:46.527 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-bq4802.ko 2020-11-20T12:06:46.528 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9052.ko 2020-11-20T12:06:46.528 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9055.ko 2020-11-20T12:06:46.528 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9063.ko 2020-11-20T12:06:46.528 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1286.ko 2020-11-20T12:06:46.528 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1307.ko 2020-11-20T12:06:46.529 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1374.ko 2020-11-20T12:06:46.529 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1511.ko 2020-11-20T12:06:46.529 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1553.ko 2020-11-20T12:06:46.529 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1672.ko 2020-11-20T12:06:46.530 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1742.ko 2020-11-20T12:06:46.530 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds2404.ko 2020-11-20T12:06:46.530 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds3232.ko 2020-11-20T12:06:46.530 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-em3027.ko 2020-11-20T12:06:46.530 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-fm3130.ko 2020-11-20T12:06:46.534 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-isl12022.ko 2020-11-20T12:06:46.534 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-isl1208.ko 2020-11-20T12:06:46.535 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-lp8788.ko 2020-11-20T12:06:46.535 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m41t80.ko 2020-11-20T12:06:46.535 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t35.ko 2020-11-20T12:06:46.535 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t59.ko 2020-11-20T12:06:46.535 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t86.ko 2020-11-20T12:06:46.536 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max6900.ko 2020-11-20T12:06:46.536 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8907.ko 2020-11-20T12:06:46.536 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8925.ko 2020-11-20T12:06:46.536 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8997.ko 2020-11-20T12:06:46.537 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8998.ko 2020-11-20T12:06:46.537 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-mc13xxx.ko 2020-11-20T12:06:46.537 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-msm6242.ko 2020-11-20T12:06:46.537 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-palmas.ko 2020-11-20T12:06:46.538 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf2127.ko 2020-11-20T12:06:46.538 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf50633.ko 2020-11-20T12:06:46.538 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf85063.ko 2020-11-20T12:06:46.538 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8523.ko 2020-11-20T12:06:46.538 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8563.ko 2020-11-20T12:06:46.539 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8583.ko 2020-11-20T12:06:46.539 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rc5t583.ko 2020-11-20T12:06:46.539 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rp5c01.ko 2020-11-20T12:06:46.539 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rs5c372.ko 2020-11-20T12:06:46.540 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rv3029c2.ko 2020-11-20T12:06:46.540 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rx8025.ko 2020-11-20T12:06:46.540 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rx8581.ko 2020-11-20T12:06:46.540 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-s35390a.ko 2020-11-20T12:06:46.540 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-s5m.ko 2020-11-20T12:06:46.541 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-stk17ta8.ko 2020-11-20T12:06:46.541 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps6586x.ko 2020-11-20T12:06:46.541 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps65910.ko 2020-11-20T12:06:46.541 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps80031.ko 2020-11-20T12:06:46.542 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-v3020.ko 2020-11-20T12:06:46.542 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-wm831x.ko 2020-11-20T12:06:46.542 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-wm8350.ko 2020-11-20T12:06:46.542 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-x1205.ko 2020-11-20T12:06:46.542 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ 2020-11-20T12:06:46.543 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 80673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-9xxx.ko 2020-11-20T12:06:46.543 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-sas.ko 2020-11-20T12:06:46.543 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-xxxx.ko 2020-11-20T12:06:46.543 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 132001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/BusLogic.ko 2020-11-20T12:06:46.544 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/a100u2w.ko 2020-11-20T12:06:46.544 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aacraid/ 2020-11-20T12:06:46.545 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 286449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aacraid/aacraid.ko 2020-11-20T12:06:46.545 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 101329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/advansys.ko 2020-11-20T12:06:46.545 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/ 2020-11-20T12:06:46.545 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 346097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2020-11-20T12:06:46.547 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 297009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2020-11-20T12:06:46.551 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic94xx/ 2020-11-20T12:06:46.551 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 194297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic94xx/aic94xx.ko 2020-11-20T12:06:46.553 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/am53c974.ko 2020-11-20T12:06:46.554 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/arcmsr/ 2020-11-20T12:06:46.554 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 116369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/arcmsr/arcmsr.ko 2020-11-20T12:06:46.555 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/atp870u.ko 2020-11-20T12:06:46.556 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/be2iscsi/ 2020-11-20T12:06:46.556 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 265265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2020-11-20T12:06:46.560 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bfa/ 2020-11-20T12:06:46.560 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1164361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bfa/bfa.ko 2020-11-20T12:06:46.573 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2fc/ 2020-11-20T12:06:46.573 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 252833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2020-11-20T12:06:46.576 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2i/ 2020-11-20T12:06:46.576 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 119577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2i/bnx2i.ko 2020-11-20T12:06:46.578 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ch.ko 2020-11-20T12:06:46.578 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/csiostor/ 2020-11-20T12:06:46.578 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 312289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/csiostor/csiostor.ko 2020-11-20T12:06:46.582 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/ 2020-11-20T12:06:46.583 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb3i/ 2020-11-20T12:06:46.583 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 80249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2020-11-20T12:06:46.583 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb4i/ 2020-11-20T12:06:46.584 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 110649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2020-11-20T12:06:46.585 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 153137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2020-11-20T12:06:46.586 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 76961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dc395x.ko 2020-11-20T12:06:46.588 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dmx3191d.ko 2020-11-20T12:06:46.588 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 79977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dpt_i2o.ko 2020-11-20T12:06:46.589 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esas2r/ 2020-11-20T12:06:46.589 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 213561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esas2r/esas2r.ko 2020-11-20T12:06:46.592 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esp_scsi.ko 2020-11-20T12:06:46.593 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/ 2020-11-20T12:06:46.593 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/fcoe.ko 2020-11-20T12:06:46.594 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 162897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/libfcoe.ko 2020-11-20T12:06:46.596 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fdomain.ko 2020-11-20T12:06:46.596 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fnic/ 2020-11-20T12:06:46.596 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 280281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fnic/fnic.ko 2020-11-20T12:06:46.599 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 107993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/gdth.ko 2020-11-20T12:06:46.601 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 215137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hpsa.ko 2020-11-20T12:06:46.604 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hptiop.ko 2020-11-20T12:06:46.605 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hv_storvsc.ko 2020-11-20T12:06:46.605 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/imm.ko 2020-11-20T12:06:46.606 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 48489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/initio.ko 2020-11-20T12:06:46.606 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 294193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ipr.ko 2020-11-20T12:06:46.610 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 110105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ips.ko 2020-11-20T12:06:46.612 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/isci/ 2020-11-20T12:06:46.612 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 335225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/isci/isci.ko 2020-11-20T12:06:46.616 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2020-11-20T12:06:46.616 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/iscsi_tcp.ko 2020-11-20T12:06:46.617 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libfc/ 2020-11-20T12:06:46.617 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 321337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libfc/libfc.ko 2020-11-20T12:06:46.621 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 144753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libiscsi.ko 2020-11-20T12:06:46.622 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libiscsi_tcp.ko 2020-11-20T12:06:46.623 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libsas/ 2020-11-20T12:06:46.623 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 199625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libsas/libsas.ko 2020-11-20T12:06:46.626 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/lpfc/ 2020-11-20T12:06:46.626 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 2047225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/lpfc/lpfc.ko 2020-11-20T12:06:46.652 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/ 2020-11-20T12:06:46.652 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 80961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2020-11-20T12:06:46.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2020-11-20T12:06:46.653 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 334937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2020-11-20T12:06:46.658 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid.ko 2020-11-20T12:06:46.659 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mpt3sas/ 2020-11-20T12:06:46.659 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 652873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2020-11-20T12:06:46.667 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvsas/ 2020-11-20T12:06:46.667 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 122993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvsas/mvsas.ko 2020-11-20T12:06:46.668 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 82081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvumi.ko 2020-11-20T12:06:46.670 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/ 2020-11-20T12:06:46.670 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 81921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2020-11-20T12:06:46.671 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2020-11-20T12:06:46.671 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2020-11-20T12:06:46.671 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2020-11-20T12:06:46.672 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pm8001/ 2020-11-20T12:06:46.672 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 420593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pm8001/pm80xx.ko 2020-11-20T12:06:46.676 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 137097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pmcraid.ko 2020-11-20T12:06:46.678 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ppa.ko 2020-11-20T12:06:46.678 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 58657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla1280.ko 2020-11-20T12:06:46.679 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/ 2020-11-20T12:06:46.679 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1702009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2020-11-20T12:06:46.701 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 88025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2020-11-20T12:06:46.702 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla4xxx/ 2020-11-20T12:06:46.702 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 610673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2020-11-20T12:06:46.710 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qlogicfas408.ko 2020-11-20T12:06:46.710 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/raid_class.ko 2020-11-20T12:06:46.710 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 238049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_debug.ko 2020-11-20T12:06:46.713 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 142097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_fc.ko 2020-11-20T12:06:46.714 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 265049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_iscsi.ko 2020-11-20T12:06:46.717 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 95017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_sas.ko 2020-11-20T12:06:46.718 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 72697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_spi.ko 2020-11-20T12:06:46.719 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_srp.ko 2020-11-20T12:06:46.719 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ses.ko 2020-11-20T12:06:46.720 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 118033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/st.ko 2020-11-20T12:06:46.721 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/stex.ko 2020-11-20T12:06:46.722 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/sym53c8xx_2/ 2020-11-20T12:06:46.722 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 166257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2020-11-20T12:06:46.724 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ 2020-11-20T12:06:46.725 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 404305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-core.ko 2020-11-20T12:06:46.729 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-pci.ko 2020-11-20T12:06:46.729 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-pltfrm.ko 2020-11-20T12:06:46.730 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/virtio_scsi.ko 2020-11-20T12:06:46.730 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/vmw_pvscsi.ko 2020-11-20T12:06:46.730 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/wd719x.ko 2020-11-20T12:06:46.731 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/xen-scsifront.ko 2020-11-20T12:06:46.731 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ssb/ 2020-11-20T12:06:46.732 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 127201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ssb/ssb.ko 2020-11-20T12:06:46.733 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/ 2020-11-20T12:06:46.733 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/iscsi/ 2020-11-20T12:06:46.733 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 747873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2020-11-20T12:06:46.741 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/loopback/ 2020-11-20T12:06:46.741 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/loopback/tcm_loop.ko 2020-11-20T12:06:46.742 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_file.ko 2020-11-20T12:06:46.742 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_iblock.ko 2020-11-20T12:06:46.743 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 931017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_mod.ko 2020-11-20T12:06:46.753 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_pscsi.ko 2020-11-20T12:06:46.754 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 108177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_user.ko 2020-11-20T12:06:46.755 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/tcm_fc/ 2020-11-20T12:06:46.755 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 73449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/tcm_fc/tcm_fc.ko 2020-11-20T12:06:46.756 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/ 2020-11-20T12:06:46.756 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/ 2020-11-20T12:06:46.757 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/ 2020-11-20T12:06:46.757 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2020-11-20T12:06:46.757 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2020-11-20T12:06:46.759 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2020-11-20T12:06:46.759 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2020-11-20T12:06:46.759 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2020-11-20T12:06:46.759 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2020-11-20T12:06:46.759 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_powerclamp.ko 2020-11-20T12:06:46.760 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2020-11-20T12:06:46.760 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2020-11-20T12:06:46.760 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2020-11-20T12:06:46.760 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ 2020-11-20T12:06:46.761 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 88425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/cyclades.ko 2020-11-20T12:06:46.761 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ipwireless/ 2020-11-20T12:06:46.761 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 79297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ipwireless/ipwireless.ko 2020-11-20T12:06:46.762 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/isicom.ko 2020-11-20T12:06:46.762 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/moxa.ko 2020-11-20T12:06:46.762 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/mxser.ko 2020-11-20T12:06:46.763 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 88745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_gsm.ko 2020-11-20T12:06:46.763 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_hdlc.ko 2020-11-20T12:06:46.764 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_tracerouter.ko 2020-11-20T12:06:46.764 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_tracesink.ko 2020-11-20T12:06:46.764 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 71777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/nozomi.ko 2020-11-20T12:06:46.765 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/rocket.ko 2020-11-20T12:06:46.766 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/ 2020-11-20T12:06:46.766 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/ 2020-11-20T12:06:46.766 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/8250_dw.ko 2020-11-20T12:06:46.767 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/serial_cs.ko 2020-11-20T12:06:46.767 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/altera_jtaguart.ko 2020-11-20T12:06:46.768 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/altera_uart.ko 2020-11-20T12:06:46.768 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/arc_uart.ko 2020-11-20T12:06:46.768 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/fsl_lpuart.ko 2020-11-20T12:06:46.768 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/jsm/ 2020-11-20T12:06:46.769 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 106905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/jsm/jsm.ko 2020-11-20T12:06:46.769 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/rp2.ko 2020-11-20T12:06:46.770 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/sc16is7xx.ko 2020-11-20T12:06:46.770 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 119353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclink.ko 2020-11-20T12:06:46.772 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 109697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclink_gt.ko 2020-11-20T12:06:46.773 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 110737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclinkmp.ko 2020-11-20T12:06:46.775 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/ 2020-11-20T12:06:46.775 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 42305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio.ko 2020-11-20T12:06:46.776 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_aec.ko 2020-11-20T12:06:46.776 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_cif.ko 2020-11-20T12:06:46.776 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_dmem_genirq.ko 2020-11-20T12:06:46.777 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_mf624.ko 2020-11-20T12:06:46.777 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_netx.ko 2020-11-20T12:06:46.777 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_pci_generic.ko 2020-11-20T12:06:46.777 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_pdrv_genirq.ko 2020-11-20T12:06:46.778 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_sercos3.ko 2020-11-20T12:06:46.778 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/ 2020-11-20T12:06:46.778 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/c67x00/ 2020-11-20T12:06:46.778 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/c67x00/c67x00.ko 2020-11-20T12:06:46.779 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ 2020-11-20T12:06:46.779 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 120065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc.ko 2020-11-20T12:06:46.779 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2020-11-20T12:06:46.780 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2020-11-20T12:06:46.780 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2020-11-20T12:06:46.780 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/ 2020-11-20T12:06:46.780 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/cdc-acm.ko 2020-11-20T12:06:46.781 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/cdc-wdm.ko 2020-11-20T12:06:46.781 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/usblp.ko 2020-11-20T12:06:46.781 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 93385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/usbtmc.ko 2020-11-20T12:06:46.782 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/common/ 2020-11-20T12:06:46.783 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/common/ulpi.ko 2020-11-20T12:06:46.783 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc2/ 2020-11-20T12:06:46.783 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc2/dwc2_pci.ko 2020-11-20T12:06:46.783 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/ 2020-11-20T12:06:46.783 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3-haps.ko 2020-11-20T12:06:46.784 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3-pci.ko 2020-11-20T12:06:46.784 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 432665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3.ko 2020-11-20T12:06:46.787 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/ 2020-11-20T12:06:46.787 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/ 2020-11-20T12:06:46.788 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/u_ether.ko 2020-11-20T12:06:46.788 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/u_serial.ko 2020-11-20T12:06:46.788 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2020-11-20T12:06:46.789 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2020-11-20T12:06:46.789 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2020-11-20T12:06:46.789 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2020-11-20T12:06:46.790 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 92065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2020-11-20T12:06:46.790 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2020-11-20T12:06:46.791 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 120497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2020-11-20T12:06:46.792 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2020-11-20T12:06:46.793 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2020-11-20T12:06:46.793 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2020-11-20T12:06:46.794 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2020-11-20T12:06:46.794 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 83169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2020-11-20T12:06:46.795 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2020-11-20T12:06:46.795 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 58137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2020-11-20T12:06:46.796 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2020-11-20T12:06:46.796 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/ 2020-11-20T12:06:46.797 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2020-11-20T12:06:46.797 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2020-11-20T12:06:46.797 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2020-11-20T12:06:46.798 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ether.ko 2020-11-20T12:06:46.799 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2020-11-20T12:06:46.799 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_hid.ko 2020-11-20T12:06:46.799 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2020-11-20T12:06:46.799 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2020-11-20T12:06:46.799 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2020-11-20T12:06:46.800 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_printer.ko 2020-11-20T12:06:46.800 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_serial.ko 2020-11-20T12:06:46.800 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_zero.ko 2020-11-20T12:06:46.801 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2020-11-20T12:06:46.801 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2020-11-20T12:06:46.801 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 143505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/libcomposite.ko 2020-11-20T12:06:46.802 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/ 2020-11-20T12:06:46.802 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2020-11-20T12:06:46.802 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/ 2020-11-20T12:06:46.803 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 160057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2020-11-20T12:06:46.803 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/bdc_pci.ko 2020-11-20T12:06:46.804 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/fotg210-udc.ko 2020-11-20T12:06:46.804 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/goku_udc.ko 2020-11-20T12:06:46.805 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/gr_udc.ko 2020-11-20T12:06:46.805 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2020-11-20T12:06:46.806 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/mv_udc.ko 2020-11-20T12:06:46.807 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 73193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/net2272.ko 2020-11-20T12:06:46.808 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 75857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/net2280.ko 2020-11-20T12:06:46.809 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 73825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/pch_udc.ko 2020-11-20T12:06:46.810 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 66497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2020-11-20T12:06:46.811 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 70881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2020-11-20T12:06:46.812 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 82825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2020-11-20T12:06:46.813 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 144217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/udc-core.ko 2020-11-20T12:06:46.815 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/ 2020-11-20T12:06:46.815 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/bcma-hcd.ko 2020-11-20T12:06:46.815 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 128681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/fotg210-hcd.ko 2020-11-20T12:06:46.817 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 76001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/isp116x-hcd.ko 2020-11-20T12:06:46.818 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 88825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/oxu210hp-hcd.ko 2020-11-20T12:06:46.819 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 79145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/r8a66597-hcd.ko 2020-11-20T12:06:46.820 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/sl811-hcd.ko 2020-11-20T12:06:46.821 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/sl811_cs.ko 2020-11-20T12:06:46.821 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/ssb-hcd.ko 2020-11-20T12:06:46.821 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 97993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/u132-hcd.ko 2020-11-20T12:06:46.822 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/xhci-plat-hcd.ko 2020-11-20T12:06:46.822 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/ 2020-11-20T12:06:46.823 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/mdc800.ko 2020-11-20T12:06:46.823 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/microtek.ko 2020-11-20T12:06:46.824 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ 2020-11-20T12:06:46.824 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/adutux.ko 2020-11-20T12:06:46.824 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/appledisplay.ko 2020-11-20T12:06:46.824 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/cypress_cy7c63.ko 2020-11-20T12:06:46.825 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/cytherm.ko 2020-11-20T12:06:46.825 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ehset.ko 2020-11-20T12:06:46.826 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/emi26.ko 2020-11-20T12:06:46.826 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/emi62.ko 2020-11-20T12:06:46.826 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ezusb.ko 2020-11-20T12:06:46.826 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ftdi-elan.ko 2020-11-20T12:06:46.827 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/idmouse.ko 2020-11-20T12:06:46.828 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/iowarrior.ko 2020-11-20T12:06:46.828 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/isight_firmware.ko 2020-11-20T12:06:46.828 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ldusb.ko 2020-11-20T12:06:46.828 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/legousbtower.ko 2020-11-20T12:06:46.829 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/lvstest.ko 2020-11-20T12:06:46.829 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/sisusbvga/ 2020-11-20T12:06:46.829 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2020-11-20T12:06:46.830 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/trancevibrator.ko 2020-11-20T12:06:46.830 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usb3503.ko 2020-11-20T12:06:46.830 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usblcd.ko 2020-11-20T12:06:46.830 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usbsevseg.ko 2020-11-20T12:06:46.831 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 71281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usbtest.ko 2020-11-20T12:06:46.831 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/uss720.ko 2020-11-20T12:06:46.832 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/yurex.ko 2020-11-20T12:06:46.832 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/mon/ 2020-11-20T12:06:46.832 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 69937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/mon/usbmon.ko 2020-11-20T12:06:46.832 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/musb/ 2020-11-20T12:06:46.833 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 293313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/musb/musb_hdrc.ko 2020-11-20T12:06:46.835 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/ 2020-11-20T12:06:46.836 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-generic.ko 2020-11-20T12:06:46.836 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2020-11-20T12:06:46.837 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-isp1301.ko 2020-11-20T12:06:46.837 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-tahvo.ko 2020-11-20T12:06:46.837 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ 2020-11-20T12:06:46.838 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/aircable.ko 2020-11-20T12:06:46.838 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ark3116.ko 2020-11-20T12:06:46.838 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/belkin_sa.ko 2020-11-20T12:06:46.838 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ch341.ko 2020-11-20T12:06:46.839 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 82145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cp210x.ko 2020-11-20T12:06:46.839 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cyberjack.ko 2020-11-20T12:06:46.839 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cypress_m8.ko 2020-11-20T12:06:46.840 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/digi_acceleport.ko 2020-11-20T12:06:46.840 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/empeg.ko 2020-11-20T12:06:46.841 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/f81232.ko 2020-11-20T12:06:46.841 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 174849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ftdi_sio.ko 2020-11-20T12:06:46.841 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/garmin_gps.ko 2020-11-20T12:06:46.842 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 140417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/io_edgeport.ko 2020-11-20T12:06:46.844 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 124153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/io_ti.ko 2020-11-20T12:06:46.845 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ipaq.ko 2020-11-20T12:06:46.846 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ipw.ko 2020-11-20T12:06:46.846 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ir-usb.ko 2020-11-20T12:06:46.846 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 57473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/iuu_phoenix.ko 2020-11-20T12:06:46.846 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 106265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/keyspan.ko 2020-11-20T12:06:46.847 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/keyspan_pda.ko 2020-11-20T12:06:46.848 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/kl5kusb105.ko 2020-11-20T12:06:46.848 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/kobil_sct.ko 2020-11-20T12:06:46.849 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mct_u232.ko 2020-11-20T12:06:46.849 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/metro-usb.ko 2020-11-20T12:06:46.849 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 75537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mos7720.ko 2020-11-20T12:06:46.850 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 78913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mos7840.ko 2020-11-20T12:06:46.850 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mxuport.ko 2020-11-20T12:06:46.851 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/navman.ko 2020-11-20T12:06:46.851 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/omninet.ko 2020-11-20T12:06:46.851 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/opticon.ko 2020-11-20T12:06:46.852 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 176777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/option.ko 2020-11-20T12:06:46.852 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 39665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/oti6858.ko 2020-11-20T12:06:46.853 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/pl2303.ko 2020-11-20T12:06:46.853 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/qcaux.ko 2020-11-20T12:06:46.854 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/qcserial.ko 2020-11-20T12:06:46.854 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/quatech2.ko 2020-11-20T12:06:46.855 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/safe_serial.ko 2020-11-20T12:06:46.855 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/sierra.ko 2020-11-20T12:06:46.855 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/spcp8x5.ko 2020-11-20T12:06:46.855 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ssu100.ko 2020-11-20T12:06:46.855 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/symbolserial.ko 2020-11-20T12:06:46.856 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2020-11-20T12:06:46.857 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb-serial-simple.ko 2020-11-20T12:06:46.857 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb_debug.ko 2020-11-20T12:06:46.857 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb_wwan.ko 2020-11-20T12:06:46.857 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 118233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usbserial.ko 2020-11-20T12:06:46.858 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/visor.ko 2020-11-20T12:06:46.858 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/whiteheat.ko 2020-11-20T12:06:46.859 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/wishbone-serial.ko 2020-11-20T12:06:46.859 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/xsens_mt.ko 2020-11-20T12:06:46.859 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ 2020-11-20T12:06:46.859 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/uas.ko 2020-11-20T12:06:46.860 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-alauda.ko 2020-11-20T12:06:46.861 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-cypress.ko 2020-11-20T12:06:46.861 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-datafab.ko 2020-11-20T12:06:46.861 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-eneub6250.ko 2020-11-20T12:06:46.861 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-freecom.ko 2020-11-20T12:06:46.862 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-isd200.ko 2020-11-20T12:06:46.862 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-jumpshot.ko 2020-11-20T12:06:46.862 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-karma.ko 2020-11-20T12:06:46.862 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-onetouch.ko 2020-11-20T12:06:46.863 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-realtek.ko 2020-11-20T12:06:46.863 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-sddr09.ko 2020-11-20T12:06:46.864 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-sddr55.ko 2020-11-20T12:06:46.864 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-usbat.ko 2020-11-20T12:06:46.864 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 173801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/usb-storage.ko 2020-11-20T12:06:46.864 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/ 2020-11-20T12:06:46.865 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/usbip-core.ko 2020-11-20T12:06:46.865 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 71161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/usbip-host.ko 2020-11-20T12:06:46.866 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 107913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/vhci-hcd.ko 2020-11-20T12:06:46.867 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/ 2020-11-20T12:06:46.868 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/pci/ 2020-11-20T12:06:46.868 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 149025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/pci/vfio-pci.ko 2020-11-20T12:06:46.869 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 72673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio.ko 2020-11-20T12:06:46.870 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 68809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio_iommu_type1.ko 2020-11-20T12:06:46.871 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio_virqfd.ko 2020-11-20T12:06:46.872 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/ 2020-11-20T12:06:46.872 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 99057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost.ko 2020-11-20T12:06:46.872 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_iotlb.ko 2020-11-20T12:06:46.873 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_net.ko 2020-11-20T12:06:46.873 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 84649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_scsi.ko 2020-11-20T12:06:46.874 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/ 2020-11-20T12:06:46.875 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/ 2020-11-20T12:06:46.875 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/88pm860x_bl.ko 2020-11-20T12:06:46.875 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/aat2870_bl.ko 2020-11-20T12:06:46.875 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp5520_bl.ko 2020-11-20T12:06:46.877 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp8860_bl.ko 2020-11-20T12:06:46.878 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp8870_bl.ko 2020-11-20T12:06:46.878 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/apple_bl.ko 2020-11-20T12:06:46.878 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/as3711_bl.ko 2020-11-20T12:06:46.878 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/bd6107.ko 2020-11-20T12:06:46.879 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/da903x_bl.ko 2020-11-20T12:06:46.879 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/da9052_bl.ko 2020-11-20T12:06:46.879 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/gpio_backlight.ko 2020-11-20T12:06:46.879 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/kb3886_bl.ko 2020-11-20T12:06:46.879 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lcd.ko 2020-11-20T12:06:46.880 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lm3533_bl.ko 2020-11-20T12:06:46.880 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lm3639_bl.ko 2020-11-20T12:06:46.880 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lv5207lp.ko 2020-11-20T12:06:46.880 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/max8925_bl.ko 2020-11-20T12:06:46.880 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/pandora_bl.ko 2020-11-20T12:06:46.881 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/pcf50633-backlight.ko 2020-11-20T12:06:46.881 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/platform_lcd.ko 2020-11-20T12:06:46.881 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/wm831x_bl.ko 2020-11-20T12:06:46.881 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/virtio/ 2020-11-20T12:06:46.882 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/virtio/virtio_mem.ko 2020-11-20T12:06:46.884 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/ 2020-11-20T12:06:46.884 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ 2020-11-20T12:06:46.884 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds1wm.ko 2020-11-20T12:06:46.885 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds2482.ko 2020-11-20T12:06:46.885 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds2490.ko 2020-11-20T12:06:46.885 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/matrox_w1.ko 2020-11-20T12:06:46.885 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/w1-gpio.ko 2020-11-20T12:06:46.886 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/ 2020-11-20T12:06:46.886 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2406.ko 2020-11-20T12:06:46.886 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2408.ko 2020-11-20T12:06:46.886 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2413.ko 2020-11-20T12:06:46.886 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2423.ko 2020-11-20T12:06:46.887 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2431.ko 2020-11-20T12:06:46.887 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2433.ko 2020-11-20T12:06:46.887 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2780.ko 2020-11-20T12:06:46.887 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2781.ko 2020-11-20T12:06:46.887 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds28e04.ko 2020-11-20T12:06:46.888 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_smem.ko 2020-11-20T12:06:46.888 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 59097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_therm.ko 2020-11-20T12:06:46.888 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 99985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/wire.ko 2020-11-20T12:06:46.888 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ 2020-11-20T12:06:46.889 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/acquirewdt.ko 2020-11-20T12:06:46.889 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/advantechwdt.ko 2020-11-20T12:06:46.889 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/alim1535_wdt.ko 2020-11-20T12:06:46.889 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/alim7101_wdt.ko 2020-11-20T12:06:46.889 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/cpu5wdt.ko 2020-11-20T12:06:46.890 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9052_wdt.ko 2020-11-20T12:06:46.890 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9055_wdt.ko 2020-11-20T12:06:46.890 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9063_wdt.ko 2020-11-20T12:06:46.890 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/dw_wdt.ko 2020-11-20T12:06:46.890 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/eurotechwdt.ko 2020-11-20T12:06:46.891 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/f71808e_wdt.ko 2020-11-20T12:06:46.894 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/hpwdt.ko 2020-11-20T12:06:46.894 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/i6300esb.ko 2020-11-20T12:06:46.895 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/iTCO_vendor_support.ko 2020-11-20T12:06:46.895 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/iTCO_wdt.ko 2020-11-20T12:06:46.895 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ib700wdt.ko 2020-11-20T12:06:46.895 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ibmasr.ko 2020-11-20T12:06:46.895 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ie6xx_wdt.ko 2020-11-20T12:06:46.896 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/it8712f_wdt.ko 2020-11-20T12:06:46.896 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/it87_wdt.ko 2020-11-20T12:06:46.896 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/kempld_wdt.ko 2020-11-20T12:06:46.896 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/machzwd.ko 2020-11-20T12:06:46.897 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/mena21_wdt.ko 2020-11-20T12:06:46.897 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/menf21bmc_wdt.ko 2020-11-20T12:06:46.897 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/nv_tco.ko 2020-11-20T12:06:46.897 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/of_xilinx_wdt.ko 2020-11-20T12:06:46.898 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pc87413_wdt.ko 2020-11-20T12:06:46.898 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pcwd_pci.ko 2020-11-20T12:06:46.898 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pcwd_usb.ko 2020-11-20T12:06:46.899 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/retu_wdt.ko 2020-11-20T12:06:46.899 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc60xxwdt.ko 2020-11-20T12:06:46.900 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc_epx_c3.ko 2020-11-20T12:06:46.900 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2020-11-20T12:06:46.901 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sc1200wdt.ko 2020-11-20T12:06:46.901 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sch311x_wdt.ko 2020-11-20T12:06:46.901 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/smsc37b787_wdt.ko 2020-11-20T12:06:46.902 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/softdog.ko 2020-11-20T12:06:46.902 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sp5100_tco.ko 2020-11-20T12:06:46.902 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/twl4030_wdt.ko 2020-11-20T12:06:46.903 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/via_wdt.ko 2020-11-20T12:06:46.903 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83627hf_wdt.ko 2020-11-20T12:06:46.904 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83877f_wdt.ko 2020-11-20T12:06:46.904 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83977f_wdt.ko 2020-11-20T12:06:46.904 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wafer5823wdt.ko 2020-11-20T12:06:46.905 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wdt_pci.ko 2020-11-20T12:06:46.905 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wm831x_wdt.ko 2020-11-20T12:06:46.905 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wm8350_wdt.ko 2020-11-20T12:06:46.905 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/xen_wdt.ko 2020-11-20T12:06:46.905 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/ 2020-11-20T12:06:46.906 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-evtchn.ko 2020-11-20T12:06:46.906 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-gntalloc.ko 2020-11-20T12:06:46.906 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 43137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-gntdev.ko 2020-11-20T12:06:46.906 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-pciback/ 2020-11-20T12:06:46.907 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 178577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2020-11-20T12:06:46.907 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-privcmd.ko 2020-11-20T12:06:46.908 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 74393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-scsiback.ko 2020-11-20T12:06:46.908 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xenfs/ 2020-11-20T12:06:46.908 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xenfs/xenfs.ko 2020-11-20T12:06:46.909 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ 2020-11-20T12:06:46.909 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/9p/ 2020-11-20T12:06:46.909 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 147729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/9p/9p.ko 2020-11-20T12:06:46.909 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/adfs/ 2020-11-20T12:06:46.909 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/adfs/adfs.ko 2020-11-20T12:06:46.910 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/affs/ 2020-11-20T12:06:46.910 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 162417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/affs/affs.ko 2020-11-20T12:06:46.910 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/afs/ 2020-11-20T12:06:46.910 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 972889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/afs/kafs.ko 2020-11-20T12:06:46.911 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/autofs/ 2020-11-20T12:06:46.911 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 106865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/autofs/autofs4.ko 2020-11-20T12:06:46.912 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/befs/ 2020-11-20T12:06:46.912 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/befs/befs.ko 2020-11-20T12:06:46.913 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/bfs/ 2020-11-20T12:06:46.913 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/bfs/bfs.ko 2020-11-20T12:06:46.913 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/binfmt_misc.ko 2020-11-20T12:06:46.914 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/btrfs/ 2020-11-20T12:06:46.914 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 3192737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/btrfs/btrfs.ko 2020-11-20T12:06:46.954 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cachefiles/ 2020-11-20T12:06:46.955 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 194401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cachefiles/cachefiles.ko 2020-11-20T12:06:46.956 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ceph/ 2020-11-20T12:06:46.956 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1125913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ceph/ceph.ko 2020-11-20T12:06:46.969 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cifs/ 2020-11-20T12:06:46.970 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 2585001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cifs/cifs.ko 2020-11-20T12:06:46.998 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/coda/ 2020-11-20T12:06:46.998 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 96281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/coda/coda.ko 2020-11-20T12:06:46.999 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/configfs/ 2020-11-20T12:06:46.999 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 102945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/configfs/configfs.ko 2020-11-20T12:06:47.000 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cramfs/ 2020-11-20T12:06:47.000 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cramfs/cramfs.ko 2020-11-20T12:06:47.001 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/dlm/ 2020-11-20T12:06:47.001 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 438425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/dlm/dlm.ko 2020-11-20T12:06:47.006 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/efs/ 2020-11-20T12:06:47.006 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/efs/efs.ko 2020-11-20T12:06:47.007 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/f2fs/ 2020-11-20T12:06:47.007 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1367417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/f2fs/f2fs.ko 2020-11-20T12:06:47.023 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fat/ 2020-11-20T12:06:47.024 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fat/msdos.ko 2020-11-20T12:06:47.024 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/freevxfs/ 2020-11-20T12:06:47.024 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/freevxfs/freevxfs.ko 2020-11-20T12:06:47.025 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fscache/ 2020-11-20T12:06:47.025 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 317913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fscache/fscache.ko 2020-11-20T12:06:47.027 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fuse/ 2020-11-20T12:06:47.028 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fuse/cuse.ko 2020-11-20T12:06:47.028 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/gfs2/ 2020-11-20T12:06:47.028 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1193977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/gfs2/gfs2.ko 2020-11-20T12:06:47.040 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfs/ 2020-11-20T12:06:47.040 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 141505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfs/hfs.ko 2020-11-20T12:06:47.041 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfsplus/ 2020-11-20T12:06:47.042 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 229553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfsplus/hfsplus.ko 2020-11-20T12:06:47.045 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hpfs/ 2020-11-20T12:06:47.045 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 193625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hpfs/hpfs.ko 2020-11-20T12:06:47.047 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/isofs/ 2020-11-20T12:06:47.048 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 75881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/isofs/isofs.ko 2020-11-20T12:06:47.048 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/jfs/ 2020-11-20T12:06:47.049 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 413889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/jfs/jfs.ko 2020-11-20T12:06:47.055 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/lockd/ 2020-11-20T12:06:47.055 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 268617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/lockd/lockd.ko 2020-11-20T12:06:47.060 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/minix/ 2020-11-20T12:06:47.060 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 83041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/minix/minix.ko 2020-11-20T12:06:47.060 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/ 2020-11-20T12:06:47.061 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/blocklayout/ 2020-11-20T12:06:47.061 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 87177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2020-11-20T12:06:47.062 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/filelayout/ 2020-11-20T12:06:47.062 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 64561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2020-11-20T12:06:47.063 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/flexfilelayout/ 2020-11-20T12:06:47.063 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 123673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2020-11-20T12:06:47.064 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 899209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfs.ko 2020-11-20T12:06:47.071 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 65457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv2.ko 2020-11-20T12:06:47.072 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 112873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv3.ko 2020-11-20T12:06:47.073 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1849921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv4.ko 2020-11-20T12:06:47.092 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/ 2020-11-20T12:06:47.092 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/grace.ko 2020-11-20T12:06:47.092 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/nfs_acl.ko 2020-11-20T12:06:47.092 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/nfs_ssc.ko 2020-11-20T12:06:47.093 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfsd/ 2020-11-20T12:06:47.093 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1361097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfsd/nfsd.ko 2020-11-20T12:06:47.108 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nilfs2/ 2020-11-20T12:06:47.108 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 557369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nilfs2/nilfs2.ko 2020-11-20T12:06:47.117 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/ 2020-11-20T12:06:47.118 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-celtic.ko 2020-11-20T12:06:47.118 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-centeuro.ko 2020-11-20T12:06:47.118 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-croatian.ko 2020-11-20T12:06:47.118 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-cyrillic.ko 2020-11-20T12:06:47.119 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-gaelic.ko 2020-11-20T12:06:47.119 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-greek.ko 2020-11-20T12:06:47.119 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-iceland.ko 2020-11-20T12:06:47.119 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-inuit.ko 2020-11-20T12:06:47.120 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-roman.ko 2020-11-20T12:06:47.120 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-romanian.ko 2020-11-20T12:06:47.120 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-turkish.ko 2020-11-20T12:06:47.120 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_ascii.ko 2020-11-20T12:06:47.120 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1250.ko 2020-11-20T12:06:47.121 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1251.ko 2020-11-20T12:06:47.121 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1255.ko 2020-11-20T12:06:47.121 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp737.ko 2020-11-20T12:06:47.121 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp775.ko 2020-11-20T12:06:47.121 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp850.ko 2020-11-20T12:06:47.122 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp852.ko 2020-11-20T12:06:47.122 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp855.ko 2020-11-20T12:06:47.122 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp857.ko 2020-11-20T12:06:47.122 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp860.ko 2020-11-20T12:06:47.122 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp861.ko 2020-11-20T12:06:47.123 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp862.ko 2020-11-20T12:06:47.123 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp863.ko 2020-11-20T12:06:47.123 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp864.ko 2020-11-20T12:06:47.123 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp865.ko 2020-11-20T12:06:47.124 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp866.ko 2020-11-20T12:06:47.124 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp869.ko 2020-11-20T12:06:47.124 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp874.ko 2020-11-20T12:06:47.124 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 93241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp932.ko 2020-11-20T12:06:47.125 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 141825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp936.ko 2020-11-20T12:06:47.125 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 164961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp949.ko 2020-11-20T12:06:47.126 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 117337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp950.ko 2020-11-20T12:06:47.131 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_euc-jp.ko 2020-11-20T12:06:47.131 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-1.ko 2020-11-20T12:06:47.131 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-13.ko 2020-11-20T12:06:47.132 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-14.ko 2020-11-20T12:06:47.132 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-15.ko 2020-11-20T12:06:47.132 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-2.ko 2020-11-20T12:06:47.132 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-3.ko 2020-11-20T12:06:47.133 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-4.ko 2020-11-20T12:06:47.133 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-5.ko 2020-11-20T12:06:47.133 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-6.ko 2020-11-20T12:06:47.133 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-7.ko 2020-11-20T12:06:47.133 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-9.ko 2020-11-20T12:06:47.134 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-r.ko 2020-11-20T12:06:47.134 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-ru.ko 2020-11-20T12:06:47.134 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-u.ko 2020-11-20T12:06:47.134 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_utf8.ko 2020-11-20T12:06:47.134 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ntfs/ 2020-11-20T12:06:47.135 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 212417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ntfs/ntfs.ko 2020-11-20T12:06:47.135 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ 2020-11-20T12:06:47.135 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/cluster/ 2020-11-20T12:06:47.136 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 235897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2020-11-20T12:06:47.137 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlm/ 2020-11-20T12:06:47.137 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 435145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2020-11-20T12:06:47.143 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlmfs/ 2020-11-20T12:06:47.143 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2020-11-20T12:06:47.144 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 2561401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2.ko 2020-11-20T12:06:47.172 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2020-11-20T12:06:47.172 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stack_user.ko 2020-11-20T12:06:47.173 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stackglue.ko 2020-11-20T12:06:47.173 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/omfs/ 2020-11-20T12:06:47.174 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 52561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/omfs/omfs.ko 2020-11-20T12:06:47.174 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/overlayfs/ 2020-11-20T12:06:47.174 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 288641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/overlayfs/overlay.ko 2020-11-20T12:06:47.177 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/pstore/ 2020-11-20T12:06:47.177 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 55033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/pstore/ramoops.ko 2020-11-20T12:06:47.178 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx4/ 2020-11-20T12:06:47.178 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx4/qnx4.ko 2020-11-20T12:06:47.178 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx6/ 2020-11-20T12:06:47.179 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx6/qnx6.ko 2020-11-20T12:06:47.179 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/ 2020-11-20T12:06:47.180 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_tree.ko 2020-11-20T12:06:47.180 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_v1.ko 2020-11-20T12:06:47.180 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_v2.ko 2020-11-20T12:06:47.180 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/reiserfs/ 2020-11-20T12:06:47.180 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 521233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/reiserfs/reiserfs.ko 2020-11-20T12:06:47.187 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/romfs/ 2020-11-20T12:06:47.187 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/romfs/romfs.ko 2020-11-20T12:06:47.188 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/squashfs/ 2020-11-20T12:06:47.188 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 125585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/squashfs/squashfs.ko 2020-11-20T12:06:47.189 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/sysv/ 2020-11-20T12:06:47.189 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 86857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/sysv/sysv.ko 2020-11-20T12:06:47.190 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/udf/ 2020-11-20T12:06:47.190 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 249097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/udf/udf.ko 2020-11-20T12:06:47.193 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ufs/ 2020-11-20T12:06:47.194 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 157929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ufs/ufs.ko 2020-11-20T12:06:47.195 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/xfs/ 2020-11-20T12:06:47.196 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 3641945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/xfs/xfs.ko 2020-11-20T12:06:47.236 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/ 2020-11-20T12:06:47.236 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/locking/ 2020-11-20T12:06:47.237 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 56513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/locking/locktorture.ko 2020-11-20T12:06:47.238 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/time/ 2020-11-20T12:06:47.238 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/time/test_udelay.ko 2020-11-20T12:06:47.238 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/torture.ko 2020-11-20T12:06:47.238 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ 2020-11-20T12:06:47.239 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc-itu-t.ko 2020-11-20T12:06:47.239 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc64.ko 2020-11-20T12:06:47.239 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc7.ko 2020-11-20T12:06:47.239 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc8.ko 2020-11-20T12:06:47.239 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/ 2020-11-20T12:06:47.240 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/libarc4.ko 2020-11-20T12:06:47.240 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/libdes.ko 2020-11-20T12:06:47.241 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/interval_tree_test.ko 2020-11-20T12:06:47.241 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/libcrc32c.ko 2020-11-20T12:06:47.242 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lru_cache.ko 2020-11-20T12:06:47.242 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/ 2020-11-20T12:06:47.242 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/lz4_compress.ko 2020-11-20T12:06:47.242 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/lz4hc_compress.ko 2020-11-20T12:06:47.243 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/math/ 2020-11-20T12:06:47.243 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/math/cordic.ko 2020-11-20T12:06:47.243 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/memory-notifier-error-inject.ko 2020-11-20T12:06:47.244 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/notifier-error-inject.ko 2020-11-20T12:06:47.244 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/percpu_test.ko 2020-11-20T12:06:47.244 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/pm-notifier-error-inject.ko 2020-11-20T12:06:47.244 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/raid6/ 2020-11-20T12:06:47.245 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 141225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/raid6/raid6_pq.ko 2020-11-20T12:06:47.246 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/rbtree_test.ko 2020-11-20T12:06:47.246 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/reed_solomon/ 2020-11-20T12:06:47.246 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/reed_solomon/reed_solomon.ko 2020-11-20T12:06:47.247 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test-kstrtox.ko 2020-11-20T12:06:47.247 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test-string_helpers.ko 2020-11-20T12:06:47.248 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1714593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_bpf.ko 2020-11-20T12:06:47.250 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_firmware.ko 2020-11-20T12:06:47.252 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 5545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_module.ko 2020-11-20T12:06:47.252 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_user_copy.ko 2020-11-20T12:06:47.252 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_bm.ko 2020-11-20T12:06:47.253 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_fsm.ko 2020-11-20T12:06:47.253 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_kmp.ko 2020-11-20T12:06:47.254 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/xz/ 2020-11-20T12:06:47.254 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/xz/xz_dec_test.ko 2020-11-20T12:06:47.255 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/zstd/ 2020-11-20T12:06:47.256 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 259025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/zstd/zstd_compress.ko 2020-11-20T12:06:47.256 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/mm/ 2020-11-20T12:06:47.257 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/mm/hwpoison-inject.ko 2020-11-20T12:06:47.257 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ 2020-11-20T12:06:47.257 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/ 2020-11-20T12:06:47.257 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 58657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/6lowpan.ko 2020-11-20T12:06:47.259 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_dest.ko 2020-11-20T12:06:47.259 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_fragment.ko 2020-11-20T12:06:47.259 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_hop.ko 2020-11-20T12:06:47.259 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_ipv6.ko 2020-11-20T12:06:47.260 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_mobility.ko 2020-11-20T12:06:47.260 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_routing.ko 2020-11-20T12:06:47.260 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_udp.ko 2020-11-20T12:06:47.260 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/ 2020-11-20T12:06:47.260 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/garp.ko 2020-11-20T12:06:47.261 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/mrp.ko 2020-11-20T12:06:47.261 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/p8022.ko 2020-11-20T12:06:47.261 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/psnap.ko 2020-11-20T12:06:47.261 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/stp.ko 2020-11-20T12:06:47.262 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/8021q/ 2020-11-20T12:06:47.262 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 82849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/8021q/8021q.ko 2020-11-20T12:06:47.263 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/ 2020-11-20T12:06:47.263 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 208001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet.ko 2020-11-20T12:06:47.263 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet_rdma.ko 2020-11-20T12:06:47.263 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet_virtio.ko 2020-11-20T12:06:47.263 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/batman-adv/ 2020-11-20T12:06:47.264 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 647129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/batman-adv/batman-adv.ko 2020-11-20T12:06:47.270 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/ 2020-11-20T12:06:47.271 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 81361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/br_netfilter.ko 2020-11-20T12:06:47.271 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 610185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/bridge.ko 2020-11-20T12:06:47.280 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ 2020-11-20T12:06:47.280 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_802_3.ko 2020-11-20T12:06:47.280 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_among.ko 2020-11-20T12:06:47.280 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_arp.ko 2020-11-20T12:06:47.281 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_arpreply.ko 2020-11-20T12:06:47.281 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_dnat.ko 2020-11-20T12:06:47.281 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_ip.ko 2020-11-20T12:06:47.282 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_ip6.ko 2020-11-20T12:06:47.282 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_limit.ko 2020-11-20T12:06:47.282 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_log.ko 2020-11-20T12:06:47.282 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_mark.ko 2020-11-20T12:06:47.282 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_mark_m.ko 2020-11-20T12:06:47.283 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_nflog.ko 2020-11-20T12:06:47.284 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_pkttype.ko 2020-11-20T12:06:47.284 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_redirect.ko 2020-11-20T12:06:47.285 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_snat.ko 2020-11-20T12:06:47.285 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_stp.ko 2020-11-20T12:06:47.285 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_vlan.ko 2020-11-20T12:06:47.285 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_broute.ko 2020-11-20T12:06:47.286 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_filter.ko 2020-11-20T12:06:47.286 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_nat.ko 2020-11-20T12:06:47.286 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtables.ko 2020-11-20T12:06:47.286 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nf_log_bridge.ko 2020-11-20T12:06:47.286 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2020-11-20T12:06:47.287 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2020-11-20T12:06:47.287 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ceph/ 2020-11-20T12:06:47.287 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 836073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ceph/libceph.ko 2020-11-20T12:06:47.292 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/core/ 2020-11-20T12:06:47.293 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 127473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/core/pktgen.ko 2020-11-20T12:06:47.294 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/ 2020-11-20T12:06:47.294 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 201017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp.ko 2020-11-20T12:06:47.297 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_diag.ko 2020-11-20T12:06:47.297 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_ipv4.ko 2020-11-20T12:06:47.298 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_ipv6.ko 2020-11-20T12:06:47.298 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/ 2020-11-20T12:06:47.299 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 263865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/decnet.ko 2020-11-20T12:06:47.302 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/netfilter/ 2020-11-20T12:06:47.302 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/netfilter/dn_rtmsg.ko 2020-11-20T12:06:47.302 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/ 2020-11-20T12:06:47.302 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 144145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/dsa_core.ko 2020-11-20T12:06:47.304 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_brcm.ko 2020-11-20T12:06:47.304 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_dsa.ko 2020-11-20T12:06:47.304 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_edsa.ko 2020-11-20T12:06:47.305 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_trailer.ko 2020-11-20T12:06:47.305 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/hsr/ 2020-11-20T12:06:47.305 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/hsr/hsr.ko 2020-11-20T12:06:47.306 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ 2020-11-20T12:06:47.306 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/6lowpan/ 2020-11-20T12:06:47.306 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2020-11-20T12:06:47.307 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 220801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ieee802154.ko 2020-11-20T12:06:47.308 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ieee802154_socket.ko 2020-11-20T12:06:47.308 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ 2020-11-20T12:06:47.309 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ah4.ko 2020-11-20T12:06:47.309 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/esp4.ko 2020-11-20T12:06:47.309 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 51945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/fou.ko 2020-11-20T12:06:47.310 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/gre.ko 2020-11-20T12:06:47.310 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/inet_diag.ko 2020-11-20T12:06:47.311 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_gre.ko 2020-11-20T12:06:47.312 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 64593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_tunnel.ko 2020-11-20T12:06:47.312 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_vti.ko 2020-11-20T12:06:47.313 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ipcomp.ko 2020-11-20T12:06:47.313 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ipip.ko 2020-11-20T12:06:47.313 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ 2020-11-20T12:06:47.313 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arp_tables.ko 2020-11-20T12:06:47.314 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arpt_mangle.ko 2020-11-20T12:06:47.314 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arptable_filter.ko 2020-11-20T12:06:47.315 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ip_tables.ko 2020-11-20T12:06:47.315 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 45657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_CLUSTERIP.ko 2020-11-20T12:06:47.317 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_ECN.ko 2020-11-20T12:06:47.317 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2020-11-20T12:06:47.317 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2020-11-20T12:06:47.318 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_ah.ko 2020-11-20T12:06:47.318 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2020-11-20T12:06:47.318 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_filter.ko 2020-11-20T12:06:47.318 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_mangle.ko 2020-11-20T12:06:47.318 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_nat.ko 2020-11-20T12:06:47.319 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_raw.ko 2020-11-20T12:06:47.319 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_security.ko 2020-11-20T12:06:47.319 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2020-11-20T12:06:47.319 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2020-11-20T12:06:47.319 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_flow_table_ipv4.ko 2020-11-20T12:06:47.320 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_log_arp.ko 2020-11-20T12:06:47.320 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_log_ipv4.ko 2020-11-20T12:06:47.320 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2020-11-20T12:06:47.320 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2020-11-20T12:06:47.320 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2020-11-20T12:06:47.321 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2020-11-20T12:06:47.324 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2020-11-20T12:06:47.324 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2020-11-20T12:06:47.324 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2020-11-20T12:06:47.324 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2020-11-20T12:06:47.325 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2020-11-20T12:06:47.325 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_bic.ko 2020-11-20T12:06:47.325 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_dctcp.ko 2020-11-20T12:06:47.325 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_diag.ko 2020-11-20T12:06:47.326 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_highspeed.ko 2020-11-20T12:06:47.326 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_htcp.ko 2020-11-20T12:06:47.326 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_hybla.ko 2020-11-20T12:06:47.326 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_illinois.ko 2020-11-20T12:06:47.326 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_lp.ko 2020-11-20T12:06:47.327 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_scalable.ko 2020-11-20T12:06:47.327 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_vegas.ko 2020-11-20T12:06:47.327 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_veno.ko 2020-11-20T12:06:47.327 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_westwood.ko 2020-11-20T12:06:47.328 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_yeah.ko 2020-11-20T12:06:47.328 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tunnel4.ko 2020-11-20T12:06:47.328 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/udp_diag.ko 2020-11-20T12:06:47.328 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 40649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/udp_tunnel.ko 2020-11-20T12:06:47.328 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/xfrm4_tunnel.ko 2020-11-20T12:06:47.329 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ 2020-11-20T12:06:47.329 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ah6.ko 2020-11-20T12:06:47.329 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/esp6.ko 2020-11-20T12:06:47.329 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/fou6.ko 2020-11-20T12:06:47.329 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 93417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_gre.ko 2020-11-20T12:06:47.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 101977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_tunnel.ko 2020-11-20T12:06:47.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_udp_tunnel.ko 2020-11-20T12:06:47.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_vti.ko 2020-11-20T12:06:47.330 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ipcomp6.ko 2020-11-20T12:06:47.331 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/mip6.ko 2020-11-20T12:06:47.331 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ 2020-11-20T12:06:47.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6_tables.ko 2020-11-20T12:06:47.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2020-11-20T12:06:47.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2020-11-20T12:06:47.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2020-11-20T12:06:47.335 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_ah.ko 2020-11-20T12:06:47.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2020-11-20T12:06:47.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_frag.ko 2020-11-20T12:06:47.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2020-11-20T12:06:47.336 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2020-11-20T12:06:47.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_mh.ko 2020-11-20T12:06:47.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2020-11-20T12:06:47.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_rt.ko 2020-11-20T12:06:47.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_filter.ko 2020-11-20T12:06:47.337 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2020-11-20T12:06:47.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_nat.ko 2020-11-20T12:06:47.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_raw.ko 2020-11-20T12:06:47.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_security.ko 2020-11-20T12:06:47.338 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2020-11-20T12:06:47.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2020-11-20T12:06:47.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_flow_table_ipv6.ko 2020-11-20T12:06:47.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_log_ipv6.ko 2020-11-20T12:06:47.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2020-11-20T12:06:47.339 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2020-11-20T12:06:47.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2020-11-20T12:06:47.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2020-11-20T12:06:47.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2020-11-20T12:06:47.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2020-11-20T12:06:47.340 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 90545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/sit.ko 2020-11-20T12:06:47.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/tunnel6.ko 2020-11-20T12:06:47.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/xfrm6_tunnel.ko 2020-11-20T12:06:47.341 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/key/ 2020-11-20T12:06:47.341 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 94849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/key/af_key.ko 2020-11-20T12:06:47.342 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/ 2020-11-20T12:06:47.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 127681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_core.ko 2020-11-20T12:06:47.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_debugfs.ko 2020-11-20T12:06:47.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_eth.ko 2020-11-20T12:06:47.342 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ip.ko 2020-11-20T12:06:47.343 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ip6.ko 2020-11-20T12:06:47.343 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_netlink.ko 2020-11-20T12:06:47.343 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ppp.ko 2020-11-20T12:06:47.343 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/lapb/ 2020-11-20T12:06:47.344 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/lapb/lapb.ko 2020-11-20T12:06:47.344 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/ 2020-11-20T12:06:47.344 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/llc.ko 2020-11-20T12:06:47.344 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 263433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/llc2.ko 2020-11-20T12:06:47.346 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mac802154/ 2020-11-20T12:06:47.346 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 218937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mac802154/mac802154.ko 2020-11-20T12:06:47.346 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mpls/ 2020-11-20T12:06:47.347 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mpls/mpls_gso.ko 2020-11-20T12:06:47.347 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ 2020-11-20T12:06:47.347 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ 2020-11-20T12:06:47.347 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 135633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set.ko 2020-11-20T12:06:47.348 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2020-11-20T12:06:47.348 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2020-11-20T12:06:47.348 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2020-11-20T12:06:47.348 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 101529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2020-11-20T12:06:47.348 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 102593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2020-11-20T12:06:47.349 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 103425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2020-11-20T12:06:47.349 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 104705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2020-11-20T12:06:47.349 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 112065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2020-11-20T12:06:47.349 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2020-11-20T12:06:47.350 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 108489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2020-11-20T12:06:47.350 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 112145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2020-11-20T12:06:47.351 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 110513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2020-11-20T12:06:47.351 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 110633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2020-11-20T12:06:47.351 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 112513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2020-11-20T12:06:47.352 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_list_set.ko 2020-11-20T12:06:47.352 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ 2020-11-20T12:06:47.352 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 384017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs.ko 2020-11-20T12:06:47.357 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2020-11-20T12:06:47.357 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2020-11-20T12:06:47.357 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2020-11-20T12:06:47.357 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2020-11-20T12:06:47.359 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2020-11-20T12:06:47.359 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2020-11-20T12:06:47.359 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2020-11-20T12:06:47.359 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2020-11-20T12:06:47.360 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2020-11-20T12:06:47.360 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2020-11-20T12:06:47.360 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2020-11-20T12:06:47.360 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2020-11-20T12:06:47.361 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2020-11-20T12:06:47.361 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 47153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conncount.ko 2020-11-20T12:06:47.361 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 414313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack.ko 2020-11-20T12:06:47.363 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_amanda.ko 2020-11-20T12:06:47.363 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_broadcast.ko 2020-11-20T12:06:47.363 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_ftp.ko 2020-11-20T12:06:47.364 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 118865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_h323.ko 2020-11-20T12:06:47.365 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_irc.ko 2020-11-20T12:06:47.365 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2020-11-20T12:06:47.366 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 105089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_netlink.ko 2020-11-20T12:06:47.367 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_pptp.ko 2020-11-20T12:06:47.367 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_sane.ko 2020-11-20T12:06:47.368 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 61369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_sip.ko 2020-11-20T12:06:47.369 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_snmp.ko 2020-11-20T12:06:47.369 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_tftp.ko 2020-11-20T12:06:47.369 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_dup_netdev.ko 2020-11-20T12:06:47.369 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 75841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_flow_table.ko 2020-11-20T12:06:47.370 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_flow_table_inet.ko 2020-11-20T12:06:47.370 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_log_common.ko 2020-11-20T12:06:47.371 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 85969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat.ko 2020-11-20T12:06:47.371 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_amanda.ko 2020-11-20T12:06:47.372 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_ftp.ko 2020-11-20T12:06:47.372 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_irc.ko 2020-11-20T12:06:47.372 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 27417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_sip.ko 2020-11-20T12:06:47.373 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_tftp.ko 2020-11-20T12:06:47.373 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 53193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_synproxy_core.ko 2020-11-20T12:06:47.373 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 492617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_tables.ko 2020-11-20T12:06:47.380 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink.ko 2020-11-20T12:06:47.380 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_acct.ko 2020-11-20T12:06:47.381 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_cttimeout.ko 2020-11-20T12:06:47.381 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_log.ko 2020-11-20T12:06:47.381 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_osf.ko 2020-11-20T12:06:47.382 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 60217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_queue.ko 2020-11-20T12:06:47.382 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_chain_nat.ko 2020-11-20T12:06:47.382 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_compat.ko 2020-11-20T12:06:47.383 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_counter.ko 2020-11-20T12:06:47.384 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_ct.ko 2020-11-20T12:06:47.384 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_dup_netdev.ko 2020-11-20T12:06:47.384 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_fib.ko 2020-11-20T12:06:47.384 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_fwd_netdev.ko 2020-11-20T12:06:47.384 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_hash.ko 2020-11-20T12:06:47.385 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_limit.ko 2020-11-20T12:06:47.385 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_log.ko 2020-11-20T12:06:47.385 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_masq.ko 2020-11-20T12:06:47.387 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_nat.ko 2020-11-20T12:06:47.387 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_numgen.ko 2020-11-20T12:06:47.387 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_osf.ko 2020-11-20T12:06:47.388 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_queue.ko 2020-11-20T12:06:47.388 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_redir.ko 2020-11-20T12:06:47.388 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_reject.ko 2020-11-20T12:06:47.388 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_reject_inet.ko 2020-11-20T12:06:47.388 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_socket.ko 2020-11-20T12:06:47.389 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_synproxy.ko 2020-11-20T12:06:47.389 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_tproxy.ko 2020-11-20T12:06:47.389 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 91641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/x_tables.ko 2020-11-20T12:06:47.389 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_AUDIT.ko 2020-11-20T12:06:47.390 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CHECKSUM.ko 2020-11-20T12:06:47.390 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CLASSIFY.ko 2020-11-20T12:06:47.390 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CONNSECMARK.ko 2020-11-20T12:06:47.390 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CT.ko 2020-11-20T12:06:47.390 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_DSCP.ko 2020-11-20T12:06:47.391 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_HL.ko 2020-11-20T12:06:47.391 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_HMARK.ko 2020-11-20T12:06:47.391 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 38073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_IDLETIMER.ko 2020-11-20T12:06:47.396 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_LED.ko 2020-11-20T12:06:47.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_LOG.ko 2020-11-20T12:06:47.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_MASQUERADE.ko 2020-11-20T12:06:47.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NETMAP.ko 2020-11-20T12:06:47.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NFLOG.ko 2020-11-20T12:06:47.397 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NFQUEUE.ko 2020-11-20T12:06:47.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 21489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_RATEEST.ko 2020-11-20T12:06:47.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_REDIRECT.ko 2020-11-20T12:06:47.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_SECMARK.ko 2020-11-20T12:06:47.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TCPMSS.ko 2020-11-20T12:06:47.398 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2020-11-20T12:06:47.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TEE.ko 2020-11-20T12:06:47.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TPROXY.ko 2020-11-20T12:06:47.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TRACE.ko 2020-11-20T12:06:47.399 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_addrtype.ko 2020-11-20T12:06:47.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_bpf.ko 2020-11-20T12:06:47.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cgroup.ko 2020-11-20T12:06:47.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cluster.ko 2020-11-20T12:06:47.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_comment.ko 2020-11-20T12:06:47.400 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connbytes.ko 2020-11-20T12:06:47.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connlabel.ko 2020-11-20T12:06:47.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connlimit.ko 2020-11-20T12:06:47.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connmark.ko 2020-11-20T12:06:47.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_conntrack.ko 2020-11-20T12:06:47.401 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cpu.ko 2020-11-20T12:06:47.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_dccp.ko 2020-11-20T12:06:47.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_devgroup.ko 2020-11-20T12:06:47.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_dscp.ko 2020-11-20T12:06:47.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ecn.ko 2020-11-20T12:06:47.402 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_esp.ko 2020-11-20T12:06:47.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 49377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_hashlimit.ko 2020-11-20T12:06:47.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_helper.ko 2020-11-20T12:06:47.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_hl.ko 2020-11-20T12:06:47.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ipcomp.ko 2020-11-20T12:06:47.403 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_iprange.ko 2020-11-20T12:06:47.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ipvs.ko 2020-11-20T12:06:47.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 16273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_l2tp.ko 2020-11-20T12:06:47.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_length.ko 2020-11-20T12:06:47.404 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_limit.ko 2020-11-20T12:06:47.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_mac.ko 2020-11-20T12:06:47.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_mark.ko 2020-11-20T12:06:47.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_multiport.ko 2020-11-20T12:06:47.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_nat.ko 2020-11-20T12:06:47.405 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_nfacct.ko 2020-11-20T12:06:47.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_osf.ko 2020-11-20T12:06:47.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_owner.ko 2020-11-20T12:06:47.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_physdev.ko 2020-11-20T12:06:47.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_pkttype.ko 2020-11-20T12:06:47.406 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_policy.ko 2020-11-20T12:06:47.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_quota.ko 2020-11-20T12:06:47.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_rateest.ko 2020-11-20T12:06:47.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_realm.ko 2020-11-20T12:06:47.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_recent.ko 2020-11-20T12:06:47.407 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_sctp.ko 2020-11-20T12:06:47.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_set.ko 2020-11-20T12:06:47.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_socket.ko 2020-11-20T12:06:47.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_state.ko 2020-11-20T12:06:47.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_statistic.ko 2020-11-20T12:06:47.408 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_string.ko 2020-11-20T12:06:47.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_tcpmss.ko 2020-11-20T12:06:47.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_tcpudp.ko 2020-11-20T12:06:47.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_time.ko 2020-11-20T12:06:47.409 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_u32.ko 2020-11-20T12:06:47.410 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netlink/ 2020-11-20T12:06:47.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 13329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netlink/netlink_diag.ko 2020-11-20T12:06:47.410 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/nsh/ 2020-11-20T12:06:47.410 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 11729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/nsh/nsh.ko 2020-11-20T12:06:47.410 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/ 2020-11-20T12:06:47.411 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 367929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/openvswitch.ko 2020-11-20T12:06:47.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-geneve.ko 2020-11-20T12:06:47.412 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 8505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-gre.ko 2020-11-20T12:06:47.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-vxlan.ko 2020-11-20T12:06:47.413 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/packet/ 2020-11-20T12:06:47.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/packet/af_packet_diag.ko 2020-11-20T12:06:47.413 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/ 2020-11-20T12:06:47.413 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 123433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/phonet.ko 2020-11-20T12:06:47.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/pn_pep.ko 2020-11-20T12:06:47.414 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/ 2020-11-20T12:06:47.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 239729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds.ko 2020-11-20T12:06:47.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 158753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds_rdma.ko 2020-11-20T12:06:47.414 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds_tcp.ko 2020-11-20T12:06:47.415 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rxrpc/ 2020-11-20T12:06:47.415 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 698057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rxrpc/rxrpc.ko 2020-11-20T12:06:47.417 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/ 2020-11-20T12:06:47.417 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_csum.ko 2020-11-20T12:06:47.418 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_gact.ko 2020-11-20T12:06:47.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 37049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_ipt.ko 2020-11-20T12:06:47.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 36497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_mirred.ko 2020-11-20T12:06:47.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_nat.ko 2020-11-20T12:06:47.419 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_pedit.ko 2020-11-20T12:06:47.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_police.ko 2020-11-20T12:06:47.420 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_simple.ko 2020-11-20T12:06:47.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_skbedit.ko 2020-11-20T12:06:47.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_vlan.ko 2020-11-20T12:06:47.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 24641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_basic.ko 2020-11-20T12:06:47.421 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_bpf.ko 2020-11-20T12:06:47.422 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_cgroup.ko 2020-11-20T12:06:47.422 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_flow.ko 2020-11-20T12:06:47.422 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_fw.ko 2020-11-20T12:06:47.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 34449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_route.ko 2020-11-20T12:06:47.423 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_rsvp.ko 2020-11-20T12:06:47.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 35065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_rsvp6.ko 2020-11-20T12:06:47.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 41113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_tcindex.ko 2020-11-20T12:06:47.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 54513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_u32.ko 2020-11-20T12:06:47.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_cmp.ko 2020-11-20T12:06:47.424 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 12625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_ipset.ko 2020-11-20T12:06:47.425 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 31985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_meta.ko 2020-11-20T12:06:47.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_nbyte.ko 2020-11-20T12:06:47.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_text.ko 2020-11-20T12:06:47.426 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 6409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_u32.ko 2020-11-20T12:06:47.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_cbq.ko 2020-11-20T12:06:47.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_choke.ko 2020-11-20T12:06:47.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_codel.ko 2020-11-20T12:06:47.427 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 30513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_drr.ko 2020-11-20T12:06:47.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 32857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_dsmark.ko 2020-11-20T12:06:47.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_fq.ko 2020-11-20T12:06:47.428 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_fq_codel.ko 2020-11-20T12:06:47.430 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_gred.ko 2020-11-20T12:06:47.431 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 46041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_hfsc.ko 2020-11-20T12:06:47.432 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 20081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_hhf.ko 2020-11-20T12:06:47.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 50497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_htb.ko 2020-11-20T12:06:47.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_ingress.ko 2020-11-20T12:06:47.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_mqprio.ko 2020-11-20T12:06:47.433 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_multiq.ko 2020-11-20T12:06:47.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 33361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_netem.ko 2020-11-20T12:06:47.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 23369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_pie.ko 2020-11-20T12:06:47.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_plug.ko 2020-11-20T12:06:47.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 22969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_prio.ko 2020-11-20T12:06:47.434 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 44257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_qfq.ko 2020-11-20T12:06:47.435 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 28633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_red.ko 2020-11-20T12:06:47.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_sfb.ko 2020-11-20T12:06:47.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 29041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_sfq.ko 2020-11-20T12:06:47.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 26089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_tbf.ko 2020-11-20T12:06:47.436 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 25633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_teql.ko 2020-11-20T12:06:47.437 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/ 2020-11-20T12:06:47.437 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 879937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/sctp.ko 2020-11-20T12:06:47.443 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 18081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/sctp_diag.ko 2020-11-20T12:06:47.443 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/ 2020-11-20T12:06:47.444 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/ 2020-11-20T12:06:47.444 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 334985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2020-11-20T12:06:47.447 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 62473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2020-11-20T12:06:47.448 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1415977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/sunrpc.ko 2020-11-20T12:06:47.462 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/xprtrdma/ 2020-11-20T12:06:47.463 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 644937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2020-11-20T12:06:47.468 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/ 2020-11-20T12:06:47.469 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/diag.ko 2020-11-20T12:06:47.469 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 1118753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/tipc.ko 2020-11-20T12:06:47.482 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/unix/ 2020-11-20T12:06:47.482 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 15201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/unix/unix_diag.ko 2020-11-20T12:06:47.482 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/ 2020-11-20T12:06:47.482 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 73233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2020-11-20T12:06:47.483 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 67177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2020-11-20T12:06:47.483 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 77529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock.ko 2020-11-20T12:06:47.484 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 9953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock_diag.ko 2020-11-20T12:06:47.484 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 17297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock_loopback.ko 2020-11-20T12:06:47.485 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/x25/ 2020-11-20T12:06:47.485 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 126641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/x25/x25.ko 2020-11-20T12:06:47.486 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/ 2020-11-20T12:06:47.486 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_algo.ko 2020-11-20T12:06:47.487 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 19793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_ipcomp.ko 2020-11-20T12:06:47.487 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 81177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_user.ko 2020-11-20T12:06:47.487 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/ 2020-11-20T12:06:47.488 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/lib/ 2020-11-20T12:06:47.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 14473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/lib/irqbypass.ko 2020-11-20T12:06:47.488 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 614204 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.alias 2020-11-20T12:06:47.491 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 599988 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.alias.bin 2020-11-20T12:06:47.498 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 7746 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin 2020-11-20T12:06:47.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 10228 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin.bin 2020-11-20T12:06:47.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 63287 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin.modinfo 2020-11-20T12:06:47.499 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 146392 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.dep 2020-11-20T12:06:47.500 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 235760 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.dep.bin 2020-11-20T12:06:47.502 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 210 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.devname 2020-11-20T12:06:47.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 74729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.order 2020-11-20T12:06:47.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 574 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.softdep 2020-11-20T12:06:47.503 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 238541 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.symbols 2020-11-20T12:06:47.505 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 289923 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.symbols.bin 2020-11-20T12:06:47.509 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/ 2020-11-20T12:06:47.509 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/ 2020-11-20T12:06:47.510 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/doc/ 2020-11-20T12:06:47.510 INFO:teuthology.orchestra.run.smithi116.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-20T12:06:47.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 178 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/changelog.Debian.gz 2020-11-20T12:06:47.510 INFO:teuthology.orchestra.run.smithi116.stdout:-rw-r--r-- root/root 692 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/copyright 2020-11-20T12:06:47.512 DEBUG:teuthology.task.kernel:get_image_version: 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:06:47.512 INFO:teuthology.task.kernel:searching for kernel 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:06:47.512 INFO:teuthology.orchestra.run.smithi116:> egrep '(submenu|menuentry.*5.10.0-rc2-ceph-gac4f7b07f403).*{' /boot/grub/grub.cfg 2020-11-20T12:06:47.560 INFO:teuthology.orchestra.run.smithi116.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-20T12:06:47.561 INFO:teuthology.orchestra.run.smithi116.stdout: menuentry 'Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.0-rc2-ceph-gac4f7b07f403-advanced-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-20T12:06:47.561 INFO:teuthology.orchestra.run.smithi116.stdout: menuentry 'Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.0-rc2-ceph-gac4f7b07f403-recovery-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-20T12:06:47.562 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2020-11-20T12:06:47.562 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:06:47.562 INFO:teuthology.orchestra.run.smithi116:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2020-11-20T12:06:47.603 INFO:teuthology.task.kernel:Installing kernel ac4f7b07f4039e2bfae0b119fa6b4810d4eaa4d9 on client.1... 2020-11-20T12:06:47.604 INFO:teuthology.orchestra.run.smithi173:> sudo dpkg -i /tmp/linux-image.deb 2020-11-20T12:06:48.114 INFO:teuthology.orchestra.run.smithi116.stderr:Sourcing file `/etc/default/grub' 2020-11-20T12:06:48.135 INFO:teuthology.orchestra.run.smithi116.stderr:Generating grub configuration file ... 2020-11-20T12:06:48.372 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:06:48.385 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:06:48.684 INFO:teuthology.orchestra.run.smithi116.stderr:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-20T12:06:48.696 INFO:teuthology.orchestra.run.smithi116.stderr:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-20T12:06:49.272 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package linux-image-5.10.0-rc2-ceph-gac4f7b07f403. 2020-11-20T12:06:49.313 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... 102077 files and directories currently installed.) 2020-11-20T12:06:49.315 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack /tmp/linux-image.deb ... 2020-11-20T12:06:49.404 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking linux-image-5.10.0-rc2-ceph-gac4f7b07f403 (5.10.0-rc2-ceph-gac4f7b07f403-1) ... 2020-11-20T12:06:49.913 INFO:teuthology.orchestra.run.smithi116.stderr:done 2020-11-20T12:06:52.343 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up linux-image-5.10.0-rc2-ceph-gac4f7b07f403 (5.10.0-rc2-ceph-gac4f7b07f403-1) ... 2020-11-20T12:06:52.606 INFO:teuthology.orchestra.run.smithi173.stderr:update-initramfs: Generating /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:07:04.253 INFO:teuthology.orchestra.run.smithi173.stderr:Sourcing file `/etc/default/grub' 2020-11-20T12:07:04.268 INFO:teuthology.orchestra.run.smithi173.stderr:Generating grub configuration file ... 2020-11-20T12:07:04.613 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:07:04.647 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:07:04.967 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-20T12:07:04.997 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-20T12:07:06.624 INFO:teuthology.orchestra.run.smithi173.stderr:done 2020-11-20T12:07:06.769 INFO:teuthology.orchestra.run.smithi173:> dpkg-deb -c /tmp/linux-image.deb 2020-11-20T12:07:06.778 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./ 2020-11-20T12:07:06.778 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./boot/ 2020-11-20T12:07:06.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 5164573 2020-11-13 14:15 ./boot/System.map-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:07:06.841 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 153248 2020-11-13 14:15 ./boot/config-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:07:06.843 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9847776 2020-11-13 14:15 ./boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:07:06.987 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/ 2020-11-20T12:07:06.987 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/ 2020-11-20T12:07:06.987 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/postinst.d/ 2020-11-20T12:07:06.987 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/postrm.d/ 2020-11-20T12:07:06.988 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/preinst.d/ 2020-11-20T12:07:06.988 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./etc/kernel/prerm.d/ 2020-11-20T12:07:06.988 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/ 2020-11-20T12:07:06.988 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/ 2020-11-20T12:07:06.989 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-20T12:07:06.989 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/ 2020-11-20T12:07:06.989 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/ 2020-11-20T12:07:06.989 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/ 2020-11-20T12:07:06.990 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/ 2020-11-20T12:07:06.990 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 488985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/aesni-intel.ko 2020-11-20T12:07:06.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/blowfish-x86_64.ko 2020-11-20T12:07:06.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2020-11-20T12:07:06.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2020-11-20T12:07:06.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 86201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/camellia-x86_64.ko 2020-11-20T12:07:06.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 98425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2020-11-20T12:07:06.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 143041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2020-11-20T12:07:06.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/crc32-pclmul.ko 2020-11-20T12:07:06.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2020-11-20T12:07:06.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2020-11-20T12:07:06.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2020-11-20T12:07:06.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/glue_helper.ko 2020-11-20T12:07:06.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2020-11-20T12:07:06.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-avx2.ko 2020-11-20T12:07:06.995 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2020-11-20T12:07:06.996 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha1-ssse3.ko 2020-11-20T12:07:06.996 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha256-ssse3.ko 2020-11-20T12:07:06.997 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/sha512-ssse3.ko 2020-11-20T12:07:06.997 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2020-11-20T12:07:06.998 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2020-11-20T12:07:06.998 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/crypto/twofish-x86_64.ko 2020-11-20T12:07:06.998 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/ 2020-11-20T12:07:06.999 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/ 2020-11-20T12:07:06.999 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/mce/ 2020-11-20T12:07:06.999 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2020-11-20T12:07:06.999 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/cpuid.ko 2020-11-20T12:07:07.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kernel/msr.ko 2020-11-20T12:07:07.000 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/ 2020-11-20T12:07:07.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 247537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm-amd.ko 2020-11-20T12:07:07.002 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 826857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm-intel.ko 2020-11-20T12:07:07.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1834553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/kvm/kvm.ko 2020-11-20T12:07:07.033 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/oprofile/ 2020-11-20T12:07:07.033 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 135385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/arch/x86/oprofile/oprofile.ko 2020-11-20T12:07:07.034 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ 2020-11-20T12:07:07.034 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/af_alg.ko 2020-11-20T12:07:07.035 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/algif_hash.ko 2020-11-20T12:07:07.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/algif_skcipher.ko 2020-11-20T12:07:07.036 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ansi_cprng.ko 2020-11-20T12:07:07.038 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/anubis.ko 2020-11-20T12:07:07.038 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/arc4.ko 2020-11-20T12:07:07.038 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/asymmetric_keys/ 2020-11-20T12:07:07.039 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2020-11-20T12:07:07.039 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/ 2020-11-20T12:07:07.039 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_memcpy.ko 2020-11-20T12:07:07.040 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_pq.ko 2020-11-20T12:07:07.040 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_raid6_recov.ko 2020-11-20T12:07:07.040 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_tx.ko 2020-11-20T12:07:07.040 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/async_xor.ko 2020-11-20T12:07:07.040 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/async_tx/raid6test.ko 2020-11-20T12:07:07.041 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/authenc.ko 2020-11-20T12:07:07.041 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/authencesn.ko 2020-11-20T12:07:07.043 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blake2b_generic.ko 2020-11-20T12:07:07.044 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blowfish_common.ko 2020-11-20T12:07:07.044 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/blowfish_generic.ko 2020-11-20T12:07:07.044 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/camellia_generic.ko 2020-11-20T12:07:07.045 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast5_generic.ko 2020-11-20T12:07:07.045 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast6_generic.ko 2020-11-20T12:07:07.045 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cast_common.ko 2020-11-20T12:07:07.045 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/ccm.ko 2020-11-20T12:07:07.046 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cmac.ko 2020-11-20T12:07:07.046 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crc32_generic.ko 2020-11-20T12:07:07.046 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cryptd.ko 2020-11-20T12:07:07.047 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_engine.ko 2020-11-20T12:07:07.047 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_simd.ko 2020-11-20T12:07:07.047 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/crypto_user.ko 2020-11-20T12:07:07.047 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/cts.ko 2020-11-20T12:07:07.048 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/des_generic.ko 2020-11-20T12:07:07.048 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/dh_generic.ko 2020-11-20T12:07:07.048 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/echainiv.ko 2020-11-20T12:07:07.048 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/essiv.ko 2020-11-20T12:07:07.049 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/fcrypt.ko 2020-11-20T12:07:07.051 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/khazad.ko 2020-11-20T12:07:07.051 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lrw.ko 2020-11-20T12:07:07.052 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lz4.ko 2020-11-20T12:07:07.052 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/lz4hc.ko 2020-11-20T12:07:07.052 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/md4.ko 2020-11-20T12:07:07.052 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/michael_mic.ko 2020-11-20T12:07:07.053 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/pcbc.ko 2020-11-20T12:07:07.053 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/pcrypt.ko 2020-11-20T12:07:07.054 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd128.ko 2020-11-20T12:07:07.054 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd160.ko 2020-11-20T12:07:07.054 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd256.ko 2020-11-20T12:07:07.055 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/rmd320.ko 2020-11-20T12:07:07.055 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/salsa20_generic.ko 2020-11-20T12:07:07.055 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/seed.ko 2020-11-20T12:07:07.055 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/serpent_generic.ko 2020-11-20T12:07:07.055 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 170649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tcrypt.ko 2020-11-20T12:07:07.056 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tea.ko 2020-11-20T12:07:07.056 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/tgr192.ko 2020-11-20T12:07:07.056 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/twofish_common.ko 2020-11-20T12:07:07.056 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/twofish_generic.ko 2020-11-20T12:07:07.057 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/vmac.ko 2020-11-20T12:07:07.057 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/wp512.ko 2020-11-20T12:07:07.057 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xcbc.ko 2020-11-20T12:07:07.057 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xor.ko 2020-11-20T12:07:07.058 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xts.ko 2020-11-20T12:07:07.062 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/crypto/xxhash_generic.ko 2020-11-20T12:07:07.062 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ 2020-11-20T12:07:07.063 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/ 2020-11-20T12:07:07.063 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_extlog.ko 2020-11-20T12:07:07.063 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_ipmi.ko 2020-11-20T12:07:07.063 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/acpi_pad.ko 2020-11-20T12:07:07.064 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/apei/ 2020-11-20T12:07:07.064 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/apei/einj.ko 2020-11-20T12:07:07.064 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/ec_sys.ko 2020-11-20T12:07:07.064 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/sbs.ko 2020-11-20T12:07:07.065 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/sbshc.ko 2020-11-20T12:07:07.065 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 82265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/acpi/video.ko 2020-11-20T12:07:07.065 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ 2020-11-20T12:07:07.065 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/acard-ahci.ko 2020-11-20T12:07:07.065 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 84793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ahci.ko 2020-11-20T12:07:07.066 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/ahci_platform.ko 2020-11-20T12:07:07.066 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 81825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/libahci.ko 2020-11-20T12:07:07.066 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/libahci_platform.ko 2020-11-20T12:07:07.066 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_acpi.ko 2020-11-20T12:07:07.066 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ali.ko 2020-11-20T12:07:07.067 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_amd.ko 2020-11-20T12:07:07.067 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_artop.ko 2020-11-20T12:07:07.067 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_atiixp.ko 2020-11-20T12:07:07.067 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_atp867x.ko 2020-11-20T12:07:07.068 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cmd640.ko 2020-11-20T12:07:07.068 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cmd64x.ko 2020-11-20T12:07:07.068 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_cypress.ko 2020-11-20T12:07:07.068 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_efar.ko 2020-11-20T12:07:07.069 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt366.ko 2020-11-20T12:07:07.069 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt37x.ko 2020-11-20T12:07:07.069 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt3x2n.ko 2020-11-20T12:07:07.069 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_hpt3x3.ko 2020-11-20T12:07:07.070 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_it8213.ko 2020-11-20T12:07:07.070 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_it821x.ko 2020-11-20T12:07:07.070 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_jmicron.ko 2020-11-20T12:07:07.070 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_legacy.ko 2020-11-20T12:07:07.070 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_marvell.ko 2020-11-20T12:07:07.071 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_mpiix.ko 2020-11-20T12:07:07.071 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_netcell.ko 2020-11-20T12:07:07.071 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ninja32.ko 2020-11-20T12:07:07.072 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ns87410.ko 2020-11-20T12:07:07.073 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_ns87415.ko 2020-11-20T12:07:07.073 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_oldpiix.ko 2020-11-20T12:07:07.073 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_opti.ko 2020-11-20T12:07:07.073 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_optidma.ko 2020-11-20T12:07:07.073 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pcmcia.ko 2020-11-20T12:07:07.077 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pdc2027x.ko 2020-11-20T12:07:07.077 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_pdc202xx_old.ko 2020-11-20T12:07:07.077 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_piccolo.ko 2020-11-20T12:07:07.077 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_platform.ko 2020-11-20T12:07:07.078 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_radisys.ko 2020-11-20T12:07:07.078 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_rdc.ko 2020-11-20T12:07:07.078 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_rz1000.ko 2020-11-20T12:07:07.078 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sch.ko 2020-11-20T12:07:07.078 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_serverworks.ko 2020-11-20T12:07:07.079 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sil680.ko 2020-11-20T12:07:07.079 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_sl82c105.ko 2020-11-20T12:07:07.079 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_triflex.ko 2020-11-20T12:07:07.080 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pata_via.ko 2020-11-20T12:07:07.080 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/pdc_adma.ko 2020-11-20T12:07:07.081 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_inic162x.ko 2020-11-20T12:07:07.081 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 76097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_mv.ko 2020-11-20T12:07:07.082 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_nv.ko 2020-11-20T12:07:07.082 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_promise.ko 2020-11-20T12:07:07.082 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_qstor.ko 2020-11-20T12:07:07.083 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sil.ko 2020-11-20T12:07:07.083 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sil24.ko 2020-11-20T12:07:07.084 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sis.ko 2020-11-20T12:07:07.084 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_svw.ko 2020-11-20T12:07:07.084 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_sx4.ko 2020-11-20T12:07:07.085 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_uli.ko 2020-11-20T12:07:07.085 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_via.ko 2020-11-20T12:07:07.086 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ata/sata_vsc.ko 2020-11-20T12:07:07.086 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/auxdisplay/ 2020-11-20T12:07:07.086 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/auxdisplay/ks0108.ko 2020-11-20T12:07:07.087 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/bcma/ 2020-11-20T12:07:07.087 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 123697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/bcma/bcma.ko 2020-11-20T12:07:07.088 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/ 2020-11-20T12:07:07.088 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/aoe/ 2020-11-20T12:07:07.088 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 115129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/aoe/aoe.ko 2020-11-20T12:07:07.089 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/cryptoloop.ko 2020-11-20T12:07:07.089 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/drbd/ 2020-11-20T12:07:07.090 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1050545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/drbd/drbd.ko 2020-11-20T12:07:07.091 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 176513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/floppy.ko 2020-11-20T12:07:07.091 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/mtip32xx/ 2020-11-20T12:07:07.092 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 107353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/mtip32xx/mtip32xx.ko 2020-11-20T12:07:07.092 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 112145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/nbd.ko 2020-11-20T12:07:07.093 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 92265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/null_blk.ko 2020-11-20T12:07:07.094 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 83089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/pktcdvd.ko 2020-11-20T12:07:07.094 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 260233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rbd.ko 2020-11-20T12:07:07.097 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rsxx/ 2020-11-20T12:07:07.097 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 113017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/rsxx/rsxx.ko 2020-11-20T12:07:07.098 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 140897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/skd.ko 2020-11-20T12:07:07.100 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/sx8.ko 2020-11-20T12:07:07.101 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/umem.ko 2020-11-20T12:07:07.101 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/xen-blkback/ 2020-11-20T12:07:07.102 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 101497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/xen-blkback/xen-blkback.ko 2020-11-20T12:07:07.103 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/zram/ 2020-11-20T12:07:07.103 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/block/zram/zram.ko 2020-11-20T12:07:07.105 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ 2020-11-20T12:07:07.106 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/applicom.ko 2020-11-20T12:07:07.106 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hangcheck-timer.ko 2020-11-20T12:07:07.107 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/ 2020-11-20T12:07:07.107 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/amd-rng.ko 2020-11-20T12:07:07.107 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/intel-rng.ko 2020-11-20T12:07:07.108 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/timeriomem-rng.ko 2020-11-20T12:07:07.108 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/via-rng.ko 2020-11-20T12:07:07.108 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/hw_random/virtio-rng.ko 2020-11-20T12:07:07.109 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ 2020-11-20T12:07:07.109 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_devintf.ko 2020-11-20T12:07:07.110 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 138969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2020-11-20T12:07:07.111 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2020-11-20T12:07:07.112 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 142913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_si.ko 2020-11-20T12:07:07.112 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_ssif.ko 2020-11-20T12:07:07.113 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2020-11-20T12:07:07.113 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/lp.ko 2020-11-20T12:07:07.113 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/mwave/ 2020-11-20T12:07:07.114 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 88665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/mwave/mwave.ko 2020-11-20T12:07:07.116 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/nvram.ko 2020-11-20T12:07:07.116 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/ 2020-11-20T12:07:07.116 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 102257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/cm4000_cs.ko 2020-11-20T12:07:07.117 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/cm4040_cs.ko 2020-11-20T12:07:07.117 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 86617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/pcmcia/synclink_cs.ko 2020-11-20T12:07:07.118 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/ppdev.ko 2020-11-20T12:07:07.118 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/raw.ko 2020-11-20T12:07:07.118 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tlclk.ko 2020-11-20T12:07:07.121 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/ 2020-11-20T12:07:07.121 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_atmel.ko 2020-11-20T12:07:07.121 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2020-11-20T12:07:07.122 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2020-11-20T12:07:07.122 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2020-11-20T12:07:07.122 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_infineon.ko 2020-11-20T12:07:07.122 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/tpm_nsc.ko 2020-11-20T12:07:07.122 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/tpm/xen-tpmfront.ko 2020-11-20T12:07:07.126 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/ 2020-11-20T12:07:07.126 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/xillybus_core.ko 2020-11-20T12:07:07.126 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/char/xillybus/xillybus_pcie.ko 2020-11-20T12:07:07.126 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/ 2020-11-20T12:07:07.127 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-palmas.ko 2020-11-20T12:07:07.127 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-s2mps11.ko 2020-11-20T12:07:07.127 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-si5351.ko 2020-11-20T12:07:07.127 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-twl6040.ko 2020-11-20T12:07:07.128 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/clk/clk-wm831x.ko 2020-11-20T12:07:07.128 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/ 2020-11-20T12:07:07.128 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2020-11-20T12:07:07.128 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/p4-clockmod.ko 2020-11-20T12:07:07.128 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/cpufreq/speedstep-lib.ko 2020-11-20T12:07:07.129 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ 2020-11-20T12:07:07.129 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ 2020-11-20T12:07:07.129 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ccp-crypto.ko 2020-11-20T12:07:07.131 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 203753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/ccp/ccp.ko 2020-11-20T12:07:07.131 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/padlock-aes.ko 2020-11-20T12:07:07.131 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/padlock-sha.ko 2020-11-20T12:07:07.131 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/ 2020-11-20T12:07:07.131 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_common/ 2020-11-20T12:07:07.132 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 279065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_common/intel_qat.ko 2020-11-20T12:07:07.132 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_dh895xcc/ 2020-11-20T12:07:07.132 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/qat/qat_dh895xcc/qat_dh895xcc.ko 2020-11-20T12:07:07.132 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/virtio/ 2020-11-20T12:07:07.133 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/crypto/virtio/virtio_crypto.ko 2020-11-20T12:07:07.133 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dca/ 2020-11-20T12:07:07.133 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dca/dca.ko 2020-11-20T12:07:07.133 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ 2020-11-20T12:07:07.133 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/dw/ 2020-11-20T12:07:07.134 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/dw/dw_dmac.ko 2020-11-20T12:07:07.134 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ioat/ 2020-11-20T12:07:07.134 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 124161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/dma/ioat/ioatdma.ko 2020-11-20T12:07:07.135 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/ 2020-11-20T12:07:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/amd64_edac_mod.ko 2020-11-20T12:07:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/e752x_edac.ko 2020-11-20T12:07:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/edac_mce_amd.ko 2020-11-20T12:07:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i3000_edac.ko 2020-11-20T12:07:07.136 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i3200_edac.ko 2020-11-20T12:07:07.137 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5000_edac.ko 2020-11-20T12:07:07.137 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5100_edac.ko 2020-11-20T12:07:07.137 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i5400_edac.ko 2020-11-20T12:07:07.137 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i7300_edac.ko 2020-11-20T12:07:07.138 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i7core_edac.ko 2020-11-20T12:07:07.139 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/i82975x_edac.ko 2020-11-20T12:07:07.140 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/ie31200_edac.ko 2020-11-20T12:07:07.140 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/sb_edac.ko 2020-11-20T12:07:07.140 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/edac/x38_edac.ko 2020-11-20T12:07:07.140 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/ 2020-11-20T12:07:07.140 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-gpio.ko 2020-11-20T12:07:07.141 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max14577.ko 2020-11-20T12:07:07.141 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max77693.ko 2020-11-20T12:07:07.141 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-max8997.ko 2020-11-20T12:07:07.141 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-palmas.ko 2020-11-20T12:07:07.142 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-rt8973a.ko 2020-11-20T12:07:07.142 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/extcon/extcon-sm5502.ko 2020-11-20T12:07:07.142 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/ 2020-11-20T12:07:07.142 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/dmi-sysfs.ko 2020-11-20T12:07:07.142 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/efi/ 2020-11-20T12:07:07.143 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/efi/efi-pstore.ko 2020-11-20T12:07:07.143 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/firmware/iscsi_ibft.ko 2020-11-20T12:07:07.146 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/ 2020-11-20T12:07:07.147 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-adp5520.ko 2020-11-20T12:07:07.147 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-adp5588.ko 2020-11-20T12:07:07.147 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-amd8111.ko 2020-11-20T12:07:07.147 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-arizona.ko 2020-11-20T12:07:07.147 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-da9052.ko 2020-11-20T12:07:07.148 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-da9055.ko 2020-11-20T12:07:07.148 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-dln2.ko 2020-11-20T12:07:07.148 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-f7188x.ko 2020-11-20T12:07:07.148 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-generic.ko 2020-11-20T12:07:07.149 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-ich.ko 2020-11-20T12:07:07.149 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-janz-ttl.ko 2020-11-20T12:07:07.149 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-kempld.ko 2020-11-20T12:07:07.149 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-lp3943.ko 2020-11-20T12:07:07.149 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max7300.ko 2020-11-20T12:07:07.150 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max730x.ko 2020-11-20T12:07:07.150 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-max732x.ko 2020-11-20T12:07:07.150 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-ml-ioh.ko 2020-11-20T12:07:07.150 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-pca953x.ko 2020-11-20T12:07:07.151 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-pcf857x.ko 2020-11-20T12:07:07.151 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-rdc321x.ko 2020-11-20T12:07:07.151 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-sch.ko 2020-11-20T12:07:07.151 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-sch311x.ko 2020-11-20T12:07:07.151 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-tps65912.ko 2020-11-20T12:07:07.152 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-twl4030.ko 2020-11-20T12:07:07.152 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-twl6040.ko 2020-11-20T12:07:07.152 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-viperboard.ko 2020-11-20T12:07:07.152 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-vx855.ko 2020-11-20T12:07:07.152 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm831x.ko 2020-11-20T12:07:07.153 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm8350.ko 2020-11-20T12:07:07.153 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/gpio/gpio-wm8994.ko 2020-11-20T12:07:07.153 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/ 2020-11-20T12:07:07.153 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-a4tech.ko 2020-11-20T12:07:07.154 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-apple.ko 2020-11-20T12:07:07.154 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-appleir.ko 2020-11-20T12:07:07.154 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-aureal.ko 2020-11-20T12:07:07.154 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-axff.ko 2020-11-20T12:07:07.154 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-belkin.ko 2020-11-20T12:07:07.155 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cherry.ko 2020-11-20T12:07:07.155 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-chicony.ko 2020-11-20T12:07:07.155 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cp2112.ko 2020-11-20T12:07:07.160 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-cypress.ko 2020-11-20T12:07:07.160 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-dr.ko 2020-11-20T12:07:07.160 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-elecom.ko 2020-11-20T12:07:07.160 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-elo.ko 2020-11-20T12:07:07.161 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-emsff.ko 2020-11-20T12:07:07.161 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ezkey.ko 2020-11-20T12:07:07.161 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gaff.ko 2020-11-20T12:07:07.161 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-generic.ko 2020-11-20T12:07:07.162 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gt683r.ko 2020-11-20T12:07:07.162 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-gyration.ko 2020-11-20T12:07:07.162 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtek-kbd.ko 2020-11-20T12:07:07.162 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtek-mouse.ko 2020-11-20T12:07:07.162 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-holtekff.ko 2020-11-20T12:07:07.163 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-hyperv.ko 2020-11-20T12:07:07.163 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-icade.ko 2020-11-20T12:07:07.163 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-kensington.ko 2020-11-20T12:07:07.163 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-keytouch.ko 2020-11-20T12:07:07.163 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-kye.ko 2020-11-20T12:07:07.164 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lcpower.ko 2020-11-20T12:07:07.164 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-led.ko 2020-11-20T12:07:07.164 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lenovo.ko 2020-11-20T12:07:07.164 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-lg-g15.ko 2020-11-20T12:07:07.165 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech-dj.ko 2020-11-20T12:07:07.165 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech-hidpp.ko 2020-11-20T12:07:07.165 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 93089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-logitech.ko 2020-11-20T12:07:07.165 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-magicmouse.ko 2020-11-20T12:07:07.165 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-microsoft.ko 2020-11-20T12:07:07.166 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-monterey.ko 2020-11-20T12:07:07.166 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-multitouch.ko 2020-11-20T12:07:07.166 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ntrig.ko 2020-11-20T12:07:07.166 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-ortek.ko 2020-11-20T12:07:07.166 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-penmount.ko 2020-11-20T12:07:07.167 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-petalynx.ko 2020-11-20T12:07:07.167 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-picolcd.ko 2020-11-20T12:07:07.167 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-pl.ko 2020-11-20T12:07:07.167 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-plantronics.ko 2020-11-20T12:07:07.168 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-primax.ko 2020-11-20T12:07:07.168 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-rmi.ko 2020-11-20T12:07:07.168 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-arvo.ko 2020-11-20T12:07:07.168 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-common.ko 2020-11-20T12:07:07.168 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-isku.ko 2020-11-20T12:07:07.169 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-kone.ko 2020-11-20T12:07:07.169 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-koneplus.ko 2020-11-20T12:07:07.169 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-konepure.ko 2020-11-20T12:07:07.169 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-kovaplus.ko 2020-11-20T12:07:07.169 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-lua.ko 2020-11-20T12:07:07.170 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-pyra.ko 2020-11-20T12:07:07.170 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-ryos.ko 2020-11-20T12:07:07.170 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat-savu.ko 2020-11-20T12:07:07.170 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-roccat.ko 2020-11-20T12:07:07.171 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-saitek.ko 2020-11-20T12:07:07.171 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-samsung.ko 2020-11-20T12:07:07.171 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sensor-hub.ko 2020-11-20T12:07:07.171 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sjoy.ko 2020-11-20T12:07:07.171 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sony.ko 2020-11-20T12:07:07.174 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-speedlink.ko 2020-11-20T12:07:07.174 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-steelseries.ko 2020-11-20T12:07:07.174 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-sunplus.ko 2020-11-20T12:07:07.174 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-tivo.ko 2020-11-20T12:07:07.175 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-tmff.ko 2020-11-20T12:07:07.175 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-topseed.ko 2020-11-20T12:07:07.175 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-twinhan.ko 2020-11-20T12:07:07.175 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-uclogic.ko 2020-11-20T12:07:07.176 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-waltop.ko 2020-11-20T12:07:07.176 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 133729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-wiimote.ko 2020-11-20T12:07:07.176 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-xinmo.ko 2020-11-20T12:07:07.176 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-zpff.ko 2020-11-20T12:07:07.176 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid-zydacron.ko 2020-11-20T12:07:07.177 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 284097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/hid.ko 2020-11-20T12:07:07.177 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/i2c-hid/ 2020-11-20T12:07:07.177 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/i2c-hid/i2c-hid.ko 2020-11-20T12:07:07.177 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/uhid.ko 2020-11-20T12:07:07.178 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/ 2020-11-20T12:07:07.178 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 125177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbhid.ko 2020-11-20T12:07:07.178 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbkbd.ko 2020-11-20T12:07:07.178 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/usbhid/usbmouse.ko 2020-11-20T12:07:07.178 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 222905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hid/wacom.ko 2020-11-20T12:07:07.179 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/ 2020-11-20T12:07:07.179 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_balloon.ko 2020-11-20T12:07:07.179 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_utils.ko 2020-11-20T12:07:07.180 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 277809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hv/hv_vmbus.ko 2020-11-20T12:07:07.181 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ 2020-11-20T12:07:07.181 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 65649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/abituguru.ko 2020-11-20T12:07:07.181 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/abituguru3.ko 2020-11-20T12:07:07.181 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/acpi_power_meter.ko 2020-11-20T12:07:07.181 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ad7414.ko 2020-11-20T12:07:07.182 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ad7418.ko 2020-11-20T12:07:07.182 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adc128d818.ko 2020-11-20T12:07:07.182 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1021.ko 2020-11-20T12:07:07.182 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1025.ko 2020-11-20T12:07:07.183 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 98513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1026.ko 2020-11-20T12:07:07.184 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1029.ko 2020-11-20T12:07:07.184 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm1031.ko 2020-11-20T12:07:07.184 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adm9240.ko 2020-11-20T12:07:07.184 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ads7828.ko 2020-11-20T12:07:07.185 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7410.ko 2020-11-20T12:07:07.185 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7411.ko 2020-11-20T12:07:07.185 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 75745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7462.ko 2020-11-20T12:07:07.185 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7470.ko 2020-11-20T12:07:07.186 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7475.ko 2020-11-20T12:07:07.186 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/adt7x10.ko 2020-11-20T12:07:07.187 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/amc6821.ko 2020-11-20T12:07:07.187 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/applesmc.ko 2020-11-20T12:07:07.187 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asb100.ko 2020-11-20T12:07:07.187 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asc7621.ko 2020-11-20T12:07:07.188 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/asus_atk0110.ko 2020-11-20T12:07:07.188 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/atxp1.ko 2020-11-20T12:07:07.189 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/coretemp.ko 2020-11-20T12:07:07.189 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/da9052-hwmon.ko 2020-11-20T12:07:07.189 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/da9055-hwmon.ko 2020-11-20T12:07:07.190 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/dell-smm-hwmon.ko 2020-11-20T12:07:07.190 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 87601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/dme1737.ko 2020-11-20T12:07:07.190 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ds1621.ko 2020-11-20T12:07:07.190 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ds620.ko 2020-11-20T12:07:07.191 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc1403.ko 2020-11-20T12:07:07.191 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc2103.ko 2020-11-20T12:07:07.192 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/emc6w201.ko 2020-11-20T12:07:07.192 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f71805f.ko 2020-11-20T12:07:07.192 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 94745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f71882fg.ko 2020-11-20T12:07:07.193 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/f75375s.ko 2020-11-20T12:07:07.193 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/fam15h_power.ko 2020-11-20T12:07:07.193 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 54809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/fschmd.ko 2020-11-20T12:07:07.194 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/g760a.ko 2020-11-20T12:07:07.194 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/g762.ko 2020-11-20T12:07:07.195 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/gl518sm.ko 2020-11-20T12:07:07.195 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/gl520sm.ko 2020-11-20T12:07:07.195 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/hih6130.ko 2020-11-20T12:07:07.196 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/hwmon-vid.ko 2020-11-20T12:07:07.196 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/i5500_temp.ko 2020-11-20T12:07:07.196 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/i5k_amb.ko 2020-11-20T12:07:07.196 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ibmaem.ko 2020-11-20T12:07:07.196 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ibmpex.ko 2020-11-20T12:07:07.197 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ina209.ko 2020-11-20T12:07:07.198 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ina2xx.ko 2020-11-20T12:07:07.198 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 109417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/it87.ko 2020-11-20T12:07:07.198 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/jc42.ko 2020-11-20T12:07:07.198 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/k10temp.ko 2020-11-20T12:07:07.199 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/k8temp.ko 2020-11-20T12:07:07.199 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lineage-pem.ko 2020-11-20T12:07:07.200 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm63.ko 2020-11-20T12:07:07.200 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm73.ko 2020-11-20T12:07:07.200 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm75.ko 2020-11-20T12:07:07.201 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm77.ko 2020-11-20T12:07:07.201 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm78.ko 2020-11-20T12:07:07.201 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm80.ko 2020-11-20T12:07:07.201 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm83.ko 2020-11-20T12:07:07.201 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm85.ko 2020-11-20T12:07:07.202 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm87.ko 2020-11-20T12:07:07.203 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm90.ko 2020-11-20T12:07:07.203 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm92.ko 2020-11-20T12:07:07.203 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 104761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm93.ko 2020-11-20T12:07:07.203 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95234.ko 2020-11-20T12:07:07.204 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95241.ko 2020-11-20T12:07:07.205 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/lm95245.ko 2020-11-20T12:07:07.205 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc2945.ko 2020-11-20T12:07:07.206 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4151.ko 2020-11-20T12:07:07.206 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4215.ko 2020-11-20T12:07:07.206 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4222.ko 2020-11-20T12:07:07.206 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4245.ko 2020-11-20T12:07:07.207 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4260.ko 2020-11-20T12:07:07.207 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ltc4261.ko 2020-11-20T12:07:07.207 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max16065.ko 2020-11-20T12:07:07.207 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max1619.ko 2020-11-20T12:07:07.207 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max1668.ko 2020-11-20T12:07:07.208 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max197.ko 2020-11-20T12:07:07.208 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6639.ko 2020-11-20T12:07:07.208 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6642.ko 2020-11-20T12:07:07.208 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6650.ko 2020-11-20T12:07:07.209 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/max6697.ko 2020-11-20T12:07:07.210 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/mc13783-adc.ko 2020-11-20T12:07:07.210 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/mcp3021.ko 2020-11-20T12:07:07.210 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2020-11-20T12:07:07.211 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct6683.ko 2020-11-20T12:07:07.211 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 124641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct6775.ko 2020-11-20T12:07:07.211 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/nct7802.ko 2020-11-20T12:07:07.211 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/ntc_thermistor.ko 2020-11-20T12:07:07.211 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pc87360.ko 2020-11-20T12:07:07.212 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pc87427.ko 2020-11-20T12:07:07.212 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pcf8591.ko 2020-11-20T12:07:07.212 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ 2020-11-20T12:07:07.212 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/adm1275.ko 2020-11-20T12:07:07.214 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/lm25066.ko 2020-11-20T12:07:07.214 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ltc2978.ko 2020-11-20T12:07:07.215 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max16064.ko 2020-11-20T12:07:07.215 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max34440.ko 2020-11-20T12:07:07.215 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/max8688.ko 2020-11-20T12:07:07.215 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/pmbus.ko 2020-11-20T12:07:07.216 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2020-11-20T12:07:07.216 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/tps40422.ko 2020-11-20T12:07:07.216 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ucd9000.ko 2020-11-20T12:07:07.216 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/ucd9200.ko 2020-11-20T12:07:07.216 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/pmbus/zl6100.ko 2020-11-20T12:07:07.217 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/powr1220.ko 2020-11-20T12:07:07.217 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch5627.ko 2020-11-20T12:07:07.217 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch5636.ko 2020-11-20T12:07:07.217 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sch56xx-common.ko 2020-11-20T12:07:07.218 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sht15.ko 2020-11-20T12:07:07.218 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sht21.ko 2020-11-20T12:07:07.218 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/shtc1.ko 2020-11-20T12:07:07.218 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/sis5595.ko 2020-11-20T12:07:07.218 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smm665.ko 2020-11-20T12:07:07.220 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47b397.ko 2020-11-20T12:07:07.220 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47m1.ko 2020-11-20T12:07:07.220 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/smsc47m192.ko 2020-11-20T12:07:07.221 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/thmc50.ko 2020-11-20T12:07:07.221 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp102.ko 2020-11-20T12:07:07.221 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp103.ko 2020-11-20T12:07:07.221 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp401.ko 2020-11-20T12:07:07.222 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/tmp421.ko 2020-11-20T12:07:07.222 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/via-cputemp.ko 2020-11-20T12:07:07.222 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/via686a.ko 2020-11-20T12:07:07.222 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/vt1211.ko 2020-11-20T12:07:07.222 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/vt8231.ko 2020-11-20T12:07:07.223 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83627ehf.ko 2020-11-20T12:07:07.223 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83627hf.ko 2020-11-20T12:07:07.223 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83781d.ko 2020-11-20T12:07:07.223 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83791d.ko 2020-11-20T12:07:07.225 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 71337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83792d.ko 2020-11-20T12:07:07.225 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 101689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83793.ko 2020-11-20T12:07:07.226 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 119921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83795.ko 2020-11-20T12:07:07.226 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83l785ts.ko 2020-11-20T12:07:07.226 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/w83l786ng.ko 2020-11-20T12:07:07.226 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/wm831x-hwmon.ko 2020-11-20T12:07:07.227 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/hwmon/wm8350-hwmon.ko 2020-11-20T12:07:07.227 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/ 2020-11-20T12:07:07.227 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/ 2020-11-20T12:07:07.227 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2020-11-20T12:07:07.227 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2020-11-20T12:07:07.228 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/ 2020-11-20T12:07:07.228 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali1535.ko 2020-11-20T12:07:07.228 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali1563.ko 2020-11-20T12:07:07.228 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2020-11-20T12:07:07.229 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2020-11-20T12:07:07.229 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd756.ko 2020-11-20T12:07:07.229 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-amd8111.ko 2020-11-20T12:07:07.229 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2020-11-20T12:07:07.229 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2020-11-20T12:07:07.232 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-core.ko 2020-11-20T12:07:07.232 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2020-11-20T12:07:07.232 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2020-11-20T12:07:07.233 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2020-11-20T12:07:07.233 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-dln2.ko 2020-11-20T12:07:07.233 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-gpio.ko 2020-11-20T12:07:07.233 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 64897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-i801.ko 2020-11-20T12:07:07.234 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-isch.ko 2020-11-20T12:07:07.234 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ismt.ko 2020-11-20T12:07:07.234 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-kempld.ko 2020-11-20T12:07:07.234 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2020-11-20T12:07:07.234 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-nforce2.ko 2020-11-20T12:07:07.235 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-ocores.ko 2020-11-20T12:07:07.235 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-parport.ko 2020-11-20T12:07:07.235 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2020-11-20T12:07:07.235 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-piix4.ko 2020-11-20T12:07:07.236 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2020-11-20T12:07:07.236 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-scmi.ko 2020-11-20T12:07:07.236 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-simtec.ko 2020-11-20T12:07:07.236 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis5595.ko 2020-11-20T12:07:07.236 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis630.ko 2020-11-20T12:07:07.237 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-sis96x.ko 2020-11-20T12:07:07.237 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2020-11-20T12:07:07.237 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2020-11-20T12:07:07.237 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-via.ko 2020-11-20T12:07:07.238 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-viapro.ko 2020-11-20T12:07:07.238 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-viperboard.ko 2020-11-20T12:07:07.238 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/busses/i2c-xiic.ko 2020-11-20T12:07:07.239 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-mux.ko 2020-11-20T12:07:07.239 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-smbus.ko 2020-11-20T12:07:07.240 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/i2c-stub.ko 2020-11-20T12:07:07.240 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/ 2020-11-20T12:07:07.240 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2020-11-20T12:07:07.240 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2020-11-20T12:07:07.240 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2020-11-20T12:07:07.241 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ 2020-11-20T12:07:07.241 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ 2020-11-20T12:07:07.241 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 259017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_cm.ko 2020-11-20T12:07:07.241 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 848849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_core.ko 2020-11-20T12:07:07.247 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_umad.ko 2020-11-20T12:07:07.247 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 309825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/ib_uverbs.ko 2020-11-20T12:07:07.251 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 102945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/iw_cm.ko 2020-11-20T12:07:07.252 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 263657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/rdma_cm.ko 2020-11-20T12:07:07.255 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/core/rdma_ucm.ko 2020-11-20T12:07:07.256 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ 2020-11-20T12:07:07.256 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/cxgb4/ 2020-11-20T12:07:07.257 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 512473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2020-11-20T12:07:07.263 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx4/ 2020-11-20T12:07:07.263 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 420329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2020-11-20T12:07:07.269 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx5/ 2020-11-20T12:07:07.269 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 721305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2020-11-20T12:07:07.278 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mthca/ 2020-11-20T12:07:07.278 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 287121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2020-11-20T12:07:07.282 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ocrdma/ 2020-11-20T12:07:07.282 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 162257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2020-11-20T12:07:07.284 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/usnic/ 2020-11-20T12:07:07.284 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 177449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2020-11-20T12:07:07.286 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ 2020-11-20T12:07:07.286 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ipoib/ 2020-11-20T12:07:07.286 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 272041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2020-11-20T12:07:07.289 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/iser/ 2020-11-20T12:07:07.289 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 105369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2020-11-20T12:07:07.290 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/isert/ 2020-11-20T12:07:07.291 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 114745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2020-11-20T12:07:07.292 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srp/ 2020-11-20T12:07:07.292 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 133665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2020-11-20T12:07:07.293 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srpt/ 2020-11-20T12:07:07.294 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 131153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2020-11-20T12:07:07.295 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/ 2020-11-20T12:07:07.295 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/evbug.ko 2020-11-20T12:07:07.296 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/ff-memless.ko 2020-11-20T12:07:07.297 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/ 2020-11-20T12:07:07.297 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/emu10k1-gp.ko 2020-11-20T12:07:07.297 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/fm801-gp.ko 2020-11-20T12:07:07.297 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/gameport.ko 2020-11-20T12:07:07.298 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/lightning.ko 2020-11-20T12:07:07.298 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/gameport/ns558.ko 2020-11-20T12:07:07.298 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/input-polldev.ko 2020-11-20T12:07:07.298 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/joydev.ko 2020-11-20T12:07:07.299 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/ 2020-11-20T12:07:07.300 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5520-keys.ko 2020-11-20T12:07:07.300 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5588-keys.ko 2020-11-20T12:07:07.300 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/adp5589-keys.ko 2020-11-20T12:07:07.300 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2020-11-20T12:07:07.301 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/gpio_keys.ko 2020-11-20T12:07:07.301 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2020-11-20T12:07:07.301 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lkkbd.ko 2020-11-20T12:07:07.301 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lm8323.ko 2020-11-20T12:07:07.302 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/lm8333.ko 2020-11-20T12:07:07.302 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/matrix_keypad.ko 2020-11-20T12:07:07.302 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/max7359_keypad.ko 2020-11-20T12:07:07.304 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/mcs_touchkey.ko 2020-11-20T12:07:07.304 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2020-11-20T12:07:07.304 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/newtonkbd.ko 2020-11-20T12:07:07.304 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/opencores-kbd.ko 2020-11-20T12:07:07.305 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/qt1070.ko 2020-11-20T12:07:07.305 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/qt2160.ko 2020-11-20T12:07:07.305 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/samsung-keypad.ko 2020-11-20T12:07:07.305 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/stowaway.ko 2020-11-20T12:07:07.306 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/sunkbd.ko 2020-11-20T12:07:07.306 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/tca6416-keypad.ko 2020-11-20T12:07:07.306 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/tca8418_keypad.ko 2020-11-20T12:07:07.306 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/twl4030_keypad.ko 2020-11-20T12:07:07.306 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/keyboard/xtkbd.ko 2020-11-20T12:07:07.307 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/matrix-keymap.ko 2020-11-20T12:07:07.307 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/ 2020-11-20T12:07:07.307 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/appletouch.ko 2020-11-20T12:07:07.307 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/bcm5974.ko 2020-11-20T12:07:07.307 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 124297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/cyapatp.ko 2020-11-20T12:07:07.308 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 94273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/elan_i2c.ko 2020-11-20T12:07:07.308 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/gpio_mouse.ko 2020-11-20T12:07:07.309 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 337393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/psmouse.ko 2020-11-20T12:07:07.310 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/sermouse.ko 2020-11-20T12:07:07.310 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/synaptics_i2c.ko 2020-11-20T12:07:07.310 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/synaptics_usb.ko 2020-11-20T12:07:07.311 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/mouse/vsxxxaa.ko 2020-11-20T12:07:07.311 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/rmi4/ 2020-11-20T12:07:07.311 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 113449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/rmi4/rmi_core.ko 2020-11-20T12:07:07.312 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ 2020-11-20T12:07:07.313 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/altera_ps2.ko 2020-11-20T12:07:07.313 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/arc_ps2.ko 2020-11-20T12:07:07.313 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ct82c710.ko 2020-11-20T12:07:07.313 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/hyperv-keyboard.ko 2020-11-20T12:07:07.314 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/parkbd.ko 2020-11-20T12:07:07.314 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/pcips2.ko 2020-11-20T12:07:07.314 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/ps2mult.ko 2020-11-20T12:07:07.314 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/serio_raw.ko 2020-11-20T12:07:07.316 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/serio/serport.ko 2020-11-20T12:07:07.317 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/input/sparse-keymap.ko 2020-11-20T12:07:07.317 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/ 2020-11-20T12:07:07.317 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/amd/ 2020-11-20T12:07:07.317 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/iommu/amd/iommu_v2.ko 2020-11-20T12:07:07.318 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/ 2020-11-20T12:07:07.318 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-88pm860x.ko 2020-11-20T12:07:07.318 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-adp5520.ko 2020-11-20T12:07:07.319 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-bd2802.ko 2020-11-20T12:07:07.319 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-blinkm.ko 2020-11-20T12:07:07.319 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-clevo-mail.ko 2020-11-20T12:07:07.319 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-da903x.ko 2020-11-20T12:07:07.320 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-da9052.ko 2020-11-20T12:07:07.320 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-gpio.ko 2020-11-20T12:07:07.320 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3530.ko 2020-11-20T12:07:07.320 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3533.ko 2020-11-20T12:07:07.320 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm355x.ko 2020-11-20T12:07:07.321 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lm3642.ko 2020-11-20T12:07:07.321 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lp3944.ko 2020-11-20T12:07:07.321 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-lp8788.ko 2020-11-20T12:07:07.321 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-max8997.ko 2020-11-20T12:07:07.322 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-mc13783.ko 2020-11-20T12:07:07.322 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-menf21bmc.ko 2020-11-20T12:07:07.322 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca9532.ko 2020-11-20T12:07:07.324 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca955x.ko 2020-11-20T12:07:07.324 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-pca963x.ko 2020-11-20T12:07:07.324 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-ss4200.ko 2020-11-20T12:07:07.325 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-tca6507.ko 2020-11-20T12:07:07.325 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-wm831x-status.ko 2020-11-20T12:07:07.325 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/leds-wm8350.ko 2020-11-20T12:07:07.325 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ 2020-11-20T12:07:07.325 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-audio.ko 2020-11-20T12:07:07.326 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2020-11-20T12:07:07.326 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-camera.ko 2020-11-20T12:07:07.326 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2020-11-20T12:07:07.326 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2020-11-20T12:07:07.327 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2020-11-20T12:07:07.327 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2020-11-20T12:07:07.327 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-timer.ko 2020-11-20T12:07:07.327 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/leds/trigger/ledtrig-transient.ko 2020-11-20T12:07:07.327 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/ 2020-11-20T12:07:07.328 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/bcache/ 2020-11-20T12:07:07.328 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 633113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/bcache/bcache.ko 2020-11-20T12:07:07.329 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-bio-prison.ko 2020-11-20T12:07:07.329 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-bufio.ko 2020-11-20T12:07:07.329 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-cache-smq.ko 2020-11-20T12:07:07.329 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 146689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-cache.ko 2020-11-20T12:07:07.330 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 98065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-crypt.ko 2020-11-20T12:07:07.332 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-delay.ko 2020-11-20T12:07:07.332 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-era.ko 2020-11-20T12:07:07.332 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-flakey.ko 2020-11-20T12:07:07.333 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-log-userspace.ko 2020-11-20T12:07:07.333 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-log.ko 2020-11-20T12:07:07.334 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-mirror.ko 2020-11-20T12:07:07.334 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 75609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-multipath.ko 2020-11-20T12:07:07.335 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-queue-length.ko 2020-11-20T12:07:07.335 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-raid.ko 2020-11-20T12:07:07.336 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-region-hash.ko 2020-11-20T12:07:07.336 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-round-robin.ko 2020-11-20T12:07:07.337 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-service-time.ko 2020-11-20T12:07:07.337 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 112913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-snapshot.ko 2020-11-20T12:07:07.338 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-switch.ko 2020-11-20T12:07:07.338 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 182441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-thin-pool.ko 2020-11-20T12:07:07.340 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-verity.ko 2020-11-20T12:07:07.341 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/dm-zero.ko 2020-11-20T12:07:07.341 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/faulty.ko 2020-11-20T12:07:07.341 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/linear.ko 2020-11-20T12:07:07.342 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/multipath.ko 2020-11-20T12:07:07.342 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/persistent-data/ 2020-11-20T12:07:07.343 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 179553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2020-11-20T12:07:07.343 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid0.ko 2020-11-20T12:07:07.344 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 111753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid1.ko 2020-11-20T12:07:07.345 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 156737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid10.ko 2020-11-20T12:07:07.347 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 385993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/md/raid456.ko 2020-11-20T12:07:07.353 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/ 2020-11-20T12:07:07.354 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/ 2020-11-20T12:07:07.355 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 194137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptbase.ko 2020-11-20T12:07:07.355 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptctl.ko 2020-11-20T12:07:07.356 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptfc.ko 2020-11-20T12:07:07.357 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptlan.ko 2020-11-20T12:07:07.357 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 129617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptsas.ko 2020-11-20T12:07:07.359 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptscsih.ko 2020-11-20T12:07:07.359 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/message/fusion/mptspi.ko 2020-11-20T12:07:07.361 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ 2020-11-20T12:07:07.361 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm800.ko 2020-11-20T12:07:07.361 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm805.ko 2020-11-20T12:07:07.361 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/88pm80x.ko 2020-11-20T12:07:07.362 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ab3100-otp.ko 2020-11-20T12:07:07.362 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/arizona-i2c.ko 2020-11-20T12:07:07.364 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/bcm590xx.ko 2020-11-20T12:07:07.364 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/cros_ec_dev.ko 2020-11-20T12:07:07.364 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/dln2.ko 2020-11-20T12:07:07.365 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/htc-pasic3.ko 2020-11-20T12:07:07.365 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/janz-cmodio.ko 2020-11-20T12:07:07.365 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/kempld-core.ko 2020-11-20T12:07:07.365 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lm3533-core.ko 2020-11-20T12:07:07.366 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lm3533-ctrlbank.ko 2020-11-20T12:07:07.366 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lp3943.ko 2020-11-20T12:07:07.366 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lpc_ich.ko 2020-11-20T12:07:07.366 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/lpc_sch.ko 2020-11-20T12:07:07.367 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/max8907.ko 2020-11-20T12:07:07.367 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/mc13xxx-core.ko 2020-11-20T12:07:07.367 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/mc13xxx-i2c.ko 2020-11-20T12:07:07.367 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/menf21bmc.ko 2020-11-20T12:07:07.368 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633-adc.ko 2020-11-20T12:07:07.368 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633-gpio.ko 2020-11-20T12:07:07.368 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/pcf50633.ko 2020-11-20T12:07:07.370 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/rdc321x-southbridge.ko 2020-11-20T12:07:07.370 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/retu-mfd.ko 2020-11-20T12:07:07.371 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 76449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/si476x-core.ko 2020-11-20T12:07:07.371 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/sm501.ko 2020-11-20T12:07:07.371 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/ti_am335x_tscadc.ko 2020-11-20T12:07:07.371 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/tps65010.ko 2020-11-20T12:07:07.372 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/tps6507x.ko 2020-11-20T12:07:07.372 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/viperboard.ko 2020-11-20T12:07:07.372 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/vx855.ko 2020-11-20T12:07:07.372 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/wl1273-core.ko 2020-11-20T12:07:07.373 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/mfd/wm8994.ko 2020-11-20T12:07:07.373 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ 2020-11-20T12:07:07.373 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ad525x_dpot-i2c.ko 2020-11-20T12:07:07.373 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ad525x_dpot.ko 2020-11-20T12:07:07.374 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/altera-stapl/ 2020-11-20T12:07:07.374 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2020-11-20T12:07:07.374 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/apds9802als.ko 2020-11-20T12:07:07.374 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/apds990x.ko 2020-11-20T12:07:07.374 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/bh1770glc.ko 2020-11-20T12:07:07.375 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/ 2020-11-20T12:07:07.375 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2020-11-20T12:07:07.375 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/c2port/core.ko 2020-11-20T12:07:07.378 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/cb710/ 2020-11-20T12:07:07.378 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/cb710/cb710.ko 2020-11-20T12:07:07.378 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ds1682.ko 2020-11-20T12:07:07.378 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/dummy-irq.ko 2020-11-20T12:07:07.379 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/echo/ 2020-11-20T12:07:07.379 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/echo/echo.ko 2020-11-20T12:07:07.379 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/ 2020-11-20T12:07:07.379 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/at24.ko 2020-11-20T12:07:07.380 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/eeprom.ko 2020-11-20T12:07:07.380 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2020-11-20T12:07:07.380 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/eeprom/max6875.ko 2020-11-20T12:07:07.380 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/enclosure.ko 2020-11-20T12:07:07.381 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/genwqe/ 2020-11-20T12:07:07.381 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 151177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/genwqe/genwqe_card.ko 2020-11-20T12:07:07.381 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/hmc6352.ko 2020-11-20T12:07:07.381 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/hpilo.ko 2020-11-20T12:07:07.381 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ibmasm/ 2020-11-20T12:07:07.382 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 71465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ibmasm/ibmasm.ko 2020-11-20T12:07:07.382 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ics932s401.ko 2020-11-20T12:07:07.382 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/isl29003.ko 2020-11-20T12:07:07.382 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/isl29020.ko 2020-11-20T12:07:07.383 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/ 2020-11-20T12:07:07.383 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2020-11-20T12:07:07.383 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2020-11-20T12:07:07.383 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/ 2020-11-20T12:07:07.384 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 121553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei-me.ko 2020-11-20T12:07:07.385 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei-txe.ko 2020-11-20T12:07:07.385 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 279201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/mei/mei.ko 2020-11-20T12:07:07.385 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/phantom.ko 2020-11-20T12:07:07.386 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/pvpanic.ko 2020-11-20T12:07:07.386 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ti-st/ 2020-11-20T12:07:07.386 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/ti-st/st_drv.ko 2020-11-20T12:07:07.386 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tifm_7xx1.ko 2020-11-20T12:07:07.387 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tifm_core.ko 2020-11-20T12:07:07.387 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/tsl2550.ko 2020-11-20T12:07:07.387 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_balloon.ko 2020-11-20T12:07:07.387 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_vmci/ 2020-11-20T12:07:07.387 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 179385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2020-11-20T12:07:07.388 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ 2020-11-20T12:07:07.388 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/bonding/ 2020-11-20T12:07:07.389 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 433953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/bonding/bonding.ko 2020-11-20T12:07:07.393 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/ 2020-11-20T12:07:07.393 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/b53/ 2020-11-20T12:07:07.393 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 96929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/b53/b53_common.ko 2020-11-20T12:07:07.394 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/bcm-sf2.ko 2020-11-20T12:07:07.395 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6060.ko 2020-11-20T12:07:07.395 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6xxx/ 2020-11-20T12:07:07.395 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 346377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2020-11-20T12:07:07.398 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/dummy.ko 2020-11-20T12:07:07.399 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/eql.ko 2020-11-20T12:07:07.399 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ 2020-11-20T12:07:07.399 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/ 2020-11-20T12:07:07.400 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2020-11-20T12:07:07.400 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2020-11-20T12:07:07.400 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 123353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/3c59x.ko 2020-11-20T12:07:07.401 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/3com/typhoon.ko 2020-11-20T12:07:07.402 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/ 2020-11-20T12:07:07.402 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/8390.ko 2020-11-20T12:07:07.403 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2020-11-20T12:07:07.403 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2020-11-20T12:07:07.404 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 96881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2020-11-20T12:07:07.404 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/adaptec/ 2020-11-20T12:07:07.405 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/adaptec/starfire.ko 2020-11-20T12:07:07.405 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/agere/ 2020-11-20T12:07:07.406 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/agere/et131x.ko 2020-11-20T12:07:07.406 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/alteon/ 2020-11-20T12:07:07.406 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/alteon/acenic.ko 2020-11-20T12:07:07.407 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/altera/ 2020-11-20T12:07:07.407 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/altera/altera_tse.ko 2020-11-20T12:07:07.408 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/ 2020-11-20T12:07:07.408 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/amd8111e.ko 2020-11-20T12:07:07.408 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2020-11-20T12:07:07.409 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/amd/pcnet32.ko 2020-11-20T12:07:07.409 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/ 2020-11-20T12:07:07.410 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/alx/ 2020-11-20T12:07:07.410 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 91593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2020-11-20T12:07:07.412 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1c/ 2020-11-20T12:07:07.412 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 98001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2020-11-20T12:07:07.413 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1e/ 2020-11-20T12:07:07.413 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 89785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2020-11-20T12:07:07.414 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/ 2020-11-20T12:07:07.414 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 86737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2020-11-20T12:07:07.415 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2020-11-20T12:07:07.416 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/ 2020-11-20T12:07:07.416 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/b44.ko 2020-11-20T12:07:07.417 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 167025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2020-11-20T12:07:07.418 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2020-11-20T12:07:07.419 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1528577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2020-11-20T12:07:07.439 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 130113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/cnic.ko 2020-11-20T12:07:07.441 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/genet/ 2020-11-20T12:07:07.442 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 129345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2020-11-20T12:07:07.446 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 314625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/broadcom/tg3.ko 2020-11-20T12:07:07.447 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/ 2020-11-20T12:07:07.448 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/bna/ 2020-11-20T12:07:07.448 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 330209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2020-11-20T12:07:07.452 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/ 2020-11-20T12:07:07.452 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb/ 2020-11-20T12:07:07.452 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 187345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2020-11-20T12:07:07.454 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2020-11-20T12:07:07.454 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 349953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2020-11-20T12:07:07.459 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2020-11-20T12:07:07.459 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1039081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2020-11-20T12:07:07.472 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2020-11-20T12:07:07.473 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 165321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2020-11-20T12:07:07.474 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2020-11-20T12:07:07.474 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2020-11-20T12:07:07.475 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/ 2020-11-20T12:07:07.475 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/enic/ 2020-11-20T12:07:07.475 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 205121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2020-11-20T12:07:07.477 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/ 2020-11-20T12:07:07.477 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/ 2020-11-20T12:07:07.478 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2020-11-20T12:07:07.478 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 97801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/de4x5.ko 2020-11-20T12:07:07.479 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2020-11-20T12:07:07.480 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 147153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2020-11-20T12:07:07.482 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2020-11-20T12:07:07.482 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2020-11-20T12:07:07.483 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2020-11-20T12:07:07.484 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/ 2020-11-20T12:07:07.484 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/dl2k.ko 2020-11-20T12:07:07.484 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dlink/sundance.ko 2020-11-20T12:07:07.485 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/dnet.ko 2020-11-20T12:07:07.485 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ec_bhf.ko 2020-11-20T12:07:07.486 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/ 2020-11-20T12:07:07.486 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/benet/ 2020-11-20T12:07:07.486 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 287161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2020-11-20T12:07:07.489 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ethoc.ko 2020-11-20T12:07:07.490 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fealnx.ko 2020-11-20T12:07:07.490 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fujitsu/ 2020-11-20T12:07:07.491 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2020-11-20T12:07:07.491 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ 2020-11-20T12:07:07.491 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 91697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e100.ko 2020-11-20T12:07:07.492 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000/ 2020-11-20T12:07:07.492 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 293081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2020-11-20T12:07:07.496 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000e/ 2020-11-20T12:07:07.496 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 549313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2020-11-20T12:07:07.502 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/fm10k/ 2020-11-20T12:07:07.503 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 247433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2020-11-20T12:07:07.506 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/i40e/ 2020-11-20T12:07:07.506 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 892665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2020-11-20T12:07:07.518 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/iavf/ 2020-11-20T12:07:07.518 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 250857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2020-11-20T12:07:07.521 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igb/ 2020-11-20T12:07:07.521 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 509433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igb/igb.ko 2020-11-20T12:07:07.527 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igbvf/ 2020-11-20T12:07:07.527 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2020-11-20T12:07:07.528 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgb/ 2020-11-20T12:07:07.528 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 123905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgb/ixgb.ko 2020-11-20T12:07:07.529 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbe/ 2020-11-20T12:07:07.530 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 743497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2020-11-20T12:07:07.539 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbevf/ 2020-11-20T12:07:07.539 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 147065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2020-11-20T12:07:07.540 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 82449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/jme.ko 2020-11-20T12:07:07.542 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/ 2020-11-20T12:07:07.542 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2020-11-20T12:07:07.542 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 103057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/skge.ko 2020-11-20T12:07:07.543 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 130193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/marvell/sky2.ko 2020-11-20T12:07:07.545 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/ 2020-11-20T12:07:07.545 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/ 2020-11-20T12:07:07.545 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 739097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2020-11-20T12:07:07.555 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 273433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2020-11-20T12:07:07.558 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/ 2020-11-20T12:07:07.559 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2020-11-20T12:07:07.559 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 920849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2020-11-20T12:07:07.569 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ 2020-11-20T12:07:07.569 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ks8842.ko 2020-11-20T12:07:07.570 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ks8851_mll.ko 2020-11-20T12:07:07.570 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 94705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2020-11-20T12:07:07.571 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/ 2020-11-20T12:07:07.572 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/myri10ge/ 2020-11-20T12:07:07.572 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 112513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2020-11-20T12:07:07.573 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/ 2020-11-20T12:07:07.573 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 75729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2020-11-20T12:07:07.574 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2020-11-20T12:07:07.574 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/ 2020-11-20T12:07:07.575 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 156657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/s2io.ko 2020-11-20T12:07:07.577 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/vxge/ 2020-11-20T12:07:07.577 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 221913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/neterion/vxge/vxge.ko 2020-11-20T12:07:07.580 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/nvidia/ 2020-11-20T12:07:07.580 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 142057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2020-11-20T12:07:07.581 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/ 2020-11-20T12:07:07.582 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2020-11-20T12:07:07.582 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2020-11-20T12:07:07.583 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/ 2020-11-20T12:07:07.583 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/netxen/ 2020-11-20T12:07:07.583 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 242793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2020-11-20T12:07:07.585 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 88409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2020-11-20T12:07:07.587 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2020-11-20T12:07:07.587 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 706337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2020-11-20T12:07:07.595 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/rdc/ 2020-11-20T12:07:07.596 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/rdc/r6040.ko 2020-11-20T12:07:07.596 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/ 2020-11-20T12:07:07.597 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/8139cp.ko 2020-11-20T12:07:07.597 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 83753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/8139too.ko 2020-11-20T12:07:07.597 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/atp.ko 2020-11-20T12:07:07.598 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 173241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/realtek/r8169.ko 2020-11-20T12:07:07.600 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/ 2020-11-20T12:07:07.601 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/sxgbe/ 2020-11-20T12:07:07.601 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 104689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2020-11-20T12:07:07.602 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sfc/ 2020-11-20T12:07:07.602 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 824569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sfc/sfc.ko 2020-11-20T12:07:07.612 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/silan/ 2020-11-20T12:07:07.612 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/silan/sc92031.ko 2020-11-20T12:07:07.613 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/ 2020-11-20T12:07:07.613 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/sis190.ko 2020-11-20T12:07:07.614 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 64625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sis/sis900.ko 2020-11-20T12:07:07.614 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/ 2020-11-20T12:07:07.614 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/epic100.ko 2020-11-20T12:07:07.615 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2020-11-20T12:07:07.615 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2020-11-20T12:07:07.616 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2020-11-20T12:07:07.617 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/ 2020-11-20T12:07:07.617 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/ 2020-11-20T12:07:07.617 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2020-11-20T12:07:07.618 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2020-11-20T12:07:07.618 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2020-11-20T12:07:07.619 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 381753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2020-11-20T12:07:07.622 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/ 2020-11-20T12:07:07.623 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 106913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/cassini.ko 2020-11-20T12:07:07.624 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 176753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/niu.ko 2020-11-20T12:07:07.626 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/sungem.ko 2020-11-20T12:07:07.627 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/sun/sunhme.ko 2020-11-20T12:07:07.628 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/tehuti/ 2020-11-20T12:07:07.628 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2020-11-20T12:07:07.629 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ti/ 2020-11-20T12:07:07.629 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/ti/tlan.ko 2020-11-20T12:07:07.629 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/ 2020-11-20T12:07:07.630 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 82449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/via-rhine.ko 2020-11-20T12:07:07.630 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 86073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/via/via-velocity.ko 2020-11-20T12:07:07.631 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/ 2020-11-20T12:07:07.632 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/w5100.ko 2020-11-20T12:07:07.632 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/wiznet/w5300.ko 2020-11-20T12:07:07.633 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/xircom/ 2020-11-20T12:07:07.633 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2020-11-20T12:07:07.634 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/ 2020-11-20T12:07:07.634 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/defxx.ko 2020-11-20T12:07:07.634 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/skfp/ 2020-11-20T12:07:07.634 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 236713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/fddi/skfp/skfp.ko 2020-11-20T12:07:07.637 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/geneve.ko 2020-11-20T12:07:07.638 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/hyperv/ 2020-11-20T12:07:07.638 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 227313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/hyperv/hv_netvsc.ko 2020-11-20T12:07:07.640 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ieee802154/ 2020-11-20T12:07:07.641 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ieee802154/fakelb.ko 2020-11-20T12:07:07.641 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ifb.ko 2020-11-20T12:07:07.641 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ipvlan/ 2020-11-20T12:07:07.642 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 84633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ipvlan/ipvlan.ko 2020-11-20T12:07:07.643 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/macvlan.ko 2020-11-20T12:07:07.643 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/macvtap.ko 2020-11-20T12:07:07.643 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/ 2020-11-20T12:07:07.643 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2020-11-20T12:07:07.644 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-bitbang.ko 2020-11-20T12:07:07.645 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio/mdio-gpio.ko 2020-11-20T12:07:07.647 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mdio.ko 2020-11-20T12:07:07.647 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/mii.ko 2020-11-20T12:07:07.648 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/netconsole.ko 2020-11-20T12:07:07.648 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/nlmon.ko 2020-11-20T12:07:07.648 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/pcs/ 2020-11-20T12:07:07.648 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/pcs/pcs-xpcs.ko 2020-11-20T12:07:07.649 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/ 2020-11-20T12:07:07.649 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/amd.ko 2020-11-20T12:07:07.649 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm-phy-lib.ko 2020-11-20T12:07:07.649 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm7xxx.ko 2020-11-20T12:07:07.650 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/bcm87xx.ko 2020-11-20T12:07:07.650 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/broadcom.ko 2020-11-20T12:07:07.650 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/cicada.ko 2020-11-20T12:07:07.650 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/davicom.ko 2020-11-20T12:07:07.650 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/et1011c.ko 2020-11-20T12:07:07.651 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/icplus.ko 2020-11-20T12:07:07.651 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/lxt.ko 2020-11-20T12:07:07.651 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 64985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/marvell.ko 2020-11-20T12:07:07.651 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/micrel.ko 2020-11-20T12:07:07.651 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/national.ko 2020-11-20T12:07:07.652 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/phylink.ko 2020-11-20T12:07:07.654 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/qsemi.ko 2020-11-20T12:07:07.654 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/realtek.ko 2020-11-20T12:07:07.654 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/smsc.ko 2020-11-20T12:07:07.655 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/ste10Xp.ko 2020-11-20T12:07:07.655 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/phy/vitesse.ko 2020-11-20T12:07:07.655 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/plip/ 2020-11-20T12:07:07.655 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/plip/plip.ko 2020-11-20T12:07:07.656 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ 2020-11-20T12:07:07.656 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/bsd_comp.ko 2020-11-20T12:07:07.656 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_async.ko 2020-11-20T12:07:07.656 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_deflate.ko 2020-11-20T12:07:07.656 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_mppe.ko 2020-11-20T12:07:07.657 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/ppp_synctty.ko 2020-11-20T12:07:07.657 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pppoe.ko 2020-11-20T12:07:07.657 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pppox.ko 2020-11-20T12:07:07.657 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/ppp/pptp.ko 2020-11-20T12:07:07.658 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/sb1000.ko 2020-11-20T12:07:07.658 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/slip/ 2020-11-20T12:07:07.658 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/slip/slip.ko 2020-11-20T12:07:07.658 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/sungem_phy.ko 2020-11-20T12:07:07.659 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/tap.ko 2020-11-20T12:07:07.659 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/ 2020-11-20T12:07:07.659 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 102721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team.ko 2020-11-20T12:07:07.660 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_activebackup.ko 2020-11-20T12:07:07.661 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_broadcast.ko 2020-11-20T12:07:07.661 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_loadbalance.ko 2020-11-20T12:07:07.661 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_random.ko 2020-11-20T12:07:07.661 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/team/team_mode_roundrobin.ko 2020-11-20T12:07:07.661 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/veth.ko 2020-11-20T12:07:07.662 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vmxnet3/ 2020-11-20T12:07:07.662 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 116905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vmxnet3/vmxnet3.ko 2020-11-20T12:07:07.662 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 171137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/vxlan.ko 2020-11-20T12:07:07.662 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/xen-netback/ 2020-11-20T12:07:07.663 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 143721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/net/xen-netback/xen-netback.ko 2020-11-20T12:07:07.663 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ntb/ 2020-11-20T12:07:07.663 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ntb/ntb.ko 2020-11-20T12:07:07.664 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/ 2020-11-20T12:07:07.664 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/ 2020-11-20T12:07:07.664 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 209665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/nvme-core.ko 2020-11-20T12:07:07.665 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 92745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/nvme/host/nvme.ko 2020-11-20T12:07:07.665 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/ 2020-11-20T12:07:07.665 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 130825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport.ko 2020-11-20T12:07:07.667 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_ax88796.ko 2020-11-20T12:07:07.667 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_cs.ko 2020-11-20T12:07:07.668 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 84865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_pc.ko 2020-11-20T12:07:07.669 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/parport/parport_serial.ko 2020-11-20T12:07:07.669 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/ 2020-11-20T12:07:07.669 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/ 2020-11-20T12:07:07.669 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2020-11-20T12:07:07.669 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2020-11-20T12:07:07.670 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2020-11-20T12:07:07.671 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/pci-stub.ko 2020-11-20T12:07:07.671 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pci/xen-pcifront.ko 2020-11-20T12:07:07.671 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/ 2020-11-20T12:07:07.671 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/i82092.ko 2020-11-20T12:07:07.671 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 150113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia.ko 2020-11-20T12:07:07.672 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia_core.ko 2020-11-20T12:07:07.672 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2020-11-20T12:07:07.673 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/pd6729.ko 2020-11-20T12:07:07.673 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 111129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pcmcia/yenta_socket.ko 2020-11-20T12:07:07.674 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/ 2020-11-20T12:07:07.675 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/broadcom/ 2020-11-20T12:07:07.675 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2020-11-20T12:07:07.675 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/samsung/ 2020-11-20T12:07:07.675 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2020-11-20T12:07:07.676 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/ 2020-11-20T12:07:07.676 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/intel/ 2020-11-20T12:07:07.676 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2020-11-20T12:07:07.677 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/ 2020-11-20T12:07:07.677 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/ 2020-11-20T12:07:07.678 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2020-11-20T12:07:07.678 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec.ko 2020-11-20T12:07:07.678 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2020-11-20T12:07:07.678 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2020-11-20T12:07:07.679 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2020-11-20T12:07:07.679 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2020-11-20T12:07:07.679 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2020-11-20T12:07:07.679 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/ 2020-11-20T12:07:07.679 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/acer-wmi.ko 2020-11-20T12:07:07.680 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/acerhdf.ko 2020-11-20T12:07:07.681 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/alienware-wmi.ko 2020-11-20T12:07:07.681 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/apple-gmux.ko 2020-11-20T12:07:07.681 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-laptop.ko 2020-11-20T12:07:07.682 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-nb-wmi.ko 2020-11-20T12:07:07.682 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 71465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/asus-wmi.ko 2020-11-20T12:07:07.682 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/classmate-laptop.ko 2020-11-20T12:07:07.682 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dcdbas.ko 2020-11-20T12:07:07.682 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell-smo8800.ko 2020-11-20T12:07:07.683 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell-wmi-aio.ko 2020-11-20T12:07:07.683 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/dell_rbu.ko 2020-11-20T12:07:07.684 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/eeepc-laptop.ko 2020-11-20T12:07:07.684 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/eeepc-wmi.ko 2020-11-20T12:07:07.685 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/fujitsu-laptop.ko 2020-11-20T12:07:07.685 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/fujitsu-tablet.ko 2020-11-20T12:07:07.685 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hdaps.ko 2020-11-20T12:07:07.685 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp-wireless.ko 2020-11-20T12:07:07.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp-wmi.ko 2020-11-20T12:07:07.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/hp_accel.ko 2020-11-20T12:07:07.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/ibm_rtl.ko 2020-11-20T12:07:07.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel-rst.ko 2020-11-20T12:07:07.686 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel-smartconnect.ko 2020-11-20T12:07:07.687 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel_ips.ko 2020-11-20T12:07:07.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/intel_menlow.ko 2020-11-20T12:07:07.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/msi-wmi.ko 2020-11-20T12:07:07.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/mxm-wmi.ko 2020-11-20T12:07:07.688 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/panasonic-laptop.ko 2020-11-20T12:07:07.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/samsung-laptop.ko 2020-11-20T12:07:07.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/samsung-q10.ko 2020-11-20T12:07:07.689 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 260993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/thinkpad_acpi.ko 2020-11-20T12:07:07.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/topstar-laptop.ko 2020-11-20T12:07:07.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2020-11-20T12:07:07.690 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/toshiba_haps.ko 2020-11-20T12:07:07.691 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/wmi-bmof.ko 2020-11-20T12:07:07.691 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/platform/x86/wmi.ko 2020-11-20T12:07:07.692 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/ 2020-11-20T12:07:07.692 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ 2020-11-20T12:07:07.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/88pm860x_battery.ko 2020-11-20T12:07:07.692 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/88pm860x_charger.ko 2020-11-20T12:07:07.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq2415x_charger.ko 2020-11-20T12:07:07.693 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq24190_charger.ko 2020-11-20T12:07:07.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/bq24735-charger.ko 2020-11-20T12:07:07.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/da9030_battery.ko 2020-11-20T12:07:07.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/da9052-battery.ko 2020-11-20T12:07:07.694 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2760_battery.ko 2020-11-20T12:07:07.695 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2780_battery.ko 2020-11-20T12:07:07.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2781_battery.ko 2020-11-20T12:07:07.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/ds2782_battery.ko 2020-11-20T12:07:07.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/gpio-charger.ko 2020-11-20T12:07:07.696 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/isp1704_charger.ko 2020-11-20T12:07:07.697 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/lp8727_charger.ko 2020-11-20T12:07:07.697 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max14577_charger.ko 2020-11-20T12:07:07.697 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max17040_battery.ko 2020-11-20T12:07:07.697 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max17042_battery.ko 2020-11-20T12:07:07.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max8903_charger.ko 2020-11-20T12:07:07.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/max8925_power.ko 2020-11-20T12:07:07.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/pcf50633-charger.ko 2020-11-20T12:07:07.699 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/pda_power.ko 2020-11-20T12:07:07.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/sbs-battery.ko 2020-11-20T12:07:07.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/smb347-charger.ko 2020-11-20T12:07:07.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/test_power.ko 2020-11-20T12:07:07.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/tps65090-charger.ko 2020-11-20T12:07:07.700 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm831x_backup.ko 2020-11-20T12:07:07.701 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm831x_power.ko 2020-11-20T12:07:07.701 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/power/supply/wm8350_power.ko 2020-11-20T12:07:07.703 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/ 2020-11-20T12:07:07.703 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/intel_rapl_common.ko 2020-11-20T12:07:07.703 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/powercap/intel_rapl_msr.ko 2020-11-20T12:07:07.704 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/ 2020-11-20T12:07:07.704 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/ 2020-11-20T12:07:07.704 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps-gpio.ko 2020-11-20T12:07:07.704 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps-ldisc.ko 2020-11-20T12:07:07.704 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/clients/pps_parport.ko 2020-11-20T12:07:07.705 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/pps/pps_core.ko 2020-11-20T12:07:07.705 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ 2020-11-20T12:07:07.705 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ptp.ko 2020-11-20T12:07:07.705 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ptp/ptp_kvm.ko 2020-11-20T12:07:07.706 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/ 2020-11-20T12:07:07.706 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-88pm80x.ko 2020-11-20T12:07:07.706 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-88pm860x.ko 2020-11-20T12:07:07.706 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ab3100.ko 2020-11-20T12:07:07.706 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-bq32k.ko 2020-11-20T12:07:07.707 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-bq4802.ko 2020-11-20T12:07:07.707 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9052.ko 2020-11-20T12:07:07.707 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9055.ko 2020-11-20T12:07:07.707 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-da9063.ko 2020-11-20T12:07:07.708 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1286.ko 2020-11-20T12:07:07.708 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 65017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1307.ko 2020-11-20T12:07:07.711 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1374.ko 2020-11-20T12:07:07.712 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1511.ko 2020-11-20T12:07:07.712 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1553.ko 2020-11-20T12:07:07.712 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1672.ko 2020-11-20T12:07:07.712 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds1742.ko 2020-11-20T12:07:07.712 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds2404.ko 2020-11-20T12:07:07.713 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-ds3232.ko 2020-11-20T12:07:07.713 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-em3027.ko 2020-11-20T12:07:07.713 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-fm3130.ko 2020-11-20T12:07:07.713 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-isl12022.ko 2020-11-20T12:07:07.714 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-isl1208.ko 2020-11-20T12:07:07.714 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-lp8788.ko 2020-11-20T12:07:07.714 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m41t80.ko 2020-11-20T12:07:07.714 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t35.ko 2020-11-20T12:07:07.715 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t59.ko 2020-11-20T12:07:07.715 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-m48t86.ko 2020-11-20T12:07:07.715 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max6900.ko 2020-11-20T12:07:07.715 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8907.ko 2020-11-20T12:07:07.715 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8925.ko 2020-11-20T12:07:07.716 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8997.ko 2020-11-20T12:07:07.716 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-max8998.ko 2020-11-20T12:07:07.716 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-mc13xxx.ko 2020-11-20T12:07:07.716 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-msm6242.ko 2020-11-20T12:07:07.716 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-palmas.ko 2020-11-20T12:07:07.717 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf2127.ko 2020-11-20T12:07:07.717 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf50633.ko 2020-11-20T12:07:07.717 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf85063.ko 2020-11-20T12:07:07.717 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8523.ko 2020-11-20T12:07:07.718 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8563.ko 2020-11-20T12:07:07.718 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-pcf8583.ko 2020-11-20T12:07:07.718 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rc5t583.ko 2020-11-20T12:07:07.718 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rp5c01.ko 2020-11-20T12:07:07.718 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rs5c372.ko 2020-11-20T12:07:07.719 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rv3029c2.ko 2020-11-20T12:07:07.719 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rx8025.ko 2020-11-20T12:07:07.719 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-rx8581.ko 2020-11-20T12:07:07.719 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-s35390a.ko 2020-11-20T12:07:07.719 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-s5m.ko 2020-11-20T12:07:07.720 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-stk17ta8.ko 2020-11-20T12:07:07.720 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps6586x.ko 2020-11-20T12:07:07.720 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps65910.ko 2020-11-20T12:07:07.720 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-tps80031.ko 2020-11-20T12:07:07.722 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-v3020.ko 2020-11-20T12:07:07.722 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-wm831x.ko 2020-11-20T12:07:07.722 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-wm8350.ko 2020-11-20T12:07:07.722 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/rtc/rtc-x1205.ko 2020-11-20T12:07:07.723 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ 2020-11-20T12:07:07.723 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 80673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-9xxx.ko 2020-11-20T12:07:07.723 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-sas.ko 2020-11-20T12:07:07.723 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/3w-xxxx.ko 2020-11-20T12:07:07.723 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 132001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/BusLogic.ko 2020-11-20T12:07:07.724 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/a100u2w.ko 2020-11-20T12:07:07.724 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aacraid/ 2020-11-20T12:07:07.724 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 286449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aacraid/aacraid.ko 2020-11-20T12:07:07.724 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 101329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/advansys.ko 2020-11-20T12:07:07.725 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/ 2020-11-20T12:07:07.725 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 346097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2020-11-20T12:07:07.728 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 297009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2020-11-20T12:07:07.732 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic94xx/ 2020-11-20T12:07:07.732 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 194297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/aic94xx/aic94xx.ko 2020-11-20T12:07:07.734 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/am53c974.ko 2020-11-20T12:07:07.734 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/arcmsr/ 2020-11-20T12:07:07.735 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 116369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/arcmsr/arcmsr.ko 2020-11-20T12:07:07.736 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/atp870u.ko 2020-11-20T12:07:07.736 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/be2iscsi/ 2020-11-20T12:07:07.737 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 265265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2020-11-20T12:07:07.740 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bfa/ 2020-11-20T12:07:07.740 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1164361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bfa/bfa.ko 2020-11-20T12:07:07.753 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2fc/ 2020-11-20T12:07:07.753 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 252833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2020-11-20T12:07:07.756 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2i/ 2020-11-20T12:07:07.756 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 119577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/bnx2i/bnx2i.ko 2020-11-20T12:07:07.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ch.ko 2020-11-20T12:07:07.758 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/csiostor/ 2020-11-20T12:07:07.758 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 312289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/csiostor/csiostor.ko 2020-11-20T12:07:07.762 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/ 2020-11-20T12:07:07.762 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb3i/ 2020-11-20T12:07:07.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 80249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2020-11-20T12:07:07.763 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb4i/ 2020-11-20T12:07:07.763 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 110649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2020-11-20T12:07:07.764 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 153137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2020-11-20T12:07:07.766 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 76961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dc395x.ko 2020-11-20T12:07:07.767 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dmx3191d.ko 2020-11-20T12:07:07.768 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/dpt_i2o.ko 2020-11-20T12:07:07.769 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esas2r/ 2020-11-20T12:07:07.769 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 213561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esas2r/esas2r.ko 2020-11-20T12:07:07.772 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/esp_scsi.ko 2020-11-20T12:07:07.773 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/ 2020-11-20T12:07:07.773 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/fcoe.ko 2020-11-20T12:07:07.774 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 162897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fcoe/libfcoe.ko 2020-11-20T12:07:07.775 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fdomain.ko 2020-11-20T12:07:07.776 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fnic/ 2020-11-20T12:07:07.776 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 280281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/fnic/fnic.ko 2020-11-20T12:07:07.779 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 107993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/gdth.ko 2020-11-20T12:07:07.781 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 215137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hpsa.ko 2020-11-20T12:07:07.783 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hptiop.ko 2020-11-20T12:07:07.784 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/hv_storvsc.ko 2020-11-20T12:07:07.785 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/imm.ko 2020-11-20T12:07:07.786 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 48489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/initio.ko 2020-11-20T12:07:07.786 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 294193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ipr.ko 2020-11-20T12:07:07.790 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 110105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ips.ko 2020-11-20T12:07:07.791 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/isci/ 2020-11-20T12:07:07.791 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 335225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/isci/isci.ko 2020-11-20T12:07:07.795 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2020-11-20T12:07:07.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/iscsi_tcp.ko 2020-11-20T12:07:07.796 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libfc/ 2020-11-20T12:07:07.796 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 321337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libfc/libfc.ko 2020-11-20T12:07:07.800 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 144753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libiscsi.ko 2020-11-20T12:07:07.801 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libiscsi_tcp.ko 2020-11-20T12:07:07.802 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libsas/ 2020-11-20T12:07:07.803 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 199625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/libsas/libsas.ko 2020-11-20T12:07:07.804 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/lpfc/ 2020-11-20T12:07:07.805 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 2047225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/lpfc/lpfc.ko 2020-11-20T12:07:07.831 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/ 2020-11-20T12:07:07.831 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 80961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2020-11-20T12:07:07.832 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2020-11-20T12:07:07.832 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 334937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2020-11-20T12:07:07.836 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/megaraid.ko 2020-11-20T12:07:07.837 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mpt3sas/ 2020-11-20T12:07:07.837 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 652873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2020-11-20T12:07:07.845 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvsas/ 2020-11-20T12:07:07.845 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 122993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvsas/mvsas.ko 2020-11-20T12:07:07.846 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 82081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/mvumi.ko 2020-11-20T12:07:07.847 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/ 2020-11-20T12:07:07.848 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 81921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2020-11-20T12:07:07.848 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2020-11-20T12:07:07.849 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2020-11-20T12:07:07.850 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2020-11-20T12:07:07.850 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pm8001/ 2020-11-20T12:07:07.850 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 420593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pm8001/pm80xx.ko 2020-11-20T12:07:07.854 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 137097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/pmcraid.ko 2020-11-20T12:07:07.855 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ppa.ko 2020-11-20T12:07:07.856 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla1280.ko 2020-11-20T12:07:07.856 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/ 2020-11-20T12:07:07.857 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1702009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2020-11-20T12:07:07.879 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 88025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2020-11-20T12:07:07.879 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla4xxx/ 2020-11-20T12:07:07.879 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 610673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2020-11-20T12:07:07.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/qlogicfas408.ko 2020-11-20T12:07:07.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/raid_class.ko 2020-11-20T12:07:07.887 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 238049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_debug.ko 2020-11-20T12:07:07.890 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 142097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_fc.ko 2020-11-20T12:07:07.891 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 265049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_iscsi.ko 2020-11-20T12:07:07.894 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 95017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_sas.ko 2020-11-20T12:07:07.895 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 72697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_spi.ko 2020-11-20T12:07:07.895 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/scsi_transport_srp.ko 2020-11-20T12:07:07.896 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ses.ko 2020-11-20T12:07:07.897 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 118033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/st.ko 2020-11-20T12:07:07.898 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/stex.ko 2020-11-20T12:07:07.898 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/sym53c8xx_2/ 2020-11-20T12:07:07.899 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 166257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2020-11-20T12:07:07.901 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ 2020-11-20T12:07:07.901 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 404305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-core.ko 2020-11-20T12:07:07.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-pci.ko 2020-11-20T12:07:07.905 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/ufs/ufshcd-pltfrm.ko 2020-11-20T12:07:07.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/virtio_scsi.ko 2020-11-20T12:07:07.906 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/vmw_pvscsi.ko 2020-11-20T12:07:07.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/wd719x.ko 2020-11-20T12:07:07.907 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/scsi/xen-scsifront.ko 2020-11-20T12:07:07.908 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ssb/ 2020-11-20T12:07:07.908 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 127201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/ssb/ssb.ko 2020-11-20T12:07:07.909 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/ 2020-11-20T12:07:07.910 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/iscsi/ 2020-11-20T12:07:07.910 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 747873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2020-11-20T12:07:07.917 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/loopback/ 2020-11-20T12:07:07.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/loopback/tcm_loop.ko 2020-11-20T12:07:07.918 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_file.ko 2020-11-20T12:07:07.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_iblock.ko 2020-11-20T12:07:07.919 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 931017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_mod.ko 2020-11-20T12:07:07.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_pscsi.ko 2020-11-20T12:07:07.929 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 108177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/target_core_user.ko 2020-11-20T12:07:07.931 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/tcm_fc/ 2020-11-20T12:07:07.931 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 73449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/target/tcm_fc/tcm_fc.ko 2020-11-20T12:07:07.931 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/ 2020-11-20T12:07:07.932 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/ 2020-11-20T12:07:07.932 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/ 2020-11-20T12:07:07.932 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2020-11-20T12:07:07.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2020-11-20T12:07:07.933 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2020-11-20T12:07:07.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2020-11-20T12:07:07.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2020-11-20T12:07:07.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2020-11-20T12:07:07.934 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_powerclamp.ko 2020-11-20T12:07:07.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2020-11-20T12:07:07.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2020-11-20T12:07:07.935 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2020-11-20T12:07:07.936 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ 2020-11-20T12:07:07.936 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 88425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/cyclades.ko 2020-11-20T12:07:07.936 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ipwireless/ 2020-11-20T12:07:07.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/ipwireless/ipwireless.ko 2020-11-20T12:07:07.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/isicom.ko 2020-11-20T12:07:07.937 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/moxa.ko 2020-11-20T12:07:07.938 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/mxser.ko 2020-11-20T12:07:07.938 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 88745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_gsm.ko 2020-11-20T12:07:07.939 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_hdlc.ko 2020-11-20T12:07:07.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_tracerouter.ko 2020-11-20T12:07:07.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/n_tracesink.ko 2020-11-20T12:07:07.940 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 71777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/nozomi.ko 2020-11-20T12:07:07.941 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/rocket.ko 2020-11-20T12:07:07.941 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/ 2020-11-20T12:07:07.942 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/ 2020-11-20T12:07:07.942 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/8250_dw.ko 2020-11-20T12:07:07.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/8250/serial_cs.ko 2020-11-20T12:07:07.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/altera_jtaguart.ko 2020-11-20T12:07:07.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/altera_uart.ko 2020-11-20T12:07:07.943 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/arc_uart.ko 2020-11-20T12:07:07.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/fsl_lpuart.ko 2020-11-20T12:07:07.944 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/jsm/ 2020-11-20T12:07:07.944 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 106905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/jsm/jsm.ko 2020-11-20T12:07:07.945 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/rp2.ko 2020-11-20T12:07:07.946 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/serial/sc16is7xx.ko 2020-11-20T12:07:07.946 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 119353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclink.ko 2020-11-20T12:07:07.948 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 109697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclink_gt.ko 2020-11-20T12:07:07.949 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 110737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/tty/synclinkmp.ko 2020-11-20T12:07:07.950 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/ 2020-11-20T12:07:07.950 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 42305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio.ko 2020-11-20T12:07:07.952 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_aec.ko 2020-11-20T12:07:07.952 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_cif.ko 2020-11-20T12:07:07.952 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_dmem_genirq.ko 2020-11-20T12:07:07.952 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_mf624.ko 2020-11-20T12:07:07.953 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_netx.ko 2020-11-20T12:07:07.953 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_pci_generic.ko 2020-11-20T12:07:07.953 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_pdrv_genirq.ko 2020-11-20T12:07:07.953 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/uio/uio_sercos3.ko 2020-11-20T12:07:07.953 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/ 2020-11-20T12:07:07.954 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/c67x00/ 2020-11-20T12:07:07.954 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/c67x00/c67x00.ko 2020-11-20T12:07:07.955 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ 2020-11-20T12:07:07.955 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 120065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc.ko 2020-11-20T12:07:07.955 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2020-11-20T12:07:07.955 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2020-11-20T12:07:07.956 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2020-11-20T12:07:07.956 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/ 2020-11-20T12:07:07.956 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/cdc-acm.ko 2020-11-20T12:07:07.957 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/cdc-wdm.ko 2020-11-20T12:07:07.957 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/usblp.ko 2020-11-20T12:07:07.957 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 93385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/class/usbtmc.ko 2020-11-20T12:07:07.958 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/common/ 2020-11-20T12:07:07.958 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/common/ulpi.ko 2020-11-20T12:07:07.959 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc2/ 2020-11-20T12:07:07.959 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc2/dwc2_pci.ko 2020-11-20T12:07:07.959 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/ 2020-11-20T12:07:07.959 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3-haps.ko 2020-11-20T12:07:07.959 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3-pci.ko 2020-11-20T12:07:07.960 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 432665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/dwc3/dwc3.ko 2020-11-20T12:07:07.964 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/ 2020-11-20T12:07:07.964 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/ 2020-11-20T12:07:07.964 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/u_ether.ko 2020-11-20T12:07:07.965 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/u_serial.ko 2020-11-20T12:07:07.965 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2020-11-20T12:07:07.966 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2020-11-20T12:07:07.967 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2020-11-20T12:07:07.968 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2020-11-20T12:07:07.968 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 92065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2020-11-20T12:07:07.968 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2020-11-20T12:07:07.969 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 120497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2020-11-20T12:07:07.970 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2020-11-20T12:07:07.971 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2020-11-20T12:07:07.971 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2020-11-20T12:07:07.972 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2020-11-20T12:07:07.972 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 83169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2020-11-20T12:07:07.972 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2020-11-20T12:07:07.973 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2020-11-20T12:07:07.978 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2020-11-20T12:07:07.978 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/ 2020-11-20T12:07:07.979 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2020-11-20T12:07:07.979 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2020-11-20T12:07:07.980 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2020-11-20T12:07:07.980 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ether.ko 2020-11-20T12:07:07.980 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2020-11-20T12:07:07.981 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_hid.ko 2020-11-20T12:07:07.981 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2020-11-20T12:07:07.982 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2020-11-20T12:07:07.982 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2020-11-20T12:07:07.982 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_printer.ko 2020-11-20T12:07:07.983 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_serial.ko 2020-11-20T12:07:07.983 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/g_zero.ko 2020-11-20T12:07:07.984 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2020-11-20T12:07:07.984 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2020-11-20T12:07:07.984 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 143505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/libcomposite.ko 2020-11-20T12:07:07.985 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/ 2020-11-20T12:07:07.985 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2020-11-20T12:07:07.986 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/ 2020-11-20T12:07:07.986 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 160057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2020-11-20T12:07:07.989 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/bdc/bdc_pci.ko 2020-11-20T12:07:07.989 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/fotg210-udc.ko 2020-11-20T12:07:07.990 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/goku_udc.ko 2020-11-20T12:07:07.990 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 54921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/gr_udc.ko 2020-11-20T12:07:07.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2020-11-20T12:07:07.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/mv_udc.ko 2020-11-20T12:07:07.991 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 73193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/net2272.ko 2020-11-20T12:07:07.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 75857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/net2280.ko 2020-11-20T12:07:07.992 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 73825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/pch_udc.ko 2020-11-20T12:07:07.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 66497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2020-11-20T12:07:07.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 70881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2020-11-20T12:07:07.993 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 82825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2020-11-20T12:07:07.994 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 144217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/gadget/udc/udc-core.ko 2020-11-20T12:07:07.996 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/ 2020-11-20T12:07:07.996 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/bcma-hcd.ko 2020-11-20T12:07:07.997 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 128681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/fotg210-hcd.ko 2020-11-20T12:07:07.997 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 76001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/isp116x-hcd.ko 2020-11-20T12:07:07.998 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 88825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/oxu210hp-hcd.ko 2020-11-20T12:07:07.998 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 79145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/r8a66597-hcd.ko 2020-11-20T12:07:07.998 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/sl811-hcd.ko 2020-11-20T12:07:07.999 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/sl811_cs.ko 2020-11-20T12:07:07.999 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/ssb-hcd.ko 2020-11-20T12:07:08.000 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 97993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/u132-hcd.ko 2020-11-20T12:07:08.004 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/host/xhci-plat-hcd.ko 2020-11-20T12:07:08.005 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/ 2020-11-20T12:07:08.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/mdc800.ko 2020-11-20T12:07:08.005 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/image/microtek.ko 2020-11-20T12:07:08.006 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ 2020-11-20T12:07:08.006 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/adutux.ko 2020-11-20T12:07:08.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/appledisplay.ko 2020-11-20T12:07:08.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/cypress_cy7c63.ko 2020-11-20T12:07:08.007 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/cytherm.ko 2020-11-20T12:07:08.008 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ehset.ko 2020-11-20T12:07:08.008 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/emi26.ko 2020-11-20T12:07:08.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/emi62.ko 2020-11-20T12:07:08.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ezusb.ko 2020-11-20T12:07:08.009 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ftdi-elan.ko 2020-11-20T12:07:08.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/idmouse.ko 2020-11-20T12:07:08.010 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/iowarrior.ko 2020-11-20T12:07:08.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/isight_firmware.ko 2020-11-20T12:07:08.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/ldusb.ko 2020-11-20T12:07:08.011 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/legousbtower.ko 2020-11-20T12:07:08.012 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/lvstest.ko 2020-11-20T12:07:08.012 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/sisusbvga/ 2020-11-20T12:07:08.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2020-11-20T12:07:08.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/trancevibrator.ko 2020-11-20T12:07:08.013 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usb3503.ko 2020-11-20T12:07:08.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usblcd.ko 2020-11-20T12:07:08.014 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usbsevseg.ko 2020-11-20T12:07:08.015 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 71281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/usbtest.ko 2020-11-20T12:07:08.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/uss720.ko 2020-11-20T12:07:08.020 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/misc/yurex.ko 2020-11-20T12:07:08.021 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/mon/ 2020-11-20T12:07:08.021 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 69937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/mon/usbmon.ko 2020-11-20T12:07:08.022 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/musb/ 2020-11-20T12:07:08.022 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 293313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/musb/musb_hdrc.ko 2020-11-20T12:07:08.022 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/ 2020-11-20T12:07:08.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-generic.ko 2020-11-20T12:07:08.023 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2020-11-20T12:07:08.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-isp1301.ko 2020-11-20T12:07:08.024 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/phy/phy-tahvo.ko 2020-11-20T12:07:08.024 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ 2020-11-20T12:07:08.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/aircable.ko 2020-11-20T12:07:08.025 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ark3116.ko 2020-11-20T12:07:08.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/belkin_sa.ko 2020-11-20T12:07:08.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ch341.ko 2020-11-20T12:07:08.026 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 82145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cp210x.ko 2020-11-20T12:07:08.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cyberjack.ko 2020-11-20T12:07:08.027 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/cypress_m8.ko 2020-11-20T12:07:08.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/digi_acceleport.ko 2020-11-20T12:07:08.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/empeg.ko 2020-11-20T12:07:08.028 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/f81232.ko 2020-11-20T12:07:08.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 174849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ftdi_sio.ko 2020-11-20T12:07:08.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/garmin_gps.ko 2020-11-20T12:07:08.029 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 140417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/io_edgeport.ko 2020-11-20T12:07:08.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 124153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/io_ti.ko 2020-11-20T12:07:08.030 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ipaq.ko 2020-11-20T12:07:08.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ipw.ko 2020-11-20T12:07:08.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ir-usb.ko 2020-11-20T12:07:08.031 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 57473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/iuu_phoenix.ko 2020-11-20T12:07:08.040 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 106265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/keyspan.ko 2020-11-20T12:07:08.041 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/keyspan_pda.ko 2020-11-20T12:07:08.041 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/kl5kusb105.ko 2020-11-20T12:07:08.041 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/kobil_sct.ko 2020-11-20T12:07:08.041 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mct_u232.ko 2020-11-20T12:07:08.042 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/metro-usb.ko 2020-11-20T12:07:08.042 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 75537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mos7720.ko 2020-11-20T12:07:08.042 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 78913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mos7840.ko 2020-11-20T12:07:08.042 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/mxuport.ko 2020-11-20T12:07:08.043 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/navman.ko 2020-11-20T12:07:08.043 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/omninet.ko 2020-11-20T12:07:08.043 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/opticon.ko 2020-11-20T12:07:08.043 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 176777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/option.ko 2020-11-20T12:07:08.043 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 39665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/oti6858.ko 2020-11-20T12:07:08.044 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/pl2303.ko 2020-11-20T12:07:08.044 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/qcaux.ko 2020-11-20T12:07:08.044 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/qcserial.ko 2020-11-20T12:07:08.044 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/quatech2.ko 2020-11-20T12:07:08.044 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/safe_serial.ko 2020-11-20T12:07:08.045 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/sierra.ko 2020-11-20T12:07:08.045 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/spcp8x5.ko 2020-11-20T12:07:08.045 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ssu100.ko 2020-11-20T12:07:08.045 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/symbolserial.ko 2020-11-20T12:07:08.046 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2020-11-20T12:07:08.046 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb-serial-simple.ko 2020-11-20T12:07:08.046 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb_debug.ko 2020-11-20T12:07:08.046 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usb_wwan.ko 2020-11-20T12:07:08.046 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 118233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/usbserial.ko 2020-11-20T12:07:08.047 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/visor.ko 2020-11-20T12:07:08.047 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/whiteheat.ko 2020-11-20T12:07:08.047 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/wishbone-serial.ko 2020-11-20T12:07:08.047 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/serial/xsens_mt.ko 2020-11-20T12:07:08.048 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ 2020-11-20T12:07:08.048 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/uas.ko 2020-11-20T12:07:08.048 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-alauda.ko 2020-11-20T12:07:08.048 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-cypress.ko 2020-11-20T12:07:08.048 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-datafab.ko 2020-11-20T12:07:08.049 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-eneub6250.ko 2020-11-20T12:07:08.049 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-freecom.ko 2020-11-20T12:07:08.049 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-isd200.ko 2020-11-20T12:07:08.049 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-jumpshot.ko 2020-11-20T12:07:08.049 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-karma.ko 2020-11-20T12:07:08.050 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-onetouch.ko 2020-11-20T12:07:08.050 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-realtek.ko 2020-11-20T12:07:08.050 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-sddr09.ko 2020-11-20T12:07:08.050 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-sddr55.ko 2020-11-20T12:07:08.050 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/ums-usbat.ko 2020-11-20T12:07:08.051 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 173801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/storage/usb-storage.ko 2020-11-20T12:07:08.051 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/ 2020-11-20T12:07:08.051 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/usbip-core.ko 2020-11-20T12:07:08.051 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 71161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/usbip-host.ko 2020-11-20T12:07:08.052 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 107913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/usb/usbip/vhci-hcd.ko 2020-11-20T12:07:08.056 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/ 2020-11-20T12:07:08.056 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/pci/ 2020-11-20T12:07:08.056 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 149025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/pci/vfio-pci.ko 2020-11-20T12:07:08.056 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 72673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio.ko 2020-11-20T12:07:08.057 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 68809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio_iommu_type1.ko 2020-11-20T12:07:08.057 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vfio/vfio_virqfd.ko 2020-11-20T12:07:08.057 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/ 2020-11-20T12:07:08.057 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 99057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost.ko 2020-11-20T12:07:08.058 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_iotlb.ko 2020-11-20T12:07:08.058 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_net.ko 2020-11-20T12:07:08.058 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 84649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/vhost/vhost_scsi.ko 2020-11-20T12:07:08.058 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/ 2020-11-20T12:07:08.058 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/ 2020-11-20T12:07:08.059 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/88pm860x_bl.ko 2020-11-20T12:07:08.059 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/aat2870_bl.ko 2020-11-20T12:07:08.059 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp5520_bl.ko 2020-11-20T12:07:08.059 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp8860_bl.ko 2020-11-20T12:07:08.059 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/adp8870_bl.ko 2020-11-20T12:07:08.060 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/apple_bl.ko 2020-11-20T12:07:08.060 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/as3711_bl.ko 2020-11-20T12:07:08.060 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/bd6107.ko 2020-11-20T12:07:08.060 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/da903x_bl.ko 2020-11-20T12:07:08.060 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/da9052_bl.ko 2020-11-20T12:07:08.061 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/gpio_backlight.ko 2020-11-20T12:07:08.061 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/kb3886_bl.ko 2020-11-20T12:07:08.061 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lcd.ko 2020-11-20T12:07:08.061 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lm3533_bl.ko 2020-11-20T12:07:08.062 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lm3639_bl.ko 2020-11-20T12:07:08.062 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/lv5207lp.ko 2020-11-20T12:07:08.062 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/max8925_bl.ko 2020-11-20T12:07:08.062 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/pandora_bl.ko 2020-11-20T12:07:08.062 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/pcf50633-backlight.ko 2020-11-20T12:07:08.063 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/platform_lcd.ko 2020-11-20T12:07:08.063 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/video/backlight/wm831x_bl.ko 2020-11-20T12:07:08.063 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/virtio/ 2020-11-20T12:07:08.063 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/virtio/virtio_mem.ko 2020-11-20T12:07:08.063 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/ 2020-11-20T12:07:08.064 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ 2020-11-20T12:07:08.064 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds1wm.ko 2020-11-20T12:07:08.064 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds2482.ko 2020-11-20T12:07:08.064 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/ds2490.ko 2020-11-20T12:07:08.065 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/matrox_w1.ko 2020-11-20T12:07:08.065 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/masters/w1-gpio.ko 2020-11-20T12:07:08.065 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/ 2020-11-20T12:07:08.065 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2406.ko 2020-11-20T12:07:08.065 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2408.ko 2020-11-20T12:07:08.066 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2413.ko 2020-11-20T12:07:08.066 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2423.ko 2020-11-20T12:07:08.066 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2431.ko 2020-11-20T12:07:08.066 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2433.ko 2020-11-20T12:07:08.066 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2780.ko 2020-11-20T12:07:08.067 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds2781.ko 2020-11-20T12:07:08.067 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_ds28e04.ko 2020-11-20T12:07:08.067 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_smem.ko 2020-11-20T12:07:08.067 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 59097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/slaves/w1_therm.ko 2020-11-20T12:07:08.067 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 99985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/w1/wire.ko 2020-11-20T12:07:08.068 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ 2020-11-20T12:07:08.068 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/acquirewdt.ko 2020-11-20T12:07:08.073 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/advantechwdt.ko 2020-11-20T12:07:08.073 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/alim1535_wdt.ko 2020-11-20T12:07:08.073 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/alim7101_wdt.ko 2020-11-20T12:07:08.073 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/cpu5wdt.ko 2020-11-20T12:07:08.074 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9052_wdt.ko 2020-11-20T12:07:08.074 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9055_wdt.ko 2020-11-20T12:07:08.074 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/da9063_wdt.ko 2020-11-20T12:07:08.074 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/dw_wdt.ko 2020-11-20T12:07:08.074 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/eurotechwdt.ko 2020-11-20T12:07:08.075 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/f71808e_wdt.ko 2020-11-20T12:07:08.075 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/hpwdt.ko 2020-11-20T12:07:08.075 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/i6300esb.ko 2020-11-20T12:07:08.075 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/iTCO_vendor_support.ko 2020-11-20T12:07:08.075 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/iTCO_wdt.ko 2020-11-20T12:07:08.076 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ib700wdt.ko 2020-11-20T12:07:08.076 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ibmasr.ko 2020-11-20T12:07:08.076 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/ie6xx_wdt.ko 2020-11-20T12:07:08.076 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/it8712f_wdt.ko 2020-11-20T12:07:08.077 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/it87_wdt.ko 2020-11-20T12:07:08.077 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/kempld_wdt.ko 2020-11-20T12:07:08.077 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/machzwd.ko 2020-11-20T12:07:08.077 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/mena21_wdt.ko 2020-11-20T12:07:08.077 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/menf21bmc_wdt.ko 2020-11-20T12:07:08.078 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/nv_tco.ko 2020-11-20T12:07:08.078 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/of_xilinx_wdt.ko 2020-11-20T12:07:08.078 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pc87413_wdt.ko 2020-11-20T12:07:08.078 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pcwd_pci.ko 2020-11-20T12:07:08.078 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/pcwd_usb.ko 2020-11-20T12:07:08.079 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/retu_wdt.ko 2020-11-20T12:07:08.079 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc60xxwdt.ko 2020-11-20T12:07:08.079 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc_epx_c3.ko 2020-11-20T12:07:08.079 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2020-11-20T12:07:08.080 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sc1200wdt.ko 2020-11-20T12:07:08.080 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sch311x_wdt.ko 2020-11-20T12:07:08.080 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/smsc37b787_wdt.ko 2020-11-20T12:07:08.080 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/softdog.ko 2020-11-20T12:07:08.080 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/sp5100_tco.ko 2020-11-20T12:07:08.081 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/twl4030_wdt.ko 2020-11-20T12:07:08.081 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/via_wdt.ko 2020-11-20T12:07:08.081 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83627hf_wdt.ko 2020-11-20T12:07:08.081 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83877f_wdt.ko 2020-11-20T12:07:08.081 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/w83977f_wdt.ko 2020-11-20T12:07:08.082 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wafer5823wdt.ko 2020-11-20T12:07:08.082 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wdt_pci.ko 2020-11-20T12:07:08.082 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wm831x_wdt.ko 2020-11-20T12:07:08.082 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/wm8350_wdt.ko 2020-11-20T12:07:08.082 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/watchdog/xen_wdt.ko 2020-11-20T12:07:08.083 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/ 2020-11-20T12:07:08.083 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-evtchn.ko 2020-11-20T12:07:08.083 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-gntalloc.ko 2020-11-20T12:07:08.083 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 43137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-gntdev.ko 2020-11-20T12:07:08.083 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-pciback/ 2020-11-20T12:07:08.084 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 178577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2020-11-20T12:07:08.084 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-privcmd.ko 2020-11-20T12:07:08.084 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xen-scsiback.ko 2020-11-20T12:07:08.084 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xenfs/ 2020-11-20T12:07:08.084 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/drivers/xen/xenfs/xenfs.ko 2020-11-20T12:07:08.085 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ 2020-11-20T12:07:08.085 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/9p/ 2020-11-20T12:07:08.085 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 147729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/9p/9p.ko 2020-11-20T12:07:08.085 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/adfs/ 2020-11-20T12:07:08.086 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/adfs/adfs.ko 2020-11-20T12:07:08.086 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/affs/ 2020-11-20T12:07:08.086 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 162417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/affs/affs.ko 2020-11-20T12:07:08.087 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/afs/ 2020-11-20T12:07:08.087 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 972889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/afs/kafs.ko 2020-11-20T12:07:08.087 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/autofs/ 2020-11-20T12:07:08.088 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 106865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/autofs/autofs4.ko 2020-11-20T12:07:08.088 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/befs/ 2020-11-20T12:07:08.088 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/befs/befs.ko 2020-11-20T12:07:08.088 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/bfs/ 2020-11-20T12:07:08.088 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/bfs/bfs.ko 2020-11-20T12:07:08.089 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/binfmt_misc.ko 2020-11-20T12:07:08.089 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/btrfs/ 2020-11-20T12:07:08.089 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 3192737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/btrfs/btrfs.ko 2020-11-20T12:07:08.125 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cachefiles/ 2020-11-20T12:07:08.125 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 194401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cachefiles/cachefiles.ko 2020-11-20T12:07:08.127 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ceph/ 2020-11-20T12:07:08.127 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1125913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ceph/ceph.ko 2020-11-20T12:07:08.139 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cifs/ 2020-11-20T12:07:08.139 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 2585001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cifs/cifs.ko 2020-11-20T12:07:08.165 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/coda/ 2020-11-20T12:07:08.166 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 96281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/coda/coda.ko 2020-11-20T12:07:08.166 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/configfs/ 2020-11-20T12:07:08.166 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 102945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/configfs/configfs.ko 2020-11-20T12:07:08.167 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cramfs/ 2020-11-20T12:07:08.167 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/cramfs/cramfs.ko 2020-11-20T12:07:08.168 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/dlm/ 2020-11-20T12:07:08.168 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 438425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/dlm/dlm.ko 2020-11-20T12:07:08.173 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/efs/ 2020-11-20T12:07:08.173 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/efs/efs.ko 2020-11-20T12:07:08.173 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/f2fs/ 2020-11-20T12:07:08.174 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1367417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/f2fs/f2fs.ko 2020-11-20T12:07:08.189 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fat/ 2020-11-20T12:07:08.190 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fat/msdos.ko 2020-11-20T12:07:08.190 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/freevxfs/ 2020-11-20T12:07:08.190 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/freevxfs/freevxfs.ko 2020-11-20T12:07:08.191 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fscache/ 2020-11-20T12:07:08.191 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 317913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fscache/fscache.ko 2020-11-20T12:07:08.193 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fuse/ 2020-11-20T12:07:08.193 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/fuse/cuse.ko 2020-11-20T12:07:08.193 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/gfs2/ 2020-11-20T12:07:08.194 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1193977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/gfs2/gfs2.ko 2020-11-20T12:07:08.205 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfs/ 2020-11-20T12:07:08.205 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 141505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfs/hfs.ko 2020-11-20T12:07:08.206 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfsplus/ 2020-11-20T12:07:08.207 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 229553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hfsplus/hfsplus.ko 2020-11-20T12:07:08.209 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hpfs/ 2020-11-20T12:07:08.210 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 193625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/hpfs/hpfs.ko 2020-11-20T12:07:08.212 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/isofs/ 2020-11-20T12:07:08.212 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 75881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/isofs/isofs.ko 2020-11-20T12:07:08.213 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/jfs/ 2020-11-20T12:07:08.213 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 413889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/jfs/jfs.ko 2020-11-20T12:07:08.219 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/lockd/ 2020-11-20T12:07:08.219 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 268617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/lockd/lockd.ko 2020-11-20T12:07:08.221 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/minix/ 2020-11-20T12:07:08.222 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 83041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/minix/minix.ko 2020-11-20T12:07:08.222 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/ 2020-11-20T12:07:08.223 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/blocklayout/ 2020-11-20T12:07:08.223 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 87177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2020-11-20T12:07:08.223 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/filelayout/ 2020-11-20T12:07:08.224 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 64561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2020-11-20T12:07:08.224 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/flexfilelayout/ 2020-11-20T12:07:08.225 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 123673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2020-11-20T12:07:08.226 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 899209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfs.ko 2020-11-20T12:07:08.235 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 65457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv2.ko 2020-11-20T12:07:08.235 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 112873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv3.ko 2020-11-20T12:07:08.236 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1849921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs/nfsv4.ko 2020-11-20T12:07:08.254 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/ 2020-11-20T12:07:08.254 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/grace.ko 2020-11-20T12:07:08.255 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/nfs_acl.ko 2020-11-20T12:07:08.255 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfs_common/nfs_ssc.ko 2020-11-20T12:07:08.255 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfsd/ 2020-11-20T12:07:08.255 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1361097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nfsd/nfsd.ko 2020-11-20T12:07:08.269 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nilfs2/ 2020-11-20T12:07:08.269 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 557369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nilfs2/nilfs2.ko 2020-11-20T12:07:08.278 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/ 2020-11-20T12:07:08.278 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-celtic.ko 2020-11-20T12:07:08.279 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-centeuro.ko 2020-11-20T12:07:08.279 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-croatian.ko 2020-11-20T12:07:08.279 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-cyrillic.ko 2020-11-20T12:07:08.279 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-gaelic.ko 2020-11-20T12:07:08.280 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-greek.ko 2020-11-20T12:07:08.280 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-iceland.ko 2020-11-20T12:07:08.280 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-inuit.ko 2020-11-20T12:07:08.280 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-roman.ko 2020-11-20T12:07:08.280 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-romanian.ko 2020-11-20T12:07:08.281 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/mac-turkish.ko 2020-11-20T12:07:08.281 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_ascii.ko 2020-11-20T12:07:08.281 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1250.ko 2020-11-20T12:07:08.281 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1251.ko 2020-11-20T12:07:08.281 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp1255.ko 2020-11-20T12:07:08.282 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp737.ko 2020-11-20T12:07:08.282 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp775.ko 2020-11-20T12:07:08.282 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp850.ko 2020-11-20T12:07:08.282 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp852.ko 2020-11-20T12:07:08.283 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp855.ko 2020-11-20T12:07:08.283 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp857.ko 2020-11-20T12:07:08.283 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp860.ko 2020-11-20T12:07:08.283 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp861.ko 2020-11-20T12:07:08.283 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp862.ko 2020-11-20T12:07:08.284 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp863.ko 2020-11-20T12:07:08.284 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp864.ko 2020-11-20T12:07:08.284 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp865.ko 2020-11-20T12:07:08.284 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp866.ko 2020-11-20T12:07:08.284 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp869.ko 2020-11-20T12:07:08.285 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp874.ko 2020-11-20T12:07:08.285 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 93241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp932.ko 2020-11-20T12:07:08.285 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 141825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp936.ko 2020-11-20T12:07:08.286 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 164961 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp949.ko 2020-11-20T12:07:08.287 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 117337 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_cp950.ko 2020-11-20T12:07:08.291 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_euc-jp.ko 2020-11-20T12:07:08.291 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-1.ko 2020-11-20T12:07:08.292 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-13.ko 2020-11-20T12:07:08.292 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-14.ko 2020-11-20T12:07:08.292 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-15.ko 2020-11-20T12:07:08.292 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-2.ko 2020-11-20T12:07:08.292 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-3.ko 2020-11-20T12:07:08.293 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-4.ko 2020-11-20T12:07:08.293 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-5.ko 2020-11-20T12:07:08.293 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-6.ko 2020-11-20T12:07:08.293 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-7.ko 2020-11-20T12:07:08.294 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_iso8859-9.ko 2020-11-20T12:07:08.294 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-r.ko 2020-11-20T12:07:08.294 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-ru.ko 2020-11-20T12:07:08.294 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_koi8-u.ko 2020-11-20T12:07:08.295 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/nls/nls_utf8.ko 2020-11-20T12:07:08.295 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ntfs/ 2020-11-20T12:07:08.295 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 212417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ntfs/ntfs.ko 2020-11-20T12:07:08.296 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ 2020-11-20T12:07:08.296 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/cluster/ 2020-11-20T12:07:08.296 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 235897 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2020-11-20T12:07:08.298 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlm/ 2020-11-20T12:07:08.298 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 435145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2020-11-20T12:07:08.302 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlmfs/ 2020-11-20T12:07:08.303 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2020-11-20T12:07:08.303 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 2561401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2.ko 2020-11-20T12:07:08.330 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2020-11-20T12:07:08.330 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stack_user.ko 2020-11-20T12:07:08.331 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ocfs2/ocfs2_stackglue.ko 2020-11-20T12:07:08.331 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/omfs/ 2020-11-20T12:07:08.331 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 52561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/omfs/omfs.ko 2020-11-20T12:07:08.332 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/overlayfs/ 2020-11-20T12:07:08.332 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 288641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/overlayfs/overlay.ko 2020-11-20T12:07:08.334 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/pstore/ 2020-11-20T12:07:08.335 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 55033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/pstore/ramoops.ko 2020-11-20T12:07:08.335 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx4/ 2020-11-20T12:07:08.336 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx4/qnx4.ko 2020-11-20T12:07:08.336 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx6/ 2020-11-20T12:07:08.336 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/qnx6/qnx6.ko 2020-11-20T12:07:08.337 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/ 2020-11-20T12:07:08.337 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_tree.ko 2020-11-20T12:07:08.338 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_v1.ko 2020-11-20T12:07:08.338 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/quota/quota_v2.ko 2020-11-20T12:07:08.338 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/reiserfs/ 2020-11-20T12:07:08.338 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 521233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/reiserfs/reiserfs.ko 2020-11-20T12:07:08.344 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/romfs/ 2020-11-20T12:07:08.344 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/romfs/romfs.ko 2020-11-20T12:07:08.344 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/squashfs/ 2020-11-20T12:07:08.345 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 125585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/squashfs/squashfs.ko 2020-11-20T12:07:08.346 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/sysv/ 2020-11-20T12:07:08.346 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 86857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/sysv/sysv.ko 2020-11-20T12:07:08.347 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/udf/ 2020-11-20T12:07:08.347 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 249097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/udf/udf.ko 2020-11-20T12:07:08.350 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ufs/ 2020-11-20T12:07:08.350 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 157929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/ufs/ufs.ko 2020-11-20T12:07:08.352 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/xfs/ 2020-11-20T12:07:08.352 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 3641945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/fs/xfs/xfs.ko 2020-11-20T12:07:08.390 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/ 2020-11-20T12:07:08.390 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/locking/ 2020-11-20T12:07:08.390 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 56513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/locking/locktorture.ko 2020-11-20T12:07:08.391 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/time/ 2020-11-20T12:07:08.392 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/time/test_udelay.ko 2020-11-20T12:07:08.392 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/kernel/torture.ko 2020-11-20T12:07:08.392 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ 2020-11-20T12:07:08.392 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc-itu-t.ko 2020-11-20T12:07:08.393 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc64.ko 2020-11-20T12:07:08.393 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc7.ko 2020-11-20T12:07:08.393 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crc8.ko 2020-11-20T12:07:08.393 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/ 2020-11-20T12:07:08.393 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/libarc4.ko 2020-11-20T12:07:08.394 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/crypto/libdes.ko 2020-11-20T12:07:08.395 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/interval_tree_test.ko 2020-11-20T12:07:08.395 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/libcrc32c.ko 2020-11-20T12:07:08.395 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lru_cache.ko 2020-11-20T12:07:08.396 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/ 2020-11-20T12:07:08.396 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/lz4_compress.ko 2020-11-20T12:07:08.396 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/lz4/lz4hc_compress.ko 2020-11-20T12:07:08.396 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/math/ 2020-11-20T12:07:08.396 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/math/cordic.ko 2020-11-20T12:07:08.397 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/memory-notifier-error-inject.ko 2020-11-20T12:07:08.397 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/notifier-error-inject.ko 2020-11-20T12:07:08.397 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12457 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/percpu_test.ko 2020-11-20T12:07:08.397 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/pm-notifier-error-inject.ko 2020-11-20T12:07:08.398 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/raid6/ 2020-11-20T12:07:08.398 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 141225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/raid6/raid6_pq.ko 2020-11-20T12:07:08.399 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/rbtree_test.ko 2020-11-20T12:07:08.399 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/reed_solomon/ 2020-11-20T12:07:08.400 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/reed_solomon/reed_solomon.ko 2020-11-20T12:07:08.400 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test-kstrtox.ko 2020-11-20T12:07:08.401 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test-string_helpers.ko 2020-11-20T12:07:08.401 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1714593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_bpf.ko 2020-11-20T12:07:08.403 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_firmware.ko 2020-11-20T12:07:08.404 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 5545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_module.ko 2020-11-20T12:07:08.404 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/test_user_copy.ko 2020-11-20T12:07:08.405 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_bm.ko 2020-11-20T12:07:08.405 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_fsm.ko 2020-11-20T12:07:08.405 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/ts_kmp.ko 2020-11-20T12:07:08.406 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/xz/ 2020-11-20T12:07:08.406 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/xz/xz_dec_test.ko 2020-11-20T12:07:08.406 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/zstd/ 2020-11-20T12:07:08.406 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 259025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/lib/zstd/zstd_compress.ko 2020-11-20T12:07:08.410 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/mm/ 2020-11-20T12:07:08.410 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/mm/hwpoison-inject.ko 2020-11-20T12:07:08.411 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ 2020-11-20T12:07:08.411 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/ 2020-11-20T12:07:08.411 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 58657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/6lowpan.ko 2020-11-20T12:07:08.413 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_dest.ko 2020-11-20T12:07:08.413 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_fragment.ko 2020-11-20T12:07:08.413 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_hop.ko 2020-11-20T12:07:08.414 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_ipv6.ko 2020-11-20T12:07:08.414 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_mobility.ko 2020-11-20T12:07:08.414 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_routing.ko 2020-11-20T12:07:08.414 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/6lowpan/nhc_udp.ko 2020-11-20T12:07:08.415 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/ 2020-11-20T12:07:08.415 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/garp.ko 2020-11-20T12:07:08.415 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/mrp.ko 2020-11-20T12:07:08.415 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/p8022.ko 2020-11-20T12:07:08.415 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/psnap.ko 2020-11-20T12:07:08.416 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/802/stp.ko 2020-11-20T12:07:08.416 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/8021q/ 2020-11-20T12:07:08.416 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 82849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/8021q/8021q.ko 2020-11-20T12:07:08.416 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/ 2020-11-20T12:07:08.416 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 208001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet.ko 2020-11-20T12:07:08.417 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26137 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet_rdma.ko 2020-11-20T12:07:08.417 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/9p/9pnet_virtio.ko 2020-11-20T12:07:08.418 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/batman-adv/ 2020-11-20T12:07:08.418 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 647129 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/batman-adv/batman-adv.ko 2020-11-20T12:07:08.423 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/ 2020-11-20T12:07:08.423 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 81361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/br_netfilter.ko 2020-11-20T12:07:08.424 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 610185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/bridge.ko 2020-11-20T12:07:08.432 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ 2020-11-20T12:07:08.433 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_802_3.ko 2020-11-20T12:07:08.433 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_among.ko 2020-11-20T12:07:08.433 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_arp.ko 2020-11-20T12:07:08.433 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_arpreply.ko 2020-11-20T12:07:08.433 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_dnat.ko 2020-11-20T12:07:08.434 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_ip.ko 2020-11-20T12:07:08.434 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_ip6.ko 2020-11-20T12:07:08.434 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_limit.ko 2020-11-20T12:07:08.434 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_log.ko 2020-11-20T12:07:08.435 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_mark.ko 2020-11-20T12:07:08.435 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_mark_m.ko 2020-11-20T12:07:08.435 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_nflog.ko 2020-11-20T12:07:08.436 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_pkttype.ko 2020-11-20T12:07:08.436 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_redirect.ko 2020-11-20T12:07:08.436 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_snat.ko 2020-11-20T12:07:08.436 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_stp.ko 2020-11-20T12:07:08.436 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebt_vlan.ko 2020-11-20T12:07:08.438 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_broute.ko 2020-11-20T12:07:08.438 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_filter.ko 2020-11-20T12:07:08.438 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtable_nat.ko 2020-11-20T12:07:08.438 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/ebtables.ko 2020-11-20T12:07:08.439 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nf_log_bridge.ko 2020-11-20T12:07:08.439 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2020-11-20T12:07:08.439 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2020-11-20T12:07:08.439 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ceph/ 2020-11-20T12:07:08.439 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 836073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ceph/libceph.ko 2020-11-20T12:07:08.444 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/core/ 2020-11-20T12:07:08.444 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 127473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/core/pktgen.ko 2020-11-20T12:07:08.446 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/ 2020-11-20T12:07:08.446 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 201017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp.ko 2020-11-20T12:07:08.448 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_diag.ko 2020-11-20T12:07:08.448 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_ipv4.ko 2020-11-20T12:07:08.449 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 54585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dccp/dccp_ipv6.ko 2020-11-20T12:07:08.449 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/ 2020-11-20T12:07:08.450 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 263865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/decnet.ko 2020-11-20T12:07:08.453 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/netfilter/ 2020-11-20T12:07:08.453 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/decnet/netfilter/dn_rtmsg.ko 2020-11-20T12:07:08.453 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/ 2020-11-20T12:07:08.453 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 144145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/dsa_core.ko 2020-11-20T12:07:08.455 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_brcm.ko 2020-11-20T12:07:08.455 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_dsa.ko 2020-11-20T12:07:08.455 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_edsa.ko 2020-11-20T12:07:08.455 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/dsa/tag_trailer.ko 2020-11-20T12:07:08.456 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/hsr/ 2020-11-20T12:07:08.456 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 91321 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/hsr/hsr.ko 2020-11-20T12:07:08.456 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ 2020-11-20T12:07:08.457 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/6lowpan/ 2020-11-20T12:07:08.457 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2020-11-20T12:07:08.457 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 220801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ieee802154.ko 2020-11-20T12:07:08.458 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45441 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ieee802154/ieee802154_socket.ko 2020-11-20T12:07:08.459 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ 2020-11-20T12:07:08.459 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ah4.ko 2020-11-20T12:07:08.460 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/esp4.ko 2020-11-20T12:07:08.460 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 51945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/fou.ko 2020-11-20T12:07:08.461 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/gre.ko 2020-11-20T12:07:08.461 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/inet_diag.ko 2020-11-20T12:07:08.461 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63881 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_gre.ko 2020-11-20T12:07:08.462 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 64593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_tunnel.ko 2020-11-20T12:07:08.463 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ip_vti.ko 2020-11-20T12:07:08.463 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ipcomp.ko 2020-11-20T12:07:08.464 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/ipip.ko 2020-11-20T12:07:08.464 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ 2020-11-20T12:07:08.465 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arp_tables.ko 2020-11-20T12:07:08.465 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8169 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arpt_mangle.ko 2020-11-20T12:07:08.465 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9249 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/arptable_filter.ko 2020-11-20T12:07:08.465 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ip_tables.ko 2020-11-20T12:07:08.467 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 45657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_CLUSTERIP.ko 2020-11-20T12:07:08.467 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_ECN.ko 2020-11-20T12:07:08.467 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2020-11-20T12:07:08.468 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15345 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2020-11-20T12:07:08.468 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10481 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_ah.ko 2020-11-20T12:07:08.468 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2020-11-20T12:07:08.468 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_filter.ko 2020-11-20T12:07:08.469 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_mangle.ko 2020-11-20T12:07:08.469 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_nat.ko 2020-11-20T12:07:08.469 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11001 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_raw.ko 2020-11-20T12:07:08.469 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/iptable_security.ko 2020-11-20T12:07:08.470 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2020-11-20T12:07:08.470 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2020-11-20T12:07:08.470 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_flow_table_ipv4.ko 2020-11-20T12:07:08.470 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_log_arp.ko 2020-11-20T12:07:08.470 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_log_ipv4.ko 2020-11-20T12:07:08.471 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2020-11-20T12:07:08.473 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2020-11-20T12:07:08.474 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2020-11-20T12:07:08.474 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2020-11-20T12:07:08.474 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2020-11-20T12:07:08.474 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2020-11-20T12:07:08.474 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9305 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2020-11-20T12:07:08.475 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2020-11-20T12:07:08.475 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2020-11-20T12:07:08.475 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_bic.ko 2020-11-20T12:07:08.475 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11281 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_dctcp.ko 2020-11-20T12:07:08.476 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14401 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_diag.ko 2020-11-20T12:07:08.476 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_highspeed.ko 2020-11-20T12:07:08.476 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_htcp.ko 2020-11-20T12:07:08.476 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8905 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_hybla.ko 2020-11-20T12:07:08.477 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_illinois.ko 2020-11-20T12:07:08.477 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_lp.ko 2020-11-20T12:07:08.477 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_scalable.ko 2020-11-20T12:07:08.478 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12889 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_vegas.ko 2020-11-20T12:07:08.478 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_veno.ko 2020-11-20T12:07:08.478 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_westwood.ko 2020-11-20T12:07:08.478 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tcp_yeah.ko 2020-11-20T12:07:08.479 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/tunnel4.ko 2020-11-20T12:07:08.479 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/udp_diag.ko 2020-11-20T12:07:08.479 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 40649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/udp_tunnel.ko 2020-11-20T12:07:08.479 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv4/xfrm4_tunnel.ko 2020-11-20T12:07:08.479 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ 2020-11-20T12:07:08.480 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ah6.ko 2020-11-20T12:07:08.480 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/esp6.ko 2020-11-20T12:07:08.480 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/fou6.ko 2020-11-20T12:07:08.480 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 93417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_gre.ko 2020-11-20T12:07:08.480 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 101977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_tunnel.ko 2020-11-20T12:07:08.485 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_udp_tunnel.ko 2020-11-20T12:07:08.485 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ip6_vti.ko 2020-11-20T12:07:08.485 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/ipcomp6.ko 2020-11-20T12:07:08.486 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/mip6.ko 2020-11-20T12:07:08.486 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ 2020-11-20T12:07:08.486 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6_tables.ko 2020-11-20T12:07:08.486 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2020-11-20T12:07:08.487 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2020-11-20T12:07:08.487 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2020-11-20T12:07:08.487 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_ah.ko 2020-11-20T12:07:08.487 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2020-11-20T12:07:08.488 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_frag.ko 2020-11-20T12:07:08.488 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2020-11-20T12:07:08.488 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2020-11-20T12:07:08.488 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_mh.ko 2020-11-20T12:07:08.488 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2020-11-20T12:07:08.489 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6t_rt.ko 2020-11-20T12:07:08.489 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_filter.ko 2020-11-20T12:07:08.489 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2020-11-20T12:07:08.489 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_nat.ko 2020-11-20T12:07:08.489 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_raw.ko 2020-11-20T12:07:08.490 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/ip6table_security.ko 2020-11-20T12:07:08.490 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2020-11-20T12:07:08.490 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2020-11-20T12:07:08.490 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_flow_table_ipv6.ko 2020-11-20T12:07:08.491 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_log_ipv6.ko 2020-11-20T12:07:08.491 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2020-11-20T12:07:08.491 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2020-11-20T12:07:08.491 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2020-11-20T12:07:08.491 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2020-11-20T12:07:08.492 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2020-11-20T12:07:08.492 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2020-11-20T12:07:08.492 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 90545 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/sit.ko 2020-11-20T12:07:08.492 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/tunnel6.ko 2020-11-20T12:07:08.493 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/ipv6/xfrm6_tunnel.ko 2020-11-20T12:07:08.493 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/key/ 2020-11-20T12:07:08.493 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 94849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/key/af_key.ko 2020-11-20T12:07:08.493 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/ 2020-11-20T12:07:08.494 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 127681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_core.ko 2020-11-20T12:07:08.494 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_debugfs.ko 2020-11-20T12:07:08.494 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_eth.ko 2020-11-20T12:07:08.494 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ip.ko 2020-11-20T12:07:08.495 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ip6.ko 2020-11-20T12:07:08.495 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_netlink.ko 2020-11-20T12:07:08.495 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/l2tp/l2tp_ppp.ko 2020-11-20T12:07:08.495 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/lapb/ 2020-11-20T12:07:08.495 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/lapb/lapb.ko 2020-11-20T12:07:08.496 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/ 2020-11-20T12:07:08.496 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/llc.ko 2020-11-20T12:07:08.496 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 263433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/llc/llc2.ko 2020-11-20T12:07:08.498 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mac802154/ 2020-11-20T12:07:08.498 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 218937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mac802154/mac802154.ko 2020-11-20T12:07:08.498 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mpls/ 2020-11-20T12:07:08.498 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8785 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/mpls/mpls_gso.ko 2020-11-20T12:07:08.499 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ 2020-11-20T12:07:08.499 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ 2020-11-20T12:07:08.499 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 135633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set.ko 2020-11-20T12:07:08.499 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2020-11-20T12:07:08.499 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26289 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2020-11-20T12:07:08.500 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25097 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2020-11-20T12:07:08.500 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 101529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2020-11-20T12:07:08.500 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 102593 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2020-11-20T12:07:08.500 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 103425 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2020-11-20T12:07:08.501 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 104705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2020-11-20T12:07:08.501 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 112065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2020-11-20T12:07:08.501 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2020-11-20T12:07:08.501 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 108489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2020-11-20T12:07:08.501 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 112145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2020-11-20T12:07:08.502 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 110513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2020-11-20T12:07:08.502 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 110633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2020-11-20T12:07:08.503 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 112513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2020-11-20T12:07:08.503 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipset/ip_set_list_set.ko 2020-11-20T12:07:08.503 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ 2020-11-20T12:07:08.503 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 384017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs.ko 2020-11-20T12:07:08.506 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11105 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2020-11-20T12:07:08.507 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2020-11-20T12:07:08.507 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2020-11-20T12:07:08.507 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2020-11-20T12:07:08.508 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2020-11-20T12:07:08.509 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2020-11-20T12:07:08.509 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2020-11-20T12:07:08.509 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2020-11-20T12:07:08.509 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8465 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2020-11-20T12:07:08.510 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2020-11-20T12:07:08.510 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2020-11-20T12:07:08.510 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6921 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2020-11-20T12:07:08.510 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2020-11-20T12:07:08.510 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 47153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conncount.ko 2020-11-20T12:07:08.511 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 414313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack.ko 2020-11-20T12:07:08.512 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_amanda.ko 2020-11-20T12:07:08.513 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_broadcast.ko 2020-11-20T12:07:08.513 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32945 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_ftp.ko 2020-11-20T12:07:08.513 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 118865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_h323.ko 2020-11-20T12:07:08.515 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21145 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_irc.ko 2020-11-20T12:07:08.515 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2020-11-20T12:07:08.515 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 105089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_netlink.ko 2020-11-20T12:07:08.516 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_pptp.ko 2020-11-20T12:07:08.517 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_sane.ko 2020-11-20T12:07:08.517 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 61369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_sip.ko 2020-11-20T12:07:08.518 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_snmp.ko 2020-11-20T12:07:08.518 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_conntrack_tftp.ko 2020-11-20T12:07:08.518 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_dup_netdev.ko 2020-11-20T12:07:08.519 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 75841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_flow_table.ko 2020-11-20T12:07:08.519 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_flow_table_inet.ko 2020-11-20T12:07:08.520 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_log_common.ko 2020-11-20T12:07:08.520 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 85969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat.ko 2020-11-20T12:07:08.521 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_amanda.ko 2020-11-20T12:07:08.521 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12161 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_ftp.ko 2020-11-20T12:07:08.521 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11385 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_irc.ko 2020-11-20T12:07:08.522 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 27417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_sip.ko 2020-11-20T12:07:08.522 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_nat_tftp.ko 2020-11-20T12:07:08.522 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 53193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_synproxy_core.ko 2020-11-20T12:07:08.523 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 492617 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nf_tables.ko 2020-11-20T12:07:08.528 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33153 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink.ko 2020-11-20T12:07:08.529 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_acct.ko 2020-11-20T12:07:08.529 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_cttimeout.ko 2020-11-20T12:07:08.530 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_log.ko 2020-11-20T12:07:08.530 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21665 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_osf.ko 2020-11-20T12:07:08.531 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 60217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nfnetlink_queue.ko 2020-11-20T12:07:08.531 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_chain_nat.ko 2020-11-20T12:07:08.531 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_compat.ko 2020-11-20T12:07:08.532 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19217 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_counter.ko 2020-11-20T12:07:08.533 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_ct.ko 2020-11-20T12:07:08.533 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8801 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_dup_netdev.ko 2020-11-20T12:07:08.534 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_fib.ko 2020-11-20T12:07:08.534 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_fwd_netdev.ko 2020-11-20T12:07:08.534 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_hash.ko 2020-11-20T12:07:08.534 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16761 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_limit.ko 2020-11-20T12:07:08.534 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14393 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_log.ko 2020-11-20T12:07:08.535 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14673 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_masq.ko 2020-11-20T12:07:08.535 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_nat.ko 2020-11-20T12:07:08.535 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_numgen.ko 2020-11-20T12:07:08.535 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_osf.ko 2020-11-20T12:07:08.535 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_queue.ko 2020-11-20T12:07:08.538 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_redir.ko 2020-11-20T12:07:08.538 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_reject.ko 2020-11-20T12:07:08.538 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_reject_inet.ko 2020-11-20T12:07:08.539 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_socket.ko 2020-11-20T12:07:08.539 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_synproxy.ko 2020-11-20T12:07:08.539 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/nft_tproxy.ko 2020-11-20T12:07:08.539 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 91641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/x_tables.ko 2020-11-20T12:07:08.539 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11681 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_AUDIT.ko 2020-11-20T12:07:08.540 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CHECKSUM.ko 2020-11-20T12:07:08.540 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CLASSIFY.ko 2020-11-20T12:07:08.540 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CONNSECMARK.ko 2020-11-20T12:07:08.540 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_CT.ko 2020-11-20T12:07:08.541 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10025 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_DSCP.ko 2020-11-20T12:07:08.541 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8697 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_HL.ko 2020-11-20T12:07:08.541 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_HMARK.ko 2020-11-20T12:07:08.541 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 38073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_IDLETIMER.ko 2020-11-20T12:07:08.542 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_LED.ko 2020-11-20T12:07:08.542 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_LOG.ko 2020-11-20T12:07:08.542 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_MASQUERADE.ko 2020-11-20T12:07:08.542 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10865 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NETMAP.ko 2020-11-20T12:07:08.542 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8193 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NFLOG.ko 2020-11-20T12:07:08.543 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14017 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_NFQUEUE.ko 2020-11-20T12:07:08.543 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 21489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_RATEEST.ko 2020-11-20T12:07:08.543 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_REDIRECT.ko 2020-11-20T12:07:08.543 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_SECMARK.ko 2020-11-20T12:07:08.543 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TCPMSS.ko 2020-11-20T12:07:08.544 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2020-11-20T12:07:08.544 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TEE.ko 2020-11-20T12:07:08.544 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20033 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TPROXY.ko 2020-11-20T12:07:08.548 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7353 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_TRACE.ko 2020-11-20T12:07:08.549 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_addrtype.ko 2020-11-20T12:07:08.549 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13417 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_bpf.ko 2020-11-20T12:07:08.549 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cgroup.ko 2020-11-20T12:07:08.549 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cluster.ko 2020-11-20T12:07:08.549 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6745 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_comment.ko 2020-11-20T12:07:08.550 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connbytes.ko 2020-11-20T12:07:08.550 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connlabel.ko 2020-11-20T12:07:08.550 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8585 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connlimit.ko 2020-11-20T12:07:08.550 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_connmark.ko 2020-11-20T12:07:08.550 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_conntrack.ko 2020-11-20T12:07:08.551 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_cpu.ko 2020-11-20T12:07:08.551 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_dccp.ko 2020-11-20T12:07:08.551 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_devgroup.ko 2020-11-20T12:07:08.551 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_dscp.ko 2020-11-20T12:07:08.552 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10313 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ecn.ko 2020-11-20T12:07:08.552 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10777 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_esp.ko 2020-11-20T12:07:08.552 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 49377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_hashlimit.ko 2020-11-20T12:07:08.552 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_helper.ko 2020-11-20T12:07:08.552 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7225 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_hl.ko 2020-11-20T12:07:08.553 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ipcomp.ko 2020-11-20T12:07:08.553 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_iprange.ko 2020-11-20T12:07:08.553 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11769 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_ipvs.ko 2020-11-20T12:07:08.553 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 16273 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_l2tp.ko 2020-11-20T12:07:08.554 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7185 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_length.ko 2020-11-20T12:07:08.554 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11561 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_limit.ko 2020-11-20T12:07:08.554 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_mac.ko 2020-11-20T12:07:08.554 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_mark.ko 2020-11-20T12:07:08.554 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_multiport.ko 2020-11-20T12:07:08.555 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13817 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_nat.ko 2020-11-20T12:07:08.555 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9689 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_nfacct.ko 2020-11-20T12:07:08.555 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_osf.ko 2020-11-20T12:07:08.555 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8809 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_owner.ko 2020-11-20T12:07:08.555 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9241 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_physdev.ko 2020-11-20T12:07:08.556 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6873 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_pkttype.ko 2020-11-20T12:07:08.556 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_policy.ko 2020-11-20T12:07:08.556 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_quota.ko 2020-11-20T12:07:08.556 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_rateest.ko 2020-11-20T12:07:08.556 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7553 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_realm.ko 2020-11-20T12:07:08.557 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_recent.ko 2020-11-20T12:07:08.557 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_sctp.ko 2020-11-20T12:07:08.557 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30825 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_set.ko 2020-11-20T12:07:08.557 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_socket.ko 2020-11-20T12:07:08.557 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_state.ko 2020-11-20T12:07:08.558 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_statistic.ko 2020-11-20T12:07:08.558 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7993 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_string.ko 2020-11-20T12:07:08.558 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_tcpmss.ko 2020-11-20T12:07:08.558 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_tcpudp.ko 2020-11-20T12:07:08.559 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10569 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_time.ko 2020-11-20T12:07:08.559 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netfilter/xt_u32.ko 2020-11-20T12:07:08.559 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netlink/ 2020-11-20T12:07:08.559 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 13329 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/netlink/netlink_diag.ko 2020-11-20T12:07:08.559 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/nsh/ 2020-11-20T12:07:08.560 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 11729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/nsh/nsh.ko 2020-11-20T12:07:08.560 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/ 2020-11-20T12:07:08.560 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 367929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/openvswitch.ko 2020-11-20T12:07:08.561 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9377 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-geneve.ko 2020-11-20T12:07:08.562 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 8505 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-gre.ko 2020-11-20T12:07:08.562 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10913 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/openvswitch/vport-vxlan.ko 2020-11-20T12:07:08.562 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/packet/ 2020-11-20T12:07:08.562 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/packet/af_packet_diag.ko 2020-11-20T12:07:08.563 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/ 2020-11-20T12:07:08.563 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 123433 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/phonet.ko 2020-11-20T12:07:08.563 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 54833 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/phonet/pn_pep.ko 2020-11-20T12:07:08.563 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/ 2020-11-20T12:07:08.564 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 239729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds.ko 2020-11-20T12:07:08.564 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 158753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds_rdma.ko 2020-11-20T12:07:08.564 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67537 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rds/rds_tcp.ko 2020-11-20T12:07:08.564 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rxrpc/ 2020-11-20T12:07:08.564 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 698057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/rxrpc/rxrpc.ko 2020-11-20T12:07:08.565 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/ 2020-11-20T12:07:08.565 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35121 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_csum.ko 2020-11-20T12:07:08.566 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26713 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_gact.ko 2020-11-20T12:07:08.566 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 37049 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_ipt.ko 2020-11-20T12:07:08.566 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 36497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_mirred.ko 2020-11-20T12:07:08.567 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_nat.ko 2020-11-20T12:07:08.567 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30929 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_pedit.ko 2020-11-20T12:07:08.568 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_police.ko 2020-11-20T12:07:08.568 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25577 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_simple.ko 2020-11-20T12:07:08.568 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_skbedit.ko 2020-11-20T12:07:08.569 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/act_vlan.ko 2020-11-20T12:07:08.569 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 24641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_basic.ko 2020-11-20T12:07:08.569 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34265 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_bpf.ko 2020-11-20T12:07:08.570 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18521 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_cgroup.ko 2020-11-20T12:07:08.570 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_flow.ko 2020-11-20T12:07:08.570 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28721 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_fw.ko 2020-11-20T12:07:08.571 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 34449 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_route.ko 2020-11-20T12:07:08.571 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_rsvp.ko 2020-11-20T12:07:08.571 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 35065 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_rsvp6.ko 2020-11-20T12:07:08.572 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 41113 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_tcindex.ko 2020-11-20T12:07:08.572 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 54513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/cls_u32.ko 2020-11-20T12:07:08.573 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6601 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_cmp.ko 2020-11-20T12:07:08.573 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 12625 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_ipset.ko 2020-11-20T12:07:08.573 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 31985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_meta.ko 2020-11-20T12:07:08.573 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7057 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_nbyte.ko 2020-11-20T12:07:08.574 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_text.ko 2020-11-20T12:07:08.574 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 6409 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/em_u32.ko 2020-11-20T12:07:08.574 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_cbq.ko 2020-11-20T12:07:08.575 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_choke.ko 2020-11-20T12:07:08.575 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19849 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_codel.ko 2020-11-20T12:07:08.575 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 30513 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_drr.ko 2020-11-20T12:07:08.576 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 32857 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_dsmark.ko 2020-11-20T12:07:08.576 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29609 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_fq.ko 2020-11-20T12:07:08.576 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28737 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_fq_codel.ko 2020-11-20T12:07:08.576 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29649 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_gred.ko 2020-11-20T12:07:08.577 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 46041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_hfsc.ko 2020-11-20T12:07:08.578 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 20081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_hhf.ko 2020-11-20T12:07:08.578 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 50497 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_htb.ko 2020-11-20T12:07:08.578 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15073 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_ingress.ko 2020-11-20T12:07:08.578 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22841 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_mqprio.ko 2020-11-20T12:07:08.578 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23705 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_multiq.ko 2020-11-20T12:07:08.579 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 33361 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_netem.ko 2020-11-20T12:07:08.580 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 23369 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_pie.ko 2020-11-20T12:07:08.580 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9489 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_plug.ko 2020-11-20T12:07:08.580 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 22969 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_prio.ko 2020-11-20T12:07:08.580 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 44257 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_qfq.ko 2020-11-20T12:07:08.581 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 28633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_red.ko 2020-11-20T12:07:08.581 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25009 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_sfb.ko 2020-11-20T12:07:08.581 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 29041 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_sfq.ko 2020-11-20T12:07:08.581 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 26089 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_tbf.ko 2020-11-20T12:07:08.582 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 25633 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sched/sch_teql.ko 2020-11-20T12:07:08.582 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/ 2020-11-20T12:07:08.582 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 879937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/sctp.ko 2020-11-20T12:07:08.591 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 18081 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sctp/sctp_diag.ko 2020-11-20T12:07:08.591 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/ 2020-11-20T12:07:08.592 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/ 2020-11-20T12:07:08.592 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 334985 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2020-11-20T12:07:08.595 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 62473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2020-11-20T12:07:08.595 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1415977 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/sunrpc.ko 2020-11-20T12:07:08.609 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/xprtrdma/ 2020-11-20T12:07:08.609 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 644937 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2020-11-20T12:07:08.615 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/ 2020-11-20T12:07:08.615 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9209 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/diag.ko 2020-11-20T12:07:08.615 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 1118753 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/tipc/tipc.ko 2020-11-20T12:07:08.627 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/unix/ 2020-11-20T12:07:08.628 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 15201 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/unix/unix_diag.ko 2020-11-20T12:07:08.628 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/ 2020-11-20T12:07:08.628 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 73233 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2020-11-20T12:07:08.629 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 67177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2020-11-20T12:07:08.629 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 77529 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock.ko 2020-11-20T12:07:08.629 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 9953 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock_diag.ko 2020-11-20T12:07:08.630 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 17297 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/vmw_vsock/vsock_loopback.ko 2020-11-20T12:07:08.630 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/x25/ 2020-11-20T12:07:08.630 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 126641 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/x25/x25.ko 2020-11-20T12:07:08.632 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/ 2020-11-20T12:07:08.632 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19657 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_algo.ko 2020-11-20T12:07:08.632 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 19793 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_ipcomp.ko 2020-11-20T12:07:08.633 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 81177 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/net/xfrm/xfrm_user.ko 2020-11-20T12:07:08.633 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/ 2020-11-20T12:07:08.634 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/lib/ 2020-11-20T12:07:08.634 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 14473 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/kernel/virt/lib/irqbypass.ko 2020-11-20T12:07:08.634 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 614204 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.alias 2020-11-20T12:07:08.637 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 599988 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.alias.bin 2020-11-20T12:07:08.643 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 7746 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin 2020-11-20T12:07:08.643 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 10228 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin.bin 2020-11-20T12:07:08.643 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 63287 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.builtin.modinfo 2020-11-20T12:07:08.644 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 146392 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.dep 2020-11-20T12:07:08.645 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 235760 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.dep.bin 2020-11-20T12:07:08.647 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 210 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.devname 2020-11-20T12:07:08.647 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 74729 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.order 2020-11-20T12:07:08.648 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 574 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.softdep 2020-11-20T12:07:08.648 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 238541 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.symbols 2020-11-20T12:07:08.650 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 289923 2020-11-13 14:15 ./lib/modules/5.10.0-rc2-ceph-gac4f7b07f403/modules.symbols.bin 2020-11-20T12:07:08.653 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/ 2020-11-20T12:07:08.654 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/ 2020-11-20T12:07:08.654 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/doc/ 2020-11-20T12:07:08.654 INFO:teuthology.orchestra.run.smithi173.stdout:drwxr-xr-x root/root 0 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/ 2020-11-20T12:07:08.655 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 178 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/changelog.Debian.gz 2020-11-20T12:07:08.655 INFO:teuthology.orchestra.run.smithi173.stdout:-rw-r--r-- root/root 692 2020-11-13 14:15 ./usr/share/doc/linux-image-5.10.0-rc2-ceph-gac4f7b07f403/copyright 2020-11-20T12:07:08.656 DEBUG:teuthology.task.kernel:get_image_version: 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:07:08.657 INFO:teuthology.task.kernel:searching for kernel 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:07:08.657 INFO:teuthology.orchestra.run.smithi173:> egrep '(submenu|menuentry.*5.10.0-rc2-ceph-gac4f7b07f403).*{' /boot/grub/grub.cfg 2020-11-20T12:07:08.707 INFO:teuthology.orchestra.run.smithi173.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-20T12:07:08.708 INFO:teuthology.orchestra.run.smithi173.stdout: menuentry 'Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.0-rc2-ceph-gac4f7b07f403-advanced-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-20T12:07:08.708 INFO:teuthology.orchestra.run.smithi173.stdout: menuentry 'Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-5.10.0-rc2-ceph-gac4f7b07f403-recovery-5b37dc69-d2db-44bf-9122-6cf42a1bab47' { 2020-11-20T12:07:08.708 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2020-11-20T12:07:08.709 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:07:08.709 INFO:teuthology.orchestra.run.smithi173:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2020-11-20T12:07:08.752 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi116.front.sepia.ceph.com to complete... 2020-11-20T12:07:08.752 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi173.front.sepia.ceph.com to complete... 2020-11-20T12:07:09.256 INFO:teuthology.orchestra.run.smithi173.stderr:Sourcing file `/etc/default/grub' 2020-11-20T12:07:09.276 INFO:teuthology.orchestra.run.smithi173.stderr:Generating grub configuration file ... 2020-11-20T12:07:09.518 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:07:09.528 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:07:09.834 INFO:teuthology.orchestra.run.smithi173.stderr:Found linux image: /boot/vmlinuz-4.15.0-123-generic 2020-11-20T12:07:09.847 INFO:teuthology.orchestra.run.smithi173.stderr:Found initrd image: /boot/initrd.img-4.15.0-123-generic 2020-11-20T12:07:11.122 INFO:teuthology.orchestra.run.smithi173.stderr:done 2020-11-20T12:07:42.269 INFO:teuthology.misc:Re-opening connections... 2020-11-20T12:07:42.269 INFO:teuthology.misc:trying to connect to ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:07:42.270 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T12:07:42.271 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:08:16.885 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.116 2020-11-20T12:08:16.922 INFO:teuthology.misc:trying to connect to ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:08:16.923 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T12:08:16.924 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:09:16.926 DEBUG:teuthology.orchestra.remote:timed out 2020-11-20T12:09:16.970 DEBUG:teuthology.misc:waited 94.70029735565186 2020-11-20T12:09:17.971 INFO:teuthology.misc:trying to connect to ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:09:17.971 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T12:09:17.972 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:09:18.032 INFO:teuthology.orchestra.run.smithi116:> true 2020-11-20T12:09:18.503 DEBUG:teuthology.misc:waited 96.23369216918945 2020-11-20T12:09:19.505 INFO:teuthology.misc:trying to connect to ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:09:19.505 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T12:09:19.506 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:09:19.581 INFO:teuthology.orchestra.run.smithi173:> true 2020-11-20T12:09:20.082 DEBUG:teuthology.misc:waited 97.81308555603027 2020-11-20T12:09:21.083 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2020-11-20T12:09:21.083 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.10.0-rc2-ceph-gac4f7b07f403"... 2020-11-20T12:09:21.084 INFO:teuthology.orchestra.run.smithi116:> uname -r 2020-11-20T12:09:21.102 INFO:teuthology.orchestra.run.smithi116.stdout:5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:09:21.102 DEBUG:teuthology.task.kernel:current kernel version is 5.10.0-rc2-ceph-gac4f7b07f403 vs 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:09:21.103 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-11-20T12:09:21.103 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-11-20T12:09:21.103 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2020-11-20T12:09:21.104 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "5.10.0-rc2-ceph-gac4f7b07f403"... 2020-11-20T12:09:21.104 INFO:teuthology.orchestra.run.smithi173:> uname -r 2020-11-20T12:09:21.122 INFO:teuthology.orchestra.run.smithi173.stdout:5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:09:21.122 DEBUG:teuthology.task.kernel:current kernel version is 5.10.0-rc2-ceph-gac4f7b07f403 vs 5.10.0-rc2-ceph-gac4f7b07f403 2020-11-20T12:09:21.123 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2020-11-20T12:09:21.123 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2020-11-20T12:09:22.124 INFO:teuthology.run_tasks:Running task internal.base... 2020-11-20T12:09:22.131 INFO:teuthology.task.internal:Creating test directory... 2020-11-20T12:09:22.132 INFO:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-20T12:09:22.139 INFO:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2020-11-20T12:09:22.156 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2020-11-20T12:09:22.215 INFO:teuthology.run_tasks:Running task internal.archive... 2020-11-20T12:09:22.230 INFO:teuthology.task.internal:Creating archive directory... 2020-11-20T12:09:22.231 INFO:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-20T12:09:22.238 INFO:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2020-11-20T12:09:22.268 INFO:teuthology.run_tasks:Running task internal.coredump... 2020-11-20T12:09:22.278 INFO:teuthology.task.internal:Enabling coredump saving... 2020-11-20T12:09:22.278 INFO:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-20T12:09:22.318 INFO:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2020-11-20T12:09:22.358 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T12:09:22.372 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T12:09:22.379 INFO:teuthology.orchestra.run.smithi116.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T12:09:22.401 INFO:teuthology.orchestra.run.smithi173.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2020-11-20T12:09:22.404 INFO:teuthology.run_tasks:Running task internal.sudo... 2020-11-20T12:09:22.412 INFO:teuthology.task.internal:Configuring sudo... 2020-11-20T12:09:22.412 INFO:teuthology.orchestra.run.smithi116:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-20T12:09:22.418 INFO:teuthology.orchestra.run.smithi173:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2020-11-20T12:09:22.453 INFO:teuthology.run_tasks:Running task internal.syslog... 2020-11-20T12:09:22.467 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2020-11-20T12:09:22.467 INFO:teuthology.orchestra.run.smithi116:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-20T12:09:22.474 INFO:teuthology.orchestra.run.smithi173:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2020-11-20T12:09:22.491 INFO:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T12:09:22.542 INFO:teuthology.orchestra.run.smithi116:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T12:09:22.553 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:09:22.553 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T12:09:22.566 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T12:09:22.584 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:09:22.584 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:09:22.585 INFO:teuthology.orchestra.run.smithi116.stderr:152 bytes copied, 0.000102558 s, 1.5 MB/s 2020-11-20T12:09:22.587 INFO:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2020-11-20T12:09:22.604 INFO:teuthology.orchestra.run.smithi173:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2020-11-20T12:09:22.619 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:09:22.619 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T12:09:22.679 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2020-11-20T12:09:22.702 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:09:22.702 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:09:22.702 INFO:teuthology.orchestra.run.smithi173.stderr:152 bytes copied, 0.00019426 s, 782 kB/s 2020-11-20T12:09:22.706 INFO:teuthology.orchestra.run.smithi116:> sudo service rsyslog restart 2020-11-20T12:09:22.721 INFO:teuthology.orchestra.run.smithi173:> sudo service rsyslog restart 2020-11-20T12:09:22.851 INFO:teuthology.run_tasks:Running task internal.timer... 2020-11-20T12:09:22.866 INFO:teuthology.task.internal:Starting timer... 2020-11-20T12:09:22.867 INFO:teuthology.run_tasks:Running task pcp... 2020-11-20T12:09:22.963 INFO:teuthology.run_tasks:Running task selinux... 2020-11-20T12:09:23.078 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2020-11-20T12:09:23.079 DEBUG:teuthology.task.selinux:Excluding smithi116: OS 'ubuntu' does not support SELinux 2020-11-20T12:09:23.079 DEBUG:teuthology.task.selinux:Excluding smithi173: OS 'ubuntu' does not support SELinux 2020-11-20T12:09:23.079 DEBUG:teuthology.task.selinux:Getting current SELinux state 2020-11-20T12:09:23.079 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2020-11-20T12:09:23.080 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2020-11-20T12:09:23.080 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2020-11-20T12:09:23.166 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2020-11-20T12:09:23.167 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2020-11-20T12:09:23.180 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2020-11-20T12:09:23.182 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi116.front.sepia.ceph.com,smithi173.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2020-11-20T12:09:23.527 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2020-11-20T12:09:24.475 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2020-11-20T12:09:27.462 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2020-11-20T12:09:27.481 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:27 +0000 (0:00:00.072) 0:00:00.072 ******* 2020-11-20T12:09:27.494 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:27 +0000 (0:00:00.012) 0:00:00.084 ******* 2020-11-20T12:09:29.970 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-11-20T12:09:29.972 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] 2020-11-20T12:09:30.032 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] 2020-11-20T12:09:30.082 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:30 +0000 (0:00:02.587) 0:00:02.672 ******* 2020-11-20T12:09:30.163 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:30 +0000 (0:00:00.081) 0:00:02.753 ******* 2020-11-20T12:09:30.401 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-20T12:09:30.443 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:30 +0000 (0:00:00.279) 0:00:03.033 ******* 2020-11-20T12:09:30.449 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-20T12:09:30.508 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:30 +0000 (0:00:00.064) 0:00:03.098 ******* 2020-11-20T12:09:30.988 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2020-11-20T12:09:30.990 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-20T12:09:31.042 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:31 +0000 (0:00:00.534) 0:00:03.632 ******* 2020-11-20T12:09:31.049 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"append": false, "changed": true, "comment": "cm,,,", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2020-11-20T12:09:31.091 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:31 +0000 (0:00:00.049) 0:00:03.681 ******* 2020-11-20T12:09:31.469 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2020-11-20T12:09:31.470 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.089201", "end": "2020-11-20 12:09:31.257697", "rc": 0, "start": "2020-11-20 12:09:31.168496", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-11-20T12:09:31.471 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["passwd", "-d", "cm"], "delta": "0:00:00.092483", "end": "2020-11-20 12:09:31.501510", "rc": 0, "start": "2020-11-20 12:09:31.409027", "stderr": "", "stderr_lines": [], "stdout": "passwd: password expiry information changed.", "stdout_lines": ["passwd: password expiry information changed."]} 2020-11-20T12:09:31.525 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:31 +0000 (0:00:00.433) 0:00:04.115 ******* 2020-11-20T12:09:31.587 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:31 +0000 (0:00:00.061) 0:00:04.177 ******* 2020-11-20T12:09:31.831 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2020-11-20T12:09:31.833 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T12:09:31.874 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:31 +0000 (0:00:00.287) 0:00:04.465 ******* 2020-11-20T12:09:31.881 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T12:09:31.930 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:31 +0000 (0:00:00.056) 0:00:04.521 ******* 2020-11-20T12:09:32.737 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** 2020-11-20T12:09:32.739 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605874171.8872738-5743872384350/source", "state": "file", "uid": 0} 2020-11-20T12:09:32.797 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:32 +0000 (0:00:00.866) 0:00:05.387 ******* 2020-11-20T12:09:32.805 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605874171.9443388-73330831573470/source", "state": "file", "uid": 0} 2020-11-20T12:09:32.848 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:32 +0000 (0:00:00.051) 0:00:05.438 ******* 2020-11-20T12:09:33.190 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2020-11-20T12:09:33.194 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-20T12:09:33.196 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-20T12:09:33.235 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2020-11-20T12:09:33.236 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2020-11-20T12:09:33.370 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2020-11-20T12:09:33.378 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2020-11-20T12:09:33.549 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:33 +0000 (0:00:00.700) 0:00:06.139 ******* 2020-11-20T12:09:33.589 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:33 +0000 (0:00:00.039) 0:00:06.179 ******* 2020-11-20T12:09:34.983 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2020-11-20T12:09:34.989 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] 2020-11-20T12:09:35.044 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] 2020-11-20T12:09:35.099 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2020-11-20T12:09:35.101 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:35 +0000 (0:00:01.510) 0:00:07.689 ******* 2020-11-20T12:09:35.202 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:35 +0000 (0:00:00.102) 0:00:07.792 ******* 2020-11-20T12:09:35.518 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2020-11-20T12:09:35.520 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.172235", "end": "2020-11-20 12:09:35.490853", "rc": 0, "start": "2020-11-20 12:09:35.318618", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T12:09:35.562 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:35 +0000 (0:00:00.360) 0:00:08.152 ******* 2020-11-20T12:09:35.579 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.151313", "end": "2020-11-20 12:09:35.554828", "rc": 0, "start": "2020-11-20 12:09:35.403515", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T12:09:35.621 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:35 +0000 (0:00:00.059) 0:00:08.212 ******* 2020-11-20T12:09:35.765 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2020-11-20T12:09:35.766 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.017928", "end": "2020-11-20 12:09:35.545464", "failed_when_result": false, "rc": 0, "start": "2020-11-20 12:09:35.527536", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T12:09:35.819 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:35 +0000 (0:00:00.197) 0:00:08.409 ******* 2020-11-20T12:09:35.836 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.024556", "end": "2020-11-20 12:09:35.848338", "failed_when_result": false, "rc": 0, "start": "2020-11-20 12:09:35.823782", "stderr": "", "stderr_lines": [], "stdout": "46fd2a03b54c2ce7520f33746fa0586027b558c3", "stdout_lines": ["46fd2a03b54c2ce7520f33746fa0586027b558c3"]} 2020-11-20T12:09:35.878 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:35 +0000 (0:00:00.059) 0:00:08.468 ******* 2020-11-20T12:09:35.884 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2020-11-20T12:09:35.885 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-20T12:09:35.927 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:35 +0000 (0:00:00.049) 0:00:08.518 ******* 2020-11-20T12:09:35.934 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"perform_users_role": false}, "changed": false} 2020-11-20T12:09:35.977 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:35 +0000 (0:00:00.050) 0:00:08.568 ******* 2020-11-20T12:09:35.984 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2020-11-20T12:09:35.986 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.043 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:36 +0000 (0:00:00.065) 0:00:08.634 ******* 2020-11-20T12:09:36.053 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.106 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:36 +0000 (0:00:00.062) 0:00:08.696 ******* 2020-11-20T12:09:36.114 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2020-11-20T12:09:36.116 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.167 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:36 +0000 (0:00:00.061) 0:00:08.757 ******* 2020-11-20T12:09:36.174 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.223 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:36 +0000 (0:00:00.056) 0:00:08.814 ******* 2020-11-20T12:09:36.229 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.272 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:36 +0000 (0:00:00.048) 0:00:08.862 ******* 2020-11-20T12:09:36.344 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 12:09:36 +0000 (0:00:00.051) 0:00:08.914 ******* TASK [users : Filter the managed_admin_users list] ***************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.376 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:36 +0000 (0:00:00.051) 0:00:08.966 ******* 2020-11-20T12:09:36.383 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.426 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:36 +0000 (0:00:00.050) 0:00:09.016 ******* 2020-11-20T12:09:36.432 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2020-11-20T12:09:36.435 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.474 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:36 +0000 (0:00:00.047) 0:00:09.064 ******* 2020-11-20T12:09:36.483 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.548 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:36 +0000 (0:00:00.074) 0:00:09.138 ******* 2020-11-20T12:09:36.555 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2020-11-20T12:09:36.557 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.558 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ 2020-11-20T12:09:36.559 INFO:teuthology.task.ansible.out:ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.570 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.582 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.584 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.594 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.606 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.623 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.624 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.626 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.638 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.639 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.642 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr 2020-11-20T12:09:36.644 INFO:teuthology.task.ansible.out:+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles 2020-11-20T12:09:36.646 INFO:teuthology.task.ansible.out: jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@fedora rT6gfdsBHVr0RvokJfO3mQ e634e7a31d873f310ff5326bacae6c19f95087f8215edda618f68dd2761257a9"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "i 2020-11-20T12:09:36.647 INFO:teuthology.task.ansible.out:tem": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.655 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.657 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f30 2020-11-20T12:09:36.659 INFO:teuthology.task.ansible.out:09adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.674 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.676 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.677 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAA 2020-11-20T12:09:36.678 INFO:teuthology.task.ansible.out:ABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.689 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.692 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/ 2020-11-20T12:09:36.696 INFO:teuthology.task.ansible.out:sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.703 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'davidz', 'ovpn': 'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.715 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.726 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.765 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'wusui', 'ovpn': 'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'zyan', 'ovpn': 'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyu 2020-11-20T12:09:36.770 INFO:teuthology.task.ansible.out:riw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9 2020-11-20T12:09:36.772 INFO:teuthology.task.ansible.out:+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dc 2020-11-20T12:09:36.775 INFO:teuthology.task.ansible.out:efa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name 2020-11-20T12:09:36.777 INFO:teuthology.task.ansible.out:': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af9 2020-11-20T12:09:36.779 INFO:teuthology.task.ansible.out:4ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@ti 2020-11-20T12:09:36.782 INFO:teuthology.task.ansible.out:mesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a9 2020-11-20T12:09:36.784 INFO:teuthology.task.ansible.out:53dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.788 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'nwatkins', 'ovpn': 'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.790 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.801 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.803 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jdillaman', 'ovpn': 'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.815 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.816 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.826 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.828 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.828 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi173 2020-11-20T12:09:36.831 INFO:teuthology.task.ansible.out:.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.841 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.852 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.854 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.866 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.868 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.879 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.882 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.884 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.886 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.888 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.902 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.906 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.907 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.909 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.911 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.917 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.924 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.929 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.932 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.933 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.938 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.951 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.964 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.980 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 2020-11-20T12:09:36.985 INFO:teuthology.task.ansible.out:zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:36.992 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.005 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.007 INFO:teuthology.task.ansible.out: 2020-11-20T12:09:37.007 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.009 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.009 INFO:teuthology.task.ansible.out: 2020-11-20T12:09:37.015 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.026 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.032 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.fro 2020-11-20T12:09:37.034 INFO:teuthology.task.ansible.out:nt.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.039 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.041 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.052 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.057 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.066 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774 2020-11-20T12:09:37.068 INFO:teuthology.task.ansible.out:707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.078 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'sidharthanup', 'ovpn': 'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.081 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'varsha', 'ovpn': 'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.090 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.092 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.093 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.095 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.095 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.098 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.109 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.121 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.123 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.123 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.123 INFO:teuthology.task.ansible.out: 2020-11-20T12:09:37.125 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.125 INFO:teuthology.task.ansible.out: 2020-11-20T12:09:37.132 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96c 2020-11-20T12:09:37.137 INFO:teuthology.task.ansible.out:c2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.139 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.141 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.152 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.157 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.166 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.168 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.178 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.180 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 4NMUghtas9gI46n7X0iJXA cc646e47c8b503eb38d30efa47e459076af52b4fc001c4959833f32ddc563a02"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.191 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.193 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.206 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsusel 2020-11-20T12:09:37.208 INFO:teuthology.task.ansible.out:aptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.218 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.220 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.231 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.233 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.246 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.248 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.248 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn 2020-11-20T12:09:37.250 INFO:teuthology.task.ansible.out:": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.263 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.274 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.277 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.279 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.280 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.292 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.303 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.305 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.315 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.317 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.332 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.333 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.347 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'hmunjulu', 'ovpn': 'hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hmunjulu", "ovpn": "hmunjulu@thinkpad +5kJJIhXveJw6Q8ytILUSA 546f32fb058ff40ef3113e6c258b0f1cba6611155ba0a40a55646946cdb53019"}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.374 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "prashant@cephdev JZCEN2T5TXiHnU5yLG4hBg 3f94ec1bd179dafd2d58335ac4bea35808d394d30fa9e174fe16d6b872450fe0"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.431 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.882) 0:00:10.020 ******* 2020-11-20T12:09:37.471 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.040) 0:00:10.061 ******* 2020-11-20T12:09:37.478 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2020-11-20T12:09:37.524 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.053) 0:00:10.114 ******* 2020-11-20T12:09:37.575 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.050) 0:00:10.165 ******* 2020-11-20T12:09:37.580 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2020-11-20T12:09:37.583 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.631 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.054) 0:00:10.219 ******* 2020-11-20T12:09:37.636 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.688 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.058) 0:00:10.277 ******* 2020-11-20T12:09:37.693 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2020-11-20T12:09:37.694 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.739 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.052) 0:00:10.330 ******* 2020-11-20T12:09:37.746 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.788 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.048) 0:00:10.379 ******* 2020-11-20T12:09:37.794 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2020-11-20T12:09:37.796 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.843 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.054) 0:00:10.433 ******* 2020-11-20T12:09:37.850 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.893 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.050) 0:00:10.483 ******* 2020-11-20T12:09:37.899 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2020-11-20T12:09:37.901 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.942 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.048) 0:00:10.532 ******* 2020-11-20T12:09:37.948 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:37.993 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:37 +0000 (0:00:00.050) 0:00:10.582 ******* 2020-11-20T12:09:37.998 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** 2020-11-20T12:09:38.000 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.041 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:38 +0000 (0:00:00.049) 0:00:10.632 ******* 2020-11-20T12:09:38.048 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.093 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:38 +0000 (0:00:00.051) 0:00:10.683 ******* 2020-11-20T12:09:38.099 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2020-11-20T12:09:38.100 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.100 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.101 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.101 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.115 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.125 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.127 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.138 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.140 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.150 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.152 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.162 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.164 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.177 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.190 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.191 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.226 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.227 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.239 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.263 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.265 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.265 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.266 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.266 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.280 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.282 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.282 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.282 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.283 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.283 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.284 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.284 INFO:teuthology.task.ansible.out:skipping: [sm 2020-11-20T12:09:38.287 INFO:teuthology.task.ansible.out:ithi116.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.298 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.310 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.322 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.333 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.344 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.346 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.358 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.369 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.371 INFO:teuthology.task.ansible.out: skipping: [smithi116.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.371 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.371 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.382 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.384 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:38.438 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:38 +0000 (0:00:00.344) 0:00:11.028 ******* 2020-11-20T12:09:38.479 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:38 +0000 (0:00:00.040) 0:00:11.069 ******* 2020-11-20T12:09:39.135 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2020-11-20T12:09:39.136 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-11-20T12:09:39.137 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "eefb312a9cf82af5b9a46fb88fa640babe5fe20f", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/keys-repo-sha1", "size": 40, "state": "file", "uid": 0} 2020-11-20T12:09:39.178 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.699) 0:00:11.768 ******* 2020-11-20T12:09:39.230 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.052) 0:00:11.820 ******* 2020-11-20T12:09:39.239 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2020-11-20T12:09:39.240 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => { "msg": "Host smithi116.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2020-11-20T12:09:39.296 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.065) 0:00:11.886 ******* 2020-11-20T12:09:39.302 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => { "msg": "Host smithi173.front.sepia.ceph.com is running Ubuntu 18.04 (bionic)" } 2020-11-20T12:09:39.347 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.051) 0:00:11.937 ******* 2020-11-20T12:09:39.352 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2020-11-20T12:09:39.354 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-11-20T12:09:39.400 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.053) 0:00:11.991 ******* 2020-11-20T12:09:39.406 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2020-11-20T12:09:39.449 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.048) 0:00:12.039 ******* 2020-11-20T12:09:39.454 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2020-11-20T12:09:39.456 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:39.500 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.051) 0:00:12.090 ******* 2020-11-20T12:09:39.507 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:39.551 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.051) 0:00:12.142 ******* 2020-11-20T12:09:39.557 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2020-11-20T12:09:39.559 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:39.601 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.048) 0:00:12.190 ******* 2020-11-20T12:09:39.607 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:39.649 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.048) 0:00:12.239 ******* 2020-11-20T12:09:39.655 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2020-11-20T12:09:39.657 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:39.699 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.049) 0:00:12.289 ******* 2020-11-20T12:09:39.705 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:39.748 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.049) 0:00:12.338 ******* 2020-11-20T12:09:39.754 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2020-11-20T12:09:39.756 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:39.801 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.052) 0:00:12.391 ******* 2020-11-20T12:09:39.808 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:39.850 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.048) 0:00:12.440 ******* 2020-11-20T12:09:39.856 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2020-11-20T12:09:39.857 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:39.900 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.050) 0:00:12.490 ******* 2020-11-20T12:09:39.906 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:39.952 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.052) 0:00:12.543 ******* 2020-11-20T12:09:39.958 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2020-11-20T12:09:39.961 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.002 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:39 +0000 (0:00:00.050) 0:00:12.593 ******* 2020-11-20T12:09:40.009 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.052 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.049) 0:00:12.643 ******* 2020-11-20T12:09:40.058 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2020-11-20T12:09:40.060 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.103 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.050) 0:00:12.693 ******* 2020-11-20T12:09:40.115 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.152 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.049) 0:00:12.743 ******* 2020-11-20T12:09:40.158 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2020-11-20T12:09:40.161 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.202 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.049) 0:00:12.792 ******* 2020-11-20T12:09:40.209 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.253 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.049) 0:00:12.842 ******* 2020-11-20T12:09:40.257 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2020-11-20T12:09:40.259 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:40.302 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.050) 0:00:12.893 ******* 2020-11-20T12:09:40.309 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:40.311 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:40.351 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.048) 0:00:12.941 ******* 2020-11-20T12:09:40.360 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.404 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.052) 0:00:12.994 ******* 2020-11-20T12:09:40.410 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.456 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.052) 0:00:13.046 ******* 2020-11-20T12:09:40.462 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2020-11-20T12:09:40.464 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.505 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.048) 0:00:13.095 ******* 2020-11-20T12:09:40.511 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.554 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.049) 0:00:13.144 ******* 2020-11-20T12:09:40.560 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2020-11-20T12:09:40.562 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:40.606 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.051) 0:00:13.196 ******* 2020-11-20T12:09:40.612 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:40.655 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.048) 0:00:13.245 ******* 2020-11-20T12:09:40.660 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2020-11-20T12:09:40.662 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.703 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.048) 0:00:13.293 ******* 2020-11-20T12:09:40.712 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.753 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.049) 0:00:13.343 ******* 2020-11-20T12:09:40.758 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2020-11-20T12:09:40.760 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:40.801 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.048) 0:00:13.392 ******* 2020-11-20T12:09:40.808 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:40.849 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.047) 0:00:13.439 ******* 2020-11-20T12:09:40.855 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.902 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.052) 0:00:13.492 ******* 2020-11-20T12:09:40.908 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:40.952 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.050) 0:00:13.542 ******* 2020-11-20T12:09:40.957 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2020-11-20T12:09:40.959 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.002 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:40 +0000 (0:00:00.050) 0:00:13.592 ******* 2020-11-20T12:09:41.008 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.050 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.048) 0:00:13.641 ******* 2020-11-20T12:09:41.056 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2020-11-20T12:09:41.058 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.103 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.050) 0:00:13.691 ******* 2020-11-20T12:09:41.109 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.193 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.087) 0:00:13.779 ******* 2020-11-20T12:09:41.195 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2020-11-20T12:09:41.197 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.241 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.052) 0:00:13.831 ******* 2020-11-20T12:09:41.248 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.304 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.062) 0:00:13.894 ******* 2020-11-20T12:09:41.310 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2020-11-20T12:09:41.312 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:41.355 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.051) 0:00:13.946 ******* 2020-11-20T12:09:41.363 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:41.431 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.072) 0:00:14.018 ******* 2020-11-20T12:09:41.437 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2020-11-20T12:09:41.438 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.489 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.058) 0:00:14.076 ******* 2020-11-20T12:09:41.495 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.537 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.050) 0:00:14.127 ******* 2020-11-20T12:09:41.543 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2020-11-20T12:09:41.544 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.588 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.051) 0:00:14.179 ******* 2020-11-20T12:09:41.595 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.649 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.060) 0:00:14.239 ******* 2020-11-20T12:09:41.655 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.701 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.049) 0:00:14.289 ******* 2020-11-20T12:09:41.706 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.750 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.051) 0:00:14.340 ******* 2020-11-20T12:09:41.756 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2020-11-20T12:09:41.758 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.801 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.051) 0:00:14.391 ******* 2020-11-20T12:09:41.808 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.851 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.049) 0:00:14.441 ******* 2020-11-20T12:09:41.857 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2020-11-20T12:09:41.858 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.900 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.049) 0:00:14.490 ******* 2020-11-20T12:09:41.906 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:41.950 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.049) 0:00:14.540 ******* 2020-11-20T12:09:41.956 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2020-11-20T12:09:41.958 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.002 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:41 +0000 (0:00:00.052) 0:00:14.593 ******* 2020-11-20T12:09:42.009 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.051 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.048) 0:00:14.642 ******* 2020-11-20T12:09:42.057 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2020-11-20T12:09:42.059 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:42.100 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.048) 0:00:14.691 ******* 2020-11-20T12:09:42.107 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:42.151 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.051) 0:00:14.742 ******* 2020-11-20T12:09:42.157 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2020-11-20T12:09:42.159 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.203 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.051) 0:00:14.793 ******* 2020-11-20T12:09:42.209 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.253 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.050) 0:00:14.843 ******* 2020-11-20T12:09:42.259 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2020-11-20T12:09:42.261 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.306 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.053) 0:00:14.897 ******* 2020-11-20T12:09:42.313 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.359 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.052) 0:00:14.949 ******* 2020-11-20T12:09:42.365 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2020-11-20T12:09:42.367 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.410 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.050) 0:00:15.000 ******* 2020-11-20T12:09:42.416 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.471 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.061) 0:00:15.061 ******* 2020-11-20T12:09:42.477 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2020-11-20T12:09:42.479 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.528 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.056) 0:00:15.117 ******* 2020-11-20T12:09:42.533 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.534 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.581 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.053) 0:00:15.171 ******* 2020-11-20T12:09:42.587 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2020-11-20T12:09:42.588 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.600 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.655 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.072) 0:00:15.244 ******* 2020-11-20T12:09:42.662 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.663 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "item": "epel-testing-mirrorlist", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.717 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.063) 0:00:15.307 ******* 2020-11-20T12:09:42.728 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.767 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.049) 0:00:15.357 ******* 2020-11-20T12:09:42.774 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:42.816 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:42 +0000 (0:00:00.049) 0:00:15.407 ******* 2020-11-20T12:09:42.979 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.024565", "end": "2020-11-20 12:09:42.770272", "rc": 0, "start": "2020-11-20 12:09:42.745707", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-20T12:09:43.026 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.209) 0:00:15.616 ******* 2020-11-20T12:09:43.033 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "cmd": "cat /etc/timezone", "delta": "0:00:00.018417", "end": "2020-11-20 12:09:43.040666", "rc": 0, "start": "2020-11-20 12:09:43.022249", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2020-11-20T12:09:43.078 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.052) 0:00:15.668 ******* 2020-11-20T12:09:43.086 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.136 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.058) 0:00:15.727 ******* 2020-11-20T12:09:43.144 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.194 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.055) 0:00:15.782 ******* 2020-11-20T12:09:43.198 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2020-11-20T12:09:43.200 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.247 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.053) 0:00:15.836 ******* 2020-11-20T12:09:43.253 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.296 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.049) 0:00:15.885 ******* 2020-11-20T12:09:43.302 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2020-11-20T12:09:43.306 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.345 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.049) 0:00:15.935 ******* 2020-11-20T12:09:43.352 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.394 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.048) 0:00:15.984 ******* 2020-11-20T12:09:43.402 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.461 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.063) 0:00:16.048 ******* 2020-11-20T12:09:43.468 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.509 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.051) 0:00:16.099 ******* 2020-11-20T12:09:43.517 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.591 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.077) 0:00:16.176 ******* 2020-11-20T12:09:43.593 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.636 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.049) 0:00:16.226 ******* 2020-11-20T12:09:43.643 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.706 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.069) 0:00:16.296 ******* 2020-11-20T12:09:43.716 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.848 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.142) 0:00:16.438 ******* 2020-11-20T12:09:43.854 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2020-11-20T12:09:43.856 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.906 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.058) 0:00:16.497 ******* 2020-11-20T12:09:43.913 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:43.962 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:43 +0000 (0:00:00.055) 0:00:16.552 ******* 2020-11-20T12:09:44.476 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-20T12:09:44.477 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.327289", "end": "2020-11-20 12:09:44.501007", "rc": 0, "start": "2020-11-20 12:09:44.173718", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:09:44.522 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:44 +0000 (0:00:00.559) 0:00:17.112 ******* 2020-11-20T12:09:44.551 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2020-11-20T12:09:44.553 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:44.605 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:44 +0000 (0:00:00.082) 0:00:17.195 ******* 2020-11-20T12:09:44.706 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2020-11-20T12:09:44.708 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.622886", "end": "2020-11-20 12:09:44.500995", "rc": 0, "start": "2020-11-20 12:09:43.878109", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:09:44.760 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:44 +0000 (0:00:00.155) 0:00:17.350 ******* 2020-11-20T12:09:44.788 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:44.831 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:44 +0000 (0:00:00.071) 0:00:17.421 ******* 2020-11-20T12:09:44.891 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2020-11-20T12:09:44.893 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the apt module rather than running 'apt-get'. If you need to use command because apt is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.107947", "end": "2020-11-20 12:09:44.914144", "rc": 0, "start": "2020-11-20 12:09:44.806197", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:09:44.946 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:44 +0000 (0:00:00.114) 0:00:17.536 ******* 2020-11-20T12:09:45.119 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["apt-get", "clean"], "delta": "0:00:00.109466", "end": "2020-11-20 12:09:44.910150", "rc": 0, "start": "2020-11-20 12:09:44.800684", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:09:45.161 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:45 +0000 (0:00:00.215) 0:00:17.751 ******* 2020-11-20T12:09:53.094 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2020-11-20T12:09:53.096 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1605874190, "cache_updated": true, "changed": true} 2020-11-20T12:09:53.161 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:53 +0000 (0:00:07.999) 0:00:25.751 ******* 2020-11-20T12:09:53.689 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"attempts": 1, "cache_update_time": 1605874191, "cache_updated": true, "changed": true} 2020-11-20T12:09:53.733 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:53 +0000 (0:00:00.572) 0:00:26.323 ******* 2020-11-20T12:09:54.180 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-20T12:09:54.182 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": false} 2020-11-20T12:09:54.238 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:54 +0000 (0:00:00.505) 0:00:26.828 ******* 2020-11-20T12:09:54.266 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:54.309 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:54 +0000 (0:00:00.070) 0:00:26.899 ******* 2020-11-20T12:09:54.758 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-20T12:09:54.760 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-20T12:09:54.801 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:54 +0000 (0:00:00.491) 0:00:27.390 ******* 2020-11-20T12:09:54.817 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2020-11-20T12:09:54.818 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": false} 2020-11-20T12:09:54.860 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:54 +0000 (0:00:00.059) 0:00:27.450 ******* 2020-11-20T12:09:54.886 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2020-11-20T12:09:54.888 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:54.933 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:54 +0000 (0:00:00.073) 0:00:27.524 ******* 2020-11-20T12:09:55.383 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2020-11-20T12:09:55.385 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "size": 131, "state": "file", "uid": 0} 2020-11-20T12:09:55.426 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:55 +0000 (0:00:00.493) 0:00:28.017 ******* 2020-11-20T12:09:55.836 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-20T12:09:55.838 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": false} 2020-11-20T12:09:55.894 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:55 +0000 (0:00:00.467) 0:00:28.485 ******* 2020-11-20T12:09:56.386 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-20T12:09:56.388 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-20T12:09:56.431 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2020-11-20T12:09:56.432 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": false} 2020-11-20T12:09:56.488 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:56 +0000 (0:00:00.593) 0:00:29.078 ******* 2020-11-20T12:09:56.834 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-20T12:09:56.836 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-20T12:09:56.975 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "size": 2716736, "state": "file", "uid": 0} 2020-11-20T12:09:57.254 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-20T12:09:57.299 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:57 +0000 (0:00:00.811) 0:00:29.890 ******* 2020-11-20T12:09:57.422 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "size": 1952038, "state": "file", "uid": 0} 2020-11-20T12:09:57.466 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T12:09:57.511 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:57 +0000 (0:00:00.211) 0:00:30.101 ******* 2020-11-20T12:09:57.849 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2020-11-20T12:09:57.851 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "size": 580432, "state": "file", "uid": 0} 2020-11-20T12:09:57.910 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:57 +0000 (0:00:00.398) 0:00:30.500 ******* 2020-11-20T12:09:58.034 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-20T12:09:58.036 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-11-20T12:09:58.090 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2020-11-20T12:09:58.091 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T12:09:58.137 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:58 +0000 (0:00:00.227) 0:00:30.727 ******* 2020-11-20T12:09:58.445 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi173.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-20T12:09:58.607 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "size": 12725, "state": "file", "uid": 0} 2020-11-20T12:09:58.886 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-20T12:09:58.936 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:58 +0000 (0:00:00.798) 0:00:31.525 ******* 2020-11-20T12:09:58.975 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-20T12:09:59.022 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:59 +0000 (0:00:00.086) 0:00:31.612 ******* 2020-11-20T12:09:59.052 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-20T12:09:59.054 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** ok: [smithi116.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "size": 8602, "state": "file", "uid": 0} 2020-11-20T12:09:59.096 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:59 +0000 (0:00:00.074) 0:00:31.686 ******* 2020-11-20T12:09:59.124 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:59.169 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:59 +0000 (0:00:00.072) 0:00:31.759 ******* 2020-11-20T12:09:59.216 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-20T12:09:59.218 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:59.228 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:59.239 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:59.250 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:59.291 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:59 +0000 (0:00:00.122) 0:00:31.882 ******* 2020-11-20T12:09:59.318 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-20T12:09:59.320 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:59.360 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:59 +0000 (0:00:00.068) 0:00:31.950 ******* 2020-11-20T12:09:59.452 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2020-11-20T12:09:59.454 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "size": 3605, "state": "file", "uid": 0} 2020-11-20T12:09:59.497 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:59 +0000 (0:00:00.136) 0:00:32.087 ******* 2020-11-20T12:09:59.528 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2020-11-20T12:09:59.530 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib/nagios/plugins", "nrpe_group": "nagios", "nrpe_packages": ["nagios-nrpe-server", "nagios-plugins-basic"], "nrpe_service_name": "nagios-nrpe-server", "nrpe_user": "nagios"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/apt_systems.yml"], "changed": false} 2020-11-20T12:09:59.570 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:59 +0000 (0:00:00.072) 0:00:32.160 ******* 2020-11-20T12:09:59.596 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2020-11-20T12:09:59.599 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:59.638 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:59 +0000 (0:00:00.068) 0:00:32.228 ******* 2020-11-20T12:09:59.665 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2020-11-20T12:09:59.666 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:09:59.708 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:59 +0000 (0:00:00.070) 0:00:32.298 ******* 2020-11-20T12:09:59.746 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2020-11-20T12:09:59.748 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:59.758 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:59.769 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2020-11-20T12:09:59.811 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:59 +0000 (0:00:00.102) 0:00:32.401 ******* 2020-11-20T12:09:59.838 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2020-11-20T12:09:59.840 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:09:59.879 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:09:59 +0000 (0:00:00.067) 0:00:32.469 ******* 2020-11-20T12:10:00.421 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-20T12:10:00.424 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": false} 2020-11-20T12:10:00.467 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:00 +0000 (0:00:00.588) 0:00:33.057 ******* 2020-11-20T12:10:00.938 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-20T12:10:00.982 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:00 +0000 (0:00:00.514) 0:00:33.572 ******* 2020-11-20T12:10:00.990 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu/Fedora)] ********** 2020-11-20T12:10:00.994 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": false} 2020-11-20T12:10:01.035 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:01 +0000 (0:00:00.053) 0:00:33.626 ******* 2020-11-20T12:10:01.458 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-20T12:10:01.460 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-20T12:10:01.503 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:01 +0000 (0:00:00.467) 0:00:34.093 ******* 2020-11-20T12:10:01.510 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2020-11-20T12:10:01.513 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "4880e187ff9f437cc33858eb1d0fa49a06d67be4", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "size": 180, "state": "file", "uid": 0} 2020-11-20T12:10:01.554 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:01 +0000 (0:00:00.050) 0:00:34.144 ******* 2020-11-20T12:10:01.693 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-20T12:10:01.694 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-20T12:10:01.850 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-20T12:10:01.898 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:01 +0000 (0:00:00.343) 0:00:34.488 ******* 2020-11-20T12:10:01.927 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2020-11-20T12:10:01.929 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:01.971 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:01 +0000 (0:00:00.073) 0:00:34.561 ******* 2020-11-20T12:10:02.015 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-20T12:10:02.017 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:10:02.059 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:02 +0000 (0:00:00.088) 0:00:34.649 ******* 2020-11-20T12:10:02.065 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2020-11-20T12:10:02.067 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/nagios/plugins/check_mem.sh", "size": 1964, "state": "file", "uid": 0} 2020-11-20T12:10:02.114 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:02 +0000 (0:00:00.054) 0:00:34.704 ******* 2020-11-20T12:10:02.120 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2020-11-20T12:10:02.124 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:02.165 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:02 +0000 (0:00:00.051) 0:00:34.755 ******* 2020-11-20T12:10:02.312 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2020-11-20T12:10:02.314 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "DAEMON_OPTS", "msg": ""} 2020-11-20T12:10:02.466 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "backup": "", "changed": false, "item": "NRPE_OPTS", "msg": ""} 2020-11-20T12:10:02.514 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:02 +0000 (0:00:00.348) 0:00:35.104 ******* 2020-11-20T12:10:02.546 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:02.593 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:02 +0000 (0:00:00.079) 0:00:35.183 ******* 2020-11-20T12:10:02.624 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2020-11-20T12:10:02.627 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:10:02.695 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:02 +0000 (0:00:00.101) 0:00:35.285 ******* 2020-11-20T12:10:02.702 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2020-11-20T12:10:02.710 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-20T12:10:02.749 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:02 +0000 (0:00:00.054) 0:00:35.340 ******* 2020-11-20T12:10:02.765 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:02.825 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:02 +0000 (0:00:00.075) 0:00:35.416 ******* 2020-11-20T12:10:03.534 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "a0ee4c31e844cf26108115f6af99f92f904eafbf", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "size": 1176, "state": "file", "uid": 0} 2020-11-20T12:10:03.572 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:03 +0000 (0:00:00.747) 0:00:36.163 ******* 2020-11-20T12:10:03.601 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** [WARNING]: The service (nagios-nrpe-server) is actually an init script but the system is managed by systemd fatal: [smithi173.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring 2020-11-20T12:10:03.649 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:03 +0000 (0:00:00.076) 0:00:36.239 ******* 2020-11-20T12:10:03.688 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-20T12:10:03.691 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:03.731 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:03 +0000 (0:00:00.082) 0:00:36.321 ******* 2020-11-20T12:10:03.760 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:03.802 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:03 +0000 (0:00:00.070) 0:00:36.392 ******* 2020-11-20T12:10:03.828 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-20T12:10:03.831 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:03.870 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:03 +0000 (0:00:00.068) 0:00:36.461 ******* 2020-11-20T12:10:03.897 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2020-11-20T12:10:03.899 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:03.940 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:03 +0000 (0:00:00.069) 0:00:36.530 ******* 2020-11-20T12:10:03.951 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** fatal: [smithi116.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring 2020-11-20T12:10:03.989 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:03 +0000 (0:00:00.049) 0:00:36.579 ******* 2020-11-20T12:10:03.996 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-20T12:10:03.997 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.042 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.053) 0:00:36.632 ******* 2020-11-20T12:10:04.048 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2020-11-20T12:10:04.050 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.095 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.053) 0:00:36.686 ******* 2020-11-20T12:10:04.103 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.157 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.061) 0:00:36.747 ******* 2020-11-20T12:10:04.162 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2020-11-20T12:10:04.164 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.207 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.050) 0:00:36.797 ******* 2020-11-20T12:10:04.213 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-20T12:10:04.215 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.271 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.063) 0:00:36.861 ******* 2020-11-20T12:10:04.277 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2020-11-20T12:10:04.279 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.321 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.050) 0:00:36.912 ******* 2020-11-20T12:10:04.328 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-20T12:10:04.330 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.384 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.052) 0:00:36.964 ******* TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.430 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.054) 0:00:37.019 ******* 2020-11-20T12:10:04.436 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.493 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.063) 0:00:37.083 ******* 2020-11-20T12:10:04.499 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2020-11-20T12:10:04.501 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.544 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.050) 0:00:37.134 ******* 2020-11-20T12:10:04.550 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2020-11-20T12:10:04.552 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.597 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.053) 0:00:37.187 ******* 2020-11-20T12:10:04.603 INFO:teuthology.task.ansible.out: 2020-11-20T12:10:04.604 INFO:teuthology.task.ansible.out:TASK [common : nrpe - Copy SELinux type enforcement file] ********************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.651 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.054) 0:00:37.242 ******* 2020-11-20T12:10:04.678 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2020-11-20T12:10:04.680 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.723 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.071) 0:00:37.313 ******* 2020-11-20T12:10:04.750 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2020-11-20T12:10:04.752 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.795 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.071) 0:00:37.385 ******* 2020-11-20T12:10:04.812 INFO:teuthology.task.ansible.out: TASK [common : Check for podman] *********************************************** 2020-11-20T12:10:04.815 INFO:teuthology.task.ansible.out:fatal: [smithi173.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": "podman --version", "msg": "[Errno 2] No such file or directory", "rc": 2} ...ignoring 2020-11-20T12:10:04.857 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.062) 0:00:37.447 ******* 2020-11-20T12:10:04.863 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.907 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.050) 0:00:37.497 ******* 2020-11-20T12:10:04.913 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T12:10:04.915 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:04.960 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:04 +0000 (0:00:00.051) 0:00:37.549 ******* 2020-11-20T12:10:04.968 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:05.012 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:05 +0000 (0:00:00.053) 0:00:37.602 ******* 2020-11-20T12:10:05.226 INFO:teuthology.task.ansible.out: TASK [common : Check for podman] *********************************************** fatal: [smithi116.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": "podman --version", "msg": "[Errno 2] No such file or directory", "rc": 2} ...ignoring 2020-11-20T12:10:05.269 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:05 +0000 (0:00:00.256) 0:00:37.858 ******* 2020-11-20T12:10:05.300 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:05.339 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:05 +0000 (0:00:00.071) 0:00:37.930 ******* 2020-11-20T12:10:09.545 INFO:teuthology.task.ansible.out: TASK [common : Check for docker] *********************************************** 2020-11-20T12:10:09.547 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["docker", "--version"], "delta": "0:00:04.384834", "end": "2020-11-20 12:10:09.549334", "rc": 0, "start": "2020-11-20 12:10:05.164500", "stderr": "", "stderr_lines": [], "stdout": "Docker version 19.03.6, build 369ce74a3c", "stdout_lines": ["Docker version 19.03.6, build 369ce74a3c"]} 2020-11-20T12:10:09.596 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:09 +0000 (0:00:04.256) 0:00:42.186 ******* 2020-11-20T12:10:09.637 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-20T12:10:09.679 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:09 +0000 (0:00:00.082) 0:00:42.269 ******* 2020-11-20T12:10:09.727 INFO:teuthology.task.ansible.out: TASK [common : Include encrypted variables] ************************************ 2020-11-20T12:10:09.733 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:10:09.738 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:10:09.781 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:09 +0000 (0:00:00.102) 0:00:42.372 ******* 2020-11-20T12:10:09.925 INFO:teuthology.task.ansible.out: TASK [common : Check for docker] *********************************************** 2020-11-20T12:10:09.927 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["docker", "--version"], "delta": "0:00:04.394881", "end": "2020-11-20 12:10:09.733706", "rc": 0, "start": "2020-11-20 12:10:05.338825", "stderr": "", "stderr_lines": [], "stdout": "Docker version 19.03.6, build 369ce74a3c", "stdout_lines": ["Docker version 19.03.6, build 369ce74a3c"]} 2020-11-20T12:10:09.971 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:09 +0000 (0:00:00.189) 0:00:42.561 ******* 2020-11-20T12:10:09.977 INFO:teuthology.task.ansible.out: TASK [common : Create /etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** 2020-11-20T12:10:09.980 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T12:10:10.029 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:10 +0000 (0:00:00.058) 0:00:42.619 ******* 2020-11-20T12:10:10.036 INFO:teuthology.task.ansible.out: TASK [common : set_fact] ******************************************************* 2020-11-20T12:10:10.037 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"container_mirror_cert_path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000"}, "changed": false} 2020-11-20T12:10:10.084 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:10 +0000 (0:00:00.054) 0:00:42.674 ******* 2020-11-20T12:10:10.133 INFO:teuthology.task.ansible.out: TASK [common : Include encrypted variables] ************************************ 2020-11-20T12:10:10.135 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} ok: [smithi116.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2020-11-20T12:10:10.175 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:10 +0000 (0:00:00.090) 0:00:42.765 ******* 2020-11-20T12:10:10.356 INFO:teuthology.task.ansible.out: TASK [common : Create /etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000] *** changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T12:10:10.400 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:10 +0000 (0:00:00.224) 0:00:42.990 ******* 2020-11-20T12:10:10.513 INFO:teuthology.task.ansible.out: TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-11-20T12:10:10.515 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605874210.0573041-150742093146543/source", "state": "file", "uid": 0} 2020-11-20T12:10:10.561 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:10 +0000 (0:00:00.161) 0:00:43.152 ******* 2020-11-20T12:10:10.589 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-20T12:10:10.632 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:10 +0000 (0:00:00.070) 0:00:43.222 ******* 2020-11-20T12:10:10.669 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-20T12:10:10.671 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-20T12:10:10.714 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:10 +0000 (0:00:00.080) 0:00:43.303 ******* 2020-11-20T12:10:10.751 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-20T12:10:10.753 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-11-20T12:10:10.798 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:10 +0000 (0:00:00.085) 0:00:43.388 ******* 2020-11-20T12:10:10.836 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [smithi173.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T12:10:10.878 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:10 +0000 (0:00:00.079) 0:00:43.468 ******* 2020-11-20T12:10:10.904 INFO:teuthology.task.ansible.out: TASK [common : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2020-11-20T12:10:10.906 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "checksum": "d386a5d594d8bb6c408cf08541a4eb609577a03d", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "md5sum": "e2cf8b6a8ad65ba91ea9fc53d49f1bf4", "mode": "0644", "owner": "root", "size": 2236, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1605874210.433204-277644415195068/source", "state": "file", "uid": 0} 2020-11-20T12:10:10.949 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:10 +0000 (0:00:00.071) 0:00:43.539 ******* 2020-11-20T12:10:10.975 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2020-11-20T12:10:10.978 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"apt_repos": [], "ceph_packages_to_remove": ["ceph", "ceph-common", "libcephfs1", "radosgw", "python-ceph", "python-rados", "python-cephfs", "python-rbd", "librbd1", "librados2", "ceph-fs-common-dbg", "ceph-fs-common"], "common_apt_repos": [], "common_packages": [], "nfs_service": "nfs-kernel-server", "ntp_service_name": "ntp", "packages": [], "packages_to_remove": ["multipath-tools"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "ssh"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/apt_systems.yml"], "changed": false} 2020-11-20T12:10:11.018 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.069) 0:00:43.608 ******* 2020-11-20T12:10:11.055 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2020-11-20T12:10:11.056 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml) => {"ansible_facts": {"common_packages": ["libfcgi0ldbl", "python-dev", "libev-dev", "perl", "libwww-perl", "lsb-release", "build-essential", "sysstat", "gdb", "libedit2", "cryptsetup-bin", "xfsprogs", "gdisk", "parted", "libuuid1", "libatomic-ops-dev", "git-core", "attr", "dbench", "bonnie++", "valgrind", "ant", "libtool", "automake", "gettext", "uuid-dev", "libacl1-dev", "bc", "xfsdump", "xfslibs-dev", "libattr1-dev", "quota", "libcap2-bin", "libncurses5-dev", "lvm2", "vim", "pdsh", "collectl", "blktrace", "genisoimage", "libjson-xs-perl", "xml-twig-tools", "default-jdk", "junit4", "tgt", "open-iscsi", "cifs-utils", "ipcalc", "nfs-common", "nfs-kernel-server", "software-properties-common"], "non_aarch64_common_packages": ["smbios-utils", "libfcgi", "sysprof"], "non_aarch64_packages_to_upgrade": ["libapache2-mod-fastcgi"], "packages_to_remove": ["openmpi-common"], "packages_to_upgrade": ["apt", "apache2"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu.yml"} 2020-11-20T12:10:11.059 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-20T12:10:11.100 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.081) 0:00:43.690 ******* 2020-11-20T12:10:11.149 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.049) 0:00:43.739 ******* 2020-11-20T12:10:11.156 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2020-11-20T12:10:11.157 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml) => {"ansible_facts": {"non_aarch64_packages": ["libgoogle-perftools4", "iozone3", "libssl1.0.0"], "non_aarch64_packages_to_upgrade": [], "packages": ["mpich", "qemu-system-x86", "python-virtualenv", "python-configobj", "python-gevent", "python-numpy", "python-matplotlib", "python-nose", "btrfs-tools", "lttng-tools", "libtool-bin", "docker.io", "python3-nose"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/ubuntu_18.yml"} 2020-11-20T12:10:11.211 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.061) 0:00:43.801 ******* 2020-11-20T12:10:11.249 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2020-11-20T12:10:11.251 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2020-11-20T12:10:11.294 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.084) 0:00:43.885 ******* 2020-11-20T12:10:11.343 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-20T12:10:11.345 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-20T12:10:11.391 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.096) 0:00:43.981 ******* 2020-11-20T12:10:11.431 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-20T12:10:11.433 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:11.474 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.082) 0:00:44.064 ******* 2020-11-20T12:10:11.491 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2020-11-20T12:10:11.492 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 27, "name": "sudo", "state": "present", "system": false} 2020-11-20T12:10:11.533 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.059) 0:00:44.123 ******* 2020-11-20T12:10:11.539 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-20T12:10:11.541 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:11.587 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.053) 0:00:44.177 ******* 2020-11-20T12:10:11.728 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2020-11-20T12:10:11.730 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "name": "ubuntu", "state": "present", "system": false} 2020-11-20T12:10:11.772 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.185) 0:00:44.363 ******* 2020-11-20T12:10:11.800 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2020-11-20T12:10:11.802 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:11.841 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.068) 0:00:44.431 ******* 2020-11-20T12:10:11.848 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2020-11-20T12:10:11.850 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-11-20T12:10:11.890 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.048) 0:00:44.480 ******* 2020-11-20T12:10:11.895 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2020-11-20T12:10:11.897 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:11.945 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:11 +0000 (0:00:00.055) 0:00:44.535 ******* 2020-11-20T12:10:12.166 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi116.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/sh", "state": "present", "uid": 10101} 2020-11-20T12:10:12.208 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:12 +0000 (0:00:00.263) 0:00:44.799 ******* 2020-11-20T12:10:12.437 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-20T12:10:12.439 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-20T12:10:12.480 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:12 +0000 (0:00:00.271) 0:00:45.070 ******* 2020-11-20T12:10:12.517 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-20T12:10:12.519 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi173"}, "changed": false} 2020-11-20T12:10:12.559 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:12 +0000 (0:00:00.078) 0:00:45.149 ******* 2020-11-20T12:10:12.779 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2020-11-20T12:10:12.780 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "size": 112, "state": "file", "uid": 0} 2020-11-20T12:10:12.823 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:12 +0000 (0:00:00.263) 0:00:45.413 ******* 2020-11-20T12:10:12.854 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2020-11-20T12:10:12.855 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi116"}, "changed": false} 2020-11-20T12:10:12.897 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:12 +0000 (0:00:00.073) 0:00:45.487 ******* 2020-11-20T12:10:13.433 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2020-11-20T12:10:13.435 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi173.front.sepia.ceph.com", "ansible_hostname": "smithi173", "ansible_nodename": "smithi173"}, "changed": true, "name": "smithi173"} 2020-11-20T12:10:13.478 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:13 +0000 (0:00:00.580) 0:00:46.067 ******* 2020-11-20T12:10:13.485 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi116.front.sepia.ceph.com", "ansible_hostname": "smithi116", "ansible_nodename": "smithi116"}, "changed": true, "name": "smithi116"} 2020-11-20T12:10:13.533 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:13 +0000 (0:00:00.055) 0:00:46.123 ******* 2020-11-20T12:10:13.952 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} 2020-11-20T12:10:14.586 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:13 +0000 (0:00:00.461) 0:00:46.585 ******* ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "2f7bfea2455a0d9220592aa0c21ac69c312e7471", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "size": 2646, "state": "file", "uid": 0} Friday 20 November 2020 12:10:14 +0000 (0:00:00.064) 0:00:46.650 ******* TASK [testnode : Upload ssh_config] ******************************************** ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} Friday 20 November 2020 12:10:14 +0000 (0:00:00.481) 0:00:47.131 ******* 2020-11-20T12:10:14.590 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "size": 570, "state": "file", "uid": 0} 2020-11-20T12:10:14.636 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:14 +0000 (0:00:00.094) 0:00:47.226 ******* 2020-11-20T12:10:15.132 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** ok: [smithi173.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-20T12:10:15.174 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.538) 0:00:47.765 ******* 2020-11-20T12:10:15.202 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-20T12:10:15.204 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.248 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.073) 0:00:47.838 ******* 2020-11-20T12:10:15.286 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-20T12:10:15.288 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.350 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.102) 0:00:47.940 ******* 2020-11-20T12:10:15.361 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2020-11-20T12:10:15.363 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"attempts": 1, "changed": false, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2020-11-20T12:10:15.404 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.053) 0:00:47.994 ******* 2020-11-20T12:10:15.410 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-20T12:10:15.412 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.452 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.047) 0:00:48.042 ******* 2020-11-20T12:10:15.458 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2020-11-20T12:10:15.460 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.504 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.051) 0:00:48.094 ******* 2020-11-20T12:10:15.510 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T12:10:15.512 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.562 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.058) 0:00:48.153 ******* 2020-11-20T12:10:15.573 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.627 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.064) 0:00:48.218 ******* 2020-11-20T12:10:15.634 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-20T12:10:15.637 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.677 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.049) 0:00:48.267 ******* 2020-11-20T12:10:15.685 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.725 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.048) 0:00:48.316 ******* 2020-11-20T12:10:15.733 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.775 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.049) 0:00:48.365 ******* 2020-11-20T12:10:15.780 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T12:10:15.782 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.829 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.054) 0:00:48.419 ******* 2020-11-20T12:10:15.834 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-20T12:10:15.836 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.878 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.049) 0:00:48.469 ******* 2020-11-20T12:10:15.884 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2020-11-20T12:10:15.885 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.932 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.053) 0:00:48.522 ******* 2020-11-20T12:10:15.938 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-20T12:10:15.940 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:15.982 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:15 +0000 (0:00:00.049) 0:00:48.572 ******* 2020-11-20T12:10:15.987 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T12:10:15.989 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.032 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.048) 0:00:48.620 ******* 2020-11-20T12:10:16.037 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-20T12:10:16.079 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.049) 0:00:48.670 ******* 2020-11-20T12:10:16.086 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-20T12:10:16.087 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.128 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.049) 0:00:48.719 ******* 2020-11-20T12:10:16.135 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-20T12:10:16.177 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.049) 0:00:48.768 ******* 2020-11-20T12:10:16.183 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2020-11-20T12:10:16.185 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.231 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.053) 0:00:48.821 ******* 2020-11-20T12:10:16.239 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-20T12:10:16.280 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.049) 0:00:48.871 ******* 2020-11-20T12:10:16.287 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2020-11-20T12:10:16.334 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.053) 0:00:48.924 ******* 2020-11-20T12:10:16.340 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-20T12:10:16.342 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.384 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.050) 0:00:48.975 ******* 2020-11-20T12:10:16.391 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2020-11-20T12:10:16.439 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.054) 0:00:49.029 ******* 2020-11-20T12:10:16.445 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-20T12:10:16.447 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.496 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.057) 0:00:49.086 ******* 2020-11-20T12:10:16.503 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2020-11-20T12:10:16.552 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.055) 0:00:49.142 ******* 2020-11-20T12:10:16.558 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-20T12:10:16.560 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.601 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.049) 0:00:49.191 ******* 2020-11-20T12:10:16.607 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2020-11-20T12:10:16.609 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.651 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.049) 0:00:49.241 ******* 2020-11-20T12:10:16.657 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:10:16.661 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.702 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.051) 0:00:49.292 ******* 2020-11-20T12:10:16.708 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2020-11-20T12:10:16.710 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.753 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.051) 0:00:49.343 ******* 2020-11-20T12:10:16.760 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.817 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.063) 0:00:49.407 ******* 2020-11-20T12:10:16.823 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2020-11-20T12:10:16.825 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.875 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.057) 0:00:49.464 ******* 2020-11-20T12:10:16.881 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-20T12:10:16.883 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.941 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.065) 0:00:49.530 ******* 2020-11-20T12:10:16.947 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:16.993 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:16 +0000 (0:00:00.053) 0:00:49.583 ******* 2020-11-20T12:10:17.000 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-20T12:10:17.002 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.051 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.051) 0:00:49.635 ******* TASK [testnode : Ensure yum_utils is present.] ********************************* 2020-11-20T12:10:17.057 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.095 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.050) 0:00:49.686 ******* 2020-11-20T12:10:17.102 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-20T12:10:17.104 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.147 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.051) 0:00:49.738 ******* 2020-11-20T12:10:17.154 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2020-11-20T12:10:17.156 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.199 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.051) 0:00:49.789 ******* 2020-11-20T12:10:17.206 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-20T12:10:17.208 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.251 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.052) 0:00:49.841 ******* 2020-11-20T12:10:17.257 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2020-11-20T12:10:17.259 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.300 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.049) 0:00:49.890 ******* 2020-11-20T12:10:17.306 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-20T12:10:17.308 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.353 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.052) 0:00:49.943 ******* 2020-11-20T12:10:17.359 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2020-11-20T12:10:17.361 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.402 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.049) 0:00:49.993 ******* 2020-11-20T12:10:17.409 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T12:10:17.411 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.452 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.049) 0:00:50.042 ******* 2020-11-20T12:10:17.458 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2020-11-20T12:10:17.459 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.500 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.048) 0:00:50.091 ******* 2020-11-20T12:10:17.507 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.549 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.048) 0:00:50.140 ******* 2020-11-20T12:10:17.556 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-20T12:10:17.557 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.599 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.049) 0:00:50.189 ******* 2020-11-20T12:10:17.605 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T12:10:17.607 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.651 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.049) 0:00:50.238 ******* 2020-11-20T12:10:17.654 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T12:10:17.659 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.700 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.051) 0:00:50.290 ******* 2020-11-20T12:10:17.707 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T12:10:17.708 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.752 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.052) 0:00:50.342 ******* 2020-11-20T12:10:17.758 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2020-11-20T12:10:17.760 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.804 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.051) 0:00:50.394 ******* 2020-11-20T12:10:17.811 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-20T12:10:17.812 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.863 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.058) 0:00:50.453 ******* 2020-11-20T12:10:17.869 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T12:10:17.871 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.919 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.056) 0:00:50.509 ******* 2020-11-20T12:10:17.927 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-20T12:10:17.928 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:17.972 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:17 +0000 (0:00:00.051) 0:00:50.561 ******* 2020-11-20T12:10:17.978 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T12:10:17.980 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.023 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.052) 0:00:50.613 ******* 2020-11-20T12:10:18.031 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.073 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.050) 0:00:50.663 ******* 2020-11-20T12:10:18.080 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2020-11-20T12:10:18.081 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.136 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.062) 0:00:50.726 ******* 2020-11-20T12:10:18.145 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.199 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.063) 0:00:50.790 ******* 2020-11-20T12:10:18.207 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2020-11-20T12:10:18.208 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.252 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.052) 0:00:50.843 ******* 2020-11-20T12:10:18.259 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.301 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.048) 0:00:50.891 ******* 2020-11-20T12:10:18.307 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2020-11-20T12:10:18.310 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.359 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.057) 0:00:50.949 ******* 2020-11-20T12:10:18.365 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2020-11-20T12:10:18.367 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.417 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.058) 0:00:51.007 ******* 2020-11-20T12:10:18.423 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2020-11-20T12:10:18.425 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.466 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.048) 0:00:51.056 ******* 2020-11-20T12:10:18.472 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-20T12:10:18.474 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.514 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.047) 0:00:51.104 ******* 2020-11-20T12:10:18.520 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2020-11-20T12:10:18.521 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.562 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.048) 0:00:51.153 ******* 2020-11-20T12:10:18.569 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2020-11-20T12:10:18.571 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.613 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.050) 0:00:51.203 ******* 2020-11-20T12:10:18.620 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.663 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.049) 0:00:51.253 ******* 2020-11-20T12:10:18.690 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2020-11-20T12:10:18.691 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.732 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.068) 0:00:51.322 ******* 2020-11-20T12:10:18.765 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:18.801 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.068) 0:00:51.391 ******* 2020-11-20T12:10:18.807 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2020-11-20T12:10:18.809 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.014471", "end": "2020-11-20 12:10:18.811324", "rc": 0, "start": "2020-11-20 12:10:18.796853", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:10:18.854 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.052) 0:00:51.444 ******* 2020-11-20T12:10:18.892 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-20T12:10:18.940 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:18 +0000 (0:00:00.086) 0:00:51.531 ******* 2020-11-20T12:10:19.048 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "cmd": "ls -1 /etc/apt/sources.list.d/", "delta": "0:00:00.013368", "end": "2020-11-20 12:10:18.858328", "rc": 0, "start": "2020-11-20 12:10:18.844960", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:10:19.093 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:19 +0000 (0:00:00.150) 0:00:51.681 ******* 2020-11-20T12:10:19.120 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2020-11-20T12:10:19.162 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:19 +0000 (0:00:00.070) 0:00:51.752 ******* 2020-11-20T12:10:19.470 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-20T12:10:19.516 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:19 +0000 (0:00:00.351) 0:00:52.104 ******* 2020-11-20T12:10:19.546 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-20T12:10:19.548 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:19.593 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:19 +0000 (0:00:00.075) 0:00:52.180 ******* 2020-11-20T12:10:19.692 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "ba101482124806c7f6ea9fc7b4acfda303391ad6", "dest": "/etc/apt/preferences.d/ceph.pref", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/apt/preferences.d/ceph.pref", "size": 52, "state": "file", "uid": 0} 2020-11-20T12:10:19.743 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:19 +0000 (0:00:00.153) 0:00:52.333 ******* 2020-11-20T12:10:19.770 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2020-11-20T12:10:19.773 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:19.813 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:19 +0000 (0:00:00.069) 0:00:52.403 ******* 2020-11-20T12:10:21.834 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* ok: [smithi116.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-20T12:10:21.879 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc"} 2020-11-20T12:10:23.706 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-20T12:10:23.755 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:23 +0000 (0:00:03.940) 0:00:56.343 ******* 2020-11-20T12:10:23.784 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc"} 2020-11-20T12:10:23.828 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:23 +0000 (0:00:00.074) 0:00:56.418 ******* 2020-11-20T12:10:24.805 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-20T12:10:24.807 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": false} 2020-11-20T12:10:24.850 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:24 +0000 (0:00:01.022) 0:00:57.440 ******* 2020-11-20T12:10:24.879 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-20T12:10:24.927 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:24 +0000 (0:00:00.077) 0:00:57.517 ******* 2020-11-20T12:10:24.933 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2020-11-20T12:10:24.934 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": false} 2020-11-20T12:10:24.983 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:24 +0000 (0:00:00.055) 0:00:57.573 ******* 2020-11-20T12:10:25.011 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2020-11-20T12:10:25.056 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:25 +0000 (0:00:00.071) 0:00:57.645 ******* 2020-11-20T12:10:25.170 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** [WARNING]: Consider using 'become', 'become_method', and 'become_user' rather than running sudo changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.083719", "end": "2020-11-20 12:10:24.995674", "rc": 0, "start": "2020-11-20 12:10:24.911955", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:10:25.215 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:25 +0000 (0:00:00.160) 0:00:57.805 ******* 2020-11-20T12:10:25.340 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": ["sudo", "apt-get", "clean"], "delta": "0:00:00.085089", "end": "2020-11-20 12:10:25.338227", "rc": 0, "start": "2020-11-20 12:10:25.253138", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:10:25.386 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:25 +0000 (0:00:00.170) 0:00:57.976 ******* 2020-11-20T12:10:29.257 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2020-11-20T12:10:29.258 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": false} 2020-11-20T12:10:29.299 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:29 +0000 (0:00:03.912) 0:01:01.889 ******* 2020-11-20T12:10:29.437 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": false} 2020-11-20T12:10:29.849 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:29 +0000 (0:00:00.176) 0:01:02.066 ******* 2020-11-20T12:10:41.191 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2020-11-20T12:10:41.192 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 23 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 104569 files and directorie 2020-11-20T12:10:41.194 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 23 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Read 2020-11-20T12:10:41.195 INFO:teuthology.task.ansible.out:ing database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 104569 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-20T12:10:41.235 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:41 +0000 (0:00:11.759) 0:01:13.826 ******* 2020-11-20T12:10:41.462 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following packages were automatically installed and are no longer required:\n cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good\n gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0\n libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1\n libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0\n libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0\n libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3\n libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0\n libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5\n libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils\nUse 'sudo apt autoremove' to remove them.\nThe following packages will be REMOVED:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 0 newly installed, 6 to remove and 23 not upgraded.\nAfter this operation, 47.6 MB disk space will be freed.\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 104569 files and directorie 2020-11-20T12:10:41.464 INFO:teuthology.task.ansible.out:s currently installed.)\r\nRemoving qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nRemoving librbd1 (12.2.13-0ubuntu0.18.04.4) ...\r\nRemoving librados2 (12.2.13-0ubuntu0.18.04.4) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubuntu1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "The following packages were automatically installed and are no longer required:", " cpu-checker gstreamer1.0-plugins-base gstreamer1.0-plugins-good", " gstreamer1.0-x ipxe-qemu ipxe-qemu-256k-compat-efi-roms libaa1 libavc1394-0", " libbluetooth3 libbrlapi0.6 libcacard0 libcdparanoia0 libdv4 libfdt1", " libgstreamer-plugins-base1.0-0 libgstreamer-plugins-good1.0-0", " libgstreamer1.0-0 libiec61883-0 libiscsi7 libjack-jackd2-0 libmp3lame0", " libmpg123-0 libopus0 liborc-0.4-0 libraw1394-11 libsamplerate0 libshout3", " libspeex1 libspice-server1 libtag1v5 libtag1v5-vanilla libtwolame0", " libusbredirparser1 libv4l-0 libv4lconvert0 libvisual-0.4-0 libvpx5", " libwavpack1 libxen-4.9 libxenstore3.0 libyajl2 msr-tools seabios sharutils", "Use 'sudo apt autoremove' to remove them.", "The following packages will be REMOVED:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 0 newly installed, 6 to remove and 23 not upgraded.", "After this operation, 47.6 MB disk space will be freed.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Read 2020-11-20T12:10:41.466 INFO:teuthology.task.ansible.out:ing database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(Reading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 104569 files and directories currently installed.)", "Removing qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Removing qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Removing librbd1 (12.2.13-0ubuntu0.18.04.4) ...", "Removing librados2 (12.2.13-0ubuntu0.18.04.4) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-20T12:10:41.508 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:41 +0000 (0:00:00.272) 0:01:14.098 ******* 2020-11-20T12:10:42.332 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T12:10:42.334 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": false} 2020-11-20T12:10:42.386 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:42 +0000 (0:00:00.874) 0:01:14.973 ******* 2020-11-20T12:10:42.411 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2020-11-20T12:10:42.414 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:42.458 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:42 +0000 (0:00:00.074) 0:01:15.047 ******* 2020-11-20T12:10:42.551 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* ok: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": false} 2020-11-20T12:10:42.595 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:42 +0000 (0:00:00.136) 0:01:15.184 ******* 2020-11-20T12:10:42.632 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:42.673 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:42 +0000 (0:00:00.079) 0:01:15.264 ******* 2020-11-20T12:10:53.104 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2020-11-20T12:10:53.106 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 23 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9340 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database .. 2020-11-20T12:10:53.108 INFO:teuthology.task.ansible.out:. 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 104427 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubunt 2020-11-20T12:10:53.113 INFO:teuthology.task.ansible.out:u1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 23 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9340 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(R 2020-11-20T12:10:53.115 INFO:teuthology.task.ansible.out:eading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 104427 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-20T12:10:53.170 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:53 +0000 (0:00:10.496) 0:01:25.760 ******* 2020-11-20T12:10:53.320 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": true, "stderr": "W: --force-yes is deprecated, use one of the options starting with --allow instead.\n", "stderr_lines": ["W: --force-yes is deprecated, use one of the options starting with --allow instead."], "stdout": "Reading package lists...\nBuilding dependency tree...\nReading state information...\ngit is already the newest version (1:2.17.1-1ubuntu0.7).\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils\nSuggested packages:\n samba vde2 sgabios ovmf debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\n0 upgraded, 6 newly installed, 0 to remove and 23 not upgraded.\nNeed to get 10.4 MB of archives.\nAfter this operation, 47.6 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]\nGet:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]\nGet:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]\nGet:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]\nGet:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]\nGet:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]\nFetched 10.4 MB in 1s (9615 kB/s)\nSelecting previously unselected package librados2.\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database .. 2020-11-20T12:10:53.321 INFO:teuthology.task.ansible.out:. 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 104427 files and directories currently installed.)\r\nPreparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package librbd1.\r\nPreparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...\r\nUnpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSelecting previously unselected package qemu-block-extra:amd64.\r\nPreparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-common.\r\nPreparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-system-x86.\r\nPreparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSelecting previously unselected package qemu-utils.\r\nPreparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...\r\nUnpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up librados2 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...\r\nSetting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...\r\nSetting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...\r\nProcessing triggers for man-db (2.8.3-2ubuntu0.1) ...\r\nProcessing triggers for libc-bin (2.27-3ubunt 2020-11-20T12:10:53.323 INFO:teuthology.task.ansible.out:u1.3) ...\r\n", "stdout_lines": ["Reading package lists...", "Building dependency tree...", "Reading state information...", "git is already the newest version (1:2.17.1-1ubuntu0.7).", "The following additional packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-utils", "Suggested packages:", " samba vde2 sgabios ovmf debootstrap", "The following NEW packages will be installed:", " librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86", " qemu-utils", "0 upgraded, 6 newly installed, 0 to remove and 23 not upgraded.", "Need to get 10.4 MB of archives.", "After this operation, 47.6 MB of additional disk space will be used.", "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.13-0ubuntu0.18.04.5 [2723 kB]", "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.13-0ubuntu0.18.04.5 [923 kB]", "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-block-extra amd64 1:2.11+dfsg-1ubuntu7.33 [40.1 kB]", "Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-common amd64 1:2.11+dfsg-1ubuntu7.33 [672 kB]", "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-system-x86 amd64 1:2.11+dfsg-1ubuntu7.33 [5193 kB]", "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 qemu-utils amd64 1:2.11+dfsg-1ubuntu7.33 [873 kB]", "Fetched 10.4 MB in 1s (9615 kB/s)", "Selecting previously unselected package librados2.", "(Reading database ... ", "(Reading database ... 5%", "(Reading database ... 10%", "(Reading database ... 15%", "(Reading database ... 20%", "(Reading database ... 25%", "(Reading database ... 30%", "(Reading database ... 35%", "(Reading database ... 40%", "(Reading database ... 45%", "(Reading database ... 50%", "(Reading database ... 55%", "(Reading database ... 60%", "(Reading database ... 65%", "(Reading database ... 70%", "(Reading database ... 75%", "(Reading database ... 80%", "(R 2020-11-20T12:10:53.325 INFO:teuthology.task.ansible.out:eading database ... 85%", "(Reading database ... 90%", "(Reading database ... 95%", "(Reading database ... 100%", "(Reading database ... 104427 files and directories currently installed.)", "Preparing to unpack .../0-librados2_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package librbd1.", "Preparing to unpack .../1-librbd1_12.2.13-0ubuntu0.18.04.5_amd64.deb ...", "Unpacking librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Selecting previously unselected package qemu-block-extra:amd64.", "Preparing to unpack .../2-qemu-block-extra_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-common.", "Preparing to unpack .../3-qemu-system-common_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-system-x86.", "Preparing to unpack .../4-qemu-system-x86_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Selecting previously unselected package qemu-utils.", "Preparing to unpack .../5-qemu-utils_1%3a2.11+dfsg-1ubuntu7.33_amd64.deb ...", "Unpacking qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up librados2 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up librbd1 (12.2.13-0ubuntu0.18.04.5) ...", "Setting up qemu-block-extra:amd64 (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-utils (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-common (1:2.11+dfsg-1ubuntu7.33) ...", "Setting up qemu-system-x86 (1:2.11+dfsg-1ubuntu7.33) ...", "Processing triggers for man-db (2.8.3-2ubuntu0.1) ...", "Processing triggers for libc-bin (2.27-3ubuntu1.3) ..."]} 2020-11-20T12:10:53.397 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:53 +0000 (0:00:00.227) 0:01:25.988 ******* 2020-11-20T12:10:54.238 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2020-11-20T12:10:54.240 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": false} 2020-11-20T12:10:54.296 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:54 +0000 (0:00:00.892) 0:01:26.880 ******* 2020-11-20T12:10:54.456 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": false} TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-11-20T12:10:54.497 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:54 +0000 (0:00:00.207) 0:01:27.087 ******* 2020-11-20T12:10:54.537 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:54 +0000 (0:00:00.040) 0:01:27.127 ******* 2020-11-20T12:10:54.544 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2020-11-20T12:10:54.547 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:54.587 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:54 +0000 (0:00:00.050) 0:01:27.177 ******* 2020-11-20T12:10:54.722 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2020-11-20T12:10:54.724 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "cmd": ["perl", "-pe", "if (m{^([^#]\\S*\\s+/\\s+\\S+\\s+)(\\S+)(\\s+.*)$}) { $_=\"$1$2,user_xattr$3\\n\" unless $2=~m{(^|,)user_xattr(,|$)}; }", "-i.bak", "/etc/fstab"], "rc": 0, "stdout": "skipped, since /etc/fstab.bak exists", "stdout_lines": ["skipped, since /etc/fstab.bak exists"]} 2020-11-20T12:10:54.766 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:54 +0000 (0:00:00.178) 0:01:27.356 ******* 2020-11-20T12:10:54.783 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-20T12:10:54.784 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-20T12:10:54.827 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:54 +0000 (0:00:00.061) 0:01:27.417 ******* 2020-11-20T12:10:54.837 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:54.885 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:54 +0000 (0:00:00.056) 0:01:27.474 ******* 2020-11-20T12:10:55.085 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2020-11-20T12:10:55.086 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 10102, "name": "fuse", "state": "present", "system": false} 2020-11-20T12:10:55.127 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:55 +0000 (0:00:00.243) 0:01:27.717 ******* 2020-11-20T12:10:55.358 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-20T12:10:55.402 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:55 +0000 (0:00:00.274) 0:01:27.992 ******* 2020-11-20T12:10:55.635 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-20T12:10:55.637 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-20T12:10:55.680 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:55 +0000 (0:00:00.276) 0:01:28.269 ******* 2020-11-20T12:10:55.685 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2020-11-20T12:10:55.687 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "9edff47031d0f14a9b8c813b44eb4a9296d0439f", "dest": "/etc/fuse.conf", "gid": 10102, "group": "fuse", "mode": "0644", "owner": "root", "path": "/etc/fuse.conf", "size": 366, "state": "file", "uid": 0} 2020-11-20T12:10:55.733 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:55 +0000 (0:00:00.051) 0:01:28.320 ******* 2020-11-20T12:10:55.856 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-20T12:10:55.864 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T12:10:55.909 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:55 +0000 (0:00:00.178) 0:01:28.499 ******* 2020-11-20T12:10:55.961 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2020-11-20T12:10:55.963 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1236, "groups": "fuse,kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2020-11-20T12:10:56.011 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.102) 0:01:28.602 ******* 2020-11-20T12:10:56.204 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2020-11-20T12:10:56.206 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T12:10:56.247 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.235) 0:01:28.837 ******* 2020-11-20T12:10:56.253 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-20T12:10:56.255 INFO:teuthology.task.ansible.out:[WARNING]: The service (apache2) is actually an init script but the system is managed by systemd fatal: [smithi116.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring 2020-11-20T12:10:56.321 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.067) 0:01:28.905 ******* 2020-11-20T12:10:56.343 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:56.389 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.074) 0:01:28.979 ******* 2020-11-20T12:10:56.416 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:56.457 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.068) 0:01:29.047 ******* 2020-11-20T12:10:56.484 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T12:10:56.486 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:56.526 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.068) 0:01:29.116 ******* 2020-11-20T12:10:56.542 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2020-11-20T12:10:56.544 INFO:teuthology.task.ansible.out:fatal: [smithi173.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring 2020-11-20T12:10:56.586 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.060) 0:01:29.176 ******* 2020-11-20T12:10:56.593 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:56.636 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.050) 0:01:29.227 ******* 2020-11-20T12:10:56.651 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2020-11-20T12:10:56.653 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:56.693 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.056) 0:01:29.283 ******* 2020-11-20T12:10:56.699 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T12:10:56.701 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:56.745 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.049) 0:01:29.333 ******* 2020-11-20T12:10:56.749 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2020-11-20T12:10:56.751 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:56.793 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.049) 0:01:29.383 ******* 2020-11-20T12:10:56.799 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2020-11-20T12:10:56.800 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:56.847 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.054) 0:01:29.438 ******* 2020-11-20T12:10:56.854 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2020-11-20T12:10:56.855 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:56.899 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.051) 0:01:29.489 ******* 2020-11-20T12:10:56.906 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:56.950 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.050) 0:01:29.540 ******* 2020-11-20T12:10:56.956 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:56.999 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:56 +0000 (0:00:00.049) 0:01:29.589 ******* 2020-11-20T12:10:57.005 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2020-11-20T12:10:57.006 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.052 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.052) 0:01:29.642 ******* 2020-11-20T12:10:57.059 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2020-11-20T12:10:57.061 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.103 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.051) 0:01:29.694 ******* 2020-11-20T12:10:57.109 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:10:57.111 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.156 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.052) 0:01:29.746 ******* 2020-11-20T12:10:57.164 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.223 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.064) 0:01:29.811 ******* 2020-11-20T12:10:57.353 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 12:10:57 +0000 (0:00:00.055) 0:01:29.866 ******* TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 12:10:57 +0000 (0:00:00.053) 0:01:29.920 ******* TASK [testnode : Install packages] ********************************************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.393 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.061) 0:01:29.982 ******* 2020-11-20T12:10:57.401 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.444 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.052) 0:01:30.034 ******* 2020-11-20T12:10:57.454 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.499 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.053) 0:01:30.088 ******* 2020-11-20T12:10:57.504 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.551 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.053) 0:01:30.141 ******* 2020-11-20T12:10:57.557 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-20T12:10:57.559 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.601 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.050) 0:01:30.191 ******* 2020-11-20T12:10:57.607 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2020-11-20T12:10:57.609 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.664 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.062) 0:01:30.254 ******* 2020-11-20T12:10:57.671 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T12:10:57.672 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.716 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.051) 0:01:30.306 ******* 2020-11-20T12:10:57.724 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.772 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.049) 0:01:30.355 ******* TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.815 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.049) 0:01:30.404 ******* 2020-11-20T12:10:57.820 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2020-11-20T12:10:57.821 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.867 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.053) 0:01:30.457 ******* 2020-11-20T12:10:57.876 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.931 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.063) 0:01:30.521 ******* 2020-11-20T12:10:57.938 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:57.988 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:57 +0000 (0:00:00.053) 0:01:30.574 ******* 2020-11-20T12:10:57.992 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.035 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.049) 0:01:30.624 ******* 2020-11-20T12:10:58.047 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.090 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.055) 0:01:30.680 ******* 2020-11-20T12:10:58.097 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.140 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.050) 0:01:30.730 ******* 2020-11-20T12:10:58.146 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T12:10:58.149 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.205 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.065) 0:01:30.795 ******* 2020-11-20T12:10:58.213 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.271 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.065) 0:01:30.861 ******* 2020-11-20T12:10:58.277 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T12:10:58.282 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.325 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.048) 0:01:30.910 ******* 2020-11-20T12:10:58.329 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.368 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.048) 0:01:30.959 ******* 2020-11-20T12:10:58.375 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-20T12:10:58.376 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.432 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.059) 0:01:31.018 ******* 2020-11-20T12:10:58.434 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T12:10:58.435 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.483 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.054) 0:01:31.073 ******* 2020-11-20T12:10:58.489 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T12:10:58.492 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.547 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.063) 0:01:31.137 ******* 2020-11-20T12:10:58.554 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T12:10:58.556 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.599 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.052) 0:01:31.189 ******* 2020-11-20T12:10:58.605 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2020-11-20T12:10:58.607 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.649 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.050) 0:01:31.239 ******* 2020-11-20T12:10:58.656 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.698 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.048) 0:01:31.288 ******* 2020-11-20T12:10:58.704 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2020-11-20T12:10:58.706 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.747 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.048) 0:01:31.337 ******* 2020-11-20T12:10:58.753 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.795 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.048) 0:01:31.386 ******* 2020-11-20T12:10:58.803 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.859 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.063) 0:01:31.449 ******* 2020-11-20T12:10:58.866 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T12:10:58.869 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.912 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.052) 0:01:31.502 ******* 2020-11-20T12:10:58.918 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T12:10:58.920 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:58.965 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:58 +0000 (0:00:00.052) 0:01:31.555 ******* 2020-11-20T12:10:58.971 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-20T12:10:58.974 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:59.018 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:59 +0000 (0:00:00.052) 0:01:31.608 ******* 2020-11-20T12:10:59.024 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:59.075 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:59 +0000 (0:00:00.057) 0:01:31.665 ******* 2020-11-20T12:10:59.102 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2020-11-20T12:10:59.104 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:59.165 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:59 +0000 (0:00:00.089) 0:01:31.755 ******* 2020-11-20T12:10:59.195 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2020-11-20T12:10:59.197 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Remove /etc/ceph] ********************************************* ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-20T12:10:59.244 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:59 +0000 (0:00:00.079) 0:01:31.835 ******* 2020-11-20T12:10:59.292 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:59 +0000 (0:00:00.044) 0:01:31.880 ******* 2020-11-20T12:10:59.318 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2020-11-20T12:10:59.319 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:59.363 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:59 +0000 (0:00:00.073) 0:01:31.954 ******* 2020-11-20T12:10:59.542 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "path": "/etc/ceph", "state": "absent"} 2020-11-20T12:10:59.584 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:59 +0000 (0:00:00.221) 0:01:32.175 ******* 2020-11-20T12:10:59.718 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-20T12:10:59.765 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:59 +0000 (0:00:00.180) 0:01:32.355 ******* 2020-11-20T12:10:59.792 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2020-11-20T12:10:59.794 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:10:59.833 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:10:59 +0000 (0:00:00.068) 0:01:32.424 ******* 2020-11-20T12:11:00.029 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2020-11-20T12:11:00.031 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "0b949a4622a315acfc90571fddbd013b4dc0f5f9", "dest": "/etc/exports", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/exports", "size": 578, "state": "file", "uid": 0} 2020-11-20T12:11:00.077 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:00 +0000 (0:00:00.243) 0:01:32.667 ******* 2020-11-20T12:11:00.104 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:00.145 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:00 +0000 (0:00:00.067) 0:01:32.735 ******* 2020-11-20T12:11:00.291 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2020-11-20T12:11:00.293 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-20T12:11:00.337 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:00 +0000 (0:00:00.192) 0:01:32.928 ******* 2020-11-20T12:11:00.826 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "d5677bbd13857966eb0fc86f66195b18a2b98e20", "dest": "/etc/cron.weekly/kernel-clean", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/cron.weekly/kernel-clean", "size": 752, "state": "file", "uid": 0} 2020-11-20T12:11:00.869 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:00 +0000 (0:00:00.531) 0:01:33.459 ******* 2020-11-20T12:11:01.034 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2020-11-20T12:11:01.036 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-20T12:11:01.077 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:01 +0000 (0:00:00.207) 0:01:33.667 ******* 2020-11-20T12:11:01.385 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "22e9a1446f43d0440219b9af14a66dd77fb0c2eb", "dest": "/etc/grub.d/02_force_timeout", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/grub.d/02_force_timeout", "size": 115, "state": "file", "uid": 0} 2020-11-20T12:11:01.440 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:01 +0000 (0:00:00.363) 0:01:34.030 ******* 2020-11-20T12:11:01.555 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-20T12:11:01.598 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:01 +0000 (0:00:00.158) 0:01:34.188 ******* 2020-11-20T12:11:01.773 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** ok: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T12:11:01.816 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:01 +0000 (0:00:00.217) 0:01:34.406 ******* 2020-11-20T12:11:01.843 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-20T12:11:01.845 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:01.891 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:01 +0000 (0:00:00.075) 0:01:34.482 ******* 2020-11-20T12:11:01.910 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2020-11-20T12:11:01.960 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "6449466d762bced58744df80432d189e35329803", "dest": "/etc/modules", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/modules", "size": 315, "state": "file", "uid": 0} 2020-11-20T12:11:01.968 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:01 +0000 (0:00:00.075) 0:01:34.558 ******* 2020-11-20T12:11:01.974 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.030 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.062) 0:01:34.621 ******* 2020-11-20T12:11:02.061 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.124 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.093) 0:01:34.714 ******* 2020-11-20T12:11:02.153 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2020-11-20T12:11:02.155 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.209 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.085) 0:01:34.800 ******* 2020-11-20T12:11:02.217 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2020-11-20T12:11:02.219 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T12:11:02.269 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.060) 0:01:34.860 ******* 2020-11-20T12:11:02.275 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2020-11-20T12:11:02.277 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.322 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.052) 0:01:34.912 ******* 2020-11-20T12:11:02.328 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2020-11-20T12:11:02.331 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.375 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.049) 0:01:34.962 ******* 2020-11-20T12:11:02.378 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:11:02.379 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-20T12:11:02.421 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.048) 0:01:35.011 ******* 2020-11-20T12:11:02.427 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2020-11-20T12:11:02.429 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.472 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.051) 0:01:35.063 ******* 2020-11-20T12:11:02.481 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi116.front.sepia.ceph.com] => (item=ram14) => {"ansible_loop_var": "item", "changed": false, "item": "ram14", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.493 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=ram15) => {"ansible_loop_var": "item", "changed": false, "item": "ram15", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.495 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=ram12) => {"ansible_loop_var": "item", "changed": false, "item": "ram12", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=ram13) => {"ansible_loop_var": "item", "changed": false, "item": "ram13", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.505 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=ram10) => {"ansible_loop_var": "item", "changed": false, "item": "ram10", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.507 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi116.front.sepia.ceph.com] => (item=ram11) => {"ansible_loop_var": "item", "changed": false, "item": "ram11", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.551 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.078) 0:01:35.141 ******* 2020-11-20T12:11:02.557 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=ram8) => {"ansible_loop_var": "item", "changed": false, "item": "ram8", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.558 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=ram9) => {"ansible_loop_var": "item", "changed": false, "item": "ram9", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.558 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=ram4) => {"ansible_loop_var": "item", "changed": false, "item": "ram4", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.560 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=ram5) => {"ansible_loop_var": "item", "changed": false, "item": "ram5", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=ram6) => {"ansible_loop_var": "item", "changed": false, "item": "ram6", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=ram7) => {"ansible_loop_var": "item", "changed": false, "item": "ram7", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=ram0) => {"ansible_loop_var": "item", "changed": false, "item": "ram0", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.561 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=ram1) => {"ansible_loop_var": "item", "changed": false, "item": "ram1", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.572 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=ram2) => {"ansible_loop_var": "item", "changed": false, "item": "ram2", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.574 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.577 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi116.front.sepia.ceph.com] => (item=ram3) => {"ansible_loop_var": "item", "changed": false, "item": "ram3", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.623 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.072) 0:01:35.213 ******* 2020-11-20T12:11:02.630 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.632 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.632 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.633 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.633 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.645 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.647 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} TASK [testnode : Update /etc/profile if needed.] ******************************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi116.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.694 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.070) 0:01:35.284 ******* 2020-11-20T12:11:02.700 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.702 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.703 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-20T12:11:02.806 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.111) 0:01:35.396 ******* 2020-11-20T12:11:02.812 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:11:02.827 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2020-11-20T12:11:02.873 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:02 +0000 (0:00:00.065) 0:01:35.461 ******* 2020-11-20T12:11:02.973 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi173.front.sepia.ceph.com] => (item=ram14) => {"ansible_loop_var": "item", "changed": false, "item": "ram14", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:02.984 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=ram15) => {"ansible_loop_var": "item", "changed": false, "item": "ram15", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.038 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** changed: [smithi116.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.008501", "end": "2020-11-20 12:11:03.004336", "rc": 0, "start": "2020-11-20 12:11:02.995835", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-20T12:11:03.110 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:03 +0000 (0:00:00.237) 0:01:35.699 ******* 2020-11-20T12:11:03.120 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2020-11-20T12:11:03.122 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=ram12) => {"ansible_loop_var": "item", "changed": false, "item": "ram12", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=ram13) => {"ansible_loop_var": "item", "changed": false, "item": "ram13", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=ram10) => {"ansible_loop_var": "item", "changed": false, "item": "ram10", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=ram11) => {"ansible_loop_var": "item", "changed": false, "item": "ram11", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.123 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=ram8) => {"ansible_loop_var": "item", "changed": false, "item": "ram8", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.123 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=ram9) => {"ansible_loop_var": "item", "changed": false, "item": "ram9", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.126 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=ram4) => {"ansible_loop_var": "item", "changed": false, "item": "ram4", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.132 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=ram5) => {"ansible_loop_var": "item", "changed": false, "item": "ram5", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=ram6) => {"ansible_loop_var": "item", "changed": false, "item": "ram6", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=ram7) => {"ansible_loop_var": "item", "changed": false, "item": "ram7", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.135 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=ram0) => {"ansible_loop_var": "item", "changed": false, "item": "ram0", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=ram1) => {"ansible_loop_var": "item", "changed": false, "item": "ram1", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=ram2) => {"ansible_loop_var": "item", "changed": false, "item": "ram2", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=ram3) => {"ansible_loop_var": "item", "changed": false, "item": "ram3", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.144 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.162 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi173.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.209 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:03 +0000 (0:00:00.097) 0:01:35.797 ******* 2020-11-20T12:11:03.217 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=loop3) => {"ansible_loop_var": "item", "changed": false, "item": "loop3", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=loop2) => {"ansible_loop_var": "item", "changed": false, "item": "loop2", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.230 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=loop1) => {"ansible_loop_var": "item", "changed": false, "item": "loop1", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.232 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=loop0) => {"ansible_loop_var": "item", "changed": false, "item": "loop0", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=loop7) => {"ansible_loop_var": "item", "changed": false, "item": "loop7", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.243 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=loop6) => {"ansible_loop_var": "item", "changed": false, "item": "loop6", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.244 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi173.front.sepia.ceph.com] => (item=loop5) => {"ansible_loop_var": "item", "changed": false, "item": "loop5", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.288 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:03 +0000 (0:00:00.081) 0:01:35.878 ******* 2020-11-20T12:11:03.295 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=loop4) => {"ansible_loop_var": "item", "changed": false, "item": "loop4", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.296 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2020-11-20T12:11:03.347 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:03 +0000 (0:00:00.058) 0:01:35.936 ******* 2020-11-20T12:11:03.352 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2020-11-20T12:11:03.355 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.439 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:03 +0000 (0:00:00.090) 0:01:36.027 ******* 2020-11-20T12:11:03.603 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2020-11-20T12:11:03.604 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.004418", "end": "2020-11-20 12:11:03.574311", "rc": 0, "start": "2020-11-20 12:11:03.569893", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2020-11-20T12:11:03.653 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:03 +0000 (0:00:00.215) 0:01:36.243 ******* 2020-11-20T12:11:03.693 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.738 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:03 +0000 (0:00:00.085) 0:01:36.328 ******* 2020-11-20T12:11:03.776 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2020-11-20T12:11:03.778 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.827 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:03 +0000 (0:00:00.085) 0:01:36.414 ******* 2020-11-20T12:11:03.851 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:03.898 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:03 +0000 (0:00:00.068) 0:01:36.483 ******* 2020-11-20T12:11:04.503 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2020-11-20T12:11:04.505 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": false} 2020-11-20T12:11:04.547 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:04 +0000 (0:00:00.654) 0:01:37.138 ******* 2020-11-20T12:11:04.576 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:04.620 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:04 +0000 (0:00:00.070) 0:01:37.209 ******* 2020-11-20T12:11:04.657 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ ok: [smithi116.front.sepia.ceph.com] => (item={'block_used': 14063015, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032093, 'size_available': 925827452928, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119645, 'block_size': 4096, 'inode_available': 60935331}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032093, "block_size": 4096, "block_total": 240095108, "block_used": 14063015, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935331, "inode_total": 61054976, "inode_used": 119645, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925827452928, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}} 2020-11-20T12:11:04.701 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:04 +0000 (0:00:00.081) 0:01:37.290 ******* 2020-11-20T12:11:04.908 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2020-11-20T12:11:04.910 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.031797", "end": "2020-11-20 12:11:04.748023", "rc": 0, "start": "2020-11-20 12:11:04.716226", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:04.954 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:04 +0000 (0:00:00.254) 0:01:37.545 ******* 2020-11-20T12:11:04.973 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* ok: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": false} 2020-11-20T12:11:05.018 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:05 +0000 (0:00:00.063) 0:01:37.608 ******* 2020-11-20T12:11:05.026 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** skipping: [smithi116.front.sepia.ceph.com] => (item={'block_used': 14063015, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032093, 'size_available': 925827452928, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119645, 'block_size': 4096, 'inode_available': 60935331}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032093, "block_size": 4096, "block_total": 240095108, "block_used": 14063015, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935331, "inode_total": 61054976, "inode_used": 119645, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925827452928, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.071 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:05 +0000 (0:00:00.052) 0:01:37.661 ******* 2020-11-20T12:11:05.078 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.124 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:05 +0000 (0:00:00.053) 0:01:37.714 ******* 2020-11-20T12:11:05.131 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-20T12:11:05.175 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:05 +0000 (0:00:00.051) 0:01:37.766 ******* 2020-11-20T12:11:05.182 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T12:11:05.184 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item={'block_used': 14063015, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032093, 'size_available': 925827452928, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119645, 'block_size': 4096, 'inode_available': 60935331}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032093, "block_size": 4096, "block_total": 240095108, "block_used": 14063015, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935331, "inode_total": 61054976, "inode_used": 119645, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925827452928, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}} 2020-11-20T12:11:05.234 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:05 +0000 (0:00:00.057) 0:01:37.823 ******* 2020-11-20T12:11:05.246 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram14', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram14", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.266 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram15', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram15", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.284 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram12', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram12", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.288 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram13', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram13", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram10', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram10", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram11', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': 2020-11-20T12:11:05.290 INFO:teuthology.task.ansible.out:[], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram11", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.299 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram8', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram8", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.311 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram9', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram9", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.324 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.351 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram5', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram6', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.359 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram7', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.371 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.385 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.394 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.407 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.462 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.027108", "end": "2020-11-20 12:11:05.440445", "rc": 0, "start": "2020-11-20 12:11:05.413337", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:05.502 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:05 +0000 (0:00:00.268) 0:01:38.092 ******* 2020-11-20T12:11:05.539 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2020-11-20T12:11:05.543 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'block_used': 14063015, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032093, 'size_available': 925827452928, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119645, 'block_size': 4096, 'inode_available': 60935331}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032093, "block_size": 4096, "block_total": 240095108, "block_used": 14063015, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935331, "inode_total": 61054976, "inode_used": 119645, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925827452928, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.584 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:05 +0000 (0:00:00.079) 0:01:38.172 ******* 2020-11-20T12:11:05.609 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2020-11-20T12:11:05.650 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:05 +0000 (0:00:00.068) 0:01:38.240 ******* 2020-11-20T12:11:05.711 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram14', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram14", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.722 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram15', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram15", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.734 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram12', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram12", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.745 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram13', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram13", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.757 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram10', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram10", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.759 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram11', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram11", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.760 INFO:teuthology.task.ansible.out: 2020-11-20T12:11:05.771 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram8', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram8", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.783 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram9', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram9", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.805 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.808 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram5', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.818 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram6', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.829 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram7', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.841 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.852 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.864 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:05.876 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:07.457 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvNeDA0YygLLZYRE7Gbm57KBZZoIGVe60m'], 'uuids': ['fed3cd3c-024b-4991-b9f6-f7b24acbaaa1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.882240", "end": "2020-11-20 12:11:07.285259", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvNeDA0YygLLZYRE7Gbm57KBZZoIGVe60m"], "labels": [], "masters": [], "uuids": ["fed3cd3c-024b-4991-b9f6-f7b24acbaaa1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:05.403019", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:07.872 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERdwQCUfBbGt5zZu3DXXA5XRN1JYN169wS'], 'uuids': ['0e1a046e-113f-42df-83b0-e4591f131bd7']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.825591", "end": "2020-11-20 12:11:07.853079", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERdwQCUfBbGt5zZu3DXXA5XRN1JYN169wS"], "labels": [], "masters": [], "uuids": ["0e1a046e-113f-42df-83b0-e4591f131bd7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:06.027488", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:08.802 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvK0Sb7p8YRq4MjcRNC2MmERfKveke3057'], 'uuids': ['9ae969e4-1749-4350-b033-a900efd8780a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.132563", "end": "2020-11-20 12:11:08.633797", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvK0Sb7p8YRq4MjcRNC2MmERfKveke3057"], "labels": [], "masters": [], "uuids": ["9ae969e4-1749-4350-b033-a900efd8780a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:07.501234", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:09.206 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaER1yGDVUsfQu3aEptqo9F3zViTyYiSkT5B'], 'uuids': ['52354218-2123-4e65-9a23-6e9a407d29dd']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.163909", "end": "2020-11-20 12:11:09.181858", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaER1yGDVUsfQu3aEptqo9F3zViTyYiSkT5B"], "labels": [], "masters": [], "uuids": ["52354218-2123-4e65-9a23-6e9a407d29dd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:08.017949", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:10.096 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGv0AwJjBaGAZQDC0IHoRazuOxV5yFXa2XI'], 'uuids': ['72d2a65e-04cd-49ba-9d90-aa8406340960']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.135211", "end": "2020-11-20 12:11:09.933540", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGv0AwJjBaGAZQDC0IHoRazuOxV5yFXa2XI"], "labels": [], "masters": [], "uuids": ["72d2a65e-04cd-49ba-9d90-aa8406340960"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:08.798329", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:10.546 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERQsYkdIngwGQSjCfIR01dMXznmKZi3xCQ'], 'uuids': ['32c2f0d1-b174-4463-a4b4-1116eebac357']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.164904", "end": "2020-11-20 12:11:10.523639", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERQsYkdIngwGQSjCfIR01dMXznmKZi3xCQ"], "labels": [], "masters": [], "uuids": ["32c2f0d1-b174-4463-a4b4-1116eebac357"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:09.358735", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:11.406 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvT2E8HE3NNeOk4cOyRAinKQpz82vcJPPJ'], 'uuids': ['279355c4-99f9-48ad-971d-1fcbbc4dbb0c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.140910", "end": "2020-11-20 12:11:11.244760", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvT2E8HE3NNeOk4cOyRAinKQpz82vcJPPJ"], "labels": [], "masters": [], "uuids": ["279355c4-99f9-48ad-971d-1fcbbc4dbb0c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:10.103850", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:11.939 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERJ0Go3rXcrwOweKKxHR52J9hdaJNLjrLv'], 'uuids': ['2a676e1e-a1d3-4b22-8aff-2e571ee7f00b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.224370", "end": "2020-11-20 12:11:11.913106", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERJ0Go3rXcrwOweKKxHR52J9hdaJNLjrLv"], "labels": [], "masters": [], "uuids": ["2a676e1e-a1d3-4b22-8aff-2e571ee7f00b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:10.688736", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:12.775 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvORFTXF7LjJ11FTzQUbWGzDDZj9ER3knA'], 'uuids': ['f11002fc-41df-40c2-99e0-28f16efc6d65']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.199447", "end": "2020-11-20 12:11:12.610188", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvORFTXF7LjJ11FTzQUbWGzDDZj9ER3knA"], "labels": [], "masters": [], "uuids": ["f11002fc-41df-40c2-99e0-28f16efc6d65"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:11.410741", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:12.786 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QJB5', 'wwn-0x5000c50091e1a66f'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QJB5-part1', 'wwn-0x5000c50091e1a66f-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e1a66f', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QJB5', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJB5", "wwn-0x5000c50091e1a66f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJB5-part1", "wwn-0x5000c50091e1a66f-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QJB5", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e1a66f"}}, 2020-11-20T12:11:12.788 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T12:11:12.798 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:12.799 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:12.811 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:12.813 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:12.825 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:12.838 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:12.840 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:12.850 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:13.272 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERqxnIkMhTVEUOo3sEZHrJWmOAOjbQKZ3E'], 'uuids': ['b5b6bd1f-ea40-4576-a4f4-9dbd028324b5']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.152997", "end": "2020-11-20 12:11:13.233693", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERqxnIkMhTVEUOo3sEZHrJWmOAOjbQKZ3E"], "labels": [], "masters": [], "uuids": ["b5b6bd1f-ea40-4576-a4f4-9dbd028324b5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:12.080696", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:13.281 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XM1Y', 'wwn-0x5000c50092935686'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XM1Y-part1', 'wwn-0x5000c50092935686-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50092935686', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XM1Y', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XM1Y", "wwn-0x5000c50092935686"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XM1Y-part1", "wwn-0x5000c50092935686-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XM1Y", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092935686"}}, 2020-11-20T12:11:13.285 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T12:11:13.293 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:13.307 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:13.319 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:13.331 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:13.343 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:13.366 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:13.380 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:13.394 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.183 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-v5WzK8-ZL9y-tu5D-wHeU-Wqeo-ekbk-XOrcRT', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400VJ400BGN', 'nvme-nvme.8086-50484654363230343030564a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.155095", "end": "2020-11-20 12:11:14.012756", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-v5WzK8-ZL9y-tu5D-wHeU-Wqeo-ekbk-XOrcRT", "nvme-INTEL_SSDPEDMD400G4_PHFT620400VJ400BGN", "nvme-nvme.8086-50484654363230343030564a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:12.857661", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-20T12:11:14.185 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:14.247 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:14 +0000 (0:00:08.596) 0:01:46.837 ******* 2020-11-20T12:11:14.296 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2020-11-20T12:11:14.298 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram14', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram14", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.308 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram15', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram15", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.321 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram12', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram12", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.332 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram13', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram13", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.344 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram10', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram10", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.355 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram11', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram11", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.369 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram8', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram8", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.379 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram9', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram9", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.392 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.404 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram5', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.424 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram6', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram7', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.430 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.446 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.469 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'ram3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.710 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2020-11-20T12:11:14.711 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-GS4Cs3-NLPK-38KP-9wcA-S10f-VHg6-bJf0Ke', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204019S400BGN', 'nvme-nvme.8086-50484654363230343031395334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.143807", "end": "2020-11-20 12:11:14.680899", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-GS4Cs3-NLPK-38KP-9wcA-S10f-VHg6-bJf0Ke", "nvme-INTEL_SSDPEDMD400G4_PHFT6204019S400BGN", "nvme-nvme.8086-50484654363230343031395334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:13.537092", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2020-11-20T12:11:14.713 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2020-11-20T12:11:14.783 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:14 +0000 (0:00:00.536) 0:01:47.373 ******* 2020-11-20T12:11:14.856 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram14', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram14", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.879 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram15', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram15", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.881 INFO:teuthology.task.ansible.out: 2020-11-20T12:11:14.891 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram12', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram12", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.903 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram13', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram13", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.915 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram10', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram10", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.926 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram11', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram11", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.949 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram8', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram8", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.960 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram9', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram9", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.972 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:14.994 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram5', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:15.006 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram6', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:15.028 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram7', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:15.039 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:15.061 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:15.073 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:15.085 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'ram3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '131072', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '64.00 MB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ram3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "131072", "sectorsize": "512", "size": "64.00 MB", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:16.472 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvNeDA0YygLLZYRE7Gbm57KBZZoIGVe60m'], 'uuids': ['fed3cd3c-024b-4991-b9f6-f7b24acbaaa1']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:01.815609", "end": "2020-11-20 12:11:16.310731", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvNeDA0YygLLZYRE7Gbm57KBZZoIGVe60m"], "labels": [], "masters": [], "uuids": ["fed3cd3c-024b-4991-b9f6-f7b24acbaaa1"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:14.495122", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.80228 s, 64.0 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.80228 s, 64.0 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:17.028 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '31252480', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERdwQCUfBbGt5zZu3DXXA5XRN1JYN169wS'], 'uuids': ['0e1a046e-113f-42df-83b0-e4591f131bd7']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '14.90 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:01.724821", "end": "2020-11-20 12:11:16.995463", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERdwQCUfBbGt5zZu3DXXA5XRN1JYN169wS"], "labels": [], "masters": [], "uuids": ["0e1a046e-113f-42df-83b0-e4591f131bd7"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:15.270642", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.71194 s, 67.4 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.71194 s, 67.4 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:18.374 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvK0Sb7p8YRq4MjcRNC2MmERfKveke3057'], 'uuids': ['9ae969e4-1749-4350-b033-a900efd8780a']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:01.732467", "end": "2020-11-20 12:11:18.203281", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvK0Sb7p8YRq4MjcRNC2MmERfKveke3057"], "labels": [], "masters": [], "uuids": ["9ae969e4-1749-4350-b033-a900efd8780a"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:16.470814", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.7211 s, 67.0 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.7211 s, 67.0 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:18.904 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaER1yGDVUsfQu3aEptqo9F3zViTyYiSkT5B'], 'uuids': ['52354218-2123-4e65-9a23-6e9a407d29dd']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:01.721304", "end": "2020-11-20 12:11:18.877501", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaER1yGDVUsfQu3aEptqo9F3zViTyYiSkT5B"], "labels": [], "masters": [], "uuids": ["52354218-2123-4e65-9a23-6e9a407d29dd"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:17.156197", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.7094 s, 67.5 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.7094 s, 67.5 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:20.317 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGv0AwJjBaGAZQDC0IHoRazuOxV5yFXa2XI'], 'uuids': ['72d2a65e-04cd-49ba-9d90-aa8406340960']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:01.745990", "end": "2020-11-20 12:11:20.150652", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGv0AwJjBaGAZQDC0IHoRazuOxV5yFXa2XI"], "labels": [], "masters": [], "uuids": ["72d2a65e-04cd-49ba-9d90-aa8406340960"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:18.404662", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.73263 s, 66.6 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.73263 s, 66.6 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:20.829 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERQsYkdIngwGQSjCfIR01dMXznmKZi3xCQ'], 'uuids': ['32c2f0d1-b174-4463-a4b4-1116eebac357']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:01.753303", "end": "2020-11-20 12:11:20.802231", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERQsYkdIngwGQSjCfIR01dMXznmKZi3xCQ"], "labels": [], "masters": [], "uuids": ["32c2f0d1-b174-4463-a4b4-1116eebac357"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:19.048928", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.74038 s, 66.3 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.74038 s, 66.3 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:22.208 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvT2E8HE3NNeOk4cOyRAinKQpz82vcJPPJ'], 'uuids': ['279355c4-99f9-48ad-971d-1fcbbc4dbb0c']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:01.718160", "end": "2020-11-20 12:11:22.052257", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvT2E8HE3NNeOk4cOyRAinKQpz82vcJPPJ"], "labels": [], "masters": [], "uuids": ["279355c4-99f9-48ad-971d-1fcbbc4dbb0c"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:20.334097", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.70586 s, 67.6 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.70586 s, 67.6 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:22.795 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERJ0Go3rXcrwOweKKxHR52J9hdaJNLjrLv'], 'uuids': ['2a676e1e-a1d3-4b22-8aff-2e571ee7f00b']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:01.788478", "end": "2020-11-20 12:11:22.764027", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERJ0Go3rXcrwOweKKxHR52J9hdaJNLjrLv"], "labels": [], "masters": [], "uuids": ["2a676e1e-a1d3-4b22-8aff-2e571ee7f00b"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:20.975549", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.77607 s, 64.9 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.77607 s, 64.9 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:24.153 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvORFTXF7LjJ11FTzQUbWGzDDZj9ER3knA'], 'uuids': ['f11002fc-41df-40c2-99e0-28f16efc6d65']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:01.780836", "end": "2020-11-20 12:11:23.991523", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-bCVaWISqIMqX1MlgsFo12hur4tdfHxGvORFTXF7LjJ11FTzQUbWGzDDZj9ER3knA"], "labels": [], "masters": [], "uuids": ["f11002fc-41df-40c2-99e0-28f16efc6d65"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:22.210687", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.76798 s, 65.2 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.76798 s, 65.2 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:24.214 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QJB5', 'wwn-0x5000c50091e1a66f'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QJB5-part1', 'wwn-0x5000c50091e1a66f-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50091e1a66f', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5QJB5', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJB5", "wwn-0x5000c50091e1a66f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QJB5-part1", "wwn-0x5000c50091e1a66f-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QJB5", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e1a66f"}}, 2020-11-20T12:11:24.216 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 2020-11-20T12:11:24.218 INFO:teuthology.task.ansible.out:'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.229 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.242 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.254 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.266 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.269 INFO:teuthology.task.ansible.out: 2020-11-20T12:11:24.691 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'scheduler_mode': '', 'rotational': '0', 'vendor': None, 'sectors': '187490304', 'links': {'masters': [], 'labels': [], 'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERqxnIkMhTVEUOo3sEZHrJWmOAOjbQKZ3E'], 'uuids': ['b5b6bd1f-ea40-4576-a4f4-9dbd028324b5']}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': None, 'partitions': {}, 'holders': [], 'size': '89.40 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:01.736684", "end": "2020-11-20 12:11:24.659704", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-duxrZ8I7QyX5Qb8q1VOCkVlZyRZOUaERqxnIkMhTVEUOo3sEZHrJWmOAOjbQKZ3E"], "labels": [], "masters": [], "uuids": ["b5b6bd1f-ea40-4576-a4f4-9dbd028324b5"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:22.923020", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 1.72463 s, 66.9 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 1.72463 s, 66.9 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:24.693 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': 'ATA', 'sectors': '1953525168', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XM1Y', 'wwn-0x5000c50092935686'], 'uuids': []}, 'partitions': {'sda1': {'sectorsize': 512, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'links': {'masters': [], 'labels': [], 'ids': ['ata-ST1000NM0033-9ZM173_Z1W5XM1Y-part1', 'wwn-0x5000c50092935686-part1'], 'uuids': ['5b37dc69-d2db-44bf-9122-6cf42a1bab47']}, 'sectors': '1953522688', 'start': '2048', 'holders': [], 'size': '931.51 GB'}}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'holders': [], 'wwn': '0x5000c50092935686', 'model': 'ST1000NM0033-9ZM', 'serial': 'Z1W5XM1Y', 'size': '931.51 GB'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XM1Y", "wwn-0x5000c50092935686"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XM1Y-part1", "wwn-0x5000c50092935686-part1"], "labels": [], "masters": [], "uuids": ["5b37dc69-d2db-44bf-9122-6cf42a1bab47"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XM1Y", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50092935686"}}, 2020-11-20T12:11:24.694 INFO:teuthology.task.ansible.out: "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.706 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop3', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop3", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.721 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop2', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop2", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.729 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop1', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop1", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.743 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop0', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop0", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.745 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop7', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop7", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.757 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop6', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop6", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.778 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop5', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop5", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.779 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'loop4', 'value': {'scheduler_mode': 'mq-deadline', 'rotational': '1', 'vendor': None, 'sectors': '0', 'links': {'masters': [], 'labels': [], 'ids': [], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': '', 'sectorsize': '512', 'removable': '0', 'support_discard': '0', 'model': None, 'partitions': {}, 'holders': [], 'size': '0.00 Bytes'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "loop4", "value": {"holders": [], "host": "", "links": {"ids": [], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "0", "sectorsize": "512", "size": "0.00 Bytes", "support_discard": "0", "vendor": null, "virtual": 1}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:24.855 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-v5WzK8-ZL9y-tu5D-wHeU-Wqeo-ekbk-XOrcRT', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400VJ400BGN', 'nvme-nvme.8086-50484654363230343030564a34303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.415205", "end": "2020-11-20 12:11:24.692062", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-v5WzK8-ZL9y-tu5D-wHeU-Wqeo-ekbk-XOrcRT", "nvme-INTEL_SSDPEDMD400G4_PHFT620400VJ400BGN", "nvme-nvme.8086-50484654363230343030564a34303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:24.276857", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.40199 s, 287 MB/s", "stder 2020-11-20T12:11:24.857 INFO:teuthology.task.ansible.out:r_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.40199 s, 287 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:24.916 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:24 +0000 (0:00:10.133) 0:01:57.506 ******* 2020-11-20T12:11:25.360 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'scheduler_mode': 'none', 'rotational': '0', 'vendor': None, 'sectors': '781422768', 'links': {'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4'], 'labels': [], 'ids': ['lvm-pv-uuid-GS4Cs3-NLPK-38KP-9wcA-S10f-VHg6-bJf0Ke', 'nvme-INTEL_SSDPEDMD400G4_PHFT6204019S400BGN', 'nvme-nvme.8086-50484654363230343031395334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': []}, 'sas_device_handle': None, 'sas_address': None, 'virtual': 1, 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'sectorsize': '512', 'removable': '0', 'support_discard': '512', 'model': 'INTEL SSDPEDMD400G4', 'partitions': {}, 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4'], 'size': '372.61 GB'}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.403584", "end": "2020-11-20 12:11:25.329417", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-GS4Cs3-NLPK-38KP-9wcA-S10f-VHg6-bJf0Ke", "nvme-INTEL_SSDPEDMD400G4_PHFT6204019S400BGN", "nvme-nvme.8086-50484654363230343031395334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2020-11-20 12:11:24.925833", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.395722 s, 291 MB/s", "stde 2020-11-20T12:11:25.362 INFO:teuthology.task.ansible.out:rr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.395722 s, 291 MB/s"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:25.419 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:25 +0000 (0:00:00.502) 0:01:58.009 ******* 2020-11-20T12:11:26.454 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** changed: [smithi173.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.860367", "end": "2020-11-20 12:11:26.425116", "rc": 0, "start": "2020-11-20 12:11:25.564749", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:26.495 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:26 +0000 (0:00:01.076) 0:01:59.085 ******* 2020-11-20T12:11:26.513 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:01.394569", "end": "2020-11-20 12:11:26.346384", "rc": 0, "start": "2020-11-20 12:11:24.951815", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:26.555 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:26 +0000 (0:00:00.059) 0:01:59.145 ******* 2020-11-20T12:11:26.782 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2020-11-20T12:11:26.784 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.095877", "end": "2020-11-20 12:11:26.752125", "rc": 0, "start": "2020-11-20 12:11:26.656248", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:26.825 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:26 +0000 (0:00:00.270) 0:01:59.416 ******* 2020-11-20T12:11:26.832 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.085051", "end": "2020-11-20 12:11:26.674481", "rc": 0, "start": "2020-11-20 12:11:26.589430", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:26.874 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:26 +0000 (0:00:00.047) 0:01:59.464 ******* 2020-11-20T12:11:26.881 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2020-11-20T12:11:26.942 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:26 +0000 (0:00:00.068) 0:01:59.532 ******* 2020-11-20T12:11:27.013 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.065) 0:01:59.598 ******* 2020-11-20T12:11:27.020 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.071 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.061) 0:01:59.660 ******* 2020-11-20T12:11:27.077 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.135 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.065) 0:01:59.726 ******* 2020-11-20T12:11:27.145 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.186 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.050) 0:01:59.776 ******* 2020-11-20T12:11:27.192 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.234 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.048) 0:01:59.824 ******* 2020-11-20T12:11:27.241 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.285 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.051) 0:01:59.875 ******* 2020-11-20T12:11:27.291 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.293 INFO:teuthology.task.ansible.out: 2020-11-20T12:11:27.337 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.051) 0:01:59.927 ******* 2020-11-20T12:11:27.344 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.391 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.054) 0:01:59.981 ******* 2020-11-20T12:11:27.398 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.443 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.051) 0:02:00.033 ******* 2020-11-20T12:11:27.566 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2020-11-20T12:11:27.567 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T12:11:27.611 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.167) 0:02:00.201 ******* 2020-11-20T12:11:27.627 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2020-11-20T12:11:27.671 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.060) 0:02:00.261 ******* 2020-11-20T12:11:27.677 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2020-11-20T12:11:27.679 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'block_used': 14063015, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032093, 'size_available': 925827452928, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119645, 'block_size': 4096, 'inode_available': 60935331}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032093, "block_size": 4096, "block_total": 240095108, "block_used": 14063015, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935331, "inode_total": 61054976, "inode_used": 119645, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925827452928, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.722 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.050) 0:02:00.312 ******* 2020-11-20T12:11:27.728 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'block_used': 14063015, 'uuid': '5b37dc69-d2db-44bf-9122-6cf42a1bab47', 'size_total': 983429562368, 'block_total': 240095108, 'mount': '/', 'block_available': 226032093, 'size_available': 925827452928, 'fstype': 'ext4', 'inode_total': 61054976, 'options': 'rw,relatime,no_fc,errors=remount-ro', 'device': '/dev/sda1', 'inode_used': 119645, 'block_size': 4096, 'inode_available': 60935331}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 226032093, "block_size": 4096, "block_total": 240095108, "block_used": 14063015, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60935331, "inode_total": 61054976, "inode_used": 119645, "mount": "/", "options": "rw,relatime,no_fc,errors=remount-ro", "size_available": 925827452928, "size_total": 983429562368, "uuid": "5b37dc69-d2db-44bf-9122-6cf42a1bab47"}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.773 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.050) 0:02:00.363 ******* 2020-11-20T12:11:27.779 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2020-11-20T12:11:27.781 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.822 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.049) 0:02:00.412 ******* 2020-11-20T12:11:27.828 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.871 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.048) 0:02:00.461 ******* 2020-11-20T12:11:27.877 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:11:27.879 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.921 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.049) 0:02:00.511 ******* 2020-11-20T12:11:27.927 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:27.972 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:27 +0000 (0:00:00.051) 0:02:00.562 ******* 2020-11-20T12:11:27.979 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:11:27.982 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:28.121 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:28 +0000 (0:00:00.148) 0:02:00.711 ******* 2020-11-20T12:11:28.127 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:28.171 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:28 +0000 (0:00:00.049) 0:02:00.761 ******* 2020-11-20T12:11:28.177 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2020-11-20T12:11:28.178 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:28.220 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:28 +0000 (0:00:00.049) 0:02:00.810 ******* 2020-11-20T12:11:28.226 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:28.269 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:28 +0000 (0:00:00.048) 0:02:00.859 ******* 2020-11-20T12:11:28.275 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2020-11-20T12:11:28.278 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:28.321 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:28 +0000 (0:00:00.052) 0:02:00.911 ******* 2020-11-20T12:11:28.328 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:28.370 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:28 +0000 (0:00:00.049) 0:02:00.960 ******* 2020-11-20T12:11:28.377 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2020-11-20T12:11:28.378 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:28.420 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:28 +0000 (0:00:00.049) 0:02:01.010 ******* 2020-11-20T12:11:28.426 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:28.469 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:28 +0000 (0:00:00.049) 0:02:01.059 ******* 2020-11-20T12:11:28.944 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2020-11-20T12:11:29.176 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:28 +0000 (0:00:00.523) 0:02:01.583 ******* changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} Friday 20 November 2020 12:11:29 +0000 (0:00:00.059) 0:02:01.643 ******* 2020-11-20T12:11:29.573 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2020-11-20T12:11:29.575 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:11:29.607 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:11:29.950 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:11:30.024 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:11:30.332 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:11:30.428 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:11:30.745 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:11:30.831 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:11:31.138 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:11:31.201 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:31 +0000 (0:00:02.147) 0:02:03.791 ******* 2020-11-20T12:11:31.282 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2020-11-20T12:11:31.327 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:31 +0000 (0:00:00.126) 0:02:03.917 ******* 2020-11-20T12:11:31.409 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-20T12:11:31.459 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:31 +0000 (0:00:00.129) 0:02:04.047 ******* 2020-11-20T12:11:31.507 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2020-11-20T12:11:31.509 INFO:teuthology.task.ansible.out: 2020-11-20T12:11:31.564 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:31 +0000 (0:00:00.106) 0:02:04.154 ******* 2020-11-20T12:11:31.689 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2020-11-20T12:11:31.691 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:11:31.777 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:11:31.895 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:11:31.949 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:11:32.055 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:11:32.112 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:11:32.220 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:11:32.242 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:32.291 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:32 +0000 (0:00:00.726) 0:02:04.881 ******* 2020-11-20T12:11:32.298 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2020-11-20T12:11:32.299 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:32.358 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:32 +0000 (0:00:00.067) 0:02:04.948 ******* 2020-11-20T12:11:32.470 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2020-11-20T12:11:32.473 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T12:11:32.515 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:32 +0000 (0:00:00.157) 0:02:05.106 ******* 2020-11-20T12:11:32.553 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2020-11-20T12:11:32.555 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} TASK [testnode : Create /var/lib/ceph] ***************************************** ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T12:11:32.599 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:32 +0000 (0:00:00.082) 0:02:05.188 ******* 2020-11-20T12:11:32.642 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:32 +0000 (0:00:00.043) 0:02:05.232 ******* 2020-11-20T12:11:32.649 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2020-11-20T12:11:32.694 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:32 +0000 (0:00:00.051) 0:02:05.284 ******* 2020-11-20T12:11:33.074 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2020-11-20T12:11:33.076 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true} 2020-11-20T12:11:33.121 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:33 +0000 (0:00:00.427) 0:02:05.711 ******* 2020-11-20T12:11:33.128 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true} 2020-11-20T12:11:33.172 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:33 +0000 (0:00:00.049) 0:02:05.761 ******* 2020-11-20T12:11:33.571 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-20T12:11:33.613 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:33 +0000 (0:00:00.441) 0:02:06.203 ******* 2020-11-20T12:11:33.619 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2020-11-20T12:11:33.664 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:33 +0000 (0:00:00.051) 0:02:06.254 ******* 2020-11-20T12:11:33.819 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2020-11-20T12:11:33.821 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-20T12:11:33.867 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:33 +0000 (0:00:00.202) 0:02:06.457 ******* 2020-11-20T12:11:33.873 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 1000} 2020-11-20T12:11:33.920 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:33 +0000 (0:00:00.052) 0:02:06.510 ******* 2020-11-20T12:11:34.043 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2020-11-20T12:11:34.045 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T12:11:34.102 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:34 +0000 (0:00:00.182) 0:02:06.692 ******* 2020-11-20T12:11:34.110 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/", "size": 4096, "state": "directory", "uid": 0} 2020-11-20T12:11:34.164 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:34 +0000 (0:00:00.062) 0:02:06.754 ******* 2020-11-20T12:11:34.633 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2020-11-20T12:11:34.922 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} Friday 20 November 2020 12:11:34 +0000 (0:00:00.572) 0:02:07.327 ******* ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "gid": 1236, "group": "ubuntu", "mode": "0755", "owner": "ubuntu", "path": "/home/ubuntu/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 1000} Friday 20 November 2020 12:11:34 +0000 (0:00:00.056) 0:02:07.383 ******* 2020-11-20T12:11:35.240 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2020-11-20T12:11:35.243 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-20T12:11:35.285 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:35 +0000 (0:00:00.492) 0:02:07.875 ******* 2020-11-20T12:11:35.292 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "dcd55d202d82b6e06d0555630d7a8b92d7100c36", "dest": "/root/.cpan/CPAN/MyConfig.pm", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/root/.cpan/CPAN/MyConfig.pm", "size": 2117, "state": "file", "uid": 0} 2020-11-20T12:11:35.336 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:35 +0000 (0:00:00.050) 0:02:07.926 ******* 2020-11-20T12:11:36.361 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** ok: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": false} 2020-11-20T12:11:36.405 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:36 +0000 (0:00:01.069) 0:02:08.996 ******* 2020-11-20T12:11:36.412 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": false} 2020-11-20T12:11:36.458 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:36 +0000 (0:00:00.052) 0:02:09.048 ******* 2020-11-20T12:11:36.687 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2020-11-20T12:11:36.689 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.058991", "end": "2020-11-20 12:11:36.652567", "rc": 0, "start": "2020-11-20 12:11:36.593576", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-20T12:11:36.731 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:36 +0000 (0:00:00.273) 0:02:09.321 ******* 2020-11-20T12:11:36.758 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "cmd": ["perldoc", "-l", "Amazon::S3"], "delta": "0:00:00.060515", "end": "2020-11-20 12:11:36.610117", "rc": 0, "start": "2020-11-20 12:11:36.549602", "stderr": "", "stderr_lines": [], "stdout": "/usr/local/share/perl/5.26.1/Amazon/S3.pm", "stdout_lines": ["/usr/local/share/perl/5.26.1/Amazon/S3.pm"]} 2020-11-20T12:11:36.760 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:36.800 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:36 +0000 (0:00:00.069) 0:02:09.390 ******* 2020-11-20T12:11:36.840 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:36 +0000 (0:00:00.040) 0:02:09.430 ******* 2020-11-20T12:11:36.846 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:36.890 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:36 +0000 (0:00:00.049) 0:02:09.480 ******* 2020-11-20T12:11:36.896 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2020-11-20T12:11:36.898 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:36.949 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:36 +0000 (0:00:00.058) 0:02:09.539 ******* 2020-11-20T12:11:36.955 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:37.000 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:36 +0000 (0:00:00.051) 0:02:09.590 ******* 2020-11-20T12:11:37.946 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-20T12:11:37.947 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": false} 2020-11-20T12:11:37.993 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:37 +0000 (0:00:00.993) 0:02:10.583 ******* 2020-11-20T12:11:38.027 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-20T12:11:38.029 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:38.071 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:38 +0000 (0:00:00.077) 0:02:10.661 ******* 2020-11-20T12:11:38.077 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2020-11-20T12:11:38.078 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": false} 2020-11-20T12:11:38.122 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:38 +0000 (0:00:00.051) 0:02:10.712 ******* 2020-11-20T12:11:38.149 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2020-11-20T12:11:38.151 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:38.191 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:38 +0000 (0:00:00.068) 0:02:10.781 ******* 2020-11-20T12:11:38.513 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-20T12:11:38.515 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-20T12:11:38.557 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:38 +0000 (0:00:00.366) 0:02:11.148 ******* 2020-11-20T12:11:38.584 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-20T12:11:38.586 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:38.627 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:38 +0000 (0:00:00.069) 0:02:11.217 ******* 2020-11-20T12:11:38.644 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2020-11-20T12:11:38.646 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "size": 2767, "state": "file", "uid": 0} 2020-11-20T12:11:38.693 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:38 +0000 (0:00:00.066) 0:02:11.283 ******* 2020-11-20T12:11:38.720 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2020-11-20T12:11:38.722 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:38.762 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:38 +0000 (0:00:00.068) 0:02:11.352 ******* 2020-11-20T12:11:38.925 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2020-11-20T12:11:38.929 INFO:teuthology.task.ansible.out:[WARNING]: The service (ntp) is actually an init script but the system is managed by systemd fatal: [smithi173.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring 2020-11-20T12:11:38.977 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:38 +0000 (0:00:00.212) 0:02:11.565 ******* 2020-11-20T12:11:39.002 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:11:39.004 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-20T12:11:39.046 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:39 +0000 (0:00:00.070) 0:02:11.636 ******* 2020-11-20T12:11:39.072 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:11:39.074 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Make sure ntpd is running.] *********************************** fatal: [smithi116.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Service is in unknown state", "status": {}} ...ignoring 2020-11-20T12:11:39.115 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:39 +0000 (0:00:00.069) 0:02:11.705 ******* 2020-11-20T12:11:39.154 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:39 +0000 (0:00:00.039) 0:02:11.745 ******* 2020-11-20T12:11:39.161 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2020-11-20T12:11:39.163 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2020-11-20T12:11:39.203 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:39 +0000 (0:00:00.049) 0:02:11.794 ******* 2020-11-20T12:11:39.210 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2020-11-20T12:11:39.212 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:39.254 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:39 +0000 (0:00:00.051) 0:02:11.845 ******* 2020-11-20T12:11:39.261 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:39.306 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:39 +0000 (0:00:00.051) 0:02:11.896 ******* 2020-11-20T12:11:39.343 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:39.386 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:39 +0000 (0:00:00.079) 0:02:11.976 ******* 2020-11-20T12:11:40.310 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-20T12:11:40.312 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"cache_update_time": 1605874190, "cache_updated": false, "changed": false} 2020-11-20T12:11:40.362 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:40 +0000 (0:00:00.975) 0:02:12.951 ******* 2020-11-20T12:11:40.392 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:40.443 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:40 +0000 (0:00:00.082) 0:02:13.033 ******* 2020-11-20T12:11:40.451 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2020-11-20T12:11:40.453 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"cache_update_time": 1605874191, "cache_updated": false, "changed": false} 2020-11-20T12:11:40.498 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:40 +0000 (0:00:00.054) 0:02:13.088 ******* 2020-11-20T12:11:40.525 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2020-11-20T12:11:40.527 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:40.568 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:40 +0000 (0:00:00.069) 0:02:13.158 ******* 2020-11-20T12:11:40.620 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-20T12:11:40.679 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:40 +0000 (0:00:00.110) 0:02:13.269 ******* 2020-11-20T12:11:40.748 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "size": 4096, "state": "directory", "uid": 1000} 2020-11-20T12:11:40.792 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:40 +0000 (0:00:00.113) 0:02:13.382 ******* 2020-11-20T12:11:41.162 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-20T12:11:41.208 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:41 +0000 (0:00:00.414) 0:02:13.796 ******* 2020-11-20T12:11:41.291 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1236, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "size": 73, "state": "file", "uid": 1000} 2020-11-20T12:11:41.293 INFO:teuthology.task.ansible.out: 2020-11-20T12:11:41.341 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:41 +0000 (0:00:00.135) 0:02:13.931 ******* 2020-11-20T12:11:42.476 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2020-11-20T12:11:42.478 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-20T12:11:42.521 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:42 +0000 (0:00:01.179) 0:02:15.111 ******* 2020-11-20T12:11:42.528 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "cmd": ["/usr/bin/pip", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "", "stderr_lines": [], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages\nRequirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)\nRequirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python2.7/dist-packages", "Requirement already satisfied: execnet in /usr/local/lib/python2.7/dist-packages (from remoto>=0.0.35)", "Requirement already satisfied: apipkg>=1.4 in /usr/local/lib/python2.7/dist-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2020-11-20T12:11:42.574 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:42 +0000 (0:00:00.053) 0:02:15.164 ******* 2020-11-20T12:11:43.117 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2020-11-20T12:11:43.119 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false} 2020-11-20T12:11:43.161 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:43 +0000 (0:00:00.586) 0:02:15.751 ******* 2020-11-20T12:11:43.188 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false} 2020-11-20T12:11:43.189 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-20T12:11:43.231 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:43 +0000 (0:00:00.070) 0:02:15.821 ******* 2020-11-20T12:11:43.270 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:43 +0000 (0:00:00.039) 0:02:15.860 ******* 2020-11-20T12:11:43.277 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2020-11-20T12:11:43.324 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:43 +0000 (0:00:00.053) 0:02:15.914 ******* 2020-11-20T12:11:43.468 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2020-11-20T12:11:43.470 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1605873908.5458958, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1605873908.5418959, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 2817, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1605873908.5418959, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "2282523401", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-20T12:11:43.511 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:43 +0000 (0:00:00.187) 0:02:16.102 ******* 2020-11-20T12:11:43.529 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "stat": {"atime": 1605873892.179405, "attr_flags": "e", "attributes": ["extents"], "block_size": 4096, "blocks": 8, "charset": "us-ascii", "ctime": 1605873892.1754048, "dev": 2049, "device_type": 0, "executable": false, "exists": true, "gid": 0, "gr_name": "root", "inode": 2817, "isblk": false, "ischr": false, "isdir": false, "isfifo": false, "isgid": false, "islnk": false, "isreg": true, "issock": false, "isuid": false, "mimetype": "text/plain", "mode": "0644", "mtime": 1605873892.1754048, "nlink": 1, "path": "/etc/network/interfaces", "pw_name": "root", "readable": true, "rgrp": true, "roth": true, "rusr": true, "size": 71, "uid": 0, "version": "2282523401", "wgrp": false, "woth": false, "writeable": true, "wusr": true, "xgrp": false, "xoth": false, "xusr": false}} 2020-11-20T12:11:43.574 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:43 +0000 (0:00:00.061) 0:02:16.163 ******* 2020-11-20T12:11:43.926 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2020-11-20T12:11:43.928 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => {"changed": false, "msg": ""} ok: [smithi173.front.sepia.ceph.com] => {"changed": false, "msg": ""} 2020-11-20T12:11:43.969 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:43 +0000 (0:00:00.395) 0:02:16.559 ******* 2020-11-20T12:11:44.008 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.039) 0:02:16.599 ******* 2020-11-20T12:11:44.015 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi116.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-20T12:11:44.057 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.048) 0:02:16.648 ******* 2020-11-20T12:11:44.064 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2020-11-20T12:11:44.106 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.048) 0:02:16.696 ******* 2020-11-20T12:11:44.112 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2020-11-20T12:11:44.113 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.154 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.048) 0:02:16.744 ******* 2020-11-20T12:11:44.161 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.205 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.049) 0:02:16.794 ******* 2020-11-20T12:11:44.333 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2020-11-20T12:11:44.334 INFO:teuthology.task.ansible.out:changed: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-20T12:11:44.376 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.171) 0:02:16.966 ******* 2020-11-20T12:11:44.383 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2020-11-20T12:11:44.430 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.054) 0:02:17.021 ******* 2020-11-20T12:11:44.567 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi116.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-20T12:11:44.616 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.185) 0:02:17.206 ******* 2020-11-20T12:11:44.645 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-20T12:11:44.647 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi173.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2020-11-20T12:11:44.688 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.071) 0:02:17.278 ******* 2020-11-20T12:11:44.731 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.040) 0:02:17.319 ******* 2020-11-20T12:11:44.747 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi116.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.749 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.749 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.749 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2020-11-20T12:11:44.750 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.750 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi116.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.793 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.064) 0:02:17.383 ******* 2020-11-20T12:11:44.834 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.040) 0:02:17.424 ******* 2020-11-20T12:11:44.841 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-20T12:11:44.842 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.854 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.856 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi116.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.899 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.065) 0:02:17.489 ******* 2020-11-20T12:11:44.905 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2020-11-20T12:11:44.907 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.908 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2020-11-20T12:11:44.950 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:44 +0000 (0:00:00.051) 0:02:17.540 ******* 2020-11-20T12:11:44.957 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.006 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.055) 0:02:17.596 ******* 2020-11-20T12:11:45.013 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2020-11-20T12:11:45.015 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi173.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.015 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.017 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.066 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.060) 0:02:17.656 ******* 2020-11-20T12:11:45.073 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.123 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.056) 0:02:17.713 ******* 2020-11-20T12:11:45.130 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.175 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.052) 0:02:17.765 ******* 2020-11-20T12:11:45.183 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.242 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.065) 0:02:17.831 ******* 2020-11-20T12:11:45.251 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2020-11-20T12:11:45.252 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.295 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.054) 0:02:17.886 ******* 2020-11-20T12:11:45.302 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.365 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.069) 0:02:17.955 ******* 2020-11-20T12:11:45.371 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.419 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.054) 0:02:18.010 ******* 2020-11-20T12:11:45.426 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2020-11-20T12:11:45.428 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.475 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.055) 0:02:18.065 ******* 2020-11-20T12:11:45.482 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2020-11-20T12:11:45.485 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.526 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.050) 0:02:18.116 ******* 2020-11-20T12:11:45.532 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-20T12:11:45.534 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.577 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.050) 0:02:18.167 ******* 2020-11-20T12:11:45.584 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.630 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.052) 0:02:18.220 ******* 2020-11-20T12:11:45.636 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2020-11-20T12:11:45.638 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.681 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.051) 0:02:18.271 ******* 2020-11-20T12:11:45.687 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-20T12:11:45.689 INFO:teuthology.task.ansible.out:skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.732 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.051) 0:02:18.322 ******* 2020-11-20T12:11:45.739 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2020-11-20T12:11:45.740 INFO:teuthology.task.ansible.out:skipping: [smithi116.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.782 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.049) 0:02:18.372 ******* 2020-11-20T12:11:45.789 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2020-11-20T12:11:45.836 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:45 +0000 (0:00:00.054) 0:02:18.427 ******* 2020-11-20T12:11:46.064 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi173.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} Friday 20 November 2020 12:11:45 +0000 (0:00:00.075) 0:02:18.502 ******* TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi116.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.022861", "end": "2020-11-20 12:11:45.843700", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-20 12:11:45.820839", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:46.122 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.023658", "end": "2020-11-20 12:11:46.081711", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2020-11-20 12:11:46.058053", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:46.185 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.015199", "end": "2020-11-20 12:11:46.028127", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-20 12:11:46.012928", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:46.302 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.014740", "end": "2020-11-20 12:11:46.259646", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2020-11-20 12:11:46.244906", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:46.366 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.013256", "end": "2020-11-20 12:11:46.208046", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 12:11:46.194790", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:46.471 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.014498", "end": "2020-11-20 12:11:46.433524", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2020-11-20 12:11:46.419026", "stderr": "find: '/etc/ceph': No such file or directory", "stderr_lines": ["find: '/etc/ceph': No such file or directory"], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:46.560 INFO:teuthology.task.ansible.out:ok: [smithi116.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.039008", "end": "2020-11-20 12:11:46.409710", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-20 12:11:46.370702", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:46.613 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:46 +0000 (0:00:00.700) 0:02:19.202 ******* 2020-11-20T12:11:46.640 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-20T12:11:46.682 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:46 +0000 (0:00:00.070) 0:02:19.273 ******* 2020-11-20T12:11:46.689 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2020-11-20T12:11:46.691 INFO:teuthology.task.ansible.out:ok: [smithi173.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.027529", "end": "2020-11-20 12:11:46.624351", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2020-11-20 12:11:46.596822", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2020-11-20T12:11:46.740 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:46 +0000 (0:00:00.054) 0:02:19.327 ******* 2020-11-20T12:11:46.765 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2020-11-20T12:11:46.815 INFO:teuthology.task.ansible.out:Friday 20 November 2020 12:11:46 +0000 (0:00:00.076) 0:02:19.403 ******* 2020-11-20T12:11:46.864 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** changed: [smithi116.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-20T12:11:47.011 INFO:teuthology.task.ansible.out:changed: [smithi173.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "size": 0, "state": "file", "uid": 0} 2020-11-20T12:11:47.124 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2020-11-20T12:11:47.129 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2020-11-20T12:11:47.133 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi116.front.sepia.ceph.com : ok=114 changed=30 unreachable=0 failed=0 skipped=182 rescued=0 ignored=4 smithi173.front.sepia.ceph.com : ok=114 changed=30 unreachable=0 failed=0 skipped=182 rescued=0 ignored=4 Friday 20 November 2020 12:11:47 +0000 (0:00:00.320) 0:02:19.723 ******* =============================================================================== 2020-11-20T12:11:47.135 INFO:teuthology.task.ansible.out:testnode : Ensure packages are not present. ---------------------------- 11.76s testnode : Install packages -------------------------------------------- 10.50s testnode : Blow away lingering OSD data and FSIDs ---------------------- 10.13s testnode : Zap all non-root disks --------------------------------------- 8.60s common : Update apt cache ----------------------------------------------- 8.00s common : Check for docker ----------------------------------------------- 4.26s testnode : Install apt keys --------------------------------------------- 3.94s testnode : Update apt cache. -------------------------------------------- 3.91s Gathering Facts --------------------------------------------------------- 2.59s testnode : Create logical volume(s) ------------------------------------- 2.15s Gathering Facts --------------------------------------------------------- 1.51s testnode : Install packages via pip ------------------------------------- 1.18s testnode : Remove all LVM data ------------------------------------------ 1.08s testnode : Ensure perl-doc is installed on apt systems. ----------------- 1.07s testnode : Install python-apt ------------------------------------------- 1.02s testnode : Install ntp package on apt based systems. -------------------- 0.99s testnode : Install python-pip on apt based systems. --------------------- 0.98s testnode : Install non aarch64 packages --------------------------------- 0.89s users : Create all admin users with sudo access. ------------------------ 0.88s testnode : Upgrade packages --------------------------------------------- 0.87s 2020-11-20T12:11:47.510 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi116.front.sepia.ceph.com'), Remote(name='ubuntu@smithi173.front.sepia.ceph.com')] 2020-11-20T12:11:47.511 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T12:11:47.512 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi116.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:11:47.564 INFO:teuthology.orchestra.run.smithi116:> true 2020-11-20T12:11:47.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2020-11-20T12:11:47.868 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi173.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2020-11-20T12:11:47.953 INFO:teuthology.orchestra.run.smithi173:> true 2020-11-20T12:11:48.238 INFO:teuthology.run_tasks:Running task clock... 2020-11-20T12:11:48.253 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2020-11-20T12:11:48.253 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-20T12:11:48.254 INFO:teuthology.orchestra.run.smithi116:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-20T12:11:48.335 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-20T12:11:48.336 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: Command line: ntpd -gq 2020-11-20T12:11:48.339 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: proto: precision = 0.041 usec (-24) 2020-11-20T12:11:48.340 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-20T12:11:48.341 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: restrict ::: KOD does nothing without LIMITED. 2020-11-20T12:11:48.342 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: Listen and drop on 0 v6wildcard [::]:123 2020-11-20T12:11:48.342 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-20T12:11:48.343 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: Listen normally on 2 lo 127.0.0.1:123 2020-11-20T12:11:48.343 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: Listen normally on 3 enp3s0f1 172.21.15.116:123 2020-11-20T12:11:48.343 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: Listen normally on 4 lo [::1]:123 2020-11-20T12:11:48.343 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:3f67%5]:123 2020-11-20T12:11:48.344 INFO:teuthology.orchestra.run.smithi116.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-20T12:11:48.344 INFO:teuthology.orchestra.run.smithi116.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-20T12:11:48.345 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:48 ntpd[5285]: Listening on routing socket on fd #22 for interface updates 2020-11-20T12:11:50.336 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:50 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:50.337 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:50 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:50.337 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:50 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:50.337 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:50 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:50.337 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:50 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:50.338 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:50 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:50.338 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:50 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:50.338 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:50 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:51.335 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:51 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:51.335 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:51 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:51.335 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:51 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:51.335 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:51 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:51.336 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:51 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:51.337 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:51 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:51.337 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:51 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:51.337 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:51 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:52.334 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:52 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:52.335 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:52 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:52.335 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:52 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:52.336 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:52 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:52.336 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:52 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:52.336 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:52 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:52.336 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:52 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:52.337 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:52 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:53 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:53 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:53 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:53 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:53 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:53 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:53 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:53 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:54.333 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:54 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:54.333 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:54 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:54.334 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:54 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:54.334 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:54 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:54.335 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:54 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:54.335 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:54 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:54.335 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:54 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:54.335 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:54 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:55 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:55 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:55.333 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:55 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:55.334 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:55 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:55.334 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:55 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:55.334 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:55 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:55.334 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:55 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:55.335 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:55 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:56.332 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:56 ntpd[5285]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:56.332 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:56 ntpd[5285]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:56.333 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:56 ntpd[5285]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:56.333 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:56 ntpd[5285]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:56.334 INFO:teuthology.orchestra.run.smithi116.stdout:20 Nov 12:11:56 ntpd[5285]: ntpd: time set +0.102489 s 2020-11-20T12:11:56.334 INFO:teuthology.orchestra.run.smithi116.stdout:ntpd: time set +0.102489s 2020-11-20T12:11:56.334 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:56 ntpd[5285]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-20T12:11:56.334 INFO:teuthology.orchestra.run.smithi116.stderr:20 Nov 12:11:56 ntpd[5285]: can't open /var/log/ntpstats/loopstats.20201120: Permission denied 2020-11-20T12:11:56.426 INFO:teuthology.orchestra.run.smithi116.stdout: remote refid st t when poll reach delay offset jitter 2020-11-20T12:11:56.427 INFO:teuthology.orchestra.run.smithi116.stdout:============================================================================== 2020-11-20T12:11:56.427 INFO:teuthology.orchestra.run.smithi116.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-20T12:11:56.427 INFO:teuthology.orchestra.run.smithi116.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-20T12:11:56.428 INFO:teuthology.orchestra.run.smithi116.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-20T12:11:56.428 INFO:teuthology.orchestra.run.smithi116.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-20T12:11:56.429 INFO:teuthology.orchestra.run:Running command with timeout 360 2020-11-20T12:11:56.429 INFO:teuthology.orchestra.run.smithi173:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2020-11-20T12:11:56.506 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: ntpd 4.2.8p10@1.3728-o (1): Starting 2020-11-20T12:11:56.506 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: Command line: ntpd -gq 2020-11-20T12:11:56.507 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: proto: precision = 0.090 usec (-23) 2020-11-20T12:11:56.508 INFO:teuthology.orchestra.run.smithi173.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-20T12:11:56.508 INFO:teuthology.orchestra.run.smithi173.stderr:restrict ::: KOD does nothing without LIMITED. 2020-11-20T12:11:56.509 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2020-11-20T12:11:56.509 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: restrict ::: KOD does nothing without LIMITED. 2020-11-20T12:11:56.509 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: Listen and drop on 0 v6wildcard [::]:123 2020-11-20T12:11:56.509 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2020-11-20T12:11:56.510 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: Listen normally on 2 lo 127.0.0.1:123 2020-11-20T12:11:56.510 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: Listen normally on 3 enp3s0f1 172.21.15.173:123 2020-11-20T12:11:56.510 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: Listen normally on 4 lo [::1]:123 2020-11-20T12:11:56.511 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:7259%5]:123 2020-11-20T12:11:56.511 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:11:56 ntpd[5284]: Listening on routing socket on fd #22 for interface updates 2020-11-20T12:11:57.508 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:57 ntpd[5284]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:57.508 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:57 ntpd[5284]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:57.508 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:57 ntpd[5284]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:57.509 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:57 ntpd[5284]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:57.509 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:57 ntpd[5284]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:57.509 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:57 ntpd[5284]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:57.509 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:57 ntpd[5284]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:57.510 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:57 ntpd[5284]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:59.508 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:59.508 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:59.509 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:59.509 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:59.509 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:59.509 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:59.510 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:59.510 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:11:59.511 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:11:59.511 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:11:59.511 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:11:59.511 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:11:59 ntpd[5284]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:12:01.510 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:12:01.511 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:12:01.511 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:12:01.511 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:12:01.511 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:12:01.511 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:12:01.512 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:12:01.512 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:12:01.512 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:12:01.512 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:12:01.513 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:12:01.513 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:12:01.513 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:12:01.513 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:12:01.513 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:12:01.514 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:01 ntpd[5284]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:12:03.509 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:03 ntpd[5284]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2020-11-20T12:12:03.510 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:03 ntpd[5284]: can't open /var/log/ntpstats/rawstats.20201120: Permission denied 2020-11-20T12:12:03.510 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:03 ntpd[5284]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2020-11-20T12:12:03.510 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:03 ntpd[5284]: can't open /var/log/ntpstats/peerstats.20201120: Permission denied 2020-11-20T12:12:03.511 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:03 ntpd[5284]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2020-11-20T12:12:03.511 INFO:teuthology.orchestra.run.smithi173.stdout:20 Nov 12:12:03 ntpd[5284]: ntpd: time slew +0.008902 s 2020-11-20T12:12:03.511 INFO:teuthology.orchestra.run.smithi173.stdout:ntpd: time slew +0.008902s 2020-11-20T12:12:03.512 INFO:teuthology.orchestra.run.smithi173.stderr:20 Nov 12:12:03 ntpd[5284]: can't open /var/log/ntpstats/loopstats.20201120: Permission denied 2020-11-20T12:12:03.614 INFO:teuthology.orchestra.run.smithi173.stdout: remote refid st t when poll reach delay offset jitter 2020-11-20T12:12:03.614 INFO:teuthology.orchestra.run.smithi173.stdout:============================================================================== 2020-11-20T12:12:03.615 INFO:teuthology.orchestra.run.smithi173.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-20T12:12:03.615 INFO:teuthology.orchestra.run.smithi173.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-20T12:12:03.615 INFO:teuthology.orchestra.run.smithi173.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-20T12:12:03.615 INFO:teuthology.orchestra.run.smithi173.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2020-11-20T12:12:03.616 INFO:teuthology.run_tasks:Running task install... 2020-11-20T12:12:03.626 DEBUG:teuthology.task.install:project ceph 2020-11-20T12:12:03.626 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'}} 2020-11-20T12:12:03.627 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'} 2020-11-20T12:12:03.627 INFO:teuthology.task.install:Using flavor: basic 2020-11-20T12:12:03.633 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2020-11-20T12:12:03.634 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']} 2020-11-20T12:12:03.634 INFO:teuthology.orchestra.run.smithi116:> sudo lsb_release -is 2020-11-20T12:12:03.711 INFO:teuthology.orchestra.run.smithi116.stdout:Ubuntu 2020-11-20T12:12:03.719 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-20T12:12:03.719 INFO:teuthology.orchestra.run.smithi173:> sudo lsb_release -is 2020-11-20T12:12:03.720 INFO:teuthology.orchestra.run.smithi116:> sudo apt-key list | grep Ceph 2020-11-20T12:12:03.791 INFO:teuthology.orchestra.run.smithi173.stdout:Ubuntu 2020-11-20T12:12:03.800 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-20T12:12:03.801 INFO:teuthology.orchestra.run.smithi173:> sudo apt-key list | grep Ceph 2020-11-20T12:12:03.916 INFO:teuthology.orchestra.run.smithi116.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-20T12:12:03.995 INFO:teuthology.orchestra.run.smithi173.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2020-11-20T12:12:04.050 INFO:teuthology.orchestra.run.smithi116.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-20T12:12:04.050 INFO:teuthology.orchestra.run.smithi116.stdout:uid [ unknown] Ceph.com (release key) 2020-11-20T12:12:04.052 INFO:teuthology.orchestra.run.smithi116:> sudo lsb_release -is 2020-11-20T12:12:04.122 INFO:teuthology.orchestra.run.smithi173.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2020-11-20T12:12:04.122 INFO:teuthology.orchestra.run.smithi173.stdout:uid [ unknown] Ceph.com (release key) 2020-11-20T12:12:04.123 INFO:teuthology.orchestra.run.smithi116.stdout:Ubuntu 2020-11-20T12:12:04.124 INFO:teuthology.orchestra.run.smithi173:> sudo lsb_release -is 2020-11-20T12:12:04.130 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-20T12:12:04.130 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-20T12:12:04.130 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-20T12:12:04.131 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T12:12:04.192 INFO:teuthology.orchestra.run.smithi173.stdout:Ubuntu 2020-11-20T12:12:04.200 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-20T12:12:04.201 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell on remote deb x86_64 2020-11-20T12:12:04.201 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev on remote deb x86_64 2020-11-20T12:12:04.201 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T12:12:04.294 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default/ 2020-11-20T12:12:04.295 INFO:teuthology.task.install.deb:Package version is 16.0.0-7397-gb86e691b-1bionic 2020-11-20T12:12:04.374 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default/ 2020-11-20T12:12:04.374 INFO:teuthology.task.install.deb:Package version is 16.0.0-7397-gb86e691b-1bionic 2020-11-20T12:12:04.421 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:12:04.422 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-20T12:12:04.436 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-20T12:12:04.449 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:12:04.450 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:12:04.450 INFO:teuthology.orchestra.run.smithi116.stderr:172 bytes copied, 0.000152587 s, 1.1 MB/s 2020-11-20T12:12:04.451 INFO:teuthology.orchestra.run.smithi116:> sudo apt-get update 2020-11-20T12:12:04.505 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:12:04.506 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-20T12:12:04.521 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/apt/sources.list.d/ceph.list 2020-11-20T12:12:04.535 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:12:04.536 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:12:04.536 INFO:teuthology.orchestra.run.smithi173.stderr:172 bytes copied, 0.000187197 s, 919 kB/s 2020-11-20T12:12:04.538 INFO:teuthology.orchestra.run.smithi173:> sudo apt-get update 2020-11-20T12:12:04.708 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2020-11-20T12:12:04.807 INFO:teuthology.orchestra.run.smithi116.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-20T12:12:05.079 INFO:teuthology.orchestra.run.smithi173.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic InRelease 2020-11-20T12:12:05.079 INFO:teuthology.orchestra.run.smithi173.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2020-11-20T12:12:05.079 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2020-11-20T12:12:05.080 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-20T12:12:05.080 INFO:teuthology.orchestra.run.smithi173.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-20T12:12:05.080 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.1 kB] 2020-11-20T12:12:05.080 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-11-20T12:12:05.080 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,964 B] 2020-11-20T12:12:05.081 INFO:teuthology.orchestra.run.smithi116.stdout:Ign:3 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic InRelease 2020-11-20T12:12:05.081 INFO:teuthology.orchestra.run.smithi116.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2020-11-20T12:12:05.081 INFO:teuthology.orchestra.run.smithi116.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic Release.gpg 2020-11-20T12:12:05.081 INFO:teuthology.orchestra.run.smithi116.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2020-11-20T12:12:05.082 INFO:teuthology.orchestra.run.smithi116.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 Packages [16.1 kB] 2020-11-20T12:12:05.082 INFO:teuthology.orchestra.run.smithi116.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main i386 Packages [2,964 B] 2020-11-20T12:12:05.145 INFO:teuthology.orchestra.run.smithi116.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-11-20T12:12:05.260 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB] 2020-11-20T12:12:06.001 INFO:teuthology.orchestra.run.smithi116.stdout:Fetched 190 kB in 1s (158 kB/s) 2020-11-20T12:12:06.277 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 279 kB in 1s (202 kB/s) 2020-11-20T12:12:07.547 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2020-11-20T12:12:07.613 INFO:teuthology.orchestra.run.smithi116:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-7397-gb86e691b-1bionic cephadm=16.0.0-7397-gb86e691b-1bionic ceph-mds=16.0.0-7397-gb86e691b-1bionic ceph-mgr=16.0.0-7397-gb86e691b-1bionic ceph-common=16.0.0-7397-gb86e691b-1bionic ceph-fuse=16.0.0-7397-gb86e691b-1bionic ceph-test=16.0.0-7397-gb86e691b-1bionic radosgw=16.0.0-7397-gb86e691b-1bionic python3-rados=16.0.0-7397-gb86e691b-1bionic python3-rgw=16.0.0-7397-gb86e691b-1bionic python3-cephfs=16.0.0-7397-gb86e691b-1bionic python3-rbd=16.0.0-7397-gb86e691b-1bionic libcephfs2=16.0.0-7397-gb86e691b-1bionic libcephfs-dev=16.0.0-7397-gb86e691b-1bionic librados2=16.0.0-7397-gb86e691b-1bionic librbd1=16.0.0-7397-gb86e691b-1bionic rbd-fuse=16.0.0-7397-gb86e691b-1bionic python3-cephfs=16.0.0-7397-gb86e691b-1bionic cephfs-shell=16.0.0-7397-gb86e691b-1bionic 2020-11-20T12:12:07.741 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2020-11-20T12:12:07.835 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2020-11-20T12:12:07.902 INFO:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.0.0-7397-gb86e691b-1bionic cephadm=16.0.0-7397-gb86e691b-1bionic ceph-mds=16.0.0-7397-gb86e691b-1bionic ceph-mgr=16.0.0-7397-gb86e691b-1bionic ceph-common=16.0.0-7397-gb86e691b-1bionic ceph-fuse=16.0.0-7397-gb86e691b-1bionic ceph-test=16.0.0-7397-gb86e691b-1bionic radosgw=16.0.0-7397-gb86e691b-1bionic python3-rados=16.0.0-7397-gb86e691b-1bionic python3-rgw=16.0.0-7397-gb86e691b-1bionic python3-cephfs=16.0.0-7397-gb86e691b-1bionic python3-rbd=16.0.0-7397-gb86e691b-1bionic libcephfs2=16.0.0-7397-gb86e691b-1bionic libcephfs-dev=16.0.0-7397-gb86e691b-1bionic librados2=16.0.0-7397-gb86e691b-1bionic librbd1=16.0.0-7397-gb86e691b-1bionic rbd-fuse=16.0.0-7397-gb86e691b-1bionic python3-cephfs=16.0.0-7397-gb86e691b-1bionic cephfs-shell=16.0.0-7397-gb86e691b-1bionic 2020-11-20T12:12:07.996 INFO:teuthology.orchestra.run.smithi116.stdout:Building dependency tree... 2020-11-20T12:12:07.996 INFO:teuthology.orchestra.run.smithi116.stdout:Reading state information... 2020-11-20T12:12:08.027 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2020-11-20T12:12:08.159 INFO:teuthology.orchestra.run.smithi116.stdout:The following additional packages will be installed: 2020-11-20T12:12:08.160 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2020-11-20T12:12:08.161 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2020-11-20T12:12:08.161 INFO:teuthology.orchestra.run.smithi116.stdout: libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-11-20T12:12:08.161 INFO:teuthology.orchestra.run.smithi116.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2020-11-20T12:12:08.161 INFO:teuthology.orchestra.run.smithi116.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2020-11-20T12:12:08.162 INFO:teuthology.orchestra.run.smithi116.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-20T12:12:08.162 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-20T12:12:08.162 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-20T12:12:08.162 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-20T12:12:08.162 INFO:teuthology.orchestra.run.smithi116.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-20T12:12:08.163 INFO:teuthology.orchestra.run.smithi116.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-20T12:12:08.163 INFO:teuthology.orchestra.run.smithi116.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-20T12:12:08.163 INFO:teuthology.orchestra.run.smithi116.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-20T12:12:08.163 INFO:teuthology.orchestra.run.smithi116.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-20T12:12:08.163 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-20T12:12:08.164 INFO:teuthology.orchestra.run.smithi116.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-20T12:12:08.164 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-20T12:12:08.164 INFO:teuthology.orchestra.run.smithi116.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-20T12:12:08.164 INFO:teuthology.orchestra.run.smithi116.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-20T12:12:08.164 INFO:teuthology.orchestra.run.smithi116.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-20T12:12:08.165 INFO:teuthology.orchestra.run.smithi116.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-20T12:12:08.165 INFO:teuthology.orchestra.run.smithi116.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-20T12:12:08.165 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-20T12:12:08.166 INFO:teuthology.orchestra.run.smithi116.stdout:Suggested packages: 2020-11-20T12:12:08.166 INFO:teuthology.orchestra.run.smithi116.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-20T12:12:08.166 INFO:teuthology.orchestra.run.smithi116.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-20T12:12:08.166 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-20T12:12:08.167 INFO:teuthology.orchestra.run.smithi116.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-20T12:12:08.167 INFO:teuthology.orchestra.run.smithi116.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-20T12:12:08.167 INFO:teuthology.orchestra.run.smithi116.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-20T12:12:08.167 INFO:teuthology.orchestra.run.smithi116.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-20T12:12:08.167 INFO:teuthology.orchestra.run.smithi116.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-20T12:12:08.168 INFO:teuthology.orchestra.run.smithi116.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-20T12:12:08.168 INFO:teuthology.orchestra.run.smithi116.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-20T12:12:08.168 INFO:teuthology.orchestra.run.smithi116.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-20T12:12:08.168 INFO:teuthology.orchestra.run.smithi116.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-20T12:12:08.168 INFO:teuthology.orchestra.run.smithi116.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-20T12:12:08.169 INFO:teuthology.orchestra.run.smithi116.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-20T12:12:08.169 INFO:teuthology.orchestra.run.smithi116.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-20T12:12:08.285 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2020-11-20T12:12:08.286 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2020-11-20T12:12:08.345 INFO:teuthology.orchestra.run.smithi116.stdout:The following NEW packages will be installed: 2020-11-20T12:12:08.345 INFO:teuthology.orchestra.run.smithi116.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-20T12:12:08.346 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2020-11-20T12:12:08.346 INFO:teuthology.orchestra.run.smithi116.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm 2020-11-20T12:12:08.347 INFO:teuthology.orchestra.run.smithi116.stdout: cephfs-shell jq libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 2020-11-20T12:12:08.347 INFO:teuthology.orchestra.run.smithi116.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-20T12:12:08.348 INFO:teuthology.orchestra.run.smithi116.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-20T12:12:08.348 INFO:teuthology.orchestra.run.smithi116.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-20T12:12:08.348 INFO:teuthology.orchestra.run.smithi116.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-20T12:12:08.348 INFO:teuthology.orchestra.run.smithi116.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2020-11-20T12:12:08.349 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cffi-backend python3-cherrypy3 python3-cmd2 python3-colorama 2020-11-20T12:12:08.349 INFO:teuthology.orchestra.run.smithi116.stdout: python3-cryptography python3-cycler python3-dateutil python3-decorator 2020-11-20T12:12:08.349 INFO:teuthology.orchestra.run.smithi116.stdout: python3-fasteners python3-html5lib python3-httplib2 python3-jinja2 2020-11-20T12:12:08.350 INFO:teuthology.orchestra.run.smithi116.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-jwt 2020-11-20T12:12:08.350 INFO:teuthology.orchestra.run.smithi116.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-20T12:12:08.350 INFO:teuthology.orchestra.run.smithi116.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-20T12:12:08.351 INFO:teuthology.orchestra.run.smithi116.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-20T12:12:08.351 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-20T12:12:08.351 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-20T12:12:08.351 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-20T12:12:08.352 INFO:teuthology.orchestra.run.smithi116.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-20T12:12:08.352 INFO:teuthology.orchestra.run.smithi116.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-20T12:12:08.352 INFO:teuthology.orchestra.run.smithi116.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-20T12:12:08.353 INFO:teuthology.orchestra.run.smithi116.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-20T12:12:08.353 INFO:teuthology.orchestra.run.smithi116.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-20T12:12:08.353 INFO:teuthology.orchestra.run.smithi116.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-20T12:12:08.353 INFO:teuthology.orchestra.run.smithi116.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-20T12:12:08.354 INFO:teuthology.orchestra.run.smithi116.stdout:The following packages will be upgraded: 2020-11-20T12:12:08.354 INFO:teuthology.orchestra.run.smithi116.stdout: librados2 librbd1 2020-11-20T12:12:08.456 INFO:teuthology.orchestra.run.smithi173.stdout:The following additional packages will be installed: 2020-11-20T12:12:08.456 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2020-11-20T12:12:08.457 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd jq 2020-11-20T12:12:08.458 INFO:teuthology.orchestra.run.smithi173.stdout: libjq1 libleveldb1v5 liblttng-ust0 liboath0 libonig4 librabbitmq4 2020-11-20T12:12:08.458 INFO:teuthology.orchestra.run.smithi173.stdout: libradosstriper1 librdkafka1 librgw2 libsnappy1v5 libxslt1.1 nvme-cli 2020-11-20T12:12:08.458 INFO:teuthology.orchestra.run.smithi173.stdout: python-attr python-funcsigs python-pastedeploy-tpl python-pluggy python-py 2020-11-20T12:12:08.458 INFO:teuthology.orchestra.run.smithi173.stdout: python-pytest python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-20T12:12:08.459 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2020-11-20T12:12:08.459 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-cryptography 2020-11-20T12:12:08.459 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cycler python3-dateutil python3-decorator python3-fasteners 2020-11-20T12:12:08.459 INFO:teuthology.orchestra.run.smithi173.stdout: python3-html5lib python3-httplib2 python3-jinja2 python3-joblib 2020-11-20T12:12:08.460 INFO:teuthology.orchestra.run.smithi173.stdout: python3-json-pointer python3-jsonpatch python3-jwt python3-kubernetes 2020-11-20T12:12:08.460 INFO:teuthology.orchestra.run.smithi173.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2020-11-20T12:12:08.460 INFO:teuthology.orchestra.run.smithi173.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2020-11-20T12:12:08.460 INFO:teuthology.orchestra.run.smithi173.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2020-11-20T12:12:08.460 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pastescript python3-pecan python3-pil python3-pluggy 2020-11-20T12:12:08.461 INFO:teuthology.orchestra.run.smithi173.stdout: python3-prettytable python3-py python3-pyasn1 python3-pyasn1-modules 2020-11-20T12:12:08.461 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyinotify python3-pyparsing python3-pyperclip python3-pytest 2020-11-20T12:12:08.461 INFO:teuthology.orchestra.run.smithi173.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2020-11-20T12:12:08.461 INFO:teuthology.orchestra.run.smithi173.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2020-11-20T12:12:08.461 INFO:teuthology.orchestra.run.smithi173.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2020-11-20T12:12:08.462 INFO:teuthology.orchestra.run.smithi173.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2020-11-20T12:12:08.462 INFO:teuthology.orchestra.run.smithi173.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2020-11-20T12:12:08.462 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2020-11-20T12:12:08.463 INFO:teuthology.orchestra.run.smithi173.stdout:Suggested packages: 2020-11-20T12:12:08.463 INFO:teuthology.orchestra.run.smithi173.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2020-11-20T12:12:08.463 INFO:teuthology.orchestra.run.smithi173.stdout: python-pytest-xdist python-mock python-cryptography-doc 2020-11-20T12:12:08.463 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2020-11-20T12:12:08.464 INFO:teuthology.orchestra.run.smithi173.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2020-11-20T12:12:08.464 INFO:teuthology.orchestra.run.smithi173.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2020-11-20T12:12:08.464 INFO:teuthology.orchestra.run.smithi173.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2020-11-20T12:12:08.464 INFO:teuthology.orchestra.run.smithi173.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2020-11-20T12:12:08.464 INFO:teuthology.orchestra.run.smithi173.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2020-11-20T12:12:08.465 INFO:teuthology.orchestra.run.smithi173.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2020-11-20T12:12:08.465 INFO:teuthology.orchestra.run.smithi173.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2020-11-20T12:12:08.465 INFO:teuthology.orchestra.run.smithi173.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2020-11-20T12:12:08.465 INFO:teuthology.orchestra.run.smithi173.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2020-11-20T12:12:08.466 INFO:teuthology.orchestra.run.smithi173.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2020-11-20T12:12:08.466 INFO:teuthology.orchestra.run.smithi173.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2020-11-20T12:12:08.466 INFO:teuthology.orchestra.run.smithi173.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2020-11-20T12:12:08.545 INFO:teuthology.orchestra.run.smithi116.stdout:2 upgraded, 114 newly installed, 0 to remove and 23 not upgraded. 2020-11-20T12:12:08.545 INFO:teuthology.orchestra.run.smithi116.stdout:Need to get 168 MB of archives. 2020-11-20T12:12:08.545 INFO:teuthology.orchestra.run.smithi116.stdout:After this operation, 782 MB of additional disk space will be used. 2020-11-20T12:12:08.546 INFO:teuthology.orchestra.run.smithi116.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-7397-gb86e691b-1bionic [1,921 kB] 2020-11-20T12:12:08.561 INFO:teuthology.orchestra.run.smithi116.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-20T12:12:08.625 INFO:teuthology.orchestra.run.smithi173.stdout:The following NEW packages will be installed: 2020-11-20T12:12:08.625 INFO:teuthology.orchestra.run.smithi173.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2020-11-20T12:12:08.625 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2020-11-20T12:12:08.626 INFO:teuthology.orchestra.run.smithi173.stdout: ceph-mgr-modules-core ceph-mgr-rook ceph-mon ceph-osd ceph-test cephadm 2020-11-20T12:12:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: cephfs-shell jq libcephfs-dev libcephfs2 libjq1 libleveldb1v5 liblttng-ust0 2020-11-20T12:12:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: liboath0 libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2020-11-20T12:12:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: libsnappy1v5 libxslt1.1 nvme-cli python-attr python-funcsigs 2020-11-20T12:12:08.627 INFO:teuthology.orchestra.run.smithi173.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2020-11-20T12:12:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2020-11-20T12:12:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2020-11-20T12:12:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cffi-backend python3-cherrypy3 python3-cmd2 python3-colorama 2020-11-20T12:12:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: python3-cryptography python3-cycler python3-dateutil python3-decorator 2020-11-20T12:12:08.628 INFO:teuthology.orchestra.run.smithi173.stdout: python3-fasteners python3-html5lib python3-httplib2 python3-jinja2 2020-11-20T12:12:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: python3-joblib python3-json-pointer python3-jsonpatch python3-jwt 2020-11-20T12:12:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2020-11-20T12:12:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2020-11-20T12:12:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2020-11-20T12:12:08.629 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2020-11-20T12:12:08.630 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2020-11-20T12:12:08.630 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pyperclip 2020-11-20T12:12:08.630 INFO:teuthology.orchestra.run.smithi173.stdout: python3-pytest python3-rados python3-rbd python3-repoze.lru python3-rgw 2020-11-20T12:12:08.630 INFO:teuthology.orchestra.run.smithi173.stdout: python3-routes python3-rsa python3-scipy python3-setuptools 2020-11-20T12:12:08.631 INFO:teuthology.orchestra.run.smithi173.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2020-11-20T12:12:08.631 INFO:teuthology.orchestra.run.smithi173.stdout: python3-sklearn python3-sklearn-lib python3-sqlalchemy python3-tempita 2020-11-20T12:12:08.631 INFO:teuthology.orchestra.run.smithi173.stdout: python3-tk python3-tz python3-uritemplate python3-waitress 2020-11-20T12:12:08.631 INFO:teuthology.orchestra.run.smithi173.stdout: python3-webencodings python3-webob python3-websocket python3-webtest 2020-11-20T12:12:08.631 INFO:teuthology.orchestra.run.smithi173.stdout: python3-werkzeug radosgw rbd-fuse socat xmlstarlet 2020-11-20T12:12:08.632 INFO:teuthology.orchestra.run.smithi173.stdout:The following packages will be upgraded: 2020-11-20T12:12:08.632 INFO:teuthology.orchestra.run.smithi173.stdout: librados2 librbd1 2020-11-20T12:12:08.737 INFO:teuthology.orchestra.run.smithi116.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-7397-gb86e691b-1bionic [3,160 kB] 2020-11-20T12:12:08.813 INFO:teuthology.orchestra.run.smithi116.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-20T12:12:08.817 INFO:teuthology.orchestra.run.smithi173.stdout:2 upgraded, 114 newly installed, 0 to remove and 23 not upgraded. 2020-11-20T12:12:08.817 INFO:teuthology.orchestra.run.smithi173.stdout:Need to get 168 MB of archives. 2020-11-20T12:12:08.817 INFO:teuthology.orchestra.run.smithi173.stdout:After this operation, 782 MB of additional disk space will be used. 2020-11-20T12:12:08.817 INFO:teuthology.orchestra.run.smithi173.stdout:Get:1 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.0.0-7397-gb86e691b-1bionic [1,921 kB] 2020-11-20T12:12:08.818 INFO:teuthology.orchestra.run.smithi116.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-20T12:12:08.820 INFO:teuthology.orchestra.run.smithi116.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-20T12:12:08.820 INFO:teuthology.orchestra.run.smithi116.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-20T12:12:08.839 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2020-11-20T12:12:08.879 INFO:teuthology.orchestra.run.smithi116.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-7397-gb86e691b-1bionic [481 kB] 2020-11-20T12:12:08.889 INFO:teuthology.orchestra.run.smithi116.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-7397-gb86e691b-1bionic [332 kB] 2020-11-20T12:12:08.908 INFO:teuthology.orchestra.run.smithi116.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-7397-gb86e691b-1bionic [20.6 kB] 2020-11-20T12:12:08.909 INFO:teuthology.orchestra.run.smithi116.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-7397-gb86e691b-1bionic [168 kB] 2020-11-20T12:12:08.912 INFO:teuthology.orchestra.run.smithi116.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-7397-gb86e691b-1bionic [27.7 kB] 2020-11-20T12:12:08.912 INFO:teuthology.orchestra.run.smithi116.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-7397-gb86e691b-1bionic [330 kB] 2020-11-20T12:12:08.932 INFO:teuthology.orchestra.run.smithi116.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-7397-gb86e691b-1bionic [3,179 kB] 2020-11-20T12:12:08.971 INFO:teuthology.orchestra.run.smithi116.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-20T12:12:08.981 INFO:teuthology.orchestra.run.smithi116.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-20T12:12:08.995 INFO:teuthology.orchestra.run.smithi116.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-20T12:12:09.059 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.0.0-7397-gb86e691b-1bionic [3,160 kB] 2020-11-20T12:12:09.079 INFO:teuthology.orchestra.run.smithi116.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-20T12:12:09.082 INFO:teuthology.orchestra.run.smithi116.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-20T12:12:09.090 INFO:teuthology.orchestra.run.smithi116.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-7397-gb86e691b-1bionic [99.1 kB] 2020-11-20T12:12:09.092 INFO:teuthology.orchestra.run.smithi116.stdout:Get:21 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-7397-gb86e691b-1bionic [414 kB] 2020-11-20T12:12:09.096 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyperclip all 1.6.0-1 [9,376 B] 2020-11-20T12:12:09.105 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cmd2 all 0.7.9-0ubuntu1 [34.2 kB] 2020-11-20T12:12:09.107 INFO:teuthology.orchestra.run.smithi116.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-20T12:12:09.116 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2020-11-20T12:12:09.116 INFO:teuthology.orchestra.run.smithi116.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-20T12:12:09.117 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2020-11-20T12:12:09.123 INFO:teuthology.orchestra.run.smithi116.stdout:Get:24 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-7397-gb86e691b-1bionic [20.7 MB] 2020-11-20T12:12:09.124 INFO:teuthology.orchestra.run.smithi116.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-20T12:12:09.135 INFO:teuthology.orchestra.run.smithi116.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-20T12:12:09.143 INFO:teuthology.orchestra.run.smithi116.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-20T12:12:09.165 INFO:teuthology.orchestra.run.smithi116.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-20T12:12:09.169 INFO:teuthology.orchestra.run.smithi116.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-20T12:12:09.210 INFO:teuthology.orchestra.run.smithi116.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-20T12:12:09.220 INFO:teuthology.orchestra.run.smithi116.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-20T12:12:09.221 INFO:teuthology.orchestra.run.smithi116.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-20T12:12:09.222 INFO:teuthology.orchestra.run.smithi116.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-20T12:12:09.223 INFO:teuthology.orchestra.run.smithi116.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-20T12:12:09.261 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2020-11-20T12:12:09.268 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2020-11-20T12:12:09.284 INFO:teuthology.orchestra.run.smithi173.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2020-11-20T12:12:09.289 INFO:teuthology.orchestra.run.smithi116.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-20T12:12:09.289 INFO:teuthology.orchestra.run.smithi116.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-20T12:12:09.290 INFO:teuthology.orchestra.run.smithi116.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-20T12:12:09.293 INFO:teuthology.orchestra.run.smithi116.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-20T12:12:09.297 INFO:teuthology.orchestra.run.smithi116.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-20T12:12:09.346 INFO:teuthology.orchestra.run.smithi173.stdout:Get:11 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.0.0-7397-gb86e691b-1bionic [481 kB] 2020-11-20T12:12:09.364 INFO:teuthology.orchestra.run.smithi173.stdout:Get:12 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.0.0-7397-gb86e691b-1bionic [332 kB] 2020-11-20T12:12:09.364 INFO:teuthology.orchestra.run.smithi173.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2020-11-20T12:12:09.365 INFO:teuthology.orchestra.run.smithi173.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2020-11-20T12:12:09.366 INFO:teuthology.orchestra.run.smithi116.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-20T12:12:09.367 INFO:teuthology.orchestra.run.smithi116.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-20T12:12:09.388 INFO:teuthology.orchestra.run.smithi116.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-20T12:12:09.388 INFO:teuthology.orchestra.run.smithi116.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-20T12:12:09.389 INFO:teuthology.orchestra.run.smithi116.stdout:Get:44 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-20T12:12:09.390 INFO:teuthology.orchestra.run.smithi173.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2020-11-20T12:12:09.398 INFO:teuthology.orchestra.run.smithi173.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2020-11-20T12:12:09.412 INFO:teuthology.orchestra.run.smithi173.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2020-11-20T12:12:09.418 INFO:teuthology.orchestra.run.smithi173.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.0.0-7397-gb86e691b-1bionic [20.6 kB] 2020-11-20T12:12:09.418 INFO:teuthology.orchestra.run.smithi173.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.0.0-7397-gb86e691b-1bionic [168 kB] 2020-11-20T12:12:09.418 INFO:teuthology.orchestra.run.smithi173.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.0.0-7397-gb86e691b-1bionic [27.7 kB] 2020-11-20T12:12:09.418 INFO:teuthology.orchestra.run.smithi173.stdout:Get:21 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.0.0-7397-gb86e691b-1bionic [330 kB] 2020-11-20T12:12:09.420 INFO:teuthology.orchestra.run.smithi173.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2020-11-20T12:12:09.426 INFO:teuthology.orchestra.run.smithi173.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2020-11-20T12:12:09.444 INFO:teuthology.orchestra.run.smithi116.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-20T12:12:09.445 INFO:teuthology.orchestra.run.smithi173.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2020-11-20T12:12:09.449 INFO:teuthology.orchestra.run.smithi173.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2020-11-20T12:12:09.450 INFO:teuthology.orchestra.run.smithi116.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-20T12:12:09.456 INFO:teuthology.orchestra.run.smithi116.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-20T12:12:09.459 INFO:teuthology.orchestra.run.smithi116.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-20T12:12:09.460 INFO:teuthology.orchestra.run.smithi116.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-20T12:12:09.464 INFO:teuthology.orchestra.run.smithi173.stdout:Get:26 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.0.0-7397-gb86e691b-1bionic [3,179 kB] 2020-11-20T12:12:09.508 INFO:teuthology.orchestra.run.smithi173.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2020-11-20T12:12:09.517 INFO:teuthology.orchestra.run.smithi173.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jwt all 1.5.3+ds1-1 [15.9 kB] 2020-11-20T12:12:09.518 INFO:teuthology.orchestra.run.smithi173.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2020-11-20T12:12:09.518 INFO:teuthology.orchestra.run.smithi173.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2020-11-20T12:12:09.519 INFO:teuthology.orchestra.run.smithi173.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-mako all 1.0.7+ds1-1 [59.3 kB] 2020-11-20T12:12:09.522 INFO:teuthology.orchestra.run.smithi116.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-20T12:12:09.522 INFO:teuthology.orchestra.run.smithi173.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2020-11-20T12:12:09.523 INFO:teuthology.orchestra.run.smithi173.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2020-11-20T12:12:09.593 INFO:teuthology.orchestra.run.smithi173.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2020-11-20T12:12:09.594 INFO:teuthology.orchestra.run.smithi173.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2020-11-20T12:12:09.597 INFO:teuthology.orchestra.run.smithi173.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2020-11-20T12:12:09.613 INFO:teuthology.orchestra.run.smithi116.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-20T12:12:09.615 INFO:teuthology.orchestra.run.smithi116.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-20T12:12:09.669 INFO:teuthology.orchestra.run.smithi173.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2020-11-20T12:12:09.672 INFO:teuthology.orchestra.run.smithi173.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2020-11-20T12:12:09.690 INFO:teuthology.orchestra.run.smithi173.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2020-11-20T12:12:09.691 INFO:teuthology.orchestra.run.smithi173.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2020-11-20T12:12:09.692 INFO:teuthology.orchestra.run.smithi173.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2020-11-20T12:12:09.692 INFO:teuthology.orchestra.run.smithi173.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2020-11-20T12:12:09.696 INFO:teuthology.orchestra.run.smithi173.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1 [174 kB] 2020-11-20T12:12:09.737 INFO:teuthology.orchestra.run.smithi173.stdout:Get:44 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.0.0-7397-gb86e691b-1bionic [99.1 kB] 2020-11-20T12:12:09.739 INFO:teuthology.orchestra.run.smithi173.stdout:Get:45 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.0.0-7397-gb86e691b-1bionic [414 kB] 2020-11-20T12:12:09.749 INFO:teuthology.orchestra.run.smithi173.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2020-11-20T12:12:09.755 INFO:teuthology.orchestra.run.smithi173.stdout:Get:47 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2020-11-20T12:12:09.755 INFO:teuthology.orchestra.run.smithi173.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2020-11-20T12:12:09.802 INFO:teuthology.orchestra.run.smithi173.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.0.0-7397-gb86e691b-1bionic [20.7 MB] 2020-11-20T12:12:09.830 INFO:teuthology.orchestra.run.smithi173.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2020-11-20T12:12:09.868 INFO:teuthology.orchestra.run.smithi173.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2020-11-20T12:12:09.868 INFO:teuthology.orchestra.run.smithi173.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2020-11-20T12:12:09.890 INFO:teuthology.orchestra.run.smithi116.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-20T12:12:09.930 INFO:teuthology.orchestra.run.smithi116.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-20T12:12:09.932 INFO:teuthology.orchestra.run.smithi116.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-20T12:12:09.965 INFO:teuthology.orchestra.run.smithi116.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-20T12:12:09.966 INFO:teuthology.orchestra.run.smithi116.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-20T12:12:09.966 INFO:teuthology.orchestra.run.smithi116.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-20T12:12:09.967 INFO:teuthology.orchestra.run.smithi116.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-20T12:12:09.967 INFO:teuthology.orchestra.run.smithi116.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-20T12:12:09.968 INFO:teuthology.orchestra.run.smithi116.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-20T12:12:09.969 INFO:teuthology.orchestra.run.smithi116.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-20T12:12:10.048 INFO:teuthology.orchestra.run.smithi116.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-20T12:12:10.051 INFO:teuthology.orchestra.run.smithi116.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-20T12:12:10.058 INFO:teuthology.orchestra.run.smithi116.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-20T12:12:10.059 INFO:teuthology.orchestra.run.smithi116.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-20T12:12:10.062 INFO:teuthology.orchestra.run.smithi116.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-20T12:12:10.062 INFO:teuthology.orchestra.run.smithi116.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-20T12:12:10.063 INFO:teuthology.orchestra.run.smithi116.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-20T12:12:10.065 INFO:teuthology.orchestra.run.smithi116.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-20T12:12:10.066 INFO:teuthology.orchestra.run.smithi116.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-20T12:12:10.126 INFO:teuthology.orchestra.run.smithi116.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-20T12:12:10.147 INFO:teuthology.orchestra.run.smithi116.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-20T12:12:10.158 INFO:teuthology.orchestra.run.smithi116.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-20T12:12:10.179 INFO:teuthology.orchestra.run.smithi173.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2020-11-20T12:12:10.179 INFO:teuthology.orchestra.run.smithi116.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-20T12:12:10.179 INFO:teuthology.orchestra.run.smithi116.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1 [183 kB] 2020-11-20T12:12:10.180 INFO:teuthology.orchestra.run.smithi116.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-20T12:12:10.180 INFO:teuthology.orchestra.run.smithi116.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-20T12:12:10.180 INFO:teuthology.orchestra.run.smithi116.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-20T12:12:10.180 INFO:teuthology.orchestra.run.smithi116.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-20T12:12:10.195 INFO:teuthology.orchestra.run.smithi116.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-7397-gb86e691b-1bionic [5,571 kB] 2020-11-20T12:12:10.197 INFO:teuthology.orchestra.run.smithi173.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2020-11-20T12:12:10.198 INFO:teuthology.orchestra.run.smithi173.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2020-11-20T12:12:10.204 INFO:teuthology.orchestra.run.smithi116.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-20T12:12:10.249 INFO:teuthology.orchestra.run.smithi116.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-20T12:12:10.250 INFO:teuthology.orchestra.run.smithi116.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-20T12:12:10.251 INFO:teuthology.orchestra.run.smithi116.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-20T12:12:10.252 INFO:teuthology.orchestra.run.smithi116.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-20T12:12:10.255 INFO:teuthology.orchestra.run.smithi116.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-20T12:12:10.257 INFO:teuthology.orchestra.run.smithi173.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2020-11-20T12:12:10.257 INFO:teuthology.orchestra.run.smithi173.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2020-11-20T12:12:10.258 INFO:teuthology.orchestra.run.smithi173.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.2 [28.6 kB] 2020-11-20T12:12:10.258 INFO:teuthology.orchestra.run.smithi173.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2020-11-20T12:12:10.259 INFO:teuthology.orchestra.run.smithi173.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2020-11-20T12:12:10.260 INFO:teuthology.orchestra.run.smithi173.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2020-11-20T12:12:10.260 INFO:teuthology.orchestra.run.smithi173.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2020-11-20T12:12:10.277 INFO:teuthology.orchestra.run.smithi116.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-20T12:12:10.278 INFO:teuthology.orchestra.run.smithi116.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-20T12:12:10.342 INFO:teuthology.orchestra.run.smithi173.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2020-11-20T12:12:10.346 INFO:teuthology.orchestra.run.smithi173.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-setuptools all 39.0.1-2 [248 kB] 2020-11-20T12:12:10.351 INFO:teuthology.orchestra.run.smithi173.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2020-11-20T12:12:10.353 INFO:teuthology.orchestra.run.smithi173.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2020-11-20T12:12:10.375 INFO:teuthology.orchestra.run.smithi116.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-20T12:12:10.406 INFO:teuthology.orchestra.run.smithi116.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-20T12:12:10.409 INFO:teuthology.orchestra.run.smithi116.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-20T12:12:10.416 INFO:teuthology.orchestra.run.smithi116.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-20T12:12:10.417 INFO:teuthology.orchestra.run.smithi116.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-20T12:12:10.418 INFO:teuthology.orchestra.run.smithi116.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-20T12:12:10.419 INFO:teuthology.orchestra.run.smithi116.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-20T12:12:10.420 INFO:teuthology.orchestra.run.smithi173.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-json-pointer all 1.10-1 [7,478 B] 2020-11-20T12:12:10.421 INFO:teuthology.orchestra.run.smithi173.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-jsonpatch all 1.19+really1.16-1fakesync1 [13.4 kB] 2020-11-20T12:12:10.422 INFO:teuthology.orchestra.run.smithi116.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-20T12:12:10.422 INFO:teuthology.orchestra.run.smithi173.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2020-11-20T12:12:10.423 INFO:teuthology.orchestra.run.smithi116.stdout:Get:98 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-20T12:12:10.425 INFO:teuthology.orchestra.run.smithi173.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2020-11-20T12:12:10.427 INFO:teuthology.orchestra.run.smithi173.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2020-11-20T12:12:10.428 INFO:teuthology.orchestra.run.smithi173.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2020-11-20T12:12:10.463 INFO:teuthology.orchestra.run.smithi116.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-7397-gb86e691b-1bionic [165 kB] 2020-11-20T12:12:10.466 INFO:teuthology.orchestra.run.smithi116.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-7397-gb86e691b-1bionic [1,365 kB] 2020-11-20T12:12:10.501 INFO:teuthology.orchestra.run.smithi173.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.2 [150 kB] 2020-11-20T12:12:10.509 INFO:teuthology.orchestra.run.smithi173.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2020-11-20T12:12:10.518 INFO:teuthology.orchestra.run.smithi173.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-colorama all 0.3.7-1 [14.9 kB] 2020-11-20T12:12:10.519 INFO:teuthology.orchestra.run.smithi173.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1 [183 kB] 2020-11-20T12:12:10.533 INFO:teuthology.orchestra.run.smithi116.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-7397-gb86e691b-1bionic [6,660 kB] 2020-11-20T12:12:10.581 INFO:teuthology.orchestra.run.smithi173.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2020-11-20T12:12:10.583 INFO:teuthology.orchestra.run.smithi173.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2020-11-20T12:12:10.584 INFO:teuthology.orchestra.run.smithi173.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2020-11-20T12:12:10.585 INFO:teuthology.orchestra.run.smithi173.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-py all 1.5.2-1 [65.1 kB] 2020-11-20T12:12:10.588 INFO:teuthology.orchestra.run.smithi173.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2020-11-20T12:12:10.592 INFO:teuthology.orchestra.run.smithi173.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2020-11-20T12:12:10.662 INFO:teuthology.orchestra.run.smithi173.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2020-11-20T12:12:10.665 INFO:teuthology.orchestra.run.smithi173.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2020-11-20T12:12:10.665 INFO:teuthology.orchestra.run.smithi173.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2020-11-20T12:12:10.669 INFO:teuthology.orchestra.run.smithi173.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.1 [1,091 kB] 2020-11-20T12:12:10.742 INFO:teuthology.orchestra.run.smithi173.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2020-11-20T12:12:10.743 INFO:teuthology.orchestra.run.smithi173.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2020-11-20T12:12:10.848 INFO:teuthology.orchestra.run.smithi173.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2020-11-20T12:12:10.848 INFO:teuthology.orchestra.run.smithi173.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2020-11-20T12:12:10.849 INFO:teuthology.orchestra.run.smithi173.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.3 [330 kB] 2020-11-20T12:12:10.855 INFO:teuthology.orchestra.run.smithi173.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2020-11-20T12:12:10.855 INFO:teuthology.orchestra.run.smithi173.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-py all 1.5.2-1 [65.2 kB] 2020-11-20T12:12:10.856 INFO:teuthology.orchestra.run.smithi173.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2020-11-20T12:12:10.857 INFO:teuthology.orchestra.run.smithi173.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2020-11-20T12:12:10.896 INFO:teuthology.orchestra.run.smithi116.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-7397-gb86e691b-1bionic [24.6 MB] 2020-11-20T12:12:10.905 INFO:teuthology.orchestra.run.smithi173.stdout:Get:96 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2020-11-20T12:12:10.908 INFO:teuthology.orchestra.run.smithi173.stdout:Get:97 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2020-11-20T12:12:11.646 INFO:teuthology.orchestra.run.smithi173.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.0.0-7397-gb86e691b-1bionic [5,571 kB] 2020-11-20T12:12:12.143 INFO:teuthology.orchestra.run.smithi173.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.0.0-7397-gb86e691b-1bionic [165 kB] 2020-11-20T12:12:12.170 INFO:teuthology.orchestra.run.smithi116.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-7397-gb86e691b-1bionic [3,808 B] 2020-11-20T12:12:12.170 INFO:teuthology.orchestra.run.smithi116.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-7397-gb86e691b-1bionic [566 kB] 2020-11-20T12:12:12.180 INFO:teuthology.orchestra.run.smithi116.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-7397-gb86e691b-1bionic [1,948 kB] 2020-11-20T12:12:12.192 INFO:teuthology.orchestra.run.smithi173.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.0.0-7397-gb86e691b-1bionic [1,365 kB] 2020-11-20T12:12:12.285 INFO:teuthology.orchestra.run.smithi173.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.0.0-7397-gb86e691b-1bionic [6,660 kB] 2020-11-20T12:12:12.299 INFO:teuthology.orchestra.run.smithi116.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-7397-gb86e691b-1bionic [51.7 kB] 2020-11-20T12:12:12.301 INFO:teuthology.orchestra.run.smithi116.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-7397-gb86e691b-1bionic [56.4 kB] 2020-11-20T12:12:12.302 INFO:teuthology.orchestra.run.smithi116.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-7397-gb86e691b-1bionic [2,878 kB] 2020-11-20T12:12:12.452 INFO:teuthology.orchestra.run.smithi116.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-7397-gb86e691b-1bionic [7,883 kB] 2020-11-20T12:12:12.852 INFO:teuthology.orchestra.run.smithi116.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-7397-gb86e691b-1bionic [3,928 B] 2020-11-20T12:12:12.853 INFO:teuthology.orchestra.run.smithi116.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-7397-gb86e691b-1bionic [22.3 kB] 2020-11-20T12:12:12.853 INFO:teuthology.orchestra.run.smithi116.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-7397-gb86e691b-1bionic [49.7 MB] 2020-11-20T12:12:12.892 INFO:teuthology.orchestra.run.smithi173.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.0.0-7397-gb86e691b-1bionic [24.6 MB] 2020-11-20T12:12:14.962 INFO:teuthology.orchestra.run.smithi173.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.0.0-7397-gb86e691b-1bionic [3,808 B] 2020-11-20T12:12:15.298 INFO:teuthology.orchestra.run.smithi173.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.0.0-7397-gb86e691b-1bionic [566 kB] 2020-11-20T12:12:15.299 INFO:teuthology.orchestra.run.smithi173.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.0.0-7397-gb86e691b-1bionic [1,948 kB] 2020-11-20T12:12:15.299 INFO:teuthology.orchestra.run.smithi173.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.0.0-7397-gb86e691b-1bionic [51.7 kB] 2020-11-20T12:12:15.300 INFO:teuthology.orchestra.run.smithi173.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.0.0-7397-gb86e691b-1bionic [56.4 kB] 2020-11-20T12:12:15.300 INFO:teuthology.orchestra.run.smithi173.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.0.0-7397-gb86e691b-1bionic [2,878 kB] 2020-11-20T12:12:15.363 INFO:teuthology.orchestra.run.smithi173.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.0.0-7397-gb86e691b-1bionic [7,883 kB] 2020-11-20T12:12:15.595 INFO:teuthology.orchestra.run.smithi116.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 16.0.0-7397-gb86e691b-1bionic [17.0 kB] 2020-11-20T12:12:15.595 INFO:teuthology.orchestra.run.smithi116.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-7397-gb86e691b-1bionic [19.5 kB] 2020-11-20T12:12:15.596 INFO:teuthology.orchestra.run.smithi116.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-7397-gb86e691b-1bionic [9,861 kB] 2020-11-20T12:12:15.977 INFO:teuthology.orchestra.run.smithi173.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.0.0-7397-gb86e691b-1bionic [3,928 B] 2020-11-20T12:12:15.978 INFO:teuthology.orchestra.run.smithi173.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-rook all 16.0.0-7397-gb86e691b-1bionic [22.3 kB] 2020-11-20T12:12:15.978 INFO:teuthology.orchestra.run.smithi173.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.0.0-7397-gb86e691b-1bionic [49.7 MB] 2020-11-20T12:12:16.157 INFO:teuthology.orchestra.run.smithi116.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-7397-gb86e691b-1bionic [77.3 kB] 2020-11-20T12:12:17.177 INFO:teuthology.orchestra.run.smithi116.stdout:Fetched 168 MB in 8s (21.7 MB/s) 2020-11-20T12:12:17.359 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-20T12:12:17.428 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104570 files and directories currently installed.) 2020-11-20T12:12:17.431 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-20T12:12:17.473 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-20T12:12:17.718 INFO:teuthology.orchestra.run.smithi173.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 cephfs-shell all 16.0.0-7397-gb86e691b-1bionic [17.0 kB] 2020-11-20T12:12:17.719 INFO:teuthology.orchestra.run.smithi173.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.0.0-7397-gb86e691b-1bionic [19.5 kB] 2020-11-20T12:12:17.719 INFO:teuthology.orchestra.run.smithi173.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.0.0-7397-gb86e691b-1bionic [9,861 kB] 2020-11-20T12:12:17.868 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-20T12:12:17.884 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-20T12:12:17.926 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-20T12:12:18.032 INFO:teuthology.orchestra.run.smithi173.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20201120.053228/b86e691bf191566922a8f08e2a5abf64fde8cfcb/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.0.0-7397-gb86e691b-1bionic [77.3 kB] 2020-11-20T12:12:18.328 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cmd2. 2020-11-20T12:12:18.345 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-20T12:12:18.387 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-20T12:12:18.739 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-20T12:12:18.755 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-20T12:12:18.815 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-20T12:12:19.057 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 168 MB in 9s (18.0 MB/s) 2020-11-20T12:12:19.175 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-20T12:12:19.184 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-20T12:12:19.238 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyparsing. 2020-11-20T12:12:19.259 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-20T12:12:19.306 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104570 files and directories currently installed.) 2020-11-20T12:12:19.309 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../000-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2020-11-20T12:12:19.352 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-20T12:12:19.677 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../005-librbd1_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:19.713 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyperclip. 2020-11-20T12:12:19.727 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../001-python3-pyperclip_1.6.0-1_all.deb ... 2020-11-20T12:12:19.783 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyperclip (1.6.0-1) ... 2020-11-20T12:12:19.804 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librbd1 (16.0.0-7397-gb86e691b-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-20T12:12:20.136 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cmd2. 2020-11-20T12:12:20.153 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../002-python3-cmd2_0.7.9-0ubuntu1_all.deb ... 2020-11-20T12:12:20.190 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-20T12:12:20.441 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../006-librados2_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:20.526 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-uritemplate. 2020-11-20T12:12:20.541 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../003-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2020-11-20T12:12:20.558 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librados2 (16.0.0-7397-gb86e691b-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-20T12:12:20.593 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2020-11-20T12:12:20.972 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2020-11-20T12:12:20.985 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../004-liblttng-ust0_2.10.1-1_amd64.deb ... 2020-11-20T12:12:21.071 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-20T12:12:21.295 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libcephfs2. 2020-11-20T12:12:21.310 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../007-libcephfs2_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:21.354 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libcephfs2 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:21.457 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../005-librbd1_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:21.600 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librbd1 (16.0.0-7397-gb86e691b-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-20T12:12:21.689 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rados. 2020-11-20T12:12:21.703 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../008-python3-rados_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:21.748 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rados (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:22.108 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-20T12:12:22.123 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:22.169 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-ceph-argparse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:22.220 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../006-librados2_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:22.363 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librados2 (16.0.0-7397-gb86e691b-1bionic) over (12.2.13-0ubuntu0.18.04.5) ... 2020-11-20T12:12:22.495 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cephfs. 2020-11-20T12:12:22.510 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../010-python3-cephfs_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:22.553 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cephfs (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:22.964 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-20T12:12:22.980 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../011-python3-ceph-common_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:23.022 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-ceph-common (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:23.033 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcephfs2. 2020-11-20T12:12:23.049 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../007-libcephfs2_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:23.091 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcephfs2 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:23.425 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-prettytable. 2020-11-20T12:12:23.441 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-20T12:12:23.460 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rados. 2020-11-20T12:12:23.474 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../008-python3-rados_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:23.483 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-20T12:12:23.519 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rados (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:23.777 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rbd. 2020-11-20T12:12:23.793 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../013-python3-rbd_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:23.827 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rbd (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:23.913 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-ceph-argparse. 2020-11-20T12:12:23.929 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../009-python3-ceph-argparse_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:23.971 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-ceph-argparse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:24.271 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-20T12:12:24.285 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-20T12:12:24.290 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cephfs. 2020-11-20T12:12:24.307 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../010-python3-cephfs_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:24.330 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-20T12:12:24.365 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cephfs (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:24.730 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-ceph-common. 2020-11-20T12:12:24.741 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-20T12:12:24.742 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../011-python3-ceph-common_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:24.755 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-20T12:12:24.784 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-ceph-common (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:24.799 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-20T12:12:25.162 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-prettytable. 2020-11-20T12:12:25.177 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../012-python3-prettytable_0.7.2-3_all.deb ... 2020-11-20T12:12:25.210 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package librgw2. 2020-11-20T12:12:25.221 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2020-11-20T12:12:25.224 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../016-librgw2_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:25.268 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking librgw2 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:25.531 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rbd. 2020-11-20T12:12:25.547 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../013-python3-rbd_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:25.581 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rbd (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:25.854 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rgw. 2020-11-20T12:12:25.868 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../017-python3-rgw_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:25.946 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rgw (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:26.009 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librabbitmq4:amd64. 2020-11-20T12:12:26.023 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../014-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2020-11-20T12:12:26.059 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-20T12:12:26.290 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-20T12:12:26.305 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-20T12:12:26.349 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-20T12:12:26.495 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librdkafka1:amd64. 2020-11-20T12:12:26.510 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../015-librdkafka1_0.11.3-1build1_amd64.deb ... 2020-11-20T12:12:26.554 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-20T12:12:26.718 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-20T12:12:26.733 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-20T12:12:26.776 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-20T12:12:26.931 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package librgw2. 2020-11-20T12:12:26.945 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../016-librgw2_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:26.990 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking librgw2 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:27.145 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package liboath0. 2020-11-20T12:12:27.159 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-20T12:12:27.229 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-20T12:12:27.550 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rgw. 2020-11-20T12:12:27.568 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../017-python3-rgw_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:27.606 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libradosstriper1. 2020-11-20T12:12:27.608 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rgw (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:27.621 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../021-libradosstriper1_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:27.656 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libradosstriper1 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:27.962 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2020-11-20T12:12:28.481 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-common. 2020-11-20T12:12:28.481 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../022-ceph-common_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:28.482 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-common (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:28.483 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../018-libsnappy1v5_1.1.7-1_amd64.deb ... 2020-11-20T12:12:28.483 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-20T12:12:28.484 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2020-11-20T12:12:28.485 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../019-libleveldb1v5_1.20-2_amd64.deb ... 2020-11-20T12:12:28.485 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2020-11-20T12:12:28.859 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package liboath0. 2020-11-20T12:12:28.874 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../020-liboath0_2.6.1-1_amd64.deb ... 2020-11-20T12:12:28.932 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking liboath0 (2.6.1-1) ... 2020-11-20T12:12:29.311 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libradosstriper1. 2020-11-20T12:12:29.326 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../021-libradosstriper1_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:29.370 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libradosstriper1 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:29.898 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-common. 2020-11-20T12:12:29.914 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../022-ceph-common_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:29.956 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-common (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:30.290 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-base. 2020-11-20T12:12:30.305 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../023-ceph-base_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:30.373 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-base (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:31.511 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-dateutil. 2020-11-20T12:12:31.528 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-20T12:12:31.571 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-20T12:12:31.965 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-20T12:12:31.983 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-20T12:12:32.031 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-20T12:12:32.112 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-base. 2020-11-20T12:12:32.126 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../023-ceph-base_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:32.193 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-base (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:32.375 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-20T12:12:32.390 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-20T12:12:32.433 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-20T12:12:32.819 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cryptography. 2020-11-20T12:12:32.834 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-20T12:12:32.878 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-20T12:12:33.266 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-dateutil. 2020-11-20T12:12:33.281 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../024-python3-dateutil_2.6.1-1_all.deb ... 2020-11-20T12:12:33.325 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2020-11-20T12:12:33.572 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-openssl. 2020-11-20T12:12:33.589 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-20T12:12:33.639 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-20T12:12:33.904 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-asn1crypto. 2020-11-20T12:12:33.918 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../025-python3-asn1crypto_0.24.0-1_all.deb ... 2020-11-20T12:12:33.961 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2020-11-20T12:12:33.975 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-20T12:12:33.989 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:34.033 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-modules-core (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:34.322 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cffi-backend. 2020-11-20T12:12:34.336 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../026-python3-cffi-backend_1.11.5-1_amd64.deb ... 2020-11-20T12:12:34.380 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2020-11-20T12:12:34.506 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-20T12:12:34.521 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-20T12:12:34.590 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-20T12:12:34.716 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cryptography. 2020-11-20T12:12:34.731 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../027-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2020-11-20T12:12:34.766 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-20T12:12:34.917 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-20T12:12:34.933 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-20T12:12:34.967 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-20T12:12:35.195 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-openssl. 2020-11-20T12:12:35.209 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../028-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2020-11-20T12:12:35.286 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2020-11-20T12:12:35.421 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jwt. 2020-11-20T12:12:35.435 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-20T12:12:35.479 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-20T12:12:35.639 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2020-11-20T12:12:35.656 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../029-ceph-mgr-modules-core_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:35.697 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-modules-core (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:35.873 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-logutils. 2020-11-20T12:12:35.888 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-20T12:12:35.932 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-20T12:12:36.151 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-bcrypt. 2020-11-20T12:12:36.166 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../030-python3-bcrypt_3.1.4-2_amd64.deb ... 2020-11-20T12:12:36.210 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2020-11-20T12:12:36.301 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-20T12:12:36.317 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-20T12:12:36.376 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-20T12:12:36.512 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cherrypy3. 2020-11-20T12:12:36.527 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../031-python3-cherrypy3_8.9.1-2_all.deb ... 2020-11-20T12:12:36.562 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2020-11-20T12:12:36.696 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-mako. 2020-11-20T12:12:36.710 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-20T12:12:36.745 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-20T12:12:36.983 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jwt. 2020-11-20T12:12:36.999 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../032-python3-jwt_1.5.3+ds1-1_all.deb ... 2020-11-20T12:12:37.057 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jwt (1.5.3+ds1-1) ... 2020-11-20T12:12:37.123 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-20T12:12:37.132 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-20T12:12:37.173 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-20T12:12:37.460 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-logutils. 2020-11-20T12:12:37.476 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-5_all.deb ... 2020-11-20T12:12:37.492 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-20T12:12:37.501 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-20T12:12:37.519 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-logutils (0.3.3-5) ... 2020-11-20T12:12:37.535 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-20T12:12:37.871 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-markupsafe. 2020-11-20T12:12:37.887 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../034-python3-markupsafe_1.0-1build1_amd64.deb ... 2020-11-20T12:12:37.912 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-webob. 2020-11-20T12:12:37.921 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2020-11-20T12:12:37.926 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-20T12:12:37.970 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-20T12:12:38.266 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-mako. 2020-11-20T12:12:38.280 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../035-python3-mako_1.0.7+ds1-1_all.deb ... 2020-11-20T12:12:38.324 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-mako (1.0.7+ds1-1) ... 2020-11-20T12:12:39.123 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-bs4. 2020-11-20T12:12:39.132 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-20T12:12:39.173 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-20T12:12:39.576 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-waitress. 2020-11-20T12:12:39.586 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-simplegeneric. 2020-11-20T12:12:39.590 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-20T12:12:39.602 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-1_all.deb ... 2020-11-20T12:12:39.634 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-20T12:12:39.644 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2020-11-20T12:12:39.988 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-singledispatch. 2020-11-20T12:12:40.003 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2_all.deb ... 2020-11-20T12:12:40.062 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-tempita. 2020-11-20T12:12:40.072 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2020-11-20T12:12:40.077 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-20T12:12:40.120 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-20T12:12:40.441 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-webob. 2020-11-20T12:12:40.455 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../038-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2020-11-20T12:12:40.456 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-paste. 2020-11-20T12:12:40.471 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-20T12:12:40.491 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2020-11-20T12:12:40.531 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-20T12:12:40.852 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-bs4. 2020-11-20T12:12:40.866 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../039-python3-bs4_4.6.0-1_all.deb ... 2020-11-20T12:12:40.909 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-20T12:12:40.919 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2020-11-20T12:12:40.923 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-20T12:12:40.967 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-20T12:12:41.280 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-waitress. 2020-11-20T12:12:41.294 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../040-python3-waitress_1.0.1-1_all.deb ... 2020-11-20T12:12:41.338 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-waitress (1.0.1-1) ... 2020-11-20T12:12:41.354 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-20T12:12:41.369 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-20T12:12:41.412 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-20T12:12:41.699 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-tempita. 2020-11-20T12:12:41.714 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2020-11-20T12:12:41.757 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-tempita (0.5.2-2) ... 2020-11-20T12:12:41.765 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-webtest. 2020-11-20T12:12:41.783 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-20T12:12:41.848 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-20T12:12:42.110 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-paste. 2020-11-20T12:12:42.129 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2020-11-20T12:12:42.185 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-20T12:12:42.267 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pecan. 2020-11-20T12:12:42.284 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-20T12:12:42.351 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-20T12:12:42.563 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2020-11-20T12:12:42.577 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2020-11-20T12:12:42.613 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2020-11-20T12:12:42.830 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-20T12:12:42.845 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-20T12:12:42.880 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-20T12:12:42.924 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pastedeploy. 2020-11-20T12:12:42.940 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2020-11-20T12:12:42.982 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2020-11-20T12:12:43.335 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-webtest. 2020-11-20T12:12:43.351 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2020-11-20T12:12:43.367 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr. 2020-11-20T12:12:43.382 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../048-ceph-mgr_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:43.394 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2020-11-20T12:12:43.425 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:43.788 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pecan. 2020-11-20T12:12:43.802 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2020-11-20T12:12:43.846 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pecan (1.2.1-2) ... 2020-11-20T12:12:43.953 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mon. 2020-11-20T12:12:43.964 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../049-ceph-mon_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:44.003 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mon (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:44.309 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-werkzeug. 2020-11-20T12:12:44.324 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1_all.deb ... 2020-11-20T12:12:44.367 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-20T12:12:44.796 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr. 2020-11-20T12:12:44.810 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../048-ceph-mgr_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:44.854 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:45.065 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-osd. 2020-11-20T12:12:45.079 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../050-ceph-osd_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:45.148 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-osd (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:45.391 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mon. 2020-11-20T12:12:45.405 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../049-ceph-mon_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:45.440 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mon (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:46.446 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-osd. 2020-11-20T12:12:46.460 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../050-ceph-osd_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:46.504 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-osd (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:47.354 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph. 2020-11-20T12:12:47.368 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../051-ceph_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:47.403 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:47.781 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-fuse. 2020-11-20T12:12:47.791 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../052-ceph-fuse_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:47.831 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-fuse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:48.309 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mds. 2020-11-20T12:12:48.327 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../053-ceph-mds_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:48.367 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mds (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:48.735 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph. 2020-11-20T12:12:48.749 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../051-ceph_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:48.793 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:48.912 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package cephadm. 2020-11-20T12:12:48.929 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../054-cephadm_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:49.029 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking cephadm (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:49.112 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-fuse. 2020-11-20T12:12:49.127 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../052-ceph-fuse_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:49.179 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-fuse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:49.373 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jinja2. 2020-11-20T12:12:49.389 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-20T12:12:49.423 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-20T12:12:49.690 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mds. 2020-11-20T12:12:49.706 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../053-ceph-mds_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:49.749 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mds (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:49.826 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-20T12:12:49.841 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:49.884 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-cephadm (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:50.289 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-20T12:12:51.228 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package cephadm. 2020-11-20T12:12:51.228 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../054-cephadm_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:12:51.228 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking cephadm (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:51.229 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jinja2. 2020-11-20T12:12:51.229 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../055-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2020-11-20T12:12:51.229 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-20T12:12:51.230 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2020-11-20T12:12:51.230 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../056-ceph-mgr-cephadm_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:51.231 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-20T12:12:51.231 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-20T12:12:51.231 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-routes. 2020-11-20T12:12:51.232 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-20T12:12:51.232 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-20T12:12:51.232 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-20T12:12:51.232 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:51.233 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-dashboard (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:51.240 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-cephadm (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:51.635 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-repoze.lru. 2020-11-20T12:12:51.649 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../057-python3-repoze.lru_0.7-1_all.deb ... 2020-11-20T12:12:51.718 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2020-11-20T12:12:52.105 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-routes. 2020-11-20T12:12:52.120 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../058-python3-routes_2.4.1-1_all.deb ... 2020-11-20T12:12:52.155 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-routes (2.4.1-1) ... 2020-11-20T12:12:52.417 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-numpy. 2020-11-20T12:12:52.434 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../060-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-20T12:12:52.475 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-20T12:12:52.508 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2020-11-20T12:12:52.522 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../059-ceph-mgr-dashboard_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:52.566 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-dashboard (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:53.181 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-decorator. 2020-11-20T12:12:53.197 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../061-python3-decorator_4.1.2-1_all.deb ... 2020-11-20T12:12:53.230 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-20T12:12:53.600 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-scipy. 2020-11-20T12:12:53.616 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../062-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-20T12:12:53.658 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-20T12:12:53.807 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-numpy. 2020-11-20T12:12:53.817 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../060-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2020-11-20T12:12:53.857 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-20T12:12:54.588 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-decorator. 2020-11-20T12:12:54.604 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../061-python3-decorator_4.1.2-1_all.deb ... 2020-11-20T12:12:54.654 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-decorator (4.1.2-1) ... 2020-11-20T12:12:54.990 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-scipy. 2020-11-20T12:12:55.006 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../062-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2020-11-20T12:12:55.048 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2020-11-20T12:12:55.057 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-20T12:12:55.074 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../063-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-20T12:12:55.116 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-20T12:12:55.611 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-joblib. 2020-11-20T12:12:55.628 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../064-python3-joblib_0.11-1_all.deb ... 2020-11-20T12:12:55.669 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-20T12:12:56.122 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-sklearn. 2020-11-20T12:12:56.138 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../065-python3-sklearn_0.19.1-3_all.deb ... 2020-11-20T12:12:56.181 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-20T12:12:56.381 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-sklearn-lib. 2020-11-20T12:12:56.396 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../063-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2020-11-20T12:12:56.439 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2020-11-20T12:12:56.879 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-20T12:12:56.895 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../066-ceph-mgr-diskprediction-local_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:56.937 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:56.993 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-joblib. 2020-11-20T12:12:57.008 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../064-python3-joblib_0.11-1_all.deb ... 2020-11-20T12:12:57.068 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-joblib (0.11-1) ... 2020-11-20T12:12:57.488 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-sklearn. 2020-11-20T12:12:57.504 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../065-python3-sklearn_0.19.1-3_all.deb ... 2020-11-20T12:12:57.546 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2020-11-20T12:12:58.219 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2020-11-20T12:12:58.223 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-monotonic. 2020-11-20T12:12:58.234 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../066-ceph-mgr-diskprediction-local_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:12:58.239 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../067-python3-monotonic_1.1-2_all.deb ... 2020-11-20T12:12:58.277 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-diskprediction-local (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:12:58.281 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-20T12:12:58.684 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-fasteners. 2020-11-20T12:12:58.701 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../068-python3-fasteners_0.12.0-3_all.deb ... 2020-11-20T12:12:58.742 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-20T12:12:59.120 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-httplib2. 2020-11-20T12:12:59.136 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../069-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-20T12:12:59.195 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-20T12:12:59.565 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-20T12:12:59.572 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-monotonic. 2020-11-20T12:12:59.581 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../070-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-20T12:12:59.587 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../067-python3-monotonic_1.1-2_all.deb ... 2020-11-20T12:12:59.623 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-20T12:12:59.631 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-monotonic (1.1-2) ... 2020-11-20T12:12:59.966 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-fasteners. 2020-11-20T12:12:59.981 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../068-python3-fasteners_0.12.0-3_all.deb ... 2020-11-20T12:13:00.016 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2020-11-20T12:13:00.026 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-20T12:13:00.040 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../071-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-20T12:13:00.084 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-20T12:13:00.369 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-httplib2. 2020-11-20T12:13:00.378 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../069-python3-httplib2_0.9.2+dfsg-1ubuntu0.2_all.deb ... 2020-11-20T12:13:00.419 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-20T12:13:00.495 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-rsa. 2020-11-20T12:13:00.511 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../072-python3-rsa_3.4.2-1_all.deb ... 2020-11-20T12:13:00.553 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-20T12:13:00.763 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyasn1. 2020-11-20T12:13:00.778 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../070-python3-pyasn1_0.4.2-3_all.deb ... 2020-11-20T12:13:00.817 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2020-11-20T12:13:00.940 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-20T12:13:00.955 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../073-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-20T12:13:00.998 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-20T12:13:01.209 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyasn1-modules. 2020-11-20T12:13:01.223 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../071-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2020-11-20T12:13:01.267 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-20T12:13:01.503 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-20T12:13:01.518 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../074-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-20T12:13:01.560 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-20T12:13:01.628 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-rsa. 2020-11-20T12:13:01.642 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../072-python3-rsa_3.4.2-1_all.deb ... 2020-11-20T12:13:01.702 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-rsa (3.4.2-1) ... 2020-11-20T12:13:01.980 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-setuptools. 2020-11-20T12:13:01.996 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../075-python3-setuptools_39.0.1-2_all.deb ... 2020-11-20T12:13:02.039 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-20T12:13:02.056 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-sqlalchemy. 2020-11-20T12:13:02.071 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../073-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2020-11-20T12:13:02.106 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-20T12:13:02.543 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-websocket. 2020-11-20T12:13:02.557 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../076-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-20T12:13:02.568 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-oauth2client. 2020-11-20T12:13:02.583 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../074-python3-oauth2client_4.1.2-3_all.deb ... 2020-11-20T12:13:02.600 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-20T12:13:02.626 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2020-11-20T12:13:02.996 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-setuptools. 2020-11-20T12:13:03.012 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../075-python3-setuptools_39.0.1-2_all.deb ... 2020-11-20T12:13:03.029 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-20T12:13:03.043 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../077-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-20T12:13:03.080 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-setuptools (39.0.1-2) ... 2020-11-20T12:13:03.086 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-20T12:13:03.550 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-websocket. 2020-11-20T12:13:03.566 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../076-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2020-11-20T12:13:03.608 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2020-11-20T12:13:03.649 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-20T12:13:03.663 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:13:03.732 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:04.011 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-kubernetes. 2020-11-20T12:13:04.026 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../077-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2020-11-20T12:13:04.069 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-20T12:13:04.101 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-20T12:13:04.118 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../079-python3-json-pointer_1.10-1_all.deb ... 2020-11-20T12:13:04.159 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-20T12:13:04.634 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2020-11-20T12:13:04.648 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../078-ceph-mgr-k8sevents_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:13:04.705 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-k8sevents (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:04.804 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-20T12:13:04.821 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../080-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-20T12:13:04.982 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-20T12:13:05.280 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-json-pointer. 2020-11-20T12:13:05.296 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../079-python3-json-pointer_1.10-1_all.deb ... 2020-11-20T12:13:05.342 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-json-pointer (1.10-1) ... 2020-11-20T12:13:05.368 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-20T12:13:05.385 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../081-ceph-mgr-rook_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:13:05.426 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-mgr-rook (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:05.754 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-jsonpatch. 2020-11-20T12:13:05.769 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../080-python3-jsonpatch_1.19+really1.16-1fakesync1_all.deb ... 2020-11-20T12:13:05.853 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-20T12:13:05.870 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../082-libonig4_6.7.0-1_amd64.deb ... 2020-11-20T12:13:05.912 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-20T12:13:05.915 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-20T12:13:06.267 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-mgr-rook. 2020-11-20T12:13:06.285 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../081-ceph-mgr-rook_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:13:06.357 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-20T12:13:06.358 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-mgr-rook (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:06.376 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../083-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-20T12:13:06.423 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-20T12:13:06.734 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package jq. 2020-11-20T12:13:06.749 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../084-jq_1.5+dfsg-2_amd64.deb ... 2020-11-20T12:13:06.761 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libonig4:amd64. 2020-11-20T12:13:06.777 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../082-libonig4_6.7.0-1_amd64.deb ... 2020-11-20T12:13:06.792 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-20T12:13:06.820 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2020-11-20T12:13:07.137 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package socat. 2020-11-20T12:13:07.157 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../085-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-20T12:13:07.189 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-20T12:13:07.223 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libjq1:amd64. 2020-11-20T12:13:07.240 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../083-libjq1_1.5+dfsg-2_amd64.deb ... 2020-11-20T12:13:07.281 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2020-11-20T12:13:07.595 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package jq. 2020-11-20T12:13:07.609 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../084-jq_1.5+dfsg-2_amd64.deb ... 2020-11-20T12:13:07.650 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-20T12:13:07.651 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking jq (1.5+dfsg-2) ... 2020-11-20T12:13:07.663 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../086-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-20T12:13:07.707 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-20T12:13:07.971 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package socat. 2020-11-20T12:13:07.984 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../085-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2020-11-20T12:13:08.028 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2020-11-20T12:13:08.043 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package xmlstarlet. 2020-11-20T12:13:08.060 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../087-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-20T12:13:08.125 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-20T12:13:08.507 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libxslt1.1:amd64. 2020-11-20T12:13:08.524 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../086-libxslt1.1_1.1.29-5ubuntu0.2_amd64.deb ... 2020-11-20T12:13:08.565 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-20T12:13:08.572 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ceph-test. 2020-11-20T12:13:08.589 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../088-ceph-test_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:13:08.655 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ceph-test (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:08.909 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package xmlstarlet. 2020-11-20T12:13:08.927 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../087-xmlstarlet_1.6.1-2_amd64.deb ... 2020-11-20T12:13:08.967 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2020-11-20T12:13:09.405 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ceph-test. 2020-11-20T12:13:09.421 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../088-ceph-test_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:13:09.471 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ceph-test (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:12.859 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-colorama. 2020-11-20T12:13:12.875 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../089-python3-colorama_0.3.7-1_all.deb ... 2020-11-20T12:13:12.917 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-20T12:13:13.239 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package cephfs-shell. 2020-11-20T12:13:13.613 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../090-cephfs-shell_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:13:13.613 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking cephfs-shell (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:13.614 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-20T12:13:13.614 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../091-libcephfs-dev_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:13:13.631 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libcephfs-dev (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:13.650 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-colorama. 2020-11-20T12:13:13.660 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../089-python3-colorama_0.3.7-1_all.deb ... 2020-11-20T12:13:13.701 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-colorama (0.3.7-1) ... 2020-11-20T12:13:13.917 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package nvme-cli. 2020-11-20T12:13:13.934 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../092-nvme-cli_1.5-1ubuntu1_amd64.deb ... 2020-11-20T12:13:13.975 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking nvme-cli (1.5-1ubuntu1) ... 2020-11-20T12:13:14.012 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package cephfs-shell. 2020-11-20T12:13:14.026 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../090-cephfs-shell_16.0.0-7397-gb86e691b-1bionic_all.deb ... 2020-11-20T12:13:14.070 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking cephfs-shell (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:14.387 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-attr. 2020-11-20T12:13:14.397 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../093-python-attr_17.4.0-2_all.deb ... 2020-11-20T12:13:14.401 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libcephfs-dev. 2020-11-20T12:13:14.415 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../091-libcephfs-dev_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:13:14.437 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-20T12:13:14.457 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libcephfs-dev (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:14.743 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package nvme-cli. 2020-11-20T12:13:14.759 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../092-nvme-cli_1.5-1ubuntu1_amd64.deb ... 2020-11-20T12:13:14.800 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking nvme-cli (1.5-1ubuntu1) ... 2020-11-20T12:13:14.848 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-funcsigs. 2020-11-20T12:13:14.863 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../094-python-funcsigs_1.0.2-4_all.deb ... 2020-11-20T12:13:14.907 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-20T12:13:15.212 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-attr. 2020-11-20T12:13:15.228 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../093-python-attr_17.4.0-2_all.deb ... 2020-11-20T12:13:15.242 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-pluggy. 2020-11-20T12:13:15.257 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../095-python-pluggy_0.6.0-1_all.deb ... 2020-11-20T12:13:15.262 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-attr (17.4.0-2) ... 2020-11-20T12:13:15.317 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-20T12:13:15.557 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-funcsigs. 2020-11-20T12:13:15.574 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../094-python-funcsigs_1.0.2-4_all.deb ... 2020-11-20T12:13:15.615 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2020-11-20T12:13:15.662 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-py. 2020-11-20T12:13:15.677 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../096-python-py_1.5.2-1_all.deb ... 2020-11-20T12:13:15.720 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-20T12:13:16.010 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-pluggy. 2020-11-20T12:13:16.029 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../095-python-pluggy_0.6.0-1_all.deb ... 2020-11-20T12:13:16.068 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-pluggy (0.6.0-1) ... 2020-11-20T12:13:16.140 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python-pytest. 2020-11-20T12:13:16.155 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../097-python-pytest_3.3.2-2_all.deb ... 2020-11-20T12:13:16.198 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-20T12:13:16.395 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-py. 2020-11-20T12:13:16.411 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../096-python-py_1.5.2-1_all.deb ... 2020-11-20T12:13:16.445 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-py (1.5.2-1) ... 2020-11-20T12:13:16.618 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-attr. 2020-11-20T12:13:16.635 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../098-python3-attr_17.4.0-2_all.deb ... 2020-11-20T12:13:16.676 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-20T12:13:16.807 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python-pytest. 2020-11-20T12:13:16.822 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../097-python-pytest_3.3.2-2_all.deb ... 2020-11-20T12:13:16.865 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python-pytest (3.3.2-2) ... 2020-11-20T12:13:17.071 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-cycler. 2020-11-20T12:13:17.086 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../099-python3-cycler_0.10.0-1_all.deb ... 2020-11-20T12:13:17.129 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-20T12:13:17.260 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-attr. 2020-11-20T12:13:17.278 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../098-python3-attr_17.4.0-2_all.deb ... 2020-11-20T12:13:17.318 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-attr (17.4.0-2) ... 2020-11-20T12:13:17.498 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-webencodings. 2020-11-20T12:13:17.514 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../100-python3-webencodings_0.5-2_all.deb ... 2020-11-20T12:13:17.582 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-20T12:13:17.679 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-cycler. 2020-11-20T12:13:17.696 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../099-python3-cycler_0.10.0-1_all.deb ... 2020-11-20T12:13:17.755 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-cycler (0.10.0-1) ... 2020-11-20T12:13:17.918 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-html5lib. 2020-11-20T12:13:17.935 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../101-python3-html5lib_0.999999999-1_all.deb ... 2020-11-20T12:13:17.976 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-20T12:13:18.074 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-webencodings. 2020-11-20T12:13:18.091 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../100-python3-webencodings_0.5-2_all.deb ... 2020-11-20T12:13:18.157 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-webencodings (0.5-2) ... 2020-11-20T12:13:18.423 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-20T12:13:18.440 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../102-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-20T12:13:18.481 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-20T12:13:18.526 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-html5lib. 2020-11-20T12:13:18.542 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../101-python3-html5lib_0.999999999-1_all.deb ... 2020-11-20T12:13:18.585 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2020-11-20T12:13:18.971 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-lxml:amd64. 2020-11-20T12:13:18.988 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../102-python3-lxml_4.2.1-1ubuntu0.1_amd64.deb ... 2020-11-20T12:13:18.993 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-tz. 2020-11-20T12:13:19.010 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../103-python3-tz_2018.3-2_all.deb ... 2020-11-20T12:13:19.033 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-20T12:13:19.051 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-20T12:13:19.437 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-20T12:13:19.454 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../104-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-20T12:13:19.495 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-20T12:13:19.558 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-tz. 2020-11-20T12:13:19.573 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../103-python3-tz_2018.3-2_all.deb ... 2020-11-20T12:13:19.616 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-tz (2018.3-2) ... 2020-11-20T12:13:19.986 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-matplotlib. 2020-11-20T12:13:20.002 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../104-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2020-11-20T12:13:20.169 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-20T12:13:20.444 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-olefile. 2020-11-20T12:13:20.460 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../105-python3-olefile_0.45.1-1_all.deb ... 2020-11-20T12:13:20.502 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-20T12:13:20.904 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pastescript. 2020-11-20T12:13:20.917 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-2_all.deb ... 2020-11-20T12:13:20.963 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-20T12:13:21.000 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-olefile. 2020-11-20T12:13:21.015 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../105-python3-olefile_0.45.1-1_all.deb ... 2020-11-20T12:13:21.059 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-olefile (0.45.1-1) ... 2020-11-20T12:13:21.385 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pastescript. 2020-11-20T12:13:21.402 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-2_all.deb ... 2020-11-20T12:13:21.416 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-20T12:13:21.433 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../107-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-20T12:13:21.443 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2020-11-20T12:13:21.474 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-20T12:13:21.856 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pil:amd64. 2020-11-20T12:13:21.870 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../107-python3-pil_5.1.0-1ubuntu0.3_amd64.deb ... 2020-11-20T12:13:21.914 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-20T12:13:21.978 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pluggy. 2020-11-20T12:13:21.994 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../108-python3-pluggy_0.6.0-1_all.deb ... 2020-11-20T12:13:22.036 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-20T12:13:22.367 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pluggy. 2020-11-20T12:13:22.385 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../108-python3-pluggy_0.6.0-1_all.deb ... 2020-11-20T12:13:22.423 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-py. 2020-11-20T12:13:22.425 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2020-11-20T12:13:22.438 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../109-python3-py_1.5.2-1_all.deb ... 2020-11-20T12:13:22.480 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-20T12:13:22.786 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-py. 2020-11-20T12:13:22.805 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../109-python3-py_1.5.2-1_all.deb ... 2020-11-20T12:13:22.844 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-py (1.5.2-1) ... 2020-11-20T12:13:22.850 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-20T12:13:22.865 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-20T12:13:22.925 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-20T12:13:23.231 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pyinotify. 2020-11-20T12:13:23.241 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1_all.deb ... 2020-11-20T12:13:23.281 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2020-11-20T12:13:23.287 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-pytest. 2020-11-20T12:13:23.303 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../111-python3-pytest_3.3.2-2_all.deb ... 2020-11-20T12:13:23.344 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-20T12:13:23.633 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-pytest. 2020-11-20T12:13:23.644 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../111-python3-pytest_3.3.2-2_all.deb ... 2020-11-20T12:13:23.683 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-pytest (3.3.2-2) ... 2020-11-20T12:13:23.765 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-simplejson. 2020-11-20T12:13:23.781 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../112-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-20T12:13:23.823 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-20T12:13:24.128 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-simplejson. 2020-11-20T12:13:24.143 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../112-python3-simplejson_3.13.2-1_amd64.deb ... 2020-11-20T12:13:24.186 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2020-11-20T12:13:24.226 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-20T12:13:24.242 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../113-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-20T12:13:24.288 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-20T12:13:24.600 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package python3-tk:amd64. 2020-11-20T12:13:24.614 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../113-python3-tk_3.6.9-1~18.04_amd64.deb ... 2020-11-20T12:13:24.657 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-20T12:13:24.780 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package radosgw. 2020-11-20T12:13:24.795 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../114-radosgw_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:13:24.829 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking radosgw (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:25.218 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package radosgw. 2020-11-20T12:13:25.228 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../114-radosgw_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:13:25.270 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking radosgw (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:25.975 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package rbd-fuse. 2020-11-20T12:13:25.991 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../115-rbd-fuse_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:13:26.033 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking rbd-fuse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:26.405 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package rbd-fuse. 2020-11-20T12:13:26.420 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../115-rbd-fuse_16.0.0-7397-gb86e691b-1bionic_amd64.deb ... 2020-11-20T12:13:26.429 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-20T12:13:26.463 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking rbd-fuse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:26.796 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-20T12:13:26.839 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-logutils (0.3.3-5) ... 2020-11-20T12:13:27.227 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1) ... 2020-11-20T12:13:27.240 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-20T12:13:27.365 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-20T12:13:27.703 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2020-11-20T12:13:27.829 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2020-11-20T12:13:28.135 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-20T12:13:28.573 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.3) ... 2020-11-20T12:13:28.636 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-20T12:13:28.995 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-20T12:13:29.067 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2020-11-20T12:13:29.346 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-20T12:13:29.418 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-webencodings (0.5-2) ... 2020-11-20T12:13:29.615 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-20T12:13:29.749 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-20T12:13:29.777 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-attr (17.4.0-2) ... 2020-11-20T12:13:30.020 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2020-11-20T12:13:30.142 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-20T12:13:30.150 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-py (1.5.2-1) ... 2020-11-20T12:13:30.540 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-20T12:13:30.581 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-waitress (1.0.1-1) ... 2020-11-20T12:13:30.915 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2020-11-20T12:13:30.962 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-20T12:13:31.292 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-olefile (0.45.1-1) ... 2020-11-20T12:13:31.338 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-ceph-argparse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:31.660 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-ceph-argparse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:31.747 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-20T12:13:32.019 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-tempita (0.5.2-2) ... 2020-11-20T12:13:32.098 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-20T12:13:32.258 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-20T12:13:32.395 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2020-11-20T12:13:32.548 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2020-11-20T12:13:33.386 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-20T12:13:33.683 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2020-11-20T12:13:33.787 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-20T12:13:34.076 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2020-11-20T12:13:34.122 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-py (1.5.2-1) ... 2020-11-20T12:13:34.427 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-py (1.5.2-1) ... 2020-11-20T12:13:34.431 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-20T12:13:34.703 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-simplejson (3.13.2-1) ... 2020-11-20T12:13:34.874 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-20T12:13:34.993 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-20T12:13:35.096 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-json-pointer (1.10-1) ... 2020-11-20T12:13:35.178 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/python3-jsonpointer to provide /usr/bin/jsonpointer (jsonpointer) in auto mode 2020-11-20T12:13:35.392 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-20T12:13:35.543 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-20T12:13:35.656 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2020-11-20T12:13:35.778 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-20T12:13:35.815 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-pluggy (0.6.0-1) ... 2020-11-20T12:13:36.083 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-colorama (0.3.7-1) ... 2020-11-20T12:13:36.112 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-20T12:13:36.447 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-20T12:13:36.451 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2020-11-20T12:13:36.802 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2020-11-20T12:13:36.856 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-20T12:13:37.186 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cycler (0.10.0-1) ... 2020-11-20T12:13:37.229 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-20T12:13:37.529 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2020-11-20T12:13:37.675 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-20T12:13:37.966 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-bs4 (4.6.0-1) ... 2020-11-20T12:13:38.102 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-20T12:13:38.367 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pluggy (0.6.0-1) ... 2020-11-20T12:13:38.453 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up cephadm (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:38.718 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up cephadm (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:39.224 INFO:teuthology.orchestra.run.smithi116.stdout:Adding system user cephadm....done 2020-11-20T12:13:39.426 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-20T12:13:39.446 INFO:teuthology.orchestra.run.smithi173.stdout:Adding system user cephadm....done 2020-11-20T12:13:39.642 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2020-11-20T12:13:39.810 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-20T12:13:39.936 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up nvme-cli (1.5-1ubuntu1) ... 2020-11-20T12:13:40.002 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.2) ... 2020-11-20T12:13:40.088 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-20T12:13:40.111 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up nvme-cli (1.5-1ubuntu1) ... 2020-11-20T12:13:40.221 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2020-11-20T12:13:40.497 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-20T12:13:40.672 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-prettytable (0.7.2-3) ... 2020-11-20T12:13:40.865 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-20T12:13:40.991 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-20T12:13:41.032 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2020-11-20T12:13:41.158 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2020-11-20T12:13:41.434 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-ceph-common (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:41.569 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-ceph-common (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:41.852 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-20T12:13:41.953 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-monotonic (1.1-2) ... 2020-11-20T12:13:42.179 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-20T12:13:42.296 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-20T12:13:42.337 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2020-11-20T12:13:42.473 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2020-11-20T12:13:42.739 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-20T12:13:42.873 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-html5lib (0.999999999-1) ... 2020-11-20T12:13:43.199 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-20T12:13:43.299 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-20T12:13:43.363 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-20T12:13:43.375 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jsonpatch (1.19+really1.16-1fakesync1) ... 2020-11-20T12:13:43.490 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/python3-jsondiff to provide /usr/bin/jsondiff (jsondiff) in auto mode 2020-11-20T12:13:43.527 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/python3-jsonpatch to provide /usr/bin/jsonpatch (jsonpatch) in auto mode 2020-11-20T12:13:43.651 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-20T12:13:43.844 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-repoze.lru (0.7-1) ... 2020-11-20T12:13:44.027 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-20T12:13:44.212 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.2) ... 2020-11-20T12:13:44.411 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-20T12:13:44.580 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2020-11-20T12:13:44.821 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-20T12:13:45.006 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-attr (17.4.0-2) ... 2020-11-20T12:13:45.230 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-20T12:13:45.370 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-funcsigs (1.0.2-4) ... 2020-11-20T12:13:45.473 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-20T12:13:45.609 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2020-11-20T12:13:45.958 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-20T12:13:46.048 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-20T12:13:46.111 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2020-11-20T12:13:46.199 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2020-11-20T12:13:46.393 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-20T12:13:46.496 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liboath0 (2.6.1-1) ... 2020-11-20T12:13:46.560 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-20T12:13:46.712 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-20T12:13:46.713 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2020-11-20T12:13:46.848 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2020-11-20T12:13:47.005 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-20T12:13:47.133 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2020-11-20T12:13:47.156 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-20T12:13:47.260 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-setuptools (39.0.1-2) ... 2020-11-20T12:13:47.674 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-20T12:13:47.770 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2020-11-20T12:13:47.800 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-20T12:13:47.896 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-decorator (4.1.2-1) ... 2020-11-20T12:13:48.185 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-20T12:13:48.238 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-tz (2018.3-2) ... 2020-11-20T12:13:48.528 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-20T12:13:48.598 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2020-11-20T12:13:48.920 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-20T12:13:48.958 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-jwt (1.5.3+ds1-1) ... 2020-11-20T12:13:49.280 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-20T12:13:49.301 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-joblib (0.11-1) ... 2020-11-20T12:13:49.735 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-20T12:13:49.748 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-routes (2.4.1-1) ... 2020-11-20T12:13:50.141 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-20T12:13:50.170 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2020-11-20T12:13:50.539 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-20T12:13:50.550 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-dateutil (2.6.1-1) ... 2020-11-20T12:13:50.932 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-20T12:13:50.976 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pyperclip (1.6.0-1) ... 2020-11-20T12:13:51.291 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-20T12:13:51.361 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2020-11-20T12:13:51.743 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-20T12:13:51.821 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-fasteners (0.12.0-3) ... 2020-11-20T12:13:52.094 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-20T12:13:52.195 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-20T12:13:52.206 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up jq (1.5+dfsg-2) ... 2020-11-20T12:13:52.332 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2020-11-20T12:13:53.662 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-20T12:13:53.803 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.1) ... 2020-11-20T12:13:54.061 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-20T12:13:54.213 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-mako (1.0.7+ds1-1) ... 2020-11-20T12:13:54.462 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-20T12:13:54.588 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-20T12:13:54.639 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2020-11-20T12:13:54.765 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pytest (3.3.2-2) ... 2020-11-20T12:13:55.070 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-20T12:13:55.250 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cmd2 (0.7.9-0ubuntu1) ... 2020-11-20T12:13:55.458 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-20T12:13:55.809 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-20T12:13:56.028 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python-pytest (3.3.2-2) ... 2020-11-20T12:13:56.362 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-20T12:13:56.363 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2020-11-20T12:13:56.463 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up xmlstarlet (1.6.1-2) ... 2020-11-20T12:13:56.471 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librados2 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:56.623 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librados2 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:56.631 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-20T12:13:56.774 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rsa (3.4.2-1) ... 2020-11-20T12:13:57.016 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcephfs2 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:57.142 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libcephfs2 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:57.142 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libcephfs-dev (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:57.285 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rados (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:57.293 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libcephfs-dev (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:57.411 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-20T12:13:57.419 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rados (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:13:57.570 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2020-11-20T12:13:57.941 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-20T12:13:58.089 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2020-11-20T12:13:58.316 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-20T12:13:58.449 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2020-11-20T12:13:58.709 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-20T12:13:58.850 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2020-11-20T12:13:59.720 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-20T12:13:59.870 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-sklearn (0.19.1-3) ... 2020-11-20T12:14:00.768 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-20T12:14:00.884 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2020-11-20T12:14:01.144 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libradosstriper1 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:01.496 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-cephfs (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:01.511 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libradosstriper1 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:01.637 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-cephfs (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:01.965 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librgw2 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:02.014 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librgw2 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:02.091 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-fuse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:02.164 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-fuse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:02.326 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-20T12:14:02.326 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-20T12:14:02.419 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-20T12:14:02.419 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2020-11-20T12:14:02.877 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up librbd1 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:02.991 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up librbd1 (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:03.028 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up cephfs-shell (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:03.117 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up cephfs-shell (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:03.138 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-20T12:14:03.227 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-20T12:14:03.264 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-20T12:14:03.270 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pecan (1.2.1-2) ... 2020-11-20T12:14:03.368 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2020-11-20T12:14:03.432 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2020-11-20T12:14:03.694 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-20T12:14:03.821 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2020-11-20T12:14:04.510 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-modules-core (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:04.636 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-20T12:14:04.699 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-modules-core (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:04.816 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-pastescript (2.0.2-2) ... 2020-11-20T12:14:05.079 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rgw (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:05.206 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up rbd-fuse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:05.243 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rgw (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:05.323 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up python3-rbd (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:05.369 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up rbd-fuse (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:05.458 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-common (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:05.520 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up python3-rbd (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:05.671 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-common (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:06.029 INFO:teuthology.orchestra.run.smithi173.stdout:Adding group ceph....done 2020-11-20T12:14:06.241 INFO:teuthology.orchestra.run.smithi116.stdout:Adding group ceph....done 2020-11-20T12:14:06.522 INFO:teuthology.orchestra.run.smithi173.stdout:Adding system user ceph....done 2020-11-20T12:14:06.766 INFO:teuthology.orchestra.run.smithi173.stdout:Setting system user ceph properties....done 2020-11-20T12:14:06.775 INFO:teuthology.orchestra.run.smithi116.stdout:Adding system user ceph....done 2020-11-20T12:14:06.978 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-20T12:14:07.044 INFO:teuthology.orchestra.run.smithi116.stdout:Setting system user ceph properties....done 2020-11-20T12:14:07.246 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2020-11-20T12:14:07.379 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-20T12:14:07.622 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2020-11-20T12:14:07.986 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up radosgw (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:08.182 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up radosgw (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:08.294 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-20T12:14:08.294 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-20T12:14:08.472 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-20T12:14:08.472 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2020-11-20T12:14:09.090 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-test (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:09.214 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-base (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:09.300 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-test (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:09.451 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-base (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:09.683 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-20T12:14:09.890 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2020-11-20T12:14:10.208 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:10.468 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-20T12:14:10.468 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-20T12:14:10.477 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:10.735 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-20T12:14:10.735 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2020-11-20T12:14:11.051 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:11.202 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-osd (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:11.271 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-diskprediction-local (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:11.445 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-osd (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:11.881 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-20T12:14:11.882 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-20T12:14:12.123 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-20T12:14:12.123 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2020-11-20T12:14:12.405 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-cephadm (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:12.600 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-rook (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:12.640 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-cephadm (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:12.757 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mds (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:12.825 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-rook (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:12.984 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mds (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:13.043 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-20T12:14:13.044 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-20T12:14:13.246 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-20T12:14:13.247 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2020-11-20T12:14:13.576 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mon (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:13.786 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mon (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:13.833 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-20T12:14:13.833 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-20T12:14:14.033 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-20T12:14:14.034 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2020-11-20T12:14:14.385 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-k8sevents (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:14.512 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph-mgr-dashboard (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:14.582 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-k8sevents (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:14.688 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ceph (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:14.721 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph-mgr-dashboard (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:14.806 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-20T12:14:14.873 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ceph (16.0.0-7397-gb86e691b-1bionic) ... 2020-11-20T12:14:15.024 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-20T12:14:15.123 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-20T12:14:15.359 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-20T12:14:18.534 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-20T12:14:18.580 INFO:teuthology.orchestra.run.smithi173.stdout:ureadahead will be reprofiled on next reboot 2020-11-20T12:14:18.660 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-20T12:14:18.777 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-20T12:14:18.823 INFO:teuthology.orchestra.run.smithi116.stdout:ureadahead will be reprofiled on next reboot 2020-11-20T12:14:18.894 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-20T12:14:19.387 INFO:teuthology.orchestra.run.smithi173.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-20T12:14:19.393 INFO:teuthology.orchestra.run.smithi173:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-20T12:14:19.523 INFO:teuthology.orchestra.run.smithi173.stdout:Reading package lists... 2020-11-20T12:14:19.623 INFO:teuthology.orchestra.run.smithi116.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-20T12:14:19.628 INFO:teuthology.orchestra.run.smithi116:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev 2020-11-20T12:14:19.759 INFO:teuthology.orchestra.run.smithi116.stdout:Reading package lists... 2020-11-20T12:14:19.781 INFO:teuthology.orchestra.run.smithi173.stdout:Building dependency tree... 2020-11-20T12:14:19.781 INFO:teuthology.orchestra.run.smithi173.stdout:Reading state information... 2020-11-20T12:14:19.949 INFO:teuthology.orchestra.run.smithi173.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-20T12:14:19.950 INFO:teuthology.orchestra.run.smithi173.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-20T12:14:19.950 INFO:teuthology.orchestra.run.smithi173.stdout:iproute2 is already the newest version (4.15.0-2ubuntu1.2). 2020-11-20T12:14:19.951 INFO:teuthology.orchestra.run.smithi173.stdout:util-linux is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-20T12:14:19.951 INFO:teuthology.orchestra.run.smithi173.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-20T12:14:19.951 INFO:teuthology.orchestra.run.smithi173.stdout:The following additional packages will be installed: 2020-11-20T12:14:19.951 INFO:teuthology.orchestra.run.smithi173.stdout: iputils-arping libbison-dev libfl-dev libfl2 libmbim-glib4 libmbim-proxy 2020-11-20T12:14:19.952 INFO:teuthology.orchestra.run.smithi173.stdout: libmm-glib0 libndp0 libnm0 libpolkit-agent-1-0 libpolkit-backend-1-0 2020-11-20T12:14:19.953 INFO:teuthology.orchestra.run.smithi173.stdout: libqmi-glib5 libqmi-proxy libteamdctl0 modemmanager network-manager-pptp 2020-11-20T12:14:19.953 INFO:teuthology.orchestra.run.smithi173.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-20T12:14:19.954 INFO:teuthology.orchestra.run.smithi173.stdout: zlib1g-dev 2020-11-20T12:14:19.954 INFO:teuthology.orchestra.run.smithi173.stdout:Suggested packages: 2020-11-20T12:14:19.954 INFO:teuthology.orchestra.run.smithi173.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils comgt 2020-11-20T12:14:19.955 INFO:teuthology.orchestra.run.smithi173.stdout: wvdial wpagui libengine-pkcs11-openssl 2020-11-20T12:14:20.012 INFO:teuthology.orchestra.run.smithi173.stdout:The following NEW packages will be installed: 2020-11-20T12:14:20.013 INFO:teuthology.orchestra.run.smithi173.stdout: bison dump flex indent iputils-arping libaio-dev libbison-dev libelf-dev 2020-11-20T12:14:20.013 INFO:teuthology.orchestra.run.smithi173.stdout: libfl-dev libfl2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 2020-11-20T12:14:20.014 INFO:teuthology.orchestra.run.smithi173.stdout: libpolkit-agent-1-0 libpolkit-backend-1-0 libqmi-glib5 libqmi-proxy 2020-11-20T12:14:20.014 INFO:teuthology.orchestra.run.smithi173.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2020-11-20T12:14:20.014 INFO:teuthology.orchestra.run.smithi173.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-20T12:14:20.015 INFO:teuthology.orchestra.run.smithi173.stdout: zlib1g-dev 2020-11-20T12:14:20.017 INFO:teuthology.orchestra.run.smithi116.stdout:Building dependency tree... 2020-11-20T12:14:20.018 INFO:teuthology.orchestra.run.smithi116.stdout:Reading state information... 2020-11-20T12:14:20.191 INFO:teuthology.orchestra.run.smithi116.stdout:libtool-bin is already the newest version (2.4.6-2). 2020-11-20T12:14:20.191 INFO:teuthology.orchestra.run.smithi116.stdout:xfslibs-dev is already the newest version (4.9.0+nmu1ubuntu2). 2020-11-20T12:14:20.191 INFO:teuthology.orchestra.run.smithi116.stdout:iproute2 is already the newest version (4.15.0-2ubuntu1.2). 2020-11-20T12:14:20.192 INFO:teuthology.orchestra.run.smithi116.stdout:util-linux is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-20T12:14:20.192 INFO:teuthology.orchestra.run.smithi116.stdout:uuid-dev is already the newest version (2.31.1-0.4ubuntu3.7). 2020-11-20T12:14:20.192 INFO:teuthology.orchestra.run.smithi116.stdout:The following additional packages will be installed: 2020-11-20T12:14:20.193 INFO:teuthology.orchestra.run.smithi116.stdout: iputils-arping libbison-dev libfl-dev libfl2 libmbim-glib4 libmbim-proxy 2020-11-20T12:14:20.193 INFO:teuthology.orchestra.run.smithi116.stdout: libmm-glib0 libndp0 libnm0 libpolkit-agent-1-0 libpolkit-backend-1-0 2020-11-20T12:14:20.193 INFO:teuthology.orchestra.run.smithi116.stdout: libqmi-glib5 libqmi-proxy libteamdctl0 modemmanager network-manager-pptp 2020-11-20T12:14:20.194 INFO:teuthology.orchestra.run.smithi116.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-20T12:14:20.195 INFO:teuthology.orchestra.run.smithi116.stdout: zlib1g-dev 2020-11-20T12:14:20.195 INFO:teuthology.orchestra.run.smithi116.stdout:Suggested packages: 2020-11-20T12:14:20.195 INFO:teuthology.orchestra.run.smithi116.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils comgt 2020-11-20T12:14:20.196 INFO:teuthology.orchestra.run.smithi116.stdout: wvdial wpagui libengine-pkcs11-openssl 2020-11-20T12:14:20.205 INFO:teuthology.orchestra.run.smithi173.stdout:0 upgraded, 31 newly installed, 0 to remove and 23 not upgraded. 2020-11-20T12:14:20.206 INFO:teuthology.orchestra.run.smithi173.stdout:Need to get 7,861 kB of archives. 2020-11-20T12:14:20.206 INFO:teuthology.orchestra.run.smithi173.stdout:After this operation, 32.0 MB of additional disk space will be used. 2020-11-20T12:14:20.206 INFO:teuthology.orchestra.run.smithi173.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-20T12:14:20.254 INFO:teuthology.orchestra.run.smithi116.stdout:The following NEW packages will be installed: 2020-11-20T12:14:20.255 INFO:teuthology.orchestra.run.smithi116.stdout: bison dump flex indent iputils-arping libaio-dev libbison-dev libelf-dev 2020-11-20T12:14:20.255 INFO:teuthology.orchestra.run.smithi116.stdout: libfl-dev libfl2 libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 2020-11-20T12:14:20.255 INFO:teuthology.orchestra.run.smithi116.stdout: libpolkit-agent-1-0 libpolkit-backend-1-0 libqmi-glib5 libqmi-proxy 2020-11-20T12:14:20.256 INFO:teuthology.orchestra.run.smithi116.stdout: libssl-dev libteamdctl0 modemmanager network-manager network-manager-pptp 2020-11-20T12:14:20.256 INFO:teuthology.orchestra.run.smithi116.stdout: policykit-1 ppp pptp-linux usb-modeswitch usb-modeswitch-data wpasupplicant 2020-11-20T12:14:20.256 INFO:teuthology.orchestra.run.smithi116.stdout: zlib1g-dev 2020-11-20T12:14:20.443 INFO:teuthology.orchestra.run.smithi116.stdout:0 upgraded, 31 newly installed, 0 to remove and 23 not upgraded. 2020-11-20T12:14:20.443 INFO:teuthology.orchestra.run.smithi116.stdout:Need to get 7,861 kB of archives. 2020-11-20T12:14:20.443 INFO:teuthology.orchestra.run.smithi116.stdout:After this operation, 32.0 MB of additional disk space will be used. 2020-11-20T12:14:20.443 INFO:teuthology.orchestra.run.smithi116.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/main amd64 flex amd64 2.6.4-6 [316 kB] 2020-11-20T12:14:20.651 INFO:teuthology.orchestra.run.smithi173.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-20T12:14:20.731 INFO:teuthology.orchestra.run.smithi173.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-20T12:14:20.777 INFO:teuthology.orchestra.run.smithi173.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-20T12:14:20.796 INFO:teuthology.orchestra.run.smithi173.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-20T12:14:20.809 INFO:teuthology.orchestra.run.smithi173.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 iputils-arping amd64 3:20161105-1ubuntu3 [29.7 kB] 2020-11-20T12:14:20.811 INFO:teuthology.orchestra.run.smithi173.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-20T12:14:20.812 INFO:teuthology.orchestra.run.smithi173.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-20T12:14:20.827 INFO:teuthology.orchestra.run.smithi173.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-20T12:14:20.831 INFO:teuthology.orchestra.run.smithi173.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-20T12:14:20.832 INFO:teuthology.orchestra.run.smithi173.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-20T12:14:20.832 INFO:teuthology.orchestra.run.smithi173.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-glib4 amd64 1.18.0-1~ubuntu18.04.1 [91.4 kB] 2020-11-20T12:14:20.882 INFO:teuthology.orchestra.run.smithi116.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbison-dev amd64 2:3.0.4.dfsg-1build1 [339 kB] 2020-11-20T12:14:20.894 INFO:teuthology.orchestra.run.smithi173.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-proxy amd64 1.18.0-1~ubuntu18.04.1 [5,572 B] 2020-11-20T12:14:20.895 INFO:teuthology.orchestra.run.smithi173.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmm-glib0 amd64 1.10.0-1~ubuntu18.04.2 [179 kB] 2020-11-20T12:14:20.903 INFO:teuthology.orchestra.run.smithi173.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libndp0 amd64 1.6-1 [10.7 kB] 2020-11-20T12:14:20.904 INFO:teuthology.orchestra.run.smithi173.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnm0 amd64 1.10.6-2ubuntu1.4 [298 kB] 2020-11-20T12:14:20.916 INFO:teuthology.orchestra.run.smithi173.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-agent-1-0 amd64 0.105-20ubuntu0.18.04.5 [14.9 kB] 2020-11-20T12:14:20.962 INFO:teuthology.orchestra.run.smithi116.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic/main amd64 bison amd64 2:3.0.4.dfsg-1build1 [266 kB] 2020-11-20T12:14:20.973 INFO:teuthology.orchestra.run.smithi173.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-backend-1-0 amd64 0.105-20ubuntu0.18.04.5 [36.3 kB] 2020-11-20T12:14:20.975 INFO:teuthology.orchestra.run.smithi173.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-glib5 amd64 1.22.0-1.2~ubuntu18.04.1 [494 kB] 2020-11-20T12:14:20.993 INFO:teuthology.orchestra.run.smithi173.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-proxy amd64 1.22.0-1.2~ubuntu18.04.1 [5,632 B] 2020-11-20T12:14:20.994 INFO:teuthology.orchestra.run.smithi173.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-20T12:14:21.009 INFO:teuthology.orchestra.run.smithi116.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/universe amd64 dump amd64 0.4b46-3 [129 kB] 2020-11-20T12:14:21.023 INFO:teuthology.orchestra.run.smithi116.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 indent amd64 2.2.11-5 [80.7 kB] 2020-11-20T12:14:21.032 INFO:teuthology.orchestra.run.smithi116.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 iputils-arping amd64 3:20161105-1ubuntu3 [29.7 kB] 2020-11-20T12:14:21.035 INFO:teuthology.orchestra.run.smithi116.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libaio-dev amd64 0.3.110-5ubuntu0.1 [12.8 kB] 2020-11-20T12:14:21.036 INFO:teuthology.orchestra.run.smithi116.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 zlib1g-dev amd64 1:1.2.11.dfsg-0ubuntu2 [176 kB] 2020-11-20T12:14:21.051 INFO:teuthology.orchestra.run.smithi116.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libelf-dev amd64 0.170-0.4ubuntu0.1 [57.3 kB] 2020-11-20T12:14:21.056 INFO:teuthology.orchestra.run.smithi116.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl2 amd64 2.6.4-6 [11.4 kB] 2020-11-20T12:14:21.057 INFO:teuthology.orchestra.run.smithi116.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/main amd64 libfl-dev amd64 2.6.4-6 [6,320 B] 2020-11-20T12:14:21.058 INFO:teuthology.orchestra.run.smithi116.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-glib4 amd64 1.18.0-1~ubuntu18.04.1 [91.4 kB] 2020-11-20T12:14:21.060 INFO:teuthology.orchestra.run.smithi173.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libteamdctl0 amd64 1.26-1 [11.6 kB] 2020-11-20T12:14:21.062 INFO:teuthology.orchestra.run.smithi173.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 modemmanager amd64 1.10.0-1~ubuntu18.04.2 [728 kB] 2020-11-20T12:14:21.093 INFO:teuthology.orchestra.run.smithi173.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 wpasupplicant amd64 2:2.6-15ubuntu2.5 [954 kB] 2020-11-20T12:14:21.117 INFO:teuthology.orchestra.run.smithi173.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 policykit-1 amd64 0.105-20ubuntu0.18.04.5 [53.5 kB] 2020-11-20T12:14:21.118 INFO:teuthology.orchestra.run.smithi173.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 network-manager amd64 1.10.6-2ubuntu1.4 [1,505 kB] 2020-11-20T12:14:21.119 INFO:teuthology.orchestra.run.smithi116.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmbim-proxy amd64 1.18.0-1~ubuntu18.04.1 [5,572 B] 2020-11-20T12:14:21.120 INFO:teuthology.orchestra.run.smithi116.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libmm-glib0 amd64 1.10.0-1~ubuntu18.04.2 [179 kB] 2020-11-20T12:14:21.128 INFO:teuthology.orchestra.run.smithi116.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic/main amd64 libndp0 amd64 1.6-1 [10.7 kB] 2020-11-20T12:14:21.129 INFO:teuthology.orchestra.run.smithi116.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libnm0 amd64 1.10.6-2ubuntu1.4 [298 kB] 2020-11-20T12:14:21.139 INFO:teuthology.orchestra.run.smithi116.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-agent-1-0 amd64 0.105-20ubuntu0.18.04.5 [14.9 kB] 2020-11-20T12:14:21.148 INFO:teuthology.orchestra.run.smithi173.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ppp amd64 2.4.7-2+2ubuntu1.3 [328 kB] 2020-11-20T12:14:21.174 INFO:teuthology.orchestra.run.smithi173.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 pptp-linux amd64 1.9.0+ds-2 [41.8 kB] 2020-11-20T12:14:21.174 INFO:teuthology.orchestra.run.smithi173.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 network-manager-pptp amd64 1.2.6-1 [30.0 kB] 2020-11-20T12:14:21.175 INFO:teuthology.orchestra.run.smithi173.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch-data all 20170806-2 [30.7 kB] 2020-11-20T12:14:21.198 INFO:teuthology.orchestra.run.smithi116.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libpolkit-backend-1-0 amd64 0.105-20ubuntu0.18.04.5 [36.3 kB] 2020-11-20T12:14:21.199 INFO:teuthology.orchestra.run.smithi116.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-glib5 amd64 1.22.0-1.2~ubuntu18.04.1 [494 kB] 2020-11-20T12:14:21.212 INFO:teuthology.orchestra.run.smithi173.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu1 [53.6 kB] 2020-11-20T12:14:21.217 INFO:teuthology.orchestra.run.smithi116.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libqmi-proxy amd64 1.22.0-1.2~ubuntu18.04.1 [5,632 B] 2020-11-20T12:14:21.218 INFO:teuthology.orchestra.run.smithi116.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libssl-dev amd64 1.1.1-1ubuntu2.1~18.04.6 [1,566 kB] 2020-11-20T12:14:21.284 INFO:teuthology.orchestra.run.smithi116.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic/main amd64 libteamdctl0 amd64 1.26-1 [11.6 kB] 2020-11-20T12:14:21.285 INFO:teuthology.orchestra.run.smithi116.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 modemmanager amd64 1.10.0-1~ubuntu18.04.2 [728 kB] 2020-11-20T12:14:21.310 INFO:teuthology.orchestra.run.smithi116.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 wpasupplicant amd64 2:2.6-15ubuntu2.5 [954 kB] 2020-11-20T12:14:21.338 INFO:teuthology.orchestra.run.smithi116.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 policykit-1 amd64 0.105-20ubuntu0.18.04.5 [53.5 kB] 2020-11-20T12:14:21.339 INFO:teuthology.orchestra.run.smithi116.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 network-manager amd64 1.10.6-2ubuntu1.4 [1,505 kB] 2020-11-20T12:14:21.369 INFO:teuthology.orchestra.run.smithi116.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 ppp amd64 2.4.7-2+2ubuntu1.3 [328 kB] 2020-11-20T12:14:21.395 INFO:teuthology.orchestra.run.smithi116.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic/main amd64 pptp-linux amd64 1.9.0+ds-2 [41.8 kB] 2020-11-20T12:14:21.397 INFO:teuthology.orchestra.run.smithi116.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 network-manager-pptp amd64 1.2.6-1 [30.0 kB] 2020-11-20T12:14:21.399 INFO:teuthology.orchestra.run.smithi116.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch-data all 20170806-2 [30.7 kB] 2020-11-20T12:14:21.432 INFO:teuthology.orchestra.run.smithi116.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 usb-modeswitch amd64 2.5.2+repack0-2ubuntu1 [53.6 kB] 2020-11-20T12:14:21.908 INFO:teuthology.orchestra.run.smithi173.stdout:Fetched 7,861 kB in 1s (6,697 kB/s) 2020-11-20T12:14:22.051 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package flex. 2020-11-20T12:14:22.123 INFO:teuthology.orchestra.run.smithi116.stdout:Fetched 7,861 kB in 1s (6,815 kB/s) 2020-11-20T12:14:22.125 INFO:teuthology.orchestra.run.smithi173.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 112494 files and directories currently installed.) 2020-11-20T12:14:22.128 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-20T12:14:22.182 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking flex (2.6.4-6) ... 2020-11-20T12:14:22.277 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package flex. 2020-11-20T12:14:22.350 INFO:teuthology.orchestra.run.smithi116.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 112494 files and directories currently installed.) 2020-11-20T12:14:22.353 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../00-flex_2.6.4-6_amd64.deb ... 2020-11-20T12:14:22.390 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking flex (2.6.4-6) ... 2020-11-20T12:14:22.678 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-20T12:14:22.693 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-20T12:14:22.736 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-20T12:14:22.861 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libbison-dev:amd64. 2020-11-20T12:14:22.877 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../01-libbison-dev_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-20T12:14:22.933 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-20T12:14:23.148 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package bison. 2020-11-20T12:14:23.165 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-20T12:14:23.218 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-20T12:14:23.322 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package bison. 2020-11-20T12:14:23.332 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../02-bison_2%3a3.0.4.dfsg-1build1_amd64.deb ... 2020-11-20T12:14:23.384 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking bison (2:3.0.4.dfsg-1build1) ... 2020-11-20T12:14:23.660 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package dump. 2020-11-20T12:14:23.678 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-20T12:14:23.743 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking dump (0.4b46-3) ... 2020-11-20T12:14:23.878 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package dump. 2020-11-20T12:14:24.509 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package indent. 2020-11-20T12:14:24.509 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-20T12:14:24.509 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking indent (2.2.11-5) ... 2020-11-20T12:14:24.510 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package iputils-arping. 2020-11-20T12:14:24.510 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../03-dump_0.4b46-3_amd64.deb ... 2020-11-20T12:14:24.510 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking dump (0.4b46-3) ... 2020-11-20T12:14:24.510 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package indent. 2020-11-20T12:14:24.511 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../04-indent_2.2.11-5_amd64.deb ... 2020-11-20T12:14:24.511 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking indent (2.2.11-5) ... 2020-11-20T12:14:24.517 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../05-iputils-arping_3%3a20161105-1ubuntu3_amd64.deb ... 2020-11-20T12:14:24.559 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking iputils-arping (3:20161105-1ubuntu3) ... 2020-11-20T12:14:24.765 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package iputils-arping. 2020-11-20T12:14:24.781 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../05-iputils-arping_3%3a20161105-1ubuntu3_amd64.deb ... 2020-11-20T12:14:24.824 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking iputils-arping (3:20161105-1ubuntu3) ... 2020-11-20T12:14:24.878 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-20T12:14:24.894 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-20T12:14:24.936 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-20T12:14:25.109 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libaio-dev:amd64. 2020-11-20T12:14:25.124 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../06-libaio-dev_0.3.110-5ubuntu0.1_amd64.deb ... 2020-11-20T12:14:25.159 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-20T12:14:25.232 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-20T12:14:25.247 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-20T12:14:25.290 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-20T12:14:25.512 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package zlib1g-dev:amd64. 2020-11-20T12:14:25.521 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../07-zlib1g-dev_1%3a1.2.11.dfsg-0ubuntu2_amd64.deb ... 2020-11-20T12:14:25.564 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-20T12:14:25.660 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-20T12:14:25.677 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-20T12:14:25.719 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-20T12:14:25.934 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libelf-dev:amd64. 2020-11-20T12:14:25.952 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../08-libelf-dev_0.170-0.4ubuntu0.1_amd64.deb ... 2020-11-20T12:14:26.017 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-20T12:14:26.188 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-20T12:14:26.207 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2020-11-20T12:14:26.299 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-20T12:14:26.420 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libfl2:amd64. 2020-11-20T12:14:26.436 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../09-libfl2_2.6.4-6_amd64.deb ... 2020-11-20T12:14:26.550 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-20T12:14:26.564 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-20T12:14:26.608 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-20T12:14:26.893 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libfl2:amd64 (2.6.4-6) ... 2020-11-20T12:14:27.198 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libfl-dev:amd64. 2020-11-20T12:14:27.214 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../10-libfl-dev_2.6.4-6_amd64.deb ... 2020-11-20T12:14:27.257 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libfl-dev:amd64 (2.6.4-6) ... 2020-11-20T12:14:27.328 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2020-11-20T12:14:27.343 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../11-libmbim-glib4_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-20T12:14:27.386 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-20T12:14:27.659 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2020-11-20T12:14:27.669 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../11-libmbim-glib4_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-20T12:14:27.706 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmbim-proxy. 2020-11-20T12:14:27.710 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-20T12:14:27.722 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../12-libmbim-proxy_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-20T12:14:27.764 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-20T12:14:28.037 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libmbim-proxy. 2020-11-20T12:14:28.054 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../12-libmbim-proxy_1.18.0-1~ubuntu18.04.1_amd64.deb ... 2020-11-20T12:14:28.095 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-20T12:14:28.125 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libmm-glib0:amd64. 2020-11-20T12:14:28.141 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../13-libmm-glib0_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-20T12:14:28.184 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-20T12:14:28.523 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libmm-glib0:amd64. 2020-11-20T12:14:28.539 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../13-libmm-glib0_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-20T12:14:28.582 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-20T12:14:28.612 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libndp0:amd64. 2020-11-20T12:14:28.628 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../14-libndp0_1.6-1_amd64.deb ... 2020-11-20T12:14:28.687 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libndp0:amd64 (1.6-1) ... 2020-11-20T12:14:29.044 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libndp0:amd64. 2020-11-20T12:14:29.060 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../14-libndp0_1.6-1_amd64.deb ... 2020-11-20T12:14:29.082 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libnm0:amd64. 2020-11-20T12:14:29.098 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../15-libnm0_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-20T12:14:29.101 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libndp0:amd64 (1.6-1) ... 2020-11-20T12:14:29.140 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-20T12:14:29.504 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libnm0:amd64. 2020-11-20T12:14:29.520 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../15-libnm0_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-20T12:14:29.554 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-20T12:14:29.576 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libpolkit-agent-1-0:amd64. 2020-11-20T12:14:29.592 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../16-libpolkit-agent-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-20T12:14:29.635 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:29.974 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libpolkit-agent-1-0:amd64. 2020-11-20T12:14:29.979 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libpolkit-backend-1-0:amd64. 2020-11-20T12:14:29.991 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../16-libpolkit-agent-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-20T12:14:29.994 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../17-libpolkit-backend-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-20T12:14:30.032 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:30.063 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:30.429 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libpolkit-backend-1-0:amd64. 2020-11-20T12:14:30.442 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../17-libpolkit-backend-1-0_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-20T12:14:30.449 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2020-11-20T12:14:30.465 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../18-libqmi-glib5_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-20T12:14:30.501 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:30.508 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-20T12:14:30.911 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libqmi-proxy. 2020-11-20T12:14:30.912 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2020-11-20T12:14:30.926 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../19-libqmi-proxy_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-20T12:14:30.928 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../18-libqmi-glib5_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-20T12:14:30.969 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-20T12:14:30.996 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-20T12:14:31.280 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-20T12:14:31.297 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../20-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-20T12:14:31.347 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-20T12:14:31.348 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libqmi-proxy. 2020-11-20T12:14:31.364 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../19-libqmi-proxy_1.22.0-1.2~ubuntu18.04.1_amd64.deb ... 2020-11-20T12:14:31.407 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-20T12:14:31.751 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libssl-dev:amd64. 2020-11-20T12:14:31.766 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../20-libssl-dev_1.1.1-1ubuntu2.1~18.04.6_amd64.deb ... 2020-11-20T12:14:31.809 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-20T12:14:31.934 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package libteamdctl0:amd64. 2020-11-20T12:14:31.949 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../21-libteamdctl0_1.26-1_amd64.deb ... 2020-11-20T12:14:31.992 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking libteamdctl0:amd64 (1.26-1) ... 2020-11-20T12:14:32.572 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package libteamdctl0:amd64. 2020-11-20T12:14:32.588 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../21-libteamdctl0_1.26-1_amd64.deb ... 2020-11-20T12:14:32.630 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking libteamdctl0:amd64 (1.26-1) ... 2020-11-20T12:14:32.639 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package modemmanager. 2020-11-20T12:14:32.656 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../22-modemmanager_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-20T12:14:32.720 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-20T12:14:33.100 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package modemmanager. 2020-11-20T12:14:33.114 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../22-modemmanager_1.10.0-1~ubuntu18.04.2_amd64.deb ... 2020-11-20T12:14:33.184 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-20T12:14:33.404 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package wpasupplicant. 2020-11-20T12:14:33.421 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../23-wpasupplicant_2%3a2.6-15ubuntu2.5_amd64.deb ... 2020-11-20T12:14:33.730 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-20T12:14:33.931 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package wpasupplicant. 2020-11-20T12:14:33.946 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../23-wpasupplicant_2%3a2.6-15ubuntu2.5_amd64.deb ... 2020-11-20T12:14:34.279 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-20T12:14:34.292 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package policykit-1. 2020-11-20T12:14:34.309 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../24-policykit-1_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-20T12:14:34.374 INFO:teuthology.orchestra.run.smithi173.stdout:Unit polkit.service does not exist, proceeding anyway. 2020-11-20T12:14:34.375 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /run/systemd/system/polkit.service -> /dev/null. 2020-11-20T12:14:34.627 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:34.869 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package policykit-1. 2020-11-20T12:14:34.887 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../24-policykit-1_0.105-20ubuntu0.18.04.5_amd64.deb ... 2020-11-20T12:14:34.947 INFO:teuthology.orchestra.run.smithi116.stdout:Unit polkit.service does not exist, proceeding anyway. 2020-11-20T12:14:34.947 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /run/systemd/system/polkit.service -> /dev/null. 2020-11-20T12:14:35.073 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package network-manager. 2020-11-20T12:14:35.089 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../25-network-manager_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-20T12:14:35.179 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:35.188 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking network-manager (1.10.6-2ubuntu1.4) ... 2020-11-20T12:14:35.690 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package network-manager. 2020-11-20T12:14:35.705 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../25-network-manager_1.10.6-2ubuntu1.4_amd64.deb ... 2020-11-20T12:14:35.835 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking network-manager (1.10.6-2ubuntu1.4) ... 2020-11-20T12:14:35.964 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package ppp. 2020-11-20T12:14:35.981 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../26-ppp_2.4.7-2+2ubuntu1.3_amd64.deb ... 2020-11-20T12:14:36.530 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package ppp. 2020-11-20T12:14:36.546 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../26-ppp_2.4.7-2+2ubuntu1.3_amd64.deb ... 2020-11-20T12:14:36.571 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-20T12:14:37.003 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package pptp-linux. 2020-11-20T12:14:37.018 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../27-pptp-linux_1.9.0+ds-2_amd64.deb ... 2020-11-20T12:14:37.078 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking pptp-linux (1.9.0+ds-2) ... 2020-11-20T12:14:37.119 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-20T12:14:37.669 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package pptp-linux. 2020-11-20T12:14:37.685 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../27-pptp-linux_1.9.0+ds-2_amd64.deb ... 2020-11-20T12:14:37.708 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package network-manager-pptp. 2020-11-20T12:14:37.719 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking pptp-linux (1.9.0+ds-2) ... 2020-11-20T12:14:37.722 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../28-network-manager-pptp_1.2.6-1_amd64.deb ... 2020-11-20T12:14:37.785 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking network-manager-pptp (1.2.6-1) ... 2020-11-20T12:14:38.043 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package usb-modeswitch-data. 2020-11-20T12:14:38.059 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../29-usb-modeswitch-data_20170806-2_all.deb ... 2020-11-20T12:14:38.102 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking usb-modeswitch-data (20170806-2) ... 2020-11-20T12:14:38.180 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package network-manager-pptp. 2020-11-20T12:14:38.190 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../28-network-manager-pptp_1.2.6-1_amd64.deb ... 2020-11-20T12:14:38.254 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking network-manager-pptp (1.2.6-1) ... 2020-11-20T12:14:38.580 INFO:teuthology.orchestra.run.smithi173.stdout:Selecting previously unselected package usb-modeswitch. 2020-11-20T12:14:38.597 INFO:teuthology.orchestra.run.smithi173.stdout:Preparing to unpack .../30-usb-modeswitch_2.5.2+repack0-2ubuntu1_amd64.deb ... 2020-11-20T12:14:38.616 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package usb-modeswitch-data. 2020-11-20T12:14:38.632 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../29-usb-modeswitch-data_20170806-2_all.deb ... 2020-11-20T12:14:38.662 INFO:teuthology.orchestra.run.smithi173.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-20T12:14:38.683 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking usb-modeswitch-data (20170806-2) ... 2020-11-20T12:14:39.000 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-20T12:14:39.129 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-20T12:14:39.145 INFO:teuthology.orchestra.run.smithi116.stdout:Selecting previously unselected package usb-modeswitch. 2020-11-20T12:14:39.162 INFO:teuthology.orchestra.run.smithi116.stdout:Preparing to unpack .../30-usb-modeswitch_2.5.2+repack0-2ubuntu1_amd64.deb ... 2020-11-20T12:14:39.227 INFO:teuthology.orchestra.run.smithi116.stdout:Unpacking usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-20T12:14:39.604 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libmbim-glib4:amd64 (1.18.0-1~ubuntu18.04.1) ... 2020-11-20T12:14:39.758 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up ppp (2.4.7-2+2ubuntu1.3) ... 2020-11-20T12:14:40.187 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2020-11-20T12:14:40.452 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up flex (2.6.4-6) ... 2020-11-20T12:14:40.604 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:40.737 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up dump (0.4b46-3) ... 2020-11-20T12:14:40.846 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/pppd-dns.service -> /lib/systemd/system/pppd-dns.service. 2020-11-20T12:14:40.854 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-20T12:14:40.922 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-20T12:14:41.073 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-20T12:14:41.081 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up flex (2.6.4-6) ... 2020-11-20T12:14:41.208 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up pptp-linux (1.9.0+ds-2) ... 2020-11-20T12:14:41.233 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libpolkit-agent-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:41.359 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up dump (0.4b46-3) ... 2020-11-20T12:14:41.401 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-20T12:14:41.500 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2020-11-20T12:14:41.568 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libnm0:amd64 (1.10.6-2ubuntu1.4) ... 2020-11-20T12:14:41.719 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libssl-dev:amd64 (1.1.1-1ubuntu2.1~18.04.6) ... 2020-11-20T12:14:41.841 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-20T12:14:41.841 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-20T12:14:41.844 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up pptp-linux (1.9.0+ds-2) ... 2020-11-20T12:14:42.029 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up wpasupplicant (2:2.6-15ubuntu2.5) ... 2020-11-20T12:14:42.378 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-20T12:14:42.467 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-20T12:14:42.467 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2020-11-20T12:14:42.513 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up usb-modeswitch-data (20170806-2) ... 2020-11-20T12:14:42.639 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-20T12:14:42.857 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:42.992 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-20T12:14:43.032 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libmbim-proxy (1.18.0-1~ubuntu18.04.1) ... 2020-11-20T12:14:43.142 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up indent (2.2.11-5) ... 2020-11-20T12:14:43.192 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up usb-modeswitch-data (20170806-2) ... 2020-11-20T12:14:43.268 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-20T12:14:43.318 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up usb-modeswitch (2.5.2+repack0-2ubuntu1) ... 2020-11-20T12:14:43.394 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-20T12:14:43.529 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-20T12:14:43.544 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libpolkit-backend-1-0:amd64 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:43.628 INFO:teuthology.orchestra.run.smithi173.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-20T12:14:43.695 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libmm-glib0:amd64 (1.10.0-1~ubuntu18.04.2) ... 2020-11-20T12:14:43.697 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-20T12:14:43.821 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up indent (2.2.11-5) ... 2020-11-20T12:14:43.848 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libndp0:amd64 (1.6-1) ... 2020-11-20T12:14:43.955 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libbison-dev:amd64 (2:3.0.4.dfsg-1build1) ... 2020-11-20T12:14:43.974 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up iputils-arping (3:20161105-1ubuntu3) ... 2020-11-20T12:14:44.065 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libfl2:amd64 (2.6.4-6) ... 2020-11-20T12:14:44.142 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libteamdctl0:amd64 (1.26-1) ... 2020-11-20T12:14:44.199 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up bison (2:3.0.4.dfsg-1build1) ... 2020-11-20T12:14:44.268 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-20T12:14:44.298 INFO:teuthology.orchestra.run.smithi116.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2020-11-20T12:14:44.367 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libaio-dev:amd64 (0.3.110-5ubuntu0.1) ... 2020-11-20T12:14:44.394 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:44.518 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libndp0:amd64 (1.6-1) ... 2020-11-20T12:14:44.627 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up iputils-arping (3:20161105-1ubuntu3) ... 2020-11-20T12:14:44.771 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libteamdctl0:amd64 (1.26-1) ... 2020-11-20T12:14:44.776 INFO:teuthology.orchestra.run.smithi173.stdout:Removed /run/systemd/system/polkit.service. 2020-11-20T12:14:44.887 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up zlib1g-dev:amd64 (1:1.2.11.dfsg-0ubuntu2) ... 2020-11-20T12:14:45.047 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up policykit-1 (0.105-20ubuntu0.18.04.5) ... 2020-11-20T12:14:45.274 INFO:teuthology.orchestra.run.smithi173.stdout:polkit.service is a disabled or a static unit not running, not starting it. 2020-11-20T12:14:45.459 INFO:teuthology.orchestra.run.smithi116.stdout:Removed /run/systemd/system/polkit.service. 2020-11-20T12:14:45.779 INFO:teuthology.orchestra.run.smithi173.stdout:polkit.service is a disabled or a static unit, not starting it. 2020-11-20T12:14:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-20T12:14:45.781 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-20T12:14:45.907 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-20T12:14:45.993 INFO:teuthology.orchestra.run.smithi116.stdout:polkit.service is a disabled or a static unit not running, not starting it. 2020-11-20T12:14:46.050 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-20T12:14:46.176 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up network-manager (1.10.6-2ubuntu1.4) ... 2020-11-20T12:14:46.273 INFO:teuthology.orchestra.run.smithi116.stdout:polkit.service is a disabled or a static unit, not starting it. 2020-11-20T12:14:46.333 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqmi-glib5:amd64 (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-20T12:14:46.484 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libelf-dev:amd64 (0.170-0.4ubuntu0.1) ... 2020-11-20T12:14:46.602 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libqmi-proxy (1.22.0-1.2~ubuntu18.04.1) ... 2020-11-20T12:14:46.737 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up libfl-dev:amd64 (2.6.4-6) ... 2020-11-20T12:14:46.839 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up network-manager (1.10.6-2ubuntu1.4) ... 2020-11-20T12:14:47.115 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:47.116 INFO:teuthology.orchestra.run.smithi173.stdout:The following network interfaces were found in /etc/network/interfaces 2020-11-20T12:14:47.117 INFO:teuthology.orchestra.run.smithi173.stdout:which means they are currently configured by ifupdown: 2020-11-20T12:14:47.117 INFO:teuthology.orchestra.run.smithi173.stdout:- enp3s0f1 2020-11-20T12:14:47.119 INFO:teuthology.orchestra.run.smithi173.stdout:If you want to manage those interfaces with NetworkManager instead 2020-11-20T12:14:47.121 INFO:teuthology.orchestra.run.smithi173.stdout:remove their configuration from /etc/network/interfaces. 2020-11-20T12:14:47.121 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:47.291 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2020-11-20T12:14:47.682 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2020-11-20T12:14:47.776 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:47.776 INFO:teuthology.orchestra.run.smithi116.stdout:The following network interfaces were found in /etc/network/interfaces 2020-11-20T12:14:47.776 INFO:teuthology.orchestra.run.smithi116.stdout:which means they are currently configured by ifupdown: 2020-11-20T12:14:47.777 INFO:teuthology.orchestra.run.smithi116.stdout:- enp3s0f1 2020-11-20T12:14:47.777 INFO:teuthology.orchestra.run.smithi116.stdout:If you want to manage those interfaces with NetworkManager instead 2020-11-20T12:14:47.777 INFO:teuthology.orchestra.run.smithi116.stdout:remove their configuration from /etc/network/interfaces. 2020-11-20T12:14:47.777 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:47.944 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2020-11-20T12:14:48.081 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2020-11-20T12:14:48.312 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2020-11-20T12:14:48.686 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2020-11-20T12:14:49.173 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up network-manager-pptp (1.2.6-1) ... 2020-11-20T12:14:49.356 INFO:teuthology.orchestra.run.smithi173.stdout:Setting up modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-20T12:14:49.686 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2020-11-20T12:14:49.687 INFO:teuthology.orchestra.run.smithi173.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2020-11-20T12:14:49.758 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up network-manager-pptp (1.2.6-1) ... 2020-11-20T12:14:49.976 INFO:teuthology.orchestra.run.smithi116.stdout:Setting up modemmanager (1.10.0-1~ubuntu18.04.2) ... 2020-11-20T12:14:50.261 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2020-11-20T12:14:50.261 INFO:teuthology.orchestra.run.smithi116.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2020-11-20T12:14:50.317 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-20T12:14:50.652 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-20T12:14:50.912 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for systemd (237-3ubuntu10.42) ... 2020-11-20T12:14:51.254 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2020-11-20T12:14:54.887 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for dbus (1.12.2-1ubuntu1.2) ... 2020-11-20T12:14:54.987 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2020-11-20T12:14:55.232 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for dbus (1.12.2-1ubuntu1.2) ... 2020-11-20T12:14:55.340 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2020-11-20T12:14:55.464 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-20T12:14:55.581 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-20T12:14:55.775 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2020-11-20T12:14:55.892 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for install-info (6.5.0.dfsg.1-2) ... 2020-11-20T12:14:55.974 INFO:teuthology.orchestra.run.smithi173.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-20T12:14:56.310 INFO:teuthology.orchestra.run.smithi116.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.3) ... 2020-11-20T12:14:56.683 INFO:teuthology.orchestra.run.smithi173.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-20T12:14:56.689 DEBUG:teuthology.parallel:result is None 2020-11-20T12:14:56.986 INFO:teuthology.orchestra.run.smithi116.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2020-11-20T12:14:56.992 DEBUG:teuthology.parallel:result is None 2020-11-20T12:14:56.993 INFO:teuthology.orchestra.run.smithi116:> sudo lsb_release -is 2020-11-20T12:14:57.063 INFO:teuthology.orchestra.run.smithi116.stdout:Ubuntu 2020-11-20T12:14:57.072 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-20T12:14:57.072 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T12:14:57.257 INFO:teuthology.orchestra.run.smithi116:> dpkg-query -W -f '${Version}' ceph 2020-11-20T12:14:57.297 INFO:teuthology.orchestra.run.smithi116.stdout:16.0.0-7397-gb86e691b-1bionic 2020-11-20T12:14:57.298 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7397-gb86e691b-1bionic 2020-11-20T12:14:57.298 INFO:teuthology.task.install:The correct ceph version 16.0.0-7397-gb86e691b-1bionic is installed. 2020-11-20T12:14:57.299 INFO:teuthology.orchestra.run.smithi173:> sudo lsb_release -is 2020-11-20T12:14:57.391 INFO:teuthology.orchestra.run.smithi173.stdout:Ubuntu 2020-11-20T12:14:57.399 DEBUG:teuthology.misc:System to be installed: Ubuntu 2020-11-20T12:14:57.400 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=b86e691bf191566922a8f08e2a5abf64fde8cfcb 2020-11-20T12:14:57.592 INFO:teuthology.orchestra.run.smithi173:> dpkg-query -W -f '${Version}' ceph 2020-11-20T12:14:57.642 INFO:teuthology.orchestra.run.smithi173.stdout:16.0.0-7397-gb86e691b-1bionic 2020-11-20T12:14:57.642 INFO:teuthology.packaging:The installed version of ceph is 16.0.0-7397-gb86e691b-1bionic 2020-11-20T12:14:57.643 INFO:teuthology.task.install:The correct ceph version 16.0.0-7397-gb86e691b-1bionic is installed. 2020-11-20T12:14:57.644 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2020-11-20T12:14:57.644 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:14:57.645 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T12:14:57.662 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T12:14:57.678 INFO:teuthology.orchestra.run.smithi116.stderr:24+1 records in 2020-11-20T12:14:57.678 INFO:teuthology.orchestra.run.smithi116.stderr:24+1 records out 2020-11-20T12:14:57.679 INFO:teuthology.orchestra.run.smithi116.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000767348 s, 16.5 MB/s 2020-11-20T12:14:57.680 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:14:57.681 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T12:14:57.704 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2020-11-20T12:14:57.719 INFO:teuthology.orchestra.run.smithi173.stderr:24+1 records in 2020-11-20T12:14:57.720 INFO:teuthology.orchestra.run.smithi173.stderr:24+1 records out 2020-11-20T12:14:57.720 INFO:teuthology.orchestra.run.smithi173.stderr:12667 bytes (13 kB, 12 KiB) copied, 0.000708421 s, 17.9 MB/s 2020-11-20T12:14:57.722 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2020-11-20T12:14:57.722 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:14:57.723 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/daemon-helper 2020-11-20T12:14:57.741 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-20T12:14:57.754 INFO:teuthology.orchestra.run.smithi116.stderr:5+1 records in 2020-11-20T12:14:57.754 INFO:teuthology.orchestra.run.smithi116.stderr:5+1 records out 2020-11-20T12:14:57.754 INFO:teuthology.orchestra.run.smithi116.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 0.000262532 s, 11.6 MB/s 2020-11-20T12:14:57.756 INFO:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-20T12:14:57.786 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:14:57.787 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/daemon-helper 2020-11-20T12:14:57.803 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/usr/bin/daemon-helper 2020-11-20T12:14:57.819 INFO:teuthology.orchestra.run.smithi173.stderr:5+1 records in 2020-11-20T12:14:57.819 INFO:teuthology.orchestra.run.smithi173.stderr:5+1 records out 2020-11-20T12:14:57.819 INFO:teuthology.orchestra.run.smithi173.stderr:3058 bytes (3.1 kB, 3.0 KiB) copied, 0.000324107 s, 9.4 MB/s 2020-11-20T12:14:57.821 INFO:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/daemon-helper 2020-11-20T12:14:57.897 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2020-11-20T12:14:57.897 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:14:57.898 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T12:14:57.917 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T12:14:57.929 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:14:57.930 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:14:57.931 INFO:teuthology.orchestra.run.smithi116.stderr:299 bytes copied, 0.00015964 s, 1.9 MB/s 2020-11-20T12:14:57.934 INFO:teuthology.orchestra.run.smithi116:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-20T12:14:58.011 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:14:58.012 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T12:14:58.033 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/usr/bin/adjust-ulimits 2020-11-20T12:14:58.046 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:14:58.046 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:14:58.047 INFO:teuthology.orchestra.run.smithi173.stderr:299 bytes copied, 0.00014614 s, 2.0 MB/s 2020-11-20T12:14:58.048 INFO:teuthology.orchestra.run.smithi173:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2020-11-20T12:14:58.135 INFO:teuthology.run_tasks:Running task ceph... 2020-11-20T12:14:58.197 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2020-11-20T12:14:58.198 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 777 /var/log/ceph 2020-11-20T12:14:58.211 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 777 /var/log/ceph 2020-11-20T12:14:58.247 INFO:tasks.ceph:Disabling ceph logrotate... 2020-11-20T12:14:58.247 INFO:teuthology.orchestra.run.smithi116:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-20T12:14:58.291 INFO:teuthology.orchestra.run.smithi173:> sudo rm -f -- /etc/logrotate.d/ceph 2020-11-20T12:14:58.328 INFO:tasks.ceph:Creating extra log directories... 2020-11-20T12:14:58.329 INFO:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-20T12:14:58.372 INFO:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2020-11-20T12:14:58.451 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2020-11-20T12:14:58.452 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2020-11-20T12:14:58.452 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2020-11-20T12:14:58.453 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:14:58.453 INFO:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T12:14:58.474 INFO:teuthology.orchestra.run.smithi116.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T12:14:58.476 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:14:58.476 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:14:58.477 INFO:teuthology.orchestra.run.smithi116.stderr:408 bytes copied, 0.000101597 s, 4.0 MB/s 2020-11-20T12:14:58.478 INFO:teuthology.orchestra.run.smithi116:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-20T12:14:58.618 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:14:58.620 INFO:teuthology.orchestra.run.smithi173:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T12:14:58.639 INFO:teuthology.orchestra.run.smithi173.stderr:+ dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2020-11-20T12:14:58.643 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:14:58.643 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:14:58.644 INFO:teuthology.orchestra.run.smithi173.stderr:408 bytes copied, 0.000102801 s, 4.0 MB/s 2020-11-20T12:14:58.644 INFO:teuthology.orchestra.run.smithi173:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2020-11-20T12:14:58.776 INFO:tasks.ceph:Creating ceph cluster ceph... 2020-11-20T12:14:58.777 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2020-11-20T12:14:58.777 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi/5640336', 'branch': 'wip-pdonnell-testing-20201120.053228', 'description': 'fs/volumes/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/kclient/{mount overrides/{distro/testing/{flavor/ubuntu_latest k-testing} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{frag_enable no_client_pidfile whitelist_health whitelist_wrongly_marked_down} tasks/volume-client}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '5640336', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2020-11-20_08:39:42-fs-wip-pdonnell-testing-20201120.053228-distro-basic-smithi', 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '18.04', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20201120.053228'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20201120.053228', 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 95, 'repo': 'git://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20201120.053228', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-pdonnell-testing-20201120.053228/qa', 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb', 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell'], 'rpm': ['python3-cephfs']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'sha1': 'b86e691bf191566922a8f08e2a5abf64fde8cfcb'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volume_client']}}], 'teuthology_branch': 'master', 'timestamp': '2020-11-20_08:39:42', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.21172', 'targets': {'smithi116.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSQ6rwLDP6wHpBzj/BMCabw7CJZ5RrVUWL6TJg6UPwAIYJiyVyCYAAhQcdWkhAmr1NA9onXfbiMEAfgCTWdmNXvm1krr2mxps5atb0Ysoama3Au3nD06o+XNn2uOpbk4IIL+c9W7Rb4uithZkr3oVws9oDcOLokt+XznhRAvi0ql8nzgiVeakbObZyiHqdzQXU8PLq4A9i80rCeKOGAhpekkg3DcRRZjeKRgPkvOIIFLSyK3KdxxWnVd++kGVjZd3wlI0KmvGhTNES7sm7z115aQOgS0IxVivad0unAoDTYngOsWyAEioT2Mez7A4uVkx90vHm8z5Dwwvn22Dxmnzt', 'smithi173.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRY9tyR9uY3qAegbul+z/sE02z16putaUgDL8Gea4ZwyArv4XkNs/CHTDbgCYxVoa7OhzxZwCAC47Gkrt6UOjTfjvznSBdSDqEhSwfAIWFaLkZ2K7Wx/TM2BZz6otRv5nYA9hPt0PeV8IDWk06mZqOIPPGr+sxtU+MW8jiByrvO/hQD0oknWXIxs+kdP8avuqxzg/KMyzhYC9D7sDDhOX0grZ43pbDy9tLC3gulOpvXQ3RvQwVsDdW1IxqEQxRcQidY8sLj6X6bffoP1Qb/1fRemuhGLMqBYxTYmlCIWQETj4XY/jhH6bfQzJ4XmveY62sj05hO/ZB/gy9vuJfqWSf'}} 2020-11-20T12:14:58.778 INFO:teuthology.orchestra.run.smithi116:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-20T12:14:58.791 INFO:teuthology.orchestra.run.smithi173:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2020-11-20T12:14:58.816 INFO:teuthology.orchestra.run.smithi116:> sudo install -d -m0777 -- /var/run/ceph 2020-11-20T12:14:58.861 INFO:teuthology.orchestra.run.smithi173:> sudo install -d -m0777 -- /var/run/ceph 2020-11-20T12:14:58.897 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:14:58.897 INFO:teuthology.orchestra.run.smithi116:> dd if=/scratch_devs of=/dev/stdout 2020-11-20T12:14:58.950 INFO:teuthology.orchestra.run.smithi116.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-20T12:14:58.953 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/vg_nvme/lv_1 2020-11-20T12:14:58.954 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/vg_nvme/lv_2 2020-11-20T12:14:58.954 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/vg_nvme/lv_3 2020-11-20T12:14:58.955 INFO:teuthology.orchestra.run.smithi116.stdout:/dev/vg_nvme/lv_4 2020-11-20T12:14:58.955 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:14:58.956 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:14:58.956 INFO:teuthology.orchestra.run.smithi116.stderr:72 bytes copied, 0.00018216 s, 395 kB/s 2020-11-20T12:14:58.956 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-20T12:14:58.957 INFO:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-20T12:14:59.020 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-20T12:14:59.020 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:14:59.021 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 497 Links: 1 2020-11-20T12:14:59.021 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:14:59.021 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2020-11-20 12:11:29.678485827 +0000 2020-11-20T12:14:59.021 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2020-11-20 12:11:29.466379832 +0000 2020-11-20T12:14:59.021 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2020-11-20 12:11:29.466379832 +0000 2020-11-20T12:14:59.022 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2020-11-20T12:14:59.037 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2020-11-20T12:14:59.037 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2020-11-20T12:14:59.037 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000738284 s, 694 kB/s 2020-11-20T12:14:59.046 INFO:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-20T12:14:59.068 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-20T12:14:59.069 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:14:59.069 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 503 Links: 1 2020-11-20T12:14:59.069 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:14:59.070 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2020-11-20 12:11:30.082687818 +0000 2020-11-20T12:14:59.070 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2020-11-20 12:11:29.878585824 +0000 2020-11-20T12:14:59.070 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2020-11-20 12:11:29.878585824 +0000 2020-11-20T12:14:59.070 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2020-11-20T12:14:59.083 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2020-11-20T12:14:59.084 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2020-11-20T12:14:59.084 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000569404 s, 899 kB/s 2020-11-20T12:14:59.093 INFO:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-20T12:14:59.162 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-20T12:14:59.162 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:14:59.162 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 509 Links: 1 2020-11-20T12:14:59.163 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:14:59.163 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2020-11-20 12:11:30.482887810 +0000 2020-11-20T12:14:59.163 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2020-11-20 12:11:30.286789813 +0000 2020-11-20T12:14:59.163 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2020-11-20 12:11:30.286789813 +0000 2020-11-20T12:14:59.164 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2020-11-20T12:14:59.177 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2020-11-20T12:14:59.178 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2020-11-20T12:14:59.178 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000569556 s, 899 kB/s 2020-11-20T12:14:59.187 INFO:teuthology.orchestra.run.smithi116:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-20T12:14:59.208 INFO:teuthology.orchestra.run.smithi116.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-20T12:14:59.208 INFO:teuthology.orchestra.run.smithi116.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:14:59.208 INFO:teuthology.orchestra.run.smithi116.stdout:Device: 5h/5d Inode: 515 Links: 1 2020-11-20T12:14:59.209 INFO:teuthology.orchestra.run.smithi116.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:14:59.209 INFO:teuthology.orchestra.run.smithi116.stdout:Access: 2020-11-20 12:11:30.931111799 +0000 2020-11-20T12:14:59.209 INFO:teuthology.orchestra.run.smithi116.stdout:Modify: 2020-11-20 12:11:30.690991806 +0000 2020-11-20T12:14:59.209 INFO:teuthology.orchestra.run.smithi116.stdout:Change: 2020-11-20 12:11:30.690991806 +0000 2020-11-20T12:14:59.210 INFO:teuthology.orchestra.run.smithi116.stdout: Birth: - 2020-11-20T12:14:59.224 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records in 2020-11-20T12:14:59.224 INFO:teuthology.orchestra.run.smithi116.stderr:1+0 records out 2020-11-20T12:14:59.224 INFO:teuthology.orchestra.run.smithi116.stderr:512 bytes copied, 0.000682604 s, 750 kB/s 2020-11-20T12:14:59.233 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T12:14:59.234 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:14:59.234 INFO:teuthology.orchestra.run.smithi173:> dd if=/scratch_devs of=/dev/stdout 2020-11-20T12:14:59.252 INFO:teuthology.orchestra.run.smithi173.stderr:+ dd if=/scratch_devs of=/dev/stdout 2020-11-20T12:14:59.255 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/vg_nvme/lv_1 2020-11-20T12:14:59.255 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/vg_nvme/lv_2 2020-11-20T12:14:59.255 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/vg_nvme/lv_3 2020-11-20T12:14:59.256 INFO:teuthology.orchestra.run.smithi173.stdout:/dev/vg_nvme/lv_4 2020-11-20T12:14:59.256 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:14:59.256 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:14:59.256 INFO:teuthology.orchestra.run.smithi173.stderr:72 bytes copied, 0.000168406 s, 428 kB/s 2020-11-20T12:14:59.257 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2020-11-20T12:14:59.257 INFO:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2020-11-20T12:14:59.317 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2020-11-20T12:14:59.318 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:14:59.318 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 497 Links: 1 2020-11-20T12:14:59.318 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:14:59.319 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2020-11-20 12:11:29.742642919 +0000 2020-11-20T12:14:59.319 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2020-11-20 12:11:29.546740922 +0000 2020-11-20T12:14:59.319 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2020-11-20 12:11:29.546740922 +0000 2020-11-20T12:14:59.319 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2020-11-20T12:14:59.335 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2020-11-20T12:14:59.336 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2020-11-20T12:14:59.336 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000557863 s, 918 kB/s 2020-11-20T12:14:59.345 INFO:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2020-11-20T12:14:59.366 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2020-11-20T12:14:59.366 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:14:59.366 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 503 Links: 1 2020-11-20T12:14:59.367 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:14:59.367 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2020-11-20 12:11:30.118454909 +0000 2020-11-20T12:14:59.367 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2020-11-20 12:11:29.930548913 +0000 2020-11-20T12:14:59.367 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2020-11-20 12:11:29.930548913 +0000 2020-11-20T12:14:59.367 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2020-11-20T12:14:59.387 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2020-11-20T12:14:59.387 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2020-11-20T12:14:59.387 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000751689 s, 681 kB/s 2020-11-20T12:14:59.395 INFO:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2020-11-20T12:14:59.410 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2020-11-20T12:14:59.411 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:14:59.411 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 509 Links: 1 2020-11-20T12:14:59.411 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:14:59.411 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2020-11-20 12:11:30.522252902 +0000 2020-11-20T12:14:59.412 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2020-11-20 12:11:30.314356906 +0000 2020-11-20T12:14:59.412 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2020-11-20 12:11:30.314356906 +0000 2020-11-20T12:14:59.412 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2020-11-20T12:14:59.431 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2020-11-20T12:14:59.432 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2020-11-20T12:14:59.432 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000573566 s, 893 kB/s 2020-11-20T12:14:59.440 INFO:teuthology.orchestra.run.smithi173:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2020-11-20T12:14:59.457 INFO:teuthology.orchestra.run.smithi173.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2020-11-20T12:14:59.458 INFO:teuthology.orchestra.run.smithi173.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2020-11-20T12:14:59.458 INFO:teuthology.orchestra.run.smithi173.stdout:Device: 5h/5d Inode: 515 Links: 1 2020-11-20T12:14:59.458 INFO:teuthology.orchestra.run.smithi173.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2020-11-20T12:14:59.458 INFO:teuthology.orchestra.run.smithi173.stdout:Access: 2020-11-20 12:11:30.730148897 +0000 2020-11-20T12:14:59.458 INFO:teuthology.orchestra.run.smithi173.stdout:Modify: 2020-11-20 12:11:30.726150897 +0000 2020-11-20T12:14:59.459 INFO:teuthology.orchestra.run.smithi173.stdout:Change: 2020-11-20 12:11:30.726150897 +0000 2020-11-20T12:14:59.459 INFO:teuthology.orchestra.run.smithi173.stdout: Birth: - 2020-11-20T12:14:59.478 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records in 2020-11-20T12:14:59.478 INFO:teuthology.orchestra.run.smithi173.stderr:1+0 records out 2020-11-20T12:14:59.479 INFO:teuthology.orchestra.run.smithi173.stderr:512 bytes copied, 0.000696287 s, 735 kB/s 2020-11-20T12:14:59.489 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T12:14:59.490 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-20T12:14:59.490 INFO:tasks.ceph:Generating config... 2020-11-20T12:14:59.491 INFO:tasks.ceph:[client] client mount timeout = 600 2020-11-20T12:14:59.492 INFO:tasks.ceph:[client] debug client = 20 2020-11-20T12:14:59.492 INFO:tasks.ceph:[client] debug ms = 1 2020-11-20T12:14:59.492 INFO:tasks.ceph:[client] pid file = 2020-11-20T12:14:59.493 INFO:tasks.ceph:[client] rados mon op timeout = 15m 2020-11-20T12:14:59.493 INFO:tasks.ceph:[client] rados osd op timeout = 15m 2020-11-20T12:14:59.494 INFO:tasks.ceph:[global] ms die on skipped message = False 2020-11-20T12:14:59.494 INFO:tasks.ceph:[mds] debug mds = 20 2020-11-20T12:14:59.494 INFO:tasks.ceph:[mds] debug ms = 1 2020-11-20T12:14:59.494 INFO:tasks.ceph:[mds] mds bal frag = True 2020-11-20T12:14:59.495 INFO:tasks.ceph:[mds] mds bal fragment size max = 10000 2020-11-20T12:14:59.495 INFO:tasks.ceph:[mds] mds bal merge size = 5 2020-11-20T12:14:59.495 INFO:tasks.ceph:[mds] mds bal split bits = 3 2020-11-20T12:14:59.495 INFO:tasks.ceph:[mds] mds bal split size = 100 2020-11-20T12:14:59.496 INFO:tasks.ceph:[mds] mds debug frag = True 2020-11-20T12:14:59.496 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2020-11-20T12:14:59.496 INFO:tasks.ceph:[mds] mds op complaint time = 180 2020-11-20T12:14:59.497 INFO:tasks.ceph:[mds] mds verify scatter = True 2020-11-20T12:14:59.497 INFO:tasks.ceph:[mds] osd op complaint time = 180 2020-11-20T12:14:59.497 INFO:tasks.ceph:[mds] rados mon op timeout = 15m 2020-11-20T12:14:59.497 INFO:tasks.ceph:[mds] rados osd op timeout = 15m 2020-11-20T12:14:59.498 INFO:tasks.ceph:[mgr] debug mgr = 20 2020-11-20T12:14:59.498 INFO:tasks.ceph:[mgr] debug ms = 1 2020-11-20T12:14:59.498 INFO:tasks.ceph:[mon] debug mon = 20 2020-11-20T12:14:59.499 INFO:tasks.ceph:[mon] debug ms = 1 2020-11-20T12:14:59.499 INFO:tasks.ceph:[mon] debug paxos = 20 2020-11-20T12:14:59.499 INFO:tasks.ceph:[mon] mon op complaint time = 120 2020-11-20T12:14:59.499 INFO:tasks.ceph:[osd] bdev async discard = True 2020-11-20T12:14:59.500 INFO:tasks.ceph:[osd] bdev enable discard = True 2020-11-20T12:14:59.500 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2020-11-20T12:14:59.500 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2020-11-20T12:14:59.500 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2020-11-20T12:14:59.501 INFO:tasks.ceph:[osd] debug bluefs = 20 2020-11-20T12:14:59.503 INFO:tasks.ceph:[osd] debug bluestore = 20 2020-11-20T12:14:59.503 INFO:tasks.ceph:[osd] debug filestore = 20 2020-11-20T12:14:59.503 INFO:tasks.ceph:[osd] debug journal = 20 2020-11-20T12:14:59.504 INFO:tasks.ceph:[osd] debug ms = 20 2020-11-20T12:14:59.504 INFO:tasks.ceph:[osd] debug osd = 25 2020-11-20T12:14:59.504 INFO:tasks.ceph:[osd] debug rocksdb = 10 2020-11-20T12:14:59.504 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2020-11-20T12:14:59.505 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2020-11-20T12:14:59.505 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2020-11-20T12:14:59.505 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2020-11-20T12:14:59.506 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2020-11-20T12:14:59.506 INFO:tasks.ceph:[osd] osd op complaint time = 180 2020-11-20T12:14:59.506 INFO:tasks.ceph:Setting up mon.a... 2020-11-20T12:14:59.507 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2020-11-20T12:14:59.563 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.keyring 2020-11-20T12:14:59.569 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2020-11-20T12:14:59.625 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T12:14:59.652 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.116'), ('mon.b', '172.21.15.173'), ('mon.c', '[v2:172.21.15.173:3301,v1:172.21.15.173:6790]')] 2020-11-20T12:14:59.653 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug filestore': 20, 'debug journal': 20, 'debug ms': 20, 'debug osd': 25, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds bal frag': True, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}} 2020-11-20T12:14:59.654 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:14:59.654 INFO:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-20T12:14:59.665 INFO:teuthology.orchestra.run.smithi116.stderr:+ dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2020-11-20T12:14:59.668 INFO:teuthology.orchestra.run.smithi116.stderr:6+1 records in 2020-11-20T12:14:59.668 INFO:teuthology.orchestra.run.smithi116.stderr:6+1 records out 2020-11-20T12:14:59.668 INFO:teuthology.orchestra.run.smithi116.stderr:3569 bytes (3.6 kB, 3.5 KiB) copied, 0.000303524 s, 11.8 MB/s 2020-11-20T12:14:59.669 INFO:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.116 --add b 172.21.15.173 --addv c '[v2:172.21.15.173:3301,v1:172.21.15.173:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:14:59.725 INFO:teuthology.orchestra.run.smithi116.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:14:59.726 INFO:teuthology.orchestra.run.smithi116.stdout:monmaptool: generated fsid 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:14:59.726 INFO:teuthology.orchestra.run.smithi116.stdout:epoch 0 2020-11-20T12:14:59.726 INFO:teuthology.orchestra.run.smithi116.stdout:fsid 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:14:59.726 INFO:teuthology.orchestra.run.smithi116.stdout:last_changed 2020-11-20T12:14:59.725236+0000 2020-11-20T12:14:59.727 INFO:teuthology.orchestra.run.smithi116.stdout:created 2020-11-20T12:14:59.725236+0000 2020-11-20T12:14:59.727 INFO:teuthology.orchestra.run.smithi116.stdout:min_mon_release 0 (unknown) 2020-11-20T12:14:59.727 INFO:teuthology.orchestra.run.smithi116.stdout:election_strategy: 1 2020-11-20T12:14:59.727 INFO:teuthology.orchestra.run.smithi116.stdout:0: [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] mon.a 2020-11-20T12:14:59.727 INFO:teuthology.orchestra.run.smithi116.stdout:1: [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] mon.b 2020-11-20T12:14:59.728 INFO:teuthology.orchestra.run.smithi116.stdout:2: [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] mon.c 2020-11-20T12:14:59.728 INFO:teuthology.orchestra.run.smithi116.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2020-11-20T12:14:59.729 INFO:teuthology.orchestra.run.smithi116:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2020-11-20T12:14:59.744 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5ca3454c-e55c-402a-9a47-9fb04981c9e5... 2020-11-20T12:14:59.745 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:14:59.751 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:14:59.820 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:14:59.821 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:14:59.821 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:14:59.821 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:14:59.821 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.822 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:14:59.822 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.822 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:14:59.822 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.822 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:14:59.823 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:14:59.823 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.823 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:14:59.823 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:14:59.823 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.824 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:14:59.824 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:14:59.824 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.824 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:14:59.824 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:14:59.825 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.825 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:14:59.825 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.825 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:14:59.826 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:14:59.826 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:14:59.826 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:14:59.826 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:14:59.826 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:14:59.827 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:14:59.827 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:14:59.827 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:14:59.827 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:14:59.827 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.828 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:14:59.828 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.828 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:14:59.828 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.828 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:14:59.829 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.829 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:14:59.829 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:14:59.829 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:14:59.829 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:14:59.830 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:14:59.830 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:14:59.830 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.830 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:14:59.830 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:14:59.831 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.831 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:14:59.831 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:14:59.831 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.831 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:14:59.832 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:14:59.832 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.832 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:14:59.832 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:14:59.832 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:14:59.833 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.833 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:14:59.833 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:14:59.833 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.833 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:14:59.834 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.834 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:14:59.834 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.834 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:14:59.834 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.835 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:14:59.835 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:14:59.835 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.835 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:14:59.835 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:14:59.836 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:14:59.836 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:14:59.836 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:14:59.836 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:14:59.836 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:14:59.837 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:14:59.837 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:14:59.837 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:14:59.837 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:14:59.837 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:14:59.838 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:14:59.838 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:14:59.838 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:14:59.838 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:14:59.838 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:14:59.839 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:14:59.839 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:14:59.839 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:14:59.839 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.839 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:14:59.840 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:14:59.840 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:14:59.840 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:14:59.840 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:14:59.840 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:14:59.841 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:14:59.841 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:14:59.841 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.841 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:14:59.841 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:14:59.842 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:14:59.842 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:14:59.842 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:14:59.842 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:14:59.843 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:14:59.843 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:14:59.843 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:14:59.843 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:14:59.843 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:14:59.844 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:14:59.844 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:14:59.844 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:14:59.844 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:14:59.844 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:14:59.845 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:14:59.845 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:14:59.845 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:14:59.845 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:14:59.845 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:14:59.846 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:14:59.846 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:14:59.846 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:14:59.846 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:14:59.846 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:14:59.847 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:14:59.847 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:14:59.847 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:14:59.847 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:14:59.847 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:14:59.848 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:14:59.848 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:14:59.848 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:14:59.848 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:14:59.848 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:14:59.849 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:14:59.849 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:14:59.849 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:14:59.849 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:14:59.849 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:14:59.850 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:14:59.850 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:14:59.850 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:14:59.850 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:14:59.850 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:14:59.851 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:14:59.852 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:14:59.852 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:14:59.852 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:14:59.853 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:14:59.853 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.853 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:14:59.853 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.854 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:14:59.854 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.854 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:14:59.854 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:14:59.854 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.855 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:14:59.855 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:14:59.855 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.855 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:14:59.855 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:14:59.856 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.856 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:14:59.856 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:14:59.856 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.856 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:14:59.857 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.857 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:14:59.857 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:14:59.857 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:14:59.857 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:14:59.858 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:14:59.858 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:14:59.858 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:14:59.858 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:14:59.858 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:14:59.859 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:14:59.859 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.859 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:14:59.859 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.859 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:14:59.860 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.860 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:14:59.860 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.860 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:14:59.860 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:14:59.861 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:14:59.861 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:14:59.861 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:14:59.861 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:14:59.861 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.862 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:14:59.862 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:14:59.862 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.862 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:14:59.862 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:14:59.863 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.863 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:14:59.863 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:14:59.863 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.863 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:14:59.864 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:14:59.864 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:14:59.864 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.864 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:14:59.864 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:14:59.865 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.865 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:14:59.865 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.865 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:14:59.865 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.866 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:14:59.866 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.866 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:14:59.866 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:14:59.866 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.867 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:14:59.867 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:14:59.867 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:14:59.867 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:14:59.867 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:14:59.868 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:14:59.868 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:14:59.868 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:14:59.868 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:14:59.868 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:14:59.868 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:14:59.869 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:14:59.869 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:14:59.869 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:14:59.869 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:14:59.869 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:14:59.870 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:14:59.870 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:14:59.870 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:14:59.870 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:14:59.870 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.871 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:14:59.871 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:14:59.871 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:14:59.871 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:14:59.871 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:14:59.872 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:14:59.872 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:14:59.872 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:14:59.872 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.873 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:14:59.873 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:14:59.873 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:14:59.873 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:14:59.873 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:14:59.874 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:14:59.874 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:14:59.874 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:14:59.874 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:14:59.874 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:14:59.875 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:14:59.875 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:14:59.875 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:14:59.875 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:14:59.875 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:14:59.876 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:14:59.876 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:14:59.876 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:14:59.876 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:14:59.876 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:14:59.877 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:14:59.877 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:14:59.877 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:14:59.877 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:14:59.877 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:14:59.878 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:14:59.878 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:14:59.878 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:14:59.878 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:14:59.878 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:14:59.879 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:14:59.879 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:14:59.879 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:14:59.879 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:14:59.879 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:14:59.880 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:14:59.880 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:14:59.880 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:14:59.880 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:14:59.880 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:14:59.880 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:14:59.881 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:14:59.881 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:14:59.881 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:14:59.881 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:14:59.881 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:14:59.882 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:14:59.882 INFO:tasks.ceph:Creating admin key on mon.a... 2020-11-20T12:14:59.883 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2020-11-20T12:14:59.955 INFO:tasks.ceph:Copying monmap to all nodes... 2020-11-20T12:14:59.955 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:14:59.956 INFO:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-20T12:14:59.968 INFO:teuthology.orchestra.run.smithi116.stderr:+ dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2020-11-20T12:14:59.970 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.] 2020-11-20T12:14:59.971 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBDs7dfKCbUJBAAmXA+JXIw4zUt+m+mjI3SAg== 2020-11-20T12:14:59.971 INFO:teuthology.orchestra.run.smithi116.stdout:[client.admin] 2020-11-20T12:14:59.971 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBDs7dfh+KKOBAAZvcPXWly9/mxRE7NWuR85A== 2020-11-20T12:14:59.971 INFO:teuthology.orchestra.run.smithi116.stdout: caps mds = "allow *" 2020-11-20T12:14:59.972 INFO:teuthology.orchestra.run.smithi116.stdout: caps mgr = "allow *" 2020-11-20T12:14:59.972 INFO:teuthology.orchestra.run.smithi116.stdout: caps mon = "allow *" 2020-11-20T12:14:59.972 INFO:teuthology.orchestra.run.smithi116.stdout: caps osd = "allow *" 2020-11-20T12:14:59.972 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:14:59.972 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:14:59.973 INFO:teuthology.orchestra.run.smithi116.stderr:206 bytes copied, 0.000197888 s, 1.0 MB/s 2020-11-20T12:14:59.973 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:14:59.973 INFO:teuthology.orchestra.run.smithi116:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-20T12:14:59.984 INFO:teuthology.orchestra.run.smithi116.stderr:+ dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2020-11-20T12:14:59.986 INFO:teuthology.orchestra.run.smithi116.stdout: �\�EL�\@*�G��I���C��_�;:+C��_�;:+�aXa �t��tbXb �����cXc �����abc 2020-11-20T12:14:59.987 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:14:59.987 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:14:59.987 INFO:teuthology.orchestra.run.smithi116.stderr:409 bytes copied, 0.000297627 s, 1.4 MB/s 2020-11-20T12:14:59.988 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:14:59.988 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:14:59.988 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T12:14:59.989 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T12:15:00.041 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T12:15:00.060 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:00.061 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:00.061 INFO:teuthology.orchestra.run.smithi116.stderr:206 bytes copied, 0.000304061 s, 677 kB/s 2020-11-20T12:15:00.062 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T12:15:00.081 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:00.082 INFO:teuthology.orchestra.run.smithi116:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:00.091 INFO:teuthology.orchestra.run.smithi116.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:00.094 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:00.094 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:00.094 INFO:teuthology.orchestra.run.smithi116.stderr:409 bytes copied, 0.000232981 s, 1.8 MB/s 2020-11-20T12:15:00.095 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:15:00.095 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:00.095 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T12:15:00.096 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T12:15:00.110 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/ceph/ceph.keyring 2020-11-20T12:15:00.131 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:00.131 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:00.131 INFO:teuthology.orchestra.run.smithi173.stderr:206 bytes copied, 0.000196891 s, 1.0 MB/s 2020-11-20T12:15:00.133 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo chmod 0644 /etc/ceph/ceph.keyring 2020-11-20T12:15:00.152 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:00.153 INFO:teuthology.orchestra.run.smithi173:> dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:00.165 INFO:teuthology.orchestra.run.smithi173.stderr:+ dd of=/home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:00.168 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:00.168 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:00.169 INFO:teuthology.orchestra.run.smithi173.stderr:409 bytes copied, 0.000164464 s, 2.5 MB/s 2020-11-20T12:15:00.169 INFO:tasks.ceph:Setting up mon nodes... 2020-11-20T12:15:00.170 INFO:tasks.ceph:Setting up mgr nodes... 2020-11-20T12:15:00.170 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2020-11-20T12:15:00.257 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2020-11-20T12:15:00.264 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2020-11-20T12:15:00.346 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2020-11-20T12:15:00.353 INFO:tasks.ceph:Setting up mds nodes... 2020-11-20T12:15:00.353 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2020-11-20T12:15:00.442 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2020-11-20T12:15:00.448 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2020-11-20T12:15:00.476 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2020-11-20T12:15:00.603 INFO:teuthology.orchestra.run.smithi116.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2020-11-20T12:15:00.609 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2020-11-20T12:15:00.641 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2020-11-20T12:15:00.725 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2020-11-20T12:15:00.732 INFO:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2020-11-20T12:15:00.761 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2020-11-20T12:15:00.844 INFO:teuthology.orchestra.run.smithi173.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2020-11-20T12:15:00.851 INFO:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2020-11-20T12:15:00.883 INFO:tasks.ceph_client:Setting up client nodes... 2020-11-20T12:15:00.884 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2020-11-20T12:15:00.947 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.client.0.keyring 2020-11-20T12:15:00.967 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2020-11-20T12:15:01.026 INFO:teuthology.orchestra.run.smithi116.stdout:creating /etc/ceph/ceph.client.2.keyring 2020-11-20T12:15:01.050 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2020-11-20T12:15:01.114 INFO:teuthology.orchestra.run.smithi173.stdout:creating /etc/ceph/ceph.client.1.keyring 2020-11-20T12:15:01.140 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2020-11-20T12:15:01.195 INFO:teuthology.orchestra.run.smithi173.stdout:creating /etc/ceph/ceph.client.3.keyring 2020-11-20T12:15:01.219 INFO:tasks.ceph:Running mkfs on osd nodes... 2020-11-20T12:15:01.219 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi116.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi173.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2020-11-20T12:15:01.220 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2020-11-20T12:15:01.253 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T12:15:01.253 INFO:tasks.ceph:role: osd.0 2020-11-20T12:15:01.253 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:15:01.254 INFO:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-20T12:15:01.737 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:15:01.738 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:15:01.738 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-20T12:15:01.738 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:15:01.739 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:15:01.739 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-20T12:15:01.739 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:15:01.739 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:15:01.739 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:15:01.743 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2020-11-20T12:15:01.744 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2020-11-20T12:15:01.845 INFO:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2020-11-20T12:15:01.870 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2020-11-20T12:15:01.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:15:01.873 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2020-11-20T12:15:01.905 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T12:15:01.906 INFO:tasks.ceph:role: osd.1 2020-11-20T12:15:01.906 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:15:01.906 INFO:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-20T12:15:02.459 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:15:02.459 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:15:02.460 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-20T12:15:02.460 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:15:02.460 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:15:02.460 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-20T12:15:02.460 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:15:02.461 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:15:02.461 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:15:02.464 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2020-11-20T12:15:02.465 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2020-11-20T12:15:02.519 INFO:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2020-11-20T12:15:02.536 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2020-11-20T12:15:02.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:15:02.538 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2020-11-20T12:15:02.616 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T12:15:02.617 INFO:tasks.ceph:role: osd.2 2020-11-20T12:15:02.617 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:15:02.618 INFO:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-20T12:15:03.109 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:15:03.109 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:15:03.110 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-20T12:15:03.110 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:15:03.110 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:15:03.110 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-20T12:15:03.111 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:15:03.111 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:15:03.112 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:15:03.114 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2020-11-20T12:15:03.114 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2020-11-20T12:15:03.187 INFO:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2020-11-20T12:15:03.211 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2020-11-20T12:15:03.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:15:03.213 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2020-11-20T12:15:03.247 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2020-11-20T12:15:03.248 INFO:tasks.ceph:role: osd.3 2020-11-20T12:15:03.248 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:15:03.249 INFO:teuthology.orchestra.run.smithi116:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-20T12:15:03.806 INFO:teuthology.orchestra.run.smithi116.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:15:03.806 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:15:03.806 INFO:teuthology.orchestra.run.smithi116.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-20T12:15:03.807 INFO:teuthology.orchestra.run.smithi116.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:15:03.807 INFO:teuthology.orchestra.run.smithi116.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:15:03.807 INFO:teuthology.orchestra.run.smithi116.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-20T12:15:03.807 INFO:teuthology.orchestra.run.smithi116.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:15:03.808 INFO:teuthology.orchestra.run.smithi116.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:15:03.808 INFO:teuthology.orchestra.run.smithi116.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:15:03.810 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi116.front.sepia.ceph.com -o noatime 2020-11-20T12:15:03.811 INFO:teuthology.orchestra.run.smithi116:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2020-11-20T12:15:03.887 INFO:teuthology.orchestra.run.smithi116:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2020-11-20T12:15:03.909 INFO:teuthology.orchestra.run.smithi116.stderr:sudo: /sbin/restorecon: command not found 2020-11-20T12:15:03.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:15:03.910 INFO:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:03.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:03.996+0000 7f07ff2a7d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2020-11-20T12:15:04.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:03.996+0000 7f07ff2a7d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2020-11-20T12:15:04.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:03.996+0000 7f07ff2a7d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T12:15:04.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:04.000+0000 7f07ff2a7d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T12:15:04.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:04.000+0000 7f07ff2a7d80 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2020-11-20T12:15:04.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:04.008+0000 7f07ff2a7d80 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2020-11-20T12:15:07.029 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2020-11-20T12:15:07.069 INFO:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:07.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:07.188+0000 7fd8f4b05d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2020-11-20T12:15:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:07.188+0000 7fd8f4b05d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2020-11-20T12:15:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:07.188+0000 7fd8f4b05d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T12:15:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:07.188+0000 7fd8f4b05d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T12:15:07.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:07.188+0000 7fd8f4b05d80 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2020-11-20T12:15:07.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:07.188+0000 7fd8f4b05d80 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2020-11-20T12:15:10.207 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2020-11-20T12:15:10.240 INFO:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:10.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:10.348+0000 7fa557e83d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2020-11-20T12:15:10.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:10.352+0000 7fa557e83d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2020-11-20T12:15:10.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:10.352+0000 7fa557e83d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T12:15:10.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:10.352+0000 7fa557e83d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T12:15:10.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:10.352+0000 7fa557e83d80 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2020-11-20T12:15:10.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:10.352+0000 7fa557e83d80 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2020-11-20T12:15:13.362 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2020-11-20T12:15:13.397 INFO:teuthology.orchestra.run.smithi116:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:13.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:13.504+0000 7f3e5dfabd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2020-11-20T12:15:13.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:13.504+0000 7f3e5dfabd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2020-11-20T12:15:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:13.504+0000 7f3e5dfabd80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T12:15:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:13.504+0000 7f3e5dfabd80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T12:15:13.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:13.504+0000 7f3e5dfabd80 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2020-11-20T12:15:13.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:13.508+0000 7f3e5dfabd80 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2020-11-20T12:15:16.521 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2020-11-20T12:15:16.552 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2020-11-20T12:15:16.585 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T12:15:16.585 INFO:tasks.ceph:role: osd.4 2020-11-20T12:15:16.586 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:15:16.586 INFO:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2020-11-20T12:15:17.096 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:15:17.096 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:15:17.096 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-20T12:15:17.097 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:15:17.097 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:15:17.097 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-20T12:15:17.097 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:15:17.097 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:15:17.098 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:15:17.103 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2020-11-20T12:15:17.104 INFO:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2020-11-20T12:15:17.203 INFO:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2020-11-20T12:15:17.220 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2020-11-20T12:15:17.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:15:17.222 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2020-11-20T12:15:17.253 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T12:15:17.254 INFO:tasks.ceph:role: osd.5 2020-11-20T12:15:17.254 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:15:17.255 INFO:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2020-11-20T12:15:17.767 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:15:17.768 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:15:17.768 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-20T12:15:17.768 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:15:17.768 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:15:17.768 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-20T12:15:17.769 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:15:17.769 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:15:17.769 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:15:17.773 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2020-11-20T12:15:17.774 INFO:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2020-11-20T12:15:17.831 INFO:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2020-11-20T12:15:17.855 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2020-11-20T12:15:17.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:15:17.857 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2020-11-20T12:15:17.889 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T12:15:17.889 INFO:tasks.ceph:role: osd.6 2020-11-20T12:15:17.890 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:15:17.890 INFO:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2020-11-20T12:15:18.432 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:15:18.433 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:15:18.433 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-20T12:15:18.433 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:15:18.433 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:15:18.434 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-20T12:15:18.434 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:15:18.434 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:15:18.434 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:15:18.438 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2020-11-20T12:15:18.438 INFO:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2020-11-20T12:15:18.512 INFO:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2020-11-20T12:15:18.538 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2020-11-20T12:15:18.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:15:18.539 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2020-11-20T12:15:18.567 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2020-11-20T12:15:18.567 INFO:tasks.ceph:role: osd.7 2020-11-20T12:15:18.568 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:15:18.568 INFO:teuthology.orchestra.run.smithi173:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2020-11-20T12:15:19.119 INFO:teuthology.orchestra.run.smithi173.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2020-11-20T12:15:19.119 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 attr=2, projid32bit=1 2020-11-20T12:15:19.120 INFO:teuthology.orchestra.run.smithi173.stdout: = crc=1 finobt=1, sparse=0, rmapbt=0, reflink=0 2020-11-20T12:15:19.120 INFO:teuthology.orchestra.run.smithi173.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2020-11-20T12:15:19.120 INFO:teuthology.orchestra.run.smithi173.stdout: = sunit=0 swidth=0 blks 2020-11-20T12:15:19.120 INFO:teuthology.orchestra.run.smithi173.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2020-11-20T12:15:19.120 INFO:teuthology.orchestra.run.smithi173.stdout:log =internal log bsize=4096 blocks=11443, version=2 2020-11-20T12:15:19.121 INFO:teuthology.orchestra.run.smithi173.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2020-11-20T12:15:19.121 INFO:teuthology.orchestra.run.smithi173.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2020-11-20T12:15:19.127 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi173.front.sepia.ceph.com -o noatime 2020-11-20T12:15:19.127 INFO:teuthology.orchestra.run.smithi173:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2020-11-20T12:15:19.238 INFO:teuthology.orchestra.run.smithi173:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2020-11-20T12:15:19.269 INFO:teuthology.orchestra.run.smithi173.stderr:sudo: /sbin/restorecon: command not found 2020-11-20T12:15:19.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:15:19.271 INFO:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:19.402 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:19.402+0000 7f9a4e2cdd80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2020-11-20T12:15:19.407 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:19.406+0000 7f9a4e2cdd80 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2020-11-20T12:15:19.407 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:19.406+0000 7f9a4e2cdd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T12:15:19.407 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:19.406+0000 7f9a4e2cdd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T12:15:19.407 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:19.406+0000 7f9a4e2cdd80 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2020-11-20T12:15:19.418 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:19.418+0000 7f9a4e2cdd80 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2020-11-20T12:15:22.472 INFO:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2020-11-20T12:15:22.505 INFO:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:22.570 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:22.570+0000 7f49b27f3d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2020-11-20T12:15:22.571 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:22.570+0000 7f49b27f3d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2020-11-20T12:15:22.571 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:22.570+0000 7f49b27f3d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T12:15:22.572 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:22.570+0000 7f49b27f3d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T12:15:22.572 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:22.570+0000 7f49b27f3d80 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2020-11-20T12:15:22.574 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:22.574+0000 7f49b27f3d80 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2020-11-20T12:15:25.567 INFO:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2020-11-20T12:15:26.831 INFO:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:26.943 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:26.942+0000 7fd10d3b9d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2020-11-20T12:15:26.945 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:26.942+0000 7fd10d3b9d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2020-11-20T12:15:26.945 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:26.942+0000 7fd10d3b9d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T12:15:26.945 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:26.942+0000 7fd10d3b9d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T12:15:26.946 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:26.942+0000 7fd10d3b9d80 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2020-11-20T12:15:26.947 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:26.946+0000 7fd10d3b9d80 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2020-11-20T12:15:28.780 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:15:28.787 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:15:29.965 INFO:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2020-11-20T12:15:29.997 INFO:teuthology.orchestra.run.smithi173:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:30.106 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:30.106+0000 7f3b7baa0d80 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2020-11-20T12:15:30.107 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:30.106+0000 7f3b7baa0d80 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2020-11-20T12:15:30.108 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:30.106+0000 7f3b7baa0d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T12:15:30.108 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:30.106+0000 7f3b7baa0d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T12:15:30.108 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:30.106+0000 7f3b7baa0d80 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2020-11-20T12:15:30.111 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:30.110+0000 7f3b7baa0d80 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2020-11-20T12:15:33.143 INFO:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2020-11-20T12:15:33.178 INFO:tasks.ceph:Reading keys from all nodes... 2020-11-20T12:15:33.178 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:33.178 INFO:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-20T12:15:33.195 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2020-11-20T12:15:33.213 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.y] 2020-11-20T12:15:33.214 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA== 2020-11-20T12:15:33.214 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:33.214 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:33.215 INFO:teuthology.orchestra.run.smithi116.stderr:56 bytes copied, 0.0005435 s, 103 kB/s 2020-11-20T12:15:33.215 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:33.215 INFO:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2020-11-20T12:15:33.269 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2020-11-20T12:15:33.284 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:15:33.285 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw== 2020-11-20T12:15:33.285 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:33.285 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:33.285 INFO:teuthology.orchestra.run.smithi116.stderr:56 bytes copied, 0.00043187 s, 130 kB/s 2020-11-20T12:15:33.287 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:33.288 INFO:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2020-11-20T12:15:33.302 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2020-11-20T12:15:33.316 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:15:33.316 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag== 2020-11-20T12:15:33.318 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:33.318 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:33.318 INFO:teuthology.orchestra.run.smithi116.stderr:56 bytes copied, 0.000395149 s, 142 kB/s 2020-11-20T12:15:33.321 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:33.321 INFO:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-20T12:15:33.377 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2020-11-20T12:15:33.390 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.0] 2020-11-20T12:15:33.391 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ== 2020-11-20T12:15:33.391 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:33.391 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:33.392 INFO:teuthology.orchestra.run.smithi116.stderr:56 bytes copied, 0.000318868 s, 176 kB/s 2020-11-20T12:15:33.393 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:33.393 INFO:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-20T12:15:33.404 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2020-11-20T12:15:33.418 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.1] 2020-11-20T12:15:33.419 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q== 2020-11-20T12:15:33.419 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:33.419 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:33.419 INFO:teuthology.orchestra.run.smithi116.stderr:56 bytes copied, 0.000370847 s, 151 kB/s 2020-11-20T12:15:33.421 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:33.421 INFO:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-20T12:15:33.433 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2020-11-20T12:15:33.447 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.2] 2020-11-20T12:15:33.447 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw== 2020-11-20T12:15:33.448 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:33.448 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:33.448 INFO:teuthology.orchestra.run.smithi116.stderr:56 bytes copied, 0.000416464 s, 134 kB/s 2020-11-20T12:15:33.449 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:33.450 INFO:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2020-11-20T12:15:33.462 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2020-11-20T12:15:33.476 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.3] 2020-11-20T12:15:33.476 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w== 2020-11-20T12:15:33.477 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:33.477 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:33.477 INFO:teuthology.orchestra.run.smithi116.stderr:56 bytes copied, 0.000465734 s, 120 kB/s 2020-11-20T12:15:33.480 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:33.480 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-20T12:15:33.495 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2020-11-20T12:15:33.509 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr.x] 2020-11-20T12:15:33.510 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A== 2020-11-20T12:15:33.510 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:33.510 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:33.510 INFO:teuthology.orchestra.run.smithi173.stderr:56 bytes copied, 0.000619091 s, 90.5 kB/s 2020-11-20T12:15:33.511 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:33.512 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2020-11-20T12:15:33.566 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2020-11-20T12:15:33.580 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:15:33.580 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ== 2020-11-20T12:15:33.581 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:33.581 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:33.581 INFO:teuthology.orchestra.run.smithi173.stderr:56 bytes copied, 0.000431221 s, 130 kB/s 2020-11-20T12:15:33.582 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:33.583 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2020-11-20T12:15:33.594 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2020-11-20T12:15:33.607 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:15:33.608 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ== 2020-11-20T12:15:33.608 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:33.608 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:33.608 INFO:teuthology.orchestra.run.smithi173.stderr:56 bytes copied, 0.000414384 s, 135 kB/s 2020-11-20T12:15:33.610 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:33.610 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2020-11-20T12:15:33.621 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2020-11-20T12:15:33.638 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.4] 2020-11-20T12:15:33.639 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA== 2020-11-20T12:15:33.639 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:33.639 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:33.640 INFO:teuthology.orchestra.run.smithi173.stderr:56 bytes copied, 0.000188528 s, 297 kB/s 2020-11-20T12:15:33.641 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:33.641 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2020-11-20T12:15:33.653 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2020-11-20T12:15:33.668 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.5] 2020-11-20T12:15:33.668 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ== 2020-11-20T12:15:33.669 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:33.669 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:33.669 INFO:teuthology.orchestra.run.smithi173.stderr:56 bytes copied, 0.000367269 s, 152 kB/s 2020-11-20T12:15:33.670 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:33.671 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2020-11-20T12:15:33.683 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2020-11-20T12:15:33.697 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.6] 2020-11-20T12:15:33.697 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg== 2020-11-20T12:15:33.698 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:33.698 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:33.698 INFO:teuthology.orchestra.run.smithi173.stderr:56 bytes copied, 0.00042202 s, 133 kB/s 2020-11-20T12:15:33.699 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:33.700 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2020-11-20T12:15:33.754 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2020-11-20T12:15:33.768 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.7] 2020-11-20T12:15:33.769 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg== 2020-11-20T12:15:33.769 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:33.769 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:33.769 INFO:teuthology.orchestra.run.smithi173.stderr:56 bytes copied, 0.000341106 s, 164 kB/s 2020-11-20T12:15:33.771 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:33.771 INFO:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-20T12:15:33.784 INFO:teuthology.orchestra.run.smithi116.stderr:+ dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2020-11-20T12:15:33.786 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2020-11-20T12:15:33.786 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA== 2020-11-20T12:15:33.787 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:33.787 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:33.787 INFO:teuthology.orchestra.run.smithi116.stderr:59 bytes copied, 0.000170702 s, 346 kB/s 2020-11-20T12:15:33.787 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:33.788 INFO:teuthology.orchestra.run.smithi116:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2020-11-20T12:15:33.799 INFO:teuthology.orchestra.run.smithi116.stderr:+ dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2020-11-20T12:15:33.801 INFO:teuthology.orchestra.run.smithi116.stdout:[client.2] 2020-11-20T12:15:33.801 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ== 2020-11-20T12:15:33.802 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:33.802 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:33.802 INFO:teuthology.orchestra.run.smithi116.stderr:59 bytes copied, 0.000266496 s, 221 kB/s 2020-11-20T12:15:33.803 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:33.803 INFO:teuthology.orchestra.run.smithi173:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2020-11-20T12:15:33.817 INFO:teuthology.orchestra.run.smithi173.stderr:+ dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2020-11-20T12:15:33.819 INFO:teuthology.orchestra.run.smithi173.stdout:[client.1] 2020-11-20T12:15:33.819 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow== 2020-11-20T12:15:33.820 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:33.820 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:33.820 INFO:teuthology.orchestra.run.smithi173.stderr:59 bytes copied, 0.000190579 s, 310 kB/s 2020-11-20T12:15:33.821 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:33.821 INFO:teuthology.orchestra.run.smithi173:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2020-11-20T12:15:33.874 INFO:teuthology.orchestra.run.smithi173.stderr:+ dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2020-11-20T12:15:33.877 INFO:teuthology.orchestra.run.smithi173.stdout:[client.3] 2020-11-20T12:15:33.878 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ== 2020-11-20T12:15:33.878 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:33.878 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:33.878 INFO:teuthology.orchestra.run.smithi173.stderr:59 bytes copied, 0.000435895 s, 135 kB/s 2020-11-20T12:15:33.879 INFO:tasks.ceph:Adding keys to all mons... 2020-11-20T12:15:33.880 INFO:teuthology.orchestra.run.smithi116:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-20T12:15:33.886 INFO:teuthology.orchestra.run.smithi173:> sudo tee -a /etc/ceph/ceph.keyring 2020-11-20T12:15:33.908 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.y] 2020-11-20T12:15:33.908 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA== 2020-11-20T12:15:33.908 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:15:33.909 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw== 2020-11-20T12:15:33.909 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:15:33.909 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag== 2020-11-20T12:15:33.909 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.0] 2020-11-20T12:15:33.910 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ== 2020-11-20T12:15:33.910 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.1] 2020-11-20T12:15:33.910 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q== 2020-11-20T12:15:33.910 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.2] 2020-11-20T12:15:33.910 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw== 2020-11-20T12:15:33.911 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.3] 2020-11-20T12:15:33.911 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w== 2020-11-20T12:15:33.911 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr.x] 2020-11-20T12:15:33.911 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A== 2020-11-20T12:15:33.912 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:15:33.912 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ== 2020-11-20T12:15:33.912 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:15:33.912 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ== 2020-11-20T12:15:33.913 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.4] 2020-11-20T12:15:33.913 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA== 2020-11-20T12:15:33.913 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.5] 2020-11-20T12:15:33.913 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ== 2020-11-20T12:15:33.913 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.6] 2020-11-20T12:15:33.914 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg== 2020-11-20T12:15:33.914 INFO:teuthology.orchestra.run.smithi116.stdout:[osd.7] 2020-11-20T12:15:33.914 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg== 2020-11-20T12:15:33.914 INFO:teuthology.orchestra.run.smithi116.stdout:[client.0] 2020-11-20T12:15:33.914 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA== 2020-11-20T12:15:33.915 INFO:teuthology.orchestra.run.smithi116.stdout:[client.2] 2020-11-20T12:15:33.915 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ== 2020-11-20T12:15:33.915 INFO:teuthology.orchestra.run.smithi116.stdout:[client.1] 2020-11-20T12:15:33.915 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow== 2020-11-20T12:15:33.915 INFO:teuthology.orchestra.run.smithi116.stdout:[client.3] 2020-11-20T12:15:33.916 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ== 2020-11-20T12:15:33.917 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr.y] 2020-11-20T12:15:33.917 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA== 2020-11-20T12:15:33.918 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:15:33.918 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw== 2020-11-20T12:15:33.918 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:15:33.918 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag== 2020-11-20T12:15:33.919 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.0] 2020-11-20T12:15:33.919 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ== 2020-11-20T12:15:33.919 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.1] 2020-11-20T12:15:33.919 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q== 2020-11-20T12:15:33.919 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.2] 2020-11-20T12:15:33.920 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw== 2020-11-20T12:15:33.920 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.3] 2020-11-20T12:15:33.920 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w== 2020-11-20T12:15:33.920 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr.x] 2020-11-20T12:15:33.921 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A== 2020-11-20T12:15:33.921 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:15:33.922 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ== 2020-11-20T12:15:33.922 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:15:33.922 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ== 2020-11-20T12:15:33.922 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.4] 2020-11-20T12:15:33.922 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA== 2020-11-20T12:15:33.923 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.5] 2020-11-20T12:15:33.923 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ== 2020-11-20T12:15:33.923 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.6] 2020-11-20T12:15:33.923 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg== 2020-11-20T12:15:33.923 INFO:teuthology.orchestra.run.smithi173.stdout:[osd.7] 2020-11-20T12:15:33.924 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg== 2020-11-20T12:15:33.924 INFO:teuthology.orchestra.run.smithi173.stdout:[client.0] 2020-11-20T12:15:33.924 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA== 2020-11-20T12:15:33.924 INFO:teuthology.orchestra.run.smithi173.stdout:[client.2] 2020-11-20T12:15:33.924 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ== 2020-11-20T12:15:33.925 INFO:teuthology.orchestra.run.smithi173.stdout:[client.1] 2020-11-20T12:15:33.925 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow== 2020-11-20T12:15:33.925 INFO:teuthology.orchestra.run.smithi173.stdout:[client.3] 2020-11-20T12:15:33.925 INFO:teuthology.orchestra.run.smithi173.stdout: key = AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ== 2020-11-20T12:15:33.926 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T12:15:33.932 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T12:15:34.004 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:15:34.009 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:15:34.080 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:15:34.086 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:15:34.154 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.160 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.228 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.241 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.309 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.315 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.382 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.388 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.455 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T12:15:34.461 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2020-11-20T12:15:34.528 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:15:34.534 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:15:34.603 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:15:34.609 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2020-11-20T12:15:34.679 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.685 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.753 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.757 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.824 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.830 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.897 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.903 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' 2020-11-20T12:15:34.969 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T12:15:34.975 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T12:15:35.046 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T12:15:35.052 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T12:15:35.123 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T12:15:35.128 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T12:15:35.202 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T12:15:35.217 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2020-11-20T12:15:35.294 INFO:tasks.ceph:Running mkfs on mon nodes... 2020-11-20T12:15:35.295 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2020-11-20T12:15:35.328 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T12:15:35.424 INFO:teuthology.orchestra.run.smithi116:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2020-11-20T12:15:35.456 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2020-11-20T12:15:35.490 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T12:15:35.574 INFO:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2020-11-20T12:15:35.614 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2020-11-20T12:15:35.692 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2020-11-20T12:15:35.829 INFO:teuthology.orchestra.run.smithi173:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2020-11-20T12:15:35.870 INFO:teuthology.orchestra.run.smithi116:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:35.878 INFO:teuthology.orchestra.run.smithi173:> rm -- /home/ubuntu/cephtest/ceph.monmap 2020-11-20T12:15:35.930 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2020-11-20T12:15:35.931 INFO:tasks.ceph.mon.a:Restarting daemon 2020-11-20T12:15:35.932 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2020-11-20T12:15:35.938 INFO:tasks.ceph.mon.a:Started 2020-11-20T12:15:35.938 INFO:tasks.ceph.mon.b:Restarting daemon 2020-11-20T12:15:35.939 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2020-11-20T12:15:35.945 INFO:tasks.ceph.mon.b:Started 2020-11-20T12:15:35.945 INFO:tasks.ceph.mon.c:Restarting daemon 2020-11-20T12:15:35.946 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2020-11-20T12:15:35.952 INFO:tasks.ceph.mon.c:Started 2020-11-20T12:15:35.952 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2020-11-20T12:15:35.953 INFO:tasks.ceph.mgr.y:Restarting daemon 2020-11-20T12:15:35.953 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2020-11-20T12:15:35.957 INFO:tasks.ceph.mgr.y:Started 2020-11-20T12:15:35.958 INFO:tasks.ceph.mgr.x:Restarting daemon 2020-11-20T12:15:35.958 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2020-11-20T12:15:35.962 INFO:tasks.ceph.mgr.x:Started 2020-11-20T12:15:35.963 INFO:tasks.ceph:Setting crush tunables to default 2020-11-20T12:15:35.963 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd crush tunables default 2020-11-20T12:15:36.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.252+0000 7f539b040700 1 Processor -- start 2020-11-20T12:15:36.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.252+0000 7f539b040700 1 -- start start 2020-11-20T12:15:36.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.256+0000 7f539b040700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:36.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.256+0000 7f539b040700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53940c1900 con 0x7f539414c050 2020-11-20T12:15:36.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.256+0000 7f539b040700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5394155530 con 0x7f539414bc70 2020-11-20T12:15:36.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.256+0000 7f539b040700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53941557d0 con 0x7f53940c1300 2020-11-20T12:15:36.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.256+0000 7f53995dd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.256+0000 7f53995dd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42354/0 (socket says 172.21.15.116:42354) 2020-11-20T12:15:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.256+0000 7f53995dd700 1 -- 172.21.15.116:0/1927550571 learned_addr learned my addr 172.21.15.116:0/1927550571 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.256+0000 7f53995dd700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 msgr2=0x7f539415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:15:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.256+0000 7f53995dd700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 msgr2=0x7f539415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:36.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.256+0000 7f53995dd700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:36.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.256+0000 7f53995dd700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:36.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.460+0000 7f53995dd700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.460+0000 7f53995dd700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 msgr2=0x7f539415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:15:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.460+0000 7f53995dd700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 msgr2=0x7f539415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.460+0000 7f53995dd700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:36.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.460+0000 7f53995dd700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:15:36.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.864+0000 7f53995dd700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:36.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.864+0000 7f53995dd700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 msgr2=0x7f539415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:15:36.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.864+0000 7f53995dd700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 msgr2=0x7f539415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:37.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.864+0000 7f53995dd700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:37.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:36.864+0000 7f53995dd700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:15:37.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:37.668+0000 7f53995dd700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:37.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:37.668+0000 7f53995dd700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 msgr2=0x7f539415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2020-11-20T12:15:37.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:37.668+0000 7f53995dd700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 msgr2=0x7f539415c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:37.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:37.668+0000 7f53995dd700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:37.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:37.668+0000 7f53995dd700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:15:39.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f53927fc700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 msgr2=0x7f539415c5b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:39.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f53927fc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f539415c5b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:39.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f53927fc700 1 -- 172.21.15.116:0/1927550571 >> v1:172.21.15.173:6789/0 conn(0x7f539414bc70 legacy=0x7f53940678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:39.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f53927fc700 1 -- 172.21.15.116:0/1927550571 >> v1:172.21.15.116:6789/0 conn(0x7f539414c050 legacy=0x7f53940c0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:39.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f53927fc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:39.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f53927fc700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:39.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f53927fc700 1 -- 172.21.15.116:0/1927550571 --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53941557d0 con 0x7f537c0033b0 2020-11-20T12:15:39.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f53927fc700 1 -- 172.21.15.116:0/1927550571 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f537c008a40 con 0x7f537c005eb0 2020-11-20T12:15:39.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:39.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:39.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f5393fff700 1 -- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 msgr2=0x7f537c008370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:15:39.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f5393fff700 1 -- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 msgr2=0x7f537c008370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:39.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:39.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f5398ddc700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 msgr2=0x7f537c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:15:39.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f5398ddc700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 msgr2=0x7f537c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:39.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:39.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:39.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.256+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2020-11-20T12:15:39.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.460+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.460+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:39.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.460+0000 7f5393fff700 1 -- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 msgr2=0x7f537c008370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:15:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.460+0000 7f5393fff700 1 -- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 msgr2=0x7f537c008370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.460+0000 7f5398ddc700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 msgr2=0x7f537c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:15:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.460+0000 7f5398ddc700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 msgr2=0x7f537c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:39.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.460+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.460+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.460+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:15:39.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.460+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.400000 2020-11-20T12:15:39.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.864+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:39.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.864+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:39.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.864+0000 7f5398ddc700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 msgr2=0x7f537c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:15:39.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.864+0000 7f5398ddc700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 msgr2=0x7f537c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:39.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.864+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:39.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.864+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:15:39.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.864+0000 7f5393fff700 1 -- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 msgr2=0x7f537c008370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:15:39.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.864+0000 7f5393fff700 1 -- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 msgr2=0x7f537c008370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:39.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.864+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:39.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:39.864+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.800000 2020-11-20T12:15:40.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:40.668+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:40.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:40.668+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:40.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:40.668+0000 7f5393fff700 1 -- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 msgr2=0x7f537c008370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2020-11-20T12:15:40.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:40.668+0000 7f5393fff700 1 -- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 msgr2=0x7f537c008370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:40.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:40.668+0000 7f5398ddc700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 msgr2=0x7f537c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2020-11-20T12:15:40.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:40.668+0000 7f5398ddc700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 msgr2=0x7f537c005830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2020-11-20T12:15:40.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:40.668+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:40.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:40.668+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2020-11-20T12:15:40.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:40.668+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:15:40.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:40.668+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 1.600000 2020-11-20T12:15:42.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f53927fc700 1 -- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 msgr2=0x7f537c008370 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:42.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f53927fc700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c005eb0 0x7f537c008370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:42.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f53927fc700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 msgr2=0x7f537c005830 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:15:42.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f53927fc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c0033b0 0x7f537c005830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:42.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f53995dd700 1 -- 172.21.15.116:0/1927550571 reap_dead start 2020-11-20T12:15:42.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f53927fc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f537c00aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:42.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f53927fc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c006340 0x7f537c003790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:42.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f53927fc700 1 -- 172.21.15.116:0/1927550571 --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5394155a70 con 0x7f537c003c80 2020-11-20T12:15:42.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f53927fc700 1 -- 172.21.15.116:0/1927550571 --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53941557d0 con 0x7f53940c1300 2020-11-20T12:15:42.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f53927fc700 1 -- 172.21.15.116:0/1927550571 --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f537c008a40 con 0x7f537c006340 2020-11-20T12:15:42.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.116:3300/0 conn(0x7f53940c1300 0x7f537c00aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:42.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f5393fff700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c006340 0x7f537c003790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:42.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f5398ddc700 1 -- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c006340 msgr2=0x7f537c003790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:42.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c006340 0x7f537c003790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:42.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f5398ddc700 1 -- 172.21.15.116:0/1927550571 >> v1:172.21.15.116:6789/0 conn(0x7f537c003c80 legacy=0x7f537c004130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:42.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f5398ddc700 1 -- 172.21.15.116:0/1927550571 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c005930 con 0x7f53940c1300 2020-11-20T12:15:42.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.256+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f53940c1300 0x7f537c00aeb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f53940683f0 tx=0x7f5388009040).ready entity=mon.0 client_cookie=5d34202e9d8ac0ca server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:42.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.260+0000 7f53937fe700 1 -- 172.21.15.116:0/1927550571 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388016760 con 0x7f53940c1300 2020-11-20T12:15:42.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.260+0000 7f53937fe700 1 -- 172.21.15.116:0/1927550571 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53880168c0 con 0x7f53940c1300 2020-11-20T12:15:42.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.260+0000 7f53937fe700 1 -- 172.21.15.116:0/1927550571 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5388016c60 con 0x7f53940c1300 2020-11-20T12:15:42.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.260+0000 7f539b040700 1 -- 172.21.15.116:0/1927550571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f53940c1300 msgr2=0x7f537c00aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:42.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.260+0000 7f539b040700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f53940c1300 0x7f537c00aeb0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f53940683f0 tx=0x7f5388009040).stop 2020-11-20T12:15:42.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.260+0000 7f539b040700 1 -- 172.21.15.116:0/1927550571 shutdown_connections 2020-11-20T12:15:42.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.260+0000 7f539b040700 1 --2- 172.21.15.116:0/1927550571 >> v2:172.21.15.173:3300/0 conn(0x7f537c006340 0x7f537c003790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:42.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.260+0000 7f539b040700 1 --2- 172.21.15.116:0/1927550571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f53940c1300 0x7f537c00aeb0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:42.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.260+0000 7f539b040700 1 -- 172.21.15.116:0/1927550571 >> 172.21.15.116:0/1927550571 conn(0x7f5394060170 msgr2=0x7f5394060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:42.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.260+0000 7f539b040700 1 -- 172.21.15.116:0/1927550571 shutdown_connections 2020-11-20T12:15:42.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.260+0000 7f539b040700 1 -- 172.21.15.116:0/1927550571 wait complete. 2020-11-20T12:15:42.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.264+0000 7f539b040700 1 Processor -- start 2020-11-20T12:15:42.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.264+0000 7f539b040700 1 -- start start 2020-11-20T12:15:42.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.264+0000 7f539b040700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c003c80 0x7f5394147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:42.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.264+0000 7f539b040700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f537c006340 0x7f5394161a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:42.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.264+0000 7f539b040700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53940c1300 0x7f5394164710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:42.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.264+0000 7f539b040700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5394165170 con 0x7f537c006340 2020-11-20T12:15:42.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.264+0000 7f539b040700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5394165410 con 0x7f53940c1300 2020-11-20T12:15:42.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.264+0000 7f539b040700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5394161330 con 0x7f537c003c80 2020-11-20T12:15:42.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.264+0000 7f5393fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f537c006340 0x7f5394161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:42.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5393fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f537c006340 0x7f5394161a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42428/0 (socket says 172.21.15.116:42428) 2020-11-20T12:15:42.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5393fff700 1 -- 172.21.15.116:0/1961931276 learned_addr learned my addr 172.21.15.116:0/1961931276 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:42.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f53995dd700 1 --2- 172.21.15.116:0/1961931276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53940c1300 0x7f5394164710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:42.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5398ddc700 1 --2- 172.21.15.116:0/1961931276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c003c80 0x7f5394147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:42.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5393fff700 1 -- 172.21.15.116:0/1961931276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c003c80 msgr2=0x7f5394147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:42.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5393fff700 1 --2- 172.21.15.116:0/1961931276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c003c80 0x7f5394147840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:42.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5393fff700 1 -- 172.21.15.116:0/1961931276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53940c1300 msgr2=0x7f5394164710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:42.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5393fff700 1 --2- 172.21.15.116:0/1961931276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53940c1300 0x7f5394164710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:42.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5393fff700 1 -- 172.21.15.116:0/1961931276 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53941615d0 con 0x7f537c006340 2020-11-20T12:15:42.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f53995dd700 1 --2- 172.21.15.116:0/1961931276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53940c1300 0x7f5394164710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:42.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5393fff700 1 --2- 172.21.15.116:0/1961931276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f537c006340 0x7f5394161a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f5380015950 tx=0x7f5380009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:42.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5390ff9700 1 -- 172.21.15.116:0/1961931276 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5380015e80 con 0x7f537c006340 2020-11-20T12:15:42.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5390ff9700 1 -- 172.21.15.116:0/1961931276 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5380007240 con 0x7f537c006340 2020-11-20T12:15:42.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f539b040700 1 -- 172.21.15.116:0/1961931276 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f539414a3f0 con 0x7f537c006340 2020-11-20T12:15:42.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5390ff9700 1 -- 172.21.15.116:0/1961931276 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53800039a0 con 0x7f537c006340 2020-11-20T12:15:42.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f539b040700 1 -- 172.21.15.116:0/1961931276 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5394161f60 con 0x7f537c006340 2020-11-20T12:15:42.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5390ff9700 1 -- 172.21.15.116:0/1961931276 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f5380028070 con 0x7f537c006340 2020-11-20T12:15:42.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f5390ff9700 1 -- 172.21.15.116:0/1961931276 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f5380006b60 con 0x7f537c006340 2020-11-20T12:15:42.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.268+0000 7f539b040700 1 -- 172.21.15.116:0/1961931276 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f539405ca30 con 0x7f537c006340 2020-11-20T12:15:42.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.276+0000 7f5390ff9700 1 -- 172.21.15.116:0/1961931276 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f5380040880 con 0x7f537c006340 2020-11-20T12:15:42.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.356+0000 7f539b040700 1 -- 172.21.15.116:0/1961931276 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f53940a6b60 con 0x7f537c006340 2020-11-20T12:15:42.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.364+0000 7f5390ff9700 1 -- 172.21.15.116:0/1961931276 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f538002f070 con 0x7f537c006340 2020-11-20T12:15:42.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.364+0000 7f539b040700 1 -- 172.21.15.116:0/1961931276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f537c006340 msgr2=0x7f5394161a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:42.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.364+0000 7f539b040700 1 --2- 172.21.15.116:0/1961931276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f537c006340 0x7f5394161a70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f5380015950 tx=0x7f5380009040).stop 2020-11-20T12:15:42.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.364+0000 7f539b040700 1 -- 172.21.15.116:0/1961931276 shutdown_connections 2020-11-20T12:15:42.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.364+0000 7f539b040700 1 --2- 172.21.15.116:0/1961931276 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53940c1300 0x7f5394164710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:42.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.364+0000 7f539b040700 1 --2- 172.21.15.116:0/1961931276 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f537c003c80 0x7f5394147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:42.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.364+0000 7f539b040700 1 --2- 172.21.15.116:0/1961931276 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f537c006340 0x7f5394161a70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:42.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.364+0000 7f539b040700 1 -- 172.21.15.116:0/1961931276 >> 172.21.15.116:0/1961931276 conn(0x7f5394060170 msgr2=0x7f53940a1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:42.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.364+0000 7f539b040700 1 -- 172.21.15.116:0/1961931276 shutdown_connections 2020-11-20T12:15:42.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:42.364+0000 7f539b040700 1 -- 172.21.15.116:0/1961931276 wait complete. 2020-11-20T12:15:42.370 INFO:teuthology.orchestra.run.smithi116.stderr:adjusted tunables profile to default 2020-11-20T12:15:42.392 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2020-11-20T12:15:42.392 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:42.393 INFO:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-20T12:15:42.403 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2020-11-20T12:15:42.424 INFO:teuthology.orchestra.run.smithi116.stdout:b46f3e7a-a64d-4724-8c3e-ea252ab40ce2 2020-11-20T12:15:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:42.424 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:42.425 INFO:teuthology.orchestra.run.smithi116.stderr:37 bytes copied, 0.000334506 s, 111 kB/s 2020-11-20T12:15:42.426 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:42.426 INFO:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-20T12:15:42.437 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2020-11-20T12:15:42.450 INFO:teuthology.orchestra.run.smithi116.stdout:90b5585c-eff0-40f1-8574-01d58ad5d7d0 2020-11-20T12:15:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:37 bytes copied, 0.000235198 s, 157 kB/s 2020-11-20T12:15:42.451 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:42.452 INFO:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-20T12:15:42.504 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2020-11-20T12:15:42.517 INFO:teuthology.orchestra.run.smithi116.stdout:063ebbab-aa94-46f2-a8c3-774fd7d8c39c 2020-11-20T12:15:42.517 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:42.517 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:42.518 INFO:teuthology.orchestra.run.smithi116.stderr:37 bytes copied, 0.000360127 s, 103 kB/s 2020-11-20T12:15:42.519 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:15:42.519 INFO:teuthology.orchestra.run.smithi116:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2020-11-20T12:15:42.529 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2020-11-20T12:15:42.543 INFO:teuthology.orchestra.run.smithi116.stdout:5a9021b2-8618-45a1-a93d-d000a17ef73d 2020-11-20T12:15:42.543 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:15:42.543 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:15:42.544 INFO:teuthology.orchestra.run.smithi116.stderr:37 bytes copied, 0.000343692 s, 108 kB/s 2020-11-20T12:15:42.545 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:42.546 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2020-11-20T12:15:42.561 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2020-11-20T12:15:42.573 INFO:teuthology.orchestra.run.smithi173.stdout:b3238680-8ed8-4a81-aa6a-20548bb44de2 2020-11-20T12:15:42.574 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:42.574 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:42.575 INFO:teuthology.orchestra.run.smithi173.stderr:37 bytes copied, 0.000410864 s, 90.1 kB/s 2020-11-20T12:15:42.576 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:42.577 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2020-11-20T12:15:42.636 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2020-11-20T12:15:42.651 INFO:teuthology.orchestra.run.smithi173.stdout:38264535-e8e9-409b-bbc5-46f51fcfcd75 2020-11-20T12:15:42.651 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:42.651 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:42.652 INFO:teuthology.orchestra.run.smithi173.stderr:37 bytes copied, 0.000373744 s, 99.0 kB/s 2020-11-20T12:15:42.654 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:42.654 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2020-11-20T12:15:42.675 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2020-11-20T12:15:42.689 INFO:teuthology.orchestra.run.smithi173.stdout:be530a7d-5f7c-4107-9171-15f4a6a4c468 2020-11-20T12:15:43.666 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:43.667 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:43.667 INFO:teuthology.orchestra.run.smithi173.stderr:37 bytes copied, 0.000404718 s, 91.4 kB/s 2020-11-20T12:15:43.668 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:15:43.668 INFO:teuthology.orchestra.run.smithi173:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2020-11-20T12:15:43.685 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2020-11-20T12:15:43.700 INFO:teuthology.orchestra.run.smithi173.stdout:949fed52-858c-459b-8fcc-317b259ed305 2020-11-20T12:15:43.701 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:15:43.701 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:15:43.701 INFO:teuthology.orchestra.run.smithi173.stderr:37 bytes copied, 0.000431274 s, 85.8 kB/s 2020-11-20T12:15:43.703 INFO:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new b46f3e7a-a64d-4724-8c3e-ea252ab40ce2 0 2020-11-20T12:15:43.917 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.914+0000 7f38923ec700 1 Processor -- start 2020-11-20T12:15:43.917 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.914+0000 7f38923ec700 1 -- start start 2020-11-20T12:15:43.919 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f38923ec700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f388c0a0f00 0x7f388c0a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:43.921 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f38923ec700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f388c0c2df0 0x7f388c0c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:43.921 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f38923ec700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388c0a50a0 con 0x7f388c0a12e0 2020-11-20T12:15:43.921 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f38923ec700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c0c3770 con 0x7f388c0a0f00 2020-11-20T12:15:43.922 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f38923ec700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388c0a5340 con 0x7f388c0c2df0 2020-11-20T12:15:43.927 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f3891beb700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f388c0c2df0 0x7f388c0c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:43.928 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f3891beb700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f388c0c2df0 0x7f388c0c3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:46162/0 (socket says 172.21.15.173:46162) 2020-11-20T12:15:43.929 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f3891beb700 1 -- 172.21.15.173:0/1500623398 learned_addr learned my addr 172.21.15.173:0/1500623398 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:43.930 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f38913ea700 1 --2- 172.21.15.173:0/1500623398 >> v2:172.21.15.116:3300/0 conn(0x7f388c0a0f00 0x7f388c0a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:43.931 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f3891beb700 1 -- 172.21.15.173:0/1500623398 >> v2:172.21.15.116:3300/0 conn(0x7f388c0a0f00 msgr2=0x7f388c0a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:43.932 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f3891beb700 1 --2- 172.21.15.173:0/1500623398 >> v2:172.21.15.116:3300/0 conn(0x7f388c0a0f00 0x7f388c0a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:43.932 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.918+0000 7f3891beb700 1 -- 172.21.15.173:0/1500623398 >> v1:172.21.15.116:6789/0 conn(0x7f388c0a12e0 legacy=0x7f388c150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:43.933 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f3891beb700 1 -- 172.21.15.173:0/1500623398 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c0a55e0 con 0x7f388c0c2df0 2020-11-20T12:15:43.935 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f38913ea700 1 --2- 172.21.15.173:0/1500623398 >> v2:172.21.15.116:3300/0 conn(0x7f388c0a0f00 0x7f388c0a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:43.935 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f3891beb700 1 --2- 172.21.15.173:0/1500623398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0c2df0 0x7f388c0c3280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f388c063c90 tx=0x7f3884009290).ready entity=mon.1 client_cookie=c7f5f230a80a311e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:43.936 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f387bfff700 1 -- 172.21.15.173:0/1500623398 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388400bbc0 con 0x7f388c0c2df0 2020-11-20T12:15:43.937 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f387bfff700 1 -- 172.21.15.173:0/1500623398 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388400bd20 con 0x7f388c0c2df0 2020-11-20T12:15:43.938 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f387bfff700 1 -- 172.21.15.173:0/1500623398 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884011610 con 0x7f388c0c2df0 2020-11-20T12:15:43.939 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f38923ec700 1 -- 172.21.15.173:0/1500623398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0c2df0 msgr2=0x7f388c0c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:43.940 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f38923ec700 1 --2- 172.21.15.173:0/1500623398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0c2df0 0x7f388c0c3280 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f388c063c90 tx=0x7f3884009290).stop 2020-11-20T12:15:43.940 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f38923ec700 1 -- 172.21.15.173:0/1500623398 shutdown_connections 2020-11-20T12:15:43.941 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f38923ec700 1 --2- 172.21.15.173:0/1500623398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0c2df0 0x7f388c0c3280 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:43.942 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f38923ec700 1 --2- 172.21.15.173:0/1500623398 >> v2:172.21.15.116:3300/0 conn(0x7f388c0a0f00 0x7f388c0a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:43.943 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f38923ec700 1 -- 172.21.15.173:0/1500623398 >> 172.21.15.173:0/1500623398 conn(0x7f388c060c30 msgr2=0x7f388c0c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:43.943 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f38923ec700 1 -- 172.21.15.173:0/1500623398 shutdown_connections 2020-11-20T12:15:43.944 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.922+0000 7f38923ec700 1 -- 172.21.15.173:0/1500623398 wait complete. 2020-11-20T12:15:43.952 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.926+0000 7f38923ec700 1 Processor -- start 2020-11-20T12:15:43.953 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.926+0000 7f38923ec700 1 -- start start 2020-11-20T12:15:43.953 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.926+0000 7f38923ec700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f388c0a0f00 0x7f388c164e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:43.953 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.926+0000 7f38923ec700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f388c0a12e0 0x7f388c1609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:43.953 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.926+0000 7f38923ec700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0c2df0 0x7f388c160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:43.954 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.926+0000 7f38923ec700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c066f90 con 0x7f388c0a0f00 2020-11-20T12:15:43.954 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.926+0000 7f38923ec700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f388c165320 con 0x7f388c0c2df0 2020-11-20T12:15:43.954 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.926+0000 7f38923ec700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388c0bd320 con 0x7f388c0a12e0 2020-11-20T12:15:43.954 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f3891beb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0c2df0 0x7f388c160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:43.955 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f3891beb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0c2df0 0x7f388c160ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:46166/0 (socket says 172.21.15.173:46166) 2020-11-20T12:15:43.955 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f3891beb700 1 -- 172.21.15.173:0/1252288688 learned_addr learned my addr 172.21.15.173:0/1252288688 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:43.955 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f3890be9700 1 --2- 172.21.15.173:0/1252288688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f388c0a12e0 0x7f388c1609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:43.955 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f38913ea700 1 --2- 172.21.15.173:0/1252288688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f388c0a0f00 0x7f388c164e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:43.955 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f3891beb700 1 -- 172.21.15.173:0/1252288688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f388c0a12e0 msgr2=0x7f388c1609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:43.956 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f3891beb700 1 --2- 172.21.15.173:0/1252288688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f388c0a12e0 0x7f388c1609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:43.956 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f3891beb700 1 -- 172.21.15.173:0/1252288688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f388c0a0f00 msgr2=0x7f388c164e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:43.956 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f3891beb700 1 --2- 172.21.15.173:0/1252288688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f388c0a0f00 0x7f388c164e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:43.956 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f3891beb700 1 -- 172.21.15.173:0/1252288688 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388c161470 con 0x7f388c0c2df0 2020-11-20T12:15:43.957 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f3891beb700 1 --2- 172.21.15.173:0/1252288688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0c2df0 0x7f388c160ee0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f38840092d0 tx=0x7f3884016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:43.957 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f38797fa700 1 -- 172.21.15.173:0/1252288688 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388400bbc0 con 0x7f388c0c2df0 2020-11-20T12:15:43.957 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f38797fa700 1 -- 172.21.15.173:0/1252288688 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3884003290 con 0x7f388c0c2df0 2020-11-20T12:15:43.957 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f38923ec700 1 -- 172.21.15.173:0/1252288688 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388c161710 con 0x7f388c0c2df0 2020-11-20T12:15:43.958 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f38797fa700 1 -- 172.21.15.173:0/1252288688 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884010d10 con 0x7f388c0c2df0 2020-11-20T12:15:43.958 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f38923ec700 1 -- 172.21.15.173:0/1252288688 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388c161c50 con 0x7f388c0c2df0 2020-11-20T12:15:43.958 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f38797fa700 1 -- 172.21.15.173:0/1252288688 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f3884023030 con 0x7f388c0c2df0 2020-11-20T12:15:43.958 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f38797fa700 1 -- 172.21.15.173:0/1252288688 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7f38840267b0 con 0x7f388c0c2df0 2020-11-20T12:15:43.959 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.930+0000 7f38923ec700 1 -- 172.21.15.173:0/1252288688 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388c0bd460 con 0x7f388c0c2df0 2020-11-20T12:15:43.959 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:43.938+0000 7f38797fa700 1 -- 172.21.15.173:0/1252288688 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f3884021070 con 0x7f388c0c2df0 2020-11-20T12:15:44.022 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.022+0000 7f38923ec700 1 -- 172.21.15.173:0/1252288688 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b46f3e7a-a64d-4724-8c3e-ea252ab40ce2", "id": 0} v 0) v1 -- 0x7f388c044f60 con 0x7f388c0c2df0 2020-11-20T12:15:44.033 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.030+0000 7f38797fa700 1 -- 172.21.15.173:0/1252288688 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b46f3e7a-a64d-4724-8c3e-ea252ab40ce2", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3884015020 con 0x7f388c0c2df0 2020-11-20T12:15:44.034 INFO:teuthology.orchestra.run.smithi173.stdout:0 2020-11-20T12:15:44.035 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.034+0000 7f38923ec700 1 -- 172.21.15.173:0/1252288688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0c2df0 msgr2=0x7f388c160ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.035 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.034+0000 7f38923ec700 1 --2- 172.21.15.173:0/1252288688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0c2df0 0x7f388c160ee0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f38840092d0 tx=0x7f3884016000).stop 2020-11-20T12:15:44.035 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.034+0000 7f38923ec700 1 -- 172.21.15.173:0/1252288688 shutdown_connections 2020-11-20T12:15:44.035 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.034+0000 7f38923ec700 1 --2- 172.21.15.173:0/1252288688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f388c0c2df0 0x7f388c160ee0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.036 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.034+0000 7f38923ec700 1 --2- 172.21.15.173:0/1252288688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f388c0a0f00 0x7f388c164e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.036 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.034+0000 7f38923ec700 1 --2- 172.21.15.173:0/1252288688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f388c0a12e0 0x7f388c1609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.036 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.034+0000 7f38923ec700 1 -- 172.21.15.173:0/1252288688 >> 172.21.15.173:0/1252288688 conn(0x7f388c060c30 msgr2=0x7f388c0c22e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:44.036 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.034+0000 7f38923ec700 1 -- 172.21.15.173:0/1252288688 shutdown_connections 2020-11-20T12:15:44.037 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.034+0000 7f38923ec700 1 -- 172.21.15.173:0/1252288688 wait complete. 2020-11-20T12:15:44.054 INFO:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 90b5585c-eff0-40f1-8574-01d58ad5d7d0 1 2020-11-20T12:15:44.223 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.222+0000 7f4c31336700 1 Processor -- start 2020-11-20T12:15:44.223 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.222+0000 7f4c31336700 1 -- start start 2020-11-20T12:15:44.226 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c31336700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4c2c067cf0 0x7f4c2c060bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.226 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c31336700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4c2c0610b0 0x7f4c2c065040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.226 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c31336700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c2c152560 con 0x7f4c2c067910 2020-11-20T12:15:44.226 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c31336700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2c152800 con 0x7f4c2c0610b0 2020-11-20T12:15:44.226 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c31336700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c2c152aa0 con 0x7f4c2c067cf0 2020-11-20T12:15:44.228 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c2a7fc700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4c2c067cf0 0x7f4c2c060bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.228 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c2affd700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f4c2c067910 0x7f4c2c151b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.173:33230/0 (socket says 172.21.15.173:33230) 2020-11-20T12:15:44.228 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c2affd700 1 -- 172.21.15.173:0/4076068568 learned_addr learned my addr 172.21.15.173:0/4076068568 (peer_addr_for_me v1:172.21.15.173:0/0) 2020-11-20T12:15:44.229 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c2bfff700 1 --2- 172.21.15.173:0/4076068568 >> v2:172.21.15.116:3300/0 conn(0x7f4c2c0610b0 0x7f4c2c065040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.229 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c2a7fc700 1 -- 172.21.15.173:0/4076068568 >> v2:172.21.15.116:3300/0 conn(0x7f4c2c0610b0 msgr2=0x7f4c2c065040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:44.229 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c2a7fc700 1 --2- 172.21.15.173:0/4076068568 >> v2:172.21.15.116:3300/0 conn(0x7f4c2c0610b0 0x7f4c2c065040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.229 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c2a7fc700 1 -- 172.21.15.173:0/4076068568 >> v1:172.21.15.173:6789/0 conn(0x7f4c2c067910 legacy=0x7f4c2c151b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.229 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c29ffb700 1 -- 172.21.15.173:0/4076068568 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3163810694 0 0) 0x7f4c2c152560 con 0x7f4c2c067910 2020-11-20T12:15:44.230 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c2a7fc700 1 -- 172.21.15.173:0/4076068568 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c060480 con 0x7f4c2c067cf0 2020-11-20T12:15:44.230 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c2a7fc700 1 --2- 172.21.15.173:0/4076068568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c2c067cf0 0x7f4c2c060bc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4c2c0c0ae0 tx=0x7f4c20006df0).ready entity=mon.1 client_cookie=d174449b8abacec9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:44.230 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c29ffb700 1 -- 172.21.15.173:0/4076068568 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c200097b0 con 0x7f4c2c067cf0 2020-11-20T12:15:44.230 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c29ffb700 1 -- 172.21.15.173:0/4076068568 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c20009910 con 0x7f4c2c067cf0 2020-11-20T12:15:44.231 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c29ffb700 1 -- 172.21.15.173:0/4076068568 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20009c90 con 0x7f4c2c067cf0 2020-11-20T12:15:44.232 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c31336700 1 -- 172.21.15.173:0/4076068568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c2c067cf0 msgr2=0x7f4c2c060bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.232 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c31336700 1 --2- 172.21.15.173:0/4076068568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c2c067cf0 0x7f4c2c060bc0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f4c2c0c0ae0 tx=0x7f4c20006df0).stop 2020-11-20T12:15:44.232 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c31336700 1 -- 172.21.15.173:0/4076068568 shutdown_connections 2020-11-20T12:15:44.232 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c31336700 1 --2- 172.21.15.173:0/4076068568 >> v2:172.21.15.116:3300/0 conn(0x7f4c2c0610b0 0x7f4c2c065040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.233 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c31336700 1 --2- 172.21.15.173:0/4076068568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c2c067cf0 0x7f4c2c060bc0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.233 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.226+0000 7f4c31336700 1 -- 172.21.15.173:0/4076068568 >> 172.21.15.173:0/4076068568 conn(0x7f4c2c0c9a90 msgr2=0x7f4c2c0c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:44.233 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.230+0000 7f4c31336700 1 -- 172.21.15.173:0/4076068568 shutdown_connections 2020-11-20T12:15:44.233 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.230+0000 7f4c31336700 1 -- 172.21.15.173:0/4076068568 wait complete. 2020-11-20T12:15:44.234 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.230+0000 7f4c31336700 1 Processor -- start 2020-11-20T12:15:44.234 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.230+0000 7f4c31336700 1 -- start start 2020-11-20T12:15:44.236 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.230+0000 7f4c31336700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c2c0610b0 0x7f4c2c156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.236 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.230+0000 7f4c31336700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c2c067910 0x7f4c2c14d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.236 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.230+0000 7f4c31336700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c2c0abf00 0x7f4c2c0c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.236 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.230+0000 7f4c31336700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c157510 con 0x7f4c2c0abf00 2020-11-20T12:15:44.237 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.230+0000 7f4c31336700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c1577b0 con 0x7f4c2c0610b0 2020-11-20T12:15:44.237 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.230+0000 7f4c31336700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c2c157a50 con 0x7f4c2c067910 2020-11-20T12:15:44.237 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c2a7fc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c2c067910 0x7f4c2c14d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.237 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c2bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c2c0abf00 0x7f4c2c0c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.237 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c2a7fc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c2c067910 0x7f4c2c14d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:45520/0 (socket says 172.21.15.173:45520) 2020-11-20T12:15:44.238 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c2a7fc700 1 -- 172.21.15.173:0/2842088200 learned_addr learned my addr 172.21.15.173:0/2842088200 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:44.238 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c2affd700 1 --2- 172.21.15.173:0/2842088200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c2c0610b0 0x7f4c2c156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.238 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c2a7fc700 1 -- 172.21.15.173:0/2842088200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c2c0610b0 msgr2=0x7f4c2c156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.238 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c2a7fc700 1 --2- 172.21.15.173:0/2842088200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c2c0610b0 0x7f4c2c156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.239 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c2a7fc700 1 -- 172.21.15.173:0/2842088200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c2c0abf00 msgr2=0x7f4c2c0c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.239 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c2a7fc700 1 --2- 172.21.15.173:0/2842088200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c2c0abf00 0x7f4c2c0c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.239 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c2a7fc700 1 -- 172.21.15.173:0/2842088200 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c157cf0 con 0x7f4c2c067910 2020-11-20T12:15:44.239 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c2a7fc700 1 --2- 172.21.15.173:0/2842088200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c2c067910 0x7f4c2c14d200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f4c20010000 tx=0x7f4c20015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:44.239 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c12ffd700 1 -- 172.21.15.173:0/2842088200 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c20011c80 con 0x7f4c2c067910 2020-11-20T12:15:44.240 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c12ffd700 1 -- 172.21.15.173:0/2842088200 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c20011de0 con 0x7f4c2c067910 2020-11-20T12:15:44.240 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c31336700 1 -- 172.21.15.173:0/2842088200 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c2c157f90 con 0x7f4c2c067910 2020-11-20T12:15:44.240 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c12ffd700 1 -- 172.21.15.173:0/2842088200 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2001e670 con 0x7f4c2c067910 2020-11-20T12:15:44.240 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c31336700 1 -- 172.21.15.173:0/2842088200 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c2c0ac360 con 0x7f4c2c067910 2020-11-20T12:15:44.240 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c12ffd700 1 -- 172.21.15.173:0/2842088200 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f4c20025030 con 0x7f4c2c067910 2020-11-20T12:15:44.241 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c12ffd700 1 -- 172.21.15.173:0/2842088200 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7f4c200284d0 con 0x7f4c2c067910 2020-11-20T12:15:44.241 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.234+0000 7f4c31336700 1 -- 172.21.15.173:0/2842088200 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c2c158330 con 0x7f4c2c067910 2020-11-20T12:15:44.242 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.238+0000 7f4c12ffd700 1 -- 172.21.15.173:0/2842088200 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f4c2c158330 con 0x7f4c2c067910 2020-11-20T12:15:44.327 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.326+0000 7f4c31336700 1 -- 172.21.15.173:0/2842088200 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "90b5585c-eff0-40f1-8574-01d58ad5d7d0", "id": 1} v 0) v1 -- 0x7f4c2c158120 con 0x7f4c2c067910 2020-11-20T12:15:44.337 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.334+0000 7f4c12ffd700 1 -- 172.21.15.173:0/2842088200 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "90b5585c-eff0-40f1-8574-01d58ad5d7d0", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f4c200360b0 con 0x7f4c2c067910 2020-11-20T12:15:44.338 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:15:44.338 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.338+0000 7f4c31336700 1 -- 172.21.15.173:0/2842088200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c2c067910 msgr2=0x7f4c2c14d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.339 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.338+0000 7f4c31336700 1 --2- 172.21.15.173:0/2842088200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c2c067910 0x7f4c2c14d200 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f4c20010000 tx=0x7f4c20015000).stop 2020-11-20T12:15:44.339 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.338+0000 7f4c31336700 1 -- 172.21.15.173:0/2842088200 shutdown_connections 2020-11-20T12:15:44.339 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.338+0000 7f4c31336700 1 --2- 172.21.15.173:0/2842088200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c2c0abf00 0x7f4c2c0c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.339 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.338+0000 7f4c31336700 1 --2- 172.21.15.173:0/2842088200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c2c0610b0 0x7f4c2c156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.340 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.338+0000 7f4c31336700 1 --2- 172.21.15.173:0/2842088200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c2c067910 0x7f4c2c14d200 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.340 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.338+0000 7f4c31336700 1 -- 172.21.15.173:0/2842088200 >> 172.21.15.173:0/2842088200 conn(0x7f4c2c0c9a90 msgr2=0x7f4c2c14ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:44.340 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.338+0000 7f4c31336700 1 -- 172.21.15.173:0/2842088200 shutdown_connections 2020-11-20T12:15:44.340 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.338+0000 7f4c31336700 1 -- 172.21.15.173:0/2842088200 wait complete. 2020-11-20T12:15:44.362 INFO:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 063ebbab-aa94-46f2-a8c3-774fd7d8c39c 2 2020-11-20T12:15:44.525 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.522+0000 7f2054b88700 1 Processor -- start 2020-11-20T12:15:44.526 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f2054b88700 1 -- start start 2020-11-20T12:15:44.528 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f2054b88700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f205013ad30 0x7f20500a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.528 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f2054b88700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205013b110 0x7f2050151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.529 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f2054b88700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2050151fa0 0x7f2050061250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.529 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f2054b88700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2050152a80 con 0x7f205013ad30 2020-11-20T12:15:44.530 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f2054b88700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2050152540 con 0x7f2050151fa0 2020-11-20T12:15:44.530 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f2054b88700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20501527e0 con 0x7f205013b110 2020-11-20T12:15:44.532 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f204ffff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2050151fa0 0x7f2050061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.532 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f204effd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205013b110 0x7f2050151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.532 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f204ffff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2050151fa0 0x7f2050061250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:46184/0 (socket says 172.21.15.173:46184) 2020-11-20T12:15:44.533 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f204ffff700 1 -- 172.21.15.173:0/3202347073 learned_addr learned my addr 172.21.15.173:0/3202347073 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:44.533 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.526+0000 7f204f7fe700 1 --2- 172.21.15.173:0/3202347073 >> v2:172.21.15.116:3300/0 conn(0x7f205013ad30 0x7f20500a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.533 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f204ffff700 1 -- 172.21.15.173:0/3202347073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205013b110 msgr2=0x7f2050151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:44.533 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f204ffff700 1 --2- 172.21.15.173:0/3202347073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205013b110 0x7f2050151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.534 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f204ffff700 1 -- 172.21.15.173:0/3202347073 >> v2:172.21.15.116:3300/0 conn(0x7f205013ad30 msgr2=0x7f20500a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:44.534 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f204ffff700 1 --2- 172.21.15.173:0/3202347073 >> v2:172.21.15.116:3300/0 conn(0x7f205013ad30 0x7f20500a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.534 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f204ffff700 1 -- 172.21.15.173:0/3202347073 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20500ac910 con 0x7f2050151fa0 2020-11-20T12:15:44.534 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f204f7fe700 1 --2- 172.21.15.173:0/3202347073 >> v2:172.21.15.116:3300/0 conn(0x7f205013ad30 0x7f20500a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:44.534 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f204ffff700 1 --2- 172.21.15.173:0/3202347073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2050151fa0 0x7f2050061250 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f20500ab8b0 tx=0x7f2048009290).ready entity=mon.1 client_cookie=c35cfdbc0ebbe53a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:44.535 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f204e7fc700 1 -- 172.21.15.173:0/3202347073 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800bba0 con 0x7f2050151fa0 2020-11-20T12:15:44.535 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f204e7fc700 1 -- 172.21.15.173:0/3202347073 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204800bd00 con 0x7f2050151fa0 2020-11-20T12:15:44.535 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f204e7fc700 1 -- 172.21.15.173:0/3202347073 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048011610 con 0x7f2050151fa0 2020-11-20T12:15:44.535 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f2054b88700 1 -- 172.21.15.173:0/3202347073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2050151fa0 msgr2=0x7f2050061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.535 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f2054b88700 1 --2- 172.21.15.173:0/3202347073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2050151fa0 0x7f2050061250 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f20500ab8b0 tx=0x7f2048009290).stop 2020-11-20T12:15:44.536 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f2054b88700 1 -- 172.21.15.173:0/3202347073 shutdown_connections 2020-11-20T12:15:44.536 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f2054b88700 1 --2- 172.21.15.173:0/3202347073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2050151fa0 0x7f2050061250 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.536 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f2054b88700 1 --2- 172.21.15.173:0/3202347073 >> v2:172.21.15.116:3300/0 conn(0x7f205013ad30 0x7f20500a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.536 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f2054b88700 1 --2- 172.21.15.173:0/3202347073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f205013b110 0x7f2050151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.536 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f2054b88700 1 -- 172.21.15.173:0/3202347073 >> 172.21.15.173:0/3202347073 conn(0x7f20500678a0 msgr2=0x7f2050067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:44.537 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f2054b88700 1 -- 172.21.15.173:0/3202347073 shutdown_connections 2020-11-20T12:15:44.537 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.530+0000 7f2054b88700 1 -- 172.21.15.173:0/3202347073 wait complete. 2020-11-20T12:15:44.538 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.534+0000 7f2054b88700 1 Processor -- start 2020-11-20T12:15:44.538 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.534+0000 7f2054b88700 1 -- start start 2020-11-20T12:15:44.538 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f2054b88700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205013ad30 0x7f20500a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.539 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f2054b88700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f205013b110 0x7f2050161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.539 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f2054b88700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2050151fa0 0x7f20500c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.539 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f2054b88700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20500a1a90 con 0x7f205013b110 2020-11-20T12:15:44.539 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f2054b88700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20501618f0 con 0x7f205013ad30 2020-11-20T12:15:44.539 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f2054b88700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2050163cf0 con 0x7f2050151fa0 2020-11-20T12:15:44.540 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204ffff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2050151fa0 0x7f20500c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.540 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204f7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205013ad30 0x7f20500a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.540 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204ffff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2050151fa0 0x7f20500c8190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:45534/0 (socket says 172.21.15.173:45534) 2020-11-20T12:15:44.540 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204ffff700 1 -- 172.21.15.173:0/1212086646 learned_addr learned my addr 172.21.15.173:0/1212086646 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:44.542 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204effd700 1 --2- 172.21.15.173:0/1212086646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f205013b110 0x7f2050161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.542 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204ffff700 1 -- 172.21.15.173:0/1212086646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205013ad30 msgr2=0x7f20500a3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.542 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204ffff700 1 --2- 172.21.15.173:0/1212086646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205013ad30 0x7f20500a3790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.542 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204ffff700 1 -- 172.21.15.173:0/1212086646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f205013b110 msgr2=0x7f2050161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.543 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204ffff700 1 --2- 172.21.15.173:0/1212086646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f205013b110 0x7f2050161400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.543 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204ffff700 1 -- 172.21.15.173:0/1212086646 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20500a3d80 con 0x7f2050151fa0 2020-11-20T12:15:44.543 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204effd700 1 --2- 172.21.15.173:0/1212086646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f205013b110 0x7f2050161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:15:44.543 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f204ffff700 1 --2- 172.21.15.173:0/1212086646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2050151fa0 0x7f20500c8190 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f20480092d0 tx=0x7f2048016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:44.543 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f202f7fe700 1 -- 172.21.15.173:0/1212086646 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f204800bba0 con 0x7f2050151fa0 2020-11-20T12:15:44.544 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f202f7fe700 1 -- 172.21.15.173:0/1212086646 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f204800be70 con 0x7f2050151fa0 2020-11-20T12:15:44.544 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f2054b88700 1 -- 172.21.15.173:0/1212086646 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20500a4020 con 0x7f2050151fa0 2020-11-20T12:15:44.544 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f202f7fe700 1 -- 172.21.15.173:0/1212086646 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2048010da0 con 0x7f2050151fa0 2020-11-20T12:15:44.544 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f2054b88700 1 -- 172.21.15.173:0/1212086646 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20500a4560 con 0x7f2050151fa0 2020-11-20T12:15:44.544 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f202f7fe700 1 -- 172.21.15.173:0/1212086646 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f2048017070 con 0x7f2050151fa0 2020-11-20T12:15:44.545 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f202f7fe700 1 -- 172.21.15.173:0/1212086646 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7f2048009500 con 0x7f2050151fa0 2020-11-20T12:15:44.545 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.538+0000 7f2054b88700 1 -- 172.21.15.173:0/1212086646 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2050044f60 con 0x7f2050151fa0 2020-11-20T12:15:44.546 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.546+0000 7f202f7fe700 1 -- 172.21.15.173:0/1212086646 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f2048015020 con 0x7f2050151fa0 2020-11-20T12:15:44.630 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.630+0000 7f2054b88700 1 -- 172.21.15.173:0/1212086646 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "063ebbab-aa94-46f2-a8c3-774fd7d8c39c", "id": 2} v 0) v1 -- 0x7f2050069320 con 0x7f2050151fa0 2020-11-20T12:15:44.641 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.638+0000 7f202f7fe700 1 -- 172.21.15.173:0/1212086646 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "063ebbab-aa94-46f2-a8c3-774fd7d8c39c", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f204801cb10 con 0x7f2050151fa0 2020-11-20T12:15:44.641 INFO:teuthology.orchestra.run.smithi173.stdout:2 2020-11-20T12:15:44.642 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.642+0000 7f2054b88700 1 -- 172.21.15.173:0/1212086646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2050151fa0 msgr2=0x7f20500c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.642 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.642+0000 7f2054b88700 1 --2- 172.21.15.173:0/1212086646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2050151fa0 0x7f20500c8190 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f20480092d0 tx=0x7f2048016000).stop 2020-11-20T12:15:44.643 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.642+0000 7f2054b88700 1 -- 172.21.15.173:0/1212086646 shutdown_connections 2020-11-20T12:15:44.643 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.642+0000 7f2054b88700 1 --2- 172.21.15.173:0/1212086646 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2050151fa0 0x7f20500c8190 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.643 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.642+0000 7f2054b88700 1 --2- 172.21.15.173:0/1212086646 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f205013ad30 0x7f20500a3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.643 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.642+0000 7f2054b88700 1 --2- 172.21.15.173:0/1212086646 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f205013b110 0x7f2050161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.644 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.642+0000 7f2054b88700 1 -- 172.21.15.173:0/1212086646 >> 172.21.15.173:0/1212086646 conn(0x7f20500678a0 msgr2=0x7f20500adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:44.644 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.642+0000 7f2054b88700 1 -- 172.21.15.173:0/1212086646 shutdown_connections 2020-11-20T12:15:44.644 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.642+0000 7f2054b88700 1 -- 172.21.15.173:0/1212086646 wait complete. 2020-11-20T12:15:44.665 INFO:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 5a9021b2-8618-45a1-a93d-d000a17ef73d 3 2020-11-20T12:15:44.826 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.826+0000 7fcf3533b700 1 Processor -- start 2020-11-20T12:15:44.826 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.826+0000 7fcf3533b700 1 -- start start 2020-11-20T12:15:44.827 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.826+0000 7fcf3533b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fcf30061190 0x7fcf300c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.828 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.826+0000 7fcf3533b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf3014bcb0 0x7fcf30152630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.828 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.826+0000 7fcf3533b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3014c100 con 0x7fcf3014b8d0 2020-11-20T12:15:44.828 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.826+0000 7fcf3533b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf300a8190 con 0x7fcf30061190 2020-11-20T12:15:44.828 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.826+0000 7fcf3533b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf30152b20 con 0x7fcf3014bcb0 2020-11-20T12:15:44.830 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.826+0000 7fcf2f7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf3014bcb0 0x7fcf30152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.830 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.826+0000 7fcf27fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fcf30061190 0x7fcf300c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.831 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.826+0000 7fcf2effd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fcf3014b8d0 0x7fcf300c4e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.173:37230/0 (socket says 172.21.15.173:37230) 2020-11-20T12:15:44.831 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.826+0000 7fcf2effd700 1 -- 172.21.15.173:0/1215459091 learned_addr learned my addr 172.21.15.173:0/1215459091 (peer_addr_for_me v1:172.21.15.173:0/0) 2020-11-20T12:15:44.831 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf2f7fe700 1 -- 172.21.15.173:0/1215459091 >> v2:172.21.15.173:3300/0 conn(0x7fcf30061190 msgr2=0x7fcf300c6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:44.832 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf2f7fe700 1 --2- 172.21.15.173:0/1215459091 >> v2:172.21.15.173:3300/0 conn(0x7fcf30061190 0x7fcf300c6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.832 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf2e7fc700 1 -- 172.21.15.173:0/1215459091 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976941882 0 0) 0x7fcf200010d0 con 0x7fcf3014b8d0 2020-11-20T12:15:44.832 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf2f7fe700 1 -- 172.21.15.173:0/1215459091 >> v1:172.21.15.116:6789/0 conn(0x7fcf3014b8d0 legacy=0x7fcf300c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.833 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf2f7fe700 1 -- 172.21.15.173:0/1215459091 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf30061710 con 0x7fcf3014bcb0 2020-11-20T12:15:44.833 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf27fff700 1 --2- 172.21.15.173:0/1215459091 >> v2:172.21.15.173:3300/0 conn(0x7fcf30061190 0x7fcf300c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:44.833 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf2f7fe700 1 --2- 172.21.15.173:0/1215459091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf3014bcb0 0x7fcf30152630 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fcf28009c90 tx=0x7fcf280091d0).ready entity=mon.2 client_cookie=7e85f1329d269bc5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:44.834 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf2e7fc700 1 -- 172.21.15.173:0/1215459091 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2800ba70 con 0x7fcf3014bcb0 2020-11-20T12:15:44.834 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf2e7fc700 1 -- 172.21.15.173:0/1215459091 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2800bbd0 con 0x7fcf3014bcb0 2020-11-20T12:15:44.834 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf2e7fc700 1 -- 172.21.15.173:0/1215459091 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf28011400 con 0x7fcf3014bcb0 2020-11-20T12:15:44.839 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf3533b700 1 -- 172.21.15.173:0/1215459091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf3014bcb0 msgr2=0x7fcf30152630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.840 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf3533b700 1 --2- 172.21.15.173:0/1215459091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf3014bcb0 0x7fcf30152630 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fcf28009c90 tx=0x7fcf280091d0).stop 2020-11-20T12:15:44.840 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf3533b700 1 -- 172.21.15.173:0/1215459091 shutdown_connections 2020-11-20T12:15:44.840 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf3533b700 1 --2- 172.21.15.173:0/1215459091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf3014bcb0 0x7fcf30152630 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.840 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf3533b700 1 --2- 172.21.15.173:0/1215459091 >> v2:172.21.15.173:3300/0 conn(0x7fcf30061190 0x7fcf300c6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.841 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf3533b700 1 -- 172.21.15.173:0/1215459091 >> 172.21.15.173:0/1215459091 conn(0x7fcf300678a0 msgr2=0x7fcf30067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:44.841 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf3533b700 1 -- 172.21.15.173:0/1215459091 shutdown_connections 2020-11-20T12:15:44.841 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.830+0000 7fcf3533b700 1 -- 172.21.15.173:0/1215459091 wait complete. 2020-11-20T12:15:44.841 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf3533b700 1 Processor -- start 2020-11-20T12:15:44.841 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf3533b700 1 -- start start 2020-11-20T12:15:44.842 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf3533b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf30061190 0x7fcf301576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.842 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf3533b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf3014b8d0 0x7fcf30164500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.842 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf3533b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3014bcb0 0x7fcf300c1500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:44.842 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf3533b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf300c05e0 con 0x7fcf3014bcb0 2020-11-20T12:15:44.843 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf3533b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf300c0880 con 0x7fcf3014b8d0 2020-11-20T12:15:44.843 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf3533b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf300c0b20 con 0x7fcf30061190 2020-11-20T12:15:44.843 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf2effd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf30061190 0x7fcf301576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.843 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf27fff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf3014b8d0 0x7fcf30164500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.843 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf2f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3014bcb0 0x7fcf300c1500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:44.844 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf27fff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf3014b8d0 0x7fcf30164500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.173:46200/0 (socket says 172.21.15.173:46200) 2020-11-20T12:15:44.844 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf27fff700 1 -- 172.21.15.173:0/3202891775 learned_addr learned my addr 172.21.15.173:0/3202891775 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:44.844 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf2effd700 1 -- 172.21.15.173:0/3202891775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf3014b8d0 msgr2=0x7fcf30164500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.844 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf2effd700 1 --2- 172.21.15.173:0/3202891775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf3014b8d0 0x7fcf30164500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.844 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf2effd700 1 -- 172.21.15.173:0/3202891775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3014bcb0 msgr2=0x7fcf300c1500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.845 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf2effd700 1 --2- 172.21.15.173:0/3202891775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3014bcb0 0x7fcf300c1500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.845 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf2effd700 1 -- 172.21.15.173:0/3202891775 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf300c0dc0 con 0x7fcf30061190 2020-11-20T12:15:44.845 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.834+0000 7fcf27fff700 1 --2- 172.21.15.173:0/3202891775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf3014b8d0 0x7fcf30164500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:44.845 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.838+0000 7fcf2effd700 1 --2- 172.21.15.173:0/3202891775 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf30061190 0x7fcf301576f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fcf2000c760 tx=0x7fcf20007860).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:44.845 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.838+0000 7fcf26ffd700 1 -- 172.21.15.173:0/3202891775 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf200072e0 con 0x7fcf30061190 2020-11-20T12:15:44.846 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.838+0000 7fcf3533b700 1 -- 172.21.15.173:0/3202891775 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf300c1060 con 0x7fcf30061190 2020-11-20T12:15:44.846 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.838+0000 7fcf3533b700 1 -- 172.21.15.173:0/3202891775 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf30157be0 con 0x7fcf30061190 2020-11-20T12:15:44.846 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.838+0000 7fcf26ffd700 1 -- 172.21.15.173:0/3202891775 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf200055f0 con 0x7fcf30061190 2020-11-20T12:15:44.846 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.838+0000 7fcf26ffd700 1 -- 172.21.15.173:0/3202891775 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20005760 con 0x7fcf30061190 2020-11-20T12:15:44.847 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.838+0000 7fcf26ffd700 1 -- 172.21.15.173:0/3202891775 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fcf2001a060 con 0x7fcf30061190 2020-11-20T12:15:44.847 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.838+0000 7fcf26ffd700 1 -- 172.21.15.173:0/3202891775 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7fcf20005be0 con 0x7fcf30061190 2020-11-20T12:15:44.847 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.838+0000 7fcf3533b700 1 -- 172.21.15.173:0/3202891775 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf30044f60 con 0x7fcf30061190 2020-11-20T12:15:44.847 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.842+0000 7fcf26ffd700 1 -- 172.21.15.173:0/3202891775 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fcf20021080 con 0x7fcf30061190 2020-11-20T12:15:44.931 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.930+0000 7fcf3533b700 1 -- 172.21.15.173:0/3202891775 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "5a9021b2-8618-45a1-a93d-d000a17ef73d", "id": 3} v 0) v1 -- 0x7fcf300c11f0 con 0x7fcf30061190 2020-11-20T12:15:44.942 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.942+0000 7fcf26ffd700 1 -- 172.21.15.173:0/3202891775 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5a9021b2-8618-45a1-a93d-d000a17ef73d", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fcf20018070 con 0x7fcf30061190 2020-11-20T12:15:44.942 INFO:teuthology.orchestra.run.smithi173.stdout:3 2020-11-20T12:15:44.943 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.942+0000 7fcf3533b700 1 -- 172.21.15.173:0/3202891775 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf30061190 msgr2=0x7fcf301576f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:44.944 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.942+0000 7fcf3533b700 1 --2- 172.21.15.173:0/3202891775 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf30061190 0x7fcf301576f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fcf2000c760 tx=0x7fcf20007860).stop 2020-11-20T12:15:44.944 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.942+0000 7fcf3533b700 1 -- 172.21.15.173:0/3202891775 shutdown_connections 2020-11-20T12:15:44.944 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.942+0000 7fcf3533b700 1 --2- 172.21.15.173:0/3202891775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf3014bcb0 0x7fcf300c1500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.945 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.942+0000 7fcf3533b700 1 --2- 172.21.15.173:0/3202891775 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf30061190 0x7fcf301576f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.945 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.942+0000 7fcf3533b700 1 --2- 172.21.15.173:0/3202891775 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf3014b8d0 0x7fcf30164500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:44.945 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.942+0000 7fcf3533b700 1 -- 172.21.15.173:0/3202891775 >> 172.21.15.173:0/3202891775 conn(0x7fcf300678a0 msgr2=0x7fcf300c7470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:44.945 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.942+0000 7fcf3533b700 1 -- 172.21.15.173:0/3202891775 shutdown_connections 2020-11-20T12:15:44.946 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:44.942+0000 7fcf3533b700 1 -- 172.21.15.173:0/3202891775 wait complete. 2020-11-20T12:15:44.964 INFO:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new b3238680-8ed8-4a81-aa6a-20548bb44de2 4 2020-11-20T12:15:45.129 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.126+0000 7f3bbd298700 1 Processor -- start 2020-11-20T12:15:45.129 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.126+0000 7f3bbd298700 1 -- start start 2020-11-20T12:15:45.130 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bbd298700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb814f900 0x7f3bb8067cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.131 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bbd298700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb8060cb0 con 0x7f3bb8064760 2020-11-20T12:15:45.131 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bbd298700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb8060f50 con 0x7f3bb8067910 2020-11-20T12:15:45.132 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bbd298700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8060170 con 0x7f3bb814f900 2020-11-20T12:15:45.135 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bb67fc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb814f900 0x7f3bb8067cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.135 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bb67fc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb814f900 0x7f3bb8067cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:45548/0 (socket says 172.21.15.173:45548) 2020-11-20T12:15:45.135 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bb67fc700 1 -- 172.21.15.173:0/4030814791 learned_addr learned my addr 172.21.15.173:0/4030814791 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:45.136 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bb6ffd700 1 --1- 172.21.15.173:0/4030814791 >> v1:172.21.15.173:6789/0 conn(0x7f3bb8067910 0x7f3bb814f1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.173:33268/0 (socket says 172.21.15.173:33268) 2020-11-20T12:15:45.136 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bb67fc700 1 -- 172.21.15.173:0/4030814791 >> v1:172.21.15.173:6789/0 conn(0x7f3bb8067910 legacy=0x7f3bb814f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:45.137 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bb67fc700 1 -- 172.21.15.173:0/4030814791 >> v1:172.21.15.116:6789/0 conn(0x7f3bb8064760 legacy=0x7f3bb80a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.137 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bb67fc700 1 -- 172.21.15.173:0/4030814791 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb8060410 con 0x7f3bb814f900 2020-11-20T12:15:45.137 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bb67fc700 1 --2- 172.21.15.173:0/4030814791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb814f900 0x7f3bb8067cf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f3bac00ba20 tx=0x7f3bac006eb0).ready entity=mon.2 client_cookie=ab5834f2aa9a228b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:45.138 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bb5ffb700 1 -- 172.21.15.173:0/4030814791 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac009960 con 0x7f3bb814f900 2020-11-20T12:15:45.138 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bb5ffb700 1 -- 172.21.15.173:0/4030814791 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bac009ac0 con 0x7f3bb814f900 2020-11-20T12:15:45.138 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.130+0000 7f3bb5ffb700 1 -- 172.21.15.173:0/4030814791 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac009e40 con 0x7f3bb814f900 2020-11-20T12:15:45.138 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 -- 172.21.15.173:0/4030814791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb814f900 msgr2=0x7f3bb8067cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.138 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 --2- 172.21.15.173:0/4030814791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb814f900 0x7f3bb8067cf0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f3bac00ba20 tx=0x7f3bac006eb0).stop 2020-11-20T12:15:45.139 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 -- 172.21.15.173:0/4030814791 shutdown_connections 2020-11-20T12:15:45.139 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 --2- 172.21.15.173:0/4030814791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb814f900 0x7f3bb8067cf0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.139 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 -- 172.21.15.173:0/4030814791 >> 172.21.15.173:0/4030814791 conn(0x7f3bb80c5680 msgr2=0x7f3bb80c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:45.139 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 -- 172.21.15.173:0/4030814791 shutdown_connections 2020-11-20T12:15:45.140 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 -- 172.21.15.173:0/4030814791 wait complete. 2020-11-20T12:15:45.144 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 Processor -- start 2020-11-20T12:15:45.144 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 -- start start 2020-11-20T12:15:45.144 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb8064760 0x7f3bb8150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.145 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bb8067910 0x7f3bb80a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.145 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb81421c0 0x7f3bb80bc450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.146 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80a40f0 con 0x7f3bb8067910 2020-11-20T12:15:45.146 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8150a30 con 0x7f3bb8064760 2020-11-20T12:15:45.146 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.134+0000 7f3bbd298700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8150cd0 con 0x7f3bb81421c0 2020-11-20T12:15:45.147 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb7fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb81421c0 0x7f3bb80bc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.147 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb7fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb81421c0 0x7f3bb80bc450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:45554/0 (socket says 172.21.15.173:45554) 2020-11-20T12:15:45.148 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb7fff700 1 -- 172.21.15.173:0/422849115 learned_addr learned my addr 172.21.15.173:0/422849115 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:45.148 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb6ffd700 1 --2- 172.21.15.173:0/422849115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb8064760 0x7f3bb8150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.149 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb67fc700 1 --2- 172.21.15.173:0/422849115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bb8067910 0x7f3bb80a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.149 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb7fff700 1 -- 172.21.15.173:0/422849115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb8064760 msgr2=0x7f3bb8150400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.149 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb7fff700 1 --2- 172.21.15.173:0/422849115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb8064760 0x7f3bb8150400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.150 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb7fff700 1 -- 172.21.15.173:0/422849115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bb8067910 msgr2=0x7f3bb80a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.150 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb7fff700 1 --2- 172.21.15.173:0/422849115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bb8067910 0x7f3bb80a3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.150 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb7fff700 1 -- 172.21.15.173:0/422849115 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb8150f70 con 0x7f3bb81421c0 2020-11-20T12:15:45.151 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb67fc700 1 --2- 172.21.15.173:0/422849115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bb8067910 0x7f3bb80a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:45.151 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb6ffd700 1 --2- 172.21.15.173:0/422849115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb8064760 0x7f3bb8150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:45.152 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bb7fff700 1 --2- 172.21.15.173:0/422849115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb81421c0 0x7f3bb80bc450 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f3bb000ca60 tx=0x7f3bb0007f40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:45.152 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3b9effd700 1 -- 172.21.15.173:0/422849115 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb000a830 con 0x7f3bb81421c0 2020-11-20T12:15:45.152 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3b9effd700 1 -- 172.21.15.173:0/422849115 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb000a990 con 0x7f3bb81421c0 2020-11-20T12:15:45.153 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bbd298700 1 -- 172.21.15.173:0/422849115 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb8151210 con 0x7f3bb81421c0 2020-11-20T12:15:45.153 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3b9effd700 1 -- 172.21.15.173:0/422849115 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb000ad10 con 0x7f3bb81421c0 2020-11-20T12:15:45.153 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bbd298700 1 -- 172.21.15.173:0/422849115 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb8142cd0 con 0x7f3bb81421c0 2020-11-20T12:15:45.154 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3b9effd700 1 -- 172.21.15.173:0/422849115 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f3bb0019030 con 0x7f3bb81421c0 2020-11-20T12:15:45.154 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3b9effd700 1 -- 172.21.15.173:0/422849115 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f3bb0012870 con 0x7f3bb81421c0 2020-11-20T12:15:45.154 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.138+0000 7f3bbd298700 1 -- 172.21.15.173:0/422849115 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb805ca30 con 0x7f3bb81421c0 2020-11-20T12:15:45.154 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.142+0000 7f3b9effd700 1 -- 172.21.15.173:0/422849115 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f3bb0021060 con 0x7f3bb81421c0 2020-11-20T12:15:45.228 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.226+0000 7f3bbd298700 1 -- 172.21.15.173:0/422849115 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b3238680-8ed8-4a81-aa6a-20548bb44de2", "id": 4} v 0) v1 -- 0x7f3bb8068210 con 0x7f3bb81421c0 2020-11-20T12:15:45.238 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.238+0000 7f3b9effd700 1 -- 172.21.15.173:0/422849115 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b3238680-8ed8-4a81-aa6a-20548bb44de2", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3bb0017070 con 0x7f3bb81421c0 2020-11-20T12:15:45.238 INFO:teuthology.orchestra.run.smithi173.stdout:4 2020-11-20T12:15:45.240 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.238+0000 7f3bbd298700 1 -- 172.21.15.173:0/422849115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb81421c0 msgr2=0x7f3bb80bc450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.240 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.238+0000 7f3bbd298700 1 --2- 172.21.15.173:0/422849115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb81421c0 0x7f3bb80bc450 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f3bb000ca60 tx=0x7f3bb0007f40).stop 2020-11-20T12:15:45.240 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.238+0000 7f3bbd298700 1 -- 172.21.15.173:0/422849115 shutdown_connections 2020-11-20T12:15:45.240 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.238+0000 7f3bbd298700 1 --2- 172.21.15.173:0/422849115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bb81421c0 0x7f3bb80bc450 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.241 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.238+0000 7f3bbd298700 1 --2- 172.21.15.173:0/422849115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bb8064760 0x7f3bb8150400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.241 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.238+0000 7f3bbd298700 1 --2- 172.21.15.173:0/422849115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bb8067910 0x7f3bb80a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.241 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.238+0000 7f3bbd298700 1 -- 172.21.15.173:0/422849115 >> 172.21.15.173:0/422849115 conn(0x7f3bb80c5680 msgr2=0x7f3bb8063be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:45.241 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.238+0000 7f3bbd298700 1 -- 172.21.15.173:0/422849115 shutdown_connections 2020-11-20T12:15:45.242 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.238+0000 7f3bbd298700 1 -- 172.21.15.173:0/422849115 wait complete. 2020-11-20T12:15:45.262 INFO:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 38264535-e8e9-409b-bbc5-46f51fcfcd75 5 2020-11-20T12:15:45.421 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.418+0000 7f2f7e7fd700 1 Processor -- start 2020-11-20T12:15:45.421 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.418+0000 7f2f7e7fd700 1 -- start start 2020-11-20T12:15:45.423 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f7e7fd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814bf70 0x7f2f780c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.424 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f7e7fd700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f781524b0 con 0x7f2f780c0240 2020-11-20T12:15:45.424 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f7e7fd700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f78152750 con 0x7f2f7814c350 2020-11-20T12:15:45.424 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f7e7fd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f780c10e0 con 0x7f2f7814bf70 2020-11-20T12:15:45.424 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f77fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814bf70 0x7f2f780c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.425 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f77fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814bf70 0x7f2f780c0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:45560/0 (socket says 172.21.15.173:45560) 2020-11-20T12:15:45.425 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f77fff700 1 -- 172.21.15.173:0/1724467759 learned_addr learned my addr 172.21.15.173:0/1724467759 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:45.426 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f77fff700 1 -- 172.21.15.173:0/1724467759 >> v1:172.21.15.173:6789/0 conn(0x7f2f7814c350 legacy=0x7f2f78062c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:45.427 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f77fff700 1 -- 172.21.15.173:0/1724467759 >> v1:172.21.15.116:6789/0 conn(0x7f2f780c0240 legacy=0x7f2f7814edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:45.427 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f77fff700 1 -- 172.21.15.173:0/1724467759 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f781529f0 con 0x7f2f7814bf70 2020-11-20T12:15:45.427 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f77fff700 1 --2- 172.21.15.173:0/1724467759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814bf70 0x7f2f780c0bf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f2f6800ba20 tx=0x7f2f68006eb0).ready entity=mon.2 client_cookie=cb49c80d04b5dbc5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:45.427 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.422+0000 7f2f76ffd700 1 -- 172.21.15.173:0/1724467759 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68009920 con 0x7f2f7814bf70 2020-11-20T12:15:45.428 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.426+0000 7f2f76ffd700 1 -- 172.21.15.173:0/1724467759 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f68009a80 con 0x7f2f7814bf70 2020-11-20T12:15:45.428 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.426+0000 7f2f76ffd700 1 -- 172.21.15.173:0/1724467759 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68009e00 con 0x7f2f7814bf70 2020-11-20T12:15:45.428 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.426+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/1724467759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814bf70 msgr2=0x7f2f780c0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.428 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.426+0000 7f2f7e7fd700 1 --2- 172.21.15.173:0/1724467759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814bf70 0x7f2f780c0bf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f2f6800ba20 tx=0x7f2f68006eb0).stop 2020-11-20T12:15:45.428 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.426+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/1724467759 shutdown_connections 2020-11-20T12:15:45.429 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.426+0000 7f2f7e7fd700 1 --2- 172.21.15.173:0/1724467759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f7814bf70 0x7f2f780c0bf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.429 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.426+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/1724467759 >> 172.21.15.173:0/1724467759 conn(0x7f2f78060170 msgr2=0x7f2f78060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:45.429 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.426+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/1724467759 shutdown_connections 2020-11-20T12:15:45.429 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.426+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/1724467759 wait complete. 2020-11-20T12:15:45.431 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.426+0000 7f2f7e7fd700 1 Processor -- start 2020-11-20T12:15:45.431 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.426+0000 7f2f7e7fd700 1 -- start start 2020-11-20T12:15:45.431 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f7e7fd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f780c0240 0x7f2f7813dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.431 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f7e7fd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f7814bf70 0x7f2f780a9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.432 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f7e7fd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f7814c350 0x7f2f78153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.432 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f7e7fd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f781569d0 con 0x7f2f7814c350 2020-11-20T12:15:45.432 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f7e7fd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f78155f50 con 0x7f2f7814bf70 2020-11-20T12:15:45.432 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f7e7fd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f781561f0 con 0x7f2f780c0240 2020-11-20T12:15:45.433 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f77fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f780c0240 0x7f2f7813dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.433 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f77fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f780c0240 0x7f2f7813dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:45566/0 (socket says 172.21.15.173:45566) 2020-11-20T12:15:45.433 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f77fff700 1 -- 172.21.15.173:0/514137130 learned_addr learned my addr 172.21.15.173:0/514137130 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:45.433 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f777fe700 1 --2- 172.21.15.173:0/514137130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f7814bf70 0x7f2f780a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.433 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f7cd9a700 1 --2- 172.21.15.173:0/514137130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f7814c350 0x7f2f78153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.434 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f77fff700 1 -- 172.21.15.173:0/514137130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f7814bf70 msgr2=0x7f2f780a9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.434 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f77fff700 1 --2- 172.21.15.173:0/514137130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f7814bf70 0x7f2f780a9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.434 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f77fff700 1 -- 172.21.15.173:0/514137130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f7814c350 msgr2=0x7f2f78153270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.434 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f77fff700 1 --2- 172.21.15.173:0/514137130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f7814c350 0x7f2f78153270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.434 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f77fff700 1 -- 172.21.15.173:0/514137130 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f78156490 con 0x7f2f780c0240 2020-11-20T12:15:45.435 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f777fe700 1 --2- 172.21.15.173:0/514137130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f7814bf70 0x7f2f780a9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:45.435 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f7cd9a700 1 --2- 172.21.15.173:0/514137130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f7814c350 0x7f2f78153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:45.436 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f77fff700 1 --2- 172.21.15.173:0/514137130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f780c0240 0x7f2f7813dbe0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f2f68006c40 tx=0x7f2f68015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:45.436 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f57fff700 1 -- 172.21.15.173:0/514137130 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f68011e50 con 0x7f2f780c0240 2020-11-20T12:15:45.436 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f57fff700 1 -- 172.21.15.173:0/514137130 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6801d450 con 0x7f2f780c0240 2020-11-20T12:15:45.437 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f57fff700 1 -- 172.21.15.173:0/514137130 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6801d810 con 0x7f2f780c0240 2020-11-20T12:15:45.437 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/514137130 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f78156730 con 0x7f2f780c0240 2020-11-20T12:15:45.437 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/514137130 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f780a99f0 con 0x7f2f780c0240 2020-11-20T12:15:45.437 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f57fff700 1 -- 172.21.15.173:0/514137130 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f2f6800f070 con 0x7f2f780c0240 2020-11-20T12:15:45.437 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.430+0000 7f2f57fff700 1 -- 172.21.15.173:0/514137130 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7f2f6801dbc0 con 0x7f2f780c0240 2020-11-20T12:15:45.438 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.434+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/514137130 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f780a9be0 con 0x7f2f780c0240 2020-11-20T12:15:45.438 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.434+0000 7f2f57fff700 1 -- 172.21.15.173:0/514137130 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f2f68027530 con 0x7f2f780c0240 2020-11-20T12:15:45.525 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.522+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/514137130 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "38264535-e8e9-409b-bbc5-46f51fcfcd75", "id": 5} v 0) v1 -- 0x7f2f7805ca30 con 0x7f2f780c0240 2020-11-20T12:15:45.535 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.534+0000 7f2f57fff700 1 -- 172.21.15.173:0/514137130 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "38264535-e8e9-409b-bbc5-46f51fcfcd75", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2f68006020 con 0x7f2f780c0240 2020-11-20T12:15:45.535 INFO:teuthology.orchestra.run.smithi173.stdout:5 2020-11-20T12:15:45.536 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.534+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/514137130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f780c0240 msgr2=0x7f2f7813dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.537 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.534+0000 7f2f7e7fd700 1 --2- 172.21.15.173:0/514137130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f780c0240 0x7f2f7813dbe0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f2f68006c40 tx=0x7f2f68015000).stop 2020-11-20T12:15:45.537 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.534+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/514137130 shutdown_connections 2020-11-20T12:15:45.537 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.534+0000 7f2f7e7fd700 1 --2- 172.21.15.173:0/514137130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2f7814c350 0x7f2f78153270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.537 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.534+0000 7f2f7e7fd700 1 --2- 172.21.15.173:0/514137130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2f780c0240 0x7f2f7813dbe0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.538 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.534+0000 7f2f7e7fd700 1 --2- 172.21.15.173:0/514137130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2f7814bf70 0x7f2f780a9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.538 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.534+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/514137130 >> 172.21.15.173:0/514137130 conn(0x7f2f78060170 msgr2=0x7f2f780c1220 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:45.538 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.538+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/514137130 shutdown_connections 2020-11-20T12:15:45.539 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.538+0000 7f2f7e7fd700 1 -- 172.21.15.173:0/514137130 wait complete. 2020-11-20T12:15:45.565 INFO:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new be530a7d-5f7c-4107-9171-15f4a6a4c468 6 2020-11-20T12:15:45.725 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.722+0000 7fc17491d700 1 Processor -- start 2020-11-20T12:15:45.725 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.722+0000 7fc17491d700 1 -- start start 2020-11-20T12:15:45.728 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc17491d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc17014bc40 0x7fc170151d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.728 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc17491d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc1700c5410 0x7fc1700be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.729 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc17491d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1701526b0 con 0x7fc170156020 2020-11-20T12:15:45.729 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc17491d700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc170152950 con 0x7fc1700c5410 2020-11-20T12:15:45.729 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc17491d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc170152bf0 con 0x7fc17014bc40 2020-11-20T12:15:45.730 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc16e59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc17014bc40 0x7fc170151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.730 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc16dd9b700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fc170156020 0x7fc1700c4d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.173:33292/0 (socket says 172.21.15.173:33292) 2020-11-20T12:15:45.731 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc16dd9b700 1 -- 172.21.15.173:0/569755219 learned_addr learned my addr 172.21.15.173:0/569755219 (peer_addr_for_me v1:172.21.15.173:0/0) 2020-11-20T12:15:45.731 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc16ed9d700 1 --2- 172.21.15.173:0/569755219 >> v2:172.21.15.173:3300/0 conn(0x7fc1700c5410 0x7fc1700be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.731 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc16e59c700 1 -- 172.21.15.173:0/569755219 >> v2:172.21.15.173:3300/0 conn(0x7fc1700c5410 msgr2=0x7fc1700be770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:45.731 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc16e59c700 1 --2- 172.21.15.173:0/569755219 >> v2:172.21.15.173:3300/0 conn(0x7fc1700c5410 0x7fc1700be770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.731 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc16e59c700 1 -- 172.21.15.173:0/569755219 >> v1:172.21.15.173:6789/0 conn(0x7fc170156020 legacy=0x7fc1700c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.732 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.726+0000 7fc16e59c700 1 -- 172.21.15.173:0/569755219 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1700a8670 con 0x7fc17014bc40 2020-11-20T12:15:45.732 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc16e59c700 1 --2- 172.21.15.173:0/569755219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc17014bc40 0x7fc170151d70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fc16000b9a0 tx=0x7fc160006df0).ready entity=mon.2 client_cookie=68bffba805476a41 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:45.732 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc16d59a700 1 -- 172.21.15.173:0/569755219 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1600097b0 con 0x7fc17014bc40 2020-11-20T12:15:45.732 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc16d59a700 1 -- 172.21.15.173:0/569755219 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc160009910 con 0x7fc17014bc40 2020-11-20T12:15:45.732 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc16d59a700 1 -- 172.21.15.173:0/569755219 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc160009c90 con 0x7fc17014bc40 2020-11-20T12:15:45.733 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc17491d700 1 -- 172.21.15.173:0/569755219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc17014bc40 msgr2=0x7fc170151d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.733 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc17491d700 1 --2- 172.21.15.173:0/569755219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc17014bc40 0x7fc170151d70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fc16000b9a0 tx=0x7fc160006df0).stop 2020-11-20T12:15:45.734 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc17491d700 1 -- 172.21.15.173:0/569755219 shutdown_connections 2020-11-20T12:15:45.734 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc17491d700 1 --2- 172.21.15.173:0/569755219 >> v2:172.21.15.173:3300/0 conn(0x7fc1700c5410 0x7fc1700be770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.734 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc17491d700 1 --2- 172.21.15.173:0/569755219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc17014bc40 0x7fc170151d70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.735 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc17491d700 1 -- 172.21.15.173:0/569755219 >> 172.21.15.173:0/569755219 conn(0x7fc1700678a0 msgr2=0x7fc170067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:45.735 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc17491d700 1 -- 172.21.15.173:0/569755219 shutdown_connections 2020-11-20T12:15:45.735 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc17491d700 1 -- 172.21.15.173:0/569755219 wait complete. 2020-11-20T12:15:45.735 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc17491d700 1 Processor -- start 2020-11-20T12:15:45.736 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.730+0000 7fc17491d700 1 -- start start 2020-11-20T12:15:45.736 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc17491d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1700c5410 0x7fc17014d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.736 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc17491d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17014bc40 0x7fc17014c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.736 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc17491d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc170156020 0x7fc1700ac570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:45.736 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc17491d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17014f7c0 con 0x7fc170156020 2020-11-20T12:15:45.737 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc17491d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17014efe0 con 0x7fc17014bc40 2020-11-20T12:15:45.737 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc17491d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17014f280 con 0x7fc1700c5410 2020-11-20T12:15:45.738 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc16e59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1700c5410 0x7fc17014d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.739 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc16e59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1700c5410 0x7fc17014d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:45578/0 (socket says 172.21.15.173:45578) 2020-11-20T12:15:45.739 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc16e59c700 1 -- 172.21.15.173:0/577122262 learned_addr learned my addr 172.21.15.173:0/577122262 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:45.739 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc16dd9b700 1 --2- 172.21.15.173:0/577122262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17014bc40 0x7fc17014c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.739 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc16ed9d700 1 --2- 172.21.15.173:0/577122262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc170156020 0x7fc1700ac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:45.739 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc16dd9b700 1 -- 172.21.15.173:0/577122262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1700c5410 msgr2=0x7fc17014d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.740 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc16dd9b700 1 --2- 172.21.15.173:0/577122262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1700c5410 0x7fc17014d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.740 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc16dd9b700 1 -- 172.21.15.173:0/577122262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc170156020 msgr2=0x7fc1700ac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.740 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc16dd9b700 1 --2- 172.21.15.173:0/577122262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc170156020 0x7fc1700ac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.740 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc16dd9b700 1 -- 172.21.15.173:0/577122262 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17014f520 con 0x7fc17014bc40 2020-11-20T12:15:45.740 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc16dd9b700 1 --2- 172.21.15.173:0/577122262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17014bc40 0x7fc17014c1e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fc170060be0 tx=0x7fc164007a50).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:45.741 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc15e7fc700 1 -- 172.21.15.173:0/577122262 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164007320 con 0x7fc17014bc40 2020-11-20T12:15:45.741 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc15e7fc700 1 -- 172.21.15.173:0/577122262 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1640054c0 con 0x7fc17014bc40 2020-11-20T12:15:45.741 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc15e7fc700 1 -- 172.21.15.173:0/577122262 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc164005840 con 0x7fc17014bc40 2020-11-20T12:15:45.741 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.734+0000 7fc17491d700 1 -- 172.21.15.173:0/577122262 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1701615a0 con 0x7fc17014bc40 2020-11-20T12:15:45.742 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.738+0000 7fc17491d700 1 -- 172.21.15.173:0/577122262 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17014c8f0 con 0x7fc17014bc40 2020-11-20T12:15:45.742 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.738+0000 7fc15e7fc700 1 -- 172.21.15.173:0/577122262 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fc164014410 con 0x7fc17014bc40 2020-11-20T12:15:45.742 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.738+0000 7fc15e7fc700 1 -- 172.21.15.173:0/577122262 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7fc164014dd0 con 0x7fc17014bc40 2020-11-20T12:15:45.742 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.738+0000 7fc17491d700 1 -- 172.21.15.173:0/577122262 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17005ca30 con 0x7fc17014bc40 2020-11-20T12:15:45.743 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.742+0000 7fc15e7fc700 1 -- 172.21.15.173:0/577122262 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fc1640146a0 con 0x7fc17014bc40 2020-11-20T12:15:45.829 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.830+0000 7fc17491d700 1 -- 172.21.15.173:0/577122262 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd new", "uuid": "be530a7d-5f7c-4107-9171-15f4a6a4c468", "id": 6} v 0) v1 -- 0x7fc1700a8360 con 0x7fc17014bc40 2020-11-20T12:15:45.840 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.838+0000 7fc15e7fc700 1 -- 172.21.15.173:0/577122262 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "be530a7d-5f7c-4107-9171-15f4a6a4c468", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc164014840 con 0x7fc17014bc40 2020-11-20T12:15:45.840 INFO:teuthology.orchestra.run.smithi173.stdout:6 2020-11-20T12:15:45.841 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.842+0000 7fc17491d700 1 -- 172.21.15.173:0/577122262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17014bc40 msgr2=0x7fc17014c1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:45.841 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.842+0000 7fc17491d700 1 --2- 172.21.15.173:0/577122262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17014bc40 0x7fc17014c1e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fc170060be0 tx=0x7fc164007a50).stop 2020-11-20T12:15:45.842 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.842+0000 7fc17491d700 1 -- 172.21.15.173:0/577122262 shutdown_connections 2020-11-20T12:15:45.842 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.842+0000 7fc17491d700 1 --2- 172.21.15.173:0/577122262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc170156020 0x7fc1700ac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.843 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.842+0000 7fc17491d700 1 --2- 172.21.15.173:0/577122262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1700c5410 0x7fc17014d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.843 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.842+0000 7fc17491d700 1 --2- 172.21.15.173:0/577122262 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc17014bc40 0x7fc17014c1e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:45.843 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.842+0000 7fc17491d700 1 -- 172.21.15.173:0/577122262 >> 172.21.15.173:0/577122262 conn(0x7fc1700678a0 msgr2=0x7fc17014b090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:45.843 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.842+0000 7fc17491d700 1 -- 172.21.15.173:0/577122262 shutdown_connections 2020-11-20T12:15:45.843 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:45.842+0000 7fc17491d700 1 -- 172.21.15.173:0/577122262 wait complete. 2020-11-20T12:15:45.863 INFO:teuthology.orchestra.run.smithi173:> sudo ceph --cluster ceph osd new 949fed52-858c-459b-8fcc-317b259ed305 7 2020-11-20T12:15:46.018 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.018+0000 7fac456d6700 1 Processor -- start 2020-11-20T12:15:46.018 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.018+0000 7fac456d6700 1 -- start start 2020-11-20T12:15:46.020 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.018+0000 7fac456d6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a0f00 0x7fac400a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:46.021 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.018+0000 7fac456d6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fac400a12e0 0x7fac40150370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:46.021 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.018+0000 7fac456d6700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac400a50a0 con 0x7fac400c2df0 2020-11-20T12:15:46.022 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.018+0000 7fac456d6700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac400a5340 con 0x7fac400a12e0 2020-11-20T12:15:46.022 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.018+0000 7fac456d6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac400a55e0 con 0x7fac400a0f00 2020-11-20T12:15:46.023 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac44ed5700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fac400c2df0 0x7fac40149240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.173:33306/0 (socket says 172.21.15.173:33306) 2020-11-20T12:15:46.023 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac44ed5700 1 -- 172.21.15.173:0/1386479029 learned_addr learned my addr 172.21.15.173:0/1386479029 (peer_addr_for_me v1:172.21.15.173:0/0) 2020-11-20T12:15:46.023 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac3f7fe700 1 --2- 172.21.15.173:0/1386479029 >> v2:172.21.15.173:3300/0 conn(0x7fac400a12e0 0x7fac40150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:46.023 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac3ffff700 1 --2- 172.21.15.173:0/1386479029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a0f00 0x7fac400a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:46.024 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac3f7fe700 1 -- 172.21.15.173:0/1386479029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a0f00 msgr2=0x7fac400a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:46.024 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac3f7fe700 1 --2- 172.21.15.173:0/1386479029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a0f00 0x7fac400a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:46.024 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac3f7fe700 1 -- 172.21.15.173:0/1386479029 >> v1:172.21.15.173:6789/0 conn(0x7fac400c2df0 legacy=0x7fac40149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:46.024 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac3f7fe700 1 -- 172.21.15.173:0/1386479029 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac400a5880 con 0x7fac400a12e0 2020-11-20T12:15:46.024 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac3f7fe700 1 --2- 172.21.15.173:0/1386479029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac400a12e0 0x7fac40150370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fac3400b9a0 tx=0x7fac34006df0).ready entity=mon.1 client_cookie=97d41d926e74eb21 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:46.026 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac3effd700 1 -- 172.21.15.173:0/1386479029 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac340097b0 con 0x7fac400a12e0 2020-11-20T12:15:46.026 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac3effd700 1 -- 172.21.15.173:0/1386479029 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac34009910 con 0x7fac400a12e0 2020-11-20T12:15:46.026 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac3effd700 1 -- 172.21.15.173:0/1386479029 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34009c90 con 0x7fac400a12e0 2020-11-20T12:15:46.027 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac456d6700 1 -- 172.21.15.173:0/1386479029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac400a12e0 msgr2=0x7fac40150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:46.027 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac456d6700 1 --2- 172.21.15.173:0/1386479029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac400a12e0 0x7fac40150370 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7fac3400b9a0 tx=0x7fac34006df0).stop 2020-11-20T12:15:46.027 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac456d6700 1 -- 172.21.15.173:0/1386479029 shutdown_connections 2020-11-20T12:15:46.027 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac456d6700 1 --2- 172.21.15.173:0/1386479029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a0f00 0x7fac400a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:46.028 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac456d6700 1 --2- 172.21.15.173:0/1386479029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac400a12e0 0x7fac40150370 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:46.028 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac456d6700 1 -- 172.21.15.173:0/1386479029 >> 172.21.15.173:0/1386479029 conn(0x7fac40060c30 msgr2=0x7fac400c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:46.028 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac456d6700 1 -- 172.21.15.173:0/1386479029 shutdown_connections 2020-11-20T12:15:46.028 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.022+0000 7fac456d6700 1 -- 172.21.15.173:0/1386479029 wait complete. 2020-11-20T12:15:46.028 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac456d6700 1 Processor -- start 2020-11-20T12:15:46.029 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac456d6700 1 -- start start 2020-11-20T12:15:46.031 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac456d6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac400a0f00 0x7fac40062d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:46.031 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac456d6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a12e0 0x7fac4014c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:46.031 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac456d6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac400c2df0 0x7fac401622d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:46.032 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac456d6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac401630c0 con 0x7fac400c2df0 2020-11-20T12:15:46.032 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac456d6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac40061ba0 con 0x7fac400a0f00 2020-11-20T12:15:46.032 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac456d6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac40061e40 con 0x7fac400a12e0 2020-11-20T12:15:46.032 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac3f7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a12e0 0x7fac4014c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:46.032 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac3ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac400a0f00 0x7fac40062d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:46.033 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac3f7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a12e0 0x7fac4014c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.173:45590/0 (socket says 172.21.15.173:45590) 2020-11-20T12:15:46.033 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac3f7fe700 1 -- 172.21.15.173:0/3963669220 learned_addr learned my addr 172.21.15.173:0/3963669220 (peer_addr_for_me v2:172.21.15.173:0/0) 2020-11-20T12:15:46.033 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac44ed5700 1 --2- 172.21.15.173:0/3963669220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac400c2df0 0x7fac401622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:46.033 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac3f7fe700 1 -- 172.21.15.173:0/3963669220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac400a0f00 msgr2=0x7fac40062d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:46.034 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac3f7fe700 1 --2- 172.21.15.173:0/3963669220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac400a0f00 0x7fac40062d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:46.034 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac3f7fe700 1 -- 172.21.15.173:0/3963669220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac400c2df0 msgr2=0x7fac401622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:46.034 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac3f7fe700 1 --2- 172.21.15.173:0/3963669220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac400c2df0 0x7fac401622d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:46.034 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac3f7fe700 1 -- 172.21.15.173:0/3963669220 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac400620e0 con 0x7fac400a12e0 2020-11-20T12:15:46.034 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac3ffff700 1 --2- 172.21.15.173:0/3963669220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac400a0f00 0x7fac40062d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:46.035 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.026+0000 7fac3f7fe700 1 --2- 172.21.15.173:0/3963669220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a12e0 0x7fac4014c530 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fac34006e30 tx=0x7fac34015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:46.035 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.030+0000 7fac1ffff700 1 -- 172.21.15.173:0/3963669220 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac34011dd0 con 0x7fac400a12e0 2020-11-20T12:15:46.035 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.030+0000 7fac1ffff700 1 -- 172.21.15.173:0/3963669220 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac3401e460 con 0x7fac400a12e0 2020-11-20T12:15:46.035 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.030+0000 7fac456d6700 1 -- 172.21.15.173:0/3963669220 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac40062380 con 0x7fac400a12e0 2020-11-20T12:15:46.035 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.030+0000 7fac1ffff700 1 -- 172.21.15.173:0/3963669220 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3401e7e0 con 0x7fac400a12e0 2020-11-20T12:15:46.036 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.030+0000 7fac456d6700 1 -- 172.21.15.173:0/3963669220 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac400628c0 con 0x7fac400a12e0 2020-11-20T12:15:46.036 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.030+0000 7fac1ffff700 1 -- 172.21.15.173:0/3963669220 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fac34025030 con 0x7fac400a12e0 2020-11-20T12:15:46.036 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.030+0000 7fac1ffff700 1 -- 172.21.15.173:0/3963669220 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7fac3401ee30 con 0x7fac400a12e0 2020-11-20T12:15:46.036 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.030+0000 7fac456d6700 1 -- 172.21.15.173:0/3963669220 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac40044f60 con 0x7fac400a12e0 2020-11-20T12:15:46.037 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.034+0000 7fac1ffff700 1 -- 172.21.15.173:0/3963669220 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7fac3402d060 con 0x7fac400a12e0 2020-11-20T12:15:46.121 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.118+0000 7fac456d6700 1 -- 172.21.15.173:0/3963669220 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd new", "uuid": "949fed52-858c-459b-8fcc-317b259ed305", "id": 7} v 0) v1 -- 0x7fac40064080 con 0x7fac400a12e0 2020-11-20T12:15:46.130 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.130+0000 7fac1ffff700 1 -- 172.21.15.173:0/3963669220 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "949fed52-858c-459b-8fcc-317b259ed305", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fac34023070 con 0x7fac400a12e0 2020-11-20T12:15:46.131 INFO:teuthology.orchestra.run.smithi173.stdout:7 2020-11-20T12:15:46.131 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.130+0000 7fac456d6700 1 -- 172.21.15.173:0/3963669220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a12e0 msgr2=0x7fac4014c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:46.132 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.130+0000 7fac456d6700 1 --2- 172.21.15.173:0/3963669220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a12e0 0x7fac4014c530 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7fac34006e30 tx=0x7fac34015000).stop 2020-11-20T12:15:46.132 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.130+0000 7fac456d6700 1 -- 172.21.15.173:0/3963669220 shutdown_connections 2020-11-20T12:15:46.132 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.130+0000 7fac456d6700 1 --2- 172.21.15.173:0/3963669220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac400c2df0 0x7fac401622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:46.132 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.130+0000 7fac456d6700 1 --2- 172.21.15.173:0/3963669220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac400a0f00 0x7fac40062d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:46.133 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.130+0000 7fac456d6700 1 --2- 172.21.15.173:0/3963669220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac400a12e0 0x7fac4014c530 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:46.133 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.130+0000 7fac456d6700 1 -- 172.21.15.173:0/3963669220 >> 172.21.15.173:0/3963669220 conn(0x7fac40060c30 msgr2=0x7fac400a30b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:46.134 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.130+0000 7fac456d6700 1 -- 172.21.15.173:0/3963669220 shutdown_connections 2020-11-20T12:15:46.134 INFO:teuthology.orchestra.run.smithi173.stderr:2020-11-20T12:15:46.130+0000 7fac456d6700 1 -- 172.21.15.173:0/3963669220 wait complete. 2020-11-20T12:15:46.153 INFO:tasks.ceph.osd.0:Restarting daemon 2020-11-20T12:15:46.154 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2020-11-20T12:15:46.159 INFO:tasks.ceph.osd.0:Started 2020-11-20T12:15:46.159 INFO:tasks.ceph.osd.1:Restarting daemon 2020-11-20T12:15:46.160 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2020-11-20T12:15:46.164 INFO:tasks.ceph.osd.1:Started 2020-11-20T12:15:46.165 INFO:tasks.ceph.osd.2:Restarting daemon 2020-11-20T12:15:46.165 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2020-11-20T12:15:46.169 INFO:tasks.ceph.osd.2:Started 2020-11-20T12:15:46.169 INFO:tasks.ceph.osd.3:Restarting daemon 2020-11-20T12:15:46.169 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2020-11-20T12:15:46.175 INFO:tasks.ceph.osd.3:Started 2020-11-20T12:15:46.175 INFO:tasks.ceph.osd.4:Restarting daemon 2020-11-20T12:15:46.176 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2020-11-20T12:15:46.181 INFO:tasks.ceph.osd.4:Started 2020-11-20T12:15:46.182 INFO:tasks.ceph.osd.5:Restarting daemon 2020-11-20T12:15:46.182 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2020-11-20T12:15:46.188 INFO:tasks.ceph.osd.5:Started 2020-11-20T12:15:46.188 INFO:tasks.ceph.osd.6:Restarting daemon 2020-11-20T12:15:46.189 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2020-11-20T12:15:46.193 INFO:tasks.ceph.osd.6:Started 2020-11-20T12:15:46.193 INFO:tasks.ceph.osd.7:Restarting daemon 2020-11-20T12:15:46.193 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2020-11-20T12:15:46.199 INFO:tasks.ceph.osd.7:Started 2020-11-20T12:15:46.200 INFO:tasks.ceph:Waiting for OSDs to come up 2020-11-20T12:15:46.824 INFO:tasks.ceph.osd.0.smithi116.stderr:2020-11-20T12:15:46.820+0000 7fe56829dd80 -1 Falling back to public interface 2020-11-20T12:15:46.825 INFO:tasks.ceph.osd.3.smithi116.stderr:2020-11-20T12:15:46.824+0000 7f4482602d80 -1 Falling back to public interface 2020-11-20T12:15:46.829 INFO:tasks.ceph.osd.2.smithi116.stderr:2020-11-20T12:15:46.828+0000 7fadba627d80 -1 Falling back to public interface 2020-11-20T12:15:46.829 INFO:tasks.ceph.osd.4.smithi173.stderr:2020-11-20T12:15:46.826+0000 7f5a4d1f6d80 -1 Falling back to public interface 2020-11-20T12:15:46.845 INFO:tasks.ceph.osd.1.smithi116.stderr:2020-11-20T12:15:46.844+0000 7f96134e1d80 -1 Falling back to public interface 2020-11-20T12:15:46.856 INFO:tasks.ceph.osd.6.smithi173.stderr:2020-11-20T12:15:46.854+0000 7f1df00bad80 -1 Falling back to public interface 2020-11-20T12:15:46.865 INFO:tasks.ceph.osd.7.smithi173.stderr:2020-11-20T12:15:46.862+0000 7f2cf9706d80 -1 Falling back to public interface 2020-11-20T12:15:46.867 INFO:tasks.ceph.osd.5.smithi173.stderr:2020-11-20T12:15:46.866+0000 7f1a7c046d80 -1 Falling back to public interface 2020-11-20T12:15:47.007 INFO:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-20T12:15:47.167 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.164+0000 7f21a172a700 1 Processor -- start 2020-11-20T12:15:47.168 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.164+0000 7f21a172a700 1 -- start start 2020-11-20T12:15:47.168 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f21a172a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f219c14fc80 0x7f219c150060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:47.169 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f21a172a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f219c066bc0 0x7f219c0a28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:47.169 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f21a172a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f219c065fb0 con 0x7f219c0a2de0 2020-11-20T12:15:47.169 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f21a172a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f219c156d00 con 0x7f219c14fc80 2020-11-20T12:15:47.169 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f21a172a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f219c0661e0 con 0x7f219c066bc0 2020-11-20T12:15:47.170 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f219affd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f219c14fc80 0x7f219c150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:47.171 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f219affd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f219c14fc80 0x7f219c150060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42454/0 (socket says 172.21.15.116:42454) 2020-11-20T12:15:47.171 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f219affd700 1 -- 172.21.15.116:0/2650868760 learned_addr learned my addr 172.21.15.116:0/2650868760 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:47.171 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f219a7fc700 1 --2- 172.21.15.116:0/2650868760 >> v2:172.21.15.173:3300/0 conn(0x7f219c066bc0 0x7f219c0a28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:47.171 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f219affd700 1 -- 172.21.15.116:0/2650868760 >> v2:172.21.15.173:3300/0 conn(0x7f219c066bc0 msgr2=0x7f219c0a28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:47.172 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f219affd700 1 --2- 172.21.15.116:0/2650868760 >> v2:172.21.15.173:3300/0 conn(0x7f219c066bc0 0x7f219c0a28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:47.172 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f219affd700 1 -- 172.21.15.116:0/2650868760 >> v1:172.21.15.173:6789/0 conn(0x7f219c0a2de0 legacy=0x7f219c155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:47.172 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f219affd700 1 -- 172.21.15.116:0/2650868760 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c066480 con 0x7f219c14fc80 2020-11-20T12:15:47.172 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f219a7fc700 1 --2- 172.21.15.116:0/2650868760 >> v2:172.21.15.173:3300/0 conn(0x7f219c066bc0 0x7f219c0a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:47.172 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f219affd700 1 --2- 172.21.15.116:0/2650868760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f219c14fc80 0x7f219c150060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f219c15a610 tx=0x7f218c006eb0).ready entity=mon.0 client_cookie=e669e314c9d2dfe8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:47.174 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f2199ffb700 1 -- 172.21.15.116:0/2650868760 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c009960 con 0x7f219c14fc80 2020-11-20T12:15:47.174 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f2199ffb700 1 -- 172.21.15.116:0/2650868760 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218c009ac0 con 0x7f219c14fc80 2020-11-20T12:15:47.174 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f2199ffb700 1 -- 172.21.15.116:0/2650868760 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218c009e40 con 0x7f219c14fc80 2020-11-20T12:15:47.175 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f21a172a700 1 -- 172.21.15.116:0/2650868760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f219c14fc80 msgr2=0x7f219c150060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:47.175 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.168+0000 7f21a172a700 1 --2- 172.21.15.116:0/2650868760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f219c14fc80 0x7f219c150060 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f219c15a610 tx=0x7f218c006eb0).stop 2020-11-20T12:15:47.176 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.172+0000 7f21a172a700 1 -- 172.21.15.116:0/2650868760 shutdown_connections 2020-11-20T12:15:47.176 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.172+0000 7f21a172a700 1 --2- 172.21.15.116:0/2650868760 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f219c14fc80 0x7f219c150060 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:47.176 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.172+0000 7f21a172a700 1 --2- 172.21.15.116:0/2650868760 >> v2:172.21.15.173:3300/0 conn(0x7f219c066bc0 0x7f219c0a28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:47.177 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.172+0000 7f21a172a700 1 -- 172.21.15.116:0/2650868760 >> 172.21.15.116:0/2650868760 conn(0x7f219c060be0 msgr2=0x7f219c0c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:47.177 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.172+0000 7f21a172a700 1 -- 172.21.15.116:0/2650868760 shutdown_connections 2020-11-20T12:15:47.177 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.172+0000 7f21a172a700 1 -- 172.21.15.116:0/2650868760 wait complete. 2020-11-20T12:15:47.178 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.172+0000 7f21a172a700 1 Processor -- start 2020-11-20T12:15:47.178 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f21a172a700 1 -- start start 2020-11-20T12:15:47.179 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f21a172a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f219c066bc0 0x7f219c062d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:47.179 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f21a172a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f219c0a2de0 0x7f219c0c2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:47.179 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f21a172a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f219c14fc80 0x7f219c14d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:47.179 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f21a172a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0c3500 con 0x7f219c14fc80 2020-11-20T12:15:47.180 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f21a172a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0c37a0 con 0x7f219c066bc0 2020-11-20T12:15:47.180 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f21a172a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f219c0c3a40 con 0x7f219c0a2de0 2020-11-20T12:15:47.180 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f219b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f219c14fc80 0x7f219c14d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:47.180 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f219b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f219c14fc80 0x7f219c14d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42464/0 (socket says 172.21.15.116:42464) 2020-11-20T12:15:47.180 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f219b7fe700 1 -- 172.21.15.116:0/1037549935 learned_addr learned my addr 172.21.15.116:0/1037549935 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:47.181 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f219affd700 1 --2- 172.21.15.116:0/1037549935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f219c066bc0 0x7f219c062d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:47.181 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f219a7fc700 1 --2- 172.21.15.116:0/1037549935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f219c0a2de0 0x7f219c0c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:47.181 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f219b7fe700 1 -- 172.21.15.116:0/1037549935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f219c0a2de0 msgr2=0x7f219c0c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:47.181 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f219b7fe700 1 --2- 172.21.15.116:0/1037549935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f219c0a2de0 0x7f219c0c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:47.181 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f219b7fe700 1 -- 172.21.15.116:0/1037549935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f219c066bc0 msgr2=0x7f219c062d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:47.182 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f219b7fe700 1 --2- 172.21.15.116:0/1037549935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f219c066bc0 0x7f219c062d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:47.182 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f219b7fe700 1 -- 172.21.15.116:0/1037549935 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f219c0c3ce0 con 0x7f219c14fc80 2020-11-20T12:15:47.182 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f219b7fe700 1 --2- 172.21.15.116:0/1037549935 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f219c14fc80 0x7f219c14d040 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f219400ca60 tx=0x7f2194007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:47.183 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f2182ffd700 1 -- 172.21.15.116:0/1037549935 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219400a830 con 0x7f219c14fc80 2020-11-20T12:15:47.183 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f2182ffd700 1 -- 172.21.15.116:0/1037549935 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f219400a990 con 0x7f219c14fc80 2020-11-20T12:15:47.184 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f21a172a700 1 -- 172.21.15.116:0/1037549935 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f219c08ead0 con 0x7f219c14fc80 2020-11-20T12:15:47.184 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.176+0000 7f21a172a700 1 -- 172.21.15.116:0/1037549935 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f219c063200 con 0x7f219c14fc80 2020-11-20T12:15:47.184 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.180+0000 7f2182ffd700 1 -- 172.21.15.116:0/1037549935 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219400ad10 con 0x7f219c14fc80 2020-11-20T12:15:47.185 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.180+0000 7f2182ffd700 1 -- 172.21.15.116:0/1037549935 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 2) v1 ==== 33272+0+0 (secure 0 0 0) 0x7f2194019030 con 0x7f219c14fc80 2020-11-20T12:15:47.185 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.180+0000 7f21a172a700 1 -- 172.21.15.116:0/1037549935 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f219c044f60 con 0x7f219c14fc80 2020-11-20T12:15:47.186 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.180+0000 7f2182ffd700 1 -- 172.21.15.116:0/1037549935 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f219403b5a0 con 0x7f219c14fc80 2020-11-20T12:15:47.186 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.184+0000 7f2182ffd700 1 -- 172.21.15.116:0/1037549935 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71786 (secure 0 0 0) 0x7f219400fc70 con 0x7f219c14fc80 2020-11-20T12:15:47.258 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.256+0000 7f21a172a700 1 -- 172.21.15.116:0/1037549935 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f219c0c0760 con 0x7f219c14fc80 2020-11-20T12:15:47.259 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.256+0000 7f2182ffd700 1 -- 172.21.15.116:0/1037549935 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7001 (secure 0 0 0) 0x7f219400f260 con 0x7f219c14fc80 2020-11-20T12:15:47.260 INFO:teuthology.misc.health.smithi116.stdout: 2020-11-20T12:15:47.260 INFO:teuthology.misc.health.smithi116.stdout:{"epoch":10,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:15:46.123785+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:15:47.261 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.260+0000 7f21a172a700 1 -- 172.21.15.116:0/1037549935 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f219c14fc80 msgr2=0x7f219c14d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:47.262 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.260+0000 7f21a172a700 1 --2- 172.21.15.116:0/1037549935 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f219c14fc80 0x7f219c14d040 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f219400ca60 tx=0x7f2194007f40).stop 2020-11-20T12:15:47.262 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.260+0000 7f21a172a700 1 -- 172.21.15.116:0/1037549935 shutdown_connections 2020-11-20T12:15:47.262 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.260+0000 7f21a172a700 1 --2- 172.21.15.116:0/1037549935 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f219c14fc80 0x7f219c14d040 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:47.262 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.260+0000 7f21a172a700 1 --2- 172.21.15.116:0/1037549935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f219c066bc0 0x7f219c062d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:47.263 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.260+0000 7f21a172a700 1 --2- 172.21.15.116:0/1037549935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f219c0a2de0 0x7f219c0c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:47.263 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.260+0000 7f21a172a700 1 -- 172.21.15.116:0/1037549935 >> 172.21.15.116:0/1037549935 conn(0x7f219c060be0 msgr2=0x7f219c159360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:47.264 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.260+0000 7f21a172a700 1 -- 172.21.15.116:0/1037549935 shutdown_connections 2020-11-20T12:15:47.264 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:47.260+0000 7f21a172a700 1 -- 172.21.15.116:0/1037549935 wait complete. 2020-11-20T12:15:47.283 DEBUG:teuthology.misc:0 of 8 OSDs are up 2020-11-20T12:15:48.280 INFO:tasks.ceph.osd.3.smithi116.stderr:2020-11-20T12:15:48.280+0000 7f4482602d80 -1 osd.3 0 log_to_monitors {default=true} 2020-11-20T12:15:48.302 INFO:tasks.ceph.osd.2.smithi116.stderr:2020-11-20T12:15:48.300+0000 7fadba627d80 -1 osd.2 0 log_to_monitors {default=true} 2020-11-20T12:15:48.320 INFO:tasks.ceph.osd.0.smithi116.stderr:2020-11-20T12:15:48.316+0000 7fe56829dd80 -1 osd.0 0 log_to_monitors {default=true} 2020-11-20T12:15:48.324 INFO:tasks.ceph.osd.4.smithi173.stderr:2020-11-20T12:15:48.322+0000 7f5a4d1f6d80 -1 osd.4 0 log_to_monitors {default=true} 2020-11-20T12:15:48.333 INFO:tasks.ceph.osd.1.smithi116.stderr:2020-11-20T12:15:48.332+0000 7f96134e1d80 -1 osd.1 0 log_to_monitors {default=true} 2020-11-20T12:15:48.341 INFO:tasks.ceph.osd.5.smithi173.stderr:2020-11-20T12:15:48.342+0000 7f1a7c046d80 -1 osd.5 0 log_to_monitors {default=true} 2020-11-20T12:15:48.380 INFO:tasks.ceph.osd.7.smithi173.stderr:2020-11-20T12:15:48.378+0000 7f2cf9706d80 -1 osd.7 0 log_to_monitors {default=true} 2020-11-20T12:15:48.387 INFO:tasks.ceph.osd.6.smithi173.stderr:2020-11-20T12:15:48.386+0000 7f1df00bad80 -1 osd.6 0 log_to_monitors {default=true} 2020-11-20T12:15:49.562 INFO:tasks.ceph.osd.1.smithi116.stderr:2020-11-20T12:15:49.560+0000 7f960a9ab700 -1 osd.1 0 waiting for initial osdmap 2020-11-20T12:15:49.564 INFO:tasks.ceph.osd.0.smithi116.stderr:2020-11-20T12:15:49.564+0000 7fe55f767700 -1 osd.0 0 waiting for initial osdmap 2020-11-20T12:15:49.565 INFO:tasks.ceph.osd.5.smithi173.stderr:2020-11-20T12:15:49.562+0000 7f1a73510700 -1 osd.5 0 waiting for initial osdmap 2020-11-20T12:15:49.566 INFO:tasks.ceph.osd.7.smithi173.stderr:2020-11-20T12:15:49.562+0000 7f2cf0bd0700 -1 osd.7 0 waiting for initial osdmap 2020-11-20T12:15:49.566 INFO:tasks.ceph.osd.2.smithi116.stderr:2020-11-20T12:15:49.564+0000 7fadb1af1700 -1 osd.2 0 waiting for initial osdmap 2020-11-20T12:15:49.566 INFO:tasks.ceph.osd.3.smithi116.stderr:2020-11-20T12:15:49.564+0000 7f4479acc700 -1 osd.3 0 waiting for initial osdmap 2020-11-20T12:15:49.568 INFO:tasks.ceph.osd.4.smithi173.stderr:2020-11-20T12:15:49.566+0000 7f5a446c0700 -1 osd.4 0 waiting for initial osdmap 2020-11-20T12:15:49.570 INFO:tasks.ceph.osd.6.smithi173.stderr:2020-11-20T12:15:49.570+0000 7f1de7584700 -1 osd.6 0 waiting for initial osdmap 2020-11-20T12:15:49.576 INFO:tasks.ceph.osd.0.smithi116.stderr:2020-11-20T12:15:49.572+0000 7fe55befb700 -1 osd.0 13 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2020-11-20T12:15:49.577 INFO:tasks.ceph.osd.5.smithi173.stderr:2020-11-20T12:15:49.574+0000 7f1a6fca4700 -1 osd.5 13 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2020-11-20T12:15:49.577 INFO:tasks.ceph.osd.7.smithi173.stderr:2020-11-20T12:15:49.574+0000 7f2ced364700 -1 osd.7 13 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2020-11-20T12:15:49.578 INFO:tasks.ceph.osd.1.smithi116.stderr:2020-11-20T12:15:49.576+0000 7f960713f700 -1 osd.1 13 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2020-11-20T12:15:49.581 INFO:tasks.ceph.osd.2.smithi116.stderr:2020-11-20T12:15:49.580+0000 7fadae285700 -1 osd.2 13 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2020-11-20T12:15:49.582 INFO:tasks.ceph.osd.3.smithi116.stderr:2020-11-20T12:15:49.580+0000 7f4476260700 -1 osd.3 13 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2020-11-20T12:15:49.582 INFO:tasks.ceph.osd.6.smithi173.stderr:2020-11-20T12:15:49.582+0000 7f1de3d18700 -1 osd.6 13 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2020-11-20T12:15:49.583 INFO:tasks.ceph.osd.4.smithi173.stderr:2020-11-20T12:15:49.582+0000 7f5a40e54700 -1 osd.4 13 set_numa_affinity unable to identify public interface 'enp3s0f1' numa node: (0) Success 2020-11-20T12:15:54.090 INFO:teuthology.orchestra.run.smithi116:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2020-11-20T12:15:54.244 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.240+0000 7fbdf0d11700 1 Processor -- start 2020-11-20T12:15:54.549 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.240+0000 7fbdf0d11700 1 -- start start 2020-11-20T12:15:54.556 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdf0d11700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0aa790 0x7fbdec152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.557 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdf0d11700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdec152a30 con 0x7fbdec13aee0 2020-11-20T12:15:54.557 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdf0d11700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdec0aac10 con 0x7fbdec0612f0 2020-11-20T12:15:54.558 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdf0d11700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0adce0 con 0x7fbdec0aa790 2020-11-20T12:15:54.558 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdeb7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbdec13aee0 0x7fbdec0ad5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52112/0 (socket says 172.21.15.116:52112) 2020-11-20T12:15:54.559 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdeb7fe700 1 -- 172.21.15.116:0/12464283 learned_addr learned my addr 172.21.15.116:0/12464283 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:15:54.559 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdeaffd700 1 --2- 172.21.15.116:0/12464283 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0aa790 0x7fbdec152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.559 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450544343 0 0) 0x7fbdec152a30 con 0x7fbdec13aee0 2020-11-20T12:15:54.560 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdd8003240 con 0x7fbdec13aee0 2020-11-20T12:15:54.560 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1141892764 0 0) 0x7fbdd8003240 con 0x7fbdec13aee0 2020-11-20T12:15:54.561 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0aa790 msgr2=0x7fbdec152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:54.561 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 --2- 172.21.15.116:0/12464283 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0aa790 0x7fbdec152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:54.561 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 >> v1:172.21.15.173:6789/0 conn(0x7fbdec0612f0 legacy=0x7fbdec0a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.562 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec0aaeb0 con 0x7fbdec13aee0 2020-11-20T12:15:54.562 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fbddc0014e0 con 0x7fbdec13aee0 2020-11-20T12:15:54.563 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 >> v1:172.21.15.116:6789/0 conn(0x7fbdec13aee0 legacy=0x7fbdec0ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.563 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 --2- 172.21.15.116:0/12464283 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0aa790 0x7fbdd8003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.563 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 --2- 172.21.15.116:0/12464283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdd8003c70 0x7fbdd80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.563 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 --2- 172.21.15.116:0/12464283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdd80049d0 0x7fbdd8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.564 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0adce0 con 0x7fbdd80049d0 2020-11-20T12:15:54.564 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd8004200 con 0x7fbdd8003c70 2020-11-20T12:15:54.564 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdd800b730 con 0x7fbdec0aa790 2020-11-20T12:15:54.564 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdebfff700 1 --2- 172.21.15.116:0/12464283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdd80049d0 0x7fbdd8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.564 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdeaffd700 1 --2- 172.21.15.116:0/12464283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdd8003c70 0x7fbdd80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.565 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdebfff700 1 -- 172.21.15.116:0/12464283 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0aa790 msgr2=0x7fbdd8003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:54.565 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdebfff700 1 --2- 172.21.15.116:0/12464283 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0aa790 0x7fbdd8003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:15:54.565 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdebfff700 1 -- 172.21.15.116:0/12464283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdd8003c70 msgr2=0x7fbdd80044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:54.565 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdebfff700 1 --2- 172.21.15.116:0/12464283 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdd8003c70 0x7fbdd80044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:54.566 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdebfff700 1 -- 172.21.15.116:0/12464283 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdd8003240 con 0x7fbdd80049d0 2020-11-20T12:15:54.566 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.244+0000 7fbdebfff700 1 --2- 172.21.15.116:0/12464283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdd80049d0 0x7fbdd8004e50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbde4001e70 tx=0x7fbde4007f10).ready entity=mon.0 client_cookie=6ae27a0649ea0d9d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:54.567 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde4001540 con 0x7fbdd80049d0 2020-11-20T12:15:54.567 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbde4003030 con 0x7fbdd80049d0 2020-11-20T12:15:54.568 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdea7fc700 1 -- 172.21.15.116:0/12464283 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbde400cd90 con 0x7fbdd80049d0 2020-11-20T12:15:54.568 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdf0d11700 1 -- 172.21.15.116:0/12464283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdd80049d0 msgr2=0x7fbdd8004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.568 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdf0d11700 1 --2- 172.21.15.116:0/12464283 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdd80049d0 0x7fbdd8004e50 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fbde4001e70 tx=0x7fbde4007f10).stop 2020-11-20T12:15:54.568 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdebfff700 1 -- 172.21.15.116:0/12464283 reap_dead start 2020-11-20T12:15:54.569 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdf0d11700 1 -- 172.21.15.116:0/12464283 shutdown_connections 2020-11-20T12:15:54.569 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdf0d11700 1 -- 172.21.15.116:0/12464283 >> 172.21.15.116:0/12464283 conn(0x7fbdec0678a0 msgr2=0x7fbdec067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:54.569 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdf0d11700 1 -- 172.21.15.116:0/12464283 shutdown_connections 2020-11-20T12:15:54.569 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdf0d11700 1 -- 172.21.15.116:0/12464283 wait complete. 2020-11-20T12:15:54.569 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdf0d11700 1 Processor -- start 2020-11-20T12:15:54.570 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.248+0000 7fbdf0d11700 1 -- start start 2020-11-20T12:15:54.570 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdf0d11700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdec1606c0 0x7fbdec160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.570 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdf0d11700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0bfab0 0x7fbdec0bff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.570 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdf0d11700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdec0c1990 0x7fbdec0c1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.571 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdf0d11700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0c2740 con 0x7fbdec1606c0 2020-11-20T12:15:54.571 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdf0d11700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec0c29e0 con 0x7fbdec0c1990 2020-11-20T12:15:54.571 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdf0d11700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec1560d0 con 0x7fbdec0bfab0 2020-11-20T12:15:54.571 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdec1606c0 0x7fbdec160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.571 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdeb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdec1606c0 0x7fbdec160aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42692/0 (socket says 172.21.15.116:42692) 2020-11-20T12:15:54.572 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdeb7fe700 1 -- 172.21.15.116:0/1705356426 learned_addr learned my addr 172.21.15.116:0/1705356426 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:54.572 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdeaffd700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0bfab0 0x7fbdec0bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.572 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdebfff700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdec0c1990 0x7fbdec0c1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.572 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdeb7fe700 1 -- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0bfab0 msgr2=0x7fbdec0bff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.572 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdeb7fe700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0bfab0 0x7fbdec0bff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:54.573 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdeb7fe700 1 -- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdec0c1990 msgr2=0x7fbdec0c1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.574 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdeb7fe700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdec0c1990 0x7fbdec0c1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:54.574 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdeb7fe700 1 -- 172.21.15.116:0/1705356426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec0bd3a0 con 0x7fbdec1606c0 2020-11-20T12:15:54.575 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdeb7fe700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdec1606c0 0x7fbdec160aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fbddc00ca40 tx=0x7fbddc001b10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:54.575 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdcb7fe700 1 -- 172.21.15.116:0/1705356426 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc007170 con 0x7fbdec1606c0 2020-11-20T12:15:54.575 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdcb7fe700 1 -- 172.21.15.116:0/1705356426 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc0072f0 con 0x7fbdec1606c0 2020-11-20T12:15:54.576 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdf0d11700 1 -- 172.21.15.116:0/1705356426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdec0c22e0 con 0x7fbdec1606c0 2020-11-20T12:15:54.576 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdcb7fe700 1 -- 172.21.15.116:0/1705356426 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc00aa30 con 0x7fbdec1606c0 2020-11-20T12:15:54.577 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.252+0000 7fbdf0d11700 1 -- 172.21.15.116:0/1705356426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdec164700 con 0x7fbdec1606c0 2020-11-20T12:15:54.577 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.256+0000 7fbdf0d11700 1 -- 172.21.15.116:0/1705356426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdec044f60 con 0x7fbdec1606c0 2020-11-20T12:15:54.577 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.256+0000 7fbdcb7fe700 1 -- 172.21.15.116:0/1705356426 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fbddc010070 con 0x7fbdec1606c0 2020-11-20T12:15:54.578 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.256+0000 7fbdcb7fe700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbdc40557c0 0x7fbdc4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.578 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.256+0000 7fbdcb7fe700 1 -- 172.21.15.116:0/1705356426 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7fbddc06dac0 con 0x7fbdec1606c0 2020-11-20T12:15:54.578 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.256+0000 7fbdeaffd700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbdc40557c0 0x7fbdc4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.579 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.256+0000 7fbdeaffd700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbdc40557c0 0x7fbdc4057c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fbdec157e30 tx=0x7fbde0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:54.579 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.260+0000 7fbdcb7fe700 1 -- 172.21.15.116:0/1705356426 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbddc043df0 con 0x7fbdec1606c0 2020-11-20T12:15:54.580 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.420+0000 7fbdf0d11700 1 -- 172.21.15.116:0/1705356426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbdec062260 con 0x7fbdec1606c0 2020-11-20T12:15:54.580 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.420+0000 7fbdcb7fe700 1 -- 172.21.15.116:0/1705356426 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12200 (secure 0 0 0) 0x7fbddc019030 con 0x7fbdec1606c0 2020-11-20T12:15:54.580 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 -- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbdc40557c0 msgr2=0x7fbdc4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.581 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbdc40557c0 0x7fbdc4057c80 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fbdec157e30 tx=0x7fbde0008040).stop 2020-11-20T12:15:54.581 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 -- 172.21.15.116:0/1705356426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdec1606c0 msgr2=0x7fbdec160aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.582 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdec1606c0 0x7fbdec160aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7fbddc00ca40 tx=0x7fbddc001b10).stop 2020-11-20T12:15:54.582 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 -- 172.21.15.116:0/1705356426 shutdown_connections 2020-11-20T12:15:54.583 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbdc40557c0 0x7fbdc4057c80 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:54.583 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbdec0c1990 0x7fbdec0c1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:54.583 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbdec1606c0 0x7fbdec160aa0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:54.584 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 --2- 172.21.15.116:0/1705356426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbdec0bfab0 0x7fbdec0bff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:54.584 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 -- 172.21.15.116:0/1705356426 >> 172.21.15.116:0/1705356426 conn(0x7fbdec0678a0 msgr2=0x7fbdec1543e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:54.585 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 -- 172.21.15.116:0/1705356426 shutdown_connections 2020-11-20T12:15:54.585 INFO:teuthology.misc.health.smithi116.stderr:2020-11-20T12:15:54.424+0000 7fbdf0d11700 1 -- 172.21.15.116:0/1705356426 wait complete. 2020-11-20T12:15:54.586 INFO:teuthology.misc.health.smithi116.stdout: 2020-11-20T12:15:54.586 INFO:teuthology.misc.health.smithi116.stdout:{"epoch":16,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:15:52.569914+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:15:54.587 DEBUG:teuthology.misc:8 of 8 OSDs are up 2020-11-20T12:15:54.588 INFO:tasks.ceph:Creating RBD pool 2020-11-20T12:15:54.588 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd pool create rbd 8 2020-11-20T12:15:54.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.760+0000 7f4c5d83b700 1 Processor -- start 2020-11-20T12:15:54.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.760+0000 7f4c5d83b700 1 -- start start 2020-11-20T12:15:54.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.764+0000 7f4c5d83b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4c580611a0 0x7f4c58061630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.764+0000 7f4c5d83b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c580a2490 con 0x7f4c580c0c00 2020-11-20T12:15:54.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.764+0000 7f4c5d83b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c58060280 con 0x7f4c580c0fe0 2020-11-20T12:15:54.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.764+0000 7f4c5d83b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c58060520 con 0x7f4c580611a0 2020-11-20T12:15:54.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.764+0000 7f4c56ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4c580c0c00 0x7f4c580a1d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52132/0 (socket says 172.21.15.116:52132) 2020-11-20T12:15:54.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.764+0000 7f4c56ffd700 1 -- 172.21.15.116:0/3992177156 learned_addr learned my addr 172.21.15.116:0/3992177156 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:15:54.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.764+0000 7f4c577fe700 1 --2- 172.21.15.116:0/3992177156 >> v2:172.21.15.173:3300/0 conn(0x7f4c580611a0 0x7f4c58061630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c55ffb700 1 -- 172.21.15.116:0/3992177156 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479986973 0 0) 0x7f4c580a2490 con 0x7f4c580c0c00 2020-11-20T12:15:54.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c55ffb700 1 -- 172.21.15.116:0/3992177156 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c44003240 con 0x7f4c580c0c00 2020-11-20T12:15:54.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c577fe700 1 -- 172.21.15.116:0/3992177156 >> v1:172.21.15.173:6789/0 conn(0x7f4c580c0fe0 legacy=0x7f4c580c2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c577fe700 1 -- 172.21.15.116:0/3992177156 >> v1:172.21.15.116:6789/0 conn(0x7f4c580c0c00 legacy=0x7f4c580a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c577fe700 1 -- 172.21.15.116:0/3992177156 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c580607c0 con 0x7f4c580611a0 2020-11-20T12:15:54.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c577fe700 1 --2- 172.21.15.116:0/3992177156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c580611a0 0x7f4c58061630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4c50009c90 tx=0x7f4c500091d0).ready entity=mon.1 client_cookie=ebeb4cd69a59bda2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:54.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c55ffb700 1 -- 172.21.15.116:0/3992177156 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000bb00 con 0x7f4c580611a0 2020-11-20T12:15:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c55ffb700 1 -- 172.21.15.116:0/3992177156 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5000bc60 con 0x7f4c580611a0 2020-11-20T12:15:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c55ffb700 1 -- 172.21.15.116:0/3992177156 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c50011400 con 0x7f4c580611a0 2020-11-20T12:15:54.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c5d83b700 1 -- 172.21.15.116:0/3992177156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c580611a0 msgr2=0x7f4c58061630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c5d83b700 1 --2- 172.21.15.116:0/3992177156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c580611a0 0x7f4c58061630 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4c50009c90 tx=0x7f4c500091d0).stop 2020-11-20T12:15:54.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c5d83b700 1 -- 172.21.15.116:0/3992177156 shutdown_connections 2020-11-20T12:15:54.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c5d83b700 1 --2- 172.21.15.116:0/3992177156 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c580611a0 0x7f4c58061630 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:54.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c5d83b700 1 -- 172.21.15.116:0/3992177156 >> 172.21.15.116:0/3992177156 conn(0x7f4c580c14a0 msgr2=0x7f4c580c18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:54.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c5d83b700 1 -- 172.21.15.116:0/3992177156 shutdown_connections 2020-11-20T12:15:54.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.768+0000 7f4c5d83b700 1 -- 172.21.15.116:0/3992177156 wait complete. 2020-11-20T12:15:54.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.772+0000 7f4c5d83b700 1 Processor -- start 2020-11-20T12:15:54.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.772+0000 7f4c5d83b700 1 -- start start 2020-11-20T12:15:54.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.772+0000 7f4c5d83b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c580611a0 0x7f4c580a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.772+0000 7f4c5d83b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c580c0c00 0x7f4c581618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.772+0000 7f4c5d83b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c580c0fe0 0x7f4c58131340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.772+0000 7f4c5d83b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c58161de0 con 0x7f4c580c0c00 2020-11-20T12:15:54.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.772+0000 7f4c5d83b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5814db00 con 0x7f4c580c0fe0 2020-11-20T12:15:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.772+0000 7f4c5d83b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c58164830 con 0x7f4c580611a0 2020-11-20T12:15:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c567fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c580c0c00 0x7f4c581618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c567fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c580c0c00 0x7f4c581618f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42704/0 (socket says 172.21.15.116:42704) 2020-11-20T12:15:54.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c567fc700 1 -- 172.21.15.116:0/1471884205 learned_addr learned my addr 172.21.15.116:0/1471884205 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c577fe700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c580c0fe0 0x7f4c58131340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c56ffd700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c580611a0 0x7f4c580a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c567fc700 1 -- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c580611a0 msgr2=0x7f4c580a3c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c567fc700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c580611a0 0x7f4c580a3c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:54.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c567fc700 1 -- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c580c0fe0 msgr2=0x7f4c58131340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:54.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c567fc700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c580c0fe0 0x7f4c58131340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:54.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c567fc700 1 -- 172.21.15.116:0/1471884205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c58164ad0 con 0x7f4c580c0c00 2020-11-20T12:15:54.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c567fc700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c580c0c00 0x7f4c581618f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f4c4c00c6e0 tx=0x7f4c4c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:54.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c3effd700 1 -- 172.21.15.116:0/1471884205 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c007020 con 0x7f4c580c0c00 2020-11-20T12:15:54.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c3effd700 1 -- 172.21.15.116:0/1471884205 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c4c007180 con 0x7f4c580c0c00 2020-11-20T12:15:54.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c5d83b700 1 -- 172.21.15.116:0/1471884205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c58164d70 con 0x7f4c580c0c00 2020-11-20T12:15:54.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c3effd700 1 -- 172.21.15.116:0/1471884205 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4c00e560 con 0x7f4c580c0c00 2020-11-20T12:15:54.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c5d83b700 1 -- 172.21.15.116:0/1471884205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c58131830 con 0x7f4c580c0c00 2020-11-20T12:15:54.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.776+0000 7f4c5d83b700 1 -- 172.21.15.116:0/1471884205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c5814e110 con 0x7f4c580c0c00 2020-11-20T12:15:54.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.780+0000 7f4c3effd700 1 -- 172.21.15.116:0/1471884205 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4c4c010070 con 0x7f4c580c0c00 2020-11-20T12:15:54.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.780+0000 7f4c3effd700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c340557c0 0x7f4c34057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:54.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.780+0000 7f4c3effd700 1 -- 172.21.15.116:0/1471884205 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4900+0+0 (secure 0 0 0) 0x7f4c4c06d570 con 0x7f4c580c0c00 2020-11-20T12:15:54.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.780+0000 7f4c56ffd700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c340557c0 0x7f4c34057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:54.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.780+0000 7f4c56ffd700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c340557c0 0x7f4c34057c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f4c480055a0 tx=0x7f4c48009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:54.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.784+0000 7f4c3effd700 1 -- 172.21.15.116:0/1471884205 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4c4c042fd0 con 0x7f4c580c0c00 2020-11-20T12:15:54.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:54.964+0000 7f4c5d83b700 1 -- 172.21.15.116:0/1471884205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f4c5805ca30 con 0x7f4c580c0c00 2020-11-20T12:15:55.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.980+0000 7f4c3effd700 1 -- 172.21.15.116:0/1471884205 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f4c5814e110 con 0x7f4c580c0c00 2020-11-20T12:15:55.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 -- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c340557c0 msgr2=0x7f4c34057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:55.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c340557c0 0x7f4c34057c80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f4c480055a0 tx=0x7f4c48009040).stop 2020-11-20T12:15:55.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 -- 172.21.15.116:0/1471884205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c580c0c00 msgr2=0x7f4c581618f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:55.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c580c0c00 0x7f4c581618f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f4c4c00c6e0 tx=0x7f4c4c007c50).stop 2020-11-20T12:15:55.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 -- 172.21.15.116:0/1471884205 shutdown_connections 2020-11-20T12:15:55.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c340557c0 0x7f4c34057c80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:55.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c580c0fe0 0x7f4c58131340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:55.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c580611a0 0x7f4c580a3c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:55.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 --2- 172.21.15.116:0/1471884205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c580c0c00 0x7f4c581618f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:55.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 -- 172.21.15.116:0/1471884205 >> 172.21.15.116:0/1471884205 conn(0x7f4c580c14a0 msgr2=0x7f4c5813a760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:55.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 -- 172.21.15.116:0/1471884205 shutdown_connections 2020-11-20T12:15:55.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:55.984+0000 7f4c5d83b700 1 -- 172.21.15.116:0/1471884205 wait complete. 2020-11-20T12:15:55.990 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'rbd' created 2020-11-20T12:15:56.012 INFO:teuthology.orchestra.run.smithi116:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2020-11-20T12:15:56.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.184+0000 7ffbb57b6700 1 Processor -- start 2020-11-20T12:15:56.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.184+0000 7ffbb57b6700 1 -- start start 2020-11-20T12:15:56.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.188+0000 7ffbb57b6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffbb0152020 0x7ffbb0152480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.188+0000 7ffbb57b6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb01591c0 0x7ffbb015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.188+0000 7ffbb57b6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbb00a85a0 con 0x7ffbb0151c40 2020-11-20T12:15:56.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.188+0000 7ffbb57b6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbb00c5e90 con 0x7ffbb0152020 2020-11-20T12:15:56.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.188+0000 7ffbb57b6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00a96f0 con 0x7ffbb01591c0 2020-11-20T12:15:56.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.188+0000 7ffbaeffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ffbb0151c40 0x7ffbb00c5780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52146/0 (socket says 172.21.15.116:52146) 2020-11-20T12:15:56.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.188+0000 7ffbaeffd700 1 -- 172.21.15.116:0/1903147685 learned_addr learned my addr 172.21.15.116:0/1903147685 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:15:56.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.188+0000 7ffbae7fc700 1 --2- 172.21.15.116:0/1903147685 >> v2:172.21.15.116:3300/0 conn(0x7ffbb0152020 0x7ffbb0152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:56.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.188+0000 7ffbaf7fe700 1 --2- 172.21.15.116:0/1903147685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb01591c0 0x7ffbb015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:56.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbadffb700 1 -- 172.21.15.116:0/1903147685 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3843766612 0 0) 0x7ffbb00a85a0 con 0x7ffbb0151c40 2020-11-20T12:15:56.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbadffb700 1 -- 172.21.15.116:0/1903147685 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb9c003240 con 0x7ffbb0151c40 2020-11-20T12:15:56.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbae7fc700 1 -- 172.21.15.116:0/1903147685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb01591c0 msgr2=0x7ffbb015b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:56.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbae7fc700 1 --2- 172.21.15.116:0/1903147685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb01591c0 0x7ffbb015b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:56.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbae7fc700 1 -- 172.21.15.116:0/1903147685 >> v1:172.21.15.116:6789/0 conn(0x7ffbb0151c40 legacy=0x7ffbb00c5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:56.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbae7fc700 1 -- 172.21.15.116:0/1903147685 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb00a87d0 con 0x7ffbb0152020 2020-11-20T12:15:56.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbae7fc700 1 --2- 172.21.15.116:0/1903147685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbb0152020 0x7ffbb0152480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ffba400b9d0 tx=0x7ffba4006df0).ready entity=mon.0 client_cookie=4bcb05aecc9902b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:56.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbadffb700 1 -- 172.21.15.116:0/1903147685 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4009800 con 0x7ffbb0152020 2020-11-20T12:15:56.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbadffb700 1 -- 172.21.15.116:0/1903147685 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba4009960 con 0x7ffbb0152020 2020-11-20T12:15:56.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbadffb700 1 -- 172.21.15.116:0/1903147685 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4009ce0 con 0x7ffbb0152020 2020-11-20T12:15:56.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbb57b6700 1 -- 172.21.15.116:0/1903147685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbb0152020 msgr2=0x7ffbb0152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:56.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbb57b6700 1 --2- 172.21.15.116:0/1903147685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbb0152020 0x7ffbb0152480 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7ffba400b9d0 tx=0x7ffba4006df0).stop 2020-11-20T12:15:56.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbb57b6700 1 -- 172.21.15.116:0/1903147685 shutdown_connections 2020-11-20T12:15:56.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbb57b6700 1 --2- 172.21.15.116:0/1903147685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb01591c0 0x7ffbb015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:56.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbb57b6700 1 --2- 172.21.15.116:0/1903147685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbb0152020 0x7ffbb0152480 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:56.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbb57b6700 1 -- 172.21.15.116:0/1903147685 >> 172.21.15.116:0/1903147685 conn(0x7ffbb0091910 msgr2=0x7ffbb0068080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:56.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbb57b6700 1 -- 172.21.15.116:0/1903147685 shutdown_connections 2020-11-20T12:15:56.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.192+0000 7ffbb57b6700 1 -- 172.21.15.116:0/1903147685 wait complete. 2020-11-20T12:15:56.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.196+0000 7ffbb57b6700 1 Processor -- start 2020-11-20T12:15:56.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.196+0000 7ffbb57b6700 1 -- start start 2020-11-20T12:15:56.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.196+0000 7ffbb57b6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbb0151c40 0x7ffbb00c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.196+0000 7ffbb57b6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbb01591c0 0x7ffbb00c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.196+0000 7ffbb57b6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb00c7160 0x7ffbb00c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.196+0000 7ffbb57b6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00ac720 con 0x7ffbb01591c0 2020-11-20T12:15:56.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.196+0000 7ffbb57b6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00ac9c0 con 0x7ffbb0151c40 2020-11-20T12:15:56.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.196+0000 7ffbb57b6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb00acc60 con 0x7ffbb00c7160 2020-11-20T12:15:56.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.196+0000 7ffbae7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbb01591c0 0x7ffbb00c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:56.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.196+0000 7ffbae7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbb01591c0 0x7ffbb00c6c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42720/0 (socket says 172.21.15.116:42720) 2020-11-20T12:15:56.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbae7fc700 1 -- 172.21.15.116:0/3049253598 learned_addr learned my addr 172.21.15.116:0/3049253598 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:56.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbaeffd700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbb0151c40 0x7ffbb00c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:56.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbaf7fe700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb00c7160 0x7ffbb00c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:56.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbae7fc700 1 -- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb00c7160 msgr2=0x7ffbb00c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:56.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbae7fc700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb00c7160 0x7ffbb00c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:56.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbae7fc700 1 -- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbb0151c40 msgr2=0x7ffbb00c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:56.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbae7fc700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbb0151c40 0x7ffbb00c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:56.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbae7fc700 1 -- 172.21.15.116:0/3049253598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb00acf00 con 0x7ffbb01591c0 2020-11-20T12:15:56.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbae7fc700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbb01591c0 0x7ffbb00c6c70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ffba400b150 tx=0x7ffba4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:56.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffb96ffd700 1 -- 172.21.15.116:0/3049253598 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4011c80 con 0x7ffbb01591c0 2020-11-20T12:15:56.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffb96ffd700 1 -- 172.21.15.116:0/3049253598 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba4011de0 con 0x7ffbb01591c0 2020-11-20T12:15:56.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffb96ffd700 1 -- 172.21.15.116:0/3049253598 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba401e670 con 0x7ffbb01591c0 2020-11-20T12:15:56.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbb57b6700 1 -- 172.21.15.116:0/3049253598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb00ad1a0 con 0x7ffbb01591c0 2020-11-20T12:15:56.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbb57b6700 1 -- 172.21.15.116:0/3049253598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb00ac1d0 con 0x7ffbb01591c0 2020-11-20T12:15:56.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.200+0000 7ffbb57b6700 1 -- 172.21.15.116:0/3049253598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb0044f60 con 0x7ffbb01591c0 2020-11-20T12:15:56.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.204+0000 7ffb96ffd700 1 -- 172.21.15.116:0/3049253598 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ffba401a040 con 0x7ffbb01591c0 2020-11-20T12:15:56.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.204+0000 7ffb96ffd700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffb8c0557c0 0x7ffb8c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:56.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.204+0000 7ffb96ffd700 1 -- 172.21.15.116:0/3049253598 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5242+0+0 (secure 0 0 0) 0x7ffba406f7b0 con 0x7ffbb01591c0 2020-11-20T12:15:56.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.204+0000 7ffbaeffd700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffb8c0557c0 0x7ffb8c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:56.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.204+0000 7ffbaeffd700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffb8c0557c0 0x7ffb8c057c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7ffba00055a0 tx=0x7ffba0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:56.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.212+0000 7ffb96ffd700 1 -- 172.21.15.116:0/3049253598 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ffba4045930 con 0x7ffbb01591c0 2020-11-20T12:15:56.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.372+0000 7ffbb57b6700 1 -- 172.21.15.116:0/3049253598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffbb005bdf0 con 0x7ffbb01591c0 2020-11-20T12:15:56.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.992+0000 7ffb96ffd700 1 -- 172.21.15.116:0/3049253598 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7ffba4023070 con 0x7ffbb01591c0 2020-11-20T12:15:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.996+0000 7ffbb57b6700 1 -- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffb8c0557c0 msgr2=0x7ffb8c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.996+0000 7ffbb57b6700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffb8c0557c0 0x7ffb8c057c80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7ffba00055a0 tx=0x7ffba0009040).stop 2020-11-20T12:15:56.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.996+0000 7ffbb57b6700 1 -- 172.21.15.116:0/3049253598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbb01591c0 msgr2=0x7ffbb00c6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.996+0000 7ffbb57b6700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbb01591c0 0x7ffbb00c6c70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ffba400b150 tx=0x7ffba4015000).stop 2020-11-20T12:15:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.996+0000 7ffbb57b6700 1 -- 172.21.15.116:0/3049253598 shutdown_connections 2020-11-20T12:15:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.996+0000 7ffbb57b6700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffb8c0557c0 0x7ffb8c057c80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.996+0000 7ffbb57b6700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffbb00c7160 0x7ffbb00c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.996+0000 7ffbb57b6700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffbb0151c40 0x7ffbb00c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.996+0000 7ffbb57b6700 1 --2- 172.21.15.116:0/3049253598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffbb01591c0 0x7ffbb00c6c70 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0x7ffba400b150 tx=0x7ffba4015000).stop 2020-11-20T12:15:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:56.996+0000 7ffbb57b6700 1 -- 172.21.15.116:0/3049253598 >> 172.21.15.116:0/3049253598 conn(0x7ffbb0091910 msgr2=0x7ffbb0152d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.000+0000 7ffbb57b6700 1 -- 172.21.15.116:0/3049253598 shutdown_connections 2020-11-20T12:15:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.000+0000 7ffbb57b6700 1 -- 172.21.15.116:0/3049253598 wait complete. 2020-11-20T12:15:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:enabled application 'rbd' on pool 'rbd' 2020-11-20T12:15:57.026 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2020-11-20T12:15:57.026 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T12:15:57.027 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T12:15:57.031 INFO:tasks.ceph.mds.a:Started 2020-11-20T12:15:57.031 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-20T12:15:57.032 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-20T12:15:57.035 INFO:tasks.ceph.mds.c:Started 2020-11-20T12:15:57.035 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T12:15:57.036 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T12:15:57.040 INFO:tasks.ceph.mds.b:Started 2020-11-20T12:15:57.041 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-20T12:15:57.041 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-20T12:15:57.045 INFO:tasks.ceph.mds.d:Started 2020-11-20T12:15:57.045 INFO:tasks.ceph:Setting up CephFS filesystem... 2020-11-20T12:15:57.046 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:15:57.132 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:15:57.133 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:15:57.147 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2020-11-20T12:15:57.148 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2020-11-20T12:15:57.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.220+0000 7f2ab9550700 1 Processor -- start 2020-11-20T12:15:57.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.220+0000 7f2ab9550700 1 -- start start 2020-11-20T12:15:57.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab9550700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2ab40a81a0 0x7f2ab40aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:57.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab9550700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2ab4151c30 0x7f2ab40ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:57.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab9550700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab41523b0 con 0x7f2ab40aa570 2020-11-20T12:15:57.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab9550700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab40ab930 con 0x7f2ab4151c30 2020-11-20T12:15:57.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab9550700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab4152650 con 0x7f2ab40a81a0 2020-11-20T12:15:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab37fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2ab4151c30 0x7f2ab40ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab37fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2ab4151c30 0x7f2ab40ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42756/0 (socket says 172.21.15.116:42756) 2020-11-20T12:15:57.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab37fe700 1 -- 172.21.15.116:0/430421394 learned_addr learned my addr 172.21.15.116:0/430421394 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab27fc700 1 --2- 172.21.15.116:0/430421394 >> v2:172.21.15.173:3300/0 conn(0x7f2ab40a81a0 0x7f2ab40aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab37fe700 1 -- 172.21.15.116:0/430421394 >> v2:172.21.15.173:3300/0 conn(0x7f2ab40a81a0 msgr2=0x7f2ab40aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab1ffb700 1 -- 172.21.15.116:0/430421394 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036266314 0 0) 0x7f2ab41523b0 con 0x7f2ab40aa570 2020-11-20T12:15:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab37fe700 1 --2- 172.21.15.116:0/430421394 >> v2:172.21.15.173:3300/0 conn(0x7f2ab40a81a0 0x7f2ab40aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab37fe700 1 -- 172.21.15.116:0/430421394 >> v1:172.21.15.173:6789/0 conn(0x7f2ab40aa570 legacy=0x7f2ab40ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab37fe700 1 -- 172.21.15.116:0/430421394 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab41528f0 con 0x7f2ab4151c30 2020-11-20T12:15:57.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab37fe700 1 --2- 172.21.15.116:0/430421394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab4151c30 0x7f2ab40ad980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2ab4155160 tx=0x7f2aac009290).ready entity=mon.0 client_cookie=f94e785bb849b075 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.224+0000 7f2ab1ffb700 1 -- 172.21.15.116:0/430421394 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac00bc50 con 0x7f2ab4151c30 2020-11-20T12:15:57.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.228+0000 7f2ab1ffb700 1 -- 172.21.15.116:0/430421394 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac00bdb0 con 0x7f2ab4151c30 2020-11-20T12:15:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.228+0000 7f2ab1ffb700 1 -- 172.21.15.116:0/430421394 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac011610 con 0x7f2ab4151c30 2020-11-20T12:15:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.228+0000 7f2ab9550700 1 -- 172.21.15.116:0/430421394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab4151c30 msgr2=0x7f2ab40ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.228+0000 7f2ab9550700 1 --2- 172.21.15.116:0/430421394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab4151c30 0x7f2ab40ad980 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2ab4155160 tx=0x7f2aac009290).stop 2020-11-20T12:15:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.228+0000 7f2ab9550700 1 -- 172.21.15.116:0/430421394 shutdown_connections 2020-11-20T12:15:57.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.228+0000 7f2ab9550700 1 --2- 172.21.15.116:0/430421394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab4151c30 0x7f2ab40ad980 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.228+0000 7f2ab9550700 1 --2- 172.21.15.116:0/430421394 >> v2:172.21.15.173:3300/0 conn(0x7f2ab40a81a0 0x7f2ab40aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.228+0000 7f2ab9550700 1 -- 172.21.15.116:0/430421394 >> 172.21.15.116:0/430421394 conn(0x7f2ab40678a0 msgr2=0x7f2ab4067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.228+0000 7f2ab9550700 1 -- 172.21.15.116:0/430421394 shutdown_connections 2020-11-20T12:15:57.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.228+0000 7f2ab9550700 1 -- 172.21.15.116:0/430421394 wait complete. 2020-11-20T12:15:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab9550700 1 Processor -- start 2020-11-20T12:15:57.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab9550700 1 -- start start 2020-11-20T12:15:57.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab9550700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40a81a0 0x7f2ab414b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab9550700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ab40aa570 0x7f2ab41508c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab9550700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab4151c30 0x7f2ab4150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab9550700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab40a07a0 con 0x7f2ab40a81a0 2020-11-20T12:15:57.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab9550700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab40a0a40 con 0x7f2ab4151c30 2020-11-20T12:15:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab9550700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab40a0ce0 con 0x7f2ab40aa570 2020-11-20T12:15:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40a81a0 0x7f2ab414b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40a81a0 0x7f2ab414b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42758/0 (socket says 172.21.15.116:42758) 2020-11-20T12:15:57.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab2ffd700 1 -- 172.21.15.116:0/580112978 learned_addr learned my addr 172.21.15.116:0/580112978 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:57.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab27fc700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ab40aa570 0x7f2ab41508c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:57.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab37fe700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab4151c30 0x7f2ab4150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:57.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab2ffd700 1 -- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ab40aa570 msgr2=0x7f2ab41508c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab2ffd700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ab40aa570 0x7f2ab41508c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab2ffd700 1 -- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab4151c30 msgr2=0x7f2ab4150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab2ffd700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab4151c30 0x7f2ab4150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab2ffd700 1 -- 172.21.15.116:0/580112978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab40a0f80 con 0x7f2ab40a81a0 2020-11-20T12:15:57.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab27fc700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ab40aa570 0x7f2ab41508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:15:57.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab37fe700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab4151c30 0x7f2ab4150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:15:57.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.232+0000 7f2ab2ffd700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40a81a0 0x7f2ab414b0b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2aa4007fd0 tx=0x7f2aa4007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:57.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.236+0000 7f2a9affd700 1 -- 172.21.15.116:0/580112978 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400e560 con 0x7f2ab40a81a0 2020-11-20T12:15:57.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.236+0000 7f2a9affd700 1 -- 172.21.15.116:0/580112978 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa400e6e0 con 0x7f2ab40a81a0 2020-11-20T12:15:57.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.236+0000 7f2a9affd700 1 -- 172.21.15.116:0/580112978 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400eaa0 con 0x7f2ab40a81a0 2020-11-20T12:15:57.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.236+0000 7f2ab9550700 1 -- 172.21.15.116:0/580112978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab40a0490 con 0x7f2ab40a81a0 2020-11-20T12:15:57.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.236+0000 7f2ab9550700 1 -- 172.21.15.116:0/580112978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab40a1420 con 0x7f2ab40a81a0 2020-11-20T12:15:57.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.236+0000 7f2ab9550700 1 -- 172.21.15.116:0/580112978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab40a16a0 con 0x7f2ab40a81a0 2020-11-20T12:15:57.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.236+0000 7f2a9affd700 1 -- 172.21.15.116:0/580112978 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2aa4010070 con 0x7f2ab40a81a0 2020-11-20T12:15:57.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.240+0000 7f2a9affd700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a900557c0 0x7f2a90057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:57.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.240+0000 7f2a9affd700 1 -- 172.21.15.116:0/580112978 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2aa406dc30 con 0x7f2ab40a81a0 2020-11-20T12:15:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.240+0000 7f2ab27fc700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a900557c0 0x7f2a90057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:57.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.240+0000 7f2ab27fc700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a900557c0 0x7f2a90057c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f2aa8006bd0 tx=0x7f2aa8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:57.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.244+0000 7f2a9affd700 1 -- 172.21.15.116:0/580112978 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2ab40a16a0 con 0x7f2ab40a81a0 2020-11-20T12:15:57.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.408+0000 7f2ab9550700 1 -- 172.21.15.116:0/580112978 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ab40a16a0 con 0x7f2ab40a81a0 2020-11-20T12:15:57.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2a9affd700 1 -- 172.21.15.116:0/580112978 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13733 (secure 0 0 0) 0x7f2ab40a16a0 con 0x7f2ab40a81a0 2020-11-20T12:15:57.413 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:15:57.413 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":18,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:15:56.980049+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:15:57.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 -- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a900557c0 msgr2=0x7f2a90057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a900557c0 0x7f2a90057c80 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f2aa8006bd0 tx=0x7f2aa8008040).stop 2020-11-20T12:15:57.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 -- 172.21.15.116:0/580112978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40a81a0 msgr2=0x7f2ab414b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40a81a0 0x7f2ab414b0b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f2aa4007fd0 tx=0x7f2aa4007ac0).stop 2020-11-20T12:15:57.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 -- 172.21.15.116:0/580112978 shutdown_connections 2020-11-20T12:15:57.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a900557c0 0x7f2a90057c80 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ab4151c30 0x7f2ab4150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ab40a81a0 0x7f2ab414b0b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 --2- 172.21.15.116:0/580112978 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ab40aa570 0x7f2ab41508c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 -- 172.21.15.116:0/580112978 >> 172.21.15.116:0/580112978 conn(0x7f2ab40678a0 msgr2=0x7f2ab414fbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:57.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 -- 172.21.15.116:0/580112978 shutdown_connections 2020-11-20T12:15:57.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.412+0000 7f2ab9550700 1 -- 172.21.15.116:0/580112978 wait complete. 2020-11-20T12:15:57.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T12:15:57.442 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:15:57.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24f9992700 1 Processor -- start 2020-11-20T12:15:57.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24f9992700 1 -- start start 2020-11-20T12:15:57.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24f9992700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f24f40673b0 0x7f24f4067860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:57.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24f9992700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f4067e10 con 0x7f24f414efa0 2020-11-20T12:15:57.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24f9992700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24f40680b0 con 0x7f24f40c3530 2020-11-20T12:15:57.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24f9992700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f4066130 con 0x7f24f40673b0 2020-11-20T12:15:57.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24f2ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f24f414efa0 0x7f24f4066870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52200/0 (socket says 172.21.15.116:52200) 2020-11-20T12:15:57.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24f2ffd700 1 -- 172.21.15.116:0/2223306469 learned_addr learned my addr 172.21.15.116:0/2223306469 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:15:57.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24ea7fc700 1 --2- 172.21.15.116:0/2223306469 >> v2:172.21.15.116:3300/0 conn(0x7f24f40673b0 0x7f24f4067860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:57.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24f27fc700 1 -- 172.21.15.116:0/2223306469 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1847713282 0 0) 0x7f24f4067e10 con 0x7f24f414efa0 2020-11-20T12:15:57.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24f27fc700 1 -- 172.21.15.116:0/2223306469 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24d4003240 con 0x7f24f414efa0 2020-11-20T12:15:57.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24ea7fc700 1 -- 172.21.15.116:0/2223306469 >> v1:172.21.15.173:6789/0 conn(0x7f24f40c3530 legacy=0x7f24f40a2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:57.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24ea7fc700 1 -- 172.21.15.116:0/2223306469 >> v1:172.21.15.116:6789/0 conn(0x7f24f414efa0 legacy=0x7f24f4066870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.620+0000 7f24ea7fc700 1 -- 172.21.15.116:0/2223306469 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f40663d0 con 0x7f24f40673b0 2020-11-20T12:15:57.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.624+0000 7f24ea7fc700 1 --2- 172.21.15.116:0/2223306469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24f40673b0 0x7f24f4067860 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f24dc00b9a0 tx=0x7f24dc006df0).ready entity=mon.0 client_cookie=b44530fc84142497 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:57.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.624+0000 7f24f27fc700 1 -- 172.21.15.116:0/2223306469 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc009770 con 0x7f24f40673b0 2020-11-20T12:15:57.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.624+0000 7f24f27fc700 1 -- 172.21.15.116:0/2223306469 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24dc0098d0 con 0x7f24f40673b0 2020-11-20T12:15:57.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.624+0000 7f24f27fc700 1 -- 172.21.15.116:0/2223306469 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24dc009c50 con 0x7f24f40673b0 2020-11-20T12:15:57.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.624+0000 7f24f9992700 1 -- 172.21.15.116:0/2223306469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24f40673b0 msgr2=0x7f24f4067860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.624+0000 7f24f9992700 1 --2- 172.21.15.116:0/2223306469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24f40673b0 0x7f24f4067860 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f24dc00b9a0 tx=0x7f24dc006df0).stop 2020-11-20T12:15:57.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.624+0000 7f24f9992700 1 -- 172.21.15.116:0/2223306469 shutdown_connections 2020-11-20T12:15:57.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.624+0000 7f24f9992700 1 --2- 172.21.15.116:0/2223306469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24f40673b0 0x7f24f4067860 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.624+0000 7f24f9992700 1 -- 172.21.15.116:0/2223306469 >> 172.21.15.116:0/2223306469 conn(0x7f24f4060c30 msgr2=0x7f24f40c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:57.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.624+0000 7f24f9992700 1 -- 172.21.15.116:0/2223306469 shutdown_connections 2020-11-20T12:15:57.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.624+0000 7f24f9992700 1 -- 172.21.15.116:0/2223306469 wait complete. 2020-11-20T12:15:57.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f9992700 1 Processor -- start 2020-11-20T12:15:57.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f9992700 1 -- start start 2020-11-20T12:15:57.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f9992700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24f40673b0 0x7f24f40a7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:57.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f9992700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24f40c3530 0x7f24f4090820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:57.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f9992700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24f414efa0 0x7f24f4160fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:57.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f9992700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f41645b0 con 0x7f24f40673b0 2020-11-20T12:15:57.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f9992700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40a8740 con 0x7f24f414efa0 2020-11-20T12:15:57.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f9992700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f40a89e0 con 0x7f24f40c3530 2020-11-20T12:15:57.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24f40673b0 0x7f24f40a7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:57.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24f40673b0 0x7f24f40a7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42772/0 (socket says 172.21.15.116:42772) 2020-11-20T12:15:57.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f2ffd700 1 -- 172.21.15.116:0/1246035923 learned_addr learned my addr 172.21.15.116:0/1246035923 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:57.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f37fe700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24f414efa0 0x7f24f4160fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:57.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24ea7fc700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24f40c3530 0x7f24f4090820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:57.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f2ffd700 1 -- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24f40c3530 msgr2=0x7f24f4090820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f2ffd700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24f40c3530 0x7f24f4090820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f2ffd700 1 -- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24f414efa0 msgr2=0x7f24f4160fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f2ffd700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24f414efa0 0x7f24f4160fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f2ffd700 1 -- 172.21.15.116:0/1246035923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f40a8c80 con 0x7f24f40673b0 2020-11-20T12:15:57.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f37fe700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24f414efa0 0x7f24f4160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:15:57.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24ea7fc700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24f40c3530 0x7f24f4090820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:15:57.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.628+0000 7f24f2ffd700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24f40673b0 0x7f24f40a7f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f24e400c6e0 tx=0x7f24e4007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:57.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.632+0000 7f24eb7fe700 1 -- 172.21.15.116:0/1246035923 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e4007320 con 0x7f24f40673b0 2020-11-20T12:15:57.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.632+0000 7f24eb7fe700 1 -- 172.21.15.116:0/1246035923 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e4007480 con 0x7f24f40673b0 2020-11-20T12:15:57.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.632+0000 7f24f9992700 1 -- 172.21.15.116:0/1246035923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f41647a0 con 0x7f24f40673b0 2020-11-20T12:15:57.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.632+0000 7f24eb7fe700 1 -- 172.21.15.116:0/1246035923 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e4005820 con 0x7f24f40673b0 2020-11-20T12:15:57.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.632+0000 7f24f9992700 1 -- 172.21.15.116:0/1246035923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f4090d10 con 0x7f24f40673b0 2020-11-20T12:15:57.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.632+0000 7f24f9992700 1 -- 172.21.15.116:0/1246035923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f40a8e10 con 0x7f24f40673b0 2020-11-20T12:15:57.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.632+0000 7f24eb7fe700 1 -- 172.21.15.116:0/1246035923 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f24e4010030 con 0x7f24f40673b0 2020-11-20T12:15:57.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.632+0000 7f24eb7fe700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f24d00557c0 0x7f24d0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:57.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.632+0000 7f24eb7fe700 1 -- 172.21.15.116:0/1246035923 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f24e406df60 con 0x7f24f40673b0 2020-11-20T12:15:57.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.636+0000 7f24ea7fc700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f24d00557c0 0x7f24d0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:57.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.636+0000 7f24ea7fc700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f24d00557c0 0x7f24d0057c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f24f4149e50 tx=0x7f24dc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:57.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.640+0000 7f24eb7fe700 1 -- 172.21.15.116:0/1246035923 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f24e4044130 con 0x7f24f40673b0 2020-11-20T12:15:57.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.800+0000 7f24f9992700 1 -- 172.21.15.116:0/1246035923 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f24f4044f60 con 0x7f24f40673b0 2020-11-20T12:15:57.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.800+0000 7f24eb7fe700 1 -- 172.21.15.116:0/1246035923 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7f24e4047750 con 0x7f24f40673b0 2020-11-20T12:15:57.802 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:15:57.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 -- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f24d00557c0 msgr2=0x7f24d0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f24d00557c0 0x7f24d0057c80 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f24f4149e50 tx=0x7f24dc015000).stop 2020-11-20T12:15:57.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 -- 172.21.15.116:0/1246035923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24f40673b0 msgr2=0x7f24f40a7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:57.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24f40673b0 0x7f24f40a7f50 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f24e400c6e0 tx=0x7f24e4007ac0).stop 2020-11-20T12:15:57.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 -- 172.21.15.116:0/1246035923 shutdown_connections 2020-11-20T12:15:57.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f24d00557c0 0x7f24d0057c80 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f24f414efa0 0x7f24f4160fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f24f40673b0 0x7f24f40a7f50 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 --2- 172.21.15.116:0/1246035923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24f40c3530 0x7f24f4090820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:57.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 -- 172.21.15.116:0/1246035923 >> 172.21.15.116:0/1246035923 conn(0x7f24f4060c30 msgr2=0x7f24f40a5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:57.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 -- 172.21.15.116:0/1246035923 shutdown_connections 2020-11-20T12:15:57.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:57.804+0000 7f24f9992700 1 -- 172.21.15.116:0/1246035923 wait complete. 2020-11-20T12:15:57.827 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:15:58.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.008+0000 7fabfc920700 1 Processor -- start 2020-11-20T12:15:58.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabfc920700 1 -- start start 2020-11-20T12:15:58.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabfc920700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf813bc90 0x7fabf80a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabfc920700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fabf80a6300 0x7fabf81561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabfc920700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf80a7ee0 con 0x7fabf8156690 2020-11-20T12:15:58.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabfc920700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf80a7cf0 con 0x7fabf80a6300 2020-11-20T12:15:58.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabfc920700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8156f10 con 0x7fabf813bc90 2020-11-20T12:15:58.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf7fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fabf8156690 0x7fabf8065ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52218/0 (socket says 172.21.15.116:52218) 2020-11-20T12:15:58.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf7fff700 1 -- 172.21.15.116:0/395079536 learned_addr learned my addr 172.21.15.116:0/395079536 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:15:58.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf77fe700 1 --2- 172.21.15.116:0/395079536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf813bc90 0x7fabf80a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf6ffd700 1 --2- 172.21.15.116:0/395079536 >> v2:172.21.15.173:3300/0 conn(0x7fabf80a6300 0x7fabf81561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf67fc700 1 -- 172.21.15.116:0/395079536 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532994580 0 0) 0x7fabf80a7ee0 con 0x7fabf8156690 2020-11-20T12:15:58.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf67fc700 1 -- 172.21.15.116:0/395079536 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabe4003240 con 0x7fabf8156690 2020-11-20T12:15:58.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf77fe700 1 -- 172.21.15.116:0/395079536 >> v2:172.21.15.173:3300/0 conn(0x7fabf80a6300 msgr2=0x7fabf81561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:58.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf77fe700 1 --2- 172.21.15.116:0/395079536 >> v2:172.21.15.173:3300/0 conn(0x7fabf80a6300 0x7fabf81561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf77fe700 1 -- 172.21.15.116:0/395079536 >> v1:172.21.15.116:6789/0 conn(0x7fabf8156690 legacy=0x7fabf8065ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf77fe700 1 -- 172.21.15.116:0/395079536 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf80a8140 con 0x7fabf813bc90 2020-11-20T12:15:58.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf77fe700 1 --2- 172.21.15.116:0/395079536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf813bc90 0x7fabf80a5e10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fabe800b990 tx=0x7fabe8006d90).ready entity=mon.2 client_cookie=dff6c70985331d1d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:58.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.012+0000 7fabf67fc700 1 -- 172.21.15.116:0/395079536 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8006030 con 0x7fabf813bc90 2020-11-20T12:15:58.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.016+0000 7fabf67fc700 1 -- 172.21.15.116:0/395079536 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe8006190 con 0x7fabf813bc90 2020-11-20T12:15:58.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.016+0000 7fabf67fc700 1 -- 172.21.15.116:0/395079536 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8006510 con 0x7fabf813bc90 2020-11-20T12:15:58.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.016+0000 7fabfc920700 1 -- 172.21.15.116:0/395079536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf813bc90 msgr2=0x7fabf80a5e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.016+0000 7fabfc920700 1 --2- 172.21.15.116:0/395079536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf813bc90 0x7fabf80a5e10 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fabe800b990 tx=0x7fabe8006d90).stop 2020-11-20T12:15:58.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.016+0000 7fabfc920700 1 -- 172.21.15.116:0/395079536 shutdown_connections 2020-11-20T12:15:58.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.016+0000 7fabfc920700 1 --2- 172.21.15.116:0/395079536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf813bc90 0x7fabf80a5e10 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.016+0000 7fabfc920700 1 --2- 172.21.15.116:0/395079536 >> v2:172.21.15.173:3300/0 conn(0x7fabf80a6300 0x7fabf81561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.016+0000 7fabfc920700 1 -- 172.21.15.116:0/395079536 >> 172.21.15.116:0/395079536 conn(0x7fabf8065120 msgr2=0x7fabf8060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:58.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.016+0000 7fabfc920700 1 -- 172.21.15.116:0/395079536 shutdown_connections 2020-11-20T12:15:58.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.016+0000 7fabfc920700 1 -- 172.21.15.116:0/395079536 wait complete. 2020-11-20T12:15:58.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabfc920700 1 Processor -- start 2020-11-20T12:15:58.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabfc920700 1 -- start start 2020-11-20T12:15:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabfc920700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf80a6300 0x7fabf8138d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabfc920700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf813bc90 0x7fabf81658a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabfc920700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf8156690 0x7fabf8165d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabfc920700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80aa1e0 con 0x7fabf813bc90 2020-11-20T12:15:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabfc920700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80aa480 con 0x7fabf8156690 2020-11-20T12:15:58.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabfc920700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf80aa720 con 0x7fabf80a6300 2020-11-20T12:15:58.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabf6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf813bc90 0x7fabf81658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabf77fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf80a6300 0x7fabf8138d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabf6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf813bc90 0x7fabf81658a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42790/0 (socket says 172.21.15.116:42790) 2020-11-20T12:15:58.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.020+0000 7fabf6ffd700 1 -- 172.21.15.116:0/3250151481 learned_addr learned my addr 172.21.15.116:0/3250151481 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:58.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabf7fff700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf8156690 0x7fabf8165d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabf6ffd700 1 -- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf80a6300 msgr2=0x7fabf8138d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabf6ffd700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf80a6300 0x7fabf8138d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabf6ffd700 1 -- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf8156690 msgr2=0x7fabf8165d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabf6ffd700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf8156690 0x7fabf8165d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabf6ffd700 1 -- 172.21.15.116:0/3250151481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf80aa9c0 con 0x7fabf813bc90 2020-11-20T12:15:58.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabf7fff700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf8156690 0x7fabf8165d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:58.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabf6ffd700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf813bc90 0x7fabf81658a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fabec00eba0 tx=0x7fabec006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:58.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabd77fe700 1 -- 172.21.15.116:0/3250151481 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec00c8e0 con 0x7fabf813bc90 2020-11-20T12:15:58.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabd77fe700 1 -- 172.21.15.116:0/3250151481 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabec00ca40 con 0x7fabf813bc90 2020-11-20T12:15:58.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabfc920700 1 -- 172.21.15.116:0/3250151481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf80aac60 con 0x7fabf813bc90 2020-11-20T12:15:58.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabd77fe700 1 -- 172.21.15.116:0/3250151481 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabec00cdc0 con 0x7fabf813bc90 2020-11-20T12:15:58.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabfc920700 1 -- 172.21.15.116:0/3250151481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf8139260 con 0x7fabf813bc90 2020-11-20T12:15:58.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabfc920700 1 -- 172.21.15.116:0/3250151481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf8044f60 con 0x7fabf813bc90 2020-11-20T12:15:58.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.024+0000 7fabd77fe700 1 -- 172.21.15.116:0/3250151481 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fabec01a030 con 0x7fabf813bc90 2020-11-20T12:15:58.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.028+0000 7fabd77fe700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd00557f0 0x7fabd0057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.028+0000 7fabd77fe700 1 -- 172.21.15.116:0/3250151481 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fabec06df40 con 0x7fabf813bc90 2020-11-20T12:15:58.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.028+0000 7fabf77fe700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd00557f0 0x7fabd0057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.028+0000 7fabf77fe700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd00557f0 0x7fabd0057cb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fabe8006b20 tx=0x7fabe8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:58.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.032+0000 7fabd77fe700 1 -- 172.21.15.116:0/3250151481 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fabec0441c0 con 0x7fabf813bc90 2020-11-20T12:15:58.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.192+0000 7fabfc920700 1 -- 172.21.15.116:0/3250151481 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fabf80ac320 con 0x7fabf813bc90 2020-11-20T12:15:58.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.192+0000 7fabd77fe700 1 -- 172.21.15.116:0/3250151481 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7fabec0477e0 con 0x7fabf813bc90 2020-11-20T12:15:58.195 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:15:58.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 -- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd00557f0 msgr2=0x7fabd0057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd00557f0 0x7fabd0057cb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7fabe8006b20 tx=0x7fabe8015000).stop 2020-11-20T12:15:58.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 -- 172.21.15.116:0/3250151481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf813bc90 msgr2=0x7fabf81658a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf813bc90 0x7fabf81658a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fabec00eba0 tx=0x7fabec006df0).stop 2020-11-20T12:15:58.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 -- 172.21.15.116:0/3250151481 shutdown_connections 2020-11-20T12:15:58.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd00557f0 0x7fabd0057cb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf8156690 0x7fabf8165d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf80a6300 0x7fabf8138d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 --2- 172.21.15.116:0/3250151481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf813bc90 0x7fabf81658a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 -- 172.21.15.116:0/3250151481 >> 172.21.15.116:0/3250151481 conn(0x7fabf8065120 msgr2=0x7fabf813adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:58.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 -- 172.21.15.116:0/3250151481 shutdown_connections 2020-11-20T12:15:58.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.196+0000 7fabfc920700 1 -- 172.21.15.116:0/3250151481 wait complete. 2020-11-20T12:15:58.223 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-20T12:15:58.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.396+0000 7f8dc8a0a700 1 Processor -- start 2020-11-20T12:15:58.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.396+0000 7f8dc8a0a700 1 -- start start 2020-11-20T12:15:58.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc8a0a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc4153ad0 0x7f8dc4064580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc8a0a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8dc4064a70 0x7f8dc414fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc8a0a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc4153330 con 0x7f8dc41536f0 2020-11-20T12:15:58.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc8a0a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dc40a7170 con 0x7f8dc4064a70 2020-11-20T12:15:58.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc8a0a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc4150600 con 0x7f8dc4153ad0 2020-11-20T12:15:58.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc37fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8dc41536f0 0x7f8dc40a6a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52228/0 (socket says 172.21.15.116:52228) 2020-11-20T12:15:58.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc37fe700 1 -- 172.21.15.116:0/782133091 learned_addr learned my addr 172.21.15.116:0/782133091 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:15:58.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc3fff700 1 --2- 172.21.15.116:0/782133091 >> v2:172.21.15.116:3300/0 conn(0x7f8dc4064a70 0x7f8dc414fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dbbfff700 1 --2- 172.21.15.116:0/782133091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc4153ad0 0x7f8dc4064580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc3fff700 1 -- 172.21.15.116:0/782133091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc4153ad0 msgr2=0x7f8dc4064580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:58.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc3fff700 1 --2- 172.21.15.116:0/782133091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc4153ad0 0x7f8dc4064580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc2ffd700 1 -- 172.21.15.116:0/782133091 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3142432953 0 0) 0x7f8dc4153330 con 0x7f8dc41536f0 2020-11-20T12:15:58.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc3fff700 1 -- 172.21.15.116:0/782133091 >> v1:172.21.15.116:6789/0 conn(0x7f8dc41536f0 legacy=0x7f8dc40a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.400+0000 7f8dc3fff700 1 -- 172.21.15.116:0/782133091 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc40a60f0 con 0x7f8dc4064a70 2020-11-20T12:15:58.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc3fff700 1 --2- 172.21.15.116:0/782133091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dc4064a70 0x7f8dc414fa10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f8dbc009c90 tx=0x7f8dbc0091d0).ready entity=mon.0 client_cookie=b1f28348094a8e88 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:58.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc2ffd700 1 -- 172.21.15.116:0/782133091 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00ba70 con 0x7f8dc4064a70 2020-11-20T12:15:58.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc2ffd700 1 -- 172.21.15.116:0/782133091 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc00bbd0 con 0x7f8dc4064a70 2020-11-20T12:15:58.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc2ffd700 1 -- 172.21.15.116:0/782133091 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc011400 con 0x7f8dc4064a70 2020-11-20T12:15:58.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/782133091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dc4064a70 msgr2=0x7f8dc414fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc8a0a700 1 --2- 172.21.15.116:0/782133091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dc4064a70 0x7f8dc414fa10 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f8dbc009c90 tx=0x7f8dbc0091d0).stop 2020-11-20T12:15:58.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/782133091 shutdown_connections 2020-11-20T12:15:58.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc8a0a700 1 --2- 172.21.15.116:0/782133091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dc4064a70 0x7f8dc414fa10 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc8a0a700 1 --2- 172.21.15.116:0/782133091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc4153ad0 0x7f8dc4064580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/782133091 >> 172.21.15.116:0/782133091 conn(0x7f8dc4060c40 msgr2=0x7f8dc40c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:58.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/782133091 shutdown_connections 2020-11-20T12:15:58.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/782133091 wait complete. 2020-11-20T12:15:58.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc8a0a700 1 Processor -- start 2020-11-20T12:15:58.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.404+0000 7f8dc8a0a700 1 -- start start 2020-11-20T12:15:58.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dc8a0a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc4064a70 0x7f8dc4152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dc8a0a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dc41536f0 0x7f8dc4149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dc8a0a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dc4153ad0 0x7f8dc414b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dc8a0a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc4152a10 con 0x7f8dc41536f0 2020-11-20T12:15:58.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dc8a0a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc415f120 con 0x7f8dc4153ad0 2020-11-20T12:15:58.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dc8a0a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc415f3c0 con 0x7f8dc4064a70 2020-11-20T12:15:58.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dc41536f0 0x7f8dc4149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dc41536f0 0x7f8dc4149d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42802/0 (socket says 172.21.15.116:42802) 2020-11-20T12:15:58.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbbfff700 1 -- 172.21.15.116:0/910542396 learned_addr learned my addr 172.21.15.116:0/910542396 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:58.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dc3fff700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dc4153ad0 0x7f8dc414b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dc37fe700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc4064a70 0x7f8dc4152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbbfff700 1 -- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc4064a70 msgr2=0x7f8dc4152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbbfff700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc4064a70 0x7f8dc4152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbbfff700 1 -- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dc4153ad0 msgr2=0x7f8dc414b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbbfff700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dc4153ad0 0x7f8dc414b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbbfff700 1 -- 172.21.15.116:0/910542396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc415f660 con 0x7f8dc41536f0 2020-11-20T12:15:58.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dc3fff700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dc4153ad0 0x7f8dc414b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:15:58.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbbfff700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dc41536f0 0x7f8dc4149d90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f8dac00ebd0 tx=0x7f8dac006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:58.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbb7fe700 1 -- 172.21.15.116:0/910542396 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00b8d0 con 0x7f8dc41536f0 2020-11-20T12:15:58.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbb7fe700 1 -- 172.21.15.116:0/910542396 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dac00ba30 con 0x7f8dc41536f0 2020-11-20T12:15:58.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/910542396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc415f900 con 0x7f8dc41536f0 2020-11-20T12:15:58.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dbb7fe700 1 -- 172.21.15.116:0/910542396 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dac00bdb0 con 0x7f8dc41536f0 2020-11-20T12:15:58.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.408+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/910542396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc415ee20 con 0x7f8dc41536f0 2020-11-20T12:15:58.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.412+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/910542396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc4044f60 con 0x7f8dc41536f0 2020-11-20T12:15:58.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.412+0000 7f8dbb7fe700 1 -- 172.21.15.116:0/910542396 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8dac012600 con 0x7f8dc41536f0 2020-11-20T12:15:58.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.412+0000 7f8dbb7fe700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8da00557c0 0x7f8da0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.412+0000 7f8dbb7fe700 1 -- 172.21.15.116:0/910542396 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f8dac06e670 con 0x7f8dc41536f0 2020-11-20T12:15:58.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.412+0000 7f8dc37fe700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8da00557c0 0x7f8da0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.416+0000 7f8dc37fe700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8da00557c0 0x7f8da0057c80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8db40011a0 tx=0x7f8db4007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:58.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.416+0000 7f8dbb7fe700 1 -- 172.21.15.116:0/910542396 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8dac0447e0 con 0x7f8dc41536f0 2020-11-20T12:15:58.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.576+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/910542396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8dc40640b0 con 0x7f8dc41536f0 2020-11-20T12:15:58.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.580+0000 7f8dbb7fe700 1 -- 172.21.15.116:0/910542396 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+143 (secure 0 0 0) 0x7f8dac06e020 con 0x7f8dc41536f0 2020-11-20T12:15:58.580 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:15:58.581 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2020-11-20T12:15:58.581 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:15:58.581 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2020-11-20T12:15:58.581 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "device_health_metrics" 2020-11-20T12:15:58.582 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:15:58.582 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:15:58.582 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 2, 2020-11-20T12:15:58.582 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "rbd" 2020-11-20T12:15:58.582 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:15:58.583 INFO:teuthology.orchestra.run.smithi116.stdout:] 2020-11-20T12:15:58.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.580+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8da00557c0 msgr2=0x7f8da0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.580+0000 7f8dc8a0a700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8da00557c0 0x7f8da0057c80 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f8db40011a0 tx=0x7f8db4007a00).stop 2020-11-20T12:15:58.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.580+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/910542396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dc41536f0 msgr2=0x7f8dc4149d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.580+0000 7f8dc8a0a700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dc41536f0 0x7f8dc4149d90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f8dac00ebd0 tx=0x7f8dac006df0).stop 2020-11-20T12:15:58.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.584+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/910542396 shutdown_connections 2020-11-20T12:15:58.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.584+0000 7f8dc8a0a700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8da00557c0 0x7f8da0057c80 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.584+0000 7f8dc8a0a700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dc4153ad0 0x7f8dc414b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.584+0000 7f8dc8a0a700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dc4064a70 0x7f8dc4152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.584+0000 7f8dc8a0a700 1 --2- 172.21.15.116:0/910542396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dc41536f0 0x7f8dc4149d90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.584+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/910542396 >> 172.21.15.116:0/910542396 conn(0x7f8dc4060c40 msgr2=0x7f8dc40a7f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:58.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.584+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/910542396 shutdown_connections 2020-11-20T12:15:58.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.584+0000 7f8dc8a0a700 1 -- 172.21.15.116:0/910542396 wait complete. 2020-11-20T12:15:58.610 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T12:15:58.611 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-20T12:15:58.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.788+0000 7fac71980700 1 Processor -- start 2020-11-20T12:15:58.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.788+0000 7fac71980700 1 -- start start 2020-11-20T12:15:58.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.792+0000 7fac71980700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fac6c13ad30 0x7fac6c0a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.792+0000 7fac71980700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6c13b110 0x7fac6c151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.792+0000 7fac71980700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac6c152a80 con 0x7fac6c151fa0 2020-11-20T12:15:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.792+0000 7fac71980700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac6c152540 con 0x7fac6c13ad30 2020-11-20T12:15:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.792+0000 7fac71980700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c1527e0 con 0x7fac6c13b110 2020-11-20T12:15:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.792+0000 7fac7117f700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fac6c151fa0 0x7fac6c061250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52246/0 (socket says 172.21.15.116:52246) 2020-11-20T12:15:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.792+0000 7fac7117f700 1 -- 172.21.15.116:0/3425280509 learned_addr learned my addr 172.21.15.116:0/3425280509 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:15:58.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.792+0000 7fac7097e700 1 --2- 172.21.15.116:0/3425280509 >> v2:172.21.15.173:3300/0 conn(0x7fac6c13ad30 0x7fac6c0a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.792+0000 7fac6bfff700 1 --2- 172.21.15.116:0/3425280509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6c13b110 0x7fac6c151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.792+0000 7fac6b7fe700 1 -- 172.21.15.116:0/3425280509 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4154598122 0 0) 0x7fac6c152a80 con 0x7fac6c151fa0 2020-11-20T12:15:58.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac6b7fe700 1 -- 172.21.15.116:0/3425280509 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac50003240 con 0x7fac6c151fa0 2020-11-20T12:15:58.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac7097e700 1 -- 172.21.15.116:0/3425280509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6c13b110 msgr2=0x7fac6c151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:15:58.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac7097e700 1 --2- 172.21.15.116:0/3425280509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6c13b110 0x7fac6c151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac7097e700 1 -- 172.21.15.116:0/3425280509 >> v1:172.21.15.116:6789/0 conn(0x7fac6c151fa0 legacy=0x7fac6c061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac7097e700 1 -- 172.21.15.116:0/3425280509 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c0ac910 con 0x7fac6c13ad30 2020-11-20T12:15:58.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac6bfff700 1 --2- 172.21.15.116:0/3425280509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6c13b110 0x7fac6c151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:15:58.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac7097e700 1 --2- 172.21.15.116:0/3425280509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac6c13ad30 0x7fac6c0a7fb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fac6000b9a0 tx=0x7fac60006df0).ready entity=mon.1 client_cookie=723acacc1ac6c062 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:58.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac6b7fe700 1 -- 172.21.15.116:0/3425280509 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60009710 con 0x7fac6c13ad30 2020-11-20T12:15:58.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac6b7fe700 1 -- 172.21.15.116:0/3425280509 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac60009870 con 0x7fac6c13ad30 2020-11-20T12:15:58.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac6b7fe700 1 -- 172.21.15.116:0/3425280509 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac60009bf0 con 0x7fac6c13ad30 2020-11-20T12:15:58.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac71980700 1 -- 172.21.15.116:0/3425280509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac6c13ad30 msgr2=0x7fac6c0a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac71980700 1 --2- 172.21.15.116:0/3425280509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac6c13ad30 0x7fac6c0a7fb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fac6000b9a0 tx=0x7fac60006df0).stop 2020-11-20T12:15:58.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac71980700 1 -- 172.21.15.116:0/3425280509 shutdown_connections 2020-11-20T12:15:58.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac71980700 1 --2- 172.21.15.116:0/3425280509 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac6c13ad30 0x7fac6c0a7fb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac71980700 1 --2- 172.21.15.116:0/3425280509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6c13b110 0x7fac6c151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac71980700 1 -- 172.21.15.116:0/3425280509 >> 172.21.15.116:0/3425280509 conn(0x7fac6c0678a0 msgr2=0x7fac6c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:58.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac71980700 1 -- 172.21.15.116:0/3425280509 shutdown_connections 2020-11-20T12:15:58.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.796+0000 7fac71980700 1 -- 172.21.15.116:0/3425280509 wait complete. 2020-11-20T12:15:58.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.800+0000 7fac71980700 1 Processor -- start 2020-11-20T12:15:58.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.800+0000 7fac71980700 1 -- start start 2020-11-20T12:15:58.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac71980700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6c13ad30 0x7fac6c1386f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac71980700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac6c13b110 0x7fac6c0c2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac71980700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac6c151fa0 0x7fac6c0acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac71980700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c0c1470 con 0x7fac6c151fa0 2020-11-20T12:15:58.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac71980700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c0c1660 con 0x7fac6c13b110 2020-11-20T12:15:58.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac71980700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c161c40 con 0x7fac6c13ad30 2020-11-20T12:15:58.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac7117f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac6c151fa0 0x7fac6c0acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac7117f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac6c151fa0 0x7fac6c0acf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42818/0 (socket says 172.21.15.116:42818) 2020-11-20T12:15:58.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac7117f700 1 -- 172.21.15.116:0/691651138 learned_addr learned my addr 172.21.15.116:0/691651138 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:58.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac7097e700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6c13ad30 0x7fac6c1386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac6bfff700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac6c13b110 0x7fac6c0c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac7117f700 1 -- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6c13ad30 msgr2=0x7fac6c1386f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac7117f700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6c13ad30 0x7fac6c1386f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac7117f700 1 -- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac6c13b110 msgr2=0x7fac6c0c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:58.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac7117f700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac6c13b110 0x7fac6c0c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:58.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac7117f700 1 -- 172.21.15.116:0/691651138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c1619a0 con 0x7fac6c151fa0 2020-11-20T12:15:58.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac7117f700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac6c151fa0 0x7fac6c0acf70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fac6400ca60 tx=0x7fac64007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:58.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac68ff9700 1 -- 172.21.15.116:0/691651138 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64005b80 con 0x7fac6c151fa0 2020-11-20T12:15:58.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac68ff9700 1 -- 172.21.15.116:0/691651138 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac64005ce0 con 0x7fac6c151fa0 2020-11-20T12:15:58.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac68ff9700 1 -- 172.21.15.116:0/691651138 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64007320 con 0x7fac6c151fa0 2020-11-20T12:15:58.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac71980700 1 -- 172.21.15.116:0/691651138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c0c18c0 con 0x7fac6c151fa0 2020-11-20T12:15:58.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac71980700 1 -- 172.21.15.116:0/691651138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6c138c60 con 0x7fac6c151fa0 2020-11-20T12:15:58.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.804+0000 7fac71980700 1 -- 172.21.15.116:0/691651138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6c044f60 con 0x7fac6c151fa0 2020-11-20T12:15:58.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.808+0000 7fac68ff9700 1 -- 172.21.15.116:0/691651138 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fac64010030 con 0x7fac6c151fa0 2020-11-20T12:15:58.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.808+0000 7fac68ff9700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac440557c0 0x7fac44057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:58.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.808+0000 7fac68ff9700 1 -- 172.21.15.116:0/691651138 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fac6406e290 con 0x7fac6c151fa0 2020-11-20T12:15:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.808+0000 7fac7097e700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac440557c0 0x7fac44057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:58.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.812+0000 7fac7097e700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac440557c0 0x7fac44057c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fac60006e30 tx=0x7fac60015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:58.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.812+0000 7fac68ff9700 1 -- 172.21.15.116:0/691651138 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fac64044460 con 0x7fac6c151fa0 2020-11-20T12:15:58.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:58.988+0000 7fac71980700 1 -- 172.21.15.116:0/691651138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fac6c069320 con 0x7fac6c151fa0 2020-11-20T12:15:59.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.120+0000 7fac68ff9700 1 -- 172.21.15.116:0/691651138 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v20) v1 ==== 133+0+0 (secure 0 0 0) 0x7fac64047a80 con 0x7fac6c151fa0 2020-11-20T12:15:59.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 -- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac440557c0 msgr2=0x7fac44057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac440557c0 0x7fac44057c80 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fac60006e30 tx=0x7fac60015000).stop 2020-11-20T12:15:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 -- 172.21.15.116:0/691651138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac6c151fa0 msgr2=0x7fac6c0acf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac6c151fa0 0x7fac6c0acf70 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fac6400ca60 tx=0x7fac64007e20).stop 2020-11-20T12:15:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 -- 172.21.15.116:0/691651138 shutdown_connections 2020-11-20T12:15:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac440557c0 0x7fac44057c80 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac6c151fa0 0x7fac6c0acf70 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac6c13ad30 0x7fac6c1386f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:59.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 --2- 172.21.15.116:0/691651138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac6c13b110 0x7fac6c0c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:59.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 -- 172.21.15.116:0/691651138 >> 172.21.15.116:0/691651138 conn(0x7fac6c0678a0 msgr2=0x7fac6c0aa730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:59.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 -- 172.21.15.116:0/691651138 shutdown_connections 2020-11-20T12:15:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.124+0000 7fac71980700 1 -- 172.21.15.116:0/691651138 wait complete. 2020-11-20T12:15:59.134 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2020-11-20T12:15:59.154 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-20T12:15:59.264 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:15:59.271 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:15:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.336+0000 7fe757309700 1 Processor -- start 2020-11-20T12:15:59.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.336+0000 7fe757309700 1 -- start start 2020-11-20T12:15:59.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe757309700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe75014f150 0x7fe7500c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:59.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe757309700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe750155620 con 0x7fe7500bd780 2020-11-20T12:15:59.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe757309700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7501558c0 con 0x7fe75014f530 2020-11-20T12:15:59.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe757309700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe750155b60 con 0x7fe75014f150 2020-11-20T12:15:59.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe7550a5700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe75014f150 0x7fe7500c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:59.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe7550a5700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe75014f150 0x7fe7500c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42822/0 (socket says 172.21.15.116:42822) 2020-11-20T12:15:59.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe7550a5700 1 -- 172.21.15.116:0/152058262 learned_addr learned my addr 172.21.15.116:0/152058262 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:59.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe7550a5700 1 -- 172.21.15.116:0/152058262 >> v1:172.21.15.173:6789/0 conn(0x7fe75014f530 legacy=0x7fe750151ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:59.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe7550a5700 1 -- 172.21.15.116:0/152058262 >> v1:172.21.15.116:6789/0 conn(0x7fe7500bd780 legacy=0x7fe7500a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:59.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe7550a5700 1 -- 172.21.15.116:0/152058262 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe750155e00 con 0x7fe75014f150 2020-11-20T12:15:59.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe7550a5700 1 --2- 172.21.15.116:0/152058262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75014f150 0x7fe7500c87f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fe74c006fd0 tx=0x7fe74c006eb0).ready entity=mon.0 client_cookie=afb8565d1f7d4f51 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:59.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe73ffff700 1 -- 172.21.15.116:0/152058262 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c0098d0 con 0x7fe75014f150 2020-11-20T12:15:59.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe73ffff700 1 -- 172.21.15.116:0/152058262 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c009a30 con 0x7fe75014f150 2020-11-20T12:15:59.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe73ffff700 1 -- 172.21.15.116:0/152058262 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c009db0 con 0x7fe75014f150 2020-11-20T12:15:59.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe757309700 1 -- 172.21.15.116:0/152058262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75014f150 msgr2=0x7fe7500c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:59.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.340+0000 7fe757309700 1 --2- 172.21.15.116:0/152058262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75014f150 0x7fe7500c87f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fe74c006fd0 tx=0x7fe74c006eb0).stop 2020-11-20T12:15:59.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.344+0000 7fe757309700 1 -- 172.21.15.116:0/152058262 shutdown_connections 2020-11-20T12:15:59.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.344+0000 7fe757309700 1 --2- 172.21.15.116:0/152058262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75014f150 0x7fe7500c87f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:59.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.344+0000 7fe757309700 1 -- 172.21.15.116:0/152058262 >> 172.21.15.116:0/152058262 conn(0x7fe750061580 msgr2=0x7fe750061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:15:59.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.344+0000 7fe757309700 1 -- 172.21.15.116:0/152058262 shutdown_connections 2020-11-20T12:15:59.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.344+0000 7fe757309700 1 -- 172.21.15.116:0/152058262 wait complete. 2020-11-20T12:15:59.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.344+0000 7fe757309700 1 Processor -- start 2020-11-20T12:15:59.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.344+0000 7fe757309700 1 -- start start 2020-11-20T12:15:59.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe757309700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7500bd780 0x7fe750161dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:59.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe757309700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75014f150 0x7fe750150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:59.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe757309700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75014f530 0x7fe7500ac640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:59.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe757309700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7501622b0 con 0x7fe75014f150 2020-11-20T12:15:59.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe757309700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe750161420 con 0x7fe7500bd780 2020-11-20T12:15:59.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe757309700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe750165540 con 0x7fe75014f530 2020-11-20T12:15:59.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe7548a4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75014f150 0x7fe750150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:59.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe7548a4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75014f150 0x7fe750150020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42830/0 (socket says 172.21.15.116:42830) 2020-11-20T12:15:59.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe7548a4700 1 -- 172.21.15.116:0/3162067180 learned_addr learned my addr 172.21.15.116:0/3162067180 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:15:59.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe7550a5700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7500bd780 0x7fe750161dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:59.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe7558a6700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75014f530 0x7fe7500ac640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:59.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe7548a4700 1 -- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75014f530 msgr2=0x7fe7500ac640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:59.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe7548a4700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75014f530 0x7fe7500ac640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:59.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe7548a4700 1 -- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7500bd780 msgr2=0x7fe750161dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:15:59.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe7548a4700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7500bd780 0x7fe750161dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:15:59.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe7548a4700 1 -- 172.21.15.116:0/3162067180 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe750161680 con 0x7fe75014f150 2020-11-20T12:15:59.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe7548a4700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75014f150 0x7fe750150020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe74000c710 tx=0x7fe740007c20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:59.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe73d7fa700 1 -- 172.21.15.116:0/3162067180 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740007020 con 0x7fe75014f150 2020-11-20T12:15:59.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe73d7fa700 1 -- 172.21.15.116:0/3162067180 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe740007180 con 0x7fe75014f150 2020-11-20T12:15:59.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe73d7fa700 1 -- 172.21.15.116:0/3162067180 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7400064f0 con 0x7fe75014f150 2020-11-20T12:15:59.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe757309700 1 -- 172.21.15.116:0/3162067180 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe750161920 con 0x7fe75014f150 2020-11-20T12:15:59.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.348+0000 7fe757309700 1 -- 172.21.15.116:0/3162067180 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe750150730 con 0x7fe75014f150 2020-11-20T12:15:59.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.352+0000 7fe757309700 1 -- 172.21.15.116:0/3162067180 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75005ca30 con 0x7fe75014f150 2020-11-20T12:15:59.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.352+0000 7fe73d7fa700 1 -- 172.21.15.116:0/3162067180 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe740010030 con 0x7fe75014f150 2020-11-20T12:15:59.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.352+0000 7fe73d7fa700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7280557f0 0x7fe728057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:15:59.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.352+0000 7fe73d7fa700 1 -- 172.21.15.116:0/3162067180 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fe74006dc30 con 0x7fe75014f150 2020-11-20T12:15:59.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.352+0000 7fe7550a5700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7280557f0 0x7fe728057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:15:59.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.356+0000 7fe7550a5700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7280557f0 0x7fe728057cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fe74c00b1b0 tx=0x7fe74c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:15:59.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.360+0000 7fe73d7fa700 1 -- 172.21.15.116:0/3162067180 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe740043c40 con 0x7fe75014f150 2020-11-20T12:15:59.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:15:59.520+0000 7fe757309700 1 -- 172.21.15.116:0/3162067180 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fe7500ab0d0 con 0x7fe75014f150 2020-11-20T12:16:00.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.128+0000 7fe73d7fa700 1 -- 172.21.15.116:0/3162067180 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v21) v1 ==== 125+0+0 (secure 0 0 0) 0x7fe740047260 con 0x7fe75014f150 2020-11-20T12:16:00.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 -- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7280557f0 msgr2=0x7fe728057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7280557f0 0x7fe728057cb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fe74c00b1b0 tx=0x7fe74c015000).stop 2020-11-20T12:16:00.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 -- 172.21.15.116:0/3162067180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75014f150 msgr2=0x7fe750150020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75014f150 0x7fe750150020 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fe74000c710 tx=0x7fe740007c20).stop 2020-11-20T12:16:00.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 -- 172.21.15.116:0/3162067180 shutdown_connections 2020-11-20T12:16:00.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7280557f0 0x7fe728057cb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe75014f530 0x7fe7500ac640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7500bd780 0x7fe750161dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 --2- 172.21.15.116:0/3162067180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe75014f150 0x7fe750150020 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 -- 172.21.15.116:0/3162067180 >> 172.21.15.116:0/3162067180 conn(0x7fe750061580 msgr2=0x7fe75014ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:00.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 -- 172.21.15.116:0/3162067180 shutdown_connections 2020-11-20T12:16:00.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.136+0000 7fe757309700 1 -- 172.21.15.116:0/3162067180 wait complete. 2020-11-20T12:16:00.142 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2020-11-20T12:16:00.163 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T12:16:00.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.380+0000 7f549e5a1700 1 Processor -- start 2020-11-20T12:16:00.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.380+0000 7f549e5a1700 1 -- start start 2020-11-20T12:16:00.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.380+0000 7f549e5a1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f54980be350 0x7f54980c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.380+0000 7f549e5a1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54980c4b70 con 0x7f5498062620 2020-11-20T12:16:00.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.380+0000 7f549e5a1700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54980c40f0 con 0x7f54980c52f0 2020-11-20T12:16:00.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.380+0000 7f549e5a1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5498151b00 con 0x7f54980be350 2020-11-20T12:16:00.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.380+0000 7f549dda0700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5498062620 0x7f549815b1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52272/0 (socket says 172.21.15.116:52272) 2020-11-20T12:16:00.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.380+0000 7f549dda0700 1 -- 172.21.15.116:0/1019909355 learned_addr learned my addr 172.21.15.116:0/1019909355 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:00.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549d59f700 1 --2- 172.21.15.116:0/1019909355 >> v2:172.21.15.116:3300/0 conn(0x7f54980be350 0x7f54980c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103195162 0 0) 0x7f54980c4b70 con 0x7f5498062620 2020-11-20T12:16:00.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5480003240 con 0x7f5498062620 2020-11-20T12:16:00.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538207911 0 0) 0x7f54980c40f0 con 0x7f54980c52f0 2020-11-20T12:16:00.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54980c4b70 con 0x7f54980c52f0 2020-11-20T12:16:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3677407721 0 0) 0x7f5480003240 con 0x7f5498062620 2020-11-20T12:16:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 >> v2:172.21.15.116:3300/0 conn(0x7f54980be350 msgr2=0x7f54980c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 --2- 172.21.15.116:0/1019909355 >> v2:172.21.15.116:3300/0 conn(0x7f54980be350 0x7f54980c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 >> v1:172.21.15.173:6789/0 conn(0x7f54980c52f0 legacy=0x7f54980c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54980c4390 con 0x7f5498062620 2020-11-20T12:16:00.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f5490001860 con 0x7f5498062620 2020-11-20T12:16:00.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 >> v1:172.21.15.116:6789/0 conn(0x7f5498062620 legacy=0x7f549815b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54800038c0 0x7f5480003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54800044e0 0x7f5480004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5480004e50 0x7f548000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5498151b00 con 0x7f54800044e0 2020-11-20T12:16:00.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54800053b0 con 0x7f5480004e50 2020-11-20T12:16:00.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548000bca0 con 0x7f54800038c0 2020-11-20T12:16:00.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549cd9e700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54800044e0 0x7f5480004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549d59f700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54800038c0 0x7f5480003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549dda0700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5480004e50 0x7f548000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549cd9e700 1 -- 172.21.15.116:0/1019909355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54800038c0 msgr2=0x7f5480003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:00.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549cd9e700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54800038c0 0x7f5480003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549cd9e700 1 -- 172.21.15.116:0/1019909355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5480004e50 msgr2=0x7f548000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:00.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549cd9e700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5480004e50 0x7f548000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549cd9e700 1 -- 172.21.15.116:0/1019909355 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5480003240 con 0x7f54800044e0 2020-11-20T12:16:00.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549dda0700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5480004e50 0x7f548000b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:00.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549dda0700 1 -- 172.21.15.116:0/1019909355 reap_dead start 2020-11-20T12:16:00.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f549cd9e700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54800044e0 0x7f5480004960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5488001a80 tx=0x7f5488007b90).ready entity=mon.0 client_cookie=a6ce9c17719bdd96 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:00.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5488007020 con 0x7f54800044e0 2020-11-20T12:16:00.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5488007180 con 0x7f54800044e0 2020-11-20T12:16:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.384+0000 7f5487fff700 1 -- 172.21.15.116:0/1019909355 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548800c890 con 0x7f54800044e0 2020-11-20T12:16:00.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.388+0000 7f549e5a1700 1 -- 172.21.15.116:0/1019909355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54800044e0 msgr2=0x7f5480004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.388+0000 7f549e5a1700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54800044e0 0x7f5480004960 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f5488001a80 tx=0x7f5488007b90).stop 2020-11-20T12:16:00.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.388+0000 7f549e5a1700 1 -- 172.21.15.116:0/1019909355 shutdown_connections 2020-11-20T12:16:00.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.388+0000 7f549e5a1700 1 --2- 172.21.15.116:0/1019909355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54800044e0 0x7f5480004960 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.388+0000 7f549e5a1700 1 -- 172.21.15.116:0/1019909355 >> 172.21.15.116:0/1019909355 conn(0x7f5498091900 msgr2=0x7f549807ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.388+0000 7f549e5a1700 1 -- 172.21.15.116:0/1019909355 shutdown_connections 2020-11-20T12:16:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.388+0000 7f549e5a1700 1 -- 172.21.15.116:0/1019909355 wait complete. 2020-11-20T12:16:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.388+0000 7f549e5a1700 1 Processor -- start 2020-11-20T12:16:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.388+0000 7f549e5a1700 1 -- start start 2020-11-20T12:16:00.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549e5a1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54800044e0 0x7f54981544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549e5a1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5498161fb0 0x7f549814e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549e5a1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54980a7f20 0x7f54980a83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549e5a1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5498152df0 con 0x7f54980a7f20 2020-11-20T12:16:00.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549e5a1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5498153090 con 0x7f5498161fb0 2020-11-20T12:16:00.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549e5a1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5498153330 con 0x7f54800044e0 2020-11-20T12:16:00.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549dda0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54980a7f20 0x7f54980a83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549dda0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54980a7f20 0x7f54980a83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42850/0 (socket says 172.21.15.116:42850) 2020-11-20T12:16:00.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549dda0700 1 -- 172.21.15.116:0/512068131 learned_addr learned my addr 172.21.15.116:0/512068131 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:00.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549cd9e700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5498161fb0 0x7f549814e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549d59f700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54800044e0 0x7f54981544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549dda0700 1 -- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54800044e0 msgr2=0x7f54981544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549dda0700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54800044e0 0x7f54981544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549dda0700 1 -- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5498161fb0 msgr2=0x7f549814e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549dda0700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5498161fb0 0x7f549814e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549dda0700 1 -- 172.21.15.116:0/512068131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54981535d0 con 0x7f54980a7f20 2020-11-20T12:16:00.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549dda0700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54980a7f20 0x7f54980a83a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f54980607d0 tx=0x7f54980c4520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:00.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f54857fa700 1 -- 172.21.15.116:0/512068131 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549000e8a0 con 0x7f54980a7f20 2020-11-20T12:16:00.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f54857fa700 1 -- 172.21.15.116:0/512068131 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f549000ea20 con 0x7f54980a7f20 2020-11-20T12:16:00.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549e5a1700 1 -- 172.21.15.116:0/512068131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5498153870 con 0x7f54980a7f20 2020-11-20T12:16:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f54857fa700 1 -- 172.21.15.116:0/512068131 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f549000ede0 con 0x7f54980a7f20 2020-11-20T12:16:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.392+0000 7f549e5a1700 1 -- 172.21.15.116:0/512068131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54981549d0 con 0x7f54980a7f20 2020-11-20T12:16:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.396+0000 7f549e5a1700 1 -- 172.21.15.116:0/512068131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5498044f60 con 0x7f54980a7f20 2020-11-20T12:16:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.396+0000 7f54857fa700 1 -- 172.21.15.116:0/512068131 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5490001570 con 0x7f54980a7f20 2020-11-20T12:16:00.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.396+0000 7f54857fa700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54700557c0 0x7f5470057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.396+0000 7f54857fa700 1 -- 172.21.15.116:0/512068131 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f549006e410 con 0x7f54980a7f20 2020-11-20T12:16:00.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.396+0000 7f549d59f700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54700557c0 0x7f5470057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.400+0000 7f549d59f700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54700557c0 0x7f5470057c80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f5498150ae0 tx=0x7f5494008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:00.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.404+0000 7f54857fa700 1 -- 172.21.15.116:0/512068131 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5490044320 con 0x7f54980a7f20 2020-11-20T12:16:00.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.616+0000 7f549e5a1700 1 -- 172.21.15.116:0/512068131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f5498153a00 con 0x7f54980a7f20 2020-11-20T12:16:00.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.628+0000 7f54857fa700 1 -- 172.21.15.116:0/512068131 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 3 and data pool 4 v3) v1 ==== 187+0+0 (secure 0 0 0) 0x7f5490047940 con 0x7f54980a7f20 2020-11-20T12:16:00.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.640+0000 7f549e5a1700 1 -- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54700557c0 msgr2=0x7f5470057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.640+0000 7f549e5a1700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54700557c0 0x7f5470057c80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f5498150ae0 tx=0x7f5494008040).stop 2020-11-20T12:16:00.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.640+0000 7f549e5a1700 1 -- 172.21.15.116:0/512068131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54980a7f20 msgr2=0x7f54980a83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.640+0000 7f549e5a1700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54980a7f20 0x7f54980a83a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f54980607d0 tx=0x7f54980c4520).stop 2020-11-20T12:16:00.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.644+0000 7f549e5a1700 1 -- 172.21.15.116:0/512068131 shutdown_connections 2020-11-20T12:16:00.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.644+0000 7f549e5a1700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54700557c0 0x7f5470057c80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.644+0000 7f549e5a1700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f54980a7f20 0x7f54980a83a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.644+0000 7f549e5a1700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54800044e0 0x7f54981544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.644+0000 7f549e5a1700 1 --2- 172.21.15.116:0/512068131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5498161fb0 0x7f549814e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.644+0000 7f549e5a1700 1 -- 172.21.15.116:0/512068131 >> 172.21.15.116:0/512068131 conn(0x7f5498091900 msgr2=0x7f54980614f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:00.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.648+0000 7f549e5a1700 1 -- 172.21.15.116:0/512068131 shutdown_connections 2020-11-20T12:16:00.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.648+0000 7f549e5a1700 1 -- 172.21.15.116:0/512068131 wait complete. 2020-11-20T12:16:00.658 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 3 and data pool 4 2020-11-20T12:16:00.675 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:00.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.848+0000 7f3aa27b0700 1 Processor -- start 2020-11-20T12:16:00.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.848+0000 7f3aa27b0700 1 -- start start 2020-11-20T12:16:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.848+0000 7f3aa27b0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a9c0668a0 0x7f3a9c0a4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.848+0000 7f3aa27b0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3a9c0a6290 0x7f3a9c0a6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.848+0000 7f3aa27b0700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c0a3bd0 con 0x7f3a9c0c3fb0 2020-11-20T12:16:00.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa27b0700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c0a3e70 con 0x7f3a9c0a6290 2020-11-20T12:16:00.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa27b0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0a4110 con 0x7f3a9c0668a0 2020-11-20T12:16:00.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa0fad700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3a9c0a6290 0x7f3a9c0a6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa0fad700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3a9c0a6290 0x7f3a9c0a6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42866/0 (socket says 172.21.15.116:42866) 2020-11-20T12:16:00.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa0fad700 1 -- 172.21.15.116:0/2872142603 learned_addr learned my addr 172.21.15.116:0/2872142603 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:00.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa17ae700 1 --2- 172.21.15.116:0/2872142603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a9c0668a0 0x7f3a9c0a4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa0fad700 1 -- 172.21.15.116:0/2872142603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a9c0668a0 msgr2=0x7f3a9c0a4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa0fad700 1 --2- 172.21.15.116:0/2872142603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a9c0668a0 0x7f3a9c0a4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa0fad700 1 -- 172.21.15.116:0/2872142603 >> v1:172.21.15.173:6789/0 conn(0x7f3a9c0c3fb0 legacy=0x7f3a9c151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa0fad700 1 -- 172.21.15.116:0/2872142603 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c0a43b0 con 0x7f3a9c0a6290 2020-11-20T12:16:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa0fad700 1 --2- 172.21.15.116:0/2872142603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a9c0a6290 0x7f3a9c0a6710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3a8c006fd0 tx=0x7f3a8c006eb0).ready entity=mon.0 client_cookie=aaf814725e2f3d29 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3a8bfff700 1 -- 172.21.15.116:0/2872142603 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c009890 con 0x7f3a9c0a6290 2020-11-20T12:16:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3a8bfff700 1 -- 172.21.15.116:0/2872142603 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8c0099f0 con 0x7f3a9c0a6290 2020-11-20T12:16:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3a8bfff700 1 -- 172.21.15.116:0/2872142603 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8c009d70 con 0x7f3a9c0a6290 2020-11-20T12:16:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2872142603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a9c0a6290 msgr2=0x7f3a9c0a6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa27b0700 1 --2- 172.21.15.116:0/2872142603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a9c0a6290 0x7f3a9c0a6710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f3a8c006fd0 tx=0x7f3a8c006eb0).stop 2020-11-20T12:16:00.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2872142603 shutdown_connections 2020-11-20T12:16:00.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa27b0700 1 --2- 172.21.15.116:0/2872142603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a9c0668a0 0x7f3a9c0a4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa27b0700 1 --2- 172.21.15.116:0/2872142603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a9c0a6290 0x7f3a9c0a6710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2872142603 >> 172.21.15.116:0/2872142603 conn(0x7f3a9c062520 msgr2=0x7f3a9c062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:00.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2872142603 shutdown_connections 2020-11-20T12:16:00.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.852+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2872142603 wait complete. 2020-11-20T12:16:00.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.856+0000 7f3aa27b0700 1 Processor -- start 2020-11-20T12:16:00.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.856+0000 7f3aa27b0700 1 -- start start 2020-11-20T12:16:00.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa27b0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a9c0668a0 0x7f3a9c144200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa27b0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a9c0a6290 0x7f3a9c0a79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa27b0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a9c0c3fb0 0x7f3a9c0a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa27b0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0a8980 con 0x7f3a9c0c3fb0 2020-11-20T12:16:00.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa27b0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0a8440 con 0x7f3a9c0a6290 2020-11-20T12:16:00.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa27b0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0a86e0 con 0x7f3a9c0668a0 2020-11-20T12:16:00.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa1faf700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a9c0c3fb0 0x7f3a9c0a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa1faf700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a9c0c3fb0 0x7f3a9c0a0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42876/0 (socket says 172.21.15.116:42876) 2020-11-20T12:16:00.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa1faf700 1 -- 172.21.15.116:0/2950305787 learned_addr learned my addr 172.21.15.116:0/2950305787 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:00.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa0fad700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a9c0a6290 0x7f3a9c0a79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa17ae700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a9c0668a0 0x7f3a9c144200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa1faf700 1 -- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a9c0668a0 msgr2=0x7f3a9c144200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa1faf700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a9c0668a0 0x7f3a9c144200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa1faf700 1 -- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a9c0a6290 msgr2=0x7f3a9c0a79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:00.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa1faf700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a9c0a6290 0x7f3a9c0a79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:00.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa1faf700 1 -- 172.21.15.116:0/2950305787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c09d5a0 con 0x7f3a9c0c3fb0 2020-11-20T12:16:00.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa1faf700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a9c0c3fb0 0x7f3a9c0a0be0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f3a9400bae0 tx=0x7f3a940090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:00.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3a897fa700 1 -- 172.21.15.116:0/2950305787 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9400c710 con 0x7f3a9c0c3fb0 2020-11-20T12:16:00.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3a897fa700 1 -- 172.21.15.116:0/2950305787 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9400c870 con 0x7f3a9c0c3fb0 2020-11-20T12:16:00.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.860+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2950305787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c09d840 con 0x7f3a9c0c3fb0 2020-11-20T12:16:00.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.864+0000 7f3a897fa700 1 -- 172.21.15.116:0/2950305787 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9400ca20 con 0x7f3a9c0c3fb0 2020-11-20T12:16:00.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.864+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2950305787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c1446f0 con 0x7f3a9c0c3fb0 2020-11-20T12:16:00.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.864+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2950305787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c09d9d0 con 0x7f3a9c0c3fb0 2020-11-20T12:16:00.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.864+0000 7f3a897fa700 1 -- 172.21.15.116:0/2950305787 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3a9401c070 con 0x7f3a9c0c3fb0 2020-11-20T12:16:00.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.864+0000 7f3a897fa700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a740557f0 0x7f3a74057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:00.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.868+0000 7f3a897fa700 1 -- 172.21.15.116:0/2950305787 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3a9406dc70 con 0x7f3a9c0c3fb0 2020-11-20T12:16:00.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.868+0000 7f3aa17ae700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a740557f0 0x7f3a74057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:00.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.868+0000 7f3aa17ae700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a740557f0 0x7f3a74057cb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3a9800ba00 tx=0x7f3a98008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:00.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:00.872+0000 7f3a897fa700 1 -- 172.21.15.116:0/2950305787 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3a94043b10 con 0x7f3a9c0c3fb0 2020-11-20T12:16:01.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.032+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2950305787 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a9c044f60 con 0x7f3a9c0c3fb0 2020-11-20T12:16:01.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.032+0000 7f3a897fa700 1 -- 172.21.15.116:0/2950305787 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16887 (secure 0 0 0) 0x7f3a9401a030 con 0x7f3a9c0c3fb0 2020-11-20T12:16:01.035 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:01.035 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":22,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:00.619510+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:01.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a740557f0 msgr2=0x7f3a74057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a740557f0 0x7f3a74057cb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f3a9800ba00 tx=0x7f3a98008040).stop 2020-11-20T12:16:01.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2950305787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a9c0c3fb0 msgr2=0x7f3a9c0a0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a9c0c3fb0 0x7f3a9c0a0be0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f3a9400bae0 tx=0x7f3a940090f0).stop 2020-11-20T12:16:01.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2950305787 shutdown_connections 2020-11-20T12:16:01.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a740557f0 0x7f3a74057cb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a9c0c3fb0 0x7f3a9c0a0be0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a9c0668a0 0x7f3a9c144200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 --2- 172.21.15.116:0/2950305787 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a9c0a6290 0x7f3a9c0a79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2950305787 >> 172.21.15.116:0/2950305787 conn(0x7f3a9c062520 msgr2=0x7f3a9c14e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:01.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2950305787 shutdown_connections 2020-11-20T12:16:01.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.036+0000 7f3aa27b0700 1 -- 172.21.15.116:0/2950305787 wait complete. 2020-11-20T12:16:01.062 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:01.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.240+0000 7f5c10dee700 1 Processor -- start 2020-11-20T12:16:01.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.240+0000 7f5c10dee700 1 -- start start 2020-11-20T12:16:01.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.240+0000 7f5c10dee700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5c0c0a3590 0x7f5c0c156110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.240+0000 7f5c10dee700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c0c15a0f0 0x7f5c0c0ba740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.240+0000 7f5c10dee700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c0c156820 con 0x7f5c0c0bac30 2020-11-20T12:16:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.240+0000 7f5c10dee700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c0bb070 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.240+0000 7f5c10dee700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c0c156600 con 0x7f5c0c0a3590 2020-11-20T12:16:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.240+0000 7f5c09d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c0c15a0f0 0x7f5c0c0ba740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.240+0000 7f5c09d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c0c15a0f0 0x7f5c0c0ba740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42882/0 (socket says 172.21.15.116:42882) 2020-11-20T12:16:01.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.240+0000 7f5c09d9b700 1 -- 172.21.15.116:0/966302928 learned_addr learned my addr 172.21.15.116:0/966302928 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:01.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.240+0000 7f5c0a59c700 1 --2- 172.21.15.116:0/966302928 >> v2:172.21.15.173:3300/0 conn(0x7f5c0c0a3590 0x7f5c0c156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c09d9b700 1 -- 172.21.15.116:0/966302928 >> v2:172.21.15.173:3300/0 conn(0x7f5c0c0a3590 msgr2=0x7f5c0c156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:01.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c09d9b700 1 --2- 172.21.15.116:0/966302928 >> v2:172.21.15.173:3300/0 conn(0x7f5c0c0a3590 0x7f5c0c156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c09d9b700 1 -- 172.21.15.116:0/966302928 >> v1:172.21.15.173:6789/0 conn(0x7f5c0c0bac30 legacy=0x7f5c0c0b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c09d9b700 1 -- 172.21.15.116:0/966302928 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c156ac0 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c09d9b700 1 --2- 172.21.15.116:0/966302928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c0c15a0f0 0x7f5c0c0ba740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f5c0c0c1c80 tx=0x7f5c00006eb0).ready entity=mon.0 client_cookie=f2ddcf80355087e7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:01.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c0959a700 1 -- 172.21.15.116:0/966302928 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c000098d0 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c0959a700 1 -- 172.21.15.116:0/966302928 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c00009a30 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c0959a700 1 -- 172.21.15.116:0/966302928 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00009db0 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c10dee700 1 -- 172.21.15.116:0/966302928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c0c15a0f0 msgr2=0x7f5c0c0ba740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c10dee700 1 --2- 172.21.15.116:0/966302928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c0c15a0f0 0x7f5c0c0ba740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f5c0c0c1c80 tx=0x7f5c00006eb0).stop 2020-11-20T12:16:01.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c10dee700 1 -- 172.21.15.116:0/966302928 shutdown_connections 2020-11-20T12:16:01.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c10dee700 1 --2- 172.21.15.116:0/966302928 >> v2:172.21.15.173:3300/0 conn(0x7f5c0c0a3590 0x7f5c0c156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c10dee700 1 --2- 172.21.15.116:0/966302928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c0c15a0f0 0x7f5c0c0ba740 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c10dee700 1 -- 172.21.15.116:0/966302928 >> 172.21.15.116:0/966302928 conn(0x7f5c0c060c80 msgr2=0x7f5c0c0c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:01.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c10dee700 1 -- 172.21.15.116:0/966302928 shutdown_connections 2020-11-20T12:16:01.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.244+0000 7f5c10dee700 1 -- 172.21.15.116:0/966302928 wait complete. 2020-11-20T12:16:01.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c10dee700 1 Processor -- start 2020-11-20T12:16:01.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c10dee700 1 -- start start 2020-11-20T12:16:01.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c10dee700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c0c0a3590 0x7f5c0c15a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c10dee700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c0c0bac30 0x7f5c0c0915b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c10dee700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c0c15a0f0 0x7f5c0c0bcf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c10dee700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0bd590 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c10dee700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0bd830 con 0x7f5c0c0a3590 2020-11-20T12:16:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c10dee700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c0c0bdad0 con 0x7f5c0c0bac30 2020-11-20T12:16:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c0ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c0c15a0f0 0x7f5c0c0bcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c0ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c0c15a0f0 0x7f5c0c0bcf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42890/0 (socket says 172.21.15.116:42890) 2020-11-20T12:16:01.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c0ad9d700 1 -- 172.21.15.116:0/3063546077 learned_addr learned my addr 172.21.15.116:0/3063546077 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:01.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c0a59c700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c0c0a3590 0x7f5c0c15a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c09d9b700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c0c0bac30 0x7f5c0c0915b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c0ad9d700 1 -- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c0c0bac30 msgr2=0x7f5c0c0915b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c0ad9d700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c0c0bac30 0x7f5c0c0915b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c0ad9d700 1 -- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c0c0a3590 msgr2=0x7f5c0c15a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c0ad9d700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c0c0a3590 0x7f5c0c15a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c0ad9d700 1 -- 172.21.15.116:0/3063546077 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c0c0bdd70 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c0a59c700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c0c0a3590 0x7f5c0c15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:01.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c09d9b700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c0c0bac30 0x7f5c0c0915b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:01.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.248+0000 7f5c0ad9d700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c0c15a0f0 0x7f5c0c0bcf30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f5c0400ca60 tx=0x7f5c04007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:01.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.252+0000 7f5bfa7fc700 1 -- 172.21.15.116:0/3063546077 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400a8c0 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.252+0000 7f5bfa7fc700 1 -- 172.21.15.116:0/3063546077 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0400aa20 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.252+0000 7f5c10dee700 1 -- 172.21.15.116:0/3063546077 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c0c0a2e80 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.252+0000 7f5bfa7fc700 1 -- 172.21.15.116:0/3063546077 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400ada0 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.252+0000 7f5c10dee700 1 -- 172.21.15.116:0/3063546077 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c0c154700 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.252+0000 7f5c10dee700 1 -- 172.21.15.116:0/3063546077 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c0c0a1010 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.252+0000 7f5bfa7fc700 1 -- 172.21.15.116:0/3063546077 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5c04019030 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.252+0000 7f5bfa7fc700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5be00557f0 0x7f5be0057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.252+0000 7f5bfa7fc700 1 -- 172.21.15.116:0/3063546077 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5c0406df60 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.256+0000 7f5c0a59c700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5be00557f0 0x7f5be0057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.256+0000 7f5c0a59c700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5be00557f0 0x7f5be0057cb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f5bfc006fd0 tx=0x7f5bfc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:01.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.260+0000 7f5bfa7fc700 1 -- 172.21.15.116:0/3063546077 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5c0c0a1010 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.420+0000 7f5c10dee700 1 -- 172.21.15.116:0/3063546077 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c0c0a1010 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.420+0000 7f5bfa7fc700 1 -- 172.21.15.116:0/3063546077 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16887 (secure 0 0 0) 0x7f5c04025210 con 0x7f5c0c15a0f0 2020-11-20T12:16:01.423 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:01.424 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":22,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:00.619510+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:01.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 -- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5be00557f0 msgr2=0x7f5be0057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5be00557f0 0x7f5be0057cb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f5bfc006fd0 tx=0x7f5bfc008040).stop 2020-11-20T12:16:01.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 -- 172.21.15.116:0/3063546077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c0c15a0f0 msgr2=0x7f5c0c0bcf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c0c15a0f0 0x7f5c0c0bcf30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f5c0400ca60 tx=0x7f5c04007f40).stop 2020-11-20T12:16:01.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 -- 172.21.15.116:0/3063546077 shutdown_connections 2020-11-20T12:16:01.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5be00557f0 0x7f5be0057cb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c0c15a0f0 0x7f5c0c0bcf30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c0c0a3590 0x7f5c0c15a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 --2- 172.21.15.116:0/3063546077 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c0c0bac30 0x7f5c0c0915b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 -- 172.21.15.116:0/3063546077 >> 172.21.15.116:0/3063546077 conn(0x7f5c0c060c80 msgr2=0x7f5c0c0c2490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:01.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 -- 172.21.15.116:0/3063546077 shutdown_connections 2020-11-20T12:16:01.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.424+0000 7f5c10dee700 1 -- 172.21.15.116:0/3063546077 wait complete. 2020-11-20T12:16:01.452 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-20T12:16:01.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.636+0000 7efc3a14a700 1 Processor -- start 2020-11-20T12:16:01.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.636+0000 7efc3a14a700 1 -- start start 2020-11-20T12:16:01.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.636+0000 7efc3a14a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc34061250 0x7efc340616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.636+0000 7efc3a14a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc34152a80 con 0x7efc3413ad30 2020-11-20T12:16:01.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.636+0000 7efc3a14a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc34152540 con 0x7efc3413b110 2020-11-20T12:16:01.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.636+0000 7efc3a14a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc341527e0 con 0x7efc34061250 2020-11-20T12:16:01.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.636+0000 7efc39148700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7efc3413ad30 0x7efc340a7fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52328/0 (socket says 172.21.15.116:52328) 2020-11-20T12:16:01.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.636+0000 7efc39148700 1 -- 172.21.15.116:0/1383238128 learned_addr learned my addr 172.21.15.116:0/1383238128 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:01.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599609145 0 0) 0x7efc34152a80 con 0x7efc3413ad30 2020-11-20T12:16:01.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc1c003240 con 0x7efc3413ad30 2020-11-20T12:16:01.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4176122204 0 0) 0x7efc1c003240 con 0x7efc3413ad30 2020-11-20T12:16:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc34061250 msgr2=0x7efc340616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 --2- 172.21.15.116:0/1383238128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc34061250 0x7efc340616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 >> v1:172.21.15.173:6789/0 conn(0x7efc3413b110 legacy=0x7efc34151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc340ac910 con 0x7efc3413ad30 2020-11-20T12:16:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7efc300014e0 con 0x7efc3413ad30 2020-11-20T12:16:01.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 >> v1:172.21.15.116:6789/0 conn(0x7efc3413ad30 legacy=0x7efc340a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 --2- 172.21.15.116:0/1383238128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc34061250 0x7efc1c003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 --2- 172.21.15.116:0/1383238128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc1c003cc0 0x7efc1c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 --2- 172.21.15.116:0/1383238128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc1c0049d0 0x7efc1c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc341527e0 con 0x7efc1c0049d0 2020-11-20T12:16:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c004250 con 0x7efc1c003cc0 2020-11-20T12:16:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc1c00b730 con 0x7efc34061250 2020-11-20T12:16:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc39949700 1 --2- 172.21.15.116:0/1383238128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc1c0049d0 0x7efc1c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc38947700 1 --2- 172.21.15.116:0/1383238128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc1c003cc0 0x7efc1c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc39949700 1 -- 172.21.15.116:0/1383238128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc34061250 msgr2=0x7efc1c003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc39949700 1 --2- 172.21.15.116:0/1383238128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc34061250 0x7efc1c003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc39949700 1 -- 172.21.15.116:0/1383238128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc1c003cc0 msgr2=0x7efc1c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc39949700 1 --2- 172.21.15.116:0/1383238128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc1c003cc0 0x7efc1c0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc39949700 1 -- 172.21.15.116:0/1383238128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc1c003240 con 0x7efc1c0049d0 2020-11-20T12:16:01.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc39949700 1 --2- 172.21.15.116:0/1383238128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc1c0049d0 0x7efc1c004e50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7efc2c009c70 tx=0x7efc2c009220).ready entity=mon.0 client_cookie=f25168f9b443cf50 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c00bb60 con 0x7efc1c0049d0 2020-11-20T12:16:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc2c00bcc0 con 0x7efc1c0049d0 2020-11-20T12:16:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc23fff700 1 -- 172.21.15.116:0/1383238128 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc2c011650 con 0x7efc1c0049d0 2020-11-20T12:16:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc3a14a700 1 -- 172.21.15.116:0/1383238128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc1c0049d0 msgr2=0x7efc1c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc3a14a700 1 --2- 172.21.15.116:0/1383238128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc1c0049d0 0x7efc1c004e50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7efc2c009c70 tx=0x7efc2c009220).stop 2020-11-20T12:16:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.640+0000 7efc39949700 1 -- 172.21.15.116:0/1383238128 reap_dead start 2020-11-20T12:16:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.644+0000 7efc3a14a700 1 -- 172.21.15.116:0/1383238128 shutdown_connections 2020-11-20T12:16:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.644+0000 7efc3a14a700 1 -- 172.21.15.116:0/1383238128 >> 172.21.15.116:0/1383238128 conn(0x7efc340678a0 msgr2=0x7efc34067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.644+0000 7efc3a14a700 1 -- 172.21.15.116:0/1383238128 shutdown_connections 2020-11-20T12:16:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.644+0000 7efc3a14a700 1 -- 172.21.15.116:0/1383238128 wait complete. 2020-11-20T12:16:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.644+0000 7efc3a14a700 1 Processor -- start 2020-11-20T12:16:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.644+0000 7efc3a14a700 1 -- start start 2020-11-20T12:16:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc3a14a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc341381f0 0x7efc340a04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc3a14a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc341385d0 0x7efc34161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc3a14a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc3409f010 0x7efc340a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc3a14a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3409f590 con 0x7efc341381f0 2020-11-20T12:16:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc3a14a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3409f830 con 0x7efc3409f010 2020-11-20T12:16:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc3a14a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc3409fad0 con 0x7efc341385d0 2020-11-20T12:16:01.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc39148700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc341381f0 0x7efc340a04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc39148700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc341381f0 0x7efc340a04b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42908/0 (socket says 172.21.15.116:42908) 2020-11-20T12:16:01.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc38947700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc341385d0 0x7efc34161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc39148700 1 -- 172.21.15.116:0/2021076526 learned_addr learned my addr 172.21.15.116:0/2021076526 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:01.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc39949700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc3409f010 0x7efc340a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc39148700 1 -- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc341385d0 msgr2=0x7efc34161400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc39148700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc341385d0 0x7efc34161400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc39148700 1 -- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc3409f010 msgr2=0x7efc340a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:01.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc39148700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc3409f010 0x7efc340a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:01.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc39148700 1 -- 172.21.15.116:0/2021076526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc3409fd70 con 0x7efc341381f0 2020-11-20T12:16:01.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc39148700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc341381f0 0x7efc340a04b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7efc3000ca60 tx=0x7efc30001b20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:01.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc217fa700 1 -- 172.21.15.116:0/2021076526 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30007170 con 0x7efc341381f0 2020-11-20T12:16:01.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc217fa700 1 -- 172.21.15.116:0/2021076526 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc300072d0 con 0x7efc341381f0 2020-11-20T12:16:01.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc217fa700 1 -- 172.21.15.116:0/2021076526 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc30004a20 con 0x7efc341381f0 2020-11-20T12:16:01.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc3a14a700 1 -- 172.21.15.116:0/2021076526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc340a0010 con 0x7efc341381f0 2020-11-20T12:16:01.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc3a14a700 1 -- 172.21.15.116:0/2021076526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc340a1050 con 0x7efc341381f0 2020-11-20T12:16:01.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.648+0000 7efc3a14a700 1 -- 172.21.15.116:0/2021076526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc34044f60 con 0x7efc341381f0 2020-11-20T12:16:01.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.652+0000 7efc217fa700 1 -- 172.21.15.116:0/2021076526 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7efc30010070 con 0x7efc341381f0 2020-11-20T12:16:01.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.652+0000 7efc217fa700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc0c0557f0 0x7efc0c057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:01.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.652+0000 7efc217fa700 1 -- 172.21.15.116:0/2021076526 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efc3006e110 con 0x7efc341381f0 2020-11-20T12:16:01.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.652+0000 7efc38947700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc0c0557f0 0x7efc0c057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:01.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.652+0000 7efc38947700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc0c0557f0 0x7efc0c057cb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7efc34163ef0 tx=0x7efc2400c040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:01.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.656+0000 7efc217fa700 1 -- 172.21.15.116:0/2021076526 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efc30043fb0 con 0x7efc341381f0 2020-11-20T12:16:01.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:01.872+0000 7efc3a14a700 1 -- 172.21.15.116:0/2021076526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efc34069320 con 0x7efc341381f0 2020-11-20T12:16:02.179 INFO:tasks.ceph.mds.a.smithi116.stderr:2020-11-20T12:16:02.176+0000 7f3e85091700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:16:02.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.644+0000 7efc217fa700 1 -- 172.21.15.116:0/2021076526 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v6) v1 ==== 118+0+0 (secure 0 0 0) 0x7efc300475d0 con 0x7efc341381f0 2020-11-20T12:16:02.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.644+0000 7efc3a14a700 1 -- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc0c0557f0 msgr2=0x7efc0c057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:02.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.644+0000 7efc3a14a700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc0c0557f0 0x7efc0c057cb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7efc34163ef0 tx=0x7efc2400c040).stop 2020-11-20T12:16:02.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.644+0000 7efc3a14a700 1 -- 172.21.15.116:0/2021076526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc341381f0 msgr2=0x7efc340a04b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:02.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.644+0000 7efc3a14a700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc341381f0 0x7efc340a04b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7efc3000ca60 tx=0x7efc30001b20).stop 2020-11-20T12:16:02.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.648+0000 7efc3a14a700 1 -- 172.21.15.116:0/2021076526 shutdown_connections 2020-11-20T12:16:02.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.648+0000 7efc3a14a700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc0c0557f0 0x7efc0c057cb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:02.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.648+0000 7efc3a14a700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc3409f010 0x7efc340a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:02.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.648+0000 7efc3a14a700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc341381f0 0x7efc340a04b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:02.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.648+0000 7efc3a14a700 1 --2- 172.21.15.116:0/2021076526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc341385d0 0x7efc34161400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:02.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.648+0000 7efc3a14a700 1 -- 172.21.15.116:0/2021076526 >> 172.21.15.116:0/2021076526 conn(0x7efc340678a0 msgr2=0x7efc340ade70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:02.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.648+0000 7efc3a14a700 1 -- 172.21.15.116:0/2021076526 shutdown_connections 2020-11-20T12:16:02.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.648+0000 7efc3a14a700 1 -- 172.21.15.116:0/2021076526 wait complete. 2020-11-20T12:16:02.678 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:16:02.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.860+0000 7f6d6b59e700 1 Processor -- start 2020-11-20T12:16:02.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.864+0000 7f6d6b59e700 1 -- start start 2020-11-20T12:16:02.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.864+0000 7f6d6b59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6d6c14d150 0x7f6d6c063eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:02.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.864+0000 7f6d6b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c067500 0x7f6d6c0b9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:02.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.864+0000 7f6d6b59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d6c062f90 con 0x7f6d6c067070 2020-11-20T12:16:02.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.864+0000 7f6d6b59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d6c063230 con 0x7f6d6c14d150 2020-11-20T12:16:02.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.864+0000 7f6d6b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c0634d0 con 0x7f6d6c067500 2020-11-20T12:16:02.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.864+0000 7f6d69d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6d6c067070 0x7f6d6c061180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52352/0 (socket says 172.21.15.116:52352) 2020-11-20T12:16:02.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.864+0000 7f6d69d9b700 1 -- 172.21.15.116:0/1156241893 learned_addr learned my addr 172.21.15.116:0/1156241893 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:02.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.864+0000 7f6d6ad9d700 1 --2- 172.21.15.116:0/1156241893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c067500 0x7f6d6c0b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.864+0000 7f6d6a59c700 1 --2- 172.21.15.116:0/1156241893 >> v2:172.21.15.173:3300/0 conn(0x7f6d6c14d150 0x7f6d6c063eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6959a700 1 -- 172.21.15.116:0/1156241893 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393915592 0 0) 0x7f6d6c062f90 con 0x7f6d6c067070 2020-11-20T12:16:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6959a700 1 -- 172.21.15.116:0/1156241893 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d50003240 con 0x7f6d6c067070 2020-11-20T12:16:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6ad9d700 1 -- 172.21.15.116:0/1156241893 >> v2:172.21.15.173:3300/0 conn(0x7f6d6c14d150 msgr2=0x7f6d6c063eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:02.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6ad9d700 1 --2- 172.21.15.116:0/1156241893 >> v2:172.21.15.173:3300/0 conn(0x7f6d6c14d150 0x7f6d6c063eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6ad9d700 1 -- 172.21.15.116:0/1156241893 >> v1:172.21.15.116:6789/0 conn(0x7f6d6c067070 legacy=0x7f6d6c061180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6ad9d700 1 -- 172.21.15.116:0/1156241893 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c063770 con 0x7f6d6c067500 2020-11-20T12:16:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6ad9d700 1 --2- 172.21.15.116:0/1156241893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c067500 0x7f6d6c0b9850 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f6d64009c90 tx=0x7f6d640091d0).ready entity=mon.2 client_cookie=9e72daa59318940f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:02.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6959a700 1 -- 172.21.15.116:0/1156241893 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400b9c0 con 0x7f6d6c067500 2020-11-20T12:16:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6959a700 1 -- 172.21.15.116:0/1156241893 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6400bb20 con 0x7f6d6c067500 2020-11-20T12:16:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6959a700 1 -- 172.21.15.116:0/1156241893 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6400bea0 con 0x7f6d6c067500 2020-11-20T12:16:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6b59e700 1 -- 172.21.15.116:0/1156241893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c067500 msgr2=0x7f6d6c0b9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6b59e700 1 --2- 172.21.15.116:0/1156241893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c067500 0x7f6d6c0b9850 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f6d64009c90 tx=0x7f6d640091d0).stop 2020-11-20T12:16:02.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6b59e700 1 -- 172.21.15.116:0/1156241893 shutdown_connections 2020-11-20T12:16:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6b59e700 1 --2- 172.21.15.116:0/1156241893 >> v2:172.21.15.173:3300/0 conn(0x7f6d6c14d150 0x7f6d6c063eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6b59e700 1 --2- 172.21.15.116:0/1156241893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c067500 0x7f6d6c0b9850 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6b59e700 1 -- 172.21.15.116:0/1156241893 >> 172.21.15.116:0/1156241893 conn(0x7f6d6c0c1e60 msgr2=0x7f6d6c066960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6b59e700 1 -- 172.21.15.116:0/1156241893 shutdown_connections 2020-11-20T12:16:02.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.868+0000 7f6d6b59e700 1 -- 172.21.15.116:0/1156241893 wait complete. 2020-11-20T12:16:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6b59e700 1 Processor -- start 2020-11-20T12:16:02.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6b59e700 1 -- start start 2020-11-20T12:16:02.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d6c067070 0x7f6d6c165020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:02.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d6c067500 0x7f6d6c14bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:02.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c14d150 0x7f6d6c154960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:02.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c15eab0 con 0x7f6d6c067070 2020-11-20T12:16:02.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c165510 con 0x7f6d6c067500 2020-11-20T12:16:02.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6c14c240 con 0x7f6d6c14d150 2020-11-20T12:16:02.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d6c067070 0x7f6d6c165020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:02.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d6c067070 0x7f6d6c165020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42928/0 (socket says 172.21.15.116:42928) 2020-11-20T12:16:02.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6a59c700 1 -- 172.21.15.116:0/3354430023 learned_addr learned my addr 172.21.15.116:0/3354430023 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:02.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6ad9d700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c14d150 0x7f6d6c154960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d69d9b700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d6c067500 0x7f6d6c14bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6a59c700 1 -- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c14d150 msgr2=0x7f6d6c154960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6a59c700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c14d150 0x7f6d6c154960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6a59c700 1 -- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d6c067500 msgr2=0x7f6d6c14bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6a59c700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d6c067500 0x7f6d6c14bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6a59c700 1 -- 172.21.15.116:0/3354430023 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d6c161460 con 0x7f6d6c067070 2020-11-20T12:16:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d69d9b700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d6c067500 0x7f6d6c14bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6ad9d700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c14d150 0x7f6d6c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6a59c700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d6c067070 0x7f6d6c165020 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f6d5c00ebd0 tx=0x7f6d5c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d5a7fc700 1 -- 172.21.15.116:0/3354430023 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00b9a0 con 0x7f6d6c067070 2020-11-20T12:16:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d5a7fc700 1 -- 172.21.15.116:0/3354430023 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d5c00bb00 con 0x7f6d6c067070 2020-11-20T12:16:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d5a7fc700 1 -- 172.21.15.116:0/3354430023 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5c00be80 con 0x7f6d6c067070 2020-11-20T12:16:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6b59e700 1 -- 172.21.15.116:0/3354430023 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6c161700 con 0x7f6d6c067070 2020-11-20T12:16:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.872+0000 7f6d6b59e700 1 -- 172.21.15.116:0/3354430023 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6c161c40 con 0x7f6d6c067070 2020-11-20T12:16:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.876+0000 7f6d6b59e700 1 -- 172.21.15.116:0/3354430023 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6c044f60 con 0x7f6d6c067070 2020-11-20T12:16:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.876+0000 7f6d5a7fc700 1 -- 172.21.15.116:0/3354430023 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6d5c012600 con 0x7f6d6c067070 2020-11-20T12:16:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.876+0000 7f6d5a7fc700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6d400557c0 0x7f6d40057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.876+0000 7f6d5a7fc700 1 -- 172.21.15.116:0/3354430023 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6d5c06eb00 con 0x7f6d6c067070 2020-11-20T12:16:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.876+0000 7f6d69d9b700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6d400557c0 0x7f6d40057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.880+0000 7f6d69d9b700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6d400557c0 0x7f6d40057c80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f6d6c0a9db0 tx=0x7f6d60009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:02.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:02.884+0000 7f6d5a7fc700 1 -- 172.21.15.116:0/3354430023 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6d5c0449a0 con 0x7f6d6c067070 2020-11-20T12:16:03.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.088+0000 7f6d6b59e700 1 -- 172.21.15.116:0/3354430023 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6d6c067a40 con 0x7f6d6c067070 2020-11-20T12:16:03.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.088+0000 7f6d5a7fc700 1 -- 172.21.15.116:0/3354430023 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2846 (secure 0 0 0) 0x7f6d5c047fc0 con 0x7f6d6c067070 2020-11-20T12:16:03.091 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:03.091 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/2968741298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":2968741298},{"type":"v1","addr":"172.21.15.116:6835","nonce":2968741298}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1214743116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1214743116},{"type":"v1","addr":"172.21.15.173:6836","nonce":1214743116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1424291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1424291148},{"type":"v1","addr":"172.21.15.173:6837","nonce":1424291148}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:00.619934+0000","modified":"2020-11-20T12:16:02.636079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.116:6834/3819756176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3819756176},{"type":"v1","addr":"172.21.15.116:6834","nonce":3819756176}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:16:03.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 -- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6d400557c0 msgr2=0x7f6d40057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6d400557c0 0x7f6d40057c80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f6d6c0a9db0 tx=0x7f6d60009040).stop 2020-11-20T12:16:03.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 -- 172.21.15.116:0/3354430023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d6c067070 msgr2=0x7f6d6c165020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d6c067070 0x7f6d6c165020 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f6d5c00ebd0 tx=0x7f6d5c006df0).stop 2020-11-20T12:16:03.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 -- 172.21.15.116:0/3354430023 shutdown_connections 2020-11-20T12:16:03.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6d400557c0 0x7f6d40057c80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d6c14d150 0x7f6d6c154960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d6c067070 0x7f6d6c165020 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 --2- 172.21.15.116:0/3354430023 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d6c067500 0x7f6d6c14bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 -- 172.21.15.116:0/3354430023 >> 172.21.15.116:0/3354430023 conn(0x7f6d6c0c1e60 msgr2=0x7f6d6c1517d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:03.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 -- 172.21.15.116:0/3354430023 shutdown_connections 2020-11-20T12:16:03.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.092+0000 7f6d6b59e700 1 -- 172.21.15.116:0/3354430023 wait complete. 2020-11-20T12:16:03.096 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 6 2020-11-20T12:16:03.117 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.284+0000 7f66eae12700 1 Processor -- start 2020-11-20T12:16:03.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66eae12700 1 -- start start 2020-11-20T12:16:03.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66eae12700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e413f5b0 0x7f66e40c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66eae12700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f66e40c44a0 0x7f66e4066090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66eae12700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e40a5680 con 0x7f66e413f1d0 2020-11-20T12:16:03.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66eae12700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e40a42e0 con 0x7f66e40c44a0 2020-11-20T12:16:03.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66eae12700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e40a58e0 con 0x7f66e413f5b0 2020-11-20T12:16:03.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66e9e10700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f66e413f1d0 0x7f66e40a3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:48596/0 (socket says 172.21.15.116:48596) 2020-11-20T12:16:03.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66e9e10700 1 -- 172.21.15.116:0/585534976 learned_addr learned my addr 172.21.15.116:0/585534976 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:03.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66e960f700 1 --2- 172.21.15.116:0/585534976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e413f5b0 0x7f66e40c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66ea611700 1 --2- 172.21.15.116:0/585534976 >> v2:172.21.15.173:3300/0 conn(0x7f66e40c44a0 0x7f66e4066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66e8e0e700 1 -- 172.21.15.116:0/585534976 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984361603 0 0) 0x7f66e40a5680 con 0x7f66e413f1d0 2020-11-20T12:16:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.288+0000 7f66e8e0e700 1 -- 172.21.15.116:0/585534976 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66cc003240 con 0x7f66e413f1d0 2020-11-20T12:16:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66e960f700 1 -- 172.21.15.116:0/585534976 >> v2:172.21.15.173:3300/0 conn(0x7f66e40c44a0 msgr2=0x7f66e4066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66e960f700 1 --2- 172.21.15.116:0/585534976 >> v2:172.21.15.173:3300/0 conn(0x7f66e40c44a0 0x7f66e4066090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66e960f700 1 -- 172.21.15.116:0/585534976 >> v1:172.21.15.173:6789/0 conn(0x7f66e413f1d0 legacy=0x7f66e40a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66e960f700 1 -- 172.21.15.116:0/585534976 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e40a5b80 con 0x7f66e413f5b0 2020-11-20T12:16:03.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66e960f700 1 --2- 172.21.15.116:0/585534976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e413f5b0 0x7f66e40c3fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f66d400b9a0 tx=0x7f66d4006df0).ready entity=mon.2 client_cookie=101afcf5f73f5348 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:03.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66e8e0e700 1 -- 172.21.15.116:0/585534976 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4009680 con 0x7f66e413f5b0 2020-11-20T12:16:03.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66e8e0e700 1 -- 172.21.15.116:0/585534976 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d40097e0 con 0x7f66e413f5b0 2020-11-20T12:16:03.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66e8e0e700 1 -- 172.21.15.116:0/585534976 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4009b60 con 0x7f66e413f5b0 2020-11-20T12:16:03.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66eae12700 1 -- 172.21.15.116:0/585534976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e413f5b0 msgr2=0x7f66e40c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66eae12700 1 --2- 172.21.15.116:0/585534976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e413f5b0 0x7f66e40c3fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f66d400b9a0 tx=0x7f66d4006df0).stop 2020-11-20T12:16:03.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66eae12700 1 -- 172.21.15.116:0/585534976 shutdown_connections 2020-11-20T12:16:03.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66eae12700 1 --2- 172.21.15.116:0/585534976 >> v2:172.21.15.173:3300/0 conn(0x7f66e40c44a0 0x7f66e4066090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66eae12700 1 --2- 172.21.15.116:0/585534976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e413f5b0 0x7f66e40c3fb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66eae12700 1 -- 172.21.15.116:0/585534976 >> 172.21.15.116:0/585534976 conn(0x7f66e4062520 msgr2=0x7f66e4062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:03.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66eae12700 1 -- 172.21.15.116:0/585534976 shutdown_connections 2020-11-20T12:16:03.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.292+0000 7f66eae12700 1 -- 172.21.15.116:0/585534976 wait complete. 2020-11-20T12:16:03.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.296+0000 7f66eae12700 1 Processor -- start 2020-11-20T12:16:03.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.296+0000 7f66eae12700 1 -- start start 2020-11-20T12:16:03.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.296+0000 7f66eae12700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e40c44a0 0x7f66e413e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.296+0000 7f66eae12700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e413f1d0 0x7f66e4159da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.296+0000 7f66eae12700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e413f5b0 0x7f66e41685a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.296+0000 7f66eae12700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e415a800 con 0x7f66e413f1d0 2020-11-20T12:16:03.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.296+0000 7f66eae12700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e415aaa0 con 0x7f66e413f5b0 2020-11-20T12:16:03.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.296+0000 7f66eae12700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e409d610 con 0x7f66e40c44a0 2020-11-20T12:16:03.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.296+0000 7f66e960f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e413f1d0 0x7f66e4159da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.296+0000 7f66e960f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e413f1d0 0x7f66e4159da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42940/0 (socket says 172.21.15.116:42940) 2020-11-20T12:16:03.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.296+0000 7f66e960f700 1 -- 172.21.15.116:0/1087026877 learned_addr learned my addr 172.21.15.116:0/1087026877 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:03.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66ea611700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e413f5b0 0x7f66e41685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66e9e10700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e40c44a0 0x7f66e413e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66e960f700 1 -- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e40c44a0 msgr2=0x7f66e413e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66e960f700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e40c44a0 0x7f66e413e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66e960f700 1 -- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e413f5b0 msgr2=0x7f66e41685a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66e960f700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e413f5b0 0x7f66e41685a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66e960f700 1 -- 172.21.15.116:0/1087026877 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e409d8b0 con 0x7f66e413f1d0 2020-11-20T12:16:03.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66e9e10700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e40c44a0 0x7f66e413e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:03.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66e960f700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e413f1d0 0x7f66e4159da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f66e4068580 tx=0x7f66d4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:03.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66d1ffb700 1 -- 172.21.15.116:0/1087026877 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d4011c40 con 0x7f66e413f1d0 2020-11-20T12:16:03.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66d1ffb700 1 -- 172.21.15.116:0/1087026877 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d4011da0 con 0x7f66e413f1d0 2020-11-20T12:16:03.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66eae12700 1 -- 172.21.15.116:0/1087026877 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e413e790 con 0x7f66e413f1d0 2020-11-20T12:16:03.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66d1ffb700 1 -- 172.21.15.116:0/1087026877 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d401e670 con 0x7f66e413f1d0 2020-11-20T12:16:03.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66eae12700 1 -- 172.21.15.116:0/1087026877 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e415a290 con 0x7f66e413f1d0 2020-11-20T12:16:03.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.300+0000 7f66eae12700 1 -- 172.21.15.116:0/1087026877 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e409da40 con 0x7f66e413f1d0 2020-11-20T12:16:03.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.304+0000 7f66d1ffb700 1 -- 172.21.15.116:0/1087026877 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f66d4025030 con 0x7f66e413f1d0 2020-11-20T12:16:03.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.304+0000 7f66d1ffb700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66bc0557c0 0x7f66bc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.304+0000 7f66d1ffb700 1 -- 172.21.15.116:0/1087026877 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f66d406fe30 con 0x7f66e413f1d0 2020-11-20T12:16:03.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.304+0000 7f66e9e10700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66bc0557c0 0x7f66bc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.304+0000 7f66e9e10700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66bc0557c0 0x7f66bc057c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f66e00055a0 tx=0x7f66e0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:03.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.308+0000 7f66d1ffb700 1 -- 172.21.15.116:0/1087026877 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f66d4045e00 con 0x7f66e413f1d0 2020-11-20T12:16:03.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.472+0000 7f66eae12700 1 -- 172.21.15.116:0/1087026877 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66e415ac60 con 0x7f66e413f1d0 2020-11-20T12:16:03.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.472+0000 7f66d1ffb700 1 -- 172.21.15.116:0/1087026877 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f66d4031200 con 0x7f66e413f1d0 2020-11-20T12:16:03.475 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:03.475 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:03.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 -- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66bc0557c0 msgr2=0x7f66bc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66bc0557c0 0x7f66bc057c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f66e00055a0 tx=0x7f66e0009040).stop 2020-11-20T12:16:03.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 -- 172.21.15.116:0/1087026877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e413f1d0 msgr2=0x7f66e4159da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e413f1d0 0x7f66e4159da0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f66e4068580 tx=0x7f66d4015000).stop 2020-11-20T12:16:03.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 -- 172.21.15.116:0/1087026877 shutdown_connections 2020-11-20T12:16:03.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66bc0557c0 0x7f66bc057c80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e413f5b0 0x7f66e41685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e40c44a0 0x7f66e413e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 --2- 172.21.15.116:0/1087026877 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e413f1d0 0x7f66e4159da0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 -- 172.21.15.116:0/1087026877 >> 172.21.15.116:0/1087026877 conn(0x7f66e4062520 msgr2=0x7f66e40c30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:03.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 -- 172.21.15.116:0/1087026877 shutdown_connections 2020-11-20T12:16:03.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.476+0000 7f66eae12700 1 -- 172.21.15.116:0/1087026877 wait complete. 2020-11-20T12:16:03.504 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:03.505 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2020-11-20T12:16:03.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.676+0000 7f7da9bd4700 1 Processor -- start 2020-11-20T12:16:03.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da9bd4700 1 -- start start 2020-11-20T12:16:03.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da9bd4700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7da40ba140 0x7f7da40c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da9bd4700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da40c2da0 con 0x7f7da40baa30 2020-11-20T12:16:03.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da9bd4700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7da40c3040 con 0x7f7da40bae10 2020-11-20T12:16:03.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da9bd4700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da40c32e0 con 0x7f7da40ba140 2020-11-20T12:16:03.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da37fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7da40baa30 0x7f7da4066510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52380/0 (socket says 172.21.15.116:52380) 2020-11-20T12:16:03.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da37fe700 1 -- 172.21.15.116:0/530946051 learned_addr learned my addr 172.21.15.116:0/530946051 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:03.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da3fff700 1 --2- 172.21.15.116:0/530946051 >> v2:172.21.15.173:3300/0 conn(0x7f7da40ba140 0x7f7da40c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174953861 0 0) 0x7f7da40c2da0 con 0x7f7da40baa30 2020-11-20T12:16:03.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d90003240 con 0x7f7da40baa30 2020-11-20T12:16:03.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 220283468 0 0) 0x7f7d90003240 con 0x7f7da40baa30 2020-11-20T12:16:03.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 >> v2:172.21.15.173:3300/0 conn(0x7f7da40ba140 msgr2=0x7f7da40c27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:03.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da27fc700 1 --2- 172.21.15.116:0/530946051 >> v2:172.21.15.173:3300/0 conn(0x7f7da40ba140 0x7f7da40c27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 >> v1:172.21.15.173:6789/0 conn(0x7f7da40bae10 legacy=0x7f7da40c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da40c3580 con 0x7f7da40baa30 2020-11-20T12:16:03.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.680+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f7d940014e0 con 0x7f7da40baa30 2020-11-20T12:16:03.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 >> v1:172.21.15.116:6789/0 conn(0x7f7da40baa30 legacy=0x7f7da4066510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da27fc700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d90003730 0x7f7d90003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da27fc700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900044e0 0x7f7d90004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da27fc700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d90004e50 0x7f7d9000fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40c32e0 con 0x7f7d900044e0 2020-11-20T12:16:03.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90004260 con 0x7f7d90004e50 2020-11-20T12:16:03.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90010030 con 0x7f7d90003730 2020-11-20T12:16:03.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da3fff700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d90004e50 0x7f7d9000fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da37fe700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d90003730 0x7f7d90003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da2ffd700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900044e0 0x7f7d90004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da2ffd700 1 -- 172.21.15.116:0/530946051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d90003730 msgr2=0x7f7d90003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:03.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da2ffd700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d90003730 0x7f7d90003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da2ffd700 1 -- 172.21.15.116:0/530946051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d90004e50 msgr2=0x7f7d9000fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:03.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da2ffd700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d90004e50 0x7f7d9000fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da2ffd700 1 -- 172.21.15.116:0/530946051 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d90003240 con 0x7f7d900044e0 2020-11-20T12:16:03.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da3fff700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d90004e50 0x7f7d9000fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:03.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da3fff700 1 -- 172.21.15.116:0/530946051 reap_dead start 2020-11-20T12:16:03.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da2ffd700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900044e0 0x7f7d90004960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f7d98001920 tx=0x7f7d98007b90).ready entity=mon.0 client_cookie=77190ea2bfbd6687 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:03.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d98007020 con 0x7f7d900044e0 2020-11-20T12:16:03.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d98007180 con 0x7f7d900044e0 2020-11-20T12:16:03.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da27fc700 1 -- 172.21.15.116:0/530946051 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9800c830 con 0x7f7d900044e0 2020-11-20T12:16:03.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da9bd4700 1 -- 172.21.15.116:0/530946051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900044e0 msgr2=0x7f7d90004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da9bd4700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900044e0 0x7f7d90004960 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f7d98001920 tx=0x7f7d98007b90).stop 2020-11-20T12:16:03.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da9bd4700 1 -- 172.21.15.116:0/530946051 shutdown_connections 2020-11-20T12:16:03.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da9bd4700 1 --2- 172.21.15.116:0/530946051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d900044e0 0x7f7d90004960 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da9bd4700 1 -- 172.21.15.116:0/530946051 >> 172.21.15.116:0/530946051 conn(0x7f7da4060c80 msgr2=0x7f7da40c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:03.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.684+0000 7f7da9bd4700 1 -- 172.21.15.116:0/530946051 shutdown_connections 2020-11-20T12:16:03.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.688+0000 7f7da9bd4700 1 -- 172.21.15.116:0/530946051 wait complete. 2020-11-20T12:16:03.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.688+0000 7f7da9bd4700 1 Processor -- start 2020-11-20T12:16:03.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.688+0000 7f7da9bd4700 1 -- start start 2020-11-20T12:16:03.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da9bd4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d900044e0 0x7f7da40a61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da9bd4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7da40bf780 0x7f7da40bfc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da9bd4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7da415cf30 0x7f7da415d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da9bd4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da415dc40 con 0x7f7da415cf30 2020-11-20T12:16:03.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da9bd4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40bd360 con 0x7f7d900044e0 2020-11-20T12:16:03.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da9bd4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40bd600 con 0x7f7da40bf780 2020-11-20T12:16:03.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7da415cf30 0x7f7da415d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7da415cf30 0x7f7da415d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42962/0 (socket says 172.21.15.116:42962) 2020-11-20T12:16:03.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da3fff700 1 -- 172.21.15.116:0/335026637 learned_addr learned my addr 172.21.15.116:0/335026637 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:03.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da2ffd700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7da40bf780 0x7f7da40bfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da37fe700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d900044e0 0x7f7da40a61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da3fff700 1 -- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7da40bf780 msgr2=0x7f7da40bfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da3fff700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7da40bf780 0x7f7da40bfc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da3fff700 1 -- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d900044e0 msgr2=0x7f7da40a61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da3fff700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d900044e0 0x7f7da40a61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da3fff700 1 -- 172.21.15.116:0/335026637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da40bd8a0 con 0x7f7da415cf30 2020-11-20T12:16:03.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da37fe700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d900044e0 0x7f7da40a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:03.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da3fff700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7da415cf30 0x7f7da415d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7d9c009740 tx=0x7f7d9c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:03.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7d837fe700 1 -- 172.21.15.116:0/335026637 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c00fd10 con 0x7f7da415cf30 2020-11-20T12:16:03.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7d837fe700 1 -- 172.21.15.116:0/335026637 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9c00fe70 con 0x7f7da415cf30 2020-11-20T12:16:03.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da9bd4700 1 -- 172.21.15.116:0/335026637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da40bdb40 con 0x7f7da415cf30 2020-11-20T12:16:03.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7d837fe700 1 -- 172.21.15.116:0/335026637 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9c014670 con 0x7f7da415cf30 2020-11-20T12:16:03.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da9bd4700 1 -- 172.21.15.116:0/335026637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da415d880 con 0x7f7da415cf30 2020-11-20T12:16:03.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.692+0000 7f7da9bd4700 1 -- 172.21.15.116:0/335026637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da40c00f0 con 0x7f7da415cf30 2020-11-20T12:16:03.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.696+0000 7f7d837fe700 1 -- 172.21.15.116:0/335026637 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7d9c023070 con 0x7f7da415cf30 2020-11-20T12:16:03.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.696+0000 7f7d837fe700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7d7c0557c0 0x7f7d7c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:03.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.696+0000 7f7d837fe700 1 -- 172.21.15.116:0/335026637 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7d9c06f0c0 con 0x7f7da415cf30 2020-11-20T12:16:03.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.696+0000 7f7da37fe700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7d7c0557c0 0x7f7d7c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:03.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.696+0000 7f7da37fe700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7d7c0557c0 0x7f7d7c057c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7da405e230 tx=0x7f7d94009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:03.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.704+0000 7f7d837fe700 1 -- 172.21.15.116:0/335026637 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7d9c044f60 con 0x7f7da415cf30 2020-11-20T12:16:03.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.860+0000 7f7da9bd4700 1 -- 172.21.15.116:0/335026637 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7da405ca30 con 0x7f7da415cf30 2020-11-20T12:16:03.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.860+0000 7f7d837fe700 1 -- 172.21.15.116:0/335026637 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f7d9c021070 con 0x7f7da415cf30 2020-11-20T12:16:03.864 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:03.864 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:03.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.864+0000 7f7da9bd4700 1 -- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7d7c0557c0 msgr2=0x7f7d7c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.864+0000 7f7da9bd4700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7d7c0557c0 0x7f7d7c057c80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f7da405e230 tx=0x7f7d94009040).stop 2020-11-20T12:16:03.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.864+0000 7f7da9bd4700 1 -- 172.21.15.116:0/335026637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7da415cf30 msgr2=0x7f7da415d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:03.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.864+0000 7f7da9bd4700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7da415cf30 0x7f7da415d390 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f7d9c009740 tx=0x7f7d9c0091d0).stop 2020-11-20T12:16:03.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.864+0000 7f7da9bd4700 1 -- 172.21.15.116:0/335026637 shutdown_connections 2020-11-20T12:16:03.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.864+0000 7f7da9bd4700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7d7c0557c0 0x7f7d7c057c80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.864+0000 7f7da9bd4700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7da415cf30 0x7f7da415d390 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.864+0000 7f7da9bd4700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d900044e0 0x7f7da40a61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.864+0000 7f7da9bd4700 1 --2- 172.21.15.116:0/335026637 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7da40bf780 0x7f7da40bfc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:03.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.864+0000 7f7da9bd4700 1 -- 172.21.15.116:0/335026637 >> 172.21.15.116:0/335026637 conn(0x7f7da4060c80 msgr2=0x7f7da40c3be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:03.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.864+0000 7f7da9bd4700 1 -- 172.21.15.116:0/335026637 shutdown_connections 2020-11-20T12:16:03.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:03.868+0000 7f7da9bd4700 1 -- 172.21.15.116:0/335026637 wait complete. 2020-11-20T12:16:03.895 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:15:59.065285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:15:59.602311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:16:03.896 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.072+0000 7f522b59e700 1 Processor -- start 2020-11-20T12:16:04.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.072+0000 7f522b59e700 1 -- start start 2020-11-20T12:16:04.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522b59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f522c13ad30 0x7f522c0a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f522c13b110 0x7f522c151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522b59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f522c151fa0 0x7f522c061250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522b59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522c152a80 con 0x7f522c13ad30 2020-11-20T12:16:04.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522b59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f522c152540 con 0x7f522c151fa0 2020-11-20T12:16:04.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c1527e0 con 0x7f522c13b110 2020-11-20T12:16:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f522c13ad30 0x7f522c0a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f5229d9b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f522c13b110 0x7f522c151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f522c13ad30 0x7f522c0a7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42966/0 (socket says 172.21.15.116:42966) 2020-11-20T12:16:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f5229d9b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f522c13b110 0x7f522c151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:51920/0 (socket says 172.21.15.116:51920) 2020-11-20T12:16:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522a59c700 1 -- 172.21.15.116:0/500145037 learned_addr learned my addr 172.21.15.116:0/500145037 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522ad9d700 1 --2- 172.21.15.116:0/500145037 >> v2:172.21.15.173:3300/0 conn(0x7f522c151fa0 0x7f522c061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522a59c700 1 -- 172.21.15.116:0/500145037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f522c13b110 msgr2=0x7f522c151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:04.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522a59c700 1 --2- 172.21.15.116:0/500145037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f522c13b110 0x7f522c151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522a59c700 1 -- 172.21.15.116:0/500145037 >> v2:172.21.15.173:3300/0 conn(0x7f522c151fa0 msgr2=0x7f522c061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:04.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522a59c700 1 --2- 172.21.15.116:0/500145037 >> v2:172.21.15.173:3300/0 conn(0x7f522c151fa0 0x7f522c061250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522a59c700 1 -- 172.21.15.116:0/500145037 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c0ac910 con 0x7f522c13ad30 2020-11-20T12:16:04.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522ad9d700 1 --2- 172.21.15.116:0/500145037 >> v2:172.21.15.173:3300/0 conn(0x7f522c151fa0 0x7f522c061250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:04.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522a59c700 1 --2- 172.21.15.116:0/500145037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f522c13ad30 0x7f522c0a7fb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f522c0ab8b0 tx=0x7f521c006eb0).ready entity=mon.0 client_cookie=309954f202cb191a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522959a700 1 -- 172.21.15.116:0/500145037 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c0097b0 con 0x7f522c13ad30 2020-11-20T12:16:04.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522959a700 1 -- 172.21.15.116:0/500145037 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c009910 con 0x7f522c13ad30 2020-11-20T12:16:04.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.076+0000 7f522959a700 1 -- 172.21.15.116:0/500145037 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c009c90 con 0x7f522c13ad30 2020-11-20T12:16:04.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.080+0000 7f522b59e700 1 -- 172.21.15.116:0/500145037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f522c13ad30 msgr2=0x7f522c0a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.080+0000 7f522b59e700 1 --2- 172.21.15.116:0/500145037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f522c13ad30 0x7f522c0a7fb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f522c0ab8b0 tx=0x7f521c006eb0).stop 2020-11-20T12:16:04.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.080+0000 7f522b59e700 1 -- 172.21.15.116:0/500145037 shutdown_connections 2020-11-20T12:16:04.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.080+0000 7f522b59e700 1 --2- 172.21.15.116:0/500145037 >> v2:172.21.15.173:3300/0 conn(0x7f522c151fa0 0x7f522c061250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.080+0000 7f522b59e700 1 --2- 172.21.15.116:0/500145037 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f522c13ad30 0x7f522c0a7fb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.080+0000 7f522b59e700 1 --2- 172.21.15.116:0/500145037 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f522c13b110 0x7f522c151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.080+0000 7f522b59e700 1 -- 172.21.15.116:0/500145037 >> 172.21.15.116:0/500145037 conn(0x7f522c0678a0 msgr2=0x7f522c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:04.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.080+0000 7f522b59e700 1 -- 172.21.15.116:0/500145037 shutdown_connections 2020-11-20T12:16:04.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.080+0000 7f522b59e700 1 -- 172.21.15.116:0/500145037 wait complete. 2020-11-20T12:16:04.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f522b59e700 1 Processor -- start 2020-11-20T12:16:04.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f522b59e700 1 -- start start 2020-11-20T12:16:04.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f522b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f522c13ad30 0x7f522c161490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f522b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f522c13b110 0x7f522c1641a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f522b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f522c151fa0 0x7f522c0c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f522b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c165060 con 0x7f522c13b110 2020-11-20T12:16:04.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f522b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522c164b20 con 0x7f522c13ad30 2020-11-20T12:16:04.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f522b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f522c164dc0 con 0x7f522c151fa0 2020-11-20T12:16:04.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f5229d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f522c13b110 0x7f522c1641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f5229d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f522c13b110 0x7f522c1641a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42974/0 (socket says 172.21.15.116:42974) 2020-11-20T12:16:04.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f5229d9b700 1 -- 172.21.15.116:0/677361807 learned_addr learned my addr 172.21.15.116:0/677361807 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:04.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f522ad9d700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f522c151fa0 0x7f522c0c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f522a59c700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f522c13ad30 0x7f522c161490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f5229d9b700 1 -- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f522c151fa0 msgr2=0x7f522c0c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f5229d9b700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f522c151fa0 0x7f522c0c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f5229d9b700 1 -- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f522c13ad30 msgr2=0x7f522c161490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f5229d9b700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f522c13ad30 0x7f522c161490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.084+0000 7f5229d9b700 1 -- 172.21.15.116:0/677361807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f522c160d50 con 0x7f522c13b110 2020-11-20T12:16:04.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.088+0000 7f5229d9b700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f522c13b110 0x7f522c1641a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f522000ebd0 tx=0x7f5220006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.088+0000 7f521a7fc700 1 -- 172.21.15.116:0/677361807 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522000c980 con 0x7f522c13b110 2020-11-20T12:16:04.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.088+0000 7f521a7fc700 1 -- 172.21.15.116:0/677361807 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522000cae0 con 0x7f522c13b110 2020-11-20T12:16:04.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.088+0000 7f521a7fc700 1 -- 172.21.15.116:0/677361807 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522000ce60 con 0x7f522c13b110 2020-11-20T12:16:04.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.088+0000 7f522b59e700 1 -- 172.21.15.116:0/677361807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522c160ff0 con 0x7f522c13b110 2020-11-20T12:16:04.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.088+0000 7f522b59e700 1 -- 172.21.15.116:0/677361807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f522c0a0fc0 con 0x7f522c13b110 2020-11-20T12:16:04.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.088+0000 7f522b59e700 1 -- 172.21.15.116:0/677361807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522c160b90 con 0x7f522c13b110 2020-11-20T12:16:04.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.088+0000 7f521a7fc700 1 -- 172.21.15.116:0/677361807 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5220012600 con 0x7f522c13b110 2020-11-20T12:16:04.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.092+0000 7f521a7fc700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52000557c0 0x7f5200057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.092+0000 7f521a7fc700 1 -- 172.21.15.116:0/677361807 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f522006eaa0 con 0x7f522c13b110 2020-11-20T12:16:04.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.092+0000 7f522a59c700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52000557c0 0x7f5200057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.092+0000 7f522a59c700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52000557c0 0x7f5200057c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f521c006ef0 tx=0x7f521c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.096+0000 7f521a7fc700 1 -- 172.21.15.116:0/677361807 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5220044940 con 0x7f522c13b110 2020-11-20T12:16:04.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.256+0000 7f522b59e700 1 -- 172.21.15.116:0/677361807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f522c161180 con 0x7f522c13b110 2020-11-20T12:16:04.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.256+0000 7f521a7fc700 1 -- 172.21.15.116:0/677361807 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5220047f60 con 0x7f522c13b110 2020-11-20T12:16:04.258 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:04.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 -- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52000557c0 msgr2=0x7f5200057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52000557c0 0x7f5200057c80 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f521c006ef0 tx=0x7f521c015000).stop 2020-11-20T12:16:04.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 -- 172.21.15.116:0/677361807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f522c13b110 msgr2=0x7f522c1641a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f522c13b110 0x7f522c1641a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f522000ebd0 tx=0x7f5220006df0).stop 2020-11-20T12:16:04.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 -- 172.21.15.116:0/677361807 shutdown_connections 2020-11-20T12:16:04.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52000557c0 0x7f5200057c80 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f522c151fa0 0x7f522c0c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f522c13ad30 0x7f522c161490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 --2- 172.21.15.116:0/677361807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f522c13b110 0x7f522c1641a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 -- 172.21.15.116:0/677361807 >> 172.21.15.116:0/677361807 conn(0x7f522c0678a0 msgr2=0x7f522c0adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:04.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 -- 172.21.15.116:0/677361807 shutdown_connections 2020-11-20T12:16:04.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.260+0000 7f522b59e700 1 -- 172.21.15.116:0/677361807 wait complete. 2020-11-20T12:16:04.283 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.460+0000 7fe63dfb3700 1 Processor -- start 2020-11-20T12:16:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.460+0000 7fe63dfb3700 1 -- start start 2020-11-20T12:16:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe63dfb3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe63813ad30 0x7fe6380a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe63dfb3700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe638152a80 con 0x7fe638061250 2020-11-20T12:16:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe63dfb3700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe638152540 con 0x7fe63813b110 2020-11-20T12:16:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe63dfb3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6381527e0 con 0x7fe63813ad30 2020-11-20T12:16:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe63d7b2700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe638061250 0x7fe6380ad050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52418/0 (socket says 172.21.15.116:52418) 2020-11-20T12:16:04.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe63d7b2700 1 -- 172.21.15.116:0/1249122681 learned_addr learned my addr 172.21.15.116:0/1249122681 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe63cfb1700 1 --2- 172.21.15.116:0/1249122681 >> v2:172.21.15.173:3300/0 conn(0x7fe63813ad30 0x7fe6380a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 291293349 0 0) 0x7fe638152a80 con 0x7fe638061250 2020-11-20T12:16:04.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe61c003240 con 0x7fe638061250 2020-11-20T12:16:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1835662393 0 0) 0x7fe61c003240 con 0x7fe638061250 2020-11-20T12:16:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 >> v2:172.21.15.173:3300/0 conn(0x7fe63813ad30 msgr2=0x7fe6380a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 --2- 172.21.15.116:0/1249122681 >> v2:172.21.15.173:3300/0 conn(0x7fe63813ad30 0x7fe6380a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 >> v1:172.21.15.173:6789/0 conn(0x7fe63813b110 legacy=0x7fe638151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6380ac910 con 0x7fe638061250 2020-11-20T12:16:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fe630001860 con 0x7fe638061250 2020-11-20T12:16:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 >> v1:172.21.15.116:6789/0 conn(0x7fe638061250 legacy=0x7fe6380ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 --2- 172.21.15.116:0/1249122681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe61c003730 0x7fe61c003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 --2- 172.21.15.116:0/1249122681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe61c0044e0 0x7fe61c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 --2- 172.21.15.116:0/1249122681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe61c004e50 0x7fe61c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6381527e0 con 0x7fe61c0044e0 2020-11-20T12:16:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c0041d0 con 0x7fe61c003730 2020-11-20T12:16:04.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61c00bc20 con 0x7fe61c004e50 2020-11-20T12:16:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.464+0000 7fe637fff700 1 --2- 172.21.15.116:0/1249122681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe61c0044e0 0x7fe61c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe637fff700 1 -- 172.21.15.116:0/1249122681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe61c004e50 msgr2=0x7fe61c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe637fff700 1 --2- 172.21.15.116:0/1249122681 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe61c004e50 0x7fe61c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:04.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe637fff700 1 -- 172.21.15.116:0/1249122681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe61c003730 msgr2=0x7fe61c003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:04.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe63cfb1700 1 --2- 172.21.15.116:0/1249122681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe61c003730 0x7fe61c003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe637fff700 1 --2- 172.21.15.116:0/1249122681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe61c003730 0x7fe61c003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe637fff700 1 -- 172.21.15.116:0/1249122681 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c003240 con 0x7fe61c0044e0 2020-11-20T12:16:04.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe63d7b2700 1 -- 172.21.15.116:0/1249122681 reap_dead start 2020-11-20T12:16:04.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe63cfb1700 1 --2- 172.21.15.116:0/1249122681 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe61c003730 0x7fe61c003b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:04.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe637fff700 1 --2- 172.21.15.116:0/1249122681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe61c0044e0 0x7fe61c004960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fe63813b550 tx=0x7fe624007b90).ready entity=mon.0 client_cookie=bf77ca432523d39a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62400c940 con 0x7fe61c0044e0 2020-11-20T12:16:04.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe62400caa0 con 0x7fe61c0044e0 2020-11-20T12:16:04.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe6377fe700 1 -- 172.21.15.116:0/1249122681 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe62400ce60 con 0x7fe61c0044e0 2020-11-20T12:16:04.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe63dfb3700 1 -- 172.21.15.116:0/1249122681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe61c0044e0 msgr2=0x7fe61c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe63dfb3700 1 --2- 172.21.15.116:0/1249122681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe61c0044e0 0x7fe61c004960 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fe63813b550 tx=0x7fe624007b90).stop 2020-11-20T12:16:04.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe63dfb3700 1 -- 172.21.15.116:0/1249122681 shutdown_connections 2020-11-20T12:16:04.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe63dfb3700 1 --2- 172.21.15.116:0/1249122681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe61c0044e0 0x7fe61c004960 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe63dfb3700 1 -- 172.21.15.116:0/1249122681 >> 172.21.15.116:0/1249122681 conn(0x7fe6380678a0 msgr2=0x7fe638067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:04.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe63dfb3700 1 -- 172.21.15.116:0/1249122681 shutdown_connections 2020-11-20T12:16:04.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.468+0000 7fe63dfb3700 1 -- 172.21.15.116:0/1249122681 wait complete. 2020-11-20T12:16:04.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63dfb3700 1 Processor -- start 2020-11-20T12:16:04.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63dfb3700 1 -- start start 2020-11-20T12:16:04.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63dfb3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe61c0044e0 0x7fe6381607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63dfb3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe638160ca0 0x7fe6381385c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63dfb3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe638138ab0 0x7fe6380c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63dfb3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe638138f10 con 0x7fe638138ab0 2020-11-20T12:16:04.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63dfb3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6380bd360 con 0x7fe638160ca0 2020-11-20T12:16:04.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63dfb3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6380c4e00 con 0x7fe61c0044e0 2020-11-20T12:16:04.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63d7b2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe638138ab0 0x7fe6380c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63d7b2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe638138ab0 0x7fe6380c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:42996/0 (socket says 172.21.15.116:42996) 2020-11-20T12:16:04.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63d7b2700 1 -- 172.21.15.116:0/3278069721 learned_addr learned my addr 172.21.15.116:0/3278069721 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:04.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe63cfb1700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe61c0044e0 0x7fe6381607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.472+0000 7fe637fff700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe638160ca0 0x7fe6381385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe63d7b2700 1 -- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe61c0044e0 msgr2=0x7fe6381607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe63d7b2700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe61c0044e0 0x7fe6381607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe63d7b2700 1 -- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe638160ca0 msgr2=0x7fe6381385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe63d7b2700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe638160ca0 0x7fe6381385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe63d7b2700 1 -- 172.21.15.116:0/3278069721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6380c50a0 con 0x7fe638138ab0 2020-11-20T12:16:04.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe63d7b2700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe638138ab0 0x7fe6380c4130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe6300012a0 tx=0x7fe630001ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe634ff9700 1 -- 172.21.15.116:0/3278069721 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe630004a90 con 0x7fe638138ab0 2020-11-20T12:16:04.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe634ff9700 1 -- 172.21.15.116:0/3278069721 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe630004c50 con 0x7fe638138ab0 2020-11-20T12:16:04.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe63dfb3700 1 -- 172.21.15.116:0/3278069721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6380bd590 con 0x7fe638138ab0 2020-11-20T12:16:04.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe634ff9700 1 -- 172.21.15.116:0/3278069721 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe63000a890 con 0x7fe638138ab0 2020-11-20T12:16:04.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe63dfb3700 1 -- 172.21.15.116:0/3278069721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6380bfb40 con 0x7fe638138ab0 2020-11-20T12:16:04.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe63dfb3700 1 -- 172.21.15.116:0/3278069721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe638044f60 con 0x7fe638138ab0 2020-11-20T12:16:04.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.476+0000 7fe634ff9700 1 -- 172.21.15.116:0/3278069721 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe63001b060 con 0x7fe638138ab0 2020-11-20T12:16:04.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.480+0000 7fe634ff9700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe6100557f0 0x7fe610057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.480+0000 7fe634ff9700 1 -- 172.21.15.116:0/3278069721 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe63006e0e0 con 0x7fe638138ab0 2020-11-20T12:16:04.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.480+0000 7fe63cfb1700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe6100557f0 0x7fe610057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.480+0000 7fe63cfb1700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe6100557f0 0x7fe610057cb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe62c006d00 tx=0x7fe62c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.484+0000 7fe634ff9700 1 -- 172.21.15.116:0/3278069721 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe630043f80 con 0x7fe638138ab0 2020-11-20T12:16:04.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.648+0000 7fe63dfb3700 1 -- 172.21.15.116:0/3278069721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe638069320 con 0x7fe638138ab0 2020-11-20T12:16:04.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.652+0000 7fe634ff9700 1 -- 172.21.15.116:0/3278069721 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe6300475a0 con 0x7fe638138ab0 2020-11-20T12:16:04.653 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:04.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.652+0000 7fe63dfb3700 1 -- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe6100557f0 msgr2=0x7fe610057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.652+0000 7fe63dfb3700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe6100557f0 0x7fe610057cb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fe62c006d00 tx=0x7fe62c008040).stop 2020-11-20T12:16:04.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.652+0000 7fe63dfb3700 1 -- 172.21.15.116:0/3278069721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe638138ab0 msgr2=0x7fe6380c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.652+0000 7fe63dfb3700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe638138ab0 0x7fe6380c4130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fe6300012a0 tx=0x7fe630001ec0).stop 2020-11-20T12:16:04.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.652+0000 7fe63dfb3700 1 -- 172.21.15.116:0/3278069721 shutdown_connections 2020-11-20T12:16:04.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.652+0000 7fe63dfb3700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe6100557f0 0x7fe610057cb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.652+0000 7fe63dfb3700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe638138ab0 0x7fe6380c4130 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.652+0000 7fe63dfb3700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe61c0044e0 0x7fe6381607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.652+0000 7fe63dfb3700 1 --2- 172.21.15.116:0/3278069721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe638160ca0 0x7fe6381385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.652+0000 7fe63dfb3700 1 -- 172.21.15.116:0/3278069721 >> 172.21.15.116:0/3278069721 conn(0x7fe6380678a0 msgr2=0x7fe6380c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:04.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.656+0000 7fe63dfb3700 1 -- 172.21.15.116:0/3278069721 shutdown_connections 2020-11-20T12:16:04.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.656+0000 7fe63dfb3700 1 -- 172.21.15.116:0/3278069721 wait complete. 2020-11-20T12:16:04.680 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:16:04.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.856+0000 7f395ba97700 1 Processor -- start 2020-11-20T12:16:04.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.860+0000 7f395ba97700 1 -- start start 2020-11-20T12:16:04.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.860+0000 7f395ba97700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39540beb90 0x7f39540beff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.860+0000 7f395ba97700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39540baff0 con 0x7f39541502e0 2020-11-20T12:16:04.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.860+0000 7f395ba97700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3954066250 con 0x7f3954159820 2020-11-20T12:16:04.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.860+0000 7f395ba97700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39540bf4e0 con 0x7f39540beb90 2020-11-20T12:16:04.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.860+0000 7f3959833700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f39541502e0 0x7f39540ba8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52436/0 (socket says 172.21.15.116:52436) 2020-11-20T12:16:04.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.860+0000 7f3959833700 1 -- 172.21.15.116:0/2573903314 learned_addr learned my addr 172.21.15.116:0/2573903314 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:04.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.860+0000 7f3959032700 1 --2- 172.21.15.116:0/2573903314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39540beb90 0x7f39540beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.860+0000 7f3958831700 1 -- 172.21.15.116:0/2573903314 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875649792 0 0) 0x7f39540baff0 con 0x7f39541502e0 2020-11-20T12:16:04.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.860+0000 7f3958831700 1 -- 172.21.15.116:0/2573903314 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f393c003240 con 0x7f39541502e0 2020-11-20T12:16:04.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f3959032700 1 -- 172.21.15.116:0/2573903314 >> v1:172.21.15.173:6789/0 conn(0x7f3954159820 legacy=0x7f3954159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f3959032700 1 -- 172.21.15.116:0/2573903314 >> v1:172.21.15.116:6789/0 conn(0x7f39541502e0 legacy=0x7f39540ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f3959032700 1 -- 172.21.15.116:0/2573903314 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39540664f0 con 0x7f39540beb90 2020-11-20T12:16:04.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f3959032700 1 --2- 172.21.15.116:0/2573903314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39540beb90 0x7f39540beff0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f394400b9d0 tx=0x7f3944006df0).ready entity=mon.2 client_cookie=c8d62211a1582c0e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f3958831700 1 -- 172.21.15.116:0/2573903314 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944009840 con 0x7f39540beb90 2020-11-20T12:16:04.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f3958831700 1 -- 172.21.15.116:0/2573903314 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39440099a0 con 0x7f39540beb90 2020-11-20T12:16:04.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f3958831700 1 -- 172.21.15.116:0/2573903314 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3944009d20 con 0x7f39540beb90 2020-11-20T12:16:04.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f395ba97700 1 -- 172.21.15.116:0/2573903314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39540beb90 msgr2=0x7f39540beff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f395ba97700 1 --2- 172.21.15.116:0/2573903314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39540beb90 0x7f39540beff0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f394400b9d0 tx=0x7f3944006df0).stop 2020-11-20T12:16:04.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f395ba97700 1 -- 172.21.15.116:0/2573903314 shutdown_connections 2020-11-20T12:16:04.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f395ba97700 1 --2- 172.21.15.116:0/2573903314 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39540beb90 0x7f39540beff0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f395ba97700 1 -- 172.21.15.116:0/2573903314 >> 172.21.15.116:0/2573903314 conn(0x7f3954060170 msgr2=0x7f3954060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:04.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f395ba97700 1 -- 172.21.15.116:0/2573903314 shutdown_connections 2020-11-20T12:16:04.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.864+0000 7f395ba97700 1 -- 172.21.15.116:0/2573903314 wait complete. 2020-11-20T12:16:04.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.868+0000 7f395ba97700 1 Processor -- start 2020-11-20T12:16:04.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.868+0000 7f395ba97700 1 -- start start 2020-11-20T12:16:04.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.868+0000 7f395ba97700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39540beb90 0x7f39540a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.868+0000 7f395ba97700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39541502e0 0x7f3954165940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.868+0000 7f395ba97700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3954159820 0x7f395414de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.868+0000 7f395ba97700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39540a4e00 con 0x7f3954159820 2020-11-20T12:16:04.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.868+0000 7f395ba97700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3954166160 con 0x7f39541502e0 2020-11-20T12:16:04.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.868+0000 7f395ba97700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3954166400 con 0x7f39540beb90 2020-11-20T12:16:04.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f395a034700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3954159820 0x7f395414de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f3959833700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39540beb90 0x7f39540a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f395a034700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3954159820 0x7f395414de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43010/0 (socket says 172.21.15.116:43010) 2020-11-20T12:16:04.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f395a034700 1 -- 172.21.15.116:0/1207262126 learned_addr learned my addr 172.21.15.116:0/1207262126 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:04.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f3959032700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39541502e0 0x7f3954165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f395a034700 1 -- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39540beb90 msgr2=0x7f39540a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f395a034700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39540beb90 0x7f39540a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f395a034700 1 -- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39541502e0 msgr2=0x7f3954165940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:04.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f395a034700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39541502e0 0x7f3954165940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:04.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f395a034700 1 -- 172.21.15.116:0/1207262126 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39541666a0 con 0x7f3954159820 2020-11-20T12:16:04.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f395a034700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3954159820 0x7f395414de00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f394c00ca90 tx=0x7f394c007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f3941ffb700 1 -- 172.21.15.116:0/1207262126 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c00ce70 con 0x7f3954159820 2020-11-20T12:16:04.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f3941ffb700 1 -- 172.21.15.116:0/1207262126 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f394c00a7c0 con 0x7f3954159820 2020-11-20T12:16:04.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f3941ffb700 1 -- 172.21.15.116:0/1207262126 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f394c00ab60 con 0x7f3954159820 2020-11-20T12:16:04.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f395ba97700 1 -- 172.21.15.116:0/1207262126 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3954166940 con 0x7f3954159820 2020-11-20T12:16:04.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.872+0000 7f395ba97700 1 -- 172.21.15.116:0/1207262126 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3954165e30 con 0x7f3954159820 2020-11-20T12:16:04.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.876+0000 7f395ba97700 1 -- 172.21.15.116:0/1207262126 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f395405ca30 con 0x7f3954159820 2020-11-20T12:16:04.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.876+0000 7f3941ffb700 1 -- 172.21.15.116:0/1207262126 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f394c010070 con 0x7f3954159820 2020-11-20T12:16:04.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.876+0000 7f3941ffb700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f392c0557c0 0x7f392c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:04.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.876+0000 7f3941ffb700 1 -- 172.21.15.116:0/1207262126 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f394c06e4b0 con 0x7f3954159820 2020-11-20T12:16:04.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.880+0000 7f3959833700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f392c0557c0 0x7f392c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:04.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.880+0000 7f3959833700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f392c0557c0 0x7f392c057c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f3950007fd0 tx=0x7f3950009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:04.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:04.884+0000 7f3941ffb700 1 -- 172.21.15.116:0/1207262126 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f394c06bfd0 con 0x7f3954159820 2020-11-20T12:16:05.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.040+0000 7f395ba97700 1 -- 172.21.15.116:0/1207262126 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3954067e60 con 0x7f3954159820 2020-11-20T12:16:05.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.040+0000 7f3941ffb700 1 -- 172.21.15.116:0/1207262126 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f394c044240 con 0x7f3954159820 2020-11-20T12:16:05.044 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:05.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 -- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f392c0557c0 msgr2=0x7f392c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f392c0557c0 0x7f392c057c80 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f3950007fd0 tx=0x7f3950009040).stop 2020-11-20T12:16:05.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 -- 172.21.15.116:0/1207262126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3954159820 msgr2=0x7f395414de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3954159820 0x7f395414de00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f394c00ca90 tx=0x7f394c007f00).stop 2020-11-20T12:16:05.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 -- 172.21.15.116:0/1207262126 shutdown_connections 2020-11-20T12:16:05.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f392c0557c0 0x7f392c057c80 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3954159820 0x7f395414de00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f39540beb90 0x7f39540a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 --2- 172.21.15.116:0/1207262126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f39541502e0 0x7f3954165940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 -- 172.21.15.116:0/1207262126 >> 172.21.15.116:0/1207262126 conn(0x7f3954060170 msgr2=0x7f3954156e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:05.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 -- 172.21.15.116:0/1207262126 shutdown_connections 2020-11-20T12:16:05.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.044+0000 7f395ba97700 1 -- 172.21.15.116:0/1207262126 wait complete. 2020-11-20T12:16:05.070 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:16:05.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.252+0000 7f3410c8c700 1 Processor -- start 2020-11-20T12:16:05.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.252+0000 7f3410c8c700 1 -- start start 2020-11-20T12:16:05.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.252+0000 7f3410c8c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f340c14bbf0 0x7f340c0c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.252+0000 7f3410c8c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c0bf8e0 con 0x7f340c14bfd0 2020-11-20T12:16:05.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.252+0000 7f3410c8c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f340c1553a0 con 0x7f340c0bab80 2020-11-20T12:16:05.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.252+0000 7f3410c8c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c1562c0 con 0x7f340c14bbf0 2020-11-20T12:16:05.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.252+0000 7f3409d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f340c14bfd0 0x7f340c0ba470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52448/0 (socket says 172.21.15.116:52448) 2020-11-20T12:16:05.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.252+0000 7f3409d9b700 1 -- 172.21.15.116:0/3720636368 learned_addr learned my addr 172.21.15.116:0/3720636368 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:05.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340a59c700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f340c14bbf0 0x7f340c0c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 364830744 0 0) 0x7f340c0bf8e0 con 0x7f340c14bfd0 2020-11-20T12:16:05.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33f0003240 con 0x7f340c14bfd0 2020-11-20T12:16:05.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3485105117 0 0) 0x7f33f0003240 con 0x7f340c14bfd0 2020-11-20T12:16:05.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f340c14bbf0 msgr2=0x7f340c0c0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:05.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f340c14bbf0 0x7f340c0c0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 >> v1:172.21.15.173:6789/0 conn(0x7f340c0bab80 legacy=0x7f340c151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c155640 con 0x7f340c14bfd0 2020-11-20T12:16:05.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f34000014e0 con 0x7f340c14bfd0 2020-11-20T12:16:05.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 >> v1:172.21.15.116:6789/0 conn(0x7f340c14bfd0 legacy=0x7f340c0ba470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f0003760 0x7f33f0003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f340c14bbf0 0x7f33f00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33f00049d0 0x7f33f0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c1562c0 con 0x7f33f00049d0 2020-11-20T12:16:05.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f00041c0 con 0x7f33f0003760 2020-11-20T12:16:05.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f000b730 con 0x7f340c14bbf0 2020-11-20T12:16:05.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340a59c700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f0003760 0x7f33f0003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f3409d9b700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f340c14bbf0 0x7f33f00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340ad9d700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33f00049d0 0x7f33f0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340a59c700 1 -- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f340c14bbf0 msgr2=0x7f33f00044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:05.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340a59c700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f340c14bbf0 0x7f33f00044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340a59c700 1 -- 172.21.15.116:0/3720636368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33f00049d0 msgr2=0x7f33f0004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:05.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340a59c700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f33f00049d0 0x7f33f0004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340a59c700 1 -- 172.21.15.116:0/3720636368 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f0003240 con 0x7f33f0003760 2020-11-20T12:16:05.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.256+0000 7f340a59c700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f0003760 0x7f33f0003b40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f33fc00bc00 tx=0x7f33fc006df0).ready entity=mon.1 client_cookie=c1b076b149c23954 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:05.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.260+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc00b310 con 0x7f33f0003760 2020-11-20T12:16:05.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.260+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33fc00ea30 con 0x7f33f0003760 2020-11-20T12:16:05.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.260+0000 7f340959a700 1 -- 172.21.15.116:0/3720636368 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33fc00edb0 con 0x7f33f0003760 2020-11-20T12:16:05.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.260+0000 7f3410c8c700 1 -- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f0003760 msgr2=0x7f33f0003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.260+0000 7f3410c8c700 1 --2- 172.21.15.116:0/3720636368 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f33f0003760 0x7f33f0003b40 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f33fc00bc00 tx=0x7f33fc006df0).stop 2020-11-20T12:16:05.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.260+0000 7f340ad9d700 1 -- 172.21.15.116:0/3720636368 reap_dead start 2020-11-20T12:16:05.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.260+0000 7f3410c8c700 1 -- 172.21.15.116:0/3720636368 shutdown_connections 2020-11-20T12:16:05.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.260+0000 7f3410c8c700 1 -- 172.21.15.116:0/3720636368 >> 172.21.15.116:0/3720636368 conn(0x7f340c0c52e0 msgr2=0x7f340c0c56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:05.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.260+0000 7f3410c8c700 1 -- 172.21.15.116:0/3720636368 shutdown_connections 2020-11-20T12:16:05.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.260+0000 7f3410c8c700 1 -- 172.21.15.116:0/3720636368 wait complete. 2020-11-20T12:16:05.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.260+0000 7f3410c8c700 1 Processor -- start 2020-11-20T12:16:05.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3410c8c700 1 -- start start 2020-11-20T12:16:05.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3410c8c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33f0003760 0x7f340c146f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3410c8c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f340c164ba0 0x7f340c0c2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3410c8c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f340c165020 0x7f340c0c2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3410c8c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0a3050 con 0x7f340c164ba0 2020-11-20T12:16:05.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3410c8c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0a32f0 con 0x7f340c165020 2020-11-20T12:16:05.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3410c8c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f340c0a3590 con 0x7f33f0003760 2020-11-20T12:16:05.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3409d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f340c164ba0 0x7f340c0c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3409d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f340c164ba0 0x7f340c0c2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43028/0 (socket says 172.21.15.116:43028) 2020-11-20T12:16:05.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3409d9b700 1 -- 172.21.15.116:0/2891333866 learned_addr learned my addr 172.21.15.116:0/2891333866 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:05.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f340ad9d700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f340c165020 0x7f340c0c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f340a59c700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33f0003760 0x7f340c146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3409d9b700 1 -- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33f0003760 msgr2=0x7f340c146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3409d9b700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33f0003760 0x7f340c146f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3409d9b700 1 -- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f340c165020 msgr2=0x7f340c0c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3409d9b700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f340c165020 0x7f340c0c2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3409d9b700 1 -- 172.21.15.116:0/2891333866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c0a3830 con 0x7f340c164ba0 2020-11-20T12:16:05.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f340a59c700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33f0003760 0x7f340c146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:05.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3409d9b700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f340c164ba0 0x7f340c0c2050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3400004b30 tx=0x7f340c0bc1c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:05.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f33fa7fc700 1 -- 172.21.15.116:0/2891333866 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340000fa00 con 0x7f340c164ba0 2020-11-20T12:16:05.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f33fa7fc700 1 -- 172.21.15.116:0/2891333866 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340000fb60 con 0x7f340c164ba0 2020-11-20T12:16:05.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3410c8c700 1 -- 172.21.15.116:0/2891333866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f340c0a3ad0 con 0x7f340c164ba0 2020-11-20T12:16:05.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f33fa7fc700 1 -- 172.21.15.116:0/2891333866 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3400012350 con 0x7f340c164ba0 2020-11-20T12:16:05.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.264+0000 7f3410c8c700 1 -- 172.21.15.116:0/2891333866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f340c164580 con 0x7f340c164ba0 2020-11-20T12:16:05.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.268+0000 7f3410c8c700 1 -- 172.21.15.116:0/2891333866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f340c05ca30 con 0x7f340c164ba0 2020-11-20T12:16:05.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.268+0000 7f33fa7fc700 1 -- 172.21.15.116:0/2891333866 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f340001b030 con 0x7f340c164ba0 2020-11-20T12:16:05.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.268+0000 7f33fa7fc700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f33e00557c0 0x7f33e0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.268+0000 7f33fa7fc700 1 -- 172.21.15.116:0/2891333866 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f340006dea0 con 0x7f340c164ba0 2020-11-20T12:16:05.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.268+0000 7f340a59c700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f33e00557c0 0x7f33e0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.272+0000 7f340a59c700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f33e00557c0 0x7f33e0057c80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f33fc006e30 tx=0x7f33fc019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:05.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.276+0000 7f33fa7fc700 1 -- 172.21.15.116:0/2891333866 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3400043d40 con 0x7f340c164ba0 2020-11-20T12:16:05.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.436+0000 7f3410c8c700 1 -- 172.21.15.116:0/2891333866 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f340c0a3c60 con 0x7f340c164ba0 2020-11-20T12:16:05.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f33fa7fc700 1 -- 172.21.15.116:0/2891333866 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3400019030 con 0x7f340c164ba0 2020-11-20T12:16:05.440 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:05.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 -- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f33e00557c0 msgr2=0x7f33e0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f33e00557c0 0x7f33e0057c80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f33fc006e30 tx=0x7f33fc019000).stop 2020-11-20T12:16:05.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 -- 172.21.15.116:0/2891333866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f340c164ba0 msgr2=0x7f340c0c2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f340c164ba0 0x7f340c0c2050 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3400004b30 tx=0x7f340c0bc1c0).stop 2020-11-20T12:16:05.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 -- 172.21.15.116:0/2891333866 shutdown_connections 2020-11-20T12:16:05.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f33e00557c0 0x7f33e0057c80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f340c165020 0x7f340c0c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f33f0003760 0x7f340c146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 --2- 172.21.15.116:0/2891333866 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f340c164ba0 0x7f340c0c2050 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 -- 172.21.15.116:0/2891333866 >> 172.21.15.116:0/2891333866 conn(0x7f340c0c52e0 msgr2=0x7f340c14ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:05.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 -- 172.21.15.116:0/2891333866 shutdown_connections 2020-11-20T12:16:05.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.440+0000 7f3410c8c700 1 -- 172.21.15.116:0/2891333866 wait complete. 2020-11-20T12:16:05.465 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2020-11-20T12:16:05.465 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2020-11-20T12:16:05.466 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2020-11-20T12:16:05.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.632+0000 7f003afa0700 1 Processor -- start 2020-11-20T12:16:05.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.632+0000 7f003afa0700 1 -- start start 2020-11-20T12:16:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003afa0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f00340aa790 0x7f0034152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003afa0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00340612f0 0x7f00340617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003afa0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0034152a30 con 0x7f003413aee0 2020-11-20T12:16:05.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003afa0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00340adce0 con 0x7f00340aa790 2020-11-20T12:16:05.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003afa0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00340aac10 con 0x7f00340612f0 2020-11-20T12:16:05.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f0039f9e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f003413aee0 0x7f00340ad5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52468/0 (socket says 172.21.15.116:52468) 2020-11-20T12:16:05.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f0039f9e700 1 -- 172.21.15.116:0/3166551468 learned_addr learned my addr 172.21.15.116:0/3166551468 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:05.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003979d700 1 --2- 172.21.15.116:0/3166551468 >> v2:172.21.15.173:3300/0 conn(0x7f00340aa790 0x7f0034152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003a79f700 1 --2- 172.21.15.116:0/3166551468 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00340612f0 0x7f00340617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f0038f9c700 1 -- 172.21.15.116:0/3166551468 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907557640 0 0) 0x7f0034152a30 con 0x7f003413aee0 2020-11-20T12:16:05.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f0038f9c700 1 -- 172.21.15.116:0/3166551468 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f001c003240 con 0x7f003413aee0 2020-11-20T12:16:05.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003979d700 1 -- 172.21.15.116:0/3166551468 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00340612f0 msgr2=0x7f00340617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:05.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003979d700 1 --2- 172.21.15.116:0/3166551468 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00340612f0 0x7f00340617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003979d700 1 -- 172.21.15.116:0/3166551468 >> v1:172.21.15.116:6789/0 conn(0x7f003413aee0 legacy=0x7f00340ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003979d700 1 -- 172.21.15.116:0/3166551468 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00340aaeb0 con 0x7f00340aa790 2020-11-20T12:16:05.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.636+0000 7f003979d700 1 --2- 172.21.15.116:0/3166551468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00340aa790 0x7f0034152540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f002400b9a0 tx=0x7f0024006df0).ready entity=mon.1 client_cookie=336e6c5e5f6066df server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:05.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.640+0000 7f0038f9c700 1 -- 172.21.15.116:0/3166551468 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024009680 con 0x7f00340aa790 2020-11-20T12:16:05.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.640+0000 7f0038f9c700 1 -- 172.21.15.116:0/3166551468 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00240097e0 con 0x7f00340aa790 2020-11-20T12:16:05.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.640+0000 7f0038f9c700 1 -- 172.21.15.116:0/3166551468 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0024009b60 con 0x7f00340aa790 2020-11-20T12:16:05.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.640+0000 7f003afa0700 1 -- 172.21.15.116:0/3166551468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00340aa790 msgr2=0x7f0034152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.640+0000 7f003afa0700 1 --2- 172.21.15.116:0/3166551468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00340aa790 0x7f0034152540 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f002400b9a0 tx=0x7f0024006df0).stop 2020-11-20T12:16:05.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.640+0000 7f003afa0700 1 -- 172.21.15.116:0/3166551468 shutdown_connections 2020-11-20T12:16:05.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.640+0000 7f003afa0700 1 --2- 172.21.15.116:0/3166551468 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00340612f0 0x7f00340617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.640+0000 7f003afa0700 1 --2- 172.21.15.116:0/3166551468 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00340aa790 0x7f0034152540 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.640+0000 7f003afa0700 1 -- 172.21.15.116:0/3166551468 >> 172.21.15.116:0/3166551468 conn(0x7f00340678a0 msgr2=0x7f0034067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:05.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.640+0000 7f003afa0700 1 -- 172.21.15.116:0/3166551468 shutdown_connections 2020-11-20T12:16:05.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.640+0000 7f003afa0700 1 -- 172.21.15.116:0/3166551468 wait complete. 2020-11-20T12:16:05.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003afa0700 1 Processor -- start 2020-11-20T12:16:05.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003afa0700 1 -- start start 2020-11-20T12:16:05.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003afa0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00340612f0 0x7f0034150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003afa0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00340aa790 0x7f0034155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003afa0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003413aee0 0x7f0034160e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003afa0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00341512b0 con 0x7f00340612f0 2020-11-20T12:16:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003afa0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0034161a30 con 0x7f00340aa790 2020-11-20T12:16:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003afa0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0034161cd0 con 0x7f003413aee0 2020-11-20T12:16:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f0039f9e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00340612f0 0x7f0034150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f0039f9e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00340612f0 0x7f0034150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43040/0 (socket says 172.21.15.116:43040) 2020-11-20T12:16:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f0039f9e700 1 -- 172.21.15.116:0/291220892 learned_addr learned my addr 172.21.15.116:0/291220892 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003979d700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00340aa790 0x7f0034155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f0039f9e700 1 -- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003413aee0 msgr2=0x7f0034160e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003a79f700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003413aee0 0x7f0034160e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f0039f9e700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003413aee0 0x7f0034160e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f0039f9e700 1 -- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00340aa790 msgr2=0x7f0034155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f0039f9e700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00340aa790 0x7f0034155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f0039f9e700 1 -- 172.21.15.116:0/291220892 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0034135f10 con 0x7f00340612f0 2020-11-20T12:16:05.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003979d700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00340aa790 0x7f0034155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:05.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f003a79f700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003413aee0 0x7f0034160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:05.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.644+0000 7f0039f9e700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00340612f0 0x7f0034150dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f003000c710 tx=0x7f0030007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:05.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.648+0000 7f0021ffb700 1 -- 172.21.15.116:0/291220892 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00300072e0 con 0x7f00340612f0 2020-11-20T12:16:05.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.648+0000 7f0021ffb700 1 -- 172.21.15.116:0/291220892 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0030007440 con 0x7f00340612f0 2020-11-20T12:16:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.648+0000 7f0021ffb700 1 -- 172.21.15.116:0/291220892 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0030005800 con 0x7f00340612f0 2020-11-20T12:16:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.648+0000 7f003afa0700 1 -- 172.21.15.116:0/291220892 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00341361b0 con 0x7f00340612f0 2020-11-20T12:16:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.648+0000 7f003afa0700 1 -- 172.21.15.116:0/291220892 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0034156310 con 0x7f00340612f0 2020-11-20T12:16:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.648+0000 7f003afa0700 1 -- 172.21.15.116:0/291220892 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0034161e90 con 0x7f00340612f0 2020-11-20T12:16:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.648+0000 7f0021ffb700 1 -- 172.21.15.116:0/291220892 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0030010070 con 0x7f00340612f0 2020-11-20T12:16:05.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.652+0000 7f0021ffb700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f000c0557c0 0x7f000c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:05.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.652+0000 7f0021ffb700 1 -- 172.21.15.116:0/291220892 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f003006df30 con 0x7f00340612f0 2020-11-20T12:16:05.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.652+0000 7f003979d700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f000c0557c0 0x7f000c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:05.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.652+0000 7f003979d700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f000c0557c0 0x7f000c057c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0024006e30 tx=0x7f0024015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:05.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.660+0000 7f0021ffb700 1 -- 172.21.15.116:0/291220892 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0030043dd0 con 0x7f00340612f0 2020-11-20T12:16:05.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.856+0000 7f003afa0700 1 -- 172.21.15.116:0/291220892 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0034044f60 con 0x7f00340612f0 2020-11-20T12:16:05.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.864+0000 7f0021ffb700 1 -- 172.21.15.116:0/291220892 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+133248 (secure 0 0 0) 0x7f00300473f0 con 0x7f00340612f0 2020-11-20T12:16:05.868 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:05.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 -- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f000c0557c0 msgr2=0x7f000c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f000c0557c0 0x7f000c057c80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f0024006e30 tx=0x7f0024015000).stop 2020-11-20T12:16:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 -- 172.21.15.116:0/291220892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00340612f0 msgr2=0x7f0034150dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00340612f0 0x7f0034150dc0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f003000c710 tx=0x7f0030007ad0).stop 2020-11-20T12:16:05.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 -- 172.21.15.116:0/291220892 shutdown_connections 2020-11-20T12:16:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f000c0557c0 0x7f000c057c80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003413aee0 0x7f0034160e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00340612f0 0x7f0034150dc0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 --2- 172.21.15.116:0/291220892 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00340aa790 0x7f0034155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 -- 172.21.15.116:0/291220892 >> 172.21.15.116:0/291220892 conn(0x7f00340678a0 msgr2=0x7f00340c72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 -- 172.21.15.116:0/291220892 shutdown_connections 2020-11-20T12:16:05.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:05.868+0000 7f003afa0700 1 -- 172.21.15.116:0/291220892 wait complete. 2020-11-20T12:16:05.895 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":5,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6800","nonce":13860},{"type":"v1","addr":"172.21.15.173:6801","nonce":13860}]},"active_addr":"172.21.15.173:6801/13860","active_change":"2020-11-20T12:15:46.506752+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading remoto library:No module named 'remoto'","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.6.2","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_nfs":{"name":"FEATURE_TOGGLE_nfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"int","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"None","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":1038750784}]},{"addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":1448292978}]},{"addrvec":[{"type":"v2","addr":"172.21.15.173:0","nonce":992405576}]}]}} 2020-11-20T12:16:05.899 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2020-11-20T12:16:05.900 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2020-11-20T12:16:05.900 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:06.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab54c34700 1 Processor -- start 2020-11-20T12:16:06.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab54c34700 1 -- start start 2020-11-20T12:16:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab54c34700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fab50156110 0x7fab5015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab54c34700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fab500c2570 0x7fab500c29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab54c34700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5015a5e0 con 0x7fab500a3590 2020-11-20T12:16:06.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab54c34700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab50156820 con 0x7fab500c2570 2020-11-20T12:16:06.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab54c34700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab50156ac0 con 0x7fab50156110 2020-11-20T12:16:06.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab4ed9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fab500c2570 0x7fab500c29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab4ed9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fab500c2570 0x7fab500c29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43052/0 (socket says 172.21.15.116:43052) 2020-11-20T12:16:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab4ed9d700 1 -- 172.21.15.116:0/1824228417 learned_addr learned my addr 172.21.15.116:0/1824228417 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab4e59c700 1 --1- 172.21.15.116:0/1824228417 >> v1:172.21.15.173:6789/0 conn(0x7fab500a3590 0x7fab500ba740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:48714/0 (socket says 172.21.15.116:48714) 2020-11-20T12:16:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.072+0000 7fab45d9b700 1 --2- 172.21.15.116:0/1824228417 >> v2:172.21.15.173:3300/0 conn(0x7fab50156110 0x7fab5015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.076+0000 7fab4ed9d700 1 -- 172.21.15.116:0/1824228417 >> v2:172.21.15.173:3300/0 conn(0x7fab50156110 msgr2=0x7fab5015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.076+0000 7fab4ed9d700 1 --2- 172.21.15.116:0/1824228417 >> v2:172.21.15.173:3300/0 conn(0x7fab50156110 0x7fab5015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.076+0000 7fab4ed9d700 1 -- 172.21.15.116:0/1824228417 >> v1:172.21.15.173:6789/0 conn(0x7fab500a3590 legacy=0x7fab500ba740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.076+0000 7fab4ed9d700 1 -- 172.21.15.116:0/1824228417 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab50156d60 con 0x7fab500c2570 2020-11-20T12:16:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.076+0000 7fab4ed9d700 1 --2- 172.21.15.116:0/1824228417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab500c2570 0x7fab500c29b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fab500c3e40 tx=0x7fab48009290).ready entity=mon.0 client_cookie=fd8a2f71803ca183 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.076+0000 7fab4dd9b700 1 -- 172.21.15.116:0/1824228417 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4800bbc0 con 0x7fab500c2570 2020-11-20T12:16:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.076+0000 7fab4dd9b700 1 -- 172.21.15.116:0/1824228417 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4800bd20 con 0x7fab500c2570 2020-11-20T12:16:06.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.076+0000 7fab4dd9b700 1 -- 172.21.15.116:0/1824228417 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48011610 con 0x7fab500c2570 2020-11-20T12:16:06.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.080+0000 7fab54c34700 1 -- 172.21.15.116:0/1824228417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab500c2570 msgr2=0x7fab500c29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.080+0000 7fab54c34700 1 --2- 172.21.15.116:0/1824228417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab500c2570 0x7fab500c29b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fab500c3e40 tx=0x7fab48009290).stop 2020-11-20T12:16:06.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.080+0000 7fab54c34700 1 -- 172.21.15.116:0/1824228417 shutdown_connections 2020-11-20T12:16:06.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.080+0000 7fab54c34700 1 --2- 172.21.15.116:0/1824228417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab500c2570 0x7fab500c29b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.080+0000 7fab54c34700 1 --2- 172.21.15.116:0/1824228417 >> v2:172.21.15.173:3300/0 conn(0x7fab50156110 0x7fab5015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.080+0000 7fab54c34700 1 -- 172.21.15.116:0/1824228417 >> 172.21.15.116:0/1824228417 conn(0x7fab50060c80 msgr2=0x7fab500c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:06.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.080+0000 7fab54c34700 1 -- 172.21.15.116:0/1824228417 shutdown_connections 2020-11-20T12:16:06.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.080+0000 7fab54c34700 1 -- 172.21.15.116:0/1824228417 wait complete. 2020-11-20T12:16:06.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.080+0000 7fab54c34700 1 Processor -- start 2020-11-20T12:16:06.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.080+0000 7fab54c34700 1 -- start start 2020-11-20T12:16:06.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab54c34700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab500a3590 0x7fab50169510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab54c34700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab500c2570 0x7fab501652f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab54c34700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab50156110 0x7fab500a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab54c34700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab500a90d0 con 0x7fab50156110 2020-11-20T12:16:06.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab54c34700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab50165ba0 con 0x7fab500a3590 2020-11-20T12:16:06.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab54c34700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab50165e40 con 0x7fab500c2570 2020-11-20T12:16:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab4ed9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab50156110 0x7fab500a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab4ed9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab50156110 0x7fab500a4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43058/0 (socket says 172.21.15.116:43058) 2020-11-20T12:16:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab4ed9d700 1 -- 172.21.15.116:0/1111976621 learned_addr learned my addr 172.21.15.116:0/1111976621 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab45d9b700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab500c2570 0x7fab501652f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab4e59c700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab500a3590 0x7fab50169510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab4ed9d700 1 -- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab500c2570 msgr2=0x7fab501652f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab4ed9d700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab500c2570 0x7fab501652f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab4ed9d700 1 -- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab500a3590 msgr2=0x7fab50169510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab4ed9d700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab500a3590 0x7fab50169510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab4ed9d700 1 -- 172.21.15.116:0/1111976621 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab501660e0 con 0x7fab50156110 2020-11-20T12:16:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab4ed9d700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab50156110 0x7fab500a4300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fab50061120 tx=0x7fab48016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab46ffd700 1 -- 172.21.15.116:0/1111976621 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab4800bbc0 con 0x7fab50156110 2020-11-20T12:16:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab46ffd700 1 -- 172.21.15.116:0/1111976621 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab4800be90 con 0x7fab50156110 2020-11-20T12:16:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab46ffd700 1 -- 172.21.15.116:0/1111976621 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab48010da0 con 0x7fab50156110 2020-11-20T12:16:06.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab54c34700 1 -- 172.21.15.116:0/1111976621 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab500a9330 con 0x7fab50156110 2020-11-20T12:16:06.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.084+0000 7fab54c34700 1 -- 172.21.15.116:0/1111976621 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab500a47f0 con 0x7fab50156110 2020-11-20T12:16:06.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.088+0000 7fab54c34700 1 -- 172.21.15.116:0/1111976621 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab500a94f0 con 0x7fab50156110 2020-11-20T12:16:06.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.088+0000 7fab46ffd700 1 -- 172.21.15.116:0/1111976621 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fab48017070 con 0x7fab50156110 2020-11-20T12:16:06.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.088+0000 7fab46ffd700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fab200557c0 0x7fab20057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.088+0000 7fab46ffd700 1 -- 172.21.15.116:0/1111976621 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fab48070750 con 0x7fab50156110 2020-11-20T12:16:06.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.088+0000 7fab4e59c700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fab200557c0 0x7fab20057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.092+0000 7fab4e59c700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fab200557c0 0x7fab20057c80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fab40007f70 tx=0x7fab40007180).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:06.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.096+0000 7fab46ffd700 1 -- 172.21.15.116:0/1111976621 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fab48046670 con 0x7fab50156110 2020-11-20T12:16:06.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.248+0000 7fab54c34700 1 -- 172.21.15.116:0/1111976621 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab5005ca30 con 0x7fab50156110 2020-11-20T12:16:06.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab46ffd700 1 -- 172.21.15.116:0/1111976621 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fab48015020 con 0x7fab50156110 2020-11-20T12:16:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 -- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fab200557c0 msgr2=0x7fab20057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fab200557c0 0x7fab20057c80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fab40007f70 tx=0x7fab40007180).stop 2020-11-20T12:16:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 -- 172.21.15.116:0/1111976621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab50156110 msgr2=0x7fab500a4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab50156110 0x7fab500a4300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fab50061120 tx=0x7fab48016000).stop 2020-11-20T12:16:06.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 -- 172.21.15.116:0/1111976621 shutdown_connections 2020-11-20T12:16:06.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fab200557c0 0x7fab20057c80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fab50156110 0x7fab500a4300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fab500a3590 0x7fab50169510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 --2- 172.21.15.116:0/1111976621 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fab500c2570 0x7fab501652f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 -- 172.21.15.116:0/1111976621 >> 172.21.15.116:0/1111976621 conn(0x7fab50060c80 msgr2=0x7fab500a0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:06.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 -- 172.21.15.116:0/1111976621 shutdown_connections 2020-11-20T12:16:06.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.252+0000 7fab54c34700 1 -- 172.21.15.116:0/1111976621 wait complete. 2020-11-20T12:16:06.373 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:06.374 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:06.376 INFO:tasks.ceph.ceph_manager.ceph:all up! 2020-11-20T12:16:06.377 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:06.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.560+0000 7f5e02f95700 1 Processor -- start 2020-11-20T12:16:06.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.564+0000 7f5e02f95700 1 -- start start 2020-11-20T12:16:06.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.564+0000 7f5e02f95700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5dfc0c3380 0x7f5dfc0c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.564+0000 7f5e02f95700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5dfc14c280 0x7f5dfc067d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.564+0000 7f5e02f95700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dfc068790 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.564+0000 7f5e02f95700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dfc068260 con 0x7f5dfc14c280 2020-11-20T12:16:06.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.564+0000 7f5e02f95700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc0684f0 con 0x7f5dfc0c3380 2020-11-20T12:16:06.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.564+0000 7f5dfbfff700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f5dfc14f5c0 0x7f5dfc14bb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:48726/0 (socket says 172.21.15.116:48726) 2020-11-20T12:16:06.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.564+0000 7f5dfbfff700 1 -- 172.21.15.116:0/2760533741 learned_addr learned my addr 172.21.15.116:0/2760533741 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:06.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.564+0000 7f5e00d31700 1 --2- 172.21.15.116:0/2760533741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5dfc0c3380 0x7f5dfc0c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.564+0000 7f5e01532700 1 --2- 172.21.15.116:0/2760533741 >> v2:172.21.15.173:3300/0 conn(0x7f5dfc14c280 0x7f5dfc067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5dfb7fe700 1 -- 172.21.15.116:0/2760533741 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667500142 0 0) 0x7f5dfc068790 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5dfb7fe700 1 -- 172.21.15.116:0/2760533741 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5de0003240 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5e01532700 1 -- 172.21.15.116:0/2760533741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5dfc0c3380 msgr2=0x7f5dfc0c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:06.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5e01532700 1 --2- 172.21.15.116:0/2760533741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5dfc0c3380 0x7f5dfc0c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5e01532700 1 -- 172.21.15.116:0/2760533741 >> v1:172.21.15.173:6789/0 conn(0x7f5dfc14f5c0 legacy=0x7f5dfc14bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5e01532700 1 -- 172.21.15.116:0/2760533741 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc14fa40 con 0x7f5dfc14c280 2020-11-20T12:16:06.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5e01532700 1 --2- 172.21.15.116:0/2760533741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc14c280 0x7f5dfc067d70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5df4009c90 tx=0x7f5df40091d0).ready entity=mon.1 client_cookie=f9014d1024d389d3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:06.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5dfb7fe700 1 -- 172.21.15.116:0/2760533741 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df400ba50 con 0x7f5dfc14c280 2020-11-20T12:16:06.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5dfb7fe700 1 -- 172.21.15.116:0/2760533741 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df400bbb0 con 0x7f5dfc14c280 2020-11-20T12:16:06.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5dfb7fe700 1 -- 172.21.15.116:0/2760533741 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4011400 con 0x7f5dfc14c280 2020-11-20T12:16:06.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5e02f95700 1 -- 172.21.15.116:0/2760533741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc14c280 msgr2=0x7f5dfc067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5e02f95700 1 --2- 172.21.15.116:0/2760533741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc14c280 0x7f5dfc067d70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f5df4009c90 tx=0x7f5df40091d0).stop 2020-11-20T12:16:06.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5e02f95700 1 -- 172.21.15.116:0/2760533741 shutdown_connections 2020-11-20T12:16:06.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5e02f95700 1 --2- 172.21.15.116:0/2760533741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc14c280 0x7f5dfc067d70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5e02f95700 1 --2- 172.21.15.116:0/2760533741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5dfc0c3380 0x7f5dfc0c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.568+0000 7f5e02f95700 1 -- 172.21.15.116:0/2760533741 >> 172.21.15.116:0/2760533741 conn(0x7f5dfc060c80 msgr2=0x7f5dfc0c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.572+0000 7f5e02f95700 1 -- 172.21.15.116:0/2760533741 shutdown_connections 2020-11-20T12:16:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.572+0000 7f5e02f95700 1 -- 172.21.15.116:0/2760533741 wait complete. 2020-11-20T12:16:06.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.572+0000 7f5e02f95700 1 Processor -- start 2020-11-20T12:16:06.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e02f95700 1 -- start start 2020-11-20T12:16:06.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e02f95700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5dfc0c3380 0x7f5dfc1530b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e02f95700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc14c280 0x7f5dfc1361b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e02f95700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5dfc14f5c0 0x7f5dfc0c2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e02f95700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc14b0f0 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e02f95700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc14a910 con 0x7f5dfc14c280 2020-11-20T12:16:06.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e02f95700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dfc14abb0 con 0x7f5dfc0c3380 2020-11-20T12:16:06.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e01532700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5dfc14f5c0 0x7f5dfc0c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e01532700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5dfc14f5c0 0x7f5dfc0c2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43072/0 (socket says 172.21.15.116:43072) 2020-11-20T12:16:06.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e01532700 1 -- 172.21.15.116:0/3144958835 learned_addr learned my addr 172.21.15.116:0/3144958835 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:06.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5dfbfff700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc14c280 0x7f5dfc1361b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e00d31700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5dfc0c3380 0x7f5dfc1530b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e01532700 1 -- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5dfc0c3380 msgr2=0x7f5dfc1530b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e01532700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5dfc0c3380 0x7f5dfc1530b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e01532700 1 -- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc14c280 msgr2=0x7f5dfc1361b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e01532700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc14c280 0x7f5dfc1361b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e01532700 1 -- 172.21.15.116:0/3144958835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dfc14ae50 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.576+0000 7f5e01532700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5dfc14f5c0 0x7f5dfc0c2130 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f5df4009210 tx=0x7f5df4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:06.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.580+0000 7f5df8ff9700 1 -- 172.21.15.116:0/3144958835 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4010700 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.580+0000 7f5df8ff9700 1 -- 172.21.15.116:0/3144958835 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5df4010860 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.580+0000 7f5df8ff9700 1 -- 172.21.15.116:0/3144958835 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5df4010c00 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.580+0000 7f5e02f95700 1 -- 172.21.15.116:0/3144958835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dfc148f10 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.580+0000 7f5e02f95700 1 -- 172.21.15.116:0/3144958835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dfc1366a0 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.580+0000 7f5e02f95700 1 -- 172.21.15.116:0/3144958835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dfc1490a0 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.580+0000 7f5df8ff9700 1 -- 172.21.15.116:0/3144958835 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5df4017070 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.580+0000 7f5df8ff9700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dd40557c0 0x7f5dd4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.580+0000 7f5df8ff9700 1 -- 172.21.15.116:0/3144958835 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5df4070b80 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.584+0000 7f5e00d31700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dd40557c0 0x7f5dd4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.584+0000 7f5e00d31700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dd40557c0 0x7f5dd4057c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5df0006fd0 tx=0x7f5df0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:06.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.588+0000 7f5df8ff9700 1 -- 172.21.15.116:0/3144958835 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5df4046a20 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.748+0000 7f5e02f95700 1 -- 172.21.15.116:0/3144958835 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5dfc05ca30 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5df8ff9700 1 -- 172.21.15.116:0/3144958835 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f5df4015030 con 0x7f5dfc14f5c0 2020-11-20T12:16:06.753 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:06.753 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:06.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 -- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dd40557c0 msgr2=0x7f5dd4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dd40557c0 0x7f5dd4057c80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f5df0006fd0 tx=0x7f5df0008040).stop 2020-11-20T12:16:06.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 -- 172.21.15.116:0/3144958835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5dfc14f5c0 msgr2=0x7f5dfc0c2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5dfc14f5c0 0x7f5dfc0c2130 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f5df4009210 tx=0x7f5df4016000).stop 2020-11-20T12:16:06.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 -- 172.21.15.116:0/3144958835 shutdown_connections 2020-11-20T12:16:06.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dd40557c0 0x7f5dd4057c80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5dfc14f5c0 0x7f5dfc0c2130 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5dfc0c3380 0x7f5dfc1530b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 --2- 172.21.15.116:0/3144958835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5dfc14c280 0x7f5dfc1361b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 -- 172.21.15.116:0/3144958835 >> 172.21.15.116:0/3144958835 conn(0x7f5dfc060c80 msgr2=0x7f5dfc14fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:06.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 -- 172.21.15.116:0/3144958835 shutdown_connections 2020-11-20T12:16:06.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.752+0000 7f5e02f95700 1 -- 172.21.15.116:0/3144958835 wait complete. 2020-11-20T12:16:06.782 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2020-11-20T12:16:06.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.956+0000 7f9e8cf01700 1 Processor -- start 2020-11-20T12:16:06.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.956+0000 7f9e8cf01700 1 -- start start 2020-11-20T12:16:06.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.956+0000 7f9e8cf01700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9e8814f530 0x7f9e881505a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.956+0000 7f9e8cf01700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e880a6690 0x7f9e880a6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.956+0000 7f9e8cf01700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e880a9220 con 0x7f9e8814f150 2020-11-20T12:16:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.956+0000 7f9e8cf01700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e880a94c0 con 0x7f9e8814f530 2020-11-20T12:16:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.956+0000 7f9e8cf01700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880a9760 con 0x7f9e880a6690 2020-11-20T12:16:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.956+0000 7f9e877fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9e8814f150 0x7f9e880a8a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52510/0 (socket says 172.21.15.116:52510) 2020-11-20T12:16:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.956+0000 7f9e877fe700 1 -- 172.21.15.116:0/790884022 learned_addr learned my addr 172.21.15.116:0/790884022 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:06.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e7ffff700 1 --2- 172.21.15.116:0/790884022 >> v2:172.21.15.116:3300/0 conn(0x7f9e8814f530 0x7f9e881505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e87fff700 1 --2- 172.21.15.116:0/790884022 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e880a6690 0x7f9e880a6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e86ffd700 1 -- 172.21.15.116:0/790884022 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3519881165 0 0) 0x7f9e880a9220 con 0x7f9e8814f150 2020-11-20T12:16:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e86ffd700 1 -- 172.21.15.116:0/790884022 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e68003240 con 0x7f9e8814f150 2020-11-20T12:16:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e7ffff700 1 -- 172.21.15.116:0/790884022 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e880a6690 msgr2=0x7f9e880a6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e7ffff700 1 --2- 172.21.15.116:0/790884022 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e880a6690 0x7f9e880a6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e7ffff700 1 -- 172.21.15.116:0/790884022 >> v1:172.21.15.116:6789/0 conn(0x7f9e8814f150 legacy=0x7f9e880a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e7ffff700 1 -- 172.21.15.116:0/790884022 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e880a9a00 con 0x7f9e8814f530 2020-11-20T12:16:06.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e7ffff700 1 --2- 172.21.15.116:0/790884022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e8814f530 0x7f9e881505a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f9e7000b9a0 tx=0x7f9e70006df0).ready entity=mon.0 client_cookie=89202b8e9998dae4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:06.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e86ffd700 1 -- 172.21.15.116:0/790884022 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70009840 con 0x7f9e8814f530 2020-11-20T12:16:06.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e86ffd700 1 -- 172.21.15.116:0/790884022 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e700099a0 con 0x7f9e8814f530 2020-11-20T12:16:06.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e86ffd700 1 -- 172.21.15.116:0/790884022 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e70009d20 con 0x7f9e8814f530 2020-11-20T12:16:06.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e8cf01700 1 -- 172.21.15.116:0/790884022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e8814f530 msgr2=0x7f9e881505a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e8cf01700 1 --2- 172.21.15.116:0/790884022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e8814f530 0x7f9e881505a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f9e7000b9a0 tx=0x7f9e70006df0).stop 2020-11-20T12:16:06.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e8cf01700 1 -- 172.21.15.116:0/790884022 shutdown_connections 2020-11-20T12:16:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e8cf01700 1 --2- 172.21.15.116:0/790884022 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e880a6690 0x7f9e880a6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e8cf01700 1 --2- 172.21.15.116:0/790884022 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e8814f530 0x7f9e881505a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e8cf01700 1 -- 172.21.15.116:0/790884022 >> 172.21.15.116:0/790884022 conn(0x7f9e880c1370 msgr2=0x7f9e880c1750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e8cf01700 1 -- 172.21.15.116:0/790884022 shutdown_connections 2020-11-20T12:16:06.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.960+0000 7f9e8cf01700 1 -- 172.21.15.116:0/790884022 wait complete. 2020-11-20T12:16:06.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.964+0000 7f9e8cf01700 1 Processor -- start 2020-11-20T12:16:06.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.964+0000 7f9e8cf01700 1 -- start start 2020-11-20T12:16:06.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.964+0000 7f9e8cf01700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e880a6690 0x7f9e880a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.964+0000 7f9e8cf01700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e8814f150 0x7f9e8814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.964+0000 7f9e8cf01700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e8814f530 0x7f9e8814b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.964+0000 7f9e8cf01700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880a2f00 con 0x7f9e880a6690 2020-11-20T12:16:06.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.964+0000 7f9e8cf01700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880a31a0 con 0x7f9e8814f530 2020-11-20T12:16:06.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.964+0000 7f9e8cf01700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e880a3440 con 0x7f9e8814f150 2020-11-20T12:16:06.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.964+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e880a6690 0x7f9e880a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.964+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e880a6690 0x7f9e880a25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43082/0 (socket says 172.21.15.116:43082) 2020-11-20T12:16:06.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.964+0000 7f9e877fe700 1 -- 172.21.15.116:0/3760250536 learned_addr learned my addr 172.21.15.116:0/3760250536 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:06.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e87fff700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e8814f530 0x7f9e8814b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e877fe700 1 -- 172.21.15.116:0/3760250536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e8814f150 msgr2=0x7f9e8814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e877fe700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e8814f150 0x7f9e8814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:06.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e877fe700 1 -- 172.21.15.116:0/3760250536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e8814f530 msgr2=0x7f9e8814b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:06.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e877fe700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e8814f530 0x7f9e8814b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:06.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e877fe700 1 -- 172.21.15.116:0/3760250536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e88161480 con 0x7f9e880a6690 2020-11-20T12:16:06.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e877fe700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e880a6690 0x7f9e880a25f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f9e7800c6f0 tx=0x7f9e78007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e7f7fe700 1 -- 172.21.15.116:0/3760250536 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78007320 con 0x7f9e880a6690 2020-11-20T12:16:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e7f7fe700 1 -- 172.21.15.116:0/3760250536 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e78007480 con 0x7f9e880a6690 2020-11-20T12:16:06.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e8cf01700 1 -- 172.21.15.116:0/3760250536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e88161720 con 0x7f9e880a6690 2020-11-20T12:16:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e7f7fe700 1 -- 172.21.15.116:0/3760250536 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e78005800 con 0x7f9e880a6690 2020-11-20T12:16:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e8cf01700 1 -- 172.21.15.116:0/3760250536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e880a1dc0 con 0x7f9e880a6690 2020-11-20T12:16:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e8cf01700 1 -- 172.21.15.116:0/3760250536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9e88044f60 con 0x7f9e880a6690 2020-11-20T12:16:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.968+0000 7f9e7f7fe700 1 -- 172.21.15.116:0/3760250536 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9e78010030 con 0x7f9e880a6690 2020-11-20T12:16:06.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.972+0000 7f9e7f7fe700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e640557f0 0x7f9e64057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.972+0000 7f9e7f7fe700 1 -- 172.21.15.116:0/3760250536 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9e7806e4b0 con 0x7f9e880a6690 2020-11-20T12:16:06.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.972+0000 7f9e7f7fe700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.116:6800/14053,v1:172.21.15.116:6801/14053] conn(0x7f9e6405cd40 0x7f9e6405f160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:06.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.972+0000 7f9e7f7fe700 1 -- 172.21.15.116:0/3760250536 --> [v2:172.21.15.116:6800/14053,v1:172.21.15.116:6801/14053] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9e6405f7c0 con 0x7f9e6405cd40 2020-11-20T12:16:06.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.972+0000 7f9e7f7fe700 1 -- 172.21.15.116:0/3760250536 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e78070b20 con 0x7f9e880a6690 2020-11-20T12:16:06.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.972+0000 7f9e7ffff700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e640557f0 0x7f9e64057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.972+0000 7f9e87fff700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.116:6800/14053,v1:172.21.15.116:6801/14053] conn(0x7f9e6405cd40 0x7f9e6405f160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.972+0000 7f9e87fff700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.116:6800/14053,v1:172.21.15.116:6801/14053] conn(0x7f9e6405cd40 0x7f9e6405f160 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.972+0000 7f9e7ffff700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e640557f0 0x7f9e64057cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f9e880a1c70 tx=0x7f9e70015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:06.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.976+0000 7f9e7f7fe700 1 -- 172.21.15.116:0/3760250536 <== osd.0 v2:172.21.15.116:6800/14053 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f9e6405f7c0 con 0x7f9e6405cd40 2020-11-20T12:16:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.992+0000 7f9e8cf01700 1 -- 172.21.15.116:0/3760250536 --> [v2:172.21.15.116:6800/14053,v1:172.21.15.116:6801/14053] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9e881618b0 con 0x7f9e6405cd40 2020-11-20T12:16:07.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e7f7fe700 1 -- 172.21.15.116:0/3760250536 <== osd.0 v2:172.21.15.116:6800/14053 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9e881618b0 con 0x7f9e6405cd40 2020-11-20T12:16:07.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e8cf01700 1 -- 172.21.15.116:0/3760250536 >> [v2:172.21.15.116:6800/14053,v1:172.21.15.116:6801/14053] conn(0x7f9e6405cd40 msgr2=0x7f9e6405f160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e8cf01700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.116:6800/14053,v1:172.21.15.116:6801/14053] conn(0x7f9e6405cd40 0x7f9e6405f160 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e8cf01700 1 -- 172.21.15.116:0/3760250536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e640557f0 msgr2=0x7f9e64057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e8cf01700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e640557f0 0x7f9e64057cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f9e880a1c70 tx=0x7f9e70015000).stop 2020-11-20T12:16:07.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e8cf01700 1 -- 172.21.15.116:0/3760250536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e880a6690 msgr2=0x7f9e880a25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e8cf01700 1 --2- 172.21.15.116:0/3760250536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e880a6690 0x7f9e880a25f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f9e7800c6f0 tx=0x7f9e78007ad0).stop 2020-11-20T12:16:07.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e87fff700 1 -- 172.21.15.116:0/3760250536 reap_dead start 2020-11-20T12:16:07.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e8cf01700 1 -- 172.21.15.116:0/3760250536 shutdown_connections 2020-11-20T12:16:07.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e8cf01700 1 -- 172.21.15.116:0/3760250536 >> 172.21.15.116:0/3760250536 conn(0x7f9e880c1370 msgr2=0x7f9e880698a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:07.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e8cf01700 1 -- 172.21.15.116:0/3760250536 shutdown_connections 2020-11-20T12:16:07.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:06.996+0000 7f9e8cf01700 1 -- 172.21.15.116:0/3760250536 wait complete. 2020-11-20T12:16:07.023 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:07.024 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2020-11-20T12:16:07.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.200+0000 7fb8752dd700 1 Processor -- start 2020-11-20T12:16:07.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.200+0000 7fb8752dd700 1 -- start start 2020-11-20T12:16:07.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb8752dd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb87014e3f0 0x7fb870158df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb8752dd700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8700a5230 con 0x7fb8700c4530 2020-11-20T12:16:07.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb8752dd700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8700a7220 con 0x7fb8700c4150 2020-11-20T12:16:07.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb8752dd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8700a74c0 con 0x7fb87014e3f0 2020-11-20T12:16:07.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb867fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb8700c4530 0x7fb87014dce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52528/0 (socket says 172.21.15.116:52528) 2020-11-20T12:16:07.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb867fff700 1 -- 172.21.15.116:0/566581533 learned_addr learned my addr 172.21.15.116:0/566581533 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:07.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb874adc700 1 --2- 172.21.15.116:0/566581533 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb87014e3f0 0x7fb870158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb86f7fe700 1 -- 172.21.15.116:0/566581533 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2039128701 0 0) 0x7fb8700a5230 con 0x7fb8700c4530 2020-11-20T12:16:07.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb86f7fe700 1 -- 172.21.15.116:0/566581533 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb850003240 con 0x7fb8700c4530 2020-11-20T12:16:07.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb86f7fe700 1 -- 172.21.15.116:0/566581533 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1560803624 0 0) 0x7fb8700a7220 con 0x7fb8700c4150 2020-11-20T12:16:07.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb86f7fe700 1 -- 172.21.15.116:0/566581533 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8700a5230 con 0x7fb8700c4150 2020-11-20T12:16:07.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb874adc700 1 -- 172.21.15.116:0/566581533 >> v1:172.21.15.173:6789/0 conn(0x7fb8700c4150 legacy=0x7fb8700a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb874adc700 1 -- 172.21.15.116:0/566581533 >> v1:172.21.15.116:6789/0 conn(0x7fb8700c4530 legacy=0x7fb87014dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb874adc700 1 -- 172.21.15.116:0/566581533 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8700a7760 con 0x7fb87014e3f0 2020-11-20T12:16:07.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb874adc700 1 --2- 172.21.15.116:0/566581533 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb87014e3f0 0x7fb870158df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb868009cf0 tx=0x7fb8680091d0).ready entity=mon.2 client_cookie=8783261fc86d7a26 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb86f7fe700 1 -- 172.21.15.116:0/566581533 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb86800b9e0 con 0x7fb87014e3f0 2020-11-20T12:16:07.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb86f7fe700 1 -- 172.21.15.116:0/566581533 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb86800bb40 con 0x7fb87014e3f0 2020-11-20T12:16:07.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.204+0000 7fb86f7fe700 1 -- 172.21.15.116:0/566581533 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868011400 con 0x7fb87014e3f0 2020-11-20T12:16:07.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.208+0000 7fb8752dd700 1 -- 172.21.15.116:0/566581533 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb87014e3f0 msgr2=0x7fb870158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.208+0000 7fb8752dd700 1 --2- 172.21.15.116:0/566581533 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb87014e3f0 0x7fb870158df0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb868009cf0 tx=0x7fb8680091d0).stop 2020-11-20T12:16:07.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.208+0000 7fb8752dd700 1 -- 172.21.15.116:0/566581533 shutdown_connections 2020-11-20T12:16:07.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.208+0000 7fb8752dd700 1 --2- 172.21.15.116:0/566581533 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb87014e3f0 0x7fb870158df0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.208+0000 7fb8752dd700 1 -- 172.21.15.116:0/566581533 >> 172.21.15.116:0/566581533 conn(0x7fb870062520 msgr2=0x7fb870062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:07.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.208+0000 7fb8752dd700 1 -- 172.21.15.116:0/566581533 shutdown_connections 2020-11-20T12:16:07.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.208+0000 7fb8752dd700 1 -- 172.21.15.116:0/566581533 wait complete. 2020-11-20T12:16:07.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.208+0000 7fb8752dd700 1 Processor -- start 2020-11-20T12:16:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.208+0000 7fb8752dd700 1 -- start start 2020-11-20T12:16:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb8752dd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8700c4150 0x7fb870161740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb8752dd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8700c4530 0x7fb87013bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb8752dd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb87014e3f0 0x7fb870155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb8752dd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb870161c30 con 0x7fb8700c4530 2020-11-20T12:16:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb8752dd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb870152e40 con 0x7fb87014e3f0 2020-11-20T12:16:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb8752dd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8701530e0 con 0x7fb8700c4150 2020-11-20T12:16:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb867fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8700c4530 0x7fb87013bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb867fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8700c4530 0x7fb87013bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43100/0 (socket says 172.21.15.116:43100) 2020-11-20T12:16:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb867fff700 1 -- 172.21.15.116:0/983850357 learned_addr learned my addr 172.21.15.116:0/983850357 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb874adc700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb87014e3f0 0x7fb870155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb86ffff700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8700c4150 0x7fb870161740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb867fff700 1 -- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8700c4150 msgr2=0x7fb870161740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb867fff700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8700c4150 0x7fb870161740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb867fff700 1 -- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb87014e3f0 msgr2=0x7fb870155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb867fff700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb87014e3f0 0x7fb870155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb867fff700 1 -- 172.21.15.116:0/983850357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb870153380 con 0x7fb8700c4530 2020-11-20T12:16:07.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb874adc700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb87014e3f0 0x7fb870155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:07.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb86ffff700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8700c4150 0x7fb870161740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:07.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb867fff700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8700c4530 0x7fb87013bae0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fb85800c710 tx=0x7fb858007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb86cff9700 1 -- 172.21.15.116:0/983850357 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858007320 con 0x7fb8700c4530 2020-11-20T12:16:07.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb8752dd700 1 -- 172.21.15.116:0/983850357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb870153620 con 0x7fb8700c4530 2020-11-20T12:16:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb86cff9700 1 -- 172.21.15.116:0/983850357 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb858007480 con 0x7fb8700c4530 2020-11-20T12:16:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb86cff9700 1 -- 172.21.15.116:0/983850357 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb858005800 con 0x7fb8700c4530 2020-11-20T12:16:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb8752dd700 1 -- 172.21.15.116:0/983850357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb87013bfd0 con 0x7fb8700c4530 2020-11-20T12:16:07.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.212+0000 7fb8752dd700 1 -- 172.21.15.116:0/983850357 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb870044f60 con 0x7fb8700c4530 2020-11-20T12:16:07.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.216+0000 7fb86cff9700 1 -- 172.21.15.116:0/983850357 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb858010030 con 0x7fb8700c4530 2020-11-20T12:16:07.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.216+0000 7fb86cff9700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb84c0557c0 0x7fb84c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.216+0000 7fb86cff9700 1 -- 172.21.15.116:0/983850357 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb85806ed90 con 0x7fb8700c4530 2020-11-20T12:16:07.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.216+0000 7fb86cff9700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.116:6824/14069,v1:172.21.15.116:6825/14069] conn(0x7fb84c05cc90 0x7fb84c05f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.216+0000 7fb86cff9700 1 -- 172.21.15.116:0/983850357 --> [v2:172.21.15.116:6824/14069,v1:172.21.15.116:6825/14069] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb84c05f710 con 0x7fb84c05cc90 2020-11-20T12:16:07.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.216+0000 7fb86cff9700 1 -- 172.21.15.116:0/983850357 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb85806f190 con 0x7fb8700c4530 2020-11-20T12:16:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.216+0000 7fb86ffff700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb84c0557c0 0x7fb84c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.216+0000 7fb874adc700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.116:6824/14069,v1:172.21.15.116:6825/14069] conn(0x7fb84c05cc90 0x7fb84c05f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.220+0000 7fb874adc700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.116:6824/14069,v1:172.21.15.116:6825/14069] conn(0x7fb84c05cc90 0x7fb84c05f0b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.220+0000 7fb86ffff700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb84c0557c0 0x7fb84c057c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb860007fd0 tx=0x7fb860009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.220+0000 7fb86cff9700 1 -- 172.21.15.116:0/983850357 <== osd.1 v2:172.21.15.116:6824/14069 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fb84c05f710 con 0x7fb84c05cc90 2020-11-20T12:16:07.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb8752dd700 1 -- 172.21.15.116:0/983850357 --> [v2:172.21.15.116:6824/14069,v1:172.21.15.116:6825/14069] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb8700c3a10 con 0x7fb84c05cc90 2020-11-20T12:16:07.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb86cff9700 1 -- 172.21.15.116:0/983850357 <== osd.1 v2:172.21.15.116:6824/14069 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb8700c3a10 con 0x7fb84c05cc90 2020-11-20T12:16:07.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb8752dd700 1 -- 172.21.15.116:0/983850357 >> [v2:172.21.15.116:6824/14069,v1:172.21.15.116:6825/14069] conn(0x7fb84c05cc90 msgr2=0x7fb84c05f0b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb8752dd700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.116:6824/14069,v1:172.21.15.116:6825/14069] conn(0x7fb84c05cc90 0x7fb84c05f0b0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb8752dd700 1 -- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb84c0557c0 msgr2=0x7fb84c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb8752dd700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb84c0557c0 0x7fb84c057c80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb860007fd0 tx=0x7fb860009040).stop 2020-11-20T12:16:07.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb8752dd700 1 -- 172.21.15.116:0/983850357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8700c4530 msgr2=0x7fb87013bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb8752dd700 1 --2- 172.21.15.116:0/983850357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8700c4530 0x7fb87013bae0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fb85800c710 tx=0x7fb858007b70).stop 2020-11-20T12:16:07.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb874adc700 1 -- 172.21.15.116:0/983850357 reap_dead start 2020-11-20T12:16:07.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb8752dd700 1 -- 172.21.15.116:0/983850357 shutdown_connections 2020-11-20T12:16:07.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb8752dd700 1 -- 172.21.15.116:0/983850357 >> 172.21.15.116:0/983850357 conn(0x7fb870062520 msgr2=0x7fb87014d580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:07.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb8752dd700 1 -- 172.21.15.116:0/983850357 shutdown_connections 2020-11-20T12:16:07.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.240+0000 7fb8752dd700 1 -- 172.21.15.116:0/983850357 wait complete. 2020-11-20T12:16:07.268 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:07.269 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2020-11-20T12:16:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.448+0000 7fb4e115a700 1 Processor -- start 2020-11-20T12:16:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.448+0000 7fb4e115a700 1 -- start start 2020-11-20T12:16:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.448+0000 7fb4e115a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb4dc0c4150 0x7fb4dc0c4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.448+0000 7fb4e115a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb4dc0a47d0 0x7fb4dc0a4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e115a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4dc0a5230 con 0x7fb4dc14dce0 2020-11-20T12:16:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e115a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4dc0a7220 con 0x7fb4dc0c4150 2020-11-20T12:16:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e115a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4dc0a74c0 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e0959700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb4dc14dce0 0x7fb4dc14e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52542/0 (socket says 172.21.15.116:52542) 2020-11-20T12:16:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e0959700 1 -- 172.21.15.116:0/4185871044 learned_addr learned my addr 172.21.15.116:0/4185871044 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4dbfff700 1 --2- 172.21.15.116:0/4185871044 >> v2:172.21.15.116:3300/0 conn(0x7fb4dc0c4150 0x7fb4dc0c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4db7fe700 1 --2- 172.21.15.116:0/4185871044 >> v2:172.21.15.173:3300/0 conn(0x7fb4dc0a47d0 0x7fb4dc0a4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4daffd700 1 -- 172.21.15.116:0/4185871044 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710395930 0 0) 0x7fb4dc0a5230 con 0x7fb4dc14dce0 2020-11-20T12:16:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4daffd700 1 -- 172.21.15.116:0/4185871044 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4c8003240 con 0x7fb4dc14dce0 2020-11-20T12:16:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4db7fe700 1 -- 172.21.15.116:0/4185871044 >> v2:172.21.15.116:3300/0 conn(0x7fb4dc0c4150 msgr2=0x7fb4dc0c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4db7fe700 1 --2- 172.21.15.116:0/4185871044 >> v2:172.21.15.116:3300/0 conn(0x7fb4dc0c4150 0x7fb4dc0c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4db7fe700 1 -- 172.21.15.116:0/4185871044 >> v1:172.21.15.116:6789/0 conn(0x7fb4dc14dce0 legacy=0x7fb4dc14e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4db7fe700 1 -- 172.21.15.116:0/4185871044 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc0a7760 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4db7fe700 1 --2- 172.21.15.116:0/4185871044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4dc0a47d0 0x7fb4dc0a4c60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fb4dc0c4a20 tx=0x7fb4d0006df0).ready entity=mon.1 client_cookie=4078d712f8fdeb93 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4daffd700 1 -- 172.21.15.116:0/4185871044 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d00096d0 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4daffd700 1 -- 172.21.15.116:0/4185871044 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d0009830 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4daffd700 1 -- 172.21.15.116:0/4185871044 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d0009bb0 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e115a700 1 -- 172.21.15.116:0/4185871044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4dc0a47d0 msgr2=0x7fb4dc0a4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e115a700 1 --2- 172.21.15.116:0/4185871044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4dc0a47d0 0x7fb4dc0a4c60 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fb4dc0c4a20 tx=0x7fb4d0006df0).stop 2020-11-20T12:16:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e115a700 1 -- 172.21.15.116:0/4185871044 shutdown_connections 2020-11-20T12:16:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e115a700 1 --2- 172.21.15.116:0/4185871044 >> v2:172.21.15.116:3300/0 conn(0x7fb4dc0c4150 0x7fb4dc0c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e115a700 1 --2- 172.21.15.116:0/4185871044 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4dc0a47d0 0x7fb4dc0a4c60 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e115a700 1 -- 172.21.15.116:0/4185871044 >> 172.21.15.116:0/4185871044 conn(0x7fb4dc062520 msgr2=0x7fb4dc062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e115a700 1 -- 172.21.15.116:0/4185871044 shutdown_connections 2020-11-20T12:16:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.452+0000 7fb4e115a700 1 -- 172.21.15.116:0/4185871044 wait complete. 2020-11-20T12:16:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.456+0000 7fb4e115a700 1 Processor -- start 2020-11-20T12:16:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.456+0000 7fb4e115a700 1 -- start start 2020-11-20T12:16:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.456+0000 7fb4e115a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4dc0a47d0 0x7fb4dc144c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.456+0000 7fb4e115a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4dc0c4150 0x7fb4dc145130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.456+0000 7fb4e115a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4dc14dce0 0x7fb4dc146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.456+0000 7fb4e115a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc09d790 con 0x7fb4dc0c4150 2020-11-20T12:16:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.456+0000 7fb4e115a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc14d470 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.456+0000 7fb4e115a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc14a960 con 0x7fb4dc14dce0 2020-11-20T12:16:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4dbfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4dc0a47d0 0x7fb4dc144c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4db7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4dc0c4150 0x7fb4dc145130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4dbfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4dc0a47d0 0x7fb4dc144c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:42166/0 (socket says 172.21.15.116:42166) 2020-11-20T12:16:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4dbfff700 1 -- 172.21.15.116:0/3868082430 learned_addr learned my addr 172.21.15.116:0/3868082430 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4e0959700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4dc14dce0 0x7fb4dc146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4dbfff700 1 -- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4dc14dce0 msgr2=0x7fb4dc146b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4dbfff700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4dc14dce0 0x7fb4dc146b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4dbfff700 1 -- 172.21.15.116:0/3868082430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4dc0c4150 msgr2=0x7fb4dc145130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4dbfff700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb4dc0c4150 0x7fb4dc145130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4dbfff700 1 -- 172.21.15.116:0/3868082430 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc14ac00 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4e0959700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb4dc14dce0 0x7fb4dc146b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4dbfff700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4dc0a47d0 0x7fb4dc144c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fb4cc00eba0 tx=0x7fb4cc006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4bbfff700 1 -- 172.21.15.116:0/3868082430 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00c9b0 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4bbfff700 1 -- 172.21.15.116:0/3868082430 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc00cb10 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4bbfff700 1 -- 172.21.15.116:0/3868082430 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00ce90 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4e115a700 1 -- 172.21.15.116:0/3868082430 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4dc14af00 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4e115a700 1 -- 172.21.15.116:0/3868082430 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc09fdd0 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.460+0000 7fb4e115a700 1 -- 172.21.15.116:0/3868082430 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb4dc044f60 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.464+0000 7fb4bbfff700 1 -- 172.21.15.116:0/3868082430 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb4cc012600 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.464+0000 7fb4bbfff700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb4b40557c0 0x7fb4b4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.464+0000 7fb4bbfff700 1 -- 172.21.15.116:0/3868082430 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb4cc06f4a0 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.464+0000 7fb4bbfff700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.116:6814/14054,v1:172.21.15.116:6816/14054] conn(0x7fb4b405cba0 0x7fb4b405efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.464+0000 7fb4bbfff700 1 -- 172.21.15.116:0/3868082430 --> [v2:172.21.15.116:6814/14054,v1:172.21.15.116:6816/14054] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb4b405f620 con 0x7fb4b405cba0 2020-11-20T12:16:07.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.464+0000 7fb4bbfff700 1 -- 172.21.15.116:0/3868082430 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4cc06f790 con 0x7fb4dc0a47d0 2020-11-20T12:16:07.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.468+0000 7fb4db7fe700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb4b40557c0 0x7fb4b4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.468+0000 7fb4e0959700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.116:6814/14054,v1:172.21.15.116:6816/14054] conn(0x7fb4b405cba0 0x7fb4b405efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.468+0000 7fb4e0959700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.116:6814/14054,v1:172.21.15.116:6816/14054] conn(0x7fb4b405cba0 0x7fb4b405efc0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.468+0000 7fb4db7fe700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb4b40557c0 0x7fb4b4057c80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fb4d000b150 tx=0x7fb4d0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.468+0000 7fb4bbfff700 1 -- 172.21.15.116:0/3868082430 <== osd.2 v2:172.21.15.116:6814/14054 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fb4b405f620 con 0x7fb4b405cba0 2020-11-20T12:16:07.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.488+0000 7fb4e115a700 1 -- 172.21.15.116:0/3868082430 --> [v2:172.21.15.116:6814/14054,v1:172.21.15.116:6816/14054] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb4dc14e900 con 0x7fb4b405cba0 2020-11-20T12:16:07.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.488+0000 7fb4bbfff700 1 -- 172.21.15.116:0/3868082430 <== osd.2 v2:172.21.15.116:6814/14054 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb4dc14e900 con 0x7fb4b405cba0 2020-11-20T12:16:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.488+0000 7fb4e115a700 1 -- 172.21.15.116:0/3868082430 >> [v2:172.21.15.116:6814/14054,v1:172.21.15.116:6816/14054] conn(0x7fb4b405cba0 msgr2=0x7fb4b405efc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.488+0000 7fb4e115a700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.116:6814/14054,v1:172.21.15.116:6816/14054] conn(0x7fb4b405cba0 0x7fb4b405efc0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.488+0000 7fb4e115a700 1 -- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb4b40557c0 msgr2=0x7fb4b4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.488+0000 7fb4e115a700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb4b40557c0 0x7fb4b4057c80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fb4d000b150 tx=0x7fb4d0015000).stop 2020-11-20T12:16:07.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.488+0000 7fb4e115a700 1 -- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4dc0a47d0 msgr2=0x7fb4dc144c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.488+0000 7fb4e115a700 1 --2- 172.21.15.116:0/3868082430 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb4dc0a47d0 0x7fb4dc144c40 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fb4cc00eba0 tx=0x7fb4cc006df0).stop 2020-11-20T12:16:07.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.488+0000 7fb4e0959700 1 -- 172.21.15.116:0/3868082430 reap_dead start 2020-11-20T12:16:07.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.488+0000 7fb4e115a700 1 -- 172.21.15.116:0/3868082430 shutdown_connections 2020-11-20T12:16:07.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.488+0000 7fb4e115a700 1 -- 172.21.15.116:0/3868082430 >> 172.21.15.116:0/3868082430 conn(0x7fb4dc062520 msgr2=0x7fb4dc0a9460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:07.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.492+0000 7fb4e115a700 1 -- 172.21.15.116:0/3868082430 shutdown_connections 2020-11-20T12:16:07.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.492+0000 7fb4e115a700 1 -- 172.21.15.116:0/3868082430 wait complete. 2020-11-20T12:16:07.517 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:07.518 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2020-11-20T12:16:07.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.696+0000 7f7cbad4c700 1 Processor -- start 2020-11-20T12:16:07.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.696+0000 7f7cbad4c700 1 -- start start 2020-11-20T12:16:07.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cbad4c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7cb413aee0 0x7f7cb4152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cbad4c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7cb40612f0 0x7f7cb4061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cbad4c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb4152a30 con 0x7f7cb40aa790 2020-11-20T12:16:07.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cbad4c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb40adce0 con 0x7f7cb413aee0 2020-11-20T12:16:07.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cbad4c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb4061c50 con 0x7f7cb40612f0 2020-11-20T12:16:07.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cb9d4a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7cb413aee0 0x7f7cb4152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cb9d4a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7cb413aee0 0x7f7cb4152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43126/0 (socket says 172.21.15.116:43126) 2020-11-20T12:16:07.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cb9d4a700 1 -- 172.21.15.116:0/921700435 learned_addr learned my addr 172.21.15.116:0/921700435 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:07.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cba54b700 1 --2- 172.21.15.116:0/921700435 >> v2:172.21.15.173:3300/0 conn(0x7f7cb40612f0 0x7f7cb4061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cb9d4a700 1 -- 172.21.15.116:0/921700435 >> v2:172.21.15.173:3300/0 conn(0x7f7cb40612f0 msgr2=0x7f7cb4061760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:07.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cb9d4a700 1 --2- 172.21.15.116:0/921700435 >> v2:172.21.15.173:3300/0 conn(0x7f7cb40612f0 0x7f7cb4061760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cb9d4a700 1 -- 172.21.15.116:0/921700435 >> v1:172.21.15.173:6789/0 conn(0x7f7cb40aa790 legacy=0x7f7cb40ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cb9d4a700 1 -- 172.21.15.116:0/921700435 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40aac10 con 0x7f7cb413aee0 2020-11-20T12:16:07.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.700+0000 7f7cb9d4a700 1 --2- 172.21.15.116:0/921700435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cb413aee0 0x7f7cb4152540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f7cb4081760 tx=0x7f7cb0006eb0).ready entity=mon.0 client_cookie=aedd848766dc663a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.704+0000 7f7cb8d48700 1 -- 172.21.15.116:0/921700435 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0009810 con 0x7f7cb413aee0 2020-11-20T12:16:07.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.704+0000 7f7cb8d48700 1 -- 172.21.15.116:0/921700435 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb0009970 con 0x7f7cb413aee0 2020-11-20T12:16:07.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.704+0000 7f7cb8d48700 1 -- 172.21.15.116:0/921700435 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0009cf0 con 0x7f7cb413aee0 2020-11-20T12:16:07.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.704+0000 7f7cbad4c700 1 -- 172.21.15.116:0/921700435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cb413aee0 msgr2=0x7f7cb4152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.704+0000 7f7cbad4c700 1 --2- 172.21.15.116:0/921700435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cb413aee0 0x7f7cb4152540 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f7cb4081760 tx=0x7f7cb0006eb0).stop 2020-11-20T12:16:07.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.704+0000 7f7cbad4c700 1 -- 172.21.15.116:0/921700435 shutdown_connections 2020-11-20T12:16:07.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.704+0000 7f7cbad4c700 1 --2- 172.21.15.116:0/921700435 >> v2:172.21.15.173:3300/0 conn(0x7f7cb40612f0 0x7f7cb4061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.704+0000 7f7cbad4c700 1 --2- 172.21.15.116:0/921700435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cb413aee0 0x7f7cb4152540 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.704+0000 7f7cbad4c700 1 -- 172.21.15.116:0/921700435 >> 172.21.15.116:0/921700435 conn(0x7f7cb40678a0 msgr2=0x7f7cb4067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:07.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.704+0000 7f7cbad4c700 1 -- 172.21.15.116:0/921700435 shutdown_connections 2020-11-20T12:16:07.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.704+0000 7f7cbad4c700 1 -- 172.21.15.116:0/921700435 wait complete. 2020-11-20T12:16:07.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cbad4c700 1 Processor -- start 2020-11-20T12:16:07.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cbad4c700 1 -- start start 2020-11-20T12:16:07.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cbad4c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cb40612f0 0x7f7cb40c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cbad4c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb40aa790 0x7f7cb4160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cbad4c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cb40c8ae0 0x7f7cb40a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cbad4c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb40a4ba0 con 0x7f7cb40612f0 2020-11-20T12:16:07.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cbad4c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4161940 con 0x7f7cb40c8ae0 2020-11-20T12:16:07.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cbad4c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb4161be0 con 0x7f7cb40aa790 2020-11-20T12:16:07.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cb9d4a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cb40612f0 0x7f7cb40c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cb9d4a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cb40612f0 0x7f7cb40c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43130/0 (socket says 172.21.15.116:43130) 2020-11-20T12:16:07.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cb9d4a700 1 -- 172.21.15.116:0/601169203 learned_addr learned my addr 172.21.15.116:0/601169203 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:07.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cba54b700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cb40c8ae0 0x7f7cb40a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cb9549700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb40aa790 0x7f7cb4160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cb9d4a700 1 -- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb40aa790 msgr2=0x7f7cb4160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cb9d4a700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cb40aa790 0x7f7cb4160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cb9d4a700 1 -- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cb40c8ae0 msgr2=0x7f7cb40a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cb9d4a700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cb40c8ae0 0x7f7cb40a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cb9d4a700 1 -- 172.21.15.116:0/601169203 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb40a4e00 con 0x7f7cb40612f0 2020-11-20T12:16:07.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.708+0000 7f7cba54b700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cb40c8ae0 0x7f7cb40a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.712+0000 7f7cb9d4a700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cb40612f0 0x7f7cb40c85f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7cb0010000 tx=0x7f7cb0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.712+0000 7f7ca1ffb700 1 -- 172.21.15.116:0/601169203 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0011d00 con 0x7f7cb40612f0 2020-11-20T12:16:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.712+0000 7f7ca1ffb700 1 -- 172.21.15.116:0/601169203 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cb0011e60 con 0x7f7cb40612f0 2020-11-20T12:16:07.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.712+0000 7f7cbad4c700 1 -- 172.21.15.116:0/601169203 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb40a50a0 con 0x7f7cb40612f0 2020-11-20T12:16:07.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.712+0000 7f7ca1ffb700 1 -- 172.21.15.116:0/601169203 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb001e670 con 0x7f7cb40612f0 2020-11-20T12:16:07.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.712+0000 7f7cbad4c700 1 -- 172.21.15.116:0/601169203 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb41613d0 con 0x7f7cb40612f0 2020-11-20T12:16:07.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.712+0000 7f7cbad4c700 1 -- 172.21.15.116:0/601169203 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7cb4044f60 con 0x7f7cb40612f0 2020-11-20T12:16:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.712+0000 7f7ca1ffb700 1 -- 172.21.15.116:0/601169203 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7cb0025030 con 0x7f7cb40612f0 2020-11-20T12:16:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.716+0000 7f7ca1ffb700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c8c0557c0 0x7f7c8c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.716+0000 7f7ca1ffb700 1 -- 172.21.15.116:0/601169203 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7cb006ff50 con 0x7f7cb40612f0 2020-11-20T12:16:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.716+0000 7f7ca1ffb700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x7f7c8c05cd10 0x7f7c8c05f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.716+0000 7f7ca1ffb700 1 -- 172.21.15.116:0/601169203 --> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7c8c05f790 con 0x7f7c8c05cd10 2020-11-20T12:16:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.716+0000 7f7ca1ffb700 1 -- 172.21.15.116:0/601169203 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7cb00754e0 con 0x7f7cb40612f0 2020-11-20T12:16:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.716+0000 7f7cb9549700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c8c0557c0 0x7f7c8c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.716+0000 7f7cba54b700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x7f7c8c05cd10 0x7f7c8c05f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.716+0000 7f7cba54b700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x7f7c8c05cd10 0x7f7c8c05f130 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.716+0000 7f7cb9549700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c8c0557c0 0x7f7c8c057c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7ca4003450 tx=0x7f7ca4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.720+0000 7f7ca1ffb700 1 -- 172.21.15.116:0/601169203 <== osd.3 v2:172.21.15.116:6802/14055 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f7c8c05f790 con 0x7f7c8c05cd10 2020-11-20T12:16:07.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.736+0000 7f7cbad4c700 1 -- 172.21.15.116:0/601169203 --> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7cb4161680 con 0x7f7c8c05cd10 2020-11-20T12:16:07.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7ca1ffb700 1 -- 172.21.15.116:0/601169203 <== osd.3 v2:172.21.15.116:6802/14055 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7cb4161680 con 0x7f7c8c05cd10 2020-11-20T12:16:07.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7cbad4c700 1 -- 172.21.15.116:0/601169203 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x7f7c8c05cd10 msgr2=0x7f7c8c05f130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7cbad4c700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x7f7c8c05cd10 0x7f7c8c05f130 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7cbad4c700 1 -- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c8c0557c0 msgr2=0x7f7c8c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7cbad4c700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c8c0557c0 0x7f7c8c057c80 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f7ca4003450 tx=0x7f7ca4009040).stop 2020-11-20T12:16:07.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7cbad4c700 1 -- 172.21.15.116:0/601169203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cb40612f0 msgr2=0x7f7cb40c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7cbad4c700 1 --2- 172.21.15.116:0/601169203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cb40612f0 0x7f7cb40c85f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f7cb0010000 tx=0x7f7cb0015000).stop 2020-11-20T12:16:07.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7cba54b700 1 -- 172.21.15.116:0/601169203 reap_dead start 2020-11-20T12:16:07.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7cbad4c700 1 -- 172.21.15.116:0/601169203 shutdown_connections 2020-11-20T12:16:07.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7cbad4c700 1 -- 172.21.15.116:0/601169203 >> 172.21.15.116:0/601169203 conn(0x7f7cb40678a0 msgr2=0x7f7cb4153f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:07.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7cbad4c700 1 -- 172.21.15.116:0/601169203 shutdown_connections 2020-11-20T12:16:07.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.740+0000 7f7cbad4c700 1 -- 172.21.15.116:0/601169203 wait complete. 2020-11-20T12:16:07.765 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:07.766 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2020-11-20T12:16:07.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.952+0000 7f6b8d30d700 1 Processor -- start 2020-11-20T12:16:07.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.952+0000 7f6b8d30d700 1 -- start start 2020-11-20T12:16:07.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.952+0000 7f6b8d30d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6b881488f0 0x7f6b8814f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.952+0000 7f6b8d30d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b880a5220 0x7f6b88060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.952+0000 7f6b8d30d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b880a57a0 con 0x7f6b88060660 2020-11-20T12:16:07.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.952+0000 7f6b8d30d700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b880a5a40 con 0x7f6b881488f0 2020-11-20T12:16:07.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.952+0000 7f6b8d30d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b880a5ce0 con 0x7f6b880a5220 2020-11-20T12:16:07.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b87fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6b88060660 0x7f6b88067700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52576/0 (socket says 172.21.15.116:52576) 2020-11-20T12:16:07.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b87fff700 1 -- 172.21.15.116:0/894802623 learned_addr learned my addr 172.21.15.116:0/894802623 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:07.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b867fc700 1 --2- 172.21.15.116:0/894802623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b880a5220 0x7f6b88060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b86ffd700 1 --2- 172.21.15.116:0/894802623 >> v2:172.21.15.173:3300/0 conn(0x7f6b881488f0 0x7f6b8814f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b85ffb700 1 -- 172.21.15.116:0/894802623 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645877689 0 0) 0x7f6b880a57a0 con 0x7f6b88060660 2020-11-20T12:16:07.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b85ffb700 1 -- 172.21.15.116:0/894802623 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b74003240 con 0x7f6b88060660 2020-11-20T12:16:07.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b867fc700 1 -- 172.21.15.116:0/894802623 >> v2:172.21.15.173:3300/0 conn(0x7f6b881488f0 msgr2=0x7f6b8814f310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:07.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b867fc700 1 --2- 172.21.15.116:0/894802623 >> v2:172.21.15.173:3300/0 conn(0x7f6b881488f0 0x7f6b8814f310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b85ffb700 1 -- 172.21.15.116:0/894802623 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3948614466 0 0) 0x7f6b74003240 con 0x7f6b88060660 2020-11-20T12:16:07.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b867fc700 1 -- 172.21.15.116:0/894802623 >> v1:172.21.15.116:6789/0 conn(0x7f6b88060660 legacy=0x7f6b88067700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b867fc700 1 -- 172.21.15.116:0/894802623 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b880a5f80 con 0x7f6b880a5220 2020-11-20T12:16:07.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b867fc700 1 --2- 172.21.15.116:0/894802623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b880a5220 0x7f6b88060170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6b7c00b9a0 tx=0x7f6b7c006df0).ready entity=mon.2 client_cookie=69606d4e4dd036e1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b85ffb700 1 -- 172.21.15.116:0/894802623 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b80001860 con 0x7f6b880a5220 2020-11-20T12:16:07.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b85ffb700 1 -- 172.21.15.116:0/894802623 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7c011400 con 0x7f6b880a5220 2020-11-20T12:16:07.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.956+0000 7f6b85ffb700 1 -- 172.21.15.116:0/894802623 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c011780 con 0x7f6b880a5220 2020-11-20T12:16:07.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.960+0000 7f6b8d30d700 1 -- 172.21.15.116:0/894802623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b880a5220 msgr2=0x7f6b88060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.960+0000 7f6b8d30d700 1 --2- 172.21.15.116:0/894802623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b880a5220 0x7f6b88060170 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f6b7c00b9a0 tx=0x7f6b7c006df0).stop 2020-11-20T12:16:07.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.960+0000 7f6b8d30d700 1 -- 172.21.15.116:0/894802623 shutdown_connections 2020-11-20T12:16:07.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.960+0000 7f6b8d30d700 1 --2- 172.21.15.116:0/894802623 >> v2:172.21.15.173:3300/0 conn(0x7f6b881488f0 0x7f6b8814f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.960+0000 7f6b8d30d700 1 --2- 172.21.15.116:0/894802623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b880a5220 0x7f6b88060170 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.960+0000 7f6b8d30d700 1 -- 172.21.15.116:0/894802623 >> 172.21.15.116:0/894802623 conn(0x7f6b880c5670 msgr2=0x7f6b880c5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:07.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.960+0000 7f6b8d30d700 1 -- 172.21.15.116:0/894802623 shutdown_connections 2020-11-20T12:16:07.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.960+0000 7f6b8d30d700 1 -- 172.21.15.116:0/894802623 wait complete. 2020-11-20T12:16:07.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b8d30d700 1 Processor -- start 2020-11-20T12:16:07.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b8d30d700 1 -- start start 2020-11-20T12:16:07.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b8d30d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b88060660 0x7f6b88136730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b8d30d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b880a5220 0x7f6b88137220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b8d30d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b881488f0 0x7f6b88134390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b8d30d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8813e4e0 con 0x7f6b880a5220 2020-11-20T12:16:07.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b8d30d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b8813f630 con 0x7f6b88060660 2020-11-20T12:16:07.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b8d30d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b88137e00 con 0x7f6b881488f0 2020-11-20T12:16:07.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b867fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b880a5220 0x7f6b88137220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b867fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b880a5220 0x7f6b88137220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43148/0 (socket says 172.21.15.116:43148) 2020-11-20T12:16:07.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b867fc700 1 -- 172.21.15.116:0/1592713910 learned_addr learned my addr 172.21.15.116:0/1592713910 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:07.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b86ffd700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b88060660 0x7f6b88136730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b87fff700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b881488f0 0x7f6b88134390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b867fc700 1 -- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b881488f0 msgr2=0x7f6b88134390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.964+0000 7f6b867fc700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6b881488f0 0x7f6b88134390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.968+0000 7f6b867fc700 1 -- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b88060660 msgr2=0x7f6b88136730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:07.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.968+0000 7f6b867fc700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6b88060660 0x7f6b88136730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:07.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.968+0000 7f6b867fc700 1 -- 172.21.15.116:0/1592713910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b8813e710 con 0x7f6b880a5220 2020-11-20T12:16:07.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.968+0000 7f6b867fc700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b880a5220 0x7f6b88137220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f6b7c006fd0 tx=0x7f6b7c013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.968+0000 7f6b6effd700 1 -- 172.21.15.116:0/1592713910 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c009bd0 con 0x7f6b880a5220 2020-11-20T12:16:07.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.968+0000 7f6b6effd700 1 -- 172.21.15.116:0/1592713910 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7c009d30 con 0x7f6b880a5220 2020-11-20T12:16:07.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.968+0000 7f6b8d30d700 1 -- 172.21.15.116:0/1592713910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b8813e9b0 con 0x7f6b880a5220 2020-11-20T12:16:07.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.968+0000 7f6b6effd700 1 -- 172.21.15.116:0/1592713910 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7c01e670 con 0x7f6b880a5220 2020-11-20T12:16:07.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.968+0000 7f6b8d30d700 1 -- 172.21.15.116:0/1592713910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b8813f190 con 0x7f6b880a5220 2020-11-20T12:16:07.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.968+0000 7f6b8d30d700 1 -- 172.21.15.116:0/1592713910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6b8805ca30 con 0x7f6b880a5220 2020-11-20T12:16:07.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.968+0000 7f6b6effd700 1 -- 172.21.15.116:0/1592713910 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6b7c025030 con 0x7f6b880a5220 2020-11-20T12:16:07.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.972+0000 7f6b6effd700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6b640557c0 0x7f6b64057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.972+0000 7f6b6effd700 1 -- 172.21.15.116:0/1592713910 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6b7c06fb50 con 0x7f6b880a5220 2020-11-20T12:16:07.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.972+0000 7f6b6effd700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] conn(0x7f6b6405cd10 0x7f6b6405f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:07.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.972+0000 7f6b6effd700 1 -- 172.21.15.116:0/1592713910 --> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6b6405f790 con 0x7f6b6405cd10 2020-11-20T12:16:07.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.972+0000 7f6b6effd700 1 -- 172.21.15.116:0/1592713910 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b7c074220 con 0x7f6b880a5220 2020-11-20T12:16:07.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.972+0000 7f6b86ffd700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6b640557c0 0x7f6b64057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.972+0000 7f6b87fff700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] conn(0x7f6b6405cd10 0x7f6b6405f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:07.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.972+0000 7f6b86ffd700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6b640557c0 0x7f6b64057c80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f6b78006fd0 tx=0x7f6b78006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.972+0000 7f6b87fff700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] conn(0x7f6b6405cd10 0x7f6b6405f130 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:07.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.976+0000 7f6b6effd700 1 -- 172.21.15.116:0/1592713910 <== osd.4 v2:172.21.15.173:6802/14206 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f6b6405f790 con 0x7f6b6405cd10 2020-11-20T12:16:07.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.996+0000 7f6b8d30d700 1 -- 172.21.15.116:0/1592713910 --> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6b880625f0 con 0x7f6b6405cd10 2020-11-20T12:16:08.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.996+0000 7f6b6effd700 1 -- 172.21.15.116:0/1592713910 <== osd.4 v2:172.21.15.173:6802/14206 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6b880625f0 con 0x7f6b6405cd10 2020-11-20T12:16:08.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.996+0000 7f6b8d30d700 1 -- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] conn(0x7f6b6405cd10 msgr2=0x7f6b6405f130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.996+0000 7f6b8d30d700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] conn(0x7f6b6405cd10 0x7f6b6405f130 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.996+0000 7f6b8d30d700 1 -- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6b640557c0 msgr2=0x7f6b64057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.996+0000 7f6b8d30d700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6b640557c0 0x7f6b64057c80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f6b78006fd0 tx=0x7f6b78006df0).stop 2020-11-20T12:16:08.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.996+0000 7f6b8d30d700 1 -- 172.21.15.116:0/1592713910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b880a5220 msgr2=0x7f6b88137220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.996+0000 7f6b8d30d700 1 --2- 172.21.15.116:0/1592713910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6b880a5220 0x7f6b88137220 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f6b7c006fd0 tx=0x7f6b7c013000).stop 2020-11-20T12:16:08.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:07.996+0000 7f6b87fff700 1 -- 172.21.15.116:0/1592713910 reap_dead start 2020-11-20T12:16:08.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.000+0000 7f6b8d30d700 1 -- 172.21.15.116:0/1592713910 shutdown_connections 2020-11-20T12:16:08.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.000+0000 7f6b8d30d700 1 -- 172.21.15.116:0/1592713910 >> 172.21.15.116:0/1592713910 conn(0x7f6b880c5670 msgr2=0x7f6b88067700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:08.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.000+0000 7f6b8d30d700 1 -- 172.21.15.116:0/1592713910 shutdown_connections 2020-11-20T12:16:08.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.000+0000 7f6b8d30d700 1 -- 172.21.15.116:0/1592713910 wait complete. 2020-11-20T12:16:08.021 INFO:teuthology.orchestra.run.smithi116.stdout:60129542150 2020-11-20T12:16:08.022 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2020-11-20T12:16:08.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd71c2f700 1 Processor -- start 2020-11-20T12:16:08.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd71c2f700 1 -- start start 2020-11-20T12:16:08.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd71c2f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdd6c0c45a0 0x7fdd6c138670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd71c2f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdd6c138b60 0x7fdd6c0a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd71c2f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6c0a9180 con 0x7fdd6c0c41c0 2020-11-20T12:16:08.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd71c2f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c0a9420 con 0x7fdd6c138b60 2020-11-20T12:16:08.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd71c2f700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd6c0a96c0 con 0x7fdd6c0c45a0 2020-11-20T12:16:08.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd6bfff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdd6c0c45a0 0x7fdd6c138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd70c2d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdd6c0c41c0 0x7fdd6c0a5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52590/0 (socket says 172.21.15.116:52590) 2020-11-20T12:16:08.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd70c2d700 1 -- 172.21.15.116:0/3826040976 learned_addr learned my addr 172.21.15.116:0/3826040976 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:08.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd7142e700 1 --2- 172.21.15.116:0/3826040976 >> v2:172.21.15.116:3300/0 conn(0x7fdd6c138b60 0x7fdd6c0a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd6bfff700 1 -- 172.21.15.116:0/3826040976 >> v2:172.21.15.116:3300/0 conn(0x7fdd6c138b60 msgr2=0x7fdd6c0a7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:08.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd6bfff700 1 --2- 172.21.15.116:0/3826040976 >> v2:172.21.15.116:3300/0 conn(0x7fdd6c138b60 0x7fdd6c0a7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd6bfff700 1 -- 172.21.15.116:0/3826040976 >> v1:172.21.15.116:6789/0 conn(0x7fdd6c0c41c0 legacy=0x7fdd6c0a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd6bfff700 1 -- 172.21.15.116:0/3826040976 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0a9960 con 0x7fdd6c0c45a0 2020-11-20T12:16:08.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.200+0000 7fdd6b7fe700 1 -- 172.21.15.116:0/3826040976 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2653628512 0 0) 0x7fdd6c0a9180 con 0x7fdd6c0c41c0 2020-11-20T12:16:08.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd6bfff700 1 --2- 172.21.15.116:0/3826040976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd6c0c45a0 0x7fdd6c138670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fdd6c152e20 tx=0x7fdd54006df0).ready entity=mon.1 client_cookie=35c3a30d26fd8334 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd6b7fe700 1 -- 172.21.15.116:0/3826040976 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd540097b0 con 0x7fdd6c0c45a0 2020-11-20T12:16:08.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd6b7fe700 1 -- 172.21.15.116:0/3826040976 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd54009910 con 0x7fdd6c0c45a0 2020-11-20T12:16:08.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd6b7fe700 1 -- 172.21.15.116:0/3826040976 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd54009c90 con 0x7fdd6c0c45a0 2020-11-20T12:16:08.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd71c2f700 1 -- 172.21.15.116:0/3826040976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd6c0c45a0 msgr2=0x7fdd6c138670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd71c2f700 1 --2- 172.21.15.116:0/3826040976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd6c0c45a0 0x7fdd6c138670 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fdd6c152e20 tx=0x7fdd54006df0).stop 2020-11-20T12:16:08.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd71c2f700 1 -- 172.21.15.116:0/3826040976 shutdown_connections 2020-11-20T12:16:08.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd71c2f700 1 --2- 172.21.15.116:0/3826040976 >> v2:172.21.15.116:3300/0 conn(0x7fdd6c138b60 0x7fdd6c0a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd71c2f700 1 --2- 172.21.15.116:0/3826040976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd6c0c45a0 0x7fdd6c138670 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd71c2f700 1 -- 172.21.15.116:0/3826040976 >> 172.21.15.116:0/3826040976 conn(0x7fdd6c062520 msgr2=0x7fdd6c062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:08.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd71c2f700 1 -- 172.21.15.116:0/3826040976 shutdown_connections 2020-11-20T12:16:08.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd71c2f700 1 -- 172.21.15.116:0/3826040976 wait complete. 2020-11-20T12:16:08.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd71c2f700 1 Processor -- start 2020-11-20T12:16:08.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.204+0000 7fdd71c2f700 1 -- start start 2020-11-20T12:16:08.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd71c2f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd6c0c41c0 0x7fdd6c0a1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd71c2f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd6c138b60 0x7fdd6c0a1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd71c2f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd6c0a0d50 0x7fdd6c137690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd71c2f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c09e1b0 con 0x7fdd6c0a0d50 2020-11-20T12:16:08.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd71c2f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c09e450 con 0x7fdd6c138b60 2020-11-20T12:16:08.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd71c2f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c09e6f0 con 0x7fdd6c0c41c0 2020-11-20T12:16:08.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd7142e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd6c0a0d50 0x7fdd6c137690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd7142e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd6c0a0d50 0x7fdd6c137690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43162/0 (socket says 172.21.15.116:43162) 2020-11-20T12:16:08.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd7142e700 1 -- 172.21.15.116:0/718581380 learned_addr learned my addr 172.21.15.116:0/718581380 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:08.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd70c2d700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd6c0c41c0 0x7fdd6c0a1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd6bfff700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd6c138b60 0x7fdd6c0a1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd7142e700 1 -- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd6c0c41c0 msgr2=0x7fdd6c0a1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd7142e700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdd6c0c41c0 0x7fdd6c0a1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd7142e700 1 -- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd6c138b60 msgr2=0x7fdd6c0a1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd7142e700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd6c138b60 0x7fdd6c0a1a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd7142e700 1 -- 172.21.15.116:0/718581380 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c09e990 con 0x7fdd6c0a0d50 2020-11-20T12:16:08.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd6bfff700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdd6c138b60 0x7fdd6c0a1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:08.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd7142e700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd6c0a0d50 0x7fdd6c137690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fdd6400ef50 tx=0x7fdd640091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd68ff9700 1 -- 172.21.15.116:0/718581380 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6400cc80 con 0x7fdd6c0a0d50 2020-11-20T12:16:08.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd68ff9700 1 -- 172.21.15.116:0/718581380 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6400cde0 con 0x7fdd6c0a0d50 2020-11-20T12:16:08.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd71c2f700 1 -- 172.21.15.116:0/718581380 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c09ec30 con 0x7fdd6c0a0d50 2020-11-20T12:16:08.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd68ff9700 1 -- 172.21.15.116:0/718581380 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64012620 con 0x7fdd6c0a0d50 2020-11-20T12:16:08.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.208+0000 7fdd71c2f700 1 -- 172.21.15.116:0/718581380 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c137b80 con 0x7fdd6c0a0d50 2020-11-20T12:16:08.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.212+0000 7fdd71c2f700 1 -- 172.21.15.116:0/718581380 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdd6c044f60 con 0x7fdd6c0a0d50 2020-11-20T12:16:08.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.212+0000 7fdd68ff9700 1 -- 172.21.15.116:0/718581380 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fdd64022070 con 0x7fdd6c0a0d50 2020-11-20T12:16:08.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.212+0000 7fdd68ff9700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdd480557c0 0x7fdd48057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.212+0000 7fdd68ff9700 1 -- 172.21.15.116:0/718581380 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdd6406efa0 con 0x7fdd6c0a0d50 2020-11-20T12:16:08.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.212+0000 7fdd68ff9700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:6818/14215,v1:172.21.15.173:6821/14215] conn(0x7fdd4805cd10 0x7fdd4805f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.212+0000 7fdd68ff9700 1 -- 172.21.15.116:0/718581380 --> [v2:172.21.15.173:6818/14215,v1:172.21.15.173:6821/14215] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdd4805f790 con 0x7fdd4805cd10 2020-11-20T12:16:08.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.212+0000 7fdd68ff9700 1 -- 172.21.15.116:0/718581380 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd64071610 con 0x7fdd6c0a0d50 2020-11-20T12:16:08.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.212+0000 7fdd70c2d700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdd480557c0 0x7fdd48057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.216+0000 7fdd6bfff700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:6818/14215,v1:172.21.15.173:6821/14215] conn(0x7fdd4805cd10 0x7fdd4805f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.216+0000 7fdd70c2d700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdd480557c0 0x7fdd48057c80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fdd5c0011a0 tx=0x7fdd5c0079e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.216+0000 7fdd6bfff700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:6818/14215,v1:172.21.15.173:6821/14215] conn(0x7fdd4805cd10 0x7fdd4805f130 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.216+0000 7fdd68ff9700 1 -- 172.21.15.116:0/718581380 <== osd.5 v2:172.21.15.173:6818/14215 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fdd4805f790 con 0x7fdd4805cd10 2020-11-20T12:16:08.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.236+0000 7fdd71c2f700 1 -- 172.21.15.116:0/718581380 --> [v2:172.21.15.173:6818/14215,v1:172.21.15.173:6821/14215] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdd6c1380f0 con 0x7fdd4805cd10 2020-11-20T12:16:08.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd68ff9700 1 -- 172.21.15.116:0/718581380 <== osd.5 v2:172.21.15.173:6818/14215 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdd6c1380f0 con 0x7fdd4805cd10 2020-11-20T12:16:08.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd71c2f700 1 -- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:6818/14215,v1:172.21.15.173:6821/14215] conn(0x7fdd4805cd10 msgr2=0x7fdd4805f130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd71c2f700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:6818/14215,v1:172.21.15.173:6821/14215] conn(0x7fdd4805cd10 0x7fdd4805f130 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd71c2f700 1 -- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdd480557c0 msgr2=0x7fdd48057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd71c2f700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdd480557c0 0x7fdd48057c80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fdd5c0011a0 tx=0x7fdd5c0079e0).stop 2020-11-20T12:16:08.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd71c2f700 1 -- 172.21.15.116:0/718581380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd6c0a0d50 msgr2=0x7fdd6c137690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd71c2f700 1 --2- 172.21.15.116:0/718581380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdd6c0a0d50 0x7fdd6c137690 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fdd6400ef50 tx=0x7fdd640091d0).stop 2020-11-20T12:16:08.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd7142e700 1 -- 172.21.15.116:0/718581380 reap_dead start 2020-11-20T12:16:08.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd71c2f700 1 -- 172.21.15.116:0/718581380 shutdown_connections 2020-11-20T12:16:08.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd71c2f700 1 -- 172.21.15.116:0/718581380 >> 172.21.15.116:0/718581380 conn(0x7fdd6c062520 msgr2=0x7fdd6c0a7f90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:08.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd71c2f700 1 -- 172.21.15.116:0/718581380 shutdown_connections 2020-11-20T12:16:08.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.240+0000 7fdd71c2f700 1 -- 172.21.15.116:0/718581380 wait complete. 2020-11-20T12:16:08.268 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:08.268 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2020-11-20T12:16:08.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.448+0000 7fb33b59e700 1 Processor -- start 2020-11-20T12:16:08.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.448+0000 7fb33b59e700 1 -- start start 2020-11-20T12:16:08.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.452+0000 7fb33b59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb33c13aee0 0x7fb33c152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.452+0000 7fb33b59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb33c0612f0 0x7fb33c061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.452+0000 7fb33b59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb33c152a30 con 0x7fb33c0aa790 2020-11-20T12:16:08.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.452+0000 7fb33b59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c0adce0 con 0x7fb33c0612f0 2020-11-20T12:16:08.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.452+0000 7fb33b59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb33c061c50 con 0x7fb33c13aee0 2020-11-20T12:16:08.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33ad9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb33c0612f0 0x7fb33c061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33a59c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb33c13aee0 0x7fb33c152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33ad9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb33c0612f0 0x7fb33c061760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43172/0 (socket says 172.21.15.116:43172) 2020-11-20T12:16:08.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33ad9d700 1 -- 172.21.15.116:0/447190017 learned_addr learned my addr 172.21.15.116:0/447190017 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:08.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb339d9b700 1 --1- 172.21.15.116:0/447190017 >> v1:172.21.15.116:6789/0 conn(0x7fb33c0aa790 0x7fb33c0ad5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52604/0 (socket says 172.21.15.116:52604) 2020-11-20T12:16:08.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33ad9d700 1 -- 172.21.15.116:0/447190017 >> v2:172.21.15.173:3300/0 conn(0x7fb33c13aee0 msgr2=0x7fb33c152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:08.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33ad9d700 1 --2- 172.21.15.116:0/447190017 >> v2:172.21.15.173:3300/0 conn(0x7fb33c13aee0 0x7fb33c152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33ad9d700 1 -- 172.21.15.116:0/447190017 >> v1:172.21.15.116:6789/0 conn(0x7fb33c0aa790 legacy=0x7fb33c0ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33ad9d700 1 -- 172.21.15.116:0/447190017 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c0aac10 con 0x7fb33c0612f0 2020-11-20T12:16:08.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33ad9d700 1 --2- 172.21.15.116:0/447190017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb33c0612f0 0x7fb33c061760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fb33c081470 tx=0x7fb334009290).ready entity=mon.0 client_cookie=bc66c7be0deec55d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33959a700 1 -- 172.21.15.116:0/447190017 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb33400bbc0 con 0x7fb33c0612f0 2020-11-20T12:16:08.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33959a700 1 -- 172.21.15.116:0/447190017 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb33400bd20 con 0x7fb33c0612f0 2020-11-20T12:16:08.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33959a700 1 -- 172.21.15.116:0/447190017 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb334011610 con 0x7fb33c0612f0 2020-11-20T12:16:08.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33b59e700 1 -- 172.21.15.116:0/447190017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb33c0612f0 msgr2=0x7fb33c061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33b59e700 1 --2- 172.21.15.116:0/447190017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb33c0612f0 0x7fb33c061760 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fb33c081470 tx=0x7fb334009290).stop 2020-11-20T12:16:08.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33b59e700 1 -- 172.21.15.116:0/447190017 shutdown_connections 2020-11-20T12:16:08.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33b59e700 1 --2- 172.21.15.116:0/447190017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb33c0612f0 0x7fb33c061760 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33b59e700 1 --2- 172.21.15.116:0/447190017 >> v2:172.21.15.173:3300/0 conn(0x7fb33c13aee0 0x7fb33c152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.456+0000 7fb33b59e700 1 -- 172.21.15.116:0/447190017 >> 172.21.15.116:0/447190017 conn(0x7fb33c0678a0 msgr2=0x7fb33c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:08.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33b59e700 1 -- 172.21.15.116:0/447190017 shutdown_connections 2020-11-20T12:16:08.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33b59e700 1 -- 172.21.15.116:0/447190017 wait complete. 2020-11-20T12:16:08.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33b59e700 1 Processor -- start 2020-11-20T12:16:08.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33b59e700 1 -- start start 2020-11-20T12:16:08.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb33c0612f0 0x7fb33c0a40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb33c0aa790 0x7fb33c0a0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb33c13aee0 0x7fb33c0a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0a61b0 con 0x7fb33c0aa790 2020-11-20T12:16:08.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0a22e0 con 0x7fb33c13aee0 2020-11-20T12:16:08.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c0a2580 con 0x7fb33c0612f0 2020-11-20T12:16:08.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb339d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb33c0aa790 0x7fb33c0a0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb339d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb33c0aa790 0x7fb33c0a0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43174/0 (socket says 172.21.15.116:43174) 2020-11-20T12:16:08.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb339d9b700 1 -- 172.21.15.116:0/2886337118 learned_addr learned my addr 172.21.15.116:0/2886337118 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:08.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33a59c700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb33c0612f0 0x7fb33c0a40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.460+0000 7fb33ad9d700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb33c13aee0 0x7fb33c0a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb339d9b700 1 -- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb33c0612f0 msgr2=0x7fb33c0a40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb339d9b700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb33c0612f0 0x7fb33c0a40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb339d9b700 1 -- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb33c13aee0 msgr2=0x7fb33c0a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb339d9b700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb33c13aee0 0x7fb33c0a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb339d9b700 1 -- 172.21.15.116:0/2886337118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb33c0a2820 con 0x7fb33c0aa790 2020-11-20T12:16:08.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb339d9b700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb33c0aa790 0x7fb33c0a0810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fb33000c710 tx=0x7fb330007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb32a7fc700 1 -- 172.21.15.116:0/2886337118 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb330007320 con 0x7fb33c0aa790 2020-11-20T12:16:08.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb32a7fc700 1 -- 172.21.15.116:0/2886337118 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb330005530 con 0x7fb33c0aa790 2020-11-20T12:16:08.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb32a7fc700 1 -- 172.21.15.116:0/2886337118 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3300058b0 con 0x7fb33c0aa790 2020-11-20T12:16:08.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb33b59e700 1 -- 172.21.15.116:0/2886337118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c0a2ac0 con 0x7fb33c0aa790 2020-11-20T12:16:08.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb33b59e700 1 -- 172.21.15.116:0/2886337118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c0a45e0 con 0x7fb33c0aa790 2020-11-20T12:16:08.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb33b59e700 1 -- 172.21.15.116:0/2886337118 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb33c044f60 con 0x7fb33c0aa790 2020-11-20T12:16:08.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb32a7fc700 1 -- 172.21.15.116:0/2886337118 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb330010030 con 0x7fb33c0aa790 2020-11-20T12:16:08.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.464+0000 7fb32a7fc700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3100557c0 0x7fb310057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.468+0000 7fb32a7fc700 1 -- 172.21.15.116:0/2886337118 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb33006e380 con 0x7fb33c0aa790 2020-11-20T12:16:08.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.468+0000 7fb32a7fc700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:6810/14207,v1:172.21.15.173:6811/14207] conn(0x7fb31005cd10 0x7fb31005f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.468+0000 7fb32a7fc700 1 -- 172.21.15.116:0/2886337118 --> [v2:172.21.15.173:6810/14207,v1:172.21.15.173:6811/14207] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb31005f790 con 0x7fb31005cd10 2020-11-20T12:16:08.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.468+0000 7fb32a7fc700 1 -- 172.21.15.116:0/2886337118 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3300709f0 con 0x7fb33c0aa790 2020-11-20T12:16:08.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.468+0000 7fb33a59c700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3100557c0 0x7fb310057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.468+0000 7fb33ad9d700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:6810/14207,v1:172.21.15.173:6811/14207] conn(0x7fb31005cd10 0x7fb31005f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.468+0000 7fb33a59c700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3100557c0 0x7fb310057c80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fb32c006fd0 tx=0x7fb32c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.468+0000 7fb33ad9d700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:6810/14207,v1:172.21.15.173:6811/14207] conn(0x7fb31005cd10 0x7fb31005f130 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.472+0000 7fb32a7fc700 1 -- 172.21.15.116:0/2886337118 <== osd.6 v2:172.21.15.173:6810/14207 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7fb31005f790 con 0x7fb31005cd10 2020-11-20T12:16:08.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.488+0000 7fb33b59e700 1 -- 172.21.15.116:0/2886337118 --> [v2:172.21.15.173:6810/14207,v1:172.21.15.173:6811/14207] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb33c0a31a0 con 0x7fb31005cd10 2020-11-20T12:16:08.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb32a7fc700 1 -- 172.21.15.116:0/2886337118 <== osd.6 v2:172.21.15.173:6810/14207 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb33c0a31a0 con 0x7fb31005cd10 2020-11-20T12:16:08.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb33b59e700 1 -- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:6810/14207,v1:172.21.15.173:6811/14207] conn(0x7fb31005cd10 msgr2=0x7fb31005f130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb33b59e700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:6810/14207,v1:172.21.15.173:6811/14207] conn(0x7fb31005cd10 0x7fb31005f130 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb33b59e700 1 -- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3100557c0 msgr2=0x7fb310057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb33b59e700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3100557c0 0x7fb310057c80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fb32c006fd0 tx=0x7fb32c008040).stop 2020-11-20T12:16:08.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb33b59e700 1 -- 172.21.15.116:0/2886337118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb33c0aa790 msgr2=0x7fb33c0a0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb33b59e700 1 --2- 172.21.15.116:0/2886337118 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb33c0aa790 0x7fb33c0a0810 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fb33000c710 tx=0x7fb330007c40).stop 2020-11-20T12:16:08.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb33ad9d700 1 -- 172.21.15.116:0/2886337118 reap_dead start 2020-11-20T12:16:08.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb33b59e700 1 -- 172.21.15.116:0/2886337118 shutdown_connections 2020-11-20T12:16:08.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb33b59e700 1 -- 172.21.15.116:0/2886337118 >> 172.21.15.116:0/2886337118 conn(0x7fb33c0678a0 msgr2=0x7fb33c0c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:08.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb33b59e700 1 -- 172.21.15.116:0/2886337118 shutdown_connections 2020-11-20T12:16:08.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.492+0000 7fb33b59e700 1 -- 172.21.15.116:0/2886337118 wait complete. 2020-11-20T12:16:08.516 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:08.516 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2020-11-20T12:16:08.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.692+0000 7f78d9dc4700 1 Processor -- start 2020-11-20T12:16:08.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.692+0000 7f78d9dc4700 1 -- start start 2020-11-20T12:16:08.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d9dc4700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f78d4061700 0x7f78d40b9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d9dc4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f78d40b9d40 0x7f78d41510b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d9dc4700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d40ba2b0 con 0x7f78d4061320 2020-11-20T12:16:08.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d9dc4700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d40ba550 con 0x7f78d40b9d40 2020-11-20T12:16:08.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d9dc4700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d40ba7f0 con 0x7f78d4061700 2020-11-20T12:16:08.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d95c3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f78d40b9d40 0x7f78d41510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d95c3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f78d40b9d40 0x7f78d41510b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43186/0 (socket says 172.21.15.116:43186) 2020-11-20T12:16:08.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d95c3700 1 -- 172.21.15.116:0/1918467364 learned_addr learned my addr 172.21.15.116:0/1918467364 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:08.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d3fff700 1 --2- 172.21.15.116:0/1918467364 >> v2:172.21.15.173:3300/0 conn(0x7f78d4061700 0x7f78d40b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d95c3700 1 -- 172.21.15.116:0/1918467364 >> v2:172.21.15.173:3300/0 conn(0x7f78d4061700 msgr2=0x7f78d40b9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:08.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d95c3700 1 --2- 172.21.15.116:0/1918467364 >> v2:172.21.15.173:3300/0 conn(0x7f78d4061700 0x7f78d40b9850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d95c3700 1 -- 172.21.15.116:0/1918467364 >> v1:172.21.15.173:6789/0 conn(0x7f78d4061320 legacy=0x7f78d40bb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d95c3700 1 -- 172.21.15.116:0/1918467364 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d40baa90 con 0x7f78d40b9d40 2020-11-20T12:16:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.696+0000 7f78d95c3700 1 --2- 172.21.15.116:0/1918467364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78d40b9d40 0x7f78d41510b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f78d41550f0 tx=0x7f78cc009290).ready entity=mon.0 client_cookie=a6e0cf0746ef2563 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.700+0000 7f78d37fe700 1 -- 172.21.15.116:0/1918467364 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc00bbc0 con 0x7f78d40b9d40 2020-11-20T12:16:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.700+0000 7f78d37fe700 1 -- 172.21.15.116:0/1918467364 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78cc00bd20 con 0x7f78d40b9d40 2020-11-20T12:16:08.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.700+0000 7f78d37fe700 1 -- 172.21.15.116:0/1918467364 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78cc011610 con 0x7f78d40b9d40 2020-11-20T12:16:08.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.700+0000 7f78d9dc4700 1 -- 172.21.15.116:0/1918467364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78d40b9d40 msgr2=0x7f78d41510b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.700+0000 7f78d9dc4700 1 --2- 172.21.15.116:0/1918467364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78d40b9d40 0x7f78d41510b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f78d41550f0 tx=0x7f78cc009290).stop 2020-11-20T12:16:08.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.700+0000 7f78d9dc4700 1 -- 172.21.15.116:0/1918467364 shutdown_connections 2020-11-20T12:16:08.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.700+0000 7f78d9dc4700 1 --2- 172.21.15.116:0/1918467364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78d40b9d40 0x7f78d41510b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.700+0000 7f78d9dc4700 1 --2- 172.21.15.116:0/1918467364 >> v2:172.21.15.173:3300/0 conn(0x7f78d4061700 0x7f78d40b9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.700+0000 7f78d9dc4700 1 -- 172.21.15.116:0/1918467364 >> 172.21.15.116:0/1918467364 conn(0x7f78d40c1e60 msgr2=0x7f78d4066960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:08.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.700+0000 7f78d9dc4700 1 -- 172.21.15.116:0/1918467364 shutdown_connections 2020-11-20T12:16:08.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.700+0000 7f78d9dc4700 1 -- 172.21.15.116:0/1918467364 wait complete. 2020-11-20T12:16:08.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d9dc4700 1 Processor -- start 2020-11-20T12:16:08.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d9dc4700 1 -- start start 2020-11-20T12:16:08.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d9dc4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78d4061320 0x7f78d4155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d9dc4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78d4061700 0x7f78d414b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d9dc4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78d40b9d40 0x7f78d4160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d9dc4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40a5c30 con 0x7f78d4061320 2020-11-20T12:16:08.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d9dc4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d40a1ca0 con 0x7f78d40b9d40 2020-11-20T12:16:08.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d9dc4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d4161610 con 0x7f78d4061700 2020-11-20T12:16:08.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d8dc2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78d4061320 0x7f78d4155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d3fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78d4061700 0x7f78d414b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d8dc2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78d4061320 0x7f78d4155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43192/0 (socket says 172.21.15.116:43192) 2020-11-20T12:16:08.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d8dc2700 1 -- 172.21.15.116:0/4070401057 learned_addr learned my addr 172.21.15.116:0/4070401057 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:08.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.704+0000 7f78d95c3700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78d40b9d40 0x7f78d4160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d3fff700 1 -- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78d40b9d40 msgr2=0x7f78d4160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d3fff700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78d40b9d40 0x7f78d4160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d3fff700 1 -- 172.21.15.116:0/4070401057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78d4061320 msgr2=0x7f78d4155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d3fff700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78d4061320 0x7f78d4155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d3fff700 1 -- 172.21.15.116:0/4070401057 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d41618b0 con 0x7f78d4061700 2020-11-20T12:16:08.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d95c3700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78d40b9d40 0x7f78d4160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:08.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d3fff700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78d4061700 0x7f78d414b730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f78c000b820 tx=0x7f78c0008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d0ff9700 1 -- 172.21.15.116:0/4070401057 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c000f5e0 con 0x7f78d4061700 2020-11-20T12:16:08.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d0ff9700 1 -- 172.21.15.116:0/4070401057 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c000f740 con 0x7f78d4061700 2020-11-20T12:16:08.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d9dc4700 1 -- 172.21.15.116:0/4070401057 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d4161b50 con 0x7f78d4061700 2020-11-20T12:16:08.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d0ff9700 1 -- 172.21.15.116:0/4070401057 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c000fac0 con 0x7f78d4061700 2020-11-20T12:16:08.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d9dc4700 1 -- 172.21.15.116:0/4070401057 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d414bc20 con 0x7f78d4061700 2020-11-20T12:16:08.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.708+0000 7f78d9dc4700 1 -- 172.21.15.116:0/4070401057 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f78d4044f60 con 0x7f78d4061700 2020-11-20T12:16:08.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.712+0000 7f78d0ff9700 1 -- 172.21.15.116:0/4070401057 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f78c001a030 con 0x7f78d4061700 2020-11-20T12:16:08.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.712+0000 7f78d0ff9700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78ac0557c0 0x7f78ac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.712+0000 7f78d0ff9700 1 -- 172.21.15.116:0/4070401057 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f78c006e810 con 0x7f78d4061700 2020-11-20T12:16:08.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.712+0000 7f78d0ff9700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:6815/14231,v1:172.21.15.173:6817/14231] conn(0x7f78ac05cd10 0x7f78ac05f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.712+0000 7f78d0ff9700 1 -- 172.21.15.116:0/4070401057 --> [v2:172.21.15.173:6815/14231,v1:172.21.15.173:6817/14231] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f78ac05f790 con 0x7f78ac05cd10 2020-11-20T12:16:08.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.712+0000 7f78d0ff9700 1 -- 172.21.15.116:0/4070401057 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f78c006ec10 con 0x7f78d4061700 2020-11-20T12:16:08.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.712+0000 7f78d8dc2700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78ac0557c0 0x7f78ac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.712+0000 7f78d95c3700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:6815/14231,v1:172.21.15.173:6817/14231] conn(0x7f78ac05cd10 0x7f78ac05f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.716+0000 7f78d8dc2700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78ac0557c0 0x7f78ac057c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f78c8003740 tx=0x7f78c8007c50).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.716+0000 7f78d95c3700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:6815/14231,v1:172.21.15.173:6817/14231] conn(0x7f78ac05cd10 0x7f78ac05f130 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.716+0000 7f78d0ff9700 1 -- 172.21.15.116:0/4070401057 <== osd.7 v2:172.21.15.173:6815/14231 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+22725 (crc 0 0 0) 0x7f78ac05f790 con 0x7f78ac05cd10 2020-11-20T12:16:08.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.736+0000 7f78d9dc4700 1 -- 172.21.15.116:0/4070401057 --> [v2:172.21.15.173:6815/14231,v1:172.21.15.173:6817/14231] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f78d40a6050 con 0x7f78ac05cd10 2020-11-20T12:16:08.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d0ff9700 1 -- 172.21.15.116:0/4070401057 <== osd.7 v2:172.21.15.173:6815/14231 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f78cc00be90 con 0x7f78ac05cd10 2020-11-20T12:16:08.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d9dc4700 1 -- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:6815/14231,v1:172.21.15.173:6817/14231] conn(0x7f78ac05cd10 msgr2=0x7f78ac05f130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d9dc4700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:6815/14231,v1:172.21.15.173:6817/14231] conn(0x7f78ac05cd10 0x7f78ac05f130 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d9dc4700 1 -- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78ac0557c0 msgr2=0x7f78ac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d9dc4700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78ac0557c0 0x7f78ac057c80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f78c8003740 tx=0x7f78c8007c50).stop 2020-11-20T12:16:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d9dc4700 1 -- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78d4061700 msgr2=0x7f78d414b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d9dc4700 1 --2- 172.21.15.116:0/4070401057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78d4061700 0x7f78d414b730 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f78c000b820 tx=0x7f78c0008040).stop 2020-11-20T12:16:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d95c3700 1 -- 172.21.15.116:0/4070401057 reap_dead start 2020-11-20T12:16:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d9dc4700 1 -- 172.21.15.116:0/4070401057 shutdown_connections 2020-11-20T12:16:08.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d9dc4700 1 -- 172.21.15.116:0/4070401057 >> 172.21.15.116:0/4070401057 conn(0x7f78d40c1e60 msgr2=0x7f78d40647e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:08.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d9dc4700 1 -- 172.21.15.116:0/4070401057 shutdown_connections 2020-11-20T12:16:08.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.740+0000 7f78d9dc4700 1 -- 172.21.15.116:0/4070401057 wait complete. 2020-11-20T12:16:08.768 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:08.769 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2020-11-20T12:16:08.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.948+0000 7f102d286700 1 Processor -- start 2020-11-20T12:16:08.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.948+0000 7f102d286700 1 -- start start 2020-11-20T12:16:08.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.948+0000 7f102d286700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f102814b9d0 0x7f1028060cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.948+0000 7f102d286700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280c2e80 con 0x7f102814bdb0 2020-11-20T12:16:08.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.948+0000 7f102d286700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280602f0 con 0x7f10280c2900 2020-11-20T12:16:08.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.948+0000 7f102d286700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10280c3120 con 0x7f102814b9d0 2020-11-20T12:16:08.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.948+0000 7f101f7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f102814bdb0 0x7f1028064860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52638/0 (socket says 172.21.15.116:52638) 2020-11-20T12:16:08.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.948+0000 7f101f7fe700 1 -- 172.21.15.116:0/2757289012 learned_addr learned my addr 172.21.15.116:0/2757289012 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:08.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.948+0000 7f1027fff700 1 --2- 172.21.15.116:0/2757289012 >> v2:172.21.15.116:3300/0 conn(0x7f102814b9d0 0x7f1028060cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f10277fe700 1 -- 172.21.15.116:0/2757289012 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097742839 0 0) 0x7f10280c2e80 con 0x7f102814bdb0 2020-11-20T12:16:08.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f10277fe700 1 -- 172.21.15.116:0/2757289012 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1008003240 con 0x7f102814bdb0 2020-11-20T12:16:08.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f1027fff700 1 -- 172.21.15.116:0/2757289012 >> v1:172.21.15.173:6789/0 conn(0x7f10280c2900 legacy=0x7f10281550c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f10277fe700 1 -- 172.21.15.116:0/2757289012 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228517563 0 0) 0x7f10280602f0 con 0x7f10280c2900 2020-11-20T12:16:08.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f1027fff700 1 -- 172.21.15.116:0/2757289012 >> v1:172.21.15.116:6789/0 conn(0x7f102814bdb0 legacy=0x7f1028064860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f1027fff700 1 -- 172.21.15.116:0/2757289012 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1028060590 con 0x7f102814b9d0 2020-11-20T12:16:08.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f1027fff700 1 --2- 172.21.15.116:0/2757289012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f102814b9d0 0x7f1028060cd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f101800b9a0 tx=0x7f1018006df0).ready entity=mon.0 client_cookie=2ef910fb07b00d09 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f10277fe700 1 -- 172.21.15.116:0/2757289012 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10180097b0 con 0x7f102814b9d0 2020-11-20T12:16:08.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f10277fe700 1 -- 172.21.15.116:0/2757289012 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1018009910 con 0x7f102814b9d0 2020-11-20T12:16:08.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f10277fe700 1 -- 172.21.15.116:0/2757289012 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1018009c90 con 0x7f102814b9d0 2020-11-20T12:16:08.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f102d286700 1 -- 172.21.15.116:0/2757289012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f102814b9d0 msgr2=0x7f1028060cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f102d286700 1 --2- 172.21.15.116:0/2757289012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f102814b9d0 0x7f1028060cd0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f101800b9a0 tx=0x7f1018006df0).stop 2020-11-20T12:16:08.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f102d286700 1 -- 172.21.15.116:0/2757289012 shutdown_connections 2020-11-20T12:16:08.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f102d286700 1 --2- 172.21.15.116:0/2757289012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f102814b9d0 0x7f1028060cd0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f102d286700 1 -- 172.21.15.116:0/2757289012 >> 172.21.15.116:0/2757289012 conn(0x7f10280c44e0 msgr2=0x7f10280c48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:08.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f102d286700 1 -- 172.21.15.116:0/2757289012 shutdown_connections 2020-11-20T12:16:08.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.952+0000 7f102d286700 1 -- 172.21.15.116:0/2757289012 wait complete. 2020-11-20T12:16:08.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.956+0000 7f102d286700 1 Processor -- start 2020-11-20T12:16:08.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.956+0000 7f102d286700 1 -- start start 2020-11-20T12:16:08.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f102d286700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10280c2900 0x7f102815e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f102d286700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f102814bdb0 0x7f10280bb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f102d286700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10280bc1d0 0x7f10280bbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f102d286700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102815ebb0 con 0x7f102814bdb0 2020-11-20T12:16:08.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f102d286700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280beac0 con 0x7f10280c2900 2020-11-20T12:16:08.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f102d286700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10281613d0 con 0x7f10280bc1d0 2020-11-20T12:16:08.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f101f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f102814bdb0 0x7f10280bb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f101f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f102814bdb0 0x7f10280bb5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43208/0 (socket says 172.21.15.116:43208) 2020-11-20T12:16:08.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f101f7fe700 1 -- 172.21.15.116:0/2601522224 learned_addr learned my addr 172.21.15.116:0/2601522224 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:08.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f1027fff700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10280c2900 0x7f102815e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f102ca85700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10280bc1d0 0x7f10280bbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f101f7fe700 1 -- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10280bc1d0 msgr2=0x7f10280bbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f101f7fe700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10280bc1d0 0x7f10280bbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f101f7fe700 1 -- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10280c2900 msgr2=0x7f102815e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:08.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f101f7fe700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10280c2900 0x7f102815e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:08.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f101f7fe700 1 -- 172.21.15.116:0/2601522224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1028161670 con 0x7f102814bdb0 2020-11-20T12:16:08.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f1027fff700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10280c2900 0x7f102815e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:08.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f102ca85700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10280bc1d0 0x7f10280bbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:08.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f101f7fe700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f102814bdb0 0x7f10280bb5e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f101000c710 tx=0x7f1010007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f1024ff9700 1 -- 172.21.15.116:0/2601522224 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10100072e0 con 0x7f102814bdb0 2020-11-20T12:16:08.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f1024ff9700 1 -- 172.21.15.116:0/2601522224 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1010007440 con 0x7f102814bdb0 2020-11-20T12:16:08.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f102d286700 1 -- 172.21.15.116:0/2601522224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1028161910 con 0x7f102814bdb0 2020-11-20T12:16:08.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f1024ff9700 1 -- 172.21.15.116:0/2601522224 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1010005800 con 0x7f102814bdb0 2020-11-20T12:16:08.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.960+0000 7f102d286700 1 -- 172.21.15.116:0/2601522224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1028161e50 con 0x7f102814bdb0 2020-11-20T12:16:08.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.964+0000 7f102d286700 1 -- 172.21.15.116:0/2601522224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1028044f60 con 0x7f102814bdb0 2020-11-20T12:16:08.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.964+0000 7f1024ff9700 1 -- 172.21.15.116:0/2601522224 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1010010070 con 0x7f102814bdb0 2020-11-20T12:16:08.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.964+0000 7f1024ff9700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10040557c0 0x7f1004057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:08.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.964+0000 7f1024ff9700 1 -- 172.21.15.116:0/2601522224 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f101006dee0 con 0x7f102814bdb0 2020-11-20T12:16:08.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.964+0000 7f1027fff700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10040557c0 0x7f1004057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:08.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.968+0000 7f1027fff700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10040557c0 0x7f1004057c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f102805d6a0 tx=0x7f1018015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:08.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:08.972+0000 7f1024ff9700 1 -- 172.21.15.116:0/2601522224 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1010043e00 con 0x7f102814bdb0 2020-11-20T12:16:09.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.128+0000 7f102d286700 1 -- 172.21.15.116:0/2601522224 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1028155fb0 con 0x7f102814bdb0 2020-11-20T12:16:09.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.128+0000 7f1024ff9700 1 -- 172.21.15.116:0/2601522224 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1010047420 con 0x7f102814bdb0 2020-11-20T12:16:09.131 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:09.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 -- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10040557c0 msgr2=0x7f1004057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10040557c0 0x7f1004057c80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f102805d6a0 tx=0x7f1018015000).stop 2020-11-20T12:16:09.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 -- 172.21.15.116:0/2601522224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f102814bdb0 msgr2=0x7f10280bb5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f102814bdb0 0x7f10280bb5e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f101000c710 tx=0x7f1010007aa0).stop 2020-11-20T12:16:09.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 -- 172.21.15.116:0/2601522224 shutdown_connections 2020-11-20T12:16:09.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10040557c0 0x7f1004057c80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10280bc1d0 0x7f10280bbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10280c2900 0x7f102815e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 --2- 172.21.15.116:0/2601522224 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f102814bdb0 0x7f10280bb5e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 -- 172.21.15.116:0/2601522224 >> 172.21.15.116:0/2601522224 conn(0x7f10280c44e0 msgr2=0x7f102814b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:09.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 -- 172.21.15.116:0/2601522224 shutdown_connections 2020-11-20T12:16:09.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.132+0000 7f102d286700 1 -- 172.21.15.116:0/2601522224 wait complete. 2020-11-20T12:16:09.159 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.0 2020-11-20T12:16:09.159 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2020-11-20T12:16:09.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f7541e31700 1 Processor -- start 2020-11-20T12:16:09.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f7541e31700 1 -- start start 2020-11-20T12:16:09.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f7541e31700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f753c13aee0 0x7f753c152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:09.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f7541e31700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f753c0aa790 0x7f753c0ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:09.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f7541e31700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753c152a30 con 0x7f753c0612f0 2020-11-20T12:16:09.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f7541e31700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753c0aac10 con 0x7f753c13aee0 2020-11-20T12:16:09.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f7541e31700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753c0aaeb0 con 0x7f753c0aa790 2020-11-20T12:16:09.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f7540e2f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f753c13aee0 0x7f753c152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:09.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f7541630700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f753c0612f0 0x7f753c0a7ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52652/0 (socket says 172.21.15.116:52652) 2020-11-20T12:16:09.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f7541630700 1 -- 172.21.15.116:0/242307161 learned_addr learned my addr 172.21.15.116:0/242307161 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:09.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f753bfff700 1 --2- 172.21.15.116:0/242307161 >> v2:172.21.15.173:3300/0 conn(0x7f753c0aa790 0x7f753c0ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:09.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f753b7fe700 1 -- 172.21.15.116:0/242307161 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2364859227 0 0) 0x7f753c152a30 con 0x7f753c0612f0 2020-11-20T12:16:09.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.324+0000 7f753b7fe700 1 -- 172.21.15.116:0/242307161 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7520003240 con 0x7f753c0612f0 2020-11-20T12:16:09.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7540e2f700 1 -- 172.21.15.116:0/242307161 >> v2:172.21.15.173:3300/0 conn(0x7f753c0aa790 msgr2=0x7f753c0ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:09.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7540e2f700 1 --2- 172.21.15.116:0/242307161 >> v2:172.21.15.173:3300/0 conn(0x7f753c0aa790 0x7f753c0ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7540e2f700 1 -- 172.21.15.116:0/242307161 >> v1:172.21.15.116:6789/0 conn(0x7f753c0612f0 legacy=0x7f753c0a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7540e2f700 1 -- 172.21.15.116:0/242307161 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c0acbf0 con 0x7f753c13aee0 2020-11-20T12:16:09.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7540e2f700 1 --2- 172.21.15.116:0/242307161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f753c13aee0 0x7f753c152540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f753c0ae630 tx=0x7f7530006df0).ready entity=mon.0 client_cookie=78a6501cfb13c362 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:09.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f753b7fe700 1 -- 172.21.15.116:0/242307161 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530009640 con 0x7f753c13aee0 2020-11-20T12:16:09.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f753b7fe700 1 -- 172.21.15.116:0/242307161 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75300097a0 con 0x7f753c13aee0 2020-11-20T12:16:09.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f753b7fe700 1 -- 172.21.15.116:0/242307161 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7530009b20 con 0x7f753c13aee0 2020-11-20T12:16:09.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7541e31700 1 -- 172.21.15.116:0/242307161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f753c13aee0 msgr2=0x7f753c152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7541e31700 1 --2- 172.21.15.116:0/242307161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f753c13aee0 0x7f753c152540 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f753c0ae630 tx=0x7f7530006df0).stop 2020-11-20T12:16:09.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7541e31700 1 -- 172.21.15.116:0/242307161 shutdown_connections 2020-11-20T12:16:09.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7541e31700 1 --2- 172.21.15.116:0/242307161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f753c13aee0 0x7f753c152540 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7541e31700 1 --2- 172.21.15.116:0/242307161 >> v2:172.21.15.173:3300/0 conn(0x7f753c0aa790 0x7f753c0ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7541e31700 1 -- 172.21.15.116:0/242307161 >> 172.21.15.116:0/242307161 conn(0x7f753c0678a0 msgr2=0x7f753c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7541e31700 1 -- 172.21.15.116:0/242307161 shutdown_connections 2020-11-20T12:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.328+0000 7f7541e31700 1 -- 172.21.15.116:0/242307161 wait complete. 2020-11-20T12:16:09.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7541e31700 1 Processor -- start 2020-11-20T12:16:09.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7541e31700 1 -- start start 2020-11-20T12:16:09.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7541e31700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f753c0612f0 0x7f753c0a5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:09.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7541e31700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f753c0aa790 0x7f753c0a92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:09.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7541e31700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f753c0a97c0 0x7f753c0a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:09.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7541e31700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0a5940 con 0x7f753c0aa790 2020-11-20T12:16:09.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7541e31700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0a5b90 con 0x7f753c0612f0 2020-11-20T12:16:09.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7541e31700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0a5e30 con 0x7f753c0a97c0 2020-11-20T12:16:09.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f753bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f753c0aa790 0x7f753c0a92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:09.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f753bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f753c0aa790 0x7f753c0a92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43222/0 (socket says 172.21.15.116:43222) 2020-11-20T12:16:09.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f753bfff700 1 -- 172.21.15.116:0/1507455073 learned_addr learned my addr 172.21.15.116:0/1507455073 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:09.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7540e2f700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f753c0612f0 0x7f753c0a5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:09.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7541630700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f753c0a97c0 0x7f753c0a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:09.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f753bfff700 1 -- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f753c0a97c0 msgr2=0x7f753c0a6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f753bfff700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f753c0a97c0 0x7f753c0a6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f753bfff700 1 -- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f753c0612f0 msgr2=0x7f753c0a5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f753bfff700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f753c0612f0 0x7f753c0a5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f753bfff700 1 -- 172.21.15.116:0/1507455073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c0a60d0 con 0x7f753c0aa790 2020-11-20T12:16:09.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7541630700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f753c0a97c0 0x7f753c0a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:09.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f753bfff700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f753c0aa790 0x7f753c0a92d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f752800ebd0 tx=0x7f7528006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:09.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7538ff9700 1 -- 172.21.15.116:0/1507455073 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752800b9a0 con 0x7f753c0aa790 2020-11-20T12:16:09.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7538ff9700 1 -- 172.21.15.116:0/1507455073 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752800bb00 con 0x7f753c0aa790 2020-11-20T12:16:09.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7538ff9700 1 -- 172.21.15.116:0/1507455073 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752800be80 con 0x7f753c0aa790 2020-11-20T12:16:09.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.332+0000 7f7541e31700 1 -- 172.21.15.116:0/1507455073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753c0a6370 con 0x7f753c0aa790 2020-11-20T12:16:09.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.336+0000 7f7541e31700 1 -- 172.21.15.116:0/1507455073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c0a29a0 con 0x7f753c0aa790 2020-11-20T12:16:09.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.336+0000 7f7541e31700 1 -- 172.21.15.116:0/1507455073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753c044f60 con 0x7f753c0aa790 2020-11-20T12:16:09.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.336+0000 7f7538ff9700 1 -- 172.21.15.116:0/1507455073 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7528012600 con 0x7f753c0aa790 2020-11-20T12:16:09.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.336+0000 7f7538ff9700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75140557c0 0x7f7514057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:09.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.336+0000 7f7538ff9700 1 -- 172.21.15.116:0/1507455073 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f752806ea60 con 0x7f753c0aa790 2020-11-20T12:16:09.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.340+0000 7f7540e2f700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75140557c0 0x7f7514057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:09.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.340+0000 7f7540e2f700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75140557c0 0x7f7514057c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f753c0c5f90 tx=0x7f7530015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:09.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.344+0000 7f7538ff9700 1 -- 172.21.15.116:0/1507455073 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f75280448a0 con 0x7f753c0aa790 2020-11-20T12:16:09.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.504+0000 7f7541e31700 1 -- 172.21.15.116:0/1507455073 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f753c060500 con 0x7f753c0aa790 2020-11-20T12:16:09.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.504+0000 7f7538ff9700 1 -- 172.21.15.116:0/1507455073 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7528047ec0 con 0x7f753c0aa790 2020-11-20T12:16:09.506 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:09.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 -- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75140557c0 msgr2=0x7f7514057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75140557c0 0x7f7514057c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f753c0c5f90 tx=0x7f7530015000).stop 2020-11-20T12:16:09.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 -- 172.21.15.116:0/1507455073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f753c0aa790 msgr2=0x7f753c0a92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f753c0aa790 0x7f753c0a92d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f752800ebd0 tx=0x7f7528006df0).stop 2020-11-20T12:16:09.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 -- 172.21.15.116:0/1507455073 shutdown_connections 2020-11-20T12:16:09.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75140557c0 0x7f7514057c80 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f753c0a97c0 0x7f753c0a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f753c0612f0 0x7f753c0a5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 --2- 172.21.15.116:0/1507455073 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f753c0aa790 0x7f753c0a92d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 -- 172.21.15.116:0/1507455073 >> 172.21.15.116:0/1507455073 conn(0x7f753c0678a0 msgr2=0x7f753c13a020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:09.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 -- 172.21.15.116:0/1507455073 shutdown_connections 2020-11-20T12:16:09.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.508+0000 7f7541e31700 1 -- 172.21.15.116:0/1507455073 wait complete. 2020-11-20T12:16:09.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.1 2020-11-20T12:16:09.535 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2020-11-20T12:16:09.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.708+0000 7fd60f054700 1 Processor -- start 2020-11-20T12:16:09.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.708+0000 7fd60f054700 1 -- start start 2020-11-20T12:16:09.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.712+0000 7fd60f054700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd608138d80 0x7fd60815a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:09.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.712+0000 7fd60f054700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6080a8c80 con 0x7fd6080c41c0 2020-11-20T12:16:09.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.712+0000 7fd60f054700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6080a9180 con 0x7fd6080c45a0 2020-11-20T12:16:09.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.712+0000 7fd60f054700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6080a9420 con 0x7fd608138d80 2020-11-20T12:16:09.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.712+0000 7fd60e052700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd6080c41c0 0x7fd6080a5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52662/0 (socket says 172.21.15.116:52662) 2020-11-20T12:16:09.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.712+0000 7fd60e052700 1 -- 172.21.15.116:0/906059481 learned_addr learned my addr 172.21.15.116:0/906059481 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:09.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.712+0000 7fd60e853700 1 --2- 172.21.15.116:0/906059481 >> v2:172.21.15.116:3300/0 conn(0x7fd608138d80 0x7fd60815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:09.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.712+0000 7fd60e853700 1 -- 172.21.15.116:0/906059481 >> v1:172.21.15.173:6789/0 conn(0x7fd6080c45a0 legacy=0x7fd608138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:09.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.712+0000 7fd60e853700 1 -- 172.21.15.116:0/906059481 >> v1:172.21.15.116:6789/0 conn(0x7fd6080c41c0 legacy=0x7fd6080a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.712+0000 7fd60e853700 1 -- 172.21.15.116:0/906059481 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080a96c0 con 0x7fd608138d80 2020-11-20T12:16:09.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.712+0000 7fd60e853700 1 --2- 172.21.15.116:0/906059481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd608138d80 0x7fd60815a030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fd600009c90 tx=0x7fd6000091d0).ready entity=mon.0 client_cookie=10701eb1a01e0652 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:09.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.716+0000 7fd60d050700 1 -- 172.21.15.116:0/906059481 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60000bbc0 con 0x7fd608138d80 2020-11-20T12:16:09.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.716+0000 7fd60d050700 1 -- 172.21.15.116:0/906059481 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60000bd20 con 0x7fd608138d80 2020-11-20T12:16:09.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.716+0000 7fd60d050700 1 -- 172.21.15.116:0/906059481 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd600011610 con 0x7fd608138d80 2020-11-20T12:16:09.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.716+0000 7fd60f054700 1 -- 172.21.15.116:0/906059481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd608138d80 msgr2=0x7fd60815a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.716+0000 7fd60f054700 1 --2- 172.21.15.116:0/906059481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd608138d80 0x7fd60815a030 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fd600009c90 tx=0x7fd6000091d0).stop 2020-11-20T12:16:09.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.716+0000 7fd60f054700 1 -- 172.21.15.116:0/906059481 shutdown_connections 2020-11-20T12:16:09.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.716+0000 7fd60f054700 1 --2- 172.21.15.116:0/906059481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd608138d80 0x7fd60815a030 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.716+0000 7fd60f054700 1 -- 172.21.15.116:0/906059481 >> 172.21.15.116:0/906059481 conn(0x7fd608062520 msgr2=0x7fd608062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:09.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.716+0000 7fd60f054700 1 -- 172.21.15.116:0/906059481 shutdown_connections 2020-11-20T12:16:09.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.716+0000 7fd60f054700 1 -- 172.21.15.116:0/906059481 wait complete. 2020-11-20T12:16:09.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.716+0000 7fd60f054700 1 Processor -- start 2020-11-20T12:16:09.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60f054700 1 -- start start 2020-11-20T12:16:09.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60f054700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6080c41c0 0x7fd608136d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:09.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60f054700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6080c45a0 0x7fd608062900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:09.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60f054700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd608138d80 0x7fd608062df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:09.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60f054700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080b9fb0 con 0x7fd6080c45a0 2020-11-20T12:16:09.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60f054700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080b9090 con 0x7fd608138d80 2020-11-20T12:16:09.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60f054700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6080b9330 con 0x7fd6080c41c0 2020-11-20T12:16:09.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60d851700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6080c45a0 0x7fd608062900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:09.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60d851700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6080c45a0 0x7fd608062900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43236/0 (socket says 172.21.15.116:43236) 2020-11-20T12:16:09.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60d851700 1 -- 172.21.15.116:0/1198165638 learned_addr learned my addr 172.21.15.116:0/1198165638 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:09.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60e853700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd608138d80 0x7fd608062df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:09.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60e052700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6080c41c0 0x7fd608136d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:09.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60d851700 1 -- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6080c41c0 msgr2=0x7fd608136d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60d851700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6080c41c0 0x7fd608136d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60d851700 1 -- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd608138d80 msgr2=0x7fd608062df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60d851700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd608138d80 0x7fd608062df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.720+0000 7fd60d851700 1 -- 172.21.15.116:0/1198165638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6080b95d0 con 0x7fd6080c45a0 2020-11-20T12:16:09.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.724+0000 7fd60d851700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6080c45a0 0x7fd608062900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fd5f8001e70 tx=0x7fd5f8007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:09.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.724+0000 7fd5f67fc700 1 -- 172.21.15.116:0/1198165638 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f8007020 con 0x7fd6080c45a0 2020-11-20T12:16:09.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.724+0000 7fd5f67fc700 1 -- 172.21.15.116:0/1198165638 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5f8007180 con 0x7fd6080c45a0 2020-11-20T12:16:09.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.724+0000 7fd60f054700 1 -- 172.21.15.116:0/1198165638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6080b9870 con 0x7fd6080c45a0 2020-11-20T12:16:09.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.724+0000 7fd5f67fc700 1 -- 172.21.15.116:0/1198165638 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5f800e560 con 0x7fd6080c45a0 2020-11-20T12:16:09.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.724+0000 7fd60f054700 1 -- 172.21.15.116:0/1198165638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6081371f0 con 0x7fd6080c45a0 2020-11-20T12:16:09.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.724+0000 7fd60f054700 1 -- 172.21.15.116:0/1198165638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd608044f60 con 0x7fd6080c45a0 2020-11-20T12:16:09.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.724+0000 7fd5f67fc700 1 -- 172.21.15.116:0/1198165638 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd5f8010070 con 0x7fd6080c45a0 2020-11-20T12:16:09.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.724+0000 7fd5f67fc700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5e00557c0 0x7fd5e0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:09.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.728+0000 7fd5f67fc700 1 -- 172.21.15.116:0/1198165638 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd5f806db30 con 0x7fd6080c45a0 2020-11-20T12:16:09.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.728+0000 7fd60e052700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5e00557c0 0x7fd5e0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:09.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.728+0000 7fd60e052700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5e00557c0 0x7fd5e0057c80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd6040045d0 tx=0x7fd604007b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:09.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.732+0000 7fd5f67fc700 1 -- 172.21.15.116:0/1198165638 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd5f8042fd0 con 0x7fd6080c45a0 2020-11-20T12:16:09.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.888+0000 7fd60f054700 1 -- 172.21.15.116:0/1198165638 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd6080c3a10 con 0x7fd6080c45a0 2020-11-20T12:16:09.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.888+0000 7fd5f67fc700 1 -- 172.21.15.116:0/1198165638 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd5f80438c0 con 0x7fd6080c45a0 2020-11-20T12:16:09.891 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:09.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 -- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5e00557c0 msgr2=0x7fd5e0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5e00557c0 0x7fd5e0057c80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fd6040045d0 tx=0x7fd604007b80).stop 2020-11-20T12:16:09.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 -- 172.21.15.116:0/1198165638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6080c45a0 msgr2=0x7fd608062900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:09.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6080c45a0 0x7fd608062900 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fd5f8001e70 tx=0x7fd5f8007c50).stop 2020-11-20T12:16:09.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 -- 172.21.15.116:0/1198165638 shutdown_connections 2020-11-20T12:16:09.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5e00557c0 0x7fd5e0057c80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd608138d80 0x7fd608062df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6080c41c0 0x7fd608136d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 --2- 172.21.15.116:0/1198165638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6080c45a0 0x7fd608062900 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:09.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 -- 172.21.15.116:0/1198165638 >> 172.21.15.116:0/1198165638 conn(0x7fd608062520 msgr2=0x7fd6080a4800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:09.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 -- 172.21.15.116:0/1198165638 shutdown_connections 2020-11-20T12:16:09.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:09.892+0000 7fd60f054700 1 -- 172.21.15.116:0/1198165638 wait complete. 2020-11-20T12:16:09.918 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.2 2020-11-20T12:16:09.918 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2020-11-20T12:16:10.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.084+0000 7f406b59e700 1 Processor -- start 2020-11-20T12:16:10.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.084+0000 7f406b59e700 1 -- start start 2020-11-20T12:16:10.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.084+0000 7f406b59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f406c138d80 0x7f406c15a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.084+0000 7f406b59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c0a8c80 con 0x7f406c0c45a0 2020-11-20T12:16:10.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.084+0000 7f406b59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f406c0a9180 con 0x7f406c0c41c0 2020-11-20T12:16:10.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.084+0000 7f406b59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c0a9420 con 0x7f406c138d80 2020-11-20T12:16:10.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4061d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f406c0c45a0 0x7f406c138670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52676/0 (socket says 172.21.15.116:52676) 2020-11-20T12:16:10.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4061d9b700 1 -- 172.21.15.116:0/2582715955 learned_addr learned my addr 172.21.15.116:0/2582715955 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:10.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f406ad9d700 1 --2- 172.21.15.116:0/2582715955 >> v2:172.21.15.116:3300/0 conn(0x7f406c138d80 0x7f406c15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1068348547 0 0) 0x7f406c0a8c80 con 0x7f406c0c45a0 2020-11-20T12:16:10.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f404c003240 con 0x7f406c0c45a0 2020-11-20T12:16:10.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4155567724 0 0) 0x7f406c0a9180 con 0x7f406c0c41c0 2020-11-20T12:16:10.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f406c0a8c80 con 0x7f406c0c41c0 2020-11-20T12:16:10.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3617515186 0 0) 0x7f404c003240 con 0x7f406c0c45a0 2020-11-20T12:16:10.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 >> v2:172.21.15.116:3300/0 conn(0x7f406c138d80 msgr2=0x7f406c15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:10.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 --2- 172.21.15.116:0/2582715955 >> v2:172.21.15.116:3300/0 conn(0x7f406c138d80 0x7f406c15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 >> v1:172.21.15.173:6789/0 conn(0x7f406c0c41c0 legacy=0x7f406c0a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c0a96c0 con 0x7f406c0c45a0 2020-11-20T12:16:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f40540014e0 con 0x7f406c0c45a0 2020-11-20T12:16:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 >> v1:172.21.15.116:6789/0 conn(0x7f406c0c45a0 legacy=0x7f406c138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f404c003910 0x7f404c003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 0x7f404c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f404c004e50 0x7f404c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c0a9420 con 0x7f404c0044e0 2020-11-20T12:16:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f404c00bc20 con 0x7f404c003910 2020-11-20T12:16:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f404c00bd60 con 0x7f404c004e50 2020-11-20T12:16:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f406ad9d700 1 --2- 172.21.15.116:0/2582715955 >> v2:172.21.15.116:3300/0 conn(0x7f406c138d80 0x7f406c15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:10.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4061d9b700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 0x7f404c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f406a59c700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f404c003910 0x7f404c003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f406ad9d700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f404c004e50 0x7f404c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4061d9b700 1 -- 172.21.15.116:0/2582715955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f404c004e50 msgr2=0x7f404c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4061d9b700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f404c004e50 0x7f404c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4061d9b700 1 -- 172.21.15.116:0/2582715955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f404c003910 msgr2=0x7f404c003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4061d9b700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f404c003910 0x7f404c003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f4061d9b700 1 -- 172.21.15.116:0/2582715955 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c003240 con 0x7f404c0044e0 2020-11-20T12:16:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f406ad9d700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f404c004e50 0x7f404c00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.088+0000 7f406ad9d700 1 -- 172.21.15.116:0/2582715955 reap_dead start 2020-11-20T12:16:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.092+0000 7f4061d9b700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 0x7f404c004960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f405400c160 tx=0x7f406c0a9310).ready entity=mon.0 client_cookie=475dd748264b629d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.092+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054007750 con 0x7f404c0044e0 2020-11-20T12:16:10.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.092+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40540070d0 con 0x7f404c0044e0 2020-11-20T12:16:10.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.092+0000 7f4069d9b700 1 -- 172.21.15.116:0/2582715955 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4054007490 con 0x7f404c0044e0 2020-11-20T12:16:10.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.092+0000 7f406b59e700 1 -- 172.21.15.116:0/2582715955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 msgr2=0x7f404c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.092+0000 7f406b59e700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 0x7f404c004960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f405400c160 tx=0x7f406c0a9310).stop 2020-11-20T12:16:10.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.092+0000 7f406b59e700 1 -- 172.21.15.116:0/2582715955 shutdown_connections 2020-11-20T12:16:10.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.092+0000 7f406b59e700 1 --2- 172.21.15.116:0/2582715955 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 0x7f404c004960 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.092+0000 7f406b59e700 1 -- 172.21.15.116:0/2582715955 >> 172.21.15.116:0/2582715955 conn(0x7f406c062520 msgr2=0x7f406c062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:10.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.092+0000 7f406b59e700 1 -- 172.21.15.116:0/2582715955 shutdown_connections 2020-11-20T12:16:10.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.092+0000 7f406b59e700 1 -- 172.21.15.116:0/2582715955 wait complete. 2020-11-20T12:16:10.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406b59e700 1 Processor -- start 2020-11-20T12:16:10.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406b59e700 1 -- start start 2020-11-20T12:16:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 0x7f406c136ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f406c0b9060 0x7f406c0a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f406c0a27d0 0x7f406c0a2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c15fcf0 con 0x7f404c0044e0 2020-11-20T12:16:10.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c15f070 con 0x7f406c0a27d0 2020-11-20T12:16:10.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c15f310 con 0x7f406c0b9060 2020-11-20T12:16:10.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 0x7f406c136ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 0x7f406c136ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43254/0 (socket says 172.21.15.116:43254) 2020-11-20T12:16:10.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406a59c700 1 -- 172.21.15.116:0/909585730 learned_addr learned my addr 172.21.15.116:0/909585730 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:10.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f4061d9b700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f406c0b9060 0x7f406c0a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406a59c700 1 -- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f406c0b9060 msgr2=0x7f406c0a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406a59c700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f406c0b9060 0x7f406c0a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406a59c700 1 -- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f406c0a27d0 msgr2=0x7f406c0a2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406a59c700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f406c0a27d0 0x7f406c0a2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406a59c700 1 -- 172.21.15.116:0/909585730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c15f5b0 con 0x7f404c0044e0 2020-11-20T12:16:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f406a59c700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 0x7f406c136ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f405c00e860 tx=0x7f405c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.096+0000 7f4062ffd700 1 -- 172.21.15.116:0/909585730 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c0105e0 con 0x7f404c0044e0 2020-11-20T12:16:10.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.100+0000 7f4062ffd700 1 -- 172.21.15.116:0/909585730 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c010740 con 0x7f404c0044e0 2020-11-20T12:16:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.100+0000 7f406b59e700 1 -- 172.21.15.116:0/909585730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406c15f850 con 0x7f404c0044e0 2020-11-20T12:16:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.100+0000 7f4062ffd700 1 -- 172.21.15.116:0/909585730 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c010ac0 con 0x7f404c0044e0 2020-11-20T12:16:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.100+0000 7f406b59e700 1 -- 172.21.15.116:0/909585730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c15ed30 con 0x7f404c0044e0 2020-11-20T12:16:10.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.100+0000 7f406b59e700 1 -- 172.21.15.116:0/909585730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406c0a9b40 con 0x7f404c0044e0 2020-11-20T12:16:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.100+0000 7f4062ffd700 1 -- 172.21.15.116:0/909585730 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f405c01b030 con 0x7f404c0044e0 2020-11-20T12:16:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.100+0000 7f4062ffd700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f40480557c0 0x7f4048057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.100+0000 7f4062ffd700 1 -- 172.21.15.116:0/909585730 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f405c06e3d0 con 0x7f404c0044e0 2020-11-20T12:16:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.100+0000 7f4061d9b700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f40480557c0 0x7f4048057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.104+0000 7f4061d9b700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f40480557c0 0x7f4048057c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f406c1539a0 tx=0x7f4054015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.108+0000 7f4062ffd700 1 -- 172.21.15.116:0/909585730 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f406c0a9b40 con 0x7f404c0044e0 2020-11-20T12:16:10.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.268+0000 7f406b59e700 1 -- 172.21.15.116:0/909585730 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f406c0a9b40 con 0x7f404c0044e0 2020-11-20T12:16:10.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.268+0000 7f4062ffd700 1 -- 172.21.15.116:0/909585730 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f406c0a9b40 con 0x7f404c0044e0 2020-11-20T12:16:10.272 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 -- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f40480557c0 msgr2=0x7f4048057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f40480557c0 0x7f4048057c80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f406c1539a0 tx=0x7f4054015000).stop 2020-11-20T12:16:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 -- 172.21.15.116:0/909585730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 msgr2=0x7f406c136ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 0x7f406c136ab0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f405c00e860 tx=0x7f405c009040).stop 2020-11-20T12:16:10.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 -- 172.21.15.116:0/909585730 shutdown_connections 2020-11-20T12:16:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f40480557c0 0x7f4048057c80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f406c0a27d0 0x7f406c0a2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f404c0044e0 0x7f406c136ab0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 --2- 172.21.15.116:0/909585730 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f406c0b9060 0x7f406c0a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 -- 172.21.15.116:0/909585730 >> 172.21.15.116:0/909585730 conn(0x7f406c062520 msgr2=0x7f406c1378e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 -- 172.21.15.116:0/909585730 shutdown_connections 2020-11-20T12:16:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.272+0000 7f406b59e700 1 -- 172.21.15.116:0/909585730 wait complete. 2020-11-20T12:16:10.299 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2020-11-20T12:16:10.300 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2020-11-20T12:16:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feaedd52700 1 Processor -- start 2020-11-20T12:16:10.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feaedd52700 1 -- start start 2020-11-20T12:16:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feaedd52700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7feae813aee0 0x7feae8152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feaedd52700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feae80aa790 0x7feae80ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feaedd52700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feae8152a30 con 0x7feae80612f0 2020-11-20T12:16:10.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feaedd52700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae80aac10 con 0x7feae80aa790 2020-11-20T12:16:10.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feaedd52700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feae80aaeb0 con 0x7feae813aee0 2020-11-20T12:16:10.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feae7fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feae80aa790 0x7feae80ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feaed551700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7feae80612f0 0x7feae80a7ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52696/0 (socket says 172.21.15.116:52696) 2020-11-20T12:16:10.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feaed551700 1 -- 172.21.15.116:0/2066309232 learned_addr learned my addr 172.21.15.116:0/2066309232 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:10.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feae7fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feae80aa790 0x7feae80ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43262/0 (socket says 172.21.15.116:43262) 2020-11-20T12:16:10.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feaecd50700 1 --2- 172.21.15.116:0/2066309232 >> v2:172.21.15.173:3300/0 conn(0x7feae813aee0 0x7feae8152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feae7fff700 1 -- 172.21.15.116:0/2066309232 >> v2:172.21.15.173:3300/0 conn(0x7feae813aee0 msgr2=0x7feae8152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:10.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feae7fff700 1 --2- 172.21.15.116:0/2066309232 >> v2:172.21.15.173:3300/0 conn(0x7feae813aee0 0x7feae8152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feae7fff700 1 -- 172.21.15.116:0/2066309232 >> v1:172.21.15.116:6789/0 conn(0x7feae80612f0 legacy=0x7feae80a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.476+0000 7feae7fff700 1 -- 172.21.15.116:0/2066309232 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae80acbf0 con 0x7feae80aa790 2020-11-20T12:16:10.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feae7fff700 1 --2- 172.21.15.116:0/2066309232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feae80aa790 0x7feae80ad5d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7feae8081470 tx=0x7fead8006df0).ready entity=mon.0 client_cookie=1bd0d09de51d85db server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:10.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feae77fe700 1 -- 172.21.15.116:0/2066309232 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead80097b0 con 0x7feae80aa790 2020-11-20T12:16:10.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feae77fe700 1 -- 172.21.15.116:0/2066309232 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead8009910 con 0x7feae80aa790 2020-11-20T12:16:10.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feae77fe700 1 -- 172.21.15.116:0/2066309232 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead8009c90 con 0x7feae80aa790 2020-11-20T12:16:10.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feaedd52700 1 -- 172.21.15.116:0/2066309232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feae80aa790 msgr2=0x7feae80ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feaedd52700 1 --2- 172.21.15.116:0/2066309232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feae80aa790 0x7feae80ad5d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7feae8081470 tx=0x7fead8006df0).stop 2020-11-20T12:16:10.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feaedd52700 1 -- 172.21.15.116:0/2066309232 shutdown_connections 2020-11-20T12:16:10.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feaedd52700 1 --2- 172.21.15.116:0/2066309232 >> v2:172.21.15.173:3300/0 conn(0x7feae813aee0 0x7feae8152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feaedd52700 1 --2- 172.21.15.116:0/2066309232 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feae80aa790 0x7feae80ad5d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feaedd52700 1 -- 172.21.15.116:0/2066309232 >> 172.21.15.116:0/2066309232 conn(0x7feae80678a0 msgr2=0x7feae8067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:10.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feaedd52700 1 -- 172.21.15.116:0/2066309232 shutdown_connections 2020-11-20T12:16:10.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.480+0000 7feaedd52700 1 -- 172.21.15.116:0/2066309232 wait complete. 2020-11-20T12:16:10.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaedd52700 1 Processor -- start 2020-11-20T12:16:10.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaedd52700 1 -- start start 2020-11-20T12:16:10.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaedd52700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feae80612f0 0x7feae8153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaedd52700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feae80aa790 0x7feae8160c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaedd52700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae813aee0 0x7feae8161120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaedd52700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae80a4e80 con 0x7feae80612f0 2020-11-20T12:16:10.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaedd52700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae81616c0 con 0x7feae80aa790 2020-11-20T12:16:10.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaedd52700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae8161960 con 0x7feae813aee0 2020-11-20T12:16:10.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaecd50700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feae80612f0 0x7feae8153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaecd50700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feae80612f0 0x7feae8153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43270/0 (socket says 172.21.15.116:43270) 2020-11-20T12:16:10.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaecd50700 1 -- 172.21.15.116:0/1526204618 learned_addr learned my addr 172.21.15.116:0/1526204618 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:10.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaed551700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae813aee0 0x7feae8161120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feae7fff700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feae80aa790 0x7feae8160c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaecd50700 1 -- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae813aee0 msgr2=0x7feae8161120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaecd50700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae813aee0 0x7feae8161120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaecd50700 1 -- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feae80aa790 msgr2=0x7feae8160c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaecd50700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feae80aa790 0x7feae8160c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.484+0000 7feaecd50700 1 -- 172.21.15.116:0/1526204618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae80a50e0 con 0x7feae80612f0 2020-11-20T12:16:10.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.488+0000 7feaecd50700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feae80612f0 0x7feae8153a50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fead000d8e0 tx=0x7fead0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:10.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.488+0000 7feae4ff9700 1 -- 172.21.15.116:0/1526204618 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead00125f0 con 0x7feae80612f0 2020-11-20T12:16:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.488+0000 7feae4ff9700 1 -- 172.21.15.116:0/1526204618 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead0012750 con 0x7feae80612f0 2020-11-20T12:16:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.488+0000 7feaedd52700 1 -- 172.21.15.116:0/1526204618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae80a5380 con 0x7feae80612f0 2020-11-20T12:16:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.488+0000 7feae4ff9700 1 -- 172.21.15.116:0/1526204618 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead0012af0 con 0x7feae80612f0 2020-11-20T12:16:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.488+0000 7feaedd52700 1 -- 172.21.15.116:0/1526204618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae8153f40 con 0x7feae80612f0 2020-11-20T12:16:10.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.488+0000 7feaedd52700 1 -- 172.21.15.116:0/1526204618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae8044f60 con 0x7feae80612f0 2020-11-20T12:16:10.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.488+0000 7feae4ff9700 1 -- 172.21.15.116:0/1526204618 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fead0013480 con 0x7feae80612f0 2020-11-20T12:16:10.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.492+0000 7feae4ff9700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feac40557c0 0x7feac4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.492+0000 7feae4ff9700 1 -- 172.21.15.116:0/1526204618 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fead006eaf0 con 0x7feae80612f0 2020-11-20T12:16:10.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.492+0000 7feae7fff700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feac40557c0 0x7feac4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.492+0000 7feae7fff700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feac40557c0 0x7feac4057c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fead8006e30 tx=0x7fead8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:10.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.496+0000 7feae4ff9700 1 -- 172.21.15.116:0/1526204618 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fead0044930 con 0x7feae80612f0 2020-11-20T12:16:10.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feaedd52700 1 -- 172.21.15.116:0/1526204618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7feae8060500 con 0x7feae80612f0 2020-11-20T12:16:10.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feae4ff9700 1 -- 172.21.15.116:0/1526204618 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fead0012e60 con 0x7feae80612f0 2020-11-20T12:16:10.657 INFO:teuthology.orchestra.run.smithi116.stdout:60129542150 2020-11-20T12:16:10.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feaedd52700 1 -- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feac40557c0 msgr2=0x7feac4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feaedd52700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feac40557c0 0x7feac4057c80 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fead8006e30 tx=0x7fead8015000).stop 2020-11-20T12:16:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feaedd52700 1 -- 172.21.15.116:0/1526204618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feae80612f0 msgr2=0x7feae8153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feaedd52700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feae80612f0 0x7feae8153a50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fead000d8e0 tx=0x7fead0008040).stop 2020-11-20T12:16:10.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feaedd52700 1 -- 172.21.15.116:0/1526204618 shutdown_connections 2020-11-20T12:16:10.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feaedd52700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feac40557c0 0x7feac4057c80 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feaedd52700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feae813aee0 0x7feae8161120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feaedd52700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feae80612f0 0x7feae8153a50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feaedd52700 1 --2- 172.21.15.116:0/1526204618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feae80aa790 0x7feae8160c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.656+0000 7feaedd52700 1 -- 172.21.15.116:0/1526204618 >> 172.21.15.116:0/1526204618 conn(0x7feae80678a0 msgr2=0x7feae80c6020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:10.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.660+0000 7feaedd52700 1 -- 172.21.15.116:0/1526204618 shutdown_connections 2020-11-20T12:16:10.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.660+0000 7feaedd52700 1 -- 172.21.15.116:0/1526204618 wait complete. 2020-11-20T12:16:10.682 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542150 for osd.4 2020-11-20T12:16:10.683 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2020-11-20T12:16:10.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.860+0000 7fd62e547700 1 Processor -- start 2020-11-20T12:16:10.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.860+0000 7fd62e547700 1 -- start start 2020-11-20T12:16:10.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.860+0000 7fd62e547700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd6280c45a0 0x7fd628138670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.860+0000 7fd62e547700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd628138b60 0x7fd6280a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.860+0000 7fd62e547700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6280a9180 con 0x7fd6280c41c0 2020-11-20T12:16:10.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.860+0000 7fd62e547700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6280a9420 con 0x7fd6280c45a0 2020-11-20T12:16:10.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.860+0000 7fd62e547700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6280a96c0 con 0x7fd628138b60 2020-11-20T12:16:10.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62dd46700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd628138b60 0x7fd6280a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62cd44700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd6280c45a0 0x7fd628138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62dd46700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd628138b60 0x7fd6280a7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:52230/0 (socket says 172.21.15.116:52230) 2020-11-20T12:16:10.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62d545700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fd6280c41c0 0x7fd6280a5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:48942/0 (socket says 172.21.15.116:48942) 2020-11-20T12:16:10.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62dd46700 1 -- 172.21.15.116:0/3344911773 learned_addr learned my addr 172.21.15.116:0/3344911773 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:10.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62cd44700 1 -- 172.21.15.116:0/3344911773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd628138b60 msgr2=0x7fd6280a7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:10.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62cd44700 1 --2- 172.21.15.116:0/3344911773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd628138b60 0x7fd6280a7c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62cd44700 1 -- 172.21.15.116:0/3344911773 >> v1:172.21.15.173:6789/0 conn(0x7fd6280c41c0 legacy=0x7fd6280a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62cd44700 1 -- 172.21.15.116:0/3344911773 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6280a9960 con 0x7fd6280c45a0 2020-11-20T12:16:10.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62cd44700 1 --2- 172.21.15.116:0/3344911773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6280c45a0 0x7fd628138670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fd61800b9a0 tx=0x7fd618006df0).ready entity=mon.1 client_cookie=733f4680315f9f75 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:10.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd617fff700 1 -- 172.21.15.116:0/3344911773 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6180097b0 con 0x7fd6280c45a0 2020-11-20T12:16:10.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd617fff700 1 -- 172.21.15.116:0/3344911773 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd618009910 con 0x7fd6280c45a0 2020-11-20T12:16:10.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd617fff700 1 -- 172.21.15.116:0/3344911773 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd618009c90 con 0x7fd6280c45a0 2020-11-20T12:16:10.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62e547700 1 -- 172.21.15.116:0/3344911773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6280c45a0 msgr2=0x7fd628138670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62e547700 1 --2- 172.21.15.116:0/3344911773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6280c45a0 0x7fd628138670 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fd61800b9a0 tx=0x7fd618006df0).stop 2020-11-20T12:16:10.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62e547700 1 -- 172.21.15.116:0/3344911773 shutdown_connections 2020-11-20T12:16:10.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62e547700 1 --2- 172.21.15.116:0/3344911773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd628138b60 0x7fd6280a7c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62e547700 1 --2- 172.21.15.116:0/3344911773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6280c45a0 0x7fd628138670 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.864+0000 7fd62e547700 1 -- 172.21.15.116:0/3344911773 >> 172.21.15.116:0/3344911773 conn(0x7fd628062520 msgr2=0x7fd628062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:10.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.868+0000 7fd62e547700 1 -- 172.21.15.116:0/3344911773 shutdown_connections 2020-11-20T12:16:10.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.868+0000 7fd62e547700 1 -- 172.21.15.116:0/3344911773 wait complete. 2020-11-20T12:16:10.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.868+0000 7fd62e547700 1 Processor -- start 2020-11-20T12:16:10.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.868+0000 7fd62e547700 1 -- start start 2020-11-20T12:16:10.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62e547700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6280c41c0 0x7fd628130ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62e547700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd628138b60 0x7fd62812eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62e547700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd628151fc0 0x7fd62814eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62e547700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd628139220 con 0x7fd628151fc0 2020-11-20T12:16:10.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62e547700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62812f3c0 con 0x7fd6280c41c0 2020-11-20T12:16:10.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62e547700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62812fb60 con 0x7fd628138b60 2020-11-20T12:16:10.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62dd46700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd628151fc0 0x7fd62814eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62dd46700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd628151fc0 0x7fd62814eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43286/0 (socket says 172.21.15.116:43286) 2020-11-20T12:16:10.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62dd46700 1 -- 172.21.15.116:0/3473651467 learned_addr learned my addr 172.21.15.116:0/3473651467 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:10.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62cd44700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd628138b60 0x7fd62812eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62d545700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6280c41c0 0x7fd628130ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62dd46700 1 -- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd628138b60 msgr2=0x7fd62812eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62dd46700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd628138b60 0x7fd62812eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62dd46700 1 -- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6280c41c0 msgr2=0x7fd628130ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:10.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62dd46700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6280c41c0 0x7fd628130ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:10.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62dd46700 1 -- 172.21.15.116:0/3473651467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62812f620 con 0x7fd628151fc0 2020-11-20T12:16:10.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62dd46700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd628151fc0 0x7fd62814eb00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fd62000ecf0 tx=0x7fd620009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:10.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd6157fa700 1 -- 172.21.15.116:0/3473651467 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62000caa0 con 0x7fd628151fc0 2020-11-20T12:16:10.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd6157fa700 1 -- 172.21.15.116:0/3473651467 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62000cc00 con 0x7fd628151fc0 2020-11-20T12:16:10.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd6157fa700 1 -- 172.21.15.116:0/3473651467 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6200123f0 con 0x7fd628151fc0 2020-11-20T12:16:10.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62e547700 1 -- 172.21.15.116:0/3473651467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62812f8c0 con 0x7fd628151fc0 2020-11-20T12:16:10.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.872+0000 7fd62e547700 1 -- 172.21.15.116:0/3473651467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd628152400 con 0x7fd628151fc0 2020-11-20T12:16:10.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.876+0000 7fd62e547700 1 -- 172.21.15.116:0/3473651467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd628044f60 con 0x7fd628151fc0 2020-11-20T12:16:10.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.876+0000 7fd6157fa700 1 -- 172.21.15.116:0/3473651467 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd620022070 con 0x7fd628151fc0 2020-11-20T12:16:10.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.876+0000 7fd6157fa700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6000557c0 0x7fd600057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:10.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.876+0000 7fd6157fa700 1 -- 172.21.15.116:0/3473651467 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd62006eeb0 con 0x7fd628151fc0 2020-11-20T12:16:10.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.876+0000 7fd62d545700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6000557c0 0x7fd600057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:10.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.876+0000 7fd62d545700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6000557c0 0x7fd600057c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd6240045d0 tx=0x7fd624007b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:10.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:10.884+0000 7fd6157fa700 1 -- 172.21.15.116:0/3473651467 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd620044c40 con 0x7fd628151fc0 2020-11-20T12:16:11.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 -- 172.21.15.116:0/3473651467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fd6280a8e30 con 0x7fd628151fc0 2020-11-20T12:16:11.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd6157fa700 1 -- 172.21.15.116:0/3473651467 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd620016070 con 0x7fd628151fc0 2020-11-20T12:16:11.046 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:11.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 -- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6000557c0 msgr2=0x7fd600057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6000557c0 0x7fd600057c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fd6240045d0 tx=0x7fd624007b80).stop 2020-11-20T12:16:11.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 -- 172.21.15.116:0/3473651467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd628151fc0 msgr2=0x7fd62814eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd628151fc0 0x7fd62814eb00 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fd62000ecf0 tx=0x7fd620009290).stop 2020-11-20T12:16:11.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 -- 172.21.15.116:0/3473651467 shutdown_connections 2020-11-20T12:16:11.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6000557c0 0x7fd600057c80 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd628151fc0 0x7fd62814eb00 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6280c41c0 0x7fd628130ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 --2- 172.21.15.116:0/3473651467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd628138b60 0x7fd62812eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 -- 172.21.15.116:0/3473651467 >> 172.21.15.116:0/3473651467 conn(0x7fd628062520 msgr2=0x7fd628152e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:11.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 -- 172.21.15.116:0/3473651467 shutdown_connections 2020-11-20T12:16:11.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.044+0000 7fd62e547700 1 -- 172.21.15.116:0/3473651467 wait complete. 2020-11-20T12:16:11.070 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.5 2020-11-20T12:16:11.070 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2020-11-20T12:16:11.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.240+0000 7efef759e700 1 Processor -- start 2020-11-20T12:16:11.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.240+0000 7efef759e700 1 -- start start 2020-11-20T12:16:11.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef8138d80 0x7efef815a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef759e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef80a8c80 con 0x7efef80c45a0 2020-11-20T12:16:11.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef759e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef80a9180 con 0x7efef80c41c0 2020-11-20T12:16:11.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef80a9420 con 0x7efef8138d80 2020-11-20T12:16:11.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef5d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7efef80c45a0 0x7efef8138670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52724/0 (socket says 172.21.15.116:52724) 2020-11-20T12:16:11.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef5d9b700 1 -- 172.21.15.116:0/2742129244 learned_addr learned my addr 172.21.15.116:0/2742129244 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:11.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef6d9d700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef8138d80 0x7efef815a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 612704406 0 0) 0x7efef80a8c80 con 0x7efef80c45a0 2020-11-20T12:16:11.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efedc003240 con 0x7efef80c45a0 2020-11-20T12:16:11.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1115749052 0 0) 0x7efedc003240 con 0x7efef80c45a0 2020-11-20T12:16:11.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef8138d80 msgr2=0x7efef815a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:11.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef8138d80 0x7efef815a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 >> v1:172.21.15.173:6789/0 conn(0x7efef80c41c0 legacy=0x7efef80a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80a96c0 con 0x7efef80c45a0 2020-11-20T12:16:11.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7efeec0014e0 con 0x7efef80c45a0 2020-11-20T12:16:11.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 >> v1:172.21.15.116:6789/0 conn(0x7efef80c45a0 legacy=0x7efef8138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efedc003760 0x7efedc003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efedc0044e0 0x7efedc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef8138d80 0x7efedc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80a9420 con 0x7efedc0044e0 2020-11-20T12:16:11.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc005390 con 0x7efedc003760 2020-11-20T12:16:11.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efedc004320 con 0x7efef8138d80 2020-11-20T12:16:11.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef6d9d700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef8138d80 0x7efedc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef5d9b700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efedc0044e0 0x7efedc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.244+0000 7efef659c700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efedc003760 0x7efedc003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef6d9d700 1 -- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efedc003760 msgr2=0x7efedc003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:11.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef6d9d700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efedc003760 0x7efedc003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef6d9d700 1 -- 172.21.15.116:0/2742129244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efedc0044e0 msgr2=0x7efedc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:11.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef6d9d700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efedc0044e0 0x7efedc004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef6d9d700 1 -- 172.21.15.116:0/2742129244 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efedc003240 con 0x7efef8138d80 2020-11-20T12:16:11.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef6d9d700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef8138d80 0x7efedc004e50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7efef0009ea0 tx=0x7efef00091d0).ready entity=mon.2 client_cookie=9778d14c9c090c87 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:11.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef00095c0 con 0x7efef8138d80 2020-11-20T12:16:11.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efef000ec40 con 0x7efef8138d80 2020-11-20T12:16:11.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef559a700 1 -- 172.21.15.116:0/2742129244 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efef00123f0 con 0x7efef8138d80 2020-11-20T12:16:11.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef759e700 1 -- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef8138d80 msgr2=0x7efedc004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef759e700 1 --2- 172.21.15.116:0/2742129244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef8138d80 0x7efedc004e50 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7efef0009ea0 tx=0x7efef00091d0).stop 2020-11-20T12:16:11.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef6d9d700 1 -- 172.21.15.116:0/2742129244 reap_dead start 2020-11-20T12:16:11.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef759e700 1 -- 172.21.15.116:0/2742129244 shutdown_connections 2020-11-20T12:16:11.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef759e700 1 -- 172.21.15.116:0/2742129244 >> 172.21.15.116:0/2742129244 conn(0x7efef8062520 msgr2=0x7efef8062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:11.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef759e700 1 -- 172.21.15.116:0/2742129244 shutdown_connections 2020-11-20T12:16:11.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.248+0000 7efef759e700 1 -- 172.21.15.116:0/2742129244 wait complete. 2020-11-20T12:16:11.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef759e700 1 Processor -- start 2020-11-20T12:16:11.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef759e700 1 -- start start 2020-11-20T12:16:11.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef8138d80 0x7efef80c1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef80c1580 0x7efef8151b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efef8152010 0x7efef8135720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef815c8c0 con 0x7efef8138d80 2020-11-20T12:16:11.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef815cb60 con 0x7efef8152010 2020-11-20T12:16:11.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef815ce00 con 0x7efef80c1580 2020-11-20T12:16:11.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef8138d80 0x7efef80c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef8138d80 0x7efef80c1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43302/0 (socket says 172.21.15.116:43302) 2020-11-20T12:16:11.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef659c700 1 -- 172.21.15.116:0/3208896299 learned_addr learned my addr 172.21.15.116:0/3208896299 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:11.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.252+0000 7efef6d9d700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efef8152010 0x7efef8135720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efef5d9b700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef80c1580 0x7efef8151b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efef659c700 1 -- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef80c1580 msgr2=0x7efef8151b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efef659c700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef80c1580 0x7efef8151b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efef659c700 1 -- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efef8152010 msgr2=0x7efef8135720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efef659c700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efef8152010 0x7efef8135720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efef659c700 1 -- 172.21.15.116:0/3208896299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef815d0a0 con 0x7efef8138d80 2020-11-20T12:16:11.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efef659c700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef8138d80 0x7efef80c1090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7efee800f900 tx=0x7efee8007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:11.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efee67fc700 1 -- 172.21.15.116:0/3208896299 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8002750 con 0x7efef8138d80 2020-11-20T12:16:11.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efee67fc700 1 -- 172.21.15.116:0/3208896299 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee80028b0 con 0x7efef8138d80 2020-11-20T12:16:11.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efef759e700 1 -- 172.21.15.116:0/3208896299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef815d340 con 0x7efef8138d80 2020-11-20T12:16:11.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efee67fc700 1 -- 172.21.15.116:0/3208896299 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8002c30 con 0x7efef8138d80 2020-11-20T12:16:11.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efef759e700 1 -- 172.21.15.116:0/3208896299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef8136aa0 con 0x7efef8138d80 2020-11-20T12:16:11.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.256+0000 7efef759e700 1 -- 172.21.15.116:0/3208896299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef8044f60 con 0x7efef8138d80 2020-11-20T12:16:11.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.260+0000 7efee67fc700 1 -- 172.21.15.116:0/3208896299 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7efee801b030 con 0x7efef8138d80 2020-11-20T12:16:11.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.260+0000 7efee67fc700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efecc0557c0 0x7efecc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.260+0000 7efee67fc700 1 -- 172.21.15.116:0/3208896299 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efee806e260 con 0x7efef8138d80 2020-11-20T12:16:11.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.260+0000 7efef5d9b700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efecc0557c0 0x7efecc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.260+0000 7efef5d9b700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efecc0557c0 0x7efecc057c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7efef80a2c80 tx=0x7efeec009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:11.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.264+0000 7efee67fc700 1 -- 172.21.15.116:0/3208896299 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efee80441b0 con 0x7efef8138d80 2020-11-20T12:16:11.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.424+0000 7efef759e700 1 -- 172.21.15.116:0/3208896299 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7efef80a8e30 con 0x7efef8138d80 2020-11-20T12:16:11.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.424+0000 7efee67fc700 1 -- 172.21.15.116:0/3208896299 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efee80477d0 con 0x7efef8138d80 2020-11-20T12:16:11.426 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:11.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 -- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efecc0557c0 msgr2=0x7efecc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efecc0557c0 0x7efecc057c80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7efef80a2c80 tx=0x7efeec009040).stop 2020-11-20T12:16:11.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 -- 172.21.15.116:0/3208896299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef8138d80 msgr2=0x7efef80c1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef8138d80 0x7efef80c1090 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7efee800f900 tx=0x7efee8007b90).stop 2020-11-20T12:16:11.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 -- 172.21.15.116:0/3208896299 shutdown_connections 2020-11-20T12:16:11.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efecc0557c0 0x7efecc057c80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efef8152010 0x7efef8135720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efef8138d80 0x7efef80c1090 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 --2- 172.21.15.116:0/3208896299 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efef80c1580 0x7efef8151b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 -- 172.21.15.116:0/3208896299 >> 172.21.15.116:0/3208896299 conn(0x7efef8062520 msgr2=0x7efef80a4180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:11.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 -- 172.21.15.116:0/3208896299 shutdown_connections 2020-11-20T12:16:11.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.428+0000 7efef759e700 1 -- 172.21.15.116:0/3208896299 wait complete. 2020-11-20T12:16:11.451 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.6 2020-11-20T12:16:11.452 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2020-11-20T12:16:11.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.632+0000 7efe931bc700 1 Processor -- start 2020-11-20T12:16:11.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.632+0000 7efe931bc700 1 -- start start 2020-11-20T12:16:11.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.632+0000 7efe931bc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c13f1d0 0x7efe8c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.632+0000 7efe931bc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe8c0a40c0 0x7efe8c0c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.632+0000 7efe931bc700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe8c0a5680 con 0x7efe8c13f5b0 2020-11-20T12:16:11.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.632+0000 7efe931bc700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe8c0a58e0 con 0x7efe8c0a40c0 2020-11-20T12:16:11.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.632+0000 7efe931bc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c0a5b80 con 0x7efe8c13f1d0 2020-11-20T12:16:11.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.632+0000 7efe919b9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe8c0a40c0 0x7efe8c0c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.632+0000 7efe929bb700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7efe8c13f5b0 0x7efe8c1594c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:48974/0 (socket says 172.21.15.116:48974) 2020-11-20T12:16:11.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe929bb700 1 -- 172.21.15.116:0/3387707456 learned_addr learned my addr 172.21.15.116:0/3387707456 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:11.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe921ba700 1 --2- 172.21.15.116:0/3387707456 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c13f1d0 0x7efe8c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe919b9700 1 -- 172.21.15.116:0/3387707456 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c13f1d0 msgr2=0x7efe8c0a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:11.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe919b9700 1 --2- 172.21.15.116:0/3387707456 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c13f1d0 0x7efe8c0a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe919b9700 1 -- 172.21.15.116:0/3387707456 >> v1:172.21.15.173:6789/0 conn(0x7efe8c13f5b0 legacy=0x7efe8c1594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe919b9700 1 -- 172.21.15.116:0/3387707456 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c0a5e20 con 0x7efe8c0a40c0 2020-11-20T12:16:11.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe911b8700 1 -- 172.21.15.116:0/3387707456 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452966797 0 0) 0x7efe8c0a5680 con 0x7efe8c13f5b0 2020-11-20T12:16:11.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe919b9700 1 --2- 172.21.15.116:0/3387707456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe8c0a40c0 0x7efe8c0c3fb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7efe7c00b9a0 tx=0x7efe7c006df0).ready entity=mon.0 client_cookie=19b05cadac68304d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:11.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe911b8700 1 -- 172.21.15.116:0/3387707456 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c0097b0 con 0x7efe8c0a40c0 2020-11-20T12:16:11.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe911b8700 1 -- 172.21.15.116:0/3387707456 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7c009910 con 0x7efe8c0a40c0 2020-11-20T12:16:11.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe911b8700 1 -- 172.21.15.116:0/3387707456 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7c009c90 con 0x7efe8c0a40c0 2020-11-20T12:16:11.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe931bc700 1 -- 172.21.15.116:0/3387707456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe8c0a40c0 msgr2=0x7efe8c0c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe931bc700 1 --2- 172.21.15.116:0/3387707456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe8c0a40c0 0x7efe8c0c3fb0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7efe7c00b9a0 tx=0x7efe7c006df0).stop 2020-11-20T12:16:11.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe931bc700 1 -- 172.21.15.116:0/3387707456 shutdown_connections 2020-11-20T12:16:11.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe931bc700 1 --2- 172.21.15.116:0/3387707456 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c13f1d0 0x7efe8c0a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe931bc700 1 --2- 172.21.15.116:0/3387707456 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe8c0a40c0 0x7efe8c0c3fb0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe931bc700 1 -- 172.21.15.116:0/3387707456 >> 172.21.15.116:0/3387707456 conn(0x7efe8c062520 msgr2=0x7efe8c062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:11.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.636+0000 7efe931bc700 1 -- 172.21.15.116:0/3387707456 shutdown_connections 2020-11-20T12:16:11.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.640+0000 7efe931bc700 1 -- 172.21.15.116:0/3387707456 wait complete. 2020-11-20T12:16:11.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.640+0000 7efe931bc700 1 Processor -- start 2020-11-20T12:16:11.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.640+0000 7efe931bc700 1 -- start start 2020-11-20T12:16:11.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe931bc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe8c13f1d0 0x7efe8c138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe931bc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c13f5b0 0x7efe8c0a6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe931bc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe8c0a74c0 0x7efe8c0c2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe931bc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c0a7950 con 0x7efe8c13f1d0 2020-11-20T12:16:11.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe931bc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c13c2d0 con 0x7efe8c0a74c0 2020-11-20T12:16:11.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe931bc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c13c570 con 0x7efe8c13f5b0 2020-11-20T12:16:11.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe921ba700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe8c13f1d0 0x7efe8c138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe921ba700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe8c13f1d0 0x7efe8c138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43316/0 (socket says 172.21.15.116:43316) 2020-11-20T12:16:11.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe921ba700 1 -- 172.21.15.116:0/2753472841 learned_addr learned my addr 172.21.15.116:0/2753472841 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:11.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe919b9700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c13f5b0 0x7efe8c0a6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe929bb700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe8c0a74c0 0x7efe8c0c2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe921ba700 1 -- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c13f5b0 msgr2=0x7efe8c0a6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe921ba700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c13f5b0 0x7efe8c0a6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe921ba700 1 -- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe8c0a74c0 msgr2=0x7efe8c0c2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe921ba700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe8c0a74c0 0x7efe8c0c2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe921ba700 1 -- 172.21.15.116:0/2753472841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c13c810 con 0x7efe8c13f1d0 2020-11-20T12:16:11.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe921ba700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe8c13f1d0 0x7efe8c138d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7efe8800db60 tx=0x7efe88008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:11.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe7a7fc700 1 -- 172.21.15.116:0/2753472841 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800be30 con 0x7efe8c13f1d0 2020-11-20T12:16:11.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe7a7fc700 1 -- 172.21.15.116:0/2753472841 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe88009790 con 0x7efe8c13f1d0 2020-11-20T12:16:11.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe7a7fc700 1 -- 172.21.15.116:0/2753472841 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe88009b30 con 0x7efe8c13f1d0 2020-11-20T12:16:11.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe931bc700 1 -- 172.21.15.116:0/2753472841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8c13cab0 con 0x7efe8c13f1d0 2020-11-20T12:16:11.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.644+0000 7efe931bc700 1 -- 172.21.15.116:0/2753472841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8c0a9c50 con 0x7efe8c13f1d0 2020-11-20T12:16:11.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.648+0000 7efe931bc700 1 -- 172.21.15.116:0/2753472841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8c13cc40 con 0x7efe8c13f1d0 2020-11-20T12:16:11.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.648+0000 7efe7a7fc700 1 -- 172.21.15.116:0/2753472841 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7efe88013480 con 0x7efe8c13f1d0 2020-11-20T12:16:11.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.648+0000 7efe7a7fc700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe640557f0 0x7efe64057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:11.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.652+0000 7efe7a7fc700 1 -- 172.21.15.116:0/2753472841 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efe8806eb40 con 0x7efe8c13f1d0 2020-11-20T12:16:11.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.652+0000 7efe919b9700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe640557f0 0x7efe64057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:11.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.652+0000 7efe919b9700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe640557f0 0x7efe64057cb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7efe8c062070 tx=0x7efe7c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:11.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.656+0000 7efe7a7fc700 1 -- 172.21.15.116:0/2753472841 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efe88044980 con 0x7efe8c13f1d0 2020-11-20T12:16:11.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.816+0000 7efe931bc700 1 -- 172.21.15.116:0/2753472841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7efe8c044f60 con 0x7efe8c13f1d0 2020-11-20T12:16:11.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.816+0000 7efe7a7fc700 1 -- 172.21.15.116:0/2753472841 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7efe88009ea0 con 0x7efe8c13f1d0 2020-11-20T12:16:11.820 INFO:teuthology.orchestra.run.smithi116.stdout:60129542149 2020-11-20T12:16:11.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.820+0000 7efe931bc700 1 -- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe640557f0 msgr2=0x7efe64057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.820+0000 7efe931bc700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe640557f0 0x7efe64057cb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7efe8c062070 tx=0x7efe7c015000).stop 2020-11-20T12:16:11.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.820+0000 7efe931bc700 1 -- 172.21.15.116:0/2753472841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe8c13f1d0 msgr2=0x7efe8c138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:11.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.820+0000 7efe931bc700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe8c13f1d0 0x7efe8c138d10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7efe8800db60 tx=0x7efe88008040).stop 2020-11-20T12:16:11.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.820+0000 7efe931bc700 1 -- 172.21.15.116:0/2753472841 shutdown_connections 2020-11-20T12:16:11.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.820+0000 7efe931bc700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe640557f0 0x7efe64057cb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.820+0000 7efe931bc700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe8c0a74c0 0x7efe8c0c2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.820+0000 7efe931bc700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe8c13f1d0 0x7efe8c138d10 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.820+0000 7efe931bc700 1 --2- 172.21.15.116:0/2753472841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe8c13f5b0 0x7efe8c0a6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:11.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.820+0000 7efe931bc700 1 -- 172.21.15.116:0/2753472841 >> 172.21.15.116:0/2753472841 conn(0x7efe8c062520 msgr2=0x7efe8c0c3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:11.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.824+0000 7efe931bc700 1 -- 172.21.15.116:0/2753472841 shutdown_connections 2020-11-20T12:16:11.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:11.824+0000 7efe931bc700 1 -- 172.21.15.116:0/2753472841 wait complete. 2020-11-20T12:16:11.846 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.7 2020-11-20T12:16:11.847 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2020-11-20T12:16:11.847 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-20T12:16:12.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.012+0000 7f07415e6700 1 Processor -- start 2020-11-20T12:16:12.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.012+0000 7f07415e6700 1 -- start start 2020-11-20T12:16:12.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.012+0000 7f07415e6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f073c0a9c80 0x7f073c0ab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.012+0000 7f07415e6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f073c0c5c70 0x7f073c1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.012+0000 7f07415e6700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073c0abd80 con 0x7f073c0aa060 2020-11-20T12:16:12.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.012+0000 7f07415e6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073c0afaf0 con 0x7f073c0a9c80 2020-11-20T12:16:12.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.012+0000 7f07415e6700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073c0ac020 con 0x7f073c0c5c70 2020-11-20T12:16:12.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.012+0000 7f073affd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f073c0a9c80 0x7f073c0ab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.012+0000 7f073affd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f073c0a9c80 0x7f073c0ab590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43324/0 (socket says 172.21.15.116:43324) 2020-11-20T12:16:12.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.012+0000 7f073affd700 1 -- 172.21.15.116:0/2628342120 learned_addr learned my addr 172.21.15.116:0/2628342120 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:12.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.012+0000 7f073bfff700 1 --2- 172.21.15.116:0/2628342120 >> v2:172.21.15.173:3300/0 conn(0x7f073c0c5c70 0x7f073c1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f073affd700 1 -- 172.21.15.116:0/2628342120 >> v2:172.21.15.173:3300/0 conn(0x7f073c0c5c70 msgr2=0x7f073c1551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:12.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f073affd700 1 --2- 172.21.15.116:0/2628342120 >> v2:172.21.15.173:3300/0 conn(0x7f073c0c5c70 0x7f073c1551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f0739ffb700 1 -- 172.21.15.116:0/2628342120 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040936504 0 0) 0x7f073c0abd80 con 0x7f073c0aa060 2020-11-20T12:16:12.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f073affd700 1 -- 172.21.15.116:0/2628342120 >> v1:172.21.15.173:6789/0 conn(0x7f073c0aa060 legacy=0x7f073c064120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f073affd700 1 -- 172.21.15.116:0/2628342120 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c0ac2c0 con 0x7f073c0a9c80 2020-11-20T12:16:12.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f073affd700 1 --2- 172.21.15.116:0/2628342120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f073c0a9c80 0x7f073c0ab590 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f073c0688b0 tx=0x7f072c006eb0).ready entity=mon.0 client_cookie=5d4c79d51a74de84 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:12.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f0739ffb700 1 -- 172.21.15.116:0/2628342120 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c009810 con 0x7f073c0a9c80 2020-11-20T12:16:12.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f0739ffb700 1 -- 172.21.15.116:0/2628342120 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c009970 con 0x7f073c0a9c80 2020-11-20T12:16:12.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f0739ffb700 1 -- 172.21.15.116:0/2628342120 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c009cf0 con 0x7f073c0a9c80 2020-11-20T12:16:12.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f07415e6700 1 -- 172.21.15.116:0/2628342120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f073c0a9c80 msgr2=0x7f073c0ab590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f07415e6700 1 --2- 172.21.15.116:0/2628342120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f073c0a9c80 0x7f073c0ab590 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f073c0688b0 tx=0x7f072c006eb0).stop 2020-11-20T12:16:12.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f07415e6700 1 -- 172.21.15.116:0/2628342120 shutdown_connections 2020-11-20T12:16:12.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f07415e6700 1 --2- 172.21.15.116:0/2628342120 >> v2:172.21.15.173:3300/0 conn(0x7f073c0c5c70 0x7f073c1551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f07415e6700 1 --2- 172.21.15.116:0/2628342120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f073c0a9c80 0x7f073c0ab590 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f07415e6700 1 -- 172.21.15.116:0/2628342120 >> 172.21.15.116:0/2628342120 conn(0x7f073c0c9a90 msgr2=0x7f073c0c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:12.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f07415e6700 1 -- 172.21.15.116:0/2628342120 shutdown_connections 2020-11-20T12:16:12.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.016+0000 7f07415e6700 1 -- 172.21.15.116:0/2628342120 wait complete. 2020-11-20T12:16:12.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f07415e6700 1 Processor -- start 2020-11-20T12:16:12.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f07415e6700 1 -- start start 2020-11-20T12:16:12.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f07415e6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f073c0aa060 0x7f073c14ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f07415e6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f073c0c5c70 0x7f073c0c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f07415e6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f073c0c8f00 0x7f073c14d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f07415e6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c151d50 con 0x7f073c0aa060 2020-11-20T12:16:12.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f07415e6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c151ff0 con 0x7f073c0c8f00 2020-11-20T12:16:12.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f07415e6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c152290 con 0x7f073c0c5c70 2020-11-20T12:16:12.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f073affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f073c0aa060 0x7f073c14ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f073affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f073c0aa060 0x7f073c14ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43332/0 (socket says 172.21.15.116:43332) 2020-11-20T12:16:12.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f073affd700 1 -- 172.21.15.116:0/2383101577 learned_addr learned my addr 172.21.15.116:0/2383101577 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:12.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f073a7fc700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f073c0c5c70 0x7f073c0c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.020+0000 7f073bfff700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f073c0c8f00 0x7f073c14d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f073affd700 1 -- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f073c0c5c70 msgr2=0x7f073c0c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f073affd700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f073c0c5c70 0x7f073c0c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f073affd700 1 -- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f073c0c8f00 msgr2=0x7f073c14d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f073affd700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f073c0c8f00 0x7f073c14d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f073affd700 1 -- 172.21.15.116:0/2383101577 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c14c380 con 0x7f073c0aa060 2020-11-20T12:16:12.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f073affd700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f073c0aa060 0x7f073c14ef70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f072c006ef0 tx=0x7f072c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:12.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f0722ffd700 1 -- 172.21.15.116:0/2383101577 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c011d00 con 0x7f073c0aa060 2020-11-20T12:16:12.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f0722ffd700 1 -- 172.21.15.116:0/2383101577 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f072c011e60 con 0x7f073c0aa060 2020-11-20T12:16:12.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f07415e6700 1 -- 172.21.15.116:0/2383101577 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c14f460 con 0x7f073c0aa060 2020-11-20T12:16:12.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f0722ffd700 1 -- 172.21.15.116:0/2383101577 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c01e670 con 0x7f073c0aa060 2020-11-20T12:16:12.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f07415e6700 1 -- 172.21.15.116:0/2383101577 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c1528a0 con 0x7f073c0aa060 2020-11-20T12:16:12.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f07415e6700 1 -- 172.21.15.116:0/2383101577 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c152450 con 0x7f073c0aa060 2020-11-20T12:16:12.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.024+0000 7f0722ffd700 1 -- 172.21.15.116:0/2383101577 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f072c025030 con 0x7f073c0aa060 2020-11-20T12:16:12.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.028+0000 7f0722ffd700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07180557c0 0x7f0718057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.028+0000 7f0722ffd700 1 -- 172.21.15.116:0/2383101577 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f072c06fcf0 con 0x7f073c0aa060 2020-11-20T12:16:12.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.028+0000 7f073a7fc700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07180557c0 0x7f0718057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.028+0000 7f073a7fc700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07180557c0 0x7f0718057c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f0730007fd0 tx=0x7f0730007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:12.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.032+0000 7f0722ffd700 1 -- 172.21.15.116:0/2383101577 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f072c045c40 con 0x7f073c0aa060 2020-11-20T12:16:12.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.188+0000 7f07415e6700 1 -- 172.21.15.116:0/2383101577 --> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f073c05ca30 con 0x7f07180557c0 2020-11-20T12:16:12.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.200+0000 7f0722ffd700 1 -- 172.21.15.116:0/2383101577 <== mgr.4100 v2:172.21.15.173:6800/13860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82113 (secure 0 0 0) 0x7f073c05ca30 con 0x7f07180557c0 2020-11-20T12:16:12.202 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:12.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 -- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07180557c0 msgr2=0x7f0718057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07180557c0 0x7f0718057c80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f0730007fd0 tx=0x7f0730007a00).stop 2020-11-20T12:16:12.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 -- 172.21.15.116:0/2383101577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f073c0aa060 msgr2=0x7f073c14ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f073c0aa060 0x7f073c14ef70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f072c006ef0 tx=0x7f072c015000).stop 2020-11-20T12:16:12.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 -- 172.21.15.116:0/2383101577 shutdown_connections 2020-11-20T12:16:12.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07180557c0 0x7f0718057c80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f073c0c8f00 0x7f073c14d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f073c0aa060 0x7f073c14ef70 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 --2- 172.21.15.116:0/2383101577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f073c0c5c70 0x7f073c0c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 -- 172.21.15.116:0/2383101577 >> 172.21.15.116:0/2383101577 conn(0x7f073c0c9a90 msgr2=0x7f073c0ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:12.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 -- 172.21.15.116:0/2383101577 shutdown_connections 2020-11-20T12:16:12.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.204+0000 7f07415e6700 1 -- 172.21.15.116:0/2383101577 wait complete. 2020-11-20T12:16:12.208 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2020-11-20T12:16:12.230 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2020-11-20T12:16:10.531099+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":40824,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":38848,"kb_avail":754933896,"statfs":{"total":773094113280,"available":773052309504,"internally_reserved":0,"allocated":1957888,"data_stored":709474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39780352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.902409"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.651738+0000","last_change":"2020-11-20T12:16:00.644962+0000","last_active":"2020-11-20T12:16:00.651738+0000","last_peered":"2020-11-20T12:16:00.651738+0000","last_clean":"2020-11-20T12:16:00.651738+0000","last_became_active":"2020-11-20T12:16:00.641439+0000","last_became_peered":"2020-11-20T12:16:00.641439+0000","last_unstale":"2020-11-20T12:16:00.651738+0000","last_undegraded":"2020-11-20T12:16:00.651738+0000","last_fullsized":"2020-11-20T12:16:00.651738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.631922+0000","last_change":"2020-11-20T12:15:57.093803+0000","last_active":"2020-11-20T12:16:00.631922+0000","last_peered":"2020-11-20T12:16:00.631922+0000","last_clean":"2020-11-20T12:16:00.631922+0000","last_became_active":"2020-11-20T12:15:57.092999+0000","last_became_peered":"2020-11-20T12:15:57.092999+0000","last_unstale":"2020-11-20T12:16:00.631922+0000","last_undegraded":"2020-11-20T12:16:00.631922+0000","last_fullsized":"2020-11-20T12:16:00.631922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'6","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.653205+0000","last_change":"2020-11-20T12:16:00.143825+0000","last_active":"2020-11-20T12:16:00.653205+0000","last_peered":"2020-11-20T12:16:00.653205+0000","last_clean":"2020-11-20T12:16:00.653205+0000","last_became_active":"2020-11-20T12:16:00.143437+0000","last_became_peered":"2020-11-20T12:16:00.143437+0000","last_unstale":"2020-11-20T12:16:00.653205+0000","last_undegraded":"2020-11-20T12:16:00.653205+0000","last_fullsized":"2020-11-20T12:16:00.653205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.651842+0000","last_change":"2020-11-20T12:16:00.648746+0000","last_active":"2020-11-20T12:16:00.651842+0000","last_peered":"2020-11-20T12:16:00.651842+0000","last_clean":"2020-11-20T12:16:00.651842+0000","last_became_active":"2020-11-20T12:16:00.646406+0000","last_became_peered":"2020-11-20T12:16:00.646406+0000","last_unstale":"2020-11-20T12:16:00.651842+0000","last_undegraded":"2020-11-20T12:16:00.651842+0000","last_fullsized":"2020-11-20T12:16:00.651842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.632256+0000","last_change":"2020-11-20T12:15:57.004405+0000","last_active":"2020-11-20T12:16:00.632256+0000","last_peered":"2020-11-20T12:16:00.632256+0000","last_clean":"2020-11-20T12:16:00.632256+0000","last_became_active":"2020-11-20T12:15:57.003840+0000","last_became_peered":"2020-11-20T12:15:57.003840+0000","last_unstale":"2020-11-20T12:16:00.632256+0000","last_undegraded":"2020-11-20T12:16:00.632256+0000","last_fullsized":"2020-11-20T12:16:00.632256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'5","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.650055+0000","last_change":"2020-11-20T12:16:00.142756+0000","last_active":"2020-11-20T12:16:00.650055+0000","last_peered":"2020-11-20T12:16:00.650055+0000","last_clean":"2020-11-20T12:16:00.650055+0000","last_became_active":"2020-11-20T12:16:00.142474+0000","last_became_peered":"2020-11-20T12:16:00.142474+0000","last_unstale":"2020-11-20T12:16:00.650055+0000","last_undegraded":"2020-11-20T12:16:00.650055+0000","last_fullsized":"2020-11-20T12:16:00.650055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.652019+0000","last_change":"2020-11-20T12:16:00.646591+0000","last_active":"2020-11-20T12:16:00.652019+0000","last_peered":"2020-11-20T12:16:00.652019+0000","last_clean":"2020-11-20T12:16:00.652019+0000","last_became_active":"2020-11-20T12:16:00.644332+0000","last_became_peered":"2020-11-20T12:16:00.644332+0000","last_unstale":"2020-11-20T12:16:00.652019+0000","last_undegraded":"2020-11-20T12:16:00.652019+0000","last_fullsized":"2020-11-20T12:16:00.652019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.638828+0000","last_change":"2020-11-20T12:15:57.003854+0000","last_active":"2020-11-20T12:16:00.638828+0000","last_peered":"2020-11-20T12:16:00.638828+0000","last_clean":"2020-11-20T12:16:00.638828+0000","last_became_active":"2020-11-20T12:15:57.003016+0000","last_became_peered":"2020-11-20T12:15:57.003016+0000","last_unstale":"2020-11-20T12:16:00.638828+0000","last_undegraded":"2020-11-20T12:16:00.638828+0000","last_fullsized":"2020-11-20T12:16:00.638828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.656478+0000","last_change":"2020-11-20T12:16:00.150225+0000","last_active":"2020-11-20T12:16:00.656478+0000","last_peered":"2020-11-20T12:16:00.656478+0000","last_clean":"2020-11-20T12:16:00.656478+0000","last_became_active":"2020-11-20T12:16:00.149849+0000","last_became_peered":"2020-11-20T12:16:00.149849+0000","last_unstale":"2020-11-20T12:16:00.656478+0000","last_undegraded":"2020-11-20T12:16:00.656478+0000","last_fullsized":"2020-11-20T12:16:00.656478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.640628+0000","last_change":"2020-11-20T12:16:00.639235+0000","last_active":"2020-11-20T12:16:00.640628+0000","last_peered":"2020-11-20T12:16:00.640628+0000","last_clean":"2020-11-20T12:16:00.640628+0000","last_became_active":"2020-11-20T12:16:00.638779+0000","last_became_peered":"2020-11-20T12:16:00.638779+0000","last_unstale":"2020-11-20T12:16:00.640628+0000","last_undegraded":"2020-11-20T12:16:00.640628+0000","last_fullsized":"2020-11-20T12:16:00.640628+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.632430+0000","last_change":"2020-11-20T12:15:57.006495+0000","last_active":"2020-11-20T12:16:00.632430+0000","last_peered":"2020-11-20T12:16:00.632430+0000","last_clean":"2020-11-20T12:16:00.632430+0000","last_became_active":"2020-11-20T12:15:57.006030+0000","last_became_peered":"2020-11-20T12:15:57.006030+0000","last_unstale":"2020-11-20T12:16:00.632430+0000","last_undegraded":"2020-11-20T12:16:00.632430+0000","last_fullsized":"2020-11-20T12:16:00.632430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.655963+0000","last_change":"2020-11-20T12:15:52.377671+0000","last_active":"2020-11-20T12:16:00.655963+0000","last_peered":"2020-11-20T12:16:00.655963+0000","last_clean":"2020-11-20T12:16:00.655963+0000","last_became_active":"2020-11-20T12:15:52.376821+0000","last_became_peered":"2020-11-20T12:15:52.376821+0000","last_unstale":"2020-11-20T12:16:00.655963+0000","last_undegraded":"2020-11-20T12:16:00.655963+0000","last_fullsized":"2020-11-20T12:16:00.655963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:47.518661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:47.518661+0000","last_clean_scrub_stamp":"2020-11-20T12:15:47.518661+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"22'1","reported_seq":"11","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.653871+0000","last_change":"2020-11-20T12:16:00.143747+0000","last_active":"2020-11-20T12:16:00.653871+0000","last_peered":"2020-11-20T12:16:00.653871+0000","last_clean":"2020-11-20T12:16:00.653871+0000","last_became_active":"2020-11-20T12:16:00.143309+0000","last_became_peered":"2020-11-20T12:16:00.143309+0000","last_unstale":"2020-11-20T12:16:00.653871+0000","last_undegraded":"2020-11-20T12:16:00.653871+0000","last_fullsized":"2020-11-20T12:16:00.653871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.653741+0000","last_change":"2020-11-20T12:16:00.651925+0000","last_active":"2020-11-20T12:16:00.653741+0000","last_peered":"2020-11-20T12:16:00.653741+0000","last_clean":"2020-11-20T12:16:00.653741+0000","last_became_active":"2020-11-20T12:16:00.651527+0000","last_became_peered":"2020-11-20T12:16:00.651527+0000","last_unstale":"2020-11-20T12:16:00.653741+0000","last_undegraded":"2020-11-20T12:16:00.653741+0000","last_fullsized":"2020-11-20T12:16:00.653741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.655968+0000","last_change":"2020-11-20T12:15:57.003949+0000","last_active":"2020-11-20T12:16:00.655968+0000","last_peered":"2020-11-20T12:16:00.655968+0000","last_clean":"2020-11-20T12:16:00.655968+0000","last_became_active":"2020-11-20T12:15:57.003587+0000","last_became_peered":"2020-11-20T12:15:57.003587+0000","last_unstale":"2020-11-20T12:16:00.655968+0000","last_undegraded":"2020-11-20T12:16:00.655968+0000","last_fullsized":"2020-11-20T12:16:00.655968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.633327+0000","last_change":"2020-11-20T12:16:00.145553+0000","last_active":"2020-11-20T12:16:00.633327+0000","last_peered":"2020-11-20T12:16:00.633327+0000","last_clean":"2020-11-20T12:16:00.633327+0000","last_became_active":"2020-11-20T12:16:00.144929+0000","last_became_peered":"2020-11-20T12:16:00.144929+0000","last_unstale":"2020-11-20T12:16:00.633327+0000","last_undegraded":"2020-11-20T12:16:00.633327+0000","last_fullsized":"2020-11-20T12:16:00.633327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.640639+0000","last_change":"2020-11-20T12:16:00.638051+0000","last_active":"2020-11-20T12:16:00.640639+0000","last_peered":"2020-11-20T12:16:00.640639+0000","last_clean":"2020-11-20T12:16:00.640639+0000","last_became_active":"2020-11-20T12:16:00.637685+0000","last_became_peered":"2020-11-20T12:16:00.637685+0000","last_unstale":"2020-11-20T12:16:00.640639+0000","last_undegraded":"2020-11-20T12:16:00.640639+0000","last_fullsized":"2020-11-20T12:16:00.640639+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.655784+0000","last_change":"2020-11-20T12:15:57.093800+0000","last_active":"2020-11-20T12:16:00.655784+0000","last_peered":"2020-11-20T12:16:00.655784+0000","last_clean":"2020-11-20T12:16:00.655784+0000","last_became_active":"2020-11-20T12:15:57.093230+0000","last_became_peered":"2020-11-20T12:15:57.093230+0000","last_unstale":"2020-11-20T12:16:00.655784+0000","last_undegraded":"2020-11-20T12:16:00.655784+0000","last_fullsized":"2020-11-20T12:16:00.655784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.660253+0000","last_change":"2020-11-20T12:16:00.150292+0000","last_active":"2020-11-20T12:16:00.660253+0000","last_peered":"2020-11-20T12:16:00.660253+0000","last_clean":"2020-11-20T12:16:00.660253+0000","last_became_active":"2020-11-20T12:16:00.149758+0000","last_became_peered":"2020-11-20T12:16:00.149758+0000","last_unstale":"2020-11-20T12:16:00.660253+0000","last_undegraded":"2020-11-20T12:16:00.660253+0000","last_fullsized":"2020-11-20T12:16:00.660253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.660074+0000","last_change":"2020-11-20T12:16:00.658553+0000","last_active":"2020-11-20T12:16:00.660074+0000","last_peered":"2020-11-20T12:16:00.660074+0000","last_clean":"2020-11-20T12:16:00.660074+0000","last_became_active":"2020-11-20T12:16:00.658218+0000","last_became_peered":"2020-11-20T12:16:00.658218+0000","last_unstale":"2020-11-20T12:16:00.660074+0000","last_undegraded":"2020-11-20T12:16:00.660074+0000","last_fullsized":"2020-11-20T12:16:00.660074+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"22'6","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.656872+0000","last_change":"2020-11-20T12:16:00.153258+0000","last_active":"2020-11-20T12:16:00.656872+0000","last_peered":"2020-11-20T12:16:00.656872+0000","last_clean":"2020-11-20T12:16:00.656872+0000","last_became_active":"2020-11-20T12:16:00.152908+0000","last_became_peered":"2020-11-20T12:16:00.152908+0000","last_unstale":"2020-11-20T12:16:00.656872+0000","last_undegraded":"2020-11-20T12:16:00.656872+0000","last_fullsized":"2020-11-20T12:16:00.656872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.632090+0000","last_change":"2020-11-20T12:15:57.006042+0000","last_active":"2020-11-20T12:16:00.632090+0000","last_peered":"2020-11-20T12:16:00.632090+0000","last_clean":"2020-11-20T12:16:00.632090+0000","last_became_active":"2020-11-20T12:15:57.005582+0000","last_became_peered":"2020-11-20T12:15:57.005582+0000","last_unstale":"2020-11-20T12:16:00.632090+0000","last_undegraded":"2020-11-20T12:16:00.632090+0000","last_fullsized":"2020-11-20T12:16:00.632090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.640173+0000","last_change":"2020-11-20T12:16:00.637895+0000","last_active":"2020-11-20T12:16:00.640173+0000","last_peered":"2020-11-20T12:16:00.640173+0000","last_clean":"2020-11-20T12:16:00.640173+0000","last_became_active":"2020-11-20T12:16:00.637292+0000","last_became_peered":"2020-11-20T12:16:00.637292+0000","last_unstale":"2020-11-20T12:16:00.640173+0000","last_undegraded":"2020-11-20T12:16:00.640173+0000","last_fullsized":"2020-11-20T12:16:00.640173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.635165+0000","last_change":"2020-11-20T12:15:57.006654+0000","last_active":"2020-11-20T12:16:00.635165+0000","last_peered":"2020-11-20T12:16:00.635165+0000","last_clean":"2020-11-20T12:16:00.635165+0000","last_became_active":"2020-11-20T12:15:57.006200+0000","last_became_peered":"2020-11-20T12:15:57.006200+0000","last_unstale":"2020-11-20T12:16:00.635165+0000","last_undegraded":"2020-11-20T12:16:00.635165+0000","last_fullsized":"2020-11-20T12:16:00.635165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'4","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T12:16:01.718956+0000","last_change":"2020-11-20T12:16:00.145290+0000","last_active":"2020-11-20T12:16:01.718956+0000","last_peered":"2020-11-20T12:16:01.718956+0000","last_clean":"2020-11-20T12:16:01.718956+0000","last_became_active":"2020-11-20T12:16:00.144853+0000","last_became_peered":"2020-11-20T12:16:00.144853+0000","last_unstale":"2020-11-20T12:16:01.718956+0000","last_undegraded":"2020-11-20T12:16:01.718956+0000","last_fullsized":"2020-11-20T12:16:01.718956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":88834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5430000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1789999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.512}]}]},{"osd":6,"up_from":14,"seq":60129542149,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":237568,"data_stored":88259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3090000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.76}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8889999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.222}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.956}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.552}]}]},{"osd":1,"up_from":14,"seq":60129542149,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":89191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4279999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9649999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.508}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0449999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4510000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2250000000000001}]}]},{"osd":0,"up_from":14,"seq":60129542149,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5104,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366736,"statfs":{"total":96636764160,"available":96631537664,"internally_reserved":0,"allocated":237568,"data_stored":87690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.024}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.472}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9490000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0590000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4390000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4380000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9289999999999998}]}]},{"osd":2,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":89794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0350000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1190000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3450000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5960000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9409999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.952}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2080000000000002}]}]},{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5120,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366720,"statfs":{"total":96636764160,"available":96631521280,"internally_reserved":0,"allocated":253952,"data_stored":88994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6890000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.978}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3410000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3799999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.286}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.157}]}]},{"osd":4,"up_from":14,"seq":60129542150,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5108,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366732,"statfs":{"total":96636764160,"available":96631533568,"internally_reserved":0,"allocated":241664,"data_stored":88293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0089999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.141}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9249999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5070000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.2460000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7629999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542149,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":88419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.552}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.978}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1739999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9929999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8690000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.4649999999999999}]}]}]}} 2020-11-20T12:16:12.233 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2020-11-20T12:16:12.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.404+0000 7fbb63e9e700 1 Processor -- start 2020-11-20T12:16:12.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.404+0000 7fbb63e9e700 1 -- start start 2020-11-20T12:16:12.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb63e9e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbb5c0a3690 0x7fbb5c152740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb63e9e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbb5c0c2450 0x7fbb5c0c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb63e9e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5c0bed50 con 0x7fbb5c0be7d0 2020-11-20T12:16:12.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb63e9e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5c0beff0 con 0x7fbb5c0c2450 2020-11-20T12:16:12.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb63e9e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5c0bf290 con 0x7fbb5c0a3690 2020-11-20T12:16:12.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb61439700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbb5c0be7d0 0x7fbb5c0b9130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52774/0 (socket says 172.21.15.116:52774) 2020-11-20T12:16:12.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb6243b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbb5c0c2450 0x7fbb5c0c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb61439700 1 -- 172.21.15.116:0/2438597618 learned_addr learned my addr 172.21.15.116:0/2438597618 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:12.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb61c3a700 1 --2- 172.21.15.116:0/2438597618 >> v2:172.21.15.173:3300/0 conn(0x7fbb5c0a3690 0x7fbb5c152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb6243b700 1 -- 172.21.15.116:0/2438597618 >> v2:172.21.15.173:3300/0 conn(0x7fbb5c0a3690 msgr2=0x7fbb5c152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:12.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb6243b700 1 --2- 172.21.15.116:0/2438597618 >> v2:172.21.15.173:3300/0 conn(0x7fbb5c0a3690 0x7fbb5c152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb6243b700 1 -- 172.21.15.116:0/2438597618 >> v1:172.21.15.116:6789/0 conn(0x7fbb5c0be7d0 legacy=0x7fbb5c0b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb6243b700 1 -- 172.21.15.116:0/2438597618 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c0bf530 con 0x7fbb5c0c2450 2020-11-20T12:16:12.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb60c38700 1 -- 172.21.15.116:0/2438597618 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4253981431 0 0) 0x7fbb5c0bed50 con 0x7fbb5c0be7d0 2020-11-20T12:16:12.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb6243b700 1 --2- 172.21.15.116:0/2438597618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb5c0c2450 0x7fbb5c0c28e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fbb5c0c1c80 tx=0x7fbb540091d0).ready entity=mon.0 client_cookie=1d5b901e5738277 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:12.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb60c38700 1 -- 172.21.15.116:0/2438597618 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5400bad0 con 0x7fbb5c0c2450 2020-11-20T12:16:12.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.408+0000 7fbb60c38700 1 -- 172.21.15.116:0/2438597618 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5400bc30 con 0x7fbb5c0c2450 2020-11-20T12:16:12.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.412+0000 7fbb60c38700 1 -- 172.21.15.116:0/2438597618 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54011400 con 0x7fbb5c0c2450 2020-11-20T12:16:12.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.412+0000 7fbb63e9e700 1 -- 172.21.15.116:0/2438597618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb5c0c2450 msgr2=0x7fbb5c0c28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.412+0000 7fbb63e9e700 1 --2- 172.21.15.116:0/2438597618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb5c0c2450 0x7fbb5c0c28e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fbb5c0c1c80 tx=0x7fbb540091d0).stop 2020-11-20T12:16:12.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.412+0000 7fbb63e9e700 1 -- 172.21.15.116:0/2438597618 shutdown_connections 2020-11-20T12:16:12.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.412+0000 7fbb63e9e700 1 --2- 172.21.15.116:0/2438597618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb5c0c2450 0x7fbb5c0c28e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.412+0000 7fbb63e9e700 1 --2- 172.21.15.116:0/2438597618 >> v2:172.21.15.173:3300/0 conn(0x7fbb5c0a3690 0x7fbb5c152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.412+0000 7fbb63e9e700 1 -- 172.21.15.116:0/2438597618 >> 172.21.15.116:0/2438597618 conn(0x7fbb5c060c80 msgr2=0x7fbb5c0c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:12.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.412+0000 7fbb63e9e700 1 -- 172.21.15.116:0/2438597618 shutdown_connections 2020-11-20T12:16:12.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.412+0000 7fbb63e9e700 1 -- 172.21.15.116:0/2438597618 wait complete. 2020-11-20T12:16:12.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb63e9e700 1 Processor -- start 2020-11-20T12:16:12.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb63e9e700 1 -- start start 2020-11-20T12:16:12.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb63e9e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb5c0a3690 0x7fbb5c0bd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb63e9e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb5c0be7d0 0x7fbb5c14d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb63e9e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb5c0c2450 0x7fbb5c164300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb63e9e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c0bddb0 con 0x7fbb5c0c2450 2020-11-20T12:16:12.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb63e9e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c0bdfd0 con 0x7fbb5c0a3690 2020-11-20T12:16:12.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb63e9e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5c153c70 con 0x7fbb5c0be7d0 2020-11-20T12:16:12.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb6243b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb5c0c2450 0x7fbb5c164300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb6243b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb5c0c2450 0x7fbb5c164300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43346/0 (socket says 172.21.15.116:43346) 2020-11-20T12:16:12.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb6243b700 1 -- 172.21.15.116:0/733837934 learned_addr learned my addr 172.21.15.116:0/733837934 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:12.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb61439700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb5c0be7d0 0x7fbb5c14d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb61c3a700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb5c0a3690 0x7fbb5c0bd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb6243b700 1 -- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb5c0be7d0 msgr2=0x7fbb5c14d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb6243b700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb5c0be7d0 0x7fbb5c14d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb6243b700 1 -- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb5c0a3690 msgr2=0x7fbb5c0bd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb6243b700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb5c0a3690 0x7fbb5c0bd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb6243b700 1 -- 172.21.15.116:0/733837934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5c153f10 con 0x7fbb5c0c2450 2020-11-20T12:16:12.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb6243b700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb5c0c2450 0x7fbb5c164300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fbb5c061120 tx=0x7fbb54016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:12.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb49ffb700 1 -- 172.21.15.116:0/733837934 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54010800 con 0x7fbb5c0c2450 2020-11-20T12:16:12.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb49ffb700 1 -- 172.21.15.116:0/733837934 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb54010960 con 0x7fbb5c0c2450 2020-11-20T12:16:12.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb63e9e700 1 -- 172.21.15.116:0/733837934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb5c1541b0 con 0x7fbb5c0c2450 2020-11-20T12:16:12.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.416+0000 7fbb49ffb700 1 -- 172.21.15.116:0/733837934 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb54010d00 con 0x7fbb5c0c2450 2020-11-20T12:16:12.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.420+0000 7fbb63e9e700 1 -- 172.21.15.116:0/733837934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb5c14d5e0 con 0x7fbb5c0c2450 2020-11-20T12:16:12.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.420+0000 7fbb63e9e700 1 -- 172.21.15.116:0/733837934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb5c05ca30 con 0x7fbb5c0c2450 2020-11-20T12:16:12.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.420+0000 7fbb49ffb700 1 -- 172.21.15.116:0/733837934 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fbb54017070 con 0x7fbb5c0c2450 2020-11-20T12:16:12.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.420+0000 7fbb49ffb700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbb340557c0 0x7fbb34057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.424+0000 7fbb49ffb700 1 -- 172.21.15.116:0/733837934 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbb54070db0 con 0x7fbb5c0c2450 2020-11-20T12:16:12.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.424+0000 7fbb61c3a700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbb340557c0 0x7fbb34057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.424+0000 7fbb61c3a700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbb340557c0 0x7fbb34057c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fbb58006fd0 tx=0x7fbb58008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:12.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.428+0000 7fbb49ffb700 1 -- 172.21.15.116:0/733837934 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbb54075070 con 0x7fbb5c0c2450 2020-11-20T12:16:12.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.580+0000 7fbb63e9e700 1 -- 172.21.15.116:0/733837934 --> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb5c154650 con 0x7fbb340557c0 2020-11-20T12:16:12.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.588+0000 7fbb49ffb700 1 -- 172.21.15.116:0/733837934 <== mgr.4100 v2:172.21.15.173:6800/13860 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+82062 (secure 0 0 0) 0x7fbb5800c6d0 con 0x7fbb340557c0 2020-11-20T12:16:12.590 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:12.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.588+0000 7fbb63e9e700 1 -- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbb340557c0 msgr2=0x7fbb34057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.588+0000 7fbb63e9e700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbb340557c0 0x7fbb34057c80 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fbb58006fd0 tx=0x7fbb58008040).stop 2020-11-20T12:16:12.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.588+0000 7fbb63e9e700 1 -- 172.21.15.116:0/733837934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb5c0c2450 msgr2=0x7fbb5c164300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.588+0000 7fbb63e9e700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb5c0c2450 0x7fbb5c164300 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fbb5c061120 tx=0x7fbb54016000).stop 2020-11-20T12:16:12.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.592+0000 7fbb63e9e700 1 -- 172.21.15.116:0/733837934 shutdown_connections 2020-11-20T12:16:12.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.592+0000 7fbb63e9e700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbb340557c0 0x7fbb34057c80 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.592+0000 7fbb63e9e700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbb5c0c2450 0x7fbb5c164300 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.592+0000 7fbb63e9e700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbb5c0a3690 0x7fbb5c0bd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.592+0000 7fbb63e9e700 1 --2- 172.21.15.116:0/733837934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbb5c0be7d0 0x7fbb5c14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.592+0000 7fbb63e9e700 1 -- 172.21.15.116:0/733837934 >> 172.21.15.116:0/733837934 conn(0x7fbb5c060c80 msgr2=0x7fbb5c0a12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:12.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.592+0000 7fbb63e9e700 1 -- 172.21.15.116:0/733837934 shutdown_connections 2020-11-20T12:16:12.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.592+0000 7fbb63e9e700 1 -- 172.21.15.116:0/733837934 wait complete. 2020-11-20T12:16:12.595 INFO:teuthology.orchestra.run.smithi116.stderr:dumped all 2020-11-20T12:16:12.617 INFO:teuthology.orchestra.run.smithi116.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2020-11-20T12:16:12.531928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":40,"ondisk_log_size":40,"up":50,"acting":50,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":50,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":40824,"kb_used_data":1912,"kb_used_omap":0,"kb_used_meta":38848,"kb_avail":754933896,"statfs":{"total":773094113280,"available":773052309504,"internally_reserved":0,"allocated":1957888,"data_stored":709474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":39780352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.891793"},"pg_stats":[{"pgid":"4.2","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.651738+0000","last_change":"2020-11-20T12:16:00.644962+0000","last_active":"2020-11-20T12:16:00.651738+0000","last_peered":"2020-11-20T12:16:00.651738+0000","last_clean":"2020-11-20T12:16:00.651738+0000","last_became_active":"2020-11-20T12:16:00.641439+0000","last_became_peered":"2020-11-20T12:16:00.641439+0000","last_unstale":"2020-11-20T12:16:00.651738+0000","last_undegraded":"2020-11-20T12:16:00.651738+0000","last_fullsized":"2020-11-20T12:16:00.651738+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.631922+0000","last_change":"2020-11-20T12:15:57.093803+0000","last_active":"2020-11-20T12:16:00.631922+0000","last_peered":"2020-11-20T12:16:00.631922+0000","last_clean":"2020-11-20T12:16:00.631922+0000","last_became_active":"2020-11-20T12:15:57.092999+0000","last_became_peered":"2020-11-20T12:15:57.092999+0000","last_unstale":"2020-11-20T12:16:00.631922+0000","last_undegraded":"2020-11-20T12:16:00.631922+0000","last_fullsized":"2020-11-20T12:16:00.631922+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"22'6","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.653205+0000","last_change":"2020-11-20T12:16:00.143825+0000","last_active":"2020-11-20T12:16:00.653205+0000","last_peered":"2020-11-20T12:16:00.653205+0000","last_clean":"2020-11-20T12:16:00.653205+0000","last_became_active":"2020-11-20T12:16:00.143437+0000","last_became_peered":"2020-11-20T12:16:00.143437+0000","last_unstale":"2020-11-20T12:16:00.653205+0000","last_undegraded":"2020-11-20T12:16:00.653205+0000","last_fullsized":"2020-11-20T12:16:00.653205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":160,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.651842+0000","last_change":"2020-11-20T12:16:00.648746+0000","last_active":"2020-11-20T12:16:00.651842+0000","last_peered":"2020-11-20T12:16:00.651842+0000","last_clean":"2020-11-20T12:16:00.651842+0000","last_became_active":"2020-11-20T12:16:00.646406+0000","last_became_peered":"2020-11-20T12:16:00.646406+0000","last_unstale":"2020-11-20T12:16:00.651842+0000","last_undegraded":"2020-11-20T12:16:00.651842+0000","last_fullsized":"2020-11-20T12:16:00.651842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.632256+0000","last_change":"2020-11-20T12:15:57.004405+0000","last_active":"2020-11-20T12:16:00.632256+0000","last_peered":"2020-11-20T12:16:00.632256+0000","last_clean":"2020-11-20T12:16:00.632256+0000","last_became_active":"2020-11-20T12:15:57.003840+0000","last_became_peered":"2020-11-20T12:15:57.003840+0000","last_unstale":"2020-11-20T12:16:00.632256+0000","last_undegraded":"2020-11-20T12:16:00.632256+0000","last_fullsized":"2020-11-20T12:16:00.632256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"22'5","reported_seq":"15","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.650055+0000","last_change":"2020-11-20T12:16:00.142756+0000","last_active":"2020-11-20T12:16:00.650055+0000","last_peered":"2020-11-20T12:16:00.650055+0000","last_clean":"2020-11-20T12:16:00.650055+0000","last_became_active":"2020-11-20T12:16:00.142474+0000","last_became_peered":"2020-11-20T12:16:00.142474+0000","last_unstale":"2020-11-20T12:16:00.650055+0000","last_undegraded":"2020-11-20T12:16:00.650055+0000","last_fullsized":"2020-11-20T12:16:00.650055+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.652019+0000","last_change":"2020-11-20T12:16:00.646591+0000","last_active":"2020-11-20T12:16:00.652019+0000","last_peered":"2020-11-20T12:16:00.652019+0000","last_clean":"2020-11-20T12:16:00.652019+0000","last_became_active":"2020-11-20T12:16:00.644332+0000","last_became_peered":"2020-11-20T12:16:00.644332+0000","last_unstale":"2020-11-20T12:16:00.652019+0000","last_undegraded":"2020-11-20T12:16:00.652019+0000","last_fullsized":"2020-11-20T12:16:00.652019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.638828+0000","last_change":"2020-11-20T12:15:57.003854+0000","last_active":"2020-11-20T12:16:00.638828+0000","last_peered":"2020-11-20T12:16:00.638828+0000","last_clean":"2020-11-20T12:16:00.638828+0000","last_became_active":"2020-11-20T12:15:57.003016+0000","last_became_peered":"2020-11-20T12:15:57.003016+0000","last_unstale":"2020-11-20T12:16:00.638828+0000","last_undegraded":"2020-11-20T12:16:00.638828+0000","last_fullsized":"2020-11-20T12:16:00.638828+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.656478+0000","last_change":"2020-11-20T12:16:00.150225+0000","last_active":"2020-11-20T12:16:00.656478+0000","last_peered":"2020-11-20T12:16:00.656478+0000","last_clean":"2020-11-20T12:16:00.656478+0000","last_became_active":"2020-11-20T12:16:00.149849+0000","last_became_peered":"2020-11-20T12:16:00.149849+0000","last_unstale":"2020-11-20T12:16:00.656478+0000","last_undegraded":"2020-11-20T12:16:00.656478+0000","last_fullsized":"2020-11-20T12:16:00.656478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.640628+0000","last_change":"2020-11-20T12:16:00.639235+0000","last_active":"2020-11-20T12:16:00.640628+0000","last_peered":"2020-11-20T12:16:00.640628+0000","last_clean":"2020-11-20T12:16:00.640628+0000","last_became_active":"2020-11-20T12:16:00.638779+0000","last_became_peered":"2020-11-20T12:16:00.638779+0000","last_unstale":"2020-11-20T12:16:00.640628+0000","last_undegraded":"2020-11-20T12:16:00.640628+0000","last_fullsized":"2020-11-20T12:16:00.640628+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.632430+0000","last_change":"2020-11-20T12:15:57.006495+0000","last_active":"2020-11-20T12:16:00.632430+0000","last_peered":"2020-11-20T12:16:00.632430+0000","last_clean":"2020-11-20T12:16:00.632430+0000","last_became_active":"2020-11-20T12:15:57.006030+0000","last_became_peered":"2020-11-20T12:15:57.006030+0000","last_unstale":"2020-11-20T12:16:00.632430+0000","last_undegraded":"2020-11-20T12:16:00.632430+0000","last_fullsized":"2020-11-20T12:16:00.632430+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'2","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.655963+0000","last_change":"2020-11-20T12:15:52.377671+0000","last_active":"2020-11-20T12:16:00.655963+0000","last_peered":"2020-11-20T12:16:00.655963+0000","last_clean":"2020-11-20T12:16:00.655963+0000","last_became_active":"2020-11-20T12:15:52.376821+0000","last_became_peered":"2020-11-20T12:15:52.376821+0000","last_unstale":"2020-11-20T12:16:00.655963+0000","last_undegraded":"2020-11-20T12:16:00.655963+0000","last_fullsized":"2020-11-20T12:16:00.655963+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:47.518661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:47.518661+0000","last_clean_scrub_stamp":"2020-11-20T12:15:47.518661+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"22'1","reported_seq":"11","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.653871+0000","last_change":"2020-11-20T12:16:00.143747+0000","last_active":"2020-11-20T12:16:00.653871+0000","last_peered":"2020-11-20T12:16:00.653871+0000","last_clean":"2020-11-20T12:16:00.653871+0000","last_became_active":"2020-11-20T12:16:00.143309+0000","last_became_peered":"2020-11-20T12:16:00.143309+0000","last_unstale":"2020-11-20T12:16:00.653871+0000","last_undegraded":"2020-11-20T12:16:00.653871+0000","last_fullsized":"2020-11-20T12:16:00.653871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.653741+0000","last_change":"2020-11-20T12:16:00.651925+0000","last_active":"2020-11-20T12:16:00.653741+0000","last_peered":"2020-11-20T12:16:00.653741+0000","last_clean":"2020-11-20T12:16:00.653741+0000","last_became_active":"2020-11-20T12:16:00.651527+0000","last_became_peered":"2020-11-20T12:16:00.651527+0000","last_unstale":"2020-11-20T12:16:00.653741+0000","last_undegraded":"2020-11-20T12:16:00.653741+0000","last_fullsized":"2020-11-20T12:16:00.653741+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.655968+0000","last_change":"2020-11-20T12:15:57.003949+0000","last_active":"2020-11-20T12:16:00.655968+0000","last_peered":"2020-11-20T12:16:00.655968+0000","last_clean":"2020-11-20T12:16:00.655968+0000","last_became_active":"2020-11-20T12:15:57.003587+0000","last_became_peered":"2020-11-20T12:15:57.003587+0000","last_unstale":"2020-11-20T12:16:00.655968+0000","last_undegraded":"2020-11-20T12:16:00.655968+0000","last_fullsized":"2020-11-20T12:16:00.655968+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":"10","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.633327+0000","last_change":"2020-11-20T12:16:00.145553+0000","last_active":"2020-11-20T12:16:00.633327+0000","last_peered":"2020-11-20T12:16:00.633327+0000","last_clean":"2020-11-20T12:16:00.633327+0000","last_became_active":"2020-11-20T12:16:00.144929+0000","last_became_peered":"2020-11-20T12:16:00.144929+0000","last_unstale":"2020-11-20T12:16:00.633327+0000","last_undegraded":"2020-11-20T12:16:00.633327+0000","last_fullsized":"2020-11-20T12:16:00.633327+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.640639+0000","last_change":"2020-11-20T12:16:00.638051+0000","last_active":"2020-11-20T12:16:00.640639+0000","last_peered":"2020-11-20T12:16:00.640639+0000","last_clean":"2020-11-20T12:16:00.640639+0000","last_became_active":"2020-11-20T12:16:00.637685+0000","last_became_peered":"2020-11-20T12:16:00.637685+0000","last_unstale":"2020-11-20T12:16:00.640639+0000","last_undegraded":"2020-11-20T12:16:00.640639+0000","last_fullsized":"2020-11-20T12:16:00.640639+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.655784+0000","last_change":"2020-11-20T12:15:57.093800+0000","last_active":"2020-11-20T12:16:00.655784+0000","last_peered":"2020-11-20T12:16:00.655784+0000","last_clean":"2020-11-20T12:16:00.655784+0000","last_became_active":"2020-11-20T12:15:57.093230+0000","last_became_peered":"2020-11-20T12:15:57.093230+0000","last_unstale":"2020-11-20T12:16:00.655784+0000","last_undegraded":"2020-11-20T12:16:00.655784+0000","last_fullsized":"2020-11-20T12:16:00.655784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"22'8","reported_seq":"18","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.660253+0000","last_change":"2020-11-20T12:16:00.150292+0000","last_active":"2020-11-20T12:16:00.660253+0000","last_peered":"2020-11-20T12:16:00.660253+0000","last_clean":"2020-11-20T12:16:00.660253+0000","last_became_active":"2020-11-20T12:16:00.149758+0000","last_became_peered":"2020-11-20T12:16:00.149758+0000","last_unstale":"2020-11-20T12:16:00.660253+0000","last_undegraded":"2020-11-20T12:16:00.660253+0000","last_fullsized":"2020-11-20T12:16:00.660253+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":1535,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.660074+0000","last_change":"2020-11-20T12:16:00.658553+0000","last_active":"2020-11-20T12:16:00.660074+0000","last_peered":"2020-11-20T12:16:00.660074+0000","last_clean":"2020-11-20T12:16:00.660074+0000","last_became_active":"2020-11-20T12:16:00.658218+0000","last_became_peered":"2020-11-20T12:16:00.658218+0000","last_unstale":"2020-11-20T12:16:00.660074+0000","last_undegraded":"2020-11-20T12:16:00.660074+0000","last_fullsized":"2020-11-20T12:16:00.660074+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"22'6","reported_seq":"16","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.656872+0000","last_change":"2020-11-20T12:16:00.153258+0000","last_active":"2020-11-20T12:16:00.656872+0000","last_peered":"2020-11-20T12:16:00.656872+0000","last_clean":"2020-11-20T12:16:00.656872+0000","last_became_active":"2020-11-20T12:16:00.152908+0000","last_became_peered":"2020-11-20T12:16:00.152908+0000","last_unstale":"2020-11-20T12:16:00.656872+0000","last_undegraded":"2020-11-20T12:16:00.656872+0000","last_fullsized":"2020-11-20T12:16:00.656872+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":6,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":575,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.632090+0000","last_change":"2020-11-20T12:15:57.006042+0000","last_active":"2020-11-20T12:16:00.632090+0000","last_peered":"2020-11-20T12:16:00.632090+0000","last_clean":"2020-11-20T12:16:00.632090+0000","last_became_active":"2020-11-20T12:15:57.005582+0000","last_became_peered":"2020-11-20T12:15:57.005582+0000","last_unstale":"2020-11-20T12:16:00.632090+0000","last_undegraded":"2020-11-20T12:16:00.632090+0000","last_fullsized":"2020-11-20T12:16:00.632090+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":"9","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.640173+0000","last_change":"2020-11-20T12:16:00.637895+0000","last_active":"2020-11-20T12:16:00.640173+0000","last_peered":"2020-11-20T12:16:00.640173+0000","last_clean":"2020-11-20T12:16:00.640173+0000","last_became_active":"2020-11-20T12:16:00.637292+0000","last_became_peered":"2020-11-20T12:16:00.637292+0000","last_unstale":"2020-11-20T12:16:00.640173+0000","last_undegraded":"2020-11-20T12:16:00.640173+0000","last_fullsized":"2020-11-20T12:16:00.640173+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:16:00.121206+0000","last_clean_scrub_stamp":"2020-11-20T12:16:00.121206+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":"13","reported_epoch":"22","state":"active+clean","last_fresh":"2020-11-20T12:16:00.635165+0000","last_change":"2020-11-20T12:15:57.006654+0000","last_active":"2020-11-20T12:16:00.635165+0000","last_peered":"2020-11-20T12:16:00.635165+0000","last_clean":"2020-11-20T12:16:00.635165+0000","last_became_active":"2020-11-20T12:15:57.006200+0000","last_became_peered":"2020-11-20T12:15:57.006200+0000","last_unstale":"2020-11-20T12:16:00.635165+0000","last_undegraded":"2020-11-20T12:16:00.635165+0000","last_fullsized":"2020-11-20T12:16:00.635165+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:55.966228+0000","last_clean_scrub_stamp":"2020-11-20T12:15:55.966228+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'4","reported_seq":"14","reported_epoch":"23","state":"active+clean","last_fresh":"2020-11-20T12:16:01.718956+0000","last_change":"2020-11-20T12:16:00.145290+0000","last_active":"2020-11-20T12:16:01.718956+0000","last_peered":"2020-11-20T12:16:01.718956+0000","last_clean":"2020-11-20T12:16:01.718956+0000","last_became_active":"2020-11-20T12:16:00.144853+0000","last_became_peered":"2020-11-20T12:16:00.144853+0000","last_unstale":"2020-11-20T12:16:01.718956+0000","last_undegraded":"2020-11-20T12:16:01.718956+0000","last_fullsized":"2020-11-20T12:16:01.718956+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2020-11-20T12:15:59.115587+0000","last_clean_scrub_stamp":"2020-11-20T12:15:59.115587+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":3,"num_pg":8,"stat_sum":{"num_bytes":2304,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542150,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":88834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.5430000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8220000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1789999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7649999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.512}]}]},{"osd":6,"up_from":14,"seq":60129542150,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5040,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":94366800,"statfs":{"total":96636764160,"available":96631603200,"internally_reserved":0,"allocated":237568,"data_stored":88259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.716}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3090000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.76}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8889999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.222}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.956}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.552}]}]},{"osd":1,"up_from":14,"seq":60129542150,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":89191,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.647}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.744}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1240000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4359999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8660000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3530000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1299999999999999}]}]},{"osd":0,"up_from":14,"seq":60129542150,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5104,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366736,"statfs":{"total":96636764160,"available":96631537664,"internally_reserved":0,"allocated":237568,"data_stored":87690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.708}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.853}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.9470000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0859999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.0329999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1420000000000003}]}]},{"osd":2,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5116,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366724,"statfs":{"total":96636764160,"available":96631525376,"internally_reserved":0,"allocated":249856,"data_stored":89794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4419999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7609999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.927}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1030000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.234}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5150000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8769999999999998}]}]},{"osd":3,"up_from":14,"seq":60129542150,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5120,"kb_used_data":248,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366720,"statfs":{"total":96636764160,"available":96631521280,"internally_reserved":0,"allocated":253952,"data_stored":88994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.319}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.615}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3740000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1629999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1710000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7370000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0140000000000002}]}]},{"osd":4,"up_from":14,"seq":60129542151,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5108,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366732,"statfs":{"total":96636764160,"available":96631533568,"internally_reserved":0,"allocated":241664,"data_stored":88293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.044}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1859999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2240000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7470000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.298}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4079999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3319999999999999}]}]},{"osd":5,"up_from":14,"seq":60129542150,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":5112,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":94366728,"statfs":{"total":96636764160,"available":96631529472,"internally_reserved":0,"allocated":245760,"data_stored":88419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.097}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2549999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.929}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.052}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4039999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6030000000000002}]}]}]}} 2020-11-20T12:16:12.624 INFO:tasks.ceph.ceph_manager.ceph:clean! 2020-11-20T12:16:12.625 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2020-11-20T12:16:12.626 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2020-11-20T12:16:12.629 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2020-11-20T12:16:12.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.808+0000 7fd41cf3c700 1 Processor -- start 2020-11-20T12:16:12.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.808+0000 7fd41cf3c700 1 -- start start 2020-11-20T12:16:12.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.808+0000 7fd41cf3c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd41814f440 0x7fd4180a9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd41cf3c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd418063200 0x7fd418063690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd41cf3c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd418068160 con 0x7fd4180a9500 2020-11-20T12:16:12.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd41cf3c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd418063b80 con 0x7fd41814f440 2020-11-20T12:16:12.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd41cf3c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd418063d80 con 0x7fd418063200 2020-11-20T12:16:12.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd417fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd418063200 0x7fd418063690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd4177fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd41814f440 0x7fd4180a9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd416ffd700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fd4180a9500 0x7fd4180a6e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49016/0 (socket says 172.21.15.116:49016) 2020-11-20T12:16:12.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd416ffd700 1 -- 172.21.15.116:0/2371935116 learned_addr learned my addr 172.21.15.116:0/2371935116 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:12.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd417fff700 1 -- 172.21.15.116:0/2371935116 >> v2:172.21.15.173:3300/0 conn(0x7fd41814f440 msgr2=0x7fd4180a9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:12.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd417fff700 1 --2- 172.21.15.116:0/2371935116 >> v2:172.21.15.173:3300/0 conn(0x7fd41814f440 0x7fd4180a9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd417fff700 1 -- 172.21.15.116:0/2371935116 >> v1:172.21.15.173:6789/0 conn(0x7fd4180a9500 legacy=0x7fd4180a6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd417fff700 1 -- 172.21.15.116:0/2371935116 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4180642c0 con 0x7fd418063200 2020-11-20T12:16:12.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd4167fc700 1 -- 172.21.15.116:0/2371935116 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409098260 0 0) 0x7fd418068160 con 0x7fd4180a9500 2020-11-20T12:16:12.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd417fff700 1 --2- 172.21.15.116:0/2371935116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd418063200 0x7fd418063690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd410009c90 tx=0x7fd4100091d0).ready entity=mon.2 client_cookie=2c0e7c1ca5b3263a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:12.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd4167fc700 1 -- 172.21.15.116:0/2371935116 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41000ba70 con 0x7fd418063200 2020-11-20T12:16:12.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd4167fc700 1 -- 172.21.15.116:0/2371935116 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41000bbd0 con 0x7fd418063200 2020-11-20T12:16:12.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.812+0000 7fd4167fc700 1 -- 172.21.15.116:0/2371935116 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd410011400 con 0x7fd418063200 2020-11-20T12:16:12.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.816+0000 7fd41cf3c700 1 -- 172.21.15.116:0/2371935116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd418063200 msgr2=0x7fd418063690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.816+0000 7fd41cf3c700 1 --2- 172.21.15.116:0/2371935116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd418063200 0x7fd418063690 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd410009c90 tx=0x7fd4100091d0).stop 2020-11-20T12:16:12.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.816+0000 7fd41cf3c700 1 -- 172.21.15.116:0/2371935116 shutdown_connections 2020-11-20T12:16:12.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.816+0000 7fd41cf3c700 1 --2- 172.21.15.116:0/2371935116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd418063200 0x7fd418063690 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.816+0000 7fd41cf3c700 1 --2- 172.21.15.116:0/2371935116 >> v2:172.21.15.173:3300/0 conn(0x7fd41814f440 0x7fd4180a9010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.816+0000 7fd41cf3c700 1 -- 172.21.15.116:0/2371935116 >> 172.21.15.116:0/2371935116 conn(0x7fd4180c50b0 msgr2=0x7fd4180c54c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:12.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.816+0000 7fd41cf3c700 1 -- 172.21.15.116:0/2371935116 shutdown_connections 2020-11-20T12:16:12.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.816+0000 7fd41cf3c700 1 -- 172.21.15.116:0/2371935116 wait complete. 2020-11-20T12:16:12.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd41cf3c700 1 Processor -- start 2020-11-20T12:16:12.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd41cf3c700 1 -- start start 2020-11-20T12:16:12.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd41cf3c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd418063200 0x7fd4181696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd41cf3c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4180a9500 0x7fd418149290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd41cf3c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd41814f440 0x7fd4180a9a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd41cf3c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418169bd0 con 0x7fd4180a9500 2020-11-20T12:16:12.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd41cf3c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd418165b30 con 0x7fd418063200 2020-11-20T12:16:12.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd41cf3c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd418165dd0 con 0x7fd41814f440 2020-11-20T12:16:12.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd416ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4180a9500 0x7fd418149290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd4177fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd418063200 0x7fd4181696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd416ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4180a9500 0x7fd418149290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43358/0 (socket says 172.21.15.116:43358) 2020-11-20T12:16:12.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd416ffd700 1 -- 172.21.15.116:0/37068972 learned_addr learned my addr 172.21.15.116:0/37068972 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:12.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.820+0000 7fd417fff700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd41814f440 0x7fd4180a9a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd416ffd700 1 -- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd41814f440 msgr2=0x7fd4180a9a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd416ffd700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd41814f440 0x7fd4180a9a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd416ffd700 1 -- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd418063200 msgr2=0x7fd4181696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:12.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd416ffd700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd418063200 0x7fd4181696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:12.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd416ffd700 1 -- 172.21.15.116:0/37068972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd418166070 con 0x7fd4180a9500 2020-11-20T12:16:12.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd4177fe700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd418063200 0x7fd4181696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:12.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd416ffd700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4180a9500 0x7fd418149290 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fd40c00c720 tx=0x7fd40c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:12.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd3f77fe700 1 -- 172.21.15.116:0/37068972 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c0072e0 con 0x7fd4180a9500 2020-11-20T12:16:12.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd3f77fe700 1 -- 172.21.15.116:0/37068972 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40c007440 con 0x7fd4180a9500 2020-11-20T12:16:12.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd41cf3c700 1 -- 172.21.15.116:0/37068972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd418166310 con 0x7fd4180a9500 2020-11-20T12:16:12.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd3f77fe700 1 -- 172.21.15.116:0/37068972 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40c005800 con 0x7fd4180a9500 2020-11-20T12:16:12.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd41cf3c700 1 -- 172.21.15.116:0/37068972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd418149780 con 0x7fd4180a9500 2020-11-20T12:16:12.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd41cf3c700 1 -- 172.21.15.116:0/37068972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd418044f60 con 0x7fd4180a9500 2020-11-20T12:16:12.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.824+0000 7fd3f77fe700 1 -- 172.21.15.116:0/37068972 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd40c01a060 con 0x7fd4180a9500 2020-11-20T12:16:12.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.828+0000 7fd3f77fe700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3f00557c0 0x7fd3f0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:12.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.828+0000 7fd3f77fe700 1 -- 172.21.15.116:0/37068972 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd40c06de40 con 0x7fd4180a9500 2020-11-20T12:16:12.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.828+0000 7fd4177fe700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3f00557c0 0x7fd3f0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:12.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.828+0000 7fd4177fe700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3f00557c0 0x7fd3f0057c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd40800c150 tx=0x7fd40800f040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:12.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:12.832+0000 7fd3f77fe700 1 -- 172.21.15.116:0/37068972 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd40c043d90 con 0x7fd4180a9500 2020-11-20T12:16:13.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.044+0000 7fd41cf3c700 1 -- 172.21.15.116:0/37068972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fd418067710 con 0x7fd4180a9500 2020-11-20T12:16:13.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.044+0000 7fd3f77fe700 1 -- 172.21.15.116:0/37068972 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7fd40c018070 con 0x7fd4180a9500 2020-11-20T12:16:13.046 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:13.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.044+0000 7fd41cf3c700 1 -- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3f00557c0 msgr2=0x7fd3f0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.044+0000 7fd41cf3c700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3f00557c0 0x7fd3f0057c80 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd40800c150 tx=0x7fd40800f040).stop 2020-11-20T12:16:13.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.044+0000 7fd41cf3c700 1 -- 172.21.15.116:0/37068972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4180a9500 msgr2=0x7fd418149290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.044+0000 7fd41cf3c700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4180a9500 0x7fd418149290 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fd40c00c720 tx=0x7fd40c007ac0).stop 2020-11-20T12:16:13.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.048+0000 7fd41cf3c700 1 -- 172.21.15.116:0/37068972 shutdown_connections 2020-11-20T12:16:13.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.048+0000 7fd41cf3c700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3f00557c0 0x7fd3f0057c80 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.048+0000 7fd41cf3c700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd41814f440 0x7fd4180a9a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.048+0000 7fd41cf3c700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd418063200 0x7fd4181696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.048+0000 7fd41cf3c700 1 --2- 172.21.15.116:0/37068972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4180a9500 0x7fd418149290 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.048+0000 7fd41cf3c700 1 -- 172.21.15.116:0/37068972 >> 172.21.15.116:0/37068972 conn(0x7fd4180c50b0 msgr2=0x7fd41814b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:13.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.048+0000 7fd41cf3c700 1 -- 172.21.15.116:0/37068972 shutdown_connections 2020-11-20T12:16:13.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.048+0000 7fd41cf3c700 1 -- 172.21.15.116:0/37068972 wait complete. 2020-11-20T12:16:13.069 INFO:teuthology.orchestra.run.smithi116.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2020-11-20T12:16:13.070 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2020-11-20T12:16:13.070 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:13.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.240+0000 7f759370b700 1 Processor -- start 2020-11-20T12:16:13.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.244+0000 7f759370b700 1 -- start start 2020-11-20T12:16:13.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.244+0000 7f759370b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f758c1518e0 0x7f758c0a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.244+0000 7f759370b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f758c151cc0 0x7f758c152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.244+0000 7f759370b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f758c14d6d0 0x7f758c14db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.244+0000 7f759370b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c0a39c0 con 0x7f758c151cc0 2020-11-20T12:16:13.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.244+0000 7f759370b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f758c152630 con 0x7f758c14d6d0 2020-11-20T12:16:13.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.244+0000 7f759370b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c14e050 con 0x7f758c1518e0 2020-11-20T12:16:13.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f7590ca6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f758c151cc0 0x7f758c152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f7590ca6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f758c151cc0 0x7f758c152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43366/0 (socket says 172.21.15.116:43366) 2020-11-20T12:16:13.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f7590ca6700 1 -- 172.21.15.116:0/3917754622 learned_addr learned my addr 172.21.15.116:0/3917754622 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:13.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f7591ca8700 1 --2- 172.21.15.116:0/3917754622 >> v2:172.21.15.173:3300/0 conn(0x7f758c14d6d0 0x7f758c14db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f75914a7700 1 --2- 172.21.15.116:0/3917754622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f758c1518e0 0x7f758c0a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f7590ca6700 1 -- 172.21.15.116:0/3917754622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f758c1518e0 msgr2=0x7f758c0a34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:13.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f7590ca6700 1 --2- 172.21.15.116:0/3917754622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f758c1518e0 0x7f758c0a34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f7590ca6700 1 -- 172.21.15.116:0/3917754622 >> v2:172.21.15.173:3300/0 conn(0x7f758c14d6d0 msgr2=0x7f758c14db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:13.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f7590ca6700 1 --2- 172.21.15.116:0/3917754622 >> v2:172.21.15.173:3300/0 conn(0x7f758c14d6d0 0x7f758c14db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f7590ca6700 1 -- 172.21.15.116:0/3917754622 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0bf360 con 0x7f758c151cc0 2020-11-20T12:16:13.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f7590ca6700 1 --2- 172.21.15.116:0/3917754622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f758c151cc0 0x7f758c152140 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f758c0c4090 tx=0x7f7578006eb0).ready entity=mon.0 client_cookie=1e838b372244057f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:13.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f758bfff700 1 -- 172.21.15.116:0/3917754622 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75780098d0 con 0x7f758c151cc0 2020-11-20T12:16:13.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f758bfff700 1 -- 172.21.15.116:0/3917754622 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7578009a30 con 0x7f758c151cc0 2020-11-20T12:16:13.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f758bfff700 1 -- 172.21.15.116:0/3917754622 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7578009db0 con 0x7f758c151cc0 2020-11-20T12:16:13.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f759370b700 1 -- 172.21.15.116:0/3917754622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f758c151cc0 msgr2=0x7f758c152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f759370b700 1 --2- 172.21.15.116:0/3917754622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f758c151cc0 0x7f758c152140 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f758c0c4090 tx=0x7f7578006eb0).stop 2020-11-20T12:16:13.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f759370b700 1 -- 172.21.15.116:0/3917754622 shutdown_connections 2020-11-20T12:16:13.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f759370b700 1 --2- 172.21.15.116:0/3917754622 >> v2:172.21.15.173:3300/0 conn(0x7f758c14d6d0 0x7f758c14db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f759370b700 1 --2- 172.21.15.116:0/3917754622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f758c1518e0 0x7f758c0a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f759370b700 1 --2- 172.21.15.116:0/3917754622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f758c151cc0 0x7f758c152140 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f759370b700 1 -- 172.21.15.116:0/3917754622 >> 172.21.15.116:0/3917754622 conn(0x7f758c060c80 msgr2=0x7f758c0c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:13.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f759370b700 1 -- 172.21.15.116:0/3917754622 shutdown_connections 2020-11-20T12:16:13.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.248+0000 7f759370b700 1 -- 172.21.15.116:0/3917754622 wait complete. 2020-11-20T12:16:13.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f759370b700 1 Processor -- start 2020-11-20T12:16:13.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f759370b700 1 -- start start 2020-11-20T12:16:13.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f759370b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f758c14d6d0 0x7f758c08ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f759370b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f758c1518e0 0x7f758c1481a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f759370b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f758c151cc0 0x7f758c0c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f759370b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c08f220 con 0x7f758c151cc0 2020-11-20T12:16:13.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f759370b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f758c08f4c0 con 0x7f758c14d6d0 2020-11-20T12:16:13.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f759370b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f758c08f760 con 0x7f758c1518e0 2020-11-20T12:16:13.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f7591ca8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f758c151cc0 0x7f758c0c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f7591ca8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f758c151cc0 0x7f758c0c0810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43374/0 (socket says 172.21.15.116:43374) 2020-11-20T12:16:13.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f7591ca8700 1 -- 172.21.15.116:0/2658933164 learned_addr learned my addr 172.21.15.116:0/2658933164 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:13.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f75914a7700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f758c14d6d0 0x7f758c08ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.252+0000 7f7590ca6700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f758c1518e0 0x7f758c1481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f7591ca8700 1 -- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f758c1518e0 msgr2=0x7f758c1481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f7591ca8700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f758c1518e0 0x7f758c1481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f7591ca8700 1 -- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f758c14d6d0 msgr2=0x7f758c08ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f7591ca8700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f758c14d6d0 0x7f758c08ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f7591ca8700 1 -- 172.21.15.116:0/2658933164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c08fa00 con 0x7f758c151cc0 2020-11-20T12:16:13.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f7591ca8700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f758c151cc0 0x7f758c0c0810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f757c00eb70 tx=0x7f757c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:13.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f75897fa700 1 -- 172.21.15.116:0/2658933164 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c00c670 con 0x7f758c151cc0 2020-11-20T12:16:13.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f75897fa700 1 -- 172.21.15.116:0/2658933164 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757c00c7d0 con 0x7f758c151cc0 2020-11-20T12:16:13.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f75897fa700 1 -- 172.21.15.116:0/2658933164 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757c00cb70 con 0x7f758c151cc0 2020-11-20T12:16:13.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f759370b700 1 -- 172.21.15.116:0/2658933164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f758c08fca0 con 0x7f758c151cc0 2020-11-20T12:16:13.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f759370b700 1 -- 172.21.15.116:0/2658933164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f758c148690 con 0x7f758c151cc0 2020-11-20T12:16:13.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f759370b700 1 -- 172.21.15.116:0/2658933164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f758c05ca30 con 0x7f758c151cc0 2020-11-20T12:16:13.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.256+0000 7f75897fa700 1 -- 172.21.15.116:0/2658933164 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f757c01d460 con 0x7f758c151cc0 2020-11-20T12:16:13.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.260+0000 7f75897fa700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75600557c0 0x7f7560057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.260+0000 7f75897fa700 1 -- 172.21.15.116:0/2658933164 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f757c06e660 con 0x7f758c151cc0 2020-11-20T12:16:13.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.260+0000 7f75914a7700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75600557c0 0x7f7560057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.260+0000 7f75914a7700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75600557c0 0x7f7560057c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f758400ba00 tx=0x7f7584008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:13.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.264+0000 7f75897fa700 1 -- 172.21.15.116:0/2658933164 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f757c0444a0 con 0x7f758c151cc0 2020-11-20T12:16:13.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.424+0000 7f759370b700 1 -- 172.21.15.116:0/2658933164 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f758c0616a0 con 0x7f758c151cc0 2020-11-20T12:16:13.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f75897fa700 1 -- 172.21.15.116:0/2658933164 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f757c047ac0 con 0x7f758c151cc0 2020-11-20T12:16:13.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 -- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75600557c0 msgr2=0x7f7560057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75600557c0 0x7f7560057c80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f758400ba00 tx=0x7f7584008040).stop 2020-11-20T12:16:13.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 -- 172.21.15.116:0/2658933164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f758c151cc0 msgr2=0x7f758c0c0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f758c151cc0 0x7f758c0c0810 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f757c00eb70 tx=0x7f757c008040).stop 2020-11-20T12:16:13.432 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:13.432 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:13.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 -- 172.21.15.116:0/2658933164 shutdown_connections 2020-11-20T12:16:13.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f75600557c0 0x7f7560057c80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f758c151cc0 0x7f758c0c0810 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f758c14d6d0 0x7f758c08ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 --2- 172.21.15.116:0/2658933164 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f758c1518e0 0x7f758c1481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 -- 172.21.15.116:0/2658933164 >> 172.21.15.116:0/2658933164 conn(0x7f758c060c80 msgr2=0x7f758c14ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:13.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 -- 172.21.15.116:0/2658933164 shutdown_connections 2020-11-20T12:16:13.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.428+0000 7f759370b700 1 -- 172.21.15.116:0/2658933164 wait complete. 2020-11-20T12:16:13.456 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:15:59.065285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:15:59.602311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:16:13.457 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:13.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.640+0000 7f5b0e73e700 1 Processor -- start 2020-11-20T12:16:13.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.644+0000 7f5b0e73e700 1 -- start start 2020-11-20T12:16:13.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.644+0000 7f5b0e73e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5b080a6330 0x7f5b080a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0e73e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5b080a2db0 0x7f5b081550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0e73e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b080a3400 con 0x7f5b080a5f50 2020-11-20T12:16:13.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0e73e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b080a36a0 con 0x7f5b080a6330 2020-11-20T12:16:13.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0e73e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b08066130 con 0x7f5b080a2db0 2020-11-20T12:16:13.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0d73c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5b080a5f50 0x7f5b080c1bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52812/0 (socket says 172.21.15.116:52812) 2020-11-20T12:16:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0d73c700 1 -- 172.21.15.116:0/423631796 learned_addr learned my addr 172.21.15.116:0/423631796 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0cf3b700 1 --2- 172.21.15.116:0/423631796 >> v2:172.21.15.116:3300/0 conn(0x7f5b080a6330 0x7f5b080a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0df3d700 1 --2- 172.21.15.116:0/423631796 >> v2:172.21.15.173:3300/0 conn(0x7f5b080a2db0 0x7f5b081550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5af7fff700 1 -- 172.21.15.116:0/423631796 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874962301 0 0) 0x7f5b080a3400 con 0x7f5b080a5f50 2020-11-20T12:16:13.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5af7fff700 1 -- 172.21.15.116:0/423631796 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5af0003240 con 0x7f5b080a5f50 2020-11-20T12:16:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0cf3b700 1 -- 172.21.15.116:0/423631796 >> v2:172.21.15.173:3300/0 conn(0x7f5b080a2db0 msgr2=0x7f5b081550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0cf3b700 1 --2- 172.21.15.116:0/423631796 >> v2:172.21.15.173:3300/0 conn(0x7f5b080a2db0 0x7f5b081550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0cf3b700 1 -- 172.21.15.116:0/423631796 >> v1:172.21.15.116:6789/0 conn(0x7f5b080a5f50 legacy=0x7f5b080c1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0cf3b700 1 -- 172.21.15.116:0/423631796 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b080663d0 con 0x7f5b080a6330 2020-11-20T12:16:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5b0cf3b700 1 --2- 172.21.15.116:0/423631796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b080a6330 0x7f5b080a28c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f5b08065bc0 tx=0x7f5af8006df0).ready entity=mon.0 client_cookie=290c67fc36ccb03f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5af7fff700 1 -- 172.21.15.116:0/423631796 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af80096d0 con 0x7f5b080a6330 2020-11-20T12:16:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5af7fff700 1 -- 172.21.15.116:0/423631796 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af8009830 con 0x7f5b080a6330 2020-11-20T12:16:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.648+0000 7f5af7fff700 1 -- 172.21.15.116:0/423631796 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af8009bb0 con 0x7f5b080a6330 2020-11-20T12:16:13.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.652+0000 7f5b0e73e700 1 -- 172.21.15.116:0/423631796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b080a6330 msgr2=0x7f5b080a28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.652+0000 7f5b0e73e700 1 --2- 172.21.15.116:0/423631796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b080a6330 0x7f5b080a28c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f5b08065bc0 tx=0x7f5af8006df0).stop 2020-11-20T12:16:13.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.652+0000 7f5b0e73e700 1 -- 172.21.15.116:0/423631796 shutdown_connections 2020-11-20T12:16:13.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.652+0000 7f5b0e73e700 1 --2- 172.21.15.116:0/423631796 >> v2:172.21.15.173:3300/0 conn(0x7f5b080a2db0 0x7f5b081550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.652+0000 7f5b0e73e700 1 --2- 172.21.15.116:0/423631796 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b080a6330 0x7f5b080a28c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.652+0000 7f5b0e73e700 1 -- 172.21.15.116:0/423631796 >> 172.21.15.116:0/423631796 conn(0x7f5b08060c30 msgr2=0x7f5b080c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:13.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.652+0000 7f5b0e73e700 1 -- 172.21.15.116:0/423631796 shutdown_connections 2020-11-20T12:16:13.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.652+0000 7f5b0e73e700 1 -- 172.21.15.116:0/423631796 wait complete. 2020-11-20T12:16:13.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.652+0000 7f5b0e73e700 1 Processor -- start 2020-11-20T12:16:13.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.652+0000 7f5b0e73e700 1 -- start start 2020-11-20T12:16:13.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0e73e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b080a2db0 0x7f5b08161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0e73e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b080a5f50 0x7f5b08160710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0e73e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b080a6330 0x7f5b08066790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0e73e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b08164890 con 0x7f5b080a5f50 2020-11-20T12:16:13.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0e73e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b08164b30 con 0x7f5b080a6330 2020-11-20T12:16:13.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0e73e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b08164dd0 con 0x7f5b080a2db0 2020-11-20T12:16:13.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0cf3b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b080a5f50 0x7f5b08160710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0cf3b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b080a5f50 0x7f5b08160710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43388/0 (socket says 172.21.15.116:43388) 2020-11-20T12:16:13.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0cf3b700 1 -- 172.21.15.116:0/3669838142 learned_addr learned my addr 172.21.15.116:0/3669838142 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:13.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0df3d700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b080a6330 0x7f5b08066790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0d73c700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b080a2db0 0x7f5b08161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0cf3b700 1 -- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b080a2db0 msgr2=0x7f5b08161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0cf3b700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b080a2db0 0x7f5b08161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0cf3b700 1 -- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b080a6330 msgr2=0x7f5b08066790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0cf3b700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b080a6330 0x7f5b08066790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0cf3b700 1 -- 172.21.15.116:0/3669838142 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b08165070 con 0x7f5b080a5f50 2020-11-20T12:16:13.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0cf3b700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b080a5f50 0x7f5b08160710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f5b080c1ac0 tx=0x7f5af8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:13.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5af57fa700 1 -- 172.21.15.116:0/3669838142 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af801e460 con 0x7f5b080a5f50 2020-11-20T12:16:13.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0e73e700 1 -- 172.21.15.116:0/3669838142 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b08165310 con 0x7f5b080a5f50 2020-11-20T12:16:13.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5af57fa700 1 -- 172.21.15.116:0/3669838142 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af801e5c0 con 0x7f5b080a5f50 2020-11-20T12:16:13.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5af57fa700 1 -- 172.21.15.116:0/3669838142 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af801e940 con 0x7f5b080a5f50 2020-11-20T12:16:13.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.656+0000 7f5b0e73e700 1 -- 172.21.15.116:0/3669838142 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b08150600 con 0x7f5b080a5f50 2020-11-20T12:16:13.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.660+0000 7f5b0e73e700 1 -- 172.21.15.116:0/3669838142 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b08044f60 con 0x7f5b080a5f50 2020-11-20T12:16:13.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.660+0000 7f5af57fa700 1 -- 172.21.15.116:0/3669838142 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5af8025030 con 0x7f5b080a5f50 2020-11-20T12:16:13.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.660+0000 7f5af57fa700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ae0055840 0x7f5ae0057d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:13.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.660+0000 7f5af57fa700 1 -- 172.21.15.116:0/3669838142 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5af806ff90 con 0x7f5b080a5f50 2020-11-20T12:16:13.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.660+0000 7f5b0d73c700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ae0055840 0x7f5ae0057d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:13.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.664+0000 7f5b0d73c700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ae0055840 0x7f5ae0057d00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5b040055a0 tx=0x7f5b04009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:13.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.664+0000 7f5af57fa700 1 -- 172.21.15.116:0/3669838142 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5af8045e50 con 0x7f5b080a5f50 2020-11-20T12:16:13.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.824+0000 7f5b0e73e700 1 -- 172.21.15.116:0/3669838142 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5b08064080 con 0x7f5b080a5f50 2020-11-20T12:16:13.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.824+0000 7f5af57fa700 1 -- 172.21.15.116:0/3669838142 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5af8023020 con 0x7f5b080a5f50 2020-11-20T12:16:13.827 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:13.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.828+0000 7f5b0e73e700 1 -- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ae0055840 msgr2=0x7f5ae0057d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.828+0000 7f5b0e73e700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ae0055840 0x7f5ae0057d00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5b040055a0 tx=0x7f5b04009040).stop 2020-11-20T12:16:13.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.828+0000 7f5b0e73e700 1 -- 172.21.15.116:0/3669838142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b080a5f50 msgr2=0x7f5b08160710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:13.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.828+0000 7f5b0e73e700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b080a5f50 0x7f5b08160710 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f5b080c1ac0 tx=0x7f5af8015000).stop 2020-11-20T12:16:13.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.828+0000 7f5b0e73e700 1 -- 172.21.15.116:0/3669838142 shutdown_connections 2020-11-20T12:16:13.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.828+0000 7f5b0e73e700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ae0055840 0x7f5ae0057d00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.828+0000 7f5b0e73e700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b080a6330 0x7f5b08066790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.828+0000 7f5b0e73e700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b080a2db0 0x7f5b08161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.828+0000 7f5b0e73e700 1 --2- 172.21.15.116:0/3669838142 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b080a5f50 0x7f5b08160710 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:13.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.828+0000 7f5b0e73e700 1 -- 172.21.15.116:0/3669838142 >> 172.21.15.116:0/3669838142 conn(0x7f5b08060c30 msgr2=0x7f5b080a0d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:13.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.832+0000 7f5b0e73e700 1 -- 172.21.15.116:0/3669838142 shutdown_connections 2020-11-20T12:16:13.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:13.832+0000 7f5b0e73e700 1 -- 172.21.15.116:0/3669838142 wait complete. 2020-11-20T12:16:13.856 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:14.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.040+0000 7f841ae96700 1 Processor -- start 2020-11-20T12:16:14.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.040+0000 7f841ae96700 1 -- start start 2020-11-20T12:16:14.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f841ae96700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f841414d020 0x7f84140c3a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f841ae96700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84140c2b30 con 0x7f84140c2690 2020-11-20T12:16:14.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f841ae96700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84140c2dd0 con 0x7f84141504f0 2020-11-20T12:16:14.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f841ae96700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84140c3070 con 0x7f841414d020 2020-11-20T12:16:14.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f8413fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f84140c2690 0x7f84140a6800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52828/0 (socket says 172.21.15.116:52828) 2020-11-20T12:16:14.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f8413fff700 1 -- 172.21.15.116:0/3132978450 learned_addr learned my addr 172.21.15.116:0/3132978450 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:14.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f8418c32700 1 --2- 172.21.15.116:0/3132978450 >> v2:172.21.15.116:3300/0 conn(0x7f841414d020 0x7f84140c3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f84137fe700 1 -- 172.21.15.116:0/3132978450 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2875916506 0 0) 0x7f84140c2b30 con 0x7f84140c2690 2020-11-20T12:16:14.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f84137fe700 1 -- 172.21.15.116:0/3132978450 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83f8003240 con 0x7f84140c2690 2020-11-20T12:16:14.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f8418c32700 1 -- 172.21.15.116:0/3132978450 >> v1:172.21.15.173:6789/0 conn(0x7f84141504f0 legacy=0x7f84140c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f8418c32700 1 -- 172.21.15.116:0/3132978450 >> v1:172.21.15.116:6789/0 conn(0x7f84140c2690 legacy=0x7f84140a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f84137fe700 1 -- 172.21.15.116:0/3132978450 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1954336807 0 0) 0x7f83f8003240 con 0x7f84140c2690 2020-11-20T12:16:14.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.044+0000 7f8418c32700 1 -- 172.21.15.116:0/3132978450 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84140c3310 con 0x7f841414d020 2020-11-20T12:16:14.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.048+0000 7f8418c32700 1 --2- 172.21.15.116:0/3132978450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f841414d020 0x7f84140c3a50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f840800b990 tx=0x7f8408006d90).ready entity=mon.0 client_cookie=73831cb7fb9a33eb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:14.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.048+0000 7f84137fe700 1 -- 172.21.15.116:0/3132978450 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84000014e0 con 0x7f841414d020 2020-11-20T12:16:14.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.048+0000 7f84137fe700 1 -- 172.21.15.116:0/3132978450 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84080094a0 con 0x7f841414d020 2020-11-20T12:16:14.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.048+0000 7f84137fe700 1 -- 172.21.15.116:0/3132978450 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800bee0 con 0x7f841414d020 2020-11-20T12:16:14.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.048+0000 7f841ae96700 1 -- 172.21.15.116:0/3132978450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f841414d020 msgr2=0x7f84140c3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.048+0000 7f841ae96700 1 --2- 172.21.15.116:0/3132978450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f841414d020 0x7f84140c3a50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f840800b990 tx=0x7f8408006d90).stop 2020-11-20T12:16:14.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.048+0000 7f841ae96700 1 -- 172.21.15.116:0/3132978450 shutdown_connections 2020-11-20T12:16:14.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.048+0000 7f841ae96700 1 --2- 172.21.15.116:0/3132978450 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f841414d020 0x7f84140c3a50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.048+0000 7f841ae96700 1 -- 172.21.15.116:0/3132978450 >> 172.21.15.116:0/3132978450 conn(0x7f841408f950 msgr2=0x7f8414060950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:14.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.048+0000 7f841ae96700 1 -- 172.21.15.116:0/3132978450 shutdown_connections 2020-11-20T12:16:14.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.048+0000 7f841ae96700 1 -- 172.21.15.116:0/3132978450 wait complete. 2020-11-20T12:16:14.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f841ae96700 1 Processor -- start 2020-11-20T12:16:14.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f841ae96700 1 -- start start 2020-11-20T12:16:14.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f841ae96700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84140c2690 0x7f8414061280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f841ae96700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84141504f0 0x7f84140abcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f841ae96700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8414165330 0x7f84140a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f841ae96700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8414061770 con 0x7f8414165330 2020-11-20T12:16:14.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f841ae96700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8414061a00 con 0x7f84141504f0 2020-11-20T12:16:14.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f841ae96700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8414061ca0 con 0x7f84140c2690 2020-11-20T12:16:14.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8419433700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8414165330 0x7f84140a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8419433700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8414165330 0x7f84140a93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43402/0 (socket says 172.21.15.116:43402) 2020-11-20T12:16:14.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8419433700 1 -- 172.21.15.116:0/3931370943 learned_addr learned my addr 172.21.15.116:0/3931370943 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:14.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8413fff700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84141504f0 0x7f84140abcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8418c32700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84140c2690 0x7f8414061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8419433700 1 -- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84140c2690 msgr2=0x7f8414061280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8419433700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84140c2690 0x7f8414061280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8419433700 1 -- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84141504f0 msgr2=0x7f84140abcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8419433700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84141504f0 0x7f84140abcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8419433700 1 -- 172.21.15.116:0/3931370943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8414061f40 con 0x7f8414165330 2020-11-20T12:16:14.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8419433700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8414165330 0x7f84140a93c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f840c00ca90 tx=0x7f840c001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:14.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.052+0000 7f8410ff9700 1 -- 172.21.15.116:0/3931370943 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00a730 con 0x7f8414165330 2020-11-20T12:16:14.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.056+0000 7f8410ff9700 1 -- 172.21.15.116:0/3931370943 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c00a890 con 0x7f8414165330 2020-11-20T12:16:14.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.056+0000 7f841ae96700 1 -- 172.21.15.116:0/3931370943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84140621e0 con 0x7f8414165330 2020-11-20T12:16:14.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.056+0000 7f8410ff9700 1 -- 172.21.15.116:0/3931370943 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00ac30 con 0x7f8414165330 2020-11-20T12:16:14.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.056+0000 7f841ae96700 1 -- 172.21.15.116:0/3931370943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84140ac1e0 con 0x7f8414165330 2020-11-20T12:16:14.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.056+0000 7f841ae96700 1 -- 172.21.15.116:0/3931370943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84140c72d0 con 0x7f8414165330 2020-11-20T12:16:14.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.056+0000 7f8410ff9700 1 -- 172.21.15.116:0/3931370943 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f840c019030 con 0x7f8414165330 2020-11-20T12:16:14.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.056+0000 7f8410ff9700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83ec0557c0 0x7f83ec057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.056+0000 7f8410ff9700 1 -- 172.21.15.116:0/3931370943 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f840c06dd60 con 0x7f8414165330 2020-11-20T12:16:14.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.056+0000 7f8418c32700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83ec0557c0 0x7f83ec057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.060+0000 7f8418c32700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83ec0557c0 0x7f83ec057c80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f841405c420 tx=0x7f8408015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:14.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.060+0000 7f8410ff9700 1 -- 172.21.15.116:0/3931370943 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f840c043cb0 con 0x7f8414165330 2020-11-20T12:16:14.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.220+0000 7f841ae96700 1 -- 172.21.15.116:0/3931370943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f841405ca30 con 0x7f8414165330 2020-11-20T12:16:14.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.220+0000 7f8410ff9700 1 -- 172.21.15.116:0/3931370943 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f840c0472d0 con 0x7f8414165330 2020-11-20T12:16:14.222 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:14.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 -- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83ec0557c0 msgr2=0x7f83ec057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83ec0557c0 0x7f83ec057c80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f841405c420 tx=0x7f8408015000).stop 2020-11-20T12:16:14.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 -- 172.21.15.116:0/3931370943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8414165330 msgr2=0x7f84140a93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8414165330 0x7f84140a93c0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f840c00ca90 tx=0x7f840c001360).stop 2020-11-20T12:16:14.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 -- 172.21.15.116:0/3931370943 shutdown_connections 2020-11-20T12:16:14.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83ec0557c0 0x7f83ec057c80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8414165330 0x7f84140a93c0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84140c2690 0x7f8414061280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 --2- 172.21.15.116:0/3931370943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84141504f0 0x7f84140abcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 -- 172.21.15.116:0/3931370943 >> 172.21.15.116:0/3931370943 conn(0x7f841408f950 msgr2=0x7f84141477a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:14.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 -- 172.21.15.116:0/3931370943 shutdown_connections 2020-11-20T12:16:14.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.224+0000 7f841ae96700 1 -- 172.21.15.116:0/3931370943 wait complete. 2020-11-20T12:16:14.249 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:16:14.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.428+0000 7fb2a64cc700 1 Processor -- start 2020-11-20T12:16:14.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.428+0000 7fb2a64cc700 1 -- start start 2020-11-20T12:16:14.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.432+0000 7fb2a64cc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a00c03f0 0x7fb2a0156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.432+0000 7fb2a64cc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb2a00c1bb0 0x7fb2a00677c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.432+0000 7fb2a64cc700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2a00a38c0 con 0x7fb2a00a28c0 2020-11-20T12:16:14.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.432+0000 7fb2a64cc700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2a0156e30 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.432+0000 7fb2a64cc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a00a2e40 con 0x7fb2a00c03f0 2020-11-20T12:16:14.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.432+0000 7fb2a4cc9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb2a00c1bb0 0x7fb2a00677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.432+0000 7fb2a4cc9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb2a00c1bb0 0x7fb2a00677c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43410/0 (socket says 172.21.15.116:43410) 2020-11-20T12:16:14.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.432+0000 7fb2a4cc9700 1 -- 172.21.15.116:0/1637660863 learned_addr learned my addr 172.21.15.116:0/1637660863 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:14.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.432+0000 7fb2a54ca700 1 --2- 172.21.15.116:0/1637660863 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a00c03f0 0x7fb2a0156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb28ffff700 1 -- 172.21.15.116:0/1637660863 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 554086397 0 0) 0x7fb2a00a38c0 con 0x7fb2a00a28c0 2020-11-20T12:16:14.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb28ffff700 1 -- 172.21.15.116:0/1637660863 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb288003240 con 0x7fb2a00a28c0 2020-11-20T12:16:14.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a4cc9700 1 -- 172.21.15.116:0/1637660863 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a00c03f0 msgr2=0x7fb2a0156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:14.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a4cc9700 1 --2- 172.21.15.116:0/1637660863 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a00c03f0 0x7fb2a0156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a4cc9700 1 -- 172.21.15.116:0/1637660863 >> v1:172.21.15.173:6789/0 conn(0x7fb2a00a28c0 legacy=0x7fb2a01594d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a4cc9700 1 -- 172.21.15.116:0/1637660863 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a00a30e0 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a4cc9700 1 --2- 172.21.15.116:0/1637660863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a00c1bb0 0x7fb2a00677c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fb290006fd0 tx=0x7fb290006eb0).ready entity=mon.0 client_cookie=ae16ad8b6d5f4c96 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:14.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb28ffff700 1 -- 172.21.15.116:0/1637660863 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2900097b0 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb28ffff700 1 -- 172.21.15.116:0/1637660863 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb290009910 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb28ffff700 1 -- 172.21.15.116:0/1637660863 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb290009c90 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1637660863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a00c1bb0 msgr2=0x7fb2a00677c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a64cc700 1 --2- 172.21.15.116:0/1637660863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a00c1bb0 0x7fb2a00677c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fb290006fd0 tx=0x7fb290006eb0).stop 2020-11-20T12:16:14.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1637660863 shutdown_connections 2020-11-20T12:16:14.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a64cc700 1 --2- 172.21.15.116:0/1637660863 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a00c03f0 0x7fb2a0156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a64cc700 1 --2- 172.21.15.116:0/1637660863 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a00c1bb0 0x7fb2a00677c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1637660863 >> 172.21.15.116:0/1637660863 conn(0x7fb2a0060c30 msgr2=0x7fb2a00c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:14.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1637660863 shutdown_connections 2020-11-20T12:16:14.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.436+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1637660863 wait complete. 2020-11-20T12:16:14.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.440+0000 7fb2a64cc700 1 Processor -- start 2020-11-20T12:16:14.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.440+0000 7fb2a64cc700 1 -- start start 2020-11-20T12:16:14.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.440+0000 7fb2a64cc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a00a28c0 0x7fb2a008f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.440+0000 7fb2a64cc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a00c03f0 0x7fb2a0164c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.440+0000 7fb2a64cc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a00c1bb0 0x7fb2a015b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.440+0000 7fb2a64cc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a008fe70 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.440+0000 7fb2a64cc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a0165630 con 0x7fb2a00c03f0 2020-11-20T12:16:14.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.440+0000 7fb2a64cc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a01658d0 con 0x7fb2a00a28c0 2020-11-20T12:16:14.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a5ccb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a00c1bb0 0x7fb2a015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a5ccb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a00c1bb0 0x7fb2a015b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43416/0 (socket says 172.21.15.116:43416) 2020-11-20T12:16:14.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a5ccb700 1 -- 172.21.15.116:0/1873818752 learned_addr learned my addr 172.21.15.116:0/1873818752 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:14.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a54ca700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a00a28c0 0x7fb2a008f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a4cc9700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a00c03f0 0x7fb2a0164c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a5ccb700 1 -- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a00a28c0 msgr2=0x7fb2a008f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a5ccb700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a00a28c0 0x7fb2a008f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a5ccb700 1 -- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a00c03f0 msgr2=0x7fb2a0164c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a5ccb700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a00c03f0 0x7fb2a0164c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a5ccb700 1 -- 172.21.15.116:0/1873818752 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a0165b70 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a5ccb700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a00c1bb0 0x7fb2a015b600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fb29800ca90 tx=0x7fb298007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:14.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb28d7fa700 1 -- 172.21.15.116:0/1873818752 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298002240 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb28d7fa700 1 -- 172.21.15.116:0/1873818752 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2980023a0 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1873818752 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a0091730 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb28d7fa700 1 -- 172.21.15.116:0/1873818752 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb298001820 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1873818752 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a008fc00 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.444+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1873818752 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a0165d00 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.448+0000 7fb28d7fa700 1 -- 172.21.15.116:0/1873818752 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb298019030 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.448+0000 7fb28d7fa700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb27c0557c0 0x7fb27c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.448+0000 7fb28d7fa700 1 -- 172.21.15.116:0/1873818752 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb29806dd50 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.448+0000 7fb2a54ca700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb27c0557c0 0x7fb27c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.448+0000 7fb2a54ca700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb27c0557c0 0x7fb27c057c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb2a0155890 tx=0x7fb29c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:14.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.452+0000 7fb28d7fa700 1 -- 172.21.15.116:0/1873818752 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb298043d20 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.608+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1873818752 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb2a0044f60 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.608+0000 7fb28d7fa700 1 -- 172.21.15.116:0/1873818752 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7fb298047340 con 0x7fb2a00c1bb0 2020-11-20T12:16:14.611 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:14.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb27c0557c0 msgr2=0x7fb27c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb27c0557c0 0x7fb27c057c80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fb2a0155890 tx=0x7fb29c006df0).stop 2020-11-20T12:16:14.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1873818752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a00c1bb0 msgr2=0x7fb2a015b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a00c1bb0 0x7fb2a015b600 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fb29800ca90 tx=0x7fb298007dd0).stop 2020-11-20T12:16:14.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1873818752 shutdown_connections 2020-11-20T12:16:14.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb27c0557c0 0x7fb27c057c80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a00c1bb0 0x7fb2a015b600 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a00a28c0 0x7fb2a008f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 --2- 172.21.15.116:0/1873818752 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a00c03f0 0x7fb2a0164c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1873818752 >> 172.21.15.116:0/1873818752 conn(0x7fb2a0060c30 msgr2=0x7fb2a0066730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:14.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1873818752 shutdown_connections 2020-11-20T12:16:14.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.612+0000 7fb2a64cc700 1 -- 172.21.15.116:0/1873818752 wait complete. 2020-11-20T12:16:14.635 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:16:14.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.812+0000 7f8278842700 1 Processor -- start 2020-11-20T12:16:14.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.812+0000 7f8278842700 1 -- start start 2020-11-20T12:16:14.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8278842700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f827414e040 0x7f82740c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8278842700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f827414e420 0x7f82740c31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8278842700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82740a1010 con 0x7f82740a4fd0 2020-11-20T12:16:14.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8278842700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82740a5ce0 con 0x7f827414e420 2020-11-20T12:16:14.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8278842700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82740a5f80 con 0x7f827414e040 2020-11-20T12:16:14.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f827359e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f82740a4fd0 0x7f82740a5480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52856/0 (socket says 172.21.15.116:52856) 2020-11-20T12:16:14.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f827359e700 1 -- 172.21.15.116:0/2420524128 learned_addr learned my addr 172.21.15.116:0/2420524128 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:14.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f827259c700 1 --2- 172.21.15.116:0/2420524128 >> v2:172.21.15.116:3300/0 conn(0x7f827414e420 0x7f82740c31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8272d9d700 1 --2- 172.21.15.116:0/2420524128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f827414e040 0x7f82740c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8271d9b700 1 -- 172.21.15.116:0/2420524128 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592346314 0 0) 0x7f82740a1010 con 0x7f82740a4fd0 2020-11-20T12:16:14.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8271d9b700 1 -- 172.21.15.116:0/2420524128 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8260003240 con 0x7f82740a4fd0 2020-11-20T12:16:14.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f827259c700 1 -- 172.21.15.116:0/2420524128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f827414e040 msgr2=0x7f82740c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:14.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f827259c700 1 --2- 172.21.15.116:0/2420524128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f827414e040 0x7f82740c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f827259c700 1 -- 172.21.15.116:0/2420524128 >> v1:172.21.15.116:6789/0 conn(0x7f82740a4fd0 legacy=0x7f82740a5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8271d9b700 1 -- 172.21.15.116:0/2420524128 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2930718078 0 0) 0x7f8260003240 con 0x7f82740a4fd0 2020-11-20T12:16:14.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f827259c700 1 -- 172.21.15.116:0/2420524128 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82740a1270 con 0x7f827414e420 2020-11-20T12:16:14.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8272d9d700 1 --2- 172.21.15.116:0/2420524128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f827414e040 0x7f82740c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:14.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f827259c700 1 --2- 172.21.15.116:0/2420524128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f827414e420 0x7f82740c31b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f826800b9a0 tx=0x7f8268006df0).ready entity=mon.0 client_cookie=989d18c6811924f7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:14.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8271d9b700 1 -- 172.21.15.116:0/2420524128 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826c001860 con 0x7f827414e420 2020-11-20T12:16:14.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8271d9b700 1 -- 172.21.15.116:0/2420524128 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8268011400 con 0x7f827414e420 2020-11-20T12:16:14.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.816+0000 7f8271d9b700 1 -- 172.21.15.116:0/2420524128 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8268011780 con 0x7f827414e420 2020-11-20T12:16:14.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.820+0000 7f8278842700 1 -- 172.21.15.116:0/2420524128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f827414e420 msgr2=0x7f82740c31b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.820+0000 7f8278842700 1 --2- 172.21.15.116:0/2420524128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f827414e420 0x7f82740c31b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f826800b9a0 tx=0x7f8268006df0).stop 2020-11-20T12:16:14.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.820+0000 7f8278842700 1 -- 172.21.15.116:0/2420524128 shutdown_connections 2020-11-20T12:16:14.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.820+0000 7f8278842700 1 --2- 172.21.15.116:0/2420524128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f827414e040 0x7f82740c2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.820+0000 7f8278842700 1 --2- 172.21.15.116:0/2420524128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f827414e420 0x7f82740c31b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.820+0000 7f8278842700 1 -- 172.21.15.116:0/2420524128 >> 172.21.15.116:0/2420524128 conn(0x7f8274060c30 msgr2=0x7f82740c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:14.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.820+0000 7f8278842700 1 -- 172.21.15.116:0/2420524128 shutdown_connections 2020-11-20T12:16:14.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.820+0000 7f8278842700 1 -- 172.21.15.116:0/2420524128 wait complete. 2020-11-20T12:16:14.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.820+0000 7f8278842700 1 Processor -- start 2020-11-20T12:16:14.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8278842700 1 -- start start 2020-11-20T12:16:14.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8278842700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82740a4fd0 0x7f827413fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8278842700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f827414e040 0x7f8274091350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8278842700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f827414e420 0x7f8274067220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8278842700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f827408fac0 con 0x7f82740a4fd0 2020-11-20T12:16:14.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8278842700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82740906d0 con 0x7f827414e420 2020-11-20T12:16:14.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8278842700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8274090970 con 0x7f827414e040 2020-11-20T12:16:14.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8272d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82740a4fd0 0x7f827413fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8272d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82740a4fd0 0x7f827413fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43426/0 (socket says 172.21.15.116:43426) 2020-11-20T12:16:14.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8272d9d700 1 -- 172.21.15.116:0/2805973669 learned_addr learned my addr 172.21.15.116:0/2805973669 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:14.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f827359e700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f827414e420 0x7f8274067220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f827259c700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f827414e040 0x7f8274091350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8272d9d700 1 -- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f827414e040 msgr2=0x7f8274091350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8272d9d700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f827414e040 0x7f8274091350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8272d9d700 1 -- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f827414e420 msgr2=0x7f8274067220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:14.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8272d9d700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f827414e420 0x7f8274067220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:14.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8272d9d700 1 -- 172.21.15.116:0/2805973669 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8274090c10 con 0x7f82740a4fd0 2020-11-20T12:16:14.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f827359e700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f827414e420 0x7f8274067220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:14.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.824+0000 7f8272d9d700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82740a4fd0 0x7f827413fa10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f826400ef50 tx=0x7f8264006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:14.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.828+0000 7f825affd700 1 -- 172.21.15.116:0/2805973669 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826400cbf0 con 0x7f82740a4fd0 2020-11-20T12:16:14.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.828+0000 7f825affd700 1 -- 172.21.15.116:0/2805973669 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f826400cd50 con 0x7f82740a4fd0 2020-11-20T12:16:14.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.828+0000 7f8278842700 1 -- 172.21.15.116:0/2805973669 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8274090eb0 con 0x7f82740a4fd0 2020-11-20T12:16:14.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.828+0000 7f825affd700 1 -- 172.21.15.116:0/2805973669 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264012600 con 0x7f82740a4fd0 2020-11-20T12:16:14.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.828+0000 7f8278842700 1 -- 172.21.15.116:0/2805973669 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8274091840 con 0x7f82740a4fd0 2020-11-20T12:16:14.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.828+0000 7f8278842700 1 -- 172.21.15.116:0/2805973669 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8274044f60 con 0x7f82740a4fd0 2020-11-20T12:16:14.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.828+0000 7f825affd700 1 -- 172.21.15.116:0/2805973669 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8264012970 con 0x7f82740a4fd0 2020-11-20T12:16:14.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.832+0000 7f825affd700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82500557f0 0x7f8250057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:14.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.832+0000 7f825affd700 1 -- 172.21.15.116:0/2805973669 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f826406ed70 con 0x7f82740a4fd0 2020-11-20T12:16:14.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.832+0000 7f827259c700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82500557f0 0x7f8250057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:14.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.832+0000 7f827259c700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82500557f0 0x7f8250057cb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f826800b9a0 tx=0x7f8268013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:14.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.832+0000 7f825affd700 1 -- 172.21.15.116:0/2805973669 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8264044c10 con 0x7f82740a4fd0 2020-11-20T12:16:14.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f8278842700 1 -- 172.21.15.116:0/2805973669 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f82740a6220 con 0x7f82740a4fd0 2020-11-20T12:16:14.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f825affd700 1 -- 172.21.15.116:0/2805973669 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8264018270 con 0x7f82740a4fd0 2020-11-20T12:16:14.997 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:15.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f8278842700 1 -- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82500557f0 msgr2=0x7f8250057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:15.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f8278842700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82500557f0 0x7f8250057cb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f826800b9a0 tx=0x7f8268013000).stop 2020-11-20T12:16:15.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f8278842700 1 -- 172.21.15.116:0/2805973669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82740a4fd0 msgr2=0x7f827413fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:15.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f8278842700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82740a4fd0 0x7f827413fa10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f826400ef50 tx=0x7f8264006df0).stop 2020-11-20T12:16:15.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f8278842700 1 -- 172.21.15.116:0/2805973669 shutdown_connections 2020-11-20T12:16:15.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f8278842700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82500557f0 0x7f8250057cb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f8278842700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f827414e420 0x7f8274067220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f8278842700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82740a4fd0 0x7f827413fa10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f8278842700 1 --2- 172.21.15.116:0/2805973669 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f827414e040 0x7f8274091350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:14.996+0000 7f8278842700 1 -- 172.21.15.116:0/2805973669 >> 172.21.15.116:0/2805973669 conn(0x7f8274060c30 msgr2=0x7f82740c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:15.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.000+0000 7f8278842700 1 -- 172.21.15.116:0/2805973669 shutdown_connections 2020-11-20T12:16:15.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.000+0000 7f8278842700 1 -- 172.21.15.116:0/2805973669 wait complete. 2020-11-20T12:16:15.024 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:16:15.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9cdf88700 1 Processor -- start 2020-11-20T12:16:15.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9cdf88700 1 -- start start 2020-11-20T12:16:15.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9cdf88700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb9c81536f0 0x7fb9c8153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:15.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9cdf88700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9c80a6ee0 0x7fb9c814fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:15.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9cdf88700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9c8153330 con 0x7fb9c80a6a60 2020-11-20T12:16:15.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9cdf88700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9c8150600 con 0x7fb9c81536f0 2020-11-20T12:16:15.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9cdf88700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c81508a0 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9c7fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb9c80a6a60 0x7fb9c8064580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52868/0 (socket says 172.21.15.116:52868) 2020-11-20T12:16:15.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9c7fff700 1 -- 172.21.15.116:0/1505567466 learned_addr learned my addr 172.21.15.116:0/1505567466 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:15.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9ccf86700 1 --2- 172.21.15.116:0/1505567466 >> v2:172.21.15.173:3300/0 conn(0x7fb9c81536f0 0x7fb9c8153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:15.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9cd787700 1 --2- 172.21.15.116:0/1505567466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9c80a6ee0 0x7fb9c814fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:15.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9c77fe700 1 -- 172.21.15.116:0/1505567466 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4161831893 0 0) 0x7fb9c8153330 con 0x7fb9c80a6a60 2020-11-20T12:16:15.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9c77fe700 1 -- 172.21.15.116:0/1505567466 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ac003240 con 0x7fb9c80a6a60 2020-11-20T12:16:15.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9ccf86700 1 -- 172.21.15.116:0/1505567466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9c80a6ee0 msgr2=0x7fb9c814fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:15.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9ccf86700 1 --2- 172.21.15.116:0/1505567466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9c80a6ee0 0x7fb9c814fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.184+0000 7fb9ccf86700 1 -- 172.21.15.116:0/1505567466 >> v1:172.21.15.116:6789/0 conn(0x7fb9c80a6a60 legacy=0x7fb9c8064580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:15.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9ccf86700 1 -- 172.21.15.116:0/1505567466 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c80a60f0 con 0x7fb9c81536f0 2020-11-20T12:16:15.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9ccf86700 1 --2- 172.21.15.116:0/1505567466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9c81536f0 0x7fb9c8153ad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb9bc009c90 tx=0x7fb9bc0091d0).ready entity=mon.1 client_cookie=8a770ab2f0e50a51 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:15.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9c77fe700 1 -- 172.21.15.116:0/1505567466 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc00b9c0 con 0x7fb9c81536f0 2020-11-20T12:16:15.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9c77fe700 1 -- 172.21.15.116:0/1505567466 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9bc00bb20 con 0x7fb9c81536f0 2020-11-20T12:16:15.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9c77fe700 1 -- 172.21.15.116:0/1505567466 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc00bea0 con 0x7fb9c81536f0 2020-11-20T12:16:15.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9cdf88700 1 -- 172.21.15.116:0/1505567466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9c81536f0 msgr2=0x7fb9c8153ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:15.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9cdf88700 1 --2- 172.21.15.116:0/1505567466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9c81536f0 0x7fb9c8153ad0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb9bc009c90 tx=0x7fb9bc0091d0).stop 2020-11-20T12:16:15.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9cdf88700 1 -- 172.21.15.116:0/1505567466 shutdown_connections 2020-11-20T12:16:15.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9cdf88700 1 --2- 172.21.15.116:0/1505567466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9c81536f0 0x7fb9c8153ad0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9cdf88700 1 --2- 172.21.15.116:0/1505567466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9c80a6ee0 0x7fb9c814fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9cdf88700 1 -- 172.21.15.116:0/1505567466 >> 172.21.15.116:0/1505567466 conn(0x7fb9c8060c40 msgr2=0x7fb9c80c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:15.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9cdf88700 1 -- 172.21.15.116:0/1505567466 shutdown_connections 2020-11-20T12:16:15.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.188+0000 7fb9cdf88700 1 -- 172.21.15.116:0/1505567466 wait complete. 2020-11-20T12:16:15.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cdf88700 1 Processor -- start 2020-11-20T12:16:15.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cdf88700 1 -- start start 2020-11-20T12:16:15.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cdf88700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9c80a6a60 0x7fb9c81552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:15.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cdf88700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c80a6ee0 0x7fb9c81607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:15.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cdf88700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9c81536f0 0x7fb9c8160ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:15.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cdf88700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c8154900 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cdf88700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c81653f0 con 0x7fb9c80a6a60 2020-11-20T12:16:15.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cdf88700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c8154b60 con 0x7fb9c81536f0 2020-11-20T12:16:15.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c80a6ee0 0x7fb9c81607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:15.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c80a6ee0 0x7fb9c81607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43442/0 (socket says 172.21.15.116:43442) 2020-11-20T12:16:15.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c7fff700 1 -- 172.21.15.116:0/4078522220 learned_addr learned my addr 172.21.15.116:0/4078522220 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:15.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9ccf86700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9c80a6a60 0x7fb9c81552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:15.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cd787700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9c81536f0 0x7fb9c8160ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:15.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c7fff700 1 -- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9c81536f0 msgr2=0x7fb9c8160ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:15.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c7fff700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9c81536f0 0x7fb9c8160ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c7fff700 1 -- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9c80a6a60 msgr2=0x7fb9c81552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:15.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c7fff700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9c80a6a60 0x7fb9c81552a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c7fff700 1 -- 172.21.15.116:0/4078522220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c8154e00 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cd787700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9c81536f0 0x7fb9c8160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:15.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c7fff700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c80a6ee0 0x7fb9c81607f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fb9b400c6a0 tx=0x7fb9b4007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:15.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c4ff9700 1 -- 172.21.15.116:0/4078522220 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40072e0 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c4ff9700 1 -- 172.21.15.116:0/4078522220 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b4007440 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cdf88700 1 -- 172.21.15.116:0/4078522220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c8165690 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9c4ff9700 1 -- 172.21.15.116:0/4078522220 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4005820 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.192+0000 7fb9cdf88700 1 -- 172.21.15.116:0/4078522220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c8155930 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.196+0000 7fb9cdf88700 1 -- 172.21.15.116:0/4078522220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c8154f90 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.196+0000 7fb9c4ff9700 1 -- 172.21.15.116:0/4078522220 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb9b4010070 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.196+0000 7fb9c4ff9700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a00557c0 0x7fb9a0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:15.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.196+0000 7fb9c4ff9700 1 -- 172.21.15.116:0/4078522220 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb9b406dd00 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.200+0000 7fb9ccf86700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a00557c0 0x7fb9a0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:15.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.200+0000 7fb9ccf86700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a00557c0 0x7fb9a0057c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb9bc006ee0 tx=0x7fb9bc016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:15.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.204+0000 7fb9c4ff9700 1 -- 172.21.15.116:0/4078522220 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb9b4071030 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.412+0000 7fb9cdf88700 1 -- 172.21.15.116:0/4078522220 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb9c8044f60 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.412+0000 7fb9c4ff9700 1 -- 172.21.15.116:0/4078522220 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2846 (secure 0 0 0) 0x7fb9b4043ba0 con 0x7fb9c80a6ee0 2020-11-20T12:16:15.414 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:15.415 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/2968741298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":2968741298},{"type":"v1","addr":"172.21.15.116:6835","nonce":2968741298}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1214743116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1214743116},{"type":"v1","addr":"172.21.15.173:6836","nonce":1214743116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1424291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1424291148},{"type":"v1","addr":"172.21.15.173:6837","nonce":1424291148}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:00.619934+0000","modified":"2020-11-20T12:16:02.636079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.116:6834/3819756176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3819756176},{"type":"v1","addr":"172.21.15.116:6834","nonce":3819756176}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:16:15.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 -- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a00557c0 msgr2=0x7fb9a0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:15.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a00557c0 0x7fb9a0057c80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb9bc006ee0 tx=0x7fb9bc016000).stop 2020-11-20T12:16:15.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 -- 172.21.15.116:0/4078522220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c80a6ee0 msgr2=0x7fb9c81607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:15.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c80a6ee0 0x7fb9c81607f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fb9b400c6a0 tx=0x7fb9b4007ac0).stop 2020-11-20T12:16:15.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 -- 172.21.15.116:0/4078522220 shutdown_connections 2020-11-20T12:16:15.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a00557c0 0x7fb9a0057c80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9c81536f0 0x7fb9c8160ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9c80a6a60 0x7fb9c81552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 --2- 172.21.15.116:0/4078522220 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9c80a6ee0 0x7fb9c81607f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:15.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 -- 172.21.15.116:0/4078522220 >> 172.21.15.116:0/4078522220 conn(0x7fb9c8060c40 msgr2=0x7fb9c80a7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:15.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 -- 172.21.15.116:0/4078522220 shutdown_connections 2020-11-20T12:16:15.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:15.416+0000 7fb9cdf88700 1 -- 172.21.15.116:0/4078522220 wait complete. 2020-11-20T12:16:15.421 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 6 2020-11-20T12:16:15.847 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:00.619934+0000', 'modified': '2020-11-20T12:16:02.636079+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4245}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4245': {'gid': 4245, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.116:6834/3819756176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 3819756176}, {'type': 'v1', 'addr': '172.21.15.116:6834', 'nonce': 3819756176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:16:15.848 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:16:15.849 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:16:16.042 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4245,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.400411037,"mdsmap_epoch":6,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":18.888256905999999} 2020-11-20T12:16:16.043 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4245, "mdsmap_epoch": 6, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 15.400411037, "state": "up:active", "uptime": 18.888256906, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:16:16.044 INFO:teuthology.run_tasks:Running task kclient... 2020-11-20T12:16:16.063 INFO:tasks.kclient:Mounting kernel clients... 2020-11-20T12:16:16.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.064 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:16:16.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.082 INFO:teuthology.orchestra.run.smithi116:> sudo ip link delete ceph-brx 2020-11-20T12:16:16.157 INFO:teuthology.orchestra.run.smithi116.stderr:Cannot find device "ceph-brx" 2020-11-20T12:16:16.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:16:16.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.160 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:16:16.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.178 INFO:teuthology.orchestra.run.smithi116:> sudo ip link delete ceph-brx 2020-11-20T12:16:16.215 INFO:teuthology.orchestra.run.smithi116.stderr:Cannot find device "ceph-brx" 2020-11-20T12:16:16.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:16:16.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.220 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:16:16.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.238 INFO:teuthology.orchestra.run.smithi173:> sudo ip link delete ceph-brx 2020-11-20T12:16:16.268 INFO:teuthology.orchestra.run.smithi173.stderr:Cannot find device "ceph-brx" 2020-11-20T12:16:16.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:16:16.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.272 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:16:16.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.288 INFO:teuthology.orchestra.run.smithi173:> sudo ip link delete ceph-brx 2020-11-20T12:16:16.329 INFO:teuthology.orchestra.run.smithi173.stderr:Cannot find device "ceph-brx" 2020-11-20T12:16:16.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:16:16.332 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:16:16.332 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:16:16.333 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:16:16.333 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:16:16.334 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:16:16.334 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2020-11-20T12:16:16.334 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:16:16.334 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:16:16.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.335 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:16:16.361 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:16:16.361 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:16:16.362 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:16:16.362 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:16.362 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:16:16.362 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:16.362 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:16.363 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:16.363 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:16.363 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:16.363 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:16.363 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:16.364 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:16:16.364 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:16.364 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:16:16.364 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:16.365 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:16:16.365 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:16.365 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:16:16.365 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:16.365 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:16:16.366 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:16.366 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-11-20T12:16:16.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.367 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:16.367 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:16:16.367 INFO:teuthology.orchestra.run.smithi116:> sudo ip link add name ceph-brx type bridge 2020-11-20T12:16:16.368 INFO:teuthology.orchestra.run.smithi116:> sudo ip addr flush dev ceph-brx 2020-11-20T12:16:16.368 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set ceph-brx up 2020-11-20T12:16:16.368 INFO:teuthology.orchestra.run.smithi116:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2020-11-20T12:16:16.369 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:16:16.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.519 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:16:16.553 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:16:16.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.557 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:16:16.645 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:16:16.645 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:16:16.646 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:16:16.646 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:16:16.646 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:16:16.647 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:16:16.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.648 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:16.648 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:16:16.649 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:16:16.649 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:16:16.650 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:16:16.650 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:16:16.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.794 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:16:16.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.812 INFO:teuthology.orchestra.run.smithi116:> ip netns list-id 2020-11-20T12:16:16.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.868 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:16.868 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:16:16.869 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2020-11-20T12:16:16.870 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2020-11-20T12:16:16.870 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:16:16.957 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2020-11-20T12:16:16.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:16.958 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:16.959 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:16:16.959 INFO:teuthology.orchestra.run.smithi116:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2020-11-20T12:16:16.960 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2020-11-20T12:16:16.960 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2020-11-20T12:16:16.960 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2020-11-20T12:16:16.961 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2020-11-20T12:16:16.961 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:16:17.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:17.135 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:17.135 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:16:17.136 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2020-11-20T12:16:17.136 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set dev brx.0 master ceph-brx 2020-11-20T12:16:17.136 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:16:17.219 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:17.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.392+0000 7f95fde1d700 1 Processor -- start 2020-11-20T12:16:17.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.392+0000 7f95fde1d700 1 -- start start 2020-11-20T12:16:17.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95fde1d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f813aee0 0x7f95f8152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95fde1d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f8152a30 con 0x7f95f80aa790 2020-11-20T12:16:17.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95fde1d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f80aac10 con 0x7f95f80612f0 2020-11-20T12:16:17.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95fde1d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80adce0 con 0x7f95f813aee0 2020-11-20T12:16:17.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f7fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f95f80aa790 0x7f95f80ad5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52884/0 (socket says 172.21.15.116:52884) 2020-11-20T12:16:17.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f7fff700 1 -- 172.21.15.116:0/4238014355 learned_addr learned my addr 172.21.15.116:0/4238014355 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:17.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95fce1b700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f813aee0 0x7f95f8152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195360023 0 0) 0x7f95f8152a30 con 0x7f95f80aa790 2020-11-20T12:16:17.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95dc003240 con 0x7f95f80aa790 2020-11-20T12:16:17.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 817983218 0 0) 0x7f95dc003240 con 0x7f95f80aa790 2020-11-20T12:16:17.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f813aee0 msgr2=0x7f95f8152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:17.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f813aee0 0x7f95f8152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 >> v1:172.21.15.173:6789/0 conn(0x7f95f80612f0 legacy=0x7f95f80a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f80aaeb0 con 0x7f95f80aa790 2020-11-20T12:16:17.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f95e40014e0 con 0x7f95f80aa790 2020-11-20T12:16:17.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 >> v1:172.21.15.116:6789/0 conn(0x7f95f80aa790 legacy=0x7f95f80ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc003760 0x7f95dc003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95dc0044e0 0x7f95dc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f813aee0 0x7f95dc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80adce0 con 0x7f95dc003760 2020-11-20T12:16:17.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc004340 con 0x7f95dc0044e0 2020-11-20T12:16:17.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95dc00b730 con 0x7f95f813aee0 2020-11-20T12:16:17.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95fce1b700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc003760 0x7f95dc003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95f7fff700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95dc0044e0 0x7f95dc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.396+0000 7f95fd61c700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f813aee0 0x7f95dc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fce1b700 1 -- 172.21.15.116:0/4238014355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f813aee0 msgr2=0x7f95dc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:17.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fce1b700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f813aee0 0x7f95dc004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fce1b700 1 -- 172.21.15.116:0/4238014355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95dc0044e0 msgr2=0x7f95dc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:17.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fce1b700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95dc0044e0 0x7f95dc004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fce1b700 1 -- 172.21.15.116:0/4238014355 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95dc003240 con 0x7f95dc003760 2020-11-20T12:16:17.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fce1b700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc003760 0x7f95dc003b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f95ec00bc00 tx=0x7f95ec006df0).ready entity=mon.0 client_cookie=a5ac46a48bf5502e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:17.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00b310 con 0x7f95dc003760 2020-11-20T12:16:17.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95ec00ea30 con 0x7f95dc003760 2020-11-20T12:16:17.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95f77fe700 1 -- 172.21.15.116:0/4238014355 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95ec00edb0 con 0x7f95dc003760 2020-11-20T12:16:17.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fde1d700 1 -- 172.21.15.116:0/4238014355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc003760 msgr2=0x7f95dc003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fde1d700 1 --2- 172.21.15.116:0/4238014355 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95dc003760 0x7f95dc003b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f95ec00bc00 tx=0x7f95ec006df0).stop 2020-11-20T12:16:17.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fd61c700 1 -- 172.21.15.116:0/4238014355 reap_dead start 2020-11-20T12:16:17.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fde1d700 1 -- 172.21.15.116:0/4238014355 shutdown_connections 2020-11-20T12:16:17.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fde1d700 1 -- 172.21.15.116:0/4238014355 >> 172.21.15.116:0/4238014355 conn(0x7f95f80678a0 msgr2=0x7f95f8067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:17.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fde1d700 1 -- 172.21.15.116:0/4238014355 shutdown_connections 2020-11-20T12:16:17.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.400+0000 7f95fde1d700 1 -- 172.21.15.116:0/4238014355 wait complete. 2020-11-20T12:16:17.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fde1d700 1 Processor -- start 2020-11-20T12:16:17.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fde1d700 1 -- start start 2020-11-20T12:16:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fde1d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f8150db0 0x7f95f80a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fde1d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f80a31d0 0x7f95f80a1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fde1d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f80a1fc0 0x7f95f812d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fde1d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80a0bb0 con 0x7f95f80a1fc0 2020-11-20T12:16:17.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fde1d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80a0e50 con 0x7f95f8150db0 2020-11-20T12:16:17.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fde1d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80a10f0 con 0x7f95f80a31d0 2020-11-20T12:16:17.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fd61c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f80a1fc0 0x7f95f812d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fd61c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f80a1fc0 0x7f95f812d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43464/0 (socket says 172.21.15.116:43464) 2020-11-20T12:16:17.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fd61c700 1 -- 172.21.15.116:0/356539408 learned_addr learned my addr 172.21.15.116:0/356539408 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:17.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95f7fff700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f80a31d0 0x7f95f80a1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.404+0000 7f95fce1b700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f8150db0 0x7f95f80a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95fd61c700 1 -- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f80a31d0 msgr2=0x7f95f80a1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95fd61c700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f80a31d0 0x7f95f80a1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95fd61c700 1 -- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f8150db0 msgr2=0x7f95f80a2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95fd61c700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f8150db0 0x7f95f80a2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95fd61c700 1 -- 172.21.15.116:0/356539408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f80a1390 con 0x7f95f80a1fc0 2020-11-20T12:16:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95fd61c700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f80a1fc0 0x7f95f812d730 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f95f8062480 tx=0x7f95f0007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95f4ff9700 1 -- 172.21.15.116:0/356539408 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0017070 con 0x7f95f80a1fc0 2020-11-20T12:16:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95f4ff9700 1 -- 172.21.15.116:0/356539408 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f000ee90 con 0x7f95f80a1fc0 2020-11-20T12:16:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95fde1d700 1 -- 172.21.15.116:0/356539408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f80a1630 con 0x7f95f80a1fc0 2020-11-20T12:16:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95f4ff9700 1 -- 172.21.15.116:0/356539408 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f0002250 con 0x7f95f80a1fc0 2020-11-20T12:16:17.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95fde1d700 1 -- 172.21.15.116:0/356539408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f8151190 con 0x7f95f80a1fc0 2020-11-20T12:16:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95fde1d700 1 -- 172.21.15.116:0/356539408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f8044f60 con 0x7f95f80a1fc0 2020-11-20T12:16:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.408+0000 7f95f4ff9700 1 -- 172.21.15.116:0/356539408 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f95f000c7b0 con 0x7f95f80a1fc0 2020-11-20T12:16:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.412+0000 7f95f4ff9700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95d00557c0 0x7f95d0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.412+0000 7f95f4ff9700 1 -- 172.21.15.116:0/356539408 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f95f006e300 con 0x7f95f80a1fc0 2020-11-20T12:16:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.412+0000 7f95fce1b700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95d00557c0 0x7f95d0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.412+0000 7f95fce1b700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95d00557c0 0x7f95d0057c80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f95f805e230 tx=0x7f95ec019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:17.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.416+0000 7f95f4ff9700 1 -- 172.21.15.116:0/356539408 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f95f00441a0 con 0x7f95f80a1fc0 2020-11-20T12:16:17.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.576+0000 7f95fde1d700 1 -- 172.21.15.116:0/356539408 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95f8060500 con 0x7f95f80a1fc0 2020-11-20T12:16:17.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.580+0000 7f95f4ff9700 1 -- 172.21.15.116:0/356539408 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f95f00477c0 con 0x7f95f80a1fc0 2020-11-20T12:16:17.581 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:17.581 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:17.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.580+0000 7f95fde1d700 1 -- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95d00557c0 msgr2=0x7f95d0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.580+0000 7f95fde1d700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95d00557c0 0x7f95d0057c80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f95f805e230 tx=0x7f95ec019000).stop 2020-11-20T12:16:17.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.580+0000 7f95fde1d700 1 -- 172.21.15.116:0/356539408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f80a1fc0 msgr2=0x7f95f812d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.580+0000 7f95fde1d700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f80a1fc0 0x7f95f812d730 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f95f8062480 tx=0x7f95f0007f10).stop 2020-11-20T12:16:17.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.580+0000 7f95fde1d700 1 -- 172.21.15.116:0/356539408 shutdown_connections 2020-11-20T12:16:17.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.580+0000 7f95fde1d700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95d00557c0 0x7f95d0057c80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.580+0000 7f95fde1d700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f95f80a1fc0 0x7f95f812d730 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.580+0000 7f95fde1d700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f95f8150db0 0x7f95f80a2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.580+0000 7f95fde1d700 1 --2- 172.21.15.116:0/356539408 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f95f80a31d0 0x7f95f80a1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.580+0000 7f95fde1d700 1 -- 172.21.15.116:0/356539408 >> 172.21.15.116:0/356539408 conn(0x7f95f80678a0 msgr2=0x7f95f8069690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:17.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.584+0000 7f95fde1d700 1 -- 172.21.15.116:0/356539408 shutdown_connections 2020-11-20T12:16:17.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.584+0000 7f95fde1d700 1 -- 172.21.15.116:0/356539408 wait complete. 2020-11-20T12:16:17.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:15:59.065285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:15:59.602311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:16:17.609 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.800+0000 7f3b96711700 1 Processor -- start 2020-11-20T12:16:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.800+0000 7f3b96711700 1 -- start start 2020-11-20T12:16:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.800+0000 7f3b96711700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b9014fce0 0x7f3b90156990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.800+0000 7f3b96711700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3b900630f0 0x7f3b900635a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.800+0000 7f3b96711700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b900678b0 con 0x7f3b901500c0 2020-11-20T12:16:17.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.800+0000 7f3b96711700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b90067b50 con 0x7f3b900630f0 2020-11-20T12:16:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.800+0000 7f3b96711700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b90067df0 con 0x7f3b9014fce0 2020-11-20T12:16:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.800+0000 7f3b94f0e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3b901500c0 0x7f3b900670c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52904/0 (socket says 172.21.15.116:52904) 2020-11-20T12:16:17.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.800+0000 7f3b94f0e700 1 -- 172.21.15.116:0/3010284048 learned_addr learned my addr 172.21.15.116:0/3010284048 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b95f10700 1 --2- 172.21.15.116:0/3010284048 >> v2:172.21.15.173:3300/0 conn(0x7f3b900630f0 0x7f3b900635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b7ffff700 1 -- 172.21.15.116:0/3010284048 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136658016 0 0) 0x7f3b900678b0 con 0x7f3b901500c0 2020-11-20T12:16:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b7ffff700 1 -- 172.21.15.116:0/3010284048 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b78003240 con 0x7f3b901500c0 2020-11-20T12:16:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b9570f700 1 --2- 172.21.15.116:0/3010284048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b9014fce0 0x7f3b90156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b95f10700 1 -- 172.21.15.116:0/3010284048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b9014fce0 msgr2=0x7f3b90156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:17.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b95f10700 1 --2- 172.21.15.116:0/3010284048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b9014fce0 0x7f3b90156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b7ffff700 1 -- 172.21.15.116:0/3010284048 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 738391990 0 0) 0x7f3b78003240 con 0x7f3b901500c0 2020-11-20T12:16:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b95f10700 1 -- 172.21.15.116:0/3010284048 >> v1:172.21.15.116:6789/0 conn(0x7f3b901500c0 legacy=0x7f3b900670c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b95f10700 1 -- 172.21.15.116:0/3010284048 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b900602f0 con 0x7f3b900630f0 2020-11-20T12:16:17.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b95f10700 1 --2- 172.21.15.116:0/3010284048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b900630f0 0x7f3b900635a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f3b88009c90 tx=0x7f3b880091d0).ready entity=mon.1 client_cookie=3c3b3933b764e61d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:17.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b7ffff700 1 -- 172.21.15.116:0/3010284048 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b800014e0 con 0x7f3b900630f0 2020-11-20T12:16:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b7ffff700 1 -- 172.21.15.116:0/3010284048 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8800b7f0 con 0x7f3b900630f0 2020-11-20T12:16:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b7ffff700 1 -- 172.21.15.116:0/3010284048 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8800bb70 con 0x7f3b900630f0 2020-11-20T12:16:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b96711700 1 -- 172.21.15.116:0/3010284048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b900630f0 msgr2=0x7f3b900635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b96711700 1 --2- 172.21.15.116:0/3010284048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b900630f0 0x7f3b900635a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f3b88009c90 tx=0x7f3b880091d0).stop 2020-11-20T12:16:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b96711700 1 -- 172.21.15.116:0/3010284048 shutdown_connections 2020-11-20T12:16:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b96711700 1 --2- 172.21.15.116:0/3010284048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b900630f0 0x7f3b900635a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b96711700 1 --2- 172.21.15.116:0/3010284048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b9014fce0 0x7f3b90156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b96711700 1 -- 172.21.15.116:0/3010284048 >> 172.21.15.116:0/3010284048 conn(0x7f3b90061d50 msgr2=0x7f3b900c50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:17.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.804+0000 7f3b96711700 1 -- 172.21.15.116:0/3010284048 shutdown_connections 2020-11-20T12:16:17.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.808+0000 7f3b96711700 1 -- 172.21.15.116:0/3010284048 wait complete. 2020-11-20T12:16:17.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.808+0000 7f3b96711700 1 Processor -- start 2020-11-20T12:16:17.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.808+0000 7f3b96711700 1 -- start start 2020-11-20T12:16:17.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b96711700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b900630f0 0x7f3b9014f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b96711700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b9014fce0 0x7f3b90159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b96711700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b901500c0 0x7f3b90159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b96711700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b900a7cd0 con 0x7f3b900630f0 2020-11-20T12:16:17.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b96711700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b90159650 con 0x7f3b9014fce0 2020-11-20T12:16:17.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b96711700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9015a130 con 0x7f3b901500c0 2020-11-20T12:16:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b9570f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b900630f0 0x7f3b9014f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b9570f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b900630f0 0x7f3b9014f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43474/0 (socket says 172.21.15.116:43474) 2020-11-20T12:16:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b9570f700 1 -- 172.21.15.116:0/2035412615 learned_addr learned my addr 172.21.15.116:0/2035412615 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:17.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b95f10700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b901500c0 0x7f3b90159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b94f0e700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b9014fce0 0x7f3b90159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b9570f700 1 -- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b901500c0 msgr2=0x7f3b90159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b9570f700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b901500c0 0x7f3b90159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b9570f700 1 -- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b9014fce0 msgr2=0x7f3b90159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b9570f700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b9014fce0 0x7f3b90159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b9570f700 1 -- 172.21.15.116:0/2035412615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b90159870 con 0x7f3b900630f0 2020-11-20T12:16:17.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b9570f700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b900630f0 0x7f3b9014f580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f3b8c00db60 tx=0x7f3b8c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:17.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b7d7fa700 1 -- 172.21.15.116:0/2035412615 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c009820 con 0x7f3b900630f0 2020-11-20T12:16:17.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b7d7fa700 1 -- 172.21.15.116:0/2035412615 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b8c009980 con 0x7f3b900630f0 2020-11-20T12:16:17.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b7d7fa700 1 -- 172.21.15.116:0/2035412615 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c009d20 con 0x7f3b900630f0 2020-11-20T12:16:17.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b96711700 1 -- 172.21.15.116:0/2035412615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b900a74f0 con 0x7f3b900630f0 2020-11-20T12:16:17.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b96711700 1 -- 172.21.15.116:0/2035412615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b900a7a30 con 0x7f3b900630f0 2020-11-20T12:16:17.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.812+0000 7f3b96711700 1 -- 172.21.15.116:0/2035412615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b90044f60 con 0x7f3b900630f0 2020-11-20T12:16:17.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.816+0000 7f3b7d7fa700 1 -- 172.21.15.116:0/2035412615 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3b8c013690 con 0x7f3b900630f0 2020-11-20T12:16:17.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.816+0000 7f3b7d7fa700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b680557c0 0x7f3b68057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:17.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.816+0000 7f3b7d7fa700 1 -- 172.21.15.116:0/2035412615 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3b8c06ed80 con 0x7f3b900630f0 2020-11-20T12:16:17.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.816+0000 7f3b94f0e700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b680557c0 0x7f3b68057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:17.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.816+0000 7f3b94f0e700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b680557c0 0x7f3b68057c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f3b901584c0 tx=0x7f3b80009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:17.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.820+0000 7f3b7d7fa700 1 -- 172.21.15.116:0/2035412615 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3b8c044c20 con 0x7f3b900630f0 2020-11-20T12:16:17.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.976+0000 7f3b96711700 1 -- 172.21.15.116:0/2035412615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3b900675f0 con 0x7f3b900630f0 2020-11-20T12:16:17.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.976+0000 7f3b7d7fa700 1 -- 172.21.15.116:0/2035412615 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3b8c070020 con 0x7f3b900630f0 2020-11-20T12:16:17.980 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:17.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 -- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b680557c0 msgr2=0x7f3b68057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b680557c0 0x7f3b68057c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f3b901584c0 tx=0x7f3b80009040).stop 2020-11-20T12:16:17.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 -- 172.21.15.116:0/2035412615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b900630f0 msgr2=0x7f3b9014f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:17.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b900630f0 0x7f3b9014f580 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f3b8c00db60 tx=0x7f3b8c008040).stop 2020-11-20T12:16:17.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 -- 172.21.15.116:0/2035412615 shutdown_connections 2020-11-20T12:16:17.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b680557c0 0x7f3b68057c80 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b901500c0 0x7f3b90159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b900630f0 0x7f3b9014f580 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 --2- 172.21.15.116:0/2035412615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b9014fce0 0x7f3b90159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:17.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 -- 172.21.15.116:0/2035412615 >> 172.21.15.116:0/2035412615 conn(0x7f3b90061d50 msgr2=0x7f3b900ba030 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:17.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 -- 172.21.15.116:0/2035412615 shutdown_connections 2020-11-20T12:16:17.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:17.980+0000 7f3b96711700 1 -- 172.21.15.116:0/2035412615 wait complete. 2020-11-20T12:16:18.005 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:18.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.184+0000 7f3d3b81d700 1 Processor -- start 2020-11-20T12:16:18.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.184+0000 7f3d3b81d700 1 -- start start 2020-11-20T12:16:18.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d3b81d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3d3414f150 0x7f3d340c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d3b81d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3d340bd780 0x7f3d340c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d3b81d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d3414f530 0x7f3d340a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d3b81d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d34155620 con 0x7f3d340bd780 2020-11-20T12:16:18.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d3b81d700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d341558c0 con 0x7f3d3414f150 2020-11-20T12:16:18.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d3b81d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d34155b60 con 0x7f3d3414f530 2020-11-20T12:16:18.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d38db8700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3d340bd780 0x7f3d340c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d38db8700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3d340bd780 0x7f3d340c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43484/0 (socket says 172.21.15.116:43484) 2020-11-20T12:16:18.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d38db8700 1 -- 172.21.15.116:0/2412065000 learned_addr learned my addr 172.21.15.116:0/2412065000 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:18.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d395b9700 1 --2- 172.21.15.116:0/2412065000 >> v2:172.21.15.173:3300/0 conn(0x7f3d3414f150 0x7f3d340c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d39dba700 1 --2- 172.21.15.116:0/2412065000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d3414f530 0x7f3d340a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d38db8700 1 -- 172.21.15.116:0/2412065000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d3414f530 msgr2=0x7f3d340a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:18.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d38db8700 1 --2- 172.21.15.116:0/2412065000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d3414f530 0x7f3d340a74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d38db8700 1 -- 172.21.15.116:0/2412065000 >> v2:172.21.15.173:3300/0 conn(0x7f3d3414f150 msgr2=0x7f3d340c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:18.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d38db8700 1 --2- 172.21.15.116:0/2412065000 >> v2:172.21.15.173:3300/0 conn(0x7f3d3414f150 0x7f3d340c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d38db8700 1 -- 172.21.15.116:0/2412065000 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d34155e00 con 0x7f3d340bd780 2020-11-20T12:16:18.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d395b9700 1 --2- 172.21.15.116:0/2412065000 >> v2:172.21.15.173:3300/0 conn(0x7f3d3414f150 0x7f3d340c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:18.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d38db8700 1 --2- 172.21.15.116:0/2412065000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d340bd780 0x7f3d340c8ce0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f3d34081540 tx=0x7f3d24006eb0).ready entity=mon.0 client_cookie=ed3f727a8bef8c67 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:18.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d23fff700 1 -- 172.21.15.116:0/2412065000 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24009960 con 0x7f3d340bd780 2020-11-20T12:16:18.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d23fff700 1 -- 172.21.15.116:0/2412065000 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d24009ac0 con 0x7f3d340bd780 2020-11-20T12:16:18.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.188+0000 7f3d23fff700 1 -- 172.21.15.116:0/2412065000 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24009e40 con 0x7f3d340bd780 2020-11-20T12:16:18.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.192+0000 7f3d3b81d700 1 -- 172.21.15.116:0/2412065000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d340bd780 msgr2=0x7f3d340c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.192+0000 7f3d3b81d700 1 --2- 172.21.15.116:0/2412065000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d340bd780 0x7f3d340c8ce0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f3d34081540 tx=0x7f3d24006eb0).stop 2020-11-20T12:16:18.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.192+0000 7f3d3b81d700 1 -- 172.21.15.116:0/2412065000 shutdown_connections 2020-11-20T12:16:18.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.192+0000 7f3d3b81d700 1 --2- 172.21.15.116:0/2412065000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d3414f530 0x7f3d340a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.192+0000 7f3d3b81d700 1 --2- 172.21.15.116:0/2412065000 >> v2:172.21.15.173:3300/0 conn(0x7f3d3414f150 0x7f3d340c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.192+0000 7f3d3b81d700 1 --2- 172.21.15.116:0/2412065000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d340bd780 0x7f3d340c8ce0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.192+0000 7f3d3b81d700 1 -- 172.21.15.116:0/2412065000 >> 172.21.15.116:0/2412065000 conn(0x7f3d34061580 msgr2=0x7f3d34061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:18.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.192+0000 7f3d3b81d700 1 -- 172.21.15.116:0/2412065000 shutdown_connections 2020-11-20T12:16:18.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.192+0000 7f3d3b81d700 1 -- 172.21.15.116:0/2412065000 wait complete. 2020-11-20T12:16:18.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d3b81d700 1 Processor -- start 2020-11-20T12:16:18.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d3b81d700 1 -- start start 2020-11-20T12:16:18.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d3b81d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d340bd780 0x7f3d340ad190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d3b81d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d3414f150 0x7f3d3414a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d3b81d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d3414f530 0x7f3d34157750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d3b81d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d34165820 con 0x7f3d3414f150 2020-11-20T12:16:18.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d3b81d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d340ac510 con 0x7f3d3414f530 2020-11-20T12:16:18.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d3b81d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d340ac7b0 con 0x7f3d340bd780 2020-11-20T12:16:18.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d38db8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d3414f150 0x7f3d3414a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d38db8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d3414f150 0x7f3d3414a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43488/0 (socket says 172.21.15.116:43488) 2020-11-20T12:16:18.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d38db8700 1 -- 172.21.15.116:0/4180572762 learned_addr learned my addr 172.21.15.116:0/4180572762 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:18.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d395b9700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d340bd780 0x7f3d340ad190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d39dba700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d3414f530 0x7f3d34157750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d38db8700 1 -- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d340bd780 msgr2=0x7f3d340ad190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d38db8700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d340bd780 0x7f3d340ad190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d38db8700 1 -- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d3414f530 msgr2=0x7f3d34157750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d38db8700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d3414f530 0x7f3d34157750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d38db8700 1 -- 172.21.15.116:0/4180572762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d340aca50 con 0x7f3d3414f150 2020-11-20T12:16:18.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d395b9700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d340bd780 0x7f3d340ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:18.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.196+0000 7f3d39dba700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d3414f530 0x7f3d34157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:18.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.200+0000 7f3d38db8700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d3414f150 0x7f3d3414a880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f3d24006ef0 tx=0x7f3d24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:18.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.200+0000 7f3d217fa700 1 -- 172.21.15.116:0/4180572762 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d24011e90 con 0x7f3d3414f150 2020-11-20T12:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.200+0000 7f3d217fa700 1 -- 172.21.15.116:0/4180572762 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2401e460 con 0x7f3d3414f150 2020-11-20T12:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.200+0000 7f3d3b81d700 1 -- 172.21.15.116:0/4180572762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d340accf0 con 0x7f3d3414f150 2020-11-20T12:16:18.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.200+0000 7f3d217fa700 1 -- 172.21.15.116:0/4180572762 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2401e800 con 0x7f3d3414f150 2020-11-20T12:16:18.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.200+0000 7f3d3b81d700 1 -- 172.21.15.116:0/4180572762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3414af10 con 0x7f3d3414f150 2020-11-20T12:16:18.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.200+0000 7f3d3b81d700 1 -- 172.21.15.116:0/4180572762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d340ace80 con 0x7f3d3414f150 2020-11-20T12:16:18.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.200+0000 7f3d217fa700 1 -- 172.21.15.116:0/4180572762 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3d24025030 con 0x7f3d3414f150 2020-11-20T12:16:18.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.204+0000 7f3d217fa700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d0c0557c0 0x7f3d0c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.204+0000 7f3d217fa700 1 -- 172.21.15.116:0/4180572762 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3d240703e0 con 0x7f3d3414f150 2020-11-20T12:16:18.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.204+0000 7f3d395b9700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d0c0557c0 0x7f3d0c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.204+0000 7f3d395b9700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d0c0557c0 0x7f3d0c057c80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3d3000bac0 tx=0x7f3d30008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:18.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.208+0000 7f3d217fa700 1 -- 172.21.15.116:0/4180572762 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3d24046170 con 0x7f3d3414f150 2020-11-20T12:16:18.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.368+0000 7f3d3b81d700 1 -- 172.21.15.116:0/4180572762 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3d3405ca30 con 0x7f3d3414f150 2020-11-20T12:16:18.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.368+0000 7f3d217fa700 1 -- 172.21.15.116:0/4180572762 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3d24023020 con 0x7f3d3414f150 2020-11-20T12:16:18.372 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:18.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.372+0000 7f3d3b81d700 1 -- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d0c0557c0 msgr2=0x7f3d0c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.372+0000 7f3d3b81d700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d0c0557c0 0x7f3d0c057c80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f3d3000bac0 tx=0x7f3d30008040).stop 2020-11-20T12:16:18.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.372+0000 7f3d3b81d700 1 -- 172.21.15.116:0/4180572762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d3414f150 msgr2=0x7f3d3414a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.372+0000 7f3d3b81d700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d3414f150 0x7f3d3414a880 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f3d24006ef0 tx=0x7f3d24015000).stop 2020-11-20T12:16:18.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.376+0000 7f3d3b81d700 1 -- 172.21.15.116:0/4180572762 shutdown_connections 2020-11-20T12:16:18.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.376+0000 7f3d3b81d700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d0c0557c0 0x7f3d0c057c80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.376+0000 7f3d3b81d700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d3414f530 0x7f3d34157750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.376+0000 7f3d3b81d700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d340bd780 0x7f3d340ad190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.376+0000 7f3d3b81d700 1 --2- 172.21.15.116:0/4180572762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d3414f150 0x7f3d3414a880 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.376+0000 7f3d3b81d700 1 -- 172.21.15.116:0/4180572762 >> 172.21.15.116:0/4180572762 conn(0x7f3d34061580 msgr2=0x7f3d3414f990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.376+0000 7f3d3b81d700 1 -- 172.21.15.116:0/4180572762 shutdown_connections 2020-11-20T12:16:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.376+0000 7f3d3b81d700 1 -- 172.21.15.116:0/4180572762 wait complete. 2020-11-20T12:16:18.400 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:16:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.584+0000 7fc2a562a700 1 Processor -- start 2020-11-20T12:16:18.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.584+0000 7fc2a562a700 1 -- start start 2020-11-20T12:16:18.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.584+0000 7fc2a562a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 0x7fc2a00673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.584+0000 7fc2a562a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc2a01538d0 0x7fc2a014fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.584+0000 7fc2a562a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2a00683c0 con 0x7fc2a0065fb0 2020-11-20T12:16:18.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.584+0000 7fc2a562a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2a0153d80 con 0x7fc2a01538d0 2020-11-20T12:16:18.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.584+0000 7fc2a562a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a0066ed0 con 0x7fc2a014f460 2020-11-20T12:16:18.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29e7fc700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fc2a0065fb0 0x7fc2a00a2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49162/0 (socket says 172.21.15.116:49162) 2020-11-20T12:16:18.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29e7fc700 1 -- 172.21.15.116:0/3318130046 learned_addr learned my addr 172.21.15.116:0/3318130046 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:18.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29effd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 0x7fc2a00673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29f7fe700 1 --2- 172.21.15.116:0/3318130046 >> v2:172.21.15.173:3300/0 conn(0x7fc2a01538d0 0x7fc2a014fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29dffb700 1 -- 172.21.15.116:0/3318130046 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2648929886 0 0) 0x7fc2a00683c0 con 0x7fc2a0065fb0 2020-11-20T12:16:18.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29dffb700 1 -- 172.21.15.116:0/3318130046 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc28c003240 con 0x7fc2a0065fb0 2020-11-20T12:16:18.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29effd700 1 -- 172.21.15.116:0/3318130046 >> v2:172.21.15.173:3300/0 conn(0x7fc2a01538d0 msgr2=0x7fc2a014fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:18.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29effd700 1 --2- 172.21.15.116:0/3318130046 >> v2:172.21.15.173:3300/0 conn(0x7fc2a01538d0 0x7fc2a014fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29effd700 1 -- 172.21.15.116:0/3318130046 >> v1:172.21.15.173:6789/0 conn(0x7fc2a0065fb0 legacy=0x7fc2a00a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29effd700 1 -- 172.21.15.116:0/3318130046 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a014f840 con 0x7fc2a014f460 2020-11-20T12:16:18.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29effd700 1 --2- 172.21.15.116:0/3318130046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 0x7fc2a00673b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc29000b9a0 tx=0x7fc290006df0).ready entity=mon.2 client_cookie=9654347a8b3440a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:18.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29dffb700 1 -- 172.21.15.116:0/3318130046 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290009680 con 0x7fc2a014f460 2020-11-20T12:16:18.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29dffb700 1 -- 172.21.15.116:0/3318130046 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2900097e0 con 0x7fc2a014f460 2020-11-20T12:16:18.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc29dffb700 1 -- 172.21.15.116:0/3318130046 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc290009b60 con 0x7fc2a014f460 2020-11-20T12:16:18.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc2a562a700 1 -- 172.21.15.116:0/3318130046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 msgr2=0x7fc2a00673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc2a562a700 1 --2- 172.21.15.116:0/3318130046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 0x7fc2a00673b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc29000b9a0 tx=0x7fc290006df0).stop 2020-11-20T12:16:18.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc2a562a700 1 -- 172.21.15.116:0/3318130046 shutdown_connections 2020-11-20T12:16:18.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc2a562a700 1 --2- 172.21.15.116:0/3318130046 >> v2:172.21.15.173:3300/0 conn(0x7fc2a01538d0 0x7fc2a014fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc2a562a700 1 --2- 172.21.15.116:0/3318130046 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 0x7fc2a00673b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.588+0000 7fc2a562a700 1 -- 172.21.15.116:0/3318130046 >> 172.21.15.116:0/3318130046 conn(0x7fc2a0060c30 msgr2=0x7fc2a00c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:18.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.592+0000 7fc2a562a700 1 -- 172.21.15.116:0/3318130046 shutdown_connections 2020-11-20T12:16:18.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.592+0000 7fc2a562a700 1 -- 172.21.15.116:0/3318130046 wait complete. 2020-11-20T12:16:18.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.592+0000 7fc2a562a700 1 Processor -- start 2020-11-20T12:16:18.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.592+0000 7fc2a562a700 1 -- start start 2020-11-20T12:16:18.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc2a562a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a0065fb0 0x7fc2a0144b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc2a562a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 0x7fc2a00bd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc2a562a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2a01538d0 0x7fc2a008fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc2a562a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a008ebb0 con 0x7fc2a01538d0 2020-11-20T12:16:18.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc2a562a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a008ee50 con 0x7fc2a0065fb0 2020-11-20T12:16:18.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc2a562a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2a008f0f0 con 0x7fc2a014f460 2020-11-20T12:16:18.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2a01538d0 0x7fc2a008fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29e7fc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 0x7fc2a00bd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2a01538d0 0x7fc2a008fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43506/0 (socket says 172.21.15.116:43506) 2020-11-20T12:16:18.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29f7fe700 1 -- 172.21.15.116:0/1010238888 learned_addr learned my addr 172.21.15.116:0/1010238888 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:18.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29effd700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a0065fb0 0x7fc2a0144b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29e7fc700 1 -- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a0065fb0 msgr2=0x7fc2a0144b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29e7fc700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a0065fb0 0x7fc2a0144b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29e7fc700 1 -- 172.21.15.116:0/1010238888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2a01538d0 msgr2=0x7fc2a008fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29e7fc700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2a01538d0 0x7fc2a008fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29e7fc700 1 -- 172.21.15.116:0/1010238888 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a008f390 con 0x7fc2a014f460 2020-11-20T12:16:18.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29f7fe700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2a01538d0 0x7fc2a008fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc29e7fc700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 0x7fc2a00bd340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fc2a0065430 tx=0x7fc294007a20).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc286ffd700 1 -- 172.21.15.116:0/1010238888 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2940072e0 con 0x7fc2a014f460 2020-11-20T12:16:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc286ffd700 1 -- 172.21.15.116:0/1010238888 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc294007440 con 0x7fc2a014f460 2020-11-20T12:16:18.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc2a562a700 1 -- 172.21.15.116:0/1010238888 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2a008f630 con 0x7fc2a014f460 2020-11-20T12:16:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc286ffd700 1 -- 172.21.15.116:0/1010238888 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc294005820 con 0x7fc2a014f460 2020-11-20T12:16:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.596+0000 7fc2a562a700 1 -- 172.21.15.116:0/1010238888 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a00637b0 con 0x7fc2a014f460 2020-11-20T12:16:18.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.600+0000 7fc2a562a700 1 -- 172.21.15.116:0/1010238888 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2a0044f60 con 0x7fc2a014f460 2020-11-20T12:16:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.600+0000 7fc286ffd700 1 -- 172.21.15.116:0/1010238888 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc29401a060 con 0x7fc2a014f460 2020-11-20T12:16:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.600+0000 7fc286ffd700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc27c0557c0 0x7fc27c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.604+0000 7fc286ffd700 1 -- 172.21.15.116:0/1010238888 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc29406e0a0 con 0x7fc2a014f460 2020-11-20T12:16:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.604+0000 7fc29effd700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc27c0557c0 0x7fc27c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.604+0000 7fc29effd700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc27c0557c0 0x7fc27c057c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fc290006e30 tx=0x7fc290015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:18.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.612+0000 7fc286ffd700 1 -- 172.21.15.116:0/1010238888 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc294043eb0 con 0x7fc2a014f460 2020-11-20T12:16:18.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.772+0000 7fc2a562a700 1 -- 172.21.15.116:0/1010238888 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc2a00c0690 con 0x7fc2a014f460 2020-11-20T12:16:18.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.772+0000 7fc286ffd700 1 -- 172.21.15.116:0/1010238888 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7fc294018070 con 0x7fc2a014f460 2020-11-20T12:16:18.776 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:18.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 -- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc27c0557c0 msgr2=0x7fc27c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc27c0557c0 0x7fc27c057c80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fc290006e30 tx=0x7fc290015000).stop 2020-11-20T12:16:18.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 -- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 msgr2=0x7fc2a00bd340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 0x7fc2a00bd340 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fc2a0065430 tx=0x7fc294007a20).stop 2020-11-20T12:16:18.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 -- 172.21.15.116:0/1010238888 shutdown_connections 2020-11-20T12:16:18.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc27c0557c0 0x7fc27c057c80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2a01538d0 0x7fc2a008fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2a0065fb0 0x7fc2a0144b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 --2- 172.21.15.116:0/1010238888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2a014f460 0x7fc2a00bd340 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 -- 172.21.15.116:0/1010238888 >> 172.21.15.116:0/1010238888 conn(0x7fc2a0060c30 msgr2=0x7fc2a00c32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:18.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 -- 172.21.15.116:0/1010238888 shutdown_connections 2020-11-20T12:16:18.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.776+0000 7fc2a562a700 1 -- 172.21.15.116:0/1010238888 wait complete. 2020-11-20T12:16:18.802 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:16:18.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.984+0000 7f771cdbe700 1 Processor -- start 2020-11-20T12:16:18.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.984+0000 7f771cdbe700 1 -- start start 2020-11-20T12:16:18.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f771cdbe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7718066b70 0x7f77180a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f771cdbe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7718066ff0 0x7f77180a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f771cdbe700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771814fc40 con 0x7f771814f860 2020-11-20T12:16:18.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f771cdbe700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77180c0590 con 0x7f7718066b70 2020-11-20T12:16:18.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f771cdbe700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77180c0830 con 0x7f7718066ff0 2020-11-20T12:16:18.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f7715d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7718066b70 0x7f77180a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f7715d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7718066b70 0x7f77180a5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43512/0 (socket says 172.21.15.116:43512) 2020-11-20T12:16:18.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f7715d9b700 1 -- 172.21.15.116:0/4070747755 learned_addr learned my addr 172.21.15.116:0/4070747755 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:18.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f771659c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f771814f860 0x7f77180c14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52944/0 (socket says 172.21.15.116:52944) 2020-11-20T12:16:18.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f7716d9d700 1 --2- 172.21.15.116:0/4070747755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7718066ff0 0x7f77180a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f7715d9b700 1 -- 172.21.15.116:0/4070747755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7718066ff0 msgr2=0x7f77180a64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:18.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f7715d9b700 1 --2- 172.21.15.116:0/4070747755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7718066ff0 0x7f77180a64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f7715d9b700 1 -- 172.21.15.116:0/4070747755 >> v1:172.21.15.116:6789/0 conn(0x7f771814f860 legacy=0x7f77180c14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f7715d9b700 1 -- 172.21.15.116:0/4070747755 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77180c0ad0 con 0x7f7718066b70 2020-11-20T12:16:18.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f7715d9b700 1 --2- 172.21.15.116:0/4070747755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7718066b70 0x7f77180a5fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f770c006fd0 tx=0x7f770c006eb0).ready entity=mon.0 client_cookie=dd510c4df2bcec0d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:18.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f771559a700 1 -- 172.21.15.116:0/4070747755 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c0098d0 con 0x7f7718066b70 2020-11-20T12:16:18.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f771559a700 1 -- 172.21.15.116:0/4070747755 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f770c009a30 con 0x7f7718066b70 2020-11-20T12:16:18.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f771559a700 1 -- 172.21.15.116:0/4070747755 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f770c009db0 con 0x7f7718066b70 2020-11-20T12:16:18.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f771cdbe700 1 -- 172.21.15.116:0/4070747755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7718066b70 msgr2=0x7f77180a5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:18.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.988+0000 7f771cdbe700 1 --2- 172.21.15.116:0/4070747755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7718066b70 0x7f77180a5fb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f770c006fd0 tx=0x7f770c006eb0).stop 2020-11-20T12:16:18.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.992+0000 7f771cdbe700 1 -- 172.21.15.116:0/4070747755 shutdown_connections 2020-11-20T12:16:18.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.992+0000 7f771cdbe700 1 --2- 172.21.15.116:0/4070747755 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7718066ff0 0x7f77180a64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.992+0000 7f771cdbe700 1 --2- 172.21.15.116:0/4070747755 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7718066b70 0x7f77180a5fb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:18.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.992+0000 7f771cdbe700 1 -- 172.21.15.116:0/4070747755 >> 172.21.15.116:0/4070747755 conn(0x7f7718060170 msgr2=0x7f7718060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:18.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.992+0000 7f771cdbe700 1 -- 172.21.15.116:0/4070747755 shutdown_connections 2020-11-20T12:16:18.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.992+0000 7f771cdbe700 1 -- 172.21.15.116:0/4070747755 wait complete. 2020-11-20T12:16:18.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.992+0000 7f771cdbe700 1 Processor -- start 2020-11-20T12:16:18.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.992+0000 7f771cdbe700 1 -- start start 2020-11-20T12:16:18.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771cdbe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7718066b70 0x7f77180a3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771cdbe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7718066ff0 0x7f77180a27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771cdbe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f771814f860 0x7f771814af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:18.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771cdbe700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77180a1bd0 con 0x7f7718066b70 2020-11-20T12:16:18.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771cdbe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77180bdd30 con 0x7f771814f860 2020-11-20T12:16:18.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771cdbe700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771814fde0 con 0x7f7718066ff0 2020-11-20T12:16:18.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7718066b70 0x7f77180a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:18.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7718066b70 0x7f77180a3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43516/0 (socket says 172.21.15.116:43516) 2020-11-20T12:16:18.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771659c700 1 -- 172.21.15.116:0/1556211000 learned_addr learned my addr 172.21.15.116:0/1556211000 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:18.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f7716d9d700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f771814f860 0x7f771814af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:19.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f7715d9b700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7718066ff0 0x7f77180a27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:19.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771659c700 1 -- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7718066ff0 msgr2=0x7f77180a27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:19.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771659c700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7718066ff0 0x7f77180a27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771659c700 1 -- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f771814f860 msgr2=0x7f771814af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:19.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771659c700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f771814f860 0x7f771814af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771659c700 1 -- 172.21.15.116:0/1556211000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718150080 con 0x7f7718066b70 2020-11-20T12:16:19.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771659c700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7718066b70 0x7f77180a3640 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f770800c710 tx=0x7f7708007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:19.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f77067fc700 1 -- 172.21.15.116:0/1556211000 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080072e0 con 0x7f7718066b70 2020-11-20T12:16:19.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f77067fc700 1 -- 172.21.15.116:0/1556211000 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7708005530 con 0x7f7718066b70 2020-11-20T12:16:19.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f77067fc700 1 -- 172.21.15.116:0/1556211000 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080058b0 con 0x7f7718066b70 2020-11-20T12:16:19.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771cdbe700 1 -- 172.21.15.116:0/1556211000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7718150320 con 0x7f7718066b70 2020-11-20T12:16:19.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:18.996+0000 7f771cdbe700 1 -- 172.21.15.116:0/1556211000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77180a2d20 con 0x7f7718066b70 2020-11-20T12:16:19.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.000+0000 7f771cdbe700 1 -- 172.21.15.116:0/1556211000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f771805ca30 con 0x7f7718066b70 2020-11-20T12:16:19.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.000+0000 7f77067fc700 1 -- 172.21.15.116:0/1556211000 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7708010070 con 0x7f7718066b70 2020-11-20T12:16:19.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.000+0000 7f77067fc700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76ec0557c0 0x7f76ec057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:19.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.000+0000 7f77067fc700 1 -- 172.21.15.116:0/1556211000 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f770806dcd0 con 0x7f7718066b70 2020-11-20T12:16:19.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.004+0000 7f7715d9b700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76ec0557c0 0x7f76ec057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:19.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.004+0000 7f7715d9b700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76ec0557c0 0x7f76ec057c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f77180bc530 tx=0x7f770c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:19.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.008+0000 7f77067fc700 1 -- 172.21.15.116:0/1556211000 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7708043a60 con 0x7f7718066b70 2020-11-20T12:16:19.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.172+0000 7f771cdbe700 1 -- 172.21.15.116:0/1556211000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f771814fc40 con 0x7f7718066b70 2020-11-20T12:16:19.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.172+0000 7f77067fc700 1 -- 172.21.15.116:0/1556211000 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f771814fc40 con 0x7f7718066b70 2020-11-20T12:16:19.176 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 -- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76ec0557c0 msgr2=0x7f76ec057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76ec0557c0 0x7f76ec057c80 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f77180bc530 tx=0x7f770c015000).stop 2020-11-20T12:16:19.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 -- 172.21.15.116:0/1556211000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7718066b70 msgr2=0x7f77180a3640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:19.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7718066b70 0x7f77180a3640 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f770800c710 tx=0x7f7708007c40).stop 2020-11-20T12:16:19.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 -- 172.21.15.116:0/1556211000 shutdown_connections 2020-11-20T12:16:19.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76ec0557c0 0x7f76ec057c80 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f771814f860 0x7f771814af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7718066b70 0x7f77180a3640 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 --2- 172.21.15.116:0/1556211000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7718066ff0 0x7f77180a27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 -- 172.21.15.116:0/1556211000 >> 172.21.15.116:0/1556211000 conn(0x7f7718060170 msgr2=0x7f77180c3350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:19.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 -- 172.21.15.116:0/1556211000 shutdown_connections 2020-11-20T12:16:19.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.176+0000 7f771cdbe700 1 -- 172.21.15.116:0/1556211000 wait complete. 2020-11-20T12:16:19.202 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:16:19.203 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:16:19.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.372+0000 7ff152411700 1 Processor -- start 2020-11-20T12:16:19.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.372+0000 7ff152411700 1 -- start start 2020-11-20T12:16:19.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.372+0000 7ff152411700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff14c13f1d0 0x7ff14c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:19.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.372+0000 7ff152411700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff14c0a40c0 0x7ff14c0c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:19.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.372+0000 7ff152411700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff14c0a5680 con 0x7ff14c13f5b0 2020-11-20T12:16:19.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.372+0000 7ff152411700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff14c0a58e0 con 0x7ff14c0a40c0 2020-11-20T12:16:19.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.372+0000 7ff152411700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff14c0a5b80 con 0x7ff14c13f1d0 2020-11-20T12:16:19.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.372+0000 7ff150c0e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff14c0a40c0 0x7ff14c0c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:19.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.372+0000 7ff151c10700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7ff14c13f5b0 0x7ff14c1594c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49190/0 (socket says 172.21.15.116:49190) 2020-11-20T12:16:19.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.372+0000 7ff151c10700 1 -- 172.21.15.116:0/2351102717 learned_addr learned my addr 172.21.15.116:0/2351102717 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:19.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.372+0000 7ff15140f700 1 --2- 172.21.15.116:0/2351102717 >> v2:172.21.15.173:3300/0 conn(0x7ff14c13f1d0 0x7ff14c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:19.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff13bfff700 1 -- 172.21.15.116:0/2351102717 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245023657 0 0) 0x7ff14c0a5680 con 0x7ff14c13f5b0 2020-11-20T12:16:19.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff150c0e700 1 -- 172.21.15.116:0/2351102717 >> v2:172.21.15.173:3300/0 conn(0x7ff14c13f1d0 msgr2=0x7ff14c0a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:19.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff150c0e700 1 --2- 172.21.15.116:0/2351102717 >> v2:172.21.15.173:3300/0 conn(0x7ff14c13f1d0 0x7ff14c0a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff150c0e700 1 -- 172.21.15.116:0/2351102717 >> v1:172.21.15.173:6789/0 conn(0x7ff14c13f5b0 legacy=0x7ff14c1594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:19.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff150c0e700 1 -- 172.21.15.116:0/2351102717 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14c0a5e20 con 0x7ff14c0a40c0 2020-11-20T12:16:19.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff150c0e700 1 --2- 172.21.15.116:0/2351102717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14c0a40c0 0x7ff14c0c3fb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7ff14c065960 tx=0x7ff13c006df0).ready entity=mon.0 client_cookie=ce42d99754aed05b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:19.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff13bfff700 1 -- 172.21.15.116:0/2351102717 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c0097b0 con 0x7ff14c0a40c0 2020-11-20T12:16:19.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff13bfff700 1 -- 172.21.15.116:0/2351102717 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c009910 con 0x7ff14c0a40c0 2020-11-20T12:16:19.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff13bfff700 1 -- 172.21.15.116:0/2351102717 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c009c90 con 0x7ff14c0a40c0 2020-11-20T12:16:19.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff152411700 1 -- 172.21.15.116:0/2351102717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14c0a40c0 msgr2=0x7ff14c0c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:19.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff152411700 1 --2- 172.21.15.116:0/2351102717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14c0a40c0 0x7ff14c0c3fb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7ff14c065960 tx=0x7ff13c006df0).stop 2020-11-20T12:16:19.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff152411700 1 -- 172.21.15.116:0/2351102717 shutdown_connections 2020-11-20T12:16:19.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff152411700 1 --2- 172.21.15.116:0/2351102717 >> v2:172.21.15.173:3300/0 conn(0x7ff14c13f1d0 0x7ff14c0a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff152411700 1 --2- 172.21.15.116:0/2351102717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14c0a40c0 0x7ff14c0c3fb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff152411700 1 -- 172.21.15.116:0/2351102717 >> 172.21.15.116:0/2351102717 conn(0x7ff14c062520 msgr2=0x7ff14c062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:19.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff152411700 1 -- 172.21.15.116:0/2351102717 shutdown_connections 2020-11-20T12:16:19.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.376+0000 7ff152411700 1 -- 172.21.15.116:0/2351102717 wait complete. 2020-11-20T12:16:19.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff152411700 1 Processor -- start 2020-11-20T12:16:19.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff152411700 1 -- start start 2020-11-20T12:16:19.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff152411700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff14c13f1d0 0x7ff14c0a9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:19.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff152411700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff14c13f5b0 0x7ff14c156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:19.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff152411700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14c165170 0x7ff14c0a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:19.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff152411700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c156f80 con 0x7ff14c165170 2020-11-20T12:16:19.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff152411700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c1571a0 con 0x7ff14c13f1d0 2020-11-20T12:16:19.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff152411700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c157440 con 0x7ff14c13f5b0 2020-11-20T12:16:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff151c10700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14c165170 0x7ff14c0a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff151c10700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14c165170 0x7ff14c0a2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43530/0 (socket says 172.21.15.116:43530) 2020-11-20T12:16:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff151c10700 1 -- 172.21.15.116:0/1510036992 learned_addr learned my addr 172.21.15.116:0/1510036992 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff15140f700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff14c13f1d0 0x7ff14c0a9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.380+0000 7ff150c0e700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff14c13f5b0 0x7ff14c156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff151c10700 1 -- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff14c13f5b0 msgr2=0x7ff14c156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff151c10700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff14c13f5b0 0x7ff14c156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff151c10700 1 -- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff14c13f1d0 msgr2=0x7ff14c0a9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:19.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff151c10700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff14c13f1d0 0x7ff14c0a9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff151c10700 1 -- 172.21.15.116:0/1510036992 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff14c1576e0 con 0x7ff14c165170 2020-11-20T12:16:19.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff151c10700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14c165170 0x7ff14c0a2e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7ff14400cae0 tx=0x7ff144007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:19.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff1397fa700 1 -- 172.21.15.116:0/1510036992 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144005b80 con 0x7ff14c165170 2020-11-20T12:16:19.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff152411700 1 -- 172.21.15.116:0/1510036992 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14c157980 con 0x7ff14c165170 2020-11-20T12:16:19.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff1397fa700 1 -- 172.21.15.116:0/1510036992 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff144005ce0 con 0x7ff14c165170 2020-11-20T12:16:19.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff1397fa700 1 -- 172.21.15.116:0/1510036992 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144007340 con 0x7ff14c165170 2020-11-20T12:16:19.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff152411700 1 -- 172.21.15.116:0/1510036992 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14c0a3300 con 0x7ff14c165170 2020-11-20T12:16:19.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff152411700 1 -- 172.21.15.116:0/1510036992 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff14c044f60 con 0x7ff14c165170 2020-11-20T12:16:19.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.384+0000 7ff1397fa700 1 -- 172.21.15.116:0/1510036992 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff144010030 con 0x7ff14c165170 2020-11-20T12:16:19.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.388+0000 7ff1397fa700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1240557f0 0x7ff124057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:19.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.388+0000 7ff1397fa700 1 -- 172.21.15.116:0/1510036992 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff14406ec20 con 0x7ff14c165170 2020-11-20T12:16:19.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.388+0000 7ff15140f700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1240557f0 0x7ff124057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:19.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.388+0000 7ff15140f700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1240557f0 0x7ff124057cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff148006fd0 tx=0x7ff148008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:19.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.388+0000 7ff1397fa700 1 -- 172.21.15.116:0/1510036992 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff144044b40 con 0x7ff14c165170 2020-11-20T12:16:19.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.596+0000 7ff152411700 1 -- 172.21.15.116:0/1510036992 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff14c061580 con 0x7ff14c165170 2020-11-20T12:16:19.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.596+0000 7ff1397fa700 1 -- 172.21.15.116:0/1510036992 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2846 (secure 0 0 0) 0x7ff144048160 con 0x7ff14c165170 2020-11-20T12:16:19.598 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:19.599 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/2968741298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":2968741298},{"type":"v1","addr":"172.21.15.116:6835","nonce":2968741298}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1214743116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1214743116},{"type":"v1","addr":"172.21.15.173:6836","nonce":1214743116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1424291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1424291148},{"type":"v1","addr":"172.21.15.173:6837","nonce":1424291148}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:00.619934+0000","modified":"2020-11-20T12:16:02.636079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.116:6834/3819756176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3819756176},{"type":"v1","addr":"172.21.15.116:6834","nonce":3819756176}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:16:19.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 -- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1240557f0 msgr2=0x7ff124057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:19.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1240557f0 0x7ff124057cb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff148006fd0 tx=0x7ff148008040).stop 2020-11-20T12:16:19.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 -- 172.21.15.116:0/1510036992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14c165170 msgr2=0x7ff14c0a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:19.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14c165170 0x7ff14c0a2e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7ff14400cae0 tx=0x7ff144007f70).stop 2020-11-20T12:16:19.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 -- 172.21.15.116:0/1510036992 shutdown_connections 2020-11-20T12:16:19.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1240557f0 0x7ff124057cb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff14c165170 0x7ff14c0a2e10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff14c13f1d0 0x7ff14c0a9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 --2- 172.21.15.116:0/1510036992 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff14c13f5b0 0x7ff14c156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:19.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 -- 172.21.15.116:0/1510036992 >> 172.21.15.116:0/1510036992 conn(0x7ff14c062520 msgr2=0x7ff14c0675b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:19.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 -- 172.21.15.116:0/1510036992 shutdown_connections 2020-11-20T12:16:19.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:19.600+0000 7ff152411700 1 -- 172.21.15.116:0/1510036992 wait complete. 2020-11-20T12:16:19.604 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 6 2020-11-20T12:16:20.027 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:00.619934+0000', 'modified': '2020-11-20T12:16:02.636079+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4245}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4245': {'gid': 4245, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.116:6834/3819756176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 3819756176}, {'type': 'v1', 'addr': '172.21.15.116:6834', 'nonce': 3819756176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:16:20.028 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:16:20.028 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:16:20.220 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4245,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":19.578186839000001,"mdsmap_epoch":6,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":23.06601745} 2020-11-20T12:16:20.220 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4245, "mdsmap_epoch": 6, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 19.578186839, "state": "up:active", "uptime": 23.06601745, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:16:20.221 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:16:20.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.222 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-20T12:16:20.238 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:16:20.238 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:20.311 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:20.311 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:16:20.414 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:20.414 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:16:20.419 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:16:20.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.420 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-20T12:16:20.451 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:16:20.451 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:16:20.452 INFO:tasks.cephfs.mount:self.client.name = client.2 2020-11-20T12:16:20.452 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2020-11-20T12:16:20.453 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:16:20.453 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2020-11-20T12:16:20.453 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:16:20.453 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:16:20.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.454 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:16:20.468 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:16:20.468 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:16:20.469 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:16:20.469 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.469 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:16:20.469 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.469 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:20.470 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:20.470 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:20.470 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:20.470 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:20.470 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:20.471 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:16:20.471 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:20.471 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:16:20.471 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.471 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:16:20.472 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.472 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:16:20.472 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:20.472 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:16:20.472 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.473 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:16:20.473 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:20.473 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:16:20.473 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.474 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:16:20.474 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.474 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:16:20.474 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:16:20.474 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:16:20.475 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.476 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:16:20.508 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:16:20.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.512 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:16:20.573 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:16:20.574 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:16:20.574 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:16:20.574 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:16:20.575 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:16:20.575 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:16:20.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.576 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:20.577 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:16:20.577 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:16:20.577 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:16:20.578 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:16:20.578 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:16:20.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.682 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:16:20.696 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:16:20.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.698 INFO:teuthology.orchestra.run.smithi116:> ip netns list-id 2020-11-20T12:16:20.760 INFO:teuthology.orchestra.run.smithi116.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2020-11-20T12:16:20.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.762 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:20.762 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:16:20.763 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2020-11-20T12:16:20.763 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2020-11-20T12:16:20.763 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:16:20.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.844 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2020-11-20T12:16:20.874 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:16:20.874 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:16:20.874 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:16:20.875 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.875 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:16:20.875 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.875 INFO:teuthology.orchestra.run.smithi116.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:16:20.875 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether d6:75:c1:a8:ef:cc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:16:20.876 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-11-20T12:16:20.876 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.876 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::d475:c1ff:fea8:efcc/64 scope link 2020-11-20T12:16:20.876 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.878 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2020-11-20T12:16:20.952 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:16:20.953 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:16:20.953 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:16:20.953 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.953 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:16:20.954 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.954 INFO:teuthology.orchestra.run.smithi116.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:16:20.954 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether d6:75:c1:a8:ef:cc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:16:20.954 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-11-20T12:16:20.955 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.955 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::d475:c1ff:fea8:efcc/64 scope link 2020-11-20T12:16:20.955 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:20.957 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2020-11-20T12:16:20.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:20.958 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:20.958 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:16:20.958 INFO:teuthology.orchestra.run.smithi116:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2020-11-20T12:16:20.959 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2020-11-20T12:16:20.959 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2020-11-20T12:16:20.959 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2020-11-20T12:16:20.960 INFO:teuthology.orchestra.run.smithi116:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2020-11-20T12:16:20.960 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:16:21.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:21.104 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:21.104 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:16:21.105 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2020-11-20T12:16:21.105 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set dev brx.1 master ceph-brx 2020-11-20T12:16:21.105 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:16:21.180 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:21.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.356+0000 7f9cf6455700 1 Processor -- start 2020-11-20T12:16:21.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.356+0000 7f9cf6455700 1 -- start start 2020-11-20T12:16:21.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9cf6455700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9cf00baa30 0x7f9cf0066510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:21.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9cf6455700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cf00c27e0 0x7f9cf0066a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:21.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9cf6455700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cf00c2da0 con 0x7f9cf00bae10 2020-11-20T12:16:21.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9cf6455700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf00c3040 con 0x7f9cf00baa30 2020-11-20T12:16:21.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9cf6455700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00c32e0 con 0x7f9cf00c27e0 2020-11-20T12:16:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9ceffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9cf00baa30 0x7f9cf0066510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:21.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9ceffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9cf00baa30 0x7f9cf0066510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43544/0 (socket says 172.21.15.116:43544) 2020-11-20T12:16:21.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9ceffff700 1 -- 172.21.15.116:0/719659014 learned_addr learned my addr 172.21.15.116:0/719659014 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:21.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9cf49f2700 1 --2- 172.21.15.116:0/719659014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cf00c27e0 0x7f9cf0066a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:21.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9cef7fe700 1 -- 172.21.15.116:0/719659014 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 157740356 0 0) 0x7f9cf00c2da0 con 0x7f9cf00bae10 2020-11-20T12:16:21.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9ceffff700 1 -- 172.21.15.116:0/719659014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cf00c27e0 msgr2=0x7f9cf0066a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:21.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9ceffff700 1 --2- 172.21.15.116:0/719659014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cf00c27e0 0x7f9cf0066a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9ceffff700 1 -- 172.21.15.116:0/719659014 >> v1:172.21.15.173:6789/0 conn(0x7f9cf00bae10 legacy=0x7f9cf00c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9ceffff700 1 -- 172.21.15.116:0/719659014 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf00c3580 con 0x7f9cf00baa30 2020-11-20T12:16:21.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9cf49f2700 1 --2- 172.21.15.116:0/719659014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cf00c27e0 0x7f9cf0066a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:21.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9ceffff700 1 --2- 172.21.15.116:0/719659014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cf00baa30 0x7f9cf0066510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f9ce0006fd0 tx=0x7f9ce0006eb0).ready entity=mon.0 client_cookie=b3bb5bedd4362259 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:21.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9cef7fe700 1 -- 172.21.15.116:0/719659014 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00098d0 con 0x7f9cf00baa30 2020-11-20T12:16:21.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9cef7fe700 1 -- 172.21.15.116:0/719659014 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce0009a30 con 0x7f9cf00baa30 2020-11-20T12:16:21.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.360+0000 7f9cef7fe700 1 -- 172.21.15.116:0/719659014 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0009db0 con 0x7f9cf00baa30 2020-11-20T12:16:21.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.364+0000 7f9cf6455700 1 -- 172.21.15.116:0/719659014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cf00baa30 msgr2=0x7f9cf0066510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.364+0000 7f9cf6455700 1 --2- 172.21.15.116:0/719659014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cf00baa30 0x7f9cf0066510 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f9ce0006fd0 tx=0x7f9ce0006eb0).stop 2020-11-20T12:16:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.364+0000 7f9cf6455700 1 -- 172.21.15.116:0/719659014 shutdown_connections 2020-11-20T12:16:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.364+0000 7f9cf6455700 1 --2- 172.21.15.116:0/719659014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cf00c27e0 0x7f9cf0066a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.364+0000 7f9cf6455700 1 --2- 172.21.15.116:0/719659014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cf00baa30 0x7f9cf0066510 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.364+0000 7f9cf6455700 1 -- 172.21.15.116:0/719659014 >> 172.21.15.116:0/719659014 conn(0x7f9cf0060c80 msgr2=0x7f9cf00c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.364+0000 7f9cf6455700 1 -- 172.21.15.116:0/719659014 shutdown_connections 2020-11-20T12:16:21.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.364+0000 7f9cf6455700 1 -- 172.21.15.116:0/719659014 wait complete. 2020-11-20T12:16:21.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.364+0000 7f9cf6455700 1 Processor -- start 2020-11-20T12:16:21.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.364+0000 7f9cf6455700 1 -- start start 2020-11-20T12:16:21.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.364+0000 7f9cf6455700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cf00bae10 0x7f9cf00a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:21.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9cf6455700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cf00c27e0 0x7f9cf014f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:21.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9cf6455700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cf0164830 0x7f9cf0164c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:21.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9cf6455700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0165290 con 0x7f9cf00c27e0 2020-11-20T12:16:21.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9cf6455700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf0165530 con 0x7f9cf0164830 2020-11-20T12:16:21.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9cf6455700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf00a9570 con 0x7f9cf00bae10 2020-11-20T12:16:21.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9ce77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cf00c27e0 0x7f9cf014f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9ce77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cf00c27e0 0x7f9cf014f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43548/0 (socket says 172.21.15.116:43548) 2020-11-20T12:16:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9ce77fe700 1 -- 172.21.15.116:0/2088729523 learned_addr learned my addr 172.21.15.116:0/2088729523 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9cf49f2700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cf0164830 0x7f9cf0164c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9ceffff700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cf00bae10 0x7f9cf00a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9ce77fe700 1 -- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cf00bae10 msgr2=0x7f9cf00a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9ce77fe700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cf00bae10 0x7f9cf00a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9ce77fe700 1 -- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cf0164830 msgr2=0x7f9cf0164c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9ce77fe700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cf0164830 0x7f9cf0164c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9ce77fe700 1 -- 172.21.15.116:0/2088729523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf00a9810 con 0x7f9cf00c27e0 2020-11-20T12:16:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9ce77fe700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cf00c27e0 0x7f9cf014f0c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f9cd8007fd0 tx=0x7f9cd8007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:21.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9cecff9700 1 -- 172.21.15.116:0/2088729523 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd800e560 con 0x7f9cf00c27e0 2020-11-20T12:16:21.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9cecff9700 1 -- 172.21.15.116:0/2088729523 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd800e6e0 con 0x7f9cf00c27e0 2020-11-20T12:16:21.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9cf6455700 1 -- 172.21.15.116:0/2088729523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf00aa1a0 con 0x7f9cf00c27e0 2020-11-20T12:16:21.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9cecff9700 1 -- 172.21.15.116:0/2088729523 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd800eaa0 con 0x7f9cf00c27e0 2020-11-20T12:16:21.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.368+0000 7f9cf6455700 1 -- 172.21.15.116:0/2088729523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf00c03a0 con 0x7f9cf00c27e0 2020-11-20T12:16:21.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.372+0000 7f9cf6455700 1 -- 172.21.15.116:0/2088729523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf01656f0 con 0x7f9cf00c27e0 2020-11-20T12:16:21.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.372+0000 7f9cecff9700 1 -- 172.21.15.116:0/2088729523 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9cd8019030 con 0x7f9cf00c27e0 2020-11-20T12:16:21.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.376+0000 7f9cecff9700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ccc0557c0 0x7f9ccc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:21.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.376+0000 7f9cecff9700 1 -- 172.21.15.116:0/2088729523 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9cd806e420 con 0x7f9cf00c27e0 2020-11-20T12:16:21.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.376+0000 7f9ceffff700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ccc0557c0 0x7f9ccc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:21.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.376+0000 7f9ceffff700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ccc0557c0 0x7f9ccc057c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f9cf00610c0 tx=0x7f9ce0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:21.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.376+0000 7f9cecff9700 1 -- 172.21.15.116:0/2088729523 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9cd8044230 con 0x7f9cf00c27e0 2020-11-20T12:16:21.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.540+0000 7f9cf6455700 1 -- 172.21.15.116:0/2088729523 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cf00a93b0 con 0x7f9cf00c27e0 2020-11-20T12:16:21.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cecff9700 1 -- 172.21.15.116:0/2088729523 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f9cd8047850 con 0x7f9cf00c27e0 2020-11-20T12:16:21.544 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:21.544 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:21.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 -- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ccc0557c0 msgr2=0x7f9ccc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ccc0557c0 0x7f9ccc057c80 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f9cf00610c0 tx=0x7f9ce0015000).stop 2020-11-20T12:16:21.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 -- 172.21.15.116:0/2088729523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cf00c27e0 msgr2=0x7f9cf014f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cf00c27e0 0x7f9cf014f0c0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f9cd8007fd0 tx=0x7f9cd8007ac0).stop 2020-11-20T12:16:21.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 -- 172.21.15.116:0/2088729523 shutdown_connections 2020-11-20T12:16:21.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ccc0557c0 0x7f9ccc057c80 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cf0164830 0x7f9cf0164c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cf00bae10 0x7f9cf00a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 --2- 172.21.15.116:0/2088729523 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cf00c27e0 0x7f9cf014f0c0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 -- 172.21.15.116:0/2088729523 >> 172.21.15.116:0/2088729523 conn(0x7f9cf0060c80 msgr2=0x7f9cf0137630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:21.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 -- 172.21.15.116:0/2088729523 shutdown_connections 2020-11-20T12:16:21.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.544+0000 7f9cf6455700 1 -- 172.21.15.116:0/2088729523 wait complete. 2020-11-20T12:16:21.572 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:15:59.065285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:15:59.602311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:16:21.573 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:21.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.752+0000 7fa8a277b700 1 Processor -- start 2020-11-20T12:16:21.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa8a277b700 1 -- start start 2020-11-20T12:16:21.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa8a277b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa89c0a5f40 0x7fa89c0bac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:21.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa8a277b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89c1537e0 con 0x7fa89c153340 2020-11-20T12:16:21.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa8a277b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa89c153a80 con 0x7fa89c0a5b60 2020-11-20T12:16:21.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa8a277b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa89c153d20 con 0x7fa89c0a5f40 2020-11-20T12:16:21.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa89b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa89c0a5f40 0x7fa89c0bac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:21.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa89b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa89c0a5f40 0x7fa89c0bac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43556/0 (socket says 172.21.15.116:43556) 2020-11-20T12:16:21.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa89b7fe700 1 -- 172.21.15.116:0/2246925700 learned_addr learned my addr 172.21.15.116:0/2246925700 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:21.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa8a0d18700 1 --1- 172.21.15.116:0/2246925700 >> v1:172.21.15.116:6789/0 conn(0x7fa89c153340 0x7fa89c064e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:52992/0 (socket says 172.21.15.116:52992) 2020-11-20T12:16:21.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa89b7fe700 1 -- 172.21.15.116:0/2246925700 >> v1:172.21.15.173:6789/0 conn(0x7fa89c0a5b60 legacy=0x7fa89c0638a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa89b7fe700 1 -- 172.21.15.116:0/2246925700 >> v1:172.21.15.116:6789/0 conn(0x7fa89c153340 legacy=0x7fa89c064e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa89affd700 1 -- 172.21.15.116:0/2246925700 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3497486873 0 0) 0x7fa89c1537e0 con 0x7fa89c153340 2020-11-20T12:16:21.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa89b7fe700 1 -- 172.21.15.116:0/2246925700 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c153fc0 con 0x7fa89c0a5f40 2020-11-20T12:16:21.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa89b7fe700 1 --2- 172.21.15.116:0/2246925700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa89c0a5f40 0x7fa89c0bac60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fa89000ba20 tx=0x7fa890006eb0).ready entity=mon.0 client_cookie=acd65887a545ac25 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:21.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa89affd700 1 -- 172.21.15.116:0/2246925700 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890009890 con 0x7fa89c0a5f40 2020-11-20T12:16:21.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa89affd700 1 -- 172.21.15.116:0/2246925700 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8900099f0 con 0x7fa89c0a5f40 2020-11-20T12:16:21.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.756+0000 7fa89affd700 1 -- 172.21.15.116:0/2246925700 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890009d70 con 0x7fa89c0a5f40 2020-11-20T12:16:21.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.760+0000 7fa8a277b700 1 -- 172.21.15.116:0/2246925700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa89c0a5f40 msgr2=0x7fa89c0bac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.760+0000 7fa8a277b700 1 --2- 172.21.15.116:0/2246925700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa89c0a5f40 0x7fa89c0bac60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fa89000ba20 tx=0x7fa890006eb0).stop 2020-11-20T12:16:21.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.760+0000 7fa8a277b700 1 -- 172.21.15.116:0/2246925700 shutdown_connections 2020-11-20T12:16:21.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.760+0000 7fa8a277b700 1 --2- 172.21.15.116:0/2246925700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa89c0a5f40 0x7fa89c0bac60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.760+0000 7fa8a277b700 1 -- 172.21.15.116:0/2246925700 >> 172.21.15.116:0/2246925700 conn(0x7fa89c061d50 msgr2=0x7fa89c0c1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:21.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.760+0000 7fa8a277b700 1 -- 172.21.15.116:0/2246925700 shutdown_connections 2020-11-20T12:16:21.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.760+0000 7fa8a277b700 1 -- 172.21.15.116:0/2246925700 wait complete. 2020-11-20T12:16:21.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.760+0000 7fa8a277b700 1 Processor -- start 2020-11-20T12:16:21.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.760+0000 7fa8a277b700 1 -- start start 2020-11-20T12:16:21.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa8a277b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa89c0a5b60 0x7fa89c151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:21.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa8a277b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa89c153340 0x7fa89c0aac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:21.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa8a277b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa89c160980 0x7fa89c160dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:21.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa8a277b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c151fe0 con 0x7fa89c0a5b60 2020-11-20T12:16:21.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa8a277b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c152280 con 0x7fa89c153340 2020-11-20T12:16:21.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa8a277b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa89c152520 con 0x7fa89c160980 2020-11-20T12:16:21.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa89bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa89c0a5b60 0x7fa89c151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:21.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa89bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa89c0a5b60 0x7fa89c151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43562/0 (socket says 172.21.15.116:43562) 2020-11-20T12:16:21.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa89bfff700 1 -- 172.21.15.116:0/2593222483 learned_addr learned my addr 172.21.15.116:0/2593222483 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:21.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa89b7fe700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa89c153340 0x7fa89c0aac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:21.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa8a0d18700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa89c160980 0x7fa89c160dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:21.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa89bfff700 1 -- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa89c160980 msgr2=0x7fa89c160dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa89bfff700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa89c160980 0x7fa89c160dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa89bfff700 1 -- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa89c153340 msgr2=0x7fa89c0aac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa89bfff700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa89c153340 0x7fa89c0aac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa89bfff700 1 -- 172.21.15.116:0/2593222483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c1527c0 con 0x7fa89c0a5b60 2020-11-20T12:16:21.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa89bfff700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa89c0a5b60 0x7fa89c151a40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fa88c001e70 tx=0x7fa88c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:21.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa87bfff700 1 -- 172.21.15.116:0/2593222483 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c007020 con 0x7fa89c0a5b60 2020-11-20T12:16:21.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa87bfff700 1 -- 172.21.15.116:0/2593222483 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c007180 con 0x7fa89c0a5b60 2020-11-20T12:16:21.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa8a277b700 1 -- 172.21.15.116:0/2593222483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa89c152a60 con 0x7fa89c0a5b60 2020-11-20T12:16:21.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa87bfff700 1 -- 172.21.15.116:0/2593222483 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c00e560 con 0x7fa89c0a5b60 2020-11-20T12:16:21.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.764+0000 7fa8a277b700 1 -- 172.21.15.116:0/2593222483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa89c14dda0 con 0x7fa89c0a5b60 2020-11-20T12:16:21.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.768+0000 7fa8a277b700 1 -- 172.21.15.116:0/2593222483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa89c152bf0 con 0x7fa89c0a5b60 2020-11-20T12:16:21.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.768+0000 7fa87bfff700 1 -- 172.21.15.116:0/2593222483 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa88c018020 con 0x7fa89c0a5b60 2020-11-20T12:16:21.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.768+0000 7fa87bfff700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8740557c0 0x7fa874057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:21.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.768+0000 7fa87bfff700 1 -- 172.21.15.116:0/2593222483 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa88c06e000 con 0x7fa89c0a5b60 2020-11-20T12:16:21.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.772+0000 7fa89b7fe700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8740557c0 0x7fa874057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:21.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.772+0000 7fa89b7fe700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8740557c0 0x7fa874057c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa89c067000 tx=0x7fa890015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:21.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.776+0000 7fa87bfff700 1 -- 172.21.15.116:0/2593222483 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa88c043e40 con 0x7fa89c0a5b60 2020-11-20T12:16:21.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.936+0000 7fa8a277b700 1 -- 172.21.15.116:0/2593222483 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa89c05ca30 con 0x7fa89c0a5b60 2020-11-20T12:16:21.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.936+0000 7fa87bfff700 1 -- 172.21.15.116:0/2593222483 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa88c016030 con 0x7fa89c0a5b60 2020-11-20T12:16:21.938 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:21.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 -- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8740557c0 msgr2=0x7fa874057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8740557c0 0x7fa874057c80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7fa89c067000 tx=0x7fa890015000).stop 2020-11-20T12:16:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 -- 172.21.15.116:0/2593222483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa89c0a5b60 msgr2=0x7fa89c151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa89c0a5b60 0x7fa89c151a40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7fa88c001e70 tx=0x7fa88c007c50).stop 2020-11-20T12:16:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 -- 172.21.15.116:0/2593222483 shutdown_connections 2020-11-20T12:16:21.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8740557c0 0x7fa874057c80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa89c160980 0x7fa89c160dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa89c0a5b60 0x7fa89c151a40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 --2- 172.21.15.116:0/2593222483 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa89c153340 0x7fa89c0aac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:21.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 -- 172.21.15.116:0/2593222483 >> 172.21.15.116:0/2593222483 conn(0x7fa89c061d50 msgr2=0x7fa89c0c52f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 -- 172.21.15.116:0/2593222483 shutdown_connections 2020-11-20T12:16:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:21.940+0000 7fa8a277b700 1 -- 172.21.15.116:0/2593222483 wait complete. 2020-11-20T12:16:21.964 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:22.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf9f25700 1 Processor -- start 2020-11-20T12:16:22.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf9f25700 1 -- start start 2020-11-20T12:16:22.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf9f25700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8cf414e040 0x7f8cf40c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf9f25700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf40a1010 con 0x7f8cf414e420 2020-11-20T12:16:22.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf9f25700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cf40a5ce0 con 0x7f8cf40a56e0 2020-11-20T12:16:22.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf9f25700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cf40a5f80 con 0x7f8cf414e040 2020-11-20T12:16:22.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf8f23700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8cf414e040 0x7f8cf40c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf8f23700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8cf414e040 0x7f8cf40c2cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43570/0 (socket says 172.21.15.116:43570) 2020-11-20T12:16:22.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf8f23700 1 -- 172.21.15.116:0/3009787625 learned_addr learned my addr 172.21.15.116:0/3009787625 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:22.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cebfff700 1 --1- 172.21.15.116:0/3009787625 >> v1:172.21.15.116:6789/0 conn(0x7f8cf414e420 0x7f8cf40a4fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53006/0 (socket says 172.21.15.116:53006) 2020-11-20T12:16:22.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf8f23700 1 -- 172.21.15.116:0/3009787625 >> v1:172.21.15.173:6789/0 conn(0x7f8cf40a56e0 legacy=0x7f8cf4157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:22.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf8f23700 1 -- 172.21.15.116:0/3009787625 >> v1:172.21.15.116:6789/0 conn(0x7f8cf414e420 legacy=0x7f8cf40a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf8f23700 1 -- 172.21.15.116:0/3009787625 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf40a1270 con 0x7f8cf414e040 2020-11-20T12:16:22.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8cf8f23700 1 --2- 172.21.15.116:0/3009787625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf414e040 0x7f8cf40c2cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8cf0006fd0 tx=0x7f8cf0006eb0).ready entity=mon.0 client_cookie=5e948815d893fff9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:22.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8ceb7fe700 1 -- 172.21.15.116:0/3009787625 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0009890 con 0x7f8cf414e040 2020-11-20T12:16:22.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8ceb7fe700 1 -- 172.21.15.116:0/3009787625 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf00099f0 con 0x7f8cf414e040 2020-11-20T12:16:22.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.160+0000 7f8ceb7fe700 1 -- 172.21.15.116:0/3009787625 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0009d70 con 0x7f8cf414e040 2020-11-20T12:16:22.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.164+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3009787625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf414e040 msgr2=0x7f8cf40c2cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.164+0000 7f8cf9f25700 1 --2- 172.21.15.116:0/3009787625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf414e040 0x7f8cf40c2cc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f8cf0006fd0 tx=0x7f8cf0006eb0).stop 2020-11-20T12:16:22.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.164+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3009787625 shutdown_connections 2020-11-20T12:16:22.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.164+0000 7f8cf9f25700 1 --2- 172.21.15.116:0/3009787625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf414e040 0x7f8cf40c2cc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.164+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3009787625 >> 172.21.15.116:0/3009787625 conn(0x7f8cf4060c30 msgr2=0x7f8cf40c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:22.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.164+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3009787625 shutdown_connections 2020-11-20T12:16:22.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.164+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3009787625 wait complete. 2020-11-20T12:16:22.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.168+0000 7f8cf9f25700 1 Processor -- start 2020-11-20T12:16:22.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.168+0000 7f8cf9f25700 1 -- start start 2020-11-20T12:16:22.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.168+0000 7f8cf9f25700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cf40a56e0 0x7f8cf408ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.168+0000 7f8cf9f25700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cf414e040 0x7f8cf40a8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.168+0000 7f8cf9f25700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf414e420 0x7f8cf40a8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.168+0000 7f8cf9f25700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf4091220 con 0x7f8cf414e420 2020-11-20T12:16:22.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.168+0000 7f8cf9f25700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf40a7e00 con 0x7f8cf414e040 2020-11-20T12:16:22.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.168+0000 7f8cf9f25700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf40914a0 con 0x7f8cf40a56e0 2020-11-20T12:16:22.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.168+0000 7f8cf9724700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf414e420 0x7f8cf40a8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.168+0000 7f8cf9724700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf414e420 0x7f8cf40a8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43580/0 (socket says 172.21.15.116:43580) 2020-11-20T12:16:22.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.168+0000 7f8cf9724700 1 -- 172.21.15.116:0/3897876748 learned_addr learned my addr 172.21.15.116:0/3897876748 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:22.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8cf8f23700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cf40a56e0 0x7f8cf408ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8cebfff700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cf414e040 0x7f8cf40a8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8cf8f23700 1 -- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cf414e040 msgr2=0x7f8cf40a8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8cf8f23700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cf414e040 0x7f8cf40a8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8cf8f23700 1 -- 172.21.15.116:0/3897876748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf414e420 msgr2=0x7f8cf40a8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8cf8f23700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf414e420 0x7f8cf40a8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8cf8f23700 1 -- 172.21.15.116:0/3897876748 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf4091740 con 0x7f8cf40a56e0 2020-11-20T12:16:22.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8cf8f23700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cf40a56e0 0x7f8cf408ebd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f8cf0006fd0 tx=0x7f8cf0015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:22.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8ce8ff9700 1 -- 172.21.15.116:0/3897876748 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0011e20 con 0x7f8cf40a56e0 2020-11-20T12:16:22.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8ce8ff9700 1 -- 172.21.15.116:0/3897876748 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf001d3c0 con 0x7f8cf40a56e0 2020-11-20T12:16:22.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8ce8ff9700 1 -- 172.21.15.116:0/3897876748 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf001d780 con 0x7f8cf40a56e0 2020-11-20T12:16:22.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3897876748 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf40919e0 con 0x7f8cf40a56e0 2020-11-20T12:16:22.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.172+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3897876748 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf408f0c0 con 0x7f8cf40a56e0 2020-11-20T12:16:22.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.176+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3897876748 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf4091b70 con 0x7f8cf40a56e0 2020-11-20T12:16:22.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.176+0000 7f8ce8ff9700 1 -- 172.21.15.116:0/3897876748 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8cf001a040 con 0x7f8cf40a56e0 2020-11-20T12:16:22.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.176+0000 7f8ce8ff9700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ccc0557c0 0x7f8ccc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.176+0000 7f8ce8ff9700 1 -- 172.21.15.116:0/3897876748 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8cf0070320 con 0x7f8cf40a56e0 2020-11-20T12:16:22.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.176+0000 7f8cebfff700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ccc0557c0 0x7f8ccc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.180+0000 7f8cebfff700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ccc0557c0 0x7f8ccc057c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f8cf415b050 tx=0x7f8ce4007b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:22.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.188+0000 7f8ce8ff9700 1 -- 172.21.15.116:0/3897876748 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8cf0049d20 con 0x7f8cf40a56e0 2020-11-20T12:16:22.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.348+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3897876748 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8cf4044f60 con 0x7f8cf40a56e0 2020-11-20T12:16:22.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.348+0000 7f8ce8ff9700 1 -- 172.21.15.116:0/3897876748 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8cf002d020 con 0x7f8cf40a56e0 2020-11-20T12:16:22.349 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:22.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.348+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ccc0557c0 msgr2=0x7f8ccc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.348+0000 7f8cf9f25700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ccc0557c0 0x7f8ccc057c80 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7f8cf415b050 tx=0x7f8ce4007b80).stop 2020-11-20T12:16:22.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.348+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cf40a56e0 msgr2=0x7f8cf408ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.348+0000 7f8cf9f25700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cf40a56e0 0x7f8cf408ebd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f8cf0006fd0 tx=0x7f8cf0015000).stop 2020-11-20T12:16:22.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.352+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3897876748 shutdown_connections 2020-11-20T12:16:22.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.352+0000 7f8cf9f25700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ccc0557c0 0x7f8ccc057c80 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.352+0000 7f8cf9f25700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cf414e420 0x7f8cf40a8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.352+0000 7f8cf9f25700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cf40a56e0 0x7f8cf408ebd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.352+0000 7f8cf9f25700 1 --2- 172.21.15.116:0/3897876748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cf414e040 0x7f8cf40a8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.352+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3897876748 >> 172.21.15.116:0/3897876748 conn(0x7f8cf4060c30 msgr2=0x7f8cf4063540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:22.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.352+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3897876748 shutdown_connections 2020-11-20T12:16:22.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.352+0000 7f8cf9f25700 1 -- 172.21.15.116:0/3897876748 wait complete. 2020-11-20T12:16:22.378 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:16:22.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.540+0000 7fcf004df700 1 Processor -- start 2020-11-20T12:16:22.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.540+0000 7fcf004df700 1 -- start start 2020-11-20T12:16:22.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.544+0000 7fcf004df700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fcef814da70 0x7fcef8067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.544+0000 7fcf004df700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcef80c11a0 0x7fcef80baa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.544+0000 7fcf004df700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef80baf00 con 0x7fcef80c0dc0 2020-11-20T12:16:22.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.544+0000 7fcf004df700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef80663d0 con 0x7fcef80c11a0 2020-11-20T12:16:22.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.544+0000 7fcf004df700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef8066670 con 0x7fcef814da70 2020-11-20T12:16:22.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.544+0000 7fcefea7c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcef80c11a0 0x7fcef80baa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.544+0000 7fcefea7c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcef80c11a0 0x7fcef80baa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43588/0 (socket says 172.21.15.116:43588) 2020-11-20T12:16:22.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.544+0000 7fcefea7c700 1 -- 172.21.15.116:0/3010685184 learned_addr learned my addr 172.21.15.116:0/3010685184 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:22.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.544+0000 7fcefda7a700 1 --2- 172.21.15.116:0/3010685184 >> v2:172.21.15.173:3300/0 conn(0x7fcef814da70 0x7fcef8067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcefea7c700 1 -- 172.21.15.116:0/3010685184 >> v2:172.21.15.173:3300/0 conn(0x7fcef814da70 msgr2=0x7fcef8067ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:22.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcefea7c700 1 --2- 172.21.15.116:0/3010685184 >> v2:172.21.15.173:3300/0 conn(0x7fcef814da70 0x7fcef8067ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcefea7c700 1 -- 172.21.15.116:0/3010685184 >> v1:172.21.15.173:6789/0 conn(0x7fcef80c0dc0 legacy=0x7fcef814d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcefea7c700 1 -- 172.21.15.116:0/3010685184 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8066910 con 0x7fcef80c11a0 2020-11-20T12:16:22.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcefd279700 1 -- 172.21.15.116:0/3010685184 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4054180388 0 0) 0x7fcef80baf00 con 0x7fcef80c0dc0 2020-11-20T12:16:22.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcefea7c700 1 --2- 172.21.15.116:0/3010685184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef80c11a0 0x7fcef80baa10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fcef80661e0 tx=0x7fcef4006eb0).ready entity=mon.0 client_cookie=f377a9c11c78a0a9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:22.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcefd279700 1 -- 172.21.15.116:0/3010685184 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4009810 con 0x7fcef80c11a0 2020-11-20T12:16:22.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcefd279700 1 -- 172.21.15.116:0/3010685184 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef4009970 con 0x7fcef80c11a0 2020-11-20T12:16:22.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcefd279700 1 -- 172.21.15.116:0/3010685184 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4009cf0 con 0x7fcef80c11a0 2020-11-20T12:16:22.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcf004df700 1 -- 172.21.15.116:0/3010685184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef80c11a0 msgr2=0x7fcef80baa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcf004df700 1 --2- 172.21.15.116:0/3010685184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef80c11a0 0x7fcef80baa10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fcef80661e0 tx=0x7fcef4006eb0).stop 2020-11-20T12:16:22.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcf004df700 1 -- 172.21.15.116:0/3010685184 shutdown_connections 2020-11-20T12:16:22.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcf004df700 1 --2- 172.21.15.116:0/3010685184 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef80c11a0 0x7fcef80baa10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcf004df700 1 --2- 172.21.15.116:0/3010685184 >> v2:172.21.15.173:3300/0 conn(0x7fcef814da70 0x7fcef8067ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcf004df700 1 -- 172.21.15.116:0/3010685184 >> 172.21.15.116:0/3010685184 conn(0x7fcef8060170 msgr2=0x7fcef8060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:22.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcf004df700 1 -- 172.21.15.116:0/3010685184 shutdown_connections 2020-11-20T12:16:22.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.548+0000 7fcf004df700 1 -- 172.21.15.116:0/3010685184 wait complete. 2020-11-20T12:16:22.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcf004df700 1 Processor -- start 2020-11-20T12:16:22.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcf004df700 1 -- start start 2020-11-20T12:16:22.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcf004df700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcef80c0dc0 0x7fcef8166140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcf004df700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef80c11a0 0x7fcef8145670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcf004df700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef814da70 0x7fcef8142420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcf004df700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8165520 con 0x7fcef814da70 2020-11-20T12:16:22.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcf004df700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8169810 con 0x7fcef80c0dc0 2020-11-20T12:16:22.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcf004df700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef8165760 con 0x7fcef80c11a0 2020-11-20T12:16:22.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcefea7c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef814da70 0x7fcef8142420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcefea7c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef814da70 0x7fcef8142420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43594/0 (socket says 172.21.15.116:43594) 2020-11-20T12:16:22.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcefea7c700 1 -- 172.21.15.116:0/322236512 learned_addr learned my addr 172.21.15.116:0/322236512 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:22.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcefda7a700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef80c11a0 0x7fcef8145670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.552+0000 7fcefe27b700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcef80c0dc0 0x7fcef8166140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcefea7c700 1 -- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef80c11a0 msgr2=0x7fcef8145670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcefea7c700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef80c11a0 0x7fcef8145670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcefea7c700 1 -- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcef80c0dc0 msgr2=0x7fcef8166140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcefea7c700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcef80c0dc0 0x7fcef8166140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcefea7c700 1 -- 172.21.15.116:0/322236512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef8165a00 con 0x7fcef814da70 2020-11-20T12:16:22.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcefe27b700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcef80c0dc0 0x7fcef8166140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:22.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcefea7c700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef814da70 0x7fcef8142420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fcef4006ef0 tx=0x7fcef4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:22.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcee67fc700 1 -- 172.21.15.116:0/322236512 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef4006be0 con 0x7fcef814da70 2020-11-20T12:16:22.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcee67fc700 1 -- 172.21.15.116:0/322236512 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef401d5d0 con 0x7fcef814da70 2020-11-20T12:16:22.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcee67fc700 1 -- 172.21.15.116:0/322236512 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef401d950 con 0x7fcef814da70 2020-11-20T12:16:22.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcf004df700 1 -- 172.21.15.116:0/322236512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef8165ca0 con 0x7fcef814da70 2020-11-20T12:16:22.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcf004df700 1 -- 172.21.15.116:0/322236512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef8145d80 con 0x7fcef814da70 2020-11-20T12:16:22.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcf004df700 1 -- 172.21.15.116:0/322236512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef805ca30 con 0x7fcef814da70 2020-11-20T12:16:22.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcee67fc700 1 -- 172.21.15.116:0/322236512 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fcef4025030 con 0x7fcef814da70 2020-11-20T12:16:22.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.556+0000 7fcee67fc700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fced00557c0 0x7fced0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.560+0000 7fcee67fc700 1 -- 172.21.15.116:0/322236512 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcef406fff0 con 0x7fcef814da70 2020-11-20T12:16:22.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.560+0000 7fcefe27b700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fced00557c0 0x7fced0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.560+0000 7fcefe27b700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fced00557c0 0x7fced0057c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fcef8169bb0 tx=0x7fcee8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:22.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.564+0000 7fcee67fc700 1 -- 172.21.15.116:0/322236512 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcef4045ee0 con 0x7fcef814da70 2020-11-20T12:16:22.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.720+0000 7fcf004df700 1 -- 172.21.15.116:0/322236512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcef8165e30 con 0x7fcef814da70 2020-11-20T12:16:22.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.720+0000 7fcee67fc700 1 -- 172.21.15.116:0/322236512 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7fcef4023020 con 0x7fcef814da70 2020-11-20T12:16:22.723 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:22.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 -- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fced00557c0 msgr2=0x7fced0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fced00557c0 0x7fced0057c80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fcef8169bb0 tx=0x7fcee8009040).stop 2020-11-20T12:16:22.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 -- 172.21.15.116:0/322236512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef814da70 msgr2=0x7fcef8142420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef814da70 0x7fcef8142420 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fcef4006ef0 tx=0x7fcef4015000).stop 2020-11-20T12:16:22.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 -- 172.21.15.116:0/322236512 shutdown_connections 2020-11-20T12:16:22.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fced00557c0 0x7fced0057c80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcef814da70 0x7fcef8142420 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcef80c0dc0 0x7fcef8166140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 --2- 172.21.15.116:0/322236512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcef80c11a0 0x7fcef8145670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 -- 172.21.15.116:0/322236512 >> 172.21.15.116:0/322236512 conn(0x7fcef8060170 msgr2=0x7fcef80c2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:22.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 -- 172.21.15.116:0/322236512 shutdown_connections 2020-11-20T12:16:22.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.724+0000 7fcf004df700 1 -- 172.21.15.116:0/322236512 wait complete. 2020-11-20T12:16:22.750 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:16:22.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.924+0000 7ff91d44a700 1 Processor -- start 2020-11-20T12:16:22.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.924+0000 7ff91d44a700 1 -- start start 2020-11-20T12:16:22.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff91d44a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814dba0 0x7ff918152470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff91d44a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff91814df80 0x7ff9180bf370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff91d44a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff918066f20 con 0x7ff9180bf860 2020-11-20T12:16:22.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff91d44a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff918151a90 con 0x7ff91814df80 2020-11-20T12:16:22.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff91d44a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff918151d30 con 0x7ff91814dba0 2020-11-20T12:16:22.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff916ffd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814dba0 0x7ff918152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff916ffd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814dba0 0x7ff918152470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:52548/0 (socket says 172.21.15.116:52548) 2020-11-20T12:16:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff916ffd700 1 -- 172.21.15.116:0/242702056 learned_addr learned my addr 172.21.15.116:0/242702056 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:22.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff9167fc700 1 --2- 172.21.15.116:0/242702056 >> v2:172.21.15.173:3300/0 conn(0x7ff91814df80 0x7ff9180bf370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff916ffd700 1 -- 172.21.15.116:0/242702056 >> v2:172.21.15.173:3300/0 conn(0x7ff91814df80 msgr2=0x7ff9180bf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff916ffd700 1 --2- 172.21.15.116:0/242702056 >> v2:172.21.15.173:3300/0 conn(0x7ff91814df80 0x7ff9180bf370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff916ffd700 1 -- 172.21.15.116:0/242702056 >> v1:172.21.15.173:6789/0 conn(0x7ff9180bf860 legacy=0x7ff918066250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff916ffd700 1 -- 172.21.15.116:0/242702056 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff918151fd0 con 0x7ff91814dba0 2020-11-20T12:16:22.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff916ffd700 1 --2- 172.21.15.116:0/242702056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814dba0 0x7ff918152470 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7ff90800dcf0 tx=0x7ff908006e40).ready entity=mon.2 client_cookie=55e90ad295e83711 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff915ffb700 1 -- 172.21.15.116:0/242702056 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9080153f0 con 0x7ff91814dba0 2020-11-20T12:16:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff915ffb700 1 -- 172.21.15.116:0/242702056 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff908015550 con 0x7ff91814dba0 2020-11-20T12:16:22.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.928+0000 7ff915ffb700 1 -- 172.21.15.116:0/242702056 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9080158d0 con 0x7ff91814dba0 2020-11-20T12:16:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.932+0000 7ff91d44a700 1 -- 172.21.15.116:0/242702056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814dba0 msgr2=0x7ff918152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.932+0000 7ff91d44a700 1 --2- 172.21.15.116:0/242702056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814dba0 0x7ff918152470 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7ff90800dcf0 tx=0x7ff908006e40).stop 2020-11-20T12:16:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.932+0000 7ff91d44a700 1 -- 172.21.15.116:0/242702056 shutdown_connections 2020-11-20T12:16:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.932+0000 7ff91d44a700 1 --2- 172.21.15.116:0/242702056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814dba0 0x7ff918152470 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.932+0000 7ff91d44a700 1 --2- 172.21.15.116:0/242702056 >> v2:172.21.15.173:3300/0 conn(0x7ff91814df80 0x7ff9180bf370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.932+0000 7ff91d44a700 1 -- 172.21.15.116:0/242702056 >> 172.21.15.116:0/242702056 conn(0x7ff918060170 msgr2=0x7ff918060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.932+0000 7ff91d44a700 1 -- 172.21.15.116:0/242702056 shutdown_connections 2020-11-20T12:16:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.932+0000 7ff91d44a700 1 -- 172.21.15.116:0/242702056 wait complete. 2020-11-20T12:16:22.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff91d44a700 1 Processor -- start 2020-11-20T12:16:22.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff91d44a700 1 -- start start 2020-11-20T12:16:22.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff91d44a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9180bf860 0x7ff9180c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff91d44a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff91814dba0 0x7ff9180a8420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff91d44a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814df80 0x7ff91815e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff91d44a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180a72d0 con 0x7ff9180bf860 2020-11-20T12:16:22.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff91d44a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180a7500 con 0x7ff91814dba0 2020-11-20T12:16:22.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff91d44a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9180a77a0 con 0x7ff91814df80 2020-11-20T12:16:22.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff916ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9180bf860 0x7ff9180c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff916ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9180bf860 0x7ff9180c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43604/0 (socket says 172.21.15.116:43604) 2020-11-20T12:16:22.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff916ffd700 1 -- 172.21.15.116:0/68595147 learned_addr learned my addr 172.21.15.116:0/68595147 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:22.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff9167fc700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff91814dba0 0x7ff9180a8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.936+0000 7ff9177fe700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814df80 0x7ff91815e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff916ffd700 1 -- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814df80 msgr2=0x7ff91815e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff916ffd700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814df80 0x7ff91815e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff916ffd700 1 -- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff91814dba0 msgr2=0x7ff9180a8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:22.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff916ffd700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff91814dba0 0x7ff9180a8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:22.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff916ffd700 1 -- 172.21.15.116:0/68595147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9180a7a40 con 0x7ff9180bf860 2020-11-20T12:16:22.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff916ffd700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9180bf860 0x7ff9180c46b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7ff90800d4a0 tx=0x7ff908018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:22.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff8feffd700 1 -- 172.21.15.116:0/68595147 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff908015790 con 0x7ff9180bf860 2020-11-20T12:16:22.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff8feffd700 1 -- 172.21.15.116:0/68595147 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff90800c840 con 0x7ff9180bf860 2020-11-20T12:16:22.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff91d44a700 1 -- 172.21.15.116:0/68595147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9180a7ce0 con 0x7ff9180bf860 2020-11-20T12:16:22.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff8feffd700 1 -- 172.21.15.116:0/68595147 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff90800cc00 con 0x7ff9180bf860 2020-11-20T12:16:22.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff91d44a700 1 -- 172.21.15.116:0/68595147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff91815e570 con 0x7ff9180bf860 2020-11-20T12:16:22.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff91d44a700 1 -- 172.21.15.116:0/68595147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff91805ca30 con 0x7ff9180bf860 2020-11-20T12:16:22.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.940+0000 7ff8feffd700 1 -- 172.21.15.116:0/68595147 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff908026030 con 0x7ff9180bf860 2020-11-20T12:16:22.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.944+0000 7ff8feffd700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8f40557c0 0x7ff8f4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:22.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.944+0000 7ff8feffd700 1 -- 172.21.15.116:0/68595147 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff908071a10 con 0x7ff9180bf860 2020-11-20T12:16:22.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.944+0000 7ff9167fc700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8f40557c0 0x7ff8f4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:22.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.944+0000 7ff9167fc700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8f40557c0 0x7ff8f4057c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7ff9180a3db0 tx=0x7ff90c006f90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:22.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:22.948+0000 7ff8feffd700 1 -- 172.21.15.116:0/68595147 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff9080478b0 con 0x7ff9180bf860 2020-11-20T12:16:23.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.108+0000 7ff91d44a700 1 -- 172.21.15.116:0/68595147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff918067860 con 0x7ff9180bf860 2020-11-20T12:16:23.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff8feffd700 1 -- 172.21.15.116:0/68595147 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff908024020 con 0x7ff9180bf860 2020-11-20T12:16:23.112 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:23.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 -- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8f40557c0 msgr2=0x7ff8f4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:23.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8f40557c0 0x7ff8f4057c80 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7ff9180a3db0 tx=0x7ff90c006f90).stop 2020-11-20T12:16:23.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 -- 172.21.15.116:0/68595147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9180bf860 msgr2=0x7ff9180c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:23.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9180bf860 0x7ff9180c46b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7ff90800d4a0 tx=0x7ff908018000).stop 2020-11-20T12:16:23.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 -- 172.21.15.116:0/68595147 shutdown_connections 2020-11-20T12:16:23.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8f40557c0 0x7ff8f4057c80 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:23.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff91814df80 0x7ff91815e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:23.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9180bf860 0x7ff9180c46b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:23.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 --2- 172.21.15.116:0/68595147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff91814dba0 0x7ff9180a8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:23.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 -- 172.21.15.116:0/68595147 >> 172.21.15.116:0/68595147 conn(0x7ff918060170 msgr2=0x7ff918150240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:23.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 -- 172.21.15.116:0/68595147 shutdown_connections 2020-11-20T12:16:23.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.112+0000 7ff91d44a700 1 -- 172.21.15.116:0/68595147 wait complete. 2020-11-20T12:16:23.139 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:16:23.140 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:16:23.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.308+0000 7f3139672700 1 Processor -- start 2020-11-20T12:16:23.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.308+0000 7f3139672700 1 -- start start 2020-11-20T12:16:23.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3139672700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31341557e0 0x7f31341526d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:23.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3139672700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3134155da0 con 0x7f313414f210 2020-11-20T12:16:23.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3139672700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3134151cf0 con 0x7f313414ee30 2020-11-20T12:16:23.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3139672700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3134151ad0 con 0x7f31341557e0 2020-11-20T12:16:23.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3133fff700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f313414ee30 0x7f31340c1b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49276/0 (socket says 172.21.15.116:49276) 2020-11-20T12:16:23.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3133fff700 1 -- 172.21.15.116:0/3025602672 learned_addr learned my addr 172.21.15.116:0/3025602672 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:23.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f31337fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f313414f210 0x7f31341550d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53048/0 (socket says 172.21.15.116:53048) 2020-11-20T12:16:23.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3138e71700 1 --2- 172.21.15.116:0/3025602672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31341557e0 0x7f31341526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:23.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3132ffd700 1 -- 172.21.15.116:0/3025602672 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568741066 0 0) 0x7f3134151cf0 con 0x7f313414ee30 2020-11-20T12:16:23.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3132ffd700 1 -- 172.21.15.116:0/3025602672 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3120003240 con 0x7f313414ee30 2020-11-20T12:16:23.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3132ffd700 1 -- 172.21.15.116:0/3025602672 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3310204133 0 0) 0x7f3134155da0 con 0x7f313414f210 2020-11-20T12:16:23.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3138e71700 1 -- 172.21.15.116:0/3025602672 >> v1:172.21.15.173:6789/0 conn(0x7f313414ee30 legacy=0x7f31340c1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:23.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3138e71700 1 -- 172.21.15.116:0/3025602672 >> v1:172.21.15.116:6789/0 conn(0x7f313414f210 legacy=0x7f31341550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:23.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3138e71700 1 -- 172.21.15.116:0/3025602672 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3134151f90 con 0x7f31341557e0 2020-11-20T12:16:23.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.312+0000 7f3138e71700 1 --2- 172.21.15.116:0/3025602672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31341557e0 0x7f31341526d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f312c009cc0 tx=0x7f312c0091d0).ready entity=mon.2 client_cookie=6292f2003051a15b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:23.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3132ffd700 1 -- 172.21.15.116:0/3025602672 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c00bad0 con 0x7f31341557e0 2020-11-20T12:16:23.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3132ffd700 1 -- 172.21.15.116:0/3025602672 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c00bc30 con 0x7f31341557e0 2020-11-20T12:16:23.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3132ffd700 1 -- 172.21.15.116:0/3025602672 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c011400 con 0x7f31341557e0 2020-11-20T12:16:23.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3139672700 1 -- 172.21.15.116:0/3025602672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31341557e0 msgr2=0x7f31341526d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:23.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3139672700 1 --2- 172.21.15.116:0/3025602672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31341557e0 0x7f31341526d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f312c009cc0 tx=0x7f312c0091d0).stop 2020-11-20T12:16:23.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3139672700 1 -- 172.21.15.116:0/3025602672 shutdown_connections 2020-11-20T12:16:23.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3139672700 1 --2- 172.21.15.116:0/3025602672 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31341557e0 0x7f31341526d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:23.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3139672700 1 -- 172.21.15.116:0/3025602672 >> 172.21.15.116:0/3025602672 conn(0x7f3134061280 msgr2=0x7f3134061660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:23.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3139672700 1 -- 172.21.15.116:0/3025602672 shutdown_connections 2020-11-20T12:16:23.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3139672700 1 -- 172.21.15.116:0/3025602672 wait complete. 2020-11-20T12:16:23.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3139672700 1 Processor -- start 2020-11-20T12:16:23.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.316+0000 7f3139672700 1 -- start start 2020-11-20T12:16:23.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3139672700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f313414ee30 0x7f3134156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:23.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3139672700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f313414f210 0x7f3134160540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:23.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3139672700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31341557e0 0x7f31340c06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:23.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3139672700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3134156170 con 0x7f313414ee30 2020-11-20T12:16:23.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3139672700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3134161cb0 con 0x7f31341557e0 2020-11-20T12:16:23.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3139672700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31341563d0 con 0x7f313414f210 2020-11-20T12:16:23.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3133fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f313414ee30 0x7f3134156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:23.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3138e71700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31341557e0 0x7f31340c06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:23.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3133fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f313414ee30 0x7f3134156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43618/0 (socket says 172.21.15.116:43618) 2020-11-20T12:16:23.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3138e71700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31341557e0 0x7f31340c06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:42674/0 (socket says 172.21.15.116:42674) 2020-11-20T12:16:23.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3133fff700 1 -- 172.21.15.116:0/3219195876 learned_addr learned my addr 172.21.15.116:0/3219195876 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:23.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f31337fe700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f313414f210 0x7f3134160540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:23.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3138e71700 1 -- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f313414f210 msgr2=0x7f3134160540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:23.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3138e71700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f313414f210 0x7f3134160540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:23.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3138e71700 1 -- 172.21.15.116:0/3219195876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f313414ee30 msgr2=0x7f3134156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:23.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3138e71700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f313414ee30 0x7f3134156db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:23.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3138e71700 1 -- 172.21.15.116:0/3219195876 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3134156670 con 0x7f31341557e0 2020-11-20T12:16:23.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3138e71700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31341557e0 0x7f31340c06f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f312c009cc0 tx=0x7f312c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:23.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3113fff700 1 -- 172.21.15.116:0/3219195876 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c010700 con 0x7f31341557e0 2020-11-20T12:16:23.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3113fff700 1 -- 172.21.15.116:0/3219195876 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c010860 con 0x7f31341557e0 2020-11-20T12:16:23.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3113fff700 1 -- 172.21.15.116:0/3219195876 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c010be0 con 0x7f31341557e0 2020-11-20T12:16:23.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3139672700 1 -- 172.21.15.116:0/3219195876 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3134156910 con 0x7f31341557e0 2020-11-20T12:16:23.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.320+0000 7f3139672700 1 -- 172.21.15.116:0/3219195876 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3134160c50 con 0x7f31341557e0 2020-11-20T12:16:23.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.324+0000 7f3139672700 1 -- 172.21.15.116:0/3219195876 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3134156aa0 con 0x7f31341557e0 2020-11-20T12:16:23.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.324+0000 7f3113fff700 1 -- 172.21.15.116:0/3219195876 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f312c017070 con 0x7f31341557e0 2020-11-20T12:16:23.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.324+0000 7f3113fff700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f310c0557c0 0x7f310c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:23.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.324+0000 7f3113fff700 1 -- 172.21.15.116:0/3219195876 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f312c070d90 con 0x7f31341557e0 2020-11-20T12:16:23.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.324+0000 7f3133fff700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f310c0557c0 0x7f310c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:23.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.328+0000 7f3133fff700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f310c0557c0 0x7f310c057c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f31240010e0 tx=0x7f3124007ac0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:23.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.336+0000 7f3113fff700 1 -- 172.21.15.116:0/3219195876 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f312c076020 con 0x7f31341557e0 2020-11-20T12:16:23.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.544+0000 7f3139672700 1 -- 172.21.15.116:0/3219195876 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3134044f60 con 0x7f31341557e0 2020-11-20T12:16:23.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.544+0000 7f3113fff700 1 -- 172.21.15.116:0/3219195876 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2846 (secure 0 0 0) 0x7f312c015070 con 0x7f31341557e0 2020-11-20T12:16:23.547 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:23.548 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/2968741298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":2968741298},{"type":"v1","addr":"172.21.15.116:6835","nonce":2968741298}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1214743116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1214743116},{"type":"v1","addr":"172.21.15.173:6836","nonce":1214743116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1424291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1424291148},{"type":"v1","addr":"172.21.15.173:6837","nonce":1424291148}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:00.619934+0000","modified":"2020-11-20T12:16:02.636079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.116:6834/3819756176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3819756176},{"type":"v1","addr":"172.21.15.116:6834","nonce":3819756176}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:16:23.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 -- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f310c0557c0 msgr2=0x7f310c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:23.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f310c0557c0 0x7f310c057c80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f31240010e0 tx=0x7f3124007ac0).stop 2020-11-20T12:16:23.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 -- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31341557e0 msgr2=0x7f31340c06f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:23.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31341557e0 0x7f31340c06f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f312c009cc0 tx=0x7f312c016000).stop 2020-11-20T12:16:23.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 -- 172.21.15.116:0/3219195876 shutdown_connections 2020-11-20T12:16:23.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f310c0557c0 0x7f310c057c80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:23.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f31341557e0 0x7f31340c06f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:23.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f313414ee30 0x7f3134156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:23.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 --2- 172.21.15.116:0/3219195876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f313414f210 0x7f3134160540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:23.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 -- 172.21.15.116:0/3219195876 >> 172.21.15.116:0/3219195876 conn(0x7f3134061280 msgr2=0x7f31340c2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:23.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 -- 172.21.15.116:0/3219195876 shutdown_connections 2020-11-20T12:16:23.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:23.548+0000 7f3139672700 1 -- 172.21.15.116:0/3219195876 wait complete. 2020-11-20T12:16:23.553 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 6 2020-11-20T12:16:23.978 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:00.619934+0000', 'modified': '2020-11-20T12:16:02.636079+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4245}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4245': {'gid': 4245, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.116:6834/3819756176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 3819756176}, {'type': 'v1', 'addr': '172.21.15.116:6834', 'nonce': 3819756176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:16:23.979 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:16:23.981 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:16:24.189 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4245,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":23.549402542999999,"mdsmap_epoch":6,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":27.037232757999998} 2020-11-20T12:16:24.190 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4245, "mdsmap_epoch": 6, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 23.549402543, "state": "up:active", "uptime": 27.037232758, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:16:24.190 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:16:24.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:24.191 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.2 2020-11-20T12:16:24.217 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:16:24.218 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:24.343 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:24.343 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:16:24.364 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:24.364 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:16:24.369 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:16:24.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:24.370 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2020-11-20T12:16:24.415 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:16:24.415 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:16:24.416 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-20T12:16:24.416 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-20T12:16:24.416 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:16:24.417 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2020-11-20T12:16:24.417 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:16:24.417 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:16:24.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:24.418 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:16:24.440 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:16:24.441 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:16:24.441 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:16:24.441 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:24.442 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:16:24.442 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:24.442 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:24.442 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:24.442 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:24.443 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:24.443 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:24.443 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:24.443 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:16:24.443 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:24.444 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:16:24.444 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:24.444 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:16:24.444 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:24.445 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:16:24.445 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:24.445 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:16:24.445 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:24.446 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2020-11-20T12:16:24.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:24.447 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:24.448 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:16:24.448 INFO:teuthology.orchestra.run.smithi173:> sudo ip link add name ceph-brx type bridge 2020-11-20T12:16:24.448 INFO:teuthology.orchestra.run.smithi173:> sudo ip addr flush dev ceph-brx 2020-11-20T12:16:24.449 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set ceph-brx up 2020-11-20T12:16:24.449 INFO:teuthology.orchestra.run.smithi173:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2020-11-20T12:16:24.449 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:16:24.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:24.612 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:16:24.651 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:16:24.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:24.654 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:16:24.753 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:16:24.754 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:16:24.754 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:16:24.754 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:16:24.755 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:16:24.755 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:16:24.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:24.756 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:24.756 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:16:24.756 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:16:24.757 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:16:24.757 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:16:24.757 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:16:24.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:24.856 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:16:24.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:24.871 INFO:teuthology.orchestra.run.smithi173:> ip netns list-id 2020-11-20T12:16:24.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:24.890 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:24.891 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:16:24.891 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2020-11-20T12:16:24.891 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2020-11-20T12:16:24.892 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:16:25.014 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2020-11-20T12:16:25.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:25.015 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:25.015 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:16:25.015 INFO:teuthology.orchestra.run.smithi173:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2020-11-20T12:16:25.015 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2020-11-20T12:16:25.016 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2020-11-20T12:16:25.016 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2020-11-20T12:16:25.016 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2020-11-20T12:16:25.017 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:16:25.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:25.207 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:25.208 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:16:25.208 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2020-11-20T12:16:25.208 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set dev brx.0 master ceph-brx 2020-11-20T12:16:25.209 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:16:25.290 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:25.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.468+0000 7f18717e8700 1 Processor -- start 2020-11-20T12:16:25.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.468+0000 7f18717e8700 1 -- start start 2020-11-20T12:16:25.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.468+0000 7f18717e8700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f186c0c5c70 0x7f186c1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.468+0000 7f18717e8700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186c0abd80 con 0x7f186c0aa060 2020-11-20T12:16:25.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.468+0000 7f18717e8700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f186c0ac020 con 0x7f186c0a9c80 2020-11-20T12:16:25.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.468+0000 7f18717e8700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c0afaf0 con 0x7f186c0c5c70 2020-11-20T12:16:25.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.468+0000 7f186a7fc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f186c0aa060 0x7f186c064120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53064/0 (socket says 172.21.15.116:53064) 2020-11-20T12:16:25.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.468+0000 7f186a7fc700 1 -- 172.21.15.116:0/368200110 learned_addr learned my addr 172.21.15.116:0/368200110 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:25.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.468+0000 7f186bfff700 1 --2- 172.21.15.116:0/368200110 >> v2:172.21.15.116:3300/0 conn(0x7f186c0c5c70 0x7f186c1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f1869ffb700 1 -- 172.21.15.116:0/368200110 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018303157 0 0) 0x7f186c0abd80 con 0x7f186c0aa060 2020-11-20T12:16:25.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f1869ffb700 1 -- 172.21.15.116:0/368200110 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1858003240 con 0x7f186c0aa060 2020-11-20T12:16:25.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f186bfff700 1 -- 172.21.15.116:0/368200110 >> v1:172.21.15.173:6789/0 conn(0x7f186c0a9c80 legacy=0x7f186c0ab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f1869ffb700 1 -- 172.21.15.116:0/368200110 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662665960 0 0) 0x7f186c0ac020 con 0x7f186c0a9c80 2020-11-20T12:16:25.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f186bfff700 1 -- 172.21.15.116:0/368200110 >> v1:172.21.15.116:6789/0 conn(0x7f186c0aa060 legacy=0x7f186c064120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f186bfff700 1 -- 172.21.15.116:0/368200110 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c0ac2c0 con 0x7f186c0c5c70 2020-11-20T12:16:25.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f186bfff700 1 --2- 172.21.15.116:0/368200110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c0c5c70 0x7f186c1551a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f1864009c90 tx=0x7f18640091d0).ready entity=mon.0 client_cookie=1dd405f51e56b9b7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:25.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f1869ffb700 1 -- 172.21.15.116:0/368200110 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f186400ba70 con 0x7f186c0c5c70 2020-11-20T12:16:25.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f1869ffb700 1 -- 172.21.15.116:0/368200110 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f186400bbd0 con 0x7f186c0c5c70 2020-11-20T12:16:25.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f1869ffb700 1 -- 172.21.15.116:0/368200110 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864011400 con 0x7f186c0c5c70 2020-11-20T12:16:25.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f18717e8700 1 -- 172.21.15.116:0/368200110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c0c5c70 msgr2=0x7f186c1551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f18717e8700 1 --2- 172.21.15.116:0/368200110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c0c5c70 0x7f186c1551a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f1864009c90 tx=0x7f18640091d0).stop 2020-11-20T12:16:25.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f18717e8700 1 -- 172.21.15.116:0/368200110 shutdown_connections 2020-11-20T12:16:25.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f18717e8700 1 --2- 172.21.15.116:0/368200110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c0c5c70 0x7f186c1551a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f18717e8700 1 -- 172.21.15.116:0/368200110 >> 172.21.15.116:0/368200110 conn(0x7f186c0c9a90 msgr2=0x7f186c0c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:25.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f18717e8700 1 -- 172.21.15.116:0/368200110 shutdown_connections 2020-11-20T12:16:25.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.472+0000 7f18717e8700 1 -- 172.21.15.116:0/368200110 wait complete. 2020-11-20T12:16:25.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.476+0000 7f18717e8700 1 Processor -- start 2020-11-20T12:16:25.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.476+0000 7f18717e8700 1 -- start start 2020-11-20T12:16:25.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.476+0000 7f18717e8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f186c0a9c80 0x7f186c14c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.476+0000 7f18717e8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f186c0aa060 0x7f186c1645b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.476+0000 7f18717e8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c0c5c70 0x7f186c164aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.476+0000 7f18717e8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c14ddc0 con 0x7f186c0c5c70 2020-11-20T12:16:25.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.476+0000 7f18717e8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c14e060 con 0x7f186c0aa060 2020-11-20T12:16:25.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.476+0000 7f18717e8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c14e300 con 0x7f186c0a9c80 2020-11-20T12:16:25.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f186bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c0c5c70 0x7f186c164aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f186bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c0c5c70 0x7f186c164aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43640/0 (socket says 172.21.15.116:43640) 2020-11-20T12:16:25.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f186bfff700 1 -- 172.21.15.116:0/2036563490 learned_addr learned my addr 172.21.15.116:0/2036563490 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:25.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f186affd700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f186c0a9c80 0x7f186c14c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f186a7fc700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f186c0aa060 0x7f186c1645b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f186bfff700 1 -- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f186c0a9c80 msgr2=0x7f186c14c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f186bfff700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f186c0a9c80 0x7f186c14c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f186bfff700 1 -- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f186c0aa060 msgr2=0x7f186c1645b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f186bfff700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f186c0aa060 0x7f186c1645b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f186bfff700 1 -- 172.21.15.116:0/2036563490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c14e5a0 con 0x7f186c0c5c70 2020-11-20T12:16:25.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f186bfff700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c0c5c70 0x7f186c164aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f186c0c7590 tx=0x7f1864016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:25.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f1852ffd700 1 -- 172.21.15.116:0/2036563490 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864010700 con 0x7f186c0c5c70 2020-11-20T12:16:25.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f1852ffd700 1 -- 172.21.15.116:0/2036563490 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1864010860 con 0x7f186c0c5c70 2020-11-20T12:16:25.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f1852ffd700 1 -- 172.21.15.116:0/2036563490 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1864010be0 con 0x7f186c0c5c70 2020-11-20T12:16:25.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f18717e8700 1 -- 172.21.15.116:0/2036563490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186c14ce10 con 0x7f186c0c5c70 2020-11-20T12:16:25.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f18717e8700 1 -- 172.21.15.116:0/2036563490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c14f3e0 con 0x7f186c0c5c70 2020-11-20T12:16:25.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.480+0000 7f18717e8700 1 -- 172.21.15.116:0/2036563490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186c0af280 con 0x7f186c0c5c70 2020-11-20T12:16:25.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.484+0000 7f1852ffd700 1 -- 172.21.15.116:0/2036563490 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1864024030 con 0x7f186c0c5c70 2020-11-20T12:16:25.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.484+0000 7f1852ffd700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18480557c0 0x7f1848057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.484+0000 7f1852ffd700 1 -- 172.21.15.116:0/2036563490 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1864070260 con 0x7f186c0c5c70 2020-11-20T12:16:25.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.484+0000 7f186affd700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18480557c0 0x7f1848057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.484+0000 7f186affd700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18480557c0 0x7f1848057c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f185c007fd0 tx=0x7f185c007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:25.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.492+0000 7f1852ffd700 1 -- 172.21.15.116:0/2036563490 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f186c0af280 con 0x7f186c0c5c70 2020-11-20T12:16:25.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.656+0000 7f18717e8700 1 -- 172.21.15.116:0/2036563490 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f186c0af280 con 0x7f186c0c5c70 2020-11-20T12:16:25.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.656+0000 7f1852ffd700 1 -- 172.21.15.116:0/2036563490 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f1864030200 con 0x7f186c0c5c70 2020-11-20T12:16:25.658 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:25.659 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:25.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 -- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18480557c0 msgr2=0x7f1848057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18480557c0 0x7f1848057c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f185c007fd0 tx=0x7f185c007a00).stop 2020-11-20T12:16:25.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 -- 172.21.15.116:0/2036563490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c0c5c70 msgr2=0x7f186c164aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c0c5c70 0x7f186c164aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f186c0c7590 tx=0x7f1864016000).stop 2020-11-20T12:16:25.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 -- 172.21.15.116:0/2036563490 shutdown_connections 2020-11-20T12:16:25.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18480557c0 0x7f1848057c80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f186c0c5c70 0x7f186c164aa0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f186c0a9c80 0x7f186c14c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 --2- 172.21.15.116:0/2036563490 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f186c0aa060 0x7f186c1645b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 -- 172.21.15.116:0/2036563490 >> 172.21.15.116:0/2036563490 conn(0x7f186c0c9a90 msgr2=0x7f186c0acd90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 -- 172.21.15.116:0/2036563490 shutdown_connections 2020-11-20T12:16:25.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.660+0000 7f18717e8700 1 -- 172.21.15.116:0/2036563490 wait complete. 2020-11-20T12:16:25.686 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:15:59.065285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:15:59.602311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:16:25.687 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:25.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.868+0000 7f0945a43700 1 Processor -- start 2020-11-20T12:16:25.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.868+0000 7f0945a43700 1 -- start start 2020-11-20T12:16:25.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.868+0000 7f0945a43700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09400c1300 0x7f094015c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.868+0000 7f0945a43700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09400c1900 con 0x7f094014bc70 2020-11-20T12:16:25.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.868+0000 7f0945a43700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0940155530 con 0x7f094014c050 2020-11-20T12:16:25.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.868+0000 7f0945a43700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09401557d0 con 0x7f09400c1300 2020-11-20T12:16:25.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.868+0000 7f093effd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f094014bc70 0x7f09400678e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53078/0 (socket says 172.21.15.116:53078) 2020-11-20T12:16:25.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.868+0000 7f093effd700 1 -- 172.21.15.116:0/8378313 learned_addr learned my addr 172.21.15.116:0/8378313 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:25.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093f7fe700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09400c1300 0x7f094015c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2873828265 0 0) 0x7f09400c1900 con 0x7f094014bc70 2020-11-20T12:16:25.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f092c003240 con 0x7f094014bc70 2020-11-20T12:16:25.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 667033336 0 0) 0x7f092c003240 con 0x7f094014bc70 2020-11-20T12:16:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09400c1300 msgr2=0x7f094015c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09400c1300 0x7f094015c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 >> v1:172.21.15.173:6789/0 conn(0x7f094014c050 legacy=0x7f09400c0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0940155a70 con 0x7f094014bc70 2020-11-20T12:16:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f09300014e0 con 0x7f094014bc70 2020-11-20T12:16:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 >> v1:172.21.15.116:6789/0 conn(0x7f094014bc70 legacy=0x7f09400678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f092c003760 0x7f092c003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f092c0044e0 0x7f092c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092c004e50 0x7f092c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09401557d0 con 0x7f092c004e50 2020-11-20T12:16:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f092c004240 con 0x7f092c0044e0 2020-11-20T12:16:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f092c014430 con 0x7f092c003760 2020-11-20T12:16:25.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093f7fe700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092c004e50 0x7f092c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093e7fc700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f092c0044e0 0x7f092c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093effd700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f092c003760 0x7f092c003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093f7fe700 1 -- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f092c003760 msgr2=0x7f092c003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093f7fe700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f092c003760 0x7f092c003b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093f7fe700 1 -- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f092c0044e0 msgr2=0x7f092c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093f7fe700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f092c0044e0 0x7f092c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093f7fe700 1 -- 172.21.15.116:0/8378313 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f092c003240 con 0x7f092c004e50 2020-11-20T12:16:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093e7fc700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f092c0044e0 0x7f092c004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093f7fe700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092c004e50 0x7f092c00b730 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f0938009e60 tx=0x7f09380091d0).ready entity=mon.0 client_cookie=e51989454aa3ba14 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:25.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938009620 con 0x7f092c004e50 2020-11-20T12:16:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093800edc0 con 0x7f092c004e50 2020-11-20T12:16:25.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.872+0000 7f093dffb700 1 -- 172.21.15.116:0/8378313 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938012600 con 0x7f092c004e50 2020-11-20T12:16:25.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.876+0000 7f0945a43700 1 -- 172.21.15.116:0/8378313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092c004e50 msgr2=0x7f092c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.876+0000 7f0945a43700 1 --2- 172.21.15.116:0/8378313 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f092c004e50 0x7f092c00b730 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f0938009e60 tx=0x7f09380091d0).stop 2020-11-20T12:16:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.876+0000 7f093f7fe700 1 -- 172.21.15.116:0/8378313 reap_dead start 2020-11-20T12:16:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.876+0000 7f0945a43700 1 -- 172.21.15.116:0/8378313 shutdown_connections 2020-11-20T12:16:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.876+0000 7f0945a43700 1 -- 172.21.15.116:0/8378313 >> 172.21.15.116:0/8378313 conn(0x7f0940060170 msgr2=0x7f0940060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.876+0000 7f0945a43700 1 -- 172.21.15.116:0/8378313 shutdown_connections 2020-11-20T12:16:25.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.876+0000 7f0945a43700 1 -- 172.21.15.116:0/8378313 wait complete. 2020-11-20T12:16:25.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.876+0000 7f0945a43700 1 Processor -- start 2020-11-20T12:16:25.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.876+0000 7f0945a43700 1 -- start start 2020-11-20T12:16:25.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f0945a43700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0940141db0 0x7f0940160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f0945a43700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09400bd010 0x7f0940153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f0945a43700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0940154150 0x7f0940142580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f0945a43700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09400a3cd0 con 0x7f0940154150 2020-11-20T12:16:25.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f0945a43700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09400bd7f0 con 0x7f0940141db0 2020-11-20T12:16:25.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f0945a43700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09400bda90 con 0x7f09400bd010 2020-11-20T12:16:25.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0940154150 0x7f0940142580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093effd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0940141db0 0x7f0940160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0940154150 0x7f0940142580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43660/0 (socket says 172.21.15.116:43660) 2020-11-20T12:16:25.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093f7fe700 1 -- 172.21.15.116:0/1019155024 learned_addr learned my addr 172.21.15.116:0/1019155024 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:25.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093e7fc700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09400bd010 0x7f0940153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093f7fe700 1 -- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09400bd010 msgr2=0x7f0940153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093f7fe700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09400bd010 0x7f0940153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093f7fe700 1 -- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0940141db0 msgr2=0x7f0940160d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:25.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093f7fe700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0940141db0 0x7f0940160d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:25.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093f7fe700 1 -- 172.21.15.116:0/1019155024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09400bdd30 con 0x7f0940154150 2020-11-20T12:16:25.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093effd700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0940141db0 0x7f0940160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:25.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f093f7fe700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0940154150 0x7f0940142580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f0938009e30 tx=0x7f0938019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:25.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f0926ffd700 1 -- 172.21.15.116:0/1019155024 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0938006ca0 con 0x7f0940154150 2020-11-20T12:16:25.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f0926ffd700 1 -- 172.21.15.116:0/1019155024 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f093801a400 con 0x7f0940154150 2020-11-20T12:16:25.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f0945a43700 1 -- 172.21.15.116:0/1019155024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09400bdfd0 con 0x7f0940154150 2020-11-20T12:16:25.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f0926ffd700 1 -- 172.21.15.116:0/1019155024 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093801a7a0 con 0x7f0940154150 2020-11-20T12:16:25.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.880+0000 7f0945a43700 1 -- 172.21.15.116:0/1019155024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09400bd490 con 0x7f0940154150 2020-11-20T12:16:25.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.884+0000 7f0945a43700 1 -- 172.21.15.116:0/1019155024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094005ca30 con 0x7f0940154150 2020-11-20T12:16:25.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.884+0000 7f0926ffd700 1 -- 172.21.15.116:0/1019155024 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0938025390 con 0x7f0940154150 2020-11-20T12:16:25.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.884+0000 7f0926ffd700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f091c0557c0 0x7f091c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:25.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.884+0000 7f0926ffd700 1 -- 172.21.15.116:0/1019155024 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0938070650 con 0x7f0940154150 2020-11-20T12:16:25.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.884+0000 7f093effd700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f091c0557c0 0x7f091c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:25.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.884+0000 7f093effd700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f091c0557c0 0x7f091c057c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f09300013b0 tx=0x7f0930009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:25.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:25.888+0000 7f0926ffd700 1 -- 172.21.15.116:0/1019155024 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0938046490 con 0x7f0940154150 2020-11-20T12:16:26.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.048+0000 7f0945a43700 1 -- 172.21.15.116:0/1019155024 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f0940061e80 con 0x7f0940154150 2020-11-20T12:16:26.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0926ffd700 1 -- 172.21.15.116:0/1019155024 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f0938049ab0 con 0x7f0940154150 2020-11-20T12:16:26.052 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:26.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 -- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f091c0557c0 msgr2=0x7f091c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f091c0557c0 0x7f091c057c80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f09300013b0 tx=0x7f0930009040).stop 2020-11-20T12:16:26.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 -- 172.21.15.116:0/1019155024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0940154150 msgr2=0x7f0940142580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0940154150 0x7f0940142580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f0938009e30 tx=0x7f0938019000).stop 2020-11-20T12:16:26.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 -- 172.21.15.116:0/1019155024 shutdown_connections 2020-11-20T12:16:26.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f091c0557c0 0x7f091c057c80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0940154150 0x7f0940142580 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0940141db0 0x7f0940160d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 --2- 172.21.15.116:0/1019155024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09400bd010 0x7f0940153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 -- 172.21.15.116:0/1019155024 >> 172.21.15.116:0/1019155024 conn(0x7f0940060170 msgr2=0x7f0940150cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:26.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 -- 172.21.15.116:0/1019155024 shutdown_connections 2020-11-20T12:16:26.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.052+0000 7f0945a43700 1 -- 172.21.15.116:0/1019155024 wait complete. 2020-11-20T12:16:26.077 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:26.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.256+0000 7f00d2669700 1 Processor -- start 2020-11-20T12:16:26.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.256+0000 7f00d2669700 1 -- start start 2020-11-20T12:16:26.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.260+0000 7f00d2669700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00cc156020 0x7f00cc0c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.260+0000 7f00d2669700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f00cc14c020 0x7f00cc0c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.260+0000 7f00d2669700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00cc152480 con 0x7f00cc14bc40 2020-11-20T12:16:26.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.260+0000 7f00d2669700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00cc0c56e0 con 0x7f00cc14c020 2020-11-20T12:16:26.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.260+0000 7f00d2669700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1526b0 con 0x7f00cc156020 2020-11-20T12:16:26.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.260+0000 7f00cbfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f00cc14bc40 0x7f00cc151d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53098/0 (socket says 172.21.15.116:53098) 2020-11-20T12:16:26.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.260+0000 7f00cbfff700 1 -- 172.21.15.116:0/2192494916 learned_addr learned my addr 172.21.15.116:0/2192494916 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:26.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.260+0000 7f00d0c06700 1 --2- 172.21.15.116:0/2192494916 >> v2:172.21.15.173:3300/0 conn(0x7f00cc14c020 0x7f00cc0c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.260+0000 7f00c37fe700 1 --2- 172.21.15.116:0/2192494916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00cc156020 0x7f00cc0c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.260+0000 7f00cb7fe700 1 -- 172.21.15.116:0/2192494916 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2274716702 0 0) 0x7f00cc152480 con 0x7f00cc14bc40 2020-11-20T12:16:26.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.260+0000 7f00cb7fe700 1 -- 172.21.15.116:0/2192494916 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00ac003240 con 0x7f00cc14bc40 2020-11-20T12:16:26.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d0c06700 1 -- 172.21.15.116:0/2192494916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00cc156020 msgr2=0x7f00cc0c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:26.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d0c06700 1 --2- 172.21.15.116:0/2192494916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00cc156020 0x7f00cc0c4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d0c06700 1 -- 172.21.15.116:0/2192494916 >> v1:172.21.15.116:6789/0 conn(0x7f00cc14bc40 legacy=0x7f00cc151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d0c06700 1 -- 172.21.15.116:0/2192494916 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc152950 con 0x7f00cc14c020 2020-11-20T12:16:26.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00c37fe700 1 --2- 172.21.15.116:0/2192494916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00cc156020 0x7f00cc0c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:26.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d0c06700 1 --2- 172.21.15.116:0/2192494916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00cc14c020 0x7f00cc0c51f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f00c4009c90 tx=0x7f00c40091d0).ready entity=mon.1 client_cookie=f7f39b5ed4f056ef server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:26.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00cb7fe700 1 -- 172.21.15.116:0/2192494916 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c400b9c0 con 0x7f00cc14c020 2020-11-20T12:16:26.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00cb7fe700 1 -- 172.21.15.116:0/2192494916 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00c400bb20 con 0x7f00cc14c020 2020-11-20T12:16:26.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00cb7fe700 1 -- 172.21.15.116:0/2192494916 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00c400bea0 con 0x7f00cc14c020 2020-11-20T12:16:26.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d2669700 1 -- 172.21.15.116:0/2192494916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00cc14c020 msgr2=0x7f00cc0c51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d2669700 1 --2- 172.21.15.116:0/2192494916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00cc14c020 0x7f00cc0c51f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f00c4009c90 tx=0x7f00c40091d0).stop 2020-11-20T12:16:26.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d2669700 1 -- 172.21.15.116:0/2192494916 shutdown_connections 2020-11-20T12:16:26.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d2669700 1 --2- 172.21.15.116:0/2192494916 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00cc14c020 0x7f00cc0c51f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d2669700 1 --2- 172.21.15.116:0/2192494916 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00cc156020 0x7f00cc0c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d2669700 1 -- 172.21.15.116:0/2192494916 >> 172.21.15.116:0/2192494916 conn(0x7f00cc0678a0 msgr2=0x7f00cc067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:26.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d2669700 1 -- 172.21.15.116:0/2192494916 shutdown_connections 2020-11-20T12:16:26.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.264+0000 7f00d2669700 1 -- 172.21.15.116:0/2192494916 wait complete. 2020-11-20T12:16:26.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.268+0000 7f00d2669700 1 Processor -- start 2020-11-20T12:16:26.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.268+0000 7f00d2669700 1 -- start start 2020-11-20T12:16:26.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00d2669700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00cc14bc40 0x7f00cc149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00d2669700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00cc14c020 0x7f00cc0ab610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00d2669700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00cc156020 0x7f00cc14ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00d2669700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc161330 con 0x7f00cc14c020 2020-11-20T12:16:26.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00d2669700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc1615d0 con 0x7f00cc14bc40 2020-11-20T12:16:26.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00d2669700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00cc161870 con 0x7f00cc156020 2020-11-20T12:16:26.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c37fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00cc14c020 0x7f00cc0ab610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c37fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00cc14c020 0x7f00cc0ab610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43672/0 (socket says 172.21.15.116:43672) 2020-11-20T12:16:26.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c37fe700 1 -- 172.21.15.116:0/4088683610 learned_addr learned my addr 172.21.15.116:0/4088683610 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:26.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00d0c06700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00cc156020 0x7f00cc14ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00cbfff700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00cc14bc40 0x7f00cc149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c37fe700 1 -- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00cc156020 msgr2=0x7f00cc14ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c37fe700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00cc156020 0x7f00cc14ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c37fe700 1 -- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00cc14bc40 msgr2=0x7f00cc149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c37fe700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00cc14bc40 0x7f00cc149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c37fe700 1 -- 172.21.15.116:0/4088683610 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00cc161b10 con 0x7f00cc14c020 2020-11-20T12:16:26.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c37fe700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00cc14c020 0x7f00cc0ab610 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f00b400eca0 tx=0x7f00b4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:26.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c8ff9700 1 -- 172.21.15.116:0/4088683610 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b400cbb0 con 0x7f00cc14c020 2020-11-20T12:16:26.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c8ff9700 1 -- 172.21.15.116:0/4088683610 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b400cd10 con 0x7f00cc14c020 2020-11-20T12:16:26.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00c8ff9700 1 -- 172.21.15.116:0/4088683610 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b4006240 con 0x7f00cc14c020 2020-11-20T12:16:26.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00d2669700 1 -- 172.21.15.116:0/4088683610 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00cc0c6400 con 0x7f00cc14c020 2020-11-20T12:16:26.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00d2669700 1 -- 172.21.15.116:0/4088683610 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00cc0abb00 con 0x7f00cc14c020 2020-11-20T12:16:26.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.272+0000 7f00d2669700 1 -- 172.21.15.116:0/4088683610 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00cc05ca30 con 0x7f00cc14c020 2020-11-20T12:16:26.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.276+0000 7f00c8ff9700 1 -- 172.21.15.116:0/4088683610 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f00b4018070 con 0x7f00cc14c020 2020-11-20T12:16:26.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.276+0000 7f00c8ff9700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00a80557f0 0x7f00a8057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.276+0000 7f00c8ff9700 1 -- 172.21.15.116:0/4088683610 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f00b406e7b0 con 0x7f00cc14c020 2020-11-20T12:16:26.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.276+0000 7f00cbfff700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00a80557f0 0x7f00a8057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.276+0000 7f00cbfff700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00a80557f0 0x7f00a8057cb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f00bc0055a0 tx=0x7f00bc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:26.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.284+0000 7f00c8ff9700 1 -- 172.21.15.116:0/4088683610 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f00b4044650 con 0x7f00cc14c020 2020-11-20T12:16:26.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.440+0000 7f00d2669700 1 -- 172.21.15.116:0/4088683610 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f00cc161ca0 con 0x7f00cc14c020 2020-11-20T12:16:26.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.440+0000 7f00c8ff9700 1 -- 172.21.15.116:0/4088683610 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f00b4047c70 con 0x7f00cc14c020 2020-11-20T12:16:26.443 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:26.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 -- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00a80557f0 msgr2=0x7f00a8057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00a80557f0 0x7f00a8057cb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f00bc0055a0 tx=0x7f00bc009040).stop 2020-11-20T12:16:26.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 -- 172.21.15.116:0/4088683610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00cc14c020 msgr2=0x7f00cc0ab610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00cc14c020 0x7f00cc0ab610 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f00b400eca0 tx=0x7f00b4008040).stop 2020-11-20T12:16:26.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 -- 172.21.15.116:0/4088683610 shutdown_connections 2020-11-20T12:16:26.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00a80557f0 0x7f00a8057cb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00cc156020 0x7f00cc14ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00cc14bc40 0x7f00cc149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 --2- 172.21.15.116:0/4088683610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00cc14c020 0x7f00cc0ab610 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 -- 172.21.15.116:0/4088683610 >> 172.21.15.116:0/4088683610 conn(0x7f00cc0678a0 msgr2=0x7f00cc1510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:26.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 -- 172.21.15.116:0/4088683610 shutdown_connections 2020-11-20T12:16:26.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.444+0000 7f00d2669700 1 -- 172.21.15.116:0/4088683610 wait complete. 2020-11-20T12:16:26.471 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:16:26.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.652+0000 7f4a4a857700 1 Processor -- start 2020-11-20T12:16:26.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.652+0000 7f4a4a857700 1 -- start start 2020-11-20T12:16:26.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a4a857700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a4414f150 0x7f4a440c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a4a857700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4a440bd780 0x7f4a440c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a4a857700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a4414f530 0x7f4a440a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a4a857700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a44155620 con 0x7f4a4414f530 2020-11-20T12:16:26.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a4a857700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a441558c0 con 0x7f4a440bd780 2020-11-20T12:16:26.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a4a857700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a44155b60 con 0x7f4a4414f150 2020-11-20T12:16:26.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a48df4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a4414f530 0x7f4a440a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a48df4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a4414f530 0x7f4a440a74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43680/0 (socket says 172.21.15.116:43680) 2020-11-20T12:16:26.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a437fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4a440bd780 0x7f4a440c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a48df4700 1 -- 172.21.15.116:0/1149151078 learned_addr learned my addr 172.21.15.116:0/1149151078 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:26.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a43fff700 1 --2- 172.21.15.116:0/1149151078 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a4414f150 0x7f4a440c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a48df4700 1 -- 172.21.15.116:0/1149151078 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a4414f150 msgr2=0x7f4a440c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:26.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a48df4700 1 --2- 172.21.15.116:0/1149151078 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a4414f150 0x7f4a440c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a48df4700 1 -- 172.21.15.116:0/1149151078 >> v2:172.21.15.173:3300/0 conn(0x7f4a440bd780 msgr2=0x7f4a440c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:26.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a48df4700 1 --2- 172.21.15.116:0/1149151078 >> v2:172.21.15.173:3300/0 conn(0x7f4a440bd780 0x7f4a440c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a48df4700 1 -- 172.21.15.116:0/1149151078 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a44155e00 con 0x7f4a4414f530 2020-11-20T12:16:26.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a43fff700 1 --2- 172.21.15.116:0/1149151078 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a4414f150 0x7f4a440c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:26.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a48df4700 1 --2- 172.21.15.116:0/1149151078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a4414f530 0x7f4a440a74b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f4a440c60c0 tx=0x7f4a3c009290).ready entity=mon.0 client_cookie=96376a26c75909e7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:26.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a42ffd700 1 -- 172.21.15.116:0/1149151078 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c00bc50 con 0x7f4a4414f530 2020-11-20T12:16:26.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a42ffd700 1 -- 172.21.15.116:0/1149151078 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c00bdb0 con 0x7f4a4414f530 2020-11-20T12:16:26.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.656+0000 7f4a42ffd700 1 -- 172.21.15.116:0/1149151078 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c011610 con 0x7f4a4414f530 2020-11-20T12:16:26.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.660+0000 7f4a4a857700 1 -- 172.21.15.116:0/1149151078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a4414f530 msgr2=0x7f4a440a74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.660+0000 7f4a4a857700 1 --2- 172.21.15.116:0/1149151078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a4414f530 0x7f4a440a74b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f4a440c60c0 tx=0x7f4a3c009290).stop 2020-11-20T12:16:26.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.660+0000 7f4a4a857700 1 -- 172.21.15.116:0/1149151078 shutdown_connections 2020-11-20T12:16:26.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.660+0000 7f4a4a857700 1 --2- 172.21.15.116:0/1149151078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a4414f530 0x7f4a440a74b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.660+0000 7f4a4a857700 1 --2- 172.21.15.116:0/1149151078 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a4414f150 0x7f4a440c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.660+0000 7f4a4a857700 1 --2- 172.21.15.116:0/1149151078 >> v2:172.21.15.173:3300/0 conn(0x7f4a440bd780 0x7f4a440c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.660+0000 7f4a4a857700 1 -- 172.21.15.116:0/1149151078 >> 172.21.15.116:0/1149151078 conn(0x7f4a44061580 msgr2=0x7f4a44061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:26.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.660+0000 7f4a4a857700 1 -- 172.21.15.116:0/1149151078 shutdown_connections 2020-11-20T12:16:26.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.660+0000 7f4a4a857700 1 -- 172.21.15.116:0/1149151078 wait complete. 2020-11-20T12:16:26.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.660+0000 7f4a4a857700 1 Processor -- start 2020-11-20T12:16:26.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.660+0000 7f4a4a857700 1 -- start start 2020-11-20T12:16:26.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a4a857700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a440bd780 0x7f4a440c0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a4a857700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a4414f150 0x7f4a440c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a4a857700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a4414f530 0x7f4a4414d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a4a857700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440bfb10 con 0x7f4a440bd780 2020-11-20T12:16:26.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a4a857700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440bfdb0 con 0x7f4a4414f150 2020-11-20T12:16:26.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a4a857700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a440c0050 con 0x7f4a4414f530 2020-11-20T12:16:26.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a437fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a4414f150 0x7f4a440c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a43fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a440bd780 0x7f4a440c0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a437fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a4414f150 0x7f4a440c1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:42734/0 (socket says 172.21.15.116:42734) 2020-11-20T12:16:26.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a437fe700 1 -- 172.21.15.116:0/3790165080 learned_addr learned my addr 172.21.15.116:0/3790165080 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:26.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a48df4700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a4414f530 0x7f4a4414d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a437fe700 1 -- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a4414f530 msgr2=0x7f4a4414d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a437fe700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a4414f530 0x7f4a4414d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a437fe700 1 -- 172.21.15.116:0/3790165080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a440bd780 msgr2=0x7f4a440c0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a437fe700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a440bd780 0x7f4a440c0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a437fe700 1 -- 172.21.15.116:0/3790165080 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a440c02f0 con 0x7f4a4414f150 2020-11-20T12:16:26.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a43fff700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a440bd780 0x7f4a440c0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:26.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a437fe700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a4414f150 0x7f4a440c1ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f4a440bdb80 tx=0x7f4a38006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:26.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a23fff700 1 -- 172.21.15.116:0/3790165080 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3800c820 con 0x7f4a4414f150 2020-11-20T12:16:26.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a23fff700 1 -- 172.21.15.116:0/3790165080 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3800c980 con 0x7f4a4414f150 2020-11-20T12:16:26.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a4a857700 1 -- 172.21.15.116:0/3790165080 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a440c0590 con 0x7f4a4414f150 2020-11-20T12:16:26.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a23fff700 1 -- 172.21.15.116:0/3790165080 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3800cd00 con 0x7f4a4414f150 2020-11-20T12:16:26.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.664+0000 7f4a4a857700 1 -- 172.21.15.116:0/3790165080 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a440c1fb0 con 0x7f4a4414f150 2020-11-20T12:16:26.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.668+0000 7f4a4a857700 1 -- 172.21.15.116:0/3790165080 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a440c0770 con 0x7f4a4414f150 2020-11-20T12:16:26.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.668+0000 7f4a23fff700 1 -- 172.21.15.116:0/3790165080 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4a3801a030 con 0x7f4a4414f150 2020-11-20T12:16:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.668+0000 7f4a23fff700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a1c0557c0 0x7f4a1c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.668+0000 7f4a23fff700 1 -- 172.21.15.116:0/3790165080 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4a3806e170 con 0x7f4a4414f150 2020-11-20T12:16:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.668+0000 7f4a43fff700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a1c0557c0 0x7f4a1c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.672+0000 7f4a43fff700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a1c0557c0 0x7f4a1c057c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f4a3400bf20 tx=0x7f4a34008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:26.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.676+0000 7f4a23fff700 1 -- 172.21.15.116:0/3790165080 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4a38044140 con 0x7f4a4414f150 2020-11-20T12:16:26.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.836+0000 7f4a4a857700 1 -- 172.21.15.116:0/3790165080 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a440c0720 con 0x7f4a4414f150 2020-11-20T12:16:26.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.840+0000 7f4a23fff700 1 -- 172.21.15.116:0/3790165080 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4a3800ce60 con 0x7f4a4414f150 2020-11-20T12:16:26.840 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:26.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.840+0000 7f4a4a857700 1 -- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a1c0557c0 msgr2=0x7f4a1c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.840+0000 7f4a4a857700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a1c0557c0 0x7f4a1c057c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f4a3400bf20 tx=0x7f4a34008040).stop 2020-11-20T12:16:26.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.840+0000 7f4a4a857700 1 -- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a4414f150 msgr2=0x7f4a440c1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:26.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.840+0000 7f4a4a857700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a4414f150 0x7f4a440c1ac0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f4a440bdb80 tx=0x7f4a38006df0).stop 2020-11-20T12:16:26.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.840+0000 7f4a4a857700 1 -- 172.21.15.116:0/3790165080 shutdown_connections 2020-11-20T12:16:26.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.840+0000 7f4a4a857700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a1c0557c0 0x7f4a1c057c80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.840+0000 7f4a4a857700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a4414f530 0x7f4a4414d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.840+0000 7f4a4a857700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a440bd780 0x7f4a440c0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.840+0000 7f4a4a857700 1 --2- 172.21.15.116:0/3790165080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a4414f150 0x7f4a440c1ac0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:26.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.840+0000 7f4a4a857700 1 -- 172.21.15.116:0/3790165080 >> 172.21.15.116:0/3790165080 conn(0x7f4a44061580 msgr2=0x7f4a4414f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:26.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.844+0000 7f4a4a857700 1 -- 172.21.15.116:0/3790165080 shutdown_connections 2020-11-20T12:16:26.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:26.844+0000 7f4a4a857700 1 -- 172.21.15.116:0/3790165080 wait complete. 2020-11-20T12:16:26.870 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:16:27.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.048+0000 7fd3b82e8700 1 Processor -- start 2020-11-20T12:16:27.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.048+0000 7fd3b82e8700 1 -- start start 2020-11-20T12:16:27.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.052+0000 7fd3b82e8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3b014f150 0x7fd3b00c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.052+0000 7fd3b82e8700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd3b014f530 0x7fd3b00c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.052+0000 7fd3b82e8700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b0155620 con 0x7fd3b00bd780 2020-11-20T12:16:27.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.052+0000 7fd3b82e8700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b01558c0 con 0x7fd3b014f530 2020-11-20T12:16:27.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.052+0000 7fd3b82e8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0155b60 con 0x7fd3b014f150 2020-11-20T12:16:27.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.052+0000 7fd3b6084700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3b014f150 0x7fd3b00c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.052+0000 7fd3b5883700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fd3b00bd780 0x7fd3b00a74b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49354/0 (socket says 172.21.15.116:49354) 2020-11-20T12:16:27.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.052+0000 7fd3b5883700 1 -- 172.21.15.116:0/1138313423 learned_addr learned my addr 172.21.15.116:0/1138313423 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:27.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.052+0000 7fd3b6885700 1 --2- 172.21.15.116:0/1138313423 >> v2:172.21.15.173:3300/0 conn(0x7fd3b014f530 0x7fd3b00c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.052+0000 7fd3b5082700 1 -- 172.21.15.116:0/1138313423 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344279136 0 0) 0x7fd3b0155620 con 0x7fd3b00bd780 2020-11-20T12:16:27.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b5082700 1 -- 172.21.15.116:0/1138313423 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd398003240 con 0x7fd3b00bd780 2020-11-20T12:16:27.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b6084700 1 -- 172.21.15.116:0/1138313423 >> v2:172.21.15.173:3300/0 conn(0x7fd3b014f530 msgr2=0x7fd3b00c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:27.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b6084700 1 --2- 172.21.15.116:0/1138313423 >> v2:172.21.15.173:3300/0 conn(0x7fd3b014f530 0x7fd3b00c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b6084700 1 -- 172.21.15.116:0/1138313423 >> v1:172.21.15.173:6789/0 conn(0x7fd3b00bd780 legacy=0x7fd3b00a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:27.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b6084700 1 -- 172.21.15.116:0/1138313423 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b0155e00 con 0x7fd3b014f150 2020-11-20T12:16:27.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b6084700 1 --2- 172.21.15.116:0/1138313423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3b014f150 0x7fd3b00c87f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd3ac00b9a0 tx=0x7fd3ac006df0).ready entity=mon.2 client_cookie=dce61a7a1a13c9fd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:27.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b5082700 1 -- 172.21.15.116:0/1138313423 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac009710 con 0x7fd3b014f150 2020-11-20T12:16:27.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b5082700 1 -- 172.21.15.116:0/1138313423 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac009870 con 0x7fd3b014f150 2020-11-20T12:16:27.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b5082700 1 -- 172.21.15.116:0/1138313423 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac009bf0 con 0x7fd3b014f150 2020-11-20T12:16:27.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b82e8700 1 -- 172.21.15.116:0/1138313423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3b014f150 msgr2=0x7fd3b00c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:27.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b82e8700 1 --2- 172.21.15.116:0/1138313423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3b014f150 0x7fd3b00c87f0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fd3ac00b9a0 tx=0x7fd3ac006df0).stop 2020-11-20T12:16:27.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b82e8700 1 -- 172.21.15.116:0/1138313423 shutdown_connections 2020-11-20T12:16:27.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b82e8700 1 --2- 172.21.15.116:0/1138313423 >> v2:172.21.15.173:3300/0 conn(0x7fd3b014f530 0x7fd3b00c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b82e8700 1 --2- 172.21.15.116:0/1138313423 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3b014f150 0x7fd3b00c87f0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b82e8700 1 -- 172.21.15.116:0/1138313423 >> 172.21.15.116:0/1138313423 conn(0x7fd3b0061580 msgr2=0x7fd3b0061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:27.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b82e8700 1 -- 172.21.15.116:0/1138313423 shutdown_connections 2020-11-20T12:16:27.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.056+0000 7fd3b82e8700 1 -- 172.21.15.116:0/1138313423 wait complete. 2020-11-20T12:16:27.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.060+0000 7fd3b82e8700 1 Processor -- start 2020-11-20T12:16:27.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.060+0000 7fd3b82e8700 1 -- start start 2020-11-20T12:16:27.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.060+0000 7fd3b82e8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b00bd780 0x7fd3b00c3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.060+0000 7fd3b82e8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3b014f150 0x7fd3b0161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b82e8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b014f530 0x7fd3b01507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b82e8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0161200 con 0x7fd3b014f530 2020-11-20T12:16:27.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b82e8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0165320 con 0x7fd3b00bd780 2020-11-20T12:16:27.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b82e8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0161460 con 0x7fd3b014f150 2020-11-20T12:16:27.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b6885700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b014f530 0x7fd3b01507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b6885700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b014f530 0x7fd3b01507e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43698/0 (socket says 172.21.15.116:43698) 2020-11-20T12:16:27.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b6885700 1 -- 172.21.15.116:0/4126565855 learned_addr learned my addr 172.21.15.116:0/4126565855 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:27.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b5883700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3b014f150 0x7fd3b0161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b6084700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b00bd780 0x7fd3b00c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b6885700 1 -- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3b014f150 msgr2=0x7fd3b0161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:27.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b6885700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3b014f150 0x7fd3b0161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b6885700 1 -- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b00bd780 msgr2=0x7fd3b00c3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:27.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b6885700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b00bd780 0x7fd3b00c3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b6885700 1 -- 172.21.15.116:0/4126565855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b0161700 con 0x7fd3b014f530 2020-11-20T12:16:27.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b6885700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b014f530 0x7fd3b01507e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fd3a800ef50 tx=0x7fd3a80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:27.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd39e7fc700 1 -- 172.21.15.116:0/4126565855 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a800ccc0 con 0x7fd3b014f530 2020-11-20T12:16:27.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd39e7fc700 1 -- 172.21.15.116:0/4126565855 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a800ce20 con 0x7fd3b014f530 2020-11-20T12:16:27.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b82e8700 1 -- 172.21.15.116:0/4126565855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b0162090 con 0x7fd3b014f530 2020-11-20T12:16:27.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd39e7fc700 1 -- 172.21.15.116:0/4126565855 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8012620 con 0x7fd3b014f530 2020-11-20T12:16:27.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.064+0000 7fd3b82e8700 1 -- 172.21.15.116:0/4126565855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b0150cd0 con 0x7fd3b014f530 2020-11-20T12:16:27.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.068+0000 7fd3b82e8700 1 -- 172.21.15.116:0/4126565855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b0161890 con 0x7fd3b014f530 2020-11-20T12:16:27.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.068+0000 7fd39e7fc700 1 -- 172.21.15.116:0/4126565855 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd3a8022070 con 0x7fd3b014f530 2020-11-20T12:16:27.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.068+0000 7fd39e7fc700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 0x7fd388057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.068+0000 7fd39e7fc700 1 -- 172.21.15.116:0/4126565855 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd3a806ef30 con 0x7fd3b014f530 2020-11-20T12:16:27.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.068+0000 7fd3b6084700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 0x7fd388057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.072+0000 7fd3b6084700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 0x7fd388057c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fd3ac006e30 tx=0x7fd3ac015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:27.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.072+0000 7fd39e7fc700 1 -- 172.21.15.116:0/4126565855 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd3a8044dd0 con 0x7fd3b014f530 2020-11-20T12:16:27.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.236+0000 7fd3b82e8700 1 -- 172.21.15.116:0/4126565855 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3b005ca30 con 0x7fd3b014f530 2020-11-20T12:16:27.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.236+0000 7fd39e7fc700 1 -- 172.21.15.116:0/4126565855 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd3a8016030 con 0x7fd3b014f530 2020-11-20T12:16:27.238 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:27.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 -- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 msgr2=0x7fd388057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:27.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 0x7fd388057c80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fd3ac006e30 tx=0x7fd3ac015000).stop 2020-11-20T12:16:27.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 -- 172.21.15.116:0/4126565855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b014f530 msgr2=0x7fd3b01507e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:27.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b014f530 0x7fd3b01507e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fd3a800ef50 tx=0x7fd3a80091d0).stop 2020-11-20T12:16:27.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 -- 172.21.15.116:0/4126565855 shutdown_connections 2020-11-20T12:16:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 0x7fd388057c80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b014f530 0x7fd3b01507e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b00bd780 0x7fd3b00c3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 --2- 172.21.15.116:0/4126565855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3b014f150 0x7fd3b0161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 -- 172.21.15.116:0/4126565855 >> 172.21.15.116:0/4126565855 conn(0x7fd3b0061580 msgr2=0x7fd3b00c71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 -- 172.21.15.116:0/4126565855 shutdown_connections 2020-11-20T12:16:27.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.240+0000 7fd3b82e8700 1 -- 172.21.15.116:0/4126565855 wait complete. 2020-11-20T12:16:27.267 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:16:27.267 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:16:27.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9ebd1c6700 1 Processor -- start 2020-11-20T12:16:27.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9ebd1c6700 1 -- start start 2020-11-20T12:16:27.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9ebd1c6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9eb813aee0 0x7f9eb8152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9ebd1c6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9eb80aa790 0x7f9eb80ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9ebd1c6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb8152a30 con 0x7f9eb80612f0 2020-11-20T12:16:27.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9ebd1c6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb80aac10 con 0x7f9eb80aa790 2020-11-20T12:16:27.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9ebd1c6700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb80aaeb0 con 0x7f9eb813aee0 2020-11-20T12:16:27.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9eb77fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9eb80aa790 0x7f9eb80ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9eb77fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9eb80aa790 0x7f9eb80ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43706/0 (socket says 172.21.15.116:43706) 2020-11-20T12:16:27.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9eb77fe700 1 -- 172.21.15.116:0/216846188 learned_addr learned my addr 172.21.15.116:0/216846188 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:27.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9eb7fff700 1 --2- 172.21.15.116:0/216846188 >> v2:172.21.15.173:3300/0 conn(0x7f9eb813aee0 0x7f9eb8152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9eb77fe700 1 -- 172.21.15.116:0/216846188 >> v2:172.21.15.173:3300/0 conn(0x7f9eb813aee0 msgr2=0x7f9eb8152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:27.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9eb77fe700 1 --2- 172.21.15.116:0/216846188 >> v2:172.21.15.173:3300/0 conn(0x7f9eb813aee0 0x7f9eb8152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.448+0000 7f9eb77fe700 1 -- 172.21.15.116:0/216846188 >> v1:172.21.15.116:6789/0 conn(0x7f9eb80612f0 legacy=0x7f9eb80a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:27.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9eb77fe700 1 -- 172.21.15.116:0/216846188 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb80acbf0 con 0x7f9eb80aa790 2020-11-20T12:16:27.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9eb7fff700 1 --2- 172.21.15.116:0/216846188 >> v2:172.21.15.173:3300/0 conn(0x7f9eb813aee0 0x7f9eb8152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:27.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9eb77fe700 1 --2- 172.21.15.116:0/216846188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb80aa790 0x7f9eb80ad5d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f9eb8081470 tx=0x7f9eac006eb0).ready entity=mon.0 client_cookie=c5e45bfd8f18ab41 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:27.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9eb6ffd700 1 -- 172.21.15.116:0/216846188 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac0098d0 con 0x7f9eb80aa790 2020-11-20T12:16:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9eb6ffd700 1 -- 172.21.15.116:0/216846188 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac009a30 con 0x7f9eb80aa790 2020-11-20T12:16:27.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9eb6ffd700 1 -- 172.21.15.116:0/216846188 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac009db0 con 0x7f9eb80aa790 2020-11-20T12:16:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/216846188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb80aa790 msgr2=0x7f9eb80ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9ebd1c6700 1 --2- 172.21.15.116:0/216846188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb80aa790 0x7f9eb80ad5d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f9eb8081470 tx=0x7f9eac006eb0).stop 2020-11-20T12:16:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/216846188 shutdown_connections 2020-11-20T12:16:27.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9ebd1c6700 1 --2- 172.21.15.116:0/216846188 >> v2:172.21.15.173:3300/0 conn(0x7f9eb813aee0 0x7f9eb8152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9ebd1c6700 1 --2- 172.21.15.116:0/216846188 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb80aa790 0x7f9eb80ad5d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/216846188 >> 172.21.15.116:0/216846188 conn(0x7f9eb80678a0 msgr2=0x7f9eb8067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/216846188 shutdown_connections 2020-11-20T12:16:27.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.452+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/216846188 wait complete. 2020-11-20T12:16:27.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.456+0000 7f9ebd1c6700 1 Processor -- start 2020-11-20T12:16:27.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.456+0000 7f9ebd1c6700 1 -- start start 2020-11-20T12:16:27.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.456+0000 7f9ebd1c6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9eb80612f0 0x7f9eb81557e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.456+0000 7f9ebd1c6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb813aee0 0x7f9eb80c0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.456+0000 7f9ebd1c6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9eb80bdd70 0x7f9eb80be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.456+0000 7f9ebd1c6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80be2c0 con 0x7f9eb813aee0 2020-11-20T12:16:27.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.456+0000 7f9ebd1c6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb8152c60 con 0x7f9eb80bdd70 2020-11-20T12:16:27.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.456+0000 7f9ebd1c6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb8152f00 con 0x7f9eb80612f0 2020-11-20T12:16:27.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9eb77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb813aee0 0x7f9eb80c0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9ebc9c5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9eb80bdd70 0x7f9eb80be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9eb77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb813aee0 0x7f9eb80c0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43714/0 (socket says 172.21.15.116:43714) 2020-11-20T12:16:27.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9eb77fe700 1 -- 172.21.15.116:0/71746751 learned_addr learned my addr 172.21.15.116:0/71746751 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:27.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9eb7fff700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9eb80612f0 0x7f9eb81557e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9eb77fe700 1 -- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9eb80612f0 msgr2=0x7f9eb81557e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:27.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9eb77fe700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9eb80612f0 0x7f9eb81557e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9eb77fe700 1 -- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9eb80bdd70 msgr2=0x7f9eb80be6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:27.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9eb77fe700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9eb80bdd70 0x7f9eb80be6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9eb77fe700 1 -- 172.21.15.116:0/71746751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb81531a0 con 0x7f9eb813aee0 2020-11-20T12:16:27.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9eb77fe700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb813aee0 0x7f9eb80c0610 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f9eac010000 tx=0x7f9eac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:27.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9e97fff700 1 -- 172.21.15.116:0/71746751 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac011d50 con 0x7f9eb813aee0 2020-11-20T12:16:27.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9e97fff700 1 -- 172.21.15.116:0/71746751 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eac01e460 con 0x7f9eb813aee0 2020-11-20T12:16:27.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/71746751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb8153440 con 0x7f9eb813aee0 2020-11-20T12:16:27.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9e97fff700 1 -- 172.21.15.116:0/71746751 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eac01e7e0 con 0x7f9eb813aee0 2020-11-20T12:16:27.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/71746751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb80bf1a0 con 0x7f9eb813aee0 2020-11-20T12:16:27.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.460+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/71746751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb81535d0 con 0x7f9eb813aee0 2020-11-20T12:16:27.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.464+0000 7f9e97fff700 1 -- 172.21.15.116:0/71746751 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9eac025030 con 0x7f9eb813aee0 2020-11-20T12:16:27.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.464+0000 7f9e97fff700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e900557c0 0x7f9e90057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:27.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.464+0000 7f9e97fff700 1 -- 172.21.15.116:0/71746751 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9eac06fea0 con 0x7f9eb813aee0 2020-11-20T12:16:27.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.464+0000 7f9eb7fff700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e900557c0 0x7f9e90057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:27.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.468+0000 7f9eb7fff700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e900557c0 0x7f9e90057c80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f9ea800ca00 tx=0x7f9ea8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:27.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.472+0000 7f9e97fff700 1 -- 172.21.15.116:0/71746751 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9eac045dd0 con 0x7f9eb813aee0 2020-11-20T12:16:27.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.676+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/71746751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9eb8044f60 con 0x7f9eb813aee0 2020-11-20T12:16:27.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9e97fff700 1 -- 172.21.15.116:0/71746751 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2846 (secure 0 0 0) 0x7f9eac023030 con 0x7f9eb813aee0 2020-11-20T12:16:27.680 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:27.681 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/2968741298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":2968741298},{"type":"v1","addr":"172.21.15.116:6835","nonce":2968741298}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1214743116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1214743116},{"type":"v1","addr":"172.21.15.173:6836","nonce":1214743116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1424291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1424291148},{"type":"v1","addr":"172.21.15.173:6837","nonce":1424291148}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:00.619934+0000","modified":"2020-11-20T12:16:02.636079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.116:6834/3819756176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3819756176},{"type":"v1","addr":"172.21.15.116:6834","nonce":3819756176}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:16:27.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e900557c0 msgr2=0x7f9e90057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:27.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e900557c0 0x7f9e90057c80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f9ea800ca00 tx=0x7f9ea8008040).stop 2020-11-20T12:16:27.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/71746751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb813aee0 msgr2=0x7f9eb80c0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:27.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb813aee0 0x7f9eb80c0610 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f9eac010000 tx=0x7f9eac015000).stop 2020-11-20T12:16:27.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/71746751 shutdown_connections 2020-11-20T12:16:27.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e900557c0 0x7f9e90057c80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9eb80bdd70 0x7f9eb80be6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9eb80612f0 0x7f9eb81557e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 --2- 172.21.15.116:0/71746751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9eb813aee0 0x7f9eb80c0610 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:27.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/71746751 >> 172.21.15.116:0/71746751 conn(0x7f9eb80678a0 msgr2=0x7f9eb8139ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/71746751 shutdown_connections 2020-11-20T12:16:27.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:27.680+0000 7f9ebd1c6700 1 -- 172.21.15.116:0/71746751 wait complete. 2020-11-20T12:16:27.686 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 6 2020-11-20T12:16:28.108 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:00.619934+0000', 'modified': '2020-11-20T12:16:02.636079+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4245}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4245': {'gid': 4245, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.116:6834/3819756176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 3819756176}, {'type': 'v1', 'addr': '172.21.15.116:6834', 'nonce': 3819756176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:16:28.109 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:16:28.109 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:16:28.296 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4245,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":27.655526301999998,"mdsmap_epoch":6,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":31.143355540000002} 2020-11-20T12:16:28.297 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4245, "mdsmap_epoch": 6, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 27.655526302, "state": "up:active", "uptime": 31.14335554, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:16:28.298 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:16:28.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.298 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-20T12:16:28.318 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:16:28.318 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:28.398 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:28.398 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:16:28.506 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:28.507 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:16:28.514 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:16:28.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.515 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-20T12:16:28.543 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:16:28.543 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:16:28.543 INFO:tasks.cephfs.mount:self.client.name = client.3 2020-11-20T12:16:28.544 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2020-11-20T12:16:28.544 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:16:28.544 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2020-11-20T12:16:28.544 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:16:28.545 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:16:28.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.546 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:16:28.601 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:16:28.602 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:16:28.602 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:16:28.602 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.603 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:16:28.603 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.603 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:28.603 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:28.603 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:28.604 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:28.604 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:16:28.604 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:28.604 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:16:28.604 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:28.605 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:16:28.605 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.605 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:16:28.605 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.605 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:16:28.606 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:28.606 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:16:28.606 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.606 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:16:28.606 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:16:28.607 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:16:28.607 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.607 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:16:28.607 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.607 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:16:28.608 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:16:28.608 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:16:28.608 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.609 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:16:28.642 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:16:28.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.645 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:16:28.666 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:16:28.666 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:16:28.666 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:16:28.667 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:16:28.667 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:16:28.667 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:16:28.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.668 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:28.668 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:16:28.669 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:16:28.669 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:16:28.669 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:16:28.669 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:16:28.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.770 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:16:28.787 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:16:28.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.789 INFO:teuthology.orchestra.run.smithi173:> ip netns list-id 2020-11-20T12:16:28.799 INFO:teuthology.orchestra.run.smithi173.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2020-11-20T12:16:28.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.801 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:28.801 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:16:28.801 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2020-11-20T12:16:28.802 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2020-11-20T12:16:28.802 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:16:28.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.882 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2020-11-20T12:16:28.914 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:16:28.915 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:16:28.915 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:16:28.915 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.915 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:16:28.916 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.916 INFO:teuthology.orchestra.run.smithi173.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:16:28.916 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether da:40:0c:02:27:46 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:16:28.916 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-11-20T12:16:28.916 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.917 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::d840:cff:fe02:2746/64 scope link 2020-11-20T12:16:28.917 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.918 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2020-11-20T12:16:28.993 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:16:28.993 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:16:28.993 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:16:28.993 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.994 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:16:28.994 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.994 INFO:teuthology.orchestra.run.smithi173.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:16:28.994 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether da:40:0c:02:27:46 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:16:28.994 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2020-11-20T12:16:28.995 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.995 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::d840:cff:fe02:2746/64 scope link 2020-11-20T12:16:28.995 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:16:28.996 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2020-11-20T12:16:28.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:28.997 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:28.997 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:16:28.998 INFO:teuthology.orchestra.run.smithi173:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2020-11-20T12:16:28.998 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2020-11-20T12:16:28.998 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2020-11-20T12:16:28.998 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2020-11-20T12:16:28.999 INFO:teuthology.orchestra.run.smithi173:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2020-11-20T12:16:28.999 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:16:29.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:29.151 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:16:29.151 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:16:29.151 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2020-11-20T12:16:29.152 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set dev brx.1 master ceph-brx 2020-11-20T12:16:29.152 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:16:29.250 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:29.333 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:16:29.339 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:16:29.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.432+0000 7f4e9359e700 1 Processor -- start 2020-11-20T12:16:29.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.432+0000 7f4e9359e700 1 -- start start 2020-11-20T12:16:29.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.436+0000 7f4e9359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e9413aee0 0x7f4e94152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.436+0000 7f4e9359e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4e940aa790 0x7f4e940ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.436+0000 7f4e9359e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e940612f0 0x7f4e94061740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.436+0000 7f4e9359e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e94061c30 con 0x7f4e940612f0 2020-11-20T12:16:29.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.436+0000 7f4e9359e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e94152a30 con 0x7f4e940aa790 2020-11-20T12:16:29.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.436+0000 7f4e9359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940aac10 con 0x7f4e9413aee0 2020-11-20T12:16:29.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.436+0000 7f4e92d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e940612f0 0x7f4e94061740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.436+0000 7f4e92d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e940612f0 0x7f4e94061740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43720/0 (socket says 172.21.15.116:43720) 2020-11-20T12:16:29.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.436+0000 7f4e89d9b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4e940aa790 0x7f4e940ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.436+0000 7f4e92d9d700 1 -- 172.21.15.116:0/404996642 learned_addr learned my addr 172.21.15.116:0/404996642 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:29.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.436+0000 7f4e9259c700 1 --2- 172.21.15.116:0/404996642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e9413aee0 0x7f4e94152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e92d9d700 1 -- 172.21.15.116:0/404996642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e9413aee0 msgr2=0x7f4e94152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:29.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e92d9d700 1 --2- 172.21.15.116:0/404996642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e9413aee0 0x7f4e94152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e92d9d700 1 -- 172.21.15.116:0/404996642 >> v2:172.21.15.173:3300/0 conn(0x7f4e940aa790 msgr2=0x7f4e940ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:29.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e92d9d700 1 --2- 172.21.15.116:0/404996642 >> v2:172.21.15.173:3300/0 conn(0x7f4e940aa790 0x7f4e940ad5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e92d9d700 1 -- 172.21.15.116:0/404996642 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e940aaeb0 con 0x7f4e940612f0 2020-11-20T12:16:29.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e89d9b700 1 --2- 172.21.15.116:0/404996642 >> v2:172.21.15.173:3300/0 conn(0x7f4e940aa790 0x7f4e940ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:29.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e92d9d700 1 --2- 172.21.15.116:0/404996642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e940612f0 0x7f4e94061740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f4e940815b0 tx=0x7f4e8c009290).ready entity=mon.0 client_cookie=2296912a0bcf1aeb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:29.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e91d9b700 1 -- 172.21.15.116:0/404996642 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00bbc0 con 0x7f4e940612f0 2020-11-20T12:16:29.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e91d9b700 1 -- 172.21.15.116:0/404996642 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c00bd20 con 0x7f4e940612f0 2020-11-20T12:16:29.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e91d9b700 1 -- 172.21.15.116:0/404996642 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c011610 con 0x7f4e940612f0 2020-11-20T12:16:29.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e9359e700 1 -- 172.21.15.116:0/404996642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e940612f0 msgr2=0x7f4e94061740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:29.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e9359e700 1 --2- 172.21.15.116:0/404996642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e940612f0 0x7f4e94061740 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f4e940815b0 tx=0x7f4e8c009290).stop 2020-11-20T12:16:29.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e9359e700 1 -- 172.21.15.116:0/404996642 shutdown_connections 2020-11-20T12:16:29.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e9359e700 1 --2- 172.21.15.116:0/404996642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e940612f0 0x7f4e94061740 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e9359e700 1 --2- 172.21.15.116:0/404996642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e9413aee0 0x7f4e94152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e9359e700 1 --2- 172.21.15.116:0/404996642 >> v2:172.21.15.173:3300/0 conn(0x7f4e940aa790 0x7f4e940ad5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e9359e700 1 -- 172.21.15.116:0/404996642 >> 172.21.15.116:0/404996642 conn(0x7f4e940678a0 msgr2=0x7f4e94067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:29.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e9359e700 1 -- 172.21.15.116:0/404996642 shutdown_connections 2020-11-20T12:16:29.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.440+0000 7f4e9359e700 1 -- 172.21.15.116:0/404996642 wait complete. 2020-11-20T12:16:29.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e9359e700 1 Processor -- start 2020-11-20T12:16:29.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e9359e700 1 -- start start 2020-11-20T12:16:29.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e9359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e940612f0 0x7f4e94164490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e9359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e940aa790 0x7f4e940a1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e9359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e9413aee0 0x7f4e940a1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e9359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940a1ba0 con 0x7f4e940612f0 2020-11-20T12:16:29.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e9359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940a1e40 con 0x7f4e940aa790 2020-11-20T12:16:29.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e9359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e940a20e0 con 0x7f4e9413aee0 2020-11-20T12:16:29.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e9259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e940612f0 0x7f4e94164490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e9259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e940612f0 0x7f4e94164490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43724/0 (socket says 172.21.15.116:43724) 2020-11-20T12:16:29.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e9259c700 1 -- 172.21.15.116:0/3506163479 learned_addr learned my addr 172.21.15.116:0/3506163479 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:29.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.444+0000 7f4e92d9d700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e9413aee0 0x7f4e940a1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e89d9b700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e940aa790 0x7f4e940a1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e9259c700 1 -- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e9413aee0 msgr2=0x7f4e940a1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:29.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e9259c700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e9413aee0 0x7f4e940a1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e9259c700 1 -- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e940aa790 msgr2=0x7f4e940a1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:29.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e9259c700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e940aa790 0x7f4e940a1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e9259c700 1 -- 172.21.15.116:0/3506163479 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e940a2380 con 0x7f4e940612f0 2020-11-20T12:16:29.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e89d9b700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e940aa790 0x7f4e940a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e9259c700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e940612f0 0x7f4e94164490 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f4e8400d8e0 tx=0x7f4e84008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e8affd700 1 -- 172.21.15.116:0/3506163479 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84009820 con 0x7f4e940612f0 2020-11-20T12:16:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e8affd700 1 -- 172.21.15.116:0/3506163479 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e84009980 con 0x7f4e940612f0 2020-11-20T12:16:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e9359e700 1 -- 172.21.15.116:0/3506163479 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e940a2620 con 0x7f4e940612f0 2020-11-20T12:16:29.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e8affd700 1 -- 172.21.15.116:0/3506163479 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e84009d20 con 0x7f4e940612f0 2020-11-20T12:16:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e9359e700 1 -- 172.21.15.116:0/3506163479 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e94165030 con 0x7f4e940612f0 2020-11-20T12:16:29.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e9359e700 1 -- 172.21.15.116:0/3506163479 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e94044f60 con 0x7f4e940612f0 2020-11-20T12:16:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.448+0000 7f4e8affd700 1 -- 172.21.15.116:0/3506163479 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4e84013690 con 0x7f4e940612f0 2020-11-20T12:16:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.452+0000 7f4e8affd700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e640557c0 0x7f4e64057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.452+0000 7f4e8affd700 1 -- 172.21.15.116:0/3506163479 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4e8406eaa0 con 0x7f4e940612f0 2020-11-20T12:16:29.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.452+0000 7f4e89d9b700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e640557c0 0x7f4e64057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.452+0000 7f4e89d9b700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e640557c0 0x7f4e64057c80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f4e9412e970 tx=0x7f4e7c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:29.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.456+0000 7f4e8affd700 1 -- 172.21.15.116:0/3506163479 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4e840448c0 con 0x7f4e940612f0 2020-11-20T12:16:29.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.616+0000 7f4e9359e700 1 -- 172.21.15.116:0/3506163479 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4e94154d40 con 0x7f4e940612f0 2020-11-20T12:16:29.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e8affd700 1 -- 172.21.15.116:0/3506163479 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f4e84047ee0 con 0x7f4e940612f0 2020-11-20T12:16:29.620 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:29.620 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:29.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 -- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e640557c0 msgr2=0x7f4e64057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:29.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e640557c0 0x7f4e64057c80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f4e9412e970 tx=0x7f4e7c008040).stop 2020-11-20T12:16:29.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 -- 172.21.15.116:0/3506163479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e940612f0 msgr2=0x7f4e94164490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:29.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e940612f0 0x7f4e94164490 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f4e8400d8e0 tx=0x7f4e84008040).stop 2020-11-20T12:16:29.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 -- 172.21.15.116:0/3506163479 shutdown_connections 2020-11-20T12:16:29.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e640557c0 0x7f4e64057c80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e9413aee0 0x7f4e940a1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e940612f0 0x7f4e94164490 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 --2- 172.21.15.116:0/3506163479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e940aa790 0x7f4e940a1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 -- 172.21.15.116:0/3506163479 >> 172.21.15.116:0/3506163479 conn(0x7f4e940678a0 msgr2=0x7f4e940a8550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:29.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 -- 172.21.15.116:0/3506163479 shutdown_connections 2020-11-20T12:16:29.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.620+0000 7f4e9359e700 1 -- 172.21.15.116:0/3506163479 wait complete. 2020-11-20T12:16:29.647 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:15:59.065285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:15:59.602311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:16:29.648 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:29.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.824+0000 7f1867e17700 1 Processor -- start 2020-11-20T12:16:29.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.824+0000 7f1867e17700 1 -- start start 2020-11-20T12:16:29.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f1867e17700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f18600bf020 0x7f1860159f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f1867e17700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18600baff0 con 0x7f18600beb90 2020-11-20T12:16:29.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f1867e17700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1860066250 con 0x7f18601502e0 2020-11-20T12:16:29.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f1867e17700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1860067170 con 0x7f18600bf020 2020-11-20T12:16:29.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f18663b4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f18600bf020 0x7f1860159f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f18653b2700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f18600beb90 0x7f1860159820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53168/0 (socket says 172.21.15.116:53168) 2020-11-20T12:16:29.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f18653b2700 1 -- 172.21.15.116:0/1827489585 learned_addr learned my addr 172.21.15.116:0/1827489585 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:29.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f1864bb1700 1 -- 172.21.15.116:0/1827489585 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952738021 0 0) 0x7f18600baff0 con 0x7f18600beb90 2020-11-20T12:16:29.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f1864bb1700 1 -- 172.21.15.116:0/1827489585 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1848003240 con 0x7f18600beb90 2020-11-20T12:16:29.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f18663b4700 1 -- 172.21.15.116:0/1827489585 >> v1:172.21.15.173:6789/0 conn(0x7f18601502e0 legacy=0x7f18600ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:29.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f1864bb1700 1 -- 172.21.15.116:0/1827489585 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045604039 0 0) 0x7f1860066250 con 0x7f18601502e0 2020-11-20T12:16:29.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f18663b4700 1 -- 172.21.15.116:0/1827489585 >> v1:172.21.15.116:6789/0 conn(0x7f18600beb90 legacy=0x7f1860159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:29.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f18663b4700 1 -- 172.21.15.116:0/1827489585 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600664f0 con 0x7f18600bf020 2020-11-20T12:16:29.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f18663b4700 1 --2- 172.21.15.116:0/1827489585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18600bf020 0x7f1860159f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f1858009c90 tx=0x7f18580091d0).ready entity=mon.0 client_cookie=2a075b1969bf1048 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:29.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f1864bb1700 1 -- 172.21.15.116:0/1827489585 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185800bb00 con 0x7f18600bf020 2020-11-20T12:16:29.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.828+0000 7f1864bb1700 1 -- 172.21.15.116:0/1827489585 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185800bc60 con 0x7f18600bf020 2020-11-20T12:16:29.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.832+0000 7f1864bb1700 1 -- 172.21.15.116:0/1827489585 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858011400 con 0x7f18600bf020 2020-11-20T12:16:29.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.832+0000 7f1867e17700 1 -- 172.21.15.116:0/1827489585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18600bf020 msgr2=0x7f1860159f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:29.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.832+0000 7f1867e17700 1 --2- 172.21.15.116:0/1827489585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18600bf020 0x7f1860159f30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f1858009c90 tx=0x7f18580091d0).stop 2020-11-20T12:16:29.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.832+0000 7f1867e17700 1 -- 172.21.15.116:0/1827489585 shutdown_connections 2020-11-20T12:16:29.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.832+0000 7f1867e17700 1 --2- 172.21.15.116:0/1827489585 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18600bf020 0x7f1860159f30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.832+0000 7f1867e17700 1 -- 172.21.15.116:0/1827489585 >> 172.21.15.116:0/1827489585 conn(0x7f1860060170 msgr2=0x7f1860060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:29.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.832+0000 7f1867e17700 1 -- 172.21.15.116:0/1827489585 shutdown_connections 2020-11-20T12:16:29.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.832+0000 7f1867e17700 1 -- 172.21.15.116:0/1827489585 wait complete. 2020-11-20T12:16:29.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.836+0000 7f1867e17700 1 Processor -- start 2020-11-20T12:16:29.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.836+0000 7f1867e17700 1 -- start start 2020-11-20T12:16:29.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f1867e17700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600beb90 0x7f186014cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f1867e17700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18600bf020 0x7f18600a3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f1867e17700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18601502e0 0x7f1860165630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f1867e17700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18601660c0 con 0x7f18601502e0 2020-11-20T12:16:29.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f1867e17700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1860166360 con 0x7f18600bf020 2020-11-20T12:16:29.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f1867e17700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600a7320 con 0x7f18600beb90 2020-11-20T12:16:29.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f18663b4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18601502e0 0x7f1860165630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f18663b4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18601502e0 0x7f1860165630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43742/0 (socket says 172.21.15.116:43742) 2020-11-20T12:16:29.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f18663b4700 1 -- 172.21.15.116:0/366751379 learned_addr learned my addr 172.21.15.116:0/366751379 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:29.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f18653b2700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18600bf020 0x7f18600a3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f1865bb3700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600beb90 0x7f186014cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f18663b4700 1 -- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600beb90 msgr2=0x7f186014cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:29.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f18663b4700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600beb90 0x7f186014cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f18663b4700 1 -- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18600bf020 msgr2=0x7f18600a3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:29.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f18663b4700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18600bf020 0x7f18600a3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:29.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f18663b4700 1 -- 172.21.15.116:0/366751379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18600a75c0 con 0x7f18601502e0 2020-11-20T12:16:29.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f18653b2700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18600bf020 0x7f18600a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:29.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f18663b4700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18601502e0 0x7f1860165630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f1858009210 tx=0x7f1858016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:29.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f184dffb700 1 -- 172.21.15.116:0/366751379 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858021070 con 0x7f18601502e0 2020-11-20T12:16:29.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f184dffb700 1 -- 172.21.15.116:0/366751379 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1858010780 con 0x7f18601502e0 2020-11-20T12:16:29.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f184dffb700 1 -- 172.21.15.116:0/366751379 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1858010b40 con 0x7f18601502e0 2020-11-20T12:16:29.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f1867e17700 1 -- 172.21.15.116:0/366751379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186014d2e0 con 0x7f18601502e0 2020-11-20T12:16:29.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.840+0000 7f1867e17700 1 -- 172.21.15.116:0/366751379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18600a5ef0 con 0x7f18601502e0 2020-11-20T12:16:29.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.844+0000 7f1867e17700 1 -- 172.21.15.116:0/366751379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18600a7750 con 0x7f18601502e0 2020-11-20T12:16:29.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.844+0000 7f184dffb700 1 -- 172.21.15.116:0/366751379 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1858017070 con 0x7f18601502e0 2020-11-20T12:16:29.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.844+0000 7f184dffb700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 0x7f1838057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:29.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.844+0000 7f184dffb700 1 -- 172.21.15.116:0/366751379 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1858070b60 con 0x7f18601502e0 2020-11-20T12:16:29.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.844+0000 7f1865bb3700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 0x7f1838057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:29.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.848+0000 7f1865bb3700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 0x7f1838057c80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f185c0034b0 tx=0x7f185c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:29.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:29.852+0000 7f184dffb700 1 -- 172.21.15.116:0/366751379 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1858072e60 con 0x7f18601502e0 2020-11-20T12:16:30.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.012+0000 7f1867e17700 1 -- 172.21.15.116:0/366751379 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1860166520 con 0x7f18601502e0 2020-11-20T12:16:30.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.012+0000 7f184dffb700 1 -- 172.21.15.116:0/366751379 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f18580468f0 con 0x7f18601502e0 2020-11-20T12:16:30.014 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:30.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.012+0000 7f1867e17700 1 -- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 msgr2=0x7f1838057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.016+0000 7f1867e17700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 0x7f1838057c80 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f185c0034b0 tx=0x7f185c009040).stop 2020-11-20T12:16:30.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.016+0000 7f1867e17700 1 -- 172.21.15.116:0/366751379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18601502e0 msgr2=0x7f1860165630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.016+0000 7f1867e17700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18601502e0 0x7f1860165630 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f1858009210 tx=0x7f1858016000).stop 2020-11-20T12:16:30.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.016+0000 7f1867e17700 1 -- 172.21.15.116:0/366751379 shutdown_connections 2020-11-20T12:16:30.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.016+0000 7f1867e17700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 0x7f1838057c80 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.016+0000 7f1867e17700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18601502e0 0x7f1860165630 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.016+0000 7f1867e17700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600beb90 0x7f186014cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.016+0000 7f1867e17700 1 --2- 172.21.15.116:0/366751379 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18600bf020 0x7f18600a3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.016+0000 7f1867e17700 1 -- 172.21.15.116:0/366751379 >> 172.21.15.116:0/366751379 conn(0x7f1860060170 msgr2=0x7f18600c4180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:30.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.016+0000 7f1867e17700 1 -- 172.21.15.116:0/366751379 shutdown_connections 2020-11-20T12:16:30.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.016+0000 7f1867e17700 1 -- 172.21.15.116:0/366751379 wait complete. 2020-11-20T12:16:30.037 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:30.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.216+0000 7fbf743c9700 1 Processor -- start 2020-11-20T12:16:30.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.216+0000 7fbf743c9700 1 -- start start 2020-11-20T12:16:30.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.216+0000 7fbf743c9700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbf6c14f150 0x7fbf6c0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.216+0000 7fbf743c9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf6c0bd780 0x7fbf6c0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.216+0000 7fbf743c9700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6c155620 con 0x7fbf6c14f530 2020-11-20T12:16:30.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf743c9700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c1558c0 con 0x7fbf6c0bd780 2020-11-20T12:16:30.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf743c9700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c155b60 con 0x7fbf6c14f150 2020-11-20T12:16:30.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf71964700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf6c0bd780 0x7fbf6c0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf71964700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf6c0bd780 0x7fbf6c0c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43748/0 (socket says 172.21.15.116:43748) 2020-11-20T12:16:30.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf72165700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbf6c14f150 0x7fbf6c0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf71964700 1 -- 172.21.15.116:0/858623989 learned_addr learned my addr 172.21.15.116:0/858623989 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:30.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf72966700 1 --1- 172.21.15.116:0/858623989 >> v1:172.21.15.173:6789/0 conn(0x7fbf6c14f530 0x7fbf6c0a74b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49412/0 (socket says 172.21.15.116:49412) 2020-11-20T12:16:30.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf71964700 1 -- 172.21.15.116:0/858623989 >> v2:172.21.15.173:3300/0 conn(0x7fbf6c14f150 msgr2=0x7fbf6c0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:30.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf71964700 1 --2- 172.21.15.116:0/858623989 >> v2:172.21.15.173:3300/0 conn(0x7fbf6c14f150 0x7fbf6c0c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf71964700 1 -- 172.21.15.116:0/858623989 >> v1:172.21.15.173:6789/0 conn(0x7fbf6c14f530 legacy=0x7fbf6c0a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf71964700 1 -- 172.21.15.116:0/858623989 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c155e00 con 0x7fbf6c0bd780 2020-11-20T12:16:30.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf71964700 1 --2- 172.21.15.116:0/858623989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0bd780 0x7fbf6c0c8ce0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fbf6c081540 tx=0x7fbf5c006eb0).ready entity=mon.0 client_cookie=a815ac95ad971d51 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:30.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf71163700 1 -- 172.21.15.116:0/858623989 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c0098a0 con 0x7fbf6c0bd780 2020-11-20T12:16:30.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf71163700 1 -- 172.21.15.116:0/858623989 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5c009a00 con 0x7fbf6c0bd780 2020-11-20T12:16:30.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf71163700 1 -- 172.21.15.116:0/858623989 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5c009d80 con 0x7fbf6c0bd780 2020-11-20T12:16:30.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf743c9700 1 -- 172.21.15.116:0/858623989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0bd780 msgr2=0x7fbf6c0c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.220+0000 7fbf743c9700 1 --2- 172.21.15.116:0/858623989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0bd780 0x7fbf6c0c8ce0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fbf6c081540 tx=0x7fbf5c006eb0).stop 2020-11-20T12:16:30.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.224+0000 7fbf743c9700 1 -- 172.21.15.116:0/858623989 shutdown_connections 2020-11-20T12:16:30.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.224+0000 7fbf743c9700 1 --2- 172.21.15.116:0/858623989 >> v2:172.21.15.173:3300/0 conn(0x7fbf6c14f150 0x7fbf6c0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.224+0000 7fbf743c9700 1 --2- 172.21.15.116:0/858623989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0bd780 0x7fbf6c0c8ce0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.224+0000 7fbf743c9700 1 -- 172.21.15.116:0/858623989 >> 172.21.15.116:0/858623989 conn(0x7fbf6c061580 msgr2=0x7fbf6c061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:30.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.224+0000 7fbf743c9700 1 -- 172.21.15.116:0/858623989 shutdown_connections 2020-11-20T12:16:30.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.224+0000 7fbf743c9700 1 -- 172.21.15.116:0/858623989 wait complete. 2020-11-20T12:16:30.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.224+0000 7fbf743c9700 1 Processor -- start 2020-11-20T12:16:30.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.224+0000 7fbf743c9700 1 -- start start 2020-11-20T12:16:30.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf743c9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0bd780 0x7fbf6c13d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf743c9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf6c14f150 0x7fbf6c14d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf743c9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf6c14f530 0x7fbf6c164870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf743c9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0ab5c0 con 0x7fbf6c0bd780 2020-11-20T12:16:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf743c9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c1545b0 con 0x7fbf6c14f150 2020-11-20T12:16:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf743c9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c1653f0 con 0x7fbf6c14f530 2020-11-20T12:16:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf72165700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0bd780 0x7fbf6c13d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf72165700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0bd780 0x7fbf6c13d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43752/0 (socket says 172.21.15.116:43752) 2020-11-20T12:16:30.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf72165700 1 -- 172.21.15.116:0/3461001924 learned_addr learned my addr 172.21.15.116:0/3461001924 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:30.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf71964700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf6c14f150 0x7fbf6c14d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf72966700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf6c14f530 0x7fbf6c164870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf72165700 1 -- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf6c14f530 msgr2=0x7fbf6c164870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf72165700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf6c14f530 0x7fbf6c164870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf72165700 1 -- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf6c14f150 msgr2=0x7fbf6c14d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf72165700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf6c14f150 0x7fbf6c14d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf72165700 1 -- 172.21.15.116:0/3461001924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c165690 con 0x7fbf6c0bd780 2020-11-20T12:16:30.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf72165700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0bd780 0x7fbf6c13d9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fbf6800eba0 tx=0x7fbf68006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:30.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf5a7fc700 1 -- 172.21.15.116:0/3461001924 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf6800ca60 con 0x7fbf6c0bd780 2020-11-20T12:16:30.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.228+0000 7fbf5a7fc700 1 -- 172.21.15.116:0/3461001924 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf6800cbc0 con 0x7fbf6c0bd780 2020-11-20T12:16:30.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.232+0000 7fbf5a7fc700 1 -- 172.21.15.116:0/3461001924 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf680123f0 con 0x7fbf6c0bd780 2020-11-20T12:16:30.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.232+0000 7fbf743c9700 1 -- 172.21.15.116:0/3461001924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6c0ab7f0 con 0x7fbf6c0bd780 2020-11-20T12:16:30.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.232+0000 7fbf743c9700 1 -- 172.21.15.116:0/3461001924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6c14ded0 con 0x7fbf6c0bd780 2020-11-20T12:16:30.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.232+0000 7fbf743c9700 1 -- 172.21.15.116:0/3461001924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6c14e120 con 0x7fbf6c0bd780 2020-11-20T12:16:30.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.232+0000 7fbf5a7fc700 1 -- 172.21.15.116:0/3461001924 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fbf68012760 con 0x7fbf6c0bd780 2020-11-20T12:16:30.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.232+0000 7fbf5a7fc700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbf440557c0 0x7fbf44057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.236+0000 7fbf5a7fc700 1 -- 172.21.15.116:0/3461001924 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbf6806ebd0 con 0x7fbf6c0bd780 2020-11-20T12:16:30.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.236+0000 7fbf71964700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbf440557c0 0x7fbf44057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.236+0000 7fbf71964700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbf440557c0 0x7fbf44057c80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fbf6c0bdb80 tx=0x7fbf5c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:30.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.240+0000 7fbf5a7fc700 1 -- 172.21.15.116:0/3461001924 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbf680449f0 con 0x7fbf6c0bd780 2020-11-20T12:16:30.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.400+0000 7fbf743c9700 1 -- 172.21.15.116:0/3461001924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbf6c05ca30 con 0x7fbf6c0bd780 2020-11-20T12:16:30.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.400+0000 7fbf5a7fc700 1 -- 172.21.15.116:0/3461001924 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbf68048010 con 0x7fbf6c0bd780 2020-11-20T12:16:30.403 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:30.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 -- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbf440557c0 msgr2=0x7fbf44057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbf440557c0 0x7fbf44057c80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fbf6c0bdb80 tx=0x7fbf5c015000).stop 2020-11-20T12:16:30.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 -- 172.21.15.116:0/3461001924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0bd780 msgr2=0x7fbf6c13d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0bd780 0x7fbf6c13d9c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fbf6800eba0 tx=0x7fbf68006df0).stop 2020-11-20T12:16:30.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 -- 172.21.15.116:0/3461001924 shutdown_connections 2020-11-20T12:16:30.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbf440557c0 0x7fbf44057c80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf6c14f530 0x7fbf6c164870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf6c0bd780 0x7fbf6c13d9c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 --2- 172.21.15.116:0/3461001924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf6c14f150 0x7fbf6c14d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 -- 172.21.15.116:0/3461001924 >> 172.21.15.116:0/3461001924 conn(0x7fbf6c061580 msgr2=0x7fbf6c0a7810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:30.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 -- 172.21.15.116:0/3461001924 shutdown_connections 2020-11-20T12:16:30.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.404+0000 7fbf743c9700 1 -- 172.21.15.116:0/3461001924 wait complete. 2020-11-20T12:16:30.429 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:16:30.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.608+0000 7fa356fa3700 1 Processor -- start 2020-11-20T12:16:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.608+0000 7fa356fa3700 1 -- start start 2020-11-20T12:16:30.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa356fa3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa35013ad30 0x7fa3500a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa356fa3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa35013b110 0x7fa350151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa356fa3700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa350152a80 con 0x7fa350151fa0 2020-11-20T12:16:30.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa356fa3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa350152540 con 0x7fa35013ad30 2020-11-20T12:16:30.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa356fa3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3501527e0 con 0x7fa35013b110 2020-11-20T12:16:30.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa355fa1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa35013ad30 0x7fa3500a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa355fa1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa35013ad30 0x7fa3500a7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43760/0 (socket says 172.21.15.116:43760) 2020-11-20T12:16:30.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa355fa1700 1 -- 172.21.15.116:0/3951440357 learned_addr learned my addr 172.21.15.116:0/3951440357 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:30.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa3557a0700 1 --2- 172.21.15.116:0/3951440357 >> v2:172.21.15.173:3300/0 conn(0x7fa35013b110 0x7fa350151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa355fa1700 1 -- 172.21.15.116:0/3951440357 >> v2:172.21.15.173:3300/0 conn(0x7fa35013b110 msgr2=0x7fa350151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:30.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa355fa1700 1 --2- 172.21.15.116:0/3951440357 >> v2:172.21.15.173:3300/0 conn(0x7fa35013b110 0x7fa350151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa355fa1700 1 -- 172.21.15.116:0/3951440357 >> v1:172.21.15.173:6789/0 conn(0x7fa350151fa0 legacy=0x7fa350061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.612+0000 7fa355fa1700 1 -- 172.21.15.116:0/3951440357 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3500ac910 con 0x7fa35013ad30 2020-11-20T12:16:30.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa355fa1700 1 --2- 172.21.15.116:0/3951440357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa35013ad30 0x7fa3500a7fb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fa350062710 tx=0x7fa34c006eb0).ready entity=mon.0 client_cookie=e3f1c4021b336a87 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:30.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa354f9f700 1 -- 172.21.15.116:0/3951440357 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c009960 con 0x7fa35013ad30 2020-11-20T12:16:30.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa354f9f700 1 -- 172.21.15.116:0/3951440357 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa34c009ac0 con 0x7fa35013ad30 2020-11-20T12:16:30.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa354f9f700 1 -- 172.21.15.116:0/3951440357 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34c009e40 con 0x7fa35013ad30 2020-11-20T12:16:30.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa356fa3700 1 -- 172.21.15.116:0/3951440357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa35013ad30 msgr2=0x7fa3500a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa356fa3700 1 --2- 172.21.15.116:0/3951440357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa35013ad30 0x7fa3500a7fb0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fa350062710 tx=0x7fa34c006eb0).stop 2020-11-20T12:16:30.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa356fa3700 1 -- 172.21.15.116:0/3951440357 shutdown_connections 2020-11-20T12:16:30.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa356fa3700 1 --2- 172.21.15.116:0/3951440357 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa35013ad30 0x7fa3500a7fb0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa356fa3700 1 --2- 172.21.15.116:0/3951440357 >> v2:172.21.15.173:3300/0 conn(0x7fa35013b110 0x7fa350151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa356fa3700 1 -- 172.21.15.116:0/3951440357 >> 172.21.15.116:0/3951440357 conn(0x7fa3500678a0 msgr2=0x7fa350067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa356fa3700 1 -- 172.21.15.116:0/3951440357 shutdown_connections 2020-11-20T12:16:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.616+0000 7fa356fa3700 1 -- 172.21.15.116:0/3951440357 wait complete. 2020-11-20T12:16:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa356fa3700 1 Processor -- start 2020-11-20T12:16:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa356fa3700 1 -- start start 2020-11-20T12:16:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa356fa3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa35013ad30 0x7fa3500ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa356fa3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa35013b110 0x7fa350150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa356fa3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa350151fa0 0x7fa3501511e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa356fa3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3501516d0 con 0x7fa35013b110 2020-11-20T12:16:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa356fa3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3500a45d0 con 0x7fa350151fa0 2020-11-20T12:16:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa356fa3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3500a4870 con 0x7fa35013ad30 2020-11-20T12:16:30.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa3557a0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa35013b110 0x7fa350150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa3557a0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa35013b110 0x7fa350150cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43768/0 (socket says 172.21.15.116:43768) 2020-11-20T12:16:30.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa3557a0700 1 -- 172.21.15.116:0/1661687580 learned_addr learned my addr 172.21.15.116:0/1661687580 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:30.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa355fa1700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa35013ad30 0x7fa3500ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa3567a2700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa350151fa0 0x7fa3501511e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa3557a0700 1 -- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa35013ad30 msgr2=0x7fa3500ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa3557a0700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa35013ad30 0x7fa3500ad3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa3557a0700 1 -- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa350151fa0 msgr2=0x7fa3501511e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa3557a0700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa350151fa0 0x7fa3501511e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa3557a0700 1 -- 172.21.15.116:0/1661687580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3500a4b10 con 0x7fa35013b110 2020-11-20T12:16:30.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa3557a0700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa35013b110 0x7fa350150cf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fa34000db30 tx=0x7fa340008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:30.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa33dffb700 1 -- 172.21.15.116:0/1661687580 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340009730 con 0x7fa35013b110 2020-11-20T12:16:30.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa33dffb700 1 -- 172.21.15.116:0/1661687580 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa340009890 con 0x7fa35013b110 2020-11-20T12:16:30.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa33dffb700 1 -- 172.21.15.116:0/1661687580 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340009c10 con 0x7fa35013b110 2020-11-20T12:16:30.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa356fa3700 1 -- 172.21.15.116:0/1661687580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3500a4db0 con 0x7fa35013b110 2020-11-20T12:16:30.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.620+0000 7fa356fa3700 1 -- 172.21.15.116:0/1661687580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3500ad8b0 con 0x7fa35013b110 2020-11-20T12:16:30.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.624+0000 7fa356fa3700 1 -- 172.21.15.116:0/1661687580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3500adb30 con 0x7fa35013b110 2020-11-20T12:16:30.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.624+0000 7fa33dffb700 1 -- 172.21.15.116:0/1661687580 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa340013480 con 0x7fa35013b110 2020-11-20T12:16:30.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.624+0000 7fa33dffb700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa3280557c0 0x7fa328057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.624+0000 7fa33dffb700 1 -- 172.21.15.116:0/1661687580 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa34006ea00 con 0x7fa35013b110 2020-11-20T12:16:30.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.624+0000 7fa355fa1700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa3280557c0 0x7fa328057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:30.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.624+0000 7fa355fa1700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa3280557c0 0x7fa328057c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fa34c006ef0 tx=0x7fa34c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:30.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.628+0000 7fa33dffb700 1 -- 172.21.15.116:0/1661687580 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa340044840 con 0x7fa35013b110 2020-11-20T12:16:30.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.788+0000 7fa356fa3700 1 -- 172.21.15.116:0/1661687580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa350044f60 con 0x7fa35013b110 2020-11-20T12:16:30.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa33dffb700 1 -- 172.21.15.116:0/1661687580 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa340047e60 con 0x7fa35013b110 2020-11-20T12:16:30.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 -- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa3280557c0 msgr2=0x7fa328057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa3280557c0 0x7fa328057c80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7fa34c006ef0 tx=0x7fa34c015000).stop 2020-11-20T12:16:30.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 -- 172.21.15.116:0/1661687580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa35013b110 msgr2=0x7fa350150cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:30.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa35013b110 0x7fa350150cf0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fa34000db30 tx=0x7fa340008040).stop 2020-11-20T12:16:30.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 -- 172.21.15.116:0/1661687580 shutdown_connections 2020-11-20T12:16:30.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa3280557c0 0x7fa328057c80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa350151fa0 0x7fa3501511e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa35013ad30 0x7fa3500ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 --2- 172.21.15.116:0/1661687580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa35013b110 0x7fa350150cf0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:30.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 -- 172.21.15.116:0/1661687580 >> 172.21.15.116:0/1661687580 conn(0x7fa3500678a0 msgr2=0x7fa350139f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:30.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 -- 172.21.15.116:0/1661687580 shutdown_connections 2020-11-20T12:16:30.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.792+0000 7fa356fa3700 1 -- 172.21.15.116:0/1661687580 wait complete. 2020-11-20T12:16:30.804 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:30.818 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:16:30.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.992+0000 7f13156d7700 1 Processor -- start 2020-11-20T12:16:30.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.996+0000 7f13156d7700 1 -- start start 2020-11-20T12:16:30.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.996+0000 7f13156d7700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f131013ad30 0x7f13100a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.996+0000 7f13156d7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131013b110 0x7f1310151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:30.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.996+0000 7f13156d7700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1310152a80 con 0x7f1310151fa0 2020-11-20T12:16:31.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.996+0000 7f13156d7700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1310152540 con 0x7f131013ad30 2020-11-20T12:16:31.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.996+0000 7f13156d7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13101527e0 con 0x7f131013b110 2020-11-20T12:16:31.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.996+0000 7f1314ed6700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1310151fa0 0x7f1310061250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53210/0 (socket says 172.21.15.116:53210) 2020-11-20T12:16:31.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:30.996+0000 7f1314ed6700 1 -- 172.21.15.116:0/2988847608 learned_addr learned my addr 172.21.15.116:0/2988847608 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:31.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130ffff700 1 --2- 172.21.15.116:0/2988847608 >> v2:172.21.15.173:3300/0 conn(0x7f131013ad30 0x7f13100a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130f7fe700 1 --2- 172.21.15.116:0/2988847608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131013b110 0x7f1310151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130effd700 1 -- 172.21.15.116:0/2988847608 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 117460069 0 0) 0x7f1310152a80 con 0x7f1310151fa0 2020-11-20T12:16:31.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130effd700 1 -- 172.21.15.116:0/2988847608 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12fc003240 con 0x7f1310151fa0 2020-11-20T12:16:31.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130ffff700 1 -- 172.21.15.116:0/2988847608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131013b110 msgr2=0x7f1310151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:31.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130ffff700 1 --2- 172.21.15.116:0/2988847608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131013b110 0x7f1310151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130ffff700 1 -- 172.21.15.116:0/2988847608 >> v1:172.21.15.116:6789/0 conn(0x7f1310151fa0 legacy=0x7f1310061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130ffff700 1 -- 172.21.15.116:0/2988847608 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13100ac910 con 0x7f131013ad30 2020-11-20T12:16:31.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130f7fe700 1 --2- 172.21.15.116:0/2988847608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131013b110 0x7f1310151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:31.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130ffff700 1 --2- 172.21.15.116:0/2988847608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131013ad30 0x7f13100a7fb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f130000b9a0 tx=0x7f1300006df0).ready entity=mon.1 client_cookie=4a2bfc88f2f26d48 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:31.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130effd700 1 -- 172.21.15.116:0/2988847608 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300009710 con 0x7f131013ad30 2020-11-20T12:16:31.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130effd700 1 -- 172.21.15.116:0/2988847608 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1300009870 con 0x7f131013ad30 2020-11-20T12:16:31.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f130effd700 1 -- 172.21.15.116:0/2988847608 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300009bf0 con 0x7f131013ad30 2020-11-20T12:16:31.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f13156d7700 1 -- 172.21.15.116:0/2988847608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131013ad30 msgr2=0x7f13100a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f13156d7700 1 --2- 172.21.15.116:0/2988847608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131013ad30 0x7f13100a7fb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f130000b9a0 tx=0x7f1300006df0).stop 2020-11-20T12:16:31.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f13156d7700 1 -- 172.21.15.116:0/2988847608 shutdown_connections 2020-11-20T12:16:31.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f13156d7700 1 --2- 172.21.15.116:0/2988847608 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131013ad30 0x7f13100a7fb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f13156d7700 1 --2- 172.21.15.116:0/2988847608 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131013b110 0x7f1310151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f13156d7700 1 -- 172.21.15.116:0/2988847608 >> 172.21.15.116:0/2988847608 conn(0x7f13100678a0 msgr2=0x7f1310067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:31.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f13156d7700 1 -- 172.21.15.116:0/2988847608 shutdown_connections 2020-11-20T12:16:31.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.000+0000 7f13156d7700 1 -- 172.21.15.116:0/2988847608 wait complete. 2020-11-20T12:16:31.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f13156d7700 1 Processor -- start 2020-11-20T12:16:31.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f13156d7700 1 -- start start 2020-11-20T12:16:31.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f13156d7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131013ad30 0x7f13100bf9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f13156d7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131013b110 0x7f1310138760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f13156d7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151fa0 0x7f1310153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f13156d7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1310160b70 con 0x7f1310151fa0 2020-11-20T12:16:31.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f13156d7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13100bd590 con 0x7f131013b110 2020-11-20T12:16:31.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f13156d7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1310160dd0 con 0x7f131013ad30 2020-11-20T12:16:31.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f1314ed6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151fa0 0x7f1310153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f1314ed6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151fa0 0x7f1310153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43784/0 (socket says 172.21.15.116:43784) 2020-11-20T12:16:31.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f1314ed6700 1 -- 172.21.15.116:0/2697711612 learned_addr learned my addr 172.21.15.116:0/2697711612 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:31.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f130ffff700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131013ad30 0x7f13100bf9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f130f7fe700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131013b110 0x7f1310138760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f1314ed6700 1 -- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131013ad30 msgr2=0x7f13100bf9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f1314ed6700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131013ad30 0x7f13100bf9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f1314ed6700 1 -- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131013b110 msgr2=0x7f1310138760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f1314ed6700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131013b110 0x7f1310138760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.004+0000 7f1314ed6700 1 -- 172.21.15.116:0/2697711612 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1310161070 con 0x7f1310151fa0 2020-11-20T12:16:31.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.008+0000 7f1314ed6700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151fa0 0x7f1310153e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f1308001420 tx=0x7f1308007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:31.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.008+0000 7f12effff700 1 -- 172.21.15.116:0/2697711612 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130800ce10 con 0x7f1310151fa0 2020-11-20T12:16:31.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.008+0000 7f12effff700 1 -- 172.21.15.116:0/2697711612 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1308005b80 con 0x7f1310151fa0 2020-11-20T12:16:31.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.008+0000 7f12effff700 1 -- 172.21.15.116:0/2697711612 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1308007110 con 0x7f1310151fa0 2020-11-20T12:16:31.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.008+0000 7f13156d7700 1 -- 172.21.15.116:0/2697711612 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1310161310 con 0x7f1310151fa0 2020-11-20T12:16:31.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.008+0000 7f13156d7700 1 -- 172.21.15.116:0/2697711612 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13100bf4b0 con 0x7f1310151fa0 2020-11-20T12:16:31.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.008+0000 7f13156d7700 1 -- 172.21.15.116:0/2697711612 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1310044f60 con 0x7f1310151fa0 2020-11-20T12:16:31.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.008+0000 7f12effff700 1 -- 172.21.15.116:0/2697711612 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1308010030 con 0x7f1310151fa0 2020-11-20T12:16:31.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.012+0000 7f12effff700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12e80557c0 0x7f12e8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.012+0000 7f12effff700 1 -- 172.21.15.116:0/2697711612 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f130806e4c0 con 0x7f1310151fa0 2020-11-20T12:16:31.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.012+0000 7f130ffff700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12e80557c0 0x7f12e8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.012+0000 7f130ffff700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12e80557c0 0x7f12e8057c80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1300006e30 tx=0x7f1300015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:31.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.016+0000 7f12effff700 1 -- 172.21.15.116:0/2697711612 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1308044360 con 0x7f1310151fa0 2020-11-20T12:16:31.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.180+0000 7f13156d7700 1 -- 172.21.15.116:0/2697711612 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1310069320 con 0x7f1310151fa0 2020-11-20T12:16:31.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.180+0000 7f12effff700 1 -- 172.21.15.116:0/2697711612 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f1308047980 con 0x7f1310151fa0 2020-11-20T12:16:31.182 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:31.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.180+0000 7f13156d7700 1 -- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12e80557c0 msgr2=0x7f12e8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.180+0000 7f13156d7700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12e80557c0 0x7f12e8057c80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1300006e30 tx=0x7f1300015000).stop 2020-11-20T12:16:31.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.184+0000 7f13156d7700 1 -- 172.21.15.116:0/2697711612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151fa0 msgr2=0x7f1310153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.184+0000 7f13156d7700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151fa0 0x7f1310153e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f1308001420 tx=0x7f1308007e20).stop 2020-11-20T12:16:31.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.184+0000 7f13156d7700 1 -- 172.21.15.116:0/2697711612 shutdown_connections 2020-11-20T12:16:31.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.184+0000 7f13156d7700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12e80557c0 0x7f12e8057c80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.184+0000 7f13156d7700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151fa0 0x7f1310153e10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.184+0000 7f13156d7700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f131013ad30 0x7f13100bf9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.184+0000 7f13156d7700 1 --2- 172.21.15.116:0/2697711612 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f131013b110 0x7f1310138760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.184+0000 7f13156d7700 1 -- 172.21.15.116:0/2697711612 >> 172.21.15.116:0/2697711612 conn(0x7f13100678a0 msgr2=0x7f13100c6a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:31.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.184+0000 7f13156d7700 1 -- 172.21.15.116:0/2697711612 shutdown_connections 2020-11-20T12:16:31.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.184+0000 7f13156d7700 1 -- 172.21.15.116:0/2697711612 wait complete. 2020-11-20T12:16:31.207 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:16:31.207 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:16:31.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.376+0000 7f81d8ff8700 1 Processor -- start 2020-11-20T12:16:31.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.376+0000 7f81d8ff8700 1 -- start start 2020-11-20T12:16:31.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.376+0000 7f81d8ff8700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81d40beee0 0x7f81d40bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.376+0000 7f81d8ff8700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f81d4060170 0x7f81d414fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.376+0000 7f81d8ff8700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d40605e0 con 0x7f81d4152ee0 2020-11-20T12:16:31.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.376+0000 7f81d8ff8700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d4063e90 con 0x7f81d40beee0 2020-11-20T12:16:31.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.376+0000 7f81d8ff8700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d4060810 con 0x7f81d4060170 2020-11-20T12:16:31.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.376+0000 7f81d37fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81d40beee0 0x7f81d40bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.376+0000 7f81d2ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f81d4152ee0 0x7f81d40664c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53224/0 (socket says 172.21.15.116:53224) 2020-11-20T12:16:31.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.376+0000 7f81d2ffd700 1 -- 172.21.15.116:0/3956593681 learned_addr learned my addr 172.21.15.116:0/3956593681 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:31.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.376+0000 7f81d3fff700 1 --2- 172.21.15.116:0/3956593681 >> v2:172.21.15.173:3300/0 conn(0x7f81d4060170 0x7f81d414fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d37fe700 1 -- 172.21.15.116:0/3956593681 >> v2:172.21.15.173:3300/0 conn(0x7f81d4060170 msgr2=0x7f81d414fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:31.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d27fc700 1 -- 172.21.15.116:0/3956593681 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951931911 0 0) 0x7f81d40605e0 con 0x7f81d4152ee0 2020-11-20T12:16:31.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d37fe700 1 --2- 172.21.15.116:0/3956593681 >> v2:172.21.15.173:3300/0 conn(0x7f81d4060170 0x7f81d414fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d37fe700 1 -- 172.21.15.116:0/3956593681 >> v1:172.21.15.116:6789/0 conn(0x7f81d4152ee0 legacy=0x7f81d40664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d37fe700 1 -- 172.21.15.116:0/3956593681 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d4060ab0 con 0x7f81d40beee0 2020-11-20T12:16:31.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d37fe700 1 --2- 172.21.15.116:0/3956593681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81d40beee0 0x7f81d40bf2c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f81d4152e20 tx=0x7f81c4006df0).ready entity=mon.0 client_cookie=ea78f0932a148623 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:31.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d27fc700 1 -- 172.21.15.116:0/3956593681 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c40097b0 con 0x7f81d40beee0 2020-11-20T12:16:31.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d27fc700 1 -- 172.21.15.116:0/3956593681 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c4009910 con 0x7f81d40beee0 2020-11-20T12:16:31.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d27fc700 1 -- 172.21.15.116:0/3956593681 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c4009c90 con 0x7f81d40beee0 2020-11-20T12:16:31.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d8ff8700 1 -- 172.21.15.116:0/3956593681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81d40beee0 msgr2=0x7f81d40bf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d8ff8700 1 --2- 172.21.15.116:0/3956593681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81d40beee0 0x7f81d40bf2c0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f81d4152e20 tx=0x7f81c4006df0).stop 2020-11-20T12:16:31.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d8ff8700 1 -- 172.21.15.116:0/3956593681 shutdown_connections 2020-11-20T12:16:31.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d8ff8700 1 --2- 172.21.15.116:0/3956593681 >> v2:172.21.15.173:3300/0 conn(0x7f81d4060170 0x7f81d414fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d8ff8700 1 --2- 172.21.15.116:0/3956593681 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81d40beee0 0x7f81d40bf2c0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d8ff8700 1 -- 172.21.15.116:0/3956593681 >> 172.21.15.116:0/3956593681 conn(0x7f81d40c44e0 msgr2=0x7f81d40c48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:31.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d8ff8700 1 -- 172.21.15.116:0/3956593681 shutdown_connections 2020-11-20T12:16:31.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.380+0000 7f81d8ff8700 1 -- 172.21.15.116:0/3956593681 wait complete. 2020-11-20T12:16:31.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d8ff8700 1 Processor -- start 2020-11-20T12:16:31.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d8ff8700 1 -- start start 2020-11-20T12:16:31.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d8ff8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d4060170 0x7f81d40a85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d8ff8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81d4152ee0 0x7f81d4066df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d8ff8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81d40672e0 0x7f81d40a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d8ff8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d40c2650 con 0x7f81d40672e0 2020-11-20T12:16:31.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d8ff8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d40a8ae0 con 0x7f81d4060170 2020-11-20T12:16:31.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d8ff8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d40a5e30 con 0x7f81d4152ee0 2020-11-20T12:16:31.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81d40672e0 0x7f81d40a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81d40672e0 0x7f81d40a9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43798/0 (socket says 172.21.15.116:43798) 2020-11-20T12:16:31.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d3fff700 1 -- 172.21.15.116:0/224232147 learned_addr learned my addr 172.21.15.116:0/224232147 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:31.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d2ffd700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81d4152ee0 0x7f81d4066df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.384+0000 7f81d37fe700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d4060170 0x7f81d40a85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81d3fff700 1 -- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81d4152ee0 msgr2=0x7f81d4066df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81d3fff700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81d4152ee0 0x7f81d4066df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81d3fff700 1 -- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d4060170 msgr2=0x7f81d40a85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81d3fff700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d4060170 0x7f81d40a85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81d3fff700 1 -- 172.21.15.116:0/224232147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d40a60d0 con 0x7f81d40672e0 2020-11-20T12:16:31.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81d3fff700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81d40672e0 0x7f81d40a9d60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f81cc00ef50 tx=0x7f81cc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:31.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81b37fe700 1 -- 172.21.15.116:0/224232147 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00cb40 con 0x7f81d40672e0 2020-11-20T12:16:31.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81b37fe700 1 -- 172.21.15.116:0/224232147 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc00cca0 con 0x7f81d40672e0 2020-11-20T12:16:31.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81d8ff8700 1 -- 172.21.15.116:0/224232147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d40a6370 con 0x7f81d40672e0 2020-11-20T12:16:31.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81b37fe700 1 -- 172.21.15.116:0/224232147 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0123f0 con 0x7f81d40672e0 2020-11-20T12:16:31.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81d8ff8700 1 -- 172.21.15.116:0/224232147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d40a5af0 con 0x7f81d40672e0 2020-11-20T12:16:31.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.388+0000 7f81d8ff8700 1 -- 172.21.15.116:0/224232147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d4061310 con 0x7f81d40672e0 2020-11-20T12:16:31.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.392+0000 7f81b37fe700 1 -- 172.21.15.116:0/224232147 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f81cc022070 con 0x7f81d40672e0 2020-11-20T12:16:31.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.392+0000 7f81b37fe700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81ac0557c0 0x7f81ac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:31.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.392+0000 7f81b37fe700 1 -- 172.21.15.116:0/224232147 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f81cc06eff0 con 0x7f81d40672e0 2020-11-20T12:16:31.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.392+0000 7f81d37fe700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81ac0557c0 0x7f81ac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:31.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.392+0000 7f81d37fe700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81ac0557c0 0x7f81ac057c80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f81d40651c0 tx=0x7f81c4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:31.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.396+0000 7f81b37fe700 1 -- 172.21.15.116:0/224232147 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f81d4061310 con 0x7f81d40672e0 2020-11-20T12:16:31.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.604+0000 7f81d8ff8700 1 -- 172.21.15.116:0/224232147 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81d40a8350 con 0x7f81d40672e0 2020-11-20T12:16:31.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81b37fe700 1 -- 172.21.15.116:0/224232147 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2846 (secure 0 0 0) 0x7f81d40a8350 con 0x7f81d40672e0 2020-11-20T12:16:31.608 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:31.608 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/2968741298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":2968741298},{"type":"v1","addr":"172.21.15.116:6835","nonce":2968741298}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1214743116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1214743116},{"type":"v1","addr":"172.21.15.173:6836","nonce":1214743116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1424291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1424291148},{"type":"v1","addr":"172.21.15.173:6837","nonce":1424291148}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:00.619934+0000","modified":"2020-11-20T12:16:02.636079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.116:6834/3819756176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3819756176},{"type":"v1","addr":"172.21.15.116:6834","nonce":3819756176}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:16:31.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 -- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81ac0557c0 msgr2=0x7f81ac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81ac0557c0 0x7f81ac057c80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f81d40651c0 tx=0x7f81c4015000).stop 2020-11-20T12:16:31.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 -- 172.21.15.116:0/224232147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81d40672e0 msgr2=0x7f81d40a9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:31.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81d40672e0 0x7f81d40a9d60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f81cc00ef50 tx=0x7f81cc0091d0).stop 2020-11-20T12:16:31.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 -- 172.21.15.116:0/224232147 shutdown_connections 2020-11-20T12:16:31.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81ac0557c0 0x7f81ac057c80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81d40672e0 0x7f81d40a9d60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81d4060170 0x7f81d40a85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 --2- 172.21.15.116:0/224232147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81d4152ee0 0x7f81d4066df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:31.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 -- 172.21.15.116:0/224232147 >> 172.21.15.116:0/224232147 conn(0x7f81d40c44e0 msgr2=0x7f81d41461c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:31.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 -- 172.21.15.116:0/224232147 shutdown_connections 2020-11-20T12:16:31.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:31.608+0000 7f81d8ff8700 1 -- 172.21.15.116:0/224232147 wait complete. 2020-11-20T12:16:31.613 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 6 2020-11-20T12:16:32.037 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:00.619934+0000', 'modified': '2020-11-20T12:16:02.636079+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4245}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4245': {'gid': 4245, 'name': 'a', 'rank': 0, 'incarnation': 3, 'state': 'up:active', 'state_seq': 2, 'addr': '172.21.15.116:6834/3819756176', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 3819756176}, {'type': 'v1', 'addr': '172.21.15.116:6834', 'nonce': 3819756176}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:16:32.038 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:16:32.038 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:16:32.229 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4245,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":31.588385078999998,"mdsmap_epoch":6,"osdmap_epoch":23,"osdmap_epoch_barrier":23,"uptime":35.076223194000001} 2020-11-20T12:16:32.229 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4245, "mdsmap_epoch": 6, "osdmap_epoch": 23, "osdmap_epoch_barrier": 23, "rank_uptime": 31.588385079, "state": "up:active", "uptime": 35.076223194, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:16:32.230 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:16:32.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:32.231 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.3 2020-11-20T12:16:32.249 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:16:32.250 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:32.322 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:32.323 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:16:32.348 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:16:32.348 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:16:32.355 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:16:32.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:32.356 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2020-11-20T12:16:32.383 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2020-11-20T12:16:32.415 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:32.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.596+0000 7f64b4713700 1 Processor -- start 2020-11-20T12:16:32.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.600+0000 7f64b4713700 1 -- start start 2020-11-20T12:16:32.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.600+0000 7f64b4713700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f64ac0aa570 0x7f64ac0aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.600+0000 7f64b4713700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64ac151c30 0x7f64ac0ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.600+0000 7f64b4713700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64ac1523b0 con 0x7f64ac0a81a0 2020-11-20T12:16:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.600+0000 7f64b4713700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64ac0ab930 con 0x7f64ac0aa570 2020-11-20T12:16:32.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.600+0000 7f64b4713700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac152650 con 0x7f64ac151c30 2020-11-20T12:16:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.600+0000 7f64b1cae700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f64ac0a81a0 0x7f64ac0ad270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53240/0 (socket says 172.21.15.116:53240) 2020-11-20T12:16:32.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.600+0000 7f64b1cae700 1 -- 172.21.15.116:0/2252131507 learned_addr learned my addr 172.21.15.116:0/2252131507 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.600+0000 7f64b2cb0700 1 --2- 172.21.15.116:0/2252131507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64ac151c30 0x7f64ac0ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.600+0000 7f64b24af700 1 --2- 172.21.15.116:0/2252131507 >> v2:172.21.15.173:3300/0 conn(0x7f64ac0aa570 0x7f64ac0aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b14ad700 1 -- 172.21.15.116:0/2252131507 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169248444 0 0) 0x7f64ac1523b0 con 0x7f64ac0a81a0 2020-11-20T12:16:32.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b14ad700 1 -- 172.21.15.116:0/2252131507 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6494003240 con 0x7f64ac0a81a0 2020-11-20T12:16:32.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b2cb0700 1 -- 172.21.15.116:0/2252131507 >> v2:172.21.15.173:3300/0 conn(0x7f64ac0aa570 msgr2=0x7f64ac0aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:32.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b2cb0700 1 --2- 172.21.15.116:0/2252131507 >> v2:172.21.15.173:3300/0 conn(0x7f64ac0aa570 0x7f64ac0aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b2cb0700 1 -- 172.21.15.116:0/2252131507 >> v1:172.21.15.116:6789/0 conn(0x7f64ac0a81a0 legacy=0x7f64ac0ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b2cb0700 1 -- 172.21.15.116:0/2252131507 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac1528f0 con 0x7f64ac151c30 2020-11-20T12:16:32.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b2cb0700 1 --2- 172.21.15.116:0/2252131507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64ac151c30 0x7f64ac0ad980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f64ac0aba70 tx=0x7f64a40091d0).ready entity=mon.2 client_cookie=f1df1a6c1506b174 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b14ad700 1 -- 172.21.15.116:0/2252131507 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a400ba20 con 0x7f64ac151c30 2020-11-20T12:16:32.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b14ad700 1 -- 172.21.15.116:0/2252131507 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a400bb80 con 0x7f64ac151c30 2020-11-20T12:16:32.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b14ad700 1 -- 172.21.15.116:0/2252131507 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a4011400 con 0x7f64ac151c30 2020-11-20T12:16:32.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b4713700 1 -- 172.21.15.116:0/2252131507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64ac151c30 msgr2=0x7f64ac0ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b4713700 1 --2- 172.21.15.116:0/2252131507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64ac151c30 0x7f64ac0ad980 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f64ac0aba70 tx=0x7f64a40091d0).stop 2020-11-20T12:16:32.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b4713700 1 -- 172.21.15.116:0/2252131507 shutdown_connections 2020-11-20T12:16:32.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b4713700 1 --2- 172.21.15.116:0/2252131507 >> v2:172.21.15.173:3300/0 conn(0x7f64ac0aa570 0x7f64ac0aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b4713700 1 --2- 172.21.15.116:0/2252131507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64ac151c30 0x7f64ac0ad980 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b4713700 1 -- 172.21.15.116:0/2252131507 >> 172.21.15.116:0/2252131507 conn(0x7f64ac0678a0 msgr2=0x7f64ac067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:32.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b4713700 1 -- 172.21.15.116:0/2252131507 shutdown_connections 2020-11-20T12:16:32.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.604+0000 7f64b4713700 1 -- 172.21.15.116:0/2252131507 wait complete. 2020-11-20T12:16:32.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b4713700 1 Processor -- start 2020-11-20T12:16:32.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b4713700 1 -- start start 2020-11-20T12:16:32.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b4713700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64ac0a81a0 0x7f64ac164350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b4713700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64ac0aa570 0x7f64ac157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b4713700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64ac151c30 0x7f64ac163050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b4713700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac162130 con 0x7f64ac151c30 2020-11-20T12:16:32.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b4713700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac1623d0 con 0x7f64ac0aa570 2020-11-20T12:16:32.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b4713700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac162670 con 0x7f64ac0a81a0 2020-11-20T12:16:32.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b2cb0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64ac151c30 0x7f64ac163050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b2cb0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64ac151c30 0x7f64ac163050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43812/0 (socket says 172.21.15.116:43812) 2020-11-20T12:16:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b2cb0700 1 -- 172.21.15.116:0/2993261977 learned_addr learned my addr 172.21.15.116:0/2993261977 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b24af700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64ac0a81a0 0x7f64ac164350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.608+0000 7f64b1cae700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64ac0aa570 0x7f64ac157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f64b2cb0700 1 -- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64ac0a81a0 msgr2=0x7f64ac164350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f64b2cb0700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64ac0a81a0 0x7f64ac164350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f64b2cb0700 1 -- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64ac0aa570 msgr2=0x7f64ac157650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f64b2cb0700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64ac0aa570 0x7f64ac157650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f64b2cb0700 1 -- 172.21.15.116:0/2993261977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64ac162910 con 0x7f64ac151c30 2020-11-20T12:16:32.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f64b2cb0700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64ac151c30 0x7f64ac163050 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f64ac0624f0 tx=0x7f64a4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f649a7fc700 1 -- 172.21.15.116:0/2993261977 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a401c6c0 con 0x7f64ac151c30 2020-11-20T12:16:32.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f649a7fc700 1 -- 172.21.15.116:0/2993261977 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64a401c820 con 0x7f64ac151c30 2020-11-20T12:16:32.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f649a7fc700 1 -- 172.21.15.116:0/2993261977 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64a401cbc0 con 0x7f64ac151c30 2020-11-20T12:16:32.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f64b4713700 1 -- 172.21.15.116:0/2993261977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac162bb0 con 0x7f64ac151c30 2020-11-20T12:16:32.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f64b4713700 1 -- 172.21.15.116:0/2993261977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac15e850 con 0x7f64ac151c30 2020-11-20T12:16:32.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.612+0000 7f64b4713700 1 -- 172.21.15.116:0/2993261977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac162d40 con 0x7f64ac151c30 2020-11-20T12:16:32.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.616+0000 7f649a7fc700 1 -- 172.21.15.116:0/2993261977 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f64a4017070 con 0x7f64ac151c30 2020-11-20T12:16:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.616+0000 7f649a7fc700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64840557c0 0x7f6484057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.616+0000 7f649a7fc700 1 -- 172.21.15.116:0/2993261977 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f64a4070c00 con 0x7f64ac151c30 2020-11-20T12:16:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.616+0000 7f64b24af700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64840557c0 0x7f6484057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.616+0000 7f64b24af700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64840557c0 0x7f6484057c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f64a8006fd0 tx=0x7f64a8006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.620+0000 7f649a7fc700 1 -- 172.21.15.116:0/2993261977 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f64a4046aa0 con 0x7f64ac151c30 2020-11-20T12:16:32.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.780+0000 7f64b4713700 1 -- 172.21.15.116:0/2993261977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64ac05ca30 con 0x7f64ac151c30 2020-11-20T12:16:32.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.784+0000 7f649a7fc700 1 -- 172.21.15.116:0/2993261977 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f64a4015030 con 0x7f64ac151c30 2020-11-20T12:16:32.785 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:32.786 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:32.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.784+0000 7f64b4713700 1 -- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64840557c0 msgr2=0x7f6484057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.784+0000 7f64b4713700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64840557c0 0x7f6484057c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f64a8006fd0 tx=0x7f64a8006df0).stop 2020-11-20T12:16:32.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.784+0000 7f64b4713700 1 -- 172.21.15.116:0/2993261977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64ac151c30 msgr2=0x7f64ac163050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.784+0000 7f64b4713700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64ac151c30 0x7f64ac163050 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f64ac0624f0 tx=0x7f64a4016000).stop 2020-11-20T12:16:32.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.788+0000 7f64b4713700 1 -- 172.21.15.116:0/2993261977 shutdown_connections 2020-11-20T12:16:32.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.788+0000 7f64b4713700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64840557c0 0x7f6484057c80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.788+0000 7f64b4713700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f64ac151c30 0x7f64ac163050 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.788+0000 7f64b4713700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64ac0a81a0 0x7f64ac164350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.788+0000 7f64b4713700 1 --2- 172.21.15.116:0/2993261977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f64ac0aa570 0x7f64ac157650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.788+0000 7f64b4713700 1 -- 172.21.15.116:0/2993261977 >> 172.21.15.116:0/2993261977 conn(0x7f64ac0678a0 msgr2=0x7f64ac154230 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:32.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.788+0000 7f64b4713700 1 -- 172.21.15.116:0/2993261977 shutdown_connections 2020-11-20T12:16:32.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.788+0000 7f64b4713700 1 -- 172.21.15.116:0/2993261977 wait complete. 2020-11-20T12:16:32.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:15:59.065285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:15:59.602311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:16:32.812 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:32.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.980+0000 7f980cd5a700 1 Processor -- start 2020-11-20T12:16:32.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.980+0000 7f980cd5a700 1 -- start start 2020-11-20T12:16:32.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980cd5a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f98080c0c00 0x7f98080a1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980cd5a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98080c0fe0 0x7f98080611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980cd5a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9808060280 con 0x7f98080a2270 2020-11-20T12:16:32.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980cd5a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9808060520 con 0x7f98080c0c00 2020-11-20T12:16:32.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980cd5a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98080607c0 con 0x7f98080c0fe0 2020-11-20T12:16:32.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98080c0fe0 0x7f98080611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980659c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f98080c0c00 0x7f98080a1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980659c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f98080c0c00 0x7f98080a1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:42868/0 (socket says 172.21.15.116:42868) 2020-11-20T12:16:32.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98080c0fe0 0x7f98080611a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:52770/0 (socket says 172.21.15.116:52770) 2020-11-20T12:16:32.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980659c700 1 -- 172.21.15.116:0/48719626 learned_addr learned my addr 172.21.15.116:0/48719626 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:32.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f9805d9b700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f98080a2270 0x7f98080c2600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49482/0 (socket says 172.21.15.116:49482) 2020-11-20T12:16:32.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980659c700 1 -- 172.21.15.116:0/48719626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98080c0fe0 msgr2=0x7f98080611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:32.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980659c700 1 --2- 172.21.15.116:0/48719626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98080c0fe0 0x7f98080611a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980659c700 1 -- 172.21.15.116:0/48719626 >> v1:172.21.15.173:6789/0 conn(0x7f98080a2270 legacy=0x7f98080c2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.984+0000 7f980659c700 1 -- 172.21.15.116:0/48719626 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9808060a60 con 0x7f98080c0c00 2020-11-20T12:16:32.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980659c700 1 --2- 172.21.15.116:0/48719626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98080c0c00 0x7f98080a1d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f97f8006fd0 tx=0x7f97f8006eb0).ready entity=mon.1 client_cookie=347009d67d6c5497 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:32.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980559a700 1 -- 172.21.15.116:0/48719626 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f80098a0 con 0x7f98080c0c00 2020-11-20T12:16:32.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980559a700 1 -- 172.21.15.116:0/48719626 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f8009a00 con 0x7f98080c0c00 2020-11-20T12:16:32.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980559a700 1 -- 172.21.15.116:0/48719626 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f8009d80 con 0x7f98080c0c00 2020-11-20T12:16:32.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980cd5a700 1 -- 172.21.15.116:0/48719626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98080c0c00 msgr2=0x7f98080a1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980cd5a700 1 --2- 172.21.15.116:0/48719626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98080c0c00 0x7f98080a1d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f97f8006fd0 tx=0x7f97f8006eb0).stop 2020-11-20T12:16:32.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980cd5a700 1 -- 172.21.15.116:0/48719626 shutdown_connections 2020-11-20T12:16:32.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980cd5a700 1 --2- 172.21.15.116:0/48719626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98080c0fe0 0x7f98080611a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980cd5a700 1 --2- 172.21.15.116:0/48719626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98080c0c00 0x7f98080a1d80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980cd5a700 1 -- 172.21.15.116:0/48719626 >> 172.21.15.116:0/48719626 conn(0x7f98080c14a0 msgr2=0x7f98080c18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980cd5a700 1 -- 172.21.15.116:0/48719626 shutdown_connections 2020-11-20T12:16:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.988+0000 7f980cd5a700 1 -- 172.21.15.116:0/48719626 wait complete. 2020-11-20T12:16:32.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f980cd5a700 1 Processor -- start 2020-11-20T12:16:32.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f980cd5a700 1 -- start start 2020-11-20T12:16:32.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f980cd5a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98080c0c00 0x7f980814c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f980cd5a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98080c0fe0 0x7f980815f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f980cd5a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98080a8900 0x7f9808168f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:32.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f980cd5a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98080a92b0 con 0x7f98080a8900 2020-11-20T12:16:32.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f980cd5a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980815fe80 con 0x7f98080c0fe0 2020-11-20T12:16:32.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f980cd5a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980814eca0 con 0x7f98080c0c00 2020-11-20T12:16:32.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98080a8900 0x7f9808168f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f9806d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98080a8900 0x7f9808168f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43824/0 (socket says 172.21.15.116:43824) 2020-11-20T12:16:32.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f9806d9d700 1 -- 172.21.15.116:0/359316086 learned_addr learned my addr 172.21.15.116:0/359316086 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:32.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f9805d9b700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98080c0fe0 0x7f980815f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f980659c700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98080c0c00 0x7f980814c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:32.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f9806d9d700 1 -- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98080c0c00 msgr2=0x7f980814c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f9806d9d700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98080c0c00 0x7f980814c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f9806d9d700 1 -- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98080c0fe0 msgr2=0x7f980815f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:32.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f9806d9d700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98080c0fe0 0x7f980815f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:32.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f9806d9d700 1 -- 172.21.15.116:0/359316086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98080a94e0 con 0x7f98080a8900 2020-11-20T12:16:32.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f9805d9b700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98080c0fe0 0x7f980815f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:32.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.992+0000 7f9806d9d700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98080a8900 0x7f9808168f90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f980000ec30 tx=0x7f98000091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:33.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.996+0000 7f97f67fc700 1 -- 172.21.15.116:0/359316086 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980000cac0 con 0x7f98080a8900 2020-11-20T12:16:33.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.996+0000 7f97f67fc700 1 -- 172.21.15.116:0/359316086 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980000cc20 con 0x7f98080a8900 2020-11-20T12:16:33.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.996+0000 7f97f67fc700 1 -- 172.21.15.116:0/359316086 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98000123f0 con 0x7f98080a8900 2020-11-20T12:16:33.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.996+0000 7f980cd5a700 1 -- 172.21.15.116:0/359316086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980814ef40 con 0x7f98080a8900 2020-11-20T12:16:33.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.996+0000 7f980cd5a700 1 -- 172.21.15.116:0/359316086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980814ea10 con 0x7f98080a8900 2020-11-20T12:16:33.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.996+0000 7f980cd5a700 1 -- 172.21.15.116:0/359316086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980805ca30 con 0x7f98080a8900 2020-11-20T12:16:33.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.996+0000 7f97f67fc700 1 -- 172.21.15.116:0/359316086 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9800022070 con 0x7f98080a8900 2020-11-20T12:16:33.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:32.996+0000 7f97f67fc700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97dc0557c0 0x7f97dc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:33.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.000+0000 7f97f67fc700 1 -- 172.21.15.116:0/359316086 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f980006ee20 con 0x7f98080a8900 2020-11-20T12:16:33.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.000+0000 7f980659c700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97dc0557c0 0x7f97dc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:33.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.000+0000 7f980659c700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97dc0557c0 0x7f97dc057c80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f9808138580 tx=0x7f97f8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:33.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.004+0000 7f97f67fc700 1 -- 172.21.15.116:0/359316086 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9800044cc0 con 0x7f98080a8900 2020-11-20T12:16:33.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.168+0000 7f980cd5a700 1 -- 172.21.15.116:0/359316086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f980813b040 con 0x7f98080a8900 2020-11-20T12:16:33.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f97f67fc700 1 -- 172.21.15.116:0/359316086 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9800016070 con 0x7f98080a8900 2020-11-20T12:16:33.171 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:33.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 -- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97dc0557c0 msgr2=0x7f97dc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97dc0557c0 0x7f97dc057c80 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f9808138580 tx=0x7f97f8015000).stop 2020-11-20T12:16:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 -- 172.21.15.116:0/359316086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98080a8900 msgr2=0x7f9808168f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98080a8900 0x7f9808168f90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f980000ec30 tx=0x7f98000091d0).stop 2020-11-20T12:16:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 -- 172.21.15.116:0/359316086 shutdown_connections 2020-11-20T12:16:33.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97dc0557c0 0x7f97dc057c80 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98080a8900 0x7f9808168f90 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98080c0c00 0x7f980814c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 --2- 172.21.15.116:0/359316086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98080c0fe0 0x7f980815f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 -- 172.21.15.116:0/359316086 >> 172.21.15.116:0/359316086 conn(0x7f98080c14a0 msgr2=0x7f980813a410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 -- 172.21.15.116:0/359316086 shutdown_connections 2020-11-20T12:16:33.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.172+0000 7f980cd5a700 1 -- 172.21.15.116:0/359316086 wait complete. 2020-11-20T12:16:33.195 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:33.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.372+0000 7fb9a8936700 1 Processor -- start 2020-11-20T12:16:33.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.372+0000 7fb9a8936700 1 -- start start 2020-11-20T12:16:33.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a8936700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb9a40c5340 0x7fb9a4062a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:33.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a8936700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb9a4062f30 0x7fb9a4152de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:33.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a8936700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9a4061b20 con 0x7fb9a40c4f60 2020-11-20T12:16:33.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a8936700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a4061dc0 con 0x7fb9a40c5340 2020-11-20T12:16:33.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a8936700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9a4062060 con 0x7fb9a4062f30 2020-11-20T12:16:33.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a1d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb9a40c5340 0x7fb9a4062a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:33.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a259c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb9a40c4f60 0x7fb9a4061280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53264/0 (socket says 172.21.15.116:53264) 2020-11-20T12:16:33.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a259c700 1 -- 172.21.15.116:0/1386819682 learned_addr learned my addr 172.21.15.116:0/1386819682 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:33.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a2d9d700 1 --2- 172.21.15.116:0/1386819682 >> v2:172.21.15.173:3300/0 conn(0x7fb9a4062f30 0x7fb9a4152de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:33.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a1d9b700 1 -- 172.21.15.116:0/1386819682 >> v2:172.21.15.173:3300/0 conn(0x7fb9a4062f30 msgr2=0x7fb9a4152de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:33.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a1d9b700 1 --2- 172.21.15.116:0/1386819682 >> v2:172.21.15.173:3300/0 conn(0x7fb9a4062f30 0x7fb9a4152de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a1d9b700 1 -- 172.21.15.116:0/1386819682 >> v1:172.21.15.116:6789/0 conn(0x7fb9a40c4f60 legacy=0x7fb9a4061280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:33.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.376+0000 7fb9a1d9b700 1 -- 172.21.15.116:0/1386819682 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a4062300 con 0x7fb9a40c5340 2020-11-20T12:16:33.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a1d9b700 1 --2- 172.21.15.116:0/1386819682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a40c5340 0x7fb9a4062a40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fb9a40ac5e0 tx=0x7fb998006df0).ready entity=mon.0 client_cookie=a32a89beeecb011f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:33.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a159a700 1 -- 172.21.15.116:0/1386819682 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998009840 con 0x7fb9a40c5340 2020-11-20T12:16:33.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a159a700 1 -- 172.21.15.116:0/1386819682 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9980099a0 con 0x7fb9a40c5340 2020-11-20T12:16:33.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a159a700 1 -- 172.21.15.116:0/1386819682 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb998009d20 con 0x7fb9a40c5340 2020-11-20T12:16:33.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a8936700 1 -- 172.21.15.116:0/1386819682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a40c5340 msgr2=0x7fb9a4062a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:33.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a8936700 1 --2- 172.21.15.116:0/1386819682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a40c5340 0x7fb9a4062a40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fb9a40ac5e0 tx=0x7fb998006df0).stop 2020-11-20T12:16:33.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a8936700 1 -- 172.21.15.116:0/1386819682 shutdown_connections 2020-11-20T12:16:33.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a8936700 1 --2- 172.21.15.116:0/1386819682 >> v2:172.21.15.173:3300/0 conn(0x7fb9a4062f30 0x7fb9a4152de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a8936700 1 --2- 172.21.15.116:0/1386819682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a40c5340 0x7fb9a4062a40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a8936700 1 -- 172.21.15.116:0/1386819682 >> 172.21.15.116:0/1386819682 conn(0x7fb9a408f950 msgr2=0x7fb9a4060950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:33.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a8936700 1 -- 172.21.15.116:0/1386819682 shutdown_connections 2020-11-20T12:16:33.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.380+0000 7fb9a8936700 1 -- 172.21.15.116:0/1386819682 wait complete. 2020-11-20T12:16:33.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a8936700 1 Processor -- start 2020-11-20T12:16:33.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a8936700 1 -- start start 2020-11-20T12:16:33.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a8936700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9a4062f30 0x7fb9a4154b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:33.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a8936700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a40c4f60 0x7fb9a40a8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:33.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a8936700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a40c5340 0x7fb9a40a8600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:33.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a8936700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a4155080 con 0x7fb9a40c4f60 2020-11-20T12:16:33.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a8936700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a414b1e0 con 0x7fb9a40c5340 2020-11-20T12:16:33.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a8936700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9a4165940 con 0x7fb9a4062f30 2020-11-20T12:16:33.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a1d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a40c4f60 0x7fb9a40a8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:33.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a1d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a40c4f60 0x7fb9a40a8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43838/0 (socket says 172.21.15.116:43838) 2020-11-20T12:16:33.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a1d9b700 1 -- 172.21.15.116:0/1763415517 learned_addr learned my addr 172.21.15.116:0/1763415517 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:33.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a2d9d700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a40c5340 0x7fb9a40a8600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:33.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a259c700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9a4062f30 0x7fb9a4154b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:33.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a1d9b700 1 -- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9a4062f30 msgr2=0x7fb9a4154b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:33.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a1d9b700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9a4062f30 0x7fb9a4154b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a1d9b700 1 -- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a40c5340 msgr2=0x7fb9a40a8600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.384+0000 7fb9a1d9b700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a40c5340 0x7fb9a40a8600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.388+0000 7fb9a1d9b700 1 -- 172.21.15.116:0/1763415517 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9a4169c60 con 0x7fb9a40c4f60 2020-11-20T12:16:33.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.388+0000 7fb9a259c700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9a4062f30 0x7fb9a4154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:33.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.388+0000 7fb9a1d9b700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a40c4f60 0x7fb9a40a8110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fb99800c010 tx=0x7fb998015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:33.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.388+0000 7fb9927fc700 1 -- 172.21.15.116:0/1763415517 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99801e460 con 0x7fb9a40c4f60 2020-11-20T12:16:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.388+0000 7fb9927fc700 1 -- 172.21.15.116:0/1763415517 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb99801e5c0 con 0x7fb9a40c4f60 2020-11-20T12:16:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.388+0000 7fb9927fc700 1 -- 172.21.15.116:0/1763415517 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb99801e960 con 0x7fb9a40c4f60 2020-11-20T12:16:33.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.388+0000 7fb9a8936700 1 -- 172.21.15.116:0/1763415517 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9a4165ba0 con 0x7fb9a40c4f60 2020-11-20T12:16:33.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.388+0000 7fb9a8936700 1 -- 172.21.15.116:0/1763415517 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9a41660e0 con 0x7fb9a40c4f60 2020-11-20T12:16:33.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.388+0000 7fb9a8936700 1 -- 172.21.15.116:0/1763415517 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9a405ca30 con 0x7fb9a40c4f60 2020-11-20T12:16:33.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.388+0000 7fb9927fc700 1 -- 172.21.15.116:0/1763415517 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb998025030 con 0x7fb9a40c4f60 2020-11-20T12:16:33.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.392+0000 7fb9927fc700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9780557c0 0x7fb978057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:33.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.392+0000 7fb9927fc700 1 -- 172.21.15.116:0/1763415517 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb99806fe70 con 0x7fb9a40c4f60 2020-11-20T12:16:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.392+0000 7fb9a259c700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9780557c0 0x7fb978057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.392+0000 7fb9a259c700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9780557c0 0x7fb978057c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb994000bc0 tx=0x7fb994009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:33.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.396+0000 7fb9927fc700 1 -- 172.21.15.116:0/1763415517 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb998045d60 con 0x7fb9a40c4f60 2020-11-20T12:16:33.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.556+0000 7fb9a8936700 1 -- 172.21.15.116:0/1763415517 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb9a40662d0 con 0x7fb9a40c4f60 2020-11-20T12:16:33.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.556+0000 7fb9927fc700 1 -- 172.21.15.116:0/1763415517 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb998023070 con 0x7fb9a40c4f60 2020-11-20T12:16:33.558 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:33.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 -- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9780557c0 msgr2=0x7fb978057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:33.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9780557c0 0x7fb978057c80 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb994000bc0 tx=0x7fb994009040).stop 2020-11-20T12:16:33.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 -- 172.21.15.116:0/1763415517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a40c4f60 msgr2=0x7fb9a40a8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:33.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a40c4f60 0x7fb9a40a8110 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fb99800c010 tx=0x7fb998015000).stop 2020-11-20T12:16:33.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 -- 172.21.15.116:0/1763415517 shutdown_connections 2020-11-20T12:16:33.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9780557c0 0x7fb978057c80 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9a40c5340 0x7fb9a40a8600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9a4062f30 0x7fb9a4154b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 --2- 172.21.15.116:0/1763415517 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9a40c4f60 0x7fb9a40a8110 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:33.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 -- 172.21.15.116:0/1763415517 >> 172.21.15.116:0/1763415517 conn(0x7fb9a408f950 msgr2=0x7fb9a414b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:33.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 -- 172.21.15.116:0/1763415517 shutdown_connections 2020-11-20T12:16:33.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.560+0000 7fb9a8936700 1 -- 172.21.15.116:0/1763415517 wait complete. 2020-11-20T12:16:33.581 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:16:33.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.756+0000 7ff5234a8700 1 Processor -- start 2020-11-20T12:16:34.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.760+0000 7ff5234a8700 1 -- start start 2020-11-20T12:16:34.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.760+0000 7ff5234a8700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff51c14f150 0x7ff51c0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff5234a8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c14f530 0x7ff51c0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff5234a8700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff51c155620 con 0x7ff51c0bd780 2020-11-20T12:16:34.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff5234a8700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff51c1558c0 con 0x7ff51c14f150 2020-11-20T12:16:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff5234a8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c155b60 con 0x7ff51c14f530 2020-11-20T12:16:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff521244700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff51c14f150 0x7ff51c0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff521a45700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c14f530 0x7ff51c0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff521244700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff51c14f150 0x7ff51c0c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:42896/0 (socket says 172.21.15.116:42896) 2020-11-20T12:16:34.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff520a43700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7ff51c0bd780 0x7ff51c0a74b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49510/0 (socket says 172.21.15.116:49510) 2020-11-20T12:16:34.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff521244700 1 -- 172.21.15.116:0/1722741127 learned_addr learned my addr 172.21.15.116:0/1722741127 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:34.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff521244700 1 -- 172.21.15.116:0/1722741127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c14f530 msgr2=0x7ff51c0c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:34.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff521244700 1 --2- 172.21.15.116:0/1722741127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c14f530 0x7ff51c0c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff521244700 1 -- 172.21.15.116:0/1722741127 >> v1:172.21.15.173:6789/0 conn(0x7ff51c0bd780 legacy=0x7ff51c0a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff521244700 1 -- 172.21.15.116:0/1722741127 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c155e00 con 0x7ff51c14f150 2020-11-20T12:16:34.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff521244700 1 --2- 172.21.15.116:0/1722741127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c14f150 0x7ff51c0c87f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7ff518006fd0 tx=0x7ff518006eb0).ready entity=mon.1 client_cookie=6dbae0bad9065fdb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:34.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff50bfff700 1 -- 172.21.15.116:0/1722741127 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5180098a0 con 0x7ff51c14f150 2020-11-20T12:16:34.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff50bfff700 1 -- 172.21.15.116:0/1722741127 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff518009a00 con 0x7ff51c14f150 2020-11-20T12:16:34.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.764+0000 7ff50bfff700 1 -- 172.21.15.116:0/1722741127 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff518009d80 con 0x7ff51c14f150 2020-11-20T12:16:34.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.768+0000 7ff5234a8700 1 -- 172.21.15.116:0/1722741127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c14f150 msgr2=0x7ff51c0c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.768+0000 7ff5234a8700 1 --2- 172.21.15.116:0/1722741127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c14f150 0x7ff51c0c87f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7ff518006fd0 tx=0x7ff518006eb0).stop 2020-11-20T12:16:34.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.768+0000 7ff5234a8700 1 -- 172.21.15.116:0/1722741127 shutdown_connections 2020-11-20T12:16:34.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.768+0000 7ff5234a8700 1 --2- 172.21.15.116:0/1722741127 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c14f530 0x7ff51c0c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.768+0000 7ff5234a8700 1 --2- 172.21.15.116:0/1722741127 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c14f150 0x7ff51c0c87f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.768+0000 7ff5234a8700 1 -- 172.21.15.116:0/1722741127 >> 172.21.15.116:0/1722741127 conn(0x7ff51c061580 msgr2=0x7ff51c061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:34.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.768+0000 7ff5234a8700 1 -- 172.21.15.116:0/1722741127 shutdown_connections 2020-11-20T12:16:34.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.768+0000 7ff5234a8700 1 -- 172.21.15.116:0/1722741127 wait complete. 2020-11-20T12:16:34.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.768+0000 7ff5234a8700 1 Processor -- start 2020-11-20T12:16:34.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.768+0000 7ff5234a8700 1 -- start start 2020-11-20T12:16:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff5234a8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c0bd780 0x7ff51c152d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff5234a8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c14f150 0x7ff51c1490f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff5234a8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff51c14f530 0x7ff51c154490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff5234a8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c165b40 con 0x7ff51c14f530 2020-11-20T12:16:34.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff5234a8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c152080 con 0x7ff51c0bd780 2020-11-20T12:16:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff5234a8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff51c152320 con 0x7ff51c14f150 2020-11-20T12:16:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff521a45700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff51c14f530 0x7ff51c154490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff521a45700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff51c14f530 0x7ff51c154490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43854/0 (socket says 172.21.15.116:43854) 2020-11-20T12:16:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff521a45700 1 -- 172.21.15.116:0/125957852 learned_addr learned my addr 172.21.15.116:0/125957852 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:34.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff521244700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c0bd780 0x7ff51c152d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff520a43700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c14f150 0x7ff51c1490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff521a45700 1 -- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c14f150 msgr2=0x7ff51c1490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff521a45700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c14f150 0x7ff51c1490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff521a45700 1 -- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c0bd780 msgr2=0x7ff51c152d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff521a45700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c0bd780 0x7ff51c152d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff521a45700 1 -- 172.21.15.116:0/125957852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff51c1525c0 con 0x7ff51c14f530 2020-11-20T12:16:34.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff520a43700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c14f150 0x7ff51c1490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:34.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff521a45700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff51c14f530 0x7ff51c154490 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7ff51400ef50 tx=0x7ff5140091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:34.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff5097fa700 1 -- 172.21.15.116:0/125957852 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51400cc80 con 0x7ff51c14f530 2020-11-20T12:16:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff5097fa700 1 -- 172.21.15.116:0/125957852 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51400cde0 con 0x7ff51c14f530 2020-11-20T12:16:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff5234a8700 1 -- 172.21.15.116:0/125957852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff51c152860 con 0x7ff51c14f530 2020-11-20T12:16:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff5097fa700 1 -- 172.21.15.116:0/125957852 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff514012620 con 0x7ff51c14f530 2020-11-20T12:16:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.772+0000 7ff5234a8700 1 -- 172.21.15.116:0/125957852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff51c154980 con 0x7ff51c14f530 2020-11-20T12:16:34.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.776+0000 7ff5234a8700 1 -- 172.21.15.116:0/125957852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff51c1529f0 con 0x7ff51c14f530 2020-11-20T12:16:34.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.776+0000 7ff5097fa700 1 -- 172.21.15.116:0/125957852 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff514022070 con 0x7ff51c14f530 2020-11-20T12:16:34.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.776+0000 7ff5097fa700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4f40557c0 0x7ff4f4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.776+0000 7ff5097fa700 1 -- 172.21.15.116:0/125957852 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff51406efb0 con 0x7ff51c14f530 2020-11-20T12:16:34.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.776+0000 7ff521244700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4f40557c0 0x7ff4f4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.780+0000 7ff521244700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4f40557c0 0x7ff4f4057c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7ff518009fd0 tx=0x7ff518015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:34.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.784+0000 7ff5097fa700 1 -- 172.21.15.116:0/125957852 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff514044e50 con 0x7ff51c14f530 2020-11-20T12:16:34.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.944+0000 7ff5234a8700 1 -- 172.21.15.116:0/125957852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff51c05ca30 con 0x7ff51c14f530 2020-11-20T12:16:34.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.944+0000 7ff5097fa700 1 -- 172.21.15.116:0/125957852 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff514016020 con 0x7ff51c14f530 2020-11-20T12:16:34.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 -- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4f40557c0 msgr2=0x7ff4f4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4f40557c0 0x7ff4f4057c80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7ff518009fd0 tx=0x7ff518015000).stop 2020-11-20T12:16:34.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 -- 172.21.15.116:0/125957852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff51c14f530 msgr2=0x7ff51c154490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff51c14f530 0x7ff51c154490 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7ff51400ef50 tx=0x7ff5140091d0).stop 2020-11-20T12:16:34.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 -- 172.21.15.116:0/125957852 shutdown_connections 2020-11-20T12:16:34.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4f40557c0 0x7ff4f4057c80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff51c14f530 0x7ff51c154490 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff51c0bd780 0x7ff51c152d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 --2- 172.21.15.116:0/125957852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff51c14f150 0x7ff51c1490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 -- 172.21.15.116:0/125957852 >> 172.21.15.116:0/125957852 conn(0x7ff51c061580 msgr2=0x7ff51c14bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:34.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 -- 172.21.15.116:0/125957852 shutdown_connections 2020-11-20T12:16:34.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:33.948+0000 7ff5234a8700 1 -- 172.21.15.116:0/125957852 wait complete. 2020-11-20T12:16:34.212 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:34.212 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:16:34.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.400+0000 7ff03ec79700 1 Processor -- start 2020-11-20T12:16:34.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.400+0000 7ff03ec79700 1 -- start start 2020-11-20T12:16:34.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03ec79700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff03813ad30 0x7ff0380a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03ec79700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff03813b110 0x7ff038151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03ec79700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff038152a80 con 0x7ff038151fa0 2020-11-20T12:16:34.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03ec79700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff038152540 con 0x7ff03813b110 2020-11-20T12:16:34.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03ec79700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0381527e0 con 0x7ff03813ad30 2020-11-20T12:16:34.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03d476700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff03813b110 0x7ff038151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03dc77700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff03813ad30 0x7ff0380a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03e478700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7ff038151fa0 0x7ff038061250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49526/0 (socket says 172.21.15.116:49526) 2020-11-20T12:16:34.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03e478700 1 -- 172.21.15.116:0/3694581171 learned_addr learned my addr 172.21.15.116:0/3694581171 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:34.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03d476700 1 -- 172.21.15.116:0/3694581171 >> v2:172.21.15.173:3300/0 conn(0x7ff03813ad30 msgr2=0x7ff0380a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:34.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03d476700 1 --2- 172.21.15.116:0/3694581171 >> v2:172.21.15.173:3300/0 conn(0x7ff03813ad30 0x7ff0380a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03d476700 1 -- 172.21.15.116:0/3694581171 >> v1:172.21.15.173:6789/0 conn(0x7ff038151fa0 legacy=0x7ff038061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03d476700 1 -- 172.21.15.116:0/3694581171 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0380ac910 con 0x7ff03813b110 2020-11-20T12:16:34.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03cc75700 1 -- 172.21.15.116:0/3694581171 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2139373410 0 0) 0x7ff038152a80 con 0x7ff038151fa0 2020-11-20T12:16:34.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.404+0000 7ff03d476700 1 --2- 172.21.15.116:0/3694581171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff03813b110 0x7ff038151ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7ff038081330 tx=0x7ff028006df0).ready entity=mon.0 client_cookie=7d4629129d4e613d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:34.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03cc75700 1 -- 172.21.15.116:0/3694581171 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028009840 con 0x7ff03813b110 2020-11-20T12:16:34.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03cc75700 1 -- 172.21.15.116:0/3694581171 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0280099a0 con 0x7ff03813b110 2020-11-20T12:16:34.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03cc75700 1 -- 172.21.15.116:0/3694581171 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff028009d20 con 0x7ff03813b110 2020-11-20T12:16:34.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03ec79700 1 -- 172.21.15.116:0/3694581171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff03813b110 msgr2=0x7ff038151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03ec79700 1 --2- 172.21.15.116:0/3694581171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff03813b110 0x7ff038151ab0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7ff038081330 tx=0x7ff028006df0).stop 2020-11-20T12:16:34.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03ec79700 1 -- 172.21.15.116:0/3694581171 shutdown_connections 2020-11-20T12:16:34.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03ec79700 1 --2- 172.21.15.116:0/3694581171 >> v2:172.21.15.173:3300/0 conn(0x7ff03813ad30 0x7ff0380a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03ec79700 1 --2- 172.21.15.116:0/3694581171 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff03813b110 0x7ff038151ab0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03ec79700 1 -- 172.21.15.116:0/3694581171 >> 172.21.15.116:0/3694581171 conn(0x7ff0380678a0 msgr2=0x7ff038067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:34.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03ec79700 1 -- 172.21.15.116:0/3694581171 shutdown_connections 2020-11-20T12:16:34.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03ec79700 1 -- 172.21.15.116:0/3694581171 wait complete. 2020-11-20T12:16:34.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03ec79700 1 Processor -- start 2020-11-20T12:16:34.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.408+0000 7ff03ec79700 1 -- start start 2020-11-20T12:16:34.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03ec79700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03813ad30 0x7ff038160e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03ec79700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff038151fa0 0x7ff038164350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03ec79700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff038164840 0x7ff0380ad110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03ec79700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff038165210 con 0x7ff038164840 2020-11-20T12:16:34.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03ec79700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff038164cd0 con 0x7ff03813ad30 2020-11-20T12:16:34.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03ec79700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff038164f70 con 0x7ff038151fa0 2020-11-20T12:16:34.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03e478700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff038164840 0x7ff0380ad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03e478700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff038164840 0x7ff0380ad110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43868/0 (socket says 172.21.15.116:43868) 2020-11-20T12:16:34.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03e478700 1 -- 172.21.15.116:0/3817151346 learned_addr learned my addr 172.21.15.116:0/3817151346 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:34.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03d476700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff038151fa0 0x7ff038164350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03dc77700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03813ad30 0x7ff038160e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03e478700 1 -- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff038151fa0 msgr2=0x7ff038164350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03e478700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff038151fa0 0x7ff038164350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03e478700 1 -- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03813ad30 msgr2=0x7ff038160e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03e478700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03813ad30 0x7ff038160e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03e478700 1 -- 172.21.15.116:0/3817151346 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0380c73b0 con 0x7ff038164840 2020-11-20T12:16:34.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03e478700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff038164840 0x7ff0380ad110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7ff03000cae0 tx=0x7ff030001320).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:34.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff025ffb700 1 -- 172.21.15.116:0/3817151346 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030005b80 con 0x7ff038164840 2020-11-20T12:16:34.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff025ffb700 1 -- 172.21.15.116:0/3817151346 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff030005ce0 con 0x7ff038164840 2020-11-20T12:16:34.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03ec79700 1 -- 172.21.15.116:0/3817151346 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0380c7650 con 0x7ff038164840 2020-11-20T12:16:34.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff025ffb700 1 -- 172.21.15.116:0/3817151346 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff030007360 con 0x7ff038164840 2020-11-20T12:16:34.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.412+0000 7ff03ec79700 1 -- 172.21.15.116:0/3817151346 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0380ad600 con 0x7ff038164840 2020-11-20T12:16:34.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.416+0000 7ff03ec79700 1 -- 172.21.15.116:0/3817151346 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0380ad8b0 con 0x7ff038164840 2020-11-20T12:16:34.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.416+0000 7ff025ffb700 1 -- 172.21.15.116:0/3817151346 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff030010030 con 0x7ff038164840 2020-11-20T12:16:34.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.416+0000 7ff025ffb700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 0x7ff010057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.416+0000 7ff025ffb700 1 -- 172.21.15.116:0/3817151346 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff03006e3a0 con 0x7ff038164840 2020-11-20T12:16:34.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.416+0000 7ff03dc77700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 0x7ff010057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.416+0000 7ff03dc77700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 0x7ff010057c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7ff034006fd0 tx=0x7ff034006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:34.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.420+0000 7ff025ffb700 1 -- 172.21.15.116:0/3817151346 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff03006bfd0 con 0x7ff038164840 2020-11-20T12:16:34.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.584+0000 7ff03ec79700 1 -- 172.21.15.116:0/3817151346 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0380c77e0 con 0x7ff038164840 2020-11-20T12:16:34.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.584+0000 7ff025ffb700 1 -- 172.21.15.116:0/3817151346 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff030044240 con 0x7ff038164840 2020-11-20T12:16:34.584 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:34.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.584+0000 7ff03ec79700 1 -- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 msgr2=0x7ff010057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.584+0000 7ff03ec79700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 0x7ff010057c80 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7ff034006fd0 tx=0x7ff034006df0).stop 2020-11-20T12:16:34.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.584+0000 7ff03ec79700 1 -- 172.21.15.116:0/3817151346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff038164840 msgr2=0x7ff0380ad110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.584+0000 7ff03ec79700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff038164840 0x7ff0380ad110 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7ff03000cae0 tx=0x7ff030001320).stop 2020-11-20T12:16:34.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.588+0000 7ff03ec79700 1 -- 172.21.15.116:0/3817151346 shutdown_connections 2020-11-20T12:16:34.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.588+0000 7ff03ec79700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 0x7ff010057c80 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.588+0000 7ff03ec79700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff038164840 0x7ff0380ad110 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.588+0000 7ff03ec79700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03813ad30 0x7ff038160e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.588+0000 7ff03ec79700 1 --2- 172.21.15.116:0/3817151346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff038151fa0 0x7ff038164350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.588+0000 7ff03ec79700 1 -- 172.21.15.116:0/3817151346 >> 172.21.15.116:0/3817151346 conn(0x7ff0380678a0 msgr2=0x7ff038067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:34.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.588+0000 7ff03ec79700 1 -- 172.21.15.116:0/3817151346 shutdown_connections 2020-11-20T12:16:34.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.588+0000 7ff03ec79700 1 -- 172.21.15.116:0/3817151346 wait complete. 2020-11-20T12:16:34.611 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:34.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.784+0000 7f8c6f692700 1 Processor -- start 2020-11-20T12:16:34.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.784+0000 7f8c6f692700 1 -- start start 2020-11-20T12:16:34.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6f692700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8c680aa570 0x7f8c680aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6f692700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8c680a81a0 0x7f8c68151c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6f692700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c68152120 con 0x7f8c680ad270 2020-11-20T12:16:34.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6f692700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c680ab930 con 0x7f8c680a81a0 2020-11-20T12:16:34.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6f692700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c681523b0 con 0x7f8c680aa570 2020-11-20T12:16:34.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6cc2d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8c680a81a0 0x7f8c68151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6cc2d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8c680a81a0 0x7f8c68151c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43874/0 (socket says 172.21.15.116:43874) 2020-11-20T12:16:34.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6cc2d700 1 -- 172.21.15.116:0/943490707 learned_addr learned my addr 172.21.15.116:0/943490707 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:34.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6d42e700 1 --2- 172.21.15.116:0/943490707 >> v2:172.21.15.173:3300/0 conn(0x7f8c680aa570 0x7f8c680aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6cc2d700 1 -- 172.21.15.116:0/943490707 >> v2:172.21.15.173:3300/0 conn(0x7f8c680aa570 msgr2=0x7f8c680aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:34.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6cc2d700 1 --2- 172.21.15.116:0/943490707 >> v2:172.21.15.173:3300/0 conn(0x7f8c680aa570 0x7f8c680aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6cc2d700 1 -- 172.21.15.116:0/943490707 >> v1:172.21.15.173:6789/0 conn(0x7f8c680ad270 legacy=0x7f8c680ad720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:34.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6cc2d700 1 -- 172.21.15.116:0/943490707 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c68152650 con 0x7f8c680a81a0 2020-11-20T12:16:34.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6cc2d700 1 --2- 172.21.15.116:0/943490707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c680a81a0 0x7f8c68151c30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f8c68081740 tx=0x7f8c58006eb0).ready entity=mon.0 client_cookie=b2e3f964de878ae2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:34.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c57fff700 1 -- 172.21.15.116:0/943490707 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58009920 con 0x7f8c680a81a0 2020-11-20T12:16:34.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c57fff700 1 -- 172.21.15.116:0/943490707 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58009a80 con 0x7f8c680a81a0 2020-11-20T12:16:34.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c57fff700 1 -- 172.21.15.116:0/943490707 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58009e00 con 0x7f8c680a81a0 2020-11-20T12:16:34.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6f692700 1 -- 172.21.15.116:0/943490707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c680a81a0 msgr2=0x7f8c68151c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.788+0000 7f8c6f692700 1 --2- 172.21.15.116:0/943490707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c680a81a0 0x7f8c68151c30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f8c68081740 tx=0x7f8c58006eb0).stop 2020-11-20T12:16:34.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.792+0000 7f8c6f692700 1 -- 172.21.15.116:0/943490707 shutdown_connections 2020-11-20T12:16:34.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.792+0000 7f8c6f692700 1 --2- 172.21.15.116:0/943490707 >> v2:172.21.15.173:3300/0 conn(0x7f8c680aa570 0x7f8c680aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.792+0000 7f8c6f692700 1 --2- 172.21.15.116:0/943490707 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c680a81a0 0x7f8c68151c30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.792+0000 7f8c6f692700 1 -- 172.21.15.116:0/943490707 >> 172.21.15.116:0/943490707 conn(0x7f8c680678a0 msgr2=0x7f8c68067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:34.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.792+0000 7f8c6f692700 1 -- 172.21.15.116:0/943490707 shutdown_connections 2020-11-20T12:16:34.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.792+0000 7f8c6f692700 1 -- 172.21.15.116:0/943490707 wait complete. 2020-11-20T12:16:34.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.792+0000 7f8c6f692700 1 Processor -- start 2020-11-20T12:16:34.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.792+0000 7f8c6f692700 1 -- start start 2020-11-20T12:16:34.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6f692700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c680a81a0 0x7f8c68155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6f692700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c680aa570 0x7f8c68164f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6f692700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c680ad270 0x7f8c6814dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6f692700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c68165430 con 0x7f8c680ad270 2020-11-20T12:16:34.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6f692700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c68155c80 con 0x7f8c680aa570 2020-11-20T12:16:34.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6f692700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c68148930 con 0x7f8c680a81a0 2020-11-20T12:16:34.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6dc2f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c680ad270 0x7f8c6814dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6cc2d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c680aa570 0x7f8c68164f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6d42e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c680a81a0 0x7f8c68155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6cc2d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c680aa570 0x7f8c68164f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:42932/0 (socket says 172.21.15.116:42932) 2020-11-20T12:16:34.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6cc2d700 1 -- 172.21.15.116:0/776352795 learned_addr learned my addr 172.21.15.116:0/776352795 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:34.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6dc2f700 1 -- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c680a81a0 msgr2=0x7f8c68155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6dc2f700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c680a81a0 0x7f8c68155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6dc2f700 1 -- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c680aa570 msgr2=0x7f8c68164f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6dc2f700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c680aa570 0x7f8c68164f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6dc2f700 1 -- 172.21.15.116:0/776352795 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c68148bd0 con 0x7f8c680ad270 2020-11-20T12:16:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6dc2f700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c680ad270 0x7f8c6814dc00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f8c6000b880 tx=0x7f8c60009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:34.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c557fa700 1 -- 172.21.15.116:0/776352795 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000d5c0 con 0x7f8c680ad270 2020-11-20T12:16:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c557fa700 1 -- 172.21.15.116:0/776352795 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6000d720 con 0x7f8c680ad270 2020-11-20T12:16:34.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c557fa700 1 -- 172.21.15.116:0/776352795 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6000dae0 con 0x7f8c680ad270 2020-11-20T12:16:34.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.796+0000 7f8c6f692700 1 -- 172.21.15.116:0/776352795 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c68148e70 con 0x7f8c680ad270 2020-11-20T12:16:34.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.800+0000 7f8c6f692700 1 -- 172.21.15.116:0/776352795 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6814e170 con 0x7f8c680ad270 2020-11-20T12:16:34.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.800+0000 7f8c6f692700 1 -- 172.21.15.116:0/776352795 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6805ca30 con 0x7f8c680ad270 2020-11-20T12:16:34.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.800+0000 7f8c557fa700 1 -- 172.21.15.116:0/776352795 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8c60018030 con 0x7f8c680ad270 2020-11-20T12:16:34.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.800+0000 7f8c557fa700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c400557c0 0x7f8c40057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:34.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.804+0000 7f8c557fa700 1 -- 172.21.15.116:0/776352795 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8c6006df30 con 0x7f8c680ad270 2020-11-20T12:16:34.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.804+0000 7f8c6d42e700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c400557c0 0x7f8c40057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:34.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.804+0000 7f8c6d42e700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c400557c0 0x7f8c40057c80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f8c6400ba00 tx=0x7f8c64008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:34.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.808+0000 7f8c557fa700 1 -- 172.21.15.116:0/776352795 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8c60043e80 con 0x7f8c680ad270 2020-11-20T12:16:34.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.972+0000 7f8c6f692700 1 -- 172.21.15.116:0/776352795 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8c680605e0 con 0x7f8c680ad270 2020-11-20T12:16:34.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.972+0000 7f8c557fa700 1 -- 172.21.15.116:0/776352795 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f8c600474a0 con 0x7f8c680ad270 2020-11-20T12:16:34.974 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:34.974 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:34.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.972+0000 7f8c6f692700 1 -- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c400557c0 msgr2=0x7f8c40057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.972+0000 7f8c6f692700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c400557c0 0x7f8c40057c80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f8c6400ba00 tx=0x7f8c64008040).stop 2020-11-20T12:16:34.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.972+0000 7f8c6f692700 1 -- 172.21.15.116:0/776352795 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c680ad270 msgr2=0x7f8c6814dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:34.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.972+0000 7f8c6f692700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c680ad270 0x7f8c6814dc00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f8c6000b880 tx=0x7f8c60009060).stop 2020-11-20T12:16:34.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.976+0000 7f8c6f692700 1 -- 172.21.15.116:0/776352795 shutdown_connections 2020-11-20T12:16:34.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.976+0000 7f8c6f692700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c400557c0 0x7f8c40057c80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.976+0000 7f8c6f692700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8c680ad270 0x7f8c6814dc00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.976+0000 7f8c6f692700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8c680a81a0 0x7f8c68155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.976+0000 7f8c6f692700 1 --2- 172.21.15.116:0/776352795 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8c680aa570 0x7f8c68164f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:34.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.976+0000 7f8c6f692700 1 -- 172.21.15.116:0/776352795 >> 172.21.15.116:0/776352795 conn(0x7f8c680678a0 msgr2=0x7f8c680a91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:34.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.976+0000 7f8c6f692700 1 -- 172.21.15.116:0/776352795 shutdown_connections 2020-11-20T12:16:34.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:34.976+0000 7f8c6f692700 1 -- 172.21.15.116:0/776352795 wait complete. 2020-11-20T12:16:34.999 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:15:59.065285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:15:59.602311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:16:35.000 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:35.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.172+0000 7f9d983e6700 1 Processor -- start 2020-11-20T12:16:35.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.176+0000 7f9d983e6700 1 -- start start 2020-11-20T12:16:35.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.176+0000 7f9d983e6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9d90066b70 0x7f9d900a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.176+0000 7f9d983e6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d9014fc40 con 0x7f9d9014f860 2020-11-20T12:16:35.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d983e6700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d900c0590 con 0x7f9d90066ff0 2020-11-20T12:16:35.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d983e6700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d900c0830 con 0x7f9d90066b70 2020-11-20T12:16:35.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d96182700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9d9014f860 0x7f9d900c14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53320/0 (socket says 172.21.15.116:53320) 2020-11-20T12:16:35.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d96182700 1 -- 172.21.15.116:0/4230964397 learned_addr learned my addr 172.21.15.116:0/4230964397 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:35.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d95981700 1 --2- 172.21.15.116:0/4230964397 >> v2:172.21.15.173:3300/0 conn(0x7f9d90066b70 0x7f9d900a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d95981700 1 -- 172.21.15.116:0/4230964397 >> v1:172.21.15.173:6789/0 conn(0x7f9d90066ff0 legacy=0x7f9d900b9150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d95981700 1 -- 172.21.15.116:0/4230964397 >> v1:172.21.15.116:6789/0 conn(0x7f9d9014f860 legacy=0x7f9d900c14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d95981700 1 -- 172.21.15.116:0/4230964397 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900c0ad0 con 0x7f9d90066b70 2020-11-20T12:16:35.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d95981700 1 --2- 172.21.15.116:0/4230964397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d90066b70 0x7f9d900a5fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f9d8000b9a0 tx=0x7f9d80006df0).ready entity=mon.1 client_cookie=a4d779f4e9b41473 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:35.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d95180700 1 -- 172.21.15.116:0/4230964397 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d800097b0 con 0x7f9d90066b70 2020-11-20T12:16:35.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d95180700 1 -- 172.21.15.116:0/4230964397 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d80009910 con 0x7f9d90066b70 2020-11-20T12:16:35.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d95180700 1 -- 172.21.15.116:0/4230964397 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80009c90 con 0x7f9d90066b70 2020-11-20T12:16:35.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d983e6700 1 -- 172.21.15.116:0/4230964397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d90066b70 msgr2=0x7f9d900a5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.180+0000 7f9d983e6700 1 --2- 172.21.15.116:0/4230964397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d90066b70 0x7f9d900a5fb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f9d8000b9a0 tx=0x7f9d80006df0).stop 2020-11-20T12:16:35.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.184+0000 7f9d983e6700 1 -- 172.21.15.116:0/4230964397 shutdown_connections 2020-11-20T12:16:35.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.184+0000 7f9d983e6700 1 --2- 172.21.15.116:0/4230964397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d90066b70 0x7f9d900a5fb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.184+0000 7f9d983e6700 1 -- 172.21.15.116:0/4230964397 >> 172.21.15.116:0/4230964397 conn(0x7f9d90060170 msgr2=0x7f9d90060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:35.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.184+0000 7f9d983e6700 1 -- 172.21.15.116:0/4230964397 shutdown_connections 2020-11-20T12:16:35.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.184+0000 7f9d983e6700 1 -- 172.21.15.116:0/4230964397 wait complete. 2020-11-20T12:16:35.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.184+0000 7f9d983e6700 1 Processor -- start 2020-11-20T12:16:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.184+0000 7f9d983e6700 1 -- start start 2020-11-20T12:16:35.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d983e6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d90066b70 0x7f9d90153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d983e6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90066ff0 0x7f9d9013fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d983e6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d9014f860 0x7f9d900a7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d983e6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d900a71a0 con 0x7f9d90066ff0 2020-11-20T12:16:35.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d983e6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90153900 con 0x7f9d90066b70 2020-11-20T12:16:35.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d983e6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90153ba0 con 0x7f9d9014f860 2020-11-20T12:16:35.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d95981700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90066ff0 0x7f9d9013fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d95981700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90066ff0 0x7f9d9013fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43894/0 (socket says 172.21.15.116:43894) 2020-11-20T12:16:35.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d95981700 1 -- 172.21.15.116:0/184342682 learned_addr learned my addr 172.21.15.116:0/184342682 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:35.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d96182700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d90066b70 0x7f9d90153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d96983700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d9014f860 0x7f9d900a7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d95981700 1 -- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d9014f860 msgr2=0x7f9d900a7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d95981700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d9014f860 0x7f9d900a7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d95981700 1 -- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d90066b70 msgr2=0x7f9d90153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d95981700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d90066b70 0x7f9d90153110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d95981700 1 -- 172.21.15.116:0/184342682 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d900a73d0 con 0x7f9d90066ff0 2020-11-20T12:16:35.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d96182700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d90066b70 0x7f9d90153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:35.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d95981700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90066ff0 0x7f9d9013fbb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f9d80006b80 tx=0x7f9d80015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:35.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d7e7fc700 1 -- 172.21.15.116:0/184342682 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80011e70 con 0x7f9d90066ff0 2020-11-20T12:16:35.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d7e7fc700 1 -- 172.21.15.116:0/184342682 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d8001d5d0 con 0x7f9d90066ff0 2020-11-20T12:16:35.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d983e6700 1 -- 172.21.15.116:0/184342682 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d900a7670 con 0x7f9d90066ff0 2020-11-20T12:16:35.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d7e7fc700 1 -- 172.21.15.116:0/184342682 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d8001d970 con 0x7f9d90066ff0 2020-11-20T12:16:35.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.188+0000 7f9d983e6700 1 -- 172.21.15.116:0/184342682 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d901400a0 con 0x7f9d90066ff0 2020-11-20T12:16:35.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.192+0000 7f9d983e6700 1 -- 172.21.15.116:0/184342682 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d9005ca30 con 0x7f9d90066ff0 2020-11-20T12:16:35.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.192+0000 7f9d7e7fc700 1 -- 172.21.15.116:0/184342682 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9d80025030 con 0x7f9d90066ff0 2020-11-20T12:16:35.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.192+0000 7f9d7e7fc700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d680557c0 0x7f9d68057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.192+0000 7f9d7e7fc700 1 -- 172.21.15.116:0/184342682 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9d8006ff90 con 0x7f9d90066ff0 2020-11-20T12:16:35.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.196+0000 7f9d96182700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d680557c0 0x7f9d68057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.196+0000 7f9d96182700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d680557c0 0x7f9d68057c80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f9d8c0014b0 tx=0x7f9d8c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:35.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.200+0000 7f9d7e7fc700 1 -- 172.21.15.116:0/184342682 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9d80045ee0 con 0x7f9d90066ff0 2020-11-20T12:16:35.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 -- 172.21.15.116:0/184342682 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9d900a7800 con 0x7f9d90066ff0 2020-11-20T12:16:35.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d7e7fc700 1 -- 172.21.15.116:0/184342682 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9d80023030 con 0x7f9d90066ff0 2020-11-20T12:16:35.367 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:35.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 -- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d680557c0 msgr2=0x7f9d68057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d680557c0 0x7f9d68057c80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f9d8c0014b0 tx=0x7f9d8c009040).stop 2020-11-20T12:16:35.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 -- 172.21.15.116:0/184342682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90066ff0 msgr2=0x7f9d9013fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90066ff0 0x7f9d9013fbb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f9d80006b80 tx=0x7f9d80015000).stop 2020-11-20T12:16:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 -- 172.21.15.116:0/184342682 shutdown_connections 2020-11-20T12:16:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d680557c0 0x7f9d68057c80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d9014f860 0x7f9d900a7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d90066b70 0x7f9d90153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 --2- 172.21.15.116:0/184342682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d90066ff0 0x7f9d9013fbb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 -- 172.21.15.116:0/184342682 >> 172.21.15.116:0/184342682 conn(0x7f9d90060170 msgr2=0x7f9d900c2860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.364+0000 7f9d983e6700 1 -- 172.21.15.116:0/184342682 shutdown_connections 2020-11-20T12:16:35.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.368+0000 7f9d983e6700 1 -- 172.21.15.116:0/184342682 wait complete. 2020-11-20T12:16:35.390 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:35.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.568+0000 7fdf17349700 1 Processor -- start 2020-11-20T12:16:35.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.568+0000 7fdf17349700 1 -- start start 2020-11-20T12:16:35.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.572+0000 7fdf17349700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1013b110 0x7fdf10151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.572+0000 7fdf17349700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdf10151fa0 0x7fdf10061250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.572+0000 7fdf17349700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf10152a80 con 0x7fdf1013ad30 2020-11-20T12:16:35.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.572+0000 7fdf17349700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf10152540 con 0x7fdf10151fa0 2020-11-20T12:16:35.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.572+0000 7fdf17349700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf101527e0 con 0x7fdf1013b110 2020-11-20T12:16:35.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.572+0000 7fdf16b48700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdf10151fa0 0x7fdf10061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.572+0000 7fdf16347700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdf1013ad30 0x7fdf100a7fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53334/0 (socket says 172.21.15.116:53334) 2020-11-20T12:16:35.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.572+0000 7fdf16347700 1 -- 172.21.15.116:0/688696029 learned_addr learned my addr 172.21.15.116:0/688696029 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf16b48700 1 -- 172.21.15.116:0/688696029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1013b110 msgr2=0x7fdf10151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf15b46700 1 --2- 172.21.15.116:0/688696029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1013b110 0x7fdf10151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf16b48700 1 --2- 172.21.15.116:0/688696029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1013b110 0x7fdf10151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf16b48700 1 -- 172.21.15.116:0/688696029 >> v1:172.21.15.116:6789/0 conn(0x7fdf1013ad30 legacy=0x7fdf100a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf16b48700 1 -- 172.21.15.116:0/688696029 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf100ac910 con 0x7fdf10151fa0 2020-11-20T12:16:35.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf15b46700 1 --2- 172.21.15.116:0/688696029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1013b110 0x7fdf10151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:35.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf16b48700 1 --2- 172.21.15.116:0/688696029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf10151fa0 0x7fdf10061250 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fdf08009c90 tx=0x7fdf080091d0).ready entity=mon.0 client_cookie=1a9fc4baf3daec04 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:35.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf15345700 1 -- 172.21.15.116:0/688696029 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0800bc50 con 0x7fdf10151fa0 2020-11-20T12:16:35.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf15345700 1 -- 172.21.15.116:0/688696029 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf0800bdb0 con 0x7fdf10151fa0 2020-11-20T12:16:35.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf15345700 1 -- 172.21.15.116:0/688696029 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf08011610 con 0x7fdf10151fa0 2020-11-20T12:16:35.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf17349700 1 -- 172.21.15.116:0/688696029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf10151fa0 msgr2=0x7fdf10061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf17349700 1 --2- 172.21.15.116:0/688696029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf10151fa0 0x7fdf10061250 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7fdf08009c90 tx=0x7fdf080091d0).stop 2020-11-20T12:16:35.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf17349700 1 -- 172.21.15.116:0/688696029 shutdown_connections 2020-11-20T12:16:35.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf17349700 1 --2- 172.21.15.116:0/688696029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf10151fa0 0x7fdf10061250 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf17349700 1 --2- 172.21.15.116:0/688696029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1013b110 0x7fdf10151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf17349700 1 -- 172.21.15.116:0/688696029 >> 172.21.15.116:0/688696029 conn(0x7fdf100678a0 msgr2=0x7fdf10067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:35.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf17349700 1 -- 172.21.15.116:0/688696029 shutdown_connections 2020-11-20T12:16:35.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.576+0000 7fdf17349700 1 -- 172.21.15.116:0/688696029 wait complete. 2020-11-20T12:16:35.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf17349700 1 Processor -- start 2020-11-20T12:16:35.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf17349700 1 -- start start 2020-11-20T12:16:35.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf17349700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1013ad30 0x7fdf101389c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf17349700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf1013b110 0x7fdf100a9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf17349700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf10151fa0 0x7fdf100a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf17349700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf10138eb0 con 0x7fdf1013b110 2020-11-20T12:16:35.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf17349700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf10164eb0 con 0x7fdf10151fa0 2020-11-20T12:16:35.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf17349700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf10160e00 con 0x7fdf1013ad30 2020-11-20T12:16:35.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf15b46700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf1013b110 0x7fdf100a9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf15b46700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf1013b110 0x7fdf100a9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43910/0 (socket says 172.21.15.116:43910) 2020-11-20T12:16:35.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf15b46700 1 -- 172.21.15.116:0/137687545 learned_addr learned my addr 172.21.15.116:0/137687545 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:35.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf16347700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1013ad30 0x7fdf101389c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf16b48700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf10151fa0 0x7fdf100a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf15b46700 1 -- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1013ad30 msgr2=0x7fdf101389c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf15b46700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1013ad30 0x7fdf101389c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf15b46700 1 -- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf10151fa0 msgr2=0x7fdf100a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf15b46700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf10151fa0 0x7fdf100a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf15b46700 1 -- 172.21.15.116:0/137687545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf101610a0 con 0x7fdf1013b110 2020-11-20T12:16:35.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf15b46700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf1013b110 0x7fdf100a9600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fdf0000cb50 tx=0x7fdf00008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:35.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdefe7fc700 1 -- 172.21.15.116:0/137687545 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf000077a0 con 0x7fdf1013b110 2020-11-20T12:16:35.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdefe7fc700 1 -- 172.21.15.116:0/137687545 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf00007900 con 0x7fdf1013b110 2020-11-20T12:16:35.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.580+0000 7fdf17349700 1 -- 172.21.15.116:0/137687545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf10161340 con 0x7fdf1013b110 2020-11-20T12:16:35.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.584+0000 7fdefe7fc700 1 -- 172.21.15.116:0/137687545 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf00007c80 con 0x7fdf1013b110 2020-11-20T12:16:35.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.584+0000 7fdf17349700 1 -- 172.21.15.116:0/137687545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf101617e0 con 0x7fdf1013b110 2020-11-20T12:16:35.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.584+0000 7fdf17349700 1 -- 172.21.15.116:0/137687545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf10161a90 con 0x7fdf1013b110 2020-11-20T12:16:35.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.584+0000 7fdefe7fc700 1 -- 172.21.15.116:0/137687545 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fdf0001b460 con 0x7fdf1013b110 2020-11-20T12:16:35.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.584+0000 7fdefe7fc700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdee80557c0 0x7fdee8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.584+0000 7fdefe7fc700 1 -- 172.21.15.116:0/137687545 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdf0006ea90 con 0x7fdf1013b110 2020-11-20T12:16:35.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.588+0000 7fdf16347700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdee80557c0 0x7fdee8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.588+0000 7fdf16347700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdee80557c0 0x7fdee8057c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fdf0c00c530 tx=0x7fdf0c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:35.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.592+0000 7fdefe7fc700 1 -- 172.21.15.116:0/137687545 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fdf000447c0 con 0x7fdf1013b110 2020-11-20T12:16:35.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.752+0000 7fdf17349700 1 -- 172.21.15.116:0/137687545 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdf10044f60 con 0x7fdf1013b110 2020-11-20T12:16:35.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.752+0000 7fdefe7fc700 1 -- 172.21.15.116:0/137687545 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdf00047de0 con 0x7fdf1013b110 2020-11-20T12:16:35.753 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:35.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.752+0000 7fdf17349700 1 -- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdee80557c0 msgr2=0x7fdee8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.752+0000 7fdf17349700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdee80557c0 0x7fdee8057c80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fdf0c00c530 tx=0x7fdf0c009040).stop 2020-11-20T12:16:35.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.752+0000 7fdf17349700 1 -- 172.21.15.116:0/137687545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf1013b110 msgr2=0x7fdf100a9600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.752+0000 7fdf17349700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf1013b110 0x7fdf100a9600 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7fdf0000cb50 tx=0x7fdf00008040).stop 2020-11-20T12:16:35.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.756+0000 7fdf17349700 1 -- 172.21.15.116:0/137687545 shutdown_connections 2020-11-20T12:16:35.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.756+0000 7fdf17349700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdee80557c0 0x7fdee8057c80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.756+0000 7fdf17349700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf10151fa0 0x7fdf100a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.756+0000 7fdf17349700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf1013ad30 0x7fdf101389c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.756+0000 7fdf17349700 1 --2- 172.21.15.116:0/137687545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf1013b110 0x7fdf100a9600 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.756+0000 7fdf17349700 1 -- 172.21.15.116:0/137687545 >> 172.21.15.116:0/137687545 conn(0x7fdf100678a0 msgr2=0x7fdf100acf70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:35.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.756+0000 7fdf17349700 1 -- 172.21.15.116:0/137687545 shutdown_connections 2020-11-20T12:16:35.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.756+0000 7fdf17349700 1 -- 172.21.15.116:0/137687545 wait complete. 2020-11-20T12:16:35.778 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:16:35.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.960+0000 7f57b2f1c700 1 Processor -- start 2020-11-20T12:16:35.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b2f1c700 1 -- start start 2020-11-20T12:16:35.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b2f1c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f57ac14f150 0x7f57ac0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b2f1c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57ac0bd780 0x7f57ac0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b2f1c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f57ac14f530 0x7f57ac0a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b2f1c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ac155620 con 0x7f57ac14f530 2020-11-20T12:16:35.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b2f1c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57ac1558c0 con 0x7f57ac14f150 2020-11-20T12:16:35.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b2f1c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac155b60 con 0x7f57ac0bd780 2020-11-20T12:16:35.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b14b9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f57ac14f530 0x7f57ac0a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b14b9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f57ac14f530 0x7f57ac0a74b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43918/0 (socket says 172.21.15.116:43918) 2020-11-20T12:16:35.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b14b9700 1 -- 172.21.15.116:0/3476845634 learned_addr learned my addr 172.21.15.116:0/3476845634 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:35.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57abfff700 1 --2- 172.21.15.116:0/3476845634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57ac0bd780 0x7f57ac0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b0cb8700 1 --2- 172.21.15.116:0/3476845634 >> v2:172.21.15.173:3300/0 conn(0x7f57ac14f150 0x7f57ac0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b14b9700 1 -- 172.21.15.116:0/3476845634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57ac0bd780 msgr2=0x7f57ac0c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:35.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b14b9700 1 --2- 172.21.15.116:0/3476845634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57ac0bd780 0x7f57ac0c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b14b9700 1 -- 172.21.15.116:0/3476845634 >> v2:172.21.15.173:3300/0 conn(0x7f57ac14f150 msgr2=0x7f57ac0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:35.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b14b9700 1 --2- 172.21.15.116:0/3476845634 >> v2:172.21.15.173:3300/0 conn(0x7f57ac14f150 0x7f57ac0c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b14b9700 1 -- 172.21.15.116:0/3476845634 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac155e00 con 0x7f57ac14f530 2020-11-20T12:16:35.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57b14b9700 1 --2- 172.21.15.116:0/3476845634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57ac14f530 0x7f57ac0a74b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f57ac081540 tx=0x7f57a4009290).ready entity=mon.0 client_cookie=237871cfe30ebc29 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:35.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57ab7fe700 1 -- 172.21.15.116:0/3476845634 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a400bc50 con 0x7f57ac14f530 2020-11-20T12:16:35.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57ab7fe700 1 -- 172.21.15.116:0/3476845634 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a400bdb0 con 0x7f57ac14f530 2020-11-20T12:16:35.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.964+0000 7f57ab7fe700 1 -- 172.21.15.116:0/3476845634 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4011610 con 0x7f57ac14f530 2020-11-20T12:16:35.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.968+0000 7f57b2f1c700 1 -- 172.21.15.116:0/3476845634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57ac14f530 msgr2=0x7f57ac0a74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.968+0000 7f57b2f1c700 1 --2- 172.21.15.116:0/3476845634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57ac14f530 0x7f57ac0a74b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f57ac081540 tx=0x7f57a4009290).stop 2020-11-20T12:16:35.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.968+0000 7f57b2f1c700 1 -- 172.21.15.116:0/3476845634 shutdown_connections 2020-11-20T12:16:35.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.968+0000 7f57b2f1c700 1 --2- 172.21.15.116:0/3476845634 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57ac14f530 0x7f57ac0a74b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.968+0000 7f57b2f1c700 1 --2- 172.21.15.116:0/3476845634 >> v2:172.21.15.173:3300/0 conn(0x7f57ac14f150 0x7f57ac0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.968+0000 7f57b2f1c700 1 --2- 172.21.15.116:0/3476845634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57ac0bd780 0x7f57ac0c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.968+0000 7f57b2f1c700 1 -- 172.21.15.116:0/3476845634 >> 172.21.15.116:0/3476845634 conn(0x7f57ac061580 msgr2=0x7f57ac061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:35.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.968+0000 7f57b2f1c700 1 -- 172.21.15.116:0/3476845634 shutdown_connections 2020-11-20T12:16:35.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.968+0000 7f57b2f1c700 1 -- 172.21.15.116:0/3476845634 wait complete. 2020-11-20T12:16:35.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.968+0000 7f57b2f1c700 1 Processor -- start 2020-11-20T12:16:35.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.968+0000 7f57b2f1c700 1 -- start start 2020-11-20T12:16:35.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b2f1c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac0bd780 0x7f57ac0a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b2f1c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57ac14f150 0x7f57ac0ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b2f1c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57ac14f530 0x7f57ac0ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b2f1c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0c01a0 con 0x7f57ac14f530 2020-11-20T12:16:35.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b2f1c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0c0390 con 0x7f57ac0bd780 2020-11-20T12:16:35.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b2f1c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57ac0c0630 con 0x7f57ac14f150 2020-11-20T12:16:35.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b14b9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57ac14f530 0x7f57ac0ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57abfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57ac14f150 0x7f57ac0ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b14b9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57ac14f530 0x7f57ac0ada30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43924/0 (socket says 172.21.15.116:43924) 2020-11-20T12:16:35.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b14b9700 1 -- 172.21.15.116:0/2505865899 learned_addr learned my addr 172.21.15.116:0/2505865899 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:35.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b0cb8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac0bd780 0x7f57ac0a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b14b9700 1 -- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57ac14f150 msgr2=0x7f57ac0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b14b9700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57ac14f150 0x7f57ac0ad540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b14b9700 1 -- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac0bd780 msgr2=0x7f57ac0a76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:35.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b14b9700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac0bd780 0x7f57ac0a76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:35.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b14b9700 1 -- 172.21.15.116:0/2505865899 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57ac0c08d0 con 0x7f57ac14f530 2020-11-20T12:16:35.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b14b9700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57ac14f530 0x7f57ac0ada30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f57a40092d0 tx=0x7f57a4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:35.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57a8ff9700 1 -- 172.21.15.116:0/2505865899 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a400bc50 con 0x7f57ac14f530 2020-11-20T12:16:35.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57a8ff9700 1 -- 172.21.15.116:0/2505865899 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a4010ae0 con 0x7f57ac14f530 2020-11-20T12:16:35.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b2f1c700 1 -- 172.21.15.116:0/2505865899 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ac0c0b70 con 0x7f57ac14f530 2020-11-20T12:16:35.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57a8ff9700 1 -- 172.21.15.116:0/2505865899 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a4010e80 con 0x7f57ac14f530 2020-11-20T12:16:35.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.972+0000 7f57b2f1c700 1 -- 172.21.15.116:0/2505865899 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ac0c12b0 con 0x7f57ac14f530 2020-11-20T12:16:35.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.976+0000 7f57b2f1c700 1 -- 172.21.15.116:0/2505865899 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ac05ca30 con 0x7f57ac14f530 2020-11-20T12:16:35.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.976+0000 7f57a8ff9700 1 -- 172.21.15.116:0/2505865899 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f57a4017070 con 0x7f57ac14f530 2020-11-20T12:16:35.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.976+0000 7f57a8ff9700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57840557c0 0x7f5784057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:35.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.976+0000 7f57a8ff9700 1 -- 172.21.15.116:0/2505865899 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f57a40704f0 con 0x7f57ac14f530 2020-11-20T12:16:35.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.976+0000 7f57b0cb8700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57840557c0 0x7f5784057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:35.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.980+0000 7f57b0cb8700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57840557c0 0x7f5784057c80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f57a000ba00 tx=0x7f57a0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:35.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:35.984+0000 7f57a8ff9700 1 -- 172.21.15.116:0/2505865899 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f57a4046390 con 0x7f57ac14f530 2020-11-20T12:16:36.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.140+0000 7f57b2f1c700 1 -- 172.21.15.116:0/2505865899 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57ac0ab0d0 con 0x7f57ac14f530 2020-11-20T12:16:36.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.140+0000 7f57a8ff9700 1 -- 172.21.15.116:0/2505865899 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f57a4015020 con 0x7f57ac14f530 2020-11-20T12:16:36.142 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:36.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 -- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57840557c0 msgr2=0x7f5784057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57840557c0 0x7f5784057c80 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f57a000ba00 tx=0x7f57a0008040).stop 2020-11-20T12:16:36.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 -- 172.21.15.116:0/2505865899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57ac14f530 msgr2=0x7f57ac0ada30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57ac14f530 0x7f57ac0ada30 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f57a40092d0 tx=0x7f57a4016000).stop 2020-11-20T12:16:36.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 -- 172.21.15.116:0/2505865899 shutdown_connections 2020-11-20T12:16:36.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57840557c0 0x7f5784057c80 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57ac14f530 0x7f57ac0ada30 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57ac0bd780 0x7f57ac0a76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 --2- 172.21.15.116:0/2505865899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57ac14f150 0x7f57ac0ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 -- 172.21.15.116:0/2505865899 >> 172.21.15.116:0/2505865899 conn(0x7f57ac061580 msgr2=0x7f57ac0c6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:36.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 -- 172.21.15.116:0/2505865899 shutdown_connections 2020-11-20T12:16:36.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.144+0000 7f57b2f1c700 1 -- 172.21.15.116:0/2505865899 wait complete. 2020-11-20T12:16:36.173 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:16:36.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.344+0000 7fa150bf4700 1 Processor -- start 2020-11-20T12:16:36.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.344+0000 7fa150bf4700 1 -- start start 2020-11-20T12:16:36.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa150bf4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa14c0c1bb0 0x7fa14c156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa150bf4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa14c0a28c0 0x7fa14c066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa150bf4700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa14c0a38c0 con 0x7fa14c0c03f0 2020-11-20T12:16:36.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa150bf4700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa14c156e30 con 0x7fa14c0c1bb0 2020-11-20T12:16:36.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa150bf4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0a2e40 con 0x7fa14c0a28c0 2020-11-20T12:16:36.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa14b7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa14c0c03f0 0x7fa14c0677c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53362/0 (socket says 172.21.15.116:53362) 2020-11-20T12:16:36.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa14b7fe700 1 -- 172.21.15.116:0/2071007665 learned_addr learned my addr 172.21.15.116:0/2071007665 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:36.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa143fff700 1 --2- 172.21.15.116:0/2071007665 >> v2:172.21.15.116:3300/0 conn(0x7fa14c0c1bb0 0x7fa14c156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa14bfff700 1 --2- 172.21.15.116:0/2071007665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa14c0a28c0 0x7fa14c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa14affd700 1 -- 172.21.15.116:0/2071007665 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27137788 0 0) 0x7fa14c0a38c0 con 0x7fa14c0c03f0 2020-11-20T12:16:36.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa14affd700 1 -- 172.21.15.116:0/2071007665 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa12c003240 con 0x7fa14c0c03f0 2020-11-20T12:16:36.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa14affd700 1 -- 172.21.15.116:0/2071007665 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2819540808 0 0) 0x7fa12c003240 con 0x7fa14c0c03f0 2020-11-20T12:16:36.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa143fff700 1 -- 172.21.15.116:0/2071007665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa14c0a28c0 msgr2=0x7fa14c066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:36.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa143fff700 1 --2- 172.21.15.116:0/2071007665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa14c0a28c0 0x7fa14c066570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa143fff700 1 -- 172.21.15.116:0/2071007665 >> v1:172.21.15.116:6789/0 conn(0x7fa14c0c03f0 legacy=0x7fa14c0677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa143fff700 1 -- 172.21.15.116:0/2071007665 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0a30e0 con 0x7fa14c0c1bb0 2020-11-20T12:16:36.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.348+0000 7fa143fff700 1 --2- 172.21.15.116:0/2071007665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14c0c1bb0 0x7fa14c156940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fa13400b9a0 tx=0x7fa134006df0).ready entity=mon.0 client_cookie=6fa2160a362337a4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:36.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.352+0000 7fa14affd700 1 -- 172.21.15.116:0/2071007665 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0014e0 con 0x7fa14c0c1bb0 2020-11-20T12:16:36.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.352+0000 7fa14affd700 1 -- 172.21.15.116:0/2071007665 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa134011400 con 0x7fa14c0c1bb0 2020-11-20T12:16:36.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.352+0000 7fa14affd700 1 -- 172.21.15.116:0/2071007665 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134011780 con 0x7fa14c0c1bb0 2020-11-20T12:16:36.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.352+0000 7fa150bf4700 1 -- 172.21.15.116:0/2071007665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14c0c1bb0 msgr2=0x7fa14c156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.352+0000 7fa150bf4700 1 --2- 172.21.15.116:0/2071007665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14c0c1bb0 0x7fa14c156940 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fa13400b9a0 tx=0x7fa134006df0).stop 2020-11-20T12:16:36.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.352+0000 7fa150bf4700 1 -- 172.21.15.116:0/2071007665 shutdown_connections 2020-11-20T12:16:36.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.352+0000 7fa150bf4700 1 --2- 172.21.15.116:0/2071007665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa14c0a28c0 0x7fa14c066570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.352+0000 7fa150bf4700 1 --2- 172.21.15.116:0/2071007665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14c0c1bb0 0x7fa14c156940 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.352+0000 7fa150bf4700 1 -- 172.21.15.116:0/2071007665 >> 172.21.15.116:0/2071007665 conn(0x7fa14c060c30 msgr2=0x7fa14c0c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:36.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.352+0000 7fa150bf4700 1 -- 172.21.15.116:0/2071007665 shutdown_connections 2020-11-20T12:16:36.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.352+0000 7fa150bf4700 1 -- 172.21.15.116:0/2071007665 wait complete. 2020-11-20T12:16:36.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa150bf4700 1 Processor -- start 2020-11-20T12:16:36.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa150bf4700 1 -- start start 2020-11-20T12:16:36.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa150bf4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa14c0a28c0 0x7fa14c149100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa150bf4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14c0c03f0 0x7fa14c08f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa150bf4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa14c0c1bb0 0x7fa14c0a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa150bf4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0a3870 con 0x7fa14c0c03f0 2020-11-20T12:16:36.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa150bf4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0a7390 con 0x7fa14c0c1bb0 2020-11-20T12:16:36.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa150bf4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0a7630 con 0x7fa14c0a28c0 2020-11-20T12:16:36.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa143fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14c0c03f0 0x7fa14c08f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa143fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14c0c03f0 0x7fa14c08f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43938/0 (socket says 172.21.15.116:43938) 2020-11-20T12:16:36.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa143fff700 1 -- 172.21.15.116:0/1008238841 learned_addr learned my addr 172.21.15.116:0/1008238841 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:36.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa14b7fe700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa14c0a28c0 0x7fa14c149100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa14bfff700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa14c0c1bb0 0x7fa14c0a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa143fff700 1 -- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa14c0a28c0 msgr2=0x7fa14c149100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa143fff700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa14c0a28c0 0x7fa14c149100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa143fff700 1 -- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa14c0c1bb0 msgr2=0x7fa14c0a92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa143fff700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa14c0c1bb0 0x7fa14c0a92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.356+0000 7fa143fff700 1 -- 172.21.15.116:0/1008238841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa14c0a3aa0 con 0x7fa14c0c03f0 2020-11-20T12:16:36.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.360+0000 7fa143fff700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14c0c03f0 0x7fa14c08f6d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fa134006fd0 tx=0x7fa134013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:36.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.360+0000 7fa1437fe700 1 -- 172.21.15.116:0/1008238841 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa134009b70 con 0x7fa14c0c03f0 2020-11-20T12:16:36.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.360+0000 7fa1437fe700 1 -- 172.21.15.116:0/1008238841 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa134009cd0 con 0x7fa14c0c03f0 2020-11-20T12:16:36.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.360+0000 7fa1437fe700 1 -- 172.21.15.116:0/1008238841 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13401d5d0 con 0x7fa14c0c03f0 2020-11-20T12:16:36.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.360+0000 7fa150bf4700 1 -- 172.21.15.116:0/1008238841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14c0a3d40 con 0x7fa14c0c03f0 2020-11-20T12:16:36.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.360+0000 7fa150bf4700 1 -- 172.21.15.116:0/1008238841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c0a6f60 con 0x7fa14c0c03f0 2020-11-20T12:16:36.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.360+0000 7fa150bf4700 1 -- 172.21.15.116:0/1008238841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14c0a3ed0 con 0x7fa14c0c03f0 2020-11-20T12:16:36.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.360+0000 7fa1437fe700 1 -- 172.21.15.116:0/1008238841 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa13401a040 con 0x7fa14c0c03f0 2020-11-20T12:16:36.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.364+0000 7fa1437fe700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1280557f0 0x7fa128057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.364+0000 7fa1437fe700 1 -- 172.21.15.116:0/1008238841 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa13406f840 con 0x7fa14c0c03f0 2020-11-20T12:16:36.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.364+0000 7fa14b7fe700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1280557f0 0x7fa128057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.364+0000 7fa14b7fe700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1280557f0 0x7fa128057cb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fa13c004a10 tx=0x7fa13c001280).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:36.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.368+0000 7fa1437fe700 1 -- 172.21.15.116:0/1008238841 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa1340456e0 con 0x7fa14c0c03f0 2020-11-20T12:16:36.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.524+0000 7fa150bf4700 1 -- 172.21.15.116:0/1008238841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa14c044f60 con 0x7fa14c0c03f0 2020-11-20T12:16:36.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa1437fe700 1 -- 172.21.15.116:0/1008238841 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa134024030 con 0x7fa14c0c03f0 2020-11-20T12:16:36.527 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:36.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 -- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1280557f0 msgr2=0x7fa128057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1280557f0 0x7fa128057cb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fa13c004a10 tx=0x7fa13c001280).stop 2020-11-20T12:16:36.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 -- 172.21.15.116:0/1008238841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14c0c03f0 msgr2=0x7fa14c08f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14c0c03f0 0x7fa14c08f6d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fa134006fd0 tx=0x7fa134013000).stop 2020-11-20T12:16:36.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 -- 172.21.15.116:0/1008238841 shutdown_connections 2020-11-20T12:16:36.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1280557f0 0x7fa128057cb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa14c0c1bb0 0x7fa14c0a92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa14c0a28c0 0x7fa14c149100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 --2- 172.21.15.116:0/1008238841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa14c0c03f0 0x7fa14c08f6d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 -- 172.21.15.116:0/1008238841 >> 172.21.15.116:0/1008238841 conn(0x7fa14c060c30 msgr2=0x7fa14c067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:36.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 -- 172.21.15.116:0/1008238841 shutdown_connections 2020-11-20T12:16:36.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.528+0000 7fa150bf4700 1 -- 172.21.15.116:0/1008238841 wait complete. 2020-11-20T12:16:36.553 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:36.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.728+0000 7f7b070fe700 1 Processor -- start 2020-11-20T12:16:36.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.728+0000 7f7b070fe700 1 -- start start 2020-11-20T12:16:36.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.728+0000 7f7b070fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7b000c41c0 0x7f7b000c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.728+0000 7f7b070fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7b00138d80 0x7f7b000a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.728+0000 7f7b070fe700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b000a9180 con 0x7f7b000a5a10 2020-11-20T12:16:36.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.728+0000 7f7b070fe700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b000a6380 con 0x7f7b000c41c0 2020-11-20T12:16:36.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.728+0000 7f7b070fe700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b000a8c80 con 0x7f7b00138d80 2020-11-20T12:16:36.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b060fc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7b000c41c0 0x7f7b000c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b068fd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7b00138d80 0x7f7b000a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b068fd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7b00138d80 0x7f7b000a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:42994/0 (socket says 172.21.15.116:42994) 2020-11-20T12:16:36.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b068fd700 1 -- 172.21.15.116:0/2648800740 learned_addr learned my addr 172.21.15.116:0/2648800740 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:36.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b058fb700 1 --1- 172.21.15.116:0/2648800740 >> v1:172.21.15.173:6789/0 conn(0x7f7b000a5a10 0x7f7b00138670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49608/0 (socket says 172.21.15.116:49608) 2020-11-20T12:16:36.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b060fc700 1 -- 172.21.15.116:0/2648800740 >> v2:172.21.15.173:3300/0 conn(0x7f7b00138d80 msgr2=0x7f7b000a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:36.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b060fc700 1 --2- 172.21.15.116:0/2648800740 >> v2:172.21.15.173:3300/0 conn(0x7f7b00138d80 0x7f7b000a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b060fc700 1 -- 172.21.15.116:0/2648800740 >> v1:172.21.15.173:6789/0 conn(0x7f7b000a5a10 legacy=0x7f7b00138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b060fc700 1 -- 172.21.15.116:0/2648800740 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b000a9420 con 0x7f7b000c41c0 2020-11-20T12:16:36.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b060fc700 1 --2- 172.21.15.116:0/2648800740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b000c41c0 0x7f7b000c45a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f7b00139230 tx=0x7f7afc006df0).ready entity=mon.0 client_cookie=592bb273a2f45399 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:36.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b050fa700 1 -- 172.21.15.116:0/2648800740 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc0097b0 con 0x7f7b000c41c0 2020-11-20T12:16:36.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b050fa700 1 -- 172.21.15.116:0/2648800740 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc009910 con 0x7f7b000c41c0 2020-11-20T12:16:36.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b050fa700 1 -- 172.21.15.116:0/2648800740 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc009c90 con 0x7f7b000c41c0 2020-11-20T12:16:36.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b070fe700 1 -- 172.21.15.116:0/2648800740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b000c41c0 msgr2=0x7f7b000c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.732+0000 7f7b070fe700 1 --2- 172.21.15.116:0/2648800740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b000c41c0 0x7f7b000c45a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f7b00139230 tx=0x7f7afc006df0).stop 2020-11-20T12:16:36.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.736+0000 7f7b070fe700 1 -- 172.21.15.116:0/2648800740 shutdown_connections 2020-11-20T12:16:36.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.736+0000 7f7b070fe700 1 --2- 172.21.15.116:0/2648800740 >> v2:172.21.15.173:3300/0 conn(0x7f7b00138d80 0x7f7b000a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.736+0000 7f7b070fe700 1 --2- 172.21.15.116:0/2648800740 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b000c41c0 0x7f7b000c45a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.736+0000 7f7b070fe700 1 -- 172.21.15.116:0/2648800740 >> 172.21.15.116:0/2648800740 conn(0x7f7b00062520 msgr2=0x7f7b00062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:36.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.736+0000 7f7b070fe700 1 -- 172.21.15.116:0/2648800740 shutdown_connections 2020-11-20T12:16:36.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.736+0000 7f7b070fe700 1 -- 172.21.15.116:0/2648800740 wait complete. 2020-11-20T12:16:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.736+0000 7f7b070fe700 1 Processor -- start 2020-11-20T12:16:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.736+0000 7f7b070fe700 1 -- start start 2020-11-20T12:16:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b070fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b000a5a10 0x7f7b000c09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b070fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b000c41c0 0x7f7b000a25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b070fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b00138d80 0x7f7b000a2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b070fe700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000bac80 con 0x7f7b000a5a10 2020-11-20T12:16:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b070fe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b000baf20 con 0x7f7b000c41c0 2020-11-20T12:16:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b070fe700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0009d610 con 0x7f7b00138d80 2020-11-20T12:16:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b060fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b000a5a10 0x7f7b000c09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b060fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b000a5a10 0x7f7b000c09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43948/0 (socket says 172.21.15.116:43948) 2020-11-20T12:16:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b060fc700 1 -- 172.21.15.116:0/3340885739 learned_addr learned my addr 172.21.15.116:0/3340885739 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b058fb700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b000c41c0 0x7f7b000a25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b068fd700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b00138d80 0x7f7b000a2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b060fc700 1 -- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b00138d80 msgr2=0x7f7b000a2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b060fc700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b00138d80 0x7f7b000a2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b060fc700 1 -- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b000c41c0 msgr2=0x7f7b000a25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b060fc700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b000c41c0 0x7f7b000a25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b060fc700 1 -- 172.21.15.116:0/3340885739 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0009d8b0 con 0x7f7b000a5a10 2020-11-20T12:16:36.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b060fc700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b000a5a10 0x7f7b000c09f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f7b000c1ad0 tx=0x7f7afc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:36.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7aee7fc700 1 -- 172.21.15.116:0/3340885739 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc01e460 con 0x7f7b000a5a10 2020-11-20T12:16:36.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7aee7fc700 1 -- 172.21.15.116:0/3340885739 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7afc01e5c0 con 0x7f7b000a5a10 2020-11-20T12:16:36.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7aee7fc700 1 -- 172.21.15.116:0/3340885739 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7afc01e940 con 0x7f7b000a5a10 2020-11-20T12:16:36.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b070fe700 1 -- 172.21.15.116:0/3340885739 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b000a00d0 con 0x7f7b000a5a10 2020-11-20T12:16:36.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.740+0000 7f7b070fe700 1 -- 172.21.15.116:0/3340885739 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b000b9ed0 con 0x7f7b000a5a10 2020-11-20T12:16:36.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.744+0000 7f7b070fe700 1 -- 172.21.15.116:0/3340885739 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0009da40 con 0x7f7b000a5a10 2020-11-20T12:16:36.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.744+0000 7f7aee7fc700 1 -- 172.21.15.116:0/3340885739 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7afc025030 con 0x7f7b000a5a10 2020-11-20T12:16:36.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.744+0000 7f7aee7fc700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7ad80557f0 0x7f7ad8057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:36.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.744+0000 7f7aee7fc700 1 -- 172.21.15.116:0/3340885739 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7afc06fe30 con 0x7f7b000a5a10 2020-11-20T12:16:36.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.744+0000 7f7b058fb700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7ad80557f0 0x7f7ad8057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:36.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.748+0000 7f7b058fb700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7ad80557f0 0x7f7ad8057cb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f7b000a9c60 tx=0x7f7af0007180).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:36.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.748+0000 7f7aee7fc700 1 -- 172.21.15.116:0/3340885739 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7afc045d80 con 0x7f7b000a5a10 2020-11-20T12:16:36.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.912+0000 7f7b070fe700 1 -- 172.21.15.116:0/3340885739 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b00044f60 con 0x7f7b000a5a10 2020-11-20T12:16:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.912+0000 7f7aee7fc700 1 -- 172.21.15.116:0/3340885739 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f7afc031200 con 0x7f7b000a5a10 2020-11-20T12:16:36.916 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:36.916 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 -- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7ad80557f0 msgr2=0x7f7ad8057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7ad80557f0 0x7f7ad8057cb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f7b000a9c60 tx=0x7f7af0007180).stop 2020-11-20T12:16:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 -- 172.21.15.116:0/3340885739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b000a5a10 msgr2=0x7f7b000c09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b000a5a10 0x7f7b000c09f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f7b000c1ad0 tx=0x7f7afc015000).stop 2020-11-20T12:16:36.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 -- 172.21.15.116:0/3340885739 shutdown_connections 2020-11-20T12:16:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7ad80557f0 0x7f7ad8057cb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b00138d80 0x7f7b000a2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b000a5a10 0x7f7b000c09f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 --2- 172.21.15.116:0/3340885739 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b000c41c0 0x7f7b000a25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 -- 172.21.15.116:0/3340885739 >> 172.21.15.116:0/3340885739 conn(0x7f7b00062520 msgr2=0x7f7b00137b10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:36.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 -- 172.21.15.116:0/3340885739 shutdown_connections 2020-11-20T12:16:36.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:36.916+0000 7f7b070fe700 1 -- 172.21.15.116:0/3340885739 wait complete. 2020-11-20T12:16:36.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:15:59.065285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:15:59.602311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:16:36.943 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:37.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.128+0000 7fad224be700 1 Processor -- start 2020-11-20T12:16:37.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.128+0000 7fad224be700 1 -- start start 2020-11-20T12:16:37.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad224be700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fad1c13b110 0x7fad1c151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad224be700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1c152a80 con 0x7fad1c151fa0 2020-11-20T12:16:37.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad224be700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1c152540 con 0x7fad1c13ad30 2020-11-20T12:16:37.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad224be700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1c1527e0 con 0x7fad1c13b110 2020-11-20T12:16:37.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad20cbb700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fad1c13b110 0x7fad1c151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad21cbd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fad1c151fa0 0x7fad1c061250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53394/0 (socket says 172.21.15.116:53394) 2020-11-20T12:16:37.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad21cbd700 1 -- 172.21.15.116:0/2192395441 learned_addr learned my addr 172.21.15.116:0/2192395441 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:37.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad20cbb700 1 -- 172.21.15.116:0/2192395441 >> v1:172.21.15.173:6789/0 conn(0x7fad1c13ad30 legacy=0x7fad1c0a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad20cbb700 1 -- 172.21.15.116:0/2192395441 >> v1:172.21.15.116:6789/0 conn(0x7fad1c151fa0 legacy=0x7fad1c061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad20cbb700 1 -- 172.21.15.116:0/2192395441 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c0ac910 con 0x7fad1c13b110 2020-11-20T12:16:37.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad20cbb700 1 --2- 172.21.15.116:0/2192395441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad1c13b110 0x7fad1c151ab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fad0c00b9a0 tx=0x7fad0c006df0).ready entity=mon.0 client_cookie=907d8fb284d640da server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:37.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad0bfff700 1 -- 172.21.15.116:0/2192395441 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c009840 con 0x7fad1c13b110 2020-11-20T12:16:37.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad0bfff700 1 -- 172.21.15.116:0/2192395441 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0c0099a0 con 0x7fad1c13b110 2020-11-20T12:16:37.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad0bfff700 1 -- 172.21.15.116:0/2192395441 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0c009d20 con 0x7fad1c13b110 2020-11-20T12:16:37.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad224be700 1 -- 172.21.15.116:0/2192395441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad1c13b110 msgr2=0x7fad1c151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad224be700 1 --2- 172.21.15.116:0/2192395441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad1c13b110 0x7fad1c151ab0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fad0c00b9a0 tx=0x7fad0c006df0).stop 2020-11-20T12:16:37.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad224be700 1 -- 172.21.15.116:0/2192395441 shutdown_connections 2020-11-20T12:16:37.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad224be700 1 --2- 172.21.15.116:0/2192395441 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad1c13b110 0x7fad1c151ab0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.132+0000 7fad224be700 1 -- 172.21.15.116:0/2192395441 >> 172.21.15.116:0/2192395441 conn(0x7fad1c0678a0 msgr2=0x7fad1c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:37.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.136+0000 7fad224be700 1 -- 172.21.15.116:0/2192395441 shutdown_connections 2020-11-20T12:16:37.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.136+0000 7fad224be700 1 -- 172.21.15.116:0/2192395441 wait complete. 2020-11-20T12:16:37.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.136+0000 7fad224be700 1 Processor -- start 2020-11-20T12:16:37.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.136+0000 7fad224be700 1 -- start start 2020-11-20T12:16:37.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad224be700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad1c13ad30 0x7fad1c0a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad224be700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad1c151fa0 0x7fad1c0a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad224be700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad1c155860 0x7fad1c137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad224be700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c09f120 con 0x7fad1c13ad30 2020-11-20T12:16:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad224be700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c09f3c0 con 0x7fad1c155860 2020-11-20T12:16:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad224be700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c09f660 con 0x7fad1c151fa0 2020-11-20T12:16:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad214bc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad1c13ad30 0x7fad1c0a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad214bc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad1c13ad30 0x7fad1c0a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43962/0 (socket says 172.21.15.116:43962) 2020-11-20T12:16:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad214bc700 1 -- 172.21.15.116:0/663381309 learned_addr learned my addr 172.21.15.116:0/663381309 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad20cbb700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad1c151fa0 0x7fad1c0a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad21cbd700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad1c155860 0x7fad1c137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad214bc700 1 -- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad1c151fa0 msgr2=0x7fad1c0a4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad214bc700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad1c151fa0 0x7fad1c0a4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad214bc700 1 -- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad1c155860 msgr2=0x7fad1c137bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad214bc700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad1c155860 0x7fad1c137bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.140+0000 7fad214bc700 1 -- 172.21.15.116:0/663381309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c09f900 con 0x7fad1c13ad30 2020-11-20T12:16:37.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.144+0000 7fad214bc700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad1c13ad30 0x7fad1c0a0040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fad18001e70 tx=0x7fad18007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:37.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.144+0000 7fad097fa700 1 -- 172.21.15.116:0/663381309 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad18007020 con 0x7fad1c13ad30 2020-11-20T12:16:37.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.144+0000 7fad097fa700 1 -- 172.21.15.116:0/663381309 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad18007180 con 0x7fad1c13ad30 2020-11-20T12:16:37.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.144+0000 7fad097fa700 1 -- 172.21.15.116:0/663381309 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad180064f0 con 0x7fad1c13ad30 2020-11-20T12:16:37.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.144+0000 7fad224be700 1 -- 172.21.15.116:0/663381309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c09fba0 con 0x7fad1c13ad30 2020-11-20T12:16:37.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.144+0000 7fad224be700 1 -- 172.21.15.116:0/663381309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c0a5490 con 0x7fad1c13ad30 2020-11-20T12:16:37.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.144+0000 7fad224be700 1 -- 172.21.15.116:0/663381309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c09fd30 con 0x7fad1c13ad30 2020-11-20T12:16:37.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.144+0000 7fad097fa700 1 -- 172.21.15.116:0/663381309 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fad18010070 con 0x7fad1c13ad30 2020-11-20T12:16:37.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.148+0000 7fad097fa700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7facf40557c0 0x7facf4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.148+0000 7fad097fa700 1 -- 172.21.15.116:0/663381309 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fad1806d960 con 0x7fad1c13ad30 2020-11-20T12:16:37.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.148+0000 7fad20cbb700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7facf40557c0 0x7facf4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.148+0000 7fad20cbb700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7facf40557c0 0x7facf4057c80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fad1c13b550 tx=0x7fad0c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:37.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.152+0000 7fad097fa700 1 -- 172.21.15.116:0/663381309 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fad18042fd0 con 0x7fad1c13ad30 2020-11-20T12:16:37.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.312+0000 7fad224be700 1 -- 172.21.15.116:0/663381309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fad1c044f60 con 0x7fad1c13ad30 2020-11-20T12:16:37.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.312+0000 7fad097fa700 1 -- 172.21.15.116:0/663381309 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7fad18043800 con 0x7fad1c13ad30 2020-11-20T12:16:37.313 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:37.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.312+0000 7fad224be700 1 -- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7facf40557c0 msgr2=0x7facf4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.312+0000 7fad224be700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7facf40557c0 0x7facf4057c80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fad1c13b550 tx=0x7fad0c015000).stop 2020-11-20T12:16:37.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.312+0000 7fad224be700 1 -- 172.21.15.116:0/663381309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad1c13ad30 msgr2=0x7fad1c0a0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.312+0000 7fad224be700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad1c13ad30 0x7fad1c0a0040 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fad18001e70 tx=0x7fad18007c50).stop 2020-11-20T12:16:37.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.316+0000 7fad224be700 1 -- 172.21.15.116:0/663381309 shutdown_connections 2020-11-20T12:16:37.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.316+0000 7fad224be700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7facf40557c0 0x7facf4057c80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.316+0000 7fad224be700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad1c155860 0x7fad1c137bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.316+0000 7fad224be700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad1c13ad30 0x7fad1c0a0040 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.316+0000 7fad224be700 1 --2- 172.21.15.116:0/663381309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad1c151fa0 0x7fad1c0a4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.316+0000 7fad224be700 1 -- 172.21.15.116:0/663381309 >> 172.21.15.116:0/663381309 conn(0x7fad1c0678a0 msgr2=0x7fad1c0a7ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:37.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.316+0000 7fad224be700 1 -- 172.21.15.116:0/663381309 shutdown_connections 2020-11-20T12:16:37.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.316+0000 7fad224be700 1 -- 172.21.15.116:0/663381309 wait complete. 2020-11-20T12:16:37.341 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:37.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.516+0000 7fa0e5c72700 1 Processor -- start 2020-11-20T12:16:37.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.516+0000 7fa0e5c72700 1 -- start start 2020-11-20T12:16:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0e5c72700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0e014f860 0x7fa0e00c14b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0e5c72700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0e0066b70 0x7fa0e00a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0e5c72700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa0e0066ff0 0x7fa0e00a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0e5c72700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e014fc40 con 0x7fa0e014f860 2020-11-20T12:16:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0e5c72700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e00c0590 con 0x7fa0e0066ff0 2020-11-20T12:16:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0e5c72700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00c0830 con 0x7fa0e0066b70 2020-11-20T12:16:37.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0df7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0e014f860 0x7fa0e00c14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0df7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0e014f860 0x7fa0e00c14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43970/0 (socket says 172.21.15.116:43970) 2020-11-20T12:16:37.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0df7fe700 1 -- 172.21.15.116:0/3366502059 learned_addr learned my addr 172.21.15.116:0/3366502059 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:37.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0deffd700 1 --2- 172.21.15.116:0/3366502059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0e0066b70 0x7fa0e00a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0dffff700 1 --2- 172.21.15.116:0/3366502059 >> v2:172.21.15.173:3300/0 conn(0x7fa0e0066ff0 0x7fa0e00a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0df7fe700 1 -- 172.21.15.116:0/3366502059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0e0066b70 msgr2=0x7fa0e00a5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:37.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0df7fe700 1 --2- 172.21.15.116:0/3366502059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0e0066b70 0x7fa0e00a5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0df7fe700 1 -- 172.21.15.116:0/3366502059 >> v2:172.21.15.173:3300/0 conn(0x7fa0e0066ff0 msgr2=0x7fa0e00a64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:37.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0df7fe700 1 --2- 172.21.15.116:0/3366502059 >> v2:172.21.15.173:3300/0 conn(0x7fa0e0066ff0 0x7fa0e00a64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0df7fe700 1 -- 172.21.15.116:0/3366502059 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e00c0ad0 con 0x7fa0e014f860 2020-11-20T12:16:37.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.520+0000 7fa0dffff700 1 --2- 172.21.15.116:0/3366502059 >> v2:172.21.15.173:3300/0 conn(0x7fa0e0066ff0 0x7fa0e00a64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:37.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0df7fe700 1 --2- 172.21.15.116:0/3366502059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e014f860 0x7fa0e00c14b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fa0e00bb240 tx=0x7fa0d0006eb0).ready entity=mon.0 client_cookie=e85e63f07870b582 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:37.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0de7fc700 1 -- 172.21.15.116:0/3366502059 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0009810 con 0x7fa0e014f860 2020-11-20T12:16:37.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0de7fc700 1 -- 172.21.15.116:0/3366502059 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d0009970 con 0x7fa0e014f860 2020-11-20T12:16:37.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0de7fc700 1 -- 172.21.15.116:0/3366502059 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0009cf0 con 0x7fa0e014f860 2020-11-20T12:16:37.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0e5c72700 1 -- 172.21.15.116:0/3366502059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e014f860 msgr2=0x7fa0e00c14b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0e5c72700 1 --2- 172.21.15.116:0/3366502059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e014f860 0x7fa0e00c14b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fa0e00bb240 tx=0x7fa0d0006eb0).stop 2020-11-20T12:16:37.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0e5c72700 1 -- 172.21.15.116:0/3366502059 shutdown_connections 2020-11-20T12:16:37.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0e5c72700 1 --2- 172.21.15.116:0/3366502059 >> v2:172.21.15.173:3300/0 conn(0x7fa0e0066ff0 0x7fa0e00a64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0e5c72700 1 --2- 172.21.15.116:0/3366502059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e014f860 0x7fa0e00c14b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0e5c72700 1 --2- 172.21.15.116:0/3366502059 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0e0066b70 0x7fa0e00a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0e5c72700 1 -- 172.21.15.116:0/3366502059 >> 172.21.15.116:0/3366502059 conn(0x7fa0e0060170 msgr2=0x7fa0e0060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:37.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0e5c72700 1 -- 172.21.15.116:0/3366502059 shutdown_connections 2020-11-20T12:16:37.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.524+0000 7fa0e5c72700 1 -- 172.21.15.116:0/3366502059 wait complete. 2020-11-20T12:16:37.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0e5c72700 1 Processor -- start 2020-11-20T12:16:37.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0e5c72700 1 -- start start 2020-11-20T12:16:37.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0e5c72700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0e0066b70 0x7fa0e01526e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0e5c72700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e0066ff0 0x7fa0e00b9150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0e5c72700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0e00b9640 0x7fa0e00bbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0e5c72700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e00b9aa0 con 0x7fa0e0066ff0 2020-11-20T12:16:37.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0e5c72700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e0151030 con 0x7fa0e00b9640 2020-11-20T12:16:37.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0e5c72700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e0162a30 con 0x7fa0e0066b70 2020-11-20T12:16:37.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0deffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e0066ff0 0x7fa0e00b9150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0deffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e0066ff0 0x7fa0e00b9150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43978/0 (socket says 172.21.15.116:43978) 2020-11-20T12:16:37.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0deffd700 1 -- 172.21.15.116:0/4162932435 learned_addr learned my addr 172.21.15.116:0/4162932435 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:37.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0df7fe700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0e0066b70 0x7fa0e01526e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0dffff700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0e00b9640 0x7fa0e00bbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0deffd700 1 -- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0e0066b70 msgr2=0x7fa0e01526e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0deffd700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0e0066b70 0x7fa0e01526e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0deffd700 1 -- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0e00b9640 msgr2=0x7fa0e00bbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0deffd700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0e00b9640 0x7fa0e00bbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.528+0000 7fa0deffd700 1 -- 172.21.15.116:0/4162932435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e0162c90 con 0x7fa0e0066ff0 2020-11-20T12:16:37.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.532+0000 7fa0deffd700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e0066ff0 0x7fa0e00b9150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fa0d400ebd0 tx=0x7fa0d4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:37.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.532+0000 7fa0bf7fe700 1 -- 172.21.15.116:0/4162932435 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d400ca60 con 0x7fa0e0066ff0 2020-11-20T12:16:37.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.532+0000 7fa0e5c72700 1 -- 172.21.15.116:0/4162932435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e0162f30 con 0x7fa0e0066ff0 2020-11-20T12:16:37.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.532+0000 7fa0bf7fe700 1 -- 172.21.15.116:0/4162932435 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0d400cbc0 con 0x7fa0e0066ff0 2020-11-20T12:16:37.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.532+0000 7fa0bf7fe700 1 -- 172.21.15.116:0/4162932435 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d40123f0 con 0x7fa0e0066ff0 2020-11-20T12:16:37.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.532+0000 7fa0e5c72700 1 -- 172.21.15.116:0/4162932435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e0163470 con 0x7fa0e0066ff0 2020-11-20T12:16:37.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.532+0000 7fa0e5c72700 1 -- 172.21.15.116:0/4162932435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e005ca30 con 0x7fa0e0066ff0 2020-11-20T12:16:37.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.532+0000 7fa0bf7fe700 1 -- 172.21.15.116:0/4162932435 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa0d4012760 con 0x7fa0e0066ff0 2020-11-20T12:16:37.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.536+0000 7fa0bf7fe700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0b80557c0 0x7fa0b8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.536+0000 7fa0bf7fe700 1 -- 172.21.15.116:0/4162932435 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa0d406eb40 con 0x7fa0e0066ff0 2020-11-20T12:16:37.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.536+0000 7fa0df7fe700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0b80557c0 0x7fa0b8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.536+0000 7fa0df7fe700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0b80557c0 0x7fa0b8057c80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fa0e00c4150 tx=0x7fa0d0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:37.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.540+0000 7fa0bf7fe700 1 -- 172.21.15.116:0/4162932435 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa0d40449e0 con 0x7fa0e0066ff0 2020-11-20T12:16:37.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.700+0000 7fa0e5c72700 1 -- 172.21.15.116:0/4162932435 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa0e0061e80 con 0x7fa0e0066ff0 2020-11-20T12:16:37.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0bf7fe700 1 -- 172.21.15.116:0/4162932435 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa0d4048000 con 0x7fa0e0066ff0 2020-11-20T12:16:37.704 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:37.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 -- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0b80557c0 msgr2=0x7fa0b8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0b80557c0 0x7fa0b8057c80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fa0e00c4150 tx=0x7fa0d0015000).stop 2020-11-20T12:16:37.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 -- 172.21.15.116:0/4162932435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e0066ff0 msgr2=0x7fa0e00b9150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e0066ff0 0x7fa0e00b9150 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7fa0d400ebd0 tx=0x7fa0d4006df0).stop 2020-11-20T12:16:37.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 -- 172.21.15.116:0/4162932435 shutdown_connections 2020-11-20T12:16:37.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0b80557c0 0x7fa0b8057c80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0e00b9640 0x7fa0e00bbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0e0066b70 0x7fa0e01526e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 --2- 172.21.15.116:0/4162932435 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0e0066ff0 0x7fa0e00b9150 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 -- 172.21.15.116:0/4162932435 >> 172.21.15.116:0/4162932435 conn(0x7fa0e0060170 msgr2=0x7fa0e00ba830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:37.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 -- 172.21.15.116:0/4162932435 shutdown_connections 2020-11-20T12:16:37.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.704+0000 7fa0e5c72700 1 -- 172.21.15.116:0/4162932435 wait complete. 2020-11-20T12:16:37.730 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:16:37.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.912+0000 7fdf3dab5700 1 Processor -- start 2020-11-20T12:16:37.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.912+0000 7fdf3dab5700 1 -- start start 2020-11-20T12:16:37.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.912+0000 7fdf3dab5700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdf3813b110 0x7fdf38151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.912+0000 7fdf3dab5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf38151fa0 0x7fdf38061250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.912+0000 7fdf3dab5700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf38152a80 con 0x7fdf3813ad30 2020-11-20T12:16:37.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.912+0000 7fdf3dab5700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf38152540 con 0x7fdf3813b110 2020-11-20T12:16:37.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.912+0000 7fdf3dab5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf381527e0 con 0x7fdf38151fa0 2020-11-20T12:16:37.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3cab3700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdf3813ad30 0x7fdf380a7fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53418/0 (socket says 172.21.15.116:53418) 2020-11-20T12:16:37.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3cab3700 1 -- 172.21.15.116:0/491519368 learned_addr learned my addr 172.21.15.116:0/491519368 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:37.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf2ffff700 1 --2- 172.21.15.116:0/491519368 >> v2:172.21.15.173:3300/0 conn(0x7fdf3813b110 0x7fdf38151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3d2b4700 1 --2- 172.21.15.116:0/491519368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf38151fa0 0x7fdf38061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf2f7fe700 1 -- 172.21.15.116:0/491519368 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1269184717 0 0) 0x7fdf38152a80 con 0x7fdf3813ad30 2020-11-20T12:16:37.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf2f7fe700 1 -- 172.21.15.116:0/491519368 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf24003240 con 0x7fdf3813ad30 2020-11-20T12:16:37.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3d2b4700 1 -- 172.21.15.116:0/491519368 >> v2:172.21.15.173:3300/0 conn(0x7fdf3813b110 msgr2=0x7fdf38151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:37.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3d2b4700 1 --2- 172.21.15.116:0/491519368 >> v2:172.21.15.173:3300/0 conn(0x7fdf3813b110 0x7fdf38151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3d2b4700 1 -- 172.21.15.116:0/491519368 >> v1:172.21.15.116:6789/0 conn(0x7fdf3813ad30 legacy=0x7fdf380a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3d2b4700 1 -- 172.21.15.116:0/491519368 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380ac910 con 0x7fdf38151fa0 2020-11-20T12:16:37.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf2ffff700 1 --2- 172.21.15.116:0/491519368 >> v2:172.21.15.173:3300/0 conn(0x7fdf3813b110 0x7fdf38151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:37.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3d2b4700 1 --2- 172.21.15.116:0/491519368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf38151fa0 0x7fdf38061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fdf3000b9a0 tx=0x7fdf30006df0).ready entity=mon.2 client_cookie=fce21058c7badf4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:37.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf2f7fe700 1 -- 172.21.15.116:0/491519368 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30009710 con 0x7fdf38151fa0 2020-11-20T12:16:37.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf2f7fe700 1 -- 172.21.15.116:0/491519368 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf30009870 con 0x7fdf38151fa0 2020-11-20T12:16:37.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf2f7fe700 1 -- 172.21.15.116:0/491519368 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30009bf0 con 0x7fdf38151fa0 2020-11-20T12:16:37.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3dab5700 1 -- 172.21.15.116:0/491519368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf38151fa0 msgr2=0x7fdf38061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3dab5700 1 --2- 172.21.15.116:0/491519368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf38151fa0 0x7fdf38061250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fdf3000b9a0 tx=0x7fdf30006df0).stop 2020-11-20T12:16:37.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3dab5700 1 -- 172.21.15.116:0/491519368 shutdown_connections 2020-11-20T12:16:37.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3dab5700 1 --2- 172.21.15.116:0/491519368 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf38151fa0 0x7fdf38061250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3dab5700 1 --2- 172.21.15.116:0/491519368 >> v2:172.21.15.173:3300/0 conn(0x7fdf3813b110 0x7fdf38151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3dab5700 1 -- 172.21.15.116:0/491519368 >> 172.21.15.116:0/491519368 conn(0x7fdf380678a0 msgr2=0x7fdf38067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:37.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3dab5700 1 -- 172.21.15.116:0/491519368 shutdown_connections 2020-11-20T12:16:37.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.916+0000 7fdf3dab5700 1 -- 172.21.15.116:0/491519368 wait complete. 2020-11-20T12:16:37.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.920+0000 7fdf3dab5700 1 Processor -- start 2020-11-20T12:16:37.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.920+0000 7fdf3dab5700 1 -- start start 2020-11-20T12:16:37.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.920+0000 7fdf3dab5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3813ad30 0x7fdf38154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.920+0000 7fdf3dab5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf3813b110 0x7fdf38151070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.920+0000 7fdf3dab5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf38151fa0 0x7fdf380a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.920+0000 7fdf3dab5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38151560 con 0x7fdf38151fa0 2020-11-20T12:16:37.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.920+0000 7fdf3dab5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf380a15f0 con 0x7fdf3813b110 2020-11-20T12:16:37.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.920+0000 7fdf3dab5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3809f190 con 0x7fdf3813ad30 2020-11-20T12:16:37.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3d2b4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf38151fa0 0x7fdf380a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3cab3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3813ad30 0x7fdf38154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3d2b4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf38151fa0 0x7fdf380a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:43994/0 (socket says 172.21.15.116:43994) 2020-11-20T12:16:37.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3d2b4700 1 -- 172.21.15.116:0/1336360201 learned_addr learned my addr 172.21.15.116:0/1336360201 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:37.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf2ffff700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf3813b110 0x7fdf38151070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3d2b4700 1 -- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3813ad30 msgr2=0x7fdf38154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3d2b4700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3813ad30 0x7fdf38154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3d2b4700 1 -- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf3813b110 msgr2=0x7fdf38151070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:37.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3d2b4700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf3813b110 0x7fdf38151070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:37.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3d2b4700 1 -- 172.21.15.116:0/1336360201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf3809f430 con 0x7fdf38151fa0 2020-11-20T12:16:37.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3d2b4700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf38151fa0 0x7fdf380a0b90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fdf3000c010 tx=0x7fdf30015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:37.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf2cff9700 1 -- 172.21.15.116:0/1336360201 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3001e460 con 0x7fdf38151fa0 2020-11-20T12:16:37.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf2cff9700 1 -- 172.21.15.116:0/1336360201 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf3001e5c0 con 0x7fdf38151fa0 2020-11-20T12:16:37.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3dab5700 1 -- 172.21.15.116:0/1336360201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf3809f6d0 con 0x7fdf38151fa0 2020-11-20T12:16:37.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf2cff9700 1 -- 172.21.15.116:0/1336360201 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3001e940 con 0x7fdf38151fa0 2020-11-20T12:16:37.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3dab5700 1 -- 172.21.15.116:0/1336360201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf380a1080 con 0x7fdf38151fa0 2020-11-20T12:16:37.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.924+0000 7fdf3dab5700 1 -- 172.21.15.116:0/1336360201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf3809f860 con 0x7fdf38151fa0 2020-11-20T12:16:37.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.928+0000 7fdf2cff9700 1 -- 172.21.15.116:0/1336360201 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fdf30025030 con 0x7fdf38151fa0 2020-11-20T12:16:37.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.928+0000 7fdf2cff9700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf100557c0 0x7fdf10057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:37.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.928+0000 7fdf2cff9700 1 -- 172.21.15.116:0/1336360201 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdf3006fc60 con 0x7fdf38151fa0 2020-11-20T12:16:37.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.928+0000 7fdf3cab3700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf100557c0 0x7fdf10057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:37.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.928+0000 7fdf3cab3700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf100557c0 0x7fdf10057c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fdf38156d90 tx=0x7fdf34009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:37.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:37.936+0000 7fdf2cff9700 1 -- 172.21.15.116:0/1336360201 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fdf30045bb0 con 0x7fdf38151fa0 2020-11-20T12:16:38.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf3dab5700 1 -- 172.21.15.116:0/1336360201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdf38044f60 con 0x7fdf38151fa0 2020-11-20T12:16:38.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf2cff9700 1 -- 172.21.15.116:0/1336360201 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7fdf30023020 con 0x7fdf38151fa0 2020-11-20T12:16:38.096 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:38.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf3dab5700 1 -- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf100557c0 msgr2=0x7fdf10057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf3dab5700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf100557c0 0x7fdf10057c80 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fdf38156d90 tx=0x7fdf34009040).stop 2020-11-20T12:16:38.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf3dab5700 1 -- 172.21.15.116:0/1336360201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf38151fa0 msgr2=0x7fdf380a0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf3dab5700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf38151fa0 0x7fdf380a0b90 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fdf3000c010 tx=0x7fdf30015000).stop 2020-11-20T12:16:38.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf3dab5700 1 -- 172.21.15.116:0/1336360201 shutdown_connections 2020-11-20T12:16:38.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf3dab5700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf100557c0 0x7fdf10057c80 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf3dab5700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf38151fa0 0x7fdf380a0b90 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf3dab5700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf3813ad30 0x7fdf38154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf3dab5700 1 --2- 172.21.15.116:0/1336360201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf3813b110 0x7fdf38151070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.096+0000 7fdf3dab5700 1 -- 172.21.15.116:0/1336360201 >> 172.21.15.116:0/1336360201 conn(0x7fdf380678a0 msgr2=0x7fdf38138dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:38.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.100+0000 7fdf3dab5700 1 -- 172.21.15.116:0/1336360201 shutdown_connections 2020-11-20T12:16:38.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.100+0000 7fdf3dab5700 1 -- 172.21.15.116:0/1336360201 wait complete. 2020-11-20T12:16:38.124 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:16:38.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.308+0000 7f790329e700 1 Processor -- start 2020-11-20T12:16:38.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.308+0000 7f790329e700 1 -- start start 2020-11-20T12:16:38.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f790329e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78fc0bd780 0x7f78fc0a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f790329e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f78fc14f530 0x7f78fc0a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f790329e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78fc155620 con 0x7f78fc14f150 2020-11-20T12:16:38.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f790329e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78fc1558c0 con 0x7f78fc14f530 2020-11-20T12:16:38.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f790329e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc155b60 con 0x7f78fc0bd780 2020-11-20T12:16:38.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f790183b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f78fc14f530 0x7f78fc0a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f790103a700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f78fc14f150 0x7f78fc0c87f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53432/0 (socket says 172.21.15.116:53432) 2020-11-20T12:16:38.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f790103a700 1 -- 172.21.15.116:0/1493434358 learned_addr learned my addr 172.21.15.116:0/1493434358 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:38.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f7900839700 1 --2- 172.21.15.116:0/1493434358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78fc0bd780 0x7f78fc0a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f78ebfff700 1 -- 172.21.15.116:0/1493434358 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 419411139 0 0) 0x7f78fc155620 con 0x7f78fc14f150 2020-11-20T12:16:38.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f78ebfff700 1 -- 172.21.15.116:0/1493434358 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f78e4003240 con 0x7f78fc14f150 2020-11-20T12:16:38.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f7900839700 1 -- 172.21.15.116:0/1493434358 >> v2:172.21.15.116:3300/0 conn(0x7f78fc14f530 msgr2=0x7f78fc0a79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:38.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f7900839700 1 --2- 172.21.15.116:0/1493434358 >> v2:172.21.15.116:3300/0 conn(0x7f78fc14f530 0x7f78fc0a79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f7900839700 1 -- 172.21.15.116:0/1493434358 >> v1:172.21.15.116:6789/0 conn(0x7f78fc14f150 legacy=0x7f78fc0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.312+0000 7f7900839700 1 -- 172.21.15.116:0/1493434358 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc155e00 con 0x7f78fc0bd780 2020-11-20T12:16:38.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f7900839700 1 --2- 172.21.15.116:0/1493434358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78fc0bd780 0x7f78fc0a74b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f78ec00b9a0 tx=0x7f78ec006df0).ready entity=mon.2 client_cookie=4d6da8bdff0d56f1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:38.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f78ebfff700 1 -- 172.21.15.116:0/1493434358 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec009680 con 0x7f78fc0bd780 2020-11-20T12:16:38.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f78ebfff700 1 -- 172.21.15.116:0/1493434358 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ec0097e0 con 0x7f78fc0bd780 2020-11-20T12:16:38.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f78ebfff700 1 -- 172.21.15.116:0/1493434358 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ec009b60 con 0x7f78fc0bd780 2020-11-20T12:16:38.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f790329e700 1 -- 172.21.15.116:0/1493434358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78fc0bd780 msgr2=0x7f78fc0a74b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f790329e700 1 --2- 172.21.15.116:0/1493434358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78fc0bd780 0x7f78fc0a74b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f78ec00b9a0 tx=0x7f78ec006df0).stop 2020-11-20T12:16:38.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f790329e700 1 -- 172.21.15.116:0/1493434358 shutdown_connections 2020-11-20T12:16:38.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f790329e700 1 --2- 172.21.15.116:0/1493434358 >> v2:172.21.15.116:3300/0 conn(0x7f78fc14f530 0x7f78fc0a79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f790329e700 1 --2- 172.21.15.116:0/1493434358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78fc0bd780 0x7f78fc0a74b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f790329e700 1 -- 172.21.15.116:0/1493434358 >> 172.21.15.116:0/1493434358 conn(0x7f78fc061580 msgr2=0x7f78fc061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:38.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f790329e700 1 -- 172.21.15.116:0/1493434358 shutdown_connections 2020-11-20T12:16:38.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f790329e700 1 -- 172.21.15.116:0/1493434358 wait complete. 2020-11-20T12:16:38.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.316+0000 7f790329e700 1 Processor -- start 2020-11-20T12:16:38.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790329e700 1 -- start start 2020-11-20T12:16:38.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790329e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78fc0bd780 0x7f78fc0c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790329e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78fc14f150 0x7f78fc0ada50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790329e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78fc14f530 0x7f78fc14b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790329e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc0c32a0 con 0x7f78fc0bd780 2020-11-20T12:16:38.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790329e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc1612c0 con 0x7f78fc14f530 2020-11-20T12:16:38.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790329e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78fc161560 con 0x7f78fc14f150 2020-11-20T12:16:38.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790103a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78fc0bd780 0x7f78fc0c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790103a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78fc0bd780 0x7f78fc0c2d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44004/0 (socket says 172.21.15.116:44004) 2020-11-20T12:16:38.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790103a700 1 -- 172.21.15.116:0/2119365864 learned_addr learned my addr 172.21.15.116:0/2119365864 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:38.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f7900839700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78fc14f150 0x7f78fc0ada50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790183b700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78fc14f530 0x7f78fc14b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790103a700 1 -- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78fc14f150 msgr2=0x7f78fc0ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790103a700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78fc14f150 0x7f78fc0ada50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790103a700 1 -- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78fc14f530 msgr2=0x7f78fc14b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790103a700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78fc14f530 0x7f78fc14b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790103a700 1 -- 172.21.15.116:0/2119365864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78fc161800 con 0x7f78fc0bd780 2020-11-20T12:16:38.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790103a700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78fc0bd780 0x7f78fc0c2d00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f78f800c6a0 tx=0x7f78f8007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:38.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f78e97fa700 1 -- 172.21.15.116:0/2119365864 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f80072e0 con 0x7f78fc0bd780 2020-11-20T12:16:38.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f78e97fa700 1 -- 172.21.15.116:0/2119365864 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78f8007440 con 0x7f78fc0bd780 2020-11-20T12:16:38.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f78e97fa700 1 -- 172.21.15.116:0/2119365864 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78f8005820 con 0x7f78fc0bd780 2020-11-20T12:16:38.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790329e700 1 -- 172.21.15.116:0/2119365864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78fc0adf40 con 0x7f78fc0bd780 2020-11-20T12:16:38.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.320+0000 7f790329e700 1 -- 172.21.15.116:0/2119365864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78fc14b9f0 con 0x7f78fc0bd780 2020-11-20T12:16:38.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.324+0000 7f790329e700 1 -- 172.21.15.116:0/2119365864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78fc161100 con 0x7f78fc0bd780 2020-11-20T12:16:38.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.324+0000 7f78e97fa700 1 -- 172.21.15.116:0/2119365864 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f78f8010070 con 0x7f78fc0bd780 2020-11-20T12:16:38.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.328+0000 7f78e97fa700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78d40557c0 0x7f78d4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.328+0000 7f78e97fa700 1 -- 172.21.15.116:0/2119365864 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f78f806dda0 con 0x7f78fc0bd780 2020-11-20T12:16:38.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.328+0000 7f7900839700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78d40557c0 0x7f78d4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.328+0000 7f7900839700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78d40557c0 0x7f78d4057c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f78fc0bdb80 tx=0x7f78ec015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:38.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.332+0000 7f78e97fa700 1 -- 172.21.15.116:0/2119365864 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f78f8043c40 con 0x7f78fc0bd780 2020-11-20T12:16:38.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.492+0000 7f790329e700 1 -- 172.21.15.116:0/2119365864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78fc05ca30 con 0x7f78fc0bd780 2020-11-20T12:16:38.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.492+0000 7f78e97fa700 1 -- 172.21.15.116:0/2119365864 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f78f8047260 con 0x7f78fc0bd780 2020-11-20T12:16:38.493 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:38.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.492+0000 7f790329e700 1 -- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78d40557c0 msgr2=0x7f78d4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.492+0000 7f790329e700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78d40557c0 0x7f78d4057c80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f78fc0bdb80 tx=0x7f78ec015000).stop 2020-11-20T12:16:38.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.496+0000 7f790329e700 1 -- 172.21.15.116:0/2119365864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78fc0bd780 msgr2=0x7f78fc0c2d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.496+0000 7f790329e700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78fc0bd780 0x7f78fc0c2d00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f78f800c6a0 tx=0x7f78f8007ac0).stop 2020-11-20T12:16:38.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.496+0000 7f790329e700 1 -- 172.21.15.116:0/2119365864 shutdown_connections 2020-11-20T12:16:38.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.496+0000 7f790329e700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78d40557c0 0x7f78d4057c80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.496+0000 7f790329e700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78fc14f530 0x7f78fc14b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.496+0000 7f790329e700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f78fc0bd780 0x7f78fc0c2d00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.496+0000 7f790329e700 1 --2- 172.21.15.116:0/2119365864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78fc14f150 0x7f78fc0ada50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.496+0000 7f790329e700 1 -- 172.21.15.116:0/2119365864 >> 172.21.15.116:0/2119365864 conn(0x7f78fc061580 msgr2=0x7f78fc0c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:38.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.496+0000 7f790329e700 1 -- 172.21.15.116:0/2119365864 shutdown_connections 2020-11-20T12:16:38.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.496+0000 7f790329e700 1 -- 172.21.15.116:0/2119365864 wait complete. 2020-11-20T12:16:38.519 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:38.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58621e9700 1 Processor -- start 2020-11-20T12:16:38.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58621e9700 1 -- start start 2020-11-20T12:16:38.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58621e9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c13aee0 0x7f585c152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58621e9700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f585c0aa790 0x7f585c0ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58621e9700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f585c152a30 con 0x7f585c0612f0 2020-11-20T12:16:38.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58621e9700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f585c0aac10 con 0x7f585c0aa790 2020-11-20T12:16:38.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58621e9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0aaeb0 con 0x7f585c13aee0 2020-11-20T12:16:38.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58609e6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f585c0aa790 0x7f585c0ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58611e7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c13aee0 0x7f585c152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58619e8700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f585c0612f0 0x7f585c0a7ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49678/0 (socket says 172.21.15.116:49678) 2020-11-20T12:16:38.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58611e7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c13aee0 0x7f585c152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:52964/0 (socket says 172.21.15.116:52964) 2020-11-20T12:16:38.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58609e6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f585c0aa790 0x7f585c0ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:43064/0 (socket says 172.21.15.116:43064) 2020-11-20T12:16:38.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.688+0000 7f58611e7700 1 -- 172.21.15.116:0/138799830 learned_addr learned my addr 172.21.15.116:0/138799830 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:38.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58609e6700 1 -- 172.21.15.116:0/138799830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c13aee0 msgr2=0x7f585c152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:38.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58609e6700 1 --2- 172.21.15.116:0/138799830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c13aee0 0x7f585c152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58609e6700 1 -- 172.21.15.116:0/138799830 >> v1:172.21.15.173:6789/0 conn(0x7f585c0612f0 legacy=0x7f585c0a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58609e6700 1 -- 172.21.15.116:0/138799830 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c0acbf0 con 0x7f585c0aa790 2020-11-20T12:16:38.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58611e7700 1 --2- 172.21.15.116:0/138799830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c13aee0 0x7f585c152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:38.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58609e6700 1 --2- 172.21.15.116:0/138799830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f585c0aa790 0x7f585c0ad5d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f584c00b9a0 tx=0x7f584c006df0).ready entity=mon.1 client_cookie=1f028035be551312 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:38.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f584bfff700 1 -- 172.21.15.116:0/138799830 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c0097b0 con 0x7f585c0aa790 2020-11-20T12:16:38.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f584bfff700 1 -- 172.21.15.116:0/138799830 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f584c009910 con 0x7f585c0aa790 2020-11-20T12:16:38.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f584bfff700 1 -- 172.21.15.116:0/138799830 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c009c90 con 0x7f585c0aa790 2020-11-20T12:16:38.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58621e9700 1 -- 172.21.15.116:0/138799830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f585c0aa790 msgr2=0x7f585c0ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58621e9700 1 --2- 172.21.15.116:0/138799830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f585c0aa790 0x7f585c0ad5d0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f584c00b9a0 tx=0x7f584c006df0).stop 2020-11-20T12:16:38.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58621e9700 1 -- 172.21.15.116:0/138799830 shutdown_connections 2020-11-20T12:16:38.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58621e9700 1 --2- 172.21.15.116:0/138799830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c13aee0 0x7f585c152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58621e9700 1 --2- 172.21.15.116:0/138799830 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f585c0aa790 0x7f585c0ad5d0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58621e9700 1 -- 172.21.15.116:0/138799830 >> 172.21.15.116:0/138799830 conn(0x7f585c0678a0 msgr2=0x7f585c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:38.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58621e9700 1 -- 172.21.15.116:0/138799830 shutdown_connections 2020-11-20T12:16:38.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.692+0000 7f58621e9700 1 -- 172.21.15.116:0/138799830 wait complete. 2020-11-20T12:16:38.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.696+0000 7f58621e9700 1 Processor -- start 2020-11-20T12:16:38.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.696+0000 7f58621e9700 1 -- start start 2020-11-20T12:16:38.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.696+0000 7f58621e9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c0612f0 0x7f585c150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.696+0000 7f58621e9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f585c0aa790 0x7f585c0658a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.696+0000 7f58621e9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f585c13aee0 0x7f585c063f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.696+0000 7f58621e9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585c066990 con 0x7f585c13aee0 2020-11-20T12:16:38.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.696+0000 7f58621e9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585c0661b0 con 0x7f585c0aa790 2020-11-20T12:16:38.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.696+0000 7f58621e9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585c066450 con 0x7f585c0612f0 2020-11-20T12:16:38.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58619e8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f585c13aee0 0x7f585c063f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58619e8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f585c13aee0 0x7f585c063f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44022/0 (socket says 172.21.15.116:44022) 2020-11-20T12:16:38.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58619e8700 1 -- 172.21.15.116:0/1473047731 learned_addr learned my addr 172.21.15.116:0/1473047731 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:38.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58609e6700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f585c0aa790 0x7f585c0658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58611e7700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c0612f0 0x7f585c150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58619e8700 1 -- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c0612f0 msgr2=0x7f585c150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58619e8700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c0612f0 0x7f585c150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58619e8700 1 -- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f585c0aa790 msgr2=0x7f585c0658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58619e8700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f585c0aa790 0x7f585c0658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58619e8700 1 -- 172.21.15.116:0/1473047731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c0666f0 con 0x7f585c13aee0 2020-11-20T12:16:38.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58619e8700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f585c13aee0 0x7f585c063f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f585400ca60 tx=0x7f5854007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:38.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58497fa700 1 -- 172.21.15.116:0/1473047731 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854005ac0 con 0x7f585c13aee0 2020-11-20T12:16:38.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58497fa700 1 -- 172.21.15.116:0/1473047731 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5854005c20 con 0x7f585c13aee0 2020-11-20T12:16:38.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58621e9700 1 -- 172.21.15.116:0/1473047731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585c156060 con 0x7f585c13aee0 2020-11-20T12:16:38.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58497fa700 1 -- 172.21.15.116:0/1473047731 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5854007110 con 0x7f585c13aee0 2020-11-20T12:16:38.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58621e9700 1 -- 172.21.15.116:0/1473047731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585c151250 con 0x7f585c13aee0 2020-11-20T12:16:38.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.700+0000 7f58621e9700 1 -- 172.21.15.116:0/1473047731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585c1561f0 con 0x7f585c13aee0 2020-11-20T12:16:38.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.704+0000 7f58497fa700 1 -- 172.21.15.116:0/1473047731 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5854010030 con 0x7f585c13aee0 2020-11-20T12:16:38.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.704+0000 7f58497fa700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58340557c0 0x7f5834057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:38.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.704+0000 7f58497fa700 1 -- 172.21.15.116:0/1473047731 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f585406e640 con 0x7f585c13aee0 2020-11-20T12:16:38.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.704+0000 7f58611e7700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58340557c0 0x7f5834057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:38.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.704+0000 7f58611e7700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58340557c0 0x7f5834057c80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f5858006fd0 tx=0x7f5858008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:38.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.708+0000 7f58497fa700 1 -- 172.21.15.116:0/1473047731 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f58540444e0 con 0x7f585c13aee0 2020-11-20T12:16:38.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.872+0000 7f58621e9700 1 -- 172.21.15.116:0/1473047731 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f585c044f60 con 0x7f585c13aee0 2020-11-20T12:16:38.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.872+0000 7f58497fa700 1 -- 172.21.15.116:0/1473047731 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f5854047b00 con 0x7f585c13aee0 2020-11-20T12:16:38.873 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:38.874 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:38.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.872+0000 7f58621e9700 1 -- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58340557c0 msgr2=0x7f5834057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.872+0000 7f58621e9700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58340557c0 0x7f5834057c80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f5858006fd0 tx=0x7f5858008040).stop 2020-11-20T12:16:38.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.872+0000 7f58621e9700 1 -- 172.21.15.116:0/1473047731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f585c13aee0 msgr2=0x7f585c063f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:38.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.872+0000 7f58621e9700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f585c13aee0 0x7f585c063f70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f585400ca60 tx=0x7f5854007f00).stop 2020-11-20T12:16:38.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.876+0000 7f58621e9700 1 -- 172.21.15.116:0/1473047731 shutdown_connections 2020-11-20T12:16:38.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.876+0000 7f58621e9700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58340557c0 0x7f5834057c80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.876+0000 7f58621e9700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f585c13aee0 0x7f585c063f70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.876+0000 7f58621e9700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f585c0612f0 0x7f585c150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.876+0000 7f58621e9700 1 --2- 172.21.15.116:0/1473047731 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f585c0aa790 0x7f585c0658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:38.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.876+0000 7f58621e9700 1 -- 172.21.15.116:0/1473047731 >> 172.21.15.116:0/1473047731 conn(0x7f585c0678a0 msgr2=0x7f585c069690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:38.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.876+0000 7f58621e9700 1 -- 172.21.15.116:0/1473047731 shutdown_connections 2020-11-20T12:16:38.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:38.876+0000 7f58621e9700 1 -- 172.21.15.116:0/1473047731 wait complete. 2020-11-20T12:16:38.898 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:15:59.065285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:15:59.602311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:16:38.899 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:39.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.076+0000 7fd4fe499700 1 Processor -- start 2020-11-20T12:16:39.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.076+0000 7fd4fe499700 1 -- start start 2020-11-20T12:16:39.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.080+0000 7fd4fe499700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd4f8063a40 0x7fd4f8063e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.080+0000 7fd4fe499700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f80baec0 con 0x7fd4f8155eb0 2020-11-20T12:16:39.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.080+0000 7fd4fe499700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f81554d0 con 0x7fd4f8151280 2020-11-20T12:16:39.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.080+0000 7fd4fe499700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f8156310 con 0x7fd4f8063a40 2020-11-20T12:16:39.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.080+0000 7fd4effff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd4f8155eb0 0x7fd4f80ba460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53460/0 (socket says 172.21.15.116:53460) 2020-11-20T12:16:39.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.080+0000 7fd4effff700 1 -- 172.21.15.116:0/2697865583 learned_addr learned my addr 172.21.15.116:0/2697865583 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:39.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.080+0000 7fd4f7fff700 1 --2- 172.21.15.116:0/2697865583 >> v2:172.21.15.173:3300/0 conn(0x7fd4f8063a40 0x7fd4f8063e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.080+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3548918333 0 0) 0x7fd4f80baec0 con 0x7fd4f8155eb0 2020-11-20T12:16:39.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.080+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4d8003240 con 0x7fd4f8155eb0 2020-11-20T12:16:39.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3513358894 0 0) 0x7fd4d8003240 con 0x7fd4f8155eb0 2020-11-20T12:16:39.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 >> v2:172.21.15.173:3300/0 conn(0x7fd4f8063a40 msgr2=0x7fd4f8063e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:39.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 --2- 172.21.15.116:0/2697865583 >> v2:172.21.15.173:3300/0 conn(0x7fd4f8063a40 0x7fd4f8063e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 >> v1:172.21.15.173:6789/0 conn(0x7fd4f8151280 legacy=0x7fd4f80bf9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8155770 con 0x7fd4f8155eb0 2020-11-20T12:16:39.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fd4e00014e0 con 0x7fd4f8155eb0 2020-11-20T12:16:39.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 >> v1:172.21.15.116:6789/0 conn(0x7fd4f8155eb0 legacy=0x7fd4f80ba460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 --2- 172.21.15.116:0/2697865583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d8003800 0x7fd4d8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 --2- 172.21.15.116:0/2697865583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4d80044e0 0x7fd4d8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 --2- 172.21.15.116:0/2697865583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4d8004e50 0x7fd4d800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d8005300 con 0x7fd4d8004e50 2020-11-20T12:16:39.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d800bc20 con 0x7fd4d8003800 2020-11-20T12:16:39.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d800bd60 con 0x7fd4d80044e0 2020-11-20T12:16:39.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f7fff700 1 --2- 172.21.15.116:0/2697865583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d8003800 0x7fd4d8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4fca36700 1 --2- 172.21.15.116:0/2697865583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4d8004e50 0x7fd4d800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4effff700 1 --2- 172.21.15.116:0/2697865583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4d80044e0 0x7fd4d8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f7fff700 1 -- 172.21.15.116:0/2697865583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4d80044e0 msgr2=0x7fd4d8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:39.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f7fff700 1 --2- 172.21.15.116:0/2697865583 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4d80044e0 0x7fd4d8004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f7fff700 1 -- 172.21.15.116:0/2697865583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4d8004e50 msgr2=0x7fd4d800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:39.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f7fff700 1 --2- 172.21.15.116:0/2697865583 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4d8004e50 0x7fd4d800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f7fff700 1 -- 172.21.15.116:0/2697865583 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d8003240 con 0x7fd4d8003800 2020-11-20T12:16:39.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4fca36700 1 -- 172.21.15.116:0/2697865583 reap_dead start 2020-11-20T12:16:39.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f7fff700 1 --2- 172.21.15.116:0/2697865583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d8003800 0x7fd4d8003c20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fd4e800bc30 tx=0x7fd4e8006df0).ready entity=mon.1 client_cookie=bfb0b9b958226df0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:39.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800b310 con 0x7fd4d8003800 2020-11-20T12:16:39.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e800ea90 con 0x7fd4d8003800 2020-11-20T12:16:39.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.084+0000 7fd4f77fe700 1 -- 172.21.15.116:0/2697865583 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800ee10 con 0x7fd4d8003800 2020-11-20T12:16:39.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.088+0000 7fd4fe499700 1 -- 172.21.15.116:0/2697865583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d8003800 msgr2=0x7fd4d8003c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.088+0000 7fd4fe499700 1 --2- 172.21.15.116:0/2697865583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d8003800 0x7fd4d8003c20 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fd4e800bc30 tx=0x7fd4e8006df0).stop 2020-11-20T12:16:39.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.088+0000 7fd4fe499700 1 -- 172.21.15.116:0/2697865583 shutdown_connections 2020-11-20T12:16:39.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.088+0000 7fd4fe499700 1 --2- 172.21.15.116:0/2697865583 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d8003800 0x7fd4d8003c20 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.088+0000 7fd4fe499700 1 -- 172.21.15.116:0/2697865583 >> 172.21.15.116:0/2697865583 conn(0x7fd4f8061d50 msgr2=0x7fd4f80c1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:39.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.088+0000 7fd4fe499700 1 -- 172.21.15.116:0/2697865583 shutdown_connections 2020-11-20T12:16:39.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.088+0000 7fd4fe499700 1 -- 172.21.15.116:0/2697865583 wait complete. 2020-11-20T12:16:39.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.088+0000 7fd4fe499700 1 Processor -- start 2020-11-20T12:16:39.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.088+0000 7fd4fe499700 1 -- start start 2020-11-20T12:16:39.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4fe499700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d8003800 0x7fd4f8164d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4fe499700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4f81548d0 0x7fd4f80aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4fe499700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4f8149d90 0x7fd4f80c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4fe499700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f81550f0 con 0x7fd4f81548d0 2020-11-20T12:16:39.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4fe499700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f80a8e20 con 0x7fd4d8003800 2020-11-20T12:16:39.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4fe499700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f8154e50 con 0x7fd4f8149d90 2020-11-20T12:16:39.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4effff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4f81548d0 0x7fd4f80aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4effff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4f81548d0 0x7fd4f80aaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44040/0 (socket says 172.21.15.116:44040) 2020-11-20T12:16:39.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4effff700 1 -- 172.21.15.116:0/3849082910 learned_addr learned my addr 172.21.15.116:0/3849082910 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:39.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4f7fff700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d8003800 0x7fd4f8164d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4fca36700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4f8149d90 0x7fd4f80c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4effff700 1 -- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4f8149d90 msgr2=0x7fd4f80c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4effff700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4f8149d90 0x7fd4f80c44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4effff700 1 -- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d8003800 msgr2=0x7fd4f8164d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4effff700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d8003800 0x7fd4f8164d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4effff700 1 -- 172.21.15.116:0/3849082910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f8164390 con 0x7fd4f81548d0 2020-11-20T12:16:39.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4fca36700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4f8149d90 0x7fd4f80c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4effff700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4f81548d0 0x7fd4f80aaff0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fd4e000eae0 tx=0x7fd4f8155660).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4f4ff9700 1 -- 172.21.15.116:0/3849082910 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000c810 con 0x7fd4f81548d0 2020-11-20T12:16:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4f4ff9700 1 -- 172.21.15.116:0/3849082910 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e000c970 con 0x7fd4f81548d0 2020-11-20T12:16:39.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4f4ff9700 1 -- 172.21.15.116:0/3849082910 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000cd10 con 0x7fd4f81548d0 2020-11-20T12:16:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4fe499700 1 -- 172.21.15.116:0/3849082910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f8164630 con 0x7fd4f81548d0 2020-11-20T12:16:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.092+0000 7fd4fe499700 1 -- 172.21.15.116:0/3849082910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f814a210 con 0x7fd4f81548d0 2020-11-20T12:16:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.096+0000 7fd4fe499700 1 -- 172.21.15.116:0/3849082910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f805ca30 con 0x7fd4f81548d0 2020-11-20T12:16:39.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.096+0000 7fd4f4ff9700 1 -- 172.21.15.116:0/3849082910 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd4e0014640 con 0x7fd4f81548d0 2020-11-20T12:16:39.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.096+0000 7fd4f4ff9700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4d40557c0 0x7fd4d4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.096+0000 7fd4f4ff9700 1 -- 172.21.15.116:0/3849082910 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd4e006e2c0 con 0x7fd4f81548d0 2020-11-20T12:16:39.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.096+0000 7fd4f7fff700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4d40557c0 0x7fd4d4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.100+0000 7fd4f7fff700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4d40557c0 0x7fd4d4057c80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd4e8006e30 tx=0x7fd4e8019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:39.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.104+0000 7fd4f4ff9700 1 -- 172.21.15.116:0/3849082910 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd4e0044050 con 0x7fd4f81548d0 2020-11-20T12:16:39.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.260+0000 7fd4fe499700 1 -- 172.21.15.116:0/3849082910 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd4f8062290 con 0x7fd4f81548d0 2020-11-20T12:16:39.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4f4ff9700 1 -- 172.21.15.116:0/3849082910 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd4e0047670 con 0x7fd4f81548d0 2020-11-20T12:16:39.263 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:39.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 -- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4d40557c0 msgr2=0x7fd4d4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4d40557c0 0x7fd4d4057c80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fd4e8006e30 tx=0x7fd4e8019000).stop 2020-11-20T12:16:39.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 -- 172.21.15.116:0/3849082910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4f81548d0 msgr2=0x7fd4f80aaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4f81548d0 0x7fd4f80aaff0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fd4e000eae0 tx=0x7fd4f8155660).stop 2020-11-20T12:16:39.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 -- 172.21.15.116:0/3849082910 shutdown_connections 2020-11-20T12:16:39.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4d40557c0 0x7fd4d4057c80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4f8149d90 0x7fd4f80c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4d8003800 0x7fd4f8164d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 --2- 172.21.15.116:0/3849082910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4f81548d0 0x7fd4f80aaff0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 -- 172.21.15.116:0/3849082910 >> 172.21.15.116:0/3849082910 conn(0x7fd4f8061d50 msgr2=0x7fd4f80c5250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:39.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 -- 172.21.15.116:0/3849082910 shutdown_connections 2020-11-20T12:16:39.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.264+0000 7fd4fe499700 1 -- 172.21.15.116:0/3849082910 wait complete. 2020-11-20T12:16:39.289 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:39.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.468+0000 7f2c9d3bb700 1 Processor -- start 2020-11-20T12:16:39.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.468+0000 7f2c9d3bb700 1 -- start start 2020-11-20T12:16:39.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.468+0000 7f2c9d3bb700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2c980c1bb0 0x7f2c98156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.468+0000 7f2c9d3bb700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2c980a28c0 0x7f2c98066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.468+0000 7f2c9d3bb700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c980a38c0 con 0x7f2c980c03f0 2020-11-20T12:16:39.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.468+0000 7f2c9d3bb700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c98156e30 con 0x7f2c980a28c0 2020-11-20T12:16:39.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.468+0000 7f2c9d3bb700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c980a2e40 con 0x7f2c980c1bb0 2020-11-20T12:16:39.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c97fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2c980c03f0 0x7f2c980677c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53480/0 (socket says 172.21.15.116:53480) 2020-11-20T12:16:39.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c97fff700 1 -- 172.21.15.116:0/4229514525 learned_addr learned my addr 172.21.15.116:0/4229514525 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:39.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9cbba700 1 --2- 172.21.15.116:0/4229514525 >> v2:172.21.15.116:3300/0 conn(0x7f2c980a28c0 0x7f2c98066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c977fe700 1 --2- 172.21.15.116:0/4229514525 >> v2:172.21.15.173:3300/0 conn(0x7f2c980c1bb0 0x7f2c98156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c96ffd700 1 -- 172.21.15.116:0/4229514525 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987752995 0 0) 0x7f2c980a38c0 con 0x7f2c980c03f0 2020-11-20T12:16:39.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c96ffd700 1 -- 172.21.15.116:0/4229514525 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c84003240 con 0x7f2c980c03f0 2020-11-20T12:16:39.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9cbba700 1 -- 172.21.15.116:0/4229514525 >> v2:172.21.15.173:3300/0 conn(0x7f2c980c1bb0 msgr2=0x7f2c98156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:39.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9cbba700 1 --2- 172.21.15.116:0/4229514525 >> v2:172.21.15.173:3300/0 conn(0x7f2c980c1bb0 0x7f2c98156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9cbba700 1 -- 172.21.15.116:0/4229514525 >> v1:172.21.15.116:6789/0 conn(0x7f2c980c03f0 legacy=0x7f2c980677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9cbba700 1 -- 172.21.15.116:0/4229514525 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980a30e0 con 0x7f2c980a28c0 2020-11-20T12:16:39.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9cbba700 1 --2- 172.21.15.116:0/4229514525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c980a28c0 0x7f2c98066570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f2c980c3f80 tx=0x7f2c900091d0).ready entity=mon.0 client_cookie=266ced27d8d86ea1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:39.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c96ffd700 1 -- 172.21.15.116:0/4229514525 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9000ba20 con 0x7f2c980a28c0 2020-11-20T12:16:39.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c96ffd700 1 -- 172.21.15.116:0/4229514525 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9000bb80 con 0x7f2c980a28c0 2020-11-20T12:16:39.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c96ffd700 1 -- 172.21.15.116:0/4229514525 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c90011400 con 0x7f2c980a28c0 2020-11-20T12:16:39.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/4229514525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c980a28c0 msgr2=0x7f2c98066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9d3bb700 1 --2- 172.21.15.116:0/4229514525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c980a28c0 0x7f2c98066570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f2c980c3f80 tx=0x7f2c900091d0).stop 2020-11-20T12:16:39.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/4229514525 shutdown_connections 2020-11-20T12:16:39.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9d3bb700 1 --2- 172.21.15.116:0/4229514525 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c980a28c0 0x7f2c98066570 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9d3bb700 1 --2- 172.21.15.116:0/4229514525 >> v2:172.21.15.173:3300/0 conn(0x7f2c980c1bb0 0x7f2c98156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/4229514525 >> 172.21.15.116:0/4229514525 conn(0x7f2c98060c30 msgr2=0x7f2c980c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:39.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/4229514525 shutdown_connections 2020-11-20T12:16:39.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.472+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/4229514525 wait complete. 2020-11-20T12:16:39.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.476+0000 7f2c9d3bb700 1 Processor -- start 2020-11-20T12:16:39.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.476+0000 7f2c9d3bb700 1 -- start start 2020-11-20T12:16:39.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.476+0000 7f2c9d3bb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c980a28c0 0x7f2c98151a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.476+0000 7f2c9d3bb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c980c03f0 0x7f2c98062570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.476+0000 7f2c9d3bb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c980c1bb0 0x7f2c98153d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.476+0000 7f2c9d3bb700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c980a7200 con 0x7f2c980a28c0 2020-11-20T12:16:39.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.476+0000 7f2c9d3bb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c98063470 con 0x7f2c980c1bb0 2020-11-20T12:16:39.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.476+0000 7f2c9d3bb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c98062f30 con 0x7f2c980c03f0 2020-11-20T12:16:39.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.476+0000 7f2c97fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c980a28c0 0x7f2c98151a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c977fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c980c03f0 0x7f2c98062570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c9cbba700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c980c1bb0 0x7f2c98153d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c977fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c980c03f0 0x7f2c98062570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:53002/0 (socket says 172.21.15.116:53002) 2020-11-20T12:16:39.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c977fe700 1 -- 172.21.15.116:0/3179545874 learned_addr learned my addr 172.21.15.116:0/3179545874 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:39.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c97fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c980a28c0 0x7f2c98151a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44056/0 (socket says 172.21.15.116:44056) 2020-11-20T12:16:39.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c977fe700 1 -- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c980c1bb0 msgr2=0x7f2c98153d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c977fe700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c980c1bb0 0x7f2c98153d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c977fe700 1 -- 172.21.15.116:0/3179545874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c980a28c0 msgr2=0x7f2c98151a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c977fe700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c980a28c0 0x7f2c98151a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c977fe700 1 -- 172.21.15.116:0/3179545874 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c980631d0 con 0x7f2c980c03f0 2020-11-20T12:16:39.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c97fff700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c980a28c0 0x7f2c98151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:39.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c977fe700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c980c03f0 0x7f2c98062570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2c8c00ec00 tx=0x7f2c8c006df0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:39.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c77fff700 1 -- 172.21.15.116:0/3179545874 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00b8d0 con 0x7f2c980c03f0 2020-11-20T12:16:39.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c77fff700 1 -- 172.21.15.116:0/3179545874 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c8c00ba30 con 0x7f2c980c03f0 2020-11-20T12:16:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c77fff700 1 -- 172.21.15.116:0/3179545874 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c8c00bdb0 con 0x7f2c980c03f0 2020-11-20T12:16:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/3179545874 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c980a73f0 con 0x7f2c980c03f0 2020-11-20T12:16:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.480+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/3179545874 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c98151fc0 con 0x7f2c980c03f0 2020-11-20T12:16:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.484+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/3179545874 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c98044f60 con 0x7f2c980c03f0 2020-11-20T12:16:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.484+0000 7f2c77fff700 1 -- 172.21.15.116:0/3179545874 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2c8c033750 con 0x7f2c980c03f0 2020-11-20T12:16:39.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.484+0000 7f2c77fff700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c700557c0 0x7f2c70057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:39.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.484+0000 7f2c77fff700 1 -- 172.21.15.116:0/3179545874 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2c8c06ec80 con 0x7f2c980c03f0 2020-11-20T12:16:39.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.484+0000 7f2c97fff700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c700557c0 0x7f2c70057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:39.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.488+0000 7f2c97fff700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c700557c0 0x7f2c70057c80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2c880055a0 tx=0x7f2c88009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:39.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.492+0000 7f2c77fff700 1 -- 172.21.15.116:0/3179545874 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2c8c044a90 con 0x7f2c980c03f0 2020-11-20T12:16:39.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.648+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/3179545874 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2c98064080 con 0x7f2c980c03f0 2020-11-20T12:16:39.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.652+0000 7f2c77fff700 1 -- 172.21.15.116:0/3179545874 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2c8c0480b0 con 0x7f2c980c03f0 2020-11-20T12:16:39.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.652+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c700557c0 msgr2=0x7f2c70057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.652+0000 7f2c9d3bb700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c700557c0 0x7f2c70057c80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2c880055a0 tx=0x7f2c88009040).stop 2020-11-20T12:16:39.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.652+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c980c03f0 msgr2=0x7f2c98062570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.652+0000 7f2c9d3bb700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c980c03f0 0x7f2c98062570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f2c8c00ec00 tx=0x7f2c8c006df0).stop 2020-11-20T12:16:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.652+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/3179545874 shutdown_connections 2020-11-20T12:16:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.652+0000 7f2c9d3bb700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c700557c0 0x7f2c70057c80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.652+0000 7f2c9d3bb700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c980c1bb0 0x7f2c98153d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.652+0000 7f2c9d3bb700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c980a28c0 0x7f2c98151a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.652+0000 7f2c9d3bb700 1 --2- 172.21.15.116:0/3179545874 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c980c03f0 0x7f2c98062570 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.652+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/3179545874 >> 172.21.15.116:0/3179545874 conn(0x7f2c98060c30 msgr2=0x7f2c980c2050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.656+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/3179545874 shutdown_connections 2020-11-20T12:16:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:39.656+0000 7f2c9d3bb700 1 -- 172.21.15.116:0/3179545874 wait complete. 2020-11-20T12:16:39.935 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:39.935 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:16:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.120+0000 7ffaa9d79700 1 Processor -- start 2020-11-20T12:16:40.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.120+0000 7ffaa9d79700 1 -- start start 2020-11-20T12:16:40.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa9d79700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 0x7ffaa4152470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa9d79700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa4066f20 con 0x7ffaa414df80 2020-11-20T12:16:40.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa9d79700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaa4151a90 con 0x7ffaa414dba0 2020-11-20T12:16:40.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa9d79700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa4151d30 con 0x7ffaa4066960 2020-11-20T12:16:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa2ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ffaa414df80 0x7ffaa4066250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53496/0 (socket says 172.21.15.116:53496) 2020-11-20T12:16:40.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa2ffd700 1 -- 172.21.15.116:0/1737667818 learned_addr learned my addr 172.21.15.116:0/1737667818 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa3fff700 1 --2- 172.21.15.116:0/1737667818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 0x7ffaa4152470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa27fc700 1 -- 172.21.15.116:0/1737667818 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2313174996 0 0) 0x7ffaa4066f20 con 0x7ffaa414df80 2020-11-20T12:16:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa27fc700 1 -- 172.21.15.116:0/1737667818 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa90003240 con 0x7ffaa414df80 2020-11-20T12:16:40.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa27fc700 1 -- 172.21.15.116:0/1737667818 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749624104 0 0) 0x7ffaa4151a90 con 0x7ffaa414dba0 2020-11-20T12:16:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa27fc700 1 -- 172.21.15.116:0/1737667818 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffaa4066f20 con 0x7ffaa414dba0 2020-11-20T12:16:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa3fff700 1 -- 172.21.15.116:0/1737667818 >> v1:172.21.15.173:6789/0 conn(0x7ffaa414dba0 legacy=0x7ffaa40bf370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa3fff700 1 -- 172.21.15.116:0/1737667818 >> v1:172.21.15.116:6789/0 conn(0x7ffaa414df80 legacy=0x7ffaa4066250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa27fc700 1 -- 172.21.15.116:0/1737667818 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1937499717 0 0) 0x7ffa90003240 con 0x7ffaa414df80 2020-11-20T12:16:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa3fff700 1 -- 172.21.15.116:0/1737667818 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa4151fd0 con 0x7ffaa4066960 2020-11-20T12:16:40.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa3fff700 1 --2- 172.21.15.116:0/1737667818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 0x7ffaa4152470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ffa9c009cc0 tx=0x7ffa9c0091d0).ready entity=mon.2 client_cookie=de3a7240a55fbe3b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:40.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa27fc700 1 -- 172.21.15.116:0/1737667818 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa980014e0 con 0x7ffaa4066960 2020-11-20T12:16:40.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa27fc700 1 -- 172.21.15.116:0/1737667818 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa9c00b7a0 con 0x7ffaa4066960 2020-11-20T12:16:40.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.124+0000 7ffaa27fc700 1 -- 172.21.15.116:0/1737667818 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa9c00bb20 con 0x7ffaa4066960 2020-11-20T12:16:40.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.128+0000 7ffaa9d79700 1 -- 172.21.15.116:0/1737667818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 msgr2=0x7ffaa4152470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.128+0000 7ffaa9d79700 1 --2- 172.21.15.116:0/1737667818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 0x7ffaa4152470 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7ffa9c009cc0 tx=0x7ffa9c0091d0).stop 2020-11-20T12:16:40.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.128+0000 7ffaa9d79700 1 -- 172.21.15.116:0/1737667818 shutdown_connections 2020-11-20T12:16:40.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.128+0000 7ffaa9d79700 1 --2- 172.21.15.116:0/1737667818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 0x7ffaa4152470 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.128+0000 7ffaa9d79700 1 -- 172.21.15.116:0/1737667818 >> 172.21.15.116:0/1737667818 conn(0x7ffaa4060170 msgr2=0x7ffaa4060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:40.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.128+0000 7ffaa9d79700 1 -- 172.21.15.116:0/1737667818 shutdown_connections 2020-11-20T12:16:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.128+0000 7ffaa9d79700 1 -- 172.21.15.116:0/1737667818 wait complete. 2020-11-20T12:16:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.128+0000 7ffaa9d79700 1 Processor -- start 2020-11-20T12:16:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa9d79700 1 -- start start 2020-11-20T12:16:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa9d79700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 0x7ffaa414c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa9d79700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa414dba0 0x7ffaa409db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa9d79700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaa414df80 0x7ffaa409dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa9d79700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa4162a00 con 0x7ffaa414dba0 2020-11-20T12:16:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa9d79700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa4161b20 con 0x7ffaa414df80 2020-11-20T12:16:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa9d79700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaa4161d80 con 0x7ffaa4066960 2020-11-20T12:16:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa414dba0 0x7ffaa409db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa414dba0 0x7ffaa409db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44070/0 (socket says 172.21.15.116:44070) 2020-11-20T12:16:40.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa2ffd700 1 -- 172.21.15.116:0/2525809751 learned_addr learned my addr 172.21.15.116:0/2525809751 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:40.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa37fe700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 0x7ffaa414c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa3fff700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaa414df80 0x7ffaa409dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa2ffd700 1 -- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 msgr2=0x7ffaa414c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa2ffd700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 0x7ffaa414c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa2ffd700 1 -- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaa414df80 msgr2=0x7ffaa409dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa2ffd700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaa414df80 0x7ffaa409dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa2ffd700 1 -- 172.21.15.116:0/2525809751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaa4162020 con 0x7ffaa414dba0 2020-11-20T12:16:40.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa3fff700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaa414df80 0x7ffaa409dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:40.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa37fe700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 0x7ffaa414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:40.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa2ffd700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa414dba0 0x7ffaa409db00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7ffa9800cae0 tx=0x7ffa980019a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:40.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffa837fe700 1 -- 172.21.15.116:0/2525809751 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98007320 con 0x7ffaa414dba0 2020-11-20T12:16:40.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffa837fe700 1 -- 172.21.15.116:0/2525809751 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa98007480 con 0x7ffaa414dba0 2020-11-20T12:16:40.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffa837fe700 1 -- 172.21.15.116:0/2525809751 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa98005800 con 0x7ffaa414dba0 2020-11-20T12:16:40.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa9d79700 1 -- 172.21.15.116:0/2525809751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaa41622c0 con 0x7ffaa414dba0 2020-11-20T12:16:40.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.132+0000 7ffaa9d79700 1 -- 172.21.15.116:0/2525809751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaa409d610 con 0x7ffaa414dba0 2020-11-20T12:16:40.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.136+0000 7ffaa9d79700 1 -- 172.21.15.116:0/2525809751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaa4162760 con 0x7ffaa414dba0 2020-11-20T12:16:40.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.136+0000 7ffa837fe700 1 -- 172.21.15.116:0/2525809751 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ffa98010030 con 0x7ffaa414dba0 2020-11-20T12:16:40.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.136+0000 7ffa837fe700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa7c0557c0 0x7ffa7c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.136+0000 7ffa837fe700 1 -- 172.21.15.116:0/2525809751 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffa9806e6c0 con 0x7ffaa414dba0 2020-11-20T12:16:40.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.136+0000 7ffaa37fe700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa7c0557c0 0x7ffa7c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.140+0000 7ffaa37fe700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa7c0557c0 0x7ffa7c057c80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7ffaa40aa4a0 tx=0x7ffa94009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:40.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.144+0000 7ffa837fe700 1 -- 172.21.15.116:0/2525809751 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ffaa4162760 con 0x7ffaa414dba0 2020-11-20T12:16:40.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.304+0000 7ffaa9d79700 1 -- 172.21.15.116:0/2525809751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffaa4162760 con 0x7ffaa414dba0 2020-11-20T12:16:40.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.304+0000 7ffa837fe700 1 -- 172.21.15.116:0/2525809751 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7ffaa4162760 con 0x7ffaa414dba0 2020-11-20T12:16:40.306 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:40.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 -- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa7c0557c0 msgr2=0x7ffa7c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa7c0557c0 0x7ffa7c057c80 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7ffaa40aa4a0 tx=0x7ffa94009040).stop 2020-11-20T12:16:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 -- 172.21.15.116:0/2525809751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa414dba0 msgr2=0x7ffaa409db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa414dba0 0x7ffaa409db00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7ffa9800cae0 tx=0x7ffa980019a0).stop 2020-11-20T12:16:40.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 -- 172.21.15.116:0/2525809751 shutdown_connections 2020-11-20T12:16:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa7c0557c0 0x7ffa7c057c80 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffaa414df80 0x7ffaa409dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffaa4066960 0x7ffaa414c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 --2- 172.21.15.116:0/2525809751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffaa414dba0 0x7ffaa409db00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 -- 172.21.15.116:0/2525809751 >> 172.21.15.116:0/2525809751 conn(0x7ffaa4060170 msgr2=0x7ffaa40bef30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:40.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 -- 172.21.15.116:0/2525809751 shutdown_connections 2020-11-20T12:16:40.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.308+0000 7ffaa9d79700 1 -- 172.21.15.116:0/2525809751 wait complete. 2020-11-20T12:16:40.333 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:16:40.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.544+0000 7fc27206f700 1 Processor -- start 2020-11-20T12:16:40.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.544+0000 7fc27206f700 1 -- start start 2020-11-20T12:16:40.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27206f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc26c13ad30 0x7fc26c0a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27206f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc26c13b110 0x7fc26c151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27206f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26c152a80 con 0x7fc26c151fa0 2020-11-20T12:16:40.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27206f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc26c152540 con 0x7fc26c13b110 2020-11-20T12:16:40.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27206f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c1527e0 con 0x7fc26c13ad30 2020-11-20T12:16:40.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27086c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc26c13b110 0x7fc26c151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27086c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc26c13b110 0x7fc26c151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44078/0 (socket says 172.21.15.116:44078) 2020-11-20T12:16:40.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27086c700 1 -- 172.21.15.116:0/3116561222 learned_addr learned my addr 172.21.15.116:0/3116561222 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:40.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27106d700 1 --2- 172.21.15.116:0/3116561222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc26c13ad30 0x7fc26c0a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27086c700 1 -- 172.21.15.116:0/3116561222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc26c13ad30 msgr2=0x7fc26c0a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:40.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27086c700 1 --2- 172.21.15.116:0/3116561222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc26c13ad30 0x7fc26c0a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27086c700 1 -- 172.21.15.116:0/3116561222 >> v1:172.21.15.173:6789/0 conn(0x7fc26c151fa0 legacy=0x7fc26c061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27086c700 1 -- 172.21.15.116:0/3116561222 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c0ac910 con 0x7fc26c13b110 2020-11-20T12:16:40.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.548+0000 7fc27086c700 1 --2- 172.21.15.116:0/3116561222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc26c13b110 0x7fc26c151ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fc25c006fd0 tx=0x7fc25c006eb0).ready entity=mon.0 client_cookie=cd9afc8fbe0dd35e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:40.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.552+0000 7fc25bfff700 1 -- 172.21.15.116:0/3116561222 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c0098a0 con 0x7fc26c13b110 2020-11-20T12:16:40.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.552+0000 7fc25bfff700 1 -- 172.21.15.116:0/3116561222 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c009a00 con 0x7fc26c13b110 2020-11-20T12:16:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.552+0000 7fc25bfff700 1 -- 172.21.15.116:0/3116561222 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c009d80 con 0x7fc26c13b110 2020-11-20T12:16:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.552+0000 7fc27206f700 1 -- 172.21.15.116:0/3116561222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc26c13b110 msgr2=0x7fc26c151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.552+0000 7fc27206f700 1 --2- 172.21.15.116:0/3116561222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc26c13b110 0x7fc26c151ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fc25c006fd0 tx=0x7fc25c006eb0).stop 2020-11-20T12:16:40.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.552+0000 7fc27206f700 1 -- 172.21.15.116:0/3116561222 shutdown_connections 2020-11-20T12:16:40.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.552+0000 7fc27206f700 1 --2- 172.21.15.116:0/3116561222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc26c13ad30 0x7fc26c0a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.552+0000 7fc27206f700 1 --2- 172.21.15.116:0/3116561222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc26c13b110 0x7fc26c151ab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.552+0000 7fc27206f700 1 -- 172.21.15.116:0/3116561222 >> 172.21.15.116:0/3116561222 conn(0x7fc26c0678a0 msgr2=0x7fc26c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.552+0000 7fc27206f700 1 -- 172.21.15.116:0/3116561222 shutdown_connections 2020-11-20T12:16:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.552+0000 7fc27206f700 1 -- 172.21.15.116:0/3116561222 wait complete. 2020-11-20T12:16:40.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27206f700 1 Processor -- start 2020-11-20T12:16:40.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27206f700 1 -- start start 2020-11-20T12:16:40.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27206f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc26c13ad30 0x7fc26c13a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27206f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc26c13b110 0x7fc26c0a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27206f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc26c151fa0 0x7fc26c0a8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27206f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c13a580 con 0x7fc26c13b110 2020-11-20T12:16:40.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27206f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c09f440 con 0x7fc26c151fa0 2020-11-20T12:16:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27206f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26c09f6e0 con 0x7fc26c13ad30 2020-11-20T12:16:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27086c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc26c13b110 0x7fc26c0a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27086c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc26c13b110 0x7fc26c0a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44080/0 (socket says 172.21.15.116:44080) 2020-11-20T12:16:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27086c700 1 -- 172.21.15.116:0/2148813418 learned_addr learned my addr 172.21.15.116:0/2148813418 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:40.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27086c700 1 -- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc26c13ad30 msgr2=0x7fc26c13a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27106d700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc26c13ad30 0x7fc26c13a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27186e700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc26c151fa0 0x7fc26c0a8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27086c700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc26c13ad30 0x7fc26c13a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27086c700 1 -- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc26c151fa0 msgr2=0x7fc26c0a8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27086c700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc26c151fa0 0x7fc26c0a8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27086c700 1 -- 172.21.15.116:0/2148813418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26c09f980 con 0x7fc26c13b110 2020-11-20T12:16:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27186e700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc26c151fa0 0x7fc26c0a8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:40.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.556+0000 7fc27106d700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc26c13ad30 0x7fc26c13a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.560+0000 7fc27086c700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc26c13b110 0x7fc26c0a6860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fc25c006ef0 tx=0x7fc25c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.560+0000 7fc2597fa700 1 -- 172.21.15.116:0/2148813418 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c011e90 con 0x7fc26c13b110 2020-11-20T12:16:40.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.560+0000 7fc2597fa700 1 -- 172.21.15.116:0/2148813418 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc25c01e460 con 0x7fc26c13b110 2020-11-20T12:16:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.560+0000 7fc2597fa700 1 -- 172.21.15.116:0/2148813418 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c01e7e0 con 0x7fc26c13b110 2020-11-20T12:16:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.560+0000 7fc27206f700 1 -- 172.21.15.116:0/2148813418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26c09fc20 con 0x7fc26c13b110 2020-11-20T12:16:40.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.560+0000 7fc27206f700 1 -- 172.21.15.116:0/2148813418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc26c09f010 con 0x7fc26c13b110 2020-11-20T12:16:40.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.560+0000 7fc27206f700 1 -- 172.21.15.116:0/2148813418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26c044f60 con 0x7fc26c13b110 2020-11-20T12:16:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.560+0000 7fc2597fa700 1 -- 172.21.15.116:0/2148813418 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc25c01a040 con 0x7fc26c13b110 2020-11-20T12:16:40.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.564+0000 7fc2597fa700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2440557c0 0x7fc244057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.564+0000 7fc2597fa700 1 -- 172.21.15.116:0/2148813418 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc25c06fb50 con 0x7fc26c13b110 2020-11-20T12:16:40.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.564+0000 7fc27106d700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2440557c0 0x7fc244057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.564+0000 7fc27106d700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2440557c0 0x7fc244057c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fc268006bd0 tx=0x7fc268008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:40.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.568+0000 7fc2597fa700 1 -- 172.21.15.116:0/2148813418 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc25c0459f0 con 0x7fc26c13b110 2020-11-20T12:16:40.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 -- 172.21.15.116:0/2148813418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc26c069320 con 0x7fc26c13b110 2020-11-20T12:16:40.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc2597fa700 1 -- 172.21.15.116:0/2148813418 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc25c023030 con 0x7fc26c13b110 2020-11-20T12:16:40.732 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:40.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 -- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2440557c0 msgr2=0x7fc244057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2440557c0 0x7fc244057c80 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fc268006bd0 tx=0x7fc268008040).stop 2020-11-20T12:16:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 -- 172.21.15.116:0/2148813418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc26c13b110 msgr2=0x7fc26c0a6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc26c13b110 0x7fc26c0a6860 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fc25c006ef0 tx=0x7fc25c015000).stop 2020-11-20T12:16:40.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 -- 172.21.15.116:0/2148813418 shutdown_connections 2020-11-20T12:16:40.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2440557c0 0x7fc244057c80 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc26c151fa0 0x7fc26c0a8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc26c13ad30 0x7fc26c13a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 --2- 172.21.15.116:0/2148813418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc26c13b110 0x7fc26c0a6860 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 -- 172.21.15.116:0/2148813418 >> 172.21.15.116:0/2148813418 conn(0x7fc26c0678a0 msgr2=0x7fc26c0c5740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:40.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 -- 172.21.15.116:0/2148813418 shutdown_connections 2020-11-20T12:16:40.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.732+0000 7fc27206f700 1 -- 172.21.15.116:0/2148813418 wait complete. 2020-11-20T12:16:40.781 INFO:tasks.cephfs_test_runner:Starting test: test_15303 (tasks.cephfs.test_volume_client.TestVolumeClient) 2020-11-20T12:16:40.782 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_15303' 2020-11-20T12:16:40.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.964+0000 7f84ba273700 1 Processor -- start 2020-11-20T12:16:40.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.968+0000 7f84ba273700 1 -- start start 2020-11-20T12:16:40.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.968+0000 7f84ba273700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f84b4151d70 0x7f84b41521e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.968+0000 7f84ba273700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b40a3bd0 con 0x7f84b40a4bc0 2020-11-20T12:16:40.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.968+0000 7f84ba273700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b40a3e70 con 0x7f84b40668a0 2020-11-20T12:16:40.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.968+0000 7f84ba273700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b40a5040 con 0x7f84b4151d70 2020-11-20T12:16:40.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b8a70700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f84b40a4bc0 0x7f84b40c3fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53524/0 (socket says 172.21.15.116:53524) 2020-11-20T12:16:40.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b8a70700 1 -- 172.21.15.116:0/1227326126 learned_addr learned my addr 172.21.15.116:0/1227326126 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:40.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3115229385 0 0) 0x7f84b40a3bd0 con 0x7f84b40a4bc0 2020-11-20T12:16:40.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f849c003240 con 0x7f84b40a4bc0 2020-11-20T12:16:40.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b9a72700 1 --2- 172.21.15.116:0/1227326126 >> v2:172.21.15.116:3300/0 conn(0x7f84b4151d70 0x7f84b41521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 555877714 0 0) 0x7f849c003240 con 0x7f84b40a4bc0 2020-11-20T12:16:40.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 >> v2:172.21.15.116:3300/0 conn(0x7f84b4151d70 msgr2=0x7f84b41521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:40.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 --2- 172.21.15.116:0/1227326126 >> v2:172.21.15.116:3300/0 conn(0x7f84b4151d70 0x7f84b41521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 >> v1:172.21.15.173:6789/0 conn(0x7f84b40668a0 legacy=0x7f84b40a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40a4110 con 0x7f84b40a4bc0 2020-11-20T12:16:40.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f84a40014e0 con 0x7f84b40a4bc0 2020-11-20T12:16:40.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b9a72700 1 --2- 172.21.15.116:0/1227326126 >> v2:172.21.15.116:3300/0 conn(0x7f84b4151d70 0x7f84b41521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:40.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 >> v1:172.21.15.116:6789/0 conn(0x7f84b40a4bc0 legacy=0x7f84b40c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f849c003730 0x7f849c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f849c0044e0 0x7f849c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f849c004e50 0x7f849c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c004260 con 0x7f849c003730 2020-11-20T12:16:40.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849c00bc20 con 0x7f849c004e50 2020-11-20T12:16:40.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f849c00bd60 con 0x7f849c0044e0 2020-11-20T12:16:40.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b9271700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f849c003730 0x7f849c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b8a70700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f849c0044e0 0x7f849c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b9a72700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f849c004e50 0x7f849c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b9271700 1 -- 172.21.15.116:0/1227326126 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f849c0044e0 msgr2=0x7f849c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:40.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b9271700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f849c0044e0 0x7f849c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b9271700 1 -- 172.21.15.116:0/1227326126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f849c004e50 msgr2=0x7f849c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:40.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b9271700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f849c004e50 0x7f849c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b9271700 1 -- 172.21.15.116:0/1227326126 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f849c003240 con 0x7f849c003730 2020-11-20T12:16:40.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b9a72700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f849c004e50 0x7f849c00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:40.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.972+0000 7f84b9a72700 1 -- 172.21.15.116:0/1227326126 reap_dead start 2020-11-20T12:16:40.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84b9271700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f849c003730 0x7f849c003b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f84b0001920 tx=0x7f84b0007b90).ready entity=mon.0 client_cookie=1adee500d39e1281 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:40.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0007020 con 0x7f849c003730 2020-11-20T12:16:40.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b0007180 con 0x7f849c003730 2020-11-20T12:16:40.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84a3fff700 1 -- 172.21.15.116:0/1227326126 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000c830 con 0x7f849c003730 2020-11-20T12:16:40.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84ba273700 1 -- 172.21.15.116:0/1227326126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f849c003730 msgr2=0x7f849c003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84ba273700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f849c003730 0x7f849c003b50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f84b0001920 tx=0x7f84b0007b90).stop 2020-11-20T12:16:40.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84ba273700 1 -- 172.21.15.116:0/1227326126 shutdown_connections 2020-11-20T12:16:40.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84ba273700 1 --2- 172.21.15.116:0/1227326126 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f849c003730 0x7f849c003b50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84ba273700 1 -- 172.21.15.116:0/1227326126 >> 172.21.15.116:0/1227326126 conn(0x7f84b4062520 msgr2=0x7f84b4062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:40.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84ba273700 1 -- 172.21.15.116:0/1227326126 shutdown_connections 2020-11-20T12:16:40.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84ba273700 1 -- 172.21.15.116:0/1227326126 wait complete. 2020-11-20T12:16:40.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84ba273700 1 Processor -- start 2020-11-20T12:16:40.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.976+0000 7f84ba273700 1 -- start start 2020-11-20T12:16:40.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84ba273700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b414deb0 0x7f84b4153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84ba273700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b41534a0 0x7f84b40a8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84ba273700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b40a8c60 0x7f84b41521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84ba273700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b40a90e0 con 0x7f84b414deb0 2020-11-20T12:16:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84ba273700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b41542a0 con 0x7f84b40a8c60 2020-11-20T12:16:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84ba273700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b4156830 con 0x7f84b41534a0 2020-11-20T12:16:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84b9271700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b414deb0 0x7f84b4153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84b9271700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b414deb0 0x7f84b4153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44100/0 (socket says 172.21.15.116:44100) 2020-11-20T12:16:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84b9271700 1 -- 172.21.15.116:0/780398358 learned_addr learned my addr 172.21.15.116:0/780398358 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84b8a70700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b41534a0 0x7f84b40a8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84b9a72700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b40a8c60 0x7f84b41521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84b9271700 1 -- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b41534a0 msgr2=0x7f84b40a8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84b9271700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b41534a0 0x7f84b40a8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84b9271700 1 -- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b40a8c60 msgr2=0x7f84b41521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84b9271700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b40a8c60 0x7f84b41521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84b9271700 1 -- 172.21.15.116:0/780398358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b41544c0 con 0x7f84b414deb0 2020-11-20T12:16:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84b9271700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b414deb0 0x7f84b4153db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f84b0010000 tx=0x7f84b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84a17fa700 1 -- 172.21.15.116:0/780398358 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b000c3c0 con 0x7f84b414deb0 2020-11-20T12:16:40.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84a17fa700 1 -- 172.21.15.116:0/780398358 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84b00075a0 con 0x7f84b414deb0 2020-11-20T12:16:40.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84ba273700 1 -- 172.21.15.116:0/780398358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b4154760 con 0x7f84b414deb0 2020-11-20T12:16:40.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84a17fa700 1 -- 172.21.15.116:0/780398358 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b0013e00 con 0x7f84b414deb0 2020-11-20T12:16:40.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.980+0000 7f84ba273700 1 -- 172.21.15.116:0/780398358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b4154ca0 con 0x7f84b414deb0 2020-11-20T12:16:40.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.984+0000 7f84ba273700 1 -- 172.21.15.116:0/780398358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b4044f60 con 0x7f84b414deb0 2020-11-20T12:16:40.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.984+0000 7f84a17fa700 1 -- 172.21.15.116:0/780398358 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f84b0010030 con 0x7f84b414deb0 2020-11-20T12:16:40.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.984+0000 7f84a17fa700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f848c0557c0 0x7f848c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:40.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.984+0000 7f84a17fa700 1 -- 172.21.15.116:0/780398358 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f84b006fb00 con 0x7f84b414deb0 2020-11-20T12:16:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.984+0000 7f84b8a70700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f848c0557c0 0x7f848c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.988+0000 7f84b8a70700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f848c0557c0 0x7f848c057c80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f84a4000bc0 tx=0x7f84b40680f0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:40.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:40.988+0000 7f84a17fa700 1 -- 172.21.15.116:0/780398358 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f84b00459a0 con 0x7f84b414deb0 2020-11-20T12:16:41.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.204+0000 7f84ba273700 1 -- 172.21.15.116:0/780398358 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_15303"]} v 0) v1 -- 0x7f84b4061580 con 0x7f84b414deb0 2020-11-20T12:16:41.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.260+0000 7f84a17fa700 1 -- 172.21.15.116:0/780398358 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_15303"]}]=0 v54) v1 ==== 141+0+0 (secure 0 0 0) 0x7f84b0048fc0 con 0x7f84b414deb0 2020-11-20T12:16:41.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 -- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f848c0557c0 msgr2=0x7f848c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:41.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f848c0557c0 0x7f848c057c80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f84a4000bc0 tx=0x7f84b40680f0).stop 2020-11-20T12:16:41.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 -- 172.21.15.116:0/780398358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b414deb0 msgr2=0x7f84b4153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:41.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b414deb0 0x7f84b4153db0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f84b0010000 tx=0x7f84b0015000).stop 2020-11-20T12:16:41.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 -- 172.21.15.116:0/780398358 shutdown_connections 2020-11-20T12:16:41.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f848c0557c0 0x7f848c057c80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b40a8c60 0x7f84b41521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b414deb0 0x7f84b4153db0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 --2- 172.21.15.116:0/780398358 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b41534a0 0x7f84b40a8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 -- 172.21.15.116:0/780398358 >> 172.21.15.116:0/780398358 conn(0x7f84b4062520 msgr2=0x7f84b40a5000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:41.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 -- 172.21.15.116:0/780398358 shutdown_connections 2020-11-20T12:16:41.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.264+0000 7f84ba273700 1 -- 172.21.15.116:0/780398358 wait complete. 2020-11-20T12:16:41.290 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-20T12:16:41.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.488+0000 7f2b68c1c700 1 Processor -- start 2020-11-20T12:16:41.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.488+0000 7f2b68c1c700 1 -- start start 2020-11-20T12:16:41.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.488+0000 7f2b68c1c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2b6414f150 0x7f2b640c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:41.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.488+0000 7f2b68c1c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2b640bd780 0x7f2b640c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:41.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b68c1c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b64155620 con 0x7f2b6414f530 2020-11-20T12:16:41.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b68c1c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b641558c0 con 0x7f2b640bd780 2020-11-20T12:16:41.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b68c1c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b64155b60 con 0x7f2b6414f150 2020-11-20T12:16:41.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b62d9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2b6414f530 0x7f2b640a74b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53544/0 (socket says 172.21.15.116:53544) 2020-11-20T12:16:41.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b62d9d700 1 -- 172.21.15.116:0/3087406067 learned_addr learned my addr 172.21.15.116:0/3087406067 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:41.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b6259c700 1 --2- 172.21.15.116:0/3087406067 >> v2:172.21.15.173:3300/0 conn(0x7f2b6414f150 0x7f2b640c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:41.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b61d9b700 1 --2- 172.21.15.116:0/3087406067 >> v2:172.21.15.116:3300/0 conn(0x7f2b640bd780 0x7f2b640c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:41.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b6159a700 1 -- 172.21.15.116:0/3087406067 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987361738 0 0) 0x7f2b64155620 con 0x7f2b6414f530 2020-11-20T12:16:41.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b6159a700 1 -- 172.21.15.116:0/3087406067 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b48003240 con 0x7f2b6414f530 2020-11-20T12:16:41.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b6259c700 1 -- 172.21.15.116:0/3087406067 >> v2:172.21.15.116:3300/0 conn(0x7f2b640bd780 msgr2=0x7f2b640c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:41.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b6259c700 1 --2- 172.21.15.116:0/3087406067 >> v2:172.21.15.116:3300/0 conn(0x7f2b640bd780 0x7f2b640c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b6259c700 1 -- 172.21.15.116:0/3087406067 >> v1:172.21.15.116:6789/0 conn(0x7f2b6414f530 legacy=0x7f2b640a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:41.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b6259c700 1 -- 172.21.15.116:0/3087406067 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b64155e00 con 0x7f2b6414f150 2020-11-20T12:16:41.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b6259c700 1 --2- 172.21.15.116:0/3087406067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6414f150 0x7f2b640c87f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f2b64081540 tx=0x7f2b54006df0).ready entity=mon.1 client_cookie=d0c56e9be0af33bb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:41.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b6159a700 1 -- 172.21.15.116:0/3087406067 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b540096d0 con 0x7f2b6414f150 2020-11-20T12:16:41.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b6159a700 1 -- 172.21.15.116:0/3087406067 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b54009830 con 0x7f2b6414f150 2020-11-20T12:16:41.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b6159a700 1 -- 172.21.15.116:0/3087406067 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b54009bb0 con 0x7f2b6414f150 2020-11-20T12:16:41.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3087406067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6414f150 msgr2=0x7f2b640c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:41.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.492+0000 7f2b68c1c700 1 --2- 172.21.15.116:0/3087406067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6414f150 0x7f2b640c87f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f2b64081540 tx=0x7f2b54006df0).stop 2020-11-20T12:16:41.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.496+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3087406067 shutdown_connections 2020-11-20T12:16:41.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.496+0000 7f2b68c1c700 1 --2- 172.21.15.116:0/3087406067 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6414f150 0x7f2b640c87f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.496+0000 7f2b68c1c700 1 --2- 172.21.15.116:0/3087406067 >> v2:172.21.15.116:3300/0 conn(0x7f2b640bd780 0x7f2b640c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.496+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3087406067 >> 172.21.15.116:0/3087406067 conn(0x7f2b64061580 msgr2=0x7f2b64061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:41.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.496+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3087406067 shutdown_connections 2020-11-20T12:16:41.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.496+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3087406067 wait complete. 2020-11-20T12:16:41.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.496+0000 7f2b68c1c700 1 Processor -- start 2020-11-20T12:16:41.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.496+0000 7f2b68c1c700 1 -- start start 2020-11-20T12:16:41.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b68c1c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b640bd780 0x7f2b64166860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:41.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b68c1c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b6414f150 0x7f2b640ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:41.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b68c1c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6414f530 0x7f2b640a9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:41.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b68c1c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64169bf0 con 0x7f2b640bd780 2020-11-20T12:16:41.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b68c1c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64169170 con 0x7f2b6414f530 2020-11-20T12:16:41.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b68c1c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b64169410 con 0x7f2b6414f150 2020-11-20T12:16:41.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b6259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b640bd780 0x7f2b64166860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:41.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b6259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b640bd780 0x7f2b64166860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44114/0 (socket says 172.21.15.116:44114) 2020-11-20T12:16:41.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b6259c700 1 -- 172.21.15.116:0/3679064888 learned_addr learned my addr 172.21.15.116:0/3679064888 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:41.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b61d9b700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b6414f150 0x7f2b640ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:41.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b62d9d700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6414f530 0x7f2b640a9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:41.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b6259c700 1 -- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b6414f150 msgr2=0x7f2b640ae0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:41.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b6259c700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b6414f150 0x7f2b640ae0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b6259c700 1 -- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6414f530 msgr2=0x7f2b640a9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:41.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b6259c700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6414f530 0x7f2b640a9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b6259c700 1 -- 172.21.15.116:0/3679064888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b641696b0 con 0x7f2b640bd780 2020-11-20T12:16:41.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b61d9b700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b6414f150 0x7f2b640ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:41.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b6259c700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b640bd780 0x7f2b64166860 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f2b54006e30 tx=0x7f2b54015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:41.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b527fc700 1 -- 172.21.15.116:0/3679064888 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5401d5d0 con 0x7f2b640bd780 2020-11-20T12:16:41.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b527fc700 1 -- 172.21.15.116:0/3679064888 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b5401d730 con 0x7f2b640bd780 2020-11-20T12:16:41.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3679064888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b64169950 con 0x7f2b640bd780 2020-11-20T12:16:41.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b527fc700 1 -- 172.21.15.116:0/3679064888 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b5401dab0 con 0x7f2b640bd780 2020-11-20T12:16:41.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.500+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3679064888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b64168c60 con 0x7f2b640bd780 2020-11-20T12:16:41.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.504+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3679064888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b64168ee0 con 0x7f2b640bd780 2020-11-20T12:16:41.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.504+0000 7f2b527fc700 1 -- 172.21.15.116:0/3679064888 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2b54026030 con 0x7f2b640bd780 2020-11-20T12:16:41.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.504+0000 7f2b527fc700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b380557c0 0x7f2b38057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:41.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.504+0000 7f2b527fc700 1 -- 172.21.15.116:0/3679064888 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2b540700b0 con 0x7f2b640bd780 2020-11-20T12:16:41.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.504+0000 7f2b61d9b700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b380557c0 0x7f2b38057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.508+0000 7f2b61d9b700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b380557c0 0x7f2b38057c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f2b640ae8c0 tx=0x7f2b58008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.512+0000 7f2b527fc700 1 -- 172.21.15.116:0/3679064888 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2b54045f50 con 0x7f2b640bd780 2020-11-20T12:16:41.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.676+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3679064888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f2b6405ca30 con 0x7f2b640bd780 2020-11-20T12:16:41.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.684+0000 7f2b527fc700 1 -- 172.21.15.116:0/3679064888 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b540240c0 con 0x7f2b640bd780 2020-11-20T12:16:41.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b380557c0 msgr2=0x7f2b38057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:41.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b380557c0 0x7f2b38057c80 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f2b640ae8c0 tx=0x7f2b58008040).stop 2020-11-20T12:16:41.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3679064888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b640bd780 msgr2=0x7f2b64166860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:41.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b640bd780 0x7f2b64166860 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f2b54006e30 tx=0x7f2b54015000).stop 2020-11-20T12:16:41.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3679064888 shutdown_connections 2020-11-20T12:16:41.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b380557c0 0x7f2b38057c80 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b6414f530 0x7f2b640a9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b640bd780 0x7f2b64166860 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 --2- 172.21.15.116:0/3679064888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b6414f150 0x7f2b640ae0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:41.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3679064888 >> 172.21.15.116:0/3679064888 conn(0x7f2b64061580 msgr2=0x7f2b640c7ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:41.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3679064888 shutdown_connections 2020-11-20T12:16:41.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:41.688+0000 7f2b68c1c700 1 -- 172.21.15.116:0/3679064888 wait complete. 2020-11-20T12:16:41.712 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:16:41.713 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:16:41.762 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:16:41.807 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:16:41.807 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-20T12:16:41.808 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:16:41.808 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2020-11-20T12:16:41.908 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:16:41.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:41.909 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:16:41.942 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-20T12:16:41.942 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:16:41.943 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2020-11-20T12:16:42.028 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:16:42.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:42.029 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T12:16:42.062 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2020-11-20T12:16:42.063 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:16:42.063 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2020-11-20T12:16:42.128 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:16:42.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:42.130 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T12:16:42.154 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2020-11-20T12:16:42.155 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:16:42.156 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2020-11-20T12:16:42.237 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:16:42.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:16:42.238 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T12:16:42.255 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2020-11-20T12:16:42.255 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:16:42.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.428+0000 7f9eac993700 1 Processor -- start 2020-11-20T12:16:42.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.428+0000 7f9eac993700 1 -- start start 2020-11-20T12:16:42.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9eac993700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9ea813f1d0 0x7f9ea80a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9eac993700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9ea813f5b0 0x7f9ea8066090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9eac993700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea80a5680 con 0x7f9ea80a40c0 2020-11-20T12:16:42.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9eac993700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea80a58e0 con 0x7f9ea813f1d0 2020-11-20T12:16:42.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9eac993700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea80a5b80 con 0x7f9ea813f5b0 2020-11-20T12:16:42.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9ea6ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9ea80a40c0 0x7f9ea80c3fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53556/0 (socket says 172.21.15.116:53556) 2020-11-20T12:16:42.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9ea6ffd700 1 -- 172.21.15.116:0/4158808744 learned_addr learned my addr 172.21.15.116:0/4158808744 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:42.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9ea77fe700 1 --2- 172.21.15.116:0/4158808744 >> v2:172.21.15.116:3300/0 conn(0x7f9ea813f1d0 0x7f9ea80a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9ea7fff700 1 --2- 172.21.15.116:0/4158808744 >> v2:172.21.15.173:3300/0 conn(0x7f9ea813f5b0 0x7f9ea8066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9ea67fc700 1 -- 172.21.15.116:0/4158808744 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3531594990 0 0) 0x7f9ea80a5680 con 0x7f9ea80a40c0 2020-11-20T12:16:42.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9ea77fe700 1 -- 172.21.15.116:0/4158808744 >> v2:172.21.15.173:3300/0 conn(0x7f9ea813f5b0 msgr2=0x7f9ea8066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9ea77fe700 1 --2- 172.21.15.116:0/4158808744 >> v2:172.21.15.173:3300/0 conn(0x7f9ea813f5b0 0x7f9ea8066090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9ea77fe700 1 -- 172.21.15.116:0/4158808744 >> v1:172.21.15.116:6789/0 conn(0x7f9ea80a40c0 legacy=0x7f9ea80c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9ea77fe700 1 -- 172.21.15.116:0/4158808744 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea80a5e20 con 0x7f9ea813f1d0 2020-11-20T12:16:42.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9ea77fe700 1 --2- 172.21.15.116:0/4158808744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea813f1d0 0x7f9ea80a3bd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f9ea80a5470 tx=0x7f9e98006df0).ready entity=mon.0 client_cookie=ac462f7a3697e52a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:42.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.432+0000 7f9ea67fc700 1 -- 172.21.15.116:0/4158808744 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e980097b0 con 0x7f9ea813f1d0 2020-11-20T12:16:42.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.436+0000 7f9ea67fc700 1 -- 172.21.15.116:0/4158808744 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98009910 con 0x7f9ea813f1d0 2020-11-20T12:16:42.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.436+0000 7f9ea67fc700 1 -- 172.21.15.116:0/4158808744 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98009c90 con 0x7f9ea813f1d0 2020-11-20T12:16:42.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.436+0000 7f9eac993700 1 -- 172.21.15.116:0/4158808744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea813f1d0 msgr2=0x7f9ea80a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:42.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.436+0000 7f9eac993700 1 --2- 172.21.15.116:0/4158808744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea813f1d0 0x7f9ea80a3bd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f9ea80a5470 tx=0x7f9e98006df0).stop 2020-11-20T12:16:42.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.436+0000 7f9eac993700 1 -- 172.21.15.116:0/4158808744 shutdown_connections 2020-11-20T12:16:42.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.436+0000 7f9eac993700 1 --2- 172.21.15.116:0/4158808744 >> v2:172.21.15.173:3300/0 conn(0x7f9ea813f5b0 0x7f9ea8066090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.436+0000 7f9eac993700 1 --2- 172.21.15.116:0/4158808744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea813f1d0 0x7f9ea80a3bd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.436+0000 7f9eac993700 1 -- 172.21.15.116:0/4158808744 >> 172.21.15.116:0/4158808744 conn(0x7f9ea8062520 msgr2=0x7f9ea8062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:42.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.436+0000 7f9eac993700 1 -- 172.21.15.116:0/4158808744 shutdown_connections 2020-11-20T12:16:42.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.436+0000 7f9eac993700 1 -- 172.21.15.116:0/4158808744 wait complete. 2020-11-20T12:16:42.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9eac993700 1 Processor -- start 2020-11-20T12:16:42.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9eac993700 1 -- start start 2020-11-20T12:16:42.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9eac993700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ea813f1d0 0x7f9ea81554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9eac993700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea813f5b0 0x7f9ea80a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9eac993700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea809ed70 0x7f9ea809f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9eac993700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea809f6c0 con 0x7f9ea809ed70 2020-11-20T12:16:42.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9eac993700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea809e650 con 0x7f9ea813f5b0 2020-11-20T12:16:42.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9eac993700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea809d490 con 0x7f9ea813f1d0 2020-11-20T12:16:42.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9ea7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea809ed70 0x7f9ea809f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9ea7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea809ed70 0x7f9ea809f1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44132/0 (socket says 172.21.15.116:44132) 2020-11-20T12:16:42.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9ea7fff700 1 -- 172.21.15.116:0/4059554977 learned_addr learned my addr 172.21.15.116:0/4059554977 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:42.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9ea6ffd700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea813f5b0 0x7f9ea80a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9ea77fe700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ea813f1d0 0x7f9ea81554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9ea7fff700 1 -- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ea813f1d0 msgr2=0x7f9ea81554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.440+0000 7f9ea7fff700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ea813f1d0 0x7f9ea81554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.444+0000 7f9ea7fff700 1 -- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea813f5b0 msgr2=0x7f9ea80a9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:42.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.444+0000 7f9ea7fff700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea813f5b0 0x7f9ea80a9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.444+0000 7f9ea7fff700 1 -- 172.21.15.116:0/4059554977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea809d730 con 0x7f9ea809ed70 2020-11-20T12:16:42.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.444+0000 7f9ea7fff700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea809ed70 0x7f9ea809f1d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f9ea000ec60 tx=0x7f9ea00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:42.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.444+0000 7f9e877fe700 1 -- 172.21.15.116:0/4059554977 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea000cac0 con 0x7f9ea809ed70 2020-11-20T12:16:42.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.444+0000 7f9e877fe700 1 -- 172.21.15.116:0/4059554977 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ea000cc20 con 0x7f9ea809ed70 2020-11-20T12:16:42.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.444+0000 7f9eac993700 1 -- 172.21.15.116:0/4059554977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea809d9d0 con 0x7f9ea809ed70 2020-11-20T12:16:42.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.444+0000 7f9e877fe700 1 -- 172.21.15.116:0/4059554977 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea00123f0 con 0x7f9ea809ed70 2020-11-20T12:16:42.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.444+0000 7f9eac993700 1 -- 172.21.15.116:0/4059554977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea809df10 con 0x7f9ea809ed70 2020-11-20T12:16:42.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.444+0000 7f9eac993700 1 -- 172.21.15.116:0/4059554977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea813ee60 con 0x7f9ea809ed70 2020-11-20T12:16:42.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.448+0000 7f9e877fe700 1 -- 172.21.15.116:0/4059554977 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9ea0022070 con 0x7f9ea809ed70 2020-11-20T12:16:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.448+0000 7f9e877fe700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e800557c0 0x7f9e80057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.448+0000 7f9e877fe700 1 -- 172.21.15.116:0/4059554977 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9ea006edf0 con 0x7f9ea809ed70 2020-11-20T12:16:42.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.448+0000 7f9ea77fe700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e800557c0 0x7f9e80057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.452+0000 7f9ea77fe700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e800557c0 0x7f9e80057c80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f9e98006e30 tx=0x7f9e98015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:42.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.452+0000 7f9e877fe700 1 -- 172.21.15.116:0/4059554977 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9ea813ee60 con 0x7f9ea809ed70 2020-11-20T12:16:42.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.660+0000 7f9eac993700 1 -- 172.21.15.116:0/4059554977 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9ea813ee60 con 0x7f9ea809ed70 2020-11-20T12:16:42.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.660+0000 7f9e877fe700 1 -- 172.21.15.116:0/4059554977 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2846 (secure 0 0 0) 0x7f9ea813ee60 con 0x7f9ea809ed70 2020-11-20T12:16:42.662 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:42.663 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/2968741298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":2968741298},{"type":"v1","addr":"172.21.15.116:6835","nonce":2968741298}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1214743116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1214743116},{"type":"v1","addr":"172.21.15.173:6836","nonce":1214743116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1424291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1424291148},{"type":"v1","addr":"172.21.15.173:6837","nonce":1424291148}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:00.619934+0000","modified":"2020-11-20T12:16:02.636079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.116:6834/3819756176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3819756176},{"type":"v1","addr":"172.21.15.116:6834","nonce":3819756176}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:16:42.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 -- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e800557c0 msgr2=0x7f9e80057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:42.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e800557c0 0x7f9e80057c80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f9e98006e30 tx=0x7f9e98015000).stop 2020-11-20T12:16:42.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 -- 172.21.15.116:0/4059554977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea809ed70 msgr2=0x7f9ea809f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:42.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea809ed70 0x7f9ea809f1d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f9ea000ec60 tx=0x7f9ea00091d0).stop 2020-11-20T12:16:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 -- 172.21.15.116:0/4059554977 shutdown_connections 2020-11-20T12:16:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e800557c0 0x7f9e80057c80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ea809ed70 0x7f9ea809f1d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ea813f1d0 0x7f9ea81554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 --2- 172.21.15.116:0/4059554977 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ea813f5b0 0x7f9ea80a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 -- 172.21.15.116:0/4059554977 >> 172.21.15.116:0/4059554977 conn(0x7f9ea8062520 msgr2=0x7f9ea80c3380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:42.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 -- 172.21.15.116:0/4059554977 shutdown_connections 2020-11-20T12:16:42.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.664+0000 7f9eac993700 1 -- 172.21.15.116:0/4059554977 wait complete. 2020-11-20T12:16:42.668 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 6 2020-11-20T12:16:42.692 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:42.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.856+0000 7f86f9292700 1 Processor -- start 2020-11-20T12:16:42.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.856+0000 7f86f9292700 1 -- start start 2020-11-20T12:16:42.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.856+0000 7f86f9292700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f86f414b8d0 0x7f86f40c6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.856+0000 7f86f9292700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86f414bcb0 0x7f86f4152630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.856+0000 7f86f9292700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86f414c100 con 0x7f86f4061190 2020-11-20T12:16:42.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.856+0000 7f86f9292700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86f40a8190 con 0x7f86f414b8d0 2020-11-20T12:16:42.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.856+0000 7f86f9292700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f4152b20 con 0x7f86f414bcb0 2020-11-20T12:16:42.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f2ffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f86f414b8d0 0x7f86f40c6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f27fc700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f86f4061190 0x7f86f40c4e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49800/0 (socket says 172.21.15.116:49800) 2020-11-20T12:16:42.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f27fc700 1 -- 172.21.15.116:0/947261558 learned_addr learned my addr 172.21.15.116:0/947261558 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:42.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f37fe700 1 --2- 172.21.15.116:0/947261558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86f414bcb0 0x7f86f4152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f1ffb700 1 -- 172.21.15.116:0/947261558 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599164541 0 0) 0x7f86e80010d0 con 0x7f86f4061190 2020-11-20T12:16:42.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f1ffb700 1 -- 172.21.15.116:0/947261558 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86e0003240 con 0x7f86f4061190 2020-11-20T12:16:42.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f2ffd700 1 -- 172.21.15.116:0/947261558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86f414bcb0 msgr2=0x7f86f4152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:42.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f2ffd700 1 --2- 172.21.15.116:0/947261558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86f414bcb0 0x7f86f4152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f2ffd700 1 -- 172.21.15.116:0/947261558 >> v1:172.21.15.173:6789/0 conn(0x7f86f4061190 legacy=0x7f86f40c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:42.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f2ffd700 1 -- 172.21.15.116:0/947261558 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f4061710 con 0x7f86f414b8d0 2020-11-20T12:16:42.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f2ffd700 1 --2- 172.21.15.116:0/947261558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f414b8d0 0x7f86f40c6910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f86e400b9a0 tx=0x7f86e4006df0).ready entity=mon.0 client_cookie=427c87db7185a119 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:42.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f1ffb700 1 -- 172.21.15.116:0/947261558 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4009680 con 0x7f86f414b8d0 2020-11-20T12:16:42.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f1ffb700 1 -- 172.21.15.116:0/947261558 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e40097e0 con 0x7f86f414b8d0 2020-11-20T12:16:42.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f1ffb700 1 -- 172.21.15.116:0/947261558 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4009b60 con 0x7f86f414b8d0 2020-11-20T12:16:42.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f9292700 1 -- 172.21.15.116:0/947261558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f414b8d0 msgr2=0x7f86f40c6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:42.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f9292700 1 --2- 172.21.15.116:0/947261558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f414b8d0 0x7f86f40c6910 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f86e400b9a0 tx=0x7f86e4006df0).stop 2020-11-20T12:16:42.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f9292700 1 -- 172.21.15.116:0/947261558 shutdown_connections 2020-11-20T12:16:42.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f9292700 1 --2- 172.21.15.116:0/947261558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86f414bcb0 0x7f86f4152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f9292700 1 --2- 172.21.15.116:0/947261558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f414b8d0 0x7f86f40c6910 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f9292700 1 -- 172.21.15.116:0/947261558 >> 172.21.15.116:0/947261558 conn(0x7f86f40678a0 msgr2=0x7f86f4067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:42.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f9292700 1 -- 172.21.15.116:0/947261558 shutdown_connections 2020-11-20T12:16:42.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.860+0000 7f86f9292700 1 -- 172.21.15.116:0/947261558 wait complete. 2020-11-20T12:16:42.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f9292700 1 Processor -- start 2020-11-20T12:16:42.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f9292700 1 -- start start 2020-11-20T12:16:42.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f9292700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f4061190 0x7f86f4161c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f9292700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86f414b8d0 0x7f86f4156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f9292700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86f414bcb0 0x7f86f414aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f9292700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f4157310 con 0x7f86f4061190 2020-11-20T12:16:42.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f9292700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86f41575b0 con 0x7f86f414bcb0 2020-11-20T12:16:42.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f9292700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86f4157850 con 0x7f86f414b8d0 2020-11-20T12:16:42.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f4061190 0x7f86f4161c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f4061190 0x7f86f4161c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44144/0 (socket says 172.21.15.116:44144) 2020-11-20T12:16:42.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f2ffd700 1 -- 172.21.15.116:0/2913613385 learned_addr learned my addr 172.21.15.116:0/2913613385 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:42.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f27fc700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86f414b8d0 0x7f86f4156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.864+0000 7f86f37fe700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86f414bcb0 0x7f86f414aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86f2ffd700 1 -- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86f414b8d0 msgr2=0x7f86f4156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:42.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86f2ffd700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86f414b8d0 0x7f86f4156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86f2ffd700 1 -- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86f414bcb0 msgr2=0x7f86f414aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:42.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86f2ffd700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86f414bcb0 0x7f86f414aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:42.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86f2ffd700 1 -- 172.21.15.116:0/2913613385 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f4157af0 con 0x7f86f4061190 2020-11-20T12:16:42.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86f2ffd700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f4061190 0x7f86f4161c10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f86e4009380 tx=0x7f86e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:42.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86daffd700 1 -- 172.21.15.116:0/2913613385 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e4011cc0 con 0x7f86f4061190 2020-11-20T12:16:42.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86daffd700 1 -- 172.21.15.116:0/2913613385 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86e4011e20 con 0x7f86f4061190 2020-11-20T12:16:42.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86f9292700 1 -- 172.21.15.116:0/2913613385 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86f4156fa0 con 0x7f86f4061190 2020-11-20T12:16:42.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86daffd700 1 -- 172.21.15.116:0/2913613385 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86e401d6a0 con 0x7f86f4061190 2020-11-20T12:16:42.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86f9292700 1 -- 172.21.15.116:0/2913613385 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86f40ad810 con 0x7f86f4061190 2020-11-20T12:16:42.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.868+0000 7f86f9292700 1 -- 172.21.15.116:0/2913613385 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86f4157c80 con 0x7f86f4061190 2020-11-20T12:16:42.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.872+0000 7f86daffd700 1 -- 172.21.15.116:0/2913613385 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f86e4024030 con 0x7f86f4061190 2020-11-20T12:16:42.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.872+0000 7f86daffd700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86d00557c0 0x7f86d0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:42.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.872+0000 7f86daffd700 1 -- 172.21.15.116:0/2913613385 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f86e406fba0 con 0x7f86f4061190 2020-11-20T12:16:42.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.872+0000 7f86f27fc700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86d00557c0 0x7f86d0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:42.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.872+0000 7f86f27fc700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86d00557c0 0x7f86d0057c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f86e8007fd0 tx=0x7f86e8000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:42.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:42.876+0000 7f86daffd700 1 -- 172.21.15.116:0/2913613385 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f86e40459c0 con 0x7f86f4061190 2020-11-20T12:16:43.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.036+0000 7f86f9292700 1 -- 172.21.15.116:0/2913613385 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86f4044f60 con 0x7f86f4061190 2020-11-20T12:16:43.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.040+0000 7f86daffd700 1 -- 172.21.15.116:0/2913613385 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f86e4048fe0 con 0x7f86f4061190 2020-11-20T12:16:43.041 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:43.042 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:43.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.040+0000 7f86f9292700 1 -- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86d00557c0 msgr2=0x7f86d0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.040+0000 7f86f9292700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86d00557c0 0x7f86d0057c80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f86e8007fd0 tx=0x7f86e8000b80).stop 2020-11-20T12:16:43.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.040+0000 7f86f9292700 1 -- 172.21.15.116:0/2913613385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f4061190 msgr2=0x7f86f4161c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.040+0000 7f86f9292700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f4061190 0x7f86f4161c10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f86e4009380 tx=0x7f86e4015000).stop 2020-11-20T12:16:43.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.040+0000 7f86f9292700 1 -- 172.21.15.116:0/2913613385 shutdown_connections 2020-11-20T12:16:43.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.040+0000 7f86f9292700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86d00557c0 0x7f86d0057c80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.040+0000 7f86f9292700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86f414bcb0 0x7f86f414aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.040+0000 7f86f9292700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f86f4061190 0x7f86f4161c10 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.040+0000 7f86f9292700 1 --2- 172.21.15.116:0/2913613385 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f86f414b8d0 0x7f86f4156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.044+0000 7f86f9292700 1 -- 172.21.15.116:0/2913613385 >> 172.21.15.116:0/2913613385 conn(0x7f86f40678a0 msgr2=0x7f86f40a9590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.044+0000 7f86f9292700 1 -- 172.21.15.116:0/2913613385 shutdown_connections 2020-11-20T12:16:43.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.044+0000 7f86f9292700 1 -- 172.21.15.116:0/2913613385 wait complete. 2020-11-20T12:16:43.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:15:59.065285+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:15:59.602311+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:16:43.068 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.244+0000 7f2b3e411700 1 Processor -- start 2020-11-20T12:16:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.244+0000 7f2b3e411700 1 -- start start 2020-11-20T12:16:43.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.244+0000 7f2b3e411700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2b380a0f00 0x7f2b380a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.244+0000 7f2b3e411700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b380a50a0 con 0x7f2b380a12e0 2020-11-20T12:16:43.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.244+0000 7f2b3e411700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b380a5340 con 0x7f2b380c2df0 2020-11-20T12:16:43.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.244+0000 7f2b3e411700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b380a55e0 con 0x7f2b380a0f00 2020-11-20T12:16:43.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b3cc0e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2b380a12e0 0x7f2b38150370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53586/0 (socket says 172.21.15.116:53586) 2020-11-20T12:16:43.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b3cc0e700 1 -- 172.21.15.116:0/2747627303 learned_addr learned my addr 172.21.15.116:0/2747627303 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:43.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b3dc10700 1 --1- 172.21.15.116:0/2747627303 >> v1:172.21.15.173:6789/0 conn(0x7f2b380c2df0 0x7f2b38149240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49814/0 (socket says 172.21.15.116:49814) 2020-11-20T12:16:43.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b3d40f700 1 --2- 172.21.15.116:0/2747627303 >> v2:172.21.15.173:3300/0 conn(0x7f2b380a0f00 0x7f2b380a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b27fff700 1 -- 172.21.15.116:0/2747627303 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1898438785 0 0) 0x7f2b380a5340 con 0x7f2b380c2df0 2020-11-20T12:16:43.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b27fff700 1 -- 172.21.15.116:0/2747627303 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b20003240 con 0x7f2b380c2df0 2020-11-20T12:16:43.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b27fff700 1 -- 172.21.15.116:0/2747627303 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375353974 0 0) 0x7f2b380a50a0 con 0x7f2b380a12e0 2020-11-20T12:16:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b27fff700 1 -- 172.21.15.116:0/2747627303 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b380a5340 con 0x7f2b380a12e0 2020-11-20T12:16:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b3d40f700 1 -- 172.21.15.116:0/2747627303 >> v1:172.21.15.173:6789/0 conn(0x7f2b380c2df0 legacy=0x7f2b38149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b3d40f700 1 -- 172.21.15.116:0/2747627303 >> v1:172.21.15.116:6789/0 conn(0x7f2b380a12e0 legacy=0x7f2b38150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b27fff700 1 -- 172.21.15.116:0/2747627303 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3009275585 0 0) 0x7f2b380a5340 con 0x7f2b380a12e0 2020-11-20T12:16:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b3d40f700 1 -- 172.21.15.116:0/2747627303 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b380a5880 con 0x7f2b380a0f00 2020-11-20T12:16:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b3d40f700 1 --2- 172.21.15.116:0/2747627303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b380a0f00 0x7f2b380a5fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2b3400b9a0 tx=0x7f2b34006df0).ready entity=mon.1 client_cookie=2217819ae21e2c6d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b27fff700 1 -- 172.21.15.116:0/2747627303 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b280014e0 con 0x7f2b380a0f00 2020-11-20T12:16:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b27fff700 1 -- 172.21.15.116:0/2747627303 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b34011400 con 0x7f2b380a0f00 2020-11-20T12:16:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.248+0000 7f2b27fff700 1 -- 172.21.15.116:0/2747627303 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b34011780 con 0x7f2b380a0f00 2020-11-20T12:16:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.252+0000 7f2b3e411700 1 -- 172.21.15.116:0/2747627303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b380a0f00 msgr2=0x7f2b380a5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.252+0000 7f2b3e411700 1 --2- 172.21.15.116:0/2747627303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b380a0f00 0x7f2b380a5fc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f2b3400b9a0 tx=0x7f2b34006df0).stop 2020-11-20T12:16:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.252+0000 7f2b3e411700 1 -- 172.21.15.116:0/2747627303 shutdown_connections 2020-11-20T12:16:43.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.252+0000 7f2b3e411700 1 --2- 172.21.15.116:0/2747627303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b380a0f00 0x7f2b380a5fc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.252+0000 7f2b3e411700 1 -- 172.21.15.116:0/2747627303 >> 172.21.15.116:0/2747627303 conn(0x7f2b38060c30 msgr2=0x7f2b380c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.252+0000 7f2b3e411700 1 -- 172.21.15.116:0/2747627303 shutdown_connections 2020-11-20T12:16:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.252+0000 7f2b3e411700 1 -- 172.21.15.116:0/2747627303 wait complete. 2020-11-20T12:16:43.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.256+0000 7f2b3e411700 1 Processor -- start 2020-11-20T12:16:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.256+0000 7f2b3e411700 1 -- start start 2020-11-20T12:16:43.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.256+0000 7f2b3e411700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b380a12e0 0x7f2b38063070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.256+0000 7f2b3e411700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b380c2df0 0x7f2b38067b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.256+0000 7f2b3e411700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b38066420 0x7f2b3814add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.256+0000 7f2b3e411700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b38066c10 con 0x7f2b38066420 2020-11-20T12:16:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.256+0000 7f2b3e411700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b38066eb0 con 0x7f2b380c2df0 2020-11-20T12:16:43.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.256+0000 7f2b3e411700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b38067150 con 0x7f2b380a12e0 2020-11-20T12:16:43.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3dc10700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b38066420 0x7f2b3814add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3d40f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b380a12e0 0x7f2b38063070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3dc10700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b38066420 0x7f2b3814add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44158/0 (socket says 172.21.15.116:44158) 2020-11-20T12:16:43.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3dc10700 1 -- 172.21.15.116:0/1144101759 learned_addr learned my addr 172.21.15.116:0/1144101759 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:43.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3cc0e700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b380c2df0 0x7f2b38067b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3d40f700 1 -- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b380c2df0 msgr2=0x7f2b38067b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3d40f700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b380c2df0 0x7f2b38067b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3d40f700 1 -- 172.21.15.116:0/1144101759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b38066420 msgr2=0x7f2b3814add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3d40f700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b38066420 0x7f2b3814add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3d40f700 1 -- 172.21.15.116:0/1144101759 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b380673f0 con 0x7f2b380a12e0 2020-11-20T12:16:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3dc10700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b38066420 0x7f2b3814add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3d40f700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b380a12e0 0x7f2b38063070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f2b34012000 tx=0x7f2b34013000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b257fa700 1 -- 172.21.15.116:0/1144101759 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b34010a60 con 0x7f2b380a12e0 2020-11-20T12:16:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b257fa700 1 -- 172.21.15.116:0/1144101759 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b34010bc0 con 0x7f2b380a12e0 2020-11-20T12:16:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3e411700 1 -- 172.21.15.116:0/1144101759 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b38067690 con 0x7f2b380a12e0 2020-11-20T12:16:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b257fa700 1 -- 172.21.15.116:0/1144101759 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3401d3c0 con 0x7f2b380a12e0 2020-11-20T12:16:43.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.260+0000 7f2b3e411700 1 -- 172.21.15.116:0/1144101759 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b38066860 con 0x7f2b380a12e0 2020-11-20T12:16:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.264+0000 7f2b3e411700 1 -- 172.21.15.116:0/1144101759 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b38067820 con 0x7f2b380a12e0 2020-11-20T12:16:43.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.264+0000 7f2b257fa700 1 -- 172.21.15.116:0/1144101759 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2b3401a040 con 0x7f2b380a12e0 2020-11-20T12:16:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.264+0000 7f2b257fa700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b100557f0 0x7f2b10057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.264+0000 7f2b257fa700 1 -- 172.21.15.116:0/1144101759 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2b3406f790 con 0x7f2b380a12e0 2020-11-20T12:16:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.264+0000 7f2b3cc0e700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b100557f0 0x7f2b10057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.268+0000 7f2b3cc0e700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b100557f0 0x7f2b10057cb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f2b380aa1c0 tx=0x7f2b28001e50).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.272+0000 7f2b257fa700 1 -- 172.21.15.116:0/1144101759 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2b34045630 con 0x7f2b380a12e0 2020-11-20T12:16:43.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.432+0000 7f2b3e411700 1 -- 172.21.15.116:0/1144101759 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2b38044f60 con 0x7f2b380a12e0 2020-11-20T12:16:43.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b257fa700 1 -- 172.21.15.116:0/1144101759 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v23) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2b34024030 con 0x7f2b380a12e0 2020-11-20T12:16:43.436 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:43.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 -- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b100557f0 msgr2=0x7f2b10057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b100557f0 0x7f2b10057cb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f2b380aa1c0 tx=0x7f2b28001e50).stop 2020-11-20T12:16:43.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 -- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b380a12e0 msgr2=0x7f2b38063070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b380a12e0 0x7f2b38063070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f2b34012000 tx=0x7f2b34013000).stop 2020-11-20T12:16:43.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 -- 172.21.15.116:0/1144101759 shutdown_connections 2020-11-20T12:16:43.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2b100557f0 0x7f2b10057cb0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b38066420 0x7f2b3814add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b380a12e0 0x7f2b38063070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 --2- 172.21.15.116:0/1144101759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b380c2df0 0x7f2b38067b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 -- 172.21.15.116:0/1144101759 >> 172.21.15.116:0/1144101759 conn(0x7f2b38060c30 msgr2=0x7f2b380bfb00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 -- 172.21.15.116:0/1144101759 shutdown_connections 2020-11-20T12:16:43.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.436+0000 7f2b3e411700 1 -- 172.21.15.116:0/1144101759 wait complete. 2020-11-20T12:16:43.465 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:43.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.632+0000 7f61ce51a700 1 Processor -- start 2020-11-20T12:16:43.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.636+0000 7f61ce51a700 1 -- start start 2020-11-20T12:16:43.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.636+0000 7f61ce51a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f61c80c1bb0 0x7f61c8156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.636+0000 7f61ce51a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c80a38c0 con 0x7f61c80a28c0 2020-11-20T12:16:43.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.636+0000 7f61ce51a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c8156e30 con 0x7f61c80c03f0 2020-11-20T12:16:43.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.636+0000 7f61ce51a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c80a2e40 con 0x7f61c80c1bb0 2020-11-20T12:16:43.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.636+0000 7f61cdd19700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f61c80a28c0 0x7f61c8066570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53600/0 (socket says 172.21.15.116:53600) 2020-11-20T12:16:43.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.636+0000 7f61cdd19700 1 -- 172.21.15.116:0/2330436581 learned_addr learned my addr 172.21.15.116:0/2330436581 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:43.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.636+0000 7f61ccd17700 1 --2- 172.21.15.116:0/2330436581 >> v2:172.21.15.173:3300/0 conn(0x7f61c80c1bb0 0x7f61c8156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61b7fff700 1 -- 172.21.15.116:0/2330436581 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604462541 0 0) 0x7f61c80a38c0 con 0x7f61c80a28c0 2020-11-20T12:16:43.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61b7fff700 1 -- 172.21.15.116:0/2330436581 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b0003240 con 0x7f61c80a28c0 2020-11-20T12:16:43.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61b7fff700 1 -- 172.21.15.116:0/2330436581 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1965027694 0 0) 0x7f61b0003240 con 0x7f61c80a28c0 2020-11-20T12:16:43.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61ccd17700 1 -- 172.21.15.116:0/2330436581 >> v1:172.21.15.173:6789/0 conn(0x7f61c80c03f0 legacy=0x7f61c80677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61ccd17700 1 -- 172.21.15.116:0/2330436581 >> v1:172.21.15.116:6789/0 conn(0x7f61c80a28c0 legacy=0x7f61c8066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61ccd17700 1 -- 172.21.15.116:0/2330436581 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c80a30e0 con 0x7f61c80c1bb0 2020-11-20T12:16:43.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61ccd17700 1 --2- 172.21.15.116:0/2330436581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c80c1bb0 0x7f61c8156940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f61b800b9a0 tx=0x7f61b8006df0).ready entity=mon.1 client_cookie=99ca471776dbfc96 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61b7fff700 1 -- 172.21.15.116:0/2330436581 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8009840 con 0x7f61c80c1bb0 2020-11-20T12:16:43.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61b7fff700 1 -- 172.21.15.116:0/2330436581 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b80099a0 con 0x7f61c80c1bb0 2020-11-20T12:16:43.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61b7fff700 1 -- 172.21.15.116:0/2330436581 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b8009d20 con 0x7f61c80c1bb0 2020-11-20T12:16:43.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61ce51a700 1 -- 172.21.15.116:0/2330436581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c80c1bb0 msgr2=0x7f61c8156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61ce51a700 1 --2- 172.21.15.116:0/2330436581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c80c1bb0 0x7f61c8156940 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f61b800b9a0 tx=0x7f61b8006df0).stop 2020-11-20T12:16:43.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61ce51a700 1 -- 172.21.15.116:0/2330436581 shutdown_connections 2020-11-20T12:16:43.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61ce51a700 1 --2- 172.21.15.116:0/2330436581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c80c1bb0 0x7f61c8156940 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61ce51a700 1 -- 172.21.15.116:0/2330436581 >> 172.21.15.116:0/2330436581 conn(0x7f61c8060c30 msgr2=0x7f61c80c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61ce51a700 1 -- 172.21.15.116:0/2330436581 shutdown_connections 2020-11-20T12:16:43.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.640+0000 7f61ce51a700 1 -- 172.21.15.116:0/2330436581 wait complete. 2020-11-20T12:16:43.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.644+0000 7f61ce51a700 1 Processor -- start 2020-11-20T12:16:43.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.644+0000 7f61ce51a700 1 -- start start 2020-11-20T12:16:43.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.644+0000 7f61ce51a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c80a28c0 0x7f61c8062fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.644+0000 7f61ce51a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61c80c03f0 0x7f61c8090580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.644+0000 7f61ce51a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80c1bb0 0x7f61c80a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.644+0000 7f61ce51a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c8090a70 con 0x7f61c80c1bb0 2020-11-20T12:16:43.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.644+0000 7f61ce51a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80a96a0 con 0x7f61c80a28c0 2020-11-20T12:16:43.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.644+0000 7f61ce51a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c80a7450 con 0x7f61c80c03f0 2020-11-20T12:16:43.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.644+0000 7f61cdd19700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80c1bb0 0x7f61c80a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.644+0000 7f61cdd19700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80c1bb0 0x7f61c80a6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44172/0 (socket says 172.21.15.116:44172) 2020-11-20T12:16:43.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.644+0000 7f61cdd19700 1 -- 172.21.15.116:0/2379336115 learned_addr learned my addr 172.21.15.116:0/2379336115 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:43.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61cd518700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c80a28c0 0x7f61c8062fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61cdd19700 1 -- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61c80c03f0 msgr2=0x7f61c8090580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61cdd19700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61c80c03f0 0x7f61c8090580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:43.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61cdd19700 1 -- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c80a28c0 msgr2=0x7f61c8062fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61cdd19700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c80a28c0 0x7f61c8062fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61cdd19700 1 -- 172.21.15.116:0/2379336115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c80a76e0 con 0x7f61c80c1bb0 2020-11-20T12:16:43.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61cdd19700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80c1bb0 0x7f61c80a6f60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f61c000cae0 tx=0x7f61c0007a70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61b57fa700 1 -- 172.21.15.116:0/2379336115 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0004a90 con 0x7f61c80c1bb0 2020-11-20T12:16:43.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61b57fa700 1 -- 172.21.15.116:0/2379336115 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c0004bf0 con 0x7f61c80c1bb0 2020-11-20T12:16:43.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61b57fa700 1 -- 172.21.15.116:0/2379336115 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0007170 con 0x7f61c80c1bb0 2020-11-20T12:16:43.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61ce51a700 1 -- 172.21.15.116:0/2379336115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c80a7c20 con 0x7f61c80c1bb0 2020-11-20T12:16:43.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61ce51a700 1 -- 172.21.15.116:0/2379336115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c808f220 con 0x7f61c80c1bb0 2020-11-20T12:16:43.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61ce51a700 1 -- 172.21.15.116:0/2379336115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c808f410 con 0x7f61c80c1bb0 2020-11-20T12:16:43.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.648+0000 7f61b57fa700 1 -- 172.21.15.116:0/2379336115 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f61c0010070 con 0x7f61c80c1bb0 2020-11-20T12:16:43.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.652+0000 7f61b57fa700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557f0 0x7f61a0057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:43.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.652+0000 7f61b57fa700 1 -- 172.21.15.116:0/2379336115 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f61c006e330 con 0x7f61c80c1bb0 2020-11-20T12:16:43.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.652+0000 7f61cd518700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557f0 0x7f61a0057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:43.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.652+0000 7f61cd518700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557f0 0x7f61a0057cb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f61c4003530 tx=0x7f61c4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:43.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.656+0000 7f61b57fa700 1 -- 172.21.15.116:0/2379336115 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f61c00441d0 con 0x7f61c80c1bb0 2020-11-20T12:16:43.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.816+0000 7f61ce51a700 1 -- 172.21.15.116:0/2379336115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f61c8044f60 con 0x7f61c80c1bb0 2020-11-20T12:16:43.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.816+0000 7f61b57fa700 1 -- 172.21.15.116:0/2379336115 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f61c00477f0 con 0x7f61c80c1bb0 2020-11-20T12:16:43.816 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:43.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.816+0000 7f61ce51a700 1 -- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557f0 msgr2=0x7f61a0057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.816+0000 7f61ce51a700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557f0 0x7f61a0057cb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f61c4003530 tx=0x7f61c4009040).stop 2020-11-20T12:16:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.816+0000 7f61ce51a700 1 -- 172.21.15.116:0/2379336115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80c1bb0 msgr2=0x7f61c80a6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.816+0000 7f61ce51a700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80c1bb0 0x7f61c80a6f60 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f61c000cae0 tx=0x7f61c0007a70).stop 2020-11-20T12:16:43.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.820+0000 7f61ce51a700 1 -- 172.21.15.116:0/2379336115 shutdown_connections 2020-11-20T12:16:43.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.820+0000 7f61ce51a700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557f0 0x7f61a0057cb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.820+0000 7f61ce51a700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c80c1bb0 0x7f61c80a6f60 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.820+0000 7f61ce51a700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c80a28c0 0x7f61c8062fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:43.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.820+0000 7f61ce51a700 1 --2- 172.21.15.116:0/2379336115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61c80c03f0 0x7f61c8090580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:43.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.820+0000 7f61ce51a700 1 -- 172.21.15.116:0/2379336115 >> 172.21.15.116:0/2379336115 conn(0x7f61c8060c30 msgr2=0x7f61c81550e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:43.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.820+0000 7f61ce51a700 1 -- 172.21.15.116:0/2379336115 shutdown_connections 2020-11-20T12:16:43.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:43.820+0000 7f61ce51a700 1 -- 172.21.15.116:0/2379336115 wait complete. 2020-11-20T12:16:43.843 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:16:44.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.020+0000 7f92df55b700 1 Processor -- start 2020-11-20T12:16:44.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.020+0000 7f92df55b700 1 -- start start 2020-11-20T12:16:44.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.020+0000 7f92df55b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92d80a0f00 0x7f92d80a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.020+0000 7f92df55b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f92d80a12e0 0x7f92d8150370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.020+0000 7f92df55b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d80a50a0 con 0x7f92d80c2df0 2020-11-20T12:16:44.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.020+0000 7f92df55b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d80a5340 con 0x7f92d80a12e0 2020-11-20T12:16:44.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.020+0000 7f92df55b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d80a55e0 con 0x7f92d80a0f00 2020-11-20T12:16:44.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.020+0000 7f92ddd58700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f92d80a12e0 0x7f92d8150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.020+0000 7f92ddd58700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f92d80a12e0 0x7f92d8150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44176/0 (socket says 172.21.15.116:44176) 2020-11-20T12:16:44.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.020+0000 7f92ddd58700 1 -- 172.21.15.116:0/1088177772 learned_addr learned my addr 172.21.15.116:0/1088177772 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:44.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92de559700 1 --2- 172.21.15.116:0/1088177772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92d80a0f00 0x7f92d80a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92ddd58700 1 -- 172.21.15.116:0/1088177772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92d80a0f00 msgr2=0x7f92d80a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:44.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92ddd58700 1 --2- 172.21.15.116:0/1088177772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92d80a0f00 0x7f92d80a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92ddd58700 1 -- 172.21.15.116:0/1088177772 >> v1:172.21.15.173:6789/0 conn(0x7f92d80c2df0 legacy=0x7f92d8149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92ddd58700 1 -- 172.21.15.116:0/1088177772 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d80a5880 con 0x7f92d80a12e0 2020-11-20T12:16:44.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92ddd58700 1 --2- 172.21.15.116:0/1088177772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92d80a12e0 0x7f92d8150370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f92c8006fd0 tx=0x7f92c8006eb0).ready entity=mon.0 client_cookie=2359c27f2679dd45 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:44.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92dd557700 1 -- 172.21.15.116:0/1088177772 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c80098d0 con 0x7f92d80a12e0 2020-11-20T12:16:44.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92dd557700 1 -- 172.21.15.116:0/1088177772 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c8009a30 con 0x7f92d80a12e0 2020-11-20T12:16:44.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92dd557700 1 -- 172.21.15.116:0/1088177772 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8009db0 con 0x7f92d80a12e0 2020-11-20T12:16:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92df55b700 1 -- 172.21.15.116:0/1088177772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92d80a12e0 msgr2=0x7f92d8150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92df55b700 1 --2- 172.21.15.116:0/1088177772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92d80a12e0 0x7f92d8150370 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f92c8006fd0 tx=0x7f92c8006eb0).stop 2020-11-20T12:16:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92df55b700 1 -- 172.21.15.116:0/1088177772 shutdown_connections 2020-11-20T12:16:44.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92df55b700 1 --2- 172.21.15.116:0/1088177772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92d80a0f00 0x7f92d80a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92df55b700 1 --2- 172.21.15.116:0/1088177772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92d80a12e0 0x7f92d8150370 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92df55b700 1 -- 172.21.15.116:0/1088177772 >> 172.21.15.116:0/1088177772 conn(0x7f92d8060c30 msgr2=0x7f92d80c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:44.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92df55b700 1 -- 172.21.15.116:0/1088177772 shutdown_connections 2020-11-20T12:16:44.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.024+0000 7f92df55b700 1 -- 172.21.15.116:0/1088177772 wait complete. 2020-11-20T12:16:44.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92df55b700 1 Processor -- start 2020-11-20T12:16:44.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92df55b700 1 -- start start 2020-11-20T12:16:44.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92df55b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92d80a0f00 0x7f92d8091230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92df55b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92d80a12e0 0x7f92d814ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92df55b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92d80c2df0 0x7f92d8146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92df55b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d815dab0 con 0x7f92d80c2df0 2020-11-20T12:16:44.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92df55b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d814dfb0 con 0x7f92d80a0f00 2020-11-20T12:16:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92df55b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d814e250 con 0x7f92d80a12e0 2020-11-20T12:16:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92ded5a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92d80c2df0 0x7f92d8146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92ded5a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92d80c2df0 0x7f92d8146f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44186/0 (socket says 172.21.15.116:44186) 2020-11-20T12:16:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92ded5a700 1 -- 172.21.15.116:0/2244753019 learned_addr learned my addr 172.21.15.116:0/2244753019 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:44.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92de559700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92d80a0f00 0x7f92d8091230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92ddd58700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92d80a12e0 0x7f92d814ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92ded5a700 1 -- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92d80a12e0 msgr2=0x7f92d814ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92ded5a700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92d80a12e0 0x7f92d814ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92ded5a700 1 -- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92d80a0f00 msgr2=0x7f92d8091230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92ded5a700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92d80a0f00 0x7f92d8091230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.028+0000 7f92ded5a700 1 -- 172.21.15.116:0/2244753019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d814e4f0 con 0x7f92d80c2df0 2020-11-20T12:16:44.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.032+0000 7f92ded5a700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92d80c2df0 0x7f92d8146f00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f92d000ca60 tx=0x7f92d0007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:44.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.032+0000 7f92c67fc700 1 -- 172.21.15.116:0/2244753019 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d000a770 con 0x7f92d80c2df0 2020-11-20T12:16:44.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.032+0000 7f92c67fc700 1 -- 172.21.15.116:0/2244753019 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d000a8d0 con 0x7f92d80c2df0 2020-11-20T12:16:44.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.032+0000 7f92df55b700 1 -- 172.21.15.116:0/2244753019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d814e790 con 0x7f92d80c2df0 2020-11-20T12:16:44.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.032+0000 7f92c67fc700 1 -- 172.21.15.116:0/2244753019 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d000ac50 con 0x7f92d80c2df0 2020-11-20T12:16:44.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.032+0000 7f92df55b700 1 -- 172.21.15.116:0/2244753019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d814f120 con 0x7f92d80c2df0 2020-11-20T12:16:44.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.032+0000 7f92df55b700 1 -- 172.21.15.116:0/2244753019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d8044f60 con 0x7f92d80c2df0 2020-11-20T12:16:44.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.032+0000 7f92c67fc700 1 -- 172.21.15.116:0/2244753019 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f92d0019030 con 0x7f92d80c2df0 2020-11-20T12:16:44.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.032+0000 7f92c67fc700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92b00557c0 0x7f92b0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.032+0000 7f92c67fc700 1 -- 172.21.15.116:0/2244753019 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f92d006ddb0 con 0x7f92d80c2df0 2020-11-20T12:16:44.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.036+0000 7f92de559700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92b00557c0 0x7f92b0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.036+0000 7f92de559700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92b00557c0 0x7f92b0057c80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f92d400da50 tx=0x7f92d4006f90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:44.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.040+0000 7f92c67fc700 1 -- 172.21.15.116:0/2244753019 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f92d0043d00 con 0x7f92d80c2df0 2020-11-20T12:16:44.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.200+0000 7f92df55b700 1 -- 172.21.15.116:0/2244753019 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92d8064080 con 0x7f92d80c2df0 2020-11-20T12:16:44.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.200+0000 7f92c67fc700 1 -- 172.21.15.116:0/2244753019 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+10 (secure 0 0 0) 0x7f92d0047320 con 0x7f92d80c2df0 2020-11-20T12:16:44.202 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 -- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92b00557c0 msgr2=0x7f92b0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92b00557c0 0x7f92b0057c80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f92d400da50 tx=0x7f92d4006f90).stop 2020-11-20T12:16:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 -- 172.21.15.116:0/2244753019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92d80c2df0 msgr2=0x7f92d8146f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92d80c2df0 0x7f92d8146f00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f92d000ca60 tx=0x7f92d0007f40).stop 2020-11-20T12:16:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 -- 172.21.15.116:0/2244753019 shutdown_connections 2020-11-20T12:16:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92b00557c0 0x7f92b0057c80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f92d80c2df0 0x7f92d8146f00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f92d80a0f00 0x7f92d8091230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 --2- 172.21.15.116:0/2244753019 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f92d80a12e0 0x7f92d814ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 -- 172.21.15.116:0/2244753019 >> 172.21.15.116:0/2244753019 conn(0x7f92d8060c30 msgr2=0x7f92d81490d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:44.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 -- 172.21.15.116:0/2244753019 shutdown_connections 2020-11-20T12:16:44.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.204+0000 7f92df55b700 1 -- 172.21.15.116:0/2244753019 wait complete. 2020-11-20T12:16:44.230 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:16:44.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.400+0000 7f5b8f3cf700 1 Processor -- start 2020-11-20T12:16:44.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.400+0000 7f5b8f3cf700 1 -- start start 2020-11-20T12:16:44.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b8f3cf700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5b880c1300 0x7f5b8815c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b8f3cf700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b880c1900 con 0x7f5b8814bc70 2020-11-20T12:16:44.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b8f3cf700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b88155530 con 0x7f5b8814c050 2020-11-20T12:16:44.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b8f3cf700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b881557d0 con 0x7f5b880c1300 2020-11-20T12:16:44.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b8d16b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5b8814bc70 0x7f5b880678e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53624/0 (socket says 172.21.15.116:53624) 2020-11-20T12:16:44.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b8d16b700 1 -- 172.21.15.116:0/2264866606 learned_addr learned my addr 172.21.15.116:0/2264866606 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:44.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b8d96c700 1 --2- 172.21.15.116:0/2264866606 >> v2:172.21.15.173:3300/0 conn(0x7f5b880c1300 0x7f5b8815c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1929537647 0 0) 0x7f5b880c1900 con 0x7f5b8814bc70 2020-11-20T12:16:44.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b70003240 con 0x7f5b8814bc70 2020-11-20T12:16:44.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157392839 0 0) 0x7f5b88155530 con 0x7f5b8814c050 2020-11-20T12:16:44.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b880c1900 con 0x7f5b8814c050 2020-11-20T12:16:44.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2526937929 0 0) 0x7f5b70003240 con 0x7f5b8814bc70 2020-11-20T12:16:44.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 >> v2:172.21.15.173:3300/0 conn(0x7f5b880c1300 msgr2=0x7f5b8815c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:44.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 --2- 172.21.15.116:0/2264866606 >> v2:172.21.15.173:3300/0 conn(0x7f5b880c1300 0x7f5b8815c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 >> v1:172.21.15.173:6789/0 conn(0x7f5b8814c050 legacy=0x7f5b880c0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b88155a70 con 0x7f5b8814bc70 2020-11-20T12:16:44.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f5b840014e0 con 0x7f5b8814bc70 2020-11-20T12:16:44.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 >> v1:172.21.15.116:6789/0 conn(0x7f5b8814bc70 legacy=0x7f5b880678e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 --2- 172.21.15.116:0/2264866606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b70003910 0x7f5b70003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 --2- 172.21.15.116:0/2264866606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b700044e0 0x7f5b70004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 --2- 172.21.15.116:0/2264866606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b70004e50 0x7f5b7000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b881557d0 con 0x7f5b700044e0 2020-11-20T12:16:44.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7000bc20 con 0x7f5b70004e50 2020-11-20T12:16:44.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.404+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7000bd60 con 0x7f5b70003910 2020-11-20T12:16:44.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8c96a700 1 --2- 172.21.15.116:0/2264866606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b700044e0 0x7f5b70004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8d16b700 1 --2- 172.21.15.116:0/2264866606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b70003910 0x7f5b70003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8c96a700 1 -- 172.21.15.116:0/2264866606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b70003910 msgr2=0x7f5b70003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:44.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8c96a700 1 --2- 172.21.15.116:0/2264866606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b70003910 0x7f5b70003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8c96a700 1 -- 172.21.15.116:0/2264866606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b70004e50 msgr2=0x7f5b7000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:44.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8c96a700 1 --2- 172.21.15.116:0/2264866606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b70004e50 0x7f5b7000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:44.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8d96c700 1 -- 172.21.15.116:0/2264866606 reap_dead start 2020-11-20T12:16:44.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8c96a700 1 -- 172.21.15.116:0/2264866606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b70003240 con 0x7f5b700044e0 2020-11-20T12:16:44.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8d16b700 1 --2- 172.21.15.116:0/2264866606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b70003910 0x7f5b70003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:44.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8c96a700 1 --2- 172.21.15.116:0/2264866606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b700044e0 0x7f5b70004960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f5b8814c470 tx=0x7f5b78007b70).ready entity=mon.0 client_cookie=9529bf4e7af8dcbd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:44.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800c8f0 con 0x7f5b700044e0 2020-11-20T12:16:44.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7800ca50 con 0x7f5b700044e0 2020-11-20T12:16:44.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b77fff700 1 -- 172.21.15.116:0/2264866606 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7800cdd0 con 0x7f5b700044e0 2020-11-20T12:16:44.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/2264866606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b700044e0 msgr2=0x7f5b70004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8f3cf700 1 --2- 172.21.15.116:0/2264866606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b700044e0 0x7f5b70004960 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f5b8814c470 tx=0x7f5b78007b70).stop 2020-11-20T12:16:44.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/2264866606 shutdown_connections 2020-11-20T12:16:44.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8f3cf700 1 --2- 172.21.15.116:0/2264866606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b700044e0 0x7f5b70004960 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/2264866606 >> 172.21.15.116:0/2264866606 conn(0x7f5b88060170 msgr2=0x7f5b88060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:44.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/2264866606 shutdown_connections 2020-11-20T12:16:44.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.408+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/2264866606 wait complete. 2020-11-20T12:16:44.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.412+0000 7f5b8f3cf700 1 Processor -- start 2020-11-20T12:16:44.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.412+0000 7f5b8f3cf700 1 -- start start 2020-11-20T12:16:44.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.412+0000 7f5b8f3cf700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b700044e0 0x7f5b8814d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.412+0000 7f5b8f3cf700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b8814cf80 0x7f5b880c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.412+0000 7f5b8f3cf700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b8814d400 0x7f5b880a9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.412+0000 7f5b8f3cf700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880a74b0 con 0x7f5b8814cf80 2020-11-20T12:16:44.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.412+0000 7f5b8f3cf700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880a7750 con 0x7f5b8814d400 2020-11-20T12:16:44.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.412+0000 7f5b8f3cf700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880a79f0 con 0x7f5b700044e0 2020-11-20T12:16:44.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8c96a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b8814cf80 0x7f5b880c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8c96a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b8814cf80 0x7f5b880c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44204/0 (socket says 172.21.15.116:44204) 2020-11-20T12:16:44.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8c96a700 1 -- 172.21.15.116:0/4245418128 learned_addr learned my addr 172.21.15.116:0/4245418128 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:44.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8d16b700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b700044e0 0x7f5b8814d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8d96c700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b8814d400 0x7f5b880a9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8c96a700 1 -- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b700044e0 msgr2=0x7f5b8814d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8c96a700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b700044e0 0x7f5b8814d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8c96a700 1 -- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b8814d400 msgr2=0x7f5b880a9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8c96a700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b8814d400 0x7f5b880a9760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8c96a700 1 -- 172.21.15.116:0/4245418128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b880a7c90 con 0x7f5b8814cf80 2020-11-20T12:16:44.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8c96a700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b8814cf80 0x7f5b880c4320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f5b78012f90 tx=0x7f5b78015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:44.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b757fa700 1 -- 172.21.15.116:0/4245418128 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78001ea0 con 0x7f5b8814cf80 2020-11-20T12:16:44.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b757fa700 1 -- 172.21.15.116:0/4245418128 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b7800c3c0 con 0x7f5b8814cf80 2020-11-20T12:16:44.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/4245418128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b880a7f30 con 0x7f5b8814cf80 2020-11-20T12:16:44.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b757fa700 1 -- 172.21.15.116:0/4245418128 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b7801b790 con 0x7f5b8814cf80 2020-11-20T12:16:44.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/4245418128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b880c4810 con 0x7f5b8814cf80 2020-11-20T12:16:44.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.416+0000 7f5b757fa700 1 -- 172.21.15.116:0/4245418128 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5b78010070 con 0x7f5b8814cf80 2020-11-20T12:16:44.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.420+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/4245418128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b8805ca30 con 0x7f5b8814cf80 2020-11-20T12:16:44.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.420+0000 7f5b757fa700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5b600557c0 0x7f5b60057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.420+0000 7f5b757fa700 1 -- 172.21.15.116:0/4245418128 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5b7806f410 con 0x7f5b8814cf80 2020-11-20T12:16:44.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.420+0000 7f5b8d16b700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5b600557c0 0x7f5b60057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.420+0000 7f5b8d16b700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5b600557c0 0x7f5b60057c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f5b84000fb0 tx=0x7f5b84007040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:44.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.424+0000 7f5b757fa700 1 -- 172.21.15.116:0/4245418128 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5b78045220 con 0x7f5b8814cf80 2020-11-20T12:16:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.584+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/4245418128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5b88061e80 con 0x7f5b8814cf80 2020-11-20T12:16:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.584+0000 7f5b757fa700 1 -- 172.21.15.116:0/4245418128 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5b780170a0 con 0x7f5b8814cf80 2020-11-20T12:16:44.586 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:44.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5b600557c0 msgr2=0x7f5b60057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5b600557c0 0x7f5b60057c80 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f5b84000fb0 tx=0x7f5b84007040).stop 2020-11-20T12:16:44.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/4245418128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b8814cf80 msgr2=0x7f5b880c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b8814cf80 0x7f5b880c4320 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f5b78012f90 tx=0x7f5b78015000).stop 2020-11-20T12:16:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/4245418128 shutdown_connections 2020-11-20T12:16:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5b600557c0 0x7f5b60057c80 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5b8814d400 0x7f5b880a9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5b700044e0 0x7f5b8814d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 --2- 172.21.15.116:0/4245418128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5b8814cf80 0x7f5b880c4320 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/4245418128 >> 172.21.15.116:0/4245418128 conn(0x7f5b88060170 msgr2=0x7f5b8814ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/4245418128 shutdown_connections 2020-11-20T12:16:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.588+0000 7f5b8f3cf700 1 -- 172.21.15.116:0/4245418128 wait complete. 2020-11-20T12:16:44.615 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:16:44.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.788+0000 7f08e18e2700 1 Processor -- start 2020-11-20T12:16:44.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.788+0000 7f08e18e2700 1 -- start start 2020-11-20T12:16:44.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.792+0000 7f08e18e2700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f08dc14daf0 0x7f08dc14ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.792+0000 7f08e18e2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc065fb0 0x7f08dc066440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.792+0000 7f08e18e2700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08dc14ef60 con 0x7f08dc14d710 2020-11-20T12:16:44.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.792+0000 7f08e18e2700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08dc0c05d0 con 0x7f08dc14daf0 2020-11-20T12:16:44.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.792+0000 7f08e18e2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc0c0870 con 0x7f08dc065fb0 2020-11-20T12:16:44.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.792+0000 7f08daffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f08dc14d710 0x7f08dc0a31b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53644/0 (socket says 172.21.15.116:53644) 2020-11-20T12:16:44.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.792+0000 7f08daffd700 1 -- 172.21.15.116:0/2840564854 learned_addr learned my addr 172.21.15.116:0/2840564854 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:44.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.792+0000 7f08da7fc700 1 --2- 172.21.15.116:0/2840564854 >> v2:172.21.15.173:3300/0 conn(0x7f08dc14daf0 0x7f08dc14ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.792+0000 7f08db7fe700 1 --2- 172.21.15.116:0/2840564854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc065fb0 0x7f08dc066440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.796+0000 7f08d9ffb700 1 -- 172.21.15.116:0/2840564854 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030223985 0 0) 0x7f08dc14ef60 con 0x7f08dc14d710 2020-11-20T12:16:44.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.796+0000 7f08da7fc700 1 -- 172.21.15.116:0/2840564854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc065fb0 msgr2=0x7f08dc066440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:44.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.796+0000 7f08da7fc700 1 --2- 172.21.15.116:0/2840564854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc065fb0 0x7f08dc066440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.796+0000 7f08da7fc700 1 -- 172.21.15.116:0/2840564854 >> v1:172.21.15.116:6789/0 conn(0x7f08dc14d710 legacy=0x7f08dc0a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.796+0000 7f08da7fc700 1 -- 172.21.15.116:0/2840564854 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc14e090 con 0x7f08dc14daf0 2020-11-20T12:16:44.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.796+0000 7f08da7fc700 1 --2- 172.21.15.116:0/2840564854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc14daf0 0x7f08dc14ea70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f08d000b9a0 tx=0x7f08d0006df0).ready entity=mon.1 client_cookie=91b873ca02943768 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:44.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.796+0000 7f08d9ffb700 1 -- 172.21.15.116:0/2840564854 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0009840 con 0x7f08dc14daf0 2020-11-20T12:16:44.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.796+0000 7f08d9ffb700 1 -- 172.21.15.116:0/2840564854 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d00099a0 con 0x7f08dc14daf0 2020-11-20T12:16:44.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.796+0000 7f08d9ffb700 1 -- 172.21.15.116:0/2840564854 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0009d20 con 0x7f08dc14daf0 2020-11-20T12:16:44.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.800+0000 7f08e18e2700 1 -- 172.21.15.116:0/2840564854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc14daf0 msgr2=0x7f08dc14ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.800+0000 7f08e18e2700 1 --2- 172.21.15.116:0/2840564854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc14daf0 0x7f08dc14ea70 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f08d000b9a0 tx=0x7f08d0006df0).stop 2020-11-20T12:16:44.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.800+0000 7f08e18e2700 1 -- 172.21.15.116:0/2840564854 shutdown_connections 2020-11-20T12:16:44.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.800+0000 7f08e18e2700 1 --2- 172.21.15.116:0/2840564854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc065fb0 0x7f08dc066440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.800+0000 7f08e18e2700 1 --2- 172.21.15.116:0/2840564854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc14daf0 0x7f08dc14ea70 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.800+0000 7f08e18e2700 1 -- 172.21.15.116:0/2840564854 >> 172.21.15.116:0/2840564854 conn(0x7f08dc060c30 msgr2=0x7f08dc0c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:44.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.800+0000 7f08e18e2700 1 -- 172.21.15.116:0/2840564854 shutdown_connections 2020-11-20T12:16:44.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.800+0000 7f08e18e2700 1 -- 172.21.15.116:0/2840564854 wait complete. 2020-11-20T12:16:44.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.800+0000 7f08e18e2700 1 Processor -- start 2020-11-20T12:16:44.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.800+0000 7f08e18e2700 1 -- start start 2020-11-20T12:16:44.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08e18e2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08dc065fb0 0x7f08dc0beff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08e18e2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc14d710 0x7f08dc151610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08e18e2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc14daf0 0x7f08dc090e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08e18e2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc0bf970 con 0x7f08dc065fb0 2020-11-20T12:16:44.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08e18e2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc0bfc10 con 0x7f08dc14daf0 2020-11-20T12:16:44.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08e18e2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc0bfeb0 con 0x7f08dc14d710 2020-11-20T12:16:44.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08daffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08dc065fb0 0x7f08dc0beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08daffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08dc065fb0 0x7f08dc0beff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44216/0 (socket says 172.21.15.116:44216) 2020-11-20T12:16:44.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08daffd700 1 -- 172.21.15.116:0/4059060482 learned_addr learned my addr 172.21.15.116:0/4059060482 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:44.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08da7fc700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc14d710 0x7f08dc151610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08db7fe700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc14daf0 0x7f08dc090e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08daffd700 1 -- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc14d710 msgr2=0x7f08dc151610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08daffd700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc14d710 0x7f08dc151610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08daffd700 1 -- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc14daf0 msgr2=0x7f08dc090e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:44.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08daffd700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc14daf0 0x7f08dc090e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:44.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08daffd700 1 -- 172.21.15.116:0/4059060482 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc0c0150 con 0x7f08dc065fb0 2020-11-20T12:16:44.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08da7fc700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc14d710 0x7f08dc151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:44.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08daffd700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08dc065fb0 0x7f08dc0beff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f08cc00c720 tx=0x7f08cc007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:44.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08c2ffd700 1 -- 172.21.15.116:0/4059060482 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc0072e0 con 0x7f08dc065fb0 2020-11-20T12:16:44.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08c2ffd700 1 -- 172.21.15.116:0/4059060482 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08cc007440 con 0x7f08dc065fb0 2020-11-20T12:16:44.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08e18e2700 1 -- 172.21.15.116:0/4059060482 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc08f1d0 con 0x7f08dc065fb0 2020-11-20T12:16:44.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08c2ffd700 1 -- 172.21.15.116:0/4059060482 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08cc005800 con 0x7f08dc065fb0 2020-11-20T12:16:44.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.804+0000 7f08e18e2700 1 -- 172.21.15.116:0/4059060482 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc151b00 con 0x7f08dc065fb0 2020-11-20T12:16:44.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.808+0000 7f08e18e2700 1 -- 172.21.15.116:0/4059060482 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc044f60 con 0x7f08dc065fb0 2020-11-20T12:16:44.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.808+0000 7f08c2ffd700 1 -- 172.21.15.116:0/4059060482 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f08cc01a060 con 0x7f08dc065fb0 2020-11-20T12:16:44.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.812+0000 7f08c2ffd700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08b80557c0 0x7f08b8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:44.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.812+0000 7f08c2ffd700 1 -- 172.21.15.116:0/4059060482 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f08cc06df20 con 0x7f08dc065fb0 2020-11-20T12:16:44.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.812+0000 7f08da7fc700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08b80557c0 0x7f08b8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:44.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.812+0000 7f08da7fc700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08b80557c0 0x7f08b8057c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f08d0006e30 tx=0x7f08d0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:44.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:44.816+0000 7f08c2ffd700 1 -- 172.21.15.116:0/4059060482 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f08cc043dc0 con 0x7f08dc065fb0 2020-11-20T12:16:45.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.028+0000 7f08e18e2700 1 -- 172.21.15.116:0/4059060482 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08dc08f360 con 0x7f08dc065fb0 2020-11-20T12:16:45.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.028+0000 7f08c2ffd700 1 -- 172.21.15.116:0/4059060482 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+2846 (secure 0 0 0) 0x7f08cc018030 con 0x7f08dc065fb0 2020-11-20T12:16:45.030 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:45.030 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4229,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/2968741298","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":2968741298},{"type":"v1","addr":"172.21.15.116:6835","nonce":2968741298}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4235,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1214743116","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1214743116},{"type":"v1","addr":"172.21.15.173:6836","nonce":1214743116}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4},{"gid":4263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1424291148","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1424291148},{"type":"v1","addr":"172.21.15.173:6837","nonce":1424291148}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":4}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:00.619934+0000","modified":"2020-11-20T12:16:02.636079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4245},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4245":{"gid":4245,"name":"a","rank":0,"incarnation":3,"state":"up:active","state_seq":2,"addr":"172.21.15.116:6834/3819756176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3819756176},{"type":"v1","addr":"172.21.15.116:6834","nonce":3819756176}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2020-11-20T12:16:45.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 -- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08b80557c0 msgr2=0x7f08b8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08b80557c0 0x7f08b8057c80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f08d0006e30 tx=0x7f08d0015000).stop 2020-11-20T12:16:45.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 -- 172.21.15.116:0/4059060482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08dc065fb0 msgr2=0x7f08dc0beff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08dc065fb0 0x7f08dc0beff0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f08cc00c720 tx=0x7f08cc007ac0).stop 2020-11-20T12:16:45.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 -- 172.21.15.116:0/4059060482 shutdown_connections 2020-11-20T12:16:45.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08b80557c0 0x7f08b8057c80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08dc14daf0 0x7f08dc090e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08dc065fb0 0x7f08dc0beff0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 --2- 172.21.15.116:0/4059060482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08dc14d710 0x7f08dc151610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 -- 172.21.15.116:0/4059060482 >> 172.21.15.116:0/4059060482 conn(0x7f08dc060c30 msgr2=0x7f08dc0a5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:45.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 -- 172.21.15.116:0/4059060482 shutdown_connections 2020-11-20T12:16:45.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.032+0000 7f08e18e2700 1 -- 172.21.15.116:0/4059060482 wait complete. 2020-11-20T12:16:45.039 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 6 2020-11-20T12:16:45.060 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:45.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.236+0000 7f15eac72700 1 Processor -- start 2020-11-20T12:16:45.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.236+0000 7f15eac72700 1 -- start start 2020-11-20T12:16:45.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15eac72700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40657c0 0x7f15e40c2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15eac72700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f15e40c0510 0x7f15e4064bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15eac72700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15e40c09a0 con 0x7f15e40c1eb0 2020-11-20T12:16:45.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15eac72700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15e40c2e20 con 0x7f15e40c0510 2020-11-20T12:16:45.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15eac72700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40c30c0 con 0x7f15e40657c0 2020-11-20T12:16:45.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e9c70700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f15e40c1eb0 0x7f15e414f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53658/0 (socket says 172.21.15.116:53658) 2020-11-20T12:16:45.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e9c70700 1 -- 172.21.15.116:0/2749527665 learned_addr learned my addr 172.21.15.116:0/2749527665 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:45.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15ea471700 1 --2- 172.21.15.116:0/2749527665 >> v2:172.21.15.116:3300/0 conn(0x7f15e40c0510 0x7f15e4064bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e946f700 1 --2- 172.21.15.116:0/2749527665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40657c0 0x7f15e40c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821168099 0 0) 0x7f15e40c09a0 con 0x7f15e40c1eb0 2020-11-20T12:16:45.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15cc003240 con 0x7f15e40c1eb0 2020-11-20T12:16:45.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3126182216 0 0) 0x7f15cc003240 con 0x7f15e40c1eb0 2020-11-20T12:16:45.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40657c0 msgr2=0x7f15e40c2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:45.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 --2- 172.21.15.116:0/2749527665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40657c0 0x7f15e40c2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 >> v2:172.21.15.116:3300/0 conn(0x7f15e40c0510 msgr2=0x7f15e4064bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:45.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 --2- 172.21.15.116:0/2749527665 >> v2:172.21.15.116:3300/0 conn(0x7f15e40c0510 0x7f15e4064bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e40c3360 con 0x7f15e40c1eb0 2020-11-20T12:16:45.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f15e00014e0 con 0x7f15e40c1eb0 2020-11-20T12:16:45.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15ea471700 1 --2- 172.21.15.116:0/2749527665 >> v2:172.21.15.116:3300/0 conn(0x7f15e40c0510 0x7f15e4064bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:45.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 >> v1:172.21.15.116:6789/0 conn(0x7f15e40c1eb0 legacy=0x7f15e414f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 --2- 172.21.15.116:0/2749527665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40657c0 0x7f15cc003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 --2- 172.21.15.116:0/2749527665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15cc003c40 0x7f15cc0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 --2- 172.21.15.116:0/2749527665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15cc0049d0 0x7f15cc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40c2e20 con 0x7f15cc003c40 2020-11-20T12:16:45.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40c30c0 con 0x7f15cc0049d0 2020-11-20T12:16:45.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.240+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc004200 con 0x7f15e40657c0 2020-11-20T12:16:45.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15e946f700 1 --2- 172.21.15.116:0/2749527665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15cc003c40 0x7f15cc0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15ea471700 1 --2- 172.21.15.116:0/2749527665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15cc0049d0 0x7f15cc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15e946f700 1 -- 172.21.15.116:0/2749527665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40657c0 msgr2=0x7f15cc003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:45.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15e946f700 1 --2- 172.21.15.116:0/2749527665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40657c0 0x7f15cc003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:45.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15e946f700 1 -- 172.21.15.116:0/2749527665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15cc0049d0 msgr2=0x7f15cc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:45.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15e946f700 1 --2- 172.21.15.116:0/2749527665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15cc0049d0 0x7f15cc004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15e946f700 1 -- 172.21.15.116:0/2749527665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc003240 con 0x7f15cc003c40 2020-11-20T12:16:45.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15e946f700 1 --2- 172.21.15.116:0/2749527665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15cc003c40 0x7f15cc0044e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f15d400bc00 tx=0x7f15d4006df0).ready entity=mon.0 client_cookie=4abdc557b6d8250e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:45.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d400b310 con 0x7f15cc003c40 2020-11-20T12:16:45.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d400eae0 con 0x7f15cc003c40 2020-11-20T12:16:45.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15e8c6e700 1 -- 172.21.15.116:0/2749527665 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d400ee60 con 0x7f15cc003c40 2020-11-20T12:16:45.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15eac72700 1 -- 172.21.15.116:0/2749527665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15cc003c40 msgr2=0x7f15cc0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15eac72700 1 --2- 172.21.15.116:0/2749527665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15cc003c40 0x7f15cc0044e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f15d400bc00 tx=0x7f15d4006df0).stop 2020-11-20T12:16:45.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15ea471700 1 -- 172.21.15.116:0/2749527665 reap_dead start 2020-11-20T12:16:45.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15eac72700 1 -- 172.21.15.116:0/2749527665 shutdown_connections 2020-11-20T12:16:45.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15eac72700 1 -- 172.21.15.116:0/2749527665 >> 172.21.15.116:0/2749527665 conn(0x7f15e4061280 msgr2=0x7f15e4061660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:45.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15eac72700 1 -- 172.21.15.116:0/2749527665 shutdown_connections 2020-11-20T12:16:45.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.244+0000 7f15eac72700 1 -- 172.21.15.116:0/2749527665 wait complete. 2020-11-20T12:16:45.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.248+0000 7f15eac72700 1 Processor -- start 2020-11-20T12:16:45.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.248+0000 7f15eac72700 1 -- start start 2020-11-20T12:16:45.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.248+0000 7f15eac72700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15cc003c40 0x7f15e4160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.248+0000 7f15eac72700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40a75a0 0x7f15e41506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15eac72700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40aa8b0 0x7f15e40aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15eac72700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e41480a0 con 0x7f15e40a75a0 2020-11-20T12:16:45.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15eac72700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e4150fc0 con 0x7f15cc003c40 2020-11-20T12:16:45.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15eac72700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e4151260 con 0x7f15e40aa8b0 2020-11-20T12:16:45.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15e946f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40a75a0 0x7f15e41506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15e946f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40a75a0 0x7f15e41506f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44238/0 (socket says 172.21.15.116:44238) 2020-11-20T12:16:45.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15e946f700 1 -- 172.21.15.116:0/3522794802 learned_addr learned my addr 172.21.15.116:0/3522794802 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:45.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15e9c70700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15cc003c40 0x7f15e4160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15ea471700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40aa8b0 0x7f15e40aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15e946f700 1 -- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40aa8b0 msgr2=0x7f15e40aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15e946f700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40aa8b0 0x7f15e40aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15e946f700 1 -- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15cc003c40 msgr2=0x7f15e4160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15e946f700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15cc003c40 0x7f15e4160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15e946f700 1 -- 172.21.15.116:0/3522794802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e4151500 con 0x7f15e40a75a0 2020-11-20T12:16:45.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15e9c70700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15cc003c40 0x7f15e4160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:45.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15e946f700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40a75a0 0x7f15e41506f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f15cc005460 tx=0x7f15d4019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:45.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15d1ffb700 1 -- 172.21.15.116:0/3522794802 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d400b390 con 0x7f15e40a75a0 2020-11-20T12:16:45.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15d1ffb700 1 -- 172.21.15.116:0/3522794802 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d401e630 con 0x7f15e40a75a0 2020-11-20T12:16:45.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15d1ffb700 1 -- 172.21.15.116:0/3522794802 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d401e7a0 con 0x7f15e40a75a0 2020-11-20T12:16:45.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15eac72700 1 -- 172.21.15.116:0/3522794802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15e40bce10 con 0x7f15e40a75a0 2020-11-20T12:16:45.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.252+0000 7f15eac72700 1 -- 172.21.15.116:0/3522794802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15e4150be0 con 0x7f15e40a75a0 2020-11-20T12:16:45.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.256+0000 7f15eac72700 1 -- 172.21.15.116:0/3522794802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15e40c3540 con 0x7f15e40a75a0 2020-11-20T12:16:45.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.256+0000 7f15d1ffb700 1 -- 172.21.15.116:0/3522794802 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f15d401ebd0 con 0x7f15e40a75a0 2020-11-20T12:16:45.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.256+0000 7f15d1ffb700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15bc0557c0 0x7f15bc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.256+0000 7f15d1ffb700 1 -- 172.21.15.116:0/3522794802 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f15d4070bf0 con 0x7f15e40a75a0 2020-11-20T12:16:45.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.256+0000 7f15e9c70700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15bc0557c0 0x7f15bc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.256+0000 7f15e9c70700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15bc0557c0 0x7f15bc057c80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f15e0001e90 tx=0x7f15e0001160).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:45.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.260+0000 7f15d1ffb700 1 -- 172.21.15.116:0/3522794802 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f15d4046b10 con 0x7f15e40a75a0 2020-11-20T12:16:45.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.420+0000 7f15eac72700 1 -- 172.21.15.116:0/3522794802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15e4044f60 con 0x7f15e40a75a0 2020-11-20T12:16:45.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.420+0000 7f15d1ffb700 1 -- 172.21.15.116:0/3522794802 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f15d401ee60 con 0x7f15e40a75a0 2020-11-20T12:16:45.424 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:45.424 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":23,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:01.624522+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:15:59.065285+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2020-11-20T12:15:59.602311+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:45.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 -- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15bc0557c0 msgr2=0x7f15bc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15bc0557c0 0x7f15bc057c80 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f15e0001e90 tx=0x7f15e0001160).stop 2020-11-20T12:16:45.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 -- 172.21.15.116:0/3522794802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40a75a0 msgr2=0x7f15e41506f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40a75a0 0x7f15e41506f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f15cc005460 tx=0x7f15d4019000).stop 2020-11-20T12:16:45.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 -- 172.21.15.116:0/3522794802 shutdown_connections 2020-11-20T12:16:45.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15bc0557c0 0x7f15bc057c80 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e40aa8b0 0x7f15e40aad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15cc003c40 0x7f15e4160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 --2- 172.21.15.116:0/3522794802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40a75a0 0x7f15e41506f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 -- 172.21.15.116:0/3522794802 >> 172.21.15.116:0/3522794802 conn(0x7f15e4061280 msgr2=0x7f15e4151b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:45.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 -- 172.21.15.116:0/3522794802 shutdown_connections 2020-11-20T12:16:45.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.424+0000 7f15eac72700 1 -- 172.21.15.116:0/3522794802 wait complete. 2020-11-20T12:16:45.450 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-20T12:16:45.451 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-20T12:16:45.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.632+0000 7fb7f6910700 1 Processor -- start 2020-11-20T12:16:45.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.632+0000 7fb7f6910700 1 -- start start 2020-11-20T12:16:45.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.632+0000 7fb7f6910700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f013f5b0 0x7fb7f00c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.632+0000 7fb7f6910700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb7f00c44a0 0x7fb7f0066090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.632+0000 7fb7f6910700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7f00a5680 con 0x7fb7f013f1d0 2020-11-20T12:16:45.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.632+0000 7fb7f6910700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7f00a42e0 con 0x7fb7f00c44a0 2020-11-20T12:16:45.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.632+0000 7fb7f6910700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f00a58e0 con 0x7fb7f013f5b0 2020-11-20T12:16:45.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f610f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb7f00c44a0 0x7fb7f0066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f510d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f013f5b0 0x7fb7f00c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f610f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb7f00c44a0 0x7fb7f0066090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:43298/0 (socket says 172.21.15.116:43298) 2020-11-20T12:16:45.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f610f700 1 -- 172.21.15.116:0/3526078820 learned_addr learned my addr 172.21.15.116:0/3526078820 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:45.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f590e700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fb7f013f1d0 0x7fb7f00a3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:49908/0 (socket says 172.21.15.116:49908) 2020-11-20T12:16:45.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f490c700 1 -- 172.21.15.116:0/3526078820 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979883726 0 0) 0x7fb7f00a5680 con 0x7fb7f013f1d0 2020-11-20T12:16:45.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f610f700 1 -- 172.21.15.116:0/3526078820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f013f5b0 msgr2=0x7fb7f00c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:45.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f610f700 1 --2- 172.21.15.116:0/3526078820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f013f5b0 0x7fb7f00c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f610f700 1 -- 172.21.15.116:0/3526078820 >> v1:172.21.15.173:6789/0 conn(0x7fb7f013f1d0 legacy=0x7fb7f00a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f610f700 1 -- 172.21.15.116:0/3526078820 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f00a5b80 con 0x7fb7f00c44a0 2020-11-20T12:16:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f510d700 1 --2- 172.21.15.116:0/3526078820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f013f5b0 0x7fb7f00c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f610f700 1 --2- 172.21.15.116:0/3526078820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7f00c44a0 0x7fb7f0066090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fb7e8009d50 tx=0x7fb7e8009290).ready entity=mon.1 client_cookie=f044b4709c36894 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f490c700 1 -- 172.21.15.116:0/3526078820 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e800bbc0 con 0x7fb7f00c44a0 2020-11-20T12:16:45.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f490c700 1 -- 172.21.15.116:0/3526078820 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e800bd20 con 0x7fb7f00c44a0 2020-11-20T12:16:45.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f490c700 1 -- 172.21.15.116:0/3526078820 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8011610 con 0x7fb7f00c44a0 2020-11-20T12:16:45.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f6910700 1 -- 172.21.15.116:0/3526078820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7f00c44a0 msgr2=0x7fb7f0066090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.636+0000 7fb7f6910700 1 --2- 172.21.15.116:0/3526078820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7f00c44a0 0x7fb7f0066090 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fb7e8009d50 tx=0x7fb7e8009290).stop 2020-11-20T12:16:45.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.640+0000 7fb7f6910700 1 -- 172.21.15.116:0/3526078820 shutdown_connections 2020-11-20T12:16:45.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.640+0000 7fb7f6910700 1 --2- 172.21.15.116:0/3526078820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7f00c44a0 0x7fb7f0066090 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.640+0000 7fb7f6910700 1 --2- 172.21.15.116:0/3526078820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f013f5b0 0x7fb7f00c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.640+0000 7fb7f6910700 1 -- 172.21.15.116:0/3526078820 >> 172.21.15.116:0/3526078820 conn(0x7fb7f0062520 msgr2=0x7fb7f0062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:45.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.640+0000 7fb7f6910700 1 -- 172.21.15.116:0/3526078820 shutdown_connections 2020-11-20T12:16:45.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.640+0000 7fb7f6910700 1 -- 172.21.15.116:0/3526078820 wait complete. 2020-11-20T12:16:45.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.640+0000 7fb7f6910700 1 Processor -- start 2020-11-20T12:16:45.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.640+0000 7fb7f6910700 1 -- start start 2020-11-20T12:16:45.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f6910700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f00c44a0 0x7fb7f009f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f6910700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7f013f1d0 0x7fb7f013d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f6910700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f013f5b0 0x7fb7f013d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f6910700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0165020 con 0x7fb7f013f5b0 2020-11-20T12:16:45.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f6910700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f0165210 con 0x7fb7f013f1d0 2020-11-20T12:16:45.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f6910700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f01654b0 con 0x7fb7f00c44a0 2020-11-20T12:16:45.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f610f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f013f5b0 0x7fb7f013d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f610f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f013f5b0 0x7fb7f013d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44254/0 (socket says 172.21.15.116:44254) 2020-11-20T12:16:45.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f610f700 1 -- 172.21.15.116:0/3624510574 learned_addr learned my addr 172.21.15.116:0/3624510574 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:45.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f590e700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f00c44a0 0x7fb7f009f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f510d700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7f013f1d0 0x7fb7f013d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f610f700 1 -- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f00c44a0 msgr2=0x7fb7f009f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f610f700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f00c44a0 0x7fb7f009f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f610f700 1 -- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7f013f1d0 msgr2=0x7fb7f013d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:45.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f610f700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7f013f1d0 0x7fb7f013d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:45.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f610f700 1 -- 172.21.15.116:0/3624510574 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7f0165750 con 0x7fb7f013f5b0 2020-11-20T12:16:45.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f590e700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f00c44a0 0x7fb7f009f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:45.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f610f700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f013f5b0 0x7fb7f013d5d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fb7e8009fd0 tx=0x7fb7e8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:45.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7ddffb700 1 -- 172.21.15.116:0/3624510574 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8011e70 con 0x7fb7f013f5b0 2020-11-20T12:16:45.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7ddffb700 1 -- 172.21.15.116:0/3624510574 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7e800be90 con 0x7fb7f013f5b0 2020-11-20T12:16:45.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f6910700 1 -- 172.21.15.116:0/3624510574 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f01659f0 con 0x7fb7f013f5b0 2020-11-20T12:16:45.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7ddffb700 1 -- 172.21.15.116:0/3624510574 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7e8010da0 con 0x7fb7f013f5b0 2020-11-20T12:16:45.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.644+0000 7fb7f6910700 1 -- 172.21.15.116:0/3624510574 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f009fa00 con 0x7fb7f013f5b0 2020-11-20T12:16:45.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.648+0000 7fb7f6910700 1 -- 172.21.15.116:0/3624510574 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f0165b80 con 0x7fb7f013f5b0 2020-11-20T12:16:45.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.648+0000 7fb7ddffb700 1 -- 172.21.15.116:0/3624510574 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb7e8017070 con 0x7fb7f013f5b0 2020-11-20T12:16:45.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.648+0000 7fb7ddffb700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7c80557c0 0x7fb7c8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:45.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.648+0000 7fb7ddffb700 1 -- 172.21.15.116:0/3624510574 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb7e8070690 con 0x7fb7f013f5b0 2020-11-20T12:16:45.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.648+0000 7fb7f590e700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7c80557c0 0x7fb7c8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:45.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.652+0000 7fb7f590e700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7c80557c0 0x7fb7c8057c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fb7ec0011a0 tx=0x7fb7ec007120).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:45.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.656+0000 7fb7ddffb700 1 -- 172.21.15.116:0/3624510574 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb7e80465b0 con 0x7fb7f013f5b0 2020-11-20T12:16:45.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:45.864+0000 7fb7f6910700 1 -- 172.21.15.116:0/3624510574 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb7f0044f60 con 0x7fb7f013f5b0 2020-11-20T12:16:46.296 INFO:tasks.ceph.mon.a.smithi116.stderr:2020-11-20T12:16:46.296+0000 7fc111d1b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-20T12:16:46.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.304+0000 7fb7ddffb700 1 -- 172.21.15.116:0/3624510574 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb7e8015070 con 0x7fb7f013f5b0 2020-11-20T12:16:46.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 -- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7c80557c0 msgr2=0x7fb7c8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:46.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7c80557c0 0x7fb7c8057c80 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fb7ec0011a0 tx=0x7fb7ec007120).stop 2020-11-20T12:16:46.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 -- 172.21.15.116:0/3624510574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f013f5b0 msgr2=0x7fb7f013d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:46.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f013f5b0 0x7fb7f013d5d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fb7e8009fd0 tx=0x7fb7e8016000).stop 2020-11-20T12:16:46.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 -- 172.21.15.116:0/3624510574 shutdown_connections 2020-11-20T12:16:46.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7c80557c0 0x7fb7c8057c80 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:46.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb7f013f5b0 0x7fb7f013d5d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:46.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7f00c44a0 0x7fb7f009f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:46.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 --2- 172.21.15.116:0/3624510574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb7f013f1d0 0x7fb7f013d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:46.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 -- 172.21.15.116:0/3624510574 >> 172.21.15.116:0/3624510574 conn(0x7fb7f0062520 msgr2=0x7fb7f0158aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:46.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 -- 172.21.15.116:0/3624510574 shutdown_connections 2020-11-20T12:16:46.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.308+0000 7fb7f6910700 1 -- 172.21.15.116:0/3624510574 wait complete. 2020-11-20T12:16:46.313 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-20T12:16:46.313 INFO:tasks.ceph.mds.a.smithi116.stderr: -2016> 2020-11-20T12:16:02.176+0000 7f3e85091700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:16:46.334 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-20T12:16:46.356 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:16:46.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.512+0000 7f2aaa17f700 1 Processor -- start 2020-11-20T12:16:46.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.512+0000 7f2aaa17f700 1 -- start start 2020-11-20T12:16:46.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aaa17f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2aa40c03f0 0x7f2aa4156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:46.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aaa17f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2aa40a28c0 0x7f2aa4066570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:46.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aaa17f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aa40a38c0 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aaa17f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa4156e30 con 0x7f2aa40c03f0 2020-11-20T12:16:46.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aaa17f700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aa40a2e40 con 0x7f2aa40a28c0 2020-11-20T12:16:46.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aa897c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2aa40c1bb0 0x7f2aa40677c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53706/0 (socket says 172.21.15.116:53706) 2020-11-20T12:16:46.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aa917d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2aa40c03f0 0x7f2aa4156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:46.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aa897c700 1 -- 172.21.15.116:0/2753081903 learned_addr learned my addr 172.21.15.116:0/2753081903 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:46.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aa997e700 1 --2- 172.21.15.116:0/2753081903 >> v2:172.21.15.173:3300/0 conn(0x7f2aa40a28c0 0x7f2aa4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:46.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2a93fff700 1 -- 172.21.15.116:0/2753081903 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203880889 0 0) 0x7f2aa40a38c0 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2a93fff700 1 -- 172.21.15.116:0/2753081903 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a8c003240 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aa917d700 1 -- 172.21.15.116:0/2753081903 >> v2:172.21.15.173:3300/0 conn(0x7f2aa40a28c0 msgr2=0x7f2aa4066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:46.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aa917d700 1 --2- 172.21.15.116:0/2753081903 >> v2:172.21.15.173:3300/0 conn(0x7f2aa40a28c0 0x7f2aa4066570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:46.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aa917d700 1 -- 172.21.15.116:0/2753081903 >> v1:172.21.15.116:6789/0 conn(0x7f2aa40c1bb0 legacy=0x7f2aa40677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:46.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aa917d700 1 -- 172.21.15.116:0/2753081903 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa40a30e0 con 0x7f2aa40c03f0 2020-11-20T12:16:46.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aa997e700 1 --2- 172.21.15.116:0/2753081903 >> v2:172.21.15.173:3300/0 conn(0x7f2aa40a28c0 0x7f2aa4066570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:46.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aa917d700 1 --2- 172.21.15.116:0/2753081903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa40c03f0 0x7f2aa4156940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f2aa40c49e0 tx=0x7f2aa0006df0).ready entity=mon.0 client_cookie=1254c2b92c2548ee server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:46.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2a93fff700 1 -- 172.21.15.116:0/2753081903 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0009840 con 0x7f2aa40c03f0 2020-11-20T12:16:46.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2a93fff700 1 -- 172.21.15.116:0/2753081903 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa00099a0 con 0x7f2aa40c03f0 2020-11-20T12:16:46.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2a93fff700 1 -- 172.21.15.116:0/2753081903 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0009d20 con 0x7f2aa40c03f0 2020-11-20T12:16:46.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aaa17f700 1 -- 172.21.15.116:0/2753081903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa40c03f0 msgr2=0x7f2aa4156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:46.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aaa17f700 1 --2- 172.21.15.116:0/2753081903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa40c03f0 0x7f2aa4156940 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f2aa40c49e0 tx=0x7f2aa0006df0).stop 2020-11-20T12:16:46.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aaa17f700 1 -- 172.21.15.116:0/2753081903 shutdown_connections 2020-11-20T12:16:46.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aaa17f700 1 --2- 172.21.15.116:0/2753081903 >> v2:172.21.15.173:3300/0 conn(0x7f2aa40a28c0 0x7f2aa4066570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:46.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aaa17f700 1 --2- 172.21.15.116:0/2753081903 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa40c03f0 0x7f2aa4156940 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:46.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.516+0000 7f2aaa17f700 1 -- 172.21.15.116:0/2753081903 >> 172.21.15.116:0/2753081903 conn(0x7f2aa4060c30 msgr2=0x7f2aa40c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:46.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.520+0000 7f2aaa17f700 1 -- 172.21.15.116:0/2753081903 shutdown_connections 2020-11-20T12:16:46.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.520+0000 7f2aaa17f700 1 -- 172.21.15.116:0/2753081903 wait complete. 2020-11-20T12:16:46.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.520+0000 7f2aaa17f700 1 Processor -- start 2020-11-20T12:16:46.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.520+0000 7f2aaa17f700 1 -- start start 2020-11-20T12:16:46.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aaa17f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aa40a28c0 0x7f2aa4169540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:46.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aaa17f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa40c1bb0 0x7f2aa40a97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:46.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aaa17f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aa40a9cc0 0x7f2aa40a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:46.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aaa17f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa4168bd0 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aaa17f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa408fa40 con 0x7f2aa40a28c0 2020-11-20T12:16:46.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aaa17f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa40a7a70 con 0x7f2aa40a9cc0 2020-11-20T12:16:46.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa897c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa40c1bb0 0x7f2aa40a97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:46.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa897c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa40c1bb0 0x7f2aa40a97d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44278/0 (socket says 172.21.15.116:44278) 2020-11-20T12:16:46.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa897c700 1 -- 172.21.15.116:0/3939386031 learned_addr learned my addr 172.21.15.116:0/3939386031 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:46.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa917d700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aa40a28c0 0x7f2aa4169540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:46.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa997e700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aa40a9cc0 0x7f2aa40a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:46.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa897c700 1 -- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aa40a9cc0 msgr2=0x7f2aa40a6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:46.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa897c700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aa40a9cc0 0x7f2aa40a6f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:46.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa897c700 1 -- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aa40a28c0 msgr2=0x7f2aa4169540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:46.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa897c700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aa40a28c0 0x7f2aa4169540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:46.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa897c700 1 -- 172.21.15.116:0/3939386031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa40a7d10 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa917d700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aa40a28c0 0x7f2aa4169540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:46.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aa897c700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa40c1bb0 0x7f2aa40a97d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f2a9400c6e0 tx=0x7f2a94007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:46.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2a917fa700 1 -- 172.21.15.116:0/3939386031 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a940072e0 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2a917fa700 1 -- 172.21.15.116:0/3939386031 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a94007440 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aaa17f700 1 -- 172.21.15.116:0/3939386031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aa4168e00 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2a917fa700 1 -- 172.21.15.116:0/3939386031 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a94005820 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.524+0000 7f2aaa17f700 1 -- 172.21.15.116:0/3939386031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aa4169a30 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.528+0000 7f2aaa17f700 1 -- 172.21.15.116:0/3939386031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aa4044f60 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.528+0000 7f2a917fa700 1 -- 172.21.15.116:0/3939386031 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2a94010070 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.528+0000 7f2a917fa700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a7c0557f0 0x7f2a7c057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:46.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.528+0000 7f2a917fa700 1 -- 172.21.15.116:0/3939386031 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2a9406dc70 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.528+0000 7f2aa917d700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a7c0557f0 0x7f2a7c057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:46.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.532+0000 7f2aa917d700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a7c0557f0 0x7f2a7c057cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f2aa4063c40 tx=0x7f2aa0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:46.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.536+0000 7f2a917fa700 1 -- 172.21.15.116:0/3939386031 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2a94043ac0 con 0x7f2aa40c1bb0 2020-11-20T12:16:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:46.744+0000 7f2aaa17f700 1 -- 172.21.15.116:0/3939386031 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2aa4064080 con 0x7f2aa40c1bb0 2020-11-20T12:16:47.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.324+0000 7f2a917fa700 1 -- 172.21.15.116:0/3939386031 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2a940470e0 con 0x7f2aa40c1bb0 2020-11-20T12:16:47.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 -- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a7c0557f0 msgr2=0x7f2a7c057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:47.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a7c0557f0 0x7f2a7c057cb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f2aa4063c40 tx=0x7f2aa0015000).stop 2020-11-20T12:16:47.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 -- 172.21.15.116:0/3939386031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa40c1bb0 msgr2=0x7f2aa40a97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:47.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa40c1bb0 0x7f2aa40a97d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f2a9400c6e0 tx=0x7f2a94007ac0).stop 2020-11-20T12:16:47.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 -- 172.21.15.116:0/3939386031 shutdown_connections 2020-11-20T12:16:47.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a7c0557f0 0x7f2a7c057cb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:47.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2aa40a9cc0 0x7f2aa40a6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:47.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2aa40a28c0 0x7f2aa4169540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:47.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 --2- 172.21.15.116:0/3939386031 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2aa40c1bb0 0x7f2aa40a97d0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:47.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 -- 172.21.15.116:0/3939386031 >> 172.21.15.116:0/3939386031 conn(0x7f2aa4060c30 msgr2=0x7f2aa4155460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:47.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 -- 172.21.15.116:0/3939386031 shutdown_connections 2020-11-20T12:16:47.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.328+0000 7f2aaa17f700 1 -- 172.21.15.116:0/3939386031 wait complete. 2020-11-20T12:16:47.354 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-20T12:16:47.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.584+0000 7fb2c19be700 1 Processor -- start 2020-11-20T12:16:47.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.584+0000 7fb2c19be700 1 -- start start 2020-11-20T12:16:47.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2c19be700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb2bc0c3af0 0x7fb2bc15c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:47.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2c19be700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc0c4b70 con 0x7fb2bc0be350 2020-11-20T12:16:47.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2c19be700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc0c40f0 con 0x7fb2bc062620 2020-11-20T12:16:47.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2c19be700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc151b00 con 0x7fb2bc0c3af0 2020-11-20T12:16:47.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2c09bc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb2bc0be350 0x7fb2bc0c4e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53718/0 (socket says 172.21.15.116:53718) 2020-11-20T12:16:47.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2c09bc700 1 -- 172.21.15.116:0/1790941212 learned_addr learned my addr 172.21.15.116:0/1790941212 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:47.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2c11bd700 1 --2- 172.21.15.116:0/1790941212 >> v2:172.21.15.173:3300/0 conn(0x7fb2bc0c3af0 0x7fb2bc15c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:47.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790592310 0 0) 0x7fb2bc0c4b70 con 0x7fb2bc0be350 2020-11-20T12:16:47.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a8003240 con 0x7fb2bc0be350 2020-11-20T12:16:47.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2861113156 0 0) 0x7fb2a8003240 con 0x7fb2bc0be350 2020-11-20T12:16:47.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 >> v2:172.21.15.173:3300/0 conn(0x7fb2bc0c3af0 msgr2=0x7fb2bc15c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:47.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 --2- 172.21.15.116:0/1790941212 >> v2:172.21.15.173:3300/0 conn(0x7fb2bc0c3af0 0x7fb2bc15c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:47.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 >> v1:172.21.15.173:6789/0 conn(0x7fb2bc062620 legacy=0x7fb2bc0c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:47.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0c4390 con 0x7fb2bc0be350 2020-11-20T12:16:47.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fb2b8004c60 con 0x7fb2bc0be350 2020-11-20T12:16:47.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 >> v1:172.21.15.116:6789/0 conn(0x7fb2bc0be350 legacy=0x7fb2bc0c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:47.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a8003730 0x7fb2a8003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:47.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a80044e0 0x7fb2a8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:47.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a8004e50 0x7fb2a800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:47.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc151b00 con 0x7fb2a80044e0 2020-11-20T12:16:47.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a8004240 con 0x7fb2a8003730 2020-11-20T12:16:47.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a800bc20 con 0x7fb2a8004e50 2020-11-20T12:16:47.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2b3fff700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a80044e0 0x7fb2a8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:47.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.588+0000 7fb2c09bc700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a8003730 0x7fb2a8003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:47.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2c11bd700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a8004e50 0x7fb2a800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:47.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2b3fff700 1 -- 172.21.15.116:0/1790941212 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a8004e50 msgr2=0x7fb2a800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:47.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2b3fff700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a8004e50 0x7fb2a800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:47.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2b3fff700 1 -- 172.21.15.116:0/1790941212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a8003730 msgr2=0x7fb2a8003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:47.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2b3fff700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a8003730 0x7fb2a8003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:47.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2b3fff700 1 -- 172.21.15.116:0/1790941212 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a8003240 con 0x7fb2a80044e0 2020-11-20T12:16:47.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2c11bd700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2a8004e50 0x7fb2a800b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:47.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2c11bd700 1 -- 172.21.15.116:0/1790941212 reap_dead start 2020-11-20T12:16:47.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2b3fff700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a80044e0 0x7fb2a8004960 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fb2ac002860 tx=0x7fb2ac007b90).ready entity=mon.0 client_cookie=3acc4e6d8b3173c3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:47.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac007020 con 0x7fb2a80044e0 2020-11-20T12:16:47.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac0071a0 con 0x7fb2a80044e0 2020-11-20T12:16:47.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2b37fe700 1 -- 172.21.15.116:0/1790941212 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac00c7f0 con 0x7fb2a80044e0 2020-11-20T12:16:47.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2c19be700 1 -- 172.21.15.116:0/1790941212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a80044e0 msgr2=0x7fb2a8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:47.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2c19be700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a80044e0 0x7fb2a8004960 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7fb2ac002860 tx=0x7fb2ac007b90).stop 2020-11-20T12:16:47.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2c19be700 1 -- 172.21.15.116:0/1790941212 shutdown_connections 2020-11-20T12:16:47.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2c19be700 1 --2- 172.21.15.116:0/1790941212 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2a80044e0 0x7fb2a8004960 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:47.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2c19be700 1 -- 172.21.15.116:0/1790941212 >> 172.21.15.116:0/1790941212 conn(0x7fb2bc091900 msgr2=0x7fb2bc07ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:47.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2c19be700 1 -- 172.21.15.116:0/1790941212 shutdown_connections 2020-11-20T12:16:47.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.592+0000 7fb2c19be700 1 -- 172.21.15.116:0/1790941212 wait complete. 2020-11-20T12:16:47.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.596+0000 7fb2c19be700 1 Processor -- start 2020-11-20T12:16:47.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.596+0000 7fb2c19be700 1 -- start start 2020-11-20T12:16:47.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.596+0000 7fb2c19be700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a80044e0 0x7fb2bc0a8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:47.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.596+0000 7fb2c19be700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2bc152eb0 0x7fb2bc153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:47.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.596+0000 7fb2c19be700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2bc0aa500 0x7fb2bc0aa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:47.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2c19be700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc14fae0 con 0x7fb2bc152eb0 2020-11-20T12:16:47.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2c19be700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc153820 con 0x7fb2a80044e0 2020-11-20T12:16:47.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2c19be700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0bf7c0 con 0x7fb2bc0aa500 2020-11-20T12:16:47.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2bc152eb0 0x7fb2bc153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:47.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2bc152eb0 0x7fb2bc153330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44298/0 (socket says 172.21.15.116:44298) 2020-11-20T12:16:47.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b3fff700 1 -- 172.21.15.116:0/2130285267 learned_addr learned my addr 172.21.15.116:0/2130285267 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:47.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2c09bc700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a80044e0 0x7fb2bc0a8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:47.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2c11bd700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2bc0aa500 0x7fb2bc0aa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:47.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b3fff700 1 -- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2bc0aa500 msgr2=0x7fb2bc0aa980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:47.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b3fff700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2bc0aa500 0x7fb2bc0aa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:47.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b3fff700 1 -- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a80044e0 msgr2=0x7fb2bc0a8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:47.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b3fff700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a80044e0 0x7fb2bc0a8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:47.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b3fff700 1 -- 172.21.15.116:0/2130285267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc153a30 con 0x7fb2bc152eb0 2020-11-20T12:16:47.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b3fff700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2bc152eb0 0x7fb2bc153330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fb2ac013a90 tx=0x7fb2ac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:47.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b0ff9700 1 -- 172.21.15.116:0/2130285267 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac020480 con 0x7fb2bc152eb0 2020-11-20T12:16:47.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b0ff9700 1 -- 172.21.15.116:0/2130285267 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac0205e0 con 0x7fb2bc152eb0 2020-11-20T12:16:47.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2c19be700 1 -- 172.21.15.116:0/2130285267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc153cd0 con 0x7fb2bc152eb0 2020-11-20T12:16:47.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2b0ff9700 1 -- 172.21.15.116:0/2130285267 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac020980 con 0x7fb2bc152eb0 2020-11-20T12:16:47.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2c19be700 1 -- 172.21.15.116:0/2130285267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc154210 con 0x7fb2bc152eb0 2020-11-20T12:16:47.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.600+0000 7fb2c19be700 1 -- 172.21.15.116:0/2130285267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc0aae70 con 0x7fb2bc152eb0 2020-11-20T12:16:47.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.604+0000 7fb2b0ff9700 1 -- 172.21.15.116:0/2130285267 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb2ac010030 con 0x7fb2bc152eb0 2020-11-20T12:16:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.604+0000 7fb2b0ff9700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2940557f0 0x7fb294057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.604+0000 7fb2b0ff9700 1 -- 172.21.15.116:0/2130285267 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb2ac070150 con 0x7fb2bc152eb0 2020-11-20T12:16:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.604+0000 7fb2c09bc700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2940557f0 0x7fb294057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:47.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.604+0000 7fb2c09bc700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2940557f0 0x7fb294057cb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fb2bc05e230 tx=0x7fb2b8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:47.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.612+0000 7fb2b0ff9700 1 -- 172.21.15.116:0/2130285267 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb2ac045fa0 con 0x7fb2bc152eb0 2020-11-20T12:16:47.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:47.772+0000 7fb2c19be700 1 -- 172.21.15.116:0/2130285267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb2bc044f60 con 0x7fb2bc152eb0 2020-11-20T12:16:48.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.340+0000 7fb2b0ff9700 1 -- 172.21.15.116:0/2130285267 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb2ac0495c0 con 0x7fb2bc152eb0 2020-11-20T12:16:48.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 -- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2940557f0 msgr2=0x7fb294057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:48.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2940557f0 0x7fb294057cb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fb2bc05e230 tx=0x7fb2b8009040).stop 2020-11-20T12:16:48.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 -- 172.21.15.116:0/2130285267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2bc152eb0 msgr2=0x7fb2bc153330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:48.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2bc152eb0 0x7fb2bc153330 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7fb2ac013a90 tx=0x7fb2ac015000).stop 2020-11-20T12:16:48.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 -- 172.21.15.116:0/2130285267 shutdown_connections 2020-11-20T12:16:48.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2940557f0 0x7fb294057cb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:48.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb2bc0aa500 0x7fb2bc0aa980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:48.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2a80044e0 0x7fb2bc0a8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:48.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 --2- 172.21.15.116:0/2130285267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb2bc152eb0 0x7fb2bc153330 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:48.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 -- 172.21.15.116:0/2130285267 >> 172.21.15.116:0/2130285267 conn(0x7fb2bc091900 msgr2=0x7fb2bc0a75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:48.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 -- 172.21.15.116:0/2130285267 shutdown_connections 2020-11-20T12:16:48.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.344+0000 7fb2c19be700 1 -- 172.21.15.116:0/2130285267 wait complete. 2020-11-20T12:16:48.348 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2020-11-20T12:16:48.371 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-20T12:16:48.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.552+0000 7ff8fd984700 1 Processor -- start 2020-11-20T12:16:48.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.552+0000 7ff8fd984700 1 -- start start 2020-11-20T12:16:48.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.552+0000 7ff8fd984700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8f81501e0 0x7ff8f80a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:48.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.552+0000 7ff8fd984700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff8f80c04c0 0x7ff8f80673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:48.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8fd984700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8f80a37a0 con 0x7ff8f81505c0 2020-11-20T12:16:48.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8fd984700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8f80a0de0 con 0x7ff8f80c04c0 2020-11-20T12:16:48.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8fd984700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f80a0fd0 con 0x7ff8f81501e0 2020-11-20T12:16:48.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f77fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff8f81505c0 0x7ff8f8152ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53742/0 (socket says 172.21.15.116:53742) 2020-11-20T12:16:48.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f77fe700 1 -- 172.21.15.116:0/2051614008 learned_addr learned my addr 172.21.15.116:0/2051614008 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:48.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f67fc700 1 --2- 172.21.15.116:0/2051614008 >> v2:172.21.15.116:3300/0 conn(0x7ff8f80c04c0 0x7ff8f80673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:48.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f5ffb700 1 -- 172.21.15.116:0/2051614008 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568809720 0 0) 0x7ff8f80a37a0 con 0x7ff8f81505c0 2020-11-20T12:16:48.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f6ffd700 1 --2- 172.21.15.116:0/2051614008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8f81501e0 0x7ff8f80a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:48.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f5ffb700 1 -- 172.21.15.116:0/2051614008 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8e4003240 con 0x7ff8f81505c0 2020-11-20T12:16:48.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f67fc700 1 -- 172.21.15.116:0/2051614008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8f81501e0 msgr2=0x7ff8f80a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:48.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f67fc700 1 --2- 172.21.15.116:0/2051614008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8f81501e0 0x7ff8f80a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:48.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f67fc700 1 -- 172.21.15.116:0/2051614008 >> v1:172.21.15.116:6789/0 conn(0x7ff8f81505c0 legacy=0x7ff8f8152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:48.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f67fc700 1 -- 172.21.15.116:0/2051614008 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f80a1270 con 0x7ff8f80c04c0 2020-11-20T12:16:48.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f67fc700 1 --2- 172.21.15.116:0/2051614008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f80c04c0 0x7ff8f80673b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7ff8ec00b9a0 tx=0x7ff8ec006df0).ready entity=mon.0 client_cookie=afabff39cb553627 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:48.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f5ffb700 1 -- 172.21.15.116:0/2051614008 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0097b0 con 0x7ff8f80c04c0 2020-11-20T12:16:48.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f5ffb700 1 -- 172.21.15.116:0/2051614008 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec009910 con 0x7ff8f80c04c0 2020-11-20T12:16:48.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8f5ffb700 1 -- 172.21.15.116:0/2051614008 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec009c90 con 0x7ff8f80c04c0 2020-11-20T12:16:48.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8fd984700 1 -- 172.21.15.116:0/2051614008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f80c04c0 msgr2=0x7ff8f80673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:48.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8fd984700 1 --2- 172.21.15.116:0/2051614008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f80c04c0 0x7ff8f80673b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7ff8ec00b9a0 tx=0x7ff8ec006df0).stop 2020-11-20T12:16:48.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8fd984700 1 -- 172.21.15.116:0/2051614008 shutdown_connections 2020-11-20T12:16:48.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8fd984700 1 --2- 172.21.15.116:0/2051614008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8f81501e0 0x7ff8f80a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:48.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8fd984700 1 --2- 172.21.15.116:0/2051614008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f80c04c0 0x7ff8f80673b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:48.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8fd984700 1 -- 172.21.15.116:0/2051614008 >> 172.21.15.116:0/2051614008 conn(0x7ff8f8060c30 msgr2=0x7ff8f80c4010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:48.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8fd984700 1 -- 172.21.15.116:0/2051614008 shutdown_connections 2020-11-20T12:16:48.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.556+0000 7ff8fd984700 1 -- 172.21.15.116:0/2051614008 wait complete. 2020-11-20T12:16:48.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.560+0000 7ff8fd984700 1 Processor -- start 2020-11-20T12:16:48.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.560+0000 7ff8fd984700 1 -- start start 2020-11-20T12:16:48.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8fd984700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f81501e0 0x7ff8f8062da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:48.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8fd984700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8f81505c0 0x7ff8f8162160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:48.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8fd984700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8f8150a80 0x7ff8f8061d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:48.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8fd984700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f8162950 con 0x7ff8f81501e0 2020-11-20T12:16:48.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8fd984700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f8162bf0 con 0x7ff8f81505c0 2020-11-20T12:16:48.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8fd984700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8f8162e90 con 0x7ff8f8150a80 2020-11-20T12:16:48.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8f6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f81501e0 0x7ff8f8062da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:48.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8f6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f81501e0 0x7ff8f8062da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44310/0 (socket says 172.21.15.116:44310) 2020-11-20T12:16:48.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8f6ffd700 1 -- 172.21.15.116:0/430793058 learned_addr learned my addr 172.21.15.116:0/430793058 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:48.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8f67fc700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8f81505c0 0x7ff8f8162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:48.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8f77fe700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8f8150a80 0x7ff8f8061d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:48.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8f6ffd700 1 -- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8f8150a80 msgr2=0x7ff8f8061d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:48.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8f6ffd700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8f8150a80 0x7ff8f8061d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:48.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8f6ffd700 1 -- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8f81505c0 msgr2=0x7ff8f8162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:48.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8f6ffd700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8f81505c0 0x7ff8f8162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:48.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8f6ffd700 1 -- 172.21.15.116:0/430793058 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8f808f980 con 0x7ff8f81501e0 2020-11-20T12:16:48.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8f6ffd700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f81501e0 0x7ff8f8062da0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7ff8e800db60 tx=0x7ff8e8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:48.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8deffd700 1 -- 172.21.15.116:0/430793058 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8009820 con 0x7ff8f81501e0 2020-11-20T12:16:48.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8fd984700 1 -- 172.21.15.116:0/430793058 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8f8162650 con 0x7ff8f81501e0 2020-11-20T12:16:48.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8deffd700 1 -- 172.21.15.116:0/430793058 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8e8009980 con 0x7ff8f81501e0 2020-11-20T12:16:48.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8deffd700 1 -- 172.21.15.116:0/430793058 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8e8009d20 con 0x7ff8f81501e0 2020-11-20T12:16:48.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.564+0000 7ff8fd984700 1 -- 172.21.15.116:0/430793058 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8f8062230 con 0x7ff8f81501e0 2020-11-20T12:16:48.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.568+0000 7ff8fd984700 1 -- 172.21.15.116:0/430793058 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8f808fb10 con 0x7ff8f81501e0 2020-11-20T12:16:48.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.568+0000 7ff8deffd700 1 -- 172.21.15.116:0/430793058 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff8e8013690 con 0x7ff8f81501e0 2020-11-20T12:16:48.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.568+0000 7ff8deffd700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8d40557c0 0x7ff8d4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:48.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.568+0000 7ff8deffd700 1 -- 172.21.15.116:0/430793058 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5721+0+0 (secure 0 0 0) 0x7ff8e806eca0 con 0x7ff8f81501e0 2020-11-20T12:16:48.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.568+0000 7ff8f67fc700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8d40557c0 0x7ff8d4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:48.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.568+0000 7ff8f67fc700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8d40557c0 0x7ff8d4057c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7ff8f80c0a80 tx=0x7ff8ec015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.576+0000 7ff8deffd700 1 -- 172.21.15.116:0/430793058 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff8e8073070 con 0x7ff8f81501e0 2020-11-20T12:16:48.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:48.736+0000 7ff8fd984700 1 -- 172.21.15.116:0/430793058 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff8f8044f60 con 0x7ff8f81501e0 2020-11-20T12:16:49.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.348+0000 7ff8deffd700 1 -- 172.21.15.116:0/430793058 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff8e8044c40 con 0x7ff8f81501e0 2020-11-20T12:16:49.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 -- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8d40557c0 msgr2=0x7ff8d4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:49.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8d40557c0 0x7ff8d4057c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7ff8f80c0a80 tx=0x7ff8ec015000).stop 2020-11-20T12:16:49.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 -- 172.21.15.116:0/430793058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f81501e0 msgr2=0x7ff8f8062da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:49.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f81501e0 0x7ff8f8062da0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7ff8e800db60 tx=0x7ff8e8008040).stop 2020-11-20T12:16:49.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 -- 172.21.15.116:0/430793058 shutdown_connections 2020-11-20T12:16:49.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8d40557c0 0x7ff8d4057c80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:49.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8f8150a80 0x7ff8f8061d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:49.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8f81501e0 0x7ff8f8062da0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:49.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 --2- 172.21.15.116:0/430793058 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8f81505c0 0x7ff8f8162160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:49.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 -- 172.21.15.116:0/430793058 >> 172.21.15.116:0/430793058 conn(0x7ff8f8060c30 msgr2=0x7ff8f80be3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:49.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 -- 172.21.15.116:0/430793058 shutdown_connections 2020-11-20T12:16:49.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.352+0000 7ff8fd984700 1 -- 172.21.15.116:0/430793058 wait complete. 2020-11-20T12:16:49.356 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2020-11-20T12:16:49.378 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T12:16:49.379 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-20T12:16:49.379 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-20T12:16:49.379 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:16:49.380 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-20T12:16:49.380 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-20T12:16:49.380 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-20T12:16:49.380 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:16:49.381 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T12:16:49.381 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-20T12:16:49.381 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-20T12:16:49.382 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:16:49.382 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-20T12:16:49.382 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-20T12:16:49.382 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-20T12:16:49.383 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:16:49.402 INFO:tasks.ceph.mds.b:Stopped 2020-11-20T12:16:49.402 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T12:16:49.405 INFO:tasks.ceph.mds.c:Stopped 2020-11-20T12:16:49.405 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-20T12:16:49.406 INFO:tasks.ceph.mds.a:Stopped 2020-11-20T12:16:49.407 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T12:16:49.407 INFO:tasks.ceph.mds.b:Started 2020-11-20T12:16:49.408 DEBUG:teuthology.parallel:result is None 2020-11-20T12:16:49.409 INFO:tasks.ceph.mds.d:Stopped 2020-11-20T12:16:49.409 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-20T12:16:49.413 INFO:tasks.ceph.mds.d:Started 2020-11-20T12:16:49.414 DEBUG:teuthology.parallel:result is None 2020-11-20T12:16:49.451 INFO:tasks.ceph.mds.c:Started 2020-11-20T12:16:49.452 DEBUG:teuthology.parallel:result is None 2020-11-20T12:16:49.456 INFO:tasks.ceph.mds.a:Started 2020-11-20T12:16:49.456 DEBUG:teuthology.parallel:result is None 2020-11-20T12:16:49.457 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-20T12:16:49.509 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2020-11-20T12:16:49.606 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:16:49.606 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:16:49.607 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2020-11-20T12:16:49.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.632+0000 7f7e3e0c2700 1 Processor -- start 2020-11-20T12:16:49.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.632+0000 7f7e3e0c2700 1 -- start start 2020-11-20T12:16:49.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.632+0000 7f7e3e0c2700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7e3813aee0 0x7f7e38152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:49.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7e380aa790 0x7f7e380ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:49.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e380612f0 0x7f7e380617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:49.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e38152a30 con 0x7f7e380aa790 2020-11-20T12:16:49.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e380aac10 con 0x7f7e3813aee0 2020-11-20T12:16:49.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380aaeb0 con 0x7f7e380612f0 2020-11-20T12:16:49.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3c8bf700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7e380aa790 0x7f7e380ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:49.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3d8c1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e380612f0 0x7f7e380617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:49.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3d8c1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e380612f0 0x7f7e380617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:53296/0 (socket says 172.21.15.116:53296) 2020-11-20T12:16:49.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3d8c1700 1 -- 172.21.15.116:0/1471158116 learned_addr learned my addr 172.21.15.116:0/1471158116 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:49.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3d0c0700 1 --2- 172.21.15.116:0/1471158116 >> v2:172.21.15.173:3300/0 conn(0x7f7e3813aee0 0x7f7e38152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:49.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3c8bf700 1 -- 172.21.15.116:0/1471158116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e380612f0 msgr2=0x7f7e380617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:49.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3c8bf700 1 --2- 172.21.15.116:0/1471158116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e380612f0 0x7f7e380617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:49.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3c8bf700 1 -- 172.21.15.116:0/1471158116 >> v2:172.21.15.173:3300/0 conn(0x7f7e3813aee0 msgr2=0x7f7e38152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:49.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3c8bf700 1 --2- 172.21.15.116:0/1471158116 >> v2:172.21.15.173:3300/0 conn(0x7f7e3813aee0 0x7f7e38152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:49.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3c8bf700 1 -- 172.21.15.116:0/1471158116 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e380acbf0 con 0x7f7e380aa790 2020-11-20T12:16:49.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3d0c0700 1 --2- 172.21.15.116:0/1471158116 >> v2:172.21.15.173:3300/0 conn(0x7f7e3813aee0 0x7f7e38152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:49.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3c8bf700 1 --2- 172.21.15.116:0/1471158116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e380aa790 0x7f7e380ad5d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f7e38081470 tx=0x7f7e28006df0).ready entity=mon.0 client_cookie=d31c0bf9e03fcd4d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:49.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e27fff700 1 -- 172.21.15.116:0/1471158116 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28009640 con 0x7f7e380aa790 2020-11-20T12:16:49.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e27fff700 1 -- 172.21.15.116:0/1471158116 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e280097a0 con 0x7f7e380aa790 2020-11-20T12:16:49.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e27fff700 1 -- 172.21.15.116:0/1471158116 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28009b20 con 0x7f7e380aa790 2020-11-20T12:16:49.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/1471158116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e380aa790 msgr2=0x7f7e380ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:49.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 --2- 172.21.15.116:0/1471158116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e380aa790 0x7f7e380ad5d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f7e38081470 tx=0x7f7e28006df0).stop 2020-11-20T12:16:49.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/1471158116 shutdown_connections 2020-11-20T12:16:49.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 --2- 172.21.15.116:0/1471158116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e380612f0 0x7f7e380617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:49.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 --2- 172.21.15.116:0/1471158116 >> v2:172.21.15.173:3300/0 conn(0x7f7e3813aee0 0x7f7e38152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:49.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 --2- 172.21.15.116:0/1471158116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e380aa790 0x7f7e380ad5d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:49.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.636+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/1471158116 >> 172.21.15.116:0/1471158116 conn(0x7f7e380678a0 msgr2=0x7f7e38067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:49.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.640+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/1471158116 shutdown_connections 2020-11-20T12:16:49.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.640+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/1471158116 wait complete. 2020-11-20T12:16:49.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.640+0000 7f7e3e0c2700 1 Processor -- start 2020-11-20T12:16:49.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.640+0000 7f7e3e0c2700 1 -- start start 2020-11-20T12:16:49.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3e0c2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e380612f0 0x7f7e380a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:49.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3e0c2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e380aa790 0x7f7e3809f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3e0c2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e3813aee0 0x7f7e38160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3e0c2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e380a3e70 con 0x7f7e3813aee0 2020-11-20T12:16:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3e0c2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e38161650 con 0x7f7e380aa790 2020-11-20T12:16:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3e0c2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e381618f0 con 0x7f7e380612f0 2020-11-20T12:16:49.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3d8c1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e3813aee0 0x7f7e38160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:49.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3c8bf700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e380aa790 0x7f7e3809f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:49.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3d8c1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e3813aee0 0x7f7e38160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44352/0 (socket says 172.21.15.116:44352) 2020-11-20T12:16:49.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3d8c1700 1 -- 172.21.15.116:0/3712585677 learned_addr learned my addr 172.21.15.116:0/3712585677 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:49.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3d0c0700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e380612f0 0x7f7e380a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:49.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3d8c1700 1 -- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e380612f0 msgr2=0x7f7e380a3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:49.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3d8c1700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e380612f0 0x7f7e380a3980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:49.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3d8c1700 1 -- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e380aa790 msgr2=0x7f7e3809f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:49.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3d8c1700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e380aa790 0x7f7e3809f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:49.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3d8c1700 1 -- 172.21.15.116:0/3712585677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3809f240 con 0x7f7e3813aee0 2020-11-20T12:16:49.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3d8c1700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e3813aee0 0x7f7e38160b40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f7e3000ecf0 tx=0x7f7e30009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:49.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e257fa700 1 -- 172.21.15.116:0/3712585677 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e3000ca10 con 0x7f7e3813aee0 2020-11-20T12:16:49.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/3712585677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3809f4e0 con 0x7f7e3813aee0 2020-11-20T12:16:49.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e257fa700 1 -- 172.21.15.116:0/3712585677 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e3000cb70 con 0x7f7e3813aee0 2020-11-20T12:16:49.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e257fa700 1 -- 172.21.15.116:0/3712585677 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e300123f0 con 0x7f7e3813aee0 2020-11-20T12:16:49.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.644+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/3712585677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3809fe70 con 0x7f7e3813aee0 2020-11-20T12:16:49.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.648+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/3712585677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e38161ab0 con 0x7f7e3813aee0 2020-11-20T12:16:49.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.648+0000 7f7e257fa700 1 -- 172.21.15.116:0/3712585677 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7e30022070 con 0x7f7e3813aee0 2020-11-20T12:16:49.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.648+0000 7f7e257fa700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e100557c0 0x7f7e10057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:49.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.648+0000 7f7e257fa700 1 -- 172.21.15.116:0/3712585677 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f7e3006eb90 con 0x7f7e3813aee0 2020-11-20T12:16:49.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.652+0000 7f7e3d0c0700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e100557c0 0x7f7e10057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:49.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.652+0000 7f7e3d0c0700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e100557c0 0x7f7e10057c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f7e34006fd0 tx=0x7f7e34006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:49.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.656+0000 7f7e257fa700 1 -- 172.21.15.116:0/3712585677 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7e30044c00 con 0x7f7e3813aee0 2020-11-20T12:16:49.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:49.828+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/3712585677 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7e38044f60 con 0x7f7e3813aee0 2020-11-20T12:16:50.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.540+0000 7f7e257fa700 1 -- 172.21.15.116:0/3712585677 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7e30016070 con 0x7f7e3813aee0 2020-11-20T12:16:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.544+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e100557c0 msgr2=0x7f7e10057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.544+0000 7f7e3e0c2700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e100557c0 0x7f7e10057c80 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f7e34006fd0 tx=0x7f7e34006df0).stop 2020-11-20T12:16:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.544+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/3712585677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e3813aee0 msgr2=0x7f7e38160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:50.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.544+0000 7f7e3e0c2700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e3813aee0 0x7f7e38160b40 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f7e3000ecf0 tx=0x7f7e30009290).stop 2020-11-20T12:16:50.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.544+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/3712585677 shutdown_connections 2020-11-20T12:16:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.544+0000 7f7e3e0c2700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e100557c0 0x7f7e10057c80 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.544+0000 7f7e3e0c2700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e3813aee0 0x7f7e38160b40 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.544+0000 7f7e3e0c2700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e380612f0 0x7f7e380a3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.544+0000 7f7e3e0c2700 1 --2- 172.21.15.116:0/3712585677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e380aa790 0x7f7e3809f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:50.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.544+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/3712585677 >> 172.21.15.116:0/3712585677 conn(0x7f7e380678a0 msgr2=0x7f7e380a8b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:50.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.548+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/3712585677 shutdown_connections 2020-11-20T12:16:50.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.548+0000 7f7e3e0c2700 1 -- 172.21.15.116:0/3712585677 wait complete. 2020-11-20T12:16:50.549 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2020-11-20T12:16:50.572 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-20T12:16:50.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.744+0000 7fa6782ce700 1 Processor -- start 2020-11-20T12:16:50.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.744+0000 7fa6782ce700 1 -- start start 2020-11-20T12:16:50.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa6782ce700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa6701518e0 0x7fa6700a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:50.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa6782ce700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa67014dde0 0x7fa670152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa6782ce700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa670152630 con 0x7fa670151cc0 2020-11-20T12:16:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa6782ce700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6700a39c0 con 0x7fa67014dde0 2020-11-20T12:16:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa6782ce700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6700bf360 con 0x7fa6701518e0 2020-11-20T12:16:50.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa67686b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa67014dde0 0x7fa670152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa675869700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa670151cc0 0x7fa67014d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53794/0 (socket says 172.21.15.116:53794) 2020-11-20T12:16:50.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa675869700 1 -- 172.21.15.116:0/1948423463 learned_addr learned my addr 172.21.15.116:0/1948423463 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa67606a700 1 --2- 172.21.15.116:0/1948423463 >> v2:172.21.15.173:3300/0 conn(0x7fa6701518e0 0x7fa6700a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:50.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa67686b700 1 -- 172.21.15.116:0/1948423463 >> v2:172.21.15.173:3300/0 conn(0x7fa6701518e0 msgr2=0x7fa6700a34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa67686b700 1 --2- 172.21.15.116:0/1948423463 >> v2:172.21.15.173:3300/0 conn(0x7fa6701518e0 0x7fa6700a34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:50.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa67686b700 1 -- 172.21.15.116:0/1948423463 >> v1:172.21.15.116:6789/0 conn(0x7fa670151cc0 legacy=0x7fa67014d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.748+0000 7fa67686b700 1 -- 172.21.15.116:0/1948423463 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700bf600 con 0x7fa67014dde0 2020-11-20T12:16:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa67686b700 1 --2- 172.21.15.116:0/1948423463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67014dde0 0x7fa670152140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fa6700c5200 tx=0x7fa6680091d0).ready entity=mon.0 client_cookie=d9edfb074642d6b3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa675068700 1 -- 172.21.15.116:0/1948423463 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800ba70 con 0x7fa67014dde0 2020-11-20T12:16:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa675068700 1 -- 172.21.15.116:0/1948423463 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66800bbd0 con 0x7fa67014dde0 2020-11-20T12:16:50.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa675068700 1 -- 172.21.15.116:0/1948423463 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa668011400 con 0x7fa67014dde0 2020-11-20T12:16:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa6782ce700 1 -- 172.21.15.116:0/1948423463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67014dde0 msgr2=0x7fa670152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa6782ce700 1 --2- 172.21.15.116:0/1948423463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67014dde0 0x7fa670152140 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fa6700c5200 tx=0x7fa6680091d0).stop 2020-11-20T12:16:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa6782ce700 1 -- 172.21.15.116:0/1948423463 shutdown_connections 2020-11-20T12:16:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa6782ce700 1 --2- 172.21.15.116:0/1948423463 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67014dde0 0x7fa670152140 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:50.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa6782ce700 1 --2- 172.21.15.116:0/1948423463 >> v2:172.21.15.173:3300/0 conn(0x7fa6701518e0 0x7fa6700a34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa6782ce700 1 -- 172.21.15.116:0/1948423463 >> 172.21.15.116:0/1948423463 conn(0x7fa670060c80 msgr2=0x7fa6700c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa6782ce700 1 -- 172.21.15.116:0/1948423463 shutdown_connections 2020-11-20T12:16:50.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.752+0000 7fa6782ce700 1 -- 172.21.15.116:0/1948423463 wait complete. 2020-11-20T12:16:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa6782ce700 1 Processor -- start 2020-11-20T12:16:50.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa6782ce700 1 -- start start 2020-11-20T12:16:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa6782ce700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa67014dde0 0x7fa6701486d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa6782ce700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6701518e0 0x7fa67014b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa6782ce700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa670151cc0 0x7fa6700c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa6782ce700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa670148bc0 con 0x7fa6701518e0 2020-11-20T12:16:50.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa6782ce700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700a3d40 con 0x7fa67014dde0 2020-11-20T12:16:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa6782ce700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700a3fe0 con 0x7fa670151cc0 2020-11-20T12:16:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa675869700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6701518e0 0x7fa67014b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa675869700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6701518e0 0x7fa67014b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44364/0 (socket says 172.21.15.116:44364) 2020-11-20T12:16:50.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa675869700 1 -- 172.21.15.116:0/2193153664 learned_addr learned my addr 172.21.15.116:0/2193153664 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa67606a700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa67014dde0 0x7fa6701486d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa67686b700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa670151cc0 0x7fa6700c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa675869700 1 -- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa670151cc0 msgr2=0x7fa6700c40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa675869700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa670151cc0 0x7fa6700c40c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:50.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa675869700 1 -- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa67014dde0 msgr2=0x7fa6701486d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:50.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa675869700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa67014dde0 0x7fa6701486d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:50.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa675869700 1 -- 172.21.15.116:0/2193153664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700a4280 con 0x7fa6701518e0 2020-11-20T12:16:50.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.756+0000 7fa67686b700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa670151cc0 0x7fa6700c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:50.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.760+0000 7fa675869700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6701518e0 0x7fa67014b760 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fa66000c710 tx=0x7fa660007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.760+0000 7fa65e7fc700 1 -- 172.21.15.116:0/2193153664 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6600072e0 con 0x7fa6701518e0 2020-11-20T12:16:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.760+0000 7fa65e7fc700 1 -- 172.21.15.116:0/2193153664 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa660007440 con 0x7fa6701518e0 2020-11-20T12:16:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.760+0000 7fa6782ce700 1 -- 172.21.15.116:0/2193153664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6700a4520 con 0x7fa6701518e0 2020-11-20T12:16:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.760+0000 7fa65e7fc700 1 -- 172.21.15.116:0/2193153664 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa660005800 con 0x7fa6701518e0 2020-11-20T12:16:50.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.760+0000 7fa6782ce700 1 -- 172.21.15.116:0/2193153664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67014c8a0 con 0x7fa6701518e0 2020-11-20T12:16:50.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.760+0000 7fa6782ce700 1 -- 172.21.15.116:0/2193153664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67005ca30 con 0x7fa6701518e0 2020-11-20T12:16:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.760+0000 7fa65e7fc700 1 -- 172.21.15.116:0/2193153664 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa660010070 con 0x7fa6701518e0 2020-11-20T12:16:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.764+0000 7fa65e7fc700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6480557c0 0x7fa648057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.764+0000 7fa65e7fc700 1 -- 172.21.15.116:0/2193153664 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa66006da10 con 0x7fa6701518e0 2020-11-20T12:16:50.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.764+0000 7fa67606a700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6480557c0 0x7fa648057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:50.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.764+0000 7fa67606a700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6480557c0 0x7fa648057c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fa66c00cd00 tx=0x7fa66c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:50.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.772+0000 7fa65e7fc700 1 -- 172.21.15.116:0/2193153664 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa660043be0 con 0x7fa6701518e0 2020-11-20T12:16:50.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.980+0000 7fa6782ce700 1 -- 172.21.15.116:0/2193153664 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa67014cd80 con 0x7fa6701518e0 2020-11-20T12:16:50.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa65e7fc700 1 -- 172.21.15.116:0/2193153664 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6485 (secure 0 0 0) 0x7fa660007de0 con 0x7fa6701518e0 2020-11-20T12:16:50.984 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:50.985 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:16:50.985 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2020-11-20T12:16:50.985 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:50.986 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2020-11-20T12:16:50.986 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw==", 2020-11-20T12:16:50.986 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:50.987 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:16:50.987 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:50.987 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:50.988 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:50.988 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:50.988 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:50.989 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:50.989 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2020-11-20T12:16:50.989 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ==", 2020-11-20T12:16:50.990 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:50.990 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:16:50.990 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:50.991 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:50.991 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:50.992 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:50.992 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:50.992 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:50.993 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2020-11-20T12:16:50.993 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag==", 2020-11-20T12:16:50.994 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:50.994 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:16:50.994 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:50.995 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:50.995 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:50.995 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:50.995 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:50.996 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:50.996 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2020-11-20T12:16:50.996 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ==", 2020-11-20T12:16:50.997 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:50.997 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:16:50.998 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:50.998 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:50.998 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:50.999 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:50.999 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:50.999 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.000 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2020-11-20T12:16:51.000 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ==", 2020-11-20T12:16:51.000 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.000 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:51.001 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:51.002 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:51.002 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.003 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.003 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.003 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2020-11-20T12:16:51.004 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q==", 2020-11-20T12:16:51.004 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.004 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:51.005 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:51.005 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:51.005 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.006 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.006 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.006 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2020-11-20T12:16:51.007 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw==", 2020-11-20T12:16:51.007 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.007 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:51.007 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:51.008 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:51.008 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.008 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.009 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.009 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2020-11-20T12:16:51.009 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w==", 2020-11-20T12:16:51.010 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.010 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:51.010 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:51.011 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:51.011 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.012 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.012 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.013 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2020-11-20T12:16:51.013 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA==", 2020-11-20T12:16:51.013 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.014 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:51.014 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:51.014 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:51.015 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.015 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.015 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.016 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2020-11-20T12:16:51.016 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ==", 2020-11-20T12:16:51.016 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.017 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:51.017 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:51.017 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:51.018 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.018 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.018 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.019 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2020-11-20T12:16:51.019 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg==", 2020-11-20T12:16:51.019 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.020 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:51.020 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:51.020 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:51.021 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.021 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.021 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.022 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2020-11-20T12:16:51.022 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg==", 2020-11-20T12:16:51.023 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.023 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:51.023 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:51.024 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:51.024 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.025 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.025 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.025 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2020-11-20T12:16:51.026 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA==", 2020-11-20T12:16:51.026 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.026 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:16:51.026 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:16:51.027 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:16:51.027 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:16:51.027 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.028 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.028 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.028 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2020-11-20T12:16:51.028 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow==", 2020-11-20T12:16:51.029 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.029 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:16:51.029 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:16:51.029 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:16:51.030 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:16:51.030 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.030 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.030 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.030 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2020-11-20T12:16:51.031 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ==", 2020-11-20T12:16:51.031 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.031 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:16:51.031 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:16:51.031 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:16:51.032 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:16:51.032 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.032 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.032 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.032 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2020-11-20T12:16:51.033 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ==", 2020-11-20T12:16:51.033 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.034 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:16:51.034 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:16:51.034 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:16:51.035 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:16:51.035 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.036 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.036 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.036 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2020-11-20T12:16:51.037 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBDs7dfh+KKOBAAZvcPXWly9/mxRE7NWuR85A==", 2020-11-20T12:16:51.038 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.038 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:16:51.038 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:16:51.039 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:16:51.039 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:51.039 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.041 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.041 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.042 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-mds", 2020-11-20T12:16:51.043 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBts7dfxAqhFRAA9kgikGXOtRCMLFdNrGrjhg==", 2020-11-20T12:16:51.043 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.044 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-mds" 2020-11-20T12:16:51.044 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.044 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.044 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.044 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-mgr", 2020-11-20T12:16:51.045 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBts7df6SKhFRAANZez/iXFjTKj3cHT6Z/fCw==", 2020-11-20T12:16:51.046 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.046 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-mgr" 2020-11-20T12:16:51.046 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.046 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.047 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.047 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-osd", 2020-11-20T12:16:51.047 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBts7dfFzmhFRAApi1S70SCBqnnsaEDzmtWNg==", 2020-11-20T12:16:51.047 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.047 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-osd" 2020-11-20T12:16:51.048 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.048 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.048 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.048 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-rbd", 2020-11-20T12:16:51.049 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBts7dfVU+hFRAAuawmwe/YT9uCt4wOTb24+g==", 2020-11-20T12:16:51.049 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.049 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-rbd" 2020-11-20T12:16:51.049 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.049 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.050 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.050 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-rbd-mirror", 2020-11-20T12:16:51.050 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBts7dfO2mhFRAAXJkt5QjMhHg8N4MCKxZVMw==", 2020-11-20T12:16:51.050 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.050 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2020-11-20T12:16:51.051 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.051 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.051 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.051 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.bootstrap-rgw", 2020-11-20T12:16:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBts7df5n+hFRAA9aTTcVbfZMhPcLCwywZIgg==", 2020-11-20T12:16:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile bootstrap-rgw" 2020-11-20T12:16:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.052 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.053 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.053 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2020-11-20T12:16:51.053 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A==", 2020-11-20T12:16:51.053 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.053 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:16:51.054 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:16:51.054 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:51.054 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.054 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:51.054 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:51.055 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2020-11-20T12:16:51.055 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA==", 2020-11-20T12:16:51.055 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:16:51.055 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:16:51.055 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:16:51.056 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:16:51.056 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.056 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:51.057 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:16:51.057 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:16:51.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 -- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6480557c0 msgr2=0x7fa648057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6480557c0 0x7fa648057c80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7fa66c00cd00 tx=0x7fa66c008040).stop 2020-11-20T12:16:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 -- 172.21.15.116:0/2193153664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6701518e0 msgr2=0x7fa67014b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6701518e0 0x7fa67014b760 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7fa66000c710 tx=0x7fa660007ca0).stop 2020-11-20T12:16:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 -- 172.21.15.116:0/2193153664 shutdown_connections 2020-11-20T12:16:51.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6480557c0 0x7fa648057c80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa670151cc0 0x7fa6700c40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa67014dde0 0x7fa6701486d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 --2- 172.21.15.116:0/2193153664 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6701518e0 0x7fa67014b760 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 -- 172.21.15.116:0/2193153664 >> 172.21.15.116:0/2193153664 conn(0x7fa670060c80 msgr2=0x7fa6700c2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:51.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 -- 172.21.15.116:0/2193153664 shutdown_connections 2020-11-20T12:16:51.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:50.984+0000 7fa6782ce700 1 -- 172.21.15.116:0/2193153664 wait complete. 2020-11-20T12:16:51.061 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2020-11-20T12:16:51.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.244+0000 7fbeaa911700 1 Processor -- start 2020-11-20T12:16:51.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.244+0000 7fbeaa911700 1 -- start start 2020-11-20T12:16:51.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbeaa911700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbea413f5b0 0x7fbea40c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbeaa911700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea40c44a0 0x7fbea4066090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbeaa911700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbea40a5680 con 0x7fbea413f1d0 2020-11-20T12:16:51.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbeaa911700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbea40a42e0 con 0x7fbea413f5b0 2020-11-20T12:16:51.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbeaa911700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea40a58e0 con 0x7fbea40c44a0 2020-11-20T12:16:51.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea990f700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbea413f1d0 0x7fbea40a3bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53804/0 (socket says 172.21.15.116:53804) 2020-11-20T12:16:51.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea990f700 1 -- 172.21.15.116:0/2430926862 learned_addr learned my addr 172.21.15.116:0/2430926862 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:51.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416512776 0 0) 0x7fbea40a5680 con 0x7fbea413f1d0 2020-11-20T12:16:51.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe8c003240 con 0x7fbea413f1d0 2020-11-20T12:16:51.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbeaa110700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea40c44a0 0x7fbea4066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea910e700 1 --2- 172.21.15.116:0/2430926862 >> v2:172.21.15.173:3300/0 conn(0x7fbea413f5b0 0x7fbea40c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 628051508 0 0) 0x7fbe8c003240 con 0x7fbea413f1d0 2020-11-20T12:16:51.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea40c44a0 msgr2=0x7fbea4066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:51.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea40c44a0 0x7fbea4066090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 >> v2:172.21.15.173:3300/0 conn(0x7fbea413f5b0 msgr2=0x7fbea40c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:51.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 --2- 172.21.15.116:0/2430926862 >> v2:172.21.15.173:3300/0 conn(0x7fbea413f5b0 0x7fbea40c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea40a5b80 con 0x7fbea413f1d0 2020-11-20T12:16:51.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fbea00014e0 con 0x7fbea413f1d0 2020-11-20T12:16:51.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 >> v1:172.21.15.116:6789/0 conn(0x7fbea413f1d0 legacy=0x7fbea40a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea40c44a0 0x7fbe8c0037b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe8c003d10 0x7fbe8c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe8c0049d0 0x7fbe8c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea40a42e0 con 0x7fbe8c003d10 2020-11-20T12:16:51.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea40a58e0 con 0x7fbe8c0049d0 2020-11-20T12:16:51.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe8c0042d0 con 0x7fbea40c44a0 2020-11-20T12:16:51.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea910e700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe8c003d10 0x7fbe8c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea990f700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea40c44a0 0x7fbe8c0037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbeaa110700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe8c0049d0 0x7fbe8c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea910e700 1 -- 172.21.15.116:0/2430926862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea40c44a0 msgr2=0x7fbe8c0037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:51.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea910e700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea40c44a0 0x7fbe8c0037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea910e700 1 -- 172.21.15.116:0/2430926862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe8c0049d0 msgr2=0x7fbe8c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:51.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea910e700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe8c0049d0 0x7fbe8c004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.248+0000 7fbea910e700 1 -- 172.21.15.116:0/2430926862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe8c003240 con 0x7fbe8c003d10 2020-11-20T12:16:51.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.252+0000 7fbea910e700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe8c003d10 0x7fbe8c0044e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fbe9400ba80 tx=0x7fbe94008040).ready entity=mon.0 client_cookie=da7c3136b518cf80 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:51.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.252+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9400d960 con 0x7fbe8c003d10 2020-11-20T12:16:51.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.252+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9400dac0 con 0x7fbe8c003d10 2020-11-20T12:16:51.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.252+0000 7fbea890d700 1 -- 172.21.15.116:0/2430926862 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9400de40 con 0x7fbe8c003d10 2020-11-20T12:16:51.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.252+0000 7fbeaa911700 1 -- 172.21.15.116:0/2430926862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe8c003d10 msgr2=0x7fbe8c0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.252+0000 7fbeaa911700 1 --2- 172.21.15.116:0/2430926862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe8c003d10 0x7fbe8c0044e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fbe9400ba80 tx=0x7fbe94008040).stop 2020-11-20T12:16:51.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.252+0000 7fbeaa110700 1 -- 172.21.15.116:0/2430926862 reap_dead start 2020-11-20T12:16:51.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.252+0000 7fbeaa911700 1 -- 172.21.15.116:0/2430926862 shutdown_connections 2020-11-20T12:16:51.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.252+0000 7fbeaa911700 1 -- 172.21.15.116:0/2430926862 >> 172.21.15.116:0/2430926862 conn(0x7fbea4062520 msgr2=0x7fbea4062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:51.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.252+0000 7fbeaa911700 1 -- 172.21.15.116:0/2430926862 shutdown_connections 2020-11-20T12:16:51.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.252+0000 7fbeaa911700 1 -- 172.21.15.116:0/2430926862 wait complete. 2020-11-20T12:16:51.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.256+0000 7fbeaa911700 1 Processor -- start 2020-11-20T12:16:51.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.256+0000 7fbeaa911700 1 -- start start 2020-11-20T12:16:51.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.256+0000 7fbeaa911700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea41555a0 0x7fbea4165ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.256+0000 7fbeaa911700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbea40a8f30 0x7fbea409e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.256+0000 7fbeaa911700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea409eb50 0x7fbea409efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.256+0000 7fbeaa911700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea40a96b0 con 0x7fbea40a8f30 2020-11-20T12:16:51.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.256+0000 7fbeaa911700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea409d610 con 0x7fbea409eb50 2020-11-20T12:16:51.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.256+0000 7fbeaa911700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbea409d8b0 con 0x7fbea41555a0 2020-11-20T12:16:51.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbea910e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbea40a8f30 0x7fbea409e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbea910e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbea40a8f30 0x7fbea409e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44382/0 (socket says 172.21.15.116:44382) 2020-11-20T12:16:51.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbea910e700 1 -- 172.21.15.116:0/3279755373 learned_addr learned my addr 172.21.15.116:0/3279755373 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:51.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbea990f700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea41555a0 0x7fbea4165ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbeaa110700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea409eb50 0x7fbea409efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbea910e700 1 -- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea41555a0 msgr2=0x7fbea4165ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbea910e700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea41555a0 0x7fbea4165ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbea910e700 1 -- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea409eb50 msgr2=0x7fbea409efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbea910e700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea409eb50 0x7fbea409efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbea910e700 1 -- 172.21.15.116:0/3279755373 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea409db50 con 0x7fbea40a8f30 2020-11-20T12:16:51.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbeaa110700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea409eb50 0x7fbea409efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:51.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbea910e700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbea40a8f30 0x7fbea409e290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fbe9400ba50 tx=0x7fbe94008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:51.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbe91ffb700 1 -- 172.21.15.116:0/3279755373 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9401d620 con 0x7fbea40a8f30 2020-11-20T12:16:51.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbe91ffb700 1 -- 172.21.15.116:0/3279755373 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9401d780 con 0x7fbea40a8f30 2020-11-20T12:16:51.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbeaa911700 1 -- 172.21.15.116:0/3279755373 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea409ddf0 con 0x7fbea40a8f30 2020-11-20T12:16:51.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbe91ffb700 1 -- 172.21.15.116:0/3279755373 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9401db20 con 0x7fbea40a8f30 2020-11-20T12:16:51.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbeaa911700 1 -- 172.21.15.116:0/3279755373 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea4155980 con 0x7fbea40a8f30 2020-11-20T12:16:51.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbeaa911700 1 -- 172.21.15.116:0/3279755373 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbea4044f60 con 0x7fbea40a8f30 2020-11-20T12:16:51.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.260+0000 7fbe91ffb700 1 -- 172.21.15.116:0/3279755373 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fbe940123e0 con 0x7fbea40a8f30 2020-11-20T12:16:51.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.264+0000 7fbe91ffb700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe7c0557c0 0x7fbe7c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.264+0000 7fbe91ffb700 1 -- 172.21.15.116:0/3279755373 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fbe9406ff00 con 0x7fbea40a8f30 2020-11-20T12:16:51.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.264+0000 7fbea990f700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe7c0557c0 0x7fbe7c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.264+0000 7fbea990f700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe7c0557c0 0x7fbe7c057c80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fbea0000dd0 tx=0x7fbea0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:51.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.268+0000 7fbe91ffb700 1 -- 172.21.15.116:0/3279755373 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbe94045f60 con 0x7fbea40a8f30 2020-11-20T12:16:51.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.484+0000 7fbeaa911700 1 -- 172.21.15.116:0/3279755373 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fbea4061580 con 0x7fbea40a8f30 2020-11-20T12:16:51.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbe91ffb700 1 -- 172.21.15.116:0/3279755373 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v3) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbe94049580 con 0x7fbea40a8f30 2020-11-20T12:16:51.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 -- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe7c0557c0 msgr2=0x7fbe7c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe7c0557c0 0x7fbe7c057c80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7fbea0000dd0 tx=0x7fbea0009040).stop 2020-11-20T12:16:51.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 -- 172.21.15.116:0/3279755373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbea40a8f30 msgr2=0x7fbea409e290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbea40a8f30 0x7fbea409e290 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7fbe9400ba50 tx=0x7fbe94008040).stop 2020-11-20T12:16:51.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 -- 172.21.15.116:0/3279755373 shutdown_connections 2020-11-20T12:16:51.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe7c0557c0 0x7fbe7c057c80 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbea409eb50 0x7fbea409efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbea41555a0 0x7fbea4165ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 --2- 172.21.15.116:0/3279755373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbea40a8f30 0x7fbea409e290 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 -- 172.21.15.116:0/3279755373 >> 172.21.15.116:0/3279755373 conn(0x7fbea4062520 msgr2=0x7fbea40c3490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:51.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 -- 172.21.15.116:0/3279755373 shutdown_connections 2020-11-20T12:16:51.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.492+0000 7fbeaa911700 1 -- 172.21.15.116:0/3279755373 wait complete. 2020-11-20T12:16:51.499 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:16:51.516 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2020-11-20T12:16:51.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.696+0000 7f223efc1700 1 Processor -- start 2020-11-20T12:16:51.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.696+0000 7f223efc1700 1 -- start start 2020-11-20T12:16:51.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223efc1700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f223813aee0 0x7f2238152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223efc1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380612f0 0x7f22380617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223efc1700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2238152a30 con 0x7f22380aa790 2020-11-20T12:16:51.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223efc1700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22380adce0 con 0x7f223813aee0 2020-11-20T12:16:51.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223efc1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22380aac10 con 0x7f22380612f0 2020-11-20T12:16:51.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223e7c0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380612f0 0x7f22380617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223dfbf700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f223813aee0 0x7f2238152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223d7be700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f22380aa790 0x7f22380ad5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:50056/0 (socket says 172.21.15.116:50056) 2020-11-20T12:16:51.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223d7be700 1 -- 172.21.15.116:0/912860481 learned_addr learned my addr 172.21.15.116:0/912860481 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:51.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223cfbd700 1 -- 172.21.15.116:0/912860481 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689625884 0 0) 0x7f2238152a30 con 0x7f22380aa790 2020-11-20T12:16:51.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223cfbd700 1 -- 172.21.15.116:0/912860481 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2220003240 con 0x7f22380aa790 2020-11-20T12:16:51.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223e7c0700 1 -- 172.21.15.116:0/912860481 >> v2:172.21.15.173:3300/0 conn(0x7f223813aee0 msgr2=0x7f2238152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:51.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223e7c0700 1 --2- 172.21.15.116:0/912860481 >> v2:172.21.15.173:3300/0 conn(0x7f223813aee0 0x7f2238152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223e7c0700 1 -- 172.21.15.116:0/912860481 >> v1:172.21.15.173:6789/0 conn(0x7f22380aa790 legacy=0x7f22380ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223e7c0700 1 -- 172.21.15.116:0/912860481 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22380aaeb0 con 0x7f22380612f0 2020-11-20T12:16:51.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.700+0000 7f223e7c0700 1 --2- 172.21.15.116:0/912860481 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380612f0 0x7f22380617a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f2230009c90 tx=0x7f22300091d0).ready entity=mon.2 client_cookie=f45714e6b0e20c8b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:51.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223cfbd700 1 -- 172.21.15.116:0/912860481 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223000b9c0 con 0x7f22380612f0 2020-11-20T12:16:51.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223cfbd700 1 -- 172.21.15.116:0/912860481 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223000bb20 con 0x7f22380612f0 2020-11-20T12:16:51.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223cfbd700 1 -- 172.21.15.116:0/912860481 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223000bea0 con 0x7f22380612f0 2020-11-20T12:16:51.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223efc1700 1 -- 172.21.15.116:0/912860481 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380612f0 msgr2=0x7f22380617a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223efc1700 1 --2- 172.21.15.116:0/912860481 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380612f0 0x7f22380617a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f2230009c90 tx=0x7f22300091d0).stop 2020-11-20T12:16:51.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223efc1700 1 -- 172.21.15.116:0/912860481 shutdown_connections 2020-11-20T12:16:51.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223efc1700 1 --2- 172.21.15.116:0/912860481 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380612f0 0x7f22380617a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223efc1700 1 --2- 172.21.15.116:0/912860481 >> v2:172.21.15.173:3300/0 conn(0x7f223813aee0 0x7f2238152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223efc1700 1 -- 172.21.15.116:0/912860481 >> 172.21.15.116:0/912860481 conn(0x7f22380678a0 msgr2=0x7f2238067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:51.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223efc1700 1 -- 172.21.15.116:0/912860481 shutdown_connections 2020-11-20T12:16:51.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223efc1700 1 -- 172.21.15.116:0/912860481 wait complete. 2020-11-20T12:16:51.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.704+0000 7f223efc1700 1 Processor -- start 2020-11-20T12:16:51.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223efc1700 1 -- start start 2020-11-20T12:16:51.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223efc1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22380612f0 0x7f223809f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223efc1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380aa790 0x7f22380c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223efc1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f223813aee0 0x7f22380a26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223efc1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223809f080 con 0x7f22380612f0 2020-11-20T12:16:51.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223efc1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2238161480 con 0x7f223813aee0 2020-11-20T12:16:51.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223efc1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2238161720 con 0x7f22380aa790 2020-11-20T12:16:51.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223d7be700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380aa790 0x7f22380c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223dfbf700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22380612f0 0x7f223809f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223d7be700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380aa790 0x7f22380c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:53346/0 (socket says 172.21.15.116:53346) 2020-11-20T12:16:51.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223d7be700 1 -- 172.21.15.116:0/3384178161 learned_addr learned my addr 172.21.15.116:0/3384178161 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:51.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223e7c0700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f223813aee0 0x7f22380a26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223dfbf700 1 -- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380aa790 msgr2=0x7f22380c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223dfbf700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380aa790 0x7f22380c7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223dfbf700 1 -- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f223813aee0 msgr2=0x7f22380a26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223dfbf700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f223813aee0 0x7f22380a26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223dfbf700 1 -- 172.21.15.116:0/3384178161 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f223809f2b0 con 0x7f22380612f0 2020-11-20T12:16:51.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223dfbf700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22380612f0 0x7f223809f9f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f223400eba0 tx=0x7f2234006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:51.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f2225ffb700 1 -- 172.21.15.116:0/3384178161 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223400b810 con 0x7f22380612f0 2020-11-20T12:16:51.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f2225ffb700 1 -- 172.21.15.116:0/3384178161 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223400b970 con 0x7f22380612f0 2020-11-20T12:16:51.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f2225ffb700 1 -- 172.21.15.116:0/3384178161 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223400bcf0 con 0x7f22380612f0 2020-11-20T12:16:51.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.708+0000 7f223efc1700 1 -- 172.21.15.116:0/3384178161 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223809f550 con 0x7f22380612f0 2020-11-20T12:16:51.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.712+0000 7f223efc1700 1 -- 172.21.15.116:0/3384178161 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223809fee0 con 0x7f22380612f0 2020-11-20T12:16:51.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.712+0000 7f223efc1700 1 -- 172.21.15.116:0/3384178161 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2238044f60 con 0x7f22380612f0 2020-11-20T12:16:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.712+0000 7f2225ffb700 1 -- 172.21.15.116:0/3384178161 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2234012600 con 0x7f22380612f0 2020-11-20T12:16:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.712+0000 7f2225ffb700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22100557c0 0x7f2210057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.712+0000 7f2225ffb700 1 -- 172.21.15.116:0/3384178161 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f223406e500 con 0x7f22380612f0 2020-11-20T12:16:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.716+0000 7f223d7be700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22100557c0 0x7f2210057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:51.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.716+0000 7f223d7be700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22100557c0 0x7f2210057c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f22380c7d20 tx=0x7f2228000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:51.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.720+0000 7f2225ffb700 1 -- 172.21.15.116:0/3384178161 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2234044670 con 0x7f22380612f0 2020-11-20T12:16:51.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.928+0000 7f223efc1700 1 -- 172.21.15.116:0/3384178161 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f2238060500 con 0x7f22380612f0 2020-11-20T12:16:51.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.940+0000 7f2225ffb700 1 -- 172.21.15.116:0/3384178161 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v4) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2234047c90 con 0x7f22380612f0 2020-11-20T12:16:51.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 -- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22100557c0 msgr2=0x7f2210057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22100557c0 0x7f2210057c80 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f22380c7d20 tx=0x7f2228000b80).stop 2020-11-20T12:16:51.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 -- 172.21.15.116:0/3384178161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22380612f0 msgr2=0x7f223809f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:51.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22380612f0 0x7f223809f9f0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f223400eba0 tx=0x7f2234006df0).stop 2020-11-20T12:16:51.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 -- 172.21.15.116:0/3384178161 shutdown_connections 2020-11-20T12:16:51.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22100557c0 0x7f2210057c80 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f223813aee0 0x7f22380a26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f22380612f0 0x7f223809f9f0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 --2- 172.21.15.116:0/3384178161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f22380aa790 0x7f22380c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:51.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 -- 172.21.15.116:0/3384178161 >> 172.21.15.116:0/3384178161 conn(0x7f22380678a0 msgr2=0x7f22380616d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:51.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 -- 172.21.15.116:0/3384178161 shutdown_connections 2020-11-20T12:16:51.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:51.944+0000 7f223efc1700 1 -- 172.21.15.116:0/3384178161 wait complete. 2020-11-20T12:16:51.948 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:16:51.967 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2020-11-20T12:16:52.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a6cbc2700 1 Processor -- start 2020-11-20T12:16:52.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a6cbc2700 1 -- start start 2020-11-20T12:16:52.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a6cbc2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a68064400 0x7f0a680a41c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a6cbc2700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0a680a9910 0x7f0a681581c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a6cbc2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a680a5e30 con 0x7f0a680a9490 2020-11-20T12:16:52.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a6cbc2700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a68068020 con 0x7f0a680a9910 2020-11-20T12:16:52.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a6cbc2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a680a60d0 con 0x7f0a68064400 2020-11-20T12:16:52.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a67fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0a680a9910 0x7f0a681581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a677fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a68064400 0x7f0a680a41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a67fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0a680a9910 0x7f0a681581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:43458/0 (socket says 172.21.15.116:43458) 2020-11-20T12:16:52.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a67fff700 1 -- 172.21.15.116:0/3504658730 learned_addr learned my addr 172.21.15.116:0/3504658730 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:52.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a67fff700 1 -- 172.21.15.116:0/3504658730 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a68064400 msgr2=0x7f0a680a41c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:52.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a67fff700 1 --2- 172.21.15.116:0/3504658730 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a68064400 0x7f0a680a41c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a67fff700 1 -- 172.21.15.116:0/3504658730 >> v1:172.21.15.173:6789/0 conn(0x7f0a680a9490 legacy=0x7f0a680c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:52.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.140+0000 7f0a67fff700 1 -- 172.21.15.116:0/3504658730 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a680a6370 con 0x7f0a680a9910 2020-11-20T12:16:52.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a67fff700 1 --2- 172.21.15.116:0/3504658730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a680a9910 0x7f0a681581c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f0a6000d2f0 tx=0x7f0a6000b530).ready entity=mon.1 client_cookie=583616f3ad62df9d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:52.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a667fc700 1 -- 172.21.15.116:0/3504658730 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60010470 con 0x7f0a680a9910 2020-11-20T12:16:52.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a667fc700 1 -- 172.21.15.116:0/3504658730 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a600105d0 con 0x7f0a680a9910 2020-11-20T12:16:52.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a667fc700 1 -- 172.21.15.116:0/3504658730 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60010950 con 0x7f0a680a9910 2020-11-20T12:16:52.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/3504658730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a680a9910 msgr2=0x7f0a681581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:52.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a6cbc2700 1 --2- 172.21.15.116:0/3504658730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a680a9910 0x7f0a681581c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f0a6000d2f0 tx=0x7f0a6000b530).stop 2020-11-20T12:16:52.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/3504658730 shutdown_connections 2020-11-20T12:16:52.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a6cbc2700 1 --2- 172.21.15.116:0/3504658730 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a680a9910 0x7f0a681581c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a6cbc2700 1 --2- 172.21.15.116:0/3504658730 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a68064400 0x7f0a680a41c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/3504658730 >> 172.21.15.116:0/3504658730 conn(0x7f0a68062520 msgr2=0x7f0a68062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:52.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/3504658730 shutdown_connections 2020-11-20T12:16:52.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.144+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/3504658730 wait complete. 2020-11-20T12:16:52.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.148+0000 7f0a6cbc2700 1 Processor -- start 2020-11-20T12:16:52.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.148+0000 7f0a6cbc2700 1 -- start start 2020-11-20T12:16:52.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.148+0000 7f0a6cbc2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a68064400 0x7f0a6809f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.148+0000 7f0a6cbc2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a680a9490 0x7f0a68146b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.148+0000 7f0a6cbc2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a680a9910 0x7f0a68149de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.148+0000 7f0a6cbc2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6809e610 con 0x7f0a68064400 2020-11-20T12:16:52.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.148+0000 7f0a6cbc2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6814fe20 con 0x7f0a680a9490 2020-11-20T12:16:52.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.148+0000 7f0a6cbc2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6809e870 con 0x7f0a680a9910 2020-11-20T12:16:52.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.148+0000 7f0a677fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a68064400 0x7f0a6809f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.148+0000 7f0a677fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a68064400 0x7f0a6809f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44410/0 (socket says 172.21.15.116:44410) 2020-11-20T12:16:52.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.148+0000 7f0a677fe700 1 -- 172.21.15.116:0/664072255 learned_addr learned my addr 172.21.15.116:0/664072255 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:52.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a67fff700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a680a9910 0x7f0a68149de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a66ffd700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a680a9490 0x7f0a68146b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a677fe700 1 -- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a680a9910 msgr2=0x7f0a68149de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:52.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a677fe700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a680a9910 0x7f0a68149de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a677fe700 1 -- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a680a9490 msgr2=0x7f0a68146b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:52.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a677fe700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a680a9490 0x7f0a68146b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a677fe700 1 -- 172.21.15.116:0/664072255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6809eb10 con 0x7f0a68064400 2020-11-20T12:16:52.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a677fe700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a68064400 0x7f0a6809f250 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f0a5800eba0 tx=0x7f0a58006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:52.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a5f7fe700 1 -- 172.21.15.116:0/664072255 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5800c9b0 con 0x7f0a68064400 2020-11-20T12:16:52.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a5f7fe700 1 -- 172.21.15.116:0/664072255 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a5800cb10 con 0x7f0a68064400 2020-11-20T12:16:52.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/664072255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a6809edb0 con 0x7f0a68064400 2020-11-20T12:16:52.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a5f7fe700 1 -- 172.21.15.116:0/664072255 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a5800ce90 con 0x7f0a68064400 2020-11-20T12:16:52.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/664072255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a68147080 con 0x7f0a68064400 2020-11-20T12:16:52.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.152+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/664072255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a68147330 con 0x7f0a68064400 2020-11-20T12:16:52.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.156+0000 7f0a5f7fe700 1 -- 172.21.15.116:0/664072255 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0a58012600 con 0x7f0a68064400 2020-11-20T12:16:52.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.156+0000 7f0a5f7fe700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0a440557c0 0x7f0a44057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.156+0000 7f0a5f7fe700 1 -- 172.21.15.116:0/664072255 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f0a5806e850 con 0x7f0a68064400 2020-11-20T12:16:52.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.156+0000 7f0a66ffd700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0a440557c0 0x7f0a44057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.160+0000 7f0a66ffd700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0a440557c0 0x7f0a44057c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f0a6814b3f0 tx=0x7f0a50009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:52.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.160+0000 7f0a5f7fe700 1 -- 172.21.15.116:0/664072255 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0a68147330 con 0x7f0a68064400 2020-11-20T12:16:52.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.376+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/664072255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f0a68147330 con 0x7f0a68064400 2020-11-20T12:16:52.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.380+0000 7f0a5f7fe700 1 -- 172.21.15.116:0/664072255 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v5) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0a68147330 con 0x7f0a68064400 2020-11-20T12:16:52.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0a440557c0 msgr2=0x7f0a44057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:52.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0a440557c0 0x7f0a44057c80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f0a6814b3f0 tx=0x7f0a50009040).stop 2020-11-20T12:16:52.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/664072255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a68064400 msgr2=0x7f0a6809f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:52.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a68064400 0x7f0a6809f250 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f0a5800eba0 tx=0x7f0a58006df0).stop 2020-11-20T12:16:52.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/664072255 shutdown_connections 2020-11-20T12:16:52.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0a440557c0 0x7f0a44057c80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0a680a9910 0x7f0a68149de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0a68064400 0x7f0a6809f250 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 --2- 172.21.15.116:0/664072255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0a680a9490 0x7f0a68146b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/664072255 >> 172.21.15.116:0/664072255 conn(0x7f0a68062520 msgr2=0x7f0a68066900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:52.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/664072255 shutdown_connections 2020-11-20T12:16:52.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.384+0000 7f0a6cbc2700 1 -- 172.21.15.116:0/664072255 wait complete. 2020-11-20T12:16:52.390 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:16:52.409 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2020-11-20T12:16:52.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.584+0000 7fa68f59e700 1 Processor -- start 2020-11-20T12:16:52.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.584+0000 7fa68f59e700 1 -- start start 2020-11-20T12:16:52.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.588+0000 7fa68f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa690064580 0x7fa6900a7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.588+0000 7fa68f59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa690062c50 0x7fa6900a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.588+0000 7fa68f59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6900a8430 con 0x7fa690064960 2020-11-20T12:16:52.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.588+0000 7fa68f59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6901551f0 con 0x7fa690062c50 2020-11-20T12:16:52.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.588+0000 7fa68f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6900a86d0 con 0x7fa690064580 2020-11-20T12:16:52.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.588+0000 7fa68ed9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa690062c50 0x7fa6900a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.588+0000 7fa68ed9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa690062c50 0x7fa6900a62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44422/0 (socket says 172.21.15.116:44422) 2020-11-20T12:16:52.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.588+0000 7fa68ed9d700 1 -- 172.21.15.116:0/3848293480 learned_addr learned my addr 172.21.15.116:0/3848293480 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:52.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.588+0000 7fa68e59c700 1 --2- 172.21.15.116:0/3848293480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa690064580 0x7fa6900a7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.588+0000 7fa68ed9d700 1 -- 172.21.15.116:0/3848293480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa690064580 msgr2=0x7fa6900a7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:52.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68ed9d700 1 --2- 172.21.15.116:0/3848293480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa690064580 0x7fa6900a7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68ed9d700 1 -- 172.21.15.116:0/3848293480 >> v1:172.21.15.173:6789/0 conn(0x7fa690064960 legacy=0x7fa690151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:52.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68ed9d700 1 -- 172.21.15.116:0/3848293480 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6900a8970 con 0x7fa690062c50 2020-11-20T12:16:52.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68ed9d700 1 --2- 172.21.15.116:0/3848293480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa690062c50 0x7fa6900a62b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fa688009d50 tx=0x7fa688009290).ready entity=mon.0 client_cookie=2901c29741163202 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:52.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68d59a700 1 -- 172.21.15.116:0/3848293480 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800bc50 con 0x7fa690062c50 2020-11-20T12:16:52.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68d59a700 1 -- 172.21.15.116:0/3848293480 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68800bdb0 con 0x7fa690062c50 2020-11-20T12:16:52.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68d59a700 1 -- 172.21.15.116:0/3848293480 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa688011610 con 0x7fa690062c50 2020-11-20T12:16:52.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68f59e700 1 -- 172.21.15.116:0/3848293480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa690062c50 msgr2=0x7fa6900a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:52.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68f59e700 1 --2- 172.21.15.116:0/3848293480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa690062c50 0x7fa6900a62b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fa688009d50 tx=0x7fa688009290).stop 2020-11-20T12:16:52.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68f59e700 1 -- 172.21.15.116:0/3848293480 shutdown_connections 2020-11-20T12:16:52.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68f59e700 1 --2- 172.21.15.116:0/3848293480 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa690062c50 0x7fa6900a62b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68f59e700 1 --2- 172.21.15.116:0/3848293480 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa690064580 0x7fa6900a7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68f59e700 1 -- 172.21.15.116:0/3848293480 >> 172.21.15.116:0/3848293480 conn(0x7fa690060c40 msgr2=0x7fa6900c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:52.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68f59e700 1 -- 172.21.15.116:0/3848293480 shutdown_connections 2020-11-20T12:16:52.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.592+0000 7fa68f59e700 1 -- 172.21.15.116:0/3848293480 wait complete. 2020-11-20T12:16:52.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.596+0000 7fa68f59e700 1 Processor -- start 2020-11-20T12:16:52.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.596+0000 7fa68f59e700 1 -- start start 2020-11-20T12:16:52.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa690062c50 0x7fa6900aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68f59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa690064580 0x7fa6900c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68f59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa690064960 0x7fa69014dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68f59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa690161070 con 0x7fa690064580 2020-11-20T12:16:52.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68f59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa690161310 con 0x7fa690064960 2020-11-20T12:16:52.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6901615b0 con 0x7fa690062c50 2020-11-20T12:16:52.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68dd9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa690064580 0x7fa6900c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68dd9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa690064580 0x7fa6900c2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44426/0 (socket says 172.21.15.116:44426) 2020-11-20T12:16:52.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68dd9b700 1 -- 172.21.15.116:0/1617527663 learned_addr learned my addr 172.21.15.116:0/1617527663 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:52.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68ed9d700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa690064960 0x7fa69014dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68e59c700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa690062c50 0x7fa6900aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68dd9b700 1 -- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa690062c50 msgr2=0x7fa6900aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:52.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68dd9b700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa690062c50 0x7fa6900aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68dd9b700 1 -- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa690064960 msgr2=0x7fa69014dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:52.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68dd9b700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa690064960 0x7fa69014dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68dd9b700 1 -- 172.21.15.116:0/1617527663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa690161850 con 0x7fa690064580 2020-11-20T12:16:52.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68dd9b700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa690064580 0x7fa6900c2ca0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fa68400c710 tx=0x7fa684007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:52.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa67e7fc700 1 -- 172.21.15.116:0/1617527663 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68400f470 con 0x7fa690064580 2020-11-20T12:16:52.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa67e7fc700 1 -- 172.21.15.116:0/1617527663 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68400f5d0 con 0x7fa690064580 2020-11-20T12:16:52.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa67e7fc700 1 -- 172.21.15.116:0/1617527663 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68400f970 con 0x7fa690064580 2020-11-20T12:16:52.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68f59e700 1 -- 172.21.15.116:0/1617527663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa690161af0 con 0x7fa690064580 2020-11-20T12:16:52.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.600+0000 7fa68f59e700 1 -- 172.21.15.116:0/1617527663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6900ab0d0 con 0x7fa690064580 2020-11-20T12:16:52.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.604+0000 7fa68f59e700 1 -- 172.21.15.116:0/1617527663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa690161f90 con 0x7fa690064580 2020-11-20T12:16:52.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.604+0000 7fa67e7fc700 1 -- 172.21.15.116:0/1617527663 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa684010070 con 0x7fa690064580 2020-11-20T12:16:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.604+0000 7fa67e7fc700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6640557f0 0x7fa664057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.604+0000 7fa67e7fc700 1 -- 172.21.15.116:0/1617527663 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa68406d690 con 0x7fa690064580 2020-11-20T12:16:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.608+0000 7fa68e59c700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6640557f0 0x7fa664057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:52.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.608+0000 7fa68e59c700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6640557f0 0x7fa664057cb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fa680006fd0 tx=0x7fa680006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:52.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.608+0000 7fa67e7fc700 1 -- 172.21.15.116:0/1617527663 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa684042fd0 con 0x7fa690064580 2020-11-20T12:16:52.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.820+0000 7fa68f59e700 1 -- 172.21.15.116:0/1617527663 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fa690044f60 con 0x7fa690064580 2020-11-20T12:16:52.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.828+0000 7fa67e7fc700 1 -- 172.21.15.116:0/1617527663 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v6) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa690161f90 con 0x7fa690064580 2020-11-20T12:16:52.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.828+0000 7fa68f59e700 1 -- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6640557f0 msgr2=0x7fa664057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:52.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.828+0000 7fa68f59e700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6640557f0 0x7fa664057cb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fa680006fd0 tx=0x7fa680006df0).stop 2020-11-20T12:16:52.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.828+0000 7fa68f59e700 1 -- 172.21.15.116:0/1617527663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa690064580 msgr2=0x7fa6900c2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:52.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.828+0000 7fa68f59e700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa690064580 0x7fa6900c2ca0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fa68400c710 tx=0x7fa684007bd0).stop 2020-11-20T12:16:52.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.828+0000 7fa68f59e700 1 -- 172.21.15.116:0/1617527663 shutdown_connections 2020-11-20T12:16:52.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.828+0000 7fa68f59e700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6640557f0 0x7fa664057cb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.828+0000 7fa68f59e700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa690064960 0x7fa69014dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.828+0000 7fa68f59e700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa690062c50 0x7fa6900aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.828+0000 7fa68f59e700 1 --2- 172.21.15.116:0/1617527663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa690064580 0x7fa6900c2ca0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:52.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.828+0000 7fa68f59e700 1 -- 172.21.15.116:0/1617527663 >> 172.21.15.116:0/1617527663 conn(0x7fa690060c40 msgr2=0x7fa69014ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:52.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.832+0000 7fa68f59e700 1 -- 172.21.15.116:0/1617527663 shutdown_connections 2020-11-20T12:16:52.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:52.832+0000 7fa68f59e700 1 -- 172.21.15.116:0/1617527663 wait complete. 2020-11-20T12:16:52.834 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:16:52.856 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2020-11-20T12:16:53.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.028+0000 7fa19d04d700 1 Processor -- start 2020-11-20T12:16:53.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.028+0000 7fa19d04d700 1 -- start start 2020-11-20T12:16:53.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.028+0000 7fa19d04d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa1980aa790 0x7fa198152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.028+0000 7fa19d04d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa198152a30 con 0x7fa1980612f0 2020-11-20T12:16:53.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.028+0000 7fa19d04d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1980aac10 con 0x7fa19813aee0 2020-11-20T12:16:53.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.028+0000 7fa19d04d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1980adce0 con 0x7fa1980aa790 2020-11-20T12:16:53.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa18ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa1980aa790 0x7fa198152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa197fff700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fa19813aee0 0x7fa1980ad5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:50096/0 (socket says 172.21.15.116:50096) 2020-11-20T12:16:53.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa197fff700 1 -- 172.21.15.116:0/3850809625 learned_addr learned my addr 172.21.15.116:0/3850809625 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:53.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa19c84c700 1 --1- 172.21.15.116:0/3850809625 >> v1:172.21.15.116:6789/0 conn(0x7fa1980612f0 0x7fa1980a7ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53870/0 (socket says 172.21.15.116:53870) 2020-11-20T12:16:53.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa1977fe700 1 -- 172.21.15.116:0/3850809625 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2211065532 0 0) 0x7fa198152a30 con 0x7fa1980612f0 2020-11-20T12:16:53.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa1977fe700 1 -- 172.21.15.116:0/3850809625 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa178003240 con 0x7fa1980612f0 2020-11-20T12:16:53.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa18ffff700 1 -- 172.21.15.116:0/3850809625 >> v1:172.21.15.173:6789/0 conn(0x7fa19813aee0 legacy=0x7fa1980ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa1977fe700 1 -- 172.21.15.116:0/3850809625 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4173725891 0 0) 0x7fa1980aac10 con 0x7fa19813aee0 2020-11-20T12:16:53.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa18ffff700 1 -- 172.21.15.116:0/3850809625 >> v1:172.21.15.116:6789/0 conn(0x7fa1980612f0 legacy=0x7fa1980a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa18ffff700 1 -- 172.21.15.116:0/3850809625 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1980aaeb0 con 0x7fa1980aa790 2020-11-20T12:16:53.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa18ffff700 1 --2- 172.21.15.116:0/3850809625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1980aa790 0x7fa198152540 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fa18000b9a0 tx=0x7fa180006df0).ready entity=mon.0 client_cookie=7ee5c8f6c44446cf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:53.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa1977fe700 1 -- 172.21.15.116:0/3850809625 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1800097b0 con 0x7fa1980aa790 2020-11-20T12:16:53.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa1977fe700 1 -- 172.21.15.116:0/3850809625 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa180009910 con 0x7fa1980aa790 2020-11-20T12:16:53.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa1977fe700 1 -- 172.21.15.116:0/3850809625 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa180009c90 con 0x7fa1980aa790 2020-11-20T12:16:53.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa19d04d700 1 -- 172.21.15.116:0/3850809625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1980aa790 msgr2=0x7fa198152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa19d04d700 1 --2- 172.21.15.116:0/3850809625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1980aa790 0x7fa198152540 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fa18000b9a0 tx=0x7fa180006df0).stop 2020-11-20T12:16:53.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa19d04d700 1 -- 172.21.15.116:0/3850809625 shutdown_connections 2020-11-20T12:16:53.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa19d04d700 1 --2- 172.21.15.116:0/3850809625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1980aa790 0x7fa198152540 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa19d04d700 1 -- 172.21.15.116:0/3850809625 >> 172.21.15.116:0/3850809625 conn(0x7fa1980678a0 msgr2=0x7fa198067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:53.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa19d04d700 1 -- 172.21.15.116:0/3850809625 shutdown_connections 2020-11-20T12:16:53.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.032+0000 7fa19d04d700 1 -- 172.21.15.116:0/3850809625 wait complete. 2020-11-20T12:16:53.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.036+0000 7fa19d04d700 1 Processor -- start 2020-11-20T12:16:53.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.036+0000 7fa19d04d700 1 -- start start 2020-11-20T12:16:53.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.036+0000 7fa19d04d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1980612f0 0x7fa1980a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.036+0000 7fa19d04d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa19813aee0 0x7fa1981616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.036+0000 7fa19d04d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa198164390 0x7fa1981647f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.036+0000 7fa19d04d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198160d80 con 0x7fa1980612f0 2020-11-20T12:16:53.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.036+0000 7fa19d04d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198164df0 con 0x7fa19813aee0 2020-11-20T12:16:53.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.036+0000 7fa19d04d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa198165090 con 0x7fa198164390 2020-11-20T12:16:53.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa197fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1980612f0 0x7fa1980a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa197fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1980612f0 0x7fa1980a1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44438/0 (socket says 172.21.15.116:44438) 2020-11-20T12:16:53.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa197fff700 1 -- 172.21.15.116:0/2835054062 learned_addr learned my addr 172.21.15.116:0/2835054062 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:53.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa18ffff700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa19813aee0 0x7fa1981616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa19c84c700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa198164390 0x7fa1981647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa197fff700 1 -- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa198164390 msgr2=0x7fa1981647f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa197fff700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa198164390 0x7fa1981647f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa197fff700 1 -- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa19813aee0 msgr2=0x7fa1981616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa197fff700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa19813aee0 0x7fa1981616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa197fff700 1 -- 172.21.15.116:0/2835054062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198160fb0 con 0x7fa1980612f0 2020-11-20T12:16:53.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa18ffff700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa19813aee0 0x7fa1981616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:53.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa197fff700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1980612f0 0x7fa1980a1cf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fa18800c710 tx=0x7fa188007ce0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:53.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa194ff9700 1 -- 172.21.15.116:0/2835054062 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188007320 con 0x7fa1980612f0 2020-11-20T12:16:53.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa194ff9700 1 -- 172.21.15.116:0/2835054062 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa188007480 con 0x7fa1980612f0 2020-11-20T12:16:53.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa19d04d700 1 -- 172.21.15.116:0/2835054062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa198161250 con 0x7fa1980612f0 2020-11-20T12:16:53.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa194ff9700 1 -- 172.21.15.116:0/2835054062 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188005800 con 0x7fa1980612f0 2020-11-20T12:16:53.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.040+0000 7fa19d04d700 1 -- 172.21.15.116:0/2835054062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa19809f280 con 0x7fa1980612f0 2020-11-20T12:16:53.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.044+0000 7fa19d04d700 1 -- 172.21.15.116:0/2835054062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa198165250 con 0x7fa1980612f0 2020-11-20T12:16:53.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.044+0000 7fa194ff9700 1 -- 172.21.15.116:0/2835054062 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa188010030 con 0x7fa1980612f0 2020-11-20T12:16:53.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.044+0000 7fa194ff9700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1740557c0 0x7fa174057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.044+0000 7fa194ff9700 1 -- 172.21.15.116:0/2835054062 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa18806e1a0 con 0x7fa1980612f0 2020-11-20T12:16:53.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.044+0000 7fa18ffff700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1740557c0 0x7fa174057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.044+0000 7fa18ffff700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1740557c0 0x7fa174057c80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fa198062a70 tx=0x7fa180015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:53.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.052+0000 7fa194ff9700 1 -- 172.21.15.116:0/2835054062 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa188044260 con 0x7fa1980612f0 2020-11-20T12:16:53.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.260+0000 7fa19d04d700 1 -- 172.21.15.116:0/2835054062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fa198044f60 con 0x7fa1980612f0 2020-11-20T12:16:53.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa194ff9700 1 -- 172.21.15.116:0/2835054062 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa188047880 con 0x7fa1980612f0 2020-11-20T12:16:53.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 -- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1740557c0 msgr2=0x7fa174057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1740557c0 0x7fa174057c80 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7fa198062a70 tx=0x7fa180015000).stop 2020-11-20T12:16:53.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 -- 172.21.15.116:0/2835054062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1980612f0 msgr2=0x7fa1980a1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1980612f0 0x7fa1980a1cf0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fa18800c710 tx=0x7fa188007ce0).stop 2020-11-20T12:16:53.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 -- 172.21.15.116:0/2835054062 shutdown_connections 2020-11-20T12:16:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1740557c0 0x7fa174057c80 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa198164390 0x7fa1981647f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa1980612f0 0x7fa1980a1cf0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 --2- 172.21.15.116:0/2835054062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa19813aee0 0x7fa1981616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 -- 172.21.15.116:0/2835054062 >> 172.21.15.116:0/2835054062 conn(0x7fa1980678a0 msgr2=0x7fa1980c6ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:53.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 -- 172.21.15.116:0/2835054062 shutdown_connections 2020-11-20T12:16:53.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.268+0000 7fa19d04d700 1 -- 172.21.15.116:0/2835054062 wait complete. 2020-11-20T12:16:53.273 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:16:53.296 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2020-11-20T12:16:53.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.480+0000 7f1e1a793700 1 Processor -- start 2020-11-20T12:16:53.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.480+0000 7f1e1a793700 1 -- start start 2020-11-20T12:16:53.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e1a793700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e140aa790 0x7f1e14152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e1a793700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e14152a30 con 0x7f1e140612f0 2020-11-20T12:16:53.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e1a793700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e140aac10 con 0x7f1e1413aee0 2020-11-20T12:16:53.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e1a793700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e140adce0 con 0x7f1e140aa790 2020-11-20T12:16:53.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e19f92700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1e140612f0 0x7f1e140a7ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53882/0 (socket says 172.21.15.116:53882) 2020-11-20T12:16:53.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e19f92700 1 -- 172.21.15.116:0/2315940032 learned_addr learned my addr 172.21.15.116:0/2315940032 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:53.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e18f90700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e140aa790 0x7f1e14152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 714963307 0 0) 0x7f1e14152a30 con 0x7f1e140612f0 2020-11-20T12:16:53.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dfc003240 con 0x7f1e140612f0 2020-11-20T12:16:53.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537196598 0 0) 0x7f1e140aac10 con 0x7f1e1413aee0 2020-11-20T12:16:53.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e14152a30 con 0x7f1e1413aee0 2020-11-20T12:16:53.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 767823499 0 0) 0x7f1dfc003240 con 0x7f1e140612f0 2020-11-20T12:16:53.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e140aa790 msgr2=0x7f1e14152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:53.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e140aa790 0x7f1e14152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 >> v1:172.21.15.173:6789/0 conn(0x7f1e1413aee0 legacy=0x7f1e140ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e140aaeb0 con 0x7f1e140612f0 2020-11-20T12:16:53.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f1e0c001860 con 0x7f1e140612f0 2020-11-20T12:16:53.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 >> v1:172.21.15.116:6789/0 conn(0x7f1e140612f0 legacy=0x7f1e140a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc0038f0 0x7f1dfc003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dfc0044e0 0x7f1dfc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e140aa790 0x7f1dfc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e140adce0 con 0x7f1dfc0038f0 2020-11-20T12:16:53.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc005340 con 0x7f1dfc0044e0 2020-11-20T12:16:53.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dfc00b730 con 0x7f1e140aa790 2020-11-20T12:16:53.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e19791700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc0038f0 0x7f1dfc003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e18f90700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dfc0044e0 0x7f1dfc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.484+0000 7f1e19f92700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e140aa790 0x7f1dfc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e19791700 1 -- 172.21.15.116:0/2315940032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e140aa790 msgr2=0x7f1dfc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:53.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e19791700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e140aa790 0x7f1dfc004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e19791700 1 -- 172.21.15.116:0/2315940032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dfc0044e0 msgr2=0x7f1dfc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:53.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e19791700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1dfc0044e0 0x7f1dfc004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e19791700 1 -- 172.21.15.116:0/2315940032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dfc003240 con 0x7f1dfc0038f0 2020-11-20T12:16:53.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e19791700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc0038f0 0x7f1dfc003cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f1e10001a80 tx=0x7f1e10007b90).ready entity=mon.0 client_cookie=e5271fa1c282865f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:53.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e10007020 con 0x7f1dfc0038f0 2020-11-20T12:16:53.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e10007180 con 0x7f1dfc0038f0 2020-11-20T12:16:53.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e03fff700 1 -- 172.21.15.116:0/2315940032 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1000c830 con 0x7f1dfc0038f0 2020-11-20T12:16:53.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e1a793700 1 -- 172.21.15.116:0/2315940032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc0038f0 msgr2=0x7f1dfc003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e1a793700 1 --2- 172.21.15.116:0/2315940032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1dfc0038f0 0x7f1dfc003cf0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f1e10001a80 tx=0x7f1e10007b90).stop 2020-11-20T12:16:53.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e19f92700 1 -- 172.21.15.116:0/2315940032 reap_dead start 2020-11-20T12:16:53.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e1a793700 1 -- 172.21.15.116:0/2315940032 shutdown_connections 2020-11-20T12:16:53.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e1a793700 1 -- 172.21.15.116:0/2315940032 >> 172.21.15.116:0/2315940032 conn(0x7f1e140678a0 msgr2=0x7f1e14067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:53.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.488+0000 7f1e1a793700 1 -- 172.21.15.116:0/2315940032 shutdown_connections 2020-11-20T12:16:53.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.492+0000 7f1e1a793700 1 -- 172.21.15.116:0/2315940032 wait complete. 2020-11-20T12:16:53.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.492+0000 7f1e1a793700 1 Processor -- start 2020-11-20T12:16:53.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.492+0000 7f1e1a793700 1 -- start start 2020-11-20T12:16:53.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e1a793700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e14139740 0x7f1e141612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e1a793700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e140a0190 0x7f1e140a2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e1a793700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e140a2560 0x7f1e14163e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e1a793700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e1409fbd0 con 0x7f1e140a2560 2020-11-20T12:16:53.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e1a793700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e141606f0 con 0x7f1e140a0190 2020-11-20T12:16:53.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e1a793700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e140a0630 con 0x7f1e14139740 2020-11-20T12:16:53.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e19f92700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e140a2560 0x7f1e14163e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e19f92700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e140a2560 0x7f1e14163e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44462/0 (socket says 172.21.15.116:44462) 2020-11-20T12:16:53.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e19f92700 1 -- 172.21.15.116:0/1711371806 learned_addr learned my addr 172.21.15.116:0/1711371806 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:53.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e19791700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e14139740 0x7f1e141612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e18f90700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e140a0190 0x7f1e140a2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e19f92700 1 -- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e14139740 msgr2=0x7f1e141612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e19f92700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e14139740 0x7f1e141612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e19f92700 1 -- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e140a0190 msgr2=0x7f1e140a2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e19f92700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e140a0190 0x7f1e140a2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e19f92700 1 -- 172.21.15.116:0/1711371806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e140a08d0 con 0x7f1e140a2560 2020-11-20T12:16:53.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e19f92700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e140a2560 0x7f1e14163e40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f1e0c001310 tx=0x7f1e0c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:53.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e017fa700 1 -- 172.21.15.116:0/1711371806 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c00fbe0 con 0x7f1e140a2560 2020-11-20T12:16:53.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e017fa700 1 -- 172.21.15.116:0/1711371806 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e0c00fd40 con 0x7f1e140a2560 2020-11-20T12:16:53.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.496+0000 7f1e1a793700 1 -- 172.21.15.116:0/1711371806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e140a0b70 con 0x7f1e140a2560 2020-11-20T12:16:53.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.500+0000 7f1e017fa700 1 -- 172.21.15.116:0/1711371806 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e0c0125a0 con 0x7f1e140a2560 2020-11-20T12:16:53.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.500+0000 7f1e1a793700 1 -- 172.21.15.116:0/1711371806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e140a10b0 con 0x7f1e140a2560 2020-11-20T12:16:53.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.500+0000 7f1e1a793700 1 -- 172.21.15.116:0/1711371806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e14044f60 con 0x7f1e140a2560 2020-11-20T12:16:53.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.500+0000 7f1e017fa700 1 -- 172.21.15.116:0/1711371806 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1e0c01b030 con 0x7f1e140a2560 2020-11-20T12:16:53.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.500+0000 7f1e017fa700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1dec0557c0 0x7f1dec057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.500+0000 7f1e017fa700 1 -- 172.21.15.116:0/1711371806 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f1e0c06fdb0 con 0x7f1e140a2560 2020-11-20T12:16:53.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.504+0000 7f1e19791700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1dec0557c0 0x7f1dec057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.504+0000 7f1e19791700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1dec0557c0 0x7f1dec057c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f1e1405e230 tx=0x7f1e10016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:53.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.508+0000 7f1e017fa700 1 -- 172.21.15.116:0/1711371806 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1e0c073060 con 0x7f1e140a2560 2020-11-20T12:16:53.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.724+0000 7f1e1a793700 1 -- 172.21.15.116:0/1711371806 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f1e140a1e50 con 0x7f1e140a2560 2020-11-20T12:16:53.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.736+0000 7f1e017fa700 1 -- 172.21.15.116:0/1711371806 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1e0c019030 con 0x7f1e140a2560 2020-11-20T12:16:53.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.736+0000 7f1e1a793700 1 -- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1dec0557c0 msgr2=0x7f1dec057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.736+0000 7f1e1a793700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1dec0557c0 0x7f1dec057c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f1e1405e230 tx=0x7f1e10016000).stop 2020-11-20T12:16:53.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.736+0000 7f1e1a793700 1 -- 172.21.15.116:0/1711371806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e140a2560 msgr2=0x7f1e14163e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.736+0000 7f1e1a793700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e140a2560 0x7f1e14163e40 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f1e0c001310 tx=0x7f1e0c007ad0).stop 2020-11-20T12:16:53.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.736+0000 7f1e1a793700 1 -- 172.21.15.116:0/1711371806 shutdown_connections 2020-11-20T12:16:53.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.736+0000 7f1e1a793700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1dec0557c0 0x7f1dec057c80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.736+0000 7f1e1a793700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e140a2560 0x7f1e14163e40 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.736+0000 7f1e1a793700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e14139740 0x7f1e141612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.736+0000 7f1e1a793700 1 --2- 172.21.15.116:0/1711371806 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e140a0190 0x7f1e140a2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.736+0000 7f1e1a793700 1 -- 172.21.15.116:0/1711371806 >> 172.21.15.116:0/1711371806 conn(0x7f1e140678a0 msgr2=0x7f1e140617a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:53.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.740+0000 7f1e1a793700 1 -- 172.21.15.116:0/1711371806 shutdown_connections 2020-11-20T12:16:53.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.740+0000 7f1e1a793700 1 -- 172.21.15.116:0/1711371806 wait complete. 2020-11-20T12:16:53.742 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:16:53.763 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:53.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.944+0000 7fe2495c7700 1 Processor -- start 2020-11-20T12:16:53.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.944+0000 7fe2495c7700 1 -- start start 2020-11-20T12:16:53.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.948+0000 7fe2495c7700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe244149900 0x7fe24414f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.948+0000 7fe2495c7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe244153960 0x7fe244151f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.948+0000 7fe2495c7700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe244150090 con 0x7fe244150ab0 2020-11-20T12:16:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.948+0000 7fe2495c7700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24414fe70 con 0x7fe244149900 2020-11-20T12:16:53.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.948+0000 7fe2495c7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe244150330 con 0x7fe244153960 2020-11-20T12:16:53.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.948+0000 7fe242ffd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe244149900 0x7fe24414f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.948+0000 7fe2427fc700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fe244150ab0 0x7fe244067700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:50132/0 (socket says 172.21.15.116:50132) 2020-11-20T12:16:53.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.948+0000 7fe242ffd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe244149900 0x7fe24414f980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:43518/0 (socket says 172.21.15.116:43518) 2020-11-20T12:16:53.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.948+0000 7fe243fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe244153960 0x7fe244151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.948+0000 7fe2427fc700 1 -- 172.21.15.116:0/808117057 learned_addr learned my addr 172.21.15.116:0/808117057 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:53.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe242ffd700 1 -- 172.21.15.116:0/808117057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe244153960 msgr2=0x7fe244151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:53.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe242ffd700 1 --2- 172.21.15.116:0/808117057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe244153960 0x7fe244151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe242ffd700 1 -- 172.21.15.116:0/808117057 >> v1:172.21.15.173:6789/0 conn(0x7fe244150ab0 legacy=0x7fe244067700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe242ffd700 1 -- 172.21.15.116:0/808117057 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2441505d0 con 0x7fe244149900 2020-11-20T12:16:53.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe242ffd700 1 --2- 172.21.15.116:0/808117057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244149900 0x7fe24414f980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fe23400b9a0 tx=0x7fe234006df0).ready entity=mon.1 client_cookie=bddfc32ae3903b62 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:53.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe241ffb700 1 -- 172.21.15.116:0/808117057 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340097b0 con 0x7fe244149900 2020-11-20T12:16:53.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe241ffb700 1 -- 172.21.15.116:0/808117057 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234009910 con 0x7fe244149900 2020-11-20T12:16:53.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe241ffb700 1 -- 172.21.15.116:0/808117057 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234009c90 con 0x7fe244149900 2020-11-20T12:16:53.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe2495c7700 1 -- 172.21.15.116:0/808117057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244149900 msgr2=0x7fe24414f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe2495c7700 1 --2- 172.21.15.116:0/808117057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244149900 0x7fe24414f980 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7fe23400b9a0 tx=0x7fe234006df0).stop 2020-11-20T12:16:53.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe2495c7700 1 -- 172.21.15.116:0/808117057 shutdown_connections 2020-11-20T12:16:53.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe2495c7700 1 --2- 172.21.15.116:0/808117057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe244153960 0x7fe244151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe2495c7700 1 --2- 172.21.15.116:0/808117057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244149900 0x7fe24414f980 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe2495c7700 1 -- 172.21.15.116:0/808117057 >> 172.21.15.116:0/808117057 conn(0x7fe2440c5670 msgr2=0x7fe2440c5a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:53.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe2495c7700 1 -- 172.21.15.116:0/808117057 shutdown_connections 2020-11-20T12:16:53.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.952+0000 7fe2495c7700 1 -- 172.21.15.116:0/808117057 wait complete. 2020-11-20T12:16:53.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe2495c7700 1 Processor -- start 2020-11-20T12:16:53.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe2495c7700 1 -- start start 2020-11-20T12:16:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe2495c7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe244149900 0x7fe24414d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe2495c7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe244150ab0 0x7fe2441524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe2495c7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244153960 0x7fe2441529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe2495c7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe244152ed0 con 0x7fe244150ab0 2020-11-20T12:16:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe2495c7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe244165690 con 0x7fe244153960 2020-11-20T12:16:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe2495c7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2441615e0 con 0x7fe244149900 2020-11-20T12:16:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe2427fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe244150ab0 0x7fe2441524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe2427fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe244150ab0 0x7fe2441524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44474/0 (socket says 172.21.15.116:44474) 2020-11-20T12:16:53.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe2427fc700 1 -- 172.21.15.116:0/3682024756 learned_addr learned my addr 172.21.15.116:0/3682024756 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:53.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.956+0000 7fe242ffd700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe244149900 0x7fe24414d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe243fff700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244153960 0x7fe2441529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe2427fc700 1 -- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe244149900 msgr2=0x7fe24414d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe2427fc700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe244149900 0x7fe24414d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe2427fc700 1 -- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244153960 msgr2=0x7fe2441529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:53.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe2427fc700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244153960 0x7fe2441529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:53.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe2427fc700 1 -- 172.21.15.116:0/3682024756 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe244161880 con 0x7fe244150ab0 2020-11-20T12:16:53.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe243fff700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244153960 0x7fe2441529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:53.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe242ffd700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe244149900 0x7fe24414d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:53.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe2427fc700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe244150ab0 0x7fe2441524f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fe23800c710 tx=0x7fe238007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:53.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe22affd700 1 -- 172.21.15.116:0/3682024756 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238007320 con 0x7fe244150ab0 2020-11-20T12:16:53.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe22affd700 1 -- 172.21.15.116:0/3682024756 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe238007480 con 0x7fe244150ab0 2020-11-20T12:16:53.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe2495c7700 1 -- 172.21.15.116:0/3682024756 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe244161b80 con 0x7fe244150ab0 2020-11-20T12:16:53.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe22affd700 1 -- 172.21.15.116:0/3682024756 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe238005800 con 0x7fe244150ab0 2020-11-20T12:16:53.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe2495c7700 1 -- 172.21.15.116:0/3682024756 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24414e030 con 0x7fe244150ab0 2020-11-20T12:16:53.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.960+0000 7fe2495c7700 1 -- 172.21.15.116:0/3682024756 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24405ca30 con 0x7fe244150ab0 2020-11-20T12:16:53.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.964+0000 7fe22affd700 1 -- 172.21.15.116:0/3682024756 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe238010030 con 0x7fe244150ab0 2020-11-20T12:16:53.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.964+0000 7fe22affd700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2200557c0 0x7fe220057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:53.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.964+0000 7fe22affd700 1 -- 172.21.15.116:0/3682024756 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe23806dfe0 con 0x7fe244150ab0 2020-11-20T12:16:53.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.964+0000 7fe242ffd700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2200557c0 0x7fe220057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:53.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.964+0000 7fe242ffd700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2200557c0 0x7fe220057c80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fe234006e30 tx=0x7fe234015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:53.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:53.968+0000 7fe22affd700 1 -- 172.21.15.116:0/3682024756 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe238044040 con 0x7fe244150ab0 2020-11-20T12:16:54.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.128+0000 7fe2495c7700 1 -- 172.21.15.116:0/3682024756 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2440a74c0 con 0x7fe244150ab0 2020-11-20T12:16:54.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe22affd700 1 -- 172.21.15.116:0/3682024756 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+13723 (secure 0 0 0) 0x7fe238047660 con 0x7fe244150ab0 2020-11-20T12:16:54.132 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:54.132 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":30,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:53.587003+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:54.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 -- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2200557c0 msgr2=0x7fe220057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2200557c0 0x7fe220057c80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fe234006e30 tx=0x7fe234015000).stop 2020-11-20T12:16:54.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 -- 172.21.15.116:0/3682024756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe244150ab0 msgr2=0x7fe2441524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe244150ab0 0x7fe2441524f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fe23800c710 tx=0x7fe238007c40).stop 2020-11-20T12:16:54.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 -- 172.21.15.116:0/3682024756 shutdown_connections 2020-11-20T12:16:54.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2200557c0 0x7fe220057c80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe244153960 0x7fe2441529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe244149900 0x7fe24414d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 --2- 172.21.15.116:0/3682024756 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe244150ab0 0x7fe2441524f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 -- 172.21.15.116:0/3682024756 >> 172.21.15.116:0/3682024756 conn(0x7fe2440c5670 msgr2=0x7fe24414e730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:54.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 -- 172.21.15.116:0/3682024756 shutdown_connections 2020-11-20T12:16:54.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.132+0000 7fe2495c7700 1 -- 172.21.15.116:0/3682024756 wait complete. 2020-11-20T12:16:54.159 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T12:16:54.160 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:16:54.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d8782700 1 Processor -- start 2020-11-20T12:16:54.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d8782700 1 -- start start 2020-11-20T12:16:54.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d8782700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f09d014bf70 0x7f09d00c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d8782700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f09d014c350 0x7f09d00c0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d8782700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09d00c0730 con 0x7f09d0062c00 2020-11-20T12:16:54.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d8782700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d00c10e0 con 0x7f09d014c350 2020-11-20T12:16:54.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d8782700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09d01524b0 con 0x7f09d014bf70 2020-11-20T12:16:54.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d5d1d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f09d014c350 0x7f09d00c0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d5d1d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f09d014c350 0x7f09d00c0240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44480/0 (socket says 172.21.15.116:44480) 2020-11-20T12:16:54.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d5d1d700 1 -- 172.21.15.116:0/2540407514 learned_addr learned my addr 172.21.15.116:0/2540407514 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:54.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d651e700 1 --2- 172.21.15.116:0/2540407514 >> v2:172.21.15.173:3300/0 conn(0x7f09d014bf70 0x7f09d00c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d5d1d700 1 -- 172.21.15.116:0/2540407514 >> v2:172.21.15.173:3300/0 conn(0x7f09d014bf70 msgr2=0x7f09d00c0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:54.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.340+0000 7f09d5d1d700 1 --2- 172.21.15.116:0/2540407514 >> v2:172.21.15.173:3300/0 conn(0x7f09d014bf70 0x7f09d00c0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d5d1d700 1 -- 172.21.15.116:0/2540407514 >> v1:172.21.15.116:6789/0 conn(0x7f09d0062c00 legacy=0x7f09d014edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d5d1d700 1 -- 172.21.15.116:0/2540407514 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d0152750 con 0x7f09d014c350 2020-11-20T12:16:54.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d551c700 1 -- 172.21.15.116:0/2540407514 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3110893553 0 0) 0x7f09d00c0730 con 0x7f09d0062c00 2020-11-20T12:16:54.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d5d1d700 1 --2- 172.21.15.116:0/2540407514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09d014c350 0x7f09d00c0240 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f09d00c5260 tx=0x7f09c0006eb0).ready entity=mon.0 client_cookie=7ef3823e4e0b6c08 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d551c700 1 -- 172.21.15.116:0/2540407514 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c00098d0 con 0x7f09d014c350 2020-11-20T12:16:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d551c700 1 -- 172.21.15.116:0/2540407514 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09c0009a30 con 0x7f09d014c350 2020-11-20T12:16:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d551c700 1 -- 172.21.15.116:0/2540407514 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09c0009db0 con 0x7f09d014c350 2020-11-20T12:16:54.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d8782700 1 -- 172.21.15.116:0/2540407514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09d014c350 msgr2=0x7f09d00c0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d8782700 1 --2- 172.21.15.116:0/2540407514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09d014c350 0x7f09d00c0240 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f09d00c5260 tx=0x7f09c0006eb0).stop 2020-11-20T12:16:54.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d8782700 1 -- 172.21.15.116:0/2540407514 shutdown_connections 2020-11-20T12:16:54.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d8782700 1 --2- 172.21.15.116:0/2540407514 >> v2:172.21.15.173:3300/0 conn(0x7f09d014bf70 0x7f09d00c0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d8782700 1 --2- 172.21.15.116:0/2540407514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09d014c350 0x7f09d00c0240 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d8782700 1 -- 172.21.15.116:0/2540407514 >> 172.21.15.116:0/2540407514 conn(0x7f09d0060170 msgr2=0x7f09d0060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:54.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d8782700 1 -- 172.21.15.116:0/2540407514 shutdown_connections 2020-11-20T12:16:54.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.344+0000 7f09d8782700 1 -- 172.21.15.116:0/2540407514 wait complete. 2020-11-20T12:16:54.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.348+0000 7f09d8782700 1 Processor -- start 2020-11-20T12:16:54.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.348+0000 7f09d8782700 1 -- start start 2020-11-20T12:16:54.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.348+0000 7f09d8782700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09d0062c00 0x7f09d00bd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d8782700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d014bf70 0x7f09d00ba0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d8782700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09d014c350 0x7f09d00c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d8782700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00bc6d0 con 0x7f09d0062c00 2020-11-20T12:16:54.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d8782700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00bc970 con 0x7f09d014bf70 2020-11-20T12:16:54.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d8782700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d00bcc10 con 0x7f09d014c350 2020-11-20T12:16:54.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d651e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09d0062c00 0x7f09d00bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d651e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09d0062c00 0x7f09d00bd350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44486/0 (socket says 172.21.15.116:44486) 2020-11-20T12:16:54.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d651e700 1 -- 172.21.15.116:0/3364248244 learned_addr learned my addr 172.21.15.116:0/3364248244 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:54.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d6d1f700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09d014c350 0x7f09d00c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d5d1d700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d014bf70 0x7f09d00ba0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d651e700 1 -- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09d014c350 msgr2=0x7f09d00c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d651e700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09d014c350 0x7f09d00c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d651e700 1 -- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d014bf70 msgr2=0x7f09d00ba0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d651e700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d014bf70 0x7f09d00ba0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d651e700 1 -- 172.21.15.116:0/3364248244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d00bceb0 con 0x7f09d0062c00 2020-11-20T12:16:54.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d651e700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09d0062c00 0x7f09d00bd350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f09cc00db60 tx=0x7f09cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:54.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09be7fc700 1 -- 172.21.15.116:0/3364248244 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc009820 con 0x7f09d0062c00 2020-11-20T12:16:54.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09be7fc700 1 -- 172.21.15.116:0/3364248244 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09cc009980 con 0x7f09d0062c00 2020-11-20T12:16:54.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d8782700 1 -- 172.21.15.116:0/3364248244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d00bc3c0 con 0x7f09d0062c00 2020-11-20T12:16:54.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09be7fc700 1 -- 172.21.15.116:0/3364248244 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09cc009d20 con 0x7f09d0062c00 2020-11-20T12:16:54.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d8782700 1 -- 172.21.15.116:0/3364248244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d00ba750 con 0x7f09d0062c00 2020-11-20T12:16:54.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.352+0000 7f09d8782700 1 -- 172.21.15.116:0/3364248244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d00bd040 con 0x7f09d0062c00 2020-11-20T12:16:54.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.356+0000 7f09be7fc700 1 -- 172.21.15.116:0/3364248244 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f09cc013690 con 0x7f09d0062c00 2020-11-20T12:16:54.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.356+0000 7f09be7fc700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a80557c0 0x7f09a8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.356+0000 7f09be7fc700 1 -- 172.21.15.116:0/3364248244 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f09cc06ea60 con 0x7f09d0062c00 2020-11-20T12:16:54.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.356+0000 7f09d5d1d700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a80557c0 0x7f09a8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.356+0000 7f09d5d1d700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a80557c0 0x7f09a8057c80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f09d00c5260 tx=0x7f09c0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:54.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.360+0000 7f09be7fc700 1 -- 172.21.15.116:0/3364248244 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f09cc044c30 con 0x7f09d0062c00 2020-11-20T12:16:54.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.524+0000 7f09d8782700 1 -- 172.21.15.116:0/3364248244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f09d005ca30 con 0x7f09d0062c00 2020-11-20T12:16:54.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.524+0000 7f09be7fc700 1 -- 172.21.15.116:0/3364248244 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v30) v1 ==== 110+0+10 (secure 0 0 0) 0x7f09cc048250 con 0x7f09d0062c00 2020-11-20T12:16:54.526 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:16:54.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 -- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a80557c0 msgr2=0x7f09a8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a80557c0 0x7f09a8057c80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f09d00c5260 tx=0x7f09c0015000).stop 2020-11-20T12:16:54.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 -- 172.21.15.116:0/3364248244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09d0062c00 msgr2=0x7f09d00bd350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09d0062c00 0x7f09d00bd350 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f09cc00db60 tx=0x7f09cc008040).stop 2020-11-20T12:16:54.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 -- 172.21.15.116:0/3364248244 shutdown_connections 2020-11-20T12:16:54.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a80557c0 0x7f09a8057c80 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09d014c350 0x7f09d00c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09d0062c00 0x7f09d00bd350 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 --2- 172.21.15.116:0/3364248244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09d014bf70 0x7f09d00ba0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 -- 172.21.15.116:0/3364248244 >> 172.21.15.116:0/3364248244 conn(0x7f09d0060170 msgr2=0x7f09d014b180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:54.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 -- 172.21.15.116:0/3364248244 shutdown_connections 2020-11-20T12:16:54.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.528+0000 7f09d8782700 1 -- 172.21.15.116:0/3364248244 wait complete. 2020-11-20T12:16:54.549 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:16:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.724+0000 7f1d808df700 1 Processor -- start 2020-11-20T12:16:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.724+0000 7f1d808df700 1 -- start start 2020-11-20T12:16:54.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d808df700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1d7c13ad30 0x7f1d7c0a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d808df700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d7c13b110 0x7f1d7c151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d808df700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d7c152a80 con 0x7f1d7c151fa0 2020-11-20T12:16:54.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d808df700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c152540 con 0x7f1d7c13ad30 2020-11-20T12:16:54.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d808df700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c1527e0 con 0x7f1d7c13b110 2020-11-20T12:16:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d7b59e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1d7c151fa0 0x7f1d7c061250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53936/0 (socket says 172.21.15.116:53936) 2020-11-20T12:16:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d7b59e700 1 -- 172.21.15.116:0/2689802050 learned_addr learned my addr 172.21.15.116:0/2689802050 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:54.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d7a59c700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d7c13b110 0x7f1d7c151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d7ad9d700 1 --2- 172.21.15.116:0/2689802050 >> v2:172.21.15.173:3300/0 conn(0x7f1d7c13ad30 0x7f1d7c0a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2672248844 0 0) 0x7f1d7c152a80 con 0x7f1d7c151fa0 2020-11-20T12:16:54.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d68003240 con 0x7f1d7c151fa0 2020-11-20T12:16:54.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1738218692 0 0) 0x7f1d68003240 con 0x7f1d7c151fa0 2020-11-20T12:16:54.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d7c13b110 msgr2=0x7f1d7c151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d7c13b110 0x7f1d7c151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 >> v2:172.21.15.173:3300/0 conn(0x7f1d7c13ad30 msgr2=0x7f1d7c0a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 --2- 172.21.15.116:0/2689802050 >> v2:172.21.15.173:3300/0 conn(0x7f1d7c13ad30 0x7f1d7c0a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0ac910 con 0x7f1d7c151fa0 2020-11-20T12:16:54.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f1d74001860 con 0x7f1d7c151fa0 2020-11-20T12:16:54.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d7a59c700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d7c13b110 0x7f1d7c151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:54.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 >> v1:172.21.15.116:6789/0 conn(0x7f1d7c151fa0 legacy=0x7f1d7c061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d68003730 0x7f1d68003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d680040b0 0x7f1d68004580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d68004b80 0x7f1d68005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c152540 con 0x7f1d680040b0 2020-11-20T12:16:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c1527e0 con 0x7f1d68004b80 2020-11-20T12:16:54.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.728+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d6800b900 con 0x7f1d68003730 2020-11-20T12:16:54.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d7a59c700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d680040b0 0x7f1d68004580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d7b59e700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d68004b80 0x7f1d68005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d7ad9d700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d68003730 0x7f1d68003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d7a59c700 1 -- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d68003730 msgr2=0x7f1d68003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:54.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d7a59c700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d68003730 0x7f1d68003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d7a59c700 1 -- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d68004b80 msgr2=0x7f1d68005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:54.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d7a59c700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d68004b80 0x7f1d68005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d7a59c700 1 -- 172.21.15.116:0/2689802050 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d68003240 con 0x7f1d680040b0 2020-11-20T12:16:54.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d7a59c700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d680040b0 0x7f1d68004580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f1d7c0c5e80 tx=0x7f1d70006df0).ready entity=mon.0 client_cookie=7fa7833f9e9a4de server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:54.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7000b510 con 0x7f1d680040b0 2020-11-20T12:16:54.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d7000e8a0 con 0x7f1d680040b0 2020-11-20T12:16:54.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d79d9b700 1 -- 172.21.15.116:0/2689802050 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d7000ec20 con 0x7f1d680040b0 2020-11-20T12:16:54.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d808df700 1 -- 172.21.15.116:0/2689802050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d680040b0 msgr2=0x7f1d68004580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d808df700 1 --2- 172.21.15.116:0/2689802050 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d680040b0 0x7f1d68004580 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f1d7c0c5e80 tx=0x7f1d70006df0).stop 2020-11-20T12:16:54.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d7b59e700 1 -- 172.21.15.116:0/2689802050 reap_dead start 2020-11-20T12:16:54.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d808df700 1 -- 172.21.15.116:0/2689802050 shutdown_connections 2020-11-20T12:16:54.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d808df700 1 -- 172.21.15.116:0/2689802050 >> 172.21.15.116:0/2689802050 conn(0x7f1d7c0678a0 msgr2=0x7f1d7c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:54.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d808df700 1 -- 172.21.15.116:0/2689802050 shutdown_connections 2020-11-20T12:16:54.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.732+0000 7f1d808df700 1 -- 172.21.15.116:0/2689802050 wait complete. 2020-11-20T12:16:54.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.736+0000 7f1d808df700 1 Processor -- start 2020-11-20T12:16:54.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.736+0000 7f1d808df700 1 -- start start 2020-11-20T12:16:54.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d808df700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7c151120 0x7f1d7c151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d808df700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d7c0a4470 0x7f1d7c164760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d808df700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d7c164c50 0x7f1d7c0c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d808df700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0a37f0 con 0x7f1d7c151120 2020-11-20T12:16:54.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d808df700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0a3a90 con 0x7f1d7c164c50 2020-11-20T12:16:54.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d808df700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0a3d30 con 0x7f1d7c0a4470 2020-11-20T12:16:54.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7c151120 0x7f1d7c151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7c151120 0x7f1d7c151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44510/0 (socket says 172.21.15.116:44510) 2020-11-20T12:16:54.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7ad9d700 1 -- 172.21.15.116:0/1803486120 learned_addr learned my addr 172.21.15.116:0/1803486120 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:54.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7a59c700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d7c0a4470 0x7f1d7c164760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7b59e700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d7c164c50 0x7f1d7c0c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7ad9d700 1 -- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d7c0a4470 msgr2=0x7f1d7c164760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7ad9d700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d7c0a4470 0x7f1d7c164760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7ad9d700 1 -- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d7c164c50 msgr2=0x7f1d7c0c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7ad9d700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d7c164c50 0x7f1d7c0c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7ad9d700 1 -- 172.21.15.116:0/1803486120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c0a3fd0 con 0x7f1d7c151120 2020-11-20T12:16:54.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7b59e700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d7c164c50 0x7f1d7c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:54.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7a59c700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d7c0a4470 0x7f1d7c164760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:16:54.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d7ad9d700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7c151120 0x7f1d7c151500 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1d6c011cc0 tx=0x7f1d6c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:54.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d62ffd700 1 -- 172.21.15.116:0/1803486120 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c011e70 con 0x7f1d7c151120 2020-11-20T12:16:54.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d62ffd700 1 -- 172.21.15.116:0/1803486120 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c00fd30 con 0x7f1d7c151120 2020-11-20T12:16:54.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d808df700 1 -- 172.21.15.116:0/1803486120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c0a34e0 con 0x7f1d7c151120 2020-11-20T12:16:54.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d62ffd700 1 -- 172.21.15.116:0/1803486120 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c014650 con 0x7f1d7c151120 2020-11-20T12:16:54.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.740+0000 7f1d808df700 1 -- 172.21.15.116:0/1803486120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c1643b0 con 0x7f1d7c151120 2020-11-20T12:16:54.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.744+0000 7f1d808df700 1 -- 172.21.15.116:0/1803486120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7c044f60 con 0x7f1d7c151120 2020-11-20T12:16:54.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.744+0000 7f1d62ffd700 1 -- 172.21.15.116:0/1803486120 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1d6c014a80 con 0x7f1d7c151120 2020-11-20T12:16:54.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.744+0000 7f1d62ffd700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1d580557c0 0x7f1d58057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:54.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.744+0000 7f1d62ffd700 1 -- 172.21.15.116:0/1803486120 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f1d6c06ef50 con 0x7f1d7c151120 2020-11-20T12:16:54.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.744+0000 7f1d7a59c700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1d580557c0 0x7f1d58057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:54.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.744+0000 7f1d7a59c700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1d580557c0 0x7f1d58057c80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f1d7c0c5e80 tx=0x7f1d70019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:54.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.748+0000 7f1d62ffd700 1 -- 172.21.15.116:0/1803486120 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1d6c045120 con 0x7f1d7c151120 2020-11-20T12:16:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.908+0000 7f1d808df700 1 -- 172.21.15.116:0/1803486120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1d7c069320 con 0x7f1d7c151120 2020-11-20T12:16:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.908+0000 7f1d62ffd700 1 -- 172.21.15.116:0/1803486120 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1d6c048740 con 0x7f1d7c151120 2020-11-20T12:16:54.910 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:16:54.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 -- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1d580557c0 msgr2=0x7f1d58057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1d580557c0 0x7f1d58057c80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f1d7c0c5e80 tx=0x7f1d70019000).stop 2020-11-20T12:16:54.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 -- 172.21.15.116:0/1803486120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7c151120 msgr2=0x7f1d7c151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:54.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7c151120 0x7f1d7c151500 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1d6c011cc0 tx=0x7f1d6c006df0).stop 2020-11-20T12:16:54.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 -- 172.21.15.116:0/1803486120 shutdown_connections 2020-11-20T12:16:54.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1d580557c0 0x7f1d58057c80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1d7c164c50 0x7f1d7c0c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1d7c151120 0x7f1d7c151500 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 --2- 172.21.15.116:0/1803486120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1d7c0a4470 0x7f1d7c164760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:54.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 -- 172.21.15.116:0/1803486120 >> 172.21.15.116:0/1803486120 conn(0x7f1d7c0678a0 msgr2=0x7f1d7c0c6480 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:54.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 -- 172.21.15.116:0/1803486120 shutdown_connections 2020-11-20T12:16:54.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:54.912+0000 7f1d808df700 1 -- 172.21.15.116:0/1803486120 wait complete. 2020-11-20T12:16:54.935 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-20T12:16:55.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.112+0000 7faa8e44e700 1 Processor -- start 2020-11-20T12:16:55.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.112+0000 7faa8e44e700 1 -- start start 2020-11-20T12:16:55.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.112+0000 7faa8e44e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7faa880a81a0 0x7faa880aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.112+0000 7faa8e44e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa880ae090 con 0x7faa880aa570 2020-11-20T12:16:55.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.112+0000 7faa8e44e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa881523b0 con 0x7faa88151c30 2020-11-20T12:16:55.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.112+0000 7faa8e44e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa880ab930 con 0x7faa880a81a0 2020-11-20T12:16:55.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa87fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7faa880aa570 0x7faa880ad270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53952/0 (socket says 172.21.15.116:53952) 2020-11-20T12:16:55.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa87fff700 1 -- 172.21.15.116:0/2049500542 learned_addr learned my addr 172.21.15.116:0/2049500542 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa877fe700 1 --2- 172.21.15.116:0/2049500542 >> v2:172.21.15.173:3300/0 conn(0x7faa880a81a0 0x7faa880aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2802163363 0 0) 0x7faa880ae090 con 0x7faa880aa570 2020-11-20T12:16:55.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa74003240 con 0x7faa880aa570 2020-11-20T12:16:55.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2588737633 0 0) 0x7faa74003240 con 0x7faa880aa570 2020-11-20T12:16:55.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 >> v2:172.21.15.173:3300/0 conn(0x7faa880a81a0 msgr2=0x7faa880aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:55.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 --2- 172.21.15.116:0/2049500542 >> v2:172.21.15.173:3300/0 conn(0x7faa880a81a0 0x7faa880aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 >> v1:172.21.15.173:6789/0 conn(0x7faa88151c30 legacy=0x7faa880ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa88152650 con 0x7faa880aa570 2020-11-20T12:16:55.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7faa780014e0 con 0x7faa880aa570 2020-11-20T12:16:55.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 >> v1:172.21.15.116:6789/0 conn(0x7faa880aa570 legacy=0x7faa880ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 --2- 172.21.15.116:0/2049500542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74003730 0x7faa74003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 --2- 172.21.15.116:0/2049500542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 0x7faa74004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 --2- 172.21.15.116:0/2049500542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa74004e50 0x7faa7400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa880ab930 con 0x7faa740044e0 2020-11-20T12:16:55.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa74004260 con 0x7faa74003730 2020-11-20T12:16:55.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7400bc20 con 0x7faa74004e50 2020-11-20T12:16:55.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa877fe700 1 --2- 172.21.15.116:0/2049500542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 0x7faa74004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa87fff700 1 --2- 172.21.15.116:0/2049500542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74003730 0x7faa74003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa877fe700 1 -- 172.21.15.116:0/2049500542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa74004e50 msgr2=0x7faa7400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:55.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa877fe700 1 --2- 172.21.15.116:0/2049500542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa74004e50 0x7faa7400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:16:55.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa877fe700 1 -- 172.21.15.116:0/2049500542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74003730 msgr2=0x7faa74003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa877fe700 1 --2- 172.21.15.116:0/2049500542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74003730 0x7faa74003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa877fe700 1 -- 172.21.15.116:0/2049500542 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa74003240 con 0x7faa740044e0 2020-11-20T12:16:55.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa87fff700 1 --2- 172.21.15.116:0/2049500542 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa74003730 0x7faa74003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:16:55.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa8c9eb700 1 -- 172.21.15.116:0/2049500542 reap_dead start 2020-11-20T12:16:55.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa877fe700 1 --2- 172.21.15.116:0/2049500542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 0x7faa74004960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7faa880a85a0 tx=0x7faa7c006df0).ready entity=mon.0 client_cookie=6e0e30c3130f9a80 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:55.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c00b330 con 0x7faa740044e0 2020-11-20T12:16:55.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7c00eae0 con 0x7faa740044e0 2020-11-20T12:16:55.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.116+0000 7faa86ffd700 1 -- 172.21.15.116:0/2049500542 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7c00ee60 con 0x7faa740044e0 2020-11-20T12:16:55.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.120+0000 7faa8e44e700 1 -- 172.21.15.116:0/2049500542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 msgr2=0x7faa74004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.120+0000 7faa8e44e700 1 --2- 172.21.15.116:0/2049500542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 0x7faa74004960 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7faa880a85a0 tx=0x7faa7c006df0).stop 2020-11-20T12:16:55.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.120+0000 7faa8e44e700 1 -- 172.21.15.116:0/2049500542 shutdown_connections 2020-11-20T12:16:55.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.120+0000 7faa8e44e700 1 --2- 172.21.15.116:0/2049500542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 0x7faa74004960 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.120+0000 7faa8e44e700 1 -- 172.21.15.116:0/2049500542 >> 172.21.15.116:0/2049500542 conn(0x7faa880678a0 msgr2=0x7faa88067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:55.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.120+0000 7faa8e44e700 1 -- 172.21.15.116:0/2049500542 shutdown_connections 2020-11-20T12:16:55.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.120+0000 7faa8e44e700 1 -- 172.21.15.116:0/2049500542 wait complete. 2020-11-20T12:16:55.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.124+0000 7faa8e44e700 1 Processor -- start 2020-11-20T12:16:55.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.124+0000 7faa8e44e700 1 -- start start 2020-11-20T12:16:55.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.124+0000 7faa8e44e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 0x7faa8814cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.124+0000 7faa8e44e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa8814d110 0x7faa8814d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.124+0000 7faa8e44e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa8814dbc0 0x7faa88149be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.124+0000 7faa8e44e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa88160d90 con 0x7faa740044e0 2020-11-20T12:16:55.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.124+0000 7faa8e44e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8814f980 con 0x7faa8814d110 2020-11-20T12:16:55.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.124+0000 7faa8e44e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa8814fc20 con 0x7faa8814dbc0 2020-11-20T12:16:55.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.124+0000 7faa87fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 0x7faa8814cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.124+0000 7faa87fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 0x7faa8814cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44530/0 (socket says 172.21.15.116:44530) 2020-11-20T12:16:55.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.124+0000 7faa87fff700 1 -- 172.21.15.116:0/1654741431 learned_addr learned my addr 172.21.15.116:0/1654741431 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:55.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa877fe700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa8814d110 0x7faa8814d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa8c9eb700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa8814dbc0 0x7faa88149be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa87fff700 1 -- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa8814dbc0 msgr2=0x7faa88149be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa87fff700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa8814dbc0 0x7faa88149be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa87fff700 1 -- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa8814d110 msgr2=0x7faa8814d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa87fff700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa8814d110 0x7faa8814d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa87fff700 1 -- 172.21.15.116:0/1654741431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8814fec0 con 0x7faa740044e0 2020-11-20T12:16:55.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa87fff700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 0x7faa8814cc20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7faa7800dc20 tx=0x7faa78009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:55.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa67fff700 1 -- 172.21.15.116:0/1654741431 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa78007170 con 0x7faa740044e0 2020-11-20T12:16:55.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa67fff700 1 -- 172.21.15.116:0/1654741431 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa780072d0 con 0x7faa740044e0 2020-11-20T12:16:55.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa8e44e700 1 -- 172.21.15.116:0/1654741431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa88156310 con 0x7faa740044e0 2020-11-20T12:16:55.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa67fff700 1 -- 172.21.15.116:0/1654741431 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7800ba70 con 0x7faa740044e0 2020-11-20T12:16:55.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa8e44e700 1 -- 172.21.15.116:0/1654741431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8814e2b0 con 0x7faa740044e0 2020-11-20T12:16:55.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.128+0000 7faa8e44e700 1 -- 172.21.15.116:0/1654741431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa8805ca30 con 0x7faa740044e0 2020-11-20T12:16:55.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.132+0000 7faa67fff700 1 -- 172.21.15.116:0/1654741431 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7faa78004500 con 0x7faa740044e0 2020-11-20T12:16:55.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.132+0000 7faa67fff700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa600557c0 0x7faa60057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.132+0000 7faa67fff700 1 -- 172.21.15.116:0/1654741431 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5253+0+0 (secure 0 0 0) 0x7faa7806e6a0 con 0x7faa740044e0 2020-11-20T12:16:55.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.132+0000 7faa877fe700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa600557c0 0x7faa60057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.132+0000 7faa877fe700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa600557c0 0x7faa60057c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7faa7c006d00 tx=0x7faa7c016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:55.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.136+0000 7faa67fff700 1 -- 172.21.15.116:0/1654741431 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7faa78044870 con 0x7faa740044e0 2020-11-20T12:16:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.296+0000 7faa8e44e700 1 -- 172.21.15.116:0/1654741431 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faa880605e0 con 0x7faa740044e0 2020-11-20T12:16:55.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.296+0000 7faa67fff700 1 -- 172.21.15.116:0/1654741431 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+143 (secure 0 0 0) 0x7faa78047e90 con 0x7faa740044e0 2020-11-20T12:16:55.297 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:55.298 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2020-11-20T12:16:55.298 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:55.298 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2020-11-20T12:16:55.298 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "device_health_metrics" 2020-11-20T12:16:55.299 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:16:55.299 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:16:55.299 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 2, 2020-11-20T12:16:55.299 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "rbd" 2020-11-20T12:16:55.299 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:16:55.300 INFO:teuthology.orchestra.run.smithi116.stdout:] 2020-11-20T12:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 -- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa600557c0 msgr2=0x7faa60057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa600557c0 0x7faa60057c80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7faa7c006d00 tx=0x7faa7c016000).stop 2020-11-20T12:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 -- 172.21.15.116:0/1654741431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 msgr2=0x7faa8814cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 0x7faa8814cc20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7faa7800dc20 tx=0x7faa78009040).stop 2020-11-20T12:16:55.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 -- 172.21.15.116:0/1654741431 shutdown_connections 2020-11-20T12:16:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa600557c0 0x7faa60057c80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa8814dbc0 0x7faa88149be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740044e0 0x7faa8814cc20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 --2- 172.21.15.116:0/1654741431 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa8814d110 0x7faa8814d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 -- 172.21.15.116:0/1654741431 >> 172.21.15.116:0/1654741431 conn(0x7faa880678a0 msgr2=0x7faa88154100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:55.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 -- 172.21.15.116:0/1654741431 shutdown_connections 2020-11-20T12:16:55.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.300+0000 7faa8e44e700 1 -- 172.21.15.116:0/1654741431 wait complete. 2020-11-20T12:16:55.323 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T12:16:55.324 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-20T12:16:55.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.520+0000 7ff369f4e700 1 Processor -- start 2020-11-20T12:16:55.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.520+0000 7ff369f4e700 1 -- start start 2020-11-20T12:16:55.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.520+0000 7ff369f4e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff3640a23e0 0x7ff364149350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.520+0000 7ff369f4e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff36414fbf0 con 0x7ff3640a2000 2020-11-20T12:16:55.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff369f4e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff36414fe90 con 0x7ff364149840 2020-11-20T12:16:55.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff369f4e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff364150130 con 0x7ff3640a23e0 2020-11-20T12:16:55.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff368f4c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff3640a2000 0x7ff3640a40c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:53974/0 (socket says 172.21.15.116:53974) 2020-11-20T12:16:55.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff368f4c700 1 -- 172.21.15.116:0/2763519715 learned_addr learned my addr 172.21.15.116:0/2763519715 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:55.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff363fff700 1 --2- 172.21.15.116:0/2763519715 >> v2:172.21.15.173:3300/0 conn(0x7ff3640a23e0 0x7ff364149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff3637fe700 1 -- 172.21.15.116:0/2763519715 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565528478 0 0) 0x7ff36414fbf0 con 0x7ff3640a2000 2020-11-20T12:16:55.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff3637fe700 1 -- 172.21.15.116:0/2763519715 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff348003240 con 0x7ff3640a2000 2020-11-20T12:16:55.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff363fff700 1 -- 172.21.15.116:0/2763519715 >> v1:172.21.15.173:6789/0 conn(0x7ff364149840 legacy=0x7ff3640671a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff363fff700 1 -- 172.21.15.116:0/2763519715 >> v1:172.21.15.116:6789/0 conn(0x7ff3640a2000 legacy=0x7ff3640a40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff363fff700 1 -- 172.21.15.116:0/2763519715 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3641503d0 con 0x7ff3640a23e0 2020-11-20T12:16:55.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff363fff700 1 --2- 172.21.15.116:0/2763519715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3640a23e0 0x7ff364149350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7ff35000b9a0 tx=0x7ff350006df0).ready entity=mon.1 client_cookie=5932a521b32583ec server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:55.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff3637fe700 1 -- 172.21.15.116:0/2763519715 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff350009840 con 0x7ff3640a23e0 2020-11-20T12:16:55.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff3637fe700 1 -- 172.21.15.116:0/2763519715 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3500099a0 con 0x7ff3640a23e0 2020-11-20T12:16:55.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.524+0000 7ff3637fe700 1 -- 172.21.15.116:0/2763519715 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff350009d20 con 0x7ff3640a23e0 2020-11-20T12:16:55.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 -- 172.21.15.116:0/2763519715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3640a23e0 msgr2=0x7ff364149350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 --2- 172.21.15.116:0/2763519715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3640a23e0 0x7ff364149350 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7ff35000b9a0 tx=0x7ff350006df0).stop 2020-11-20T12:16:55.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 -- 172.21.15.116:0/2763519715 shutdown_connections 2020-11-20T12:16:55.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 --2- 172.21.15.116:0/2763519715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3640a23e0 0x7ff364149350 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 -- 172.21.15.116:0/2763519715 >> 172.21.15.116:0/2763519715 conn(0x7ff364061d50 msgr2=0x7ff3640c50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:55.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 -- 172.21.15.116:0/2763519715 shutdown_connections 2020-11-20T12:16:55.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 -- 172.21.15.116:0/2763519715 wait complete. 2020-11-20T12:16:55.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 Processor -- start 2020-11-20T12:16:55.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 -- start start 2020-11-20T12:16:55.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3640a2000 0x7ff3640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3640a23e0 0x7ff3640a8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff364149840 0x7ff364148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3640bc6c0 con 0x7ff364149840 2020-11-20T12:16:55.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3640a4a20 con 0x7ff3640a23e0 2020-11-20T12:16:55.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.528+0000 7ff369f4e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3640a6dd0 con 0x7ff3640a2000 2020-11-20T12:16:55.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff36974d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff364149840 0x7ff364148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff36974d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff364149840 0x7ff364148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44550/0 (socket says 172.21.15.116:44550) 2020-11-20T12:16:55.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff36974d700 1 -- 172.21.15.116:0/2814967890 learned_addr learned my addr 172.21.15.116:0/2814967890 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:55.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff368f4c700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3640a2000 0x7ff3640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff363fff700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3640a23e0 0x7ff3640a8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff36974d700 1 -- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3640a2000 msgr2=0x7ff3640a61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff36974d700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3640a2000 0x7ff3640a61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff36974d700 1 -- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3640a23e0 msgr2=0x7ff3640a8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:55.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff36974d700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3640a23e0 0x7ff3640a8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:55.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff36974d700 1 -- 172.21.15.116:0/2814967890 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3640a4c50 con 0x7ff364149840 2020-11-20T12:16:55.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff36974d700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff364149840 0x7ff364148780 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7ff35c00ca60 tx=0x7ff35c007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:55.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff360ff9700 1 -- 172.21.15.116:0/2814967890 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c00a770 con 0x7ff364149840 2020-11-20T12:16:55.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff360ff9700 1 -- 172.21.15.116:0/2814967890 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff35c00a8d0 con 0x7ff364149840 2020-11-20T12:16:55.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff369f4e700 1 -- 172.21.15.116:0/2814967890 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3640a4f50 con 0x7ff364149840 2020-11-20T12:16:55.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff360ff9700 1 -- 172.21.15.116:0/2814967890 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff35c00ac50 con 0x7ff364149840 2020-11-20T12:16:55.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff369f4e700 1 -- 172.21.15.116:0/2814967890 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3640a56d0 con 0x7ff364149840 2020-11-20T12:16:55.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.532+0000 7ff369f4e700 1 -- 172.21.15.116:0/2814967890 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff364044f60 con 0x7ff364149840 2020-11-20T12:16:55.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.536+0000 7ff360ff9700 1 -- 172.21.15.116:0/2814967890 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff35c019030 con 0x7ff364149840 2020-11-20T12:16:55.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.536+0000 7ff360ff9700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff33c0557c0 0x7ff33c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:55.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.536+0000 7ff360ff9700 1 -- 172.21.15.116:0/2814967890 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5253+0+0 (secure 0 0 0) 0x7ff35c06d940 con 0x7ff364149840 2020-11-20T12:16:55.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.536+0000 7ff368f4c700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff33c0557c0 0x7ff33c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:55.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.536+0000 7ff368f4c700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff33c0557c0 0x7ff33c057c80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7ff358007fd0 tx=0x7ff358009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:55.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.540+0000 7ff360ff9700 1 -- 172.21.15.116:0/2814967890 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff35c043bc0 con 0x7ff364149840 2020-11-20T12:16:55.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:55.700+0000 7ff369f4e700 1 -- 172.21.15.116:0/2814967890 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7ff3640bfee0 con 0x7ff364149840 2020-11-20T12:16:56.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.424+0000 7ff360ff9700 1 -- 172.21.15.116:0/2814967890 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v31) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff35c0471e0 con 0x7ff364149840 2020-11-20T12:16:56.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 -- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff33c0557c0 msgr2=0x7ff33c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:56.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff33c0557c0 0x7ff33c057c80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7ff358007fd0 tx=0x7ff358009040).stop 2020-11-20T12:16:56.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 -- 172.21.15.116:0/2814967890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff364149840 msgr2=0x7ff364148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:56.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff364149840 0x7ff364148780 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7ff35c00ca60 tx=0x7ff35c007f40).stop 2020-11-20T12:16:56.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 -- 172.21.15.116:0/2814967890 shutdown_connections 2020-11-20T12:16:56.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff33c0557c0 0x7ff33c057c80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:56.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff364149840 0x7ff364148780 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:56.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff3640a2000 0x7ff3640a61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:56.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 --2- 172.21.15.116:0/2814967890 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3640a23e0 0x7ff3640a8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:56.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 -- 172.21.15.116:0/2814967890 >> 172.21.15.116:0/2814967890 conn(0x7ff364061d50 msgr2=0x7ff3640c3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:56.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 -- 172.21.15.116:0/2814967890 shutdown_connections 2020-11-20T12:16:56.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.432+0000 7ff369f4e700 1 -- 172.21.15.116:0/2814967890 wait complete. 2020-11-20T12:16:56.436 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2020-11-20T12:16:56.457 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-20T12:16:56.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.636+0000 7f09397f5700 1 Processor -- start 2020-11-20T12:16:56.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.636+0000 7f09397f5700 1 -- start start 2020-11-20T12:16:56.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.636+0000 7f09397f5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f093414f150 0x7f09340c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:56.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.636+0000 7f09397f5700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f093414f530 0x7f09340c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:56.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.636+0000 7f09397f5700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0934155620 con 0x7f09340bd780 2020-11-20T12:16:56.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.636+0000 7f09397f5700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09341558c0 con 0x7f093414f530 2020-11-20T12:16:56.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.636+0000 7f09397f5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0934155b60 con 0x7f093414f150 2020-11-20T12:16:56.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0932ffd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f093414f150 0x7f09340c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:56.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f09337fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f093414f530 0x7f09340c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:56.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0932ffd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f093414f150 0x7f09340c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:53504/0 (socket says 172.21.15.116:53504) 2020-11-20T12:16:56.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f09337fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f093414f530 0x7f09340c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:43610/0 (socket says 172.21.15.116:43610) 2020-11-20T12:16:56.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f09327fc700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f09340bd780 0x7f09340a74b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:50220/0 (socket says 172.21.15.116:50220) 2020-11-20T12:16:56.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0932ffd700 1 -- 172.21.15.116:0/3231965013 learned_addr learned my addr 172.21.15.116:0/3231965013 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:56.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0932ffd700 1 -- 172.21.15.116:0/3231965013 >> v2:172.21.15.173:3300/0 conn(0x7f093414f530 msgr2=0x7f09340c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:56.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0932ffd700 1 --2- 172.21.15.116:0/3231965013 >> v2:172.21.15.173:3300/0 conn(0x7f093414f530 0x7f09340c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:56.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0932ffd700 1 -- 172.21.15.116:0/3231965013 >> v1:172.21.15.173:6789/0 conn(0x7f09340bd780 legacy=0x7f09340a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:56.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0931ffb700 1 -- 172.21.15.116:0/3231965013 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629287510 0 0) 0x7f0934155620 con 0x7f09340bd780 2020-11-20T12:16:56.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0932ffd700 1 -- 172.21.15.116:0/3231965013 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0934155e00 con 0x7f093414f150 2020-11-20T12:16:56.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0932ffd700 1 --2- 172.21.15.116:0/3231965013 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f093414f150 0x7f09340c87f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0924006fd0 tx=0x7f0924006eb0).ready entity=mon.2 client_cookie=c448e03e87d42765 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:56.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0931ffb700 1 -- 172.21.15.116:0/3231965013 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09240098a0 con 0x7f093414f150 2020-11-20T12:16:56.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0931ffb700 1 -- 172.21.15.116:0/3231965013 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0924009a00 con 0x7f093414f150 2020-11-20T12:16:56.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f0931ffb700 1 -- 172.21.15.116:0/3231965013 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924009d80 con 0x7f093414f150 2020-11-20T12:16:56.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f09397f5700 1 -- 172.21.15.116:0/3231965013 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f093414f150 msgr2=0x7f09340c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:56.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f09397f5700 1 --2- 172.21.15.116:0/3231965013 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f093414f150 0x7f09340c87f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0924006fd0 tx=0x7f0924006eb0).stop 2020-11-20T12:16:56.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f09397f5700 1 -- 172.21.15.116:0/3231965013 shutdown_connections 2020-11-20T12:16:56.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f09397f5700 1 --2- 172.21.15.116:0/3231965013 >> v2:172.21.15.173:3300/0 conn(0x7f093414f530 0x7f09340c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:56.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f09397f5700 1 --2- 172.21.15.116:0/3231965013 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f093414f150 0x7f09340c87f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:56.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f09397f5700 1 -- 172.21.15.116:0/3231965013 >> 172.21.15.116:0/3231965013 conn(0x7f0934061580 msgr2=0x7f0934061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:56.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f09397f5700 1 -- 172.21.15.116:0/3231965013 shutdown_connections 2020-11-20T12:16:56.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.640+0000 7f09397f5700 1 -- 172.21.15.116:0/3231965013 wait complete. 2020-11-20T12:16:56.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.644+0000 7f09397f5700 1 Processor -- start 2020-11-20T12:16:56.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.644+0000 7f09397f5700 1 -- start start 2020-11-20T12:16:56.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.644+0000 7f09397f5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09340bd780 0x7f09340c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:56.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.644+0000 7f09397f5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f093414f530 0x7f09340c2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:56.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.644+0000 7f09397f5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09340c22c0 0x7f093414b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:56.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.644+0000 7f09397f5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0934153310 con 0x7f093414f530 2020-11-20T12:16:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.644+0000 7f09397f5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c1220 con 0x7f09340c22c0 2020-11-20T12:16:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.644+0000 7f09397f5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09340c0a40 con 0x7f09340bd780 2020-11-20T12:16:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f09327fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f093414f530 0x7f09340c2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:56.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f09337fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09340c22c0 0x7f093414b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f0932ffd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09340bd780 0x7f09340c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f09327fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f093414f530 0x7f09340c2a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44562/0 (socket says 172.21.15.116:44562) 2020-11-20T12:16:56.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f09327fc700 1 -- 172.21.15.116:0/396823855 learned_addr learned my addr 172.21.15.116:0/396823855 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:56.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f0932ffd700 1 -- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09340c22c0 msgr2=0x7f093414b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:56.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f0932ffd700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09340c22c0 0x7f093414b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:56.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f0932ffd700 1 -- 172.21.15.116:0/396823855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f093414f530 msgr2=0x7f09340c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:56.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f0932ffd700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f093414f530 0x7f09340c2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:56.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f0932ffd700 1 -- 172.21.15.116:0/396823855 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09340c0ce0 con 0x7f09340bd780 2020-11-20T12:16:56.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f09337fe700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09340c22c0 0x7f093414b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:56.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f09327fc700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f093414f530 0x7f09340c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:56.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f0932ffd700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09340bd780 0x7f09340c1dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0924012000 tx=0x7f0924015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:56.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f091affd700 1 -- 172.21.15.116:0/396823855 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0924011d00 con 0x7f09340bd780 2020-11-20T12:16:56.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f09397f5700 1 -- 172.21.15.116:0/396823855 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09340c0f80 con 0x7f09340bd780 2020-11-20T12:16:56.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f091affd700 1 -- 172.21.15.116:0/396823855 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0924011e60 con 0x7f09340bd780 2020-11-20T12:16:56.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f091affd700 1 -- 172.21.15.116:0/396823855 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f092401e670 con 0x7f09340bd780 2020-11-20T12:16:56.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.648+0000 7f09397f5700 1 -- 172.21.15.116:0/396823855 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09340bf850 con 0x7f09340bd780 2020-11-20T12:16:56.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.652+0000 7f09397f5700 1 -- 172.21.15.116:0/396823855 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f093405ca30 con 0x7f09340bd780 2020-11-20T12:16:56.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.652+0000 7f091affd700 1 -- 172.21.15.116:0/396823855 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0924025030 con 0x7f09340bd780 2020-11-20T12:16:56.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.652+0000 7f091affd700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09100557c0 0x7f0910057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:56.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.652+0000 7f091affd700 1 -- 172.21.15.116:0/396823855 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f092406fbb0 con 0x7f09340bd780 2020-11-20T12:16:56.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.652+0000 7f09327fc700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09100557c0 0x7f0910057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:56.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.656+0000 7f09327fc700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09100557c0 0x7f0910057c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f09340c2730 tx=0x7f0928000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:56.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.660+0000 7f091affd700 1 -- 172.21.15.116:0/396823855 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0924045cd0 con 0x7f09340bd780 2020-11-20T12:16:56.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:56.820+0000 7f09397f5700 1 -- 172.21.15.116:0/396823855 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f09340c8ba0 con 0x7f09340bd780 2020-11-20T12:16:57.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.436+0000 7f091affd700 1 -- 172.21.15.116:0/396823855 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v32) v1 ==== 125+0+0 (secure 0 0 0) 0x7f0924023070 con 0x7f09340bd780 2020-11-20T12:16:57.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.440+0000 7f09397f5700 1 -- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09100557c0 msgr2=0x7f0910057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:57.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.440+0000 7f09397f5700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09100557c0 0x7f0910057c80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f09340c2730 tx=0x7f0928000b80).stop 2020-11-20T12:16:57.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.440+0000 7f09397f5700 1 -- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09340bd780 msgr2=0x7f09340c1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:57.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.440+0000 7f09397f5700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09340bd780 0x7f09340c1dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f0924012000 tx=0x7f0924015000).stop 2020-11-20T12:16:57.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.440+0000 7f09397f5700 1 -- 172.21.15.116:0/396823855 shutdown_connections 2020-11-20T12:16:57.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.440+0000 7f09397f5700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09100557c0 0x7f0910057c80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.440+0000 7f09397f5700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09340c22c0 0x7f093414b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.440+0000 7f09397f5700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09340bd780 0x7f09340c1dd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.440+0000 7f09397f5700 1 --2- 172.21.15.116:0/396823855 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f093414f530 0x7f09340c2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.440+0000 7f09397f5700 1 -- 172.21.15.116:0/396823855 >> 172.21.15.116:0/396823855 conn(0x7f0934061580 msgr2=0x7f09340c5020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:57.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.444+0000 7f09397f5700 1 -- 172.21.15.116:0/396823855 shutdown_connections 2020-11-20T12:16:57.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.444+0000 7f09397f5700 1 -- 172.21.15.116:0/396823855 wait complete. 2020-11-20T12:16:57.448 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2020-11-20T12:16:57.472 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T12:16:57.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.668+0000 7fe2e0842700 1 Processor -- start 2020-11-20T12:16:57.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.668+0000 7fe2e0842700 1 -- start start 2020-11-20T12:16:57.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.668+0000 7fe2e0842700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 0x7fe2dc0c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:57.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.668+0000 7fe2e0842700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe2dc0c52f0 0x7fe2dc0c33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:57.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.668+0000 7fe2e0842700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2dc0c4b70 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.668+0000 7fe2e0842700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2dc151b00 con 0x7fe2dc0c52f0 2020-11-20T12:16:57.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.668+0000 7fe2e0842700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0c40f0 con 0x7fe2dc0be350 2020-11-20T12:16:57.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2db7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 0x7fe2dc0c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:57.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2daffd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe2dc0c52f0 0x7fe2dc0c33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:57.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2db7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 0x7fe2dc0c4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:53518/0 (socket says 172.21.15.116:53518) 2020-11-20T12:16:57.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2db7fe700 1 -- 172.21.15.116:0/1824404772 learned_addr learned my addr 172.21.15.116:0/1824404772 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:57.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2daffd700 1 -- 172.21.15.116:0/1824404772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 msgr2=0x7fe2dc0c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:57.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2daffd700 1 --2- 172.21.15.116:0/1824404772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 0x7fe2dc0c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2daffd700 1 -- 172.21.15.116:0/1824404772 >> v1:172.21.15.173:6789/0 conn(0x7fe2dc0c38d0 legacy=0x7fe2dc061610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:57.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2da7fc700 1 -- 172.21.15.116:0/1824404772 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1576503692 0 0) 0x7fe2dc0c4b70 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2daffd700 1 -- 172.21.15.116:0/1824404772 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc0c4390 con 0x7fe2dc0c52f0 2020-11-20T12:16:57.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2db7fe700 1 --2- 172.21.15.116:0/1824404772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 0x7fe2dc0c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:57.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2daffd700 1 --2- 172.21.15.116:0/1824404772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2dc0c52f0 0x7fe2dc0c33e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fe2d000b9a0 tx=0x7fe2d0006df0).ready entity=mon.1 client_cookie=f740d5f7263f4fcd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:57.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2da7fc700 1 -- 172.21.15.116:0/1824404772 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d00097b0 con 0x7fe2dc0c52f0 2020-11-20T12:16:57.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2da7fc700 1 -- 172.21.15.116:0/1824404772 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d0009910 con 0x7fe2dc0c52f0 2020-11-20T12:16:57.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2da7fc700 1 -- 172.21.15.116:0/1824404772 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0009c90 con 0x7fe2dc0c52f0 2020-11-20T12:16:57.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2e0842700 1 -- 172.21.15.116:0/1824404772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2dc0c52f0 msgr2=0x7fe2dc0c33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:57.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2e0842700 1 --2- 172.21.15.116:0/1824404772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2dc0c52f0 0x7fe2dc0c33e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fe2d000b9a0 tx=0x7fe2d0006df0).stop 2020-11-20T12:16:57.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2e0842700 1 -- 172.21.15.116:0/1824404772 shutdown_connections 2020-11-20T12:16:57.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2e0842700 1 --2- 172.21.15.116:0/1824404772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 0x7fe2dc0c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2e0842700 1 --2- 172.21.15.116:0/1824404772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2dc0c52f0 0x7fe2dc0c33e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.672+0000 7fe2e0842700 1 -- 172.21.15.116:0/1824404772 >> 172.21.15.116:0/1824404772 conn(0x7fe2dc091900 msgr2=0x7fe2dc07ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:57.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.676+0000 7fe2e0842700 1 -- 172.21.15.116:0/1824404772 shutdown_connections 2020-11-20T12:16:57.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.676+0000 7fe2e0842700 1 -- 172.21.15.116:0/1824404772 wait complete. 2020-11-20T12:16:57.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.676+0000 7fe2e0842700 1 Processor -- start 2020-11-20T12:16:57.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.676+0000 7fe2e0842700 1 -- start start 2020-11-20T12:16:57.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2e0842700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 0x7fe2dc144ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:57.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2e0842700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2dc0c38d0 0x7fe2dc161400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:57.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2e0842700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2dc0abf40 0x7fe2dc0a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:57.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2e0842700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0ac450 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2e0842700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0ac6f0 con 0x7fe2dc0abf40 2020-11-20T12:16:57.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2e0842700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2dc0ac990 con 0x7fe2dc0be350 2020-11-20T12:16:57.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2daffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2dc0c38d0 0x7fe2dc161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:57.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2db7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 0x7fe2dc144ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:57.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2daffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2dc0c38d0 0x7fe2dc161400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44576/0 (socket says 172.21.15.116:44576) 2020-11-20T12:16:57.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2daffd700 1 -- 172.21.15.116:0/1585587998 learned_addr learned my addr 172.21.15.116:0/1585587998 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:57.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2dbfff700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2dc0abf40 0x7fe2dc0a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:57.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2daffd700 1 -- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 msgr2=0x7fe2dc144ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:57.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2daffd700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 0x7fe2dc144ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2daffd700 1 -- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2dc0abf40 msgr2=0x7fe2dc0a95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:57.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2daffd700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2dc0abf40 0x7fe2dc0a95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2daffd700 1 -- 172.21.15.116:0/1585587998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc0a9f10 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2db7fe700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 0x7fe2dc144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:16:57.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2daffd700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2dc0c38d0 0x7fe2dc161400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fe2d0006e30 tx=0x7fe2d0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:57.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.680+0000 7fe2bb7fe700 1 -- 172.21.15.116:0/1585587998 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d0011b20 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.684+0000 7fe2bb7fe700 1 -- 172.21.15.116:0/1585587998 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2d0011c80 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.684+0000 7fe2e0842700 1 -- 172.21.15.116:0/1585587998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2dc0acc30 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.684+0000 7fe2bb7fe700 1 -- 172.21.15.116:0/1585587998 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2d001e460 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.684+0000 7fe2e0842700 1 -- 172.21.15.116:0/1585587998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2dc1618f0 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.684+0000 7fe2e0842700 1 -- 172.21.15.116:0/1585587998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2dc0aa0a0 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.684+0000 7fe2bb7fe700 1 -- 172.21.15.116:0/1585587998 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe2d001a040 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.688+0000 7fe2bb7fe700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2b40557c0 0x7fe2b4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:57.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.688+0000 7fe2bb7fe700 1 -- 172.21.15.116:0/1585587998 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fe2d006fad0 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.688+0000 7fe2db7fe700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2b40557c0 0x7fe2b4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:57.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.688+0000 7fe2db7fe700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2b40557c0 0x7fe2b4057c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fe2cc006fd0 tx=0x7fe2cc006e50).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:57.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.692+0000 7fe2bb7fe700 1 -- 172.21.15.116:0/1585587998 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe2d0045a90 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.896+0000 7fe2e0842700 1 -- 172.21.15.116:0/1585587998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fe2dc044f60 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.912+0000 7fe2bb7fe700 1 -- 172.21.15.116:0/1585587998 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 5 and data pool 6 v11) v1 ==== 187+0+0 (secure 0 0 0) 0x7fe2d0023020 con 0x7fe2dc0c38d0 2020-11-20T12:16:57.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.916+0000 7fe2e0842700 1 -- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2b40557c0 msgr2=0x7fe2b4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:57.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.916+0000 7fe2e0842700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2b40557c0 0x7fe2b4057c80 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fe2cc006fd0 tx=0x7fe2cc006e50).stop 2020-11-20T12:16:57.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.916+0000 7fe2e0842700 1 -- 172.21.15.116:0/1585587998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2dc0c38d0 msgr2=0x7fe2dc161400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:57.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.916+0000 7fe2e0842700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2dc0c38d0 0x7fe2dc161400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fe2d0006e30 tx=0x7fe2d0015000).stop 2020-11-20T12:16:57.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.916+0000 7fe2e0842700 1 -- 172.21.15.116:0/1585587998 shutdown_connections 2020-11-20T12:16:57.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.916+0000 7fe2e0842700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2b40557c0 0x7fe2b4057c80 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.916+0000 7fe2e0842700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2dc0abf40 0x7fe2dc0a95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.916+0000 7fe2e0842700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2dc0be350 0x7fe2dc144ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.916+0000 7fe2e0842700 1 --2- 172.21.15.116:0/1585587998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2dc0c38d0 0x7fe2dc161400 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:57.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.916+0000 7fe2e0842700 1 -- 172.21.15.116:0/1585587998 >> 172.21.15.116:0/1585587998 conn(0x7fe2dc091900 msgr2=0x7fe2dc0a51d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:57.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.924+0000 7fe2e0842700 1 -- 172.21.15.116:0/1585587998 shutdown_connections 2020-11-20T12:16:57.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:57.924+0000 7fe2e0842700 1 -- 172.21.15.116:0/1585587998 wait complete. 2020-11-20T12:16:57.934 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 5 and data pool 6 2020-11-20T12:16:57.955 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:58.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.136+0000 7f1a6f472700 1 Processor -- start 2020-11-20T12:16:58.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.136+0000 7f1a6f472700 1 -- start start 2020-11-20T12:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.136+0000 7f1a6f472700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1a680a3690 0x7f1a68152740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.136+0000 7f1a6f472700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680c2450 0x7f1a680c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.136+0000 7f1a6f472700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a680bed50 con 0x7f1a680be7d0 2020-11-20T12:16:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.136+0000 7f1a6f472700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a680beff0 con 0x7f1a680a3690 2020-11-20T12:16:58.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.136+0000 7f1a6f472700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680bf290 con 0x7f1a680c2450 2020-11-20T12:16:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6ca0d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1a680be7d0 0x7f1a680b9130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54030/0 (socket says 172.21.15.116:54030) 2020-11-20T12:16:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6ca0d700 1 -- 172.21.15.116:0/4089542595 learned_addr learned my addr 172.21.15.116:0/4089542595 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:58.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6d20e700 1 --2- 172.21.15.116:0/4089542595 >> v2:172.21.15.173:3300/0 conn(0x7f1a680a3690 0x7f1a68152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6da0f700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680c2450 0x7f1a680c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204217265 0 0) 0x7f1a680bed50 con 0x7f1a680be7d0 2020-11-20T12:16:58.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a50003240 con 0x7f1a680be7d0 2020-11-20T12:16:58.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2269504778 0 0) 0x7f1a50003240 con 0x7f1a680be7d0 2020-11-20T12:16:58.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680c2450 msgr2=0x7f1a680c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:58.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680c2450 0x7f1a680c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 >> v2:172.21.15.173:3300/0 conn(0x7f1a680a3690 msgr2=0x7f1a68152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:58.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 --2- 172.21.15.116:0/4089542595 >> v2:172.21.15.173:3300/0 conn(0x7f1a680a3690 0x7f1a68152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680bf530 con 0x7f1a680be7d0 2020-11-20T12:16:58.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f1a580014e0 con 0x7f1a680be7d0 2020-11-20T12:16:58.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 >> v1:172.21.15.116:6789/0 conn(0x7f1a680be7d0 legacy=0x7f1a680b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680c2450 0x7f1a50003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a50003c40 0x7f1a500044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a500049d0 0x7f1a50004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680beff0 con 0x7f1a500049d0 2020-11-20T12:16:58.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680bf290 con 0x7f1a50003c40 2020-11-20T12:16:58.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a50004200 con 0x7f1a680c2450 2020-11-20T12:16:58.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6da0f700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a500049d0 0x7f1a50004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6d20e700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680c2450 0x7f1a50003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6ca0d700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a50003c40 0x7f1a500044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6da0f700 1 -- 172.21.15.116:0/4089542595 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680c2450 msgr2=0x7f1a50003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:58.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6da0f700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680c2450 0x7f1a50003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6da0f700 1 -- 172.21.15.116:0/4089542595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a50003c40 msgr2=0x7f1a500044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:58.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6da0f700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a50003c40 0x7f1a500044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.140+0000 7f1a6da0f700 1 -- 172.21.15.116:0/4089542595 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a50003240 con 0x7f1a500049d0 2020-11-20T12:16:58.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.144+0000 7f1a6da0f700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a500049d0 0x7f1a50004e50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f1a6000bc00 tx=0x7f1a60006df0).ready entity=mon.0 client_cookie=aeba429b51ef93de server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:58.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.144+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000b310 con 0x7f1a500049d0 2020-11-20T12:16:58.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.144+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6000eae0 con 0x7f1a500049d0 2020-11-20T12:16:58.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.144+0000 7f1a57fff700 1 -- 172.21.15.116:0/4089542595 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6000ee60 con 0x7f1a500049d0 2020-11-20T12:16:58.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.144+0000 7f1a6f472700 1 -- 172.21.15.116:0/4089542595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a500049d0 msgr2=0x7f1a50004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.144+0000 7f1a6f472700 1 --2- 172.21.15.116:0/4089542595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a500049d0 0x7f1a50004e50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f1a6000bc00 tx=0x7f1a60006df0).stop 2020-11-20T12:16:58.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.144+0000 7f1a6da0f700 1 -- 172.21.15.116:0/4089542595 reap_dead start 2020-11-20T12:16:58.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.144+0000 7f1a6f472700 1 -- 172.21.15.116:0/4089542595 shutdown_connections 2020-11-20T12:16:58.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.144+0000 7f1a6f472700 1 -- 172.21.15.116:0/4089542595 >> 172.21.15.116:0/4089542595 conn(0x7f1a68060c80 msgr2=0x7f1a680c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:58.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.144+0000 7f1a6f472700 1 -- 172.21.15.116:0/4089542595 shutdown_connections 2020-11-20T12:16:58.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.144+0000 7f1a6f472700 1 -- 172.21.15.116:0/4089542595 wait complete. 2020-11-20T12:16:58.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.148+0000 7f1a6f472700 1 Processor -- start 2020-11-20T12:16:58.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.148+0000 7f1a6f472700 1 -- start start 2020-11-20T12:16:58.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.148+0000 7f1a6f472700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680a1540 0x7f1a680a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.148+0000 7f1a6f472700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a68160f10 0x7f1a680a9b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.148+0000 7f1a6f472700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680bcb60 0x7f1a680bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.148+0000 7f1a6f472700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680a9520 con 0x7f1a680bcb60 2020-11-20T12:16:58.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.148+0000 7f1a6f472700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680a7490 con 0x7f1a68160f10 2020-11-20T12:16:58.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.148+0000 7f1a6f472700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680a7730 con 0x7f1a680a1540 2020-11-20T12:16:58.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6da0f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680bcb60 0x7f1a680bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6da0f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680bcb60 0x7f1a680bcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44610/0 (socket says 172.21.15.116:44610) 2020-11-20T12:16:58.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6da0f700 1 -- 172.21.15.116:0/3418053831 learned_addr learned my addr 172.21.15.116:0/3418053831 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:58.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6d20e700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680a1540 0x7f1a680a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6ca0d700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a68160f10 0x7f1a680a9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6da0f700 1 -- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680a1540 msgr2=0x7f1a680a1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6da0f700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680a1540 0x7f1a680a1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6da0f700 1 -- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a68160f10 msgr2=0x7f1a680a9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6da0f700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a68160f10 0x7f1a680a9b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6da0f700 1 -- 172.21.15.116:0/3418053831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680a79d0 con 0x7f1a680bcb60 2020-11-20T12:16:58.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6da0f700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680bcb60 0x7f1a680bcfc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f1a60006fd0 tx=0x7f1a60019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:58.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a557fa700 1 -- 172.21.15.116:0/3418053831 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6001d620 con 0x7f1a680bcb60 2020-11-20T12:16:58.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a557fa700 1 -- 172.21.15.116:0/3418053831 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a6001d780 con 0x7f1a680bcb60 2020-11-20T12:16:58.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6f472700 1 -- 172.21.15.116:0/3418053831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a680a7c70 con 0x7f1a680bcb60 2020-11-20T12:16:58.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a557fa700 1 -- 172.21.15.116:0/3418053831 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a6001d910 con 0x7f1a680bcb60 2020-11-20T12:16:58.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6f472700 1 -- 172.21.15.116:0/3418053831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a68161390 con 0x7f1a680bcb60 2020-11-20T12:16:58.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.152+0000 7f1a6f472700 1 -- 172.21.15.116:0/3418053831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6805ca30 con 0x7f1a680bcb60 2020-11-20T12:16:58.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.156+0000 7f1a557fa700 1 -- 172.21.15.116:0/3418053831 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1a6002d070 con 0x7f1a680bcb60 2020-11-20T12:16:58.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.156+0000 7f1a557fa700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a400557c0 0x7f1a40057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.156+0000 7f1a557fa700 1 -- 172.21.15.116:0/3418053831 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1a60070840 con 0x7f1a680bcb60 2020-11-20T12:16:58.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.156+0000 7f1a6d20e700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a400557c0 0x7f1a40057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.156+0000 7f1a6d20e700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a400557c0 0x7f1a40057c80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f1a640091a0 tx=0x7f1a64008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:58.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.164+0000 7f1a557fa700 1 -- 172.21.15.116:0/3418053831 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1a60046570 con 0x7f1a680bcb60 2020-11-20T12:16:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.324+0000 7f1a6f472700 1 -- 172.21.15.116:0/3418053831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1a680be550 con 0x7f1a680bcb60 2020-11-20T12:16:58.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.324+0000 7f1a557fa700 1 -- 172.21.15.116:0/3418053831 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16887 (secure 0 0 0) 0x7f1a60049b90 con 0x7f1a680bcb60 2020-11-20T12:16:58.326 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:58.326 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":33,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:57.901357+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:16:55.781578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T12:16:56.900539+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 -- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a400557c0 msgr2=0x7f1a40057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a400557c0 0x7f1a40057c80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f1a640091a0 tx=0x7f1a64008040).stop 2020-11-20T12:16:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 -- 172.21.15.116:0/3418053831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680bcb60 msgr2=0x7f1a680bcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680bcb60 0x7f1a680bcfc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f1a60006fd0 tx=0x7f1a60019000).stop 2020-11-20T12:16:58.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 -- 172.21.15.116:0/3418053831 shutdown_connections 2020-11-20T12:16:58.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a400557c0 0x7f1a40057c80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680bcb60 0x7f1a680bcfc0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a680a1540 0x7f1a680a1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 --2- 172.21.15.116:0/3418053831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a68160f10 0x7f1a680a9b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 -- 172.21.15.116:0/3418053831 >> 172.21.15.116:0/3418053831 conn(0x7f1a68060c80 msgr2=0x7f1a680b9130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:58.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 -- 172.21.15.116:0/3418053831 shutdown_connections 2020-11-20T12:16:58.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.328+0000 7f1a6f472700 1 -- 172.21.15.116:0/3418053831 wait complete. 2020-11-20T12:16:58.354 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:16:58.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.532+0000 7fb0eb59e700 1 Processor -- start 2020-11-20T12:16:58.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.532+0000 7fb0eb59e700 1 -- start start 2020-11-20T12:16:58.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.532+0000 7fb0eb59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0ec13f1d0 0x7fb0ec0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.532+0000 7fb0eb59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb0ec13f5b0 0x7fb0ec066090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.532+0000 7fb0eb59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0ec0a5680 con 0x7fb0ec0a40c0 2020-11-20T12:16:58.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.532+0000 7fb0eb59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec0a58e0 con 0x7fb0ec13f5b0 2020-11-20T12:16:58.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.532+0000 7fb0eb59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0a5b80 con 0x7fb0ec13f1d0 2020-11-20T12:16:58.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.532+0000 7fb0e9d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb0ec0a40c0 0x7fb0ec0c3fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54050/0 (socket says 172.21.15.116:54050) 2020-11-20T12:16:58.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.532+0000 7fb0e9d9b700 1 -- 172.21.15.116:0/272262138 learned_addr learned my addr 172.21.15.116:0/272262138 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:58.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.532+0000 7fb0ea59c700 1 --2- 172.21.15.116:0/272262138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0ec13f1d0 0x7fb0ec0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0ead9d700 1 --2- 172.21.15.116:0/272262138 >> v2:172.21.15.173:3300/0 conn(0x7fb0ec13f5b0 0x7fb0ec066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0e959a700 1 -- 172.21.15.116:0/272262138 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1952522401 0 0) 0x7fb0ec0a5680 con 0x7fb0ec0a40c0 2020-11-20T12:16:58.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0e959a700 1 -- 172.21.15.116:0/272262138 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0d0003240 con 0x7fb0ec0a40c0 2020-11-20T12:16:58.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0ea59c700 1 -- 172.21.15.116:0/272262138 >> v2:172.21.15.173:3300/0 conn(0x7fb0ec13f5b0 msgr2=0x7fb0ec066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:16:58.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0ea59c700 1 --2- 172.21.15.116:0/272262138 >> v2:172.21.15.173:3300/0 conn(0x7fb0ec13f5b0 0x7fb0ec066090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0ea59c700 1 -- 172.21.15.116:0/272262138 >> v1:172.21.15.116:6789/0 conn(0x7fb0ec0a40c0 legacy=0x7fb0ec0c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0ea59c700 1 -- 172.21.15.116:0/272262138 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec0a5e20 con 0x7fb0ec13f1d0 2020-11-20T12:16:58.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0ea59c700 1 --2- 172.21.15.116:0/272262138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0ec13f1d0 0x7fb0ec0a3bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb0dc00b9a0 tx=0x7fb0dc006df0).ready entity=mon.2 client_cookie=25568f611664bea3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:58.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0e959a700 1 -- 172.21.15.116:0/272262138 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc009680 con 0x7fb0ec13f1d0 2020-11-20T12:16:58.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0e959a700 1 -- 172.21.15.116:0/272262138 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc0097e0 con 0x7fb0ec13f1d0 2020-11-20T12:16:58.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0e959a700 1 -- 172.21.15.116:0/272262138 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc009b60 con 0x7fb0ec13f1d0 2020-11-20T12:16:58.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0eb59e700 1 -- 172.21.15.116:0/272262138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0ec13f1d0 msgr2=0x7fb0ec0a3bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0eb59e700 1 --2- 172.21.15.116:0/272262138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0ec13f1d0 0x7fb0ec0a3bd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fb0dc00b9a0 tx=0x7fb0dc006df0).stop 2020-11-20T12:16:58.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0eb59e700 1 -- 172.21.15.116:0/272262138 shutdown_connections 2020-11-20T12:16:58.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0eb59e700 1 --2- 172.21.15.116:0/272262138 >> v2:172.21.15.173:3300/0 conn(0x7fb0ec13f5b0 0x7fb0ec066090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0eb59e700 1 --2- 172.21.15.116:0/272262138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0ec13f1d0 0x7fb0ec0a3bd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.536+0000 7fb0eb59e700 1 -- 172.21.15.116:0/272262138 >> 172.21.15.116:0/272262138 conn(0x7fb0ec062520 msgr2=0x7fb0ec062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:58.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.540+0000 7fb0eb59e700 1 -- 172.21.15.116:0/272262138 shutdown_connections 2020-11-20T12:16:58.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.540+0000 7fb0eb59e700 1 -- 172.21.15.116:0/272262138 wait complete. 2020-11-20T12:16:58.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.540+0000 7fb0eb59e700 1 Processor -- start 2020-11-20T12:16:58.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.540+0000 7fb0eb59e700 1 -- start start 2020-11-20T12:16:58.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.540+0000 7fb0eb59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0ec13f1d0 0x7fb0ec0a83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.540+0000 7fb0eb59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0ec13f5b0 0x7fb0ec155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.540+0000 7fb0eb59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0ec0a9ab0 0x7fb0ec09f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.540+0000 7fb0eb59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0a88b0 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.540+0000 7fb0eb59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec09ede0 con 0x7fb0ec13f1d0 2020-11-20T12:16:58.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.540+0000 7fb0eb59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec09f080 con 0x7fb0ec13f5b0 2020-11-20T12:16:58.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0ead9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0ec0a9ab0 0x7fb0ec09f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0ead9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0ec0a9ab0 0x7fb0ec09f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44624/0 (socket says 172.21.15.116:44624) 2020-11-20T12:16:58.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0ead9d700 1 -- 172.21.15.116:0/1107863341 learned_addr learned my addr 172.21.15.116:0/1107863341 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:58.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0e9d9b700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0ec13f5b0 0x7fb0ec155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0ea59c700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0ec13f1d0 0x7fb0ec0a83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0ead9d700 1 -- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0ec13f5b0 msgr2=0x7fb0ec155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0ead9d700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0ec13f5b0 0x7fb0ec155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0ead9d700 1 -- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0ec13f1d0 msgr2=0x7fb0ec0a83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0ead9d700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0ec13f1d0 0x7fb0ec0a83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0ead9d700 1 -- 172.21.15.116:0/1107863341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec0a7c80 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0ead9d700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0ec0a9ab0 0x7fb0ec09f440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fb0e400ed20 tx=0x7fb0e40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:58.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0da7fc700 1 -- 172.21.15.116:0/1107863341 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400cb00 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0da7fc700 1 -- 172.21.15.116:0/1107863341 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e400cc60 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0eb59e700 1 -- 172.21.15.116:0/1107863341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec0a7f20 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0da7fc700 1 -- 172.21.15.116:0/1107863341 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e40123f0 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0eb59e700 1 -- 172.21.15.116:0/1107863341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec155910 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.544+0000 7fb0eb59e700 1 -- 172.21.15.116:0/1107863341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ec044f60 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.548+0000 7fb0da7fc700 1 -- 172.21.15.116:0/1107863341 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb0e4022070 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.548+0000 7fb0da7fc700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0c00557c0 0x7fb0c0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.548+0000 7fb0da7fc700 1 -- 172.21.15.116:0/1107863341 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb0e406ee10 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.548+0000 7fb0ea59c700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0c00557c0 0x7fb0c0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.548+0000 7fb0ea59c700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0c00557c0 0x7fb0c0057c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fb0dc006e30 tx=0x7fb0dc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:58.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.556+0000 7fb0da7fc700 1 -- 172.21.15.116:0/1107863341 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb0e4072030 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.712+0000 7fb0eb59e700 1 -- 172.21.15.116:0/1107863341 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0ec0a80b0 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.716+0000 7fb0da7fc700 1 -- 172.21.15.116:0/1107863341 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16887 (secure 0 0 0) 0x7fb0e4016070 con 0x7fb0ec0a9ab0 2020-11-20T12:16:58.717 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:16:58.717 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":33,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:57.901357+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:16:55.781578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T12:16:56.900539+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:16:58.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.716+0000 7fb0eb59e700 1 -- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0c00557c0 msgr2=0x7fb0c0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.716+0000 7fb0eb59e700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0c00557c0 0x7fb0c0057c80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fb0dc006e30 tx=0x7fb0dc015000).stop 2020-11-20T12:16:58.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.716+0000 7fb0eb59e700 1 -- 172.21.15.116:0/1107863341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0ec0a9ab0 msgr2=0x7fb0ec09f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.716+0000 7fb0eb59e700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0ec0a9ab0 0x7fb0ec09f440 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fb0e400ed20 tx=0x7fb0e40091d0).stop 2020-11-20T12:16:58.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.720+0000 7fb0eb59e700 1 -- 172.21.15.116:0/1107863341 shutdown_connections 2020-11-20T12:16:58.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.720+0000 7fb0eb59e700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0c00557c0 0x7fb0c0057c80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.720+0000 7fb0eb59e700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0ec0a9ab0 0x7fb0ec09f440 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.720+0000 7fb0eb59e700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0ec13f1d0 0x7fb0ec0a83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.720+0000 7fb0eb59e700 1 --2- 172.21.15.116:0/1107863341 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb0ec13f5b0 0x7fb0ec155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.720+0000 7fb0eb59e700 1 -- 172.21.15.116:0/1107863341 >> 172.21.15.116:0/1107863341 conn(0x7fb0ec062520 msgr2=0x7fb0ec0c3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:58.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.720+0000 7fb0eb59e700 1 -- 172.21.15.116:0/1107863341 shutdown_connections 2020-11-20T12:16:58.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.720+0000 7fb0eb59e700 1 -- 172.21.15.116:0/1107863341 wait complete. 2020-11-20T12:16:58.747 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-20T12:16:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.920+0000 7fd6271b0700 1 Processor -- start 2020-11-20T12:16:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.920+0000 7fd6271b0700 1 -- start start 2020-11-20T12:16:58.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd6271b0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6200c0fe0 0x7fd6200c2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd6271b0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6200a2490 con 0x7fd6200c2af0 2020-11-20T12:16:58.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd6271b0700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd620060280 con 0x7fd6200c0c00 2020-11-20T12:16:58.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd6271b0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd620060520 con 0x7fd6200c0fe0 2020-11-20T12:16:58.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd62574d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd6200c2af0 0x7fd6200611a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54066/0 (socket says 172.21.15.116:54066) 2020-11-20T12:16:58.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd61ffff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6200c0fe0 0x7fd6200c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd62574d700 1 -- 172.21.15.116:0/3693567323 learned_addr learned my addr 172.21.15.116:0/3693567323 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:16:58.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd61f7fe700 1 -- 172.21.15.116:0/3693567323 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822789195 0 0) 0x7fd620060280 con 0x7fd6200c0c00 2020-11-20T12:16:58.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd61f7fe700 1 -- 172.21.15.116:0/3693567323 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd604003240 con 0x7fd6200c0c00 2020-11-20T12:16:58.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd61f7fe700 1 -- 172.21.15.116:0/3693567323 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1033097741 0 0) 0x7fd6200a2490 con 0x7fd6200c2af0 2020-11-20T12:16:58.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd61ffff700 1 -- 172.21.15.116:0/3693567323 >> v1:172.21.15.173:6789/0 conn(0x7fd6200c0c00 legacy=0x7fd6200a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd61ffff700 1 -- 172.21.15.116:0/3693567323 >> v1:172.21.15.116:6789/0 conn(0x7fd6200c2af0 legacy=0x7fd6200611a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd61ffff700 1 -- 172.21.15.116:0/3693567323 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6200607c0 con 0x7fd6200c0fe0 2020-11-20T12:16:58.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd61ffff700 1 --2- 172.21.15.116:0/3693567323 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6200c0fe0 0x7fd6200c2600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fd60c00b9d0 tx=0x7fd60c006df0).ready entity=mon.2 client_cookie=f97c8614569f1179 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:58.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd61f7fe700 1 -- 172.21.15.116:0/3693567323 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c0097b0 con 0x7fd6200c0fe0 2020-11-20T12:16:58.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd61f7fe700 1 -- 172.21.15.116:0/3693567323 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c009910 con 0x7fd6200c0fe0 2020-11-20T12:16:58.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd61f7fe700 1 -- 172.21.15.116:0/3693567323 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c009c90 con 0x7fd6200c0fe0 2020-11-20T12:16:58.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd6271b0700 1 -- 172.21.15.116:0/3693567323 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6200c0fe0 msgr2=0x7fd6200c2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd6271b0700 1 --2- 172.21.15.116:0/3693567323 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6200c0fe0 0x7fd6200c2600 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fd60c00b9d0 tx=0x7fd60c006df0).stop 2020-11-20T12:16:58.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd6271b0700 1 -- 172.21.15.116:0/3693567323 shutdown_connections 2020-11-20T12:16:58.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd6271b0700 1 --2- 172.21.15.116:0/3693567323 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6200c0fe0 0x7fd6200c2600 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd6271b0700 1 -- 172.21.15.116:0/3693567323 >> 172.21.15.116:0/3693567323 conn(0x7fd6200c14a0 msgr2=0x7fd6200c18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:58.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.924+0000 7fd6271b0700 1 -- 172.21.15.116:0/3693567323 shutdown_connections 2020-11-20T12:16:58.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.928+0000 7fd6271b0700 1 -- 172.21.15.116:0/3693567323 wait complete. 2020-11-20T12:16:58.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.928+0000 7fd6271b0700 1 Processor -- start 2020-11-20T12:16:58.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.928+0000 7fd6271b0700 1 -- start start 2020-11-20T12:16:58.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd6271b0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6200c0c00 0x7fd6200a79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd6271b0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6200c0fe0 0x7fd6200a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd6271b0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6200c2af0 0x7fd6200c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd6271b0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6201360e0 con 0x7fd6200c0fe0 2020-11-20T12:16:58.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd6271b0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200a81a0 con 0x7fd6200c0c00 2020-11-20T12:16:58.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd6271b0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6200a8440 con 0x7fd6200c2af0 2020-11-20T12:16:58.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6200c0fe0 0x7fd6200a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6200c0fe0 0x7fd6200a5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44638/0 (socket says 172.21.15.116:44638) 2020-11-20T12:16:58.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61ffff700 1 -- 172.21.15.116:0/1836579791 learned_addr learned my addr 172.21.15.116:0/1836579791 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:16:58.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd62574d700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6200c2af0 0x7fd6200c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd624f4c700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6200c0c00 0x7fd6200a79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61ffff700 1 -- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6200c2af0 msgr2=0x7fd6200c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61ffff700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6200c2af0 0x7fd6200c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61ffff700 1 -- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6200c0c00 msgr2=0x7fd6200a79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61ffff700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6200c0c00 0x7fd6200a79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61ffff700 1 -- 172.21.15.116:0/1836579791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6200a86e0 con 0x7fd6200c0fe0 2020-11-20T12:16:58.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61ffff700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6200c0fe0 0x7fd6200a5c70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fd60c002fc0 tx=0x7fd60c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:58.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61cff9700 1 -- 172.21.15.116:0/1836579791 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c011c80 con 0x7fd6200c0fe0 2020-11-20T12:16:58.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61cff9700 1 -- 172.21.15.116:0/1836579791 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd60c011de0 con 0x7fd6200c0fe0 2020-11-20T12:16:58.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd6271b0700 1 -- 172.21.15.116:0/1836579791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6200a8980 con 0x7fd6200c0fe0 2020-11-20T12:16:58.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd61cff9700 1 -- 172.21.15.116:0/1836579791 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd60c01e670 con 0x7fd6200c0fe0 2020-11-20T12:16:58.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd6271b0700 1 -- 172.21.15.116:0/1836579791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6200a7f20 con 0x7fd6200c0fe0 2020-11-20T12:16:58.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.932+0000 7fd6271b0700 1 -- 172.21.15.116:0/1836579791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62005ca30 con 0x7fd6200c0fe0 2020-11-20T12:16:58.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.936+0000 7fd61cff9700 1 -- 172.21.15.116:0/1836579791 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd60c025030 con 0x7fd6200c0fe0 2020-11-20T12:16:58.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.936+0000 7fd61cff9700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5f80557c0 0x7fd5f8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:16:58.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.936+0000 7fd61cff9700 1 -- 172.21.15.116:0/1836579791 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd60c06fe10 con 0x7fd6200c0fe0 2020-11-20T12:16:58.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.936+0000 7fd624f4c700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5f80557c0 0x7fd5f8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:16:58.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.940+0000 7fd624f4c700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5f80557c0 0x7fd5f8057c80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fd614003740 tx=0x7fd614007a20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:16:58.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:58.944+0000 7fd61cff9700 1 -- 172.21.15.116:0/1836579791 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd60c045cb0 con 0x7fd6200c0fe0 2020-11-20T12:16:59.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.152+0000 7fd6271b0700 1 -- 172.21.15.116:0/1836579791 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd620068120 con 0x7fd6200c0fe0 2020-11-20T12:16:59.392 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:16:59.400 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:16:59.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.924+0000 7fd61cff9700 1 -- 172.21.15.116:0/1836579791 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd60c023020 con 0x7fd6200c0fe0 2020-11-20T12:16:59.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 -- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5f80557c0 msgr2=0x7fd5f8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:59.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5f80557c0 0x7fd5f8057c80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fd614003740 tx=0x7fd614007a20).stop 2020-11-20T12:16:59.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 -- 172.21.15.116:0/1836579791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6200c0fe0 msgr2=0x7fd6200a5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:16:59.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6200c0fe0 0x7fd6200a5c70 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fd60c002fc0 tx=0x7fd60c015000).stop 2020-11-20T12:16:59.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 -- 172.21.15.116:0/1836579791 shutdown_connections 2020-11-20T12:16:59.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5f80557c0 0x7fd5f8057c80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:59.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6200c2af0 0x7fd6200c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:59.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6200c0c00 0x7fd6200a79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:59.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 --2- 172.21.15.116:0/1836579791 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6200c0fe0 0x7fd6200a5c70 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:16:59.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 -- 172.21.15.116:0/1836579791 >> 172.21.15.116:0/1836579791 conn(0x7fd6200c14a0 msgr2=0x7fd620066ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:16:59.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 -- 172.21.15.116:0/1836579791 shutdown_connections 2020-11-20T12:16:59.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:16:59.928+0000 7fd6271b0700 1 -- 172.21.15.116:0/1836579791 wait complete. 2020-11-20T12:16:59.954 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:00.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.124+0000 7f8672394700 1 Processor -- start 2020-11-20T12:17:00.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f8672394700 1 -- start start 2020-11-20T12:17:00.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f8672394700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f866c0aa790 0x7f866c152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f8672394700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c152a30 con 0x7f866c0612f0 2020-11-20T12:17:00.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f8672394700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c0aac10 con 0x7f866c13aee0 2020-11-20T12:17:00.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f8672394700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866c0adce0 con 0x7f866c0aa790 2020-11-20T12:17:00.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f8671b93700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f866c0612f0 0x7f866c0a7ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54080/0 (socket says 172.21.15.116:54080) 2020-11-20T12:17:00.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f8671b93700 1 -- 172.21.15.116:0/3164608095 learned_addr learned my addr 172.21.15.116:0/3164608095 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:00.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f8670b91700 1 --2- 172.21.15.116:0/3164608095 >> v2:172.21.15.116:3300/0 conn(0x7f866c0aa790 0x7f866c152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f865bfff700 1 -- 172.21.15.116:0/3164608095 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3487627528 0 0) 0x7f866c152a30 con 0x7f866c0612f0 2020-11-20T12:17:00.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f865bfff700 1 -- 172.21.15.116:0/3164608095 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8654003240 con 0x7f866c0612f0 2020-11-20T12:17:00.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f8670b91700 1 -- 172.21.15.116:0/3164608095 >> v1:172.21.15.173:6789/0 conn(0x7f866c13aee0 legacy=0x7f866c0ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f8670b91700 1 -- 172.21.15.116:0/3164608095 >> v1:172.21.15.116:6789/0 conn(0x7f866c0612f0 legacy=0x7f866c0a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.128+0000 7f8670b91700 1 -- 172.21.15.116:0/3164608095 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c0aaeb0 con 0x7f866c0aa790 2020-11-20T12:17:00.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.132+0000 7f8670b91700 1 --2- 172.21.15.116:0/3164608095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c0aa790 0x7f866c152540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f865c00b9a0 tx=0x7f865c006df0).ready entity=mon.0 client_cookie=f40388ebd81a24a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:00.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.132+0000 7f865bfff700 1 -- 172.21.15.116:0/3164608095 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0097b0 con 0x7f866c0aa790 2020-11-20T12:17:00.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.132+0000 7f865bfff700 1 -- 172.21.15.116:0/3164608095 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c009910 con 0x7f866c0aa790 2020-11-20T12:17:00.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.132+0000 7f865bfff700 1 -- 172.21.15.116:0/3164608095 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c009c90 con 0x7f866c0aa790 2020-11-20T12:17:00.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.132+0000 7f8672394700 1 -- 172.21.15.116:0/3164608095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c0aa790 msgr2=0x7f866c152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.132+0000 7f8672394700 1 --2- 172.21.15.116:0/3164608095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c0aa790 0x7f866c152540 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f865c00b9a0 tx=0x7f865c006df0).stop 2020-11-20T12:17:00.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.132+0000 7f8672394700 1 -- 172.21.15.116:0/3164608095 shutdown_connections 2020-11-20T12:17:00.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.132+0000 7f8672394700 1 --2- 172.21.15.116:0/3164608095 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c0aa790 0x7f866c152540 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.132+0000 7f8672394700 1 -- 172.21.15.116:0/3164608095 >> 172.21.15.116:0/3164608095 conn(0x7f866c0678a0 msgr2=0x7f866c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:00.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.132+0000 7f8672394700 1 -- 172.21.15.116:0/3164608095 shutdown_connections 2020-11-20T12:17:00.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.132+0000 7f8672394700 1 -- 172.21.15.116:0/3164608095 wait complete. 2020-11-20T12:17:00.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8672394700 1 Processor -- start 2020-11-20T12:17:00.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8672394700 1 -- start start 2020-11-20T12:17:00.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8672394700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f866c0612f0 0x7f866c09fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8672394700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f866c13aee0 0x7f866c150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8672394700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c153480 0x7f866c153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8672394700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c161f60 con 0x7f866c153480 2020-11-20T12:17:00.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8672394700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0a0060 con 0x7f866c13aee0 2020-11-20T12:17:00.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8672394700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c161cc0 con 0x7f866c0612f0 2020-11-20T12:17:00.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8671b93700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c153480 0x7f866c153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8671b93700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c153480 0x7f866c153900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44652/0 (socket says 172.21.15.116:44652) 2020-11-20T12:17:00.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8671b93700 1 -- 172.21.15.116:0/1476249199 learned_addr learned my addr 172.21.15.116:0/1476249199 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:00.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8670b91700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f866c13aee0 0x7f866c150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8671392700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f866c0612f0 0x7f866c09fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8671b93700 1 -- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f866c0612f0 msgr2=0x7f866c09fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8671b93700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f866c0612f0 0x7f866c09fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8671b93700 1 -- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f866c13aee0 msgr2=0x7f866c150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8671b93700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f866c13aee0 0x7f866c150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8671b93700 1 -- 172.21.15.116:0/1476249199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c09f190 con 0x7f866c153480 2020-11-20T12:17:00.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.136+0000 7f8671b93700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c153480 0x7f866c153900 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f8664001420 tx=0x7f8664007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:00.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.140+0000 7f86597fa700 1 -- 172.21.15.116:0/1476249199 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664005b80 con 0x7f866c153480 2020-11-20T12:17:00.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.140+0000 7f8672394700 1 -- 172.21.15.116:0/1476249199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c09f430 con 0x7f866c153480 2020-11-20T12:17:00.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.140+0000 7f8672394700 1 -- 172.21.15.116:0/1476249199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c151220 con 0x7f866c153480 2020-11-20T12:17:00.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.140+0000 7f86597fa700 1 -- 172.21.15.116:0/1476249199 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8664007110 con 0x7f866c153480 2020-11-20T12:17:00.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.140+0000 7f86597fa700 1 -- 172.21.15.116:0/1476249199 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8664007280 con 0x7f866c153480 2020-11-20T12:17:00.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.140+0000 7f8672394700 1 -- 172.21.15.116:0/1476249199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c044f60 con 0x7f866c153480 2020-11-20T12:17:00.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.140+0000 7f86597fa700 1 -- 172.21.15.116:0/1476249199 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8664010030 con 0x7f866c153480 2020-11-20T12:17:00.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.140+0000 7f86597fa700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86440557c0 0x7f8644057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.140+0000 7f86597fa700 1 -- 172.21.15.116:0/1476249199 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f866406e790 con 0x7f866c153480 2020-11-20T12:17:00.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.144+0000 7f8671392700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86440557c0 0x7f8644057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.144+0000 7f8671392700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86440557c0 0x7f8644057c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f8668003480 tx=0x7f8668009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:00.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.148+0000 7f86597fa700 1 -- 172.21.15.116:0/1476249199 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8664044630 con 0x7f866c153480 2020-11-20T12:17:00.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.352+0000 7f8672394700 1 -- 172.21.15.116:0/1476249199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f866c060500 con 0x7f866c153480 2020-11-20T12:17:00.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f86597fa700 1 -- 172.21.15.116:0/1476249199 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2851 (secure 0 0 0) 0x7f8664047c50 con 0x7f866c153480 2020-11-20T12:17:00.356 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:00.356 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3031892761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3031892761},{"type":"v1","addr":"172.21.15.173:6836","nonce":3031892761}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1794965470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1794965470},{"type":"v1","addr":"172.21.15.173:6837","nonce":1794965470}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":9},{"gid":4995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/4143559744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":4143559744},{"type":"v1","addr":"172.21.15.116:6833","nonce":4143559744}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:57.901741+0000","modified":"2020-11-20T12:16:59.918401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4998":{"gid":4998,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6835/3994085646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3994085646},{"type":"v1","addr":"172.21.15.116:6835","nonce":3994085646}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T12:17:00.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 -- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86440557c0 msgr2=0x7f8644057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86440557c0 0x7f8644057c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f8668003480 tx=0x7f8668009040).stop 2020-11-20T12:17:00.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 -- 172.21.15.116:0/1476249199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c153480 msgr2=0x7f866c153900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c153480 0x7f866c153900 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f8664001420 tx=0x7f8664007e20).stop 2020-11-20T12:17:00.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 -- 172.21.15.116:0/1476249199 shutdown_connections 2020-11-20T12:17:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86440557c0 0x7f8644057c80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f866c153480 0x7f866c153900 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f866c0612f0 0x7f866c09fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 --2- 172.21.15.116:0/1476249199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f866c13aee0 0x7f866c150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 -- 172.21.15.116:0/1476249199 >> 172.21.15.116:0/1476249199 conn(0x7f866c0678a0 msgr2=0x7f866c138b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 -- 172.21.15.116:0/1476249199 shutdown_connections 2020-11-20T12:17:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.356+0000 7f8672394700 1 -- 172.21.15.116:0/1476249199 wait complete. 2020-11-20T12:17:00.362 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 14 2020-11-20T12:17:00.380 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:00.399 INFO:tasks.ceph.mds.a.smithi116.stderr:2020-11-20T12:17:00.400+0000 7f758246c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:17:00.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.556+0000 7f931f59e700 1 Processor -- start 2020-11-20T12:17:00.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.556+0000 7f931f59e700 1 -- start start 2020-11-20T12:17:00.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.560+0000 7f931f59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f932013aee0 0x7f9320152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.560+0000 7f931f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200612f0 0x7f93200617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.560+0000 7f931f59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9320152a30 con 0x7f93200aa790 2020-11-20T12:17:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.560+0000 7f931f59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93200adce0 con 0x7f932013aee0 2020-11-20T12:17:00.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.560+0000 7f931f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93200aac10 con 0x7f93200612f0 2020-11-20T12:17:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.560+0000 7f931ed9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200612f0 0x7f93200617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.560+0000 7f931e59c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f932013aee0 0x7f9320152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.560+0000 7f931ed9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200612f0 0x7f93200617a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:53610/0 (socket says 172.21.15.116:53610) 2020-11-20T12:17:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.560+0000 7f931e59c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f932013aee0 0x7f9320152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:43708/0 (socket says 172.21.15.116:43708) 2020-11-20T12:17:00.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.560+0000 7f931ed9d700 1 -- 172.21.15.116:0/3555081695 learned_addr learned my addr 172.21.15.116:0/3555081695 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931ed9d700 1 -- 172.21.15.116:0/3555081695 >> v2:172.21.15.173:3300/0 conn(0x7f932013aee0 msgr2=0x7f9320152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931ed9d700 1 --2- 172.21.15.116:0/3555081695 >> v2:172.21.15.173:3300/0 conn(0x7f932013aee0 0x7f9320152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931ed9d700 1 -- 172.21.15.116:0/3555081695 >> v1:172.21.15.173:6789/0 conn(0x7f93200aa790 legacy=0x7f93200ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931ed9d700 1 -- 172.21.15.116:0/3555081695 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93200aaeb0 con 0x7f93200612f0 2020-11-20T12:17:00.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931e59c700 1 --2- 172.21.15.116:0/3555081695 >> v2:172.21.15.173:3300/0 conn(0x7f932013aee0 0x7f9320152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:00.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931ed9d700 1 --2- 172.21.15.116:0/3555081695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200612f0 0x7f93200617a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f9318009d50 tx=0x7f9318009290).ready entity=mon.2 client_cookie=b1766bdfe57191f9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:00.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931d59a700 1 -- 172.21.15.116:0/3555081695 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931800bbc0 con 0x7f93200612f0 2020-11-20T12:17:00.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931d59a700 1 -- 172.21.15.116:0/3555081695 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931800bd20 con 0x7f93200612f0 2020-11-20T12:17:00.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931d59a700 1 -- 172.21.15.116:0/3555081695 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9318011610 con 0x7f93200612f0 2020-11-20T12:17:00.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931f59e700 1 -- 172.21.15.116:0/3555081695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200612f0 msgr2=0x7f93200617a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931f59e700 1 --2- 172.21.15.116:0/3555081695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200612f0 0x7f93200617a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f9318009d50 tx=0x7f9318009290).stop 2020-11-20T12:17:00.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931f59e700 1 -- 172.21.15.116:0/3555081695 shutdown_connections 2020-11-20T12:17:00.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931f59e700 1 --2- 172.21.15.116:0/3555081695 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200612f0 0x7f93200617a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931f59e700 1 --2- 172.21.15.116:0/3555081695 >> v2:172.21.15.173:3300/0 conn(0x7f932013aee0 0x7f9320152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931f59e700 1 -- 172.21.15.116:0/3555081695 >> 172.21.15.116:0/3555081695 conn(0x7f93200678a0 msgr2=0x7f9320067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:00.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931f59e700 1 -- 172.21.15.116:0/3555081695 shutdown_connections 2020-11-20T12:17:00.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.564+0000 7f931f59e700 1 -- 172.21.15.116:0/3555081695 wait complete. 2020-11-20T12:17:00.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.568+0000 7f931f59e700 1 Processor -- start 2020-11-20T12:17:00.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.568+0000 7f931f59e700 1 -- start start 2020-11-20T12:17:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931f59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93200612f0 0x7f9320065d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200aa790 0x7f9320066610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931f59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f932013aee0 0x7f9320066b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931f59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9320165e80 con 0x7f93200612f0 2020-11-20T12:17:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931f59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9320165200 con 0x7f932013aee0 2020-11-20T12:17:00.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93201654a0 con 0x7f93200aa790 2020-11-20T12:17:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93200612f0 0x7f9320065d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93200612f0 0x7f9320065d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44662/0 (socket says 172.21.15.116:44662) 2020-11-20T12:17:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931e59c700 1 -- 172.21.15.116:0/3067064948 learned_addr learned my addr 172.21.15.116:0/3067064948 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931dd9b700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200aa790 0x7f9320066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931ed9d700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f932013aee0 0x7f9320066b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931e59c700 1 -- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200aa790 msgr2=0x7f9320066610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931e59c700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200aa790 0x7f9320066610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931e59c700 1 -- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f932013aee0 msgr2=0x7f9320066b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931e59c700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f932013aee0 0x7f9320066b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931e59c700 1 -- 172.21.15.116:0/3067064948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9320165740 con 0x7f93200612f0 2020-11-20T12:17:00.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931ed9d700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f932013aee0 0x7f9320066b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:00.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931e59c700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93200612f0 0x7f9320065d60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f931000ed20 tx=0x7f9310006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f930e7fc700 1 -- 172.21.15.116:0/3067064948 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f931000cb50 con 0x7f93200612f0 2020-11-20T12:17:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f930e7fc700 1 -- 172.21.15.116:0/3067064948 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f931000ccb0 con 0x7f93200612f0 2020-11-20T12:17:00.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f930e7fc700 1 -- 172.21.15.116:0/3067064948 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9310012600 con 0x7f93200612f0 2020-11-20T12:17:00.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931f59e700 1 -- 172.21.15.116:0/3067064948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93201659e0 con 0x7f93200612f0 2020-11-20T12:17:00.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.572+0000 7f931f59e700 1 -- 172.21.15.116:0/3067064948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9320066250 con 0x7f93200612f0 2020-11-20T12:17:00.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.576+0000 7f931f59e700 1 -- 172.21.15.116:0/3067064948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9320044f60 con 0x7f93200612f0 2020-11-20T12:17:00.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.576+0000 7f930e7fc700 1 -- 172.21.15.116:0/3067064948 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9310012970 con 0x7f93200612f0 2020-11-20T12:17:00.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.576+0000 7f930e7fc700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92f40557c0 0x7f92f4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.576+0000 7f930e7fc700 1 -- 172.21.15.116:0/3067064948 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f931006ed40 con 0x7f93200612f0 2020-11-20T12:17:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.576+0000 7f931dd9b700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92f40557c0 0x7f92f4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.576+0000 7f931dd9b700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92f40557c0 0x7f92f4057c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9320157ca0 tx=0x7f9314009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:00.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.580+0000 7f930e7fc700 1 -- 172.21.15.116:0/3067064948 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9310044be0 con 0x7f93200612f0 2020-11-20T12:17:00.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.740+0000 7f931f59e700 1 -- 172.21.15.116:0/3067064948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9320060500 con 0x7f93200612f0 2020-11-20T12:17:00.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.744+0000 7f930e7fc700 1 -- 172.21.15.116:0/3067064948 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f9310048200 con 0x7f93200612f0 2020-11-20T12:17:00.745 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:00.745 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:58.907407+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:16:55.781578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T12:16:56.900539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:00.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.744+0000 7f931f59e700 1 -- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92f40557c0 msgr2=0x7f92f4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.744+0000 7f931f59e700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92f40557c0 0x7f92f4057c80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9320157ca0 tx=0x7f9314009040).stop 2020-11-20T12:17:00.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.744+0000 7f931f59e700 1 -- 172.21.15.116:0/3067064948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93200612f0 msgr2=0x7f9320065d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.744+0000 7f931f59e700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93200612f0 0x7f9320065d60 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f931000ed20 tx=0x7f9310006df0).stop 2020-11-20T12:17:00.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.748+0000 7f931f59e700 1 -- 172.21.15.116:0/3067064948 shutdown_connections 2020-11-20T12:17:00.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.748+0000 7f931f59e700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92f40557c0 0x7f92f4057c80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.748+0000 7f931f59e700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f932013aee0 0x7f9320066b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.748+0000 7f931f59e700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93200612f0 0x7f9320065d60 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.748+0000 7f931f59e700 1 --2- 172.21.15.116:0/3067064948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93200aa790 0x7f9320066610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.748+0000 7f931f59e700 1 -- 172.21.15.116:0/3067064948 >> 172.21.15.116:0/3067064948 conn(0x7f93200678a0 msgr2=0x7f9320069690 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.748+0000 7f931f59e700 1 -- 172.21.15.116:0/3067064948 shutdown_connections 2020-11-20T12:17:00.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.748+0000 7f931f59e700 1 -- 172.21.15.116:0/3067064948 wait complete. 2020-11-20T12:17:00.769 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-20T12:17:00.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.940+0000 7faa4d9c9700 1 Processor -- start 2020-11-20T12:17:00.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.940+0000 7faa4d9c9700 1 -- start start 2020-11-20T12:17:00.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.940+0000 7faa4d9c9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa4814b5a0 0x7faa48151640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.940+0000 7faa4d9c9700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa48151f20 con 0x7faa480c2a70 2020-11-20T12:17:00.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.940+0000 7faa4d9c9700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa481521c0 con 0x7faa480a3700 2020-11-20T12:17:00.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.940+0000 7faa4d9c9700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa48152460 con 0x7faa4814b5a0 2020-11-20T12:17:00.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.940+0000 7faa46ffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa4814b5a0 0x7faa48151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.940+0000 7faa467fc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7faa480c2a70 0x7faa4814e860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54106/0 (socket says 172.21.15.116:54106) 2020-11-20T12:17:00.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.940+0000 7faa467fc700 1 -- 172.21.15.116:0/349243490 learned_addr learned my addr 172.21.15.116:0/349243490 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:00.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa46ffd700 1 -- 172.21.15.116:0/349243490 >> v1:172.21.15.173:6789/0 conn(0x7faa480a3700 legacy=0x7faa48060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa46ffd700 1 -- 172.21.15.116:0/349243490 >> v1:172.21.15.116:6789/0 conn(0x7faa480c2a70 legacy=0x7faa4814e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa46ffd700 1 -- 172.21.15.116:0/349243490 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa4814e3c0 con 0x7faa4814b5a0 2020-11-20T12:17:00.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa46ffd700 1 --2- 172.21.15.116:0/349243490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4814b5a0 0x7faa48151640 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7faa3800b9a0 tx=0x7faa38006df0).ready entity=mon.0 client_cookie=127f2cdc6cb9abd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:00.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa45ffb700 1 -- 172.21.15.116:0/349243490 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa380097b0 con 0x7faa4814b5a0 2020-11-20T12:17:00.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa45ffb700 1 -- 172.21.15.116:0/349243490 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa38009910 con 0x7faa4814b5a0 2020-11-20T12:17:00.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa45ffb700 1 -- 172.21.15.116:0/349243490 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38009c90 con 0x7faa4814b5a0 2020-11-20T12:17:00.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa4d9c9700 1 -- 172.21.15.116:0/349243490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4814b5a0 msgr2=0x7faa48151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa4d9c9700 1 --2- 172.21.15.116:0/349243490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4814b5a0 0x7faa48151640 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7faa3800b9a0 tx=0x7faa38006df0).stop 2020-11-20T12:17:00.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa4d9c9700 1 -- 172.21.15.116:0/349243490 shutdown_connections 2020-11-20T12:17:00.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa4d9c9700 1 --2- 172.21.15.116:0/349243490 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa4814b5a0 0x7faa48151640 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa4d9c9700 1 -- 172.21.15.116:0/349243490 >> 172.21.15.116:0/349243490 conn(0x7faa480c1360 msgr2=0x7faa480c1770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:00.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa4d9c9700 1 -- 172.21.15.116:0/349243490 shutdown_connections 2020-11-20T12:17:00.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.944+0000 7faa4d9c9700 1 -- 172.21.15.116:0/349243490 wait complete. 2020-11-20T12:17:00.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa4d9c9700 1 Processor -- start 2020-11-20T12:17:00.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa4d9c9700 1 -- start start 2020-11-20T12:17:00.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa4d9c9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa480a3700 0x7faa4813d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa4d9c9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa480c2a70 0x7faa480a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa4d9c9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa4814b5a0 0x7faa48063590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa4d9c9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa480a4b80 con 0x7faa480a3700 2020-11-20T12:17:00.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa4d9c9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa480a4db0 con 0x7faa480c2a70 2020-11-20T12:17:00.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa4d9c9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa480a5050 con 0x7faa4814b5a0 2020-11-20T12:17:00.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa46ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa480a3700 0x7faa4813d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa46ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa480a3700 0x7faa4813d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44680/0 (socket says 172.21.15.116:44680) 2020-11-20T12:17:00.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa46ffd700 1 -- 172.21.15.116:0/445588587 learned_addr learned my addr 172.21.15.116:0/445588587 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:00.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa477fe700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa4814b5a0 0x7faa48063590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.948+0000 7faa467fc700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa480c2a70 0x7faa480a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa46ffd700 1 -- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa4814b5a0 msgr2=0x7faa48063590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa46ffd700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa4814b5a0 0x7faa48063590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa46ffd700 1 -- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa480c2a70 msgr2=0x7faa480a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:00.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa46ffd700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa480c2a70 0x7faa480a9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:00.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa46ffd700 1 -- 172.21.15.116:0/445588587 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa480a52f0 con 0x7faa480a3700 2020-11-20T12:17:00.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa467fc700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa480c2a70 0x7faa480a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:00.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa46ffd700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa480a3700 0x7faa4813d390 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7faa48066290 tx=0x7faa38015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:00.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa2effd700 1 -- 172.21.15.116:0/445588587 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3801e460 con 0x7faa480a3700 2020-11-20T12:17:00.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa2effd700 1 -- 172.21.15.116:0/445588587 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3801e5c0 con 0x7faa480a3700 2020-11-20T12:17:00.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa2effd700 1 -- 172.21.15.116:0/445588587 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3801e940 con 0x7faa480a3700 2020-11-20T12:17:00.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa4d9c9700 1 -- 172.21.15.116:0/445588587 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa480a5590 con 0x7faa480a3700 2020-11-20T12:17:00.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa4d9c9700 1 -- 172.21.15.116:0/445588587 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4813d880 con 0x7faa480a3700 2020-11-20T12:17:00.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa4d9c9700 1 -- 172.21.15.116:0/445588587 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa480a5bd0 con 0x7faa480a3700 2020-11-20T12:17:00.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.952+0000 7faa2effd700 1 -- 172.21.15.116:0/445588587 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7faa38025030 con 0x7faa480a3700 2020-11-20T12:17:00.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.956+0000 7faa2effd700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa240557c0 0x7faa24057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:00.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.956+0000 7faa2effd700 1 -- 172.21.15.116:0/445588587 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faa3806ff60 con 0x7faa480a3700 2020-11-20T12:17:00.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.956+0000 7faa467fc700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa240557c0 0x7faa24057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:00.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.956+0000 7faa467fc700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa240557c0 0x7faa24057c80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7faa480aab70 tx=0x7faa3c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:00.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:00.960+0000 7faa2effd700 1 -- 172.21.15.116:0/445588587 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7faa38045eb0 con 0x7faa480a3700 2020-11-20T12:17:01.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.192+0000 7faa4d9c9700 1 -- 172.21.15.116:0/445588587 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7faa480a5a30 con 0x7faa480a3700 2020-11-20T12:17:01.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.200+0000 7faa2effd700 1 -- 172.21.15.116:0/445588587 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 184+0+0 (secure 0 0 0) 0x7faa480a5a30 con 0x7faa480a3700 2020-11-20T12:17:01.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 -- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa240557c0 msgr2=0x7faa24057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:01.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa240557c0 0x7faa24057c80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7faa480aab70 tx=0x7faa3c009040).stop 2020-11-20T12:17:01.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 -- 172.21.15.116:0/445588587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa480a3700 msgr2=0x7faa4813d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:01.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa480a3700 0x7faa4813d390 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7faa48066290 tx=0x7faa38015000).stop 2020-11-20T12:17:01.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 -- 172.21.15.116:0/445588587 shutdown_connections 2020-11-20T12:17:01.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa240557c0 0x7faa24057c80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa4814b5a0 0x7faa48063590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa480a3700 0x7faa4813d390 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 --2- 172.21.15.116:0/445588587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa480c2a70 0x7faa480a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 -- 172.21.15.116:0/445588587 >> 172.21.15.116:0/445588587 conn(0x7faa480c1360 msgr2=0x7faa480c0010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:01.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 -- 172.21.15.116:0/445588587 shutdown_connections 2020-11-20T12:17:01.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.204+0000 7faa4d9c9700 1 -- 172.21.15.116:0/445588587 wait complete. 2020-11-20T12:17:01.207 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2020-11-20T12:17:01.228 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:01.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.408+0000 7fcbaf20c700 1 Processor -- start 2020-11-20T12:17:01.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.408+0000 7fcbaf20c700 1 -- start start 2020-11-20T12:17:01.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.408+0000 7fcbaf20c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fcba80aa790 0x7fcba8152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:01.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.408+0000 7fcbaf20c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcba80612f0 0x7fcba8061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:01.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.408+0000 7fcbaf20c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba8152a30 con 0x7fcba813aee0 2020-11-20T12:17:01.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.408+0000 7fcbaf20c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba80adce0 con 0x7fcba80612f0 2020-11-20T12:17:01.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.408+0000 7fcbaf20c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcba8061c50 con 0x7fcba80aa790 2020-11-20T12:17:01.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.408+0000 7fcbaea0b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcba80612f0 0x7fcba8061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:01.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.408+0000 7fcbaea0b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcba80612f0 0x7fcba8061760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44688/0 (socket says 172.21.15.116:44688) 2020-11-20T12:17:01.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.408+0000 7fcbaea0b700 1 -- 172.21.15.116:0/222423665 learned_addr learned my addr 172.21.15.116:0/222423665 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:01.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbada09700 1 --2- 172.21.15.116:0/222423665 >> v2:172.21.15.173:3300/0 conn(0x7fcba80aa790 0x7fcba8152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:01.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaea0b700 1 -- 172.21.15.116:0/222423665 >> v2:172.21.15.173:3300/0 conn(0x7fcba80aa790 msgr2=0x7fcba8152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:01.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaea0b700 1 --2- 172.21.15.116:0/222423665 >> v2:172.21.15.173:3300/0 conn(0x7fcba80aa790 0x7fcba8152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaea0b700 1 -- 172.21.15.116:0/222423665 >> v1:172.21.15.173:6789/0 conn(0x7fcba813aee0 legacy=0x7fcba80ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:01.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaea0b700 1 -- 172.21.15.116:0/222423665 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba80aac10 con 0x7fcba80612f0 2020-11-20T12:17:01.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaea0b700 1 --2- 172.21.15.116:0/222423665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcba80612f0 0x7fcba8061760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fcba80ae630 tx=0x7fcba0009290).ready entity=mon.0 client_cookie=b92a2e071bc1efcb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:01.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbad208700 1 -- 172.21.15.116:0/222423665 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba000bbc0 con 0x7fcba80612f0 2020-11-20T12:17:01.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbad208700 1 -- 172.21.15.116:0/222423665 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba000bd20 con 0x7fcba80612f0 2020-11-20T12:17:01.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbad208700 1 -- 172.21.15.116:0/222423665 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0011610 con 0x7fcba80612f0 2020-11-20T12:17:01.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaf20c700 1 -- 172.21.15.116:0/222423665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcba80612f0 msgr2=0x7fcba8061760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:01.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaf20c700 1 --2- 172.21.15.116:0/222423665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcba80612f0 0x7fcba8061760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fcba80ae630 tx=0x7fcba0009290).stop 2020-11-20T12:17:01.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaf20c700 1 -- 172.21.15.116:0/222423665 shutdown_connections 2020-11-20T12:17:01.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaf20c700 1 --2- 172.21.15.116:0/222423665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcba80612f0 0x7fcba8061760 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaf20c700 1 --2- 172.21.15.116:0/222423665 >> v2:172.21.15.173:3300/0 conn(0x7fcba80aa790 0x7fcba8152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaf20c700 1 -- 172.21.15.116:0/222423665 >> 172.21.15.116:0/222423665 conn(0x7fcba80678a0 msgr2=0x7fcba8067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:01.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaf20c700 1 -- 172.21.15.116:0/222423665 shutdown_connections 2020-11-20T12:17:01.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.412+0000 7fcbaf20c700 1 -- 172.21.15.116:0/222423665 wait complete. 2020-11-20T12:17:01.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.416+0000 7fcbaf20c700 1 Processor -- start 2020-11-20T12:17:01.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.416+0000 7fcbaf20c700 1 -- start start 2020-11-20T12:17:01.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.416+0000 7fcbaf20c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcba80612f0 0x7fcba81574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:01.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.416+0000 7fcbaf20c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcba80aa790 0x7fcba8150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:01.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.416+0000 7fcbaf20c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba813aee0 0x7fcba8150d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:01.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.416+0000 7fcbaf20c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba813a060 con 0x7fcba80aa790 2020-11-20T12:17:01.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.416+0000 7fcbaf20c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba8151250 con 0x7fcba813aee0 2020-11-20T12:17:01.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.416+0000 7fcbaf20c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba80a5ce0 con 0x7fcba80612f0 2020-11-20T12:17:01.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbada09700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcba80aa790 0x7fcba8150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:01.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbada09700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcba80aa790 0x7fcba8150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44692/0 (socket says 172.21.15.116:44692) 2020-11-20T12:17:01.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbada09700 1 -- 172.21.15.116:0/3221379917 learned_addr learned my addr 172.21.15.116:0/3221379917 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:01.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbaea0b700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba813aee0 0x7fcba8150d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:01.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbae20a700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcba80612f0 0x7fcba81574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbada09700 1 -- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcba80612f0 msgr2=0x7fcba81574c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbada09700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcba80612f0 0x7fcba81574c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbada09700 1 -- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba813aee0 msgr2=0x7fcba8150d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbada09700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba813aee0 0x7fcba8150d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbada09700 1 -- 172.21.15.116:0/3221379917 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba80a5f80 con 0x7fcba80aa790 2020-11-20T12:17:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbaea0b700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba813aee0 0x7fcba8150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbae20a700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcba80612f0 0x7fcba81574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbada09700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcba80aa790 0x7fcba8150870 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fcb9800ebd0 tx=0x7fcb98006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcb967fc700 1 -- 172.21.15.116:0/3221379917 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9800ca40 con 0x7fcba80aa790 2020-11-20T12:17:01.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcb967fc700 1 -- 172.21.15.116:0/3221379917 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9800cba0 con 0x7fcba80aa790 2020-11-20T12:17:01.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcb967fc700 1 -- 172.21.15.116:0/3221379917 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb980123f0 con 0x7fcba80aa790 2020-11-20T12:17:01.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbaf20c700 1 -- 172.21.15.116:0/3221379917 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba80a6220 con 0x7fcba80aa790 2020-11-20T12:17:01.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbaf20c700 1 -- 172.21.15.116:0/3221379917 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba81579b0 con 0x7fcba80aa790 2020-11-20T12:17:01.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.420+0000 7fcbaf20c700 1 -- 172.21.15.116:0/3221379917 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba8044f60 con 0x7fcba80aa790 2020-11-20T12:17:01.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.424+0000 7fcb967fc700 1 -- 172.21.15.116:0/3221379917 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fcb98012760 con 0x7fcba80aa790 2020-11-20T12:17:01.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.424+0000 7fcb967fc700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcb800557c0 0x7fcb80057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:01.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.424+0000 7fcb967fc700 1 -- 172.21.15.116:0/3221379917 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcb9806ec40 con 0x7fcba80aa790 2020-11-20T12:17:01.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.424+0000 7fcbae20a700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcb800557c0 0x7fcb80057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:01.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.424+0000 7fcbae20a700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcb800557c0 0x7fcb80057c80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fcba4001430 tx=0x7fcba4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:01.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.428+0000 7fcb967fc700 1 -- 172.21.15.116:0/3221379917 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcb98044ae0 con 0x7fcba80aa790 2020-11-20T12:17:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.644+0000 7fcbaf20c700 1 -- 172.21.15.116:0/3221379917 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcba8060500 con 0x7fcba80aa790 2020-11-20T12:17:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.644+0000 7fcb967fc700 1 -- 172.21.15.116:0/3221379917 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2851 (secure 0 0 0) 0x7fcb98048100 con 0x7fcba80aa790 2020-11-20T12:17:01.646 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:01.646 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3031892761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3031892761},{"type":"v1","addr":"172.21.15.173:6836","nonce":3031892761}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1794965470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1794965470},{"type":"v1","addr":"172.21.15.173:6837","nonce":1794965470}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":9},{"gid":4995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/4143559744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":4143559744},{"type":"v1","addr":"172.21.15.116:6833","nonce":4143559744}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:57.901741+0000","modified":"2020-11-20T12:16:59.918401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4998":{"gid":4998,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6835/3994085646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3994085646},{"type":"v1","addr":"172.21.15.116:6835","nonce":3994085646}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T12:17:01.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 -- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcb800557c0 msgr2=0x7fcb80057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcb800557c0 0x7fcb80057c80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fcba4001430 tx=0x7fcba4009040).stop 2020-11-20T12:17:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 -- 172.21.15.116:0/3221379917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcba80aa790 msgr2=0x7fcba8150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcba80aa790 0x7fcba8150870 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7fcb9800ebd0 tx=0x7fcb98006df0).stop 2020-11-20T12:17:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 -- 172.21.15.116:0/3221379917 shutdown_connections 2020-11-20T12:17:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcb800557c0 0x7fcb80057c80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcba813aee0 0x7fcba8150d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcba80612f0 0x7fcba81574c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 --2- 172.21.15.116:0/3221379917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcba80aa790 0x7fcba8150870 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 -- 172.21.15.116:0/3221379917 >> 172.21.15.116:0/3221379917 conn(0x7fcba80678a0 msgr2=0x7fcba81545b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 -- 172.21.15.116:0/3221379917 shutdown_connections 2020-11-20T12:17:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:01.648+0000 7fcbaf20c700 1 -- 172.21.15.116:0/3221379917 wait complete. 2020-11-20T12:17:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 14 2020-11-20T12:17:02.075 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:57.901741+0000', 'modified': '2020-11-20T12:16:59.918401+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4998}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4998': {'gid': 4998, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.116:6835/3994085646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3994085646}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3994085646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:17:02.075 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:17:02.076 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:17:02.268 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4998,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.3450699119999996,"mdsmap_epoch":14,"osdmap_epoch":34,"osdmap_epoch_barrier":34,"uptime":12.686153063000001} 2020-11-20T12:17:02.268 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4998, "mdsmap_epoch": 14, "osdmap_epoch": 34, "osdmap_epoch_barrier": 34, "rank_uptime": 4.345069912, "state": "up:active", "uptime": 12.686153063, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:17:02.269 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:17:02.270 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:17:02.271 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:17:02.271 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:17:02.272 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:17:02.273 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:17:02.273 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:17:02.274 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:17:02.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:02.275 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:17:02.292 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:17:02.293 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:17:02.294 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:17:02.294 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:02.295 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:17:02.295 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:02.295 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:02.296 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:02.296 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:02.297 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:02.297 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:02.297 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:02.298 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:17:02.298 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:02.299 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:17:02.299 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:02.299 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:17:02.300 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:02.300 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:17:02.300 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:02.301 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:17:02.301 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:02.302 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:17:02.302 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:02.302 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:17:02.303 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:02.303 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:17:02.304 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:02.304 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:02.304 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:17:02.305 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:17:02.305 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:02.306 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:02.306 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:17:02.306 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:17:02.307 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:02.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:02.309 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:17:02.385 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:17:02.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:02.441 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:17:02.467 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:17:02.468 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:17:02.468 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:17:02.469 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:17:02.470 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:17:02.470 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:17:02.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:02.472 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:17:02.472 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:17:02.473 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:17:02.473 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:17:02.474 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:17:02.475 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:17:02.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:02.607 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:17:02.622 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:17:02.622 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:17:02.624 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:17:02.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:02.626 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2020-11-20T12:17:02.699 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.880+0000 7f2a709e1700 1 Processor -- start 2020-11-20T12:17:02.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.880+0000 7f2a709e1700 1 -- start start 2020-11-20T12:17:02.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.880+0000 7f2a709e1700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2a6c0a4bc0 0x7f2a6c0c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:02.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.880+0000 7f2a709e1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6c0a3bd0 con 0x7f2a6c0668a0 2020-11-20T12:17:02.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.880+0000 7f2a709e1700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a6c0a3e70 con 0x7f2a6c0c44a0 2020-11-20T12:17:02.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.880+0000 7f2a709e1700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a6c0c4910 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.880+0000 7f2a6b7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2a6c0668a0 0x7f2a6c0a6290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54132/0 (socket says 172.21.15.116:54132) 2020-11-20T12:17:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.880+0000 7f2a6b7fe700 1 -- 172.21.15.116:0/4134901328 learned_addr learned my addr 172.21.15.116:0/4134901328 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:02.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6affd700 1 --2- 172.21.15.116:0/4134901328 >> v2:172.21.15.173:3300/0 conn(0x7f2a6c0a4bc0 0x7f2a6c0c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6a7fc700 1 -- 172.21.15.116:0/4134901328 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680424011 0 0) 0x7f2a6c0a3bd0 con 0x7f2a6c0668a0 2020-11-20T12:17:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6a7fc700 1 -- 172.21.15.116:0/4134901328 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a58003240 con 0x7f2a6c0668a0 2020-11-20T12:17:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6a7fc700 1 -- 172.21.15.116:0/4134901328 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1634251543 0 0) 0x7f2a6c0a3e70 con 0x7f2a6c0c44a0 2020-11-20T12:17:02.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6a7fc700 1 -- 172.21.15.116:0/4134901328 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a6c0a3bd0 con 0x7f2a6c0c44a0 2020-11-20T12:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6a7fc700 1 -- 172.21.15.116:0/4134901328 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2630186411 0 0) 0x7f2a58003240 con 0x7f2a6c0668a0 2020-11-20T12:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6affd700 1 -- 172.21.15.116:0/4134901328 >> v1:172.21.15.173:6789/0 conn(0x7f2a6c0c44a0 legacy=0x7f2a6c151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:02.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6affd700 1 -- 172.21.15.116:0/4134901328 >> v1:172.21.15.116:6789/0 conn(0x7f2a6c0668a0 legacy=0x7f2a6c0a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6affd700 1 -- 172.21.15.116:0/4134901328 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c0a4110 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6affd700 1 --2- 172.21.15.116:0/4134901328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c0a4bc0 0x7f2a6c0c3fb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f2a6000b9a0 tx=0x7f2a60006df0).ready entity=mon.1 client_cookie=19bc38274124cdf8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:02.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6a7fc700 1 -- 172.21.15.116:0/4134901328 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c0014e0 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6a7fc700 1 -- 172.21.15.116:0/4134901328 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a60011400 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a6a7fc700 1 -- 172.21.15.116:0/4134901328 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60011780 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a709e1700 1 -- 172.21.15.116:0/4134901328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c0a4bc0 msgr2=0x7f2a6c0c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a709e1700 1 --2- 172.21.15.116:0/4134901328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c0a4bc0 0x7f2a6c0c3fb0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f2a6000b9a0 tx=0x7f2a60006df0).stop 2020-11-20T12:17:02.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a709e1700 1 -- 172.21.15.116:0/4134901328 shutdown_connections 2020-11-20T12:17:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a709e1700 1 --2- 172.21.15.116:0/4134901328 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c0a4bc0 0x7f2a6c0c3fb0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a709e1700 1 -- 172.21.15.116:0/4134901328 >> 172.21.15.116:0/4134901328 conn(0x7f2a6c062520 msgr2=0x7f2a6c062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:02.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a709e1700 1 -- 172.21.15.116:0/4134901328 shutdown_connections 2020-11-20T12:17:02.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.884+0000 7f2a709e1700 1 -- 172.21.15.116:0/4134901328 wait complete. 2020-11-20T12:17:02.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a709e1700 1 Processor -- start 2020-11-20T12:17:02.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a709e1700 1 -- start start 2020-11-20T12:17:02.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a709e1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a6c0668a0 0x7f2a6c14e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:02.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a709e1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a6c0a4bc0 0x7f2a6c164900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:02.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a709e1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c0c44a0 0x7f2a6c0a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a709e1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c1533b0 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a709e1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c153650 con 0x7f2a6c0c44a0 2020-11-20T12:17:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a709e1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a6c1538f0 con 0x7f2a6c0668a0 2020-11-20T12:17:02.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a6c0a4bc0 0x7f2a6c164900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a6c0a4bc0 0x7f2a6c164900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44706/0 (socket says 172.21.15.116:44706) 2020-11-20T12:17:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6affd700 1 -- 172.21.15.116:0/1834017466 learned_addr learned my addr 172.21.15.116:0/1834017466 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6b7fe700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a6c0668a0 0x7f2a6c14e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6affd700 1 -- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a6c0668a0 msgr2=0x7f2a6c14e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:02.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6affd700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a6c0668a0 0x7f2a6c14e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:02.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6affd700 1 -- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c0c44a0 msgr2=0x7f2a6c0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:02.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6bfff700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c0c44a0 0x7f2a6c0a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:02.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6affd700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c0c44a0 0x7f2a6c0a8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:02.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6affd700 1 -- 172.21.15.116:0/1834017466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a6c153b90 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6b7fe700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a6c0668a0 0x7f2a6c14e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6bfff700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c0c44a0 0x7f2a6c0a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.888+0000 7f2a6affd700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a6c0a4bc0 0x7f2a6c164900 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f2a6000b150 tx=0x7f2a60013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.892+0000 7f2a4b7fe700 1 -- 172.21.15.116:0/1834017466 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60010c50 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.892+0000 7f2a709e1700 1 -- 172.21.15.116:0/1834017466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a6c153e30 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.892+0000 7f2a4b7fe700 1 -- 172.21.15.116:0/1834017466 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a60010db0 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.892+0000 7f2a4b7fe700 1 -- 172.21.15.116:0/1834017466 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6001d5d0 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.892+0000 7f2a709e1700 1 -- 172.21.15.116:0/1834017466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a6c14e910 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.892+0000 7f2a709e1700 1 -- 172.21.15.116:0/1834017466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a6c044f60 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.892+0000 7f2a4b7fe700 1 -- 172.21.15.116:0/1834017466 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2a6001a040 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.892+0000 7f2a4b7fe700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a440557c0 0x7f2a44057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:02.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.892+0000 7f2a4b7fe700 1 -- 172.21.15.116:0/1834017466 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2a6006faa0 con 0x7f2a6c0a4bc0 2020-11-20T12:17:02.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.892+0000 7f2a6b7fe700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a440557c0 0x7f2a44057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:02.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.896+0000 7f2a6b7fe700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a440557c0 0x7f2a44057c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f2a5c000fb0 tx=0x7f2a5c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:02.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:02.896+0000 7f2a4b7fe700 1 -- 172.21.15.116:0/1834017466 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2a60045940 con 0x7f2a6c0a4bc0 2020-11-20T12:17:03.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.060+0000 7f2a709e1700 1 -- 172.21.15.116:0/1834017466 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a6c061580 con 0x7f2a6c0a4bc0 2020-11-20T12:17:03.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a4b7fe700 1 -- 172.21.15.116:0/1834017466 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f2a60024070 con 0x7f2a6c0a4bc0 2020-11-20T12:17:03.063 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:03.064 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:58.907407+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:16:55.781578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T12:16:56.900539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:03.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 -- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a440557c0 msgr2=0x7f2a44057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a440557c0 0x7f2a44057c80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f2a5c000fb0 tx=0x7f2a5c009040).stop 2020-11-20T12:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 -- 172.21.15.116:0/1834017466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a6c0a4bc0 msgr2=0x7f2a6c164900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a6c0a4bc0 0x7f2a6c164900 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f2a6000b150 tx=0x7f2a60013000).stop 2020-11-20T12:17:03.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 -- 172.21.15.116:0/1834017466 shutdown_connections 2020-11-20T12:17:03.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a440557c0 0x7f2a44057c80 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a6c0c44a0 0x7f2a6c0a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a6c0668a0 0x7f2a6c14e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 --2- 172.21.15.116:0/1834017466 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a6c0a4bc0 0x7f2a6c164900 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 -- 172.21.15.116:0/1834017466 >> 172.21.15.116:0/1834017466 conn(0x7f2a6c062520 msgr2=0x7f2a6c0a5150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:03.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 -- 172.21.15.116:0/1834017466 shutdown_connections 2020-11-20T12:17:03.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.064+0000 7f2a709e1700 1 -- 172.21.15.116:0/1834017466 wait complete. 2020-11-20T12:17:03.090 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:16:55.781578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:16:56.900539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:17:03.091 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:03.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.268+0000 7fdb9a2f9700 1 Processor -- start 2020-11-20T12:17:03.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.268+0000 7fdb9a2f9700 1 -- start start 2020-11-20T12:17:03.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.268+0000 7fdb9a2f9700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fdb9413ad30 0x7fdb940a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.268+0000 7fdb9a2f9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94061250 0x7fdb940616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.268+0000 7fdb9a2f9700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb94152a80 con 0x7fdb9413b110 2020-11-20T12:17:03.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.268+0000 7fdb9a2f9700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb94152540 con 0x7fdb9413ad30 2020-11-20T12:17:03.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.268+0000 7fdb9a2f9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941527e0 con 0x7fdb94061250 2020-11-20T12:17:03.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.268+0000 7fdb98af6700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdb9413b110 0x7fdb94151ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54146/0 (socket says 172.21.15.116:54146) 2020-11-20T12:17:03.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.268+0000 7fdb98af6700 1 -- 172.21.15.116:0/2333109701 learned_addr learned my addr 172.21.15.116:0/2333109701 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:03.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb99af8700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94061250 0x7fdb940616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645691656 0 0) 0x7fdb94152a80 con 0x7fdb9413b110 2020-11-20T12:17:03.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb992f7700 1 --2- 172.21.15.116:0/2333109701 >> v2:172.21.15.173:3300/0 conn(0x7fdb9413ad30 0x7fdb940a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb78003240 con 0x7fdb9413b110 2020-11-20T12:17:03.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2504948789 0 0) 0x7fdb78003240 con 0x7fdb9413b110 2020-11-20T12:17:03.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94061250 msgr2=0x7fdb940616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:03.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94061250 0x7fdb940616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 >> v2:172.21.15.173:3300/0 conn(0x7fdb9413ad30 msgr2=0x7fdb940a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:03.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 --2- 172.21.15.116:0/2333109701 >> v2:172.21.15.173:3300/0 conn(0x7fdb9413ad30 0x7fdb940a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb940ac910 con 0x7fdb9413b110 2020-11-20T12:17:03.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fdb840014e0 con 0x7fdb9413b110 2020-11-20T12:17:03.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 >> v1:172.21.15.116:6789/0 conn(0x7fdb9413b110 legacy=0x7fdb94151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb78003730 0x7fdb78003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94061250 0x7fdb780044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb780049d0 0x7fdb78004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb94152540 con 0x7fdb78003730 2020-11-20T12:17:03.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb941527e0 con 0x7fdb780049d0 2020-11-20T12:17:03.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb780041d0 con 0x7fdb94061250 2020-11-20T12:17:03.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb992f7700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb78003730 0x7fdb78003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb98af6700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94061250 0x7fdb780044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb99af8700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb780049d0 0x7fdb78004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb992f7700 1 -- 172.21.15.116:0/2333109701 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94061250 msgr2=0x7fdb780044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:03.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb992f7700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94061250 0x7fdb780044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb992f7700 1 -- 172.21.15.116:0/2333109701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb780049d0 msgr2=0x7fdb78004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:03.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb992f7700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb780049d0 0x7fdb78004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb992f7700 1 -- 172.21.15.116:0/2333109701 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb78003240 con 0x7fdb78003730 2020-11-20T12:17:03.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.272+0000 7fdb992f7700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb78003730 0x7fdb78003b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fdb9000bc00 tx=0x7fdb90006df0).ready entity=mon.0 client_cookie=f64e98d1eaef717d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:03.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.276+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000b310 con 0x7fdb78003730 2020-11-20T12:17:03.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.276+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb9000ea30 con 0x7fdb78003730 2020-11-20T12:17:03.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.276+0000 7fdb8bfff700 1 -- 172.21.15.116:0/2333109701 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000edb0 con 0x7fdb78003730 2020-11-20T12:17:03.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.276+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/2333109701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb78003730 msgr2=0x7fdb78003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.276+0000 7fdb9a2f9700 1 --2- 172.21.15.116:0/2333109701 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb78003730 0x7fdb78003b50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fdb9000bc00 tx=0x7fdb90006df0).stop 2020-11-20T12:17:03.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.276+0000 7fdb99af8700 1 -- 172.21.15.116:0/2333109701 reap_dead start 2020-11-20T12:17:03.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.276+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/2333109701 shutdown_connections 2020-11-20T12:17:03.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.276+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/2333109701 >> 172.21.15.116:0/2333109701 conn(0x7fdb940678a0 msgr2=0x7fdb94067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:03.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.276+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/2333109701 shutdown_connections 2020-11-20T12:17:03.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.276+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/2333109701 wait complete. 2020-11-20T12:17:03.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb9a2f9700 1 Processor -- start 2020-11-20T12:17:03.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb9a2f9700 1 -- start start 2020-11-20T12:17:03.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb9a2f9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb940a1560 0x7fdb94164960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb9a2f9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94127b40 0x7fdb940a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb9a2f9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb940a1e30 0x7fdb9409f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb9a2f9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940a9030 con 0x7fdb940a1560 2020-11-20T12:17:03.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb9a2f9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940a92d0 con 0x7fdb940a1e30 2020-11-20T12:17:03.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb9a2f9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb940a9570 con 0x7fdb94127b40 2020-11-20T12:17:03.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb992f7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb940a1560 0x7fdb94164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb992f7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb940a1560 0x7fdb94164960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44724/0 (socket says 172.21.15.116:44724) 2020-11-20T12:17:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb992f7700 1 -- 172.21.15.116:0/1732655754 learned_addr learned my addr 172.21.15.116:0/1732655754 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb98af6700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94127b40 0x7fdb940a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb99af8700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb940a1e30 0x7fdb9409f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb992f7700 1 -- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94127b40 msgr2=0x7fdb940a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb992f7700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94127b40 0x7fdb940a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb992f7700 1 -- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb940a1e30 msgr2=0x7fdb9409f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb992f7700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb940a1e30 0x7fdb9409f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb992f7700 1 -- 172.21.15.116:0/1732655754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb940a9810 con 0x7fdb940a1560 2020-11-20T12:17:03.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.280+0000 7fdb99af8700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb940a1e30 0x7fdb9409f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:03.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.284+0000 7fdb992f7700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb940a1560 0x7fdb94164960 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fdb90006fd0 tx=0x7fdb90019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:03.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.284+0000 7fdb897fa700 1 -- 172.21.15.116:0/1732655754 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9000b390 con 0x7fdb940a1560 2020-11-20T12:17:03.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.284+0000 7fdb897fa700 1 -- 172.21.15.116:0/1732655754 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb90021070 con 0x7fdb940a1560 2020-11-20T12:17:03.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.284+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/1732655754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb94164e50 con 0x7fdb940a1560 2020-11-20T12:17:03.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.284+0000 7fdb897fa700 1 -- 172.21.15.116:0/1732655754 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb90012e70 con 0x7fdb940a1560 2020-11-20T12:17:03.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.284+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/1732655754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9409f500 con 0x7fdb940a1560 2020-11-20T12:17:03.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.284+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/1732655754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb940a99a0 con 0x7fdb940a1560 2020-11-20T12:17:03.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.284+0000 7fdb897fa700 1 -- 172.21.15.116:0/1732655754 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fdb9001d830 con 0x7fdb940a1560 2020-11-20T12:17:03.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.284+0000 7fdb897fa700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdb6c0557c0 0x7fdb6c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.284+0000 7fdb897fa700 1 -- 172.21.15.116:0/1732655754 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdb90070450 con 0x7fdb940a1560 2020-11-20T12:17:03.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.288+0000 7fdb98af6700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdb6c0557c0 0x7fdb6c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.288+0000 7fdb98af6700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdb6c0557c0 0x7fdb6c057c80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fdb940aadc0 tx=0x7fdb84007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:03.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.292+0000 7fdb897fa700 1 -- 172.21.15.116:0/1732655754 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fdb90046290 con 0x7fdb940a1560 2020-11-20T12:17:03.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.448+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/1732655754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdb94044f60 con 0x7fdb940a1560 2020-11-20T12:17:03.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.448+0000 7fdb897fa700 1 -- 172.21.15.116:0/1732655754 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v34) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdb900498b0 con 0x7fdb940a1560 2020-11-20T12:17:03.450 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:03.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdb6c0557c0 msgr2=0x7fdb6c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdb6c0557c0 0x7fdb6c057c80 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fdb940aadc0 tx=0x7fdb84007800).stop 2020-11-20T12:17:03.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/1732655754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb940a1560 msgr2=0x7fdb94164960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb940a1560 0x7fdb94164960 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7fdb90006fd0 tx=0x7fdb90019000).stop 2020-11-20T12:17:03.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/1732655754 shutdown_connections 2020-11-20T12:17:03.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdb6c0557c0 0x7fdb6c057c80 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdb940a1e30 0x7fdb9409f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdb940a1560 0x7fdb94164960 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 --2- 172.21.15.116:0/1732655754 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdb94127b40 0x7fdb940a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/1732655754 >> 172.21.15.116:0/1732655754 conn(0x7fdb940678a0 msgr2=0x7fdb94138e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/1732655754 shutdown_connections 2020-11-20T12:17:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.452+0000 7fdb9a2f9700 1 -- 172.21.15.116:0/1732655754 wait complete. 2020-11-20T12:17:03.478 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:03.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.660+0000 7f3050933700 1 Processor -- start 2020-11-20T12:17:03.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f3050933700 1 -- start start 2020-11-20T12:17:03.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f3050933700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c14bc40 0x7f304c151d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f3050933700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f304c0c5410 0x7f304c0be770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f3050933700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304c1526b0 con 0x7f304c156020 2020-11-20T12:17:03.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f3050933700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304c152950 con 0x7f304c0c5410 2020-11-20T12:17:03.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f3050933700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c152bf0 con 0x7f304c14bc40 2020-11-20T12:17:03.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f3043fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f304c156020 0x7f304c0c4d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54166/0 (socket says 172.21.15.116:54166) 2020-11-20T12:17:03.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f3043fff700 1 -- 172.21.15.116:0/822918634 learned_addr learned my addr 172.21.15.116:0/822918634 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:03.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f304a59c700 1 --2- 172.21.15.116:0/822918634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c14bc40 0x7f304c151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f304ad9d700 1 --2- 172.21.15.116:0/822918634 >> v2:172.21.15.173:3300/0 conn(0x7f304c0c5410 0x7f304c0be770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f3049d9b700 1 -- 172.21.15.116:0/822918634 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916668173 0 0) 0x7f304c1526b0 con 0x7f304c156020 2020-11-20T12:17:03.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f3049d9b700 1 -- 172.21.15.116:0/822918634 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f302c003240 con 0x7f304c156020 2020-11-20T12:17:03.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f304a59c700 1 -- 172.21.15.116:0/822918634 >> v2:172.21.15.173:3300/0 conn(0x7f304c0c5410 msgr2=0x7f304c0be770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:03.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f304a59c700 1 --2- 172.21.15.116:0/822918634 >> v2:172.21.15.173:3300/0 conn(0x7f304c0c5410 0x7f304c0be770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f304a59c700 1 -- 172.21.15.116:0/822918634 >> v1:172.21.15.116:6789/0 conn(0x7f304c156020 legacy=0x7f304c0c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.664+0000 7f304a59c700 1 -- 172.21.15.116:0/822918634 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c0a8670 con 0x7f304c14bc40 2020-11-20T12:17:03.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f304a59c700 1 --2- 172.21.15.116:0/822918634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c14bc40 0x7f304c151d70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f303c00b9a0 tx=0x7f303c006df0).ready entity=mon.2 client_cookie=de2a8d967ec46561 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:03.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3049d9b700 1 -- 172.21.15.116:0/822918634 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c009680 con 0x7f304c14bc40 2020-11-20T12:17:03.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3049d9b700 1 -- 172.21.15.116:0/822918634 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303c0097e0 con 0x7f304c14bc40 2020-11-20T12:17:03.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3049d9b700 1 -- 172.21.15.116:0/822918634 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303c009b60 con 0x7f304c14bc40 2020-11-20T12:17:03.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3050933700 1 -- 172.21.15.116:0/822918634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c14bc40 msgr2=0x7f304c151d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3050933700 1 --2- 172.21.15.116:0/822918634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c14bc40 0x7f304c151d70 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f303c00b9a0 tx=0x7f303c006df0).stop 2020-11-20T12:17:03.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3050933700 1 -- 172.21.15.116:0/822918634 shutdown_connections 2020-11-20T12:17:03.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3050933700 1 --2- 172.21.15.116:0/822918634 >> v2:172.21.15.173:3300/0 conn(0x7f304c0c5410 0x7f304c0be770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3050933700 1 --2- 172.21.15.116:0/822918634 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c14bc40 0x7f304c151d70 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3050933700 1 -- 172.21.15.116:0/822918634 >> 172.21.15.116:0/822918634 conn(0x7f304c0678a0 msgr2=0x7f304c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:03.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3050933700 1 -- 172.21.15.116:0/822918634 shutdown_connections 2020-11-20T12:17:03.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3050933700 1 -- 172.21.15.116:0/822918634 wait complete. 2020-11-20T12:17:03.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3050933700 1 Processor -- start 2020-11-20T12:17:03.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.668+0000 7f3050933700 1 -- start start 2020-11-20T12:17:03.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3050933700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c0c5410 0x7f304c14af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3050933700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c156020 0x7f304c148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3050933700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f304c1613c0 0x7f304c0c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3050933700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c0bfe20 con 0x7f304c156020 2020-11-20T12:17:03.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3050933700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304c14b430 con 0x7f304c1613c0 2020-11-20T12:17:03.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3050933700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f304c1623c0 con 0x7f304c0c5410 2020-11-20T12:17:03.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3043fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c156020 0x7f304c148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3043fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c156020 0x7f304c148160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44740/0 (socket says 172.21.15.116:44740) 2020-11-20T12:17:03.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3043fff700 1 -- 172.21.15.116:0/3549520489 learned_addr learned my addr 172.21.15.116:0/3549520489 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:03.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f304a59c700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c0c5410 0x7f304c14af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f304ad9d700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f304c1613c0 0x7f304c0c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3043fff700 1 -- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c0c5410 msgr2=0x7f304c14af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3043fff700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c0c5410 0x7f304c14af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3043fff700 1 -- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f304c1613c0 msgr2=0x7f304c0c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3043fff700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f304c1613c0 0x7f304c0c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3043fff700 1 -- 172.21.15.116:0/3549520489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304c161940 con 0x7f304c156020 2020-11-20T12:17:03.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f304a59c700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c0c5410 0x7f304c14af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:03.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3043fff700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c156020 0x7f304c148160 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f303400c710 tx=0x7f30340078b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:03.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f30427fc700 1 -- 172.21.15.116:0/3549520489 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30340072e0 con 0x7f304c156020 2020-11-20T12:17:03.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f30427fc700 1 -- 172.21.15.116:0/3549520489 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30340054c0 con 0x7f304c156020 2020-11-20T12:17:03.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f30427fc700 1 -- 172.21.15.116:0/3549520489 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3034005840 con 0x7f304c156020 2020-11-20T12:17:03.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3050933700 1 -- 172.21.15.116:0/3549520489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f304c161be0 con 0x7f304c156020 2020-11-20T12:17:03.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.672+0000 7f3050933700 1 -- 172.21.15.116:0/3549520489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f304c162120 con 0x7f304c156020 2020-11-20T12:17:03.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.676+0000 7f3050933700 1 -- 172.21.15.116:0/3549520489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304c05ca30 con 0x7f304c156020 2020-11-20T12:17:03.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.676+0000 7f30427fc700 1 -- 172.21.15.116:0/3549520489 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3034010070 con 0x7f304c156020 2020-11-20T12:17:03.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.676+0000 7f30427fc700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30280557c0 0x7f3028057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:03.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.676+0000 7f30427fc700 1 -- 172.21.15.116:0/3549520489 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f303406dd20 con 0x7f304c156020 2020-11-20T12:17:03.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.676+0000 7f304a59c700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30280557c0 0x7f3028057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:03.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.680+0000 7f304a59c700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30280557c0 0x7f3028057c80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f304c05e910 tx=0x7f303c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:03.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.684+0000 7f30427fc700 1 -- 172.21.15.116:0/3549520489 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3034043bc0 con 0x7f304c156020 2020-11-20T12:17:03.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.844+0000 7f3050933700 1 -- 172.21.15.116:0/3549520489 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f304c0a8360 con 0x7f304c156020 2020-11-20T12:17:03.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.844+0000 7f30427fc700 1 -- 172.21.15.116:0/3549520489 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f30340471e0 con 0x7f304c156020 2020-11-20T12:17:03.846 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:03.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 -- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30280557c0 msgr2=0x7f3028057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30280557c0 0x7f3028057c80 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f304c05e910 tx=0x7f303c015000).stop 2020-11-20T12:17:03.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 -- 172.21.15.116:0/3549520489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c156020 msgr2=0x7f304c148160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:03.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c156020 0x7f304c148160 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f303400c710 tx=0x7f30340078b0).stop 2020-11-20T12:17:03.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 -- 172.21.15.116:0/3549520489 shutdown_connections 2020-11-20T12:17:03.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30280557c0 0x7f3028057c80 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f304c1613c0 0x7f304c0c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f304c0c5410 0x7f304c14af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 --2- 172.21.15.116:0/3549520489 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f304c156020 0x7f304c148160 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:03.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 -- 172.21.15.116:0/3549520489 >> 172.21.15.116:0/3549520489 conn(0x7f304c0678a0 msgr2=0x7f304c152df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:03.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 -- 172.21.15.116:0/3549520489 shutdown_connections 2020-11-20T12:17:03.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:03.848+0000 7f3050933700 1 -- 172.21.15.116:0/3549520489 wait complete. 2020-11-20T12:17:03.872 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:17:04.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.056+0000 7f20fd39e700 1 Processor -- start 2020-11-20T12:17:04.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.056+0000 7f20fd39e700 1 -- start start 2020-11-20T12:17:04.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.056+0000 7f20fd39e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f20f8067910 0x7f20f8067cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.056+0000 7f20fd39e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f8060cb0 con 0x7f20f814f900 2020-11-20T12:17:04.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20fd39e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20f8060f50 con 0x7f20f8064760 2020-11-20T12:17:04.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20fd39e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20f8060170 con 0x7f20f8067910 2020-11-20T12:17:04.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20f6ffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f20f8067910 0x7f20f8067cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20f7fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f20f814f900 0x7f20f80a54d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54180/0 (socket says 172.21.15.116:54180) 2020-11-20T12:17:04.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20f7fff700 1 -- 172.21.15.116:0/2843428460 learned_addr learned my addr 172.21.15.116:0/2843428460 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20f5ffb700 1 -- 172.21.15.116:0/2843428460 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422134113 0 0) 0x7f20f8060cb0 con 0x7f20f814f900 2020-11-20T12:17:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20f6ffd700 1 -- 172.21.15.116:0/2843428460 >> v1:172.21.15.173:6789/0 conn(0x7f20f8064760 legacy=0x7f20f814f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20f6ffd700 1 -- 172.21.15.116:0/2843428460 >> v1:172.21.15.116:6789/0 conn(0x7f20f814f900 legacy=0x7f20f80a54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20f6ffd700 1 -- 172.21.15.116:0/2843428460 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8060410 con 0x7f20f8067910 2020-11-20T12:17:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20f6ffd700 1 --2- 172.21.15.116:0/2843428460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f8067910 0x7f20f8067cf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f20e800b9a0 tx=0x7f20e8006df0).ready entity=mon.0 client_cookie=ab7c22de97bcace8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:04.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20f5ffb700 1 -- 172.21.15.116:0/2843428460 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e80097b0 con 0x7f20f8067910 2020-11-20T12:17:04.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20f5ffb700 1 -- 172.21.15.116:0/2843428460 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e8009910 con 0x7f20f8067910 2020-11-20T12:17:04.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.060+0000 7f20f5ffb700 1 -- 172.21.15.116:0/2843428460 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8009c90 con 0x7f20f8067910 2020-11-20T12:17:04.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.064+0000 7f20fd39e700 1 -- 172.21.15.116:0/2843428460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f8067910 msgr2=0x7f20f8067cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.064+0000 7f20fd39e700 1 --2- 172.21.15.116:0/2843428460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f8067910 0x7f20f8067cf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f20e800b9a0 tx=0x7f20e8006df0).stop 2020-11-20T12:17:04.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.064+0000 7f20fd39e700 1 -- 172.21.15.116:0/2843428460 shutdown_connections 2020-11-20T12:17:04.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.064+0000 7f20fd39e700 1 --2- 172.21.15.116:0/2843428460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f8067910 0x7f20f8067cf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.064+0000 7f20fd39e700 1 -- 172.21.15.116:0/2843428460 >> 172.21.15.116:0/2843428460 conn(0x7f20f80c5680 msgr2=0x7f20f80c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:04.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.064+0000 7f20fd39e700 1 -- 172.21.15.116:0/2843428460 shutdown_connections 2020-11-20T12:17:04.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.064+0000 7f20fd39e700 1 -- 172.21.15.116:0/2843428460 wait complete. 2020-11-20T12:17:04.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20fd39e700 1 Processor -- start 2020-11-20T12:17:04.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20fd39e700 1 -- start start 2020-11-20T12:17:04.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20fd39e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f8064760 0x7f20f80bedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20fd39e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f8067910 0x7f20f815f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20fd39e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f814f900 0x7f20f813e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20fd39e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80bf2a0 con 0x7f20f8064760 2020-11-20T12:17:04.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20fd39e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20f815fe30 con 0x7f20f8067910 2020-11-20T12:17:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20fd39e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20f80a86d0 con 0x7f20f814f900 2020-11-20T12:17:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20f6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f8064760 0x7f20f80bedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20f67fc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f8067910 0x7f20f815f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20f7fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f814f900 0x7f20f813e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20f6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f8064760 0x7f20f80bedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44756/0 (socket says 172.21.15.116:44756) 2020-11-20T12:17:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20f6ffd700 1 -- 172.21.15.116:0/1159551165 learned_addr learned my addr 172.21.15.116:0/1159551165 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20f6ffd700 1 -- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f814f900 msgr2=0x7f20f813e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20f6ffd700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f814f900 0x7f20f813e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20f6ffd700 1 -- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f8067910 msgr2=0x7f20f815f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20f6ffd700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f8067910 0x7f20f815f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20f6ffd700 1 -- 172.21.15.116:0/1159551165 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f80a8970 con 0x7f20f8064760 2020-11-20T12:17:04.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.068+0000 7f20f6ffd700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f8064760 0x7f20f80bedb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f20e8006e30 tx=0x7f20e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:04.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.072+0000 7f20deffd700 1 -- 172.21.15.116:0/1159551165 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e8011e50 con 0x7f20f8064760 2020-11-20T12:17:04.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.072+0000 7f20deffd700 1 -- 172.21.15.116:0/1159551165 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20e801e460 con 0x7f20f8064760 2020-11-20T12:17:04.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.072+0000 7f20deffd700 1 -- 172.21.15.116:0/1159551165 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20e801e7e0 con 0x7f20f8064760 2020-11-20T12:17:04.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.072+0000 7f20fd39e700 1 -- 172.21.15.116:0/1159551165 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20f81520f0 con 0x7f20f8064760 2020-11-20T12:17:04.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.072+0000 7f20fd39e700 1 -- 172.21.15.116:0/1159551165 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20f815fff0 con 0x7f20f8064760 2020-11-20T12:17:04.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.072+0000 7f20fd39e700 1 -- 172.21.15.116:0/1159551165 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20f805ca30 con 0x7f20f8064760 2020-11-20T12:17:04.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.072+0000 7f20deffd700 1 -- 172.21.15.116:0/1159551165 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f20e8025030 con 0x7f20f8064760 2020-11-20T12:17:04.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.072+0000 7f20deffd700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f20d40557c0 0x7f20d4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.072+0000 7f20deffd700 1 -- 172.21.15.116:0/1159551165 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f20e806fec0 con 0x7f20f8064760 2020-11-20T12:17:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.072+0000 7f20f67fc700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f20d40557c0 0x7f20d4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.076+0000 7f20f67fc700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f20d40557c0 0x7f20d4057c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f20f80bea50 tx=0x7f20ec007b90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:04.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.080+0000 7f20deffd700 1 -- 172.21.15.116:0/1159551165 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f20e8045d90 con 0x7f20f8064760 2020-11-20T12:17:04.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.236+0000 7f20fd39e700 1 -- 172.21.15.116:0/1159551165 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f20f8068210 con 0x7f20f8064760 2020-11-20T12:17:04.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20deffd700 1 -- 172.21.15.116:0/1159551165 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+10 (secure 0 0 0) 0x7f20e8023070 con 0x7f20f8064760 2020-11-20T12:17:04.239 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:04.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 -- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f20d40557c0 msgr2=0x7f20d4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f20d40557c0 0x7f20d4057c80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f20f80bea50 tx=0x7f20ec007b90).stop 2020-11-20T12:17:04.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 -- 172.21.15.116:0/1159551165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f8064760 msgr2=0x7f20f80bedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f8064760 0x7f20f80bedb0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f20e8006e30 tx=0x7f20e8015000).stop 2020-11-20T12:17:04.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 -- 172.21.15.116:0/1159551165 shutdown_connections 2020-11-20T12:17:04.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f20d40557c0 0x7f20d4057c80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f20f814f900 0x7f20f813e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f20f8064760 0x7f20f80bedb0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 --2- 172.21.15.116:0/1159551165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f20f8067910 0x7f20f815f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 -- 172.21.15.116:0/1159551165 >> 172.21.15.116:0/1159551165 conn(0x7f20f80c5680 msgr2=0x7f20f80a7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:04.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 -- 172.21.15.116:0/1159551165 shutdown_connections 2020-11-20T12:17:04.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.240+0000 7f20fd39e700 1 -- 172.21.15.116:0/1159551165 wait complete. 2020-11-20T12:17:04.265 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:17:04.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.444+0000 7fb80c0a3700 1 Processor -- start 2020-11-20T12:17:04.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb80c0a3700 1 -- start start 2020-11-20T12:17:04.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb80c0a3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040c4f60 0x7fb8040c5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb80c0a3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb804152de0 0x7fb804153290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb80c0a3700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb804061b20 con 0x7fb804061280 2020-11-20T12:17:04.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb80c0a3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb804061dc0 con 0x7fb804152de0 2020-11-20T12:17:04.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb80c0a3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb804062060 con 0x7fb8040c4f60 2020-11-20T12:17:04.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb809e3f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040c4f60 0x7fb8040c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb80a640700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb804152de0 0x7fb804153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb809e3f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040c4f60 0x7fb8040c5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:53710/0 (socket says 172.21.15.116:53710) 2020-11-20T12:17:04.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb809e3f700 1 -- 172.21.15.116:0/3089839112 learned_addr learned my addr 172.21.15.116:0/3089839112 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:04.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb809e3f700 1 -- 172.21.15.116:0/3089839112 >> v2:172.21.15.173:3300/0 conn(0x7fb804152de0 msgr2=0x7fb804153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:04.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb809e3f700 1 --2- 172.21.15.116:0/3089839112 >> v2:172.21.15.173:3300/0 conn(0x7fb804152de0 0x7fb804153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb809e3f700 1 -- 172.21.15.116:0/3089839112 >> v1:172.21.15.173:6789/0 conn(0x7fb804061280 legacy=0x7fb804062a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb809e3f700 1 -- 172.21.15.116:0/3089839112 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804062300 con 0x7fb8040c4f60 2020-11-20T12:17:04.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.448+0000 7fb80a640700 1 --2- 172.21.15.116:0/3089839112 >> v2:172.21.15.173:3300/0 conn(0x7fb804152de0 0x7fb804153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb809e3f700 1 --2- 172.21.15.116:0/3089839112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040c4f60 0x7fb8040c5340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fb800006fd0 tx=0x7fb800006e50).ready entity=mon.2 client_cookie=146acc2c0d59df50 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb808e3d700 1 -- 172.21.15.116:0/3089839112 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800006030 con 0x7fb8040c4f60 2020-11-20T12:17:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb808e3d700 1 -- 172.21.15.116:0/3089839112 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb800006190 con 0x7fb8040c4f60 2020-11-20T12:17:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb808e3d700 1 -- 172.21.15.116:0/3089839112 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800006510 con 0x7fb8040c4f60 2020-11-20T12:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb80c0a3700 1 -- 172.21.15.116:0/3089839112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040c4f60 msgr2=0x7fb8040c5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb80c0a3700 1 --2- 172.21.15.116:0/3089839112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040c4f60 0x7fb8040c5340 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7fb800006fd0 tx=0x7fb800006e50).stop 2020-11-20T12:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb80c0a3700 1 -- 172.21.15.116:0/3089839112 shutdown_connections 2020-11-20T12:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb80c0a3700 1 --2- 172.21.15.116:0/3089839112 >> v2:172.21.15.173:3300/0 conn(0x7fb804152de0 0x7fb804153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb80c0a3700 1 --2- 172.21.15.116:0/3089839112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040c4f60 0x7fb8040c5340 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb80c0a3700 1 -- 172.21.15.116:0/3089839112 >> 172.21.15.116:0/3089839112 conn(0x7fb80408f950 msgr2=0x7fb804060950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb80c0a3700 1 -- 172.21.15.116:0/3089839112 shutdown_connections 2020-11-20T12:17:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.452+0000 7fb80c0a3700 1 -- 172.21.15.116:0/3089839112 wait complete. 2020-11-20T12:17:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb80c0a3700 1 Processor -- start 2020-11-20T12:17:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb80c0a3700 1 -- start start 2020-11-20T12:17:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb80c0a3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb804061280 0x7fb80414ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb80c0a3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8040c4f60 0x7fb804161c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb80c0a3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb804152de0 0x7fb80414aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb80c0a3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8041612f0 con 0x7fb804061280 2020-11-20T12:17:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb80c0a3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80414c680 con 0x7fb8040c4f60 2020-11-20T12:17:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb80c0a3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80414c920 con 0x7fb804152de0 2020-11-20T12:17:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb809e3f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb804061280 0x7fb80414ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb809e3f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb804061280 0x7fb80414ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44766/0 (socket says 172.21.15.116:44766) 2020-11-20T12:17:04.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb809e3f700 1 -- 172.21.15.116:0/949009136 learned_addr learned my addr 172.21.15.116:0/949009136 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.456+0000 7fb80a640700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb804152de0 0x7fb80414aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb80963e700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8040c4f60 0x7fb804161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb809e3f700 1 -- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb804152de0 msgr2=0x7fb80414aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb809e3f700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb804152de0 0x7fb80414aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb809e3f700 1 -- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8040c4f60 msgr2=0x7fb804161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb809e3f700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8040c4f60 0x7fb804161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb809e3f700 1 -- 172.21.15.116:0/949009136 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb804161520 con 0x7fb804061280 2020-11-20T12:17:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb809e3f700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb804061280 0x7fb80414ebe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fb800006be0 tx=0x7fb800015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/949009136 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8000058e0 con 0x7fb804061280 2020-11-20T12:17:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/949009136 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb800005a40 con 0x7fb804061280 2020-11-20T12:17:04.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/949009136 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800005de0 con 0x7fb804061280 2020-11-20T12:17:04.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb80c0a3700 1 -- 172.21.15.116:0/949009136 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8041617c0 con 0x7fb804061280 2020-11-20T12:17:04.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb80c0a3700 1 -- 172.21.15.116:0/949009136 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80414f0d0 con 0x7fb804061280 2020-11-20T12:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb80c0a3700 1 -- 172.21.15.116:0/949009136 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80405ca30 con 0x7fb804061280 2020-11-20T12:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.460+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/949009136 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb800025030 con 0x7fb804061280 2020-11-20T12:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.464+0000 7fb7f1ffb700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 0x7fb7dc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.464+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/949009136 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb80006f970 con 0x7fb804061280 2020-11-20T12:17:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.464+0000 7fb80963e700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 0x7fb7dc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.464+0000 7fb80963e700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 0x7fb7dc057c80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fb7f4000bc0 tx=0x7fb7f4007b70).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:04.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.468+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/949009136 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb8000458c0 con 0x7fb804061280 2020-11-20T12:17:04.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.624+0000 7fb80c0a3700 1 -- 172.21.15.116:0/949009136 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb8040662d0 con 0x7fb804061280 2020-11-20T12:17:04.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.624+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/949009136 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb800023030 con 0x7fb804061280 2020-11-20T12:17:04.626 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:04.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 -- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 msgr2=0x7fb7dc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 0x7fb7dc057c80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7fb7f4000bc0 tx=0x7fb7f4007b70).stop 2020-11-20T12:17:04.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 -- 172.21.15.116:0/949009136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb804061280 msgr2=0x7fb80414ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb804061280 0x7fb80414ebe0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7fb800006be0 tx=0x7fb800015000).stop 2020-11-20T12:17:04.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 -- 172.21.15.116:0/949009136 shutdown_connections 2020-11-20T12:17:04.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 0x7fb7dc057c80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb804152de0 0x7fb80414aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb804061280 0x7fb80414ebe0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 --2- 172.21.15.116:0/949009136 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8040c4f60 0x7fb804161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 -- 172.21.15.116:0/949009136 >> 172.21.15.116:0/949009136 conn(0x7fb80408f950 msgr2=0x7fb80414b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:04.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 -- 172.21.15.116:0/949009136 shutdown_connections 2020-11-20T12:17:04.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.628+0000 7fb80c0a3700 1 -- 172.21.15.116:0/949009136 wait complete. 2020-11-20T12:17:04.652 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:17:04.653 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:04.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.832+0000 7f1fadc89700 1 Processor -- start 2020-11-20T12:17:04.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.832+0000 7f1fadc89700 1 -- start start 2020-11-20T12:17:04.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fadc89700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fa813aee0 0x7f1fa8152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fadc89700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1fa80aa790 0x7f1fa80ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fadc89700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa8152a30 con 0x7f1fa80612f0 2020-11-20T12:17:04.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fadc89700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fa80aac10 con 0x7f1fa80aa790 2020-11-20T12:17:04.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fadc89700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa80aaeb0 con 0x7f1fa813aee0 2020-11-20T12:17:04.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fa7fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1fa80aa790 0x7f1fa80ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fa7fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1fa80aa790 0x7f1fa80ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44776/0 (socket says 172.21.15.116:44776) 2020-11-20T12:17:04.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1facc87700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fa813aee0 0x7f1fa8152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fa7fff700 1 -- 172.21.15.116:0/2061557173 learned_addr learned my addr 172.21.15.116:0/2061557173 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:04.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fa7fff700 1 -- 172.21.15.116:0/2061557173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fa813aee0 msgr2=0x7f1fa8152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:04.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fa7fff700 1 --2- 172.21.15.116:0/2061557173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fa813aee0 0x7f1fa8152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fa7fff700 1 -- 172.21.15.116:0/2061557173 >> v1:172.21.15.173:6789/0 conn(0x7f1fa80612f0 legacy=0x7f1fa80a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.836+0000 7f1fa7fff700 1 -- 172.21.15.116:0/2061557173 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa80acbf0 con 0x7f1fa80aa790 2020-11-20T12:17:04.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fa7fff700 1 --2- 172.21.15.116:0/2061557173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fa80aa790 0x7f1fa80ad5d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f1f94006fd0 tx=0x7f1f94006eb0).ready entity=mon.0 client_cookie=41df140cb1bf7b47 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:04.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fa77fe700 1 -- 172.21.15.116:0/2061557173 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94009810 con 0x7f1fa80aa790 2020-11-20T12:17:04.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fa77fe700 1 -- 172.21.15.116:0/2061557173 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f94009970 con 0x7f1fa80aa790 2020-11-20T12:17:04.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fa77fe700 1 -- 172.21.15.116:0/2061557173 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94009cf0 con 0x7f1fa80aa790 2020-11-20T12:17:04.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fadc89700 1 -- 172.21.15.116:0/2061557173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fa80aa790 msgr2=0x7f1fa80ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fadc89700 1 --2- 172.21.15.116:0/2061557173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fa80aa790 0x7f1fa80ad5d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f1f94006fd0 tx=0x7f1f94006eb0).stop 2020-11-20T12:17:04.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fadc89700 1 -- 172.21.15.116:0/2061557173 shutdown_connections 2020-11-20T12:17:04.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fadc89700 1 --2- 172.21.15.116:0/2061557173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fa813aee0 0x7f1fa8152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fadc89700 1 --2- 172.21.15.116:0/2061557173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fa80aa790 0x7f1fa80ad5d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fadc89700 1 -- 172.21.15.116:0/2061557173 >> 172.21.15.116:0/2061557173 conn(0x7f1fa80678a0 msgr2=0x7f1fa8067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:04.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fadc89700 1 -- 172.21.15.116:0/2061557173 shutdown_connections 2020-11-20T12:17:04.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.840+0000 7f1fadc89700 1 -- 172.21.15.116:0/2061557173 wait complete. 2020-11-20T12:17:04.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.844+0000 7f1fadc89700 1 Processor -- start 2020-11-20T12:17:04.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.844+0000 7f1fadc89700 1 -- start start 2020-11-20T12:17:04.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fadc89700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fa80612f0 0x7f1fa8164320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fadc89700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fa80aa790 0x7f1fa80c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fadc89700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fa813aee0 0x7f1fa8161500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fadc89700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8164b80 con 0x7f1fa80aa790 2020-11-20T12:17:04.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fadc89700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa8164e20 con 0x7f1fa80612f0 2020-11-20T12:17:04.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fadc89700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa81650c0 con 0x7f1fa813aee0 2020-11-20T12:17:04.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fa80aa790 0x7f1fa80c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fa80aa790 0x7f1fa80c4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44780/0 (socket says 172.21.15.116:44780) 2020-11-20T12:17:04.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa7fff700 1 -- 172.21.15.116:0/3042847343 learned_addr learned my addr 172.21.15.116:0/3042847343 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:04.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fad488700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fa813aee0 0x7f1fa8161500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1facc87700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fa80612f0 0x7f1fa8164320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa7fff700 1 -- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fa813aee0 msgr2=0x7f1fa8161500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa7fff700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fa813aee0 0x7f1fa8161500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa7fff700 1 -- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fa80612f0 msgr2=0x7f1fa8164320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:04.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa7fff700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fa80612f0 0x7f1fa8164320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:04.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa7fff700 1 -- 172.21.15.116:0/3042847343 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8161060 con 0x7f1fa80aa790 2020-11-20T12:17:04.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa7fff700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fa80aa790 0x7f1fa80c4c50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f1fa8081760 tx=0x7f1f94015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:04.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa4ff9700 1 -- 172.21.15.116:0/3042847343 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f94011c80 con 0x7f1fa80aa790 2020-11-20T12:17:04.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa4ff9700 1 -- 172.21.15.116:0/3042847343 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f94011de0 con 0x7f1fa80aa790 2020-11-20T12:17:04.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fadc89700 1 -- 172.21.15.116:0/3042847343 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa8160d20 con 0x7f1fa80aa790 2020-11-20T12:17:04.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fa4ff9700 1 -- 172.21.15.116:0/3042847343 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9401e670 con 0x7f1fa80aa790 2020-11-20T12:17:04.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.848+0000 7f1fadc89700 1 -- 172.21.15.116:0/3042847343 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa80c5140 con 0x7f1fa80aa790 2020-11-20T12:17:04.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.852+0000 7f1fadc89700 1 -- 172.21.15.116:0/3042847343 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa81611f0 con 0x7f1fa80aa790 2020-11-20T12:17:04.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.852+0000 7f1fa4ff9700 1 -- 172.21.15.116:0/3042847343 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1f94025030 con 0x7f1fa80aa790 2020-11-20T12:17:04.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.856+0000 7f1fa4ff9700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f800557c0 0x7f1f80057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:04.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.856+0000 7f1fa4ff9700 1 -- 172.21.15.116:0/3042847343 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1f9406fc80 con 0x7f1fa80aa790 2020-11-20T12:17:04.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.856+0000 7f1facc87700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f800557c0 0x7f1f80057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:04.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.856+0000 7f1facc87700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f800557c0 0x7f1f80057c80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f1f9c006fd0 tx=0x7f1f9c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:04.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:04.860+0000 7f1fa4ff9700 1 -- 172.21.15.116:0/3042847343 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1f94045bd0 con 0x7f1fa80aa790 2020-11-20T12:17:05.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.068+0000 7f1fadc89700 1 -- 172.21.15.116:0/3042847343 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1fa8044f60 con 0x7f1fa80aa790 2020-11-20T12:17:05.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fa4ff9700 1 -- 172.21.15.116:0/3042847343 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2851 (secure 0 0 0) 0x7f1f94023030 con 0x7f1fa80aa790 2020-11-20T12:17:05.071 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:05.072 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3031892761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3031892761},{"type":"v1","addr":"172.21.15.173:6836","nonce":3031892761}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1794965470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1794965470},{"type":"v1","addr":"172.21.15.173:6837","nonce":1794965470}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":9},{"gid":4995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/4143559744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":4143559744},{"type":"v1","addr":"172.21.15.116:6833","nonce":4143559744}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:57.901741+0000","modified":"2020-11-20T12:16:59.918401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4998":{"gid":4998,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6835/3994085646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3994085646},{"type":"v1","addr":"172.21.15.116:6835","nonce":3994085646}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T12:17:05.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 -- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f800557c0 msgr2=0x7f1f80057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f800557c0 0x7f1f80057c80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f1f9c006fd0 tx=0x7f1f9c006df0).stop 2020-11-20T12:17:05.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 -- 172.21.15.116:0/3042847343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fa80aa790 msgr2=0x7f1fa80c4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:05.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fa80aa790 0x7f1fa80c4c50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f1fa8081760 tx=0x7f1f94015000).stop 2020-11-20T12:17:05.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 -- 172.21.15.116:0/3042847343 shutdown_connections 2020-11-20T12:17:05.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f800557c0 0x7f1f80057c80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fa813aee0 0x7f1fa8161500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fa80612f0 0x7f1fa8164320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 --2- 172.21.15.116:0/3042847343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fa80aa790 0x7f1fa80c4c50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:05.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 -- 172.21.15.116:0/3042847343 >> 172.21.15.116:0/3042847343 conn(0x7f1fa80678a0 msgr2=0x7f1fa80a7fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:05.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 -- 172.21.15.116:0/3042847343 shutdown_connections 2020-11-20T12:17:05.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:05.072+0000 7f1fadc89700 1 -- 172.21.15.116:0/3042847343 wait complete. 2020-11-20T12:17:05.077 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 14 2020-11-20T12:17:05.505 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:57.901741+0000', 'modified': '2020-11-20T12:16:59.918401+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4998}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4998': {'gid': 4998, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.116:6835/3994085646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3994085646}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3994085646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:17:05.505 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:17:05.506 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:17:05.700 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4998,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.7772640739999996,"mdsmap_epoch":14,"osdmap_epoch":34,"osdmap_epoch_barrier":34,"uptime":16.118327516000001} 2020-11-20T12:17:05.701 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4998, "mdsmap_epoch": 14, "osdmap_epoch": 34, "osdmap_epoch_barrier": 34, "rank_uptime": 7.777264074, "state": "up:active", "uptime": 16.118327516, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:17:05.701 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:17:05.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:05.702 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-20T12:17:05.719 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:17:05.720 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:05.787 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:05.788 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:17:05.813 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:05.813 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:17:05.820 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:17:05.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:05.821 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-20T12:17:05.848 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:17:05.849 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:17:05.849 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-20T12:17:05.850 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-20T12:17:05.850 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:17:05.851 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:17:05.851 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:17:05.851 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:17:05.852 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:05.852 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:17:05.869 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:17:05.870 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:17:05.870 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:17:05.870 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:05.871 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:17:05.871 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:05.871 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:05.871 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:05.872 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:05.872 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:05.872 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:05.872 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:05.873 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:17:05.873 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:05.873 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:17:05.873 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:05.874 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:17:05.874 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:05.874 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:17:05.874 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:05.874 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:17:05.875 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:05.875 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:17:05.875 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:05.875 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:17:05.876 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:05.876 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:17:05.876 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:05.876 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:05.876 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:17:05.877 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:17:05.877 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:05.877 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:05.877 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:17:05.878 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:17:05.878 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:05.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:05.879 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:17:05.915 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:17:05.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:05.918 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:17:05.982 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:17:05.982 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:17:05.983 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:17:05.983 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:17:05.983 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:17:05.983 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:17:05.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:05.985 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:17:05.986 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:17:05.986 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:17:05.986 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:17:05.987 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:17:05.987 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:17:06.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:06.082 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:17:06.098 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:17:06.098 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:17:06.099 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:17:06.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:06.100 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2020-11-20T12:17:06.174 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.352+0000 7f79d8ec3700 1 Processor -- start 2020-11-20T12:17:06.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.356+0000 7f79d8ec3700 1 -- start start 2020-11-20T12:17:06.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.356+0000 7f79d8ec3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f79d4138d80 0x7f79d415a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:06.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.356+0000 7f79d8ec3700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d40a8c80 con 0x7f79d40c45a0 2020-11-20T12:17:06.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.356+0000 7f79d8ec3700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d40a9180 con 0x7f79d40c41c0 2020-11-20T12:17:06.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.356+0000 7f79d8ec3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d40a9420 con 0x7f79d4138d80 2020-11-20T12:17:06.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.356+0000 7f79d2ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f79d40c45a0 0x7f79d4138670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54228/0 (socket says 172.21.15.116:54228) 2020-11-20T12:17:06.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.356+0000 7f79d2ffd700 1 -- 172.21.15.116:0/2748627763 learned_addr learned my addr 172.21.15.116:0/2748627763 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:06.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.356+0000 7f79d3fff700 1 --2- 172.21.15.116:0/2748627763 >> v2:172.21.15.173:3300/0 conn(0x7f79d4138d80 0x7f79d415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:06.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.356+0000 7f79d27fc700 1 -- 172.21.15.116:0/2748627763 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550362569 0 0) 0x7f79d40a8c80 con 0x7f79d40c45a0 2020-11-20T12:17:06.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d27fc700 1 -- 172.21.15.116:0/2748627763 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c0003240 con 0x7f79d40c45a0 2020-11-20T12:17:06.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d27fc700 1 -- 172.21.15.116:0/2748627763 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839083238 0 0) 0x7f79d40a9180 con 0x7f79d40c41c0 2020-11-20T12:17:06.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d27fc700 1 -- 172.21.15.116:0/2748627763 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d40a8c80 con 0x7f79d40c41c0 2020-11-20T12:17:06.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d27fc700 1 -- 172.21.15.116:0/2748627763 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1497355761 0 0) 0x7f79c0003240 con 0x7f79d40c45a0 2020-11-20T12:17:06.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d3fff700 1 -- 172.21.15.116:0/2748627763 >> v1:172.21.15.173:6789/0 conn(0x7f79d40c41c0 legacy=0x7f79d40a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d3fff700 1 -- 172.21.15.116:0/2748627763 >> v1:172.21.15.116:6789/0 conn(0x7f79d40c45a0 legacy=0x7f79d4138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d3fff700 1 -- 172.21.15.116:0/2748627763 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d40a96c0 con 0x7f79d4138d80 2020-11-20T12:17:06.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d3fff700 1 --2- 172.21.15.116:0/2748627763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79d4138d80 0x7f79d415a030 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f79cc009c90 tx=0x7f79cc0091d0).ready entity=mon.1 client_cookie=7752338e0e2653af server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d27fc700 1 -- 172.21.15.116:0/2748627763 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c80014e0 con 0x7f79d4138d80 2020-11-20T12:17:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d27fc700 1 -- 172.21.15.116:0/2748627763 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc00b6b0 con 0x7f79d4138d80 2020-11-20T12:17:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d27fc700 1 -- 172.21.15.116:0/2748627763 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc00ba30 con 0x7f79d4138d80 2020-11-20T12:17:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d8ec3700 1 -- 172.21.15.116:0/2748627763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79d4138d80 msgr2=0x7f79d415a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d8ec3700 1 --2- 172.21.15.116:0/2748627763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79d4138d80 0x7f79d415a030 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f79cc009c90 tx=0x7f79cc0091d0).stop 2020-11-20T12:17:06.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d8ec3700 1 -- 172.21.15.116:0/2748627763 shutdown_connections 2020-11-20T12:17:06.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d8ec3700 1 --2- 172.21.15.116:0/2748627763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79d4138d80 0x7f79d415a030 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d8ec3700 1 -- 172.21.15.116:0/2748627763 >> 172.21.15.116:0/2748627763 conn(0x7f79d4062520 msgr2=0x7f79d4062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:06.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d8ec3700 1 -- 172.21.15.116:0/2748627763 shutdown_connections 2020-11-20T12:17:06.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.360+0000 7f79d8ec3700 1 -- 172.21.15.116:0/2748627763 wait complete. 2020-11-20T12:17:06.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d8ec3700 1 Processor -- start 2020-11-20T12:17:06.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d8ec3700 1 -- start start 2020-11-20T12:17:06.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d8ec3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79d40c41c0 0x7f79d41578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:06.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d8ec3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79d40c45a0 0x7f79d4151e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d8ec3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79d4138d80 0x7f79d414ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d8ec3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d4164360 con 0x7f79d4138d80 2020-11-20T12:17:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d8ec3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d4164600 con 0x7f79d40c45a0 2020-11-20T12:17:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d8ec3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d41648a0 con 0x7f79d40c41c0 2020-11-20T12:17:06.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79d4138d80 0x7f79d414ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79d4138d80 0x7f79d414ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44798/0 (socket says 172.21.15.116:44798) 2020-11-20T12:17:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d3fff700 1 -- 172.21.15.116:0/3078218397 learned_addr learned my addr 172.21.15.116:0/3078218397 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d2ffd700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79d40c45a0 0x7f79d4151e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:06.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d37fe700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79d40c41c0 0x7f79d41578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d3fff700 1 -- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79d40c41c0 msgr2=0x7f79d41578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d3fff700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79d40c41c0 0x7f79d41578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d3fff700 1 -- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79d40c45a0 msgr2=0x7f79d4151e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d3fff700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79d40c45a0 0x7f79d4151e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d3fff700 1 -- 172.21.15.116:0/3078218397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d4164b40 con 0x7f79d4138d80 2020-11-20T12:17:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.364+0000 7f79d2ffd700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79d40c45a0 0x7f79d4151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.368+0000 7f79d3fff700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79d4138d80 0x7f79d414ed90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f79d40c1ad0 tx=0x7f79cc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:06.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.368+0000 7f79b37fe700 1 -- 172.21.15.116:0/3078218397 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc01d450 con 0x7f79d4138d80 2020-11-20T12:17:06.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.368+0000 7f79b37fe700 1 -- 172.21.15.116:0/3078218397 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79cc01d5b0 con 0x7f79d4138d80 2020-11-20T12:17:06.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.368+0000 7f79d8ec3700 1 -- 172.21.15.116:0/3078218397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d4152320 con 0x7f79d4138d80 2020-11-20T12:17:06.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.368+0000 7f79b37fe700 1 -- 172.21.15.116:0/3078218397 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79cc01d950 con 0x7f79d4138d80 2020-11-20T12:17:06.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.368+0000 7f79d8ec3700 1 -- 172.21.15.116:0/3078218397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d414f280 con 0x7f79d4138d80 2020-11-20T12:17:06.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.368+0000 7f79d8ec3700 1 -- 172.21.15.116:0/3078218397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d4164cd0 con 0x7f79d4138d80 2020-11-20T12:17:06.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.368+0000 7f79b37fe700 1 -- 172.21.15.116:0/3078218397 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f79cc02d070 con 0x7f79d4138d80 2020-11-20T12:17:06.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.372+0000 7f79b37fe700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79ac0557c0 0x7f79ac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:06.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.372+0000 7f79b37fe700 1 -- 172.21.15.116:0/3078218397 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f79cc070c10 con 0x7f79d4138d80 2020-11-20T12:17:06.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.372+0000 7f79d37fe700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79ac0557c0 0x7f79ac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:06.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.372+0000 7f79d37fe700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79ac0557c0 0x7f79ac057c80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f79c4003500 tx=0x7f79c4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:06.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.376+0000 7f79b37fe700 1 -- 172.21.15.116:0/3078218397 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f79cc046ab0 con 0x7f79d4138d80 2020-11-20T12:17:06.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.544+0000 7f79d8ec3700 1 -- 172.21.15.116:0/3078218397 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79d4044f60 con 0x7f79d4138d80 2020-11-20T12:17:06.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.548+0000 7f79b37fe700 1 -- 172.21.15.116:0/3078218397 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f79cc021020 con 0x7f79d4138d80 2020-11-20T12:17:06.548 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:06.549 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:58.907407+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:16:55.781578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T12:16:56.900539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:06.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.548+0000 7f79d8ec3700 1 -- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79ac0557c0 msgr2=0x7f79ac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.548+0000 7f79d8ec3700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79ac0557c0 0x7f79ac057c80 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f79c4003500 tx=0x7f79c4009040).stop 2020-11-20T12:17:06.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.548+0000 7f79d8ec3700 1 -- 172.21.15.116:0/3078218397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79d4138d80 msgr2=0x7f79d414ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.548+0000 7f79d8ec3700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79d4138d80 0x7f79d414ed90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f79d40c1ad0 tx=0x7f79cc016000).stop 2020-11-20T12:17:06.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.552+0000 7f79d8ec3700 1 -- 172.21.15.116:0/3078218397 shutdown_connections 2020-11-20T12:17:06.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.552+0000 7f79d8ec3700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79ac0557c0 0x7f79ac057c80 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.552+0000 7f79d8ec3700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79d4138d80 0x7f79d414ed90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.552+0000 7f79d8ec3700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79d40c41c0 0x7f79d41578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.552+0000 7f79d8ec3700 1 --2- 172.21.15.116:0/3078218397 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79d40c45a0 0x7f79d4151e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.552+0000 7f79d8ec3700 1 -- 172.21.15.116:0/3078218397 >> 172.21.15.116:0/3078218397 conn(0x7f79d4062520 msgr2=0x7f79d40a3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:06.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.552+0000 7f79d8ec3700 1 -- 172.21.15.116:0/3078218397 shutdown_connections 2020-11-20T12:17:06.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.552+0000 7f79d8ec3700 1 -- 172.21.15.116:0/3078218397 wait complete. 2020-11-20T12:17:06.574 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:16:55.781578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:16:56.900539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:17:06.576 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:06.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.756+0000 7f00208b2700 1 Processor -- start 2020-11-20T12:17:06.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.756+0000 7f00208b2700 1 -- start start 2020-11-20T12:17:06.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f00208b2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f001c149bb0 0x7f001c0a7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:06.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f00208b2700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f001c0a7800 0x7f001c060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:06.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f00208b2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001c149f90 con 0x7f001c060660 2020-11-20T12:17:06.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f00208b2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001c060ab0 con 0x7f001c149bb0 2020-11-20T12:17:06.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f00208b2700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001c0a7e20 con 0x7f001c0a7800 2020-11-20T12:17:06.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f001a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f001c149bb0 0x7f001c0a7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:06.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f001a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f001c149bb0 0x7f001c0a7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44806/0 (socket says 172.21.15.116:44806) 2020-11-20T12:17:06.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f001a59c700 1 -- 172.21.15.116:0/111776414 learned_addr learned my addr 172.21.15.116:0/111776414 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:06.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f0019d9b700 1 --2- 172.21.15.116:0/111776414 >> v2:172.21.15.173:3300/0 conn(0x7f001c0a7800 0x7f001c060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:06.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f001a59c700 1 -- 172.21.15.116:0/111776414 >> v2:172.21.15.173:3300/0 conn(0x7f001c0a7800 msgr2=0x7f001c060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:06.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f001a59c700 1 --2- 172.21.15.116:0/111776414 >> v2:172.21.15.173:3300/0 conn(0x7f001c0a7800 0x7f001c060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f001a59c700 1 -- 172.21.15.116:0/111776414 >> v1:172.21.15.173:6789/0 conn(0x7f001c060660 legacy=0x7f001c1530f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f001a59c700 1 -- 172.21.15.116:0/111776414 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c14fc30 con 0x7f001c149bb0 2020-11-20T12:17:06.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.760+0000 7f001a59c700 1 --2- 172.21.15.116:0/111776414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f001c149bb0 0x7f001c0a7310 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f001c064730 tx=0x7f000c006eb0).ready entity=mon.0 client_cookie=e516b39a567b75a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:06.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.764+0000 7f001959a700 1 -- 172.21.15.116:0/111776414 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c0098d0 con 0x7f001c149bb0 2020-11-20T12:17:06.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.764+0000 7f001959a700 1 -- 172.21.15.116:0/111776414 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000c009a30 con 0x7f001c149bb0 2020-11-20T12:17:06.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.764+0000 7f001959a700 1 -- 172.21.15.116:0/111776414 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c009db0 con 0x7f001c149bb0 2020-11-20T12:17:06.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.764+0000 7f00208b2700 1 -- 172.21.15.116:0/111776414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f001c149bb0 msgr2=0x7f001c0a7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.764+0000 7f00208b2700 1 --2- 172.21.15.116:0/111776414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f001c149bb0 0x7f001c0a7310 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f001c064730 tx=0x7f000c006eb0).stop 2020-11-20T12:17:06.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.764+0000 7f00208b2700 1 -- 172.21.15.116:0/111776414 shutdown_connections 2020-11-20T12:17:06.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.764+0000 7f00208b2700 1 --2- 172.21.15.116:0/111776414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f001c149bb0 0x7f001c0a7310 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.764+0000 7f00208b2700 1 --2- 172.21.15.116:0/111776414 >> v2:172.21.15.173:3300/0 conn(0x7f001c0a7800 0x7f001c060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.764+0000 7f00208b2700 1 -- 172.21.15.116:0/111776414 >> 172.21.15.116:0/111776414 conn(0x7f001c0c52e0 msgr2=0x7f001c0c56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:06.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.764+0000 7f00208b2700 1 -- 172.21.15.116:0/111776414 shutdown_connections 2020-11-20T12:17:06.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.764+0000 7f00208b2700 1 -- 172.21.15.116:0/111776414 wait complete. 2020-11-20T12:17:06.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f00208b2700 1 Processor -- start 2020-11-20T12:17:06.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f00208b2700 1 -- start start 2020-11-20T12:17:06.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f00208b2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f001c060660 0x7f001c161e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:06.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f00208b2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f001c0a7800 0x7f001c0c1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:06.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f00208b2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f001c149bb0 0x7f001c1649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:06.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f00208b2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c165590 con 0x7f001c060660 2020-11-20T12:17:06.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f00208b2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c165830 con 0x7f001c0a7800 2020-11-20T12:17:06.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f00208b2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001c1616e0 con 0x7f001c149bb0 2020-11-20T12:17:06.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f001a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f001c060660 0x7f001c161e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:06.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f001a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f001c060660 0x7f001c161e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44812/0 (socket says 172.21.15.116:44812) 2020-11-20T12:17:06.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f001a59c700 1 -- 172.21.15.116:0/3250919858 learned_addr learned my addr 172.21.15.116:0/3250919858 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:06.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f0019d9b700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f001c0a7800 0x7f001c0c1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:06.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.768+0000 7f001ad9d700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f001c149bb0 0x7f001c1649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:06.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f001a59c700 1 -- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f001c149bb0 msgr2=0x7f001c1649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f001a59c700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f001c149bb0 0x7f001c1649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f001a59c700 1 -- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f001c0a7800 msgr2=0x7f001c0c1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f001a59c700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f001c0a7800 0x7f001c0c1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f001a59c700 1 -- 172.21.15.116:0/3250919858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c161980 con 0x7f001c060660 2020-11-20T12:17:06.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f001a59c700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f001c060660 0x7f001c161e20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f001c0c1ba0 tx=0x7f000c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:06.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f000a7fc700 1 -- 172.21.15.116:0/3250919858 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c011c80 con 0x7f001c060660 2020-11-20T12:17:06.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f000a7fc700 1 -- 172.21.15.116:0/3250919858 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000c011de0 con 0x7f001c060660 2020-11-20T12:17:06.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f00208b2700 1 -- 172.21.15.116:0/3250919858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c161410 con 0x7f001c060660 2020-11-20T12:17:06.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f000a7fc700 1 -- 172.21.15.116:0/3250919858 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000c01e670 con 0x7f001c060660 2020-11-20T12:17:06.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f00208b2700 1 -- 172.21.15.116:0/3250919858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c0c2390 con 0x7f001c060660 2020-11-20T12:17:06.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.772+0000 7f00208b2700 1 -- 172.21.15.116:0/3250919858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c0c2640 con 0x7f001c060660 2020-11-20T12:17:06.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.776+0000 7f000a7fc700 1 -- 172.21.15.116:0/3250919858 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f000c025030 con 0x7f001c060660 2020-11-20T12:17:06.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.776+0000 7f000a7fc700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efff00557c0 0x7efff0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:06.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.776+0000 7f000a7fc700 1 -- 172.21.15.116:0/3250919858 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f000c06fd00 con 0x7f001c060660 2020-11-20T12:17:06.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.776+0000 7f0019d9b700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efff00557c0 0x7efff0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:06.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.776+0000 7f0019d9b700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efff00557c0 0x7efff0057c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f001c0c0350 tx=0x7f0010008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:06.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.780+0000 7f000a7fc700 1 -- 172.21.15.116:0/3250919858 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f000c073030 con 0x7f001c060660 2020-11-20T12:17:06.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.940+0000 7f00208b2700 1 -- 172.21.15.116:0/3250919858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f001c05ca30 con 0x7f001c060660 2020-11-20T12:17:06.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.940+0000 7f000a7fc700 1 -- 172.21.15.116:0/3250919858 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v34) v1 ==== 110+0+10 (secure 0 0 0) 0x7f000c023070 con 0x7f001c060660 2020-11-20T12:17:06.942 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:06.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 -- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efff00557c0 msgr2=0x7efff0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efff00557c0 0x7efff0057c80 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f001c0c0350 tx=0x7f0010008040).stop 2020-11-20T12:17:06.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 -- 172.21.15.116:0/3250919858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f001c060660 msgr2=0x7f001c161e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:06.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f001c060660 0x7f001c161e20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f001c0c1ba0 tx=0x7f000c015000).stop 2020-11-20T12:17:06.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 -- 172.21.15.116:0/3250919858 shutdown_connections 2020-11-20T12:17:06.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efff00557c0 0x7efff0057c80 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f001c149bb0 0x7f001c1649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f001c060660 0x7f001c161e20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 --2- 172.21.15.116:0/3250919858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f001c0a7800 0x7f001c0c1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:06.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 -- 172.21.15.116:0/3250919858 >> 172.21.15.116:0/3250919858 conn(0x7f001c0c52e0 msgr2=0x7f001c0a7000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:06.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 -- 172.21.15.116:0/3250919858 shutdown_connections 2020-11-20T12:17:06.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:06.944+0000 7f00208b2700 1 -- 172.21.15.116:0/3250919858 wait complete. 2020-11-20T12:17:06.969 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:07.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.144+0000 7f4c948aa700 1 Processor -- start 2020-11-20T12:17:07.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.144+0000 7f4c948aa700 1 -- start start 2020-11-20T12:17:07.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.144+0000 7f4c948aa700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4c9013b110 0x7f4c90151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c948aa700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c90151fa0 0x7f4c90061250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c948aa700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c90152a80 con 0x7f4c9013ad30 2020-11-20T12:17:07.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c948aa700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c90152540 con 0x7f4c9013b110 2020-11-20T12:17:07.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c948aa700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c901527e0 con 0x7f4c90151fa0 2020-11-20T12:17:07.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c8ed9d700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f4c9013ad30 0x7f4c900a7fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:50484/0 (socket says 172.21.15.116:50484) 2020-11-20T12:17:07.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c87fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4c9013b110 0x7f4c90151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c8ed9d700 1 -- 172.21.15.116:0/1209219482 learned_addr learned my addr 172.21.15.116:0/1209219482 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:07.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c8f59e700 1 --2- 172.21.15.116:0/1209219482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c90151fa0 0x7f4c90061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c87fff700 1 -- 172.21.15.116:0/1209219482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c90151fa0 msgr2=0x7f4c90061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:07.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c87fff700 1 --2- 172.21.15.116:0/1209219482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c90151fa0 0x7f4c90061250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c87fff700 1 -- 172.21.15.116:0/1209219482 >> v1:172.21.15.173:6789/0 conn(0x7f4c9013ad30 legacy=0x7f4c900a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c87fff700 1 -- 172.21.15.116:0/1209219482 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900ac910 con 0x7f4c9013b110 2020-11-20T12:17:07.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.148+0000 7f4c87fff700 1 --2- 172.21.15.116:0/1209219482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9013b110 0x7f4c90151ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f4c7800b9a0 tx=0x7f4c78006df0).ready entity=mon.0 client_cookie=e24a89e58e2297a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:07.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.152+0000 7f4c8e59c700 1 -- 172.21.15.116:0/1209219482 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78009840 con 0x7f4c9013b110 2020-11-20T12:17:07.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.152+0000 7f4c8e59c700 1 -- 172.21.15.116:0/1209219482 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c780099a0 con 0x7f4c9013b110 2020-11-20T12:17:07.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.152+0000 7f4c8e59c700 1 -- 172.21.15.116:0/1209219482 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78009d20 con 0x7f4c9013b110 2020-11-20T12:17:07.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.152+0000 7f4c948aa700 1 -- 172.21.15.116:0/1209219482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9013b110 msgr2=0x7f4c90151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.152+0000 7f4c948aa700 1 --2- 172.21.15.116:0/1209219482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9013b110 0x7f4c90151ab0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f4c7800b9a0 tx=0x7f4c78006df0).stop 2020-11-20T12:17:07.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.152+0000 7f4c948aa700 1 -- 172.21.15.116:0/1209219482 shutdown_connections 2020-11-20T12:17:07.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.152+0000 7f4c948aa700 1 --2- 172.21.15.116:0/1209219482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c90151fa0 0x7f4c90061250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.152+0000 7f4c948aa700 1 --2- 172.21.15.116:0/1209219482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9013b110 0x7f4c90151ab0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.152+0000 7f4c948aa700 1 -- 172.21.15.116:0/1209219482 >> 172.21.15.116:0/1209219482 conn(0x7f4c900678a0 msgr2=0x7f4c90067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:07.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.152+0000 7f4c948aa700 1 -- 172.21.15.116:0/1209219482 shutdown_connections 2020-11-20T12:17:07.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.152+0000 7f4c948aa700 1 -- 172.21.15.116:0/1209219482 wait complete. 2020-11-20T12:17:07.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c948aa700 1 Processor -- start 2020-11-20T12:17:07.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c948aa700 1 -- start start 2020-11-20T12:17:07.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c948aa700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9013ad30 0x7f4c900a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c948aa700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c90151fa0 0x7f4c900a3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c948aa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c900a92c0 0x7f4c90136760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c948aa700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90156d20 con 0x7f4c9013ad30 2020-11-20T12:17:07.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c948aa700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90155ad0 con 0x7f4c900a92c0 2020-11-20T12:17:07.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c948aa700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90155d70 con 0x7f4c90151fa0 2020-11-20T12:17:07.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c8ed9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9013ad30 0x7f4c900a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c8ed9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9013ad30 0x7f4c900a8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44826/0 (socket says 172.21.15.116:44826) 2020-11-20T12:17:07.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c8ed9d700 1 -- 172.21.15.116:0/118560745 learned_addr learned my addr 172.21.15.116:0/118560745 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:07.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c87fff700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c90151fa0 0x7f4c900a3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c8ed9d700 1 -- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c90151fa0 msgr2=0x7f4c900a3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c8ed9d700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c90151fa0 0x7f4c900a3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c8ed9d700 1 -- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c900a92c0 msgr2=0x7f4c90136760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c8ed9d700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c900a92c0 0x7f4c90136760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c8ed9d700 1 -- 172.21.15.116:0/118560745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900a3340 con 0x7f4c9013ad30 2020-11-20T12:17:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c87fff700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c90151fa0 0x7f4c900a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c8ed9d700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9013ad30 0x7f4c900a8dd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f4c8000c710 tx=0x7f4c80007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.156+0000 7f4c86ffd700 1 -- 172.21.15.116:0/118560745 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80007320 con 0x7f4c9013ad30 2020-11-20T12:17:07.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.160+0000 7f4c86ffd700 1 -- 172.21.15.116:0/118560745 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c80007480 con 0x7f4c9013ad30 2020-11-20T12:17:07.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.160+0000 7f4c86ffd700 1 -- 172.21.15.116:0/118560745 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c80005800 con 0x7f4c9013ad30 2020-11-20T12:17:07.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.160+0000 7f4c948aa700 1 -- 172.21.15.116:0/118560745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c900a35e0 con 0x7f4c9013ad30 2020-11-20T12:17:07.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.160+0000 7f4c948aa700 1 -- 172.21.15.116:0/118560745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c900a3f70 con 0x7f4c9013ad30 2020-11-20T12:17:07.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.160+0000 7f4c948aa700 1 -- 172.21.15.116:0/118560745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c900a4220 con 0x7f4c9013ad30 2020-11-20T12:17:07.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.160+0000 7f4c86ffd700 1 -- 172.21.15.116:0/118560745 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4c80010030 con 0x7f4c9013ad30 2020-11-20T12:17:07.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.160+0000 7f4c86ffd700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c6c0557c0 0x7f4c6c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.160+0000 7f4c86ffd700 1 -- 172.21.15.116:0/118560745 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c8006e380 con 0x7f4c9013ad30 2020-11-20T12:17:07.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.164+0000 7f4c87fff700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c6c0557c0 0x7f4c6c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.164+0000 7f4c87fff700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c6c0557c0 0x7f4c6c057c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f4c9013b550 tx=0x7f4c78015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:07.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.168+0000 7f4c86ffd700 1 -- 172.21.15.116:0/118560745 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4c80044220 con 0x7f4c9013ad30 2020-11-20T12:17:07.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.328+0000 7f4c948aa700 1 -- 172.21.15.116:0/118560745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4c90044f60 con 0x7f4c9013ad30 2020-11-20T12:17:07.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.332+0000 7f4c86ffd700 1 -- 172.21.15.116:0/118560745 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4c80047840 con 0x7f4c9013ad30 2020-11-20T12:17:07.331 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 -- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c6c0557c0 msgr2=0x7f4c6c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c6c0557c0 0x7f4c6c057c80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f4c9013b550 tx=0x7f4c78015000).stop 2020-11-20T12:17:07.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 -- 172.21.15.116:0/118560745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9013ad30 msgr2=0x7f4c900a8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9013ad30 0x7f4c900a8dd0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f4c8000c710 tx=0x7f4c80007c40).stop 2020-11-20T12:17:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 -- 172.21.15.116:0/118560745 shutdown_connections 2020-11-20T12:17:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c6c0557c0 0x7f4c6c057c80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c900a92c0 0x7f4c90136760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:07.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c9013ad30 0x7f4c900a8dd0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 --2- 172.21.15.116:0/118560745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c90151fa0 0x7f4c900a3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 -- 172.21.15.116:0/118560745 >> 172.21.15.116:0/118560745 conn(0x7f4c900678a0 msgr2=0x7f4c90138930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:07.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 -- 172.21.15.116:0/118560745 shutdown_connections 2020-11-20T12:17:07.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.336+0000 7f4c948aa700 1 -- 172.21.15.116:0/118560745 wait complete. 2020-11-20T12:17:07.362 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:17:07.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.552+0000 7f4b0eb03700 1 Processor -- start 2020-11-20T12:17:07.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.556+0000 7f4b0eb03700 1 -- start start 2020-11-20T12:17:07.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0eb03700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b080a4610 0x7f4b08067a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0eb03700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4b08150f80 0x7f4b08151410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0eb03700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b080602f0 con 0x7f4b080a4230 2020-11-20T12:17:07.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0eb03700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b08060590 con 0x7f4b08150f80 2020-11-20T12:17:07.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0eb03700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08060830 con 0x7f4b080a4610 2020-11-20T12:17:07.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0db01700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4b080a4230 0x7f4b08067360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54268/0 (socket says 172.21.15.116:54268) 2020-11-20T12:17:07.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0db01700 1 -- 172.21.15.116:0/3186595685 learned_addr learned my addr 172.21.15.116:0/3186595685 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:07.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0e302700 1 --2- 172.21.15.116:0/3186595685 >> v2:172.21.15.116:3300/0 conn(0x7f4b08150f80 0x7f4b08151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0d300700 1 --2- 172.21.15.116:0/3186595685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b080a4610 0x7f4b08067a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0caff700 1 -- 172.21.15.116:0/3186595685 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863021391 0 0) 0x7f4b080602f0 con 0x7f4b080a4230 2020-11-20T12:17:07.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0caff700 1 -- 172.21.15.116:0/3186595685 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4af0003240 con 0x7f4b080a4230 2020-11-20T12:17:07.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0e302700 1 -- 172.21.15.116:0/3186595685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b080a4610 msgr2=0x7f4b08067a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:07.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0e302700 1 --2- 172.21.15.116:0/3186595685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b080a4610 0x7f4b08067a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0e302700 1 -- 172.21.15.116:0/3186595685 >> v1:172.21.15.116:6789/0 conn(0x7f4b080a4230 legacy=0x7f4b08067360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0e302700 1 -- 172.21.15.116:0/3186595685 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08060ad0 con 0x7f4b08150f80 2020-11-20T12:17:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0e302700 1 --2- 172.21.15.116:0/3186595685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b08150f80 0x7f4b08151410 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f4b00009cc0 tx=0x7f4b000091d0).ready entity=mon.0 client_cookie=6e8f91596ce28a06 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0caff700 1 -- 172.21.15.116:0/3186595685 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0000ba70 con 0x7f4b08150f80 2020-11-20T12:17:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0caff700 1 -- 172.21.15.116:0/3186595685 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0000bbd0 con 0x7f4b08150f80 2020-11-20T12:17:07.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.560+0000 7f4b0caff700 1 -- 172.21.15.116:0/3186595685 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b00011400 con 0x7f4b08150f80 2020-11-20T12:17:07.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.564+0000 7f4b0eb03700 1 -- 172.21.15.116:0/3186595685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b08150f80 msgr2=0x7f4b08151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.564+0000 7f4b0eb03700 1 --2- 172.21.15.116:0/3186595685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b08150f80 0x7f4b08151410 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f4b00009cc0 tx=0x7f4b000091d0).stop 2020-11-20T12:17:07.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.564+0000 7f4b0eb03700 1 -- 172.21.15.116:0/3186595685 shutdown_connections 2020-11-20T12:17:07.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.564+0000 7f4b0eb03700 1 --2- 172.21.15.116:0/3186595685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b08150f80 0x7f4b08151410 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.564+0000 7f4b0eb03700 1 --2- 172.21.15.116:0/3186595685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b080a4610 0x7f4b08067a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.564+0000 7f4b0eb03700 1 -- 172.21.15.116:0/3186595685 >> 172.21.15.116:0/3186595685 conn(0x7f4b08061d50 msgr2=0x7f4b080c50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:07.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.564+0000 7f4b0eb03700 1 -- 172.21.15.116:0/3186595685 shutdown_connections 2020-11-20T12:17:07.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.564+0000 7f4b0eb03700 1 -- 172.21.15.116:0/3186595685 wait complete. 2020-11-20T12:17:07.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.564+0000 7f4b0eb03700 1 Processor -- start 2020-11-20T12:17:07.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0eb03700 1 -- start start 2020-11-20T12:17:07.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0eb03700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b080a4230 0x7f4b08161310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0eb03700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b080a4610 0x7f4b080a3420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0eb03700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b08150f80 0x7f4b08163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0eb03700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b081609a0 con 0x7f4b080a4610 2020-11-20T12:17:07.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0eb03700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08164a10 con 0x7f4b080a4230 2020-11-20T12:17:07.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0eb03700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b08164cb0 con 0x7f4b08150f80 2020-11-20T12:17:07.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0d300700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b080a4610 0x7f4b080a3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0d300700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b080a4610 0x7f4b080a3420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44842/0 (socket says 172.21.15.116:44842) 2020-11-20T12:17:07.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0d300700 1 -- 172.21.15.116:0/1484454734 learned_addr learned my addr 172.21.15.116:0/1484454734 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:07.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0db01700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b080a4230 0x7f4b08161310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0e302700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b08150f80 0x7f4b08163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0d300700 1 -- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b08150f80 msgr2=0x7f4b08163fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0d300700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b08150f80 0x7f4b08163fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0d300700 1 -- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b080a4230 msgr2=0x7f4b08161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0d300700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b080a4230 0x7f4b08161310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0d300700 1 -- 172.21.15.116:0/1484454734 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b08160bd0 con 0x7f4b080a4610 2020-11-20T12:17:07.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0db01700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b080a4230 0x7f4b08161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:07.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.568+0000 7f4b0d300700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b080a4610 0x7f4b080a3420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f4af800eba0 tx=0x7f4af8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:07.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.572+0000 7f4af5ffb700 1 -- 172.21.15.116:0/1484454734 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af800ca60 con 0x7f4b080a4610 2020-11-20T12:17:07.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.572+0000 7f4af5ffb700 1 -- 172.21.15.116:0/1484454734 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af800cbc0 con 0x7f4b080a4610 2020-11-20T12:17:07.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.572+0000 7f4b0eb03700 1 -- 172.21.15.116:0/1484454734 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b08160e70 con 0x7f4b080a4610 2020-11-20T12:17:07.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.572+0000 7f4af5ffb700 1 -- 172.21.15.116:0/1484454734 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af80123f0 con 0x7f4b080a4610 2020-11-20T12:17:07.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.572+0000 7f4b0eb03700 1 -- 172.21.15.116:0/1484454734 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b08161880 con 0x7f4b080a4610 2020-11-20T12:17:07.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.572+0000 7f4b0eb03700 1 -- 172.21.15.116:0/1484454734 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b08164e70 con 0x7f4b080a4610 2020-11-20T12:17:07.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.572+0000 7f4af5ffb700 1 -- 172.21.15.116:0/1484454734 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4af80127a0 con 0x7f4b080a4610 2020-11-20T12:17:07.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.576+0000 7f4af5ffb700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ae00557f0 0x7f4ae0057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.576+0000 7f4af5ffb700 1 -- 172.21.15.116:0/1484454734 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4af806eb50 con 0x7f4b080a4610 2020-11-20T12:17:07.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.576+0000 7f4b0db01700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ae00557f0 0x7f4ae0057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.576+0000 7f4b0db01700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ae00557f0 0x7f4ae0057cb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f4b040055a0 tx=0x7f4b04009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:07.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.580+0000 7f4af5ffb700 1 -- 172.21.15.116:0/1484454734 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4af80449f0 con 0x7f4b080a4610 2020-11-20T12:17:07.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.740+0000 7f4b0eb03700 1 -- 172.21.15.116:0/1484454734 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4b08161000 con 0x7f4b080a4610 2020-11-20T12:17:07.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4af5ffb700 1 -- 172.21.15.116:0/1484454734 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4af8048010 con 0x7f4b080a4610 2020-11-20T12:17:07.744 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:07.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 -- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ae00557f0 msgr2=0x7f4ae0057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ae00557f0 0x7f4ae0057cb0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f4b040055a0 tx=0x7f4b04009040).stop 2020-11-20T12:17:07.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 -- 172.21.15.116:0/1484454734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b080a4610 msgr2=0x7f4b080a3420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b080a4610 0x7f4b080a3420 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f4af800eba0 tx=0x7f4af8006df0).stop 2020-11-20T12:17:07.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 -- 172.21.15.116:0/1484454734 shutdown_connections 2020-11-20T12:17:07.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ae00557f0 0x7f4ae0057cb0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4b08150f80 0x7f4b08163fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4b080a4230 0x7f4b08161310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 --2- 172.21.15.116:0/1484454734 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4b080a4610 0x7f4b080a3420 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 -- 172.21.15.116:0/1484454734 >> 172.21.15.116:0/1484454734 conn(0x7f4b08061d50 msgr2=0x7f4b080ba420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:07.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 -- 172.21.15.116:0/1484454734 shutdown_connections 2020-11-20T12:17:07.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.744+0000 7f4b0eb03700 1 -- 172.21.15.116:0/1484454734 wait complete. 2020-11-20T12:17:07.769 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:17:07.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.960+0000 7fd43b0e0700 1 Processor -- start 2020-11-20T12:17:07.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.960+0000 7fd43b0e0700 1 -- start start 2020-11-20T12:17:07.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd43b0e0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd43413bec0 0x7fd43413c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd43b0e0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd4340bda60 0x7fd4340bdee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd43b0e0700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4340a7d50 con 0x7fd4340c85a0 2020-11-20T12:17:07.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd43b0e0700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43415a330 con 0x7fd4340bda60 2020-11-20T12:17:07.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd43b0e0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43415a5d0 con 0x7fd43413bec0 2020-11-20T12:17:07.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd42bfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd4340bda60 0x7fd4340bdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd42bfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd4340bda60 0x7fd4340bdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44850/0 (socket says 172.21.15.116:44850) 2020-11-20T12:17:07.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd42bfff700 1 -- 172.21.15.116:0/2425440104 learned_addr learned my addr 172.21.15.116:0/2425440104 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:07.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd438e7c700 1 --2- 172.21.15.116:0/2425440104 >> v2:172.21.15.173:3300/0 conn(0x7fd43413bec0 0x7fd43413c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd42bfff700 1 -- 172.21.15.116:0/2425440104 >> v2:172.21.15.173:3300/0 conn(0x7fd43413bec0 msgr2=0x7fd43413c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:07.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd42b7fe700 1 -- 172.21.15.116:0/2425440104 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3262767755 0 0) 0x7fd4340a7d50 con 0x7fd4340c85a0 2020-11-20T12:17:07.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd42bfff700 1 --2- 172.21.15.116:0/2425440104 >> v2:172.21.15.173:3300/0 conn(0x7fd43413bec0 0x7fd43413c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd42bfff700 1 -- 172.21.15.116:0/2425440104 >> v1:172.21.15.173:6789/0 conn(0x7fd4340c85a0 legacy=0x7fd4340c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.964+0000 7fd42bfff700 1 -- 172.21.15.116:0/2425440104 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4340c4e70 con 0x7fd4340bda60 2020-11-20T12:17:07.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd42bfff700 1 --2- 172.21.15.116:0/2425440104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4340bda60 0x7fd4340bdee0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fd434081540 tx=0x7fd424006eb0).ready entity=mon.0 client_cookie=d402d9d93e50af55 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:07.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd42b7fe700 1 -- 172.21.15.116:0/2425440104 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424009810 con 0x7fd4340bda60 2020-11-20T12:17:07.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd42b7fe700 1 -- 172.21.15.116:0/2425440104 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd424009970 con 0x7fd4340bda60 2020-11-20T12:17:07.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd42b7fe700 1 -- 172.21.15.116:0/2425440104 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd424009cf0 con 0x7fd4340bda60 2020-11-20T12:17:07.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd43b0e0700 1 -- 172.21.15.116:0/2425440104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4340bda60 msgr2=0x7fd4340bdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd43b0e0700 1 --2- 172.21.15.116:0/2425440104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4340bda60 0x7fd4340bdee0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fd434081540 tx=0x7fd424006eb0).stop 2020-11-20T12:17:07.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd43b0e0700 1 -- 172.21.15.116:0/2425440104 shutdown_connections 2020-11-20T12:17:07.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd43b0e0700 1 --2- 172.21.15.116:0/2425440104 >> v2:172.21.15.173:3300/0 conn(0x7fd43413bec0 0x7fd43413c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd43b0e0700 1 --2- 172.21.15.116:0/2425440104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4340bda60 0x7fd4340bdee0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd43b0e0700 1 -- 172.21.15.116:0/2425440104 >> 172.21.15.116:0/2425440104 conn(0x7fd434061580 msgr2=0x7fd434061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:07.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd43b0e0700 1 -- 172.21.15.116:0/2425440104 shutdown_connections 2020-11-20T12:17:07.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.968+0000 7fd43b0e0700 1 -- 172.21.15.116:0/2425440104 wait complete. 2020-11-20T12:17:07.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.972+0000 7fd43b0e0700 1 Processor -- start 2020-11-20T12:17:07.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.972+0000 7fd43b0e0700 1 -- start start 2020-11-20T12:17:07.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.972+0000 7fd43b0e0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4340c85a0 0x7fd43413a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.972+0000 7fd43b0e0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd43413bec0 0x7fd4340a8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.972+0000 7fd43b0e0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4340bf690 0x7fd4340a8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.972+0000 7fd43b0e0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4340a9a50 con 0x7fd4340c85a0 2020-11-20T12:17:07.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.972+0000 7fd43b0e0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4340a91b0 con 0x7fd43413bec0 2020-11-20T12:17:07.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.972+0000 7fd43b0e0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4340a9450 con 0x7fd4340bf690 2020-11-20T12:17:07.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd438e7c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4340c85a0 0x7fd43413a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd438e7c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4340c85a0 0x7fd43413a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44854/0 (socket says 172.21.15.116:44854) 2020-11-20T12:17:07.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd438e7c700 1 -- 172.21.15.116:0/1485246255 learned_addr learned my addr 172.21.15.116:0/1485246255 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:07.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd42bfff700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd43413bec0 0x7fd4340a8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd43967d700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4340bf690 0x7fd4340a8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd438e7c700 1 -- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4340bf690 msgr2=0x7fd4340a8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd438e7c700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4340bf690 0x7fd4340a8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd438e7c700 1 -- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd43413bec0 msgr2=0x7fd4340a8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:07.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd438e7c700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd43413bec0 0x7fd4340a8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:07.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd438e7c700 1 -- 172.21.15.116:0/1485246255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4340a96f0 con 0x7fd4340c85a0 2020-11-20T12:17:07.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd438e7c700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4340c85a0 0x7fd43413a310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fd43000ebd0 tx=0x7fd430006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:07.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd428ff9700 1 -- 172.21.15.116:0/1485246255 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43000b9a0 con 0x7fd4340c85a0 2020-11-20T12:17:07.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd428ff9700 1 -- 172.21.15.116:0/1485246255 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd43000bb00 con 0x7fd4340c85a0 2020-11-20T12:17:07.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd428ff9700 1 -- 172.21.15.116:0/1485246255 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43000be80 con 0x7fd4340c85a0 2020-11-20T12:17:07.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd43b0e0700 1 -- 172.21.15.116:0/1485246255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4340a9ca0 con 0x7fd4340c85a0 2020-11-20T12:17:07.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd43b0e0700 1 -- 172.21.15.116:0/1485246255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43413a800 con 0x7fd4340c85a0 2020-11-20T12:17:07.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.976+0000 7fd43b0e0700 1 -- 172.21.15.116:0/1485246255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43405ca30 con 0x7fd4340c85a0 2020-11-20T12:17:07.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.980+0000 7fd428ff9700 1 -- 172.21.15.116:0/1485246255 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd430012600 con 0x7fd4340c85a0 2020-11-20T12:17:07.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.980+0000 7fd428ff9700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd40c0557c0 0x7fd40c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:07.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.980+0000 7fd428ff9700 1 -- 172.21.15.116:0/1485246255 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd43006eb90 con 0x7fd4340c85a0 2020-11-20T12:17:07.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.980+0000 7fd42bfff700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd40c0557c0 0x7fd40c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:07.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.980+0000 7fd42bfff700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd40c0557c0 0x7fd40c057c80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fd4340625a0 tx=0x7fd424015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:07.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:07.984+0000 7fd428ff9700 1 -- 172.21.15.116:0/1485246255 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd4300448c0 con 0x7fd4340c85a0 2020-11-20T12:17:08.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd43b0e0700 1 -- 172.21.15.116:0/1485246255 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd434060620 con 0x7fd4340c85a0 2020-11-20T12:17:08.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd428ff9700 1 -- 172.21.15.116:0/1485246255 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd430047ee0 con 0x7fd4340c85a0 2020-11-20T12:17:08.148 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:08.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd43b0e0700 1 -- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd40c0557c0 msgr2=0x7fd40c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:08.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd43b0e0700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd40c0557c0 0x7fd40c057c80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fd4340625a0 tx=0x7fd424015000).stop 2020-11-20T12:17:08.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd43b0e0700 1 -- 172.21.15.116:0/1485246255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4340c85a0 msgr2=0x7fd43413a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:08.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd43b0e0700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4340c85a0 0x7fd43413a310 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7fd43000ebd0 tx=0x7fd430006df0).stop 2020-11-20T12:17:08.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd43b0e0700 1 -- 172.21.15.116:0/1485246255 shutdown_connections 2020-11-20T12:17:08.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd43b0e0700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd40c0557c0 0x7fd40c057c80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd43b0e0700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd4340bf690 0x7fd4340a8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd43b0e0700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd4340c85a0 0x7fd43413a310 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd43b0e0700 1 --2- 172.21.15.116:0/1485246255 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd43413bec0 0x7fd4340a8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.148+0000 7fd43b0e0700 1 -- 172.21.15.116:0/1485246255 >> 172.21.15.116:0/1485246255 conn(0x7fd434061580 msgr2=0x7fd4340c3180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:08.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.152+0000 7fd43b0e0700 1 -- 172.21.15.116:0/1485246255 shutdown_connections 2020-11-20T12:17:08.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.152+0000 7fd43b0e0700 1 -- 172.21.15.116:0/1485246255 wait complete. 2020-11-20T12:17:08.176 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:17:08.177 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:08.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.356+0000 7f5c72a34700 1 Processor -- start 2020-11-20T12:17:08.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.356+0000 7f5c72a34700 1 -- start start 2020-11-20T12:17:08.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72a34700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c6c13aee0 0x7f5c6c152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:08.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72a34700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5c6c0612f0 0x7f5c6c061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:08.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72a34700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6c152a30 con 0x7f5c6c0aa790 2020-11-20T12:17:08.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72a34700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c0adce0 con 0x7f5c6c13aee0 2020-11-20T12:17:08.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72a34700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c6c061c50 con 0x7f5c6c0612f0 2020-11-20T12:17:08.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c71a32700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c6c13aee0 0x7f5c6c152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:08.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c71a32700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5c6c13aee0 0x7f5c6c152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44862/0 (socket says 172.21.15.116:44862) 2020-11-20T12:17:08.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c71a32700 1 -- 172.21.15.116:0/503273812 learned_addr learned my addr 172.21.15.116:0/503273812 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:08.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72233700 1 --2- 172.21.15.116:0/503273812 >> v2:172.21.15.173:3300/0 conn(0x7f5c6c0612f0 0x7f5c6c061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:08.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c71a32700 1 -- 172.21.15.116:0/503273812 >> v2:172.21.15.173:3300/0 conn(0x7f5c6c0612f0 msgr2=0x7f5c6c061760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:08.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c71a32700 1 --2- 172.21.15.116:0/503273812 >> v2:172.21.15.173:3300/0 conn(0x7f5c6c0612f0 0x7f5c6c061760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c71a32700 1 -- 172.21.15.116:0/503273812 >> v1:172.21.15.173:6789/0 conn(0x7f5c6c0aa790 legacy=0x7f5c6c0ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:08.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c71a32700 1 -- 172.21.15.116:0/503273812 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c0aac10 con 0x7f5c6c13aee0 2020-11-20T12:17:08.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72233700 1 --2- 172.21.15.116:0/503273812 >> v2:172.21.15.173:3300/0 conn(0x7f5c6c0612f0 0x7f5c6c061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:08.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c71a32700 1 --2- 172.21.15.116:0/503273812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c6c13aee0 0x7f5c6c152540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f5c6c081760 tx=0x7f5c68006eb0).ready entity=mon.0 client_cookie=2c0b9f6cadf14b57 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:08.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c70a30700 1 -- 172.21.15.116:0/503273812 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68009890 con 0x7f5c6c13aee0 2020-11-20T12:17:08.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c70a30700 1 -- 172.21.15.116:0/503273812 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c680099f0 con 0x7f5c6c13aee0 2020-11-20T12:17:08.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c70a30700 1 -- 172.21.15.116:0/503273812 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68009d70 con 0x7f5c6c13aee0 2020-11-20T12:17:08.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72a34700 1 -- 172.21.15.116:0/503273812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c6c13aee0 msgr2=0x7f5c6c152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:08.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72a34700 1 --2- 172.21.15.116:0/503273812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c6c13aee0 0x7f5c6c152540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f5c6c081760 tx=0x7f5c68006eb0).stop 2020-11-20T12:17:08.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72a34700 1 -- 172.21.15.116:0/503273812 shutdown_connections 2020-11-20T12:17:08.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72a34700 1 --2- 172.21.15.116:0/503273812 >> v2:172.21.15.173:3300/0 conn(0x7f5c6c0612f0 0x7f5c6c061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72a34700 1 --2- 172.21.15.116:0/503273812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c6c13aee0 0x7f5c6c152540 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.360+0000 7f5c72a34700 1 -- 172.21.15.116:0/503273812 >> 172.21.15.116:0/503273812 conn(0x7f5c6c0678a0 msgr2=0x7f5c6c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:08.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.364+0000 7f5c72a34700 1 -- 172.21.15.116:0/503273812 shutdown_connections 2020-11-20T12:17:08.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.364+0000 7f5c72a34700 1 -- 172.21.15.116:0/503273812 wait complete. 2020-11-20T12:17:08.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.364+0000 7f5c72a34700 1 Processor -- start 2020-11-20T12:17:08.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.364+0000 7f5c72a34700 1 -- start start 2020-11-20T12:17:08.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c72a34700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c6c0612f0 0x7f5c6c156560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:08.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c72a34700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c6c0aa790 0x7f5c6c0a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:08.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c72a34700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c6c13aee0 0x7f5c6c150a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:08.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c72a34700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c09f600 con 0x7f5c6c0612f0 2020-11-20T12:17:08.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c72a34700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c09f8a0 con 0x7f5c6c0aa790 2020-11-20T12:17:08.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c72a34700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c09fb40 con 0x7f5c6c13aee0 2020-11-20T12:17:08.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c71a32700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c6c0612f0 0x7f5c6c156560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:08.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c71a32700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c6c0612f0 0x7f5c6c156560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44868/0 (socket says 172.21.15.116:44868) 2020-11-20T12:17:08.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c71a32700 1 -- 172.21.15.116:0/194467467 learned_addr learned my addr 172.21.15.116:0/194467467 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:08.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c71231700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c6c0aa790 0x7f5c6c0a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:08.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c72233700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c6c13aee0 0x7f5c6c150a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:08.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c71a32700 1 -- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c6c13aee0 msgr2=0x7f5c6c150a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:08.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c71a32700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c6c13aee0 0x7f5c6c150a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c71a32700 1 -- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c6c0aa790 msgr2=0x7f5c6c0a0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:08.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c71a32700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c6c0aa790 0x7f5c6c0a0520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c71a32700 1 -- 172.21.15.116:0/194467467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c09fde0 con 0x7f5c6c0612f0 2020-11-20T12:17:08.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c72233700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c6c13aee0 0x7f5c6c150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:08.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c71a32700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c6c0612f0 0x7f5c6c156560 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f5c68006fd0 tx=0x7f5c68015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:08.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c59ffb700 1 -- 172.21.15.116:0/194467467 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68011dd0 con 0x7f5c6c0612f0 2020-11-20T12:17:08.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c72a34700 1 -- 172.21.15.116:0/194467467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c0a0080 con 0x7f5c6c0612f0 2020-11-20T12:17:08.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c59ffb700 1 -- 172.21.15.116:0/194467467 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6801e460 con 0x7f5c6c0612f0 2020-11-20T12:17:08.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c59ffb700 1 -- 172.21.15.116:0/194467467 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6801e7e0 con 0x7f5c6c0612f0 2020-11-20T12:17:08.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.368+0000 7f5c72a34700 1 -- 172.21.15.116:0/194467467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c0a0a10 con 0x7f5c6c0612f0 2020-11-20T12:17:08.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.372+0000 7f5c72a34700 1 -- 172.21.15.116:0/194467467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c0a0210 con 0x7f5c6c0612f0 2020-11-20T12:17:08.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.372+0000 7f5c59ffb700 1 -- 172.21.15.116:0/194467467 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5c68025030 con 0x7f5c6c0612f0 2020-11-20T12:17:08.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.372+0000 7f5c59ffb700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5c440557c0 0x7f5c44057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:08.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.372+0000 7f5c59ffb700 1 -- 172.21.15.116:0/194467467 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5c6806fe90 con 0x7f5c6c0612f0 2020-11-20T12:17:08.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.376+0000 7f5c71231700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5c440557c0 0x7f5c44057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:08.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.376+0000 7f5c71231700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5c440557c0 0x7f5c44057c80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f5c6c157de0 tx=0x7f5c5c007b70).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:08.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.380+0000 7f5c59ffb700 1 -- 172.21.15.116:0/194467467 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5c68045de0 con 0x7f5c6c0612f0 2020-11-20T12:17:08.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.584+0000 7f5c72a34700 1 -- 172.21.15.116:0/194467467 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c6c0a0cf0 con 0x7f5c6c0612f0 2020-11-20T12:17:08.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.584+0000 7f5c59ffb700 1 -- 172.21.15.116:0/194467467 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2851 (secure 0 0 0) 0x7f5c68023030 con 0x7f5c6c0612f0 2020-11-20T12:17:08.586 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:08.587 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3031892761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3031892761},{"type":"v1","addr":"172.21.15.173:6836","nonce":3031892761}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1794965470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1794965470},{"type":"v1","addr":"172.21.15.173:6837","nonce":1794965470}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":9},{"gid":4995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/4143559744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":4143559744},{"type":"v1","addr":"172.21.15.116:6833","nonce":4143559744}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:57.901741+0000","modified":"2020-11-20T12:16:59.918401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4998":{"gid":4998,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6835/3994085646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3994085646},{"type":"v1","addr":"172.21.15.116:6835","nonce":3994085646}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T12:17:08.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 -- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5c440557c0 msgr2=0x7f5c44057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:08.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5c440557c0 0x7f5c44057c80 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f5c6c157de0 tx=0x7f5c5c007b70).stop 2020-11-20T12:17:08.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 -- 172.21.15.116:0/194467467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c6c0612f0 msgr2=0x7f5c6c156560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:08.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c6c0612f0 0x7f5c6c156560 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f5c68006fd0 tx=0x7f5c68015000).stop 2020-11-20T12:17:08.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 -- 172.21.15.116:0/194467467 shutdown_connections 2020-11-20T12:17:08.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5c440557c0 0x7f5c44057c80 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5c6c13aee0 0x7f5c6c150a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5c6c0612f0 0x7f5c6c156560 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 --2- 172.21.15.116:0/194467467 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5c6c0aa790 0x7f5c6c0a0520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:08.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 -- 172.21.15.116:0/194467467 >> 172.21.15.116:0/194467467 conn(0x7f5c6c0678a0 msgr2=0x7f5c6c139b80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:08.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 -- 172.21.15.116:0/194467467 shutdown_connections 2020-11-20T12:17:08.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:08.588+0000 7f5c72a34700 1 -- 172.21.15.116:0/194467467 wait complete. 2020-11-20T12:17:08.593 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 14 2020-11-20T12:17:09.016 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:57.901741+0000', 'modified': '2020-11-20T12:16:59.918401+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4998}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4998': {'gid': 4998, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.116:6835/3994085646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3994085646}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3994085646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:17:09.016 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:17:09.017 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:17:09.212 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4998,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.289913486,"mdsmap_epoch":14,"osdmap_epoch":34,"osdmap_epoch_barrier":34,"uptime":19.630979259} 2020-11-20T12:17:09.212 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4998, "mdsmap_epoch": 14, "osdmap_epoch": 34, "osdmap_epoch_barrier": 34, "rank_uptime": 11.289913486, "state": "up:active", "uptime": 19.630979259, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:17:09.213 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:17:09.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:09.214 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-20T12:17:09.231 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:17:09.231 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:09.345 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:09.346 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:17:09.370 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:09.371 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:17:09.377 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:17:09.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:09.378 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-20T12:17:09.406 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:17:09.407 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:17:09.407 INFO:tasks.cephfs.mount:self.client.name = client.2 2020-11-20T12:17:09.408 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2020-11-20T12:17:09.409 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:17:09.409 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:17:09.409 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:17:09.410 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:17:09.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:09.411 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:17:09.429 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:17:09.429 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:17:09.430 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:17:09.430 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:09.430 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:17:09.430 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:09.431 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:09.431 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:09.431 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:09.432 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:09.432 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:09.432 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:09.433 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:17:09.433 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:09.433 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:17:09.433 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:09.434 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:17:09.434 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:09.434 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:17:09.434 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:09.434 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:17:09.435 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:09.435 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:17:09.435 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:09.435 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:17:09.436 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:09.436 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:17:09.436 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:09.436 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:09.437 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:17:09.437 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:17:09.437 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:09.437 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:09.438 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:17:09.438 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:17:09.438 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:09.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:09.439 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:17:09.475 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:17:09.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:09.480 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:17:09.544 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:17:09.544 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:17:09.544 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:17:09.545 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:17:09.545 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:17:09.545 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:17:09.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:09.546 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:17:09.547 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:17:09.547 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:17:09.547 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:17:09.548 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:17:09.548 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:17:09.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:09.640 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:17:09.653 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:17:09.654 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:17:09.654 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:17:09.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:09.655 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2020-11-20T12:17:09.685 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:09.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8ba05a700 1 Processor -- start 2020-11-20T12:17:09.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8ba05a700 1 -- start start 2020-11-20T12:17:09.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8ba05a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd8b40aa570 0x7fd8b40aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:09.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8ba05a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd8b40a81a0 0x7fd8b4151c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:09.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8ba05a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40ad270 0x7fd8b40ad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:09.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8ba05a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b40ab930 con 0x7fd8b40a81a0 2020-11-20T12:17:09.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8ba05a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b4152120 con 0x7fd8b40aa570 2020-11-20T12:17:09.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8ba05a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b41523b0 con 0x7fd8b40ad270 2020-11-20T12:17:09.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8aaffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd8b40a81a0 0x7fd8b4151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:09.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8b3fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40ad270 0x7fd8b40ad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:09.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8b37fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd8b40aa570 0x7fd8b40aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:09.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8b3fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40ad270 0x7fd8b40ad720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:53830/0 (socket says 172.21.15.116:53830) 2020-11-20T12:17:09.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.860+0000 7fd8b3fff700 1 -- 172.21.15.116:0/3203332006 learned_addr learned my addr 172.21.15.116:0/3203332006 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:09.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8b3fff700 1 -- 172.21.15.116:0/3203332006 >> v2:172.21.15.173:3300/0 conn(0x7fd8b40aa570 msgr2=0x7fd8b40aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:09.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8b3fff700 1 --2- 172.21.15.116:0/3203332006 >> v2:172.21.15.173:3300/0 conn(0x7fd8b40aa570 0x7fd8b40aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:09.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8b3fff700 1 -- 172.21.15.116:0/3203332006 >> v2:172.21.15.116:3300/0 conn(0x7fd8b40a81a0 msgr2=0x7fd8b4151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:09.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8b3fff700 1 --2- 172.21.15.116:0/3203332006 >> v2:172.21.15.116:3300/0 conn(0x7fd8b40a81a0 0x7fd8b4151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:09.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8b3fff700 1 -- 172.21.15.116:0/3203332006 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b4152650 con 0x7fd8b40ad270 2020-11-20T12:17:09.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8aaffd700 1 --2- 172.21.15.116:0/3203332006 >> v2:172.21.15.116:3300/0 conn(0x7fd8b40a81a0 0x7fd8b4151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:09.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8b3fff700 1 --2- 172.21.15.116:0/3203332006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40ad270 0x7fd8b40ad720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fd8b4081740 tx=0x7fd8ac009290).ready entity=mon.2 client_cookie=5db22f7543cc80e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:09.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8b2ffd700 1 -- 172.21.15.116:0/3203332006 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac00bba0 con 0x7fd8b40ad270 2020-11-20T12:17:09.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8b2ffd700 1 -- 172.21.15.116:0/3203332006 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac00bd00 con 0x7fd8b40ad270 2020-11-20T12:17:09.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8b2ffd700 1 -- 172.21.15.116:0/3203332006 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac011610 con 0x7fd8b40ad270 2020-11-20T12:17:09.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8ba05a700 1 -- 172.21.15.116:0/3203332006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40ad270 msgr2=0x7fd8b40ad720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:09.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8ba05a700 1 --2- 172.21.15.116:0/3203332006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40ad270 0x7fd8b40ad720 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fd8b4081740 tx=0x7fd8ac009290).stop 2020-11-20T12:17:09.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8ba05a700 1 -- 172.21.15.116:0/3203332006 shutdown_connections 2020-11-20T12:17:09.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8ba05a700 1 --2- 172.21.15.116:0/3203332006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40ad270 0x7fd8b40ad720 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:09.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8ba05a700 1 --2- 172.21.15.116:0/3203332006 >> v2:172.21.15.173:3300/0 conn(0x7fd8b40aa570 0x7fd8b40aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:09.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8ba05a700 1 --2- 172.21.15.116:0/3203332006 >> v2:172.21.15.116:3300/0 conn(0x7fd8b40a81a0 0x7fd8b4151c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:09.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8ba05a700 1 -- 172.21.15.116:0/3203332006 >> 172.21.15.116:0/3203332006 conn(0x7fd8b40678a0 msgr2=0x7fd8b4067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:09.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8ba05a700 1 -- 172.21.15.116:0/3203332006 shutdown_connections 2020-11-20T12:17:09.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.864+0000 7fd8ba05a700 1 -- 172.21.15.116:0/3203332006 wait complete. 2020-11-20T12:17:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.868+0000 7fd8ba05a700 1 Processor -- start 2020-11-20T12:17:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.868+0000 7fd8ba05a700 1 -- start start 2020-11-20T12:17:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8ba05a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40a81a0 0x7fd8b4164050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8ba05a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b40aa570 0x7fd8b4164540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8ba05a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b40ad270 0x7fd8b4160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:09.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8ba05a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4157700 con 0x7fd8b40ad270 2020-11-20T12:17:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8ba05a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4161d30 con 0x7fd8b40aa570 2020-11-20T12:17:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8ba05a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4161fd0 con 0x7fd8b40a81a0 2020-11-20T12:17:09.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8b3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b40ad270 0x7fd8b4160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8b37fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40a81a0 0x7fd8b4164050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8aaffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b40aa570 0x7fd8b4164540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8b3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b40ad270 0x7fd8b4160a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44886/0 (socket says 172.21.15.116:44886) 2020-11-20T12:17:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8b3fff700 1 -- 172.21.15.116:0/385760149 learned_addr learned my addr 172.21.15.116:0/385760149 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8b3fff700 1 -- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40a81a0 msgr2=0x7fd8b4164050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8b3fff700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40a81a0 0x7fd8b4164050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8b3fff700 1 -- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b40aa570 msgr2=0x7fd8b4164540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8b3fff700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b40aa570 0x7fd8b4164540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8b3fff700 1 -- 172.21.15.116:0/385760149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b4162270 con 0x7fd8b40ad270 2020-11-20T12:17:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8b37fe700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40a81a0 0x7fd8b4164050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8b3fff700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b40ad270 0x7fd8b4160a80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fd8ac0092d0 tx=0x7fd8ac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8abfff700 1 -- 172.21.15.116:0/385760149 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac00bba0 con 0x7fd8b40ad270 2020-11-20T12:17:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8abfff700 1 -- 172.21.15.116:0/385760149 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac00be70 con 0x7fd8b40ad270 2020-11-20T12:17:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8ba05a700 1 -- 172.21.15.116:0/385760149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b4160f70 con 0x7fd8b40ad270 2020-11-20T12:17:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8abfff700 1 -- 172.21.15.116:0/385760149 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac010d10 con 0x7fd8b40ad270 2020-11-20T12:17:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.872+0000 7fd8ba05a700 1 -- 172.21.15.116:0/385760149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b41614c0 con 0x7fd8b40ad270 2020-11-20T12:17:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.876+0000 7fd8ba05a700 1 -- 172.21.15.116:0/385760149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b41579a0 con 0x7fd8b40ad270 2020-11-20T12:17:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.876+0000 7fd8abfff700 1 -- 172.21.15.116:0/385760149 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd8ac017070 con 0x7fd8b40ad270 2020-11-20T12:17:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.880+0000 7fd8abfff700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8900557c0 0x7fd890057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.880+0000 7fd8abfff700 1 -- 172.21.15.116:0/385760149 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd8ac0704c0 con 0x7fd8b40ad270 2020-11-20T12:17:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.880+0000 7fd8b37fe700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8900557c0 0x7fd890057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.880+0000 7fd8b37fe700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8900557c0 0x7fd890057c80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fd8a4006fd0 tx=0x7fd8a4006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:09.884+0000 7fd8abfff700 1 -- 172.21.15.116:0/385760149 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd8ac0462e0 con 0x7fd8b40ad270 2020-11-20T12:17:10.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.044+0000 7fd8ba05a700 1 -- 172.21.15.116:0/385760149 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8b405ca30 con 0x7fd8b40ad270 2020-11-20T12:17:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.044+0000 7fd8abfff700 1 -- 172.21.15.116:0/385760149 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fd8ac015090 con 0x7fd8b40ad270 2020-11-20T12:17:10.045 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:10.045 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:58.907407+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:16:55.781578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T12:16:56.900539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:10.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.044+0000 7fd8ba05a700 1 -- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8900557c0 msgr2=0x7fd890057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.044+0000 7fd8ba05a700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8900557c0 0x7fd890057c80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fd8a4006fd0 tx=0x7fd8a4006df0).stop 2020-11-20T12:17:10.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.048+0000 7fd8ba05a700 1 -- 172.21.15.116:0/385760149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b40ad270 msgr2=0x7fd8b4160a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.048+0000 7fd8ba05a700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b40ad270 0x7fd8b4160a80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fd8ac0092d0 tx=0x7fd8ac016000).stop 2020-11-20T12:17:10.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.048+0000 7fd8ba05a700 1 -- 172.21.15.116:0/385760149 shutdown_connections 2020-11-20T12:17:10.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.048+0000 7fd8ba05a700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8900557c0 0x7fd890057c80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.048+0000 7fd8ba05a700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b40ad270 0x7fd8b4160a80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.048+0000 7fd8ba05a700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b40a81a0 0x7fd8b4164050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.048+0000 7fd8ba05a700 1 --2- 172.21.15.116:0/385760149 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8b40aa570 0x7fd8b4164540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.048+0000 7fd8ba05a700 1 -- 172.21.15.116:0/385760149 >> 172.21.15.116:0/385760149 conn(0x7fd8b40678a0 msgr2=0x7fd8b40ad9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:10.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.048+0000 7fd8ba05a700 1 -- 172.21.15.116:0/385760149 shutdown_connections 2020-11-20T12:17:10.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.048+0000 7fd8ba05a700 1 -- 172.21.15.116:0/385760149 wait complete. 2020-11-20T12:17:10.070 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:16:55.781578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:16:56.900539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:17:10.071 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:10.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.252+0000 7fe9b35ca700 1 Processor -- start 2020-11-20T12:17:10.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.252+0000 7fe9b35ca700 1 -- start start 2020-11-20T12:17:10.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.252+0000 7fe9b35ca700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe9ac14f150 0x7fe9ac0c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b35ca700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9ac14f530 0x7fe9ac0c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b35ca700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ac155620 con 0x7fe9ac0bd780 2020-11-20T12:17:10.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b35ca700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ac1558c0 con 0x7fe9ac14f530 2020-11-20T12:17:10.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b35ca700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ac155b60 con 0x7fe9ac14f150 2020-11-20T12:17:10.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b1b67700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9ac14f530 0x7fe9ac0c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b0b65700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fe9ac0bd780 0x7fe9ac0a74b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:50554/0 (socket says 172.21.15.116:50554) 2020-11-20T12:17:10.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b0b65700 1 -- 172.21.15.116:0/3599078393 learned_addr learned my addr 172.21.15.116:0/3599078393 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:10.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b1b67700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9ac14f530 0x7fe9ac0c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44894/0 (socket says 172.21.15.116:44894) 2020-11-20T12:17:10.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b1366700 1 --2- 172.21.15.116:0/3599078393 >> v2:172.21.15.173:3300/0 conn(0x7fe9ac14f150 0x7fe9ac0c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe99bfff700 1 -- 172.21.15.116:0/3599078393 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1348905925 0 0) 0x7fe9ac155620 con 0x7fe9ac0bd780 2020-11-20T12:17:10.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b1b67700 1 -- 172.21.15.116:0/3599078393 >> v2:172.21.15.173:3300/0 conn(0x7fe9ac14f150 msgr2=0x7fe9ac0c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:10.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b1b67700 1 --2- 172.21.15.116:0/3599078393 >> v2:172.21.15.173:3300/0 conn(0x7fe9ac14f150 0x7fe9ac0c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b1b67700 1 -- 172.21.15.116:0/3599078393 >> v1:172.21.15.173:6789/0 conn(0x7fe9ac0bd780 legacy=0x7fe9ac0a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b1b67700 1 -- 172.21.15.116:0/3599078393 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac155e00 con 0x7fe9ac14f530 2020-11-20T12:17:10.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe9b1b67700 1 --2- 172.21.15.116:0/3599078393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9ac14f530 0x7fe9ac0c8ce0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fe9ac081540 tx=0x7fe9a40091d0).ready entity=mon.0 client_cookie=694e70019400e67 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:10.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe99bfff700 1 -- 172.21.15.116:0/3599078393 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a400bad0 con 0x7fe9ac14f530 2020-11-20T12:17:10.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe99bfff700 1 -- 172.21.15.116:0/3599078393 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a400bc30 con 0x7fe9ac14f530 2020-11-20T12:17:10.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.256+0000 7fe99bfff700 1 -- 172.21.15.116:0/3599078393 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a4011400 con 0x7fe9ac14f530 2020-11-20T12:17:10.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3599078393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9ac14f530 msgr2=0x7fe9ac0c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 --2- 172.21.15.116:0/3599078393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9ac14f530 0x7fe9ac0c8ce0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fe9ac081540 tx=0x7fe9a40091d0).stop 2020-11-20T12:17:10.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3599078393 shutdown_connections 2020-11-20T12:17:10.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 --2- 172.21.15.116:0/3599078393 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9ac14f530 0x7fe9ac0c8ce0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 --2- 172.21.15.116:0/3599078393 >> v2:172.21.15.173:3300/0 conn(0x7fe9ac14f150 0x7fe9ac0c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3599078393 >> 172.21.15.116:0/3599078393 conn(0x7fe9ac061580 msgr2=0x7fe9ac061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:10.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3599078393 shutdown_connections 2020-11-20T12:17:10.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3599078393 wait complete. 2020-11-20T12:17:10.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 Processor -- start 2020-11-20T12:17:10.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 -- start start 2020-11-20T12:17:10.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9ac0bd780 0x7fe9ac153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9ac14f150 0x7fe9ac0bf810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9ac14f530 0x7fe9ac14ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac14c170 con 0x7fe9ac0bd780 2020-11-20T12:17:10.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac153bf0 con 0x7fe9ac14f150 2020-11-20T12:17:10.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.260+0000 7fe9b35ca700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac14c3d0 con 0x7fe9ac14f530 2020-11-20T12:17:10.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b1366700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9ac0bd780 0x7fe9ac153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b1366700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9ac0bd780 0x7fe9ac153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44900/0 (socket says 172.21.15.116:44900) 2020-11-20T12:17:10.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b1366700 1 -- 172.21.15.116:0/3882650152 learned_addr learned my addr 172.21.15.116:0/3882650152 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:10.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b0b65700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9ac14f150 0x7fe9ac0bf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b1b67700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9ac14f530 0x7fe9ac14ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b1366700 1 -- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9ac14f530 msgr2=0x7fe9ac14ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b1366700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9ac14f530 0x7fe9ac14ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b1366700 1 -- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9ac14f150 msgr2=0x7fe9ac0bf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b1366700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9ac14f150 0x7fe9ac0bf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b1366700 1 -- 172.21.15.116:0/3882650152 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac14c670 con 0x7fe9ac0bd780 2020-11-20T12:17:10.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b0b65700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9ac14f150 0x7fe9ac0bf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:10.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b1b67700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9ac14f530 0x7fe9ac14ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:10.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b1366700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9ac0bd780 0x7fe9ac153700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fe9a800eba0 tx=0x7fe9a8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:10.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9997fa700 1 -- 172.21.15.116:0/3882650152 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a800ca60 con 0x7fe9ac0bd780 2020-11-20T12:17:10.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3882650152 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ac14c910 con 0x7fe9ac0bd780 2020-11-20T12:17:10.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9997fa700 1 -- 172.21.15.116:0/3882650152 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9a800cbc0 con 0x7fe9ac0bd780 2020-11-20T12:17:10.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9997fa700 1 -- 172.21.15.116:0/3882650152 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9a80123f0 con 0x7fe9ac0bd780 2020-11-20T12:17:10.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3882650152 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ac0bfd00 con 0x7fe9ac0bd780 2020-11-20T12:17:10.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.264+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3882650152 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ac0c4a70 con 0x7fe9ac0bd780 2020-11-20T12:17:10.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.268+0000 7fe9997fa700 1 -- 172.21.15.116:0/3882650152 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe9a80127a0 con 0x7fe9ac0bd780 2020-11-20T12:17:10.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.268+0000 7fe9997fa700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9840557c0 0x7fe984057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.268+0000 7fe9997fa700 1 -- 172.21.15.116:0/3882650152 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe9a806ec80 con 0x7fe9ac0bd780 2020-11-20T12:17:10.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.268+0000 7fe9b0b65700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9840557c0 0x7fe984057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.268+0000 7fe9b0b65700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9840557c0 0x7fe984057c80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fe9ac150b40 tx=0x7fe99c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:10.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.272+0000 7fe9997fa700 1 -- 172.21.15.116:0/3882650152 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe9ac0c4a70 con 0x7fe9ac0bd780 2020-11-20T12:17:10.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.428+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3882650152 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe9ac062390 con 0x7fe9ac0bd780 2020-11-20T12:17:10.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9997fa700 1 -- 172.21.15.116:0/3882650152 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v34) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe9ac062390 con 0x7fe9ac0bd780 2020-11-20T12:17:10.431 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:10.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9840557c0 msgr2=0x7fe984057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9840557c0 0x7fe984057c80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fe9ac150b40 tx=0x7fe99c009040).stop 2020-11-20T12:17:10.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3882650152 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9ac0bd780 msgr2=0x7fe9ac153700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9ac0bd780 0x7fe9ac153700 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fe9a800eba0 tx=0x7fe9a8006df0).stop 2020-11-20T12:17:10.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3882650152 shutdown_connections 2020-11-20T12:17:10.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9840557c0 0x7fe984057c80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9ac14f530 0x7fe9ac14ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9ac0bd780 0x7fe9ac153700 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 --2- 172.21.15.116:0/3882650152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9ac14f150 0x7fe9ac0bf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3882650152 >> 172.21.15.116:0/3882650152 conn(0x7fe9ac061580 msgr2=0x7fe9ac0bdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:10.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3882650152 shutdown_connections 2020-11-20T12:17:10.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.432+0000 7fe9b35ca700 1 -- 172.21.15.116:0/3882650152 wait complete. 2020-11-20T12:17:10.457 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:10.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.632+0000 7f4da58d6700 1 Processor -- start 2020-11-20T12:17:10.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.632+0000 7f4da58d6700 1 -- start start 2020-11-20T12:17:10.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4da58d6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4da00c34a0 0x7f4da00c3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4da58d6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4da0156480 0x7f4da01568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4da58d6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4da00662c0 con 0x7f4da00673b0 2020-11-20T12:17:10.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4da58d6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da0066560 con 0x7f4da0156480 2020-11-20T12:17:10.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4da58d6700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4da0066800 con 0x7f4da00c34a0 2020-11-20T12:17:10.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9f7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4da0156480 0x7f4da01568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9f7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4da0156480 0x7f4da01568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44906/0 (socket says 172.21.15.116:44906) 2020-11-20T12:17:10.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9f7fe700 1 -- 172.21.15.116:0/2391777488 learned_addr learned my addr 172.21.15.116:0/2391777488 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:10.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9effd700 1 --2- 172.21.15.116:0/2391777488 >> v2:172.21.15.173:3300/0 conn(0x7f4da00c34a0 0x7f4da00c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9f7fe700 1 -- 172.21.15.116:0/2391777488 >> v2:172.21.15.173:3300/0 conn(0x7f4da00c34a0 msgr2=0x7f4da00c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:10.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9f7fe700 1 --2- 172.21.15.116:0/2391777488 >> v2:172.21.15.173:3300/0 conn(0x7f4da00c34a0 0x7f4da00c3880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9f7fe700 1 -- 172.21.15.116:0/2391777488 >> v1:172.21.15.116:6789/0 conn(0x7f4da00673b0 legacy=0x7f4da00a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9f7fe700 1 -- 172.21.15.116:0/2391777488 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da0066aa0 con 0x7f4da0156480 2020-11-20T12:17:10.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9f7fe700 1 --2- 172.21.15.116:0/2391777488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4da0156480 0x7f4da01568a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f4da00c1a80 tx=0x7f4d98009290).ready entity=mon.0 client_cookie=fa4cb4dc1abcf01f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:10.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9dffb700 1 -- 172.21.15.116:0/2391777488 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800bc50 con 0x7f4da0156480 2020-11-20T12:17:10.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9dffb700 1 -- 172.21.15.116:0/2391777488 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d9800bdb0 con 0x7f4da0156480 2020-11-20T12:17:10.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.636+0000 7f4d9dffb700 1 -- 172.21.15.116:0/2391777488 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98011610 con 0x7f4da0156480 2020-11-20T12:17:10.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.640+0000 7f4da58d6700 1 -- 172.21.15.116:0/2391777488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4da0156480 msgr2=0x7f4da01568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.640+0000 7f4da58d6700 1 --2- 172.21.15.116:0/2391777488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4da0156480 0x7f4da01568a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f4da00c1a80 tx=0x7f4d98009290).stop 2020-11-20T12:17:10.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.640+0000 7f4da58d6700 1 -- 172.21.15.116:0/2391777488 shutdown_connections 2020-11-20T12:17:10.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.640+0000 7f4da58d6700 1 --2- 172.21.15.116:0/2391777488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4da0156480 0x7f4da01568a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.640+0000 7f4da58d6700 1 --2- 172.21.15.116:0/2391777488 >> v2:172.21.15.173:3300/0 conn(0x7f4da00c34a0 0x7f4da00c3880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.640+0000 7f4da58d6700 1 -- 172.21.15.116:0/2391777488 >> 172.21.15.116:0/2391777488 conn(0x7f4da0060c30 msgr2=0x7f4da00c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:10.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.640+0000 7f4da58d6700 1 -- 172.21.15.116:0/2391777488 shutdown_connections 2020-11-20T12:17:10.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.640+0000 7f4da58d6700 1 -- 172.21.15.116:0/2391777488 wait complete. 2020-11-20T12:17:10.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.640+0000 7f4da58d6700 1 Processor -- start 2020-11-20T12:17:10.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4da58d6700 1 -- start start 2020-11-20T12:17:10.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4da58d6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4da00673b0 0x7f4da013dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4da58d6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4da00c34a0 0x7f4da01687c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4da58d6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4da0156480 0x7f4da0168cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4da58d6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da0162cc0 con 0x7f4da0156480 2020-11-20T12:17:10.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4da58d6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4da0166540 con 0x7f4da00673b0 2020-11-20T12:17:10.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4da58d6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4da01667e0 con 0x7f4da00c34a0 2020-11-20T12:17:10.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4da0156480 0x7f4da0168cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4da0156480 0x7f4da0168cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44914/0 (socket says 172.21.15.116:44914) 2020-11-20T12:17:10.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9f7fe700 1 -- 172.21.15.116:0/562859589 learned_addr learned my addr 172.21.15.116:0/562859589 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:10.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9effd700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4da00673b0 0x7f4da013dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9e7fc700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4da00c34a0 0x7f4da01687c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9f7fe700 1 -- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4da00c34a0 msgr2=0x7f4da01687c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9f7fe700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4da00c34a0 0x7f4da01687c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9f7fe700 1 -- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4da00673b0 msgr2=0x7f4da013dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9f7fe700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4da00673b0 0x7f4da013dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9f7fe700 1 -- 172.21.15.116:0/562859589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4da0166a80 con 0x7f4da0156480 2020-11-20T12:17:10.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9e7fc700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4da00c34a0 0x7f4da01687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:10.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.644+0000 7f4d9f7fe700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4da0156480 0x7f4da0168cb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f4d98007fd0 tx=0x7f4d98016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:10.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.648+0000 7f4d96ffd700 1 -- 172.21.15.116:0/562859589 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800b750 con 0x7f4da0156480 2020-11-20T12:17:10.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.648+0000 7f4d96ffd700 1 -- 172.21.15.116:0/562859589 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d98011e70 con 0x7f4da0156480 2020-11-20T12:17:10.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.648+0000 7f4da58d6700 1 -- 172.21.15.116:0/562859589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4da0166d20 con 0x7f4da0156480 2020-11-20T12:17:10.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.648+0000 7f4d96ffd700 1 -- 172.21.15.116:0/562859589 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d98010d60 con 0x7f4da0156480 2020-11-20T12:17:10.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.648+0000 7f4da58d6700 1 -- 172.21.15.116:0/562859589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4da013e160 con 0x7f4da0156480 2020-11-20T12:17:10.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.648+0000 7f4d94ff9700 1 -- 172.21.15.116:0/562859589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d70004f90 con 0x7f4da0156480 2020-11-20T12:17:10.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.648+0000 7f4d96ffd700 1 -- 172.21.15.116:0/562859589 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4d98017070 con 0x7f4da0156480 2020-11-20T12:17:10.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.648+0000 7f4d96ffd700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d7c0557c0 0x7f4d7c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:10.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.648+0000 7f4d96ffd700 1 -- 172.21.15.116:0/562859589 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4d980708c0 con 0x7f4da0156480 2020-11-20T12:17:10.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.652+0000 7f4d9effd700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d7c0557c0 0x7f4d7c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:10.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.652+0000 7f4d9effd700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d7c0557c0 0x7f4d7c057c80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f4d90006fd0 tx=0x7f4d90006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:10.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.656+0000 7f4d96ffd700 1 -- 172.21.15.116:0/562859589 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4d98046760 con 0x7f4da0156480 2020-11-20T12:17:10.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.812+0000 7f4d94ff9700 1 -- 172.21.15.116:0/562859589 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f4d70003b00 con 0x7f4da0156480 2020-11-20T12:17:10.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.812+0000 7f4d96ffd700 1 -- 172.21.15.116:0/562859589 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4d98015070 con 0x7f4da0156480 2020-11-20T12:17:10.813 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:10.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 -- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d7c0557c0 msgr2=0x7f4d7c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d7c0557c0 0x7f4d7c057c80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f4d90006fd0 tx=0x7f4d90006df0).stop 2020-11-20T12:17:10.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 -- 172.21.15.116:0/562859589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4da0156480 msgr2=0x7f4da0168cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:10.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4da0156480 0x7f4da0168cb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f4d98007fd0 tx=0x7f4d98016000).stop 2020-11-20T12:17:10.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 -- 172.21.15.116:0/562859589 shutdown_connections 2020-11-20T12:17:10.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d7c0557c0 0x7f4d7c057c80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4da0156480 0x7f4da0168cb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4da00673b0 0x7f4da013dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 --2- 172.21.15.116:0/562859589 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4da00c34a0 0x7f4da01687c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:10.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 -- 172.21.15.116:0/562859589 >> 172.21.15.116:0/562859589 conn(0x7f4da0060c30 msgr2=0x7f4da0155010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:10.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 -- 172.21.15.116:0/562859589 shutdown_connections 2020-11-20T12:17:10.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:10.816+0000 7f4d94ff9700 1 -- 172.21.15.116:0/562859589 wait complete. 2020-11-20T12:17:10.837 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:17:11.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.012+0000 7f3eb7669700 1 Processor -- start 2020-11-20T12:17:11.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.012+0000 7f3eb7669700 1 -- start start 2020-11-20T12:17:11.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb7669700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3eb014f150 0x7f3eb00c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb7669700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3eb00bd780 0x7f3eb00c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb7669700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb014f530 0x7f3eb00a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb7669700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb0155620 con 0x7f3eb014f150 2020-11-20T12:17:11.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb7669700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb01558c0 con 0x7f3eb00bd780 2020-11-20T12:17:11.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb7669700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0155b60 con 0x7f3eb014f530 2020-11-20T12:17:11.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb5405700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3eb014f150 0x7f3eb00c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb4c04700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3eb00bd780 0x7f3eb00c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb5405700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3eb014f150 0x7f3eb00c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44920/0 (socket says 172.21.15.116:44920) 2020-11-20T12:17:11.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb5405700 1 -- 172.21.15.116:0/1241146094 learned_addr learned my addr 172.21.15.116:0/1241146094 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:11.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb5c06700 1 --2- 172.21.15.116:0/1241146094 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb014f530 0x7f3eb00a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb4c04700 1 -- 172.21.15.116:0/1241146094 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb014f530 msgr2=0x7f3eb00a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:11.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb4c04700 1 --2- 172.21.15.116:0/1241146094 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb014f530 0x7f3eb00a74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb4c04700 1 -- 172.21.15.116:0/1241146094 >> v2:172.21.15.116:3300/0 conn(0x7f3eb014f150 msgr2=0x7f3eb00c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:11.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb4c04700 1 --2- 172.21.15.116:0/1241146094 >> v2:172.21.15.116:3300/0 conn(0x7f3eb014f150 0x7f3eb00c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb4c04700 1 -- 172.21.15.116:0/1241146094 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb0155e00 con 0x7f3eb00bd780 2020-11-20T12:17:11.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.016+0000 7f3eb4c04700 1 --2- 172.21.15.116:0/1241146094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb00bd780 0x7f3eb00c8ce0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f3eb00ae8a0 tx=0x7f3ea0006df0).ready entity=mon.1 client_cookie=ccba0ea9e34c9da1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:11.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3e9ffff700 1 -- 172.21.15.116:0/1241146094 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea00096d0 con 0x7f3eb00bd780 2020-11-20T12:17:11.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3e9ffff700 1 -- 172.21.15.116:0/1241146094 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea0009830 con 0x7f3eb00bd780 2020-11-20T12:17:11.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3e9ffff700 1 -- 172.21.15.116:0/1241146094 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0009bb0 con 0x7f3eb00bd780 2020-11-20T12:17:11.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3eb7669700 1 -- 172.21.15.116:0/1241146094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb00bd780 msgr2=0x7f3eb00c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3eb7669700 1 --2- 172.21.15.116:0/1241146094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb00bd780 0x7f3eb00c8ce0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f3eb00ae8a0 tx=0x7f3ea0006df0).stop 2020-11-20T12:17:11.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3eb7669700 1 -- 172.21.15.116:0/1241146094 shutdown_connections 2020-11-20T12:17:11.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3eb7669700 1 --2- 172.21.15.116:0/1241146094 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb014f530 0x7f3eb00a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3eb7669700 1 --2- 172.21.15.116:0/1241146094 >> v2:172.21.15.116:3300/0 conn(0x7f3eb014f150 0x7f3eb00c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3eb7669700 1 --2- 172.21.15.116:0/1241146094 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb00bd780 0x7f3eb00c8ce0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3eb7669700 1 -- 172.21.15.116:0/1241146094 >> 172.21.15.116:0/1241146094 conn(0x7f3eb0061580 msgr2=0x7f3eb0061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:11.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3eb7669700 1 -- 172.21.15.116:0/1241146094 shutdown_connections 2020-11-20T12:17:11.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.020+0000 7f3eb7669700 1 -- 172.21.15.116:0/1241146094 wait complete. 2020-11-20T12:17:11.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb7669700 1 Processor -- start 2020-11-20T12:17:11.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb7669700 1 -- start start 2020-11-20T12:17:11.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb7669700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb00bd780 0x7f3eb00ab750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb7669700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb014f150 0x7f3eb014d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb7669700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb014f530 0x7f3eb00ad280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb7669700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00c1900 con 0x7f3eb014f530 2020-11-20T12:17:11.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb7669700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00c1ba0 con 0x7f3eb014f150 2020-11-20T12:17:11.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb7669700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb00c1e40 con 0x7f3eb00bd780 2020-11-20T12:17:11.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb5c06700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb014f530 0x7f3eb00ad280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb5c06700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb014f530 0x7f3eb00ad280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44928/0 (socket says 172.21.15.116:44928) 2020-11-20T12:17:11.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb5c06700 1 -- 172.21.15.116:0/3633378644 learned_addr learned my addr 172.21.15.116:0/3633378644 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:11.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb5405700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb00bd780 0x7f3eb00ab750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.024+0000 7f3eb4c04700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb014f150 0x7f3eb014d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3eb5c06700 1 -- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb00bd780 msgr2=0x7f3eb00ab750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3eb5c06700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb00bd780 0x7f3eb00ab750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3eb5c06700 1 -- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb014f150 msgr2=0x7f3eb014d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3eb5c06700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb014f150 0x7f3eb014d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3eb5c06700 1 -- 172.21.15.116:0/3633378644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb00c20e0 con 0x7f3eb014f530 2020-11-20T12:17:11.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3eb5c06700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb014f530 0x7f3eb00ad280 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f3ea800ed20 tx=0x7f3ea80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:11.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3e9d7fa700 1 -- 172.21.15.116:0/3633378644 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea800cbb0 con 0x7f3eb014f530 2020-11-20T12:17:11.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3eb7669700 1 -- 172.21.15.116:0/3633378644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb00c2380 con 0x7f3eb014f530 2020-11-20T12:17:11.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3e9d7fa700 1 -- 172.21.15.116:0/3633378644 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea800cd10 con 0x7f3eb014f530 2020-11-20T12:17:11.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3e9d7fa700 1 -- 172.21.15.116:0/3633378644 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea8012620 con 0x7f3eb014f530 2020-11-20T12:17:11.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3eb7669700 1 -- 172.21.15.116:0/3633378644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb014dc30 con 0x7f3eb014f530 2020-11-20T12:17:11.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.028+0000 7f3eb7669700 1 -- 172.21.15.116:0/3633378644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb005ca30 con 0x7f3eb014f530 2020-11-20T12:17:11.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.032+0000 7f3e9d7fa700 1 -- 172.21.15.116:0/3633378644 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3ea8022070 con 0x7f3eb014f530 2020-11-20T12:17:11.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.032+0000 7f3e9d7fa700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e880557c0 0x7f3e88057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.032+0000 7f3e9d7fa700 1 -- 172.21.15.116:0/3633378644 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3ea806ee50 con 0x7f3eb014f530 2020-11-20T12:17:11.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.032+0000 7f3eb5405700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e880557c0 0x7f3e88057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.032+0000 7f3eb5405700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e880557c0 0x7f3e88057c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f3eac006a10 tx=0x7f3eac008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:11.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.036+0000 7f3e9d7fa700 1 -- 172.21.15.116:0/3633378644 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3ea8044cf0 con 0x7f3eb014f530 2020-11-20T12:17:11.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.192+0000 7f3eb7669700 1 -- 172.21.15.116:0/3633378644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3eb00c8fb0 con 0x7f3eb014f530 2020-11-20T12:17:11.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3e9d7fa700 1 -- 172.21.15.116:0/3633378644 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3ea8016070 con 0x7f3eb014f530 2020-11-20T12:17:11.195 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 -- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e880557c0 msgr2=0x7f3e88057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e880557c0 0x7f3e88057c80 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f3eac006a10 tx=0x7f3eac008040).stop 2020-11-20T12:17:11.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 -- 172.21.15.116:0/3633378644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb014f530 msgr2=0x7f3eb00ad280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb014f530 0x7f3eb00ad280 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f3ea800ed20 tx=0x7f3ea80091d0).stop 2020-11-20T12:17:11.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 -- 172.21.15.116:0/3633378644 shutdown_connections 2020-11-20T12:17:11.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e880557c0 0x7f3e88057c80 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eb014f530 0x7f3eb00ad280 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eb00bd780 0x7f3eb00ab750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 --2- 172.21.15.116:0/3633378644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eb014f150 0x7f3eb014d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 -- 172.21.15.116:0/3633378644 >> 172.21.15.116:0/3633378644 conn(0x7f3eb0061580 msgr2=0x7f3eb00bdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:11.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 -- 172.21.15.116:0/3633378644 shutdown_connections 2020-11-20T12:17:11.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.196+0000 7f3eb7669700 1 -- 172.21.15.116:0/3633378644 wait complete. 2020-11-20T12:17:11.222 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:17:11.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.396+0000 7f72d9106700 1 Processor -- start 2020-11-20T12:17:11.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.400+0000 7f72d9106700 1 -- start start 2020-11-20T12:17:11.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.400+0000 7f72d9106700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a6330 0x7f72d40a28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.400+0000 7f72d9106700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f72d40a2db0 0x7f72d41550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.400+0000 7f72d9106700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72d40a3400 con 0x7f72d40a5f50 2020-11-20T12:17:11.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.400+0000 7f72d9106700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72d40a36a0 con 0x7f72d40a2db0 2020-11-20T12:17:11.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.400+0000 7f72d9106700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d4066130 con 0x7f72d40a6330 2020-11-20T12:17:11.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.400+0000 7f72d37fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a6330 0x7f72d40a28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.400+0000 7f72d37fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a6330 0x7f72d40a28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:53884/0 (socket says 172.21.15.116:53884) 2020-11-20T12:17:11.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.400+0000 7f72d37fe700 1 -- 172.21.15.116:0/2824158230 learned_addr learned my addr 172.21.15.116:0/2824158230 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:11.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.400+0000 7f72d8905700 1 --2- 172.21.15.116:0/2824158230 >> v2:172.21.15.173:3300/0 conn(0x7f72d40a2db0 0x7f72d41550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d37fe700 1 -- 172.21.15.116:0/2824158230 >> v2:172.21.15.173:3300/0 conn(0x7f72d40a2db0 msgr2=0x7f72d41550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:11.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d37fe700 1 --2- 172.21.15.116:0/2824158230 >> v2:172.21.15.173:3300/0 conn(0x7f72d40a2db0 0x7f72d41550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d37fe700 1 -- 172.21.15.116:0/2824158230 >> v1:172.21.15.173:6789/0 conn(0x7f72d40a5f50 legacy=0x7f72d40c1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:11.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d37fe700 1 -- 172.21.15.116:0/2824158230 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d40663d0 con 0x7f72d40a6330 2020-11-20T12:17:11.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d37fe700 1 --2- 172.21.15.116:0/2824158230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a6330 0x7f72d40a28c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f72c8006fd0 tx=0x7f72c8006eb0).ready entity=mon.2 client_cookie=204815b5b7f2cb60 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:11.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d2ffd700 1 -- 172.21.15.116:0/2824158230 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8009920 con 0x7f72d40a6330 2020-11-20T12:17:11.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d2ffd700 1 -- 172.21.15.116:0/2824158230 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c8009a80 con 0x7f72d40a6330 2020-11-20T12:17:11.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d2ffd700 1 -- 172.21.15.116:0/2824158230 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c8009e00 con 0x7f72d40a6330 2020-11-20T12:17:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d9106700 1 -- 172.21.15.116:0/2824158230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a6330 msgr2=0x7f72d40a28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d9106700 1 --2- 172.21.15.116:0/2824158230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a6330 0x7f72d40a28c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f72c8006fd0 tx=0x7f72c8006eb0).stop 2020-11-20T12:17:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d9106700 1 -- 172.21.15.116:0/2824158230 shutdown_connections 2020-11-20T12:17:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d9106700 1 --2- 172.21.15.116:0/2824158230 >> v2:172.21.15.173:3300/0 conn(0x7f72d40a2db0 0x7f72d41550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d9106700 1 --2- 172.21.15.116:0/2824158230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a6330 0x7f72d40a28c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d9106700 1 -- 172.21.15.116:0/2824158230 >> 172.21.15.116:0/2824158230 conn(0x7f72d4060c30 msgr2=0x7f72d40c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:11.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d9106700 1 -- 172.21.15.116:0/2824158230 shutdown_connections 2020-11-20T12:17:11.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d9106700 1 -- 172.21.15.116:0/2824158230 wait complete. 2020-11-20T12:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.404+0000 7f72d9106700 1 Processor -- start 2020-11-20T12:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d9106700 1 -- start start 2020-11-20T12:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d9106700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d40a2db0 0x7f72d4150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d9106700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a5f50 0x7f72d414c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d9106700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72d413b890 0x7f72d414f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d9106700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d414ae00 con 0x7f72d413b890 2020-11-20T12:17:11.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d9106700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d414a8c0 con 0x7f72d40a2db0 2020-11-20T12:17:11.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d9106700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d414ab60 con 0x7f72d40a5f50 2020-11-20T12:17:11.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d8905700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72d413b890 0x7f72d414f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d8905700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72d413b890 0x7f72d414f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44942/0 (socket says 172.21.15.116:44942) 2020-11-20T12:17:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d3fff700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d40a2db0 0x7f72d4150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d8905700 1 -- 172.21.15.116:0/4240904238 learned_addr learned my addr 172.21.15.116:0/4240904238 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d37fe700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a5f50 0x7f72d414c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d3fff700 1 -- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a5f50 msgr2=0x7f72d414c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d3fff700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a5f50 0x7f72d414c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d3fff700 1 -- 172.21.15.116:0/4240904238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72d413b890 msgr2=0x7f72d414f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d3fff700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72d413b890 0x7f72d414f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d3fff700 1 -- 172.21.15.116:0/4240904238 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d41502f0 con 0x7f72d40a2db0 2020-11-20T12:17:11.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.408+0000 7f72d3fff700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d40a2db0 0x7f72d4150a30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f72c4009710 tx=0x7f72c4006cb0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:11.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.412+0000 7f72b3fff700 1 -- 172.21.15.116:0/4240904238 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400d490 con 0x7f72d40a2db0 2020-11-20T12:17:11.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.412+0000 7f72b3fff700 1 -- 172.21.15.116:0/4240904238 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72c4000e90 con 0x7f72d40a2db0 2020-11-20T12:17:11.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.412+0000 7f72b3fff700 1 -- 172.21.15.116:0/4240904238 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72c400d820 con 0x7f72d40a2db0 2020-11-20T12:17:11.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.412+0000 7f72d9106700 1 -- 172.21.15.116:0/4240904238 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72d4150590 con 0x7f72d40a2db0 2020-11-20T12:17:11.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.412+0000 7f72d9106700 1 -- 172.21.15.116:0/4240904238 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d414a5b0 con 0x7f72d40a2db0 2020-11-20T12:17:11.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.412+0000 7f72d9106700 1 -- 172.21.15.116:0/4240904238 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72d4067410 con 0x7f72d40a2db0 2020-11-20T12:17:11.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.412+0000 7f72b3fff700 1 -- 172.21.15.116:0/4240904238 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f72c400f030 con 0x7f72d40a2db0 2020-11-20T12:17:11.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.416+0000 7f72b3fff700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72ac0557c0 0x7f72ac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.416+0000 7f72b3fff700 1 -- 172.21.15.116:0/4240904238 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f72c406e1b0 con 0x7f72d40a2db0 2020-11-20T12:17:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.416+0000 7f72d37fe700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72ac0557c0 0x7f72ac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.416+0000 7f72d37fe700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72ac0557c0 0x7f72ac057c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f72d40c3630 tx=0x7f72c8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:11.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.424+0000 7f72b3fff700 1 -- 172.21.15.116:0/4240904238 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f72d4067410 con 0x7f72d40a2db0 2020-11-20T12:17:11.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.584+0000 7f72d9106700 1 -- 172.21.15.116:0/4240904238 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f72d4067410 con 0x7f72d40a2db0 2020-11-20T12:17:11.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.584+0000 7f72b3fff700 1 -- 172.21.15.116:0/4240904238 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+10 (secure 0 0 0) 0x7f72d4067410 con 0x7f72d40a2db0 2020-11-20T12:17:11.584 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:11.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.584+0000 7f72d9106700 1 -- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72ac0557c0 msgr2=0x7f72ac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.584+0000 7f72d9106700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72ac0557c0 0x7f72ac057c80 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f72d40c3630 tx=0x7f72c8015000).stop 2020-11-20T12:17:11.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.584+0000 7f72d9106700 1 -- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d40a2db0 msgr2=0x7f72d4150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.584+0000 7f72d9106700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d40a2db0 0x7f72d4150a30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f72c4009710 tx=0x7f72c4006cb0).stop 2020-11-20T12:17:11.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.584+0000 7f72d9106700 1 -- 172.21.15.116:0/4240904238 shutdown_connections 2020-11-20T12:17:11.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.584+0000 7f72d9106700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72ac0557c0 0x7f72ac057c80 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.588+0000 7f72d9106700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72d413b890 0x7f72d414f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.588+0000 7f72d9106700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72d40a2db0 0x7f72d4150a30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.588+0000 7f72d9106700 1 --2- 172.21.15.116:0/4240904238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72d40a5f50 0x7f72d414c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.588+0000 7f72d9106700 1 -- 172.21.15.116:0/4240904238 >> 172.21.15.116:0/4240904238 conn(0x7f72d4060c30 msgr2=0x7f72d4061ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.588+0000 7f72d9106700 1 -- 172.21.15.116:0/4240904238 shutdown_connections 2020-11-20T12:17:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.588+0000 7f72d9106700 1 -- 172.21.15.116:0/4240904238 wait complete. 2020-11-20T12:17:11.611 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:17:11.612 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:11.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.780+0000 7fbf09fc3700 1 Processor -- start 2020-11-20T12:17:11.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.780+0000 7fbf09fc3700 1 -- start start 2020-11-20T12:17:11.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf09fc3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf040baa30 0x7fbf04066510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf09fc3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbf040c27e0 0x7fbf04066a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf09fc3700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf040c2da0 con 0x7fbf040bae10 2020-11-20T12:17:11.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf09fc3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf040c3040 con 0x7fbf040baa30 2020-11-20T12:17:11.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf09fc3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf040c32e0 con 0x7fbf040c27e0 2020-11-20T12:17:11.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf037fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf040baa30 0x7fbf04066510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf037fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbf040baa30 0x7fbf04066510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44946/0 (socket says 172.21.15.116:44946) 2020-11-20T12:17:11.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf037fe700 1 -- 172.21.15.116:0/536945502 learned_addr learned my addr 172.21.15.116:0/536945502 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:11.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf03fff700 1 --2- 172.21.15.116:0/536945502 >> v2:172.21.15.173:3300/0 conn(0x7fbf040c27e0 0x7fbf04066a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf037fe700 1 -- 172.21.15.116:0/536945502 >> v2:172.21.15.173:3300/0 conn(0x7fbf040c27e0 msgr2=0x7fbf04066a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf037fe700 1 --2- 172.21.15.116:0/536945502 >> v2:172.21.15.173:3300/0 conn(0x7fbf040c27e0 0x7fbf04066a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf037fe700 1 -- 172.21.15.116:0/536945502 >> v1:172.21.15.173:6789/0 conn(0x7fbf040bae10 legacy=0x7fbf040c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf037fe700 1 -- 172.21.15.116:0/536945502 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040c3580 con 0x7fbf040baa30 2020-11-20T12:17:11.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.784+0000 7fbf037fe700 1 --2- 172.21.15.116:0/536945502 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf040baa30 0x7fbf04066510 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fbf040682f0 tx=0x7fbef4006eb0).ready entity=mon.0 client_cookie=73b7a8556e3c2828 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:11.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.788+0000 7fbf027fc700 1 -- 172.21.15.116:0/536945502 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4009810 con 0x7fbf040baa30 2020-11-20T12:17:11.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.788+0000 7fbf027fc700 1 -- 172.21.15.116:0/536945502 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef4009970 con 0x7fbf040baa30 2020-11-20T12:17:11.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.788+0000 7fbf027fc700 1 -- 172.21.15.116:0/536945502 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4009cf0 con 0x7fbf040baa30 2020-11-20T12:17:11.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.788+0000 7fbf09fc3700 1 -- 172.21.15.116:0/536945502 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf040baa30 msgr2=0x7fbf04066510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.788+0000 7fbf09fc3700 1 --2- 172.21.15.116:0/536945502 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf040baa30 0x7fbf04066510 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fbf040682f0 tx=0x7fbef4006eb0).stop 2020-11-20T12:17:11.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.788+0000 7fbf09fc3700 1 -- 172.21.15.116:0/536945502 shutdown_connections 2020-11-20T12:17:11.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.788+0000 7fbf09fc3700 1 --2- 172.21.15.116:0/536945502 >> v2:172.21.15.173:3300/0 conn(0x7fbf040c27e0 0x7fbf04066a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.788+0000 7fbf09fc3700 1 --2- 172.21.15.116:0/536945502 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf040baa30 0x7fbf04066510 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.788+0000 7fbf09fc3700 1 -- 172.21.15.116:0/536945502 >> 172.21.15.116:0/536945502 conn(0x7fbf04060c80 msgr2=0x7fbf040c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:11.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.788+0000 7fbf09fc3700 1 -- 172.21.15.116:0/536945502 shutdown_connections 2020-11-20T12:17:11.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.788+0000 7fbf09fc3700 1 -- 172.21.15.116:0/536945502 wait complete. 2020-11-20T12:17:11.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.792+0000 7fbf09fc3700 1 Processor -- start 2020-11-20T12:17:11.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.792+0000 7fbf09fc3700 1 -- start start 2020-11-20T12:17:11.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.792+0000 7fbf09fc3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf040baa30 0x7fbf040bf160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.792+0000 7fbf09fc3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf040bae10 0x7fbf040bb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.792+0000 7fbf09fc3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf040c27e0 0x7fbf040a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.792+0000 7fbf09fc3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040bfa70 con 0x7fbf040baa30 2020-11-20T12:17:11.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.792+0000 7fbf09fc3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040bfd10 con 0x7fbf040bae10 2020-11-20T12:17:11.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.792+0000 7fbf09fc3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf040bffb0 con 0x7fbf040c27e0 2020-11-20T12:17:11.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.792+0000 7fbf037fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf040baa30 0x7fbf040bf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf037fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf040baa30 0x7fbf040bf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44952/0 (socket says 172.21.15.116:44952) 2020-11-20T12:17:11.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf037fe700 1 -- 172.21.15.116:0/562334162 learned_addr learned my addr 172.21.15.116:0/562334162 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:11.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf02ffd700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf040bae10 0x7fbf040bb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf03fff700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf040c27e0 0x7fbf040a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf037fe700 1 -- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf040c27e0 msgr2=0x7fbf040a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf037fe700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf040c27e0 0x7fbf040a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf037fe700 1 -- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf040bae10 msgr2=0x7fbf040bb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:11.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf037fe700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf040bae10 0x7fbf040bb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:11.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf037fe700 1 -- 172.21.15.116:0/562334162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf040c0250 con 0x7fbf040baa30 2020-11-20T12:17:11.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf02ffd700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf040bae10 0x7fbf040bb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:11.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf037fe700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf040baa30 0x7fbf040bf160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fbef400b1b0 tx=0x7fbef4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:11.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbee37fe700 1 -- 172.21.15.116:0/562334162 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef4011dd0 con 0x7fbf040baa30 2020-11-20T12:17:11.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbee37fe700 1 -- 172.21.15.116:0/562334162 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbef401e460 con 0x7fbf040baa30 2020-11-20T12:17:11.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf09fc3700 1 -- 172.21.15.116:0/562334162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf040c04f0 con 0x7fbf040baa30 2020-11-20T12:17:11.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbee37fe700 1 -- 172.21.15.116:0/562334162 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbef401e800 con 0x7fbf040baa30 2020-11-20T12:17:11.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf09fc3700 1 -- 172.21.15.116:0/562334162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf040bb8a0 con 0x7fbf040baa30 2020-11-20T12:17:11.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbf09fc3700 1 -- 172.21.15.116:0/562334162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf04150620 con 0x7fbf040baa30 2020-11-20T12:17:11.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.796+0000 7fbee37fe700 1 -- 172.21.15.116:0/562334162 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fbef4025030 con 0x7fbf040baa30 2020-11-20T12:17:11.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.800+0000 7fbee37fe700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbedc0557c0 0x7fbedc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:11.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.800+0000 7fbee37fe700 1 -- 172.21.15.116:0/562334162 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbef406fe60 con 0x7fbf040baa30 2020-11-20T12:17:11.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.800+0000 7fbf02ffd700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbedc0557c0 0x7fbedc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:11.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.800+0000 7fbf02ffd700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbedc0557c0 0x7fbedc057c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fbf0415b860 tx=0x7fbef8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:11.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:11.804+0000 7fbee37fe700 1 -- 172.21.15.116:0/562334162 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbf04150620 con 0x7fbf040baa30 2020-11-20T12:17:12.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.004+0000 7fbf09fc3700 1 -- 172.21.15.116:0/562334162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf04150620 con 0x7fbf040baa30 2020-11-20T12:17:12.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbee37fe700 1 -- 172.21.15.116:0/562334162 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2851 (secure 0 0 0) 0x7fbf04150620 con 0x7fbf040baa30 2020-11-20T12:17:12.007 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:12.007 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3031892761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3031892761},{"type":"v1","addr":"172.21.15.173:6836","nonce":3031892761}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1794965470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1794965470},{"type":"v1","addr":"172.21.15.173:6837","nonce":1794965470}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":9},{"gid":4995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/4143559744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":4143559744},{"type":"v1","addr":"172.21.15.116:6833","nonce":4143559744}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:57.901741+0000","modified":"2020-11-20T12:16:59.918401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4998":{"gid":4998,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6835/3994085646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3994085646},{"type":"v1","addr":"172.21.15.116:6835","nonce":3994085646}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T12:17:12.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 -- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbedc0557c0 msgr2=0x7fbedc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:12.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbedc0557c0 0x7fbedc057c80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fbf0415b860 tx=0x7fbef8009040).stop 2020-11-20T12:17:12.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 -- 172.21.15.116:0/562334162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf040baa30 msgr2=0x7fbf040bf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:12.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf040baa30 0x7fbf040bf160 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fbef400b1b0 tx=0x7fbef4015000).stop 2020-11-20T12:17:12.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 -- 172.21.15.116:0/562334162 shutdown_connections 2020-11-20T12:17:12.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbedc0557c0 0x7fbedc057c80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:12.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbf040c27e0 0x7fbf040a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:12.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbf040baa30 0x7fbf040bf160 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:12.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 --2- 172.21.15.116:0/562334162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbf040bae10 0x7fbf040bb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:12.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 -- 172.21.15.116:0/562334162 >> 172.21.15.116:0/562334162 conn(0x7fbf04060c80 msgr2=0x7fbf040b9500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:12.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 -- 172.21.15.116:0/562334162 shutdown_connections 2020-11-20T12:17:12.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:12.008+0000 7fbf09fc3700 1 -- 172.21.15.116:0/562334162 wait complete. 2020-11-20T12:17:12.012 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 14 2020-11-20T12:17:12.437 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:57.901741+0000', 'modified': '2020-11-20T12:16:59.918401+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4998}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4998': {'gid': 4998, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.116:6835/3994085646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3994085646}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3994085646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:17:12.437 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:17:12.438 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:17:12.642 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4998,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.719438746,"mdsmap_epoch":14,"osdmap_epoch":34,"osdmap_epoch_barrier":34,"uptime":23.060501071000001} 2020-11-20T12:17:12.643 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4998, "mdsmap_epoch": 14, "osdmap_epoch": 34, "osdmap_epoch_barrier": 34, "rank_uptime": 14.719438746, "state": "up:active", "uptime": 23.060501071, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:17:12.644 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:17:12.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:12.649 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.2 2020-11-20T12:17:12.666 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:17:12.666 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:12.784 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:12.785 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:17:12.815 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:12.815 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:17:12.823 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:17:12.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:12.824 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2020-11-20T12:17:12.854 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:17:12.854 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:17:12.855 INFO:tasks.cephfs.mount:self.client.name = client.3 2020-11-20T12:17:12.855 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2020-11-20T12:17:12.855 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:17:12.856 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:17:12.856 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:17:12.856 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:17:12.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:12.858 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:17:12.876 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:17:12.877 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:17:12.877 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:17:12.877 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:12.877 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:17:12.878 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:12.878 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:12.878 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:12.878 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:12.879 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:12.879 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:12.879 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:12.879 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:17:12.880 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:12.880 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:17:12.880 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:12.880 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:17:12.880 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:12.881 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:17:12.881 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:12.881 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:17:12.881 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:12.881 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:17:12.882 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:12.882 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:17:12.882 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:12.882 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:17:12.882 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:12.883 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:12.883 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:17:12.883 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:17:12.883 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:12.884 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:12.884 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:17:12.884 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:17:12.884 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:12.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:12.885 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:17:12.951 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:17:12.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:12.955 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:17:12.971 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:17:12.971 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:17:12.972 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:17:12.972 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:17:12.972 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:17:12.973 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:17:12.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:12.974 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:17:12.975 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:17:12.976 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:17:12.977 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:17:12.978 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:17:12.978 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:17:13.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:13.086 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:17:13.101 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:17:13.101 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:17:13.102 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:17:13.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:13.103 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2020-11-20T12:17:13.130 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:13.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f9231ce7700 1 Processor -- start 2020-11-20T12:17:13.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f9231ce7700 1 -- start start 2020-11-20T12:17:13.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f9231ce7700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f922c0c41c0 0x7f922c0c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f9231ce7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f922c0a5a10 0x7f922c0a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f9231ce7700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f922c138670 0x7f922c138b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f9231ce7700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c0a6380 con 0x7f922c138670 2020-11-20T12:17:13.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f9231ce7700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922c0a9180 con 0x7f922c0c41c0 2020-11-20T12:17:13.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f9231ce7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c0a9420 con 0x7f922c0a5a10 2020-11-20T12:17:13.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f92314e6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f922c138670 0x7f922c138b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f922bfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f922c0a5a10 0x7f922c0a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f922bfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f922c0a5a10 0x7f922c0a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:53914/0 (socket says 172.21.15.116:53914) 2020-11-20T12:17:13.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f922bfff700 1 -- 172.21.15.116:0/1799910917 learned_addr learned my addr 172.21.15.116:0/1799910917 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:13.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.308+0000 7f9230ce5700 1 --2- 172.21.15.116:0/1799910917 >> v2:172.21.15.173:3300/0 conn(0x7f922c0c41c0 0x7f922c0c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f92314e6700 1 -- 172.21.15.116:0/1799910917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f922c0a5a10 msgr2=0x7f922c0a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:13.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f92314e6700 1 --2- 172.21.15.116:0/1799910917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f922c0a5a10 0x7f922c0a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f92314e6700 1 -- 172.21.15.116:0/1799910917 >> v2:172.21.15.173:3300/0 conn(0x7f922c0c41c0 msgr2=0x7f922c0c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:13.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f92314e6700 1 --2- 172.21.15.116:0/1799910917 >> v2:172.21.15.173:3300/0 conn(0x7f922c0c41c0 0x7f922c0c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f92314e6700 1 -- 172.21.15.116:0/1799910917 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c0a96c0 con 0x7f922c138670 2020-11-20T12:17:13.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f9230ce5700 1 --2- 172.21.15.116:0/1799910917 >> v2:172.21.15.173:3300/0 conn(0x7f922c0c41c0 0x7f922c0c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:13.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f92314e6700 1 --2- 172.21.15.116:0/1799910917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f922c138670 0x7f922c138b00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f922c065960 tx=0x7f92240091d0).ready entity=mon.0 client_cookie=c7aadd43a6a0dd4f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:13.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f922b7fe700 1 -- 172.21.15.116:0/1799910917 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f922400ba70 con 0x7f922c138670 2020-11-20T12:17:13.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f922b7fe700 1 -- 172.21.15.116:0/1799910917 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f922400bbd0 con 0x7f922c138670 2020-11-20T12:17:13.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f922b7fe700 1 -- 172.21.15.116:0/1799910917 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224011400 con 0x7f922c138670 2020-11-20T12:17:13.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f9231ce7700 1 -- 172.21.15.116:0/1799910917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f922c138670 msgr2=0x7f922c138b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:13.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f9231ce7700 1 --2- 172.21.15.116:0/1799910917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f922c138670 0x7f922c138b00 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f922c065960 tx=0x7f92240091d0).stop 2020-11-20T12:17:13.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f9231ce7700 1 -- 172.21.15.116:0/1799910917 shutdown_connections 2020-11-20T12:17:13.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f9231ce7700 1 --2- 172.21.15.116:0/1799910917 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f922c138670 0x7f922c138b00 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f9231ce7700 1 --2- 172.21.15.116:0/1799910917 >> v2:172.21.15.173:3300/0 conn(0x7f922c0c41c0 0x7f922c0c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f9231ce7700 1 --2- 172.21.15.116:0/1799910917 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f922c0a5a10 0x7f922c0a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f9231ce7700 1 -- 172.21.15.116:0/1799910917 >> 172.21.15.116:0/1799910917 conn(0x7f922c062520 msgr2=0x7f922c062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:13.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f9231ce7700 1 -- 172.21.15.116:0/1799910917 shutdown_connections 2020-11-20T12:17:13.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.312+0000 7f9231ce7700 1 -- 172.21.15.116:0/1799910917 wait complete. 2020-11-20T12:17:13.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f9231ce7700 1 Processor -- start 2020-11-20T12:17:13.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f9231ce7700 1 -- start start 2020-11-20T12:17:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f9231ce7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f922c0a5a10 0x7f922c157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f9231ce7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f922c0c41c0 0x7f922c1578f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f9231ce7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f922c138670 0x7f922c1551a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f9231ce7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c157de0 con 0x7f922c138670 2020-11-20T12:17:13.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f9231ce7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922c154000 con 0x7f922c0a5a10 2020-11-20T12:17:13.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f9231ce7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922c1542a0 con 0x7f922c0c41c0 2020-11-20T12:17:13.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f92314e6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f922c138670 0x7f922c1551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f92314e6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f922c138670 0x7f922c1551a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44974/0 (socket says 172.21.15.116:44974) 2020-11-20T12:17:13.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f92314e6700 1 -- 172.21.15.116:0/464800428 learned_addr learned my addr 172.21.15.116:0/464800428 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:13.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f9230ce5700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f922c0a5a10 0x7f922c157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f922bfff700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f922c0c41c0 0x7f922c1578f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f92314e6700 1 -- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f922c0c41c0 msgr2=0x7f922c1578f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:13.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f92314e6700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f922c0c41c0 0x7f922c1578f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f92314e6700 1 -- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f922c0a5a10 msgr2=0x7f922c157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:13.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.316+0000 7f92314e6700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f922c0a5a10 0x7f922c157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.320+0000 7f92314e6700 1 -- 172.21.15.116:0/464800428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c154540 con 0x7f922c138670 2020-11-20T12:17:13.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.320+0000 7f9230ce5700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f922c0a5a10 0x7f922c157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:13.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.320+0000 7f92314e6700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f922c138670 0x7f922c1551a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f9224009fd0 tx=0x7f9224016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:13.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.320+0000 7f9228ff9700 1 -- 172.21.15.116:0/464800428 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224010780 con 0x7f922c138670 2020-11-20T12:17:13.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.320+0000 7f9228ff9700 1 -- 172.21.15.116:0/464800428 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92240108e0 con 0x7f922c138670 2020-11-20T12:17:13.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.320+0000 7f9231ce7700 1 -- 172.21.15.116:0/464800428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922c1547e0 con 0x7f922c138670 2020-11-20T12:17:13.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.320+0000 7f9228ff9700 1 -- 172.21.15.116:0/464800428 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224010c60 con 0x7f922c138670 2020-11-20T12:17:13.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.320+0000 7f9231ce7700 1 -- 172.21.15.116:0/464800428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f922c154d20 con 0x7f922c138670 2020-11-20T12:17:13.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.320+0000 7f9231ce7700 1 -- 172.21.15.116:0/464800428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c154ee0 con 0x7f922c138670 2020-11-20T12:17:13.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.320+0000 7f9228ff9700 1 -- 172.21.15.116:0/464800428 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9224017070 con 0x7f922c138670 2020-11-20T12:17:13.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.324+0000 7f9228ff9700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92040557c0 0x7f9204057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.324+0000 7f9228ff9700 1 -- 172.21.15.116:0/464800428 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9224070cc0 con 0x7f922c138670 2020-11-20T12:17:13.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.324+0000 7f9230ce5700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92040557c0 0x7f9204057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.324+0000 7f9230ce5700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92040557c0 0x7f9204057c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f922000bac0 tx=0x7f9220008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:13.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.332+0000 7f9228ff9700 1 -- 172.21.15.116:0/464800428 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9224077080 con 0x7f922c138670 2020-11-20T12:17:13.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.492+0000 7f9231ce7700 1 -- 172.21.15.116:0/464800428 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f922c044f60 con 0x7f922c138670 2020-11-20T12:17:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9228ff9700 1 -- 172.21.15.116:0/464800428 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f9224015030 con 0x7f922c138670 2020-11-20T12:17:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 -- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92040557c0 msgr2=0x7f9204057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:13.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92040557c0 0x7f9204057c80 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f922000bac0 tx=0x7f9220008040).stop 2020-11-20T12:17:13.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 -- 172.21.15.116:0/464800428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f922c138670 msgr2=0x7f922c1551a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:13.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f922c138670 0x7f922c1551a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f9224009fd0 tx=0x7f9224016000).stop 2020-11-20T12:17:13.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 -- 172.21.15.116:0/464800428 shutdown_connections 2020-11-20T12:17:13.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92040557c0 0x7f9204057c80 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f922c138670 0x7f922c1551a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f922c0a5a10 0x7f922c157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 --2- 172.21.15.116:0/464800428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f922c0c41c0 0x7f922c1578f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 -- 172.21.15.116:0/464800428 >> 172.21.15.116:0/464800428 conn(0x7f922c062520 msgr2=0x7f922c0c3250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 -- 172.21.15.116:0/464800428 shutdown_connections 2020-11-20T12:17:13.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.496+0000 7f9231ce7700 1 -- 172.21.15.116:0/464800428 wait complete. 2020-11-20T12:17:13.505 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:13.505 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:58.907407+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:16:55.781578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T12:16:56.900539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:13.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:16:55.781578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:16:56.900539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:17:13.521 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:13.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.692+0000 7efd80d2b700 1 Processor -- start 2020-11-20T12:17:13.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.692+0000 7efd80d2b700 1 -- start start 2020-11-20T12:17:13.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd80d2b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7efd7c060e00 0x7efd7c14dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd80d2b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd7c0ba9f0 0x7efd7c063e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd80d2b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7c0b9b10 con 0x7efd7c14d930 2020-11-20T12:17:13.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd80d2b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7c0bf560 con 0x7efd7c060e00 2020-11-20T12:17:13.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd80d2b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c0b9d70 con 0x7efd7c0ba9f0 2020-11-20T12:17:13.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd7b7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7efd7c14d930 0x7efd7c0bee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54412/0 (socket says 172.21.15.116:54412) 2020-11-20T12:17:13.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd7b7fe700 1 -- 172.21.15.116:0/3367376152 learned_addr learned my addr 172.21.15.116:0/3367376152 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:13.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd73fff700 1 --2- 172.21.15.116:0/3367376152 >> v2:172.21.15.173:3300/0 conn(0x7efd7c060e00 0x7efd7c14dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd7bfff700 1 --2- 172.21.15.116:0/3367376152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd7c0ba9f0 0x7efd7c063e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd7affd700 1 -- 172.21.15.116:0/3367376152 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386812071 0 0) 0x7efd7c0b9b10 con 0x7efd7c14d930 2020-11-20T12:17:13.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd7affd700 1 -- 172.21.15.116:0/3367376152 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd5c003240 con 0x7efd7c14d930 2020-11-20T12:17:13.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd73fff700 1 -- 172.21.15.116:0/3367376152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd7c0ba9f0 msgr2=0x7efd7c063e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:13.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd73fff700 1 --2- 172.21.15.116:0/3367376152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd7c0ba9f0 0x7efd7c063e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd73fff700 1 -- 172.21.15.116:0/3367376152 >> v1:172.21.15.116:6789/0 conn(0x7efd7c14d930 legacy=0x7efd7c0bee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:13.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd73fff700 1 -- 172.21.15.116:0/3367376152 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c0ba010 con 0x7efd7c060e00 2020-11-20T12:17:13.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd73fff700 1 --2- 172.21.15.116:0/3367376152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd7c060e00 0x7efd7c14dd10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7efd6400b9a0 tx=0x7efd64006df0).ready entity=mon.1 client_cookie=3f6cf7df185ba24 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:13.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd7affd700 1 -- 172.21.15.116:0/3367376152 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64009680 con 0x7efd7c060e00 2020-11-20T12:17:13.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd7affd700 1 -- 172.21.15.116:0/3367376152 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd640097e0 con 0x7efd7c060e00 2020-11-20T12:17:13.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.696+0000 7efd7affd700 1 -- 172.21.15.116:0/3367376152 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64009b60 con 0x7efd7c060e00 2020-11-20T12:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.700+0000 7efd80d2b700 1 -- 172.21.15.116:0/3367376152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd7c060e00 msgr2=0x7efd7c14dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.700+0000 7efd80d2b700 1 --2- 172.21.15.116:0/3367376152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd7c060e00 0x7efd7c14dd10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7efd6400b9a0 tx=0x7efd64006df0).stop 2020-11-20T12:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.700+0000 7efd80d2b700 1 -- 172.21.15.116:0/3367376152 shutdown_connections 2020-11-20T12:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.700+0000 7efd80d2b700 1 --2- 172.21.15.116:0/3367376152 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd7c0ba9f0 0x7efd7c063e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.700+0000 7efd80d2b700 1 --2- 172.21.15.116:0/3367376152 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd7c060e00 0x7efd7c14dd10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.700+0000 7efd80d2b700 1 -- 172.21.15.116:0/3367376152 >> 172.21.15.116:0/3367376152 conn(0x7efd7c0c44b0 msgr2=0x7efd7c0c48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:13.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.700+0000 7efd80d2b700 1 -- 172.21.15.116:0/3367376152 shutdown_connections 2020-11-20T12:17:13.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.700+0000 7efd80d2b700 1 -- 172.21.15.116:0/3367376152 wait complete. 2020-11-20T12:17:13.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.700+0000 7efd80d2b700 1 Processor -- start 2020-11-20T12:17:13.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.700+0000 7efd80d2b700 1 -- start start 2020-11-20T12:17:13.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd80d2b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd7c0ba9f0 0x7efd7c1626c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd80d2b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c14d930 0x7efd7c0a9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd80d2b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd7c0a9b60 0x7efd7c1614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd80d2b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c162490 con 0x7efd7c14d930 2020-11-20T12:17:13.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd80d2b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c161f50 con 0x7efd7c0a9b60 2020-11-20T12:17:13.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd80d2b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c1621f0 con 0x7efd7c0ba9f0 2020-11-20T12:17:13.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd73fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c14d930 0x7efd7c0a9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd73fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c14d930 0x7efd7c0a9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:44988/0 (socket says 172.21.15.116:44988) 2020-11-20T12:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd73fff700 1 -- 172.21.15.116:0/2087724857 learned_addr learned my addr 172.21.15.116:0/2087724857 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd7b7fe700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd7c0ba9f0 0x7efd7c1626c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd7bfff700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd7c0a9b60 0x7efd7c1614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd73fff700 1 -- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd7c0ba9f0 msgr2=0x7efd7c1626c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:13.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd73fff700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd7c0ba9f0 0x7efd7c1626c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd73fff700 1 -- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd7c0a9b60 msgr2=0x7efd7c1614f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:13.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd73fff700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd7c0a9b60 0x7efd7c1614f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd73fff700 1 -- 172.21.15.116:0/2087724857 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c143d50 con 0x7efd7c14d930 2020-11-20T12:17:13.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd73fff700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c14d930 0x7efd7c0a9670 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7efd6400b150 tx=0x7efd64015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:13.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd737fe700 1 -- 172.21.15.116:0/2087724857 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd64011b70 con 0x7efd7c14d930 2020-11-20T12:17:13.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd737fe700 1 -- 172.21.15.116:0/2087724857 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd64011cd0 con 0x7efd7c14d930 2020-11-20T12:17:13.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd80d2b700 1 -- 172.21.15.116:0/2087724857 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c143ff0 con 0x7efd7c14d930 2020-11-20T12:17:13.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd737fe700 1 -- 172.21.15.116:0/2087724857 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6401e670 con 0x7efd7c14d930 2020-11-20T12:17:13.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.704+0000 7efd80d2b700 1 -- 172.21.15.116:0/2087724857 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c1619e0 con 0x7efd7c14d930 2020-11-20T12:17:13.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.708+0000 7efd80d2b700 1 -- 172.21.15.116:0/2087724857 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7c144180 con 0x7efd7c14d930 2020-11-20T12:17:13.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.708+0000 7efd737fe700 1 -- 172.21.15.116:0/2087724857 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7efd64025030 con 0x7efd7c14d930 2020-11-20T12:17:13.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.708+0000 7efd737fe700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efd580557c0 0x7efd58057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:13.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.708+0000 7efd737fe700 1 -- 172.21.15.116:0/2087724857 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efd6406fcf0 con 0x7efd7c14d930 2020-11-20T12:17:13.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.708+0000 7efd7b7fe700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efd580557c0 0x7efd58057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:13.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.712+0000 7efd7b7fe700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efd580557c0 0x7efd58057c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7efd6c007fd0 tx=0x7efd6c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:13.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.712+0000 7efd737fe700 1 -- 172.21.15.116:0/2087724857 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efd64045b10 con 0x7efd7c14d930 2020-11-20T12:17:13.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.872+0000 7efd80d2b700 1 -- 172.21.15.116:0/2087724857 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7efd7c044f60 con 0x7efd7c14d930 2020-11-20T12:17:13.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd737fe700 1 -- 172.21.15.116:0/2087724857 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v34) v1 ==== 110+0+10 (secure 0 0 0) 0x7efd64023020 con 0x7efd7c14d930 2020-11-20T12:17:13.875 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:13.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 -- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efd580557c0 msgr2=0x7efd58057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:13.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efd580557c0 0x7efd58057c80 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7efd6c007fd0 tx=0x7efd6c009040).stop 2020-11-20T12:17:13.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 -- 172.21.15.116:0/2087724857 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c14d930 msgr2=0x7efd7c0a9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:13.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c14d930 0x7efd7c0a9670 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7efd6400b150 tx=0x7efd64015000).stop 2020-11-20T12:17:13.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 -- 172.21.15.116:0/2087724857 shutdown_connections 2020-11-20T12:17:13.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efd580557c0 0x7efd58057c80 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efd7c0a9b60 0x7efd7c1614f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efd7c0ba9f0 0x7efd7c1626c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 --2- 172.21.15.116:0/2087724857 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efd7c14d930 0x7efd7c0a9670 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:13.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 -- 172.21.15.116:0/2087724857 >> 172.21.15.116:0/2087724857 conn(0x7efd7c0c44b0 msgr2=0x7efd7c0bf840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:13.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 -- 172.21.15.116:0/2087724857 shutdown_connections 2020-11-20T12:17:13.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:13.876+0000 7efd80d2b700 1 -- 172.21.15.116:0/2087724857 wait complete. 2020-11-20T12:17:13.902 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:14.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.080+0000 7f587a039700 1 Processor -- start 2020-11-20T12:17:14.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.084+0000 7f587a039700 1 -- start start 2020-11-20T12:17:14.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.084+0000 7f587a039700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f587413b110 0x7f5874151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.084+0000 7f587a039700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5874151fa0 0x7f5874061250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.084+0000 7f587a039700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5874152a80 con 0x7f587413ad30 2020-11-20T12:17:14.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f587a039700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5874152540 con 0x7f587413b110 2020-11-20T12:17:14.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f587a039700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58741527e0 con 0x7f5874151fa0 2020-11-20T12:17:14.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5879037700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f587413ad30 0x7f58740a7fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54426/0 (socket says 172.21.15.116:54426) 2020-11-20T12:17:14.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5879037700 1 -- 172.21.15.116:0/1461880110 learned_addr learned my addr 172.21.15.116:0/1461880110 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:14.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5878836700 1 --2- 172.21.15.116:0/1461880110 >> v2:172.21.15.116:3300/0 conn(0x7f587413b110 0x7f5874151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5879838700 1 --2- 172.21.15.116:0/1461880110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5874151fa0 0x7f5874061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5863fff700 1 -- 172.21.15.116:0/1461880110 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1353828003 0 0) 0x7f5874152a80 con 0x7f587413ad30 2020-11-20T12:17:14.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5863fff700 1 -- 172.21.15.116:0/1461880110 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f585c003240 con 0x7f587413ad30 2020-11-20T12:17:14.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5878836700 1 -- 172.21.15.116:0/1461880110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5874151fa0 msgr2=0x7f5874061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:14.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5878836700 1 --2- 172.21.15.116:0/1461880110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5874151fa0 0x7f5874061250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5878836700 1 -- 172.21.15.116:0/1461880110 >> v1:172.21.15.116:6789/0 conn(0x7f587413ad30 legacy=0x7f58740a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5878836700 1 -- 172.21.15.116:0/1461880110 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58740ac910 con 0x7f587413b110 2020-11-20T12:17:14.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5878836700 1 --2- 172.21.15.116:0/1461880110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f587413b110 0x7f5874151ab0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f586400b9a0 tx=0x7f5864006df0).ready entity=mon.0 client_cookie=601f31823f5b806b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:14.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5863fff700 1 -- 172.21.15.116:0/1461880110 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864009710 con 0x7f587413b110 2020-11-20T12:17:14.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5863fff700 1 -- 172.21.15.116:0/1461880110 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5864009870 con 0x7f587413b110 2020-11-20T12:17:14.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f5863fff700 1 -- 172.21.15.116:0/1461880110 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864009bf0 con 0x7f587413b110 2020-11-20T12:17:14.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f587a039700 1 -- 172.21.15.116:0/1461880110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f587413b110 msgr2=0x7f5874151ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f587a039700 1 --2- 172.21.15.116:0/1461880110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f587413b110 0x7f5874151ab0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f586400b9a0 tx=0x7f5864006df0).stop 2020-11-20T12:17:14.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f587a039700 1 -- 172.21.15.116:0/1461880110 shutdown_connections 2020-11-20T12:17:14.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f587a039700 1 --2- 172.21.15.116:0/1461880110 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5874151fa0 0x7f5874061250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f587a039700 1 --2- 172.21.15.116:0/1461880110 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f587413b110 0x7f5874151ab0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f587a039700 1 -- 172.21.15.116:0/1461880110 >> 172.21.15.116:0/1461880110 conn(0x7f58740678a0 msgr2=0x7f5874067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:14.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.088+0000 7f587a039700 1 -- 172.21.15.116:0/1461880110 shutdown_connections 2020-11-20T12:17:14.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f587a039700 1 -- 172.21.15.116:0/1461880110 wait complete. 2020-11-20T12:17:14.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f587a039700 1 Processor -- start 2020-11-20T12:17:14.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f587a039700 1 -- start start 2020-11-20T12:17:14.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f587a039700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f587413ad30 0x7f58740a3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f587a039700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5874151fa0 0x7f58740a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f587a039700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58740a4400 0x7f58740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f587a039700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58740a6740 con 0x7f5874151fa0 2020-11-20T12:17:14.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f587a039700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58740a69e0 con 0x7f587413ad30 2020-11-20T12:17:14.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f587a039700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58740a6c80 con 0x7f58740a4400 2020-11-20T12:17:14.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f5878836700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5874151fa0 0x7f58740a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f5878836700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5874151fa0 0x7f58740a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45000/0 (socket says 172.21.15.116:45000) 2020-11-20T12:17:14.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f5878836700 1 -- 172.21.15.116:0/414423076 learned_addr learned my addr 172.21.15.116:0/414423076 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:14.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f5879838700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58740a4400 0x7f58740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.092+0000 7f5879037700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f587413ad30 0x7f58740a3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f5878836700 1 -- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58740a4400 msgr2=0x7f58740a60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f5878836700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58740a4400 0x7f58740a60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f5878836700 1 -- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f587413ad30 msgr2=0x7f58740a3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f5878836700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f587413ad30 0x7f58740a3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f5878836700 1 -- 172.21.15.116:0/414423076 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587409f190 con 0x7f5874151fa0 2020-11-20T12:17:14.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f5878836700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5874151fa0 0x7f58740a5c00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f5864006e30 tx=0x7f5864015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:14.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f58617fa700 1 -- 172.21.15.116:0/414423076 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5864011c80 con 0x7f5874151fa0 2020-11-20T12:17:14.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f58617fa700 1 -- 172.21.15.116:0/414423076 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5864011de0 con 0x7f5874151fa0 2020-11-20T12:17:14.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f58617fa700 1 -- 172.21.15.116:0/414423076 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586401e690 con 0x7f5874151fa0 2020-11-20T12:17:14.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f587a039700 1 -- 172.21.15.116:0/414423076 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58740a17a0 con 0x7f5874151fa0 2020-11-20T12:17:14.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f587a039700 1 -- 172.21.15.116:0/414423076 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5874150e20 con 0x7f5874151fa0 2020-11-20T12:17:14.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f587a039700 1 -- 172.21.15.116:0/414423076 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f587409f320 con 0x7f5874151fa0 2020-11-20T12:17:14.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.096+0000 7f58617fa700 1 -- 172.21.15.116:0/414423076 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f586401a040 con 0x7f5874151fa0 2020-11-20T12:17:14.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.100+0000 7f58617fa700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f584c0557c0 0x7f584c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.100+0000 7f58617fa700 1 -- 172.21.15.116:0/414423076 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f586406f9a0 con 0x7f5874151fa0 2020-11-20T12:17:14.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.100+0000 7f5879037700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f584c0557c0 0x7f584c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.100+0000 7f5879037700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f584c0557c0 0x7f584c057c80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f58700055a0 tx=0x7f5870009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:14.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.104+0000 7f58617fa700 1 -- 172.21.15.116:0/414423076 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f58640458f0 con 0x7f5874151fa0 2020-11-20T12:17:14.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.268+0000 7f587a039700 1 -- 172.21.15.116:0/414423076 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f58740a6e40 con 0x7f5874151fa0 2020-11-20T12:17:14.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.268+0000 7f58617fa700 1 -- 172.21.15.116:0/414423076 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5864023070 con 0x7f5874151fa0 2020-11-20T12:17:14.270 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:14.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 -- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f584c0557c0 msgr2=0x7f584c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f584c0557c0 0x7f584c057c80 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f58700055a0 tx=0x7f5870009040).stop 2020-11-20T12:17:14.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 -- 172.21.15.116:0/414423076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5874151fa0 msgr2=0x7f58740a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5874151fa0 0x7f58740a5c00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f5864006e30 tx=0x7f5864015000).stop 2020-11-20T12:17:14.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 -- 172.21.15.116:0/414423076 shutdown_connections 2020-11-20T12:17:14.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f584c0557c0 0x7f584c057c80 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58740a4400 0x7f58740a60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f587413ad30 0x7f58740a3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 --2- 172.21.15.116:0/414423076 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5874151fa0 0x7f58740a5c00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 -- 172.21.15.116:0/414423076 >> 172.21.15.116:0/414423076 conn(0x7f58740678a0 msgr2=0x7f5874138a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:14.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 -- 172.21.15.116:0/414423076 shutdown_connections 2020-11-20T12:17:14.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.272+0000 7f587a039700 1 -- 172.21.15.116:0/414423076 wait complete. 2020-11-20T12:17:14.298 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:17:14.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.480+0000 7f7a6dddd700 1 Processor -- start 2020-11-20T12:17:14.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.484+0000 7f7a6dddd700 1 -- start start 2020-11-20T12:17:14.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.484+0000 7f7a6dddd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7a68063e30 0x7f7a68064290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.484+0000 7f7a6dddd700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a68064b40 con 0x7f7a6814d980 2020-11-20T12:17:14.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.484+0000 7f7a6dddd700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a68064de0 con 0x7f7a680bf230 2020-11-20T12:17:14.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.484+0000 7f7a6dddd700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a680655c0 con 0x7f7a68063e30 2020-11-20T12:17:14.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.484+0000 7f7a6cddb700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7a6814d980 0x7f7a68061180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54440/0 (socket says 172.21.15.116:54440) 2020-11-20T12:17:14.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.484+0000 7f7a6cddb700 1 -- 172.21.15.116:0/3967677968 learned_addr learned my addr 172.21.15.116:0/3967677968 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:14.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.484+0000 7f7a67fff700 1 --2- 172.21.15.116:0/3967677968 >> v2:172.21.15.173:3300/0 conn(0x7f7a68063e30 0x7f7a68064290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a677fe700 1 -- 172.21.15.116:0/3967677968 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1152638196 0 0) 0x7f7a68064b40 con 0x7f7a6814d980 2020-11-20T12:17:14.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a677fe700 1 -- 172.21.15.116:0/3967677968 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a4c004350 con 0x7f7a6814d980 2020-11-20T12:17:14.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a67fff700 1 -- 172.21.15.116:0/3967677968 >> v1:172.21.15.173:6789/0 conn(0x7f7a680bf230 legacy=0x7f7a680bf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a67fff700 1 -- 172.21.15.116:0/3967677968 >> v1:172.21.15.116:6789/0 conn(0x7f7a6814d980 legacy=0x7f7a68061180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a67fff700 1 -- 172.21.15.116:0/3967677968 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a68065080 con 0x7f7a68063e30 2020-11-20T12:17:14.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a67fff700 1 --2- 172.21.15.116:0/3967677968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a68063e30 0x7f7a68064290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f7a5400b9a0 tx=0x7f7a54006df0).ready entity=mon.1 client_cookie=53016f323a5257d7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:14.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a677fe700 1 -- 172.21.15.116:0/3967677968 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a540097b0 con 0x7f7a68063e30 2020-11-20T12:17:14.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a677fe700 1 -- 172.21.15.116:0/3967677968 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a54009910 con 0x7f7a68063e30 2020-11-20T12:17:14.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a677fe700 1 -- 172.21.15.116:0/3967677968 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a54009c90 con 0x7f7a68063e30 2020-11-20T12:17:14.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a6dddd700 1 -- 172.21.15.116:0/3967677968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a68063e30 msgr2=0x7f7a68064290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a6dddd700 1 --2- 172.21.15.116:0/3967677968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a68063e30 0x7f7a68064290 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f7a5400b9a0 tx=0x7f7a54006df0).stop 2020-11-20T12:17:14.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a6dddd700 1 -- 172.21.15.116:0/3967677968 shutdown_connections 2020-11-20T12:17:14.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a6dddd700 1 --2- 172.21.15.116:0/3967677968 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a68063e30 0x7f7a68064290 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a6dddd700 1 -- 172.21.15.116:0/3967677968 >> 172.21.15.116:0/3967677968 conn(0x7f7a680c1e60 msgr2=0x7f7a680669d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:14.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a6dddd700 1 -- 172.21.15.116:0/3967677968 shutdown_connections 2020-11-20T12:17:14.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.488+0000 7f7a6dddd700 1 -- 172.21.15.116:0/3967677968 wait complete. 2020-11-20T12:17:14.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.492+0000 7f7a6dddd700 1 Processor -- start 2020-11-20T12:17:14.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.492+0000 7f7a6dddd700 1 -- start start 2020-11-20T12:17:14.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6dddd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a68063e30 0x7f7a680bd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6dddd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a680bf230 0x7f7a680a7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6dddd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a6814d980 0x7f7a68160290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6dddd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680be8d0 con 0x7f7a68063e30 2020-11-20T12:17:14.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6dddd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680bddd0 con 0x7f7a6814d980 2020-11-20T12:17:14.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6dddd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680be390 con 0x7f7a680bf230 2020-11-20T12:17:14.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6cddb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a68063e30 0x7f7a680bd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6cddb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a68063e30 0x7f7a680bd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45012/0 (socket says 172.21.15.116:45012) 2020-11-20T12:17:14.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6cddb700 1 -- 172.21.15.116:0/236134658 learned_addr learned my addr 172.21.15.116:0/236134658 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:14.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a67fff700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a680bf230 0x7f7a680a7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6d5dc700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a6814d980 0x7f7a68160290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6cddb700 1 -- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a680bf230 msgr2=0x7f7a680a7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6cddb700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a680bf230 0x7f7a680a7970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6cddb700 1 -- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a6814d980 msgr2=0x7f7a68160290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6cddb700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a6814d980 0x7f7a68160290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6cddb700 1 -- 172.21.15.116:0/236134658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680be630 con 0x7f7a68063e30 2020-11-20T12:17:14.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a67fff700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a680bf230 0x7f7a680a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:14.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6cddb700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a68063e30 0x7f7a680bd7b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f7a5c00c710 tx=0x7f7a5c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:14.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a64ff9700 1 -- 172.21.15.116:0/236134658 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c0072e0 con 0x7f7a68063e30 2020-11-20T12:17:14.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a64ff9700 1 -- 172.21.15.116:0/236134658 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5c007440 con 0x7f7a68063e30 2020-11-20T12:17:14.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6dddd700 1 -- 172.21.15.116:0/236134658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a680be0d0 con 0x7f7a68063e30 2020-11-20T12:17:14.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a64ff9700 1 -- 172.21.15.116:0/236134658 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5c005800 con 0x7f7a68063e30 2020-11-20T12:17:14.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6dddd700 1 -- 172.21.15.116:0/236134658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a68160780 con 0x7f7a68063e30 2020-11-20T12:17:14.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.496+0000 7f7a6dddd700 1 -- 172.21.15.116:0/236134658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a68044f60 con 0x7f7a68063e30 2020-11-20T12:17:14.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.500+0000 7f7a64ff9700 1 -- 172.21.15.116:0/236134658 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7a5c01a060 con 0x7f7a68063e30 2020-11-20T12:17:14.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.500+0000 7f7a64ff9700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a400557c0 0x7f7a40057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.500+0000 7f7a64ff9700 1 -- 172.21.15.116:0/236134658 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7a5c06dfc0 con 0x7f7a68063e30 2020-11-20T12:17:14.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.500+0000 7f7a67fff700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a400557c0 0x7f7a40057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.500+0000 7f7a67fff700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a400557c0 0x7f7a40057c80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f7a54006e30 tx=0x7f7a54015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:14.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.504+0000 7f7a64ff9700 1 -- 172.21.15.116:0/236134658 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7a5c043ee0 con 0x7f7a68063e30 2020-11-20T12:17:14.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.664+0000 7f7a6dddd700 1 -- 172.21.15.116:0/236134658 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a68160970 con 0x7f7a68063e30 2020-11-20T12:17:14.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.664+0000 7f7a64ff9700 1 -- 172.21.15.116:0/236134658 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7a5c018070 con 0x7f7a68063e30 2020-11-20T12:17:14.666 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:14.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 -- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a400557c0 msgr2=0x7f7a40057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a400557c0 0x7f7a40057c80 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f7a54006e30 tx=0x7f7a54015000).stop 2020-11-20T12:17:14.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 -- 172.21.15.116:0/236134658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a68063e30 msgr2=0x7f7a680bd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a68063e30 0x7f7a680bd7b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f7a5c00c710 tx=0x7f7a5c007ad0).stop 2020-11-20T12:17:14.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 -- 172.21.15.116:0/236134658 shutdown_connections 2020-11-20T12:17:14.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a400557c0 0x7f7a40057c80 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a6814d980 0x7f7a68160290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a68063e30 0x7f7a680bd7b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 --2- 172.21.15.116:0/236134658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a680bf230 0x7f7a680a7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 -- 172.21.15.116:0/236134658 >> 172.21.15.116:0/236134658 conn(0x7f7a680c1e60 msgr2=0x7f7a6814cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:14.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 -- 172.21.15.116:0/236134658 shutdown_connections 2020-11-20T12:17:14.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.668+0000 7f7a6dddd700 1 -- 172.21.15.116:0/236134658 wait complete. 2020-11-20T12:17:14.689 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:17:14.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.876+0000 7fc25e76d700 1 Processor -- start 2020-11-20T12:17:14.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.876+0000 7fc25e76d700 1 -- start start 2020-11-20T12:17:14.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.876+0000 7fc25e76d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc25813ad30 0x7fc2580a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.876+0000 7fc25e76d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc25813b110 0x7fc258151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.876+0000 7fc25e76d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc258152a80 con 0x7fc258151fa0 2020-11-20T12:17:14.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.876+0000 7fc25e76d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc258152540 con 0x7fc25813ad30 2020-11-20T12:17:14.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.876+0000 7fc25e76d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2581527e0 con 0x7fc25813b110 2020-11-20T12:17:14.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc25d76b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc25813ad30 0x7fc2580a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc25cf6a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc25813b110 0x7fc258151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc25d76b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc25813ad30 0x7fc2580a7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45022/0 (socket says 172.21.15.116:45022) 2020-11-20T12:17:14.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc25df6c700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fc258151fa0 0x7fc258061250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:50688/0 (socket says 172.21.15.116:50688) 2020-11-20T12:17:14.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc25d76b700 1 -- 172.21.15.116:0/690247575 learned_addr learned my addr 172.21.15.116:0/690247575 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:14.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc25d76b700 1 -- 172.21.15.116:0/690247575 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc25813b110 msgr2=0x7fc258151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:14.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc25d76b700 1 --2- 172.21.15.116:0/690247575 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc25813b110 0x7fc258151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc25d76b700 1 -- 172.21.15.116:0/690247575 >> v1:172.21.15.173:6789/0 conn(0x7fc258151fa0 legacy=0x7fc258061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc25d76b700 1 -- 172.21.15.116:0/690247575 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2580ac910 con 0x7fc25813ad30 2020-11-20T12:17:14.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc25d76b700 1 --2- 172.21.15.116:0/690247575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc25813ad30 0x7fc2580a7fb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fc254006fd0 tx=0x7fc254006eb0).ready entity=mon.0 client_cookie=783c9491c18b6bc8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:14.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc247fff700 1 -- 172.21.15.116:0/690247575 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2540098a0 con 0x7fc25813ad30 2020-11-20T12:17:14.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc247fff700 1 -- 172.21.15.116:0/690247575 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254009a00 con 0x7fc25813ad30 2020-11-20T12:17:14.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.880+0000 7fc247fff700 1 -- 172.21.15.116:0/690247575 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254009d80 con 0x7fc25813ad30 2020-11-20T12:17:14.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.884+0000 7fc25e76d700 1 -- 172.21.15.116:0/690247575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc25813ad30 msgr2=0x7fc2580a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.884+0000 7fc25e76d700 1 --2- 172.21.15.116:0/690247575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc25813ad30 0x7fc2580a7fb0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fc254006fd0 tx=0x7fc254006eb0).stop 2020-11-20T12:17:14.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.884+0000 7fc25e76d700 1 -- 172.21.15.116:0/690247575 shutdown_connections 2020-11-20T12:17:14.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.884+0000 7fc25e76d700 1 --2- 172.21.15.116:0/690247575 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc25813ad30 0x7fc2580a7fb0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.884+0000 7fc25e76d700 1 --2- 172.21.15.116:0/690247575 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc25813b110 0x7fc258151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.884+0000 7fc25e76d700 1 -- 172.21.15.116:0/690247575 >> 172.21.15.116:0/690247575 conn(0x7fc2580678a0 msgr2=0x7fc258067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:14.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.884+0000 7fc25e76d700 1 -- 172.21.15.116:0/690247575 shutdown_connections 2020-11-20T12:17:14.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.884+0000 7fc25e76d700 1 -- 172.21.15.116:0/690247575 wait complete. 2020-11-20T12:17:14.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.884+0000 7fc25e76d700 1 Processor -- start 2020-11-20T12:17:14.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.884+0000 7fc25e76d700 1 -- start start 2020-11-20T12:17:14.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25e76d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc25813ad30 0x7fc2580a6830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25e76d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc25813b110 0x7fc258161640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25e76d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc258151fa0 0x7fc258164350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25e76d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc258165210 con 0x7fc25813b110 2020-11-20T12:17:14.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25e76d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc258164cd0 con 0x7fc25813ad30 2020-11-20T12:17:14.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25e76d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc258164f70 con 0x7fc258151fa0 2020-11-20T12:17:14.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25cf6a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc25813b110 0x7fc258161640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25cf6a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc25813b110 0x7fc258161640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45026/0 (socket says 172.21.15.116:45026) 2020-11-20T12:17:14.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25cf6a700 1 -- 172.21.15.116:0/27870281 learned_addr learned my addr 172.21.15.116:0/27870281 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:14.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25d76b700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc25813ad30 0x7fc2580a6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25df6c700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc258151fa0 0x7fc258164350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25cf6a700 1 -- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc258151fa0 msgr2=0x7fc258164350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25cf6a700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc258151fa0 0x7fc258164350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25cf6a700 1 -- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc25813ad30 msgr2=0x7fc2580a6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:14.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25cf6a700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc25813ad30 0x7fc2580a6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:14.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25cf6a700 1 -- 172.21.15.116:0/27870281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc258160f00 con 0x7fc25813b110 2020-11-20T12:17:14.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25cf6a700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc25813b110 0x7fc258161640 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fc24800ebd0 tx=0x7fc248006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:14.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc2457fa700 1 -- 172.21.15.116:0/27870281 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24800ca40 con 0x7fc25813b110 2020-11-20T12:17:14.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc2457fa700 1 -- 172.21.15.116:0/27870281 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc24800cba0 con 0x7fc25813b110 2020-11-20T12:17:14.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25e76d700 1 -- 172.21.15.116:0/27870281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2581611a0 con 0x7fc25813b110 2020-11-20T12:17:14.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc2457fa700 1 -- 172.21.15.116:0/27870281 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2480123f0 con 0x7fc25813b110 2020-11-20T12:17:14.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.888+0000 7fc25e76d700 1 -- 172.21.15.116:0/27870281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2580c0e80 con 0x7fc25813b110 2020-11-20T12:17:14.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.892+0000 7fc25e76d700 1 -- 172.21.15.116:0/27870281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc258044f60 con 0x7fc25813b110 2020-11-20T12:17:14.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.892+0000 7fc2457fa700 1 -- 172.21.15.116:0/27870281 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc248012760 con 0x7fc25813b110 2020-11-20T12:17:14.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.892+0000 7fc2457fa700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2300557c0 0x7fc230057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:14.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.892+0000 7fc2457fa700 1 -- 172.21.15.116:0/27870281 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc24806ecd0 con 0x7fc25813b110 2020-11-20T12:17:14.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.892+0000 7fc25d76b700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2300557c0 0x7fc230057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:14.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.896+0000 7fc25d76b700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2300557c0 0x7fc230057c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fc2581517e0 tx=0x7fc254015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:14.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:14.896+0000 7fc2457fa700 1 -- 172.21.15.116:0/27870281 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc248044a60 con 0x7fc25813b110 2020-11-20T12:17:15.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.056+0000 7fc25e76d700 1 -- 172.21.15.116:0/27870281 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc258069320 con 0x7fc25813b110 2020-11-20T12:17:15.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.056+0000 7fc2457fa700 1 -- 172.21.15.116:0/27870281 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc248048080 con 0x7fc25813b110 2020-11-20T12:17:15.058 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:15.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 -- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2300557c0 msgr2=0x7fc230057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:15.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2300557c0 0x7fc230057c80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fc2581517e0 tx=0x7fc254015000).stop 2020-11-20T12:17:15.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 -- 172.21.15.116:0/27870281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc25813b110 msgr2=0x7fc258161640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:15.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc25813b110 0x7fc258161640 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fc24800ebd0 tx=0x7fc248006df0).stop 2020-11-20T12:17:15.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 -- 172.21.15.116:0/27870281 shutdown_connections 2020-11-20T12:17:15.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2300557c0 0x7fc230057c80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc258151fa0 0x7fc258164350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc25813ad30 0x7fc2580a6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 --2- 172.21.15.116:0/27870281 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc25813b110 0x7fc258161640 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 -- 172.21.15.116:0/27870281 >> 172.21.15.116:0/27870281 conn(0x7fc2580678a0 msgr2=0x7fc2580c5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:15.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 -- 172.21.15.116:0/27870281 shutdown_connections 2020-11-20T12:17:15.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.060+0000 7fc25e76d700 1 -- 172.21.15.116:0/27870281 wait complete. 2020-11-20T12:17:15.084 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:17:15.085 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.264+0000 7fd4721fd700 1 Processor -- start 2020-11-20T12:17:15.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.264+0000 7fd4721fd700 1 -- start start 2020-11-20T12:17:15.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.264+0000 7fd4721fd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd46c138d80 0x7fd46c15a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:15.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.264+0000 7fd4721fd700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46c0a8c80 con 0x7fd46c0c41c0 2020-11-20T12:17:15.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.264+0000 7fd4721fd700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46c0a9180 con 0x7fd46c0c45a0 2020-11-20T12:17:15.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.264+0000 7fd4721fd700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46c0a9420 con 0x7fd46c138d80 2020-11-20T12:17:15.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.264+0000 7fd4711fb700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd46c0c41c0 0x7fd46c0a5a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54468/0 (socket says 172.21.15.116:54468) 2020-11-20T12:17:15.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.264+0000 7fd4711fb700 1 -- 172.21.15.116:0/2596841817 learned_addr learned my addr 172.21.15.116:0/2596841817 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:15.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd4719fc700 1 --2- 172.21.15.116:0/2596841817 >> v2:172.21.15.173:3300/0 conn(0x7fd46c138d80 0x7fd46c15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:15.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2112021419 0 0) 0x7fd46c0a8c80 con 0x7fd46c0c41c0 2020-11-20T12:17:15.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd454003240 con 0x7fd46c0c41c0 2020-11-20T12:17:15.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941239875 0 0) 0x7fd46c0a9180 con 0x7fd46c0c45a0 2020-11-20T12:17:15.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd46c0a8c80 con 0x7fd46c0c45a0 2020-11-20T12:17:15.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2935978745 0 0) 0x7fd454003240 con 0x7fd46c0c41c0 2020-11-20T12:17:15.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 >> v2:172.21.15.173:3300/0 conn(0x7fd46c138d80 msgr2=0x7fd46c15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:15.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 --2- 172.21.15.116:0/2596841817 >> v2:172.21.15.173:3300/0 conn(0x7fd46c138d80 0x7fd46c15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 >> v1:172.21.15.173:6789/0 conn(0x7fd46c0c45a0 legacy=0x7fd46c138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:15.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c0a96c0 con 0x7fd46c0c41c0 2020-11-20T12:17:15.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fd4680014e0 con 0x7fd46c0c41c0 2020-11-20T12:17:15.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 >> v1:172.21.15.116:6789/0 conn(0x7fd46c0c41c0 legacy=0x7fd46c0a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:15.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454003910 0x7fd454003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:15.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4540044e0 0x7fd454004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:15.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd454004e50 0x7fd45400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:15.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c0a9420 con 0x7fd454003910 2020-11-20T12:17:15.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45400bc20 con 0x7fd4540044e0 2020-11-20T12:17:15.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd45400bd60 con 0x7fd454004e50 2020-11-20T12:17:15.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd4711fb700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454003910 0x7fd454003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:15.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd4719fc700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd454004e50 0x7fd45400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:15.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd4709fa700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4540044e0 0x7fd454004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:15.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd4719fc700 1 -- 172.21.15.116:0/2596841817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4540044e0 msgr2=0x7fd454004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:15.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd4719fc700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd4540044e0 0x7fd454004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd4719fc700 1 -- 172.21.15.116:0/2596841817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454003910 msgr2=0x7fd454003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:15.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd4719fc700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454003910 0x7fd454003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd4719fc700 1 -- 172.21.15.116:0/2596841817 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd454003240 con 0x7fd454004e50 2020-11-20T12:17:15.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd4711fb700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454003910 0x7fd454003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:15.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.268+0000 7fd4719fc700 1 -- 172.21.15.116:0/2596841817 reap_dead start 2020-11-20T12:17:15.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.272+0000 7fd4719fc700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd454004e50 0x7fd45400b730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fd464009ea0 tx=0x7fd4640091d0).ready entity=mon.2 client_cookie=9e1cedbd82071f01 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:15.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.272+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4640095c0 con 0x7fd454004e50 2020-11-20T12:17:15.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.272+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46400ec40 con 0x7fd454004e50 2020-11-20T12:17:15.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.272+0000 7fd45bfff700 1 -- 172.21.15.116:0/2596841817 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4640123f0 con 0x7fd454004e50 2020-11-20T12:17:15.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.272+0000 7fd4721fd700 1 -- 172.21.15.116:0/2596841817 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd454004e50 msgr2=0x7fd45400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:15.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.272+0000 7fd4721fd700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd454004e50 0x7fd45400b730 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fd464009ea0 tx=0x7fd4640091d0).stop 2020-11-20T12:17:15.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.272+0000 7fd4721fd700 1 -- 172.21.15.116:0/2596841817 shutdown_connections 2020-11-20T12:17:15.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.272+0000 7fd4721fd700 1 --2- 172.21.15.116:0/2596841817 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd454004e50 0x7fd45400b730 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.272+0000 7fd4721fd700 1 -- 172.21.15.116:0/2596841817 >> 172.21.15.116:0/2596841817 conn(0x7fd46c062520 msgr2=0x7fd46c062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:15.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.272+0000 7fd4721fd700 1 -- 172.21.15.116:0/2596841817 shutdown_connections 2020-11-20T12:17:15.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.272+0000 7fd4721fd700 1 -- 172.21.15.116:0/2596841817 wait complete. 2020-11-20T12:17:15.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4721fd700 1 Processor -- start 2020-11-20T12:17:15.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4721fd700 1 -- start start 2020-11-20T12:17:15.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4721fd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454004e50 0x7fd46c0c1da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:15.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4721fd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd46c0c2290 0x7fd46c09ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:15.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4721fd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd46c0a03f0 0x7fd46c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:15.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4721fd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c162360 con 0x7fd454004e50 2020-11-20T12:17:15.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4721fd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c163210 con 0x7fd46c0a03f0 2020-11-20T12:17:15.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4721fd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c162590 con 0x7fd46c0c2290 2020-11-20T12:17:15.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4711fb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454004e50 0x7fd46c0c1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:15.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4711fb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454004e50 0x7fd46c0c1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45046/0 (socket says 172.21.15.116:45046) 2020-11-20T12:17:15.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4711fb700 1 -- 172.21.15.116:0/526876543 learned_addr learned my addr 172.21.15.116:0/526876543 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:15.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4719fc700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd46c0a03f0 0x7fd46c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:15.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4709fa700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd46c0c2290 0x7fd46c09ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:15.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4711fb700 1 -- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd46c0c2290 msgr2=0x7fd46c09ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:15.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4711fb700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd46c0c2290 0x7fd46c09ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4711fb700 1 -- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd46c0a03f0 msgr2=0x7fd46c15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:15.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4711fb700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd46c0a03f0 0x7fd46c15f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4711fb700 1 -- 172.21.15.116:0/526876543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c162830 con 0x7fd454004e50 2020-11-20T12:17:15.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4711fb700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454004e50 0x7fd46c0c1da0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fd468001610 tx=0x7fd46c0a9310).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:15.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4597fa700 1 -- 172.21.15.116:0/526876543 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46800f9f0 con 0x7fd454004e50 2020-11-20T12:17:15.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4597fa700 1 -- 172.21.15.116:0/526876543 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46800fb50 con 0x7fd454004e50 2020-11-20T12:17:15.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4721fd700 1 -- 172.21.15.116:0/526876543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46c162ad0 con 0x7fd454004e50 2020-11-20T12:17:15.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4597fa700 1 -- 172.21.15.116:0/526876543 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468012350 con 0x7fd454004e50 2020-11-20T12:17:15.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.276+0000 7fd4721fd700 1 -- 172.21.15.116:0/526876543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c0a08f0 con 0x7fd454004e50 2020-11-20T12:17:15.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.280+0000 7fd4721fd700 1 -- 172.21.15.116:0/526876543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46c0a2d90 con 0x7fd454004e50 2020-11-20T12:17:15.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.280+0000 7fd4597fa700 1 -- 172.21.15.116:0/526876543 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd46801b030 con 0x7fd454004e50 2020-11-20T12:17:15.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.280+0000 7fd4597fa700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4440557c0 0x7fd444057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:15.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.280+0000 7fd4597fa700 1 -- 172.21.15.116:0/526876543 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd46806dfe0 con 0x7fd454004e50 2020-11-20T12:17:15.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.280+0000 7fd4709fa700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4440557c0 0x7fd444057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:15.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.284+0000 7fd4709fa700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4440557c0 0x7fd444057c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fd46c0c2670 tx=0x7fd45c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:15.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.288+0000 7fd4597fa700 1 -- 172.21.15.116:0/526876543 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd468043e80 con 0x7fd454004e50 2020-11-20T12:17:15.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.496+0000 7fd4721fd700 1 -- 172.21.15.116:0/526876543 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd46c044f60 con 0x7fd454004e50 2020-11-20T12:17:15.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.496+0000 7fd4597fa700 1 -- 172.21.15.116:0/526876543 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2851 (secure 0 0 0) 0x7fd468019030 con 0x7fd454004e50 2020-11-20T12:17:15.498 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:15.498 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3031892761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3031892761},{"type":"v1","addr":"172.21.15.173:6836","nonce":3031892761}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1794965470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1794965470},{"type":"v1","addr":"172.21.15.173:6837","nonce":1794965470}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":9},{"gid":4995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/4143559744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":4143559744},{"type":"v1","addr":"172.21.15.116:6833","nonce":4143559744}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:57.901741+0000","modified":"2020-11-20T12:16:59.918401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4998":{"gid":4998,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6835/3994085646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3994085646},{"type":"v1","addr":"172.21.15.116:6835","nonce":3994085646}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T12:17:15.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 -- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4440557c0 msgr2=0x7fd444057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:15.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4440557c0 0x7fd444057c80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fd46c0c2670 tx=0x7fd45c009040).stop 2020-11-20T12:17:15.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 -- 172.21.15.116:0/526876543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454004e50 msgr2=0x7fd46c0c1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:15.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454004e50 0x7fd46c0c1da0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fd468001610 tx=0x7fd46c0a9310).stop 2020-11-20T12:17:15.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 -- 172.21.15.116:0/526876543 shutdown_connections 2020-11-20T12:17:15.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd4440557c0 0x7fd444057c80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd46c0a03f0 0x7fd46c15f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd454004e50 0x7fd46c0c1da0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 --2- 172.21.15.116:0/526876543 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd46c0c2290 0x7fd46c09ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:15.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 -- 172.21.15.116:0/526876543 >> 172.21.15.116:0/526876543 conn(0x7fd46c062520 msgr2=0x7fd46c062900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:15.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 -- 172.21.15.116:0/526876543 shutdown_connections 2020-11-20T12:17:15.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:15.500+0000 7fd4721fd700 1 -- 172.21.15.116:0/526876543 wait complete. 2020-11-20T12:17:15.502 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 14 2020-11-20T12:17:15.926 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:57.901741+0000', 'modified': '2020-11-20T12:16:59.918401+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4998}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4998': {'gid': 4998, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.116:6835/3994085646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3994085646}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3994085646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:17:15.927 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:17:15.927 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:17:16.125 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4998,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.197327504,"mdsmap_epoch":14,"osdmap_epoch":34,"osdmap_epoch_barrier":34,"uptime":26.538392905999999} 2020-11-20T12:17:16.125 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4998, "mdsmap_epoch": 14, "osdmap_epoch": 34, "osdmap_epoch_barrier": 34, "rank_uptime": 18.197327504, "state": "up:active", "uptime": 26.538392906, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:17:16.126 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:17:16.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.127 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.3 2020-11-20T12:17:16.143 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:17:16.144 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:16.255 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:16.256 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:17:16.282 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:16.282 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:17:16.288 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:17:16.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.290 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2020-11-20T12:17:16.318 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-20T12:17:16.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:17:16.319 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-20T12:17:16.411 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:17:16.412 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.412 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:17:16.432 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-20T12:17:16.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:17:16.433 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 2020-11-20T12:17:16.504 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:17:16.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.505 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T12:17:16.523 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2020-11-20T12:17:16.524 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:17:16.524 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.2 2020-11-20T12:17:16.603 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:17:16.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.604 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T12:17:16.620 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2020-11-20T12:17:16.621 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:17:16.621 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.3 2020-11-20T12:17:16.707 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:17:16.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.708 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T12:17:16.727 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:17:16.728 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:17:16.728 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:17:16.729 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:17:16.729 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:17:16.730 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:17:16.730 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:17:16.730 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:17:16.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.731 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:17:16.749 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:17:16.749 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:17:16.749 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:17:16.750 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:16.750 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:17:16.750 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:16.750 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:16.751 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:16.751 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:16.751 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:16.751 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:16.751 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:16.752 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:17:16.752 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:16.753 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:17:16.753 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:16.753 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:17:16.754 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:16.754 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:17:16.754 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:16.754 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:17:16.754 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:16.755 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:17:16.755 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:16.755 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:17:16.755 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:16.755 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:17:16.756 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:16.756 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:16.756 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:17:16.756 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:17:16.757 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:16.757 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:16.758 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:17:16.758 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:17:16.758 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:16.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.759 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:17:16.823 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:17:16.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.827 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:17:16.841 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:17:16.842 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:17:16.842 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:17:16.842 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:17:16.843 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:17:16.843 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:17:16.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.844 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:17:16.845 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:17:16.845 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:17:16.845 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:17:16.846 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:17:16.846 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:17:16.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.949 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:17:16.964 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:17:16.964 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:17:16.965 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:17:16.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:16.966 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2020-11-20T12:17:16.996 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:17.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.208+0000 7f9cd7e29700 1 Processor -- start 2020-11-20T12:17:17.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.208+0000 7f9cd7e29700 1 -- start start 2020-11-20T12:17:17.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.208+0000 7f9cd7e29700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9cd00a3690 0x7f9cd0152740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.208+0000 7f9cd7e29700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cd00c2450 0x7f9cd00c28e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.208+0000 7f9cd7e29700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd00bed50 con 0x7f9cd00be7d0 2020-11-20T12:17:17.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.208+0000 7f9cd7e29700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd00beff0 con 0x7f9cd00a3690 2020-11-20T12:17:17.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.208+0000 7f9cd7e29700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd00bf290 con 0x7f9cd00c2450 2020-11-20T12:17:17.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd5bc5700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9cd00a3690 0x7f9cd0152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd53c4700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f9cd00be7d0 0x7f9cd00b9130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:50718/0 (socket says 172.21.15.116:50718) 2020-11-20T12:17:17.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd53c4700 1 -- 172.21.15.116:0/222118215 learned_addr learned my addr 172.21.15.116:0/222118215 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:17.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd63c6700 1 --2- 172.21.15.116:0/222118215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cd00c2450 0x7f9cd00c28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd5bc5700 1 -- 172.21.15.116:0/222118215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cd00c2450 msgr2=0x7f9cd00c28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:17.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd5bc5700 1 --2- 172.21.15.116:0/222118215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cd00c2450 0x7f9cd00c28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd4bc3700 1 -- 172.21.15.116:0/222118215 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1282145338 0 0) 0x7f9cd00bed50 con 0x7f9cd00be7d0 2020-11-20T12:17:17.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd5bc5700 1 -- 172.21.15.116:0/222118215 >> v1:172.21.15.173:6789/0 conn(0x7f9cd00be7d0 legacy=0x7f9cd00b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd5bc5700 1 -- 172.21.15.116:0/222118215 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd00bf530 con 0x7f9cd00a3690 2020-11-20T12:17:17.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd5bc5700 1 --2- 172.21.15.116:0/222118215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cd00a3690 0x7f9cd0152740 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9ccc00b9a0 tx=0x7f9ccc006df0).ready entity=mon.0 client_cookie=f3ab2bcd06e85c56 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:17.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd4bc3700 1 -- 172.21.15.116:0/222118215 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc009840 con 0x7f9cd00a3690 2020-11-20T12:17:17.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd4bc3700 1 -- 172.21.15.116:0/222118215 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc0099a0 con 0x7f9cd00a3690 2020-11-20T12:17:17.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd4bc3700 1 -- 172.21.15.116:0/222118215 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc009d20 con 0x7f9cd00a3690 2020-11-20T12:17:17.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd7e29700 1 -- 172.21.15.116:0/222118215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cd00a3690 msgr2=0x7f9cd0152740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.212+0000 7f9cd7e29700 1 --2- 172.21.15.116:0/222118215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cd00a3690 0x7f9cd0152740 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f9ccc00b9a0 tx=0x7f9ccc006df0).stop 2020-11-20T12:17:17.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.216+0000 7f9cd7e29700 1 -- 172.21.15.116:0/222118215 shutdown_connections 2020-11-20T12:17:17.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.216+0000 7f9cd7e29700 1 --2- 172.21.15.116:0/222118215 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cd00c2450 0x7f9cd00c28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.216+0000 7f9cd7e29700 1 --2- 172.21.15.116:0/222118215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cd00a3690 0x7f9cd0152740 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.216+0000 7f9cd7e29700 1 -- 172.21.15.116:0/222118215 >> 172.21.15.116:0/222118215 conn(0x7f9cd0060c80 msgr2=0x7f9cd00c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:17.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.216+0000 7f9cd7e29700 1 -- 172.21.15.116:0/222118215 shutdown_connections 2020-11-20T12:17:17.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.216+0000 7f9cd7e29700 1 -- 172.21.15.116:0/222118215 wait complete. 2020-11-20T12:17:17.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.216+0000 7f9cd7e29700 1 Processor -- start 2020-11-20T12:17:17.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.216+0000 7f9cd7e29700 1 -- start start 2020-11-20T12:17:17.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd7e29700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cd00a3690 0x7f9cd00bcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd7e29700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cd00be7d0 0x7f9cd0150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd7e29700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cd00c2450 0x7f9cd01652e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd7e29700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0165c60 con 0x7f9cd00c2450 2020-11-20T12:17:17.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd7e29700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0165f00 con 0x7f9cd00be7d0 2020-11-20T12:17:17.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd7e29700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd01661a0 con 0x7f9cd00a3690 2020-11-20T12:17:17.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd63c6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cd00c2450 0x7f9cd01652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd63c6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cd00c2450 0x7f9cd01652e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45064/0 (socket says 172.21.15.116:45064) 2020-11-20T12:17:17.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd63c6700 1 -- 172.21.15.116:0/3505625445 learned_addr learned my addr 172.21.15.116:0/3505625445 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:17.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd5bc5700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cd00a3690 0x7f9cd00bcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd53c4700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cd00be7d0 0x7f9cd0150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd53c4700 1 -- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cd00a3690 msgr2=0x7f9cd00bcbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd53c4700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cd00a3690 0x7f9cd00bcbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd53c4700 1 -- 172.21.15.116:0/3505625445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cd00c2450 msgr2=0x7f9cd01652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd53c4700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cd00c2450 0x7f9cd01652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd53c4700 1 -- 172.21.15.116:0/3505625445 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd00a3a70 con 0x7f9cd00be7d0 2020-11-20T12:17:17.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd63c6700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cd00c2450 0x7f9cd01652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:17.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cd53c4700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cd00be7d0 0x7f9cd0150490 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f9cc400c760 tx=0x7f9cc4007c20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:17.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.220+0000 7f9cc1ffb700 1 -- 172.21.15.116:0/3505625445 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40072e0 con 0x7f9cd00be7d0 2020-11-20T12:17:17.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.224+0000 7f9cc1ffb700 1 -- 172.21.15.116:0/3505625445 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc4007440 con 0x7f9cd00be7d0 2020-11-20T12:17:17.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.224+0000 7f9cd7e29700 1 -- 172.21.15.116:0/3505625445 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd00a3d10 con 0x7f9cd00be7d0 2020-11-20T12:17:17.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.224+0000 7f9cc1ffb700 1 -- 172.21.15.116:0/3505625445 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc4005820 con 0x7f9cd00be7d0 2020-11-20T12:17:17.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.224+0000 7f9cd7e29700 1 -- 172.21.15.116:0/3505625445 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd01659f0 con 0x7f9cd00be7d0 2020-11-20T12:17:17.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.224+0000 7f9cd7e29700 1 -- 172.21.15.116:0/3505625445 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd005ca30 con 0x7f9cd00be7d0 2020-11-20T12:17:17.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.224+0000 7f9cc1ffb700 1 -- 172.21.15.116:0/3505625445 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9cc401a060 con 0x7f9cd00be7d0 2020-11-20T12:17:17.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.224+0000 7f9cc1ffb700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ca80557c0 0x7f9ca8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.224+0000 7f9cc1ffb700 1 -- 172.21.15.116:0/3505625445 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9cc406d9e0 con 0x7f9cd00be7d0 2020-11-20T12:17:17.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.228+0000 7f9cd5bc5700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ca80557c0 0x7f9ca8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.228+0000 7f9cd5bc5700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ca80557c0 0x7f9ca8057c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f9ccc006e30 tx=0x7f9ccc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:17.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.232+0000 7f9cc1ffb700 1 -- 172.21.15.116:0/3505625445 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9cc4043770 con 0x7f9cd00be7d0 2020-11-20T12:17:17.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.392+0000 7f9cd7e29700 1 -- 172.21.15.116:0/3505625445 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cd00aa250 con 0x7f9cd00be7d0 2020-11-20T12:17:17.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.392+0000 7f9cc1ffb700 1 -- 172.21.15.116:0/3505625445 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f9cc4018070 con 0x7f9cd00be7d0 2020-11-20T12:17:17.394 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:17.395 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:58.907407+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:16:55.781578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T12:16:56.900539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:17.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.396+0000 7f9cd7e29700 1 -- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ca80557c0 msgr2=0x7f9ca8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.396+0000 7f9cd7e29700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ca80557c0 0x7f9ca8057c80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f9ccc006e30 tx=0x7f9ccc015000).stop 2020-11-20T12:17:17.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.396+0000 7f9cd7e29700 1 -- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cd00be7d0 msgr2=0x7f9cd0150490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.396+0000 7f9cd7e29700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cd00be7d0 0x7f9cd0150490 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f9cc400c760 tx=0x7f9cc4007c20).stop 2020-11-20T12:17:17.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.396+0000 7f9cd7e29700 1 -- 172.21.15.116:0/3505625445 shutdown_connections 2020-11-20T12:17:17.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.396+0000 7f9cd7e29700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ca80557c0 0x7f9ca8057c80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.396+0000 7f9cd7e29700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cd00c2450 0x7f9cd01652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.396+0000 7f9cd7e29700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cd00a3690 0x7f9cd00bcbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.396+0000 7f9cd7e29700 1 --2- 172.21.15.116:0/3505625445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cd00be7d0 0x7f9cd0150490 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.396+0000 7f9cd7e29700 1 -- 172.21.15.116:0/3505625445 >> 172.21.15.116:0/3505625445 conn(0x7f9cd0060c80 msgr2=0x7f9cd0151040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:17.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.400+0000 7f9cd7e29700 1 -- 172.21.15.116:0/3505625445 shutdown_connections 2020-11-20T12:17:17.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.400+0000 7f9cd7e29700 1 -- 172.21.15.116:0/3505625445 wait complete. 2020-11-20T12:17:17.423 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:16:55.781578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:16:56.900539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:17:17.424 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:17.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.604+0000 7f1836617700 1 Processor -- start 2020-11-20T12:17:17.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.604+0000 7f1836617700 1 -- start start 2020-11-20T12:17:17.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.608+0000 7f1836617700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f183014f150 0x7f18300c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.608+0000 7f1836617700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f18300bd780 0x7f18300c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.608+0000 7f1836617700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1830155620 con 0x7f183014f530 2020-11-20T12:17:17.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.608+0000 7f1836617700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18301558c0 con 0x7f18300bd780 2020-11-20T12:17:17.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.608+0000 7f1836617700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1830155b60 con 0x7f183014f150 2020-11-20T12:17:17.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.608+0000 7f1834bb4700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f183014f530 0x7f18300a74b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54506/0 (socket says 172.21.15.116:54506) 2020-11-20T12:17:17.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.608+0000 7f1834bb4700 1 -- 172.21.15.116:0/2569531744 learned_addr learned my addr 172.21.15.116:0/2569531744 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:17.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.608+0000 7f182ffff700 1 --2- 172.21.15.116:0/2569531744 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f183014f150 0x7f18300c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.608+0000 7f182f7fe700 1 --2- 172.21.15.116:0/2569531744 >> v2:172.21.15.173:3300/0 conn(0x7f18300bd780 0x7f18300c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.608+0000 7f182effd700 1 -- 172.21.15.116:0/2569531744 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384535475 0 0) 0x7f1830155620 con 0x7f183014f530 2020-11-20T12:17:17.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.608+0000 7f182effd700 1 -- 172.21.15.116:0/2569531744 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f181c003240 con 0x7f183014f530 2020-11-20T12:17:17.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f182ffff700 1 -- 172.21.15.116:0/2569531744 >> v2:172.21.15.173:3300/0 conn(0x7f18300bd780 msgr2=0x7f18300c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:17.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f182ffff700 1 --2- 172.21.15.116:0/2569531744 >> v2:172.21.15.173:3300/0 conn(0x7f18300bd780 0x7f18300c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f182ffff700 1 -- 172.21.15.116:0/2569531744 >> v1:172.21.15.116:6789/0 conn(0x7f183014f530 legacy=0x7f18300a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f182ffff700 1 -- 172.21.15.116:0/2569531744 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830155e00 con 0x7f183014f150 2020-11-20T12:17:17.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f182ffff700 1 --2- 172.21.15.116:0/2569531744 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f183014f150 0x7f18300c87f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f182000b9a0 tx=0x7f1820006df0).ready entity=mon.2 client_cookie=1503156ca6db0e11 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:17.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f182effd700 1 -- 172.21.15.116:0/2569531744 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820009710 con 0x7f183014f150 2020-11-20T12:17:17.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f182effd700 1 -- 172.21.15.116:0/2569531744 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1820009870 con 0x7f183014f150 2020-11-20T12:17:17.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f182effd700 1 -- 172.21.15.116:0/2569531744 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1820009bf0 con 0x7f183014f150 2020-11-20T12:17:17.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f1836617700 1 -- 172.21.15.116:0/2569531744 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f183014f150 msgr2=0x7f18300c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f1836617700 1 --2- 172.21.15.116:0/2569531744 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f183014f150 0x7f18300c87f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f182000b9a0 tx=0x7f1820006df0).stop 2020-11-20T12:17:17.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f1836617700 1 -- 172.21.15.116:0/2569531744 shutdown_connections 2020-11-20T12:17:17.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f1836617700 1 --2- 172.21.15.116:0/2569531744 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f183014f150 0x7f18300c87f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f1836617700 1 --2- 172.21.15.116:0/2569531744 >> v2:172.21.15.173:3300/0 conn(0x7f18300bd780 0x7f18300c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f1836617700 1 -- 172.21.15.116:0/2569531744 >> 172.21.15.116:0/2569531744 conn(0x7f1830061580 msgr2=0x7f1830061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:17.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f1836617700 1 -- 172.21.15.116:0/2569531744 shutdown_connections 2020-11-20T12:17:17.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.612+0000 7f1836617700 1 -- 172.21.15.116:0/2569531744 wait complete. 2020-11-20T12:17:17.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f1836617700 1 Processor -- start 2020-11-20T12:17:17.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f1836617700 1 -- start start 2020-11-20T12:17:17.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f1836617700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18300bd780 0x7f18300ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f1836617700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f183014f150 0x7f1830157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f1836617700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f183014f530 0x7f1830157950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f1836617700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18300acb20 con 0x7f183014f150 2020-11-20T12:17:17.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f1836617700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18301657f0 con 0x7f18300bd780 2020-11-20T12:17:17.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f1836617700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1830165a90 con 0x7f183014f530 2020-11-20T12:17:17.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f182f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f183014f150 0x7f1830157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f182f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f183014f150 0x7f1830157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45078/0 (socket says 172.21.15.116:45078) 2020-11-20T12:17:17.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f182f7fe700 1 -- 172.21.15.116:0/2107552924 learned_addr learned my addr 172.21.15.116:0/2107552924 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:17.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f182ffff700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18300bd780 0x7f18300ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f1834bb4700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f183014f530 0x7f1830157950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f182f7fe700 1 -- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f183014f530 msgr2=0x7f1830157950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f182f7fe700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f183014f530 0x7f1830157950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f182f7fe700 1 -- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18300bd780 msgr2=0x7f18300ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f182f7fe700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18300bd780 0x7f18300ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.616+0000 7f182f7fe700 1 -- 172.21.15.116:0/2107552924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1830165d30 con 0x7f183014f150 2020-11-20T12:17:17.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.620+0000 7f182f7fe700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f183014f150 0x7f1830157460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f18300bdb80 tx=0x7f1824006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:17.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.620+0000 7f180ffff700 1 -- 172.21.15.116:0/2107552924 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400b8a0 con 0x7f183014f150 2020-11-20T12:17:17.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.620+0000 7f180ffff700 1 -- 172.21.15.116:0/2107552924 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f182400ba00 con 0x7f183014f150 2020-11-20T12:17:17.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.620+0000 7f180ffff700 1 -- 172.21.15.116:0/2107552924 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182400bd80 con 0x7f183014f150 2020-11-20T12:17:17.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.620+0000 7f1836617700 1 -- 172.21.15.116:0/2107552924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1830165fd0 con 0x7f183014f150 2020-11-20T12:17:17.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.620+0000 7f1836617700 1 -- 172.21.15.116:0/2107552924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1830165520 con 0x7f183014f150 2020-11-20T12:17:17.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.620+0000 7f1836617700 1 -- 172.21.15.116:0/2107552924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f183005ca30 con 0x7f183014f150 2020-11-20T12:17:17.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.620+0000 7f180ffff700 1 -- 172.21.15.116:0/2107552924 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f182401a030 con 0x7f183014f150 2020-11-20T12:17:17.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.624+0000 7f180ffff700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18080557c0 0x7f1808057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:17.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.624+0000 7f180ffff700 1 -- 172.21.15.116:0/2107552924 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f182406e110 con 0x7f183014f150 2020-11-20T12:17:17.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.624+0000 7f182ffff700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18080557c0 0x7f1808057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:17.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.624+0000 7f182ffff700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18080557c0 0x7f1808057c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f1820006e30 tx=0x7f1820015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:17.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.628+0000 7f180ffff700 1 -- 172.21.15.116:0/2107552924 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1824044060 con 0x7f183014f150 2020-11-20T12:17:17.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.792+0000 7f1836617700 1 -- 172.21.15.116:0/2107552924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f18300ab0d0 con 0x7f183014f150 2020-11-20T12:17:17.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.792+0000 7f180ffff700 1 -- 172.21.15.116:0/2107552924 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v34) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1824047680 con 0x7f183014f150 2020-11-20T12:17:17.795 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:17.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.796+0000 7f1836617700 1 -- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18080557c0 msgr2=0x7f1808057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.796+0000 7f1836617700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18080557c0 0x7f1808057c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f1820006e30 tx=0x7f1820015000).stop 2020-11-20T12:17:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.796+0000 7f1836617700 1 -- 172.21.15.116:0/2107552924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f183014f150 msgr2=0x7f1830157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.796+0000 7f1836617700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f183014f150 0x7f1830157460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f18300bdb80 tx=0x7f1824006df0).stop 2020-11-20T12:17:17.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.796+0000 7f1836617700 1 -- 172.21.15.116:0/2107552924 shutdown_connections 2020-11-20T12:17:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.796+0000 7f1836617700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18080557c0 0x7f1808057c80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.796+0000 7f1836617700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f183014f530 0x7f1830157950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.796+0000 7f1836617700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f18300bd780 0x7f18300ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.796+0000 7f1836617700 1 --2- 172.21.15.116:0/2107552924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f183014f150 0x7f1830157460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.796+0000 7f1836617700 1 -- 172.21.15.116:0/2107552924 >> 172.21.15.116:0/2107552924 conn(0x7f1830061580 msgr2=0x7f18300bdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.800+0000 7f1836617700 1 -- 172.21.15.116:0/2107552924 shutdown_connections 2020-11-20T12:17:17.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:17.800+0000 7f1836617700 1 -- 172.21.15.116:0/2107552924 wait complete. 2020-11-20T12:17:17.823 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:18.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.004+0000 7f581359e700 1 Processor -- start 2020-11-20T12:17:18.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.004+0000 7f581359e700 1 -- start start 2020-11-20T12:17:18.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581359e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f581413ad30 0x7f58140a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581359e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f581413b110 0x7f5814151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581359e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5814152a80 con 0x7f5814151fa0 2020-11-20T12:17:18.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581359e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5814152540 con 0x7f581413ad30 2020-11-20T12:17:18.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581359e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58141527e0 con 0x7f581413b110 2020-11-20T12:17:18.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f581413ad30 0x7f58140a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f581413ad30 0x7f58140a7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45084/0 (socket says 172.21.15.116:45084) 2020-11-20T12:17:18.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581259c700 1 -- 172.21.15.116:0/2157121196 learned_addr learned my addr 172.21.15.116:0/2157121196 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:18.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f5811d9b700 1 --2- 172.21.15.116:0/2157121196 >> v2:172.21.15.173:3300/0 conn(0x7f581413b110 0x7f5814151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581259c700 1 -- 172.21.15.116:0/2157121196 >> v2:172.21.15.173:3300/0 conn(0x7f581413b110 msgr2=0x7f5814151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:18.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581259c700 1 --2- 172.21.15.116:0/2157121196 >> v2:172.21.15.173:3300/0 conn(0x7f581413b110 0x7f5814151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581259c700 1 -- 172.21.15.116:0/2157121196 >> v1:172.21.15.173:6789/0 conn(0x7f5814151fa0 legacy=0x7f5814061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581259c700 1 -- 172.21.15.116:0/2157121196 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58140ac910 con 0x7f581413ad30 2020-11-20T12:17:18.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.008+0000 7f581259c700 1 --2- 172.21.15.116:0/2157121196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581413ad30 0x7f58140a7fb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f5814062710 tx=0x7f5804006eb0).ready entity=mon.0 client_cookie=df3639eec5558189 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:18.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581159a700 1 -- 172.21.15.116:0/2157121196 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58040098a0 con 0x7f581413ad30 2020-11-20T12:17:18.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581159a700 1 -- 172.21.15.116:0/2157121196 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5804009a00 con 0x7f581413ad30 2020-11-20T12:17:18.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581159a700 1 -- 172.21.15.116:0/2157121196 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5804009d80 con 0x7f581413ad30 2020-11-20T12:17:18.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581359e700 1 -- 172.21.15.116:0/2157121196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581413ad30 msgr2=0x7f58140a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581359e700 1 --2- 172.21.15.116:0/2157121196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581413ad30 0x7f58140a7fb0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f5814062710 tx=0x7f5804006eb0).stop 2020-11-20T12:17:18.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581359e700 1 -- 172.21.15.116:0/2157121196 shutdown_connections 2020-11-20T12:17:18.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581359e700 1 --2- 172.21.15.116:0/2157121196 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581413ad30 0x7f58140a7fb0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581359e700 1 --2- 172.21.15.116:0/2157121196 >> v2:172.21.15.173:3300/0 conn(0x7f581413b110 0x7f5814151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581359e700 1 -- 172.21.15.116:0/2157121196 >> 172.21.15.116:0/2157121196 conn(0x7f58140678a0 msgr2=0x7f5814067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:18.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581359e700 1 -- 172.21.15.116:0/2157121196 shutdown_connections 2020-11-20T12:17:18.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581359e700 1 -- 172.21.15.116:0/2157121196 wait complete. 2020-11-20T12:17:18.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581359e700 1 Processor -- start 2020-11-20T12:17:18.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.012+0000 7f581359e700 1 -- start start 2020-11-20T12:17:18.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f581359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f581413ad30 0x7f581409f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f581359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581413b110 0x7f5814139950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f581359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5814151fa0 0x7f58140a0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f581359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5814156db0 con 0x7f581413b110 2020-11-20T12:17:18.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f581359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5814157050 con 0x7f581413ad30 2020-11-20T12:17:18.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f581359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58141572f0 con 0x7f5814151fa0 2020-11-20T12:17:18.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f5811d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581413b110 0x7f5814139950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f5811d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581413b110 0x7f5814139950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45088/0 (socket says 172.21.15.116:45088) 2020-11-20T12:17:18.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f5811d9b700 1 -- 172.21.15.116:0/3734132421 learned_addr learned my addr 172.21.15.116:0/3734132421 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:18.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f581259c700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f581413ad30 0x7f581409f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f5812d9d700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5814151fa0 0x7f58140a0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f5811d9b700 1 -- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5814151fa0 msgr2=0x7f58140a0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f5811d9b700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5814151fa0 0x7f58140a0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f5811d9b700 1 -- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f581413ad30 msgr2=0x7f581409f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f5811d9b700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f581413ad30 0x7f581409f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f5811d9b700 1 -- 172.21.15.116:0/3734132421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5814157590 con 0x7f581413b110 2020-11-20T12:17:18.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f581259c700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f581413ad30 0x7f581409f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:18.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f5811d9b700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581413b110 0x7f5814139950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f580800ebd0 tx=0x7f5808006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:18.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f58027fc700 1 -- 172.21.15.116:0/3734132421 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580800c9b0 con 0x7f581413b110 2020-11-20T12:17:18.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f58027fc700 1 -- 172.21.15.116:0/3734132421 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f580800cb10 con 0x7f581413b110 2020-11-20T12:17:18.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f581359e700 1 -- 172.21.15.116:0/3734132421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5814157830 con 0x7f581413b110 2020-11-20T12:17:18.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f58027fc700 1 -- 172.21.15.116:0/3734132421 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f580800ce90 con 0x7f581413b110 2020-11-20T12:17:18.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.016+0000 7f581359e700 1 -- 172.21.15.116:0/3734132421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5814139e40 con 0x7f581413b110 2020-11-20T12:17:18.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.020+0000 7f581359e700 1 -- 172.21.15.116:0/3734132421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581413a0f0 con 0x7f581413b110 2020-11-20T12:17:18.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.020+0000 7f58027fc700 1 -- 172.21.15.116:0/3734132421 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5808012600 con 0x7f581413b110 2020-11-20T12:17:18.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.020+0000 7f58027fc700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57e80557c0 0x7f57e8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.020+0000 7f58027fc700 1 -- 172.21.15.116:0/3734132421 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f580806eaf0 con 0x7f581413b110 2020-11-20T12:17:18.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.024+0000 7f581259c700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57e80557c0 0x7f57e8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.024+0000 7f581259c700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57e80557c0 0x7f57e8057c80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f5804006ef0 tx=0x7f5804015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:18.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.028+0000 7f58027fc700 1 -- 172.21.15.116:0/3734132421 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5808044990 con 0x7f581413b110 2020-11-20T12:17:18.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.188+0000 7f581359e700 1 -- 172.21.15.116:0/3734132421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f581409fc50 con 0x7f581413b110 2020-11-20T12:17:18.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.188+0000 7f58027fc700 1 -- 172.21.15.116:0/3734132421 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5808047fb0 con 0x7f581413b110 2020-11-20T12:17:18.190 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:18.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 -- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57e80557c0 msgr2=0x7f57e8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57e80557c0 0x7f57e8057c80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f5804006ef0 tx=0x7f5804015000).stop 2020-11-20T12:17:18.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 -- 172.21.15.116:0/3734132421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581413b110 msgr2=0x7f5814139950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581413b110 0x7f5814139950 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f580800ebd0 tx=0x7f5808006df0).stop 2020-11-20T12:17:18.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 -- 172.21.15.116:0/3734132421 shutdown_connections 2020-11-20T12:17:18.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57e80557c0 0x7f57e8057c80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5814151fa0 0x7f58140a0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f581413ad30 0x7f581409f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 --2- 172.21.15.116:0/3734132421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f581413b110 0x7f5814139950 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 -- 172.21.15.116:0/3734132421 >> 172.21.15.116:0/3734132421 conn(0x7f58140678a0 msgr2=0x7f58140acf70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:18.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 -- 172.21.15.116:0/3734132421 shutdown_connections 2020-11-20T12:17:18.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.192+0000 7f581359e700 1 -- 172.21.15.116:0/3734132421 wait complete. 2020-11-20T12:17:18.215 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:17:18.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.384+0000 7f07cd83b700 1 Processor -- start 2020-11-20T12:17:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.384+0000 7f07cd83b700 1 -- start start 2020-11-20T12:17:18.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07cd83b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f07c8065fb0 0x7f07c80673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07cd83b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c81538d0 0x7f07c814fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07cd83b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07c80683c0 con 0x7f07c814f460 2020-11-20T12:17:18.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07cd83b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07c8066ed0 con 0x7f07c8065fb0 2020-11-20T12:17:18.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07cd83b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c8066450 con 0x7f07c81538d0 2020-11-20T12:17:18.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c6ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f07c814f460 0x7f07c80a2ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54532/0 (socket says 172.21.15.116:54532) 2020-11-20T12:17:18.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c6ffd700 1 -- 172.21.15.116:0/2047556660 learned_addr learned my addr 172.21.15.116:0/2047556660 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:18.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c67fc700 1 --2- 172.21.15.116:0/2047556660 >> v2:172.21.15.173:3300/0 conn(0x7f07c8065fb0 0x7f07c80673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c77fe700 1 --2- 172.21.15.116:0/2047556660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c81538d0 0x7f07c814fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c5ffb700 1 -- 172.21.15.116:0/2047556660 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2846295261 0 0) 0x7f07c80683c0 con 0x7f07c814f460 2020-11-20T12:17:18.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c5ffb700 1 -- 172.21.15.116:0/2047556660 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07b4003240 con 0x7f07c814f460 2020-11-20T12:17:18.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c67fc700 1 -- 172.21.15.116:0/2047556660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c81538d0 msgr2=0x7f07c814fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:18.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c67fc700 1 --2- 172.21.15.116:0/2047556660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c81538d0 0x7f07c814fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c67fc700 1 -- 172.21.15.116:0/2047556660 >> v1:172.21.15.116:6789/0 conn(0x7f07c814f460 legacy=0x7f07c80a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c67fc700 1 -- 172.21.15.116:0/2047556660 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c814f840 con 0x7f07c8065fb0 2020-11-20T12:17:18.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c67fc700 1 --2- 172.21.15.116:0/2047556660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c8065fb0 0x7f07c80673b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f07bc00b9a0 tx=0x7f07bc006df0).ready entity=mon.1 client_cookie=a593f83214ca0b7c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:18.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c5ffb700 1 -- 172.21.15.116:0/2047556660 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc009680 con 0x7f07c8065fb0 2020-11-20T12:17:18.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c5ffb700 1 -- 172.21.15.116:0/2047556660 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07bc0097e0 con 0x7f07c8065fb0 2020-11-20T12:17:18.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.388+0000 7f07c5ffb700 1 -- 172.21.15.116:0/2047556660 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07bc009b60 con 0x7f07c8065fb0 2020-11-20T12:17:18.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.392+0000 7f07cd83b700 1 -- 172.21.15.116:0/2047556660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c8065fb0 msgr2=0x7f07c80673b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.392+0000 7f07cd83b700 1 --2- 172.21.15.116:0/2047556660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c8065fb0 0x7f07c80673b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f07bc00b9a0 tx=0x7f07bc006df0).stop 2020-11-20T12:17:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.392+0000 7f07cd83b700 1 -- 172.21.15.116:0/2047556660 shutdown_connections 2020-11-20T12:17:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.392+0000 7f07cd83b700 1 --2- 172.21.15.116:0/2047556660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c81538d0 0x7f07c814fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.392+0000 7f07cd83b700 1 --2- 172.21.15.116:0/2047556660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c8065fb0 0x7f07c80673b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.392+0000 7f07cd83b700 1 -- 172.21.15.116:0/2047556660 >> 172.21.15.116:0/2047556660 conn(0x7f07c8060c30 msgr2=0x7f07c80c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.392+0000 7f07cd83b700 1 -- 172.21.15.116:0/2047556660 shutdown_connections 2020-11-20T12:17:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.392+0000 7f07cd83b700 1 -- 172.21.15.116:0/2047556660 wait complete. 2020-11-20T12:17:18.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.392+0000 7f07cd83b700 1 Processor -- start 2020-11-20T12:17:18.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.392+0000 7f07cd83b700 1 -- start start 2020-11-20T12:17:18.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07cd83b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c8065fb0 0x7f07c81509a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07cd83b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c814f460 0x7f07c80bd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07cd83b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07c81538d0 0x7f07c814d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07cd83b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c80bdb30 con 0x7f07c81538d0 2020-11-20T12:17:18.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07cd83b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07c80bddd0 con 0x7f07c814f460 2020-11-20T12:17:18.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07cd83b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07c80be070 con 0x7f07c8065fb0 2020-11-20T12:17:18.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07c77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07c81538d0 0x7f07c814d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07c77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07c81538d0 0x7f07c814d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45106/0 (socket says 172.21.15.116:45106) 2020-11-20T12:17:18.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07c77fe700 1 -- 172.21.15.116:0/3226913836 learned_addr learned my addr 172.21.15.116:0/3226913836 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:18.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07c6ffd700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c8065fb0 0x7f07c81509a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07c67fc700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c814f460 0x7f07c80bd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07c77fe700 1 -- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c8065fb0 msgr2=0x7f07c81509a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07c77fe700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c8065fb0 0x7f07c81509a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07c77fe700 1 -- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c814f460 msgr2=0x7f07c80bd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07c77fe700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c814f460 0x7f07c80bd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07c77fe700 1 -- 172.21.15.116:0/3226913836 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07c80be310 con 0x7f07c81538d0 2020-11-20T12:17:18.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07c77fe700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07c81538d0 0x7f07c814d050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f07c000ef50 tx=0x7f07c00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:18.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07aeffd700 1 -- 172.21.15.116:0/3226913836 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c000cb40 con 0x7f07c81538d0 2020-11-20T12:17:18.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07aeffd700 1 -- 172.21.15.116:0/3226913836 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c000cca0 con 0x7f07c81538d0 2020-11-20T12:17:18.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07cd83b700 1 -- 172.21.15.116:0/3226913836 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07c8154d90 con 0x7f07c81538d0 2020-11-20T12:17:18.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.396+0000 7f07aeffd700 1 -- 172.21.15.116:0/3226913836 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c00123f0 con 0x7f07c81538d0 2020-11-20T12:17:18.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.400+0000 7f07cd83b700 1 -- 172.21.15.116:0/3226913836 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07c814d540 con 0x7f07c81538d0 2020-11-20T12:17:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.400+0000 7f07cd83b700 1 -- 172.21.15.116:0/3226913836 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07c8154f20 con 0x7f07c81538d0 2020-11-20T12:17:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.400+0000 7f07aeffd700 1 -- 172.21.15.116:0/3226913836 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f07c0022070 con 0x7f07c81538d0 2020-11-20T12:17:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.400+0000 7f07aeffd700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07a40557c0 0x7f07a4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.404+0000 7f07aeffd700 1 -- 172.21.15.116:0/3226913836 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f07c006ef40 con 0x7f07c81538d0 2020-11-20T12:17:18.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.404+0000 7f07c6ffd700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07a40557c0 0x7f07a4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.404+0000 7f07c6ffd700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07a40557c0 0x7f07a4057c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f07b80055a0 tx=0x7f07b8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:18.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.408+0000 7f07aeffd700 1 -- 172.21.15.116:0/3226913836 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f07c0044de0 con 0x7f07c81538d0 2020-11-20T12:17:18.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.564+0000 7f07cd83b700 1 -- 172.21.15.116:0/3226913836 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f07c8044f60 con 0x7f07c81538d0 2020-11-20T12:17:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.564+0000 7f07aeffd700 1 -- 172.21.15.116:0/3226913836 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+10 (secure 0 0 0) 0x7f07c0016070 con 0x7f07c81538d0 2020-11-20T12:17:18.565 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:18.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.564+0000 7f07cd83b700 1 -- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07a40557c0 msgr2=0x7f07a4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.564+0000 7f07cd83b700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07a40557c0 0x7f07a4057c80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f07b80055a0 tx=0x7f07b8009040).stop 2020-11-20T12:17:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.564+0000 7f07cd83b700 1 -- 172.21.15.116:0/3226913836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07c81538d0 msgr2=0x7f07c814d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.564+0000 7f07cd83b700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07c81538d0 0x7f07c814d050 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f07c000ef50 tx=0x7f07c00091d0).stop 2020-11-20T12:17:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.568+0000 7f07cd83b700 1 -- 172.21.15.116:0/3226913836 shutdown_connections 2020-11-20T12:17:18.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.568+0000 7f07cd83b700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07a40557c0 0x7f07a4057c80 secure :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0x7f07b80055a0 tx=0x7f07b8009040).stop 2020-11-20T12:17:18.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.568+0000 7f07cd83b700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07c81538d0 0x7f07c814d050 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.568+0000 7f07cd83b700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07c8065fb0 0x7f07c81509a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.568+0000 7f07cd83b700 1 --2- 172.21.15.116:0/3226913836 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07c814f460 0x7f07c80bd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.568+0000 7f07cd83b700 1 -- 172.21.15.116:0/3226913836 >> 172.21.15.116:0/3226913836 conn(0x7f07c8060c30 msgr2=0x7f07c80c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.568+0000 7f07cd83b700 1 -- 172.21.15.116:0/3226913836 shutdown_connections 2020-11-20T12:17:18.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.568+0000 7f07cd83b700 1 -- 172.21.15.116:0/3226913836 wait complete. 2020-11-20T12:17:18.589 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:17:18.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.768+0000 7fc8b4c22700 1 Processor -- start 2020-11-20T12:17:18.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.768+0000 7fc8b4c22700 1 -- start start 2020-11-20T12:17:18.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.768+0000 7fc8b4c22700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc8b014e420 0x7fc8b00a4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.768+0000 7fc8b4c22700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b00a1010 con 0x7fc8b014e040 2020-11-20T12:17:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.768+0000 7fc8b4c22700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8b00a5ce0 con 0x7fc8b00a54c0 2020-11-20T12:17:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.768+0000 7fc8b4c22700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8b00a5f80 con 0x7fc8b014e420 2020-11-20T12:17:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.768+0000 7fc8af7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc8b014e040 0x7fc8b00c2cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54544/0 (socket says 172.21.15.116:54544) 2020-11-20T12:17:18.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.768+0000 7fc8af7fe700 1 -- 172.21.15.116:0/1563740778 learned_addr learned my addr 172.21.15.116:0/1563740778 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.768+0000 7fc8aeffd700 1 --2- 172.21.15.116:0/1563740778 >> v2:172.21.15.116:3300/0 conn(0x7fc8b014e420 0x7fc8b00a4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8aeffd700 1 -- 172.21.15.116:0/1563740778 >> v1:172.21.15.173:6789/0 conn(0x7fc8b00a54c0 legacy=0x7fc8b0157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8aeffd700 1 -- 172.21.15.116:0/1563740778 >> v1:172.21.15.116:6789/0 conn(0x7fc8b014e040 legacy=0x7fc8b00c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8aeffd700 1 -- 172.21.15.116:0/1563740778 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b00a1270 con 0x7fc8b014e420 2020-11-20T12:17:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8aeffd700 1 --2- 172.21.15.116:0/1563740778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8b014e420 0x7fc8b00a4fd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fc8a400b9a0 tx=0x7fc8a4006df0).ready entity=mon.0 client_cookie=e8a8557164660b7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:18.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8ae7fc700 1 -- 172.21.15.116:0/1563740778 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4009770 con 0x7fc8b014e420 2020-11-20T12:17:18.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8ae7fc700 1 -- 172.21.15.116:0/1563740778 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a40098d0 con 0x7fc8b014e420 2020-11-20T12:17:18.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8ae7fc700 1 -- 172.21.15.116:0/1563740778 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a4009c50 con 0x7fc8b014e420 2020-11-20T12:17:18.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8b4c22700 1 -- 172.21.15.116:0/1563740778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8b014e420 msgr2=0x7fc8b00a4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8b4c22700 1 --2- 172.21.15.116:0/1563740778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8b014e420 0x7fc8b00a4fd0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fc8a400b9a0 tx=0x7fc8a4006df0).stop 2020-11-20T12:17:18.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8b4c22700 1 -- 172.21.15.116:0/1563740778 shutdown_connections 2020-11-20T12:17:18.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8b4c22700 1 --2- 172.21.15.116:0/1563740778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8b014e420 0x7fc8b00a4fd0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8b4c22700 1 -- 172.21.15.116:0/1563740778 >> 172.21.15.116:0/1563740778 conn(0x7fc8b0060c30 msgr2=0x7fc8b00c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:18.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8b4c22700 1 -- 172.21.15.116:0/1563740778 shutdown_connections 2020-11-20T12:17:18.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.772+0000 7fc8b4c22700 1 -- 172.21.15.116:0/1563740778 wait complete. 2020-11-20T12:17:18.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.776+0000 7fc8b4c22700 1 Processor -- start 2020-11-20T12:17:18.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.776+0000 7fc8b4c22700 1 -- start start 2020-11-20T12:17:18.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8b4c22700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8b00a54c0 0x7fc8b00909d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8b4c22700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8b014e040 0x7fc8b0149a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8b4c22700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8b014e420 0x7fc8b00c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8b4c22700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0090380 con 0x7fc8b00a54c0 2020-11-20T12:17:18.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8b4c22700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b0090620 con 0x7fc8b014e420 2020-11-20T12:17:18.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8b4c22700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8b00a70e0 con 0x7fc8b014e040 2020-11-20T12:17:18.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8af7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8b00a54c0 0x7fc8b00909d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8af7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8b00a54c0 0x7fc8b00909d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45118/0 (socket says 172.21.15.116:45118) 2020-11-20T12:17:18.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8af7fe700 1 -- 172.21.15.116:0/3142577304 learned_addr learned my addr 172.21.15.116:0/3142577304 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:18.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8aeffd700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8b014e040 0x7fc8b0149a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8affff700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8b014e420 0x7fc8b00c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8af7fe700 1 -- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8b014e040 msgr2=0x7fc8b0149a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8af7fe700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8b014e040 0x7fc8b0149a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8af7fe700 1 -- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8b014e420 msgr2=0x7fc8b00c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8af7fe700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8b014e420 0x7fc8b00c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8af7fe700 1 -- 172.21.15.116:0/3142577304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b00a7380 con 0x7fc8b00a54c0 2020-11-20T12:17:18.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8af7fe700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8b00a54c0 0x7fc8b00909d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fc8a000c710 tx=0x7fc8a0007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:18.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc88f7fe700 1 -- 172.21.15.116:0/3142577304 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a00072e0 con 0x7fc8b00a54c0 2020-11-20T12:17:18.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc88f7fe700 1 -- 172.21.15.116:0/3142577304 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a0007440 con 0x7fc8b00a54c0 2020-11-20T12:17:18.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8b4c22700 1 -- 172.21.15.116:0/3142577304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8b0090ec0 con 0x7fc8b00a54c0 2020-11-20T12:17:18.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc88f7fe700 1 -- 172.21.15.116:0/3142577304 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0005800 con 0x7fc8b00a54c0 2020-11-20T12:17:18.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.780+0000 7fc8b4c22700 1 -- 172.21.15.116:0/3142577304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8b014a100 con 0x7fc8b00a54c0 2020-11-20T12:17:18.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.784+0000 7fc8b4c22700 1 -- 172.21.15.116:0/3142577304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8b0044f60 con 0x7fc8b00a54c0 2020-11-20T12:17:18.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.784+0000 7fc88f7fe700 1 -- 172.21.15.116:0/3142577304 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc8a001a060 con 0x7fc8b00a54c0 2020-11-20T12:17:18.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.784+0000 7fc88f7fe700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8880557c0 0x7fc888057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:18.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.784+0000 7fc88f7fe700 1 -- 172.21.15.116:0/3142577304 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc8a006de90 con 0x7fc8b00a54c0 2020-11-20T12:17:18.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.788+0000 7fc8aeffd700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8880557c0 0x7fc888057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:18.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.788+0000 7fc8aeffd700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8880557c0 0x7fc888057c80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fc8b014e840 tx=0x7fc8a4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:18.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.792+0000 7fc88f7fe700 1 -- 172.21.15.116:0/3142577304 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc8a0043c20 con 0x7fc8b00a54c0 2020-11-20T12:17:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.948+0000 7fc8b4c22700 1 -- 172.21.15.116:0/3142577304 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc8b00a6220 con 0x7fc8b00a54c0 2020-11-20T12:17:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.948+0000 7fc88f7fe700 1 -- 172.21.15.116:0/3142577304 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc8a0018070 con 0x7fc8b00a54c0 2020-11-20T12:17:18.950 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:18.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 -- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8880557c0 msgr2=0x7fc888057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8880557c0 0x7fc888057c80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fc8b014e840 tx=0x7fc8a4015000).stop 2020-11-20T12:17:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 -- 172.21.15.116:0/3142577304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8b00a54c0 msgr2=0x7fc8b00909d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8b00a54c0 0x7fc8b00909d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fc8a000c710 tx=0x7fc8a0007c40).stop 2020-11-20T12:17:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 -- 172.21.15.116:0/3142577304 shutdown_connections 2020-11-20T12:17:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8880557c0 0x7fc888057c80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8b014e420 0x7fc8b00c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8b00a54c0 0x7fc8b00909d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 --2- 172.21.15.116:0/3142577304 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8b014e040 0x7fc8b0149a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 -- 172.21.15.116:0/3142577304 >> 172.21.15.116:0/3142577304 conn(0x7fc8b0060c30 msgr2=0x7fc8b0062620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:18.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 -- 172.21.15.116:0/3142577304 shutdown_connections 2020-11-20T12:17:18.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:18.952+0000 7fc8b4c22700 1 -- 172.21.15.116:0/3142577304 wait complete. 2020-11-20T12:17:18.976 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:17:18.977 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.152+0000 7efe1ca3f700 1 Processor -- start 2020-11-20T12:17:19.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.152+0000 7efe1ca3f700 1 -- start start 2020-11-20T12:17:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.152+0000 7efe1ca3f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c41c0 0x7efe180c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.152+0000 7efe1ca3f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe180a9180 con 0x7efe180a5a10 2020-11-20T12:17:19.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.152+0000 7efe1ca3f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe180a9420 con 0x7efe18138d80 2020-11-20T12:17:19.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.152+0000 7efe1ca3f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe180a96c0 con 0x7efe180c41c0 2020-11-20T12:17:19.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.152+0000 7efe0e59c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7efe180a5a10 0x7efe18138670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54560/0 (socket says 172.21.15.116:54560) 2020-11-20T12:17:19.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.152+0000 7efe0e59c700 1 -- 172.21.15.116:0/606172572 learned_addr learned my addr 172.21.15.116:0/606172572 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:19.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.152+0000 7efe16d9d700 1 --2- 172.21.15.116:0/606172572 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c41c0 0x7efe180c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1659c700 1 -- 172.21.15.116:0/606172572 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404768953 0 0) 0x7efe180a9180 con 0x7efe180a5a10 2020-11-20T12:17:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1659c700 1 -- 172.21.15.116:0/606172572 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdf8003240 con 0x7efe180a5a10 2020-11-20T12:17:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe16d9d700 1 -- 172.21.15.116:0/606172572 >> v1:172.21.15.173:6789/0 conn(0x7efe18138d80 legacy=0x7efe180a7c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe16d9d700 1 -- 172.21.15.116:0/606172572 >> v1:172.21.15.116:6789/0 conn(0x7efe180a5a10 legacy=0x7efe18138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:19.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe16d9d700 1 -- 172.21.15.116:0/606172572 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180a9960 con 0x7efe180c41c0 2020-11-20T12:17:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe16d9d700 1 --2- 172.21.15.116:0/606172572 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c41c0 0x7efe180c45a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7efe0800ba00 tx=0x7efe08006df0).ready entity=mon.2 client_cookie=846a5c9fb426b7f9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1659c700 1 -- 172.21.15.116:0/606172572 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08009770 con 0x7efe180c41c0 2020-11-20T12:17:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1659c700 1 -- 172.21.15.116:0/606172572 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe080098d0 con 0x7efe180c41c0 2020-11-20T12:17:19.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1659c700 1 -- 172.21.15.116:0/606172572 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08009c50 con 0x7efe180c41c0 2020-11-20T12:17:19.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1ca3f700 1 -- 172.21.15.116:0/606172572 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c41c0 msgr2=0x7efe180c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:19.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1ca3f700 1 --2- 172.21.15.116:0/606172572 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c41c0 0x7efe180c45a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7efe0800ba00 tx=0x7efe08006df0).stop 2020-11-20T12:17:19.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1ca3f700 1 -- 172.21.15.116:0/606172572 shutdown_connections 2020-11-20T12:17:19.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1ca3f700 1 --2- 172.21.15.116:0/606172572 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c41c0 0x7efe180c45a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:19.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1ca3f700 1 -- 172.21.15.116:0/606172572 >> 172.21.15.116:0/606172572 conn(0x7efe18062520 msgr2=0x7efe18062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:19.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1ca3f700 1 -- 172.21.15.116:0/606172572 shutdown_connections 2020-11-20T12:17:19.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.156+0000 7efe1ca3f700 1 -- 172.21.15.116:0/606172572 wait complete. 2020-11-20T12:17:19.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.160+0000 7efe1ca3f700 1 Processor -- start 2020-11-20T12:17:19.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.160+0000 7efe1ca3f700 1 -- start start 2020-11-20T12:17:19.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.160+0000 7efe1ca3f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180a5a10 0x7efe1814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:19.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.160+0000 7efe1ca3f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c41c0 0x7efe180a2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:19.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.160+0000 7efe1ca3f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe18138d80 0x7efe180a2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:19.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.160+0000 7efe1ca3f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe18132870 con 0x7efe180a5a10 2020-11-20T12:17:19.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.160+0000 7efe1ca3f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe18132b10 con 0x7efe18138d80 2020-11-20T12:17:19.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.160+0000 7efe1ca3f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe18132db0 con 0x7efe180c41c0 2020-11-20T12:17:19.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe16d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180a5a10 0x7efe1814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe16d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180a5a10 0x7efe1814ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45130/0 (socket says 172.21.15.116:45130) 2020-11-20T12:17:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe16d9d700 1 -- 172.21.15.116:0/1704726998 learned_addr learned my addr 172.21.15.116:0/1704726998 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe0e59c700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c41c0 0x7efe180a2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:19.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe1759e700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe18138d80 0x7efe180a2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe16d9d700 1 -- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c41c0 msgr2=0x7efe180a2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe16d9d700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c41c0 0x7efe180a2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe16d9d700 1 -- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe18138d80 msgr2=0x7efe180a2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe16d9d700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe18138d80 0x7efe180a2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:19.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe16d9d700 1 -- 172.21.15.116:0/1704726998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe18133050 con 0x7efe180a5a10 2020-11-20T12:17:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe16d9d700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180a5a10 0x7efe1814ed90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7efe0800c010 tx=0x7efe08015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe0f7fe700 1 -- 172.21.15.116:0/1704726998 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe08011de0 con 0x7efe180a5a10 2020-11-20T12:17:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe0f7fe700 1 -- 172.21.15.116:0/1704726998 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0801e460 con 0x7efe180a5a10 2020-11-20T12:17:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe1ca3f700 1 -- 172.21.15.116:0/1704726998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe18132560 con 0x7efe180a5a10 2020-11-20T12:17:19.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe0f7fe700 1 -- 172.21.15.116:0/1704726998 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0801e7e0 con 0x7efe180a5a10 2020-11-20T12:17:19.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe1ca3f700 1 -- 172.21.15.116:0/1704726998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe181334f0 con 0x7efe180a5a10 2020-11-20T12:17:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.164+0000 7efe1ca3f700 1 -- 172.21.15.116:0/1704726998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe18044f60 con 0x7efe180a5a10 2020-11-20T12:17:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.168+0000 7efe0f7fe700 1 -- 172.21.15.116:0/1704726998 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7efe08025030 con 0x7efe180a5a10 2020-11-20T12:17:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.168+0000 7efe0f7fe700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 0x7efdf4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:19.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.168+0000 7efe0f7fe700 1 -- 172.21.15.116:0/1704726998 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efe0806ff60 con 0x7efe180a5a10 2020-11-20T12:17:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.168+0000 7efe0e59c700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 0x7efdf4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:19.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.168+0000 7efe0e59c700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 0x7efdf4057c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7efe181649c0 tx=0x7efe00009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:19.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.172+0000 7efe0f7fe700 1 -- 172.21.15.116:0/1704726998 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efe08045e00 con 0x7efe180a5a10 2020-11-20T12:17:19.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.380+0000 7efe1ca3f700 1 -- 172.21.15.116:0/1704726998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe180a8e30 con 0x7efe180a5a10 2020-11-20T12:17:19.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.380+0000 7efe0f7fe700 1 -- 172.21.15.116:0/1704726998 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2851 (secure 0 0 0) 0x7efe08023020 con 0x7efe180a5a10 2020-11-20T12:17:19.381 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:19.381 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3031892761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3031892761},{"type":"v1","addr":"172.21.15.173:6836","nonce":3031892761}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1794965470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1794965470},{"type":"v1","addr":"172.21.15.173:6837","nonce":1794965470}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":9},{"gid":4995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/4143559744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":4143559744},{"type":"v1","addr":"172.21.15.116:6833","nonce":4143559744}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:57.901741+0000","modified":"2020-11-20T12:16:59.918401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4998":{"gid":4998,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6835/3994085646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3994085646},{"type":"v1","addr":"172.21.15.116:6835","nonce":3994085646}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T12:17:19.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 -- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 msgr2=0x7efdf4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:19.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 0x7efdf4057c80 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7efe181649c0 tx=0x7efe00009040).stop 2020-11-20T12:17:19.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 -- 172.21.15.116:0/1704726998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180a5a10 msgr2=0x7efe1814ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:19.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180a5a10 0x7efe1814ed90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7efe0800c010 tx=0x7efe08015000).stop 2020-11-20T12:17:19.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 -- 172.21.15.116:0/1704726998 shutdown_connections 2020-11-20T12:17:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 0x7efdf4057c80 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe18138d80 0x7efe180a2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe180a5a10 0x7efe1814ed90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:19.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 --2- 172.21.15.116:0/1704726998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c41c0 0x7efe180a2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 -- 172.21.15.116:0/1704726998 >> 172.21.15.116:0/1704726998 conn(0x7efe18062520 msgr2=0x7efe18062900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 -- 172.21.15.116:0/1704726998 shutdown_connections 2020-11-20T12:17:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:19.384+0000 7efe1ca3f700 1 -- 172.21.15.116:0/1704726998 wait complete. 2020-11-20T12:17:19.386 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 14 2020-11-20T12:17:19.810 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:57.901741+0000', 'modified': '2020-11-20T12:16:59.918401+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4998}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4998': {'gid': 4998, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.116:6835/3994085646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3994085646}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3994085646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:17:19.810 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:17:19.811 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:17:20.010 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4998,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":22.088608419,"mdsmap_epoch":14,"osdmap_epoch":34,"osdmap_epoch_barrier":34,"uptime":30.429674364} 2020-11-20T12:17:20.010 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4998, "mdsmap_epoch": 14, "osdmap_epoch": 34, "osdmap_epoch_barrier": 34, "rank_uptime": 22.088608419, "state": "up:active", "uptime": 30.429674364, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:17:20.011 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:17:20.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:20.012 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-20T12:17:20.024 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:17:20.025 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:20.135 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:20.136 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:17:20.163 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:20.164 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:17:20.171 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:17:20.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:20.172 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-20T12:17:20.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:17:20.203 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir parent1) 2020-11-20T12:17:20.246 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:17:20.246 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir parent2) 2020-11-20T12:17:20.280 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:17:20.281 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir parent1/mydir) 2020-11-20T12:17:20.368 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:17:20.369 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:17:20.369 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-20T12:17:20.370 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-20T12:17:20.370 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:17:20.370 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:17:20.371 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:17:20.371 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:17:20.372 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:20.372 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:17:20.387 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:17:20.388 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:17:20.388 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:17:20.388 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:20.388 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:17:20.388 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:20.389 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:20.389 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:20.389 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:20.389 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:20.390 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:20.390 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:20.390 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:17:20.390 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:20.390 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:17:20.391 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:20.391 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:17:20.391 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:20.391 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:17:20.391 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:20.392 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:17:20.392 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:20.392 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:17:20.392 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:20.392 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:17:20.393 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:20.393 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:17:20.393 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:20.393 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:20.394 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:17:20.394 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:17:20.394 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:20.394 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:20.394 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:17:20.395 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:17:20.395 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:20.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:20.397 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:17:20.432 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:17:20.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:20.436 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:17:20.495 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:17:20.495 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:17:20.496 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:17:20.496 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:17:20.497 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:17:20.497 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:17:20.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:20.499 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:17:20.499 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:17:20.499 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:17:20.500 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:17:20.500 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:17:20.501 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:17:20.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:20.603 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:17:20.617 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:17:20.618 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:17:20.619 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:17:20.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:20.620 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2020-11-20T12:17:20.659 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:20.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.832+0000 7f56da3bf700 1 Processor -- start 2020-11-20T12:17:20.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.832+0000 7f56da3bf700 1 -- start start 2020-11-20T12:17:20.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56da3bf700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56d40beee0 0x7f56d40bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:20.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56da3bf700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56d4152ee0 0x7f56d4060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:20.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56da3bf700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d4063e90 con 0x7f56d40664c0 2020-11-20T12:17:20.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56da3bf700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d4060810 con 0x7f56d40beee0 2020-11-20T12:17:20.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56da3bf700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d4060ab0 con 0x7f56d4152ee0 2020-11-20T12:17:20.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56d93bd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56d40beee0 0x7f56d40bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:20.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56d93bd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56d40beee0 0x7f56d40bf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45142/0 (socket says 172.21.15.116:45142) 2020-11-20T12:17:20.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56d93bd700 1 -- 172.21.15.116:0/860079508 learned_addr learned my addr 172.21.15.116:0/860079508 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:20.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56d8bbc700 1 --2- 172.21.15.116:0/860079508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56d4152ee0 0x7f56d4060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:20.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56d93bd700 1 -- 172.21.15.116:0/860079508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56d4152ee0 msgr2=0x7f56d4060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:20.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56d93bd700 1 --2- 172.21.15.116:0/860079508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56d4152ee0 0x7f56d4060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:20.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56d93bd700 1 -- 172.21.15.116:0/860079508 >> v1:172.21.15.116:6789/0 conn(0x7f56d40664c0 legacy=0x7f56d40642f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:20.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56d93bd700 1 -- 172.21.15.116:0/860079508 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d4060d50 con 0x7f56d40beee0 2020-11-20T12:17:20.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56d8bbc700 1 --2- 172.21.15.116:0/860079508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56d4152ee0 0x7f56d4060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:20.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56d93bd700 1 --2- 172.21.15.116:0/860079508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d40beee0 0x7f56d40bf2c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f56d0006fd0 tx=0x7f56d0006eb0).ready entity=mon.0 client_cookie=79c3ca7b63377f58 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:20.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56c3fff700 1 -- 172.21.15.116:0/860079508 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d00098d0 con 0x7f56d40beee0 2020-11-20T12:17:20.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56c3fff700 1 -- 172.21.15.116:0/860079508 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56d0009a30 con 0x7f56d40beee0 2020-11-20T12:17:20.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.836+0000 7f56c3fff700 1 -- 172.21.15.116:0/860079508 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56d0009db0 con 0x7f56d40beee0 2020-11-20T12:17:20.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.840+0000 7f56da3bf700 1 -- 172.21.15.116:0/860079508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d40beee0 msgr2=0x7f56d40bf2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:20.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.840+0000 7f56da3bf700 1 --2- 172.21.15.116:0/860079508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d40beee0 0x7f56d40bf2c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f56d0006fd0 tx=0x7f56d0006eb0).stop 2020-11-20T12:17:20.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.840+0000 7f56da3bf700 1 -- 172.21.15.116:0/860079508 shutdown_connections 2020-11-20T12:17:20.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.840+0000 7f56da3bf700 1 --2- 172.21.15.116:0/860079508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d40beee0 0x7f56d40bf2c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:20.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.840+0000 7f56da3bf700 1 --2- 172.21.15.116:0/860079508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56d4152ee0 0x7f56d4060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:20.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.840+0000 7f56da3bf700 1 -- 172.21.15.116:0/860079508 >> 172.21.15.116:0/860079508 conn(0x7f56d40c44e0 msgr2=0x7f56d40c48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:20.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.840+0000 7f56da3bf700 1 -- 172.21.15.116:0/860079508 shutdown_connections 2020-11-20T12:17:20.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.840+0000 7f56da3bf700 1 -- 172.21.15.116:0/860079508 wait complete. 2020-11-20T12:17:20.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56da3bf700 1 Processor -- start 2020-11-20T12:17:20.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56da3bf700 1 -- start start 2020-11-20T12:17:20.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56da3bf700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56d40664c0 0x7f56d4161a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:20.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56da3bf700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56d40beee0 0x7f56d4160850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:20.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56da3bf700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d4152ee0 0x7f56d40a99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:20.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56da3bf700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d4161f70 con 0x7f56d4152ee0 2020-11-20T12:17:20.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56da3bf700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d40bc7f0 con 0x7f56d40664c0 2020-11-20T12:17:20.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56da3bf700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d40bca90 con 0x7f56d40beee0 2020-11-20T12:17:20.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56d9bbe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d4152ee0 0x7f56d40a99f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:20.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56d9bbe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d4152ee0 0x7f56d40a99f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45152/0 (socket says 172.21.15.116:45152) 2020-11-20T12:17:20.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56d9bbe700 1 -- 172.21.15.116:0/4198841120 learned_addr learned my addr 172.21.15.116:0/4198841120 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:20.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56d8bbc700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56d40beee0 0x7f56d4160850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:20.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.844+0000 7f56d93bd700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56d40664c0 0x7f56d4161a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:20.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56d9bbe700 1 -- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56d40beee0 msgr2=0x7f56d4160850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:20.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56d9bbe700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56d40beee0 0x7f56d4160850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:20.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56d9bbe700 1 -- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56d40664c0 msgr2=0x7f56d4161a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:20.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56d9bbe700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56d40664c0 0x7f56d4161a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:20.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56d9bbe700 1 -- 172.21.15.116:0/4198841120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d40a7ec0 con 0x7f56d4152ee0 2020-11-20T12:17:20.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56d9bbe700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d4152ee0 0x7f56d40a99f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f56cc00ca60 tx=0x7f56cc007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:20.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56c17fa700 1 -- 172.21.15.116:0/4198841120 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc00a770 con 0x7f56d4152ee0 2020-11-20T12:17:20.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56c17fa700 1 -- 172.21.15.116:0/4198841120 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56cc00a8d0 con 0x7f56d4152ee0 2020-11-20T12:17:20.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56da3bf700 1 -- 172.21.15.116:0/4198841120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d40a8160 con 0x7f56d4152ee0 2020-11-20T12:17:20.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56c17fa700 1 -- 172.21.15.116:0/4198841120 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56cc00ac50 con 0x7f56d4152ee0 2020-11-20T12:17:20.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56da3bf700 1 -- 172.21.15.116:0/4198841120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d40a7730 con 0x7f56d4152ee0 2020-11-20T12:17:20.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56da3bf700 1 -- 172.21.15.116:0/4198841120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d4044f60 con 0x7f56d4152ee0 2020-11-20T12:17:20.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.848+0000 7f56c17fa700 1 -- 172.21.15.116:0/4198841120 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f56cc019030 con 0x7f56d4152ee0 2020-11-20T12:17:20.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.852+0000 7f56c17fa700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f56ac0557f0 0x7f56ac057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:20.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.852+0000 7f56c17fa700 1 -- 172.21.15.116:0/4198841120 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f56cc06ddb0 con 0x7f56d4152ee0 2020-11-20T12:17:20.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.852+0000 7f56d93bd700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f56ac0557f0 0x7f56ac057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:20.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.852+0000 7f56d93bd700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f56ac0557f0 0x7f56ac057cb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f56d000b1b0 tx=0x7f56d0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:20.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:20.856+0000 7f56c17fa700 1 -- 172.21.15.116:0/4198841120 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f56cc043d00 con 0x7f56d4152ee0 2020-11-20T12:17:21.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.016+0000 7f56da3bf700 1 -- 172.21.15.116:0/4198841120 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56d40639c0 con 0x7f56d4152ee0 2020-11-20T12:17:21.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.016+0000 7f56c17fa700 1 -- 172.21.15.116:0/4198841120 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f56cc025200 con 0x7f56d4152ee0 2020-11-20T12:17:21.018 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:21.018 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:58.907407+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:16:55.781578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T12:16:56.900539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:21.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 -- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f56ac0557f0 msgr2=0x7f56ac057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f56ac0557f0 0x7f56ac057cb0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f56d000b1b0 tx=0x7f56d0015000).stop 2020-11-20T12:17:21.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 -- 172.21.15.116:0/4198841120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d4152ee0 msgr2=0x7f56d40a99f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d4152ee0 0x7f56d40a99f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f56cc00ca60 tx=0x7f56cc007f40).stop 2020-11-20T12:17:21.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 -- 172.21.15.116:0/4198841120 shutdown_connections 2020-11-20T12:17:21.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f56ac0557f0 0x7f56ac057cb0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56d4152ee0 0x7f56d40a99f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56d40664c0 0x7f56d4161a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 --2- 172.21.15.116:0/4198841120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56d40beee0 0x7f56d4160850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 -- 172.21.15.116:0/4198841120 >> 172.21.15.116:0/4198841120 conn(0x7f56d40c44e0 msgr2=0x7f56d4148de0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:21.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 -- 172.21.15.116:0/4198841120 shutdown_connections 2020-11-20T12:17:21.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.020+0000 7f56da3bf700 1 -- 172.21.15.116:0/4198841120 wait complete. 2020-11-20T12:17:21.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:16:55.781578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:16:56.900539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:17:21.045 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:21.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.228+0000 7f6d14f70700 1 Processor -- start 2020-11-20T12:17:21.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.228+0000 7f6d14f70700 1 -- start start 2020-11-20T12:17:21.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d14f70700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6d1014bc70 0x7f6d100678e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d14f70700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6d100c1300 0x7f6d10067dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d14f70700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d100c1900 con 0x7f6d1014c050 2020-11-20T12:17:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d14f70700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d10155530 con 0x7f6d1014bc70 2020-11-20T12:17:21.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d14f70700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d101557d0 con 0x7f6d100c1300 2020-11-20T12:17:21.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d0e59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6d1014bc70 0x7f6d100678e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d0e59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6d1014bc70 0x7f6d100678e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45156/0 (socket says 172.21.15.116:45156) 2020-11-20T12:17:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d0e59c700 1 -- 172.21.15.116:0/3298202351 learned_addr learned my addr 172.21.15.116:0/3298202351 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d0ed9d700 1 --2- 172.21.15.116:0/3298202351 >> v2:172.21.15.173:3300/0 conn(0x7f6d100c1300 0x7f6d10067dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d0e59c700 1 -- 172.21.15.116:0/3298202351 >> v2:172.21.15.173:3300/0 conn(0x7f6d100c1300 msgr2=0x7f6d10067dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d0e59c700 1 --2- 172.21.15.116:0/3298202351 >> v2:172.21.15.173:3300/0 conn(0x7f6d100c1300 0x7f6d10067dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d0e59c700 1 -- 172.21.15.116:0/3298202351 >> v1:172.21.15.173:6789/0 conn(0x7f6d1014c050 legacy=0x7f6d100c0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d0d59a700 1 -- 172.21.15.116:0/3298202351 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2403179849 0 0) 0x7f6d100c1900 con 0x7f6d1014c050 2020-11-20T12:17:21.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.232+0000 7f6d0e59c700 1 -- 172.21.15.116:0/3298202351 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d10155a70 con 0x7f6d1014bc70 2020-11-20T12:17:21.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d0e59c700 1 --2- 172.21.15.116:0/3298202351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d1014bc70 0x7f6d100678e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f6d10068420 tx=0x7f6d00006eb0).ready entity=mon.0 client_cookie=9312ff873c429613 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:21.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d0d59a700 1 -- 172.21.15.116:0/3298202351 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d000098a0 con 0x7f6d1014bc70 2020-11-20T12:17:21.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d0d59a700 1 -- 172.21.15.116:0/3298202351 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d00009a00 con 0x7f6d1014bc70 2020-11-20T12:17:21.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d0d59a700 1 -- 172.21.15.116:0/3298202351 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d00009d80 con 0x7f6d1014bc70 2020-11-20T12:17:21.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d14f70700 1 -- 172.21.15.116:0/3298202351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d1014bc70 msgr2=0x7f6d100678e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d14f70700 1 --2- 172.21.15.116:0/3298202351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d1014bc70 0x7f6d100678e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f6d10068420 tx=0x7f6d00006eb0).stop 2020-11-20T12:17:21.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d14f70700 1 -- 172.21.15.116:0/3298202351 shutdown_connections 2020-11-20T12:17:21.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d14f70700 1 --2- 172.21.15.116:0/3298202351 >> v2:172.21.15.173:3300/0 conn(0x7f6d100c1300 0x7f6d10067dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d14f70700 1 --2- 172.21.15.116:0/3298202351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d1014bc70 0x7f6d100678e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d14f70700 1 -- 172.21.15.116:0/3298202351 >> 172.21.15.116:0/3298202351 conn(0x7f6d10060170 msgr2=0x7f6d10060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:21.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d14f70700 1 -- 172.21.15.116:0/3298202351 shutdown_connections 2020-11-20T12:17:21.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.236+0000 7f6d14f70700 1 -- 172.21.15.116:0/3298202351 wait complete. 2020-11-20T12:17:21.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d14f70700 1 Processor -- start 2020-11-20T12:17:21.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d14f70700 1 -- start start 2020-11-20T12:17:21.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d14f70700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d100c1300 0x7f6d100c27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d14f70700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d1014bc70 0x7f6d100a5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d14f70700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d1014c050 0x7f6d100a6220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d14f70700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d10161100 con 0x7f6d100c1300 2020-11-20T12:17:21.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d14f70700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d100a72b0 con 0x7f6d1014c050 2020-11-20T12:17:21.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d14f70700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d100a7550 con 0x7f6d1014bc70 2020-11-20T12:17:21.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d0e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d100c1300 0x7f6d100c27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d0e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d100c1300 0x7f6d100c27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45162/0 (socket says 172.21.15.116:45162) 2020-11-20T12:17:21.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d0e59c700 1 -- 172.21.15.116:0/3822762544 learned_addr learned my addr 172.21.15.116:0/3822762544 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:21.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d0ed9d700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d1014c050 0x7f6d100a6220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d0dd9b700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d1014bc70 0x7f6d100a5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d0e59c700 1 -- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d1014bc70 msgr2=0x7f6d100a5350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d0e59c700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d1014bc70 0x7f6d100a5350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d0e59c700 1 -- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d1014c050 msgr2=0x7f6d100a6220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d0e59c700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d1014c050 0x7f6d100a6220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d0e59c700 1 -- 172.21.15.116:0/3822762544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d100a77f0 con 0x7f6d100c1300 2020-11-20T12:17:21.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.240+0000 7f6d0dd9b700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d1014bc70 0x7f6d100a5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:21.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.244+0000 7f6d0e59c700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d100c1300 0x7f6d100c27b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f6d00009fd0 tx=0x7f6d00015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:21.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.244+0000 7f6cfe7fc700 1 -- 172.21.15.116:0/3822762544 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0001e460 con 0x7f6d100c1300 2020-11-20T12:17:21.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.244+0000 7f6d14f70700 1 -- 172.21.15.116:0/3822762544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d100a5d80 con 0x7f6d100c1300 2020-11-20T12:17:21.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.248+0000 7f6cfe7fc700 1 -- 172.21.15.116:0/3822762544 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0001e5c0 con 0x7f6d100c1300 2020-11-20T12:17:21.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.248+0000 7f6d14f70700 1 -- 172.21.15.116:0/3822762544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d100aa120 con 0x7f6d100c1300 2020-11-20T12:17:21.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.248+0000 7f6cfe7fc700 1 -- 172.21.15.116:0/3822762544 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0001e940 con 0x7f6d100c1300 2020-11-20T12:17:21.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.248+0000 7f6d14f70700 1 -- 172.21.15.116:0/3822762544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1005ca30 con 0x7f6d100c1300 2020-11-20T12:17:21.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.248+0000 7f6cfe7fc700 1 -- 172.21.15.116:0/3822762544 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6d00025030 con 0x7f6d100c1300 2020-11-20T12:17:21.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.248+0000 7f6cfe7fc700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ce40557c0 0x7f6ce4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.248+0000 7f6cfe7fc700 1 -- 172.21.15.116:0/3822762544 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6d0006fec0 con 0x7f6d100c1300 2020-11-20T12:17:21.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.252+0000 7f6d0dd9b700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ce40557c0 0x7f6ce4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.252+0000 7f6d0dd9b700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ce40557c0 0x7f6ce4057c80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f6d10165850 tx=0x7f6d04009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:21.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.256+0000 7f6cfe7fc700 1 -- 172.21.15.116:0/3822762544 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6d00045e90 con 0x7f6d100c1300 2020-11-20T12:17:21.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.416+0000 7f6d14f70700 1 -- 172.21.15.116:0/3822762544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6d10061e80 con 0x7f6d100c1300 2020-11-20T12:17:21.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.416+0000 7f6cfe7fc700 1 -- 172.21.15.116:0/3822762544 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v34) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6d00023070 con 0x7f6d100c1300 2020-11-20T12:17:21.417 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:21.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 -- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ce40557c0 msgr2=0x7f6ce4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ce40557c0 0x7f6ce4057c80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f6d10165850 tx=0x7f6d04009040).stop 2020-11-20T12:17:21.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 -- 172.21.15.116:0/3822762544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d100c1300 msgr2=0x7f6d100c27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d100c1300 0x7f6d100c27b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f6d00009fd0 tx=0x7f6d00015000).stop 2020-11-20T12:17:21.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 -- 172.21.15.116:0/3822762544 shutdown_connections 2020-11-20T12:17:21.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ce40557c0 0x7f6ce4057c80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6d1014c050 0x7f6d100a6220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6d100c1300 0x7f6d100c27b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 --2- 172.21.15.116:0/3822762544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6d1014bc70 0x7f6d100a5350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 -- 172.21.15.116:0/3822762544 >> 172.21.15.116:0/3822762544 conn(0x7f6d10060170 msgr2=0x7f6d10150e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:21.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 -- 172.21.15.116:0/3822762544 shutdown_connections 2020-11-20T12:17:21.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.420+0000 7f6d14f70700 1 -- 172.21.15.116:0/3822762544 wait complete. 2020-11-20T12:17:21.443 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:21.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.616+0000 7fae08797700 1 Processor -- start 2020-11-20T12:17:21.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.616+0000 7fae08797700 1 -- start start 2020-11-20T12:17:21.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae08797700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fae001502e0 0x7fae000ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae08797700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae000badd0 0x7fae000beb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae08797700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae00066250 con 0x7fae000bf080 2020-11-20T12:17:21.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae08797700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae000bf4f0 con 0x7fae001502e0 2020-11-20T12:17:21.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae08797700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae00067170 con 0x7fae000badd0 2020-11-20T12:17:21.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae06d34700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fae000bf080 0x7fae00062440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54604/0 (socket says 172.21.15.116:54604) 2020-11-20T12:17:21.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae06d34700 1 -- 172.21.15.116:0/4017043337 learned_addr learned my addr 172.21.15.116:0/4017043337 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:21.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05d32700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae000badd0 0x7fae000beb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae06533700 1 --2- 172.21.15.116:0/4017043337 >> v2:172.21.15.173:3300/0 conn(0x7fae001502e0 0x7fae000ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3436888320 0 0) 0x7fae00066250 con 0x7fae000bf080 2020-11-20T12:17:21.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade8003240 con 0x7fae000bf080 2020-11-20T12:17:21.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3381698913 0 0) 0x7fade8003240 con 0x7fae000bf080 2020-11-20T12:17:21.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae000badd0 msgr2=0x7fae000beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:21.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae000badd0 0x7fae000beb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 >> v2:172.21.15.173:3300/0 conn(0x7fae001502e0 msgr2=0x7fae000ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:21.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 --2- 172.21.15.116:0/4017043337 >> v2:172.21.15.173:3300/0 conn(0x7fae001502e0 0x7fae000ba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae000664f0 con 0x7fae000bf080 2020-11-20T12:17:21.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fadf8001860 con 0x7fae000bf080 2020-11-20T12:17:21.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 >> v1:172.21.15.116:6789/0 conn(0x7fae000bf080 legacy=0x7fae00062440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fade80037e0 0x7fade8003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fade80044e0 0x7fade8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fade8004e50 0x7fade800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae000bf4f0 con 0x7fade80037e0 2020-11-20T12:17:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae00067170 con 0x7fade80044e0 2020-11-20T12:17:21.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade8005290 con 0x7fade8004e50 2020-11-20T12:17:21.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae06533700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fade80037e0 0x7fade8003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae06d34700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fade8004e50 0x7fade800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.620+0000 7fae05d32700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fade80044e0 0x7fade8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae06d34700 1 -- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fade80044e0 msgr2=0x7fade8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:21.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae06d34700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fade80044e0 0x7fade8004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae06d34700 1 -- 172.21.15.116:0/4017043337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fade80037e0 msgr2=0x7fade8003c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae06d34700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fade80037e0 0x7fade8003c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae06d34700 1 -- 172.21.15.116:0/4017043337 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade8003240 con 0x7fade8004e50 2020-11-20T12:17:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae06d34700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fade8004e50 0x7fade800b730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fadf800c550 tx=0x7fadf8007a50).ready entity=mon.2 client_cookie=e1940047f0f064b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8005b80 con 0x7fade8004e50 2020-11-20T12:17:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf8005ce0 con 0x7fade8004e50 2020-11-20T12:17:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae05531700 1 -- 172.21.15.116:0/4017043337 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf8007300 con 0x7fade8004e50 2020-11-20T12:17:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae08797700 1 -- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fade8004e50 msgr2=0x7fade800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae08797700 1 --2- 172.21.15.116:0/4017043337 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fade8004e50 0x7fade800b730 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fadf800c550 tx=0x7fadf8007a50).stop 2020-11-20T12:17:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae06d34700 1 -- 172.21.15.116:0/4017043337 reap_dead start 2020-11-20T12:17:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae08797700 1 -- 172.21.15.116:0/4017043337 shutdown_connections 2020-11-20T12:17:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae08797700 1 -- 172.21.15.116:0/4017043337 >> 172.21.15.116:0/4017043337 conn(0x7fae00060170 msgr2=0x7fae00060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae08797700 1 -- 172.21.15.116:0/4017043337 shutdown_connections 2020-11-20T12:17:21.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.624+0000 7fae08797700 1 -- 172.21.15.116:0/4017043337 wait complete. 2020-11-20T12:17:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.628+0000 7fae08797700 1 Processor -- start 2020-11-20T12:17:21.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.628+0000 7fae08797700 1 -- start start 2020-11-20T12:17:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.628+0000 7fae08797700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fade8004e50 0x7fae0014e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.628+0000 7fae08797700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000c3b50 0x7fae000c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.628+0000 7fae08797700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae0009d620 0x7fae0009da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.628+0000 7fae08797700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae00165100 con 0x7fae000c3b50 2020-11-20T12:17:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.628+0000 7fae08797700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae00165360 con 0x7fae0009d620 2020-11-20T12:17:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.628+0000 7fae08797700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae00165600 con 0x7fade8004e50 2020-11-20T12:17:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae05d32700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000c3b50 0x7fae000c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae05d32700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000c3b50 0x7fae000c3fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45184/0 (socket says 172.21.15.116:45184) 2020-11-20T12:17:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae05d32700 1 -- 172.21.15.116:0/3941945227 learned_addr learned my addr 172.21.15.116:0/3941945227 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae06533700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fade8004e50 0x7fae0014e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae06d34700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae0009d620 0x7fae0009da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae05d32700 1 -- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fade8004e50 msgr2=0x7fae0014e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae05d32700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fade8004e50 0x7fae0014e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae05d32700 1 -- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae0009d620 msgr2=0x7fae0009da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae05d32700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae0009d620 0x7fae0009da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae05d32700 1 -- 172.21.15.116:0/3941945227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae001658a0 con 0x7fae000c3b50 2020-11-20T12:17:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae06d34700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae0009d620 0x7fae0009da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae05d32700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000c3b50 0x7fae000c3fd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fadf0006860 tx=0x7fadf0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fadee7fc700 1 -- 172.21.15.116:0/3941945227 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0018070 con 0x7fae000c3b50 2020-11-20T12:17:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fadee7fc700 1 -- 172.21.15.116:0/3941945227 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf000faa0 con 0x7fae000c3b50 2020-11-20T12:17:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae08797700 1 -- 172.21.15.116:0/3941945227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae00165b40 con 0x7fae000c3b50 2020-11-20T12:17:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fadee7fc700 1 -- 172.21.15.116:0/3941945227 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf000fe20 con 0x7fae000c3b50 2020-11-20T12:17:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae08797700 1 -- 172.21.15.116:0/3941945227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae0014ee60 con 0x7fae000c3b50 2020-11-20T12:17:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fadee7fc700 1 -- 172.21.15.116:0/3941945227 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fadf0014650 con 0x7fae000c3b50 2020-11-20T12:17:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.632+0000 7fae08797700 1 -- 172.21.15.116:0/3941945227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae0005ca30 con 0x7fae000c3b50 2020-11-20T12:17:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.636+0000 7fadee7fc700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadd80557c0 0x7fadd8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.636+0000 7fadee7fc700 1 -- 172.21.15.116:0/3941945227 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fadf006ef10 con 0x7fae000c3b50 2020-11-20T12:17:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.636+0000 7fae06533700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadd80557c0 0x7fadd8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.636+0000 7fae06533700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadd80557c0 0x7fadd8057c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fadfc006860 tx=0x7fadfc006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.640+0000 7fadee7fc700 1 -- 172.21.15.116:0/3941945227 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fadf0044db0 con 0x7fae000c3b50 2020-11-20T12:17:21.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.812+0000 7fae08797700 1 -- 172.21.15.116:0/3941945227 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fae00067e60 con 0x7fae000c3b50 2020-11-20T12:17:21.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.812+0000 7fadee7fc700 1 -- 172.21.15.116:0/3941945227 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7fadf00483d0 con 0x7fae000c3b50 2020-11-20T12:17:21.812 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:21.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.812+0000 7fae08797700 1 -- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadd80557c0 msgr2=0x7fadd8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.812+0000 7fae08797700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadd80557c0 0x7fadd8057c80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fadfc006860 tx=0x7fadfc006df0).stop 2020-11-20T12:17:21.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.812+0000 7fae08797700 1 -- 172.21.15.116:0/3941945227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000c3b50 msgr2=0x7fae000c3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:21.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.812+0000 7fae08797700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000c3b50 0x7fae000c3fd0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fadf0006860 tx=0x7fadf0006df0).stop 2020-11-20T12:17:21.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.816+0000 7fae08797700 1 -- 172.21.15.116:0/3941945227 shutdown_connections 2020-11-20T12:17:21.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.816+0000 7fae08797700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadd80557c0 0x7fadd8057c80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.816+0000 7fae08797700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae0009d620 0x7fae0009da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.816+0000 7fae08797700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fade8004e50 0x7fae0014e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.816+0000 7fae08797700 1 --2- 172.21.15.116:0/3941945227 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae000c3b50 0x7fae000c3fd0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:21.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.816+0000 7fae08797700 1 -- 172.21.15.116:0/3941945227 >> 172.21.15.116:0/3941945227 conn(0x7fae00060170 msgr2=0x7fae000c4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:21.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.816+0000 7fae08797700 1 -- 172.21.15.116:0/3941945227 shutdown_connections 2020-11-20T12:17:21.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:21.816+0000 7fae08797700 1 -- 172.21.15.116:0/3941945227 wait complete. 2020-11-20T12:17:21.838 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:17:22.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.016+0000 7fda2cff6700 1 Processor -- start 2020-11-20T12:17:22.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.016+0000 7fda2cff6700 1 -- start start 2020-11-20T12:17:22.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.016+0000 7fda2cff6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fda280c1bb0 0x7fda28156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.016+0000 7fda2cff6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda280a38c0 con 0x7fda280c03f0 2020-11-20T12:17:22.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.016+0000 7fda2cff6700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda28156e30 con 0x7fda280a28c0 2020-11-20T12:17:22.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.016+0000 7fda2cff6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda280a2e40 con 0x7fda280c1bb0 2020-11-20T12:17:22.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda277fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fda280c03f0 0x7fda280677c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54624/0 (socket says 172.21.15.116:54624) 2020-11-20T12:17:22.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda277fe700 1 -- 172.21.15.116:0/918551774 learned_addr learned my addr 172.21.15.116:0/918551774 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:22.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda26ffd700 1 --2- 172.21.15.116:0/918551774 >> v2:172.21.15.116:3300/0 conn(0x7fda280c1bb0 0x7fda28156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda26ffd700 1 -- 172.21.15.116:0/918551774 >> v1:172.21.15.173:6789/0 conn(0x7fda280a28c0 legacy=0x7fda28066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda267fc700 1 -- 172.21.15.116:0/918551774 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552698868 0 0) 0x7fda28156e30 con 0x7fda280a28c0 2020-11-20T12:17:22.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda26ffd700 1 -- 172.21.15.116:0/918551774 >> v1:172.21.15.116:6789/0 conn(0x7fda280c03f0 legacy=0x7fda280677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda26ffd700 1 -- 172.21.15.116:0/918551774 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda280a30e0 con 0x7fda280c1bb0 2020-11-20T12:17:22.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda26ffd700 1 --2- 172.21.15.116:0/918551774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda280c1bb0 0x7fda28156940 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fda1c00b9a0 tx=0x7fda1c006df0).ready entity=mon.0 client_cookie=732d3e25f1e3275f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:22.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda267fc700 1 -- 172.21.15.116:0/918551774 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c0096b0 con 0x7fda280c1bb0 2020-11-20T12:17:22.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda267fc700 1 -- 172.21.15.116:0/918551774 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda1c009810 con 0x7fda280c1bb0 2020-11-20T12:17:22.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda267fc700 1 -- 172.21.15.116:0/918551774 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda1c009b90 con 0x7fda280c1bb0 2020-11-20T12:17:22.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda2cff6700 1 -- 172.21.15.116:0/918551774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda280c1bb0 msgr2=0x7fda28156940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda2cff6700 1 --2- 172.21.15.116:0/918551774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda280c1bb0 0x7fda28156940 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fda1c00b9a0 tx=0x7fda1c006df0).stop 2020-11-20T12:17:22.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda2cff6700 1 -- 172.21.15.116:0/918551774 shutdown_connections 2020-11-20T12:17:22.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda2cff6700 1 --2- 172.21.15.116:0/918551774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda280c1bb0 0x7fda28156940 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda2cff6700 1 -- 172.21.15.116:0/918551774 >> 172.21.15.116:0/918551774 conn(0x7fda28060c30 msgr2=0x7fda280c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:22.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda2cff6700 1 -- 172.21.15.116:0/918551774 shutdown_connections 2020-11-20T12:17:22.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.020+0000 7fda2cff6700 1 -- 172.21.15.116:0/918551774 wait complete. 2020-11-20T12:17:22.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.024+0000 7fda2cff6700 1 Processor -- start 2020-11-20T12:17:22.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.024+0000 7fda2cff6700 1 -- start start 2020-11-20T12:17:22.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda2cff6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda280a28c0 0x7fda280a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda2cff6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda280c03f0 0x7fda280a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda2cff6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda280c1bb0 0x7fda280c2560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda2cff6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda28153d60 con 0x7fda280a28c0 2020-11-20T12:17:22.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda2cff6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda28154000 con 0x7fda280c1bb0 2020-11-20T12:17:22.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda2cff6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda281542a0 con 0x7fda280c03f0 2020-11-20T12:17:22.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda277fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda280a28c0 0x7fda280a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda277fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda280a28c0 0x7fda280a77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45196/0 (socket says 172.21.15.116:45196) 2020-11-20T12:17:22.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda277fe700 1 -- 172.21.15.116:0/1743777151 learned_addr learned my addr 172.21.15.116:0/1743777151 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:22.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda26ffd700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda280c03f0 0x7fda280a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda27fff700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda280c1bb0 0x7fda280c2560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda277fe700 1 -- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda280c03f0 msgr2=0x7fda280a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda277fe700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda280c03f0 0x7fda280a7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda277fe700 1 -- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda280c1bb0 msgr2=0x7fda280c2560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda277fe700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda280c1bb0 0x7fda280c2560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda277fe700 1 -- 172.21.15.116:0/1743777151 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28154540 con 0x7fda280a28c0 2020-11-20T12:17:22.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda277fe700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda280a28c0 0x7fda280a77f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fda1800c8b0 tx=0x7fda18007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:22.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda077fe700 1 -- 172.21.15.116:0/1743777151 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18007320 con 0x7fda280a28c0 2020-11-20T12:17:22.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda077fe700 1 -- 172.21.15.116:0/1743777151 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda18007480 con 0x7fda280a28c0 2020-11-20T12:17:22.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda2cff6700 1 -- 172.21.15.116:0/1743777151 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda28090130 con 0x7fda280a28c0 2020-11-20T12:17:22.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda077fe700 1 -- 172.21.15.116:0/1743777151 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda18005800 con 0x7fda280a28c0 2020-11-20T12:17:22.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.028+0000 7fda2cff6700 1 -- 172.21.15.116:0/1743777151 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda280c2c70 con 0x7fda280a28c0 2020-11-20T12:17:22.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.032+0000 7fda2cff6700 1 -- 172.21.15.116:0/1743777151 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda28044f60 con 0x7fda280a28c0 2020-11-20T12:17:22.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.032+0000 7fda077fe700 1 -- 172.21.15.116:0/1743777151 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fda18010030 con 0x7fda280a28c0 2020-11-20T12:17:22.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.032+0000 7fda077fe700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda000557c0 0x7fda00057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.032+0000 7fda077fe700 1 -- 172.21.15.116:0/1743777151 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fda1806e3c0 con 0x7fda280a28c0 2020-11-20T12:17:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.036+0000 7fda26ffd700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda000557c0 0x7fda00057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.036+0000 7fda26ffd700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda000557c0 0x7fda00057c80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fda1c006e30 tx=0x7fda1c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:22.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.040+0000 7fda077fe700 1 -- 172.21.15.116:0/1743777151 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fda18044260 con 0x7fda280a28c0 2020-11-20T12:17:22.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 -- 172.21.15.116:0/1743777151 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fda28064080 con 0x7fda280a28c0 2020-11-20T12:17:22.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda077fe700 1 -- 172.21.15.116:0/1743777151 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+10 (secure 0 0 0) 0x7fda18047880 con 0x7fda280a28c0 2020-11-20T12:17:22.200 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:22.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 -- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda000557c0 msgr2=0x7fda00057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda000557c0 0x7fda00057c80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fda1c006e30 tx=0x7fda1c015000).stop 2020-11-20T12:17:22.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 -- 172.21.15.116:0/1743777151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda280a28c0 msgr2=0x7fda280a77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda280a28c0 0x7fda280a77f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fda1800c8b0 tx=0x7fda18007c00).stop 2020-11-20T12:17:22.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 -- 172.21.15.116:0/1743777151 shutdown_connections 2020-11-20T12:17:22.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda000557c0 0x7fda00057c80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda280c1bb0 0x7fda280c2560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda280a28c0 0x7fda280a77f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 --2- 172.21.15.116:0/1743777151 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda280c03f0 0x7fda280a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 -- 172.21.15.116:0/1743777151 >> 172.21.15.116:0/1743777151 conn(0x7fda28060c30 msgr2=0x7fda28152d60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:22.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 -- 172.21.15.116:0/1743777151 shutdown_connections 2020-11-20T12:17:22.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.200+0000 7fda2cff6700 1 -- 172.21.15.116:0/1743777151 wait complete. 2020-11-20T12:17:22.225 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:17:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.400+0000 7ff03359e700 1 Processor -- start 2020-11-20T12:17:22.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.400+0000 7ff03359e700 1 -- start start 2020-11-20T12:17:22.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.404+0000 7ff03359e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff03413ad30 0x7ff0340a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.404+0000 7ff03359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413b110 0x7ff034151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.404+0000 7ff03359e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff034151fa0 0x7ff034061250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.404+0000 7ff03359e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff034152a80 con 0x7ff034151fa0 2020-11-20T12:17:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.404+0000 7ff03359e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff034152540 con 0x7ff03413ad30 2020-11-20T12:17:22.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.404+0000 7ff03359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0341527e0 con 0x7ff03413b110 2020-11-20T12:17:22.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.404+0000 7ff031d9b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413b110 0x7ff034151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.404+0000 7ff03259c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff03413ad30 0x7ff0340a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.404+0000 7ff031d9b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413b110 0x7ff034151ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:54156/0 (socket says 172.21.15.116:54156) 2020-11-20T12:17:22.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.404+0000 7ff032d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff034151fa0 0x7ff034061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.404+0000 7ff031d9b700 1 -- 172.21.15.116:0/1967918708 learned_addr learned my addr 172.21.15.116:0/1967918708 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03259c700 1 -- 172.21.15.116:0/1967918708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413b110 msgr2=0x7ff034151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:22.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03259c700 1 --2- 172.21.15.116:0/1967918708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413b110 0x7ff034151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03259c700 1 -- 172.21.15.116:0/1967918708 >> v2:172.21.15.116:3300/0 conn(0x7ff034151fa0 msgr2=0x7ff034061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03259c700 1 --2- 172.21.15.116:0/1967918708 >> v2:172.21.15.116:3300/0 conn(0x7ff034151fa0 0x7ff034061250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03259c700 1 -- 172.21.15.116:0/1967918708 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340ac910 con 0x7ff03413ad30 2020-11-20T12:17:22.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03259c700 1 --2- 172.21.15.116:0/1967918708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03413ad30 0x7ff0340a7fb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7ff034081330 tx=0x7ff024006df0).ready entity=mon.1 client_cookie=8b388e9c2f556771 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:22.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03159a700 1 -- 172.21.15.116:0/1967918708 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0240096d0 con 0x7ff03413ad30 2020-11-20T12:17:22.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03159a700 1 -- 172.21.15.116:0/1967918708 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff024009830 con 0x7ff03413ad30 2020-11-20T12:17:22.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03159a700 1 -- 172.21.15.116:0/1967918708 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff024009bb0 con 0x7ff03413ad30 2020-11-20T12:17:22.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03359e700 1 -- 172.21.15.116:0/1967918708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03413ad30 msgr2=0x7ff0340a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03359e700 1 --2- 172.21.15.116:0/1967918708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03413ad30 0x7ff0340a7fb0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7ff034081330 tx=0x7ff024006df0).stop 2020-11-20T12:17:22.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03359e700 1 -- 172.21.15.116:0/1967918708 shutdown_connections 2020-11-20T12:17:22.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03359e700 1 --2- 172.21.15.116:0/1967918708 >> v2:172.21.15.116:3300/0 conn(0x7ff034151fa0 0x7ff034061250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03359e700 1 --2- 172.21.15.116:0/1967918708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03413ad30 0x7ff0340a7fb0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03359e700 1 --2- 172.21.15.116:0/1967918708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413b110 0x7ff034151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03359e700 1 -- 172.21.15.116:0/1967918708 >> 172.21.15.116:0/1967918708 conn(0x7ff0340678a0 msgr2=0x7ff034067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03359e700 1 -- 172.21.15.116:0/1967918708 shutdown_connections 2020-11-20T12:17:22.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.408+0000 7ff03359e700 1 -- 172.21.15.116:0/1967918708 wait complete. 2020-11-20T12:17:22.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.412+0000 7ff03359e700 1 Processor -- start 2020-11-20T12:17:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.412+0000 7ff03359e700 1 -- start start 2020-11-20T12:17:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff03359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413ad30 0x7ff03409fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff03359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03413b110 0x7ff034164f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff03359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff034151fa0 0x7ff0340a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff03359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff034138ae0 con 0x7ff034151fa0 2020-11-20T12:17:22.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff03359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340a0460 con 0x7ff03413b110 2020-11-20T12:17:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff03359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0340a0700 con 0x7ff03413ad30 2020-11-20T12:17:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff032d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff034151fa0 0x7ff0340a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff032d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff034151fa0 0x7ff0340a1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45214/0 (socket says 172.21.15.116:45214) 2020-11-20T12:17:22.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff032d9d700 1 -- 172.21.15.116:0/4033707828 learned_addr learned my addr 172.21.15.116:0/4033707828 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:22.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff031d9b700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03413b110 0x7ff034164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff03259c700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413ad30 0x7ff03409fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff032d9d700 1 -- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413ad30 msgr2=0x7ff03409fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff032d9d700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413ad30 0x7ff03409fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff032d9d700 1 -- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03413b110 msgr2=0x7ff034164f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff032d9d700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03413b110 0x7ff034164f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff032d9d700 1 -- 172.21.15.116:0/4033707828 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0340a09a0 con 0x7ff034151fa0 2020-11-20T12:17:22.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff03259c700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413ad30 0x7ff03409fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:22.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff032d9d700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff034151fa0 0x7ff0340a1ca0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7ff02c00ed20 tx=0x7ff02c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:22.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff02a7fc700 1 -- 172.21.15.116:0/4033707828 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c00eee0 con 0x7ff034151fa0 2020-11-20T12:17:22.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff02a7fc700 1 -- 172.21.15.116:0/4033707828 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff02c00cbb0 con 0x7ff034151fa0 2020-11-20T12:17:22.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff03359e700 1 -- 172.21.15.116:0/4033707828 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0340a0c40 con 0x7ff034151fa0 2020-11-20T12:17:22.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff02a7fc700 1 -- 172.21.15.116:0/4033707828 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff02c0123f0 con 0x7ff034151fa0 2020-11-20T12:17:22.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.416+0000 7ff03359e700 1 -- 172.21.15.116:0/4033707828 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff034165440 con 0x7ff034151fa0 2020-11-20T12:17:22.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.420+0000 7ff03359e700 1 -- 172.21.15.116:0/4033707828 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0340aa1b0 con 0x7ff034151fa0 2020-11-20T12:17:22.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.420+0000 7ff02a7fc700 1 -- 172.21.15.116:0/4033707828 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff02c022070 con 0x7ff034151fa0 2020-11-20T12:17:22.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.420+0000 7ff02a7fc700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 0x7ff010057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.420+0000 7ff02a7fc700 1 -- 172.21.15.116:0/4033707828 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff02c06eda0 con 0x7ff034151fa0 2020-11-20T12:17:22.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.420+0000 7ff03259c700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 0x7ff010057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.420+0000 7ff03259c700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 0x7ff010057c80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7ff024006e30 tx=0x7ff024015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:22.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.428+0000 7ff02a7fc700 1 -- 172.21.15.116:0/4033707828 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff0340aa1b0 con 0x7ff034151fa0 2020-11-20T12:17:22.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff03359e700 1 -- 172.21.15.116:0/4033707828 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0340aa1b0 con 0x7ff034151fa0 2020-11-20T12:17:22.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff02a7fc700 1 -- 172.21.15.116:0/4033707828 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff0340aa1b0 con 0x7ff034151fa0 2020-11-20T12:17:22.584 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:22.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff03359e700 1 -- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 msgr2=0x7ff010057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff03359e700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 0x7ff010057c80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7ff024006e30 tx=0x7ff024015000).stop 2020-11-20T12:17:22.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff03359e700 1 -- 172.21.15.116:0/4033707828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff034151fa0 msgr2=0x7ff0340a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff03359e700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff034151fa0 0x7ff0340a1ca0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7ff02c00ed20 tx=0x7ff02c0091d0).stop 2020-11-20T12:17:22.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff03359e700 1 -- 172.21.15.116:0/4033707828 shutdown_connections 2020-11-20T12:17:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff03359e700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff0100557c0 0x7ff010057c80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff03359e700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff034151fa0 0x7ff0340a1ca0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff03359e700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff03413ad30 0x7ff03409fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff03359e700 1 --2- 172.21.15.116:0/4033707828 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff03413b110 0x7ff034164f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.584+0000 7ff03359e700 1 -- 172.21.15.116:0/4033707828 >> 172.21.15.116:0/4033707828 conn(0x7ff0340678a0 msgr2=0x7ff0340adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:22.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.588+0000 7ff03359e700 1 -- 172.21.15.116:0/4033707828 shutdown_connections 2020-11-20T12:17:22.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.588+0000 7ff03359e700 1 -- 172.21.15.116:0/4033707828 wait complete. 2020-11-20T12:17:22.612 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:17:22.613 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:22.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.788+0000 7ff370441700 1 Processor -- start 2020-11-20T12:17:22.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.788+0000 7ff370441700 1 -- start start 2020-11-20T12:17:22.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff370441700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff3680c3380 0x7ff3680c3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff370441700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff36814c280 0x7ff368067d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff370441700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff368068790 con 0x7ff36814f5c0 2020-11-20T12:17:22.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff370441700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff368068260 con 0x7ff3680c3380 2020-11-20T12:17:22.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff370441700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680684f0 con 0x7ff36814c280 2020-11-20T12:17:22.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff36e1dd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff3680c3380 0x7ff3680c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff36e1dd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff3680c3380 0x7ff3680c3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45218/0 (socket says 172.21.15.116:45218) 2020-11-20T12:17:22.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff36e1dd700 1 -- 172.21.15.116:0/1453029663 learned_addr learned my addr 172.21.15.116:0/1453029663 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:22.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff36d9dc700 1 --1- 172.21.15.116:0/1453029663 >> v1:172.21.15.116:6789/0 conn(0x7ff36814f5c0 0x7ff36814bb70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54654/0 (socket says 172.21.15.116:54654) 2020-11-20T12:17:22.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff36e9de700 1 --2- 172.21.15.116:0/1453029663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff36814c280 0x7ff368067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff36e1dd700 1 -- 172.21.15.116:0/1453029663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff36814c280 msgr2=0x7ff368067d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:22.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff36e1dd700 1 --2- 172.21.15.116:0/1453029663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff36814c280 0x7ff368067d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff36e1dd700 1 -- 172.21.15.116:0/1453029663 >> v1:172.21.15.116:6789/0 conn(0x7ff36814f5c0 legacy=0x7ff36814bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff36e1dd700 1 -- 172.21.15.116:0/1453029663 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36814fa40 con 0x7ff3680c3380 2020-11-20T12:17:22.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff36e9de700 1 --2- 172.21.15.116:0/1453029663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff36814c280 0x7ff368067d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:22.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.792+0000 7ff36e1dd700 1 --2- 172.21.15.116:0/1453029663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3680c3380 0x7ff3680c3760 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7ff364006fd0 tx=0x7ff364006eb0).ready entity=mon.0 client_cookie=57686c4d339c5729 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:22.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.796+0000 7ff36d1db700 1 -- 172.21.15.116:0/1453029663 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364009960 con 0x7ff3680c3380 2020-11-20T12:17:22.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.796+0000 7ff36d1db700 1 -- 172.21.15.116:0/1453029663 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff364009ac0 con 0x7ff3680c3380 2020-11-20T12:17:22.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.796+0000 7ff36d1db700 1 -- 172.21.15.116:0/1453029663 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364009e40 con 0x7ff3680c3380 2020-11-20T12:17:22.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.796+0000 7ff370441700 1 -- 172.21.15.116:0/1453029663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3680c3380 msgr2=0x7ff3680c3760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.796+0000 7ff370441700 1 --2- 172.21.15.116:0/1453029663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3680c3380 0x7ff3680c3760 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7ff364006fd0 tx=0x7ff364006eb0).stop 2020-11-20T12:17:22.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.796+0000 7ff370441700 1 -- 172.21.15.116:0/1453029663 shutdown_connections 2020-11-20T12:17:22.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.796+0000 7ff370441700 1 --2- 172.21.15.116:0/1453029663 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff3680c3380 0x7ff3680c3760 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.796+0000 7ff370441700 1 --2- 172.21.15.116:0/1453029663 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff36814c280 0x7ff368067d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.796+0000 7ff370441700 1 -- 172.21.15.116:0/1453029663 >> 172.21.15.116:0/1453029663 conn(0x7ff368060c80 msgr2=0x7ff3680c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:22.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.796+0000 7ff370441700 1 -- 172.21.15.116:0/1453029663 shutdown_connections 2020-11-20T12:17:22.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.796+0000 7ff370441700 1 -- 172.21.15.116:0/1453029663 wait complete. 2020-11-20T12:17:22.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.800+0000 7ff370441700 1 Processor -- start 2020-11-20T12:17:22.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.800+0000 7ff370441700 1 -- start start 2020-11-20T12:17:22.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff370441700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3680c3380 0x7ff3680a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff370441700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff36814c280 0x7ff36814b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff370441700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff36814f5c0 0x7ff368150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff370441700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680a9530 con 0x7ff36814f5c0 2020-11-20T12:17:22.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff370441700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680a8270 con 0x7ff3680c3380 2020-11-20T12:17:22.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff370441700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3680a9790 con 0x7ff36814c280 2020-11-20T12:17:22.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff36e9de700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff36814f5c0 0x7ff368150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff36e9de700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff36814f5c0 0x7ff368150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45224/0 (socket says 172.21.15.116:45224) 2020-11-20T12:17:22.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff36e9de700 1 -- 172.21.15.116:0/721502365 learned_addr learned my addr 172.21.15.116:0/721502365 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:22.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff36e1dd700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3680c3380 0x7ff3680a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff36e9de700 1 -- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff36814c280 msgr2=0x7ff36814b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff36e9de700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff36814c280 0x7ff36814b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:22.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff36e9de700 1 -- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3680c3380 msgr2=0x7ff3680a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:22.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff36e9de700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3680c3380 0x7ff3680a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:22.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff36e9de700 1 -- 172.21.15.116:0/721502365 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3680a9a30 con 0x7ff36814f5c0 2020-11-20T12:17:22.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff36e1dd700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3680c3380 0x7ff3680a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:22.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff36e9de700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff36814f5c0 0x7ff368150180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7ff36000cba0 tx=0x7ff360008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:22.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff3567fc700 1 -- 172.21.15.116:0/721502365 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36000aa50 con 0x7ff36814f5c0 2020-11-20T12:17:22.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff3567fc700 1 -- 172.21.15.116:0/721502365 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36000abb0 con 0x7ff36814f5c0 2020-11-20T12:17:22.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff370441700 1 -- 172.21.15.116:0/721502365 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3680a9cd0 con 0x7ff36814f5c0 2020-11-20T12:17:22.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff3567fc700 1 -- 172.21.15.116:0/721502365 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff360011410 con 0x7ff36814f5c0 2020-11-20T12:17:22.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.804+0000 7ff370441700 1 -- 172.21.15.116:0/721502365 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3680a7f70 con 0x7ff36814f5c0 2020-11-20T12:17:22.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.808+0000 7ff370441700 1 -- 172.21.15.116:0/721502365 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff36805ca30 con 0x7ff36814f5c0 2020-11-20T12:17:22.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.808+0000 7ff3567fc700 1 -- 172.21.15.116:0/721502365 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff3600117c0 con 0x7ff36814f5c0 2020-11-20T12:17:22.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.808+0000 7ff3567fc700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff3400557c0 0x7ff340057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:22.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.808+0000 7ff3567fc700 1 -- 172.21.15.116:0/721502365 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff36006ec10 con 0x7ff36814f5c0 2020-11-20T12:17:22.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.808+0000 7ff36e1dd700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff3400557c0 0x7ff340057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:22.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.812+0000 7ff36e1dd700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff3400557c0 0x7ff340057c80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7ff36400b1b0 tx=0x7ff364015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:22.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:22.812+0000 7ff3567fc700 1 -- 172.21.15.116:0/721502365 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff360044ab0 con 0x7ff36814f5c0 2020-11-20T12:17:23.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.024+0000 7ff370441700 1 -- 172.21.15.116:0/721502365 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff36814eee0 con 0x7ff36814f5c0 2020-11-20T12:17:23.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.024+0000 7ff3567fc700 1 -- 172.21.15.116:0/721502365 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2851 (secure 0 0 0) 0x7ff3600480d0 con 0x7ff36814f5c0 2020-11-20T12:17:23.026 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:23.026 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3031892761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3031892761},{"type":"v1","addr":"172.21.15.173:6836","nonce":3031892761}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1794965470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1794965470},{"type":"v1","addr":"172.21.15.173:6837","nonce":1794965470}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":9},{"gid":4995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/4143559744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":4143559744},{"type":"v1","addr":"172.21.15.116:6833","nonce":4143559744}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:57.901741+0000","modified":"2020-11-20T12:16:59.918401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4998":{"gid":4998,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6835/3994085646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3994085646},{"type":"v1","addr":"172.21.15.116:6835","nonce":3994085646}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T12:17:23.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 -- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff3400557c0 msgr2=0x7ff340057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:23.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff3400557c0 0x7ff340057c80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7ff36400b1b0 tx=0x7ff364015000).stop 2020-11-20T12:17:23.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 -- 172.21.15.116:0/721502365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff36814f5c0 msgr2=0x7ff368150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff36814f5c0 0x7ff368150180 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7ff36000cba0 tx=0x7ff360008040).stop 2020-11-20T12:17:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 -- 172.21.15.116:0/721502365 shutdown_connections 2020-11-20T12:17:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff3400557c0 0x7ff340057c80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff36814f5c0 0x7ff368150180 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:23.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff3680c3380 0x7ff3680a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 --2- 172.21.15.116:0/721502365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff36814c280 0x7ff36814b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 -- 172.21.15.116:0/721502365 >> 172.21.15.116:0/721502365 conn(0x7ff368060c80 msgr2=0x7ff368150f00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 -- 172.21.15.116:0/721502365 shutdown_connections 2020-11-20T12:17:23.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:23.028+0000 7ff370441700 1 -- 172.21.15.116:0/721502365 wait complete. 2020-11-20T12:17:23.031 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 14 2020-11-20T12:17:23.451 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 14, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:16:57.901741+0000', 'modified': '2020-11-20T12:16:59.918401+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4998}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4998': {'gid': 4998, 'name': 'a', 'rank': 0, 'incarnation': 11, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.116:6835/3994085646', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3994085646}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3994085646}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:17:23.452 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:17:23.453 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:17:23.649 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":4998,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":25.726978007,"mdsmap_epoch":14,"osdmap_epoch":34,"osdmap_epoch_barrier":34,"uptime":34.068041704000002} 2020-11-20T12:17:23.650 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 4998, "mdsmap_epoch": 14, "osdmap_epoch": 34, "osdmap_epoch_barrier": 34, "rank_uptime": 25.726978007, "state": "up:active", "uptime": 34.068041704, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:17:23.651 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:17:23.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:23.652 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-20T12:17:23.668 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:17:23.669 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:23.740 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:23.741 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:17:23.767 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:23.767 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:17:23.773 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:17:23.774 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:23.774 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-20T12:17:23.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:17:23.804 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest/mnt.1 && exec sudo touch parent1/mydir/afile) 2020-11-20T12:17:23.853 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-20T12:17:23.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:17:23.854 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 2020-11-20T12:17:23.943 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:17:23.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:23.944 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T12:17:23.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:17:23.961 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo ls parent1/mydir) 2020-11-20T12:17:23.996 INFO:teuthology.orchestra.run.smithi116.stdout:afile 2020-11-20T12:17:24.000 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:17:24.001 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:17:24.080 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:17:24.119 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:17:24.120 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-20T12:17:24.120 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:17:24.121 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-20T12:17:24.199 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:17:24.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:24.200 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:17:24.219 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-20T12:17:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.460+0000 7fd3d68a8700 1 Processor -- start 2020-11-20T12:17:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.460+0000 7fd3d68a8700 1 -- start start 2020-11-20T12:17:24.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d68a8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d013ad30 0x7fd3d00a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d68a8700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd3d0061250 0x7fd3d0061700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d68a8700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d0152a80 con 0x7fd3d013b110 2020-11-20T12:17:24.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d68a8700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d0152540 con 0x7fd3d0061250 2020-11-20T12:17:24.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d68a8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d01527e0 con 0x7fd3d013ad30 2020-11-20T12:17:24.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d50a5700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd3d013b110 0x7fd3d0151ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54666/0 (socket says 172.21.15.116:54666) 2020-11-20T12:17:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d50a5700 1 -- 172.21.15.116:0/3135213200 learned_addr learned my addr 172.21.15.116:0/3135213200 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:24.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d60a7700 1 --2- 172.21.15.116:0/3135213200 >> v2:172.21.15.173:3300/0 conn(0x7fd3d0061250 0x7fd3d0061700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038827586 0 0) 0x7fd3d0152a80 con 0x7fd3d013b110 2020-11-20T12:17:24.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d58a6700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d013ad30 0x7fd3d00a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3b8003240 con 0x7fd3d013b110 2020-11-20T12:17:24.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2432274966 0 0) 0x7fd3b8003240 con 0x7fd3d013b110 2020-11-20T12:17:24.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d013ad30 msgr2=0x7fd3d00a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:24.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d013ad30 0x7fd3d00a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 >> v2:172.21.15.173:3300/0 conn(0x7fd3d0061250 msgr2=0x7fd3d0061700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:24.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 --2- 172.21.15.116:0/3135213200 >> v2:172.21.15.173:3300/0 conn(0x7fd3d0061250 0x7fd3d0061700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d00ac910 con 0x7fd3d013b110 2020-11-20T12:17:24.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fd3c00014e0 con 0x7fd3d013b110 2020-11-20T12:17:24.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 >> v1:172.21.15.116:6789/0 conn(0x7fd3d013b110 legacy=0x7fd3d0151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:24.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d013ad30 0x7fd3b8003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b8003c40 0x7fd3b80044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b80049d0 0x7fd3b8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d0152540 con 0x7fd3b80049d0 2020-11-20T12:17:24.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d01527e0 con 0x7fd3b8003c40 2020-11-20T12:17:24.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b8004200 con 0x7fd3d013ad30 2020-11-20T12:17:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d60a7700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b80049d0 0x7fd3b8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d58a6700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d013ad30 0x7fd3b8003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.464+0000 7fd3d50a5700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b8003c40 0x7fd3b80044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d60a7700 1 -- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d013ad30 msgr2=0x7fd3b8003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:24.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d60a7700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d013ad30 0x7fd3b8003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d60a7700 1 -- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b8003c40 msgr2=0x7fd3b80044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:24.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d60a7700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b8003c40 0x7fd3b80044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d60a7700 1 -- 172.21.15.116:0/3135213200 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b8003240 con 0x7fd3b80049d0 2020-11-20T12:17:24.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d50a5700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b8003c40 0x7fd3b80044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:24.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d60a7700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b80049d0 0x7fd3b8004e50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fd3c8009ea0 tx=0x7fd3c80091d0).ready entity=mon.0 client_cookie=dce55ed8b9d9e8cd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:24.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80095c0 con 0x7fd3b80049d0 2020-11-20T12:17:24.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c800ec90 con 0x7fd3b80049d0 2020-11-20T12:17:24.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d48a4700 1 -- 172.21.15.116:0/3135213200 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80123f0 con 0x7fd3b80049d0 2020-11-20T12:17:24.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d68a8700 1 -- 172.21.15.116:0/3135213200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b80049d0 msgr2=0x7fd3b8004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:24.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d68a8700 1 --2- 172.21.15.116:0/3135213200 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b80049d0 0x7fd3b8004e50 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fd3c8009ea0 tx=0x7fd3c80091d0).stop 2020-11-20T12:17:24.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d60a7700 1 -- 172.21.15.116:0/3135213200 reap_dead start 2020-11-20T12:17:24.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d68a8700 1 -- 172.21.15.116:0/3135213200 shutdown_connections 2020-11-20T12:17:24.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d68a8700 1 -- 172.21.15.116:0/3135213200 >> 172.21.15.116:0/3135213200 conn(0x7fd3d00678a0 msgr2=0x7fd3d0067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:24.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d68a8700 1 -- 172.21.15.116:0/3135213200 shutdown_connections 2020-11-20T12:17:24.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.468+0000 7fd3d68a8700 1 -- 172.21.15.116:0/3135213200 wait complete. 2020-11-20T12:17:24.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d68a8700 1 Processor -- start 2020-11-20T12:17:24.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d68a8700 1 -- start start 2020-11-20T12:17:24.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d68a8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d00a4420 0x7fd3d00a4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d68a8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d00a67c0 0x7fd3d00a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d68a8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3d00a6c40 0x7fd3d00a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d68a8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d009f190 con 0x7fd3d00a6c40 2020-11-20T12:17:24.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d68a8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d009f3f0 con 0x7fd3d00a4420 2020-11-20T12:17:24.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d68a8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d009f690 con 0x7fd3d00a67c0 2020-11-20T12:17:24.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d60a7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3d00a6c40 0x7fd3d00a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d58a6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d00a4420 0x7fd3d00a4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d60a7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3d00a6c40 0x7fd3d00a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45248/0 (socket says 172.21.15.116:45248) 2020-11-20T12:17:24.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d60a7700 1 -- 172.21.15.116:0/2006811464 learned_addr learned my addr 172.21.15.116:0/2006811464 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:24.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d50a5700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d00a67c0 0x7fd3d00a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d60a7700 1 -- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d00a67c0 msgr2=0x7fd3d00a0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:24.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d60a7700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d00a67c0 0x7fd3d00a0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d60a7700 1 -- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d00a4420 msgr2=0x7fd3d00a4800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:24.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d60a7700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d00a4420 0x7fd3d00a4800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d60a7700 1 -- 172.21.15.116:0/2006811464 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d009f930 con 0x7fd3d00a6c40 2020-11-20T12:17:24.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.472+0000 7fd3d60a7700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3d00a6c40 0x7fd3d00a0560 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fd3c8014000 tx=0x7fd3c801a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:24.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.476+0000 7fd3bdffb700 1 -- 172.21.15.116:0/2006811464 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80123f0 con 0x7fd3d00a6c40 2020-11-20T12:17:24.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.476+0000 7fd3d68a8700 1 -- 172.21.15.116:0/2006811464 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d009fbd0 con 0x7fd3d00a6c40 2020-11-20T12:17:24.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.476+0000 7fd3bdffb700 1 -- 172.21.15.116:0/2006811464 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3c8012e90 con 0x7fd3d00a6c40 2020-11-20T12:17:24.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.476+0000 7fd3bdffb700 1 -- 172.21.15.116:0/2006811464 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3c80095c0 con 0x7fd3d00a6c40 2020-11-20T12:17:24.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.476+0000 7fd3d68a8700 1 -- 172.21.15.116:0/2006811464 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d00a1a90 con 0x7fd3d00a6c40 2020-11-20T12:17:24.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.476+0000 7fd3d68a8700 1 -- 172.21.15.116:0/2006811464 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d0044f60 con 0x7fd3d00a6c40 2020-11-20T12:17:24.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.476+0000 7fd3bdffb700 1 -- 172.21.15.116:0/2006811464 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd3c802e3a0 con 0x7fd3d00a6c40 2020-11-20T12:17:24.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.476+0000 7fd3bdffb700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3a80557c0 0x7fd3a8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.476+0000 7fd3bdffb700 1 -- 172.21.15.116:0/2006811464 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd3c8070bc0 con 0x7fd3d00a6c40 2020-11-20T12:17:24.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.480+0000 7fd3d58a6700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3a80557c0 0x7fd3a8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.480+0000 7fd3d58a6700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3a80557c0 0x7fd3a8057c80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fd3cc006a90 tx=0x7fd3cc006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:24.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.484+0000 7fd3bdffb700 1 -- 172.21.15.116:0/2006811464 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd3c8046a80 con 0x7fd3d00a6c40 2020-11-20T12:17:24.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.644+0000 7fd3d68a8700 1 -- 172.21.15.116:0/2006811464 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd3d00a4060 con 0x7fd3d00a6c40 2020-11-20T12:17:24.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.652+0000 7fd3bdffb700 1 -- 172.21.15.116:0/2006811464 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd3c802a020 con 0x7fd3d00a6c40 2020-11-20T12:17:24.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 -- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3a80557c0 msgr2=0x7fd3a8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:24.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3a80557c0 0x7fd3a8057c80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fd3cc006a90 tx=0x7fd3cc006df0).stop 2020-11-20T12:17:24.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 -- 172.21.15.116:0/2006811464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3d00a6c40 msgr2=0x7fd3d00a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:24.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3d00a6c40 0x7fd3d00a0560 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fd3c8014000 tx=0x7fd3c801a000).stop 2020-11-20T12:17:24.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 -- 172.21.15.116:0/2006811464 shutdown_connections 2020-11-20T12:17:24.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3a80557c0 0x7fd3a8057c80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3d00a6c40 0x7fd3d00a0560 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3d00a4420 0x7fd3d00a4800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 --2- 172.21.15.116:0/2006811464 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3d00a67c0 0x7fd3d00a0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 -- 172.21.15.116:0/2006811464 >> 172.21.15.116:0/2006811464 conn(0x7fd3d00678a0 msgr2=0x7fd3d00ab950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:24.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 -- 172.21.15.116:0/2006811464 shutdown_connections 2020-11-20T12:17:24.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.656+0000 7fd3d68a8700 1 -- 172.21.15.116:0/2006811464 wait complete. 2020-11-20T12:17:24.680 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_15303' 2020-11-20T12:17:24.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.868+0000 7fe598bbd700 1 Processor -- start 2020-11-20T12:17:24.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.868+0000 7fe598bbd700 1 -- start start 2020-11-20T12:17:24.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.868+0000 7fe598bbd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe59414e3f0 0x7fe594158df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.868+0000 7fe598bbd700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5940a5230 con 0x7fe5940c4150 2020-11-20T12:17:24.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.868+0000 7fe598bbd700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5940a7220 con 0x7fe5940c4530 2020-11-20T12:17:24.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.868+0000 7fe598bbd700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5940a74c0 con 0x7fe59414e3f0 2020-11-20T12:17:24.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5937fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe5940c4150 0x7fe5940a47d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54686/0 (socket says 172.21.15.116:54686) 2020-11-20T12:17:24.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5937fe700 1 -- 172.21.15.116:0/509331905 learned_addr learned my addr 172.21.15.116:0/509331905 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:24.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe593fff700 1 --2- 172.21.15.116:0/509331905 >> v2:172.21.15.116:3300/0 conn(0x7fe59414e3f0 0x7fe594158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202162091 0 0) 0x7fe5940a5230 con 0x7fe5940c4150 2020-11-20T12:17:24.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe580003240 con 0x7fe5940c4150 2020-11-20T12:17:24.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712425834 0 0) 0x7fe5940a7220 con 0x7fe5940c4530 2020-11-20T12:17:24.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5940a5230 con 0x7fe5940c4530 2020-11-20T12:17:24.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1015991604 0 0) 0x7fe580003240 con 0x7fe5940c4150 2020-11-20T12:17:24.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 >> v2:172.21.15.116:3300/0 conn(0x7fe59414e3f0 msgr2=0x7fe594158df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:24.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 --2- 172.21.15.116:0/509331905 >> v2:172.21.15.116:3300/0 conn(0x7fe59414e3f0 0x7fe594158df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 >> v1:172.21.15.173:6789/0 conn(0x7fe5940c4530 legacy=0x7fe59414dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:24.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5940a7760 con 0x7fe5940c4150 2020-11-20T12:17:24.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fe5840014e0 con 0x7fe5940c4150 2020-11-20T12:17:24.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 >> v1:172.21.15.116:6789/0 conn(0x7fe5940c4150 legacy=0x7fe5940a47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:24.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 --2- 172.21.15.116:0/509331905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe580003910 0x7fe580003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 --2- 172.21.15.116:0/509331905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5800044e0 0x7fe580004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 --2- 172.21.15.116:0/509331905 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe580004e50 0x7fe58000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5940a74c0 con 0x7fe580003910 2020-11-20T12:17:24.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58000bc20 con 0x7fe5800044e0 2020-11-20T12:17:24.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe58000bd60 con 0x7fe580004e50 2020-11-20T12:17:24.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5937fe700 1 --2- 172.21.15.116:0/509331905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe580003910 0x7fe580003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe593fff700 1 --2- 172.21.15.116:0/509331905 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe580004e50 0x7fe58000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe592ffd700 1 --2- 172.21.15.116:0/509331905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5800044e0 0x7fe580004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5937fe700 1 -- 172.21.15.116:0/509331905 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe580004e50 msgr2=0x7fe58000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:24.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5937fe700 1 --2- 172.21.15.116:0/509331905 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe580004e50 0x7fe58000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5937fe700 1 -- 172.21.15.116:0/509331905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5800044e0 msgr2=0x7fe580004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:24.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5937fe700 1 --2- 172.21.15.116:0/509331905 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5800044e0 0x7fe580004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5937fe700 1 -- 172.21.15.116:0/509331905 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe580003240 con 0x7fe580003910 2020-11-20T12:17:24.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe593fff700 1 -- 172.21.15.116:0/509331905 reap_dead start 2020-11-20T12:17:24.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.872+0000 7fe5937fe700 1 --2- 172.21.15.116:0/509331905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe580003910 0x7fe580003d10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fe58400c160 tx=0x7fe5940a53c0).ready entity=mon.0 client_cookie=4d63946953bc38fa server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:24.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.876+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58400ce60 con 0x7fe580003910 2020-11-20T12:17:24.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.876+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe584007750 con 0x7fe580003910 2020-11-20T12:17:24.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.876+0000 7fe5927fc700 1 -- 172.21.15.116:0/509331905 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584007320 con 0x7fe580003910 2020-11-20T12:17:24.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.876+0000 7fe598bbd700 1 -- 172.21.15.116:0/509331905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe580003910 msgr2=0x7fe580003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:24.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.876+0000 7fe598bbd700 1 --2- 172.21.15.116:0/509331905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe580003910 0x7fe580003d10 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fe58400c160 tx=0x7fe5940a53c0).stop 2020-11-20T12:17:24.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.876+0000 7fe598bbd700 1 -- 172.21.15.116:0/509331905 shutdown_connections 2020-11-20T12:17:24.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.876+0000 7fe598bbd700 1 --2- 172.21.15.116:0/509331905 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe580003910 0x7fe580003d10 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.876+0000 7fe598bbd700 1 -- 172.21.15.116:0/509331905 >> 172.21.15.116:0/509331905 conn(0x7fe594062520 msgr2=0x7fe594062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:24.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.876+0000 7fe598bbd700 1 -- 172.21.15.116:0/509331905 shutdown_connections 2020-11-20T12:17:24.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.876+0000 7fe598bbd700 1 -- 172.21.15.116:0/509331905 wait complete. 2020-11-20T12:17:24.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.876+0000 7fe598bbd700 1 Processor -- start 2020-11-20T12:17:24.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe598bbd700 1 -- start start 2020-11-20T12:17:24.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe598bbd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe594143af0 0x7fe5940c1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe598bbd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5940c1c80 0x7fe594165300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe598bbd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5941657f0 0x7fe594168910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe598bbd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe594165c70 con 0x7fe594143af0 2020-11-20T12:17:24.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe598bbd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe594165ec0 con 0x7fe5941657f0 2020-11-20T12:17:24.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe598bbd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe594166160 con 0x7fe5940c1c80 2020-11-20T12:17:24.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5937fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe594143af0 0x7fe5940c1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5937fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe594143af0 0x7fe5940c1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45264/0 (socket says 172.21.15.116:45264) 2020-11-20T12:17:24.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5937fe700 1 -- 172.21.15.116:0/1129730913 learned_addr learned my addr 172.21.15.116:0/1129730913 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:24.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe593fff700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5941657f0 0x7fe594168910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe592ffd700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5940c1c80 0x7fe594165300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5937fe700 1 -- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5940c1c80 msgr2=0x7fe594165300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:24.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5937fe700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5940c1c80 0x7fe594165300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5937fe700 1 -- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5941657f0 msgr2=0x7fe594168910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:24.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5937fe700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5941657f0 0x7fe594168910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:24.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5937fe700 1 -- 172.21.15.116:0/1129730913 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe594166470 con 0x7fe594143af0 2020-11-20T12:17:24.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5937fe700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe594143af0 0x7fe5940c1790 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fe584007fd0 tx=0x7fe584015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:24.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5737fe700 1 -- 172.21.15.116:0/1129730913 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584006c70 con 0x7fe594143af0 2020-11-20T12:17:24.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5737fe700 1 -- 172.21.15.116:0/1129730913 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe584006dd0 con 0x7fe594143af0 2020-11-20T12:17:24.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe598bbd700 1 -- 172.21.15.116:0/1129730913 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe594166710 con 0x7fe594143af0 2020-11-20T12:17:24.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe5737fe700 1 -- 172.21.15.116:0/1129730913 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe584005260 con 0x7fe594143af0 2020-11-20T12:17:24.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.880+0000 7fe598bbd700 1 -- 172.21.15.116:0/1129730913 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5940c2a20 con 0x7fe594143af0 2020-11-20T12:17:24.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.884+0000 7fe598bbd700 1 -- 172.21.15.116:0/1129730913 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5940c2060 con 0x7fe594143af0 2020-11-20T12:17:24.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.884+0000 7fe5737fe700 1 -- 172.21.15.116:0/1129730913 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe584010070 con 0x7fe594143af0 2020-11-20T12:17:24.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.884+0000 7fe5737fe700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe56c0557c0 0x7fe56c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:24.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.884+0000 7fe5737fe700 1 -- 172.21.15.116:0/1129730913 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe58406f880 con 0x7fe594143af0 2020-11-20T12:17:24.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.888+0000 7fe592ffd700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe56c0557c0 0x7fe56c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:24.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.888+0000 7fe592ffd700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe56c0557c0 0x7fe56c057c80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fe588007fd0 tx=0x7fe588009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:24.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:24.892+0000 7fe5737fe700 1 -- 172.21.15.116:0/1129730913 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe584045610 con 0x7fe594143af0 2020-11-20T12:17:25.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.104+0000 7fe598bbd700 1 -- 172.21.15.116:0/1129730913 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_15303"]} v 0) v1 -- 0x7fe594044f60 con 0x7fe594143af0 2020-11-20T12:17:25.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.284+0000 7fe5737fe700 1 -- 172.21.15.116:0/1129730913 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_15303"]}]=0 v95) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe584048c30 con 0x7fe594143af0 2020-11-20T12:17:25.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 -- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe56c0557c0 msgr2=0x7fe56c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:25.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe56c0557c0 0x7fe56c057c80 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7fe588007fd0 tx=0x7fe588009040).stop 2020-11-20T12:17:25.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 -- 172.21.15.116:0/1129730913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe594143af0 msgr2=0x7fe5940c1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:25.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe594143af0 0x7fe5940c1790 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7fe584007fd0 tx=0x7fe584015000).stop 2020-11-20T12:17:25.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 -- 172.21.15.116:0/1129730913 shutdown_connections 2020-11-20T12:17:25.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe56c0557c0 0x7fe56c057c80 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:25.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5941657f0 0x7fe594168910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:25.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe594143af0 0x7fe5940c1790 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:25.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 --2- 172.21.15.116:0/1129730913 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5940c1c80 0x7fe594165300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:25.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 -- 172.21.15.116:0/1129730913 >> 172.21.15.116:0/1129730913 conn(0x7fe594062520 msgr2=0x7fe5940a18a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:25.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 -- 172.21.15.116:0/1129730913 shutdown_connections 2020-11-20T12:17:25.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.288+0000 7fe598bbd700 1 -- 172.21.15.116:0/1129730913 wait complete. 2020-11-20T12:17:25.313 INFO:tasks.cephfs_test_runner:test_15303 (tasks.cephfs.test_volume_client.TestVolumeClient) ... ok 2020-11-20T12:17:25.313 INFO:tasks.cephfs_test_runner:Starting test: test_21501 (tasks.cephfs.test_volume_client.TestVolumeClient) 2020-11-20T12:17:25.314 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_21501' 2020-11-20T12:17:25.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.492+0000 7fa8d4d0f700 1 Processor -- start 2020-11-20T12:17:25.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.492+0000 7fa8d4d0f700 1 -- start start 2020-11-20T12:17:25.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8d4d0f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8d013aee0 0x7fa8d0152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:25.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8d4d0f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d0152a30 con 0x7fa8d00aa790 2020-11-20T12:17:25.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8d4d0f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d00aac10 con 0x7fa8d00612f0 2020-11-20T12:17:25.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8d4d0f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d00adce0 con 0x7fa8d013aee0 2020-11-20T12:17:25.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8cf7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8d013aee0 0x7fa8d0152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:25.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8ceffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa8d00aa790 0x7fa8d00ad5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54708/0 (socket says 172.21.15.116:54708) 2020-11-20T12:17:25.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8ceffd700 1 -- 172.21.15.116:0/1773977047 learned_addr learned my addr 172.21.15.116:0/1773977047 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:25.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8ce7fc700 1 -- 172.21.15.116:0/1773977047 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122241071 0 0) 0x7fa8d0152a30 con 0x7fa8d00aa790 2020-11-20T12:17:25.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8ce7fc700 1 -- 172.21.15.116:0/1773977047 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8bc003240 con 0x7fa8d00aa790 2020-11-20T12:17:25.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8cf7fe700 1 -- 172.21.15.116:0/1773977047 >> v1:172.21.15.173:6789/0 conn(0x7fa8d00612f0 legacy=0x7fa8d00a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:25.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8cf7fe700 1 -- 172.21.15.116:0/1773977047 >> v1:172.21.15.116:6789/0 conn(0x7fa8d00aa790 legacy=0x7fa8d00ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:25.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8cf7fe700 1 -- 172.21.15.116:0/1773977047 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d00aaeb0 con 0x7fa8d013aee0 2020-11-20T12:17:25.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8cf7fe700 1 --2- 172.21.15.116:0/1773977047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d013aee0 0x7fa8d0152540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fa8c000b9a0 tx=0x7fa8c0006df0).ready entity=mon.0 client_cookie=8ad46d70556a8df8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:25.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8ce7fc700 1 -- 172.21.15.116:0/1773977047 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00097b0 con 0x7fa8d013aee0 2020-11-20T12:17:25.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8ce7fc700 1 -- 172.21.15.116:0/1773977047 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c0009910 con 0x7fa8d013aee0 2020-11-20T12:17:25.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8ce7fc700 1 -- 172.21.15.116:0/1773977047 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0009c90 con 0x7fa8d013aee0 2020-11-20T12:17:25.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/1773977047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d013aee0 msgr2=0x7fa8d0152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:25.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8d4d0f700 1 --2- 172.21.15.116:0/1773977047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d013aee0 0x7fa8d0152540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7fa8c000b9a0 tx=0x7fa8c0006df0).stop 2020-11-20T12:17:25.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/1773977047 shutdown_connections 2020-11-20T12:17:25.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8d4d0f700 1 --2- 172.21.15.116:0/1773977047 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d013aee0 0x7fa8d0152540 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:25.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/1773977047 >> 172.21.15.116:0/1773977047 conn(0x7fa8d00678a0 msgr2=0x7fa8d0067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:25.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/1773977047 shutdown_connections 2020-11-20T12:17:25.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.496+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/1773977047 wait complete. 2020-11-20T12:17:25.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.500+0000 7fa8d4d0f700 1 Processor -- start 2020-11-20T12:17:25.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.500+0000 7fa8d4d0f700 1 -- start start 2020-11-20T12:17:25.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8d4d0f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d00612f0 0x7fa8d0150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:25.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8d4d0f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d00aa790 0x7fa8d00a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:25.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8d4d0f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d013aee0 0x7fa8d00a2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:25.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8d4d0f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00a2100 con 0x7fa8d013aee0 2020-11-20T12:17:25.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8d4d0f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00a1680 con 0x7fa8d00612f0 2020-11-20T12:17:25.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8d4d0f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00a1920 con 0x7fa8d00aa790 2020-11-20T12:17:25.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8cffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d013aee0 0x7fa8d00a2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:25.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8cffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d013aee0 0x7fa8d00a2a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45282/0 (socket says 172.21.15.116:45282) 2020-11-20T12:17:25.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8cffff700 1 -- 172.21.15.116:0/3931932544 learned_addr learned my addr 172.21.15.116:0/3931932544 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:25.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8cf7fe700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d00612f0 0x7fa8d0150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:25.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8ceffd700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d00aa790 0x7fa8d00a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:25.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8cffff700 1 -- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d00aa790 msgr2=0x7fa8d00a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:25.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8cffff700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d00aa790 0x7fa8d00a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:25.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8cffff700 1 -- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d00612f0 msgr2=0x7fa8d0150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:25.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8cffff700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d00612f0 0x7fa8d0150e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:25.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8cffff700 1 -- 172.21.15.116:0/3931932544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d00a1bc0 con 0x7fa8d013aee0 2020-11-20T12:17:25.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8ceffd700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d00aa790 0x7fa8d00a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:25.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8cffff700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d013aee0 0x7fa8d00a2a50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fa8c800ca90 tx=0x7fa8c800cda0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:25.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8af7fe700 1 -- 172.21.15.116:0/3931932544 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800a7b0 con 0x7fa8d013aee0 2020-11-20T12:17:25.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8af7fe700 1 -- 172.21.15.116:0/3931932544 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c8003030 con 0x7fa8d013aee0 2020-11-20T12:17:25.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/3931932544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d00a1e60 con 0x7fa8d013aee0 2020-11-20T12:17:25.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8af7fe700 1 -- 172.21.15.116:0/3931932544 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800ab20 con 0x7fa8d013aee0 2020-11-20T12:17:25.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.504+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/3931932544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d00a0510 con 0x7fa8d013aee0 2020-11-20T12:17:25.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.508+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/3931932544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d0044f60 con 0x7fa8d013aee0 2020-11-20T12:17:25.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.508+0000 7fa8af7fe700 1 -- 172.21.15.116:0/3931932544 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa8c8019030 con 0x7fa8d013aee0 2020-11-20T12:17:25.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.508+0000 7fa8af7fe700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8a80557c0 0x7fa8a8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:25.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.508+0000 7fa8af7fe700 1 -- 172.21.15.116:0/3931932544 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa8c806de40 con 0x7fa8d013aee0 2020-11-20T12:17:25.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.512+0000 7fa8cf7fe700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8a80557c0 0x7fa8a8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:25.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.512+0000 7fa8cf7fe700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8a80557c0 0x7fa8a8057c80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fa8c0006e30 tx=0x7fa8c0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:25.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.516+0000 7fa8af7fe700 1 -- 172.21.15.116:0/3931932544 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa8c8043d10 con 0x7fa8d013aee0 2020-11-20T12:17:25.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:25.728+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/3931932544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_21501"]} v 0) v1 -- 0x7fa8d0151550 con 0x7fa8d013aee0 2020-11-20T12:17:26.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.292+0000 7fa8af7fe700 1 -- 172.21.15.116:0/3931932544 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_21501"]}]=0 v96) v1 ==== 141+0+0 (secure 0 0 0) 0x7fa8c8047330 con 0x7fa8d013aee0 2020-11-20T12:17:26.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8a80557c0 msgr2=0x7fa8a8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8a80557c0 0x7fa8a8057c80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fa8c0006e30 tx=0x7fa8c0015000).stop 2020-11-20T12:17:26.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/3931932544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d013aee0 msgr2=0x7fa8d00a2a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d013aee0 0x7fa8d00a2a50 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7fa8c800ca90 tx=0x7fa8c800cda0).stop 2020-11-20T12:17:26.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/3931932544 shutdown_connections 2020-11-20T12:17:26.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8a80557c0 0x7fa8a8057c80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d013aee0 0x7fa8d00a2a50 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d00612f0 0x7fa8d0150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 --2- 172.21.15.116:0/3931932544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d00aa790 0x7fa8d00a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/3931932544 >> 172.21.15.116:0/3931932544 conn(0x7fa8d00678a0 msgr2=0x7fa8d00c77e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:26.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/3931932544 shutdown_connections 2020-11-20T12:17:26.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.296+0000 7fa8d4d0f700 1 -- 172.21.15.116:0/3931932544 wait complete. 2020-11-20T12:17:26.324 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-20T12:17:26.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.504+0000 7f9f42a0b700 1 Processor -- start 2020-11-20T12:17:26.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.508+0000 7f9f42a0b700 1 -- start start 2020-11-20T12:17:26.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.508+0000 7f9f42a0b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9f3c14bbf0 0x7f9f3c0c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.508+0000 7f9f42a0b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3c0bf8e0 con 0x7f9f3c14bfd0 2020-11-20T12:17:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.508+0000 7f9f42a0b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f3c1553a0 con 0x7f9f3c0bab80 2020-11-20T12:17:26.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.508+0000 7f9f42a0b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f3c1562c0 con 0x7f9f3c14bbf0 2020-11-20T12:17:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.508+0000 7f9f337fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9f3c14bfd0 0x7f9f3c0ba470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54720/0 (socket says 172.21.15.116:54720) 2020-11-20T12:17:26.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.508+0000 7f9f337fe700 1 -- 172.21.15.116:0/3160092008 learned_addr learned my addr 172.21.15.116:0/3160092008 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:26.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3bfff700 1 --2- 172.21.15.116:0/3160092008 >> v2:172.21.15.173:3300/0 conn(0x7f9f3c14bbf0 0x7f9f3c0c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3960722603 0 0) 0x7f9f3c0bf8e0 con 0x7f9f3c14bfd0 2020-11-20T12:17:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f1c003240 con 0x7f9f3c14bfd0 2020-11-20T12:17:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079080495 0 0) 0x7f9f3c1553a0 con 0x7f9f3c0bab80 2020-11-20T12:17:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f3c0bf8e0 con 0x7f9f3c0bab80 2020-11-20T12:17:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 443669916 0 0) 0x7f9f1c003240 con 0x7f9f3c14bfd0 2020-11-20T12:17:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 >> v2:172.21.15.173:3300/0 conn(0x7f9f3c14bbf0 msgr2=0x7f9f3c0c0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 --2- 172.21.15.116:0/3160092008 >> v2:172.21.15.173:3300/0 conn(0x7f9f3c14bbf0 0x7f9f3c0c0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 >> v1:172.21.15.173:6789/0 conn(0x7f9f3c0bab80 legacy=0x7f9f3c151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c155640 con 0x7f9f3c14bfd0 2020-11-20T12:17:26.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f9f240014e0 con 0x7f9f3c14bfd0 2020-11-20T12:17:26.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 >> v1:172.21.15.116:6789/0 conn(0x7f9f3c14bfd0 legacy=0x7f9f3c0ba470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f1c003910 0x7f9f1c003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f1c0044e0 0x7f9f1c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f1c004e50 0x7f9f1c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c1562c0 con 0x7f9f1c003910 2020-11-20T12:17:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c00bc20 con 0x7f9f1c0044e0 2020-11-20T12:17:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1c00bd60 con 0x7f9f1c004e50 2020-11-20T12:17:26.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3bfff700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f1c003910 0x7f9f1c003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f337fe700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f1c0044e0 0x7f9f1c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f40fa8700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f1c004e50 0x7f9f1c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3bfff700 1 -- 172.21.15.116:0/3160092008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f1c004e50 msgr2=0x7f9f1c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:26.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3bfff700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f1c004e50 0x7f9f1c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3bfff700 1 -- 172.21.15.116:0/3160092008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f1c0044e0 msgr2=0x7f9f1c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:26.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3bfff700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f1c0044e0 0x7f9f1c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f3bfff700 1 -- 172.21.15.116:0/3160092008 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f1c003240 con 0x7f9f1c003910 2020-11-20T12:17:26.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.512+0000 7f9f40fa8700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f1c004e50 0x7f9f1c00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:26.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f40fa8700 1 -- 172.21.15.116:0/3160092008 reap_dead start 2020-11-20T12:17:26.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f3bfff700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f1c003910 0x7f9f1c003d10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f9f2c00bc00 tx=0x7f9f2c006df0).ready entity=mon.0 client_cookie=6774708b7d419d30 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:26.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00b310 con 0x7f9f1c003910 2020-11-20T12:17:26.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c00ea90 con 0x7f9f1c003910 2020-11-20T12:17:26.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f3b7fe700 1 -- 172.21.15.116:0/3160092008 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00ee10 con 0x7f9f1c003910 2020-11-20T12:17:26.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3160092008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f1c003910 msgr2=0x7f9f1c003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f42a0b700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f1c003910 0x7f9f1c003d10 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f9f2c00bc00 tx=0x7f9f2c006df0).stop 2020-11-20T12:17:26.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3160092008 shutdown_connections 2020-11-20T12:17:26.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f42a0b700 1 --2- 172.21.15.116:0/3160092008 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f1c003910 0x7f9f1c003d10 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3160092008 >> 172.21.15.116:0/3160092008 conn(0x7f9f3c0c52e0 msgr2=0x7f9f3c0c56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:26.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3160092008 shutdown_connections 2020-11-20T12:17:26.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.516+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3160092008 wait complete. 2020-11-20T12:17:26.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f42a0b700 1 Processor -- start 2020-11-20T12:17:26.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f42a0b700 1 -- start start 2020-11-20T12:17:26.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f42a0b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f3c0a7660 0x7f9f3c0a7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f42a0b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3c0bd510 0x7f9f3c0bd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f42a0b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3c1473e0 0x7f9f3c147840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f42a0b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c0bde80 con 0x7f9f3c0a7660 2020-11-20T12:17:26.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f42a0b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c0bcf20 con 0x7f9f3c1473e0 2020-11-20T12:17:26.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f42a0b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3c147d30 con 0x7f9f3c0bd510 2020-11-20T12:17:26.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f3bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f3c0a7660 0x7f9f3c0a7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f3bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f3c0a7660 0x7f9f3c0a7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45298/0 (socket says 172.21.15.116:45298) 2020-11-20T12:17:26.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f3bfff700 1 -- 172.21.15.116:0/3161381222 learned_addr learned my addr 172.21.15.116:0/3161381222 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:26.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f40fa8700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3c1473e0 0x7f9f3c147840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.520+0000 7f9f337fe700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3c0bd510 0x7f9f3c0bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f3bfff700 1 -- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3c0bd510 msgr2=0x7f9f3c0bd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f3bfff700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3c0bd510 0x7f9f3c0bd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f3bfff700 1 -- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3c1473e0 msgr2=0x7f9f3c147840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f3bfff700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3c1473e0 0x7f9f3c147840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f3bfff700 1 -- 172.21.15.116:0/3161381222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c0be090 con 0x7f9f3c0a7660 2020-11-20T12:17:26.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f337fe700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3c0bd510 0x7f9f3c0bd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:26.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f3bfff700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f3c0a7660 0x7f9f3c0a7a40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f9f2c006e30 tx=0x7f9f2c019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:26.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f38ff9700 1 -- 172.21.15.116:0/3161381222 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c00eed0 con 0x7f9f3c0a7660 2020-11-20T12:17:26.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f38ff9700 1 -- 172.21.15.116:0/3161381222 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2c021070 con 0x7f9f3c0a7660 2020-11-20T12:17:26.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3161381222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f3c0be330 con 0x7f9f3c0a7660 2020-11-20T12:17:26.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f38ff9700 1 -- 172.21.15.116:0/3161381222 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c01d620 con 0x7f9f3c0a7660 2020-11-20T12:17:26.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3161381222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f3c0be870 con 0x7f9f3c0a7660 2020-11-20T12:17:26.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3161381222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3c155820 con 0x7f9f3c0a7660 2020-11-20T12:17:26.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f38ff9700 1 -- 172.21.15.116:0/3161381222 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9f2c01da50 con 0x7f9f3c0a7660 2020-11-20T12:17:26.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.524+0000 7f9f38ff9700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f180557c0 0x7f9f18057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.528+0000 7f9f38ff9700 1 -- 172.21.15.116:0/3161381222 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9f2c070570 con 0x7f9f3c0a7660 2020-11-20T12:17:26.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.528+0000 7f9f337fe700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f180557c0 0x7f9f18057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.528+0000 7f9f337fe700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f180557c0 0x7f9f18057c80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f9f3c0bcb90 tx=0x7f9f24009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:26.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.532+0000 7f9f38ff9700 1 -- 172.21.15.116:0/3161381222 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9f2c046490 con 0x7f9f3c0a7660 2020-11-20T12:17:26.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.696+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3161381222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f9f3c05ca30 con 0x7f9f3c0a7660 2020-11-20T12:17:26.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.704+0000 7f9f38ff9700 1 -- 172.21.15.116:0/3161381222 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9f2c049ab0 con 0x7f9f3c0a7660 2020-11-20T12:17:26.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f180557c0 msgr2=0x7f9f18057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f180557c0 0x7f9f18057c80 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f9f3c0bcb90 tx=0x7f9f24009040).stop 2020-11-20T12:17:26.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3161381222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f3c0a7660 msgr2=0x7f9f3c0a7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f3c0a7660 0x7f9f3c0a7a40 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f9f2c006e30 tx=0x7f9f2c019000).stop 2020-11-20T12:17:26.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3161381222 shutdown_connections 2020-11-20T12:17:26.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f180557c0 0x7f9f18057c80 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9f3c1473e0 0x7f9f3c147840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9f3c0a7660 0x7f9f3c0a7a40 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 --2- 172.21.15.116:0/3161381222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9f3c0bd510 0x7f9f3c0bd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3161381222 >> 172.21.15.116:0/3161381222 conn(0x7f9f3c0c52e0 msgr2=0x7f9f3c064bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:26.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3161381222 shutdown_connections 2020-11-20T12:17:26.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.708+0000 7f9f42a0b700 1 -- 172.21.15.116:0/3161381222 wait complete. 2020-11-20T12:17:26.731 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:17:26.732 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:17:26.764 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:17:26.800 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:17:26.800 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2020-11-20T12:17:26.801 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:26.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.972+0000 7f007d6fe700 1 Processor -- start 2020-11-20T12:17:26.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.972+0000 7f007d6fe700 1 -- start start 2020-11-20T12:17:26.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f007d6fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00780657c0 0x7f00780c2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f007d6fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f00780c0510 0x7f0078064bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f007d6fe700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00780c09a0 con 0x7f00780c1eb0 2020-11-20T12:17:26.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f007d6fe700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00780c2e20 con 0x7f00780c0510 2020-11-20T12:17:26.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f007d6fe700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c30c0 con 0x7f00780657c0 2020-11-20T12:17:26.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0077fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f00780c1eb0 0x7f007814f130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54740/0 (socket says 172.21.15.116:54740) 2020-11-20T12:17:26.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0077fff700 1 -- 172.21.15.116:0/729669692 learned_addr learned my addr 172.21.15.116:0/729669692 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:26.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f007cefd700 1 --2- 172.21.15.116:0/729669692 >> v2:172.21.15.173:3300/0 conn(0x7f00780c0510 0x7f0078064bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f00777fe700 1 --2- 172.21.15.116:0/729669692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00780657c0 0x7f00780c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675502652 0 0) 0x7f00780c09a0 con 0x7f00780c1eb0 2020-11-20T12:17:26.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0064003240 con 0x7f00780c1eb0 2020-11-20T12:17:26.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 241087239 0 0) 0x7f0064003240 con 0x7f00780c1eb0 2020-11-20T12:17:26.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00780657c0 msgr2=0x7f00780c2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:26.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0076ffd700 1 --2- 172.21.15.116:0/729669692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00780657c0 0x7f00780c2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 >> v2:172.21.15.173:3300/0 conn(0x7f00780c0510 msgr2=0x7f0078064bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:26.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0076ffd700 1 --2- 172.21.15.116:0/729669692 >> v2:172.21.15.173:3300/0 conn(0x7f00780c0510 0x7f0078064bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00780c3360 con 0x7f00780c1eb0 2020-11-20T12:17:26.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f00680014e0 con 0x7f00780c1eb0 2020-11-20T12:17:26.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.976+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 >> v1:172.21.15.116:6789/0 conn(0x7f00780c1eb0 legacy=0x7f007814f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0076ffd700 1 --2- 172.21.15.116:0/729669692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00640037b0 0x7f0064003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0076ffd700 1 --2- 172.21.15.116:0/729669692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00640044e0 0x7f0064004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0076ffd700 1 --2- 172.21.15.116:0/729669692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00780657c0 0x7f0064004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c2e20 con 0x7f00640037b0 2020-11-20T12:17:26.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780c30c0 con 0x7f00640044e0 2020-11-20T12:17:26.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0064005340 con 0x7f00780657c0 2020-11-20T12:17:26.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0077fff700 1 --2- 172.21.15.116:0/729669692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00640037b0 0x7f0064003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0077fff700 1 -- 172.21.15.116:0/729669692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00780657c0 msgr2=0x7f0064004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:26.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0077fff700 1 --2- 172.21.15.116:0/729669692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00780657c0 0x7f0064004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0077fff700 1 -- 172.21.15.116:0/729669692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00640044e0 msgr2=0x7f0064004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0077fff700 1 --2- 172.21.15.116:0/729669692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00640044e0 0x7f0064004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0077fff700 1 -- 172.21.15.116:0/729669692 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0064003240 con 0x7f00640037b0 2020-11-20T12:17:26.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0077fff700 1 --2- 172.21.15.116:0/729669692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00640037b0 0x7f0064003bd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f006800c210 tx=0x7f00780c0460).ready entity=mon.0 client_cookie=c7cddd01999d045f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00680076f0 con 0x7f00640037b0 2020-11-20T12:17:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00680070d0 con 0x7f00640037b0 2020-11-20T12:17:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f0076ffd700 1 -- 172.21.15.116:0/729669692 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068007490 con 0x7f00640037b0 2020-11-20T12:17:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f007d6fe700 1 -- 172.21.15.116:0/729669692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00640037b0 msgr2=0x7f0064003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f007d6fe700 1 --2- 172.21.15.116:0/729669692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00640037b0 0x7f0064003bd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f006800c210 tx=0x7f00780c0460).stop 2020-11-20T12:17:26.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f007cefd700 1 -- 172.21.15.116:0/729669692 reap_dead start 2020-11-20T12:17:26.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f007d6fe700 1 -- 172.21.15.116:0/729669692 shutdown_connections 2020-11-20T12:17:26.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f007d6fe700 1 -- 172.21.15.116:0/729669692 >> 172.21.15.116:0/729669692 conn(0x7f0078061280 msgr2=0x7f0078061660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:26.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f007d6fe700 1 -- 172.21.15.116:0/729669692 shutdown_connections 2020-11-20T12:17:26.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.980+0000 7f007d6fe700 1 -- 172.21.15.116:0/729669692 wait complete. 2020-11-20T12:17:26.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.984+0000 7f007d6fe700 1 Processor -- start 2020-11-20T12:17:26.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.984+0000 7f007d6fe700 1 -- start start 2020-11-20T12:17:26.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.984+0000 7f007d6fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0078160550 0x7f00780a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.984+0000 7f007d6fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0078160930 0x7f00780a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.984+0000 7f007d6fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00780a2280 0x7f0078132870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.984+0000 7f007d6fe700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0078149210 con 0x7f0078160550 2020-11-20T12:17:26.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.984+0000 7f007d6fe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00780a2700 con 0x7f00780a2280 2020-11-20T12:17:26.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.984+0000 7f007d6fe700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0078149440 con 0x7f0078160930 2020-11-20T12:17:26.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.984+0000 7f0077fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0078160550 0x7f00780a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.984+0000 7f0077fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0078160550 0x7f00780a18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45318/0 (socket says 172.21.15.116:45318) 2020-11-20T12:17:26.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.984+0000 7f0077fff700 1 -- 172.21.15.116:0/3656211222 learned_addr learned my addr 172.21.15.116:0/3656211222 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:26.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f00777fe700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0078160930 0x7f00780a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f007cefd700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00780a2280 0x7f0078132870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0077fff700 1 -- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0078160930 msgr2=0x7f00780a1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0077fff700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0078160930 0x7f00780a1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0077fff700 1 -- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00780a2280 msgr2=0x7f0078132870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:26.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0077fff700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00780a2280 0x7f0078132870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:26.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0077fff700 1 -- 172.21.15.116:0/3656211222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00781496e0 con 0x7f0078160550 2020-11-20T12:17:26.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f00777fe700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0078160930 0x7f00780a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:26.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0077fff700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0078160550 0x7f00780a18a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f006800a000 tx=0x7f0068017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:26.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0057fff700 1 -- 172.21.15.116:0/3656211222 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068011470 con 0x7f0078160550 2020-11-20T12:17:26.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0057fff700 1 -- 172.21.15.116:0/3656211222 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00680115d0 con 0x7f0078160550 2020-11-20T12:17:26.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f007d6fe700 1 -- 172.21.15.116:0/3656211222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0078149980 con 0x7f0078160550 2020-11-20T12:17:26.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0057fff700 1 -- 172.21.15.116:0/3656211222 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0068011990 con 0x7f0078160550 2020-11-20T12:17:26.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f007d6fe700 1 -- 172.21.15.116:0/3656211222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0078149ec0 con 0x7f0078160550 2020-11-20T12:17:26.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0057fff700 1 -- 172.21.15.116:0/3656211222 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f006802b070 con 0x7f0078160550 2020-11-20T12:17:26.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f007d6fe700 1 -- 172.21.15.116:0/3656211222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0078044f60 con 0x7f0078160550 2020-11-20T12:17:26.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0057fff700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00500557c0 0x7f0050057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:26.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.988+0000 7f0057fff700 1 -- 172.21.15.116:0/3656211222 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0068070350 con 0x7f0078160550 2020-11-20T12:17:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.992+0000 7f00777fe700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00500557c0 0x7f0050057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.992+0000 7f00777fe700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00500557c0 0x7f0050057c80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f006c006a90 tx=0x7f006c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:26.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:26.996+0000 7f0057fff700 1 -- 172.21.15.116:0/3656211222 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f00680461f0 con 0x7f0078160550 2020-11-20T12:17:27.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.196+0000 7f007d6fe700 1 -- 172.21.15.116:0/3656211222 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0078062690 con 0x7f0078160550 2020-11-20T12:17:27.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.196+0000 7f0057fff700 1 -- 172.21.15.116:0/3656211222 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2851 (secure 0 0 0) 0x7f006801f020 con 0x7f0078160550 2020-11-20T12:17:27.198 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:27.198 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3031892761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3031892761},{"type":"v1","addr":"172.21.15.173:6836","nonce":3031892761}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1794965470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1794965470},{"type":"v1","addr":"172.21.15.173:6837","nonce":1794965470}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":9},{"gid":4995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/4143559744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":4143559744},{"type":"v1","addr":"172.21.15.116:6833","nonce":4143559744}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:57.901741+0000","modified":"2020-11-20T12:16:59.918401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4998":{"gid":4998,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6835/3994085646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3994085646},{"type":"v1","addr":"172.21.15.116:6835","nonce":3994085646}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T12:17:27.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 -- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00500557c0 msgr2=0x7f0050057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00500557c0 0x7f0050057c80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f006c006a90 tx=0x7f006c008040).stop 2020-11-20T12:17:27.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 -- 172.21.15.116:0/3656211222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0078160550 msgr2=0x7f00780a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0078160550 0x7f00780a18a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f006800a000 tx=0x7f0068017000).stop 2020-11-20T12:17:27.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 -- 172.21.15.116:0/3656211222 shutdown_connections 2020-11-20T12:17:27.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00500557c0 0x7f0050057c80 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00780a2280 0x7f0078132870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0078160550 0x7f00780a18a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 --2- 172.21.15.116:0/3656211222 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0078160930 0x7f00780a1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 -- 172.21.15.116:0/3656211222 >> 172.21.15.116:0/3656211222 conn(0x7f0078061280 msgr2=0x7f00780bb950 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:27.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 -- 172.21.15.116:0/3656211222 shutdown_connections 2020-11-20T12:17:27.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.200+0000 7f007d6fe700 1 -- 172.21.15.116:0/3656211222 wait complete. 2020-11-20T12:17:27.204 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 14 2020-11-20T12:17:27.226 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:27.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.400+0000 7f949a1ce700 1 Processor -- start 2020-11-20T12:17:27.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.400+0000 7f949a1ce700 1 -- start start 2020-11-20T12:17:27.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.404+0000 7f949a1ce700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f94941518e0 0x7f94940a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.404+0000 7f949a1ce700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9494151cc0 0x7f9494152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.404+0000 7f949a1ce700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9494152630 con 0x7f949414d6d0 2020-11-20T12:17:27.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.404+0000 7f949a1ce700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94940a39c0 con 0x7f94941518e0 2020-11-20T12:17:27.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.404+0000 7f949a1ce700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94940bf360 con 0x7f9494151cc0 2020-11-20T12:17:27.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.404+0000 7f94937fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f94941518e0 0x7f94940a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.404+0000 7f9492ffd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9494151cc0 0x7f9494152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.404+0000 7f9492ffd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9494151cc0 0x7f9494152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:44380/0 (socket says 172.21.15.116:44380) 2020-11-20T12:17:27.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.404+0000 7f9492ffd700 1 -- 172.21.15.116:0/2131123206 learned_addr learned my addr 172.21.15.116:0/2131123206 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:27.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f94937fe700 1 -- 172.21.15.116:0/2131123206 >> v2:172.21.15.173:3300/0 conn(0x7f9494151cc0 msgr2=0x7f9494152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:27.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f94937fe700 1 --2- 172.21.15.116:0/2131123206 >> v2:172.21.15.173:3300/0 conn(0x7f9494151cc0 0x7f9494152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f94937fe700 1 -- 172.21.15.116:0/2131123206 >> v1:172.21.15.173:6789/0 conn(0x7f949414d6d0 legacy=0x7f949414db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f94937fe700 1 -- 172.21.15.116:0/2131123206 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940bf600 con 0x7f94941518e0 2020-11-20T12:17:27.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f9492ffd700 1 --2- 172.21.15.116:0/2131123206 >> v2:172.21.15.173:3300/0 conn(0x7f9494151cc0 0x7f9494152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:27.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f94937fe700 1 --2- 172.21.15.116:0/2131123206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94941518e0 0x7f94940a34d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f94940a6f20 tx=0x7f9484006df0).ready entity=mon.0 client_cookie=6c35a6c7bbdc0368 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:27.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f94927fc700 1 -- 172.21.15.116:0/2131123206 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484009770 con 0x7f94941518e0 2020-11-20T12:17:27.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f94927fc700 1 -- 172.21.15.116:0/2131123206 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94840098d0 con 0x7f94941518e0 2020-11-20T12:17:27.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f94927fc700 1 -- 172.21.15.116:0/2131123206 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484009c50 con 0x7f94941518e0 2020-11-20T12:17:27.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f949a1ce700 1 -- 172.21.15.116:0/2131123206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94941518e0 msgr2=0x7f94940a34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f949a1ce700 1 --2- 172.21.15.116:0/2131123206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94941518e0 0x7f94940a34d0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f94940a6f20 tx=0x7f9484006df0).stop 2020-11-20T12:17:27.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f949a1ce700 1 -- 172.21.15.116:0/2131123206 shutdown_connections 2020-11-20T12:17:27.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f949a1ce700 1 --2- 172.21.15.116:0/2131123206 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94941518e0 0x7f94940a34d0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f949a1ce700 1 --2- 172.21.15.116:0/2131123206 >> v2:172.21.15.173:3300/0 conn(0x7f9494151cc0 0x7f9494152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f949a1ce700 1 -- 172.21.15.116:0/2131123206 >> 172.21.15.116:0/2131123206 conn(0x7f9494060c80 msgr2=0x7f94940c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:27.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f949a1ce700 1 -- 172.21.15.116:0/2131123206 shutdown_connections 2020-11-20T12:17:27.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.408+0000 7f949a1ce700 1 -- 172.21.15.116:0/2131123206 wait complete. 2020-11-20T12:17:27.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f949a1ce700 1 Processor -- start 2020-11-20T12:17:27.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f949a1ce700 1 -- start start 2020-11-20T12:17:27.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f949a1ce700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f949414d6d0 0x7f9494090c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f949a1ce700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94941518e0 0x7f9494091190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f949a1ce700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9494151cc0 0x7f94941471d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f949a1ce700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949408fdb0 con 0x7f949414d6d0 2020-11-20T12:17:27.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f949a1ce700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949408ffa0 con 0x7f94941518e0 2020-11-20T12:17:27.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f949a1ce700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9494090240 con 0x7f9494151cc0 2020-11-20T12:17:27.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f94937fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f949414d6d0 0x7f9494090c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f94937fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f949414d6d0 0x7f9494090c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45332/0 (socket says 172.21.15.116:45332) 2020-11-20T12:17:27.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f94937fe700 1 -- 172.21.15.116:0/945163831 learned_addr learned my addr 172.21.15.116:0/945163831 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:27.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f9493fff700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9494151cc0 0x7f94941471d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f9492ffd700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94941518e0 0x7f9494091190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f94937fe700 1 -- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9494151cc0 msgr2=0x7f94941471d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f94937fe700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9494151cc0 0x7f94941471d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f94937fe700 1 -- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94941518e0 msgr2=0x7f9494091190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f94937fe700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94941518e0 0x7f9494091190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f94937fe700 1 -- 172.21.15.116:0/945163831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94940904e0 con 0x7f949414d6d0 2020-11-20T12:17:27.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f9492ffd700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94941518e0 0x7f9494091190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:27.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f9493fff700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9494151cc0 0x7f94941471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:27.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.412+0000 7f94937fe700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f949414d6d0 0x7f9494090c20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f948400c010 tx=0x7f9484015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:27.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.416+0000 7f94737fe700 1 -- 172.21.15.116:0/945163831 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9484011dd0 con 0x7f949414d6d0 2020-11-20T12:17:27.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.416+0000 7f94737fe700 1 -- 172.21.15.116:0/945163831 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f948401e460 con 0x7f949414d6d0 2020-11-20T12:17:27.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.416+0000 7f94737fe700 1 -- 172.21.15.116:0/945163831 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948401e800 con 0x7f949414d6d0 2020-11-20T12:17:27.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.416+0000 7f949a1ce700 1 -- 172.21.15.116:0/945163831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9494090780 con 0x7f949414d6d0 2020-11-20T12:17:27.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.416+0000 7f949a1ce700 1 -- 172.21.15.116:0/945163831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94941462f0 con 0x7f949414d6d0 2020-11-20T12:17:27.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.416+0000 7f949a1ce700 1 -- 172.21.15.116:0/945163831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949405ca30 con 0x7f949414d6d0 2020-11-20T12:17:27.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.416+0000 7f94737fe700 1 -- 172.21.15.116:0/945163831 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f948401a040 con 0x7f949414d6d0 2020-11-20T12:17:27.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.416+0000 7f94737fe700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f946c0557c0 0x7f946c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.416+0000 7f94737fe700 1 -- 172.21.15.116:0/945163831 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f948406f810 con 0x7f949414d6d0 2020-11-20T12:17:27.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.420+0000 7f9492ffd700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f946c0557c0 0x7f946c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.420+0000 7f9492ffd700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f946c0557c0 0x7f946c057c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f9494147cc0 tx=0x7f9488008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:27.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.424+0000 7f94737fe700 1 -- 172.21.15.116:0/945163831 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9484045760 con 0x7f949414d6d0 2020-11-20T12:17:27.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.584+0000 7f949a1ce700 1 -- 172.21.15.116:0/945163831 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9494151260 con 0x7f949414d6d0 2020-11-20T12:17:27.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.584+0000 7f94737fe700 1 -- 172.21.15.116:0/945163831 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f9484030200 con 0x7f949414d6d0 2020-11-20T12:17:27.587 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:27.587 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:58.907407+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:16:55.781578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T12:16:56.900539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:27.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 -- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f946c0557c0 msgr2=0x7f946c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f946c0557c0 0x7f946c057c80 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f9494147cc0 tx=0x7f9488008040).stop 2020-11-20T12:17:27.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 -- 172.21.15.116:0/945163831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f949414d6d0 msgr2=0x7f9494090c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f949414d6d0 0x7f9494090c20 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f948400c010 tx=0x7f9484015000).stop 2020-11-20T12:17:27.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 -- 172.21.15.116:0/945163831 shutdown_connections 2020-11-20T12:17:27.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f946c0557c0 0x7f946c057c80 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9494151cc0 0x7f94941471d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f949414d6d0 0x7f9494090c20 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 --2- 172.21.15.116:0/945163831 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94941518e0 0x7f9494091190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 -- 172.21.15.116:0/945163831 >> 172.21.15.116:0/945163831 conn(0x7f9494060c80 msgr2=0x7f94940c3640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:27.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 -- 172.21.15.116:0/945163831 shutdown_connections 2020-11-20T12:17:27.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.588+0000 7f949a1ce700 1 -- 172.21.15.116:0/945163831 wait complete. 2020-11-20T12:17:27.612 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:16:55.781578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:16:56.900539+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:17:27.613 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:27.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.788+0000 7f595e7f1700 1 Processor -- start 2020-11-20T12:17:27.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.788+0000 7f595e7f1700 1 -- start start 2020-11-20T12:17:27.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595e7f1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f59580644e0 0x7f59580c0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595e7f1700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f59580648c0 0x7f59580670c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595e7f1700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59580c1620 con 0x7f5958137d60 2020-11-20T12:17:27.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595e7f1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59580c2cd0 con 0x7f59580644e0 2020-11-20T12:17:27.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595e7f1700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59580c2f70 con 0x7f59580648c0 2020-11-20T12:17:27.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595d7ef700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f59580644e0 0x7f59580c0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595d7ef700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f59580644e0 0x7f59580c0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45340/0 (socket says 172.21.15.116:45340) 2020-11-20T12:17:27.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595d7ef700 1 -- 172.21.15.116:0/2108018459 learned_addr learned my addr 172.21.15.116:0/2108018459 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:27.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595cfee700 1 --2- 172.21.15.116:0/2108018459 >> v2:172.21.15.173:3300/0 conn(0x7f59580648c0 0x7f59580670c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595d7ef700 1 -- 172.21.15.116:0/2108018459 >> v2:172.21.15.173:3300/0 conn(0x7f59580648c0 msgr2=0x7f59580670c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:27.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595d7ef700 1 --2- 172.21.15.116:0/2108018459 >> v2:172.21.15.173:3300/0 conn(0x7f59580648c0 0x7f59580670c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595d7ef700 1 -- 172.21.15.116:0/2108018459 >> v1:172.21.15.173:6789/0 conn(0x7f5958137d60 legacy=0x7f5958151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:27.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595d7ef700 1 -- 172.21.15.116:0/2108018459 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59580c3210 con 0x7f59580644e0 2020-11-20T12:17:27.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595cfee700 1 --2- 172.21.15.116:0/2108018459 >> v2:172.21.15.173:3300/0 conn(0x7f59580648c0 0x7f59580670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:27.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595d7ef700 1 --2- 172.21.15.116:0/2108018459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59580644e0 0x7f59580c0d80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f59580666f0 tx=0x7f5954006eb0).ready entity=mon.0 client_cookie=cdd5ecb5f7faf23c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:27.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f5947fff700 1 -- 172.21.15.116:0/2108018459 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954009920 con 0x7f59580644e0 2020-11-20T12:17:27.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f5947fff700 1 -- 172.21.15.116:0/2108018459 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5954009a80 con 0x7f59580644e0 2020-11-20T12:17:27.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f5947fff700 1 -- 172.21.15.116:0/2108018459 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954009e00 con 0x7f59580644e0 2020-11-20T12:17:27.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595e7f1700 1 -- 172.21.15.116:0/2108018459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59580644e0 msgr2=0x7f59580c0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595e7f1700 1 --2- 172.21.15.116:0/2108018459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59580644e0 0x7f59580c0d80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f59580666f0 tx=0x7f5954006eb0).stop 2020-11-20T12:17:27.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595e7f1700 1 -- 172.21.15.116:0/2108018459 shutdown_connections 2020-11-20T12:17:27.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595e7f1700 1 --2- 172.21.15.116:0/2108018459 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59580644e0 0x7f59580c0d80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595e7f1700 1 --2- 172.21.15.116:0/2108018459 >> v2:172.21.15.173:3300/0 conn(0x7f59580648c0 0x7f59580670c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.792+0000 7f595e7f1700 1 -- 172.21.15.116:0/2108018459 >> 172.21.15.116:0/2108018459 conn(0x7f5958061d50 msgr2=0x7f59580c50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:27.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.796+0000 7f595e7f1700 1 -- 172.21.15.116:0/2108018459 shutdown_connections 2020-11-20T12:17:27.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.796+0000 7f595e7f1700 1 -- 172.21.15.116:0/2108018459 wait complete. 2020-11-20T12:17:27.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.796+0000 7f595e7f1700 1 Processor -- start 2020-11-20T12:17:27.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.796+0000 7f595e7f1700 1 -- start start 2020-11-20T12:17:27.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.796+0000 7f595e7f1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59580644e0 0x7f59580bd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.796+0000 7f595e7f1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59580648c0 0x7f59580be830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.796+0000 7f595e7f1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5958137d60 0x7f5958134aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.796+0000 7f595e7f1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59580bd870 con 0x7f5958137d60 2020-11-20T12:17:27.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.796+0000 7f595e7f1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59580bdb10 con 0x7f59580644e0 2020-11-20T12:17:27.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.796+0000 7f595e7f1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59580bddb0 con 0x7f59580648c0 2020-11-20T12:17:27.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.796+0000 7f595dff0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5958137d60 0x7f5958134aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595dff0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5958137d60 0x7f5958134aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45350/0 (socket says 172.21.15.116:45350) 2020-11-20T12:17:27.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595cfee700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59580648c0 0x7f59580be830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595dff0700 1 -- 172.21.15.116:0/3375184190 learned_addr learned my addr 172.21.15.116:0/3375184190 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:27.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595d7ef700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59580644e0 0x7f59580bd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595dff0700 1 -- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59580648c0 msgr2=0x7f59580be830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595dff0700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59580648c0 0x7f59580be830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595dff0700 1 -- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59580644e0 msgr2=0x7f59580bd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595dff0700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59580644e0 0x7f59580bd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595dff0700 1 -- 172.21.15.116:0/3375184190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59580be050 con 0x7f5958137d60 2020-11-20T12:17:27.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595dff0700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5958137d60 0x7f5958134aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f595000bae0 tx=0x7f59500090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:27.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f59457fa700 1 -- 172.21.15.116:0/3375184190 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595000c710 con 0x7f5958137d60 2020-11-20T12:17:27.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f59457fa700 1 -- 172.21.15.116:0/3375184190 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595000c870 con 0x7f5958137d60 2020-11-20T12:17:27.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595e7f1700 1 -- 172.21.15.116:0/3375184190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59580be2f0 con 0x7f5958137d60 2020-11-20T12:17:27.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f59457fa700 1 -- 172.21.15.116:0/3375184190 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595000ca20 con 0x7f5958137d60 2020-11-20T12:17:27.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595e7f1700 1 -- 172.21.15.116:0/3375184190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5958134f90 con 0x7f5958137d60 2020-11-20T12:17:27.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.800+0000 7f595e7f1700 1 -- 172.21.15.116:0/3375184190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59580c0560 con 0x7f5958137d60 2020-11-20T12:17:27.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.804+0000 7f59457fa700 1 -- 172.21.15.116:0/3375184190 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f595001c070 con 0x7f5958137d60 2020-11-20T12:17:27.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.804+0000 7f59457fa700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59300557c0 0x7f5930057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:27.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.804+0000 7f59457fa700 1 -- 172.21.15.116:0/3375184190 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f595006dc60 con 0x7f5958137d60 2020-11-20T12:17:27.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.804+0000 7f595d7ef700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59300557c0 0x7f5930057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:27.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.804+0000 7f595d7ef700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59300557c0 0x7f5930057c80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f5954006ef0 tx=0x7f5954015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:27.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.808+0000 7f59457fa700 1 -- 172.21.15.116:0/3375184190 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f59580c0560 con 0x7f5958137d60 2020-11-20T12:17:27.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.968+0000 7f595e7f1700 1 -- 172.21.15.116:0/3375184190 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f59580c0560 con 0x7f5958137d60 2020-11-20T12:17:27.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f59457fa700 1 -- 172.21.15.116:0/3375184190 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v34) v1 ==== 110+0+10 (secure 0 0 0) 0x7f59580c0560 con 0x7f5958137d60 2020-11-20T12:17:27.972 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:27.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 -- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59300557c0 msgr2=0x7f5930057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59300557c0 0x7f5930057c80 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f5954006ef0 tx=0x7f5954015000).stop 2020-11-20T12:17:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 -- 172.21.15.116:0/3375184190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5958137d60 msgr2=0x7f5958134aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5958137d60 0x7f5958134aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f595000bae0 tx=0x7f59500090f0).stop 2020-11-20T12:17:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 -- 172.21.15.116:0/3375184190 shutdown_connections 2020-11-20T12:17:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59300557c0 0x7f5930057c80 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5958137d60 0x7f5958134aa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59580644e0 0x7f59580bd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 --2- 172.21.15.116:0/3375184190 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59580648c0 0x7f59580be830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 -- 172.21.15.116:0/3375184190 >> 172.21.15.116:0/3375184190 conn(0x7f5958061d50 msgr2=0x7f59581527c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 -- 172.21.15.116:0/3375184190 shutdown_connections 2020-11-20T12:17:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:27.972+0000 7f595e7f1700 1 -- 172.21.15.116:0/3375184190 wait complete. 2020-11-20T12:17:27.995 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:28.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.168+0000 7f2449ac5700 1 Processor -- start 2020-11-20T12:17:28.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.168+0000 7f2449ac5700 1 -- start start 2020-11-20T12:17:28.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.172+0000 7f2449ac5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24440bd780 0x7f24440a74b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.172+0000 7f2449ac5700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f244414f530 0x7f24440a79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.172+0000 7f2449ac5700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2444155620 con 0x7f244414f150 2020-11-20T12:17:28.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.172+0000 7f2449ac5700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24441558c0 con 0x7f244414f530 2020-11-20T12:17:28.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.172+0000 7f2449ac5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2444155b60 con 0x7f24440bd780 2020-11-20T12:17:28.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.172+0000 7f24437fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f244414f150 0x7f24440c87f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54788/0 (socket says 172.21.15.116:54788) 2020-11-20T12:17:28.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.172+0000 7f24437fe700 1 -- 172.21.15.116:0/1230126395 learned_addr learned my addr 172.21.15.116:0/1230126395 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:28.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.172+0000 7f2443fff700 1 --2- 172.21.15.116:0/1230126395 >> v2:172.21.15.173:3300/0 conn(0x7f244414f530 0x7f24440a79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.172+0000 7f2442ffd700 1 --2- 172.21.15.116:0/1230126395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24440bd780 0x7f24440a74b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f24427fc700 1 -- 172.21.15.116:0/1230126395 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757483990 0 0) 0x7f2444155620 con 0x7f244414f150 2020-11-20T12:17:28.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2443fff700 1 -- 172.21.15.116:0/1230126395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24440bd780 msgr2=0x7f24440a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:28.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2443fff700 1 --2- 172.21.15.116:0/1230126395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24440bd780 0x7f24440a74b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2443fff700 1 -- 172.21.15.116:0/1230126395 >> v1:172.21.15.116:6789/0 conn(0x7f244414f150 legacy=0x7f24440c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2443fff700 1 -- 172.21.15.116:0/1230126395 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2444155e00 con 0x7f244414f530 2020-11-20T12:17:28.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2443fff700 1 --2- 172.21.15.116:0/1230126395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f244414f530 0x7f24440a79a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f243c009c90 tx=0x7f243c0091d0).ready entity=mon.1 client_cookie=b4c7a99d6d8566a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:28.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f24427fc700 1 -- 172.21.15.116:0/1230126395 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c00ba70 con 0x7f244414f530 2020-11-20T12:17:28.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f24427fc700 1 -- 172.21.15.116:0/1230126395 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c00bbd0 con 0x7f244414f530 2020-11-20T12:17:28.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f24427fc700 1 -- 172.21.15.116:0/1230126395 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c011400 con 0x7f244414f530 2020-11-20T12:17:28.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2449ac5700 1 -- 172.21.15.116:0/1230126395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f244414f530 msgr2=0x7f24440a79a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2449ac5700 1 --2- 172.21.15.116:0/1230126395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f244414f530 0x7f24440a79a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f243c009c90 tx=0x7f243c0091d0).stop 2020-11-20T12:17:28.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2449ac5700 1 -- 172.21.15.116:0/1230126395 shutdown_connections 2020-11-20T12:17:28.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2449ac5700 1 --2- 172.21.15.116:0/1230126395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f244414f530 0x7f24440a79a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2449ac5700 1 --2- 172.21.15.116:0/1230126395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24440bd780 0x7f24440a74b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2449ac5700 1 -- 172.21.15.116:0/1230126395 >> 172.21.15.116:0/1230126395 conn(0x7f2444061580 msgr2=0x7f2444061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:28.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2449ac5700 1 -- 172.21.15.116:0/1230126395 shutdown_connections 2020-11-20T12:17:28.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.176+0000 7f2449ac5700 1 -- 172.21.15.116:0/1230126395 wait complete. 2020-11-20T12:17:28.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2449ac5700 1 Processor -- start 2020-11-20T12:17:28.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2449ac5700 1 -- start start 2020-11-20T12:17:28.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2449ac5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24440bd780 0x7f2444165060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2449ac5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f244414f150 0x7f24440ad4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2449ac5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244414f530 0x7f24440ae0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2449ac5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24440c0a90 con 0x7f244414f530 2020-11-20T12:17:28.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2449ac5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2444161720 con 0x7f244414f150 2020-11-20T12:17:28.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2449ac5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24441619c0 con 0x7f24440bd780 2020-11-20T12:17:28.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2443fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244414f530 0x7f24440ae0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2443fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244414f530 0x7f24440ae0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45364/0 (socket says 172.21.15.116:45364) 2020-11-20T12:17:28.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2443fff700 1 -- 172.21.15.116:0/3223282292 learned_addr learned my addr 172.21.15.116:0/3223282292 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:28.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f24437fe700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24440bd780 0x7f2444165060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2442ffd700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f244414f150 0x7f24440ad4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2443fff700 1 -- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24440bd780 msgr2=0x7f2444165060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.180+0000 7f2443fff700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24440bd780 0x7f2444165060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.184+0000 7f2443fff700 1 -- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f244414f150 msgr2=0x7f24440ad4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.184+0000 7f2443fff700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f244414f150 0x7f24440ad4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.184+0000 7f2443fff700 1 -- 172.21.15.116:0/3223282292 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2444161c60 con 0x7f244414f530 2020-11-20T12:17:28.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.184+0000 7f2443fff700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244414f530 0x7f24440ae0e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f243c011000 tx=0x7f243c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:28.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.184+0000 7f24237fe700 1 -- 172.21.15.116:0/3223282292 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c01c6c0 con 0x7f244414f530 2020-11-20T12:17:28.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.184+0000 7f24237fe700 1 -- 172.21.15.116:0/3223282292 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f243c01c820 con 0x7f244414f530 2020-11-20T12:17:28.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.184+0000 7f2449ac5700 1 -- 172.21.15.116:0/3223282292 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24440c0cf0 con 0x7f244414f530 2020-11-20T12:17:28.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.184+0000 7f24237fe700 1 -- 172.21.15.116:0/3223282292 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f243c01cba0 con 0x7f244414f530 2020-11-20T12:17:28.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.184+0000 7f2449ac5700 1 -- 172.21.15.116:0/3223282292 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24440ae5d0 con 0x7f244414f530 2020-11-20T12:17:28.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.184+0000 7f2449ac5700 1 -- 172.21.15.116:0/3223282292 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24440c9210 con 0x7f244414f530 2020-11-20T12:17:28.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.184+0000 7f24237fe700 1 -- 172.21.15.116:0/3223282292 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f243c024030 con 0x7f244414f530 2020-11-20T12:17:28.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.188+0000 7f24237fe700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f241c0557c0 0x7f241c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.188+0000 7f24237fe700 1 -- 172.21.15.116:0/3223282292 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f243c06ff50 con 0x7f244414f530 2020-11-20T12:17:28.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.188+0000 7f24437fe700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f241c0557c0 0x7f241c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.188+0000 7f24437fe700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f241c0557c0 0x7f241c057c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f24340011a0 tx=0x7f2434007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:28.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.196+0000 7f24237fe700 1 -- 172.21.15.116:0/3223282292 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f243c045ea0 con 0x7f244414f530 2020-11-20T12:17:28.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.356+0000 7f2449ac5700 1 -- 172.21.15.116:0/3223282292 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f24440c0e80 con 0x7f244414f530 2020-11-20T12:17:28.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f24237fe700 1 -- 172.21.15.116:0/3223282292 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v34) v1 ==== 92+0+10 (secure 0 0 0) 0x7f243c022070 con 0x7f244414f530 2020-11-20T12:17:28.359 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:28.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 -- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f241c0557c0 msgr2=0x7f241c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f241c0557c0 0x7f241c057c80 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f24340011a0 tx=0x7f2434007a00).stop 2020-11-20T12:17:28.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 -- 172.21.15.116:0/3223282292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244414f530 msgr2=0x7f24440ae0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244414f530 0x7f24440ae0e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f243c011000 tx=0x7f243c016000).stop 2020-11-20T12:17:28.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 -- 172.21.15.116:0/3223282292 shutdown_connections 2020-11-20T12:17:28.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f241c0557c0 0x7f241c057c80 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f244414f530 0x7f24440ae0e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f24440bd780 0x7f2444165060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 --2- 172.21.15.116:0/3223282292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f244414f150 0x7f24440ad4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 -- 172.21.15.116:0/3223282292 >> 172.21.15.116:0/3223282292 conn(0x7f2444061580 msgr2=0x7f24441506f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:28.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 -- 172.21.15.116:0/3223282292 shutdown_connections 2020-11-20T12:17:28.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.360+0000 7f2449ac5700 1 -- 172.21.15.116:0/3223282292 wait complete. 2020-11-20T12:17:28.386 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:17:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45dd21e700 1 Processor -- start 2020-11-20T12:17:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45dd21e700 1 -- start start 2020-11-20T12:17:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45dd21e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f45d814f150 0x7f45d80c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45dd21e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45d814f530 0x7f45d80c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45dd21e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45d8155620 con 0x7f45d80bd780 2020-11-20T12:17:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45dd21e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45d81558c0 con 0x7f45d814f150 2020-11-20T12:17:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45dd21e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d8155b60 con 0x7f45d814f530 2020-11-20T12:17:28.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45d6d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f45d814f150 0x7f45d80c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45d6d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f45d814f150 0x7f45d80c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45372/0 (socket says 172.21.15.116:45372) 2020-11-20T12:17:28.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45d6d9d700 1 -- 172.21.15.116:0/497354861 learned_addr learned my addr 172.21.15.116:0/497354861 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:28.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45d759e700 1 --2- 172.21.15.116:0/497354861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45d814f530 0x7f45d80c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45d6d9d700 1 -- 172.21.15.116:0/497354861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45d814f530 msgr2=0x7f45d80c8ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:28.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45d6d9d700 1 --2- 172.21.15.116:0/497354861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45d814f530 0x7f45d80c8ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45d6d9d700 1 -- 172.21.15.116:0/497354861 >> v1:172.21.15.173:6789/0 conn(0x7f45d80bd780 legacy=0x7f45d80a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.560+0000 7f45d6d9d700 1 -- 172.21.15.116:0/497354861 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d8155e00 con 0x7f45d814f150 2020-11-20T12:17:28.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45d6d9d700 1 --2- 172.21.15.116:0/497354861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45d814f150 0x7f45d80c87f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f45c8006fd0 tx=0x7f45c8006eb0).ready entity=mon.0 client_cookie=56c29beaaa0bb286 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:28.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45d659c700 1 -- 172.21.15.116:0/497354861 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8009960 con 0x7f45d814f150 2020-11-20T12:17:28.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45d659c700 1 -- 172.21.15.116:0/497354861 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c8009ac0 con 0x7f45d814f150 2020-11-20T12:17:28.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45d659c700 1 -- 172.21.15.116:0/497354861 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8009e40 con 0x7f45d814f150 2020-11-20T12:17:28.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45dd21e700 1 -- 172.21.15.116:0/497354861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45d814f150 msgr2=0x7f45d80c87f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45dd21e700 1 --2- 172.21.15.116:0/497354861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45d814f150 0x7f45d80c87f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f45c8006fd0 tx=0x7f45c8006eb0).stop 2020-11-20T12:17:28.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45dd21e700 1 -- 172.21.15.116:0/497354861 shutdown_connections 2020-11-20T12:17:28.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45dd21e700 1 --2- 172.21.15.116:0/497354861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45d814f530 0x7f45d80c8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45dd21e700 1 --2- 172.21.15.116:0/497354861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45d814f150 0x7f45d80c87f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45dd21e700 1 -- 172.21.15.116:0/497354861 >> 172.21.15.116:0/497354861 conn(0x7f45d8061580 msgr2=0x7f45d8061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45dd21e700 1 -- 172.21.15.116:0/497354861 shutdown_connections 2020-11-20T12:17:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45dd21e700 1 -- 172.21.15.116:0/497354861 wait complete. 2020-11-20T12:17:28.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45dd21e700 1 Processor -- start 2020-11-20T12:17:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.564+0000 7f45dd21e700 1 -- start start 2020-11-20T12:17:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45dd21e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45d80bd780 0x7f45d80ae1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45dd21e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45d814f150 0x7f45d80ab5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45dd21e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45d814f530 0x7f45d8161220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45dd21e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d814ba50 con 0x7f45d80bd780 2020-11-20T12:17:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45dd21e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80ac050 con 0x7f45d814f150 2020-11-20T12:17:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45dd21e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45d80ae6c0 con 0x7f45d814f530 2020-11-20T12:17:28.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45d6d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45d80bd780 0x7f45d80ae1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45d6d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45d80bd780 0x7f45d80ae1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45376/0 (socket says 172.21.15.116:45376) 2020-11-20T12:17:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45d6d9d700 1 -- 172.21.15.116:0/505873373 learned_addr learned my addr 172.21.15.116:0/505873373 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45d759e700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45d814f530 0x7f45d8161220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45cffff700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45d814f150 0x7f45d80ab5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45d6d9d700 1 -- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45d814f530 msgr2=0x7f45d8161220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45d6d9d700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45d814f530 0x7f45d8161220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45d6d9d700 1 -- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45d814f150 msgr2=0x7f45d80ab5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45d6d9d700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45d814f150 0x7f45d80ab5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45d6d9d700 1 -- 172.21.15.116:0/505873373 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d80ac2b0 con 0x7f45d80bd780 2020-11-20T12:17:28.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45d6d9d700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45d80bd780 0x7f45d80ae1d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f45c8006ef0 tx=0x7f45c8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:28.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45ceffd700 1 -- 172.21.15.116:0/505873373 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c8011dd0 con 0x7f45d80bd780 2020-11-20T12:17:28.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45ceffd700 1 -- 172.21.15.116:0/505873373 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45c801e460 con 0x7f45d80bd780 2020-11-20T12:17:28.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45dd21e700 1 -- 172.21.15.116:0/505873373 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d80ac550 con 0x7f45d80bd780 2020-11-20T12:17:28.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45ceffd700 1 -- 172.21.15.116:0/505873373 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45c801e7e0 con 0x7f45d80bd780 2020-11-20T12:17:28.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.568+0000 7f45dd21e700 1 -- 172.21.15.116:0/505873373 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d81533b0 con 0x7f45d80bd780 2020-11-20T12:17:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.572+0000 7f45dd21e700 1 -- 172.21.15.116:0/505873373 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d80ac710 con 0x7f45d80bd780 2020-11-20T12:17:28.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.572+0000 7f45ceffd700 1 -- 172.21.15.116:0/505873373 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f45c8025030 con 0x7f45d80bd780 2020-11-20T12:17:28.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.572+0000 7f45ceffd700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45b40557c0 0x7f45b4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.572+0000 7f45ceffd700 1 -- 172.21.15.116:0/505873373 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f45c806fdb0 con 0x7f45d80bd780 2020-11-20T12:17:28.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.576+0000 7f45cffff700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45b40557c0 0x7f45b4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.576+0000 7f45cffff700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45b40557c0 0x7f45b4057c80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f45d80c21e0 tx=0x7f45c0007b90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:28.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.576+0000 7f45ceffd700 1 -- 172.21.15.116:0/505873373 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f45c8045d00 con 0x7f45d80bd780 2020-11-20T12:17:28.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.732+0000 7f45dd21e700 1 -- 172.21.15.116:0/505873373 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45d805ca30 con 0x7f45d80bd780 2020-11-20T12:17:28.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45ceffd700 1 -- 172.21.15.116:0/505873373 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v34) v1 ==== 104+0+10 (secure 0 0 0) 0x7f45c8023070 con 0x7f45d80bd780 2020-11-20T12:17:28.736 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:28.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 -- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45b40557c0 msgr2=0x7f45b4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45b40557c0 0x7f45b4057c80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f45d80c21e0 tx=0x7f45c0007b90).stop 2020-11-20T12:17:28.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 -- 172.21.15.116:0/505873373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45d80bd780 msgr2=0x7f45d80ae1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45d80bd780 0x7f45d80ae1d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f45c8006ef0 tx=0x7f45c8015000).stop 2020-11-20T12:17:28.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 -- 172.21.15.116:0/505873373 shutdown_connections 2020-11-20T12:17:28.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45b40557c0 0x7f45b4057c80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45d814f530 0x7f45d8161220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45d80bd780 0x7f45d80ae1d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 --2- 172.21.15.116:0/505873373 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45d814f150 0x7f45d80ab5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 -- 172.21.15.116:0/505873373 >> 172.21.15.116:0/505873373 conn(0x7f45d8061580 msgr2=0x7f45d80c2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:28.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 -- 172.21.15.116:0/505873373 shutdown_connections 2020-11-20T12:17:28.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.736+0000 7f45dd21e700 1 -- 172.21.15.116:0/505873373 wait complete. 2020-11-20T12:17:28.760 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:17:28.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.936+0000 7f72219ce700 1 Processor -- start 2020-11-20T12:17:28.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.936+0000 7f72219ce700 1 -- start start 2020-11-20T12:17:28.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.940+0000 7f72219ce700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f721c067910 0x7f721c067cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.940+0000 7f72219ce700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f721c064760 0x7f721c0bc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.940+0000 7f72219ce700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c0bc550 con 0x7f721c14f1f0 2020-11-20T12:17:28.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.940+0000 7f72219ce700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c060cb0 con 0x7f721c067910 2020-11-20T12:17:28.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.940+0000 7f72219ce700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c060f50 con 0x7f721c064760 2020-11-20T12:17:28.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.940+0000 7f721bfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f721c14f1f0 0x7f721c0bac10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54820/0 (socket says 172.21.15.116:54820) 2020-11-20T12:17:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.940+0000 7f721bfff700 1 -- 172.21.15.116:0/2740977313 learned_addr learned my addr 172.21.15.116:0/2740977313 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.940+0000 7f721affd700 1 --2- 172.21.15.116:0/2740977313 >> v2:172.21.15.116:3300/0 conn(0x7f721c067910 0x7f721c067cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.940+0000 7f721a7fc700 1 --2- 172.21.15.116:0/2740977313 >> v2:172.21.15.173:3300/0 conn(0x7f721c064760 0x7f721c0bc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f721a7fc700 1 -- 172.21.15.116:0/2740977313 >> v2:172.21.15.116:3300/0 conn(0x7f721c067910 msgr2=0x7f721c067cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:28.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f721a7fc700 1 --2- 172.21.15.116:0/2740977313 >> v2:172.21.15.116:3300/0 conn(0x7f721c067910 0x7f721c067cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f721a7fc700 1 -- 172.21.15.116:0/2740977313 >> v1:172.21.15.116:6789/0 conn(0x7f721c14f1f0 legacy=0x7f721c0bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f721a7fc700 1 -- 172.21.15.116:0/2740977313 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c060170 con 0x7f721c064760 2020-11-20T12:17:28.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f7219ffb700 1 -- 172.21.15.116:0/2740977313 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187227956 0 0) 0x7f721c0bc550 con 0x7f721c14f1f0 2020-11-20T12:17:28.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f721a7fc700 1 --2- 172.21.15.116:0/2740977313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f721c064760 0x7f721c0bc060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f721c0681e0 tx=0x7f7210006df0).ready entity=mon.1 client_cookie=978deae5ee3bff7c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:28.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f7219ffb700 1 -- 172.21.15.116:0/2740977313 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210009840 con 0x7f721c064760 2020-11-20T12:17:28.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f7219ffb700 1 -- 172.21.15.116:0/2740977313 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72100099a0 con 0x7f721c064760 2020-11-20T12:17:28.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f7219ffb700 1 -- 172.21.15.116:0/2740977313 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7210009d20 con 0x7f721c064760 2020-11-20T12:17:28.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f72219ce700 1 -- 172.21.15.116:0/2740977313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f721c064760 msgr2=0x7f721c0bc060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f72219ce700 1 --2- 172.21.15.116:0/2740977313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f721c064760 0x7f721c0bc060 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f721c0681e0 tx=0x7f7210006df0).stop 2020-11-20T12:17:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f72219ce700 1 -- 172.21.15.116:0/2740977313 shutdown_connections 2020-11-20T12:17:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f72219ce700 1 --2- 172.21.15.116:0/2740977313 >> v2:172.21.15.116:3300/0 conn(0x7f721c067910 0x7f721c067cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f72219ce700 1 --2- 172.21.15.116:0/2740977313 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f721c064760 0x7f721c0bc060 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f72219ce700 1 -- 172.21.15.116:0/2740977313 >> 172.21.15.116:0/2740977313 conn(0x7f721c0c5680 msgr2=0x7f721c0c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:28.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f72219ce700 1 -- 172.21.15.116:0/2740977313 shutdown_connections 2020-11-20T12:17:28.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.944+0000 7f72219ce700 1 -- 172.21.15.116:0/2740977313 wait complete. 2020-11-20T12:17:28.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.948+0000 7f72219ce700 1 Processor -- start 2020-11-20T12:17:28.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.948+0000 7f72219ce700 1 -- start start 2020-11-20T12:17:28.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.948+0000 7f72219ce700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f721c064760 0x7f721c14b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.948+0000 7f72219ce700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f721c067910 0x7f721c0c06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.948+0000 7f72219ce700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f721c14f1f0 0x7f721c0c0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f72219ce700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0c44b0 con 0x7f721c064760 2020-11-20T12:17:28.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f72219ce700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0c4750 con 0x7f721c067910 2020-11-20T12:17:28.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f72219ce700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0c49f0 con 0x7f721c14f1f0 2020-11-20T12:17:28.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f721c064760 0x7f721c14b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f721c064760 0x7f721c14b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45390/0 (socket says 172.21.15.116:45390) 2020-11-20T12:17:28.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721affd700 1 -- 172.21.15.116:0/2448489844 learned_addr learned my addr 172.21.15.116:0/2448489844 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:28.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721a7fc700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f721c067910 0x7f721c0c06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721bfff700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f721c14f1f0 0x7f721c0c0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721affd700 1 -- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f721c14f1f0 msgr2=0x7f721c0c0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721affd700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f721c14f1f0 0x7f721c0c0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721affd700 1 -- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f721c067910 msgr2=0x7f721c0c06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:28.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721affd700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f721c067910 0x7f721c0c06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:28.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721affd700 1 -- 172.21.15.116:0/2448489844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c0c4c90 con 0x7f721c064760 2020-11-20T12:17:28.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721a7fc700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f721c067910 0x7f721c0c06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:28.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f721affd700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f721c064760 0x7f721c14b520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f720c00eba0 tx=0x7f720c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:28.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f7202ffd700 1 -- 172.21.15.116:0/2448489844 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c00ca60 con 0x7f721c064760 2020-11-20T12:17:28.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f7202ffd700 1 -- 172.21.15.116:0/2448489844 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c00cbc0 con 0x7f721c064760 2020-11-20T12:17:28.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f72219ce700 1 -- 172.21.15.116:0/2448489844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f721c0a18a0 con 0x7f721c064760 2020-11-20T12:17:28.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f7202ffd700 1 -- 172.21.15.116:0/2448489844 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c0123f0 con 0x7f721c064760 2020-11-20T12:17:28.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.952+0000 7f72219ce700 1 -- 172.21.15.116:0/2448489844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c0c39c0 con 0x7f721c064760 2020-11-20T12:17:28.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.956+0000 7f72219ce700 1 -- 172.21.15.116:0/2448489844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c05ca30 con 0x7f721c064760 2020-11-20T12:17:28.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.956+0000 7f7202ffd700 1 -- 172.21.15.116:0/2448489844 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f720c0127a0 con 0x7f721c064760 2020-11-20T12:17:28.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.956+0000 7f7202ffd700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71f80557c0 0x7f71f8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:28.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.956+0000 7f7202ffd700 1 -- 172.21.15.116:0/2448489844 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f720c06eb90 con 0x7f721c064760 2020-11-20T12:17:28.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.956+0000 7f721a7fc700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71f80557c0 0x7f71f8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:28.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.960+0000 7f721a7fc700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71f80557c0 0x7f71f8057c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f7210006e30 tx=0x7f7210015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:28.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:28.960+0000 7f7202ffd700 1 -- 172.21.15.116:0/2448489844 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f720c044a30 con 0x7f721c064760 2020-11-20T12:17:29.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.120+0000 7f72219ce700 1 -- 172.21.15.116:0/2448489844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f721c0c1c10 con 0x7f721c064760 2020-11-20T12:17:29.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.120+0000 7f7202ffd700 1 -- 172.21.15.116:0/2448489844 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v34) v1 ==== 100+0+10 (secure 0 0 0) 0x7f720c048050 con 0x7f721c064760 2020-11-20T12:17:29.122 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:29.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 -- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71f80557c0 msgr2=0x7f71f8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71f80557c0 0x7f71f8057c80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f7210006e30 tx=0x7f7210015000).stop 2020-11-20T12:17:29.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 -- 172.21.15.116:0/2448489844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f721c064760 msgr2=0x7f721c14b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f721c064760 0x7f721c14b520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f720c00eba0 tx=0x7f720c006df0).stop 2020-11-20T12:17:29.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 -- 172.21.15.116:0/2448489844 shutdown_connections 2020-11-20T12:17:29.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71f80557c0 0x7f71f8057c80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f721c14f1f0 0x7f721c0c0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f721c064760 0x7f721c14b520 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 --2- 172.21.15.116:0/2448489844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f721c067910 0x7f721c0c06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 -- 172.21.15.116:0/2448489844 >> 172.21.15.116:0/2448489844 conn(0x7f721c0c5680 msgr2=0x7f721c0c5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:29.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 -- 172.21.15.116:0/2448489844 shutdown_connections 2020-11-20T12:17:29.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.124+0000 7f72219ce700 1 -- 172.21.15.116:0/2448489844 wait complete. 2020-11-20T12:17:29.146 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:29.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f9a1d7700 1 Processor -- start 2020-11-20T12:17:29.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f9a1d7700 1 -- start start 2020-11-20T12:17:29.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f9a1d7700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3f9414edd0 0x7f3f9414f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f9a1d7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f941526d0 0x7f3f940c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f9a1d7700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f94155da0 con 0x7f3f940c2060 2020-11-20T12:17:29.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f9a1d7700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f94151ad0 con 0x7f3f9414edd0 2020-11-20T12:17:29.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f9a1d7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f94151cf0 con 0x7f3f941526d0 2020-11-20T12:17:29.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f991d5700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3f9414edd0 0x7f3f9414f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f991d5700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3f9414edd0 0x7f3f9414f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45396/0 (socket says 172.21.15.116:45396) 2020-11-20T12:17:29.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f991d5700 1 -- 172.21.15.116:0/2561671937 learned_addr learned my addr 172.21.15.116:0/2561671937 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:29.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f989d4700 1 --2- 172.21.15.116:0/2561671937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f941526d0 0x7f3f940c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f991d5700 1 -- 172.21.15.116:0/2561671937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f941526d0 msgr2=0x7f3f940c1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:29.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.320+0000 7f3f991d5700 1 --2- 172.21.15.116:0/2561671937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f941526d0 0x7f3f940c1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f991d5700 1 -- 172.21.15.116:0/2561671937 >> v1:172.21.15.173:6789/0 conn(0x7f3f940c2060 legacy=0x7f3f941550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f991d5700 1 -- 172.21.15.116:0/2561671937 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f94151f90 con 0x7f3f9414edd0 2020-11-20T12:17:29.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f991d5700 1 --2- 172.21.15.116:0/2561671937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f9414edd0 0x7f3f9414f1b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f3f90006fd0 tx=0x7f3f90006eb0).ready entity=mon.0 client_cookie=6e5be247eb5bfa42 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:29.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f83fff700 1 -- 172.21.15.116:0/2561671937 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f900098d0 con 0x7f3f9414edd0 2020-11-20T12:17:29.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f83fff700 1 -- 172.21.15.116:0/2561671937 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f90009a30 con 0x7f3f9414edd0 2020-11-20T12:17:29.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f83fff700 1 -- 172.21.15.116:0/2561671937 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90009db0 con 0x7f3f9414edd0 2020-11-20T12:17:29.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/2561671937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f9414edd0 msgr2=0x7f3f9414f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f9a1d7700 1 --2- 172.21.15.116:0/2561671937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f9414edd0 0x7f3f9414f1b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f3f90006fd0 tx=0x7f3f90006eb0).stop 2020-11-20T12:17:29.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/2561671937 shutdown_connections 2020-11-20T12:17:29.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f9a1d7700 1 --2- 172.21.15.116:0/2561671937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f9414edd0 0x7f3f9414f1b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f9a1d7700 1 --2- 172.21.15.116:0/2561671937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f941526d0 0x7f3f940c1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/2561671937 >> 172.21.15.116:0/2561671937 conn(0x7f3f94061280 msgr2=0x7f3f94061660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:29.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/2561671937 shutdown_connections 2020-11-20T12:17:29.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.324+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/2561671937 wait complete. 2020-11-20T12:17:29.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f9a1d7700 1 Processor -- start 2020-11-20T12:17:29.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f9a1d7700 1 -- start start 2020-11-20T12:17:29.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f9a1d7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f940c2060 0x7f3f940aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f9a1d7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f941526d0 0x7f3f940a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f9a1d7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f940a3fc0 0x7f3f94139780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f9a1d7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f940aa6d0 con 0x7f3f940c2060 2020-11-20T12:17:29.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f9a1d7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f940a94e0 con 0x7f3f941526d0 2020-11-20T12:17:29.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f9a1d7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f940a8a60 con 0x7f3f940a3fc0 2020-11-20T12:17:29.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f991d5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f940c2060 0x7f3f940aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f991d5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f940c2060 0x7f3f940aa1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45402/0 (socket says 172.21.15.116:45402) 2020-11-20T12:17:29.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f991d5700 1 -- 172.21.15.116:0/579857711 learned_addr learned my addr 172.21.15.116:0/579857711 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:29.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f989d4700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f941526d0 0x7f3f940a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.328+0000 7f3f999d6700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f940a3fc0 0x7f3f94139780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f991d5700 1 -- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f940a3fc0 msgr2=0x7f3f94139780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f991d5700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f940a3fc0 0x7f3f94139780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f991d5700 1 -- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f941526d0 msgr2=0x7f3f940a82b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f991d5700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f941526d0 0x7f3f940a82b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f991d5700 1 -- 172.21.15.116:0/579857711 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f940a8d00 con 0x7f3f940c2060 2020-11-20T12:17:29.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f991d5700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f940c2060 0x7f3f940aa1e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f3f9000b1b0 tx=0x7f3f90015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:29.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f817fa700 1 -- 172.21.15.116:0/579857711 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f90011d80 con 0x7f3f940c2060 2020-11-20T12:17:29.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f817fa700 1 -- 172.21.15.116:0/579857711 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9001e460 con 0x7f3f940c2060 2020-11-20T12:17:29.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/579857711 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f940a8fa0 con 0x7f3f940c2060 2020-11-20T12:17:29.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f817fa700 1 -- 172.21.15.116:0/579857711 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9001e800 con 0x7f3f940c2060 2020-11-20T12:17:29.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/579857711 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f940a8820 con 0x7f3f940c2060 2020-11-20T12:17:29.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.332+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/579857711 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f940a18f0 con 0x7f3f940c2060 2020-11-20T12:17:29.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.336+0000 7f3f817fa700 1 -- 172.21.15.116:0/579857711 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3f90025030 con 0x7f3f940c2060 2020-11-20T12:17:29.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.336+0000 7f3f817fa700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f6c0557f0 0x7f3f6c057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.336+0000 7f3f817fa700 1 -- 172.21.15.116:0/579857711 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3f9006fbe0 con 0x7f3f940c2060 2020-11-20T12:17:29.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.336+0000 7f3f989d4700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f6c0557f0 0x7f3f6c057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.340+0000 7f3f989d4700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f6c0557f0 0x7f3f6c057cb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f3f940a7df0 tx=0x7f3f84006f90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:29.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.344+0000 7f3f817fa700 1 -- 172.21.15.116:0/579857711 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3f940a18f0 con 0x7f3f940c2060 2020-11-20T12:17:29.462 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:17:29.468 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:17:29.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.556+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/579857711 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f940a8820 con 0x7f3f940c2060 2020-11-20T12:17:29.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f817fa700 1 -- 172.21.15.116:0/579857711 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+2851 (secure 0 0 0) 0x7f3f940a8820 con 0x7f3f940c2060 2020-11-20T12:17:29.559 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:29.560 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":14,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":4957,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3031892761","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3031892761},{"type":"v1","addr":"172.21.15.173:6836","nonce":3031892761}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10},{"gid":4960,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6837/1794965470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1794965470},{"type":"v1","addr":"172.21.15.173:6837","nonce":1794965470}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":9},{"gid":4995,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/4143559744","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":4143559744},{"type":"v1","addr":"172.21.15.116:6833","nonce":4143559744}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":10}],"filesystems":[{"mdsmap":{"epoch":14,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:16:57.901741+0000","modified":"2020-11-20T12:16:59.918401+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4998},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4998":{"gid":4998,"name":"a","rank":0,"incarnation":11,"state":"up:active","state_seq":4,"addr":"172.21.15.116:6835/3994085646","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3994085646},{"type":"v1","addr":"172.21.15.116:6835","nonce":3994085646}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2020-11-20T12:17:29.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f6c0557f0 msgr2=0x7f3f6c057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f9a1d7700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f6c0557f0 0x7f3f6c057cb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f3f940a7df0 tx=0x7f3f84006f90).stop 2020-11-20T12:17:29.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/579857711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f940c2060 msgr2=0x7f3f940aa1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f9a1d7700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f940c2060 0x7f3f940aa1e0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f3f9000b1b0 tx=0x7f3f90015000).stop 2020-11-20T12:17:29.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/579857711 shutdown_connections 2020-11-20T12:17:29.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f9a1d7700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f6c0557f0 0x7f3f6c057cb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f9a1d7700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f940a3fc0 0x7f3f94139780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f9a1d7700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f940c2060 0x7f3f940aa1e0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f9a1d7700 1 --2- 172.21.15.116:0/579857711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f941526d0 0x7f3f940a82b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/579857711 >> 172.21.15.116:0/579857711 conn(0x7f3f94061280 msgr2=0x7f3f940bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:29.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.560+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/579857711 shutdown_connections 2020-11-20T12:17:29.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.564+0000 7f3f9a1d7700 1 -- 172.21.15.116:0/579857711 wait complete. 2020-11-20T12:17:29.565 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 14 2020-11-20T12:17:29.585 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:29.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.760+0000 7f716c92f700 1 Processor -- start 2020-11-20T12:17:29.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.760+0000 7f716c92f700 1 -- start start 2020-11-20T12:17:29.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f716c92f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f71680bae10 0x7f71680c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f716c92f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f71680c25c0 0x7f71680b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f716c92f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71680c2da0 con 0x7f71680baa30 2020-11-20T12:17:29.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f716c92f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71680c3040 con 0x7f71680c25c0 2020-11-20T12:17:29.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f716c92f700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71680c32e0 con 0x7f71680bae10 2020-11-20T12:17:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f7166d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f71680c25c0 0x7f71680b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f716659c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f71680baa30 0x7f7168066510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54844/0 (socket says 172.21.15.116:54844) 2020-11-20T12:17:29.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f716659c700 1 -- 172.21.15.116:0/435493141 learned_addr learned my addr 172.21.15.116:0/435493141 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f715ffff700 1 --2- 172.21.15.116:0/435493141 >> v2:172.21.15.173:3300/0 conn(0x7f71680bae10 0x7f71680c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f7166d9d700 1 -- 172.21.15.116:0/435493141 >> v2:172.21.15.173:3300/0 conn(0x7f71680bae10 msgr2=0x7f71680c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f7166d9d700 1 --2- 172.21.15.116:0/435493141 >> v2:172.21.15.173:3300/0 conn(0x7f71680bae10 0x7f71680c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f7165d9b700 1 -- 172.21.15.116:0/435493141 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158868207 0 0) 0x7f71680c2da0 con 0x7f71680baa30 2020-11-20T12:17:29.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f7166d9d700 1 -- 172.21.15.116:0/435493141 >> v1:172.21.15.116:6789/0 conn(0x7f71680baa30 legacy=0x7f7168066510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f7166d9d700 1 -- 172.21.15.116:0/435493141 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71680c3580 con 0x7f71680c25c0 2020-11-20T12:17:29.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f7166d9d700 1 --2- 172.21.15.116:0/435493141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71680c25c0 0x7f71680b9130 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f7168068920 tx=0x7f71600091d0).ready entity=mon.0 client_cookie=24308e94ceffdcd8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:29.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f7165d9b700 1 -- 172.21.15.116:0/435493141 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716000ba70 con 0x7f71680c25c0 2020-11-20T12:17:29.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f7165d9b700 1 -- 172.21.15.116:0/435493141 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716000bbd0 con 0x7f71680c25c0 2020-11-20T12:17:29.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.764+0000 7f7165d9b700 1 -- 172.21.15.116:0/435493141 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160011400 con 0x7f71680c25c0 2020-11-20T12:17:29.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.768+0000 7f716c92f700 1 -- 172.21.15.116:0/435493141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71680c25c0 msgr2=0x7f71680b9130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.768+0000 7f716c92f700 1 --2- 172.21.15.116:0/435493141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71680c25c0 0x7f71680b9130 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f7168068920 tx=0x7f71600091d0).stop 2020-11-20T12:17:29.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.768+0000 7f716c92f700 1 -- 172.21.15.116:0/435493141 shutdown_connections 2020-11-20T12:17:29.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.768+0000 7f716c92f700 1 --2- 172.21.15.116:0/435493141 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71680c25c0 0x7f71680b9130 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.768+0000 7f716c92f700 1 --2- 172.21.15.116:0/435493141 >> v2:172.21.15.173:3300/0 conn(0x7f71680bae10 0x7f71680c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.768+0000 7f716c92f700 1 -- 172.21.15.116:0/435493141 >> 172.21.15.116:0/435493141 conn(0x7f7168060c80 msgr2=0x7f71680c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:29.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.768+0000 7f716c92f700 1 -- 172.21.15.116:0/435493141 shutdown_connections 2020-11-20T12:17:29.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.768+0000 7f716c92f700 1 -- 172.21.15.116:0/435493141 wait complete. 2020-11-20T12:17:29.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.768+0000 7f716c92f700 1 Processor -- start 2020-11-20T12:17:29.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.768+0000 7f716c92f700 1 -- start start 2020-11-20T12:17:29.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f716c92f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71680baa30 0x7f7168138bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f716c92f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71680bae10 0x7f71680a9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f716c92f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71680c25c0 0x7f71680a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f716c92f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716814f840 con 0x7f71680bae10 2020-11-20T12:17:29.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f716c92f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71680a8430 con 0x7f71680c25c0 2020-11-20T12:17:29.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f716c92f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71680a86d0 con 0x7f71680baa30 2020-11-20T12:17:29.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71680bae10 0x7f71680a9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71680bae10 0x7f71680a9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45418/0 (socket says 172.21.15.116:45418) 2020-11-20T12:17:29.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715ffff700 1 -- 172.21.15.116:0/235688587 learned_addr learned my addr 172.21.15.116:0/235688587 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:29.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f7166d9d700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71680c25c0 0x7f71680a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715ffff700 1 -- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71680baa30 msgr2=0x7f7168138bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f716659c700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71680baa30 0x7f7168138bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715ffff700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71680baa30 0x7f7168138bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715ffff700 1 -- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71680c25c0 msgr2=0x7f71680a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:29.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715ffff700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71680c25c0 0x7f71680a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:29.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715ffff700 1 -- 172.21.15.116:0/235688587 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71680a8970 con 0x7f71680bae10 2020-11-20T12:17:29.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f716659c700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71680baa30 0x7f7168138bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:29.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715ffff700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71680bae10 0x7f71680a9350 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f715000eba0 tx=0x7f7150006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:29.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715e7fc700 1 -- 172.21.15.116:0/235688587 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715000c9b0 con 0x7f71680bae10 2020-11-20T12:17:29.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715e7fc700 1 -- 172.21.15.116:0/235688587 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715000cb10 con 0x7f71680bae10 2020-11-20T12:17:29.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f715e7fc700 1 -- 172.21.15.116:0/235688587 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715000ce90 con 0x7f71680bae10 2020-11-20T12:17:29.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f716c92f700 1 -- 172.21.15.116:0/235688587 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71680a8c10 con 0x7f71680bae10 2020-11-20T12:17:29.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.772+0000 7f716c92f700 1 -- 172.21.15.116:0/235688587 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71680a9840 con 0x7f71680bae10 2020-11-20T12:17:29.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.776+0000 7f716c92f700 1 -- 172.21.15.116:0/235688587 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716805ca30 con 0x7f71680bae10 2020-11-20T12:17:29.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.776+0000 7f715e7fc700 1 -- 172.21.15.116:0/235688587 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7150012600 con 0x7f71680bae10 2020-11-20T12:17:29.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.776+0000 7f715e7fc700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71440557c0 0x7f7144057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:29.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.776+0000 7f715e7fc700 1 -- 172.21.15.116:0/235688587 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f715006eb40 con 0x7f71680bae10 2020-11-20T12:17:29.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.776+0000 7f716659c700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71440557c0 0x7f7144057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:29.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.780+0000 7f716659c700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71440557c0 0x7f7144057c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f7158001550 tx=0x7f7158009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:29.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.780+0000 7f715e7fc700 1 -- 172.21.15.116:0/235688587 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7150044870 con 0x7f71680bae10 2020-11-20T12:17:29.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.940+0000 7f716c92f700 1 -- 172.21.15.116:0/235688587 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71680616a0 con 0x7f71680bae10 2020-11-20T12:17:30.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f715e7fc700 1 -- 172.21.15.116:0/235688587 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f7150047e90 con 0x7f71680bae10 2020-11-20T12:17:30.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 -- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71440557c0 msgr2=0x7f7144057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:30.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71440557c0 0x7f7144057c80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f7158001550 tx=0x7f7158009040).stop 2020-11-20T12:17:30.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 -- 172.21.15.116:0/235688587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71680bae10 msgr2=0x7f71680a9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:30.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71680bae10 0x7f71680a9350 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f715000eba0 tx=0x7f7150006df0).stop 2020-11-20T12:17:30.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 -- 172.21.15.116:0/235688587 shutdown_connections 2020-11-20T12:17:30.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71440557c0 0x7f7144057c80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:30.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71680c25c0 0x7f71680a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:30.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71680baa30 0x7f7168138bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:30.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 --2- 172.21.15.116:0/235688587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71680bae10 0x7f71680a9350 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:30.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 -- 172.21.15.116:0/235688587 >> 172.21.15.116:0/235688587 conn(0x7f7168060c80 msgr2=0x7f7168153080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:30.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 -- 172.21.15.116:0/235688587 shutdown_connections 2020-11-20T12:17:30.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:29.944+0000 7f716c92f700 1 -- 172.21.15.116:0/235688587 wait complete. 2020-11-20T12:17:30.020 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:30.021 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":34,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:16:58.907407+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:16:55.781578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2020-11-20T12:16:56.900539+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:30.022 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-20T12:17:30.024 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-20T12:17:30.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.208+0000 7f2dc31fa700 1 Processor -- start 2020-11-20T12:17:30.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dc31fa700 1 -- start start 2020-11-20T12:17:30.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dc31fa700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2dbc0bae10 0x7f2dbc0c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:30.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dc31fa700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dbc0c25c0 0x7f2dbc0b9130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:30.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dc31fa700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dbc0c2da0 con 0x7f2dbc0baa30 2020-11-20T12:17:30.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dc31fa700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dbc0c3040 con 0x7f2dbc0bae10 2020-11-20T12:17:30.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dc31fa700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0c32e0 con 0x7f2dbc0c25c0 2020-11-20T12:17:30.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dbbfff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2dbc0bae10 0x7f2dbc0c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:30.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dbbfff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2dbc0bae10 0x7f2dbc0c20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:44478/0 (socket says 172.21.15.116:44478) 2020-11-20T12:17:30.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dbbfff700 1 -- 172.21.15.116:0/4055653191 learned_addr learned my addr 172.21.15.116:0/4055653191 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:30.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dc1797700 1 --2- 172.21.15.116:0/4055653191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dbc0c25c0 0x7f2dbc0b9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:30.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dbbfff700 1 -- 172.21.15.116:0/4055653191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dbc0c25c0 msgr2=0x7f2dbc0b9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:30.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dbbfff700 1 --2- 172.21.15.116:0/4055653191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dbc0c25c0 0x7f2dbc0b9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:30.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dbbfff700 1 -- 172.21.15.116:0/4055653191 >> v1:172.21.15.173:6789/0 conn(0x7f2dbc0baa30 legacy=0x7f2dbc066510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:30.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.212+0000 7f2dbbfff700 1 -- 172.21.15.116:0/4055653191 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc0c3580 con 0x7f2dbc0bae10 2020-11-20T12:17:30.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dbbfff700 1 --2- 172.21.15.116:0/4055653191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dbc0bae10 0x7f2dbc0c20d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f2da8006fd0 tx=0x7f2da8006eb0).ready entity=mon.1 client_cookie=c59365548978e314 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:30.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dbb7fe700 1 -- 172.21.15.116:0/4055653191 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8009810 con 0x7f2dbc0bae10 2020-11-20T12:17:30.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dbb7fe700 1 -- 172.21.15.116:0/4055653191 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da8009970 con 0x7f2dbc0bae10 2020-11-20T12:17:30.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dbb7fe700 1 -- 172.21.15.116:0/4055653191 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8009cf0 con 0x7f2dbc0bae10 2020-11-20T12:17:30.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dc31fa700 1 -- 172.21.15.116:0/4055653191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dbc0bae10 msgr2=0x7f2dbc0c20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:30.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dc31fa700 1 --2- 172.21.15.116:0/4055653191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dbc0bae10 0x7f2dbc0c20d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f2da8006fd0 tx=0x7f2da8006eb0).stop 2020-11-20T12:17:30.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dc31fa700 1 -- 172.21.15.116:0/4055653191 shutdown_connections 2020-11-20T12:17:30.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dc31fa700 1 --2- 172.21.15.116:0/4055653191 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dbc0c25c0 0x7f2dbc0b9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:30.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dc31fa700 1 --2- 172.21.15.116:0/4055653191 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dbc0bae10 0x7f2dbc0c20d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:30.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dc31fa700 1 -- 172.21.15.116:0/4055653191 >> 172.21.15.116:0/4055653191 conn(0x7f2dbc060c80 msgr2=0x7f2dbc0c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:30.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dc31fa700 1 -- 172.21.15.116:0/4055653191 shutdown_connections 2020-11-20T12:17:30.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.216+0000 7f2dc31fa700 1 -- 172.21.15.116:0/4055653191 wait complete. 2020-11-20T12:17:30.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.220+0000 7f2dc31fa700 1 Processor -- start 2020-11-20T12:17:30.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.220+0000 7f2dc31fa700 1 -- start start 2020-11-20T12:17:30.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.220+0000 7f2dc31fa700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dbc0baa30 0x7f2dbc1364d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:30.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.220+0000 7f2dc31fa700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dbc0bae10 0x7f2dbc0a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:30.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.220+0000 7f2dc31fa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dbc0c25c0 0x7f2dbc0a2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:30.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.220+0000 7f2dc31fa700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc0a4910 con 0x7f2dbc0bae10 2020-11-20T12:17:30.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.220+0000 7f2dc31fa700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc162420 con 0x7f2dbc0c25c0 2020-11-20T12:17:30.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.220+0000 7f2dc31fa700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dbc1626c0 con 0x7f2dbc0baa30 2020-11-20T12:17:30.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dbbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dbc0bae10 0x7f2dbc0a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:30.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dc0f96700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dbc0baa30 0x7f2dbc1364d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:30.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dc1797700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dbc0c25c0 0x7f2dbc0a2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:30.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dbbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dbc0bae10 0x7f2dbc0a1f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45432/0 (socket says 172.21.15.116:45432) 2020-11-20T12:17:30.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dbbfff700 1 -- 172.21.15.116:0/1638078625 learned_addr learned my addr 172.21.15.116:0/1638078625 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:30.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dbbfff700 1 -- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dbc0baa30 msgr2=0x7f2dbc1364d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:30.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dbbfff700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dbc0baa30 0x7f2dbc1364d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:30.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dbbfff700 1 -- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dbc0c25c0 msgr2=0x7f2dbc0a2460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:30.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dbbfff700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dbc0c25c0 0x7f2dbc0a2460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:30.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dbbfff700 1 -- 172.21.15.116:0/1638078625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dbc162960 con 0x7f2dbc0bae10 2020-11-20T12:17:30.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dbbfff700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dbc0bae10 0x7f2dbc0a1f70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f2da800beb0 tx=0x7f2da8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:30.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2db8ff9700 1 -- 172.21.15.116:0/1638078625 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da8011d50 con 0x7f2dbc0bae10 2020-11-20T12:17:30.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2db8ff9700 1 -- 172.21.15.116:0/1638078625 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2da801e460 con 0x7f2dbc0bae10 2020-11-20T12:17:30.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dc31fa700 1 -- 172.21.15.116:0/1638078625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dbc162c00 con 0x7f2dbc0bae10 2020-11-20T12:17:30.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2db8ff9700 1 -- 172.21.15.116:0/1638078625 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2da801e800 con 0x7f2dbc0bae10 2020-11-20T12:17:30.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dc31fa700 1 -- 172.21.15.116:0/1638078625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dbc0a3cb0 con 0x7f2dbc0bae10 2020-11-20T12:17:30.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.224+0000 7f2dc31fa700 1 -- 172.21.15.116:0/1638078625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dbc162d90 con 0x7f2dbc0bae10 2020-11-20T12:17:30.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.228+0000 7f2db8ff9700 1 -- 172.21.15.116:0/1638078625 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2da8025030 con 0x7f2dbc0bae10 2020-11-20T12:17:30.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.228+0000 7f2db8ff9700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d940557c0 0x7f2d94057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:30.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.228+0000 7f2db8ff9700 1 -- 172.21.15.116:0/1638078625 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2da806fd30 con 0x7f2dbc0bae10 2020-11-20T12:17:30.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.228+0000 7f2dc0f96700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d940557c0 0x7f2d94057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:30.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.232+0000 7f2dc0f96700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d940557c0 0x7f2d94057c80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f2db0003610 tx=0x7f2db0000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:30.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.236+0000 7f2db8ff9700 1 -- 172.21.15.116:0/1638078625 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2da8045c00 con 0x7f2dbc0bae10 2020-11-20T12:17:30.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:30.448+0000 7f2dc31fa700 1 -- 172.21.15.116:0/1638078625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2dbc05ca30 con 0x7f2dbc0bae10 2020-11-20T12:17:31.321 INFO:tasks.ceph.mon.a.smithi116.stderr:2020-11-20T12:17:31.320+0000 7fc111d1b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-20T12:17:31.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.336+0000 7f2db8ff9700 1 -- 172.21.15.116:0/1638078625 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v15) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2da8023030 con 0x7f2dbc0bae10 2020-11-20T12:17:31.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.336+0000 7f2dc31fa700 1 -- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d940557c0 msgr2=0x7f2d94057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:31.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.336+0000 7f2dc31fa700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d940557c0 0x7f2d94057c80 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f2db0003610 tx=0x7f2db0000b80).stop 2020-11-20T12:17:31.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.336+0000 7f2dc31fa700 1 -- 172.21.15.116:0/1638078625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dbc0bae10 msgr2=0x7f2dbc0a1f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:31.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.336+0000 7f2dc31fa700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dbc0bae10 0x7f2dbc0a1f70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f2da800beb0 tx=0x7f2da8015000).stop 2020-11-20T12:17:31.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.340+0000 7f2dc31fa700 1 -- 172.21.15.116:0/1638078625 shutdown_connections 2020-11-20T12:17:31.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.340+0000 7f2dc31fa700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d940557c0 0x7f2d94057c80 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:31.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.340+0000 7f2dc31fa700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2dbc0c25c0 0x7f2dbc0a2460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:31.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.340+0000 7f2dc31fa700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2dbc0baa30 0x7f2dbc1364d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:31.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.340+0000 7f2dc31fa700 1 --2- 172.21.15.116:0/1638078625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2dbc0bae10 0x7f2dbc0a1f70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:31.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.340+0000 7f2dc31fa700 1 -- 172.21.15.116:0/1638078625 >> 172.21.15.116:0/1638078625 conn(0x7f2dbc060c80 msgr2=0x7f2dbc1538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:31.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.340+0000 7f2dc31fa700 1 -- 172.21.15.116:0/1638078625 shutdown_connections 2020-11-20T12:17:31.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.340+0000 7f2dc31fa700 1 -- 172.21.15.116:0/1638078625 wait complete. 2020-11-20T12:17:31.342 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-20T12:17:31.350 INFO:tasks.ceph.mds.a.smithi116.stderr: -3786> 2020-11-20T12:17:00.400+0000 7f758246c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:17:31.363 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-20T12:17:31.395 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:17:31.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.532+0000 7f8114b8e700 1 Processor -- start 2020-11-20T12:17:31.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.532+0000 7f8114b8e700 1 -- start start 2020-11-20T12:17:31.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.532+0000 7f8114b8e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f811014e040 0x7f81100c2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:31.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.532+0000 7f8114b8e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81100a1010 con 0x7f81100a56e0 2020-11-20T12:17:31.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.532+0000 7f8114b8e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81100a5ce0 con 0x7f811014e420 2020-11-20T12:17:31.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.532+0000 7f8114b8e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81100a5f80 con 0x7f811014e040 2020-11-20T12:17:31.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.532+0000 7f810ffff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f81100a56e0 0x7f8110157e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54884/0 (socket says 172.21.15.116:54884) 2020-11-20T12:17:31.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.532+0000 7f810ffff700 1 -- 172.21.15.116:0/2371753854 learned_addr learned my addr 172.21.15.116:0/2371753854 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:31.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.532+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243494816 0 0) 0x7f81100a1010 con 0x7f81100a56e0 2020-11-20T12:17:31.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.532+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80fc003240 con 0x7f81100a56e0 2020-11-20T12:17:31.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.532+0000 7f810f7fe700 1 --2- 172.21.15.116:0/2371753854 >> v2:172.21.15.173:3300/0 conn(0x7f811014e040 0x7f81100c2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:31.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1688332138 0 0) 0x7f80fc003240 con 0x7f81100a56e0 2020-11-20T12:17:31.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 >> v2:172.21.15.173:3300/0 conn(0x7f811014e040 msgr2=0x7f81100c2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:31.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 --2- 172.21.15.116:0/2371753854 >> v2:172.21.15.173:3300/0 conn(0x7f811014e040 0x7f81100c2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:31.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 >> v1:172.21.15.173:6789/0 conn(0x7f811014e420 legacy=0x7f81100a4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:31.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81100a1270 con 0x7f81100a56e0 2020-11-20T12:17:31.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f8108001860 con 0x7f81100a56e0 2020-11-20T12:17:31.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 >> v1:172.21.15.116:6789/0 conn(0x7f81100a56e0 legacy=0x7f8110157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:31.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 --2- 172.21.15.116:0/2371753854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80fc003730 0x7f80fc003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:31.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 --2- 172.21.15.116:0/2371753854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80fc0044e0 0x7f80fc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:31.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 --2- 172.21.15.116:0/2371753854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc004e50 0x7f80fc00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:31.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81100a5f80 con 0x7f80fc004e50 2020-11-20T12:17:31.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc004260 con 0x7f80fc0044e0 2020-11-20T12:17:31.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc00bc20 con 0x7f80fc003730 2020-11-20T12:17:31.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810ffff700 1 --2- 172.21.15.116:0/2371753854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc004e50 0x7f80fc00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:31.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810f7fe700 1 --2- 172.21.15.116:0/2371753854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80fc003730 0x7f80fc003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:31.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810effd700 1 --2- 172.21.15.116:0/2371753854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80fc0044e0 0x7f80fc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:31.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810ffff700 1 -- 172.21.15.116:0/2371753854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80fc003730 msgr2=0x7f80fc003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:31.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810ffff700 1 --2- 172.21.15.116:0/2371753854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f80fc003730 0x7f80fc003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:31.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810ffff700 1 -- 172.21.15.116:0/2371753854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80fc0044e0 msgr2=0x7f80fc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:31.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810ffff700 1 --2- 172.21.15.116:0/2371753854 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80fc0044e0 0x7f80fc004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:31.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810ffff700 1 -- 172.21.15.116:0/2371753854 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80fc003240 con 0x7f80fc004e50 2020-11-20T12:17:31.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810ffff700 1 -- 172.21.15.116:0/2371753854 reap_dead start 2020-11-20T12:17:31.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810ffff700 1 --2- 172.21.15.116:0/2371753854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc004e50 0x7f80fc00b730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f810800c840 tx=0x7f81100a5e70).ready entity=mon.0 client_cookie=b5085d7b1c43bd70 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:31.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108007b50 con 0x7f80fc004e50 2020-11-20T12:17:31.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8108005b80 con 0x7f80fc004e50 2020-11-20T12:17:31.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f810e7fc700 1 -- 172.21.15.116:0/2371753854 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108007110 con 0x7f80fc004e50 2020-11-20T12:17:31.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f8114b8e700 1 -- 172.21.15.116:0/2371753854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc004e50 msgr2=0x7f80fc00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:31.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f8114b8e700 1 --2- 172.21.15.116:0/2371753854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc004e50 0x7f80fc00b730 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f810800c840 tx=0x7f81100a5e70).stop 2020-11-20T12:17:31.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f8114b8e700 1 -- 172.21.15.116:0/2371753854 shutdown_connections 2020-11-20T12:17:31.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f8114b8e700 1 --2- 172.21.15.116:0/2371753854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f80fc004e50 0x7f80fc00b730 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:31.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f8114b8e700 1 -- 172.21.15.116:0/2371753854 >> 172.21.15.116:0/2371753854 conn(0x7f8110060c30 msgr2=0x7f81100c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:31.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.536+0000 7f8114b8e700 1 -- 172.21.15.116:0/2371753854 shutdown_connections 2020-11-20T12:17:31.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.540+0000 7f8114b8e700 1 -- 172.21.15.116:0/2371753854 wait complete. 2020-11-20T12:17:31.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.540+0000 7f8114b8e700 1 Processor -- start 2020-11-20T12:17:31.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.540+0000 7f8114b8e700 1 -- start start 2020-11-20T12:17:31.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f8114b8e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80fc004e50 0x7f811013e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:31.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f8114b8e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81100c31a0 0x7f811013d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:31.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f8114b8e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81100c3620 0x7f811013dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:31.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f8114b8e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811008ebb0 con 0x7f81100c3620 2020-11-20T12:17:31.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f8114b8e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f811008ee50 con 0x7f80fc004e50 2020-11-20T12:17:31.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f8114b8e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811008f0f0 con 0x7f81100c31a0 2020-11-20T12:17:31.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f810ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81100c3620 0x7f811013dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:31.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f810ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81100c3620 0x7f811013dba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45466/0 (socket says 172.21.15.116:45466) 2020-11-20T12:17:31.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f810ffff700 1 -- 172.21.15.116:0/945982201 learned_addr learned my addr 172.21.15.116:0/945982201 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:31.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f810f7fe700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80fc004e50 0x7f811013e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:31.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f810effd700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81100c31a0 0x7f811013d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:31.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f810ffff700 1 -- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81100c31a0 msgr2=0x7f811013d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:31.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f810ffff700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81100c31a0 0x7f811013d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:31.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f810ffff700 1 -- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80fc004e50 msgr2=0x7f811013e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:31.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f810ffff700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80fc004e50 0x7f811013e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:31.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f810ffff700 1 -- 172.21.15.116:0/945982201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811008f390 con 0x7f81100c3620 2020-11-20T12:17:31.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f810ffff700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81100c3620 0x7f811013dba0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f8110061150 tx=0x7f8108014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:31.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f80ef7fe700 1 -- 172.21.15.116:0/945982201 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810801b030 con 0x7f81100c3620 2020-11-20T12:17:31.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f80ef7fe700 1 -- 172.21.15.116:0/945982201 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8108019070 con 0x7f81100c3620 2020-11-20T12:17:31.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f8114b8e700 1 -- 172.21.15.116:0/945982201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f811008f630 con 0x7f81100c3620 2020-11-20T12:17:31.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f80ef7fe700 1 -- 172.21.15.116:0/945982201 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8108013850 con 0x7f81100c3620 2020-11-20T12:17:31.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.544+0000 7f8114b8e700 1 -- 172.21.15.116:0/945982201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81100a9ce0 con 0x7f81100c3620 2020-11-20T12:17:31.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.548+0000 7f8114b8e700 1 -- 172.21.15.116:0/945982201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f811008f7c0 con 0x7f81100c3620 2020-11-20T12:17:31.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.548+0000 7f80ef7fe700 1 -- 172.21.15.116:0/945982201 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8108010030 con 0x7f81100c3620 2020-11-20T12:17:31.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.548+0000 7f80ef7fe700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f80e80557c0 0x7f80e8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:31.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.548+0000 7f80ef7fe700 1 -- 172.21.15.116:0/945982201 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f810806fe70 con 0x7f81100c3620 2020-11-20T12:17:31.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.548+0000 7f810f7fe700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f80e80557c0 0x7f80e8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:31.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.552+0000 7f810f7fe700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f80e80557c0 0x7f80e8057c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f8110062b50 tx=0x7f8100006f90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:31.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.552+0000 7f80ef7fe700 1 -- 172.21.15.116:0/945982201 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8108045cc0 con 0x7f81100c3620 2020-11-20T12:17:31.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:31.760+0000 7f8114b8e700 1 -- 172.21.15.116:0/945982201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8110044f60 con 0x7f81100c3620 2020-11-20T12:17:32.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.460+0000 7f80ef7fe700 1 -- 172.21.15.116:0/945982201 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v16) v1 ==== 104+0+0 (secure 0 0 0) 0x7f81080492e0 con 0x7f81100c3620 2020-11-20T12:17:32.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.464+0000 7f8114b8e700 1 -- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f80e80557c0 msgr2=0x7f80e8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:32.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.464+0000 7f8114b8e700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f80e80557c0 0x7f80e8057c80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f8110062b50 tx=0x7f8100006f90).stop 2020-11-20T12:17:32.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.464+0000 7f8114b8e700 1 -- 172.21.15.116:0/945982201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81100c3620 msgr2=0x7f811013dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:32.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.464+0000 7f8114b8e700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81100c3620 0x7f811013dba0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f8110061150 tx=0x7f8108014000).stop 2020-11-20T12:17:32.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.464+0000 7f8114b8e700 1 -- 172.21.15.116:0/945982201 shutdown_connections 2020-11-20T12:17:32.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.464+0000 7f8114b8e700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f80e80557c0 0x7f80e8057c80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:32.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.464+0000 7f8114b8e700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81100c3620 0x7f811013dba0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:32.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.464+0000 7f8114b8e700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f80fc004e50 0x7f811013e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:32.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.464+0000 7f8114b8e700 1 --2- 172.21.15.116:0/945982201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81100c31a0 0x7f811013d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:32.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.464+0000 7f8114b8e700 1 -- 172.21.15.116:0/945982201 >> 172.21.15.116:0/945982201 conn(0x7f8110060c30 msgr2=0x7f8110063350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:32.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.468+0000 7f8114b8e700 1 -- 172.21.15.116:0/945982201 shutdown_connections 2020-11-20T12:17:32.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.468+0000 7f8114b8e700 1 -- 172.21.15.116:0/945982201 wait complete. 2020-11-20T12:17:32.491 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-20T12:17:32.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.728+0000 7fe5a26c5700 1 Processor -- start 2020-11-20T12:17:32.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.728+0000 7fe5a26c5700 1 -- start start 2020-11-20T12:17:32.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.728+0000 7fe5a26c5700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe59c0be350 0x7fe59c0c4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:32.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a26c5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe59c062620 0x7fe59c0c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:32.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a26c5700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe59c0c4b70 con 0x7fe59c0c52f0 2020-11-20T12:17:32.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a26c5700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe59c151b00 con 0x7fe59c0be350 2020-11-20T12:17:32.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a26c5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c0c40f0 con 0x7fe59c062620 2020-11-20T12:17:32.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a0ec2700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe59c0c52f0 0x7fe59c0c33e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54906/0 (socket says 172.21.15.116:54906) 2020-11-20T12:17:32.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a0ec2700 1 -- 172.21.15.116:0/707041520 learned_addr learned my addr 172.21.15.116:0/707041520 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:32.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a1ec4700 1 --2- 172.21.15.116:0/707041520 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe59c062620 0x7fe59c0c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:32.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a16c3700 1 --2- 172.21.15.116:0/707041520 >> v2:172.21.15.173:3300/0 conn(0x7fe59c0be350 0x7fe59c0c4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:32.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe58bfff700 1 -- 172.21.15.116:0/707041520 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 33170704 0 0) 0x7fe59c0c4b70 con 0x7fe59c0c52f0 2020-11-20T12:17:32.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe58bfff700 1 -- 172.21.15.116:0/707041520 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe584003240 con 0x7fe59c0c52f0 2020-11-20T12:17:32.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a1ec4700 1 -- 172.21.15.116:0/707041520 >> v2:172.21.15.173:3300/0 conn(0x7fe59c0be350 msgr2=0x7fe59c0c4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:32.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a1ec4700 1 --2- 172.21.15.116:0/707041520 >> v2:172.21.15.173:3300/0 conn(0x7fe59c0be350 0x7fe59c0c4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:32.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a1ec4700 1 -- 172.21.15.116:0/707041520 >> v1:172.21.15.116:6789/0 conn(0x7fe59c0c52f0 legacy=0x7fe59c0c33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:32.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a1ec4700 1 -- 172.21.15.116:0/707041520 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c0c4390 con 0x7fe59c062620 2020-11-20T12:17:32.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a16c3700 1 --2- 172.21.15.116:0/707041520 >> v2:172.21.15.173:3300/0 conn(0x7fe59c0be350 0x7fe59c0c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:32.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe5a1ec4700 1 --2- 172.21.15.116:0/707041520 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe59c062620 0x7fe59c0c3af0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fe594009c90 tx=0x7fe5940091d0).ready entity=mon.2 client_cookie=1f565ce3c07fe7b4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:32.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe58bfff700 1 -- 172.21.15.116:0/707041520 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59400b9c0 con 0x7fe59c062620 2020-11-20T12:17:32.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe58bfff700 1 -- 172.21.15.116:0/707041520 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59400bb20 con 0x7fe59c062620 2020-11-20T12:17:32.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.732+0000 7fe58bfff700 1 -- 172.21.15.116:0/707041520 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59400bea0 con 0x7fe59c062620 2020-11-20T12:17:32.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.736+0000 7fe5a26c5700 1 -- 172.21.15.116:0/707041520 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe59c062620 msgr2=0x7fe59c0c3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:32.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.736+0000 7fe5a26c5700 1 --2- 172.21.15.116:0/707041520 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe59c062620 0x7fe59c0c3af0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7fe594009c90 tx=0x7fe5940091d0).stop 2020-11-20T12:17:32.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.736+0000 7fe5a26c5700 1 -- 172.21.15.116:0/707041520 shutdown_connections 2020-11-20T12:17:32.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.736+0000 7fe5a26c5700 1 --2- 172.21.15.116:0/707041520 >> v2:172.21.15.173:3300/0 conn(0x7fe59c0be350 0x7fe59c0c4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:32.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.736+0000 7fe5a26c5700 1 --2- 172.21.15.116:0/707041520 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe59c062620 0x7fe59c0c3af0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:32.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.736+0000 7fe5a26c5700 1 -- 172.21.15.116:0/707041520 >> 172.21.15.116:0/707041520 conn(0x7fe59c091900 msgr2=0x7fe59c07ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:32.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.736+0000 7fe5a26c5700 1 -- 172.21.15.116:0/707041520 shutdown_connections 2020-11-20T12:17:32.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.736+0000 7fe5a26c5700 1 -- 172.21.15.116:0/707041520 wait complete. 2020-11-20T12:17:32.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.736+0000 7fe5a26c5700 1 Processor -- start 2020-11-20T12:17:32.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.736+0000 7fe5a26c5700 1 -- start start 2020-11-20T12:17:32.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a26c5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe59c062620 0x7fe59c161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:32.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a26c5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c0be350 0x7fe59c1609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:32.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a26c5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe59c0c52f0 0x7fe59c153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:32.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a26c5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c0a8230 con 0x7fe59c0be350 2020-11-20T12:17:32.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a26c5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c0a84d0 con 0x7fe59c062620 2020-11-20T12:17:32.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a26c5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59c0a8770 con 0x7fe59c0c52f0 2020-11-20T12:17:32.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a0ec2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c0be350 0x7fe59c1609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:32.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a16c3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe59c062620 0x7fe59c161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:32.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a0ec2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c0be350 0x7fe59c1609d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45478/0 (socket says 172.21.15.116:45478) 2020-11-20T12:17:32.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a0ec2700 1 -- 172.21.15.116:0/4227993316 learned_addr learned my addr 172.21.15.116:0/4227993316 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:32.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a1ec4700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe59c0c52f0 0x7fe59c153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:32.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a0ec2700 1 -- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe59c0c52f0 msgr2=0x7fe59c153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:32.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a0ec2700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe59c0c52f0 0x7fe59c153970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:32.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a0ec2700 1 -- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe59c062620 msgr2=0x7fe59c161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:32.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a0ec2700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe59c062620 0x7fe59c161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:32.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a0ec2700 1 -- 172.21.15.116:0/4227993316 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe59c0a8a10 con 0x7fe59c0be350 2020-11-20T12:17:32.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5a0ec2700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c0be350 0x7fe59c1609d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fe58c00c710 tx=0x7fe58c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:32.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.740+0000 7fe5897fa700 1 -- 172.21.15.116:0/4227993316 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c0072e0 con 0x7fe59c0be350 2020-11-20T12:17:32.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.744+0000 7fe5a26c5700 1 -- 172.21.15.116:0/4227993316 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe59c0a7f20 con 0x7fe59c0be350 2020-11-20T12:17:32.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.744+0000 7fe5897fa700 1 -- 172.21.15.116:0/4227993316 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe58c007440 con 0x7fe59c0be350 2020-11-20T12:17:32.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.744+0000 7fe5897fa700 1 -- 172.21.15.116:0/4227993316 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe58c005800 con 0x7fe59c0be350 2020-11-20T12:17:32.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.744+0000 7fe5a26c5700 1 -- 172.21.15.116:0/4227993316 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe59c0a9bd0 con 0x7fe59c0be350 2020-11-20T12:17:32.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.744+0000 7fe5a26c5700 1 -- 172.21.15.116:0/4227993316 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe59c044f60 con 0x7fe59c0be350 2020-11-20T12:17:32.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.748+0000 7fe5897fa700 1 -- 172.21.15.116:0/4227993316 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe58c010070 con 0x7fe59c0be350 2020-11-20T12:17:32.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.748+0000 7fe5897fa700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5740557c0 0x7fe574057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:32.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.748+0000 7fe5897fa700 1 -- 172.21.15.116:0/4227993316 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe58c06dd40 con 0x7fe59c0be350 2020-11-20T12:17:32.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.748+0000 7fe5a16c3700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5740557c0 0x7fe574057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:32.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.748+0000 7fe5a16c3700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5740557c0 0x7fe574057c80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fe598006fd0 tx=0x7fe598008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:32.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.756+0000 7fe5897fa700 1 -- 172.21.15.116:0/4227993316 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe58c043b90 con 0x7fe59c0be350 2020-11-20T12:17:32.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:32.912+0000 7fe5a26c5700 1 -- 172.21.15.116:0/4227993316 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe59c082690 con 0x7fe59c0be350 2020-11-20T12:17:33.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.472+0000 7fe5897fa700 1 -- 172.21.15.116:0/4227993316 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v36) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe58c0471b0 con 0x7fe59c0be350 2020-11-20T12:17:33.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 -- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5740557c0 msgr2=0x7fe574057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:33.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5740557c0 0x7fe574057c80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fe598006fd0 tx=0x7fe598008040).stop 2020-11-20T12:17:33.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 -- 172.21.15.116:0/4227993316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c0be350 msgr2=0x7fe59c1609d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:33.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c0be350 0x7fe59c1609d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7fe58c00c710 tx=0x7fe58c007ad0).stop 2020-11-20T12:17:33.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 -- 172.21.15.116:0/4227993316 shutdown_connections 2020-11-20T12:17:33.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5740557c0 0x7fe574057c80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:33.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe59c0c52f0 0x7fe59c153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:33.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe59c062620 0x7fe59c161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:33.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 --2- 172.21.15.116:0/4227993316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe59c0be350 0x7fe59c1609d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:33.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 -- 172.21.15.116:0/4227993316 >> 172.21.15.116:0/4227993316 conn(0x7fe59c091900 msgr2=0x7fe59c0a7610 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:33.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 -- 172.21.15.116:0/4227993316 shutdown_connections 2020-11-20T12:17:33.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.476+0000 7fe5a26c5700 1 -- 172.21.15.116:0/4227993316 wait complete. 2020-11-20T12:17:33.480 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2020-11-20T12:17:33.500 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-20T12:17:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.680+0000 7f81558e0700 1 Processor -- start 2020-11-20T12:17:33.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.684+0000 7f81558e0700 1 -- start start 2020-11-20T12:17:33.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.684+0000 7f81558e0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81500c5020 0x7f8150151d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:33.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.684+0000 7f81558e0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f815014b870 0x7f815014bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:33.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.684+0000 7f81558e0700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81501523b0 con 0x7f81500c5400 2020-11-20T12:17:33.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.684+0000 7f81558e0700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8150152650 con 0x7f81500c5020 2020-11-20T12:17:33.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.684+0000 7f81558e0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81501528f0 con 0x7f815014b870 2020-11-20T12:17:33.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81500c5020 0x7f8150151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:33.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81500c5020 0x7f8150151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45486/0 (socket says 172.21.15.116:45486) 2020-11-20T12:17:33.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814effd700 1 -- 172.21.15.116:0/2259838041 learned_addr learned my addr 172.21.15.116:0/2259838041 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:33.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814f7fe700 1 --2- 172.21.15.116:0/2259838041 >> v2:172.21.15.173:3300/0 conn(0x7f815014b870 0x7f815014bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:33.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814f7fe700 1 -- 172.21.15.116:0/2259838041 >> v2:172.21.15.116:3300/0 conn(0x7f81500c5020 msgr2=0x7f8150151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:33.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814f7fe700 1 --2- 172.21.15.116:0/2259838041 >> v2:172.21.15.116:3300/0 conn(0x7f81500c5020 0x7f8150151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:33.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814f7fe700 1 -- 172.21.15.116:0/2259838041 >> v1:172.21.15.173:6789/0 conn(0x7f81500c5400 legacy=0x7f8150155d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:33.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814f7fe700 1 -- 172.21.15.116:0/2259838041 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8150152b90 con 0x7f815014b870 2020-11-20T12:17:33.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814f7fe700 1 --2- 172.21.15.116:0/2259838041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f815014b870 0x7f815014bd20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f81500c4010 tx=0x7f81480091d0).ready entity=mon.1 client_cookie=c448b73272c725ed server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:33.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814dffb700 1 -- 172.21.15.116:0/2259838041 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814800ba70 con 0x7f815014b870 2020-11-20T12:17:33.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814dffb700 1 -- 172.21.15.116:0/2259838041 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814800bbd0 con 0x7f815014b870 2020-11-20T12:17:33.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f814dffb700 1 -- 172.21.15.116:0/2259838041 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148011400 con 0x7f815014b870 2020-11-20T12:17:33.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f81558e0700 1 -- 172.21.15.116:0/2259838041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f815014b870 msgr2=0x7f815014bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:33.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f81558e0700 1 --2- 172.21.15.116:0/2259838041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f815014b870 0x7f815014bd20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f81500c4010 tx=0x7f81480091d0).stop 2020-11-20T12:17:33.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f81558e0700 1 -- 172.21.15.116:0/2259838041 shutdown_connections 2020-11-20T12:17:33.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f81558e0700 1 --2- 172.21.15.116:0/2259838041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f815014b870 0x7f815014bd20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:33.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f81558e0700 1 --2- 172.21.15.116:0/2259838041 >> v2:172.21.15.116:3300/0 conn(0x7f81500c5020 0x7f8150151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:33.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f81558e0700 1 -- 172.21.15.116:0/2259838041 >> 172.21.15.116:0/2259838041 conn(0x7f81500678a0 msgr2=0x7f8150067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:33.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f81558e0700 1 -- 172.21.15.116:0/2259838041 shutdown_connections 2020-11-20T12:17:33.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.688+0000 7f81558e0700 1 -- 172.21.15.116:0/2259838041 wait complete. 2020-11-20T12:17:33.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.692+0000 7f81558e0700 1 Processor -- start 2020-11-20T12:17:33.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.692+0000 7f81558e0700 1 -- start start 2020-11-20T12:17:33.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f81558e0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81500c5020 0x7f81500bf670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:33.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f81558e0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81500c5400 0x7f815014b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:33.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f81558e0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f815014b870 0x7f81500add10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:33.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f81558e0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81501611e0 con 0x7f815014b870 2020-11-20T12:17:33.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f81558e0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81500bfb60 con 0x7f81500c5020 2020-11-20T12:17:33.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f81558e0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8150164d10 con 0x7f81500c5400 2020-11-20T12:17:33.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f815014b870 0x7f81500add10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:33.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f815014b870 0x7f81500add10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45494/0 (socket says 172.21.15.116:45494) 2020-11-20T12:17:33.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814f7fe700 1 -- 172.21.15.116:0/1073576902 learned_addr learned my addr 172.21.15.116:0/1073576902 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:33.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814effd700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81500c5020 0x7f81500bf670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:33.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814e7fc700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81500c5400 0x7f815014b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:33.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814f7fe700 1 -- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81500c5400 msgr2=0x7f815014b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:33.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814f7fe700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81500c5400 0x7f815014b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:33.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814f7fe700 1 -- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81500c5020 msgr2=0x7f81500bf670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:33.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814f7fe700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81500c5020 0x7f81500bf670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:33.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814f7fe700 1 -- 172.21.15.116:0/1073576902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8150164fb0 con 0x7f815014b870 2020-11-20T12:17:33.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814effd700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81500c5020 0x7f81500bf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:33.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.696+0000 7f814f7fe700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f815014b870 0x7f81500add10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f8148009210 tx=0x7f8148016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:33.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.700+0000 7f8136ffd700 1 -- 172.21.15.116:0/1073576902 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814801c6c0 con 0x7f815014b870 2020-11-20T12:17:33.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.700+0000 7f8136ffd700 1 -- 172.21.15.116:0/1073576902 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f814801c820 con 0x7f815014b870 2020-11-20T12:17:33.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.700+0000 7f81558e0700 1 -- 172.21.15.116:0/1073576902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8150165250 con 0x7f815014b870 2020-11-20T12:17:33.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.700+0000 7f8136ffd700 1 -- 172.21.15.116:0/1073576902 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f814801cbc0 con 0x7f815014b870 2020-11-20T12:17:33.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.700+0000 7f81558e0700 1 -- 172.21.15.116:0/1073576902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815014b5e0 con 0x7f815014b870 2020-11-20T12:17:33.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.700+0000 7f81558e0700 1 -- 172.21.15.116:0/1073576902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8150161380 con 0x7f815014b870 2020-11-20T12:17:33.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.700+0000 7f8136ffd700 1 -- 172.21.15.116:0/1073576902 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8148017070 con 0x7f815014b870 2020-11-20T12:17:33.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.700+0000 7f8136ffd700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f812c0557f0 0x7f812c057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:33.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.700+0000 7f8136ffd700 1 -- 172.21.15.116:0/1073576902 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f8148070c40 con 0x7f815014b870 2020-11-20T12:17:33.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.704+0000 7f814effd700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f812c0557f0 0x7f812c057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.704+0000 7f814effd700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f812c0557f0 0x7f812c057cb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f8140006a10 tx=0x7f8140008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.708+0000 7f8136ffd700 1 -- 172.21.15.116:0/1073576902 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8150161380 con 0x7f815014b870 2020-11-20T12:17:33.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:33.864+0000 7f81558e0700 1 -- 172.21.15.116:0/1073576902 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8150161380 con 0x7f815014b870 2020-11-20T12:17:34.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.480+0000 7f8136ffd700 1 -- 172.21.15.116:0/1073576902 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v37) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8150161380 con 0x7f815014b870 2020-11-20T12:17:34.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 -- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f812c0557f0 msgr2=0x7f812c057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:34.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f812c0557f0 0x7f812c057cb0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f8140006a10 tx=0x7f8140008040).stop 2020-11-20T12:17:34.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 -- 172.21.15.116:0/1073576902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f815014b870 msgr2=0x7f81500add10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:34.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f815014b870 0x7f81500add10 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f8148009210 tx=0x7f8148016000).stop 2020-11-20T12:17:34.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 -- 172.21.15.116:0/1073576902 shutdown_connections 2020-11-20T12:17:34.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f812c0557f0 0x7f812c057cb0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:34.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f815014b870 0x7f81500add10 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:34.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81500c5020 0x7f81500bf670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:34.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 --2- 172.21.15.116:0/1073576902 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81500c5400 0x7f815014b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:34.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 -- 172.21.15.116:0/1073576902 >> 172.21.15.116:0/1073576902 conn(0x7f81500678a0 msgr2=0x7f81500c73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:34.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 -- 172.21.15.116:0/1073576902 shutdown_connections 2020-11-20T12:17:34.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.484+0000 7f81558e0700 1 -- 172.21.15.116:0/1073576902 wait complete. 2020-11-20T12:17:34.488 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2020-11-20T12:17:34.510 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T12:17:34.511 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-20T12:17:34.511 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-20T12:17:34.511 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:17:34.512 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-20T12:17:34.512 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-20T12:17:34.512 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-20T12:17:34.512 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:17:34.513 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T12:17:34.513 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-20T12:17:34.513 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-20T12:17:34.514 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:17:34.514 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-20T12:17:34.514 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-20T12:17:34.514 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-20T12:17:34.515 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:17:34.538 INFO:tasks.ceph.mds.c:Stopped 2020-11-20T12:17:34.538 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-20T12:17:34.539 INFO:tasks.ceph.mds.a:Stopped 2020-11-20T12:17:34.539 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T12:17:34.541 INFO:tasks.ceph.mds.d:Stopped 2020-11-20T12:17:34.541 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-20T12:17:34.542 INFO:tasks.ceph.mds.b:Stopped 2020-11-20T12:17:34.543 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T12:17:34.546 INFO:tasks.ceph.mds.c:Started 2020-11-20T12:17:34.547 INFO:tasks.ceph.mds.a:Started 2020-11-20T12:17:34.547 DEBUG:teuthology.parallel:result is None 2020-11-20T12:17:34.548 DEBUG:teuthology.parallel:result is None 2020-11-20T12:17:34.549 INFO:tasks.ceph.mds.d:Started 2020-11-20T12:17:34.549 DEBUG:teuthology.parallel:result is None 2020-11-20T12:17:34.552 INFO:tasks.ceph.mds.b:Started 2020-11-20T12:17:34.552 DEBUG:teuthology.parallel:result is None 2020-11-20T12:17:34.553 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-20T12:17:34.641 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:17:34.642 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:17:34.650 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2020-11-20T12:17:34.655 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2020-11-20T12:17:34.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.732+0000 7f7a3ef2c700 1 Processor -- start 2020-11-20T12:17:34.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.732+0000 7f7a3ef2c700 1 -- start start 2020-11-20T12:17:34.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.732+0000 7f7a3ef2c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7a38137380 0x7f7a38152120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:34.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3ef2c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a3814e790 0x7f7a38061630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:34.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3ef2c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a3814ef80 con 0x7f7a38061180 2020-11-20T12:17:34.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3ef2c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a38065120 con 0x7f7a38137380 2020-11-20T12:17:34.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3ef2c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3814ece0 con 0x7f7a3814e790 2020-11-20T12:17:34.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3df2a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7a38137380 0x7f7a38152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:34.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3e72b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a3814e790 0x7f7a38061630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:34.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3df2a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7a38137380 0x7f7a38152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:44574/0 (socket says 172.21.15.116:44574) 2020-11-20T12:17:34.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3df2a700 1 -- 172.21.15.116:0/4166134177 learned_addr learned my addr 172.21.15.116:0/4166134177 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:34.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3d729700 1 --1- 172.21.15.116:0/4166134177 >> v1:172.21.15.173:6789/0 conn(0x7f7a38061180 0x7f7a3814e080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:51188/0 (socket says 172.21.15.116:51188) 2020-11-20T12:17:34.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3e72b700 1 -- 172.21.15.116:0/4166134177 >> v2:172.21.15.173:3300/0 conn(0x7f7a38137380 msgr2=0x7f7a38152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:34.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3e72b700 1 --2- 172.21.15.116:0/4166134177 >> v2:172.21.15.173:3300/0 conn(0x7f7a38137380 0x7f7a38152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:34.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3e72b700 1 -- 172.21.15.116:0/4166134177 >> v1:172.21.15.173:6789/0 conn(0x7f7a38061180 legacy=0x7f7a3814e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3e72b700 1 -- 172.21.15.116:0/4166134177 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a38151740 con 0x7f7a3814e790 2020-11-20T12:17:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3cf28700 1 -- 172.21.15.116:0/4166134177 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3828305400 0 0) 0x7f7a3814ef80 con 0x7f7a38061180 2020-11-20T12:17:34.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3df2a700 1 --2- 172.21.15.116:0/4166134177 >> v2:172.21.15.173:3300/0 conn(0x7f7a38137380 0x7f7a38152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:34.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3e72b700 1 --2- 172.21.15.116:0/4166134177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a3814e790 0x7f7a38061630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f7a30009c90 tx=0x7f7a300091d0).ready entity=mon.2 client_cookie=8acb734e3ad5edf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:34.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3cf28700 1 -- 172.21.15.116:0/4166134177 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3000ba70 con 0x7f7a3814e790 2020-11-20T12:17:34.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.736+0000 7f7a3cf28700 1 -- 172.21.15.116:0/4166134177 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3000bbd0 con 0x7f7a3814e790 2020-11-20T12:17:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.740+0000 7f7a3cf28700 1 -- 172.21.15.116:0/4166134177 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a30011400 con 0x7f7a3814e790 2020-11-20T12:17:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.740+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/4166134177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a3814e790 msgr2=0x7f7a38061630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.740+0000 7f7a3ef2c700 1 --2- 172.21.15.116:0/4166134177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a3814e790 0x7f7a38061630 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f7a30009c90 tx=0x7f7a300091d0).stop 2020-11-20T12:17:34.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.740+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/4166134177 shutdown_connections 2020-11-20T12:17:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.740+0000 7f7a3ef2c700 1 --2- 172.21.15.116:0/4166134177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a3814e790 0x7f7a38061630 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.740+0000 7f7a3ef2c700 1 --2- 172.21.15.116:0/4166134177 >> v2:172.21.15.173:3300/0 conn(0x7f7a38137380 0x7f7a38152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:34.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.740+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/4166134177 >> 172.21.15.116:0/4166134177 conn(0x7f7a380c1e60 msgr2=0x7f7a38066960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.740+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/4166134177 shutdown_connections 2020-11-20T12:17:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.740+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/4166134177 wait complete. 2020-11-20T12:17:34.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.740+0000 7f7a3ef2c700 1 Processor -- start 2020-11-20T12:17:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.740+0000 7f7a3ef2c700 1 -- start start 2020-11-20T12:17:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3ef2c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a38061180 0x7f7a380a6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:34.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3ef2c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a38137380 0x7f7a380a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3ef2c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a3814e790 0x7f7a3812e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3ef2c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38145c80 con 0x7f7a38061180 2020-11-20T12:17:34.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3ef2c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380a6060 con 0x7f7a3814e790 2020-11-20T12:17:34.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3ef2c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380a62c0 con 0x7f7a38137380 2020-11-20T12:17:34.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3df2a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a38061180 0x7f7a380a6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3df2a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a38061180 0x7f7a380a6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45528/0 (socket says 172.21.15.116:45528) 2020-11-20T12:17:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3df2a700 1 -- 172.21.15.116:0/380539825 learned_addr learned my addr 172.21.15.116:0/380539825 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:34.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3d729700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a38137380 0x7f7a380a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:34.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3e72b700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a3814e790 0x7f7a3812e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:34.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3df2a700 1 -- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a38137380 msgr2=0x7f7a380a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:34.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3df2a700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a38137380 0x7f7a380a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:34.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3df2a700 1 -- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a3814e790 msgr2=0x7f7a3812e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:34.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3df2a700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a3814e790 0x7f7a3812e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:34.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3df2a700 1 -- 172.21.15.116:0/380539825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a380a6560 con 0x7f7a38061180 2020-11-20T12:17:34.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3df2a700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a38061180 0x7f7a380a6f40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f7a3400ef20 tx=0x7f7a34006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:34.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a25ffb700 1 -- 172.21.15.116:0/380539825 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3400cbc0 con 0x7f7a38061180 2020-11-20T12:17:34.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a25ffb700 1 -- 172.21.15.116:0/380539825 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3400cd20 con 0x7f7a38061180 2020-11-20T12:17:34.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/380539825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a380a6800 con 0x7f7a38061180 2020-11-20T12:17:34.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a25ffb700 1 -- 172.21.15.116:0/380539825 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34012600 con 0x7f7a38061180 2020-11-20T12:17:34.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.744+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/380539825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a380a74b0 con 0x7f7a38061180 2020-11-20T12:17:34.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.748+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/380539825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a380a5470 con 0x7f7a38061180 2020-11-20T12:17:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.748+0000 7f7a25ffb700 1 -- 172.21.15.116:0/380539825 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7a34022070 con 0x7f7a38061180 2020-11-20T12:17:34.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.748+0000 7f7a25ffb700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a100557c0 0x7f7a10057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:34.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.748+0000 7f7a25ffb700 1 -- 172.21.15.116:0/380539825 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f7a3406e970 con 0x7f7a38061180 2020-11-20T12:17:34.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.748+0000 7f7a3d729700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a100557c0 0x7f7a10057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:34.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.748+0000 7f7a3d729700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a100557c0 0x7f7a10057c80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f7a38064470 tx=0x7f7a28007a20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:34.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.752+0000 7f7a25ffb700 1 -- 172.21.15.116:0/380539825 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7a34044af0 con 0x7f7a38061180 2020-11-20T12:17:34.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:34.928+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/380539825 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7a38044f60 con 0x7f7a38061180 2020-11-20T12:17:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.492+0000 7f7a25ffb700 1 -- 172.21.15.116:0/380539825 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v38) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7a34048110 con 0x7f7a38061180 2020-11-20T12:17:35.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.496+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a100557c0 msgr2=0x7f7a10057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:35.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.500+0000 7f7a3ef2c700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a100557c0 0x7f7a10057c80 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f7a38064470 tx=0x7f7a28007a20).stop 2020-11-20T12:17:35.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.500+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/380539825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a38061180 msgr2=0x7f7a380a6f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:35.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.500+0000 7f7a3ef2c700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a38061180 0x7f7a380a6f40 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f7a3400ef20 tx=0x7f7a34006eb0).stop 2020-11-20T12:17:35.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.500+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/380539825 shutdown_connections 2020-11-20T12:17:35.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.500+0000 7f7a3ef2c700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a100557c0 0x7f7a10057c80 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:35.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.500+0000 7f7a3ef2c700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a3814e790 0x7f7a3812e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:35.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.500+0000 7f7a3ef2c700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a38061180 0x7f7a380a6f40 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:35.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.500+0000 7f7a3ef2c700 1 --2- 172.21.15.116:0/380539825 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a38137380 0x7f7a380a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:35.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.500+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/380539825 >> 172.21.15.116:0/380539825 conn(0x7f7a380c1e60 msgr2=0x7f7a3805ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:35.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.500+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/380539825 shutdown_connections 2020-11-20T12:17:35.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.500+0000 7f7a3ef2c700 1 -- 172.21.15.116:0/380539825 wait complete. 2020-11-20T12:17:35.502 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2020-11-20T12:17:35.524 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-20T12:17:35.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.720+0000 7fcbcb59e700 1 Processor -- start 2020-11-20T12:17:35.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.720+0000 7fcbcb59e700 1 -- start start 2020-11-20T12:17:35.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.720+0000 7fcbcb59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbcc0c45a0 0x7fcbcc138670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:35.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.720+0000 7fcbcb59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcbcc138b60 0x7fcbcc0a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:35.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.720+0000 7fcbcb59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbcc0a9180 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.720+0000 7fcbcb59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbcc0a9420 con 0x7fcbcc138b60 2020-11-20T12:17:35.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.720+0000 7fcbcb59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc0a96c0 con 0x7fcbcc0c45a0 2020-11-20T12:17:35.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcad9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcbcc138b60 0x7fcbcc0a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:35.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbc1d9b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbcc0c45a0 0x7fcbcc138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:35.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcad9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcbcc138b60 0x7fcbcc0a7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45540/0 (socket says 172.21.15.116:45540) 2020-11-20T12:17:35.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcad9d700 1 -- 172.21.15.116:0/2076791534 learned_addr learned my addr 172.21.15.116:0/2076791534 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:35.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcad9d700 1 -- 172.21.15.116:0/2076791534 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbcc0c45a0 msgr2=0x7fcbcc138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:35.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcad9d700 1 --2- 172.21.15.116:0/2076791534 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbcc0c45a0 0x7fcbcc138670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:35.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcad9d700 1 -- 172.21.15.116:0/2076791534 >> v1:172.21.15.173:6789/0 conn(0x7fcbcc0c41c0 legacy=0x7fcbcc0a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:35.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcad9d700 1 -- 172.21.15.116:0/2076791534 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc0a9960 con 0x7fcbcc138b60 2020-11-20T12:17:35.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcad9d700 1 --2- 172.21.15.116:0/2076791534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbcc138b60 0x7fcbcc0a7c70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fcbc4009d50 tx=0x7fcbc4009290).ready entity=mon.0 client_cookie=9f7298823b2d4c52 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:35.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbc9d9b700 1 -- 172.21.15.116:0/2076791534 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc400bbc0 con 0x7fcbcc138b60 2020-11-20T12:17:35.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbc9d9b700 1 -- 172.21.15.116:0/2076791534 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbc400bd20 con 0x7fcbcc138b60 2020-11-20T12:17:35.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbc9d9b700 1 -- 172.21.15.116:0/2076791534 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbc4011610 con 0x7fcbcc138b60 2020-11-20T12:17:35.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcb59e700 1 -- 172.21.15.116:0/2076791534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbcc138b60 msgr2=0x7fcbcc0a7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcb59e700 1 --2- 172.21.15.116:0/2076791534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbcc138b60 0x7fcbcc0a7c70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fcbc4009d50 tx=0x7fcbc4009290).stop 2020-11-20T12:17:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcb59e700 1 -- 172.21.15.116:0/2076791534 shutdown_connections 2020-11-20T12:17:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcb59e700 1 --2- 172.21.15.116:0/2076791534 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbcc138b60 0x7fcbcc0a7c70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcb59e700 1 --2- 172.21.15.116:0/2076791534 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbcc0c45a0 0x7fcbcc138670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:35.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcb59e700 1 -- 172.21.15.116:0/2076791534 >> 172.21.15.116:0/2076791534 conn(0x7fcbcc062520 msgr2=0x7fcbcc062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcb59e700 1 -- 172.21.15.116:0/2076791534 shutdown_connections 2020-11-20T12:17:35.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.724+0000 7fcbcb59e700 1 -- 172.21.15.116:0/2076791534 wait complete. 2020-11-20T12:17:35.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.728+0000 7fcbcb59e700 1 Processor -- start 2020-11-20T12:17:35.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.728+0000 7fcbcb59e700 1 -- start start 2020-11-20T12:17:35.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbcb59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbcc0c41c0 0x7fcbcc0a3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:35.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbcb59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc0c45a0 0x7fcbcc14ea70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:35.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbcb59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbcc138b60 0x7fcbcc137230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:35.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbcb59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc132f50 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbcb59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc1331f0 con 0x7fcbcc0c45a0 2020-11-20T12:17:35.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbcb59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbcc133780 con 0x7fcbcc138b60 2020-11-20T12:17:35.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbca59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbcc0c41c0 0x7fcbcc0a3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:35.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbca59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbcc0c41c0 0x7fcbcc0a3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45542/0 (socket says 172.21.15.116:45542) 2020-11-20T12:17:35.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbca59c700 1 -- 172.21.15.116:0/1912871168 learned_addr learned my addr 172.21.15.116:0/1912871168 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:35.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbc1d9b700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc0c45a0 0x7fcbcc14ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:35.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbcad9d700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbcc138b60 0x7fcbcc137230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:35.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbca59c700 1 -- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbcc138b60 msgr2=0x7fcbcc137230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:35.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbca59c700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbcc138b60 0x7fcbcc137230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:35.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbca59c700 1 -- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc0c45a0 msgr2=0x7fcbcc14ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:35.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbca59c700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc0c45a0 0x7fcbcc14ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:35.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbca59c700 1 -- 172.21.15.116:0/1912871168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbcc133a20 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbca59c700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbcc0c41c0 0x7fcbcc0a3230 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fcbbc00c6e0 tx=0x7fcbbc007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:35.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.732+0000 7fcbc2ffd700 1 -- 172.21.15.116:0/1912871168 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc007020 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.736+0000 7fcbc2ffd700 1 -- 172.21.15.116:0/1912871168 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbbc007180 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.736+0000 7fcbc2ffd700 1 -- 172.21.15.116:0/1912871168 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbbc00e560 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.736+0000 7fcbcb59e700 1 -- 172.21.15.116:0/1912871168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbcc133490 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.736+0000 7fcbcb59e700 1 -- 172.21.15.116:0/1912871168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbcc137ed0 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.736+0000 7fcbcb59e700 1 -- 172.21.15.116:0/1912871168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbcc133bb0 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.736+0000 7fcbc2ffd700 1 -- 172.21.15.116:0/1912871168 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fcbbc010070 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.740+0000 7fcbc2ffd700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcba80557c0 0x7fcba8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.740+0000 7fcbc2ffd700 1 -- 172.21.15.116:0/1912871168 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fcbbc06d800 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.740+0000 7fcbc1d9b700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcba80557c0 0x7fcba8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:35.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.740+0000 7fcbc1d9b700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcba80557c0 0x7fcba8057c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fcbcc0a2c20 tx=0x7fcbb4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:35.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.744+0000 7fcbc2ffd700 1 -- 172.21.15.116:0/1912871168 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcbbc042fd0 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.960+0000 7fcbcb59e700 1 -- 172.21.15.116:0/1912871168 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcbcc044f60 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbc2ffd700 1 -- 172.21.15.116:0/1912871168 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) v1 ==== 80+0+5138 (secure 0 0 0) 0x7fcbbc0438c0 con 0x7fcbcc0c41c0 2020-11-20T12:17:35.962 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:35.963 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:17:35.963 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2020-11-20T12:17:35.963 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.963 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2020-11-20T12:17:35.964 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw==", 2020-11-20T12:17:35.964 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.964 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:17:35.964 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.965 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.965 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.965 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.965 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.965 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.966 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2020-11-20T12:17:35.966 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ==", 2020-11-20T12:17:35.966 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.966 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:17:35.967 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.967 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.967 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.967 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.967 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.968 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.968 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2020-11-20T12:17:35.968 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag==", 2020-11-20T12:17:35.968 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.968 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:17:35.969 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.969 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.969 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.969 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.970 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.970 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.970 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2020-11-20T12:17:35.970 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ==", 2020-11-20T12:17:35.970 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.971 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:17:35.971 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.971 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.971 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.971 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.972 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.972 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.972 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2020-11-20T12:17:35.972 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ==", 2020-11-20T12:17:35.972 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.973 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.973 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.973 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.973 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.974 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.974 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.974 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2020-11-20T12:17:35.974 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q==", 2020-11-20T12:17:35.974 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.975 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.975 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.975 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.975 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.975 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.976 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.976 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2020-11-20T12:17:35.976 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw==", 2020-11-20T12:17:35.976 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.976 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.977 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.977 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.977 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.977 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.978 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.978 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2020-11-20T12:17:35.978 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w==", 2020-11-20T12:17:35.978 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.978 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.979 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.979 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.979 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.979 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.979 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.980 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2020-11-20T12:17:35.980 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA==", 2020-11-20T12:17:35.980 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.980 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.980 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.981 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.981 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.981 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.981 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.981 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2020-11-20T12:17:35.982 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ==", 2020-11-20T12:17:35.982 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.982 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.982 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.983 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.983 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.983 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.983 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.983 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2020-11-20T12:17:35.984 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg==", 2020-11-20T12:17:35.984 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.984 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.984 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.984 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.985 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.985 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.985 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.985 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2020-11-20T12:17:35.985 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg==", 2020-11-20T12:17:35.986 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.986 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.986 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.986 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.986 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.987 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.987 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.987 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2020-11-20T12:17:35.987 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA==", 2020-11-20T12:17:35.988 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.988 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:17:35.988 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2020-11-20T12:17:35.988 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-20T12:17:35.988 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.989 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.989 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.989 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2020-11-20T12:17:35.990 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow==", 2020-11-20T12:17:35.990 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.990 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:17:35.990 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:17:35.990 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:17:35.991 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:17:35.991 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.991 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.991 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.991 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2020-11-20T12:17:35.992 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ==", 2020-11-20T12:17:35.992 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.992 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:17:35.992 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:17:35.992 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:17:35.993 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:17:35.993 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.993 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.993 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.994 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2020-11-20T12:17:35.994 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ==", 2020-11-20T12:17:35.994 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.994 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:17:35.994 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:17:35.995 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:17:35.995 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:17:35.995 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.995 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.996 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.996 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2020-11-20T12:17:35.996 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBDs7dfh+KKOBAAZvcPXWly9/mxRE7NWuR85A==", 2020-11-20T12:17:35.996 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.997 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:17:35.997 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:17:35.997 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:17:35.997 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:35.998 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:35.998 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:35.998 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:35.998 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2020-11-20T12:17:35.998 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A==", 2020-11-20T12:17:35.999 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:35.999 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:17:35.999 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:17:35.999 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:36.000 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:36.000 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:36.000 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:36.001 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2020-11-20T12:17:36.001 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA==", 2020-11-20T12:17:36.001 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:17:36.001 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:17:36.001 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:17:36.002 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:17:36.002 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:36.002 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:36.002 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:17:36.002 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:17:36.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 -- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcba80557c0 msgr2=0x7fcba8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcba80557c0 0x7fcba8057c80 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fcbcc0a2c20 tx=0x7fcbb4008040).stop 2020-11-20T12:17:36.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 -- 172.21.15.116:0/1912871168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbcc0c41c0 msgr2=0x7fcbcc0a3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbcc0c41c0 0x7fcbcc0a3230 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7fcbbc00c6e0 tx=0x7fcbbc007c50).stop 2020-11-20T12:17:36.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 -- 172.21.15.116:0/1912871168 shutdown_connections 2020-11-20T12:17:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcba80557c0 0x7fcba8057c80 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbcc138b60 0x7fcbcc137230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbcc0c41c0 0x7fcbcc0a3230 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 --2- 172.21.15.116:0/1912871168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbcc0c45a0 0x7fcbcc14ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 -- 172.21.15.116:0/1912871168 >> 172.21.15.116:0/1912871168 conn(0x7fcbcc062520 msgr2=0x7fcbcc152e20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:36.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 -- 172.21.15.116:0/1912871168 shutdown_connections 2020-11-20T12:17:36.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:35.964+0000 7fcbcb59e700 1 -- 172.21.15.116:0/1912871168 wait complete. 2020-11-20T12:17:36.009 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:36.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.184+0000 7f74ca588700 1 Processor -- start 2020-11-20T12:17:36.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.184+0000 7f74ca588700 1 -- start start 2020-11-20T12:17:36.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74ca588700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f74c41526d0 0x7f74c40a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74ca588700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c40602f0 con 0x7f74c41522f0 2020-11-20T12:17:36.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74ca588700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c4060590 con 0x7f74c40a3ea0 2020-11-20T12:17:36.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74ca588700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c4060830 con 0x7f74c41526d0 2020-11-20T12:17:36.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c3fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f74c41522f0 0x7f74c40c28d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:54984/0 (socket says 172.21.15.116:54984) 2020-11-20T12:17:36.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c3fff700 1 -- 172.21.15.116:0/4042710851 learned_addr learned my addr 172.21.15.116:0/4042710851 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:36.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74bbfff700 1 --2- 172.21.15.116:0/4042710851 >> v2:172.21.15.173:3300/0 conn(0x7f74c41526d0 0x7f74c40a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050819761 0 0) 0x7f74c40602f0 con 0x7f74c41522f0 2020-11-20T12:17:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a4003240 con 0x7f74c41522f0 2020-11-20T12:17:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2524650324 0 0) 0x7f74a4003240 con 0x7f74c41522f0 2020-11-20T12:17:36.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 >> v2:172.21.15.173:3300/0 conn(0x7f74c41526d0 msgr2=0x7f74c40a39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 --2- 172.21.15.116:0/4042710851 >> v2:172.21.15.173:3300/0 conn(0x7f74c41526d0 0x7f74c40a39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 >> v1:172.21.15.173:6789/0 conn(0x7f74c40a3ea0 legacy=0x7f74c40a18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c4060ad0 con 0x7f74c41522f0 2020-11-20T12:17:36.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f74b40014e0 con 0x7f74c41522f0 2020-11-20T12:17:36.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 >> v1:172.21.15.116:6789/0 conn(0x7f74c41522f0 legacy=0x7f74c40c28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 --2- 172.21.15.116:0/4042710851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74a4003730 0x7f74a4003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 --2- 172.21.15.116:0/4042710851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 0x7f74a4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 --2- 172.21.15.116:0/4042710851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74a4004e50 0x7f74a400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4060830 con 0x7f74a4004e50 2020-11-20T12:17:36.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a4005360 con 0x7f74a4003730 2020-11-20T12:17:36.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.188+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a400bc50 con 0x7f74a40044e0 2020-11-20T12:17:36.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c8b25700 1 --2- 172.21.15.116:0/4042710851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74a4004e50 0x7f74a400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c3fff700 1 --2- 172.21.15.116:0/4042710851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74a4003730 0x7f74a4003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c8b25700 1 -- 172.21.15.116:0/4042710851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 msgr2=0x7f74a4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:36.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c8b25700 1 --2- 172.21.15.116:0/4042710851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 0x7f74a4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:36.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c8b25700 1 -- 172.21.15.116:0/4042710851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74a4003730 msgr2=0x7f74a4003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:36.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c8b25700 1 --2- 172.21.15.116:0/4042710851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74a4003730 0x7f74a4003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c8b25700 1 -- 172.21.15.116:0/4042710851 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a4003240 con 0x7f74a4004e50 2020-11-20T12:17:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c3fff700 1 --2- 172.21.15.116:0/4042710851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74a4003730 0x7f74a4003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c8b25700 1 -- 172.21.15.116:0/4042710851 reap_dead start 2020-11-20T12:17:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c8b25700 1 --2- 172.21.15.116:0/4042710851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74a4004e50 0x7f74a400b730 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f74c40aabc0 tx=0x7f74bc007f10).ready entity=mon.0 client_cookie=ec39184e7ff4f6a2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:36.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc001540 con 0x7f74a4004e50 2020-11-20T12:17:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc00cad0 con 0x7f74a4004e50 2020-11-20T12:17:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74c37fe700 1 -- 172.21.15.116:0/4042710851 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc00ce70 con 0x7f74a4004e50 2020-11-20T12:17:36.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74ca588700 1 -- 172.21.15.116:0/4042710851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74a4004e50 msgr2=0x7f74a400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74ca588700 1 --2- 172.21.15.116:0/4042710851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74a4004e50 0x7f74a400b730 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f74c40aabc0 tx=0x7f74bc007f10).stop 2020-11-20T12:17:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74ca588700 1 -- 172.21.15.116:0/4042710851 shutdown_connections 2020-11-20T12:17:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74ca588700 1 --2- 172.21.15.116:0/4042710851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74a4004e50 0x7f74a400b730 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74ca588700 1 -- 172.21.15.116:0/4042710851 >> 172.21.15.116:0/4042710851 conn(0x7f74c40c1470 msgr2=0x7f74c40c1880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:36.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74ca588700 1 -- 172.21.15.116:0/4042710851 shutdown_connections 2020-11-20T12:17:36.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74ca588700 1 -- 172.21.15.116:0/4042710851 wait complete. 2020-11-20T12:17:36.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.192+0000 7f74ca588700 1 Processor -- start 2020-11-20T12:17:36.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74ca588700 1 -- start start 2020-11-20T12:17:36.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74ca588700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a4004e50 0x7f74c4138430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74ca588700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74c413a6e0 0x7f74c4152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74ca588700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74c4151170 0x7f74c4136f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74ca588700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c4138920 con 0x7f74c4151170 2020-11-20T12:17:36.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74ca588700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c413ab60 con 0x7f74c413a6e0 2020-11-20T12:17:36.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74ca588700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c413adc0 con 0x7f74a4004e50 2020-11-20T12:17:36.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c8b25700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74c4151170 0x7f74c4136f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c8b25700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74c4151170 0x7f74c4136f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45566/0 (socket says 172.21.15.116:45566) 2020-11-20T12:17:36.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c8b25700 1 -- 172.21.15.116:0/190676536 learned_addr learned my addr 172.21.15.116:0/190676536 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:36.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c3fff700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a4004e50 0x7f74c4138430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74bbfff700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74c413a6e0 0x7f74c4152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c8b25700 1 -- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a4004e50 msgr2=0x7f74c4138430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c8b25700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a4004e50 0x7f74c4138430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c8b25700 1 -- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74c413a6e0 msgr2=0x7f74c4152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c8b25700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74c413a6e0 0x7f74c4152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c8b25700 1 -- 172.21.15.116:0/190676536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c4135120 con 0x7f74c4151170 2020-11-20T12:17:36.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c8b25700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74c4151170 0x7f74c4136f10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f74c4069f50 tx=0x7f74bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:36.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c0ff9700 1 -- 172.21.15.116:0/190676536 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc0125d0 con 0x7f74c4151170 2020-11-20T12:17:36.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c0ff9700 1 -- 172.21.15.116:0/190676536 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc012730 con 0x7f74c4151170 2020-11-20T12:17:36.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74c0ff9700 1 -- 172.21.15.116:0/190676536 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc012ad0 con 0x7f74c4151170 2020-11-20T12:17:36.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.196+0000 7f74ca588700 1 -- 172.21.15.116:0/190676536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c41353c0 con 0x7f74c4151170 2020-11-20T12:17:36.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.200+0000 7f74ca588700 1 -- 172.21.15.116:0/190676536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c4135900 con 0x7f74c4151170 2020-11-20T12:17:36.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.200+0000 7f74ca588700 1 -- 172.21.15.116:0/190676536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c405ca30 con 0x7f74c4151170 2020-11-20T12:17:36.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.200+0000 7f74c0ff9700 1 -- 172.21.15.116:0/190676536 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f74bc026030 con 0x7f74c4151170 2020-11-20T12:17:36.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.200+0000 7f74c0ff9700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74a00557c0 0x7f74a0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.200+0000 7f74c0ff9700 1 -- 172.21.15.116:0/190676536 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f74bc06fb20 con 0x7f74c4151170 2020-11-20T12:17:36.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.204+0000 7f74c3fff700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74a00557c0 0x7f74a0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.204+0000 7f74c3fff700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74a00557c0 0x7f74a0057c80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f74b4001e90 tx=0x7f74b4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:36.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.208+0000 7f74c0ff9700 1 -- 172.21.15.116:0/190676536 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f74bc045c60 con 0x7f74c4151170 2020-11-20T12:17:36.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.364+0000 7f74ca588700 1 -- 172.21.15.116:0/190676536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74c4152b50 con 0x7f74c4151170 2020-11-20T12:17:36.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74c0ff9700 1 -- 172.21.15.116:0/190676536 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+13723 (secure 0 0 0) 0x7f74bc049280 con 0x7f74c4151170 2020-11-20T12:17:36.367 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:36.367 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":38,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:17:35.479676+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:36.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 -- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74a00557c0 msgr2=0x7f74a0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74a00557c0 0x7f74a0057c80 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f74b4001e90 tx=0x7f74b4009040).stop 2020-11-20T12:17:36.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 -- 172.21.15.116:0/190676536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74c4151170 msgr2=0x7f74c4136f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74c4151170 0x7f74c4136f10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f74c4069f50 tx=0x7f74bc015000).stop 2020-11-20T12:17:36.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 -- 172.21.15.116:0/190676536 shutdown_connections 2020-11-20T12:17:36.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74a00557c0 0x7f74a0057c80 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74c4151170 0x7f74c4136f10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a4004e50 0x7f74c4138430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 --2- 172.21.15.116:0/190676536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74c413a6e0 0x7f74c4152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 -- 172.21.15.116:0/190676536 >> 172.21.15.116:0/190676536 conn(0x7f74c40c1470 msgr2=0x7f74c40a1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:36.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 -- 172.21.15.116:0/190676536 shutdown_connections 2020-11-20T12:17:36.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.368+0000 7f74ca588700 1 -- 172.21.15.116:0/190676536 wait complete. 2020-11-20T12:17:36.392 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T12:17:36.393 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:36.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.572+0000 7f121359e700 1 Processor -- start 2020-11-20T12:17:36.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.572+0000 7f121359e700 1 -- start start 2020-11-20T12:17:36.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.572+0000 7f121359e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1214061250 0x7f12140616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.572+0000 7f121359e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1214152a80 con 0x7f121413ad30 2020-11-20T12:17:36.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.572+0000 7f121359e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1214152540 con 0x7f121413b110 2020-11-20T12:17:36.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.572+0000 7f121359e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12141527e0 con 0x7f1214061250 2020-11-20T12:17:36.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f1212d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1214061250 0x7f12140616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f1211d9b700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f121413b110 0x7f1214151ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:51234/0 (socket says 172.21.15.116:51234) 2020-11-20T12:17:36.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f1211d9b700 1 -- 172.21.15.116:0/188707059 learned_addr learned my addr 172.21.15.116:0/188707059 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:36.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.572+0000 7f121259c700 1 --1- 172.21.15.116:0/188707059 >> v1:172.21.15.116:6789/0 conn(0x7f121413ad30 0x7f12140a7fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55006/0 (socket says 172.21.15.116:55006) 2020-11-20T12:17:36.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f1212d9d700 1 -- 172.21.15.116:0/188707059 >> v1:172.21.15.173:6789/0 conn(0x7f121413b110 legacy=0x7f1214151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f1212d9d700 1 -- 172.21.15.116:0/188707059 >> v1:172.21.15.116:6789/0 conn(0x7f121413ad30 legacy=0x7f12140a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f121159a700 1 -- 172.21.15.116:0/188707059 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651057703 0 0) 0x7f1214152a80 con 0x7f121413ad30 2020-11-20T12:17:36.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f1212d9d700 1 -- 172.21.15.116:0/188707059 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12140ac910 con 0x7f1214061250 2020-11-20T12:17:36.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f1212d9d700 1 --2- 172.21.15.116:0/188707059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1214061250 0x7f12140616e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f120c009c90 tx=0x7f120c0091d0).ready entity=mon.0 client_cookie=37143a5e45b7f69d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:36.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f121159a700 1 -- 172.21.15.116:0/188707059 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c00bbc0 con 0x7f1214061250 2020-11-20T12:17:36.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f121159a700 1 -- 172.21.15.116:0/188707059 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c00bd20 con 0x7f1214061250 2020-11-20T12:17:36.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f121159a700 1 -- 172.21.15.116:0/188707059 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c011610 con 0x7f1214061250 2020-11-20T12:17:36.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f121359e700 1 -- 172.21.15.116:0/188707059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1214061250 msgr2=0x7f12140616e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f121359e700 1 --2- 172.21.15.116:0/188707059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1214061250 0x7f12140616e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f120c009c90 tx=0x7f120c0091d0).stop 2020-11-20T12:17:36.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f121359e700 1 -- 172.21.15.116:0/188707059 shutdown_connections 2020-11-20T12:17:36.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f121359e700 1 --2- 172.21.15.116:0/188707059 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1214061250 0x7f12140616e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f121359e700 1 -- 172.21.15.116:0/188707059 >> 172.21.15.116:0/188707059 conn(0x7f12140678a0 msgr2=0x7f1214067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:36.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f121359e700 1 -- 172.21.15.116:0/188707059 shutdown_connections 2020-11-20T12:17:36.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.576+0000 7f121359e700 1 -- 172.21.15.116:0/188707059 wait complete. 2020-11-20T12:17:36.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f121359e700 1 Processor -- start 2020-11-20T12:17:36.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f121359e700 1 -- start start 2020-11-20T12:17:36.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f121359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1214061250 0x7f1214165050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f121359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f121413ad30 0x7f121409fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f121359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f121413b110 0x7f12140a1b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f121359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12140a04a0 con 0x7f121413b110 2020-11-20T12:17:36.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f121359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12140a0740 con 0x7f121413ad30 2020-11-20T12:17:36.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f121359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12140a09e0 con 0x7f1214061250 2020-11-20T12:17:36.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f1212d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f121413b110 0x7f12140a1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f1212d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f121413b110 0x7f12140a1b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45578/0 (socket says 172.21.15.116:45578) 2020-11-20T12:17:36.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f1212d9d700 1 -- 172.21.15.116:0/678012662 learned_addr learned my addr 172.21.15.116:0/678012662 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:36.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.580+0000 7f121259c700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1214061250 0x7f1214165050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f1211d9b700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f121413ad30 0x7f121409fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f1212d9d700 1 -- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1214061250 msgr2=0x7f1214165050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f1212d9d700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1214061250 0x7f1214165050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f1212d9d700 1 -- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f121413ad30 msgr2=0x7f121409fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f1212d9d700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f121413ad30 0x7f121409fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f1212d9d700 1 -- 172.21.15.116:0/678012662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12140a0c80 con 0x7f121413b110 2020-11-20T12:17:36.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f1211d9b700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f121413ad30 0x7f121409fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:36.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f1212d9d700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f121413b110 0x7f12140a1b90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f120c009fd0 tx=0x7f120c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:36.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f12027fc700 1 -- 172.21.15.116:0/678012662 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c011e30 con 0x7f121413b110 2020-11-20T12:17:36.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f12027fc700 1 -- 172.21.15.116:0/678012662 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120c010b90 con 0x7f121413b110 2020-11-20T12:17:36.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f12027fc700 1 -- 172.21.15.116:0/678012662 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120c01d480 con 0x7f121413b110 2020-11-20T12:17:36.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f121359e700 1 -- 172.21.15.116:0/678012662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12140a0150 con 0x7f121413b110 2020-11-20T12:17:36.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f121359e700 1 -- 172.21.15.116:0/678012662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12140a1560 con 0x7f121413b110 2020-11-20T12:17:36.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f121359e700 1 -- 172.21.15.116:0/678012662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1214044f60 con 0x7f121413b110 2020-11-20T12:17:36.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.584+0000 7f12027fc700 1 -- 172.21.15.116:0/678012662 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f120c017070 con 0x7f121413b110 2020-11-20T12:17:36.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.588+0000 7f12027fc700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11e80557c0 0x7f11e8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.588+0000 7f12027fc700 1 -- 172.21.15.116:0/678012662 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f120c070b30 con 0x7f121413b110 2020-11-20T12:17:36.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.588+0000 7f121259c700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11e80557c0 0x7f11e8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.588+0000 7f121259c700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11e80557c0 0x7f11e8057c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1214153a80 tx=0x7f1204007a20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:36.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.596+0000 7f12027fc700 1 -- 172.21.15.116:0/678012662 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f120c046bf0 con 0x7f121413b110 2020-11-20T12:17:36.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.760+0000 7f121359e700 1 -- 172.21.15.116:0/678012662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f12140ae420 con 0x7f121413b110 2020-11-20T12:17:36.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.760+0000 7f12027fc700 1 -- 172.21.15.116:0/678012662 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v38) v1 ==== 110+0+10 (secure 0 0 0) 0x7f120c015070 con 0x7f121413b110 2020-11-20T12:17:36.761 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:36.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 -- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11e80557c0 msgr2=0x7f11e8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11e80557c0 0x7f11e8057c80 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1214153a80 tx=0x7f1204007a20).stop 2020-11-20T12:17:36.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 -- 172.21.15.116:0/678012662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f121413b110 msgr2=0x7f12140a1b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f121413b110 0x7f12140a1b90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f120c009fd0 tx=0x7f120c016000).stop 2020-11-20T12:17:36.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 -- 172.21.15.116:0/678012662 shutdown_connections 2020-11-20T12:17:36.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11e80557c0 0x7f11e8057c80 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f121413b110 0x7f12140a1b90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1214061250 0x7f1214165050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 --2- 172.21.15.116:0/678012662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f121413ad30 0x7f121409fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 -- 172.21.15.116:0/678012662 >> 172.21.15.116:0/678012662 conn(0x7f12140678a0 msgr2=0x7f12141542b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:36.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 -- 172.21.15.116:0/678012662 shutdown_connections 2020-11-20T12:17:36.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.764+0000 7f121359e700 1 -- 172.21.15.116:0/678012662 wait complete. 2020-11-20T12:17:36.783 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:36.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.960+0000 7fc39b11b700 1 Processor -- start 2020-11-20T12:17:36.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.960+0000 7fc39b11b700 1 -- start start 2020-11-20T12:17:36.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.964+0000 7fc39b11b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc39414f530 0x7fc39415c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.964+0000 7fc39b11b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc394155620 con 0x7fc3940bd780 2020-11-20T12:17:36.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.964+0000 7fc39b11b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3941558c0 con 0x7fc39414f150 2020-11-20T12:17:36.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.964+0000 7fc39b11b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc394155b60 con 0x7fc39414f530 2020-11-20T12:17:36.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.964+0000 7fc393fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc3940bd780 0x7fc3940a74b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55018/0 (socket says 172.21.15.116:55018) 2020-11-20T12:17:36.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.964+0000 7fc393fff700 1 -- 172.21.15.116:0/1904043104 learned_addr learned my addr 172.21.15.116:0/1904043104 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:36.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.964+0000 7fc3996b8700 1 --2- 172.21.15.116:0/1904043104 >> v2:172.21.15.116:3300/0 conn(0x7fc39414f530 0x7fc39415c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.964+0000 7fc3937fe700 1 -- 172.21.15.116:0/1904043104 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1708593577 0 0) 0x7fc394155620 con 0x7fc3940bd780 2020-11-20T12:17:36.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.964+0000 7fc3937fe700 1 -- 172.21.15.116:0/1904043104 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc378003240 con 0x7fc3940bd780 2020-11-20T12:17:36.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.964+0000 7fc3937fe700 1 -- 172.21.15.116:0/1904043104 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2697910758 0 0) 0x7fc3941558c0 con 0x7fc39414f150 2020-11-20T12:17:36.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.964+0000 7fc3937fe700 1 -- 172.21.15.116:0/1904043104 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc394155620 con 0x7fc39414f150 2020-11-20T12:17:36.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc3996b8700 1 -- 172.21.15.116:0/1904043104 >> v1:172.21.15.173:6789/0 conn(0x7fc39414f150 legacy=0x7fc3940c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc3996b8700 1 -- 172.21.15.116:0/1904043104 >> v1:172.21.15.116:6789/0 conn(0x7fc3940bd780 legacy=0x7fc3940a74b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc3996b8700 1 -- 172.21.15.116:0/1904043104 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394155e00 con 0x7fc39414f530 2020-11-20T12:17:36.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc3996b8700 1 --2- 172.21.15.116:0/1904043104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39414f530 0x7fc39415c4e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fc38c009c90 tx=0x7fc38c0091d0).ready entity=mon.0 client_cookie=b7055961244b839c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:36.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc3937fe700 1 -- 172.21.15.116:0/1904043104 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c00b9c0 con 0x7fc39414f530 2020-11-20T12:17:36.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc3937fe700 1 -- 172.21.15.116:0/1904043104 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c00bb20 con 0x7fc39414f530 2020-11-20T12:17:36.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc3937fe700 1 -- 172.21.15.116:0/1904043104 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c00bea0 con 0x7fc39414f530 2020-11-20T12:17:36.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc39b11b700 1 -- 172.21.15.116:0/1904043104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39414f530 msgr2=0x7fc39415c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc39b11b700 1 --2- 172.21.15.116:0/1904043104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39414f530 0x7fc39415c4e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7fc38c009c90 tx=0x7fc38c0091d0).stop 2020-11-20T12:17:36.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc39b11b700 1 -- 172.21.15.116:0/1904043104 shutdown_connections 2020-11-20T12:17:36.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc39b11b700 1 --2- 172.21.15.116:0/1904043104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39414f530 0x7fc39415c4e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc39b11b700 1 -- 172.21.15.116:0/1904043104 >> 172.21.15.116:0/1904043104 conn(0x7fc394061580 msgr2=0x7fc394061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:36.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc39b11b700 1 -- 172.21.15.116:0/1904043104 shutdown_connections 2020-11-20T12:17:36.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.968+0000 7fc39b11b700 1 -- 172.21.15.116:0/1904043104 wait complete. 2020-11-20T12:17:36.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc39b11b700 1 Processor -- start 2020-11-20T12:17:36.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc39b11b700 1 -- start start 2020-11-20T12:17:36.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc39b11b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3940bd780 0x7fc3940c2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc39b11b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc39414f150 0x7fc39414b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc39b11b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39414f530 0x7fc3940ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc39b11b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3941661a0 con 0x7fc39414f530 2020-11-20T12:17:36.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc39b11b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940ac0a0 con 0x7fc39414f150 2020-11-20T12:17:36.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc39b11b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3940ac340 con 0x7fc3940bd780 2020-11-20T12:17:36.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc3996b8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39414f530 0x7fc3940ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc3996b8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39414f530 0x7fc3940ad900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45590/0 (socket says 172.21.15.116:45590) 2020-11-20T12:17:36.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc3996b8700 1 -- 172.21.15.116:0/1154313751 learned_addr learned my addr 172.21.15.116:0/1154313751 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:36.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc393fff700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc39414f150 0x7fc39414b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc398eb7700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3940bd780 0x7fc3940c2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc3996b8700 1 -- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3940bd780 msgr2=0x7fc3940c2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc3996b8700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3940bd780 0x7fc3940c2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc3996b8700 1 -- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc39414f150 msgr2=0x7fc39414b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:36.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc3996b8700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc39414f150 0x7fc39414b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:36.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc3996b8700 1 -- 172.21.15.116:0/1154313751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc394153020 con 0x7fc39414f530 2020-11-20T12:17:36.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc398eb7700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3940bd780 0x7fc3940c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:36.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.972+0000 7fc393fff700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc39414f150 0x7fc39414b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:36.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.976+0000 7fc3996b8700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39414f530 0x7fc3940ad900 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc3940a8390 tx=0x7fc38c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:36.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.976+0000 7fc390ff9700 1 -- 172.21.15.116:0/1154313751 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c01c5f0 con 0x7fc39414f530 2020-11-20T12:17:36.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.976+0000 7fc390ff9700 1 -- 172.21.15.116:0/1154313751 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38c01c750 con 0x7fc39414f530 2020-11-20T12:17:36.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.976+0000 7fc39b11b700 1 -- 172.21.15.116:0/1154313751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3940ab7f0 con 0x7fc39414f530 2020-11-20T12:17:36.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.976+0000 7fc390ff9700 1 -- 172.21.15.116:0/1154313751 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c01cad0 con 0x7fc39414f530 2020-11-20T12:17:36.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.976+0000 7fc39b11b700 1 -- 172.21.15.116:0/1154313751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc394165d00 con 0x7fc39414f530 2020-11-20T12:17:36.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.976+0000 7fc39b11b700 1 -- 172.21.15.116:0/1154313751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc394165f20 con 0x7fc39414f530 2020-11-20T12:17:36.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.976+0000 7fc390ff9700 1 -- 172.21.15.116:0/1154313751 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc38c017070 con 0x7fc39414f530 2020-11-20T12:17:36.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.980+0000 7fc390ff9700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc36c0557c0 0x7fc36c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:36.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.980+0000 7fc390ff9700 1 -- 172.21.15.116:0/1154313751 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc38c070660 con 0x7fc39414f530 2020-11-20T12:17:36.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.980+0000 7fc398eb7700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc36c0557c0 0x7fc36c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:36.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.980+0000 7fc398eb7700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc36c0557c0 0x7fc36c057c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fc3940a7f50 tx=0x7fc388009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:36.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:36.984+0000 7fc390ff9700 1 -- 172.21.15.116:0/1154313751 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc38c0467d0 con 0x7fc39414f530 2020-11-20T12:17:37.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.140+0000 7fc39b11b700 1 -- 172.21.15.116:0/1154313751 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc3941531b0 con 0x7fc39414f530 2020-11-20T12:17:37.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.140+0000 7fc390ff9700 1 -- 172.21.15.116:0/1154313751 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v38) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc38c015030 con 0x7fc39414f530 2020-11-20T12:17:37.141 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:37.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 -- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc36c0557c0 msgr2=0x7fc36c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:37.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc36c0557c0 0x7fc36c057c80 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fc3940a7f50 tx=0x7fc388009040).stop 2020-11-20T12:17:37.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 -- 172.21.15.116:0/1154313751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39414f530 msgr2=0x7fc3940ad900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:37.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39414f530 0x7fc3940ad900 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7fc3940a8390 tx=0x7fc38c016000).stop 2020-11-20T12:17:37.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 -- 172.21.15.116:0/1154313751 shutdown_connections 2020-11-20T12:17:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc36c0557c0 0x7fc36c057c80 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc39414f530 0x7fc3940ad900 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3940bd780 0x7fc3940c2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 --2- 172.21.15.116:0/1154313751 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc39414f150 0x7fc39414b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 -- 172.21.15.116:0/1154313751 >> 172.21.15.116:0/1154313751 conn(0x7fc394061580 msgr2=0x7fc394150880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 -- 172.21.15.116:0/1154313751 shutdown_connections 2020-11-20T12:17:37.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.144+0000 7fc39b11b700 1 -- 172.21.15.116:0/1154313751 wait complete. 2020-11-20T12:17:37.166 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-20T12:17:37.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.344+0000 7fc8fb59e700 1 Processor -- start 2020-11-20T12:17:37.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.344+0000 7fc8fb59e700 1 -- start start 2020-11-20T12:17:37.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fb59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc8fc14d150 0x7fc8fc063eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fb59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc8fc067070 0x7fc8fc061180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fb59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8fc062f90 con 0x7fc8fc067500 2020-11-20T12:17:37.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fb59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8fc063230 con 0x7fc8fc14d150 2020-11-20T12:17:37.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fb59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8fc0634d0 con 0x7fc8fc067070 2020-11-20T12:17:37.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fa59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc8fc14d150 0x7fc8fc063eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8f9d9b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc8fc067070 0x7fc8fc061180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fa59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc8fc14d150 0x7fc8fc063eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45598/0 (socket says 172.21.15.116:45598) 2020-11-20T12:17:37.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fa59c700 1 -- 172.21.15.116:0/808229246 learned_addr learned my addr 172.21.15.116:0/808229246 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:37.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fa59c700 1 -- 172.21.15.116:0/808229246 >> v2:172.21.15.173:3300/0 conn(0x7fc8fc067070 msgr2=0x7fc8fc061180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:37.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fa59c700 1 --2- 172.21.15.116:0/808229246 >> v2:172.21.15.173:3300/0 conn(0x7fc8fc067070 0x7fc8fc061180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fa59c700 1 -- 172.21.15.116:0/808229246 >> v1:172.21.15.173:6789/0 conn(0x7fc8fc067500 legacy=0x7fc8fc0b9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:37.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8f959a700 1 -- 172.21.15.116:0/808229246 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868121813 0 0) 0x7fc8fc062f90 con 0x7fc8fc067500 2020-11-20T12:17:37.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fa59c700 1 -- 172.21.15.116:0/808229246 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc063770 con 0x7fc8fc14d150 2020-11-20T12:17:37.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8fa59c700 1 --2- 172.21.15.116:0/808229246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8fc14d150 0x7fc8fc063eb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fc8fc0c26f0 tx=0x7fc8ec006eb0).ready entity=mon.0 client_cookie=7a688d42e283e42c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:37.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8f959a700 1 -- 172.21.15.116:0/808229246 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec009810 con 0x7fc8fc14d150 2020-11-20T12:17:37.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8f959a700 1 -- 172.21.15.116:0/808229246 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8ec009970 con 0x7fc8fc14d150 2020-11-20T12:17:37.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.348+0000 7fc8f959a700 1 -- 172.21.15.116:0/808229246 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec009cf0 con 0x7fc8fc14d150 2020-11-20T12:17:37.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.352+0000 7fc8fb59e700 1 -- 172.21.15.116:0/808229246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8fc14d150 msgr2=0x7fc8fc063eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:37.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.352+0000 7fc8fb59e700 1 --2- 172.21.15.116:0/808229246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8fc14d150 0x7fc8fc063eb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fc8fc0c26f0 tx=0x7fc8ec006eb0).stop 2020-11-20T12:17:37.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.352+0000 7fc8fb59e700 1 -- 172.21.15.116:0/808229246 shutdown_connections 2020-11-20T12:17:37.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.352+0000 7fc8fb59e700 1 --2- 172.21.15.116:0/808229246 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8fc14d150 0x7fc8fc063eb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.352+0000 7fc8fb59e700 1 --2- 172.21.15.116:0/808229246 >> v2:172.21.15.173:3300/0 conn(0x7fc8fc067070 0x7fc8fc061180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.352+0000 7fc8fb59e700 1 -- 172.21.15.116:0/808229246 >> 172.21.15.116:0/808229246 conn(0x7fc8fc0c1e60 msgr2=0x7fc8fc066960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:37.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.352+0000 7fc8fb59e700 1 -- 172.21.15.116:0/808229246 shutdown_connections 2020-11-20T12:17:37.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.352+0000 7fc8fb59e700 1 -- 172.21.15.116:0/808229246 wait complete. 2020-11-20T12:17:37.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8fb59e700 1 Processor -- start 2020-11-20T12:17:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8fb59e700 1 -- start start 2020-11-20T12:17:37.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8fb59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8fc067070 0x7fc8fc14abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8fb59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8fc067500 0x7fc8fc153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8fb59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8fc14d150 0x7fc8fc0bdc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8fb59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc0bf1e0 con 0x7fc8fc067500 2020-11-20T12:17:37.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8fb59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc1648e0 con 0x7fc8fc14d150 2020-11-20T12:17:37.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8fb59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8fc164b80 con 0x7fc8fc067070 2020-11-20T12:17:37.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8f9d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8fc067500 0x7fc8fc153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8f9d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8fc067500 0x7fc8fc153a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45606/0 (socket says 172.21.15.116:45606) 2020-11-20T12:17:37.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8f9d9b700 1 -- 172.21.15.116:0/162507494 learned_addr learned my addr 172.21.15.116:0/162507494 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:37.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8fa59c700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8fc067070 0x7fc8fc14abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8fad9d700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8fc14d150 0x7fc8fc0bdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8f9d9b700 1 -- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8fc067070 msgr2=0x7fc8fc14abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:37.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8f9d9b700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8fc067070 0x7fc8fc14abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8f9d9b700 1 -- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8fc14d150 msgr2=0x7fc8fc0bdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:37.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8f9d9b700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8fc14d150 0x7fc8fc0bdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8f9d9b700 1 -- 172.21.15.116:0/162507494 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8fc164e20 con 0x7fc8fc067500 2020-11-20T12:17:37.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8f9d9b700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8fc067500 0x7fc8fc153a50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fc8f000eba0 tx=0x7fc8f0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:37.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.356+0000 7fc8ea7fc700 1 -- 172.21.15.116:0/162507494 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000c8e0 con 0x7fc8fc067500 2020-11-20T12:17:37.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.360+0000 7fc8ea7fc700 1 -- 172.21.15.116:0/162507494 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8f000ca40 con 0x7fc8fc067500 2020-11-20T12:17:37.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.360+0000 7fc8ea7fc700 1 -- 172.21.15.116:0/162507494 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8f000cdc0 con 0x7fc8fc067500 2020-11-20T12:17:37.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.360+0000 7fc8fb59e700 1 -- 172.21.15.116:0/162507494 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8fc1650c0 con 0x7fc8fc067500 2020-11-20T12:17:37.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.360+0000 7fc8fb59e700 1 -- 172.21.15.116:0/162507494 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8fc1645d0 con 0x7fc8fc067500 2020-11-20T12:17:37.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.360+0000 7fc8fb59e700 1 -- 172.21.15.116:0/162507494 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8fc044f60 con 0x7fc8fc067500 2020-11-20T12:17:37.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.360+0000 7fc8ea7fc700 1 -- 172.21.15.116:0/162507494 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc8f0012600 con 0x7fc8fc067500 2020-11-20T12:17:37.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.360+0000 7fc8ea7fc700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8d00557c0 0x7fc8d0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.360+0000 7fc8ea7fc700 1 -- 172.21.15.116:0/162507494 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fc8f006e670 con 0x7fc8fc067500 2020-11-20T12:17:37.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.364+0000 7fc8fa59c700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8d00557c0 0x7fc8d0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.364+0000 7fc8fa59c700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8d00557c0 0x7fc8d0057c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fc8ec006ef0 tx=0x7fc8ec015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:37.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.368+0000 7fc8ea7fc700 1 -- 172.21.15.116:0/162507494 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc8f0044840 con 0x7fc8fc067500 2020-11-20T12:17:37.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.528+0000 7fc8fb59e700 1 -- 172.21.15.116:0/162507494 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc8fc067a40 con 0x7fc8fc067500 2020-11-20T12:17:37.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.528+0000 7fc8ea7fc700 1 -- 172.21.15.116:0/162507494 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v38) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc8f006e020 con 0x7fc8fc067500 2020-11-20T12:17:37.530 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:37.530 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2020-11-20T12:17:37.530 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:37.530 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2020-11-20T12:17:37.531 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "device_health_metrics" 2020-11-20T12:17:37.531 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:17:37.531 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:17:37.531 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 2, 2020-11-20T12:17:37.531 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "rbd" 2020-11-20T12:17:37.532 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:17:37.532 INFO:teuthology.orchestra.run.smithi116.stdout:] 2020-11-20T12:17:37.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 -- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8d00557c0 msgr2=0x7fc8d0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:37.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8d00557c0 0x7fc8d0057c80 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fc8ec006ef0 tx=0x7fc8ec015000).stop 2020-11-20T12:17:37.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 -- 172.21.15.116:0/162507494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8fc067500 msgr2=0x7fc8fc153a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:37.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8fc067500 0x7fc8fc153a50 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7fc8f000eba0 tx=0x7fc8f0006df0).stop 2020-11-20T12:17:37.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 -- 172.21.15.116:0/162507494 shutdown_connections 2020-11-20T12:17:37.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8d00557c0 0x7fc8d0057c80 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8fc14d150 0x7fc8fc0bdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8fc067070 0x7fc8fc14abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 --2- 172.21.15.116:0/162507494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8fc067500 0x7fc8fc153a50 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 -- 172.21.15.116:0/162507494 >> 172.21.15.116:0/162507494 conn(0x7fc8fc0c1e60 msgr2=0x7fc8fc150380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:37.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 -- 172.21.15.116:0/162507494 shutdown_connections 2020-11-20T12:17:37.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.532+0000 7fc8fb59e700 1 -- 172.21.15.116:0/162507494 wait complete. 2020-11-20T12:17:37.557 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T12:17:37.558 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-20T12:17:37.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.732+0000 7fb9d1305700 1 Processor -- start 2020-11-20T12:17:37.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9d1305700 1 -- start start 2020-11-20T12:17:37.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9d1305700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc060310 0x7fb9cc0606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9d1305700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb9cc0c2c30 0x7fb9cc067c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9d1305700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb9cc152530 0x7fb9cc14ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9d1305700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc068110 con 0x7fb9cc0c2c30 2020-11-20T12:17:37.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9d1305700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc0c31b0 con 0x7fb9cc152530 2020-11-20T12:17:37.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9d1305700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc067240 con 0x7fb9cc060310 2020-11-20T12:17:37.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9c3fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb9cc0c2c30 0x7fb9cc067c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9c3fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb9cc0c2c30 0x7fb9cc067c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45612/0 (socket says 172.21.15.116:45612) 2020-11-20T12:17:37.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9c3fff700 1 -- 172.21.15.116:0/850809901 learned_addr learned my addr 172.21.15.116:0/850809901 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:37.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9cbfff700 1 --2- 172.21.15.116:0/850809901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc060310 0x7fb9cc0606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.736+0000 7fb9d0b04700 1 --2- 172.21.15.116:0/850809901 >> v2:172.21.15.173:3300/0 conn(0x7fb9cc152530 0x7fb9cc14ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9c3fff700 1 -- 172.21.15.116:0/850809901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc060310 msgr2=0x7fb9cc0606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:37.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9c3fff700 1 --2- 172.21.15.116:0/850809901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc060310 0x7fb9cc0606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9c3fff700 1 -- 172.21.15.116:0/850809901 >> v2:172.21.15.173:3300/0 conn(0x7fb9cc152530 msgr2=0x7fb9cc14ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:37.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9c3fff700 1 --2- 172.21.15.116:0/850809901 >> v2:172.21.15.173:3300/0 conn(0x7fb9cc152530 0x7fb9cc14ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9c3fff700 1 -- 172.21.15.116:0/850809901 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc0674e0 con 0x7fb9cc0c2c30 2020-11-20T12:17:37.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9c3fff700 1 --2- 172.21.15.116:0/850809901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9cc0c2c30 0x7fb9cc067c20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fb9cc064ec0 tx=0x7fb9b4006eb0).ready entity=mon.0 client_cookie=e67d81f1d9c9e2de server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:37.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9cb7fe700 1 -- 172.21.15.116:0/850809901 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b40098a0 con 0x7fb9cc0c2c30 2020-11-20T12:17:37.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9cb7fe700 1 -- 172.21.15.116:0/850809901 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9b4009a00 con 0x7fb9cc0c2c30 2020-11-20T12:17:37.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9cb7fe700 1 -- 172.21.15.116:0/850809901 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9b4009d80 con 0x7fb9cc0c2c30 2020-11-20T12:17:37.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9d1305700 1 -- 172.21.15.116:0/850809901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9cc0c2c30 msgr2=0x7fb9cc067c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:37.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9d1305700 1 --2- 172.21.15.116:0/850809901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9cc0c2c30 0x7fb9cc067c20 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7fb9cc064ec0 tx=0x7fb9b4006eb0).stop 2020-11-20T12:17:37.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9d1305700 1 -- 172.21.15.116:0/850809901 shutdown_connections 2020-11-20T12:17:37.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9d1305700 1 --2- 172.21.15.116:0/850809901 >> v2:172.21.15.173:3300/0 conn(0x7fb9cc152530 0x7fb9cc14ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9d1305700 1 --2- 172.21.15.116:0/850809901 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc060310 0x7fb9cc0606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9d1305700 1 --2- 172.21.15.116:0/850809901 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9cc0c2c30 0x7fb9cc067c20 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9d1305700 1 -- 172.21.15.116:0/850809901 >> 172.21.15.116:0/850809901 conn(0x7fb9cc061d50 msgr2=0x7fb9cc0c50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:37.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9d1305700 1 -- 172.21.15.116:0/850809901 shutdown_connections 2020-11-20T12:17:37.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.740+0000 7fb9d1305700 1 -- 172.21.15.116:0/850809901 wait complete. 2020-11-20T12:17:37.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.744+0000 7fb9d1305700 1 Processor -- start 2020-11-20T12:17:37.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.744+0000 7fb9d1305700 1 -- start start 2020-11-20T12:17:37.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d1305700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9cc060310 0x7fb9cc0a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d1305700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9cc0c2c30 0x7fb9cc0ba540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d1305700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc152530 0x7fb9cc0baa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d1305700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc1373f0 con 0x7fb9cc060310 2020-11-20T12:17:37.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d1305700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc161f30 con 0x7fb9cc0c2c30 2020-11-20T12:17:37.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d1305700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc1612b0 con 0x7fb9cc152530 2020-11-20T12:17:37.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9cbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9cc060310 0x7fb9cc0a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9cbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9cc060310 0x7fb9cc0a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45622/0 (socket says 172.21.15.116:45622) 2020-11-20T12:17:37.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9cbfff700 1 -- 172.21.15.116:0/1227032272 learned_addr learned my addr 172.21.15.116:0/1227032272 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:37.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d0b04700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc152530 0x7fb9cc0baa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9c3fff700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9cc0c2c30 0x7fb9cc0ba540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d0b04700 1 -- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9cc0c2c30 msgr2=0x7fb9cc0ba540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:37.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d0b04700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9cc0c2c30 0x7fb9cc0ba540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d0b04700 1 -- 172.21.15.116:0/1227032272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9cc060310 msgr2=0x7fb9cc0a42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:37.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d0b04700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9cc060310 0x7fb9cc0a42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:37.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d0b04700 1 -- 172.21.15.116:0/1227032272 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc161550 con 0x7fb9cc152530 2020-11-20T12:17:37.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9cbfff700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9cc060310 0x7fb9cc0a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:37.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d0b04700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc152530 0x7fb9cc0baa30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb9c400ebd0 tx=0x7fb9c4008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:37.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9c8ff9700 1 -- 172.21.15.116:0/1227032272 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400c670 con 0x7fb9cc152530 2020-11-20T12:17:37.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9c8ff9700 1 -- 172.21.15.116:0/1227032272 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c400c7d0 con 0x7fb9cc152530 2020-11-20T12:17:37.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9c8ff9700 1 -- 172.21.15.116:0/1227032272 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400cb50 con 0x7fb9cc152530 2020-11-20T12:17:37.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d1305700 1 -- 172.21.15.116:0/1227032272 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc1617f0 con 0x7fb9cc152530 2020-11-20T12:17:37.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.748+0000 7fb9d1305700 1 -- 172.21.15.116:0/1227032272 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc161020 con 0x7fb9cc152530 2020-11-20T12:17:37.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.752+0000 7fb9d1305700 1 -- 172.21.15.116:0/1227032272 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc044f60 con 0x7fb9cc152530 2020-11-20T12:17:37.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.752+0000 7fb9c8ff9700 1 -- 172.21.15.116:0/1227032272 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb9c401d460 con 0x7fb9cc152530 2020-11-20T12:17:37.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.752+0000 7fb9c8ff9700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a80557c0 0x7fb9a8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:37.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.752+0000 7fb9c8ff9700 1 -- 172.21.15.116:0/1227032272 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb9c406e310 con 0x7fb9cc152530 2020-11-20T12:17:37.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.756+0000 7fb9cbfff700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a80557c0 0x7fb9a8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:37.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.756+0000 7fb9cbfff700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a80557c0 0x7fb9a8057c80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fb9bc011310 tx=0x7fb9bc006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:37.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.764+0000 7fb9c8ff9700 1 -- 172.21.15.116:0/1227032272 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb9c4044370 con 0x7fb9cc152530 2020-11-20T12:17:37.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:37.924+0000 7fb9d1305700 1 -- 172.21.15.116:0/1227032272 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb9cc0c4a20 con 0x7fb9cc152530 2020-11-20T12:17:38.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.520+0000 7fb9c8ff9700 1 -- 172.21.15.116:0/1227032272 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v39) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb9c4047990 con 0x7fb9cc152530 2020-11-20T12:17:38.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 -- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a80557c0 msgr2=0x7fb9a8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:38.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a80557c0 0x7fb9a8057c80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fb9bc011310 tx=0x7fb9bc006df0).stop 2020-11-20T12:17:38.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 -- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc152530 msgr2=0x7fb9cc0baa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:38.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc152530 0x7fb9cc0baa30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fb9c400ebd0 tx=0x7fb9c4008040).stop 2020-11-20T12:17:38.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 -- 172.21.15.116:0/1227032272 shutdown_connections 2020-11-20T12:17:38.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9a80557c0 0x7fb9a8057c80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:38.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9cc152530 0x7fb9cc0baa30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:38.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9cc060310 0x7fb9cc0a42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:38.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 --2- 172.21.15.116:0/1227032272 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9cc0c2c30 0x7fb9cc0ba540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:38.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 -- 172.21.15.116:0/1227032272 >> 172.21.15.116:0/1227032272 conn(0x7fb9cc061d50 msgr2=0x7fb9cc137a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:38.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 -- 172.21.15.116:0/1227032272 shutdown_connections 2020-11-20T12:17:38.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.524+0000 7fb9d1305700 1 -- 172.21.15.116:0/1227032272 wait complete. 2020-11-20T12:17:38.527 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2020-11-20T12:17:38.549 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-20T12:17:38.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.740+0000 7fa94cd63700 1 Processor -- start 2020-11-20T12:17:38.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.740+0000 7fa94cd63700 1 -- start start 2020-11-20T12:17:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.740+0000 7fa94cd63700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa94813b110 0x7fa948151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:38.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.740+0000 7fa94cd63700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa948152a80 con 0x7fa948151fa0 2020-11-20T12:17:38.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.740+0000 7fa94cd63700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa948152540 con 0x7fa94813ad30 2020-11-20T12:17:38.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.740+0000 7fa94cd63700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9481527e0 con 0x7fa94813b110 2020-11-20T12:17:38.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.740+0000 7fa947fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa948151fa0 0x7fa948061250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55062/0 (socket says 172.21.15.116:55062) 2020-11-20T12:17:38.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.740+0000 7fa947fff700 1 -- 172.21.15.116:0/1030719421 learned_addr learned my addr 172.21.15.116:0/1030719421 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:38.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa946ffd700 1 --2- 172.21.15.116:0/1030719421 >> v2:172.21.15.116:3300/0 conn(0x7fa94813b110 0x7fa948151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:38.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3146268563 0 0) 0x7fa948152a80 con 0x7fa948151fa0 2020-11-20T12:17:38.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa934003240 con 0x7fa948151fa0 2020-11-20T12:17:38.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1561651625 0 0) 0x7fa934003240 con 0x7fa948151fa0 2020-11-20T12:17:38.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 >> v2:172.21.15.116:3300/0 conn(0x7fa94813b110 msgr2=0x7fa948151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:38.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 --2- 172.21.15.116:0/1030719421 >> v2:172.21.15.116:3300/0 conn(0x7fa94813b110 0x7fa948151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:38.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 >> v1:172.21.15.173:6789/0 conn(0x7fa94813ad30 legacy=0x7fa9480a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:38.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9480ac910 con 0x7fa948151fa0 2020-11-20T12:17:38.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fa940001860 con 0x7fa948151fa0 2020-11-20T12:17:38.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 >> v1:172.21.15.116:6789/0 conn(0x7fa948151fa0 legacy=0x7fa948061250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:38.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa934003780 0x7fa934003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:38.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9340044e0 0x7fa934004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:38.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa934004e50 0x7fa93400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:38.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9481527e0 con 0x7fa9340044e0 2020-11-20T12:17:38.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa934005300 con 0x7fa934003780 2020-11-20T12:17:38.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93400bc20 con 0x7fa934004e50 2020-11-20T12:17:38.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa946ffd700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9340044e0 0x7fa934004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:38.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9477fe700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa934003780 0x7fa934003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:38.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa947fff700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa934004e50 0x7fa93400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:38.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa946ffd700 1 -- 172.21.15.116:0/1030719421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa934004e50 msgr2=0x7fa93400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:38.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa946ffd700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa934004e50 0x7fa93400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:38.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa946ffd700 1 -- 172.21.15.116:0/1030719421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa934003780 msgr2=0x7fa934003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:38.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa946ffd700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa934003780 0x7fa934003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:38.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa946ffd700 1 -- 172.21.15.116:0/1030719421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa934003240 con 0x7fa9340044e0 2020-11-20T12:17:38.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa947fff700 1 -- 172.21.15.116:0/1030719421 reap_dead start 2020-11-20T12:17:38.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa9477fe700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa934003780 0x7fa934003ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:38.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.744+0000 7fa946ffd700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9340044e0 0x7fa934004960 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fa93c006fd0 tx=0x7fa93c006df0).ready entity=mon.0 client_cookie=d224ae9e03f8bbb7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:38.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.748+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00b310 con 0x7fa9340044e0 2020-11-20T12:17:38.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.748+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c00e970 con 0x7fa9340044e0 2020-11-20T12:17:38.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.748+0000 7fa9467fc700 1 -- 172.21.15.116:0/1030719421 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c00ecf0 con 0x7fa9340044e0 2020-11-20T12:17:38.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.748+0000 7fa94cd63700 1 -- 172.21.15.116:0/1030719421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9340044e0 msgr2=0x7fa934004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:38.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.748+0000 7fa94cd63700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9340044e0 0x7fa934004960 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fa93c006fd0 tx=0x7fa93c006df0).stop 2020-11-20T12:17:38.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.748+0000 7fa94cd63700 1 -- 172.21.15.116:0/1030719421 shutdown_connections 2020-11-20T12:17:38.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.748+0000 7fa94cd63700 1 --2- 172.21.15.116:0/1030719421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9340044e0 0x7fa934004960 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:38.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.748+0000 7fa94cd63700 1 -- 172.21.15.116:0/1030719421 >> 172.21.15.116:0/1030719421 conn(0x7fa9480678a0 msgr2=0x7fa948067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:38.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.748+0000 7fa94cd63700 1 -- 172.21.15.116:0/1030719421 shutdown_connections 2020-11-20T12:17:38.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.748+0000 7fa94cd63700 1 -- 172.21.15.116:0/1030719421 wait complete. 2020-11-20T12:17:38.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa94cd63700 1 Processor -- start 2020-11-20T12:17:38.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa94cd63700 1 -- start start 2020-11-20T12:17:38.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa94cd63700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9480a4a10 0x7fa9480a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:38.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa94cd63700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa948160c70 0x7fa948154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:38.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa94cd63700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa948139010 0x7fa948139470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:38.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa94cd63700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa948139960 con 0x7fa9480a4a10 2020-11-20T12:17:38.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa94cd63700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9480a4df0 con 0x7fa948139010 2020-11-20T12:17:38.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa94cd63700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa948161bc0 con 0x7fa948160c70 2020-11-20T12:17:38.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa9477fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9480a4a10 0x7fa9480a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:38.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa9477fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9480a4a10 0x7fa9480a3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45638/0 (socket says 172.21.15.116:45638) 2020-11-20T12:17:38.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa9477fe700 1 -- 172.21.15.116:0/852411858 learned_addr learned my addr 172.21.15.116:0/852411858 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:38.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa947fff700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa948139010 0x7fa948139470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:38.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa9477fe700 1 -- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa948160c70 msgr2=0x7fa948154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:38.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa9477fe700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa948160c70 0x7fa948154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:38.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa9477fe700 1 -- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa948139010 msgr2=0x7fa948139470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:38.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa9477fe700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa948139010 0x7fa948139470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:38.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.752+0000 7fa9477fe700 1 -- 172.21.15.116:0/852411858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa948161140 con 0x7fa9480a4a10 2020-11-20T12:17:38.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.756+0000 7fa9477fe700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9480a4a10 0x7fa9480a3be0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fa93800fb80 tx=0x7fa938009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:38.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.756+0000 7fa9277fe700 1 -- 172.21.15.116:0/852411858 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93800d8e0 con 0x7fa9480a4a10 2020-11-20T12:17:38.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.756+0000 7fa9277fe700 1 -- 172.21.15.116:0/852411858 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93800da40 con 0x7fa9480a4a10 2020-11-20T12:17:38.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.756+0000 7fa9277fe700 1 -- 172.21.15.116:0/852411858 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93800ddc0 con 0x7fa9480a4a10 2020-11-20T12:17:38.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.756+0000 7fa94cd63700 1 -- 172.21.15.116:0/852411858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9481613e0 con 0x7fa9480a4a10 2020-11-20T12:17:38.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.756+0000 7fa94cd63700 1 -- 172.21.15.116:0/852411858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa948161920 con 0x7fa9480a4a10 2020-11-20T12:17:38.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.756+0000 7fa9257fa700 1 -- 172.21.15.116:0/852411858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa918004f90 con 0x7fa9480a4a10 2020-11-20T12:17:38.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.760+0000 7fa9277fe700 1 -- 172.21.15.116:0/852411858 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa93801a030 con 0x7fa9480a4a10 2020-11-20T12:17:38.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.760+0000 7fa9277fe700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa920055810 0x7fa920057cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:38.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.760+0000 7fa9277fe700 1 -- 172.21.15.116:0/852411858 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fa93806de50 con 0x7fa9480a4a10 2020-11-20T12:17:38.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.768+0000 7fa9277fe700 1 -- 172.21.15.116:0/852411858 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa938043ff0 con 0x7fa9480a4a10 2020-11-20T12:17:38.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.768+0000 7fa946ffd700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa920055810 0x7fa920057cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:38.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.768+0000 7fa946ffd700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa920055810 0x7fa920057cd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fa94805f7d0 tx=0x7fa93c019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:38.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:38.920+0000 7fa9257fa700 1 -- 172.21.15.116:0/852411858 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fa918003b00 con 0x7fa9480a4a10 2020-11-20T12:17:39.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.712+0000 7fa9277fe700 1 -- 172.21.15.116:0/852411858 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v43) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa938047610 con 0x7fa9480a4a10 2020-11-20T12:17:39.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.716+0000 7fa9257fa700 1 -- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa920055810 msgr2=0x7fa920057cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:39.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.716+0000 7fa9257fa700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa920055810 0x7fa920057cd0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fa94805f7d0 tx=0x7fa93c019000).stop 2020-11-20T12:17:39.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.716+0000 7fa9257fa700 1 -- 172.21.15.116:0/852411858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9480a4a10 msgr2=0x7fa9480a3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:39.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.716+0000 7fa9257fa700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9480a4a10 0x7fa9480a3be0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7fa93800fb80 tx=0x7fa938009040).stop 2020-11-20T12:17:39.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.720+0000 7fa9257fa700 1 -- 172.21.15.116:0/852411858 shutdown_connections 2020-11-20T12:17:39.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.720+0000 7fa9257fa700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa920055810 0x7fa920057cd0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:39.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.720+0000 7fa9257fa700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa948139010 0x7fa948139470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:39.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.720+0000 7fa9257fa700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9480a4a10 0x7fa9480a3be0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:39.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.720+0000 7fa9257fa700 1 --2- 172.21.15.116:0/852411858 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa948160c70 0x7fa948154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:39.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.720+0000 7fa9257fa700 1 -- 172.21.15.116:0/852411858 >> 172.21.15.116:0/852411858 conn(0x7fa9480678a0 msgr2=0x7fa9480c7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:39.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.720+0000 7fa9257fa700 1 -- 172.21.15.116:0/852411858 shutdown_connections 2020-11-20T12:17:39.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.720+0000 7fa9257fa700 1 -- 172.21.15.116:0/852411858 wait complete. 2020-11-20T12:17:39.723 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2020-11-20T12:17:39.744 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T12:17:39.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.920+0000 7fc315790700 1 Processor -- start 2020-11-20T12:17:39.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.920+0000 7fc315790700 1 -- start start 2020-11-20T12:17:39.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.924+0000 7fc315790700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3101503a0 0x7fc310065f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:39.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.924+0000 7fc315790700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3100a3400 con 0x7fc310149760 2020-11-20T12:17:39.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.924+0000 7fc315790700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3100a36a0 con 0x7fc3100c4800 2020-11-20T12:17:39.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.924+0000 7fc315790700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc310150850 con 0x7fc3101503a0 2020-11-20T12:17:39.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.924+0000 7fc30ffff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc310149760 0x7fc3100a2b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55084/0 (socket says 172.21.15.116:55084) 2020-11-20T12:17:39.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.924+0000 7fc30ffff700 1 -- 172.21.15.116:0/3114125090 learned_addr learned my addr 172.21.15.116:0/3114125090 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:39.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.924+0000 7fc314f8f700 1 --2- 172.21.15.116:0/3114125090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3101503a0 0x7fc310065f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:39.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.924+0000 7fc30f7fe700 1 -- 172.21.15.116:0/3114125090 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1978718627 0 0) 0x7fc3100a3400 con 0x7fc310149760 2020-11-20T12:17:39.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.924+0000 7fc30f7fe700 1 -- 172.21.15.116:0/3114125090 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2f0003240 con 0x7fc310149760 2020-11-20T12:17:39.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.924+0000 7fc30f7fe700 1 -- 172.21.15.116:0/3114125090 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1135441605 0 0) 0x7fc3100a36a0 con 0x7fc3100c4800 2020-11-20T12:17:39.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.924+0000 7fc30f7fe700 1 -- 172.21.15.116:0/3114125090 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3100a3400 con 0x7fc3100c4800 2020-11-20T12:17:39.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc314f8f700 1 -- 172.21.15.116:0/3114125090 >> v1:172.21.15.173:6789/0 conn(0x7fc3100c4800 legacy=0x7fc310067340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:39.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc314f8f700 1 -- 172.21.15.116:0/3114125090 >> v1:172.21.15.116:6789/0 conn(0x7fc310149760 legacy=0x7fc3100a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:39.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc314f8f700 1 -- 172.21.15.116:0/3114125090 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31014f9c0 con 0x7fc3101503a0 2020-11-20T12:17:39.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc314f8f700 1 --2- 172.21.15.116:0/3114125090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3101503a0 0x7fc310065f40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fc308009cc0 tx=0x7fc3080091d0).ready entity=mon.2 client_cookie=9c23ae63e490009c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:39.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc30f7fe700 1 -- 172.21.15.116:0/3114125090 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30800bab0 con 0x7fc3101503a0 2020-11-20T12:17:39.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc30f7fe700 1 -- 172.21.15.116:0/3114125090 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30800bc10 con 0x7fc3101503a0 2020-11-20T12:17:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc30f7fe700 1 -- 172.21.15.116:0/3114125090 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc308011400 con 0x7fc3101503a0 2020-11-20T12:17:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc315790700 1 -- 172.21.15.116:0/3114125090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3101503a0 msgr2=0x7fc310065f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc315790700 1 --2- 172.21.15.116:0/3114125090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3101503a0 0x7fc310065f40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fc308009cc0 tx=0x7fc3080091d0).stop 2020-11-20T12:17:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc315790700 1 -- 172.21.15.116:0/3114125090 shutdown_connections 2020-11-20T12:17:39.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc315790700 1 --2- 172.21.15.116:0/3114125090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3101503a0 0x7fc310065f40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:39.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc315790700 1 -- 172.21.15.116:0/3114125090 >> 172.21.15.116:0/3114125090 conn(0x7fc310060c40 msgr2=0x7fc3100c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:39.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc315790700 1 -- 172.21.15.116:0/3114125090 shutdown_connections 2020-11-20T12:17:39.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.928+0000 7fc315790700 1 -- 172.21.15.116:0/3114125090 wait complete. 2020-11-20T12:17:39.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc315790700 1 Processor -- start 2020-11-20T12:17:39.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc315790700 1 -- start start 2020-11-20T12:17:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc315790700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3100c4800 0x7fc31008f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc315790700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc310149760 0x7fc310164b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc315790700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3101503a0 0x7fc310090110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc315790700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc310165060 con 0x7fc3100c4800 2020-11-20T12:17:39.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc315790700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3101610f0 con 0x7fc310149760 2020-11-20T12:17:39.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc315790700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc310161390 con 0x7fc3101503a0 2020-11-20T12:17:39.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc30ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3100c4800 0x7fc31008f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:39.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc30ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3100c4800 0x7fc31008f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45656/0 (socket says 172.21.15.116:45656) 2020-11-20T12:17:39.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc30ffff700 1 -- 172.21.15.116:0/3455006642 learned_addr learned my addr 172.21.15.116:0/3455006642 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:39.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc314f8f700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3101503a0 0x7fc310090110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:39.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.932+0000 7fc307fff700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc310149760 0x7fc310164b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:39.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc30ffff700 1 -- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3101503a0 msgr2=0x7fc310090110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:39.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc30ffff700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3101503a0 0x7fc310090110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:39.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc30ffff700 1 -- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc310149760 msgr2=0x7fc310164b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:39.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc30ffff700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc310149760 0x7fc310164b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:39.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc30ffff700 1 -- 172.21.15.116:0/3455006642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc310161630 con 0x7fc3100c4800 2020-11-20T12:17:39.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc307fff700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc310149760 0x7fc310164b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:39.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc30ffff700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3100c4800 0x7fc31008f4f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc3000055a0 tx=0x7fc300007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:39.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc30cff9700 1 -- 172.21.15.116:0/3455006642 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3000072e0 con 0x7fc3100c4800 2020-11-20T12:17:39.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc30cff9700 1 -- 172.21.15.116:0/3455006642 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc300007440 con 0x7fc3100c4800 2020-11-20T12:17:39.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc315790700 1 -- 172.21.15.116:0/3455006642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3101618d0 con 0x7fc3100c4800 2020-11-20T12:17:39.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc30cff9700 1 -- 172.21.15.116:0/3455006642 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc300005800 con 0x7fc3100c4800 2020-11-20T12:17:39.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc315790700 1 -- 172.21.15.116:0/3455006642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc310091310 con 0x7fc3100c4800 2020-11-20T12:17:39.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.936+0000 7fc315790700 1 -- 172.21.15.116:0/3455006642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc310044f60 con 0x7fc3100c4800 2020-11-20T12:17:39.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.940+0000 7fc30cff9700 1 -- 172.21.15.116:0/3455006642 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc30001a060 con 0x7fc3100c4800 2020-11-20T12:17:39.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.940+0000 7fc30cff9700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2ec0557c0 0x7fc2ec057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:39.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.940+0000 7fc30cff9700 1 -- 172.21.15.116:0/3455006642 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fc30006de60 con 0x7fc3100c4800 2020-11-20T12:17:39.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.940+0000 7fc307fff700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2ec0557c0 0x7fc2ec057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:39.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.940+0000 7fc307fff700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2ec0557c0 0x7fc2ec057c80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fc310152620 tx=0x7fc2f8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:39.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:39.948+0000 7fc30cff9700 1 -- 172.21.15.116:0/3455006642 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc300043d70 con 0x7fc3100c4800 2020-11-20T12:17:40.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.160+0000 7fc315790700 1 -- 172.21.15.116:0/3455006642 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fc31014f630 con 0x7fc3100c4800 2020-11-20T12:17:40.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.168+0000 7fc30cff9700 1 -- 172.21.15.116:0/3455006642 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 7 and data pool 8 v18) v1 ==== 187+0+0 (secure 0 0 0) 0x7fc300018070 con 0x7fc3100c4800 2020-11-20T12:17:40.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.176+0000 7fc315790700 1 -- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2ec0557c0 msgr2=0x7fc2ec057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.176+0000 7fc315790700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2ec0557c0 0x7fc2ec057c80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fc310152620 tx=0x7fc2f8009040).stop 2020-11-20T12:17:40.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.176+0000 7fc315790700 1 -- 172.21.15.116:0/3455006642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3100c4800 msgr2=0x7fc31008f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.176+0000 7fc315790700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3100c4800 0x7fc31008f4f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc3000055a0 tx=0x7fc300007b70).stop 2020-11-20T12:17:40.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.176+0000 7fc315790700 1 -- 172.21.15.116:0/3455006642 shutdown_connections 2020-11-20T12:17:40.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.176+0000 7fc315790700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2ec0557c0 0x7fc2ec057c80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.176+0000 7fc315790700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc3101503a0 0x7fc310090110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.176+0000 7fc315790700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc3100c4800 0x7fc31008f4f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.176+0000 7fc315790700 1 --2- 172.21.15.116:0/3455006642 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc310149760 0x7fc310164b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.176+0000 7fc315790700 1 -- 172.21.15.116:0/3455006642 >> 172.21.15.116:0/3455006642 conn(0x7fc310060c40 msgr2=0x7fc3100c0340 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:40.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.180+0000 7fc315790700 1 -- 172.21.15.116:0/3455006642 shutdown_connections 2020-11-20T12:17:40.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.180+0000 7fc315790700 1 -- 172.21.15.116:0/3455006642 wait complete. 2020-11-20T12:17:40.190 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 7 and data pool 8 2020-11-20T12:17:40.212 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:40.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.456+0000 7fc205654700 1 Processor -- start 2020-11-20T12:17:40.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.456+0000 7fc205654700 1 -- start start 2020-11-20T12:17:40.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.456+0000 7fc205654700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc2001518e0 0x7fc2000a34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:40.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc20014dde0 0x7fc200152140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:40.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc200152630 con 0x7fc200151cc0 2020-11-20T12:17:40.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2000a39c0 con 0x7fc2001518e0 2020-11-20T12:17:40.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2000bf360 con 0x7fc20014dde0 2020-11-20T12:17:40.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1f7fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc200151cc0 0x7fc20014d6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55112/0 (socket says 172.21.15.116:55112) 2020-11-20T12:17:40.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1feffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc2001518e0 0x7fc2000a34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:40.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1f7fff700 1 -- 172.21.15.116:0/1818906165 learned_addr learned my addr 172.21.15.116:0/1818906165 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:40.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1ff7fe700 1 --2- 172.21.15.116:0/1818906165 >> v2:172.21.15.173:3300/0 conn(0x7fc20014dde0 0x7fc200152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:40.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1feffd700 1 -- 172.21.15.116:0/1818906165 >> v2:172.21.15.173:3300/0 conn(0x7fc20014dde0 msgr2=0x7fc200152140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:40.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1feffd700 1 --2- 172.21.15.116:0/1818906165 >> v2:172.21.15.173:3300/0 conn(0x7fc20014dde0 0x7fc200152140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1feffd700 1 -- 172.21.15.116:0/1818906165 >> v1:172.21.15.116:6789/0 conn(0x7fc200151cc0 legacy=0x7fc20014d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1feffd700 1 -- 172.21.15.116:0/1818906165 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2000bf600 con 0x7fc2001518e0 2020-11-20T12:17:40.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1feffd700 1 --2- 172.21.15.116:0/1818906165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2001518e0 0x7fc2000a34d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fc2000c0a60 tx=0x7fc1f0006e80).ready entity=mon.0 client_cookie=d8870c70326a989d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:40.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1fe7fc700 1 -- 172.21.15.116:0/1818906165 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0014070 con 0x7fc2001518e0 2020-11-20T12:17:40.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1fe7fc700 1 -- 172.21.15.116:0/1818906165 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f000a840 con 0x7fc2001518e0 2020-11-20T12:17:40.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc1fe7fc700 1 -- 172.21.15.116:0/1818906165 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f000abc0 con 0x7fc2001518e0 2020-11-20T12:17:40.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 -- 172.21.15.116:0/1818906165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2001518e0 msgr2=0x7fc2000a34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 --2- 172.21.15.116:0/1818906165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2001518e0 0x7fc2000a34d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7fc2000c0a60 tx=0x7fc1f0006e80).stop 2020-11-20T12:17:40.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 -- 172.21.15.116:0/1818906165 shutdown_connections 2020-11-20T12:17:40.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 --2- 172.21.15.116:0/1818906165 >> v2:172.21.15.173:3300/0 conn(0x7fc20014dde0 0x7fc200152140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 --2- 172.21.15.116:0/1818906165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2001518e0 0x7fc2000a34d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 -- 172.21.15.116:0/1818906165 >> 172.21.15.116:0/1818906165 conn(0x7fc200060c80 msgr2=0x7fc2000c41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:40.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 -- 172.21.15.116:0/1818906165 shutdown_connections 2020-11-20T12:17:40.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.460+0000 7fc205654700 1 -- 172.21.15.116:0/1818906165 wait complete. 2020-11-20T12:17:40.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.464+0000 7fc205654700 1 Processor -- start 2020-11-20T12:17:40.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.464+0000 7fc205654700 1 -- start start 2020-11-20T12:17:40.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc205654700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc20014dde0 0x7fc2001625c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:40.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc205654700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2001518e0 0x7fc20014bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:40.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc205654700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc200151cc0 0x7fc20014e240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:40.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc205654700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc200162ab0 con 0x7fc20014dde0 2020-11-20T12:17:40.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc205654700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc20014f160 con 0x7fc200151cc0 2020-11-20T12:17:40.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc205654700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc20014f400 con 0x7fc2001518e0 2020-11-20T12:17:40.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1feffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc20014dde0 0x7fc2001625c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:40.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1feffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc20014dde0 0x7fc2001625c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45682/0 (socket says 172.21.15.116:45682) 2020-11-20T12:17:40.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1feffd700 1 -- 172.21.15.116:0/2821203691 learned_addr learned my addr 172.21.15.116:0/2821203691 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:40.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1ff7fe700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc200151cc0 0x7fc20014e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:40.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1f7fff700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2001518e0 0x7fc20014bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:40.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1feffd700 1 -- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2001518e0 msgr2=0x7fc20014bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1feffd700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2001518e0 0x7fc20014bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1feffd700 1 -- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc200151cc0 msgr2=0x7fc20014e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1feffd700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc200151cc0 0x7fc20014e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1feffd700 1 -- 172.21.15.116:0/2821203691 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc20014f6a0 con 0x7fc20014dde0 2020-11-20T12:17:40.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1ff7fe700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc200151cc0 0x7fc20014e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:40.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1feffd700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc20014dde0 0x7fc2001625c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fc1f000d010 tx=0x7fc1f0006e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:40.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1f6ffd700 1 -- 172.21.15.116:0/2821203691 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f001c440 con 0x7fc20014dde0 2020-11-20T12:17:40.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1f6ffd700 1 -- 172.21.15.116:0/2821203691 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1f0020070 con 0x7fc20014dde0 2020-11-20T12:17:40.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc205654700 1 -- 172.21.15.116:0/2821203691 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc20014f940 con 0x7fc20014dde0 2020-11-20T12:17:40.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc1f6ffd700 1 -- 172.21.15.116:0/2821203691 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1f001c5a0 con 0x7fc20014dde0 2020-11-20T12:17:40.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.468+0000 7fc205654700 1 -- 172.21.15.116:0/2821203691 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc20014c4a0 con 0x7fc20014dde0 2020-11-20T12:17:40.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.472+0000 7fc205654700 1 -- 172.21.15.116:0/2821203691 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc20005ca30 con 0x7fc20014dde0 2020-11-20T12:17:40.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.472+0000 7fc1f6ffd700 1 -- 172.21.15.116:0/2821203691 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc1f000e040 con 0x7fc20014dde0 2020-11-20T12:17:40.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.472+0000 7fc1f6ffd700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1dc0557c0 0x7fc1dc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:40.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.472+0000 7fc1f6ffd700 1 -- 172.21.15.116:0/2821203691 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc1f0070000 con 0x7fc20014dde0 2020-11-20T12:17:40.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.472+0000 7fc1f7fff700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1dc0557c0 0x7fc1dc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:40.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.476+0000 7fc1f7fff700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1dc0557c0 0x7fc1dc057c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fc200150d70 tx=0x7fc1e8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:40.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.480+0000 7fc1f6ffd700 1 -- 172.21.15.116:0/2821203691 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc1f0045ea0 con 0x7fc20014dde0 2020-11-20T12:17:40.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.640+0000 7fc205654700 1 -- 172.21.15.116:0/2821203691 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2000682e0 con 0x7fc20014dde0 2020-11-20T12:17:40.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc1f6ffd700 1 -- 172.21.15.116:0/2821203691 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16887 (secure 0 0 0) 0x7fc1f00494c0 con 0x7fc20014dde0 2020-11-20T12:17:40.643 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:40.644 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":44,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:17:40.163067+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:17:38.002878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-20T12:17:38.995456+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:40.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 -- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1dc0557c0 msgr2=0x7fc1dc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1dc0557c0 0x7fc1dc057c80 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fc200150d70 tx=0x7fc1e8009040).stop 2020-11-20T12:17:40.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 -- 172.21.15.116:0/2821203691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc20014dde0 msgr2=0x7fc2001625c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc20014dde0 0x7fc2001625c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7fc1f000d010 tx=0x7fc1f0006e80).stop 2020-11-20T12:17:40.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 -- 172.21.15.116:0/2821203691 shutdown_connections 2020-11-20T12:17:40.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1dc0557c0 0x7fc1dc057c80 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc200151cc0 0x7fc20014e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc20014dde0 0x7fc2001625c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 --2- 172.21.15.116:0/2821203691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2001518e0 0x7fc20014bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 -- 172.21.15.116:0/2821203691 >> 172.21.15.116:0/2821203691 conn(0x7fc200060c80 msgr2=0x7fc200067b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:40.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 -- 172.21.15.116:0/2821203691 shutdown_connections 2020-11-20T12:17:40.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.644+0000 7fc205654700 1 -- 172.21.15.116:0/2821203691 wait complete. 2020-11-20T12:17:40.670 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:40.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.836+0000 7f0e12f62700 1 Processor -- start 2020-11-20T12:17:40.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.836+0000 7f0e12f62700 1 -- start start 2020-11-20T12:17:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e12f62700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e0c0aa790 0x7f0e0c152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e12f62700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0c152a30 con 0x7f0e0c13aee0 2020-11-20T12:17:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e12f62700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e0c0aac10 con 0x7f0e0c0612f0 2020-11-20T12:17:40.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e12f62700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c0adce0 con 0x7f0e0c0aa790 2020-11-20T12:17:40.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e11f60700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0e0c13aee0 0x7f0e0c0ad5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55122/0 (socket says 172.21.15.116:55122) 2020-11-20T12:17:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e11f60700 1 -- 172.21.15.116:0/3546235587 learned_addr learned my addr 172.21.15.116:0/3546235587 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e1175f700 1 --2- 172.21.15.116:0/3546235587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e0c0aa790 0x7f0e0c152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e10f5e700 1 -- 172.21.15.116:0/3546235587 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3543233324 0 0) 0x7f0e0c152a30 con 0x7f0e0c13aee0 2020-11-20T12:17:40.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e10f5e700 1 -- 172.21.15.116:0/3546235587 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0df4003240 con 0x7f0e0c13aee0 2020-11-20T12:17:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e10f5e700 1 -- 172.21.15.116:0/3546235587 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2968209279 0 0) 0x7f0e0c0aac10 con 0x7f0e0c0612f0 2020-11-20T12:17:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e1175f700 1 -- 172.21.15.116:0/3546235587 >> v1:172.21.15.173:6789/0 conn(0x7f0e0c0612f0 legacy=0x7f0e0c0a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e1175f700 1 -- 172.21.15.116:0/3546235587 >> v1:172.21.15.116:6789/0 conn(0x7f0e0c13aee0 legacy=0x7f0e0c0ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e1175f700 1 -- 172.21.15.116:0/3546235587 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c0aaeb0 con 0x7f0e0c0aa790 2020-11-20T12:17:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e1175f700 1 --2- 172.21.15.116:0/3546235587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e0c0aa790 0x7f0e0c152540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f0dfc00b9d0 tx=0x7f0dfc006df0).ready entity=mon.2 client_cookie=43f078e83162f759 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:40.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.840+0000 7f0e10f5e700 1 -- 172.21.15.116:0/3546235587 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc0097b0 con 0x7f0e0c0aa790 2020-11-20T12:17:40.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.844+0000 7f0e10f5e700 1 -- 172.21.15.116:0/3546235587 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dfc009910 con 0x7f0e0c0aa790 2020-11-20T12:17:40.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.844+0000 7f0e10f5e700 1 -- 172.21.15.116:0/3546235587 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc009c90 con 0x7f0e0c0aa790 2020-11-20T12:17:40.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.844+0000 7f0e12f62700 1 -- 172.21.15.116:0/3546235587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e0c0aa790 msgr2=0x7f0e0c152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.844+0000 7f0e12f62700 1 --2- 172.21.15.116:0/3546235587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e0c0aa790 0x7f0e0c152540 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f0dfc00b9d0 tx=0x7f0dfc006df0).stop 2020-11-20T12:17:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.844+0000 7f0e12f62700 1 -- 172.21.15.116:0/3546235587 shutdown_connections 2020-11-20T12:17:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.844+0000 7f0e12f62700 1 --2- 172.21.15.116:0/3546235587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e0c0aa790 0x7f0e0c152540 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.844+0000 7f0e12f62700 1 -- 172.21.15.116:0/3546235587 >> 172.21.15.116:0/3546235587 conn(0x7f0e0c0678a0 msgr2=0x7f0e0c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.844+0000 7f0e12f62700 1 -- 172.21.15.116:0/3546235587 shutdown_connections 2020-11-20T12:17:40.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.844+0000 7f0e12f62700 1 -- 172.21.15.116:0/3546235587 wait complete. 2020-11-20T12:17:40.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.844+0000 7f0e12f62700 1 Processor -- start 2020-11-20T12:17:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.844+0000 7f0e12f62700 1 -- start start 2020-11-20T12:17:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e12f62700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c0612f0 0x7f0e0c0bdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e12f62700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e0c13aee0 0x7f0e0c1391c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e12f62700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e0c1396b0 0x7f0e0c150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:40.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e12f62700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c0be280 con 0x7f0e0c0612f0 2020-11-20T12:17:40.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e12f62700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c0c0670 con 0x7f0e0c13aee0 2020-11-20T12:17:40.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e12f62700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e0c151ce0 con 0x7f0e0c1396b0 2020-11-20T12:17:40.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e11f60700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c0612f0 0x7f0e0c0bdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:40.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e11f60700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c0612f0 0x7f0e0c0bdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45696/0 (socket says 172.21.15.116:45696) 2020-11-20T12:17:40.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e11f60700 1 -- 172.21.15.116:0/3233048298 learned_addr learned my addr 172.21.15.116:0/3233048298 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e1175f700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e0c13aee0 0x7f0e0c1391c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e12761700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e0c1396b0 0x7f0e0c150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e11f60700 1 -- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e0c1396b0 msgr2=0x7f0e0c150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e11f60700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e0c1396b0 0x7f0e0c150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e11f60700 1 -- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e0c13aee0 msgr2=0x7f0e0c1391c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e11f60700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e0c13aee0 0x7f0e0c1391c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e11f60700 1 -- 172.21.15.116:0/3233048298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c151f80 con 0x7f0e0c0612f0 2020-11-20T12:17:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e1175f700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e0c13aee0 0x7f0e0c1391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:40.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e11f60700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c0612f0 0x7f0e0c0bdd90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f0e0800c6f0 tx=0x7f0e08007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:40.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0df9ffb700 1 -- 172.21.15.116:0/3233048298 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e080072e0 con 0x7f0e0c0612f0 2020-11-20T12:17:40.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0e12f62700 1 -- 172.21.15.116:0/3233048298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e0c152220 con 0x7f0e0c0612f0 2020-11-20T12:17:40.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0df9ffb700 1 -- 172.21.15.116:0/3233048298 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e08007440 con 0x7f0e0c0612f0 2020-11-20T12:17:40.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.848+0000 7f0df9ffb700 1 -- 172.21.15.116:0/3233048298 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08005800 con 0x7f0e0c0612f0 2020-11-20T12:17:40.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.852+0000 7f0e12f62700 1 -- 172.21.15.116:0/3233048298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e0c0c0300 con 0x7f0e0c0612f0 2020-11-20T12:17:40.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.852+0000 7f0e12f62700 1 -- 172.21.15.116:0/3233048298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0c1523b0 con 0x7f0e0c0612f0 2020-11-20T12:17:40.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.852+0000 7f0df9ffb700 1 -- 172.21.15.116:0/3233048298 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0e08010070 con 0x7f0e0c0612f0 2020-11-20T12:17:40.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.852+0000 7f0df9ffb700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0de40557c0 0x7f0de4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:40.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.852+0000 7f0df9ffb700 1 -- 172.21.15.116:0/3233048298 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0e0806dec0 con 0x7f0e0c0612f0 2020-11-20T12:17:40.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.856+0000 7f0e1175f700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0de40557c0 0x7f0de4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:40.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.856+0000 7f0e1175f700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0de40557c0 0x7f0de4057c80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f0e0c062380 tx=0x7f0dfc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:40.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:40.856+0000 7f0df9ffb700 1 -- 172.21.15.116:0/3233048298 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0e08043d60 con 0x7f0e0c0612f0 2020-11-20T12:17:41.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.016+0000 7f0e12f62700 1 -- 172.21.15.116:0/3233048298 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e0c044f60 con 0x7f0e0c0612f0 2020-11-20T12:17:41.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.016+0000 7f0df9ffb700 1 -- 172.21.15.116:0/3233048298 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16887 (secure 0 0 0) 0x7f0e08047380 con 0x7f0e0c0612f0 2020-11-20T12:17:41.017 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:41.018 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":44,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:17:40.163067+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:17:38.002878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-20T12:17:38.995456+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:41.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 -- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0de40557c0 msgr2=0x7f0de4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0de40557c0 0x7f0de4057c80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f0e0c062380 tx=0x7f0dfc015000).stop 2020-11-20T12:17:41.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 -- 172.21.15.116:0/3233048298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c0612f0 msgr2=0x7f0e0c0bdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c0612f0 0x7f0e0c0bdd90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f0e0800c6f0 tx=0x7f0e08007ad0).stop 2020-11-20T12:17:41.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 -- 172.21.15.116:0/3233048298 shutdown_connections 2020-11-20T12:17:41.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0de40557c0 0x7f0de4057c80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e0c1396b0 0x7f0e0c150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e0c0612f0 0x7f0e0c0bdd90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 --2- 172.21.15.116:0/3233048298 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e0c13aee0 0x7f0e0c1391c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 -- 172.21.15.116:0/3233048298 >> 172.21.15.116:0/3233048298 conn(0x7f0e0c0678a0 msgr2=0x7f0e0c0c6020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:41.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 -- 172.21.15.116:0/3233048298 shutdown_connections 2020-11-20T12:17:41.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.020+0000 7f0e12f62700 1 -- 172.21.15.116:0/3233048298 wait complete. 2020-11-20T12:17:41.043 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-20T12:17:41.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.208+0000 7fec52ec2700 1 Processor -- start 2020-11-20T12:17:41.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec52ec2700 1 -- start start 2020-11-20T12:17:41.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec52ec2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fec4c0a0f00 0x7fec4c0a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec52ec2700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fec4c0a12e0 0x7fec4c150370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec52ec2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4c0a50a0 con 0x7fec4c0c2df0 2020-11-20T12:17:41.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec52ec2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4c0a5340 con 0x7fec4c0a0f00 2020-11-20T12:17:41.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec52ec2700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec4c0a55e0 con 0x7fec4c0a12e0 2020-11-20T12:17:41.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec516bf700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fec4c0a12e0 0x7fec4c150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec51ec0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fec4c0a0f00 0x7fec4c0a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec516bf700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fec4c0a12e0 0x7fec4c150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:44756/0 (socket says 172.21.15.116:44756) 2020-11-20T12:17:41.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec516bf700 1 -- 172.21.15.116:0/3474202896 learned_addr learned my addr 172.21.15.116:0/3474202896 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:41.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec526c1700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fec4c0c2df0 0x7fec4c149240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:51370/0 (socket says 172.21.15.116:51370) 2020-11-20T12:17:41.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec50ebe700 1 -- 172.21.15.116:0/3474202896 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 980946580 0 0) 0x7fec4c0a50a0 con 0x7fec4c0c2df0 2020-11-20T12:17:41.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec516bf700 1 -- 172.21.15.116:0/3474202896 >> v2:172.21.15.116:3300/0 conn(0x7fec4c0a0f00 msgr2=0x7fec4c0a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:41.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec516bf700 1 --2- 172.21.15.116:0/3474202896 >> v2:172.21.15.116:3300/0 conn(0x7fec4c0a0f00 0x7fec4c0a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec516bf700 1 -- 172.21.15.116:0/3474202896 >> v1:172.21.15.173:6789/0 conn(0x7fec4c0c2df0 legacy=0x7fec4c149240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec516bf700 1 -- 172.21.15.116:0/3474202896 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c0a5880 con 0x7fec4c0a12e0 2020-11-20T12:17:41.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.212+0000 7fec516bf700 1 --2- 172.21.15.116:0/3474202896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec4c0a12e0 0x7fec4c150370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fec4c0c4770 tx=0x7fec3c006eb0).ready entity=mon.1 client_cookie=79510f948c69431c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.216+0000 7fec50ebe700 1 -- 172.21.15.116:0/3474202896 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c009810 con 0x7fec4c0a12e0 2020-11-20T12:17:41.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.216+0000 7fec50ebe700 1 -- 172.21.15.116:0/3474202896 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c009970 con 0x7fec4c0a12e0 2020-11-20T12:17:41.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.216+0000 7fec50ebe700 1 -- 172.21.15.116:0/3474202896 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c009cf0 con 0x7fec4c0a12e0 2020-11-20T12:17:41.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.216+0000 7fec52ec2700 1 -- 172.21.15.116:0/3474202896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec4c0a12e0 msgr2=0x7fec4c150370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.216+0000 7fec52ec2700 1 --2- 172.21.15.116:0/3474202896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec4c0a12e0 0x7fec4c150370 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fec4c0c4770 tx=0x7fec3c006eb0).stop 2020-11-20T12:17:41.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.216+0000 7fec52ec2700 1 -- 172.21.15.116:0/3474202896 shutdown_connections 2020-11-20T12:17:41.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.216+0000 7fec52ec2700 1 --2- 172.21.15.116:0/3474202896 >> v2:172.21.15.116:3300/0 conn(0x7fec4c0a0f00 0x7fec4c0a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.216+0000 7fec52ec2700 1 --2- 172.21.15.116:0/3474202896 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec4c0a12e0 0x7fec4c150370 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.216+0000 7fec52ec2700 1 -- 172.21.15.116:0/3474202896 >> 172.21.15.116:0/3474202896 conn(0x7fec4c060c30 msgr2=0x7fec4c0c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:41.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.216+0000 7fec52ec2700 1 -- 172.21.15.116:0/3474202896 shutdown_connections 2020-11-20T12:17:41.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.216+0000 7fec52ec2700 1 -- 172.21.15.116:0/3474202896 wait complete. 2020-11-20T12:17:41.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.220+0000 7fec52ec2700 1 Processor -- start 2020-11-20T12:17:41.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.220+0000 7fec52ec2700 1 -- start start 2020-11-20T12:17:41.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.220+0000 7fec52ec2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec4c0a0f00 0x7fec4c162cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.220+0000 7fec52ec2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c0a12e0 0x7fec4c14b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.220+0000 7fec52ec2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec4c0c2df0 0x7fec4c0c1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.220+0000 7fec52ec2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c08f220 con 0x7fec4c0a12e0 2020-11-20T12:17:41.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.220+0000 7fec52ec2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c14a310 con 0x7fec4c0a0f00 2020-11-20T12:17:41.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.220+0000 7fec52ec2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec4c08ef80 con 0x7fec4c0c2df0 2020-11-20T12:17:41.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec516bf700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c0a12e0 0x7fec4c14b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec516bf700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c0a12e0 0x7fec4c14b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45710/0 (socket says 172.21.15.116:45710) 2020-11-20T12:17:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec516bf700 1 -- 172.21.15.116:0/1246777960 learned_addr learned my addr 172.21.15.116:0/1246777960 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec51ec0700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec4c0a0f00 0x7fec4c162cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec516bf700 1 -- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec4c0c2df0 msgr2=0x7fec4c0c1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec526c1700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec4c0c2df0 0x7fec4c0c1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec516bf700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec4c0c2df0 0x7fec4c0c1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec516bf700 1 -- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec4c0a0f00 msgr2=0x7fec4c162cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:41.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec516bf700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec4c0a0f00 0x7fec4c162cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:41.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec516bf700 1 -- 172.21.15.116:0/1246777960 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec4c08f500 con 0x7fec4c0a12e0 2020-11-20T12:17:41.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec526c1700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec4c0c2df0 0x7fec4c0c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:41.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec516bf700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c0a12e0 0x7fec4c14b730 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fec3c006ef0 tx=0x7fec3c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec39ffb700 1 -- 172.21.15.116:0/1246777960 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c011e70 con 0x7fec4c0a12e0 2020-11-20T12:17:41.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec39ffb700 1 -- 172.21.15.116:0/1246777960 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c01e460 con 0x7fec4c0a12e0 2020-11-20T12:17:41.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec52ec2700 1 -- 172.21.15.116:0/1246777960 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4c08f7a0 con 0x7fec4c0a12e0 2020-11-20T12:17:41.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec39ffb700 1 -- 172.21.15.116:0/1246777960 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c01e7e0 con 0x7fec4c0a12e0 2020-11-20T12:17:41.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec52ec2700 1 -- 172.21.15.116:0/1246777960 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c163220 con 0x7fec4c0a12e0 2020-11-20T12:17:41.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.224+0000 7fec52ec2700 1 -- 172.21.15.116:0/1246777960 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec4c044f60 con 0x7fec4c0a12e0 2020-11-20T12:17:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.228+0000 7fec39ffb700 1 -- 172.21.15.116:0/1246777960 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fec3c01a040 con 0x7fec4c0a12e0 2020-11-20T12:17:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.228+0000 7fec39ffb700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec240557f0 0x7fec24057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.228+0000 7fec39ffb700 1 -- 172.21.15.116:0/1246777960 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec3c06f9a0 con 0x7fec4c0a12e0 2020-11-20T12:17:41.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.228+0000 7fec51ec0700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec240557f0 0x7fec24057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:41.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.232+0000 7fec51ec0700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec240557f0 0x7fec24057cb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fec48009cf0 tx=0x7fec48008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:41.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.236+0000 7fec39ffb700 1 -- 172.21.15.116:0/1246777960 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fec3c0458f0 con 0x7fec4c0a12e0 2020-11-20T12:17:41.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:41.444+0000 7fec52ec2700 1 -- 172.21.15.116:0/1246777960 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fec4c064080 con 0x7fec4c0a12e0 2020-11-20T12:17:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.176+0000 7fec39ffb700 1 -- 172.21.15.116:0/1246777960 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v20) v1 ==== 118+0+0 (secure 0 0 0) 0x7fec3c023070 con 0x7fec4c0a12e0 2020-11-20T12:17:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 -- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec240557f0 msgr2=0x7fec24057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec240557f0 0x7fec24057cb0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fec48009cf0 tx=0x7fec48008040).stop 2020-11-20T12:17:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 -- 172.21.15.116:0/1246777960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c0a12e0 msgr2=0x7fec4c14b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c0a12e0 0x7fec4c14b730 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fec3c006ef0 tx=0x7fec3c015000).stop 2020-11-20T12:17:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 -- 172.21.15.116:0/1246777960 shutdown_connections 2020-11-20T12:17:42.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec240557f0 0x7fec24057cb0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec4c0c2df0 0x7fec4c0c1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec4c0a0f00 0x7fec4c162cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 --2- 172.21.15.116:0/1246777960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec4c0a12e0 0x7fec4c14b730 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 -- 172.21.15.116:0/1246777960 >> 172.21.15.116:0/1246777960 conn(0x7fec4c060c30 msgr2=0x7fec4c149710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:42.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 -- 172.21.15.116:0/1246777960 shutdown_connections 2020-11-20T12:17:42.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.180+0000 7fec52ec2700 1 -- 172.21.15.116:0/1246777960 wait complete. 2020-11-20T12:17:42.207 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.384+0000 7fa77322d700 1 Processor -- start 2020-11-20T12:17:42.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.384+0000 7fa77322d700 1 -- start start 2020-11-20T12:17:42.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.384+0000 7fa77322d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa76c0c45a0 0x7fa76c138670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.384+0000 7fa77322d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa76c138b60 0x7fa76c0a7c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.384+0000 7fa77322d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa76c0a9180 con 0x7fa76c0c41c0 2020-11-20T12:17:42.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.384+0000 7fa77322d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c0a9420 con 0x7fa76c138b60 2020-11-20T12:17:42.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.384+0000 7fa77322d700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa76c0a96c0 con 0x7fa76c0c45a0 2020-11-20T12:17:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa771a2a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa76c0c45a0 0x7fa76c138670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa772a2c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa76c138b60 0x7fa76c0a7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa771a2a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa76c0c45a0 0x7fa76c138670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:44768/0 (socket says 172.21.15.116:44768) 2020-11-20T12:17:42.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa771a2a700 1 -- 172.21.15.116:0/863121290 learned_addr learned my addr 172.21.15.116:0/863121290 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa772a2c700 1 -- 172.21.15.116:0/863121290 >> v2:172.21.15.173:3300/0 conn(0x7fa76c0c45a0 msgr2=0x7fa76c138670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa772a2c700 1 --2- 172.21.15.116:0/863121290 >> v2:172.21.15.173:3300/0 conn(0x7fa76c0c45a0 0x7fa76c138670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa772a2c700 1 -- 172.21.15.116:0/863121290 >> v1:172.21.15.173:6789/0 conn(0x7fa76c0c41c0 legacy=0x7fa76c0a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:42.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa772a2c700 1 -- 172.21.15.116:0/863121290 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c0a9960 con 0x7fa76c138b60 2020-11-20T12:17:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa772a2c700 1 --2- 172.21.15.116:0/863121290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c138b60 0x7fa76c0a7c70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fa76c152e20 tx=0x7fa7640091d0).ready entity=mon.0 client_cookie=97c516af3996c474 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa771229700 1 -- 172.21.15.116:0/863121290 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76400ba30 con 0x7fa76c138b60 2020-11-20T12:17:42.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa771229700 1 -- 172.21.15.116:0/863121290 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa76400bb90 con 0x7fa76c138b60 2020-11-20T12:17:42.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa771229700 1 -- 172.21.15.116:0/863121290 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa764011400 con 0x7fa76c138b60 2020-11-20T12:17:42.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa77322d700 1 -- 172.21.15.116:0/863121290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c138b60 msgr2=0x7fa76c0a7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa77322d700 1 --2- 172.21.15.116:0/863121290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c138b60 0x7fa76c0a7c70 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7fa76c152e20 tx=0x7fa7640091d0).stop 2020-11-20T12:17:42.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa77322d700 1 -- 172.21.15.116:0/863121290 shutdown_connections 2020-11-20T12:17:42.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa77322d700 1 --2- 172.21.15.116:0/863121290 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c138b60 0x7fa76c0a7c70 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa77322d700 1 --2- 172.21.15.116:0/863121290 >> v2:172.21.15.173:3300/0 conn(0x7fa76c0c45a0 0x7fa76c138670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa77322d700 1 -- 172.21.15.116:0/863121290 >> 172.21.15.116:0/863121290 conn(0x7fa76c062520 msgr2=0x7fa76c062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:42.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.388+0000 7fa77322d700 1 -- 172.21.15.116:0/863121290 shutdown_connections 2020-11-20T12:17:42.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.392+0000 7fa77322d700 1 -- 172.21.15.116:0/863121290 wait complete. 2020-11-20T12:17:42.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.392+0000 7fa77322d700 1 Processor -- start 2020-11-20T12:17:42.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.392+0000 7fa77322d700 1 -- start start 2020-11-20T12:17:42.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.392+0000 7fa77322d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa76c0c41c0 0x7fa76c137790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.392+0000 7fa77322d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c0c45a0 0x7fa76c14e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa77322d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa76c138b60 0x7fa76c14ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa77322d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c14f260 con 0x7fa76c0c45a0 2020-11-20T12:17:42.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa77322d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c0a3600 con 0x7fa76c138b60 2020-11-20T12:17:42.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa77322d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa76c133a10 con 0x7fa76c0c41c0 2020-11-20T12:17:42.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa771a2a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c0c45a0 0x7fa76c14e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa771a2a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c0c45a0 0x7fa76c14e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45724/0 (socket says 172.21.15.116:45724) 2020-11-20T12:17:42.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa771a2a700 1 -- 172.21.15.116:0/3962898048 learned_addr learned my addr 172.21.15.116:0/3962898048 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:42.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa772a2c700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa76c138b60 0x7fa76c14ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa77222b700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa76c0c41c0 0x7fa76c137790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa771a2a700 1 -- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa76c0c41c0 msgr2=0x7fa76c137790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa771a2a700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa76c0c41c0 0x7fa76c137790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa771a2a700 1 -- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa76c138b60 msgr2=0x7fa76c14ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa771a2a700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa76c138b60 0x7fa76c14ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa771a2a700 1 -- 172.21.15.116:0/3962898048 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa76c133cb0 con 0x7fa76c0c45a0 2020-11-20T12:17:42.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa771a2a700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c0c45a0 0x7fa76c14e880 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fa75c00f840 tx=0x7fa75c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:42.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa75a7fc700 1 -- 172.21.15.116:0/3962898048 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00d6e0 con 0x7fa76c0c45a0 2020-11-20T12:17:42.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa77322d700 1 -- 172.21.15.116:0/3962898048 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76c133f50 con 0x7fa76c0c45a0 2020-11-20T12:17:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa75a7fc700 1 -- 172.21.15.116:0/3962898048 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa75c00d840 con 0x7fa76c0c45a0 2020-11-20T12:17:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa75a7fc700 1 -- 172.21.15.116:0/3962898048 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa75c00dbc0 con 0x7fa76c0c45a0 2020-11-20T12:17:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.396+0000 7fa77322d700 1 -- 172.21.15.116:0/3962898048 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76c134490 con 0x7fa76c0c45a0 2020-11-20T12:17:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.400+0000 7fa77322d700 1 -- 172.21.15.116:0/3962898048 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa76c134650 con 0x7fa76c0c45a0 2020-11-20T12:17:42.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.400+0000 7fa75a7fc700 1 -- 172.21.15.116:0/3962898048 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa75c024070 con 0x7fa76c0c45a0 2020-11-20T12:17:42.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.400+0000 7fa75a7fc700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa7440557f0 0x7fa744057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.400+0000 7fa75a7fc700 1 -- 172.21.15.116:0/3962898048 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa75c06ec70 con 0x7fa76c0c45a0 2020-11-20T12:17:42.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.400+0000 7fa77222b700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa7440557f0 0x7fa744057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.400+0000 7fa77222b700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa7440557f0 0x7fa744057cb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fa76800b630 tx=0x7fa768008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:42.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.408+0000 7fa75a7fc700 1 -- 172.21.15.116:0/3962898048 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa75c044b10 con 0x7fa76c0c45a0 2020-11-20T12:17:42.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.620+0000 7fa77322d700 1 -- 172.21.15.116:0/3962898048 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa76c044f60 con 0x7fa76c0c45a0 2020-11-20T12:17:42.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.620+0000 7fa75a7fc700 1 -- 172.21.15.116:0/3962898048 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+2849 (secure 0 0 0) 0x7fa75c018030 con 0x7fa76c0c45a0 2020-11-20T12:17:42.620 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:42.620 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1096724826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1096724826},{"type":"v1","addr":"172.21.15.173:6837","nonce":1096724826}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/1589443098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1589443098},{"type":"v1","addr":"172.21.15.173:6835","nonce":1589443098}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/966300328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":966300328},{"type":"v1","addr":"172.21.15.116:6833","nonce":966300328}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:17:40.163408+0000","modified":"2020-11-20T12:17:42.171423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5493":{"gid":5493,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/2908037349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2908037349},{"type":"v1","addr":"172.21.15.116:6835","nonce":2908037349}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-20T12:17:42.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.620+0000 7fa77322d700 1 -- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa7440557f0 msgr2=0x7fa744057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.620+0000 7fa77322d700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa7440557f0 0x7fa744057cb0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fa76800b630 tx=0x7fa768008040).stop 2020-11-20T12:17:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.620+0000 7fa77322d700 1 -- 172.21.15.116:0/3962898048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c0c45a0 msgr2=0x7fa76c14e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.620+0000 7fa77322d700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c0c45a0 0x7fa76c14e880 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fa75c00f840 tx=0x7fa75c008040).stop 2020-11-20T12:17:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.624+0000 7fa77322d700 1 -- 172.21.15.116:0/3962898048 shutdown_connections 2020-11-20T12:17:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.624+0000 7fa77322d700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa7440557f0 0x7fa744057cb0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.624+0000 7fa77322d700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa76c138b60 0x7fa76c14ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.624+0000 7fa77322d700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa76c0c41c0 0x7fa76c137790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.624+0000 7fa77322d700 1 --2- 172.21.15.116:0/3962898048 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa76c0c45a0 0x7fa76c14e880 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.624+0000 7fa77322d700 1 -- 172.21.15.116:0/3962898048 >> 172.21.15.116:0/3962898048 conn(0x7fa76c062520 msgr2=0x7fa76c0a7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.624+0000 7fa77322d700 1 -- 172.21.15.116:0/3962898048 shutdown_connections 2020-11-20T12:17:42.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.624+0000 7fa77322d700 1 -- 172.21.15.116:0/3962898048 wait complete. 2020-11-20T12:17:42.625 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 20 2020-11-20T12:17:42.647 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:42.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.816+0000 7ff48b59e700 1 Processor -- start 2020-11-20T12:17:42.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.816+0000 7ff48b59e700 1 -- start start 2020-11-20T12:17:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.816+0000 7ff48b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff48c0c41c0 0x7ff48c0c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.816+0000 7ff48b59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff48c0a5a10 0x7ff48c0a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.816+0000 7ff48b59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff48c0a9180 con 0x7ff48c138670 2020-11-20T12:17:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.816+0000 7ff48b59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff48c0a6380 con 0x7ff48c0a5a10 2020-11-20T12:17:42.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.816+0000 7ff48b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0a8c80 con 0x7ff48c0c41c0 2020-11-20T12:17:42.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff481d9b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff48c0a5a10 0x7ff48c0a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff481d9b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff48c0a5a10 0x7ff48c0a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:44784/0 (socket says 172.21.15.116:44784) 2020-11-20T12:17:42.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff48ad9d700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7ff48c138670 0x7ff48c138b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:51398/0 (socket says 172.21.15.116:51398) 2020-11-20T12:17:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff48a59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff48c0c41c0 0x7ff48c0c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff481d9b700 1 -- 172.21.15.116:0/1837112304 learned_addr learned my addr 172.21.15.116:0/1837112304 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff481d9b700 1 -- 172.21.15.116:0/1837112304 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff48c0c41c0 msgr2=0x7ff48c0c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:42.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff481d9b700 1 --2- 172.21.15.116:0/1837112304 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff48c0c41c0 0x7ff48c0c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff481d9b700 1 -- 172.21.15.116:0/1837112304 >> v1:172.21.15.173:6789/0 conn(0x7ff48c138670 legacy=0x7ff48c138b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff489d9b700 1 -- 172.21.15.116:0/1837112304 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346909155 0 0) 0x7ff48c0a9180 con 0x7ff48c138670 2020-11-20T12:17:42.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff481d9b700 1 -- 172.21.15.116:0/1837112304 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c0a9420 con 0x7ff48c0a5a10 2020-11-20T12:17:42.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff481d9b700 1 --2- 172.21.15.116:0/1837112304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c0a5a10 0x7ff48c0a5e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7ff47400ba20 tx=0x7ff474006eb0).ready entity=mon.1 client_cookie=d5d8879c1dff72ef server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:42.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff489d9b700 1 -- 172.21.15.116:0/1837112304 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4740097d0 con 0x7ff48c0a5a10 2020-11-20T12:17:42.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff489d9b700 1 -- 172.21.15.116:0/1837112304 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff474009930 con 0x7ff48c0a5a10 2020-11-20T12:17:42.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff489d9b700 1 -- 172.21.15.116:0/1837112304 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff474009cb0 con 0x7ff48c0a5a10 2020-11-20T12:17:42.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff48b59e700 1 -- 172.21.15.116:0/1837112304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c0a5a10 msgr2=0x7ff48c0a5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.820+0000 7ff48b59e700 1 --2- 172.21.15.116:0/1837112304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c0a5a10 0x7ff48c0a5e90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7ff47400ba20 tx=0x7ff474006eb0).stop 2020-11-20T12:17:42.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.824+0000 7ff48b59e700 1 -- 172.21.15.116:0/1837112304 shutdown_connections 2020-11-20T12:17:42.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.824+0000 7ff48b59e700 1 --2- 172.21.15.116:0/1837112304 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff48c0c41c0 0x7ff48c0c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.824+0000 7ff48b59e700 1 --2- 172.21.15.116:0/1837112304 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c0a5a10 0x7ff48c0a5e90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.824+0000 7ff48b59e700 1 -- 172.21.15.116:0/1837112304 >> 172.21.15.116:0/1837112304 conn(0x7ff48c062520 msgr2=0x7ff48c062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:42.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.824+0000 7ff48b59e700 1 -- 172.21.15.116:0/1837112304 shutdown_connections 2020-11-20T12:17:42.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.824+0000 7ff48b59e700 1 -- 172.21.15.116:0/1837112304 wait complete. 2020-11-20T12:17:42.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.824+0000 7ff48b59e700 1 Processor -- start 2020-11-20T12:17:42.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.824+0000 7ff48b59e700 1 -- start start 2020-11-20T12:17:42.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff48c0a5a10 0x7ff48c151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c0c41c0 0x7ff48c0a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff48c138670 0x7ff48c0a23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0a1a60 con 0x7ff48c138670 2020-11-20T12:17:42.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c0a28d0 con 0x7ff48c0c41c0 2020-11-20T12:17:42.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff48c15eec0 con 0x7ff48c0a5a10 2020-11-20T12:17:42.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff48c138670 0x7ff48c0a23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff48c138670 0x7ff48c0a23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45740/0 (socket says 172.21.15.116:45740) 2020-11-20T12:17:42.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48ad9d700 1 -- 172.21.15.116:0/1180065130 learned_addr learned my addr 172.21.15.116:0/1180065130 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:42.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff481d9b700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c0c41c0 0x7ff48c0a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48a59c700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff48c0a5a10 0x7ff48c151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48ad9d700 1 -- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff48c0a5a10 msgr2=0x7ff48c151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48ad9d700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff48c0a5a10 0x7ff48c151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48ad9d700 1 -- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c0c41c0 msgr2=0x7ff48c0a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48ad9d700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c0c41c0 0x7ff48c0a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48ad9d700 1 -- 172.21.15.116:0/1180065130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff48c15f380 con 0x7ff48c138670 2020-11-20T12:17:42.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48ad9d700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff48c138670 0x7ff48c0a23e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7ff48400cae0 tx=0x7ff484001320).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff482ffd700 1 -- 172.21.15.116:0/1180065130 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484005ac0 con 0x7ff48c138670 2020-11-20T12:17:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff482ffd700 1 -- 172.21.15.116:0/1180065130 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff484005c20 con 0x7ff48c138670 2020-11-20T12:17:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48b59e700 1 -- 172.21.15.116:0/1180065130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff48c15f620 con 0x7ff48c138670 2020-11-20T12:17:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff482ffd700 1 -- 172.21.15.116:0/1180065130 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff484007110 con 0x7ff48c138670 2020-11-20T12:17:42.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.828+0000 7ff48b59e700 1 -- 172.21.15.116:0/1180065130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff48c151c70 con 0x7ff48c138670 2020-11-20T12:17:42.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.832+0000 7ff48b59e700 1 -- 172.21.15.116:0/1180065130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff48c15f7b0 con 0x7ff48c138670 2020-11-20T12:17:42.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.832+0000 7ff482ffd700 1 -- 172.21.15.116:0/1180065130 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff484010030 con 0x7ff48c138670 2020-11-20T12:17:42.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.832+0000 7ff482ffd700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4680557c0 0x7ff468057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:42.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.832+0000 7ff482ffd700 1 -- 172.21.15.116:0/1180065130 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff48406e510 con 0x7ff48c138670 2020-11-20T12:17:42.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.832+0000 7ff48a59c700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4680557c0 0x7ff468057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.836+0000 7ff48a59c700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4680557c0 0x7ff468057c80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7ff47c006fd0 tx=0x7ff47c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:42.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:42.840+0000 7ff482ffd700 1 -- 172.21.15.116:0/1180065130 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff4840443b0 con 0x7ff48c138670 2020-11-20T12:17:43.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.000+0000 7ff48b59e700 1 -- 172.21.15.116:0/1180065130 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff48c044f60 con 0x7ff48c138670 2020-11-20T12:17:43.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.004+0000 7ff482ffd700 1 -- 172.21.15.116:0/1180065130 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16874 (secure 0 0 0) 0x7ff4840479d0 con 0x7ff48c138670 2020-11-20T12:17:43.003 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:43.004 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":45,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:17:41.167389+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:17:38.002878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-20T12:17:38.995456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:43.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 -- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4680557c0 msgr2=0x7ff468057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4680557c0 0x7ff468057c80 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7ff47c006fd0 tx=0x7ff47c006df0).stop 2020-11-20T12:17:43.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 -- 172.21.15.116:0/1180065130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff48c138670 msgr2=0x7ff48c0a23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff48c138670 0x7ff48c0a23e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7ff48400cae0 tx=0x7ff484001320).stop 2020-11-20T12:17:43.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 -- 172.21.15.116:0/1180065130 shutdown_connections 2020-11-20T12:17:43.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4680557c0 0x7ff468057c80 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff48c138670 0x7ff48c0a23e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff48c0a5a10 0x7ff48c151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 --2- 172.21.15.116:0/1180065130 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff48c0c41c0 0x7ff48c0a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 -- 172.21.15.116:0/1180065130 >> 172.21.15.116:0/1180065130 conn(0x7ff48c062520 msgr2=0x7ff48c062da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 -- 172.21.15.116:0/1180065130 shutdown_connections 2020-11-20T12:17:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.008+0000 7ff48b59e700 1 -- 172.21.15.116:0/1180065130 wait complete. 2020-11-20T12:17:43.034 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-20T12:17:43.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.236+0000 7f10ff5ba700 1 Processor -- start 2020-11-20T12:17:43.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.236+0000 7f10ff5ba700 1 -- start start 2020-11-20T12:17:43.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.236+0000 7f10ff5ba700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10f81535f0 0x7f10f80b9360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.236+0000 7f10ff5ba700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10f81539d0 0x7f10f80b9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.236+0000 7f10ff5ba700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f81598c0 con 0x7f10f8067840 2020-11-20T12:17:43.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.236+0000 7f10ff5ba700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f8157150 con 0x7f10f81539d0 2020-11-20T12:17:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.236+0000 7f10ff5ba700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f8159b60 con 0x7f10f81535f0 2020-11-20T12:17:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.236+0000 7f10fcb55700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10f81539d0 0x7f10f80b9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.236+0000 7f10fcb55700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10f81539d0 0x7f10f80b9850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45746/0 (socket says 172.21.15.116:45746) 2020-11-20T12:17:43.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.236+0000 7f10fcb55700 1 -- 172.21.15.116:0/1036065369 learned_addr learned my addr 172.21.15.116:0/1036065369 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:43.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.236+0000 7f10fdb57700 1 --1- 172.21.15.116:0/1036065369 >> v1:172.21.15.116:6789/0 conn(0x7f10f8067840 0x7f10f8067cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55182/0 (socket says 172.21.15.116:55182) 2020-11-20T12:17:43.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10fd356700 1 --2- 172.21.15.116:0/1036065369 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10f81535f0 0x7f10f80b9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10fcb55700 1 -- 172.21.15.116:0/1036065369 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10f81535f0 msgr2=0x7f10f80b9360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:43.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10fcb55700 1 --2- 172.21.15.116:0/1036065369 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10f81535f0 0x7f10f80b9360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10fcb55700 1 -- 172.21.15.116:0/1036065369 >> v1:172.21.15.116:6789/0 conn(0x7f10f8067840 legacy=0x7f10f8067cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:43.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10fcb55700 1 -- 172.21.15.116:0/1036065369 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f8159e00 con 0x7f10f81539d0 2020-11-20T12:17:43.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10fcb55700 1 --2- 172.21.15.116:0/1036065369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10f81539d0 0x7f10f80b9850 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f10e8006fd0 tx=0x7f10e8006eb0).ready entity=mon.0 client_cookie=97c555cd59f55d20 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:43.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10e7fff700 1 -- 172.21.15.116:0/1036065369 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e80098d0 con 0x7f10f81539d0 2020-11-20T12:17:43.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10e7fff700 1 -- 172.21.15.116:0/1036065369 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e8009a30 con 0x7f10f81539d0 2020-11-20T12:17:43.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10e7fff700 1 -- 172.21.15.116:0/1036065369 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e8009db0 con 0x7f10f81539d0 2020-11-20T12:17:43.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10ff5ba700 1 -- 172.21.15.116:0/1036065369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10f81539d0 msgr2=0x7f10f80b9850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10ff5ba700 1 --2- 172.21.15.116:0/1036065369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10f81539d0 0x7f10f80b9850 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f10e8006fd0 tx=0x7f10e8006eb0).stop 2020-11-20T12:17:43.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10ff5ba700 1 -- 172.21.15.116:0/1036065369 shutdown_connections 2020-11-20T12:17:43.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10ff5ba700 1 --2- 172.21.15.116:0/1036065369 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10f81535f0 0x7f10f80b9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10ff5ba700 1 --2- 172.21.15.116:0/1036065369 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10f81539d0 0x7f10f80b9850 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10ff5ba700 1 -- 172.21.15.116:0/1036065369 >> 172.21.15.116:0/1036065369 conn(0x7f10f8060170 msgr2=0x7f10f8060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10ff5ba700 1 -- 172.21.15.116:0/1036065369 shutdown_connections 2020-11-20T12:17:43.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.240+0000 7f10ff5ba700 1 -- 172.21.15.116:0/1036065369 wait complete. 2020-11-20T12:17:43.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.244+0000 7f10ff5ba700 1 Processor -- start 2020-11-20T12:17:43.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.244+0000 7f10ff5ba700 1 -- start start 2020-11-20T12:17:43.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.244+0000 7f10ff5ba700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10f8067840 0x7f10f813b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.244+0000 7f10ff5ba700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10f81535f0 0x7f10f80bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.244+0000 7f10ff5ba700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10f80bd8c0 0x7f10f80bdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.244+0000 7f10ff5ba700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80bc9a0 con 0x7f10f8067840 2020-11-20T12:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.244+0000 7f10ff5ba700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80bcc40 con 0x7f10f80bd8c0 2020-11-20T12:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.244+0000 7f10ff5ba700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f80bcee0 con 0x7f10f81535f0 2020-11-20T12:17:43.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.244+0000 7f10fd356700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10f8067840 0x7f10f813b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.244+0000 7f10fd356700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10f8067840 0x7f10f813b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45750/0 (socket says 172.21.15.116:45750) 2020-11-20T12:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.244+0000 7f10fd356700 1 -- 172.21.15.116:0/2065467784 learned_addr learned my addr 172.21.15.116:0/2065467784 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:43.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10fcb55700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10f81535f0 0x7f10f80bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10fdb57700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10f80bd8c0 0x7f10f80bdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10fd356700 1 -- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10f81535f0 msgr2=0x7f10f80bbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10fd356700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10f81535f0 0x7f10f80bbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10fd356700 1 -- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10f80bd8c0 msgr2=0x7f10f80bdce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10fd356700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10f80bd8c0 0x7f10f80bdce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10fd356700 1 -- 172.21.15.116:0/2065467784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f80bd180 con 0x7f10f8067840 2020-11-20T12:17:43.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10fd356700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10f8067840 0x7f10f813b660 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f10f400db60 tx=0x7f10f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:43.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10e57fa700 1 -- 172.21.15.116:0/2065467784 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f400be30 con 0x7f10f8067840 2020-11-20T12:17:43.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10e57fa700 1 -- 172.21.15.116:0/2065467784 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4009790 con 0x7f10f8067840 2020-11-20T12:17:43.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10ff5ba700 1 -- 172.21.15.116:0/2065467784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f80bd420 con 0x7f10f8067840 2020-11-20T12:17:43.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10e57fa700 1 -- 172.21.15.116:0/2065467784 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4009b30 con 0x7f10f8067840 2020-11-20T12:17:43.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.248+0000 7f10ff5ba700 1 -- 172.21.15.116:0/2065467784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f813bcf0 con 0x7f10f8067840 2020-11-20T12:17:43.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.252+0000 7f10ff5ba700 1 -- 172.21.15.116:0/2065467784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f80a3b10 con 0x7f10f8067840 2020-11-20T12:17:43.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.252+0000 7f10e57fa700 1 -- 172.21.15.116:0/2065467784 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f10f4013480 con 0x7f10f8067840 2020-11-20T12:17:43.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.252+0000 7f10e57fa700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d00557c0 0x7f10d0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.252+0000 7f10e57fa700 1 -- 172.21.15.116:0/2065467784 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f10f406eb60 con 0x7f10f8067840 2020-11-20T12:17:43.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.252+0000 7f10fcb55700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d00557c0 0x7f10d0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.252+0000 7f10fcb55700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d00557c0 0x7f10d0057c80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f10f80c1900 tx=0x7f10e8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:43.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.256+0000 7f10e57fa700 1 -- 172.21.15.116:0/2065467784 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f10f40449a0 con 0x7f10f8067840 2020-11-20T12:17:43.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.472+0000 7f10ff5ba700 1 -- 172.21.15.116:0/2065467784 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f10f805ca30 con 0x7f10f8067840 2020-11-20T12:17:43.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.480+0000 7f10e57fa700 1 -- 172.21.15.116:0/2065467784 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v10) v1 ==== 184+0+0 (secure 0 0 0) 0x7f10f4047fc0 con 0x7f10f8067840 2020-11-20T12:17:43.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.480+0000 7f10ff5ba700 1 -- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d00557c0 msgr2=0x7f10d0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.480+0000 7f10ff5ba700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d00557c0 0x7f10d0057c80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f10f80c1900 tx=0x7f10e8015000).stop 2020-11-20T12:17:43.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.480+0000 7f10ff5ba700 1 -- 172.21.15.116:0/2065467784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10f8067840 msgr2=0x7f10f813b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.480+0000 7f10ff5ba700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10f8067840 0x7f10f813b660 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f10f400db60 tx=0x7f10f4008040).stop 2020-11-20T12:17:43.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.484+0000 7f10ff5ba700 1 -- 172.21.15.116:0/2065467784 shutdown_connections 2020-11-20T12:17:43.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.484+0000 7f10ff5ba700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d00557c0 0x7f10d0057c80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.484+0000 7f10ff5ba700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10f80bd8c0 0x7f10f80bdce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.484+0000 7f10ff5ba700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10f8067840 0x7f10f813b660 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.484+0000 7f10ff5ba700 1 --2- 172.21.15.116:0/2065467784 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10f81535f0 0x7f10f80bbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.484+0000 7f10ff5ba700 1 -- 172.21.15.116:0/2065467784 >> 172.21.15.116:0/2065467784 conn(0x7f10f8060170 msgr2=0x7f10f8066ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.484+0000 7f10ff5ba700 1 -- 172.21.15.116:0/2065467784 shutdown_connections 2020-11-20T12:17:43.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.484+0000 7f10ff5ba700 1 -- 172.21.15.116:0/2065467784 wait complete. 2020-11-20T12:17:43.486 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2020-11-20T12:17:43.507 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:43.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.680+0000 7f52b1685700 1 Processor -- start 2020-11-20T12:17:43.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.680+0000 7f52b1685700 1 -- start start 2020-11-20T12:17:43.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.680+0000 7f52b1685700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f52ac13aee0 0x7f52ac152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.680+0000 7f52b1685700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52ac0aa790 0x7f52ac0ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.680+0000 7f52b1685700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ac152a30 con 0x7f52ac0612f0 2020-11-20T12:17:43.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.680+0000 7f52b1685700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac0aac10 con 0x7f52ac13aee0 2020-11-20T12:17:43.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.680+0000 7f52b1685700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0aaeb0 con 0x7f52ac0aa790 2020-11-20T12:17:43.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.680+0000 7f52b0e84700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f52ac0612f0 0x7f52ac0a7ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55196/0 (socket says 172.21.15.116:55196) 2020-11-20T12:17:43.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.680+0000 7f52b0e84700 1 -- 172.21.15.116:0/1186206619 learned_addr learned my addr 172.21.15.116:0/1186206619 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:43.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.680+0000 7f52a37fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52ac0aa790 0x7f52ac0ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.680+0000 7f52abfff700 1 --2- 172.21.15.116:0/1186206619 >> v2:172.21.15.173:3300/0 conn(0x7f52ac13aee0 0x7f52ac152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52a37fe700 1 -- 172.21.15.116:0/1186206619 >> v2:172.21.15.173:3300/0 conn(0x7f52ac13aee0 msgr2=0x7f52ac152540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:43.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52a37fe700 1 --2- 172.21.15.116:0/1186206619 >> v2:172.21.15.173:3300/0 conn(0x7f52ac13aee0 0x7f52ac152540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52a37fe700 1 -- 172.21.15.116:0/1186206619 >> v1:172.21.15.116:6789/0 conn(0x7f52ac0612f0 legacy=0x7f52ac0a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52a37fe700 1 -- 172.21.15.116:0/1186206619 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0acbf0 con 0x7f52ac0aa790 2020-11-20T12:17:43.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52a37fe700 1 --2- 172.21.15.116:0/1186206619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52ac0aa790 0x7f52ac0ad5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f529400b9a0 tx=0x7f5294006df0).ready entity=mon.2 client_cookie=52a768e38f1fb0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:43.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52ab7fe700 1 -- 172.21.15.116:0/1186206619 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52940097b0 con 0x7f52ac0aa790 2020-11-20T12:17:43.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52ab7fe700 1 -- 172.21.15.116:0/1186206619 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5294009910 con 0x7f52ac0aa790 2020-11-20T12:17:43.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52ab7fe700 1 -- 172.21.15.116:0/1186206619 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5294009c90 con 0x7f52ac0aa790 2020-11-20T12:17:43.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52b1685700 1 -- 172.21.15.116:0/1186206619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52ac0aa790 msgr2=0x7f52ac0ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52b1685700 1 --2- 172.21.15.116:0/1186206619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52ac0aa790 0x7f52ac0ad5d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f529400b9a0 tx=0x7f5294006df0).stop 2020-11-20T12:17:43.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52b1685700 1 -- 172.21.15.116:0/1186206619 shutdown_connections 2020-11-20T12:17:43.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52b1685700 1 --2- 172.21.15.116:0/1186206619 >> v2:172.21.15.173:3300/0 conn(0x7f52ac13aee0 0x7f52ac152540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52b1685700 1 --2- 172.21.15.116:0/1186206619 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52ac0aa790 0x7f52ac0ad5d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52b1685700 1 -- 172.21.15.116:0/1186206619 >> 172.21.15.116:0/1186206619 conn(0x7f52ac0678a0 msgr2=0x7f52ac067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:43.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52b1685700 1 -- 172.21.15.116:0/1186206619 shutdown_connections 2020-11-20T12:17:43.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.684+0000 7f52b1685700 1 -- 172.21.15.116:0/1186206619 wait complete. 2020-11-20T12:17:43.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.688+0000 7f52b1685700 1 Processor -- start 2020-11-20T12:17:43.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.688+0000 7f52b1685700 1 -- start start 2020-11-20T12:17:43.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.688+0000 7f52b1685700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52ac0612f0 0x7f52ac137d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.688+0000 7f52b1685700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52ac0aa790 0x7f52ac0c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.688+0000 7f52b1685700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52ac13aee0 0x7f52ac0c8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.688+0000 7f52b1685700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0a2e80 con 0x7f52ac0612f0 2020-11-20T12:17:43.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.688+0000 7f52b1685700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0a1f60 con 0x7f52ac13aee0 2020-11-20T12:17:43.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.688+0000 7f52b1685700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0a2200 con 0x7f52ac0aa790 2020-11-20T12:17:43.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.688+0000 7f52abfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52ac0612f0 0x7f52ac137d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.688+0000 7f52abfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52ac0612f0 0x7f52ac137d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45764/0 (socket says 172.21.15.116:45764) 2020-11-20T12:17:43.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.688+0000 7f52abfff700 1 -- 172.21.15.116:0/1538686717 learned_addr learned my addr 172.21.15.116:0/1538686717 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:43.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52b0e84700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52ac13aee0 0x7f52ac0c8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52a37fe700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52ac0aa790 0x7f52ac0c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52abfff700 1 -- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52ac0aa790 msgr2=0x7f52ac0c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52abfff700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52ac0aa790 0x7f52ac0c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52abfff700 1 -- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52ac13aee0 msgr2=0x7f52ac0c8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52abfff700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52ac13aee0 0x7f52ac0c8930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52abfff700 1 -- 172.21.15.116:0/1538686717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0a24a0 con 0x7f52ac0612f0 2020-11-20T12:17:43.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52abfff700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52ac0612f0 0x7f52ac137d40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f529c00ebd0 tx=0x7f529c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:43.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52a8ff9700 1 -- 172.21.15.116:0/1538686717 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c00b9a0 con 0x7f52ac0612f0 2020-11-20T12:17:43.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52a8ff9700 1 -- 172.21.15.116:0/1538686717 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f529c00bb00 con 0x7f52ac0612f0 2020-11-20T12:17:43.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52b1685700 1 -- 172.21.15.116:0/1538686717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ac0a2740 con 0x7f52ac0612f0 2020-11-20T12:17:43.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52a8ff9700 1 -- 172.21.15.116:0/1538686717 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529c00be80 con 0x7f52ac0612f0 2020-11-20T12:17:43.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52b1685700 1 -- 172.21.15.116:0/1538686717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac138230 con 0x7f52ac0612f0 2020-11-20T12:17:43.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.692+0000 7f52b1685700 1 -- 172.21.15.116:0/1538686717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac044f60 con 0x7f52ac0612f0 2020-11-20T12:17:43.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.696+0000 7f52a8ff9700 1 -- 172.21.15.116:0/1538686717 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f529c012600 con 0x7f52ac0612f0 2020-11-20T12:17:43.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.696+0000 7f52a8ff9700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52880557c0 0x7f5288057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:43.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.696+0000 7f52a8ff9700 1 -- 172.21.15.116:0/1538686717 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f529c06ea80 con 0x7f52ac0612f0 2020-11-20T12:17:43.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.696+0000 7f52a37fe700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52880557c0 0x7f5288057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:43.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.696+0000 7f52a37fe700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52880557c0 0x7f5288057c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f5294006e30 tx=0x7f5294015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:43.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.700+0000 7f52a8ff9700 1 -- 172.21.15.116:0/1538686717 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f529c0448c0 con 0x7f52ac0612f0 2020-11-20T12:17:43.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.920+0000 7f52b1685700 1 -- 172.21.15.116:0/1538686717 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52ac156c60 con 0x7f52ac0612f0 2020-11-20T12:17:43.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52a8ff9700 1 -- 172.21.15.116:0/1538686717 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+2849 (secure 0 0 0) 0x7f529c047ee0 con 0x7f52ac0612f0 2020-11-20T12:17:43.923 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:43.923 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1096724826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1096724826},{"type":"v1","addr":"172.21.15.173:6837","nonce":1096724826}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/1589443098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1589443098},{"type":"v1","addr":"172.21.15.173:6835","nonce":1589443098}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/966300328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":966300328},{"type":"v1","addr":"172.21.15.116:6833","nonce":966300328}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:17:40.163408+0000","modified":"2020-11-20T12:17:42.171423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5493":{"gid":5493,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/2908037349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2908037349},{"type":"v1","addr":"172.21.15.116:6835","nonce":2908037349}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-20T12:17:43.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 -- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52880557c0 msgr2=0x7f5288057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52880557c0 0x7f5288057c80 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f5294006e30 tx=0x7f5294015000).stop 2020-11-20T12:17:43.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 -- 172.21.15.116:0/1538686717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52ac0612f0 msgr2=0x7f52ac137d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:43.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52ac0612f0 0x7f52ac137d40 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f529c00ebd0 tx=0x7f529c006df0).stop 2020-11-20T12:17:43.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 -- 172.21.15.116:0/1538686717 shutdown_connections 2020-11-20T12:17:43.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52880557c0 0x7f5288057c80 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52ac13aee0 0x7f52ac0c8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52ac0612f0 0x7f52ac137d40 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 --2- 172.21.15.116:0/1538686717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52ac0aa790 0x7f52ac0c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:43.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 -- 172.21.15.116:0/1538686717 >> 172.21.15.116:0/1538686717 conn(0x7f52ac0678a0 msgr2=0x7f52ac0a98c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:43.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 -- 172.21.15.116:0/1538686717 shutdown_connections 2020-11-20T12:17:43.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:43.924+0000 7f52b1685700 1 -- 172.21.15.116:0/1538686717 wait complete. 2020-11-20T12:17:43.930 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 20 2020-11-20T12:17:44.352 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:17:40.163408+0000', 'modified': '2020-11-20T12:17:42.171423+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5493': {'gid': 5493, 'name': 'a', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/2908037349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2908037349}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 2908037349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:17:44.353 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:17:44.355 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:17:44.542 INFO:tasks.ceph.mds.a.smithi116.stderr:2020-11-20T12:17:44.544+0000 7f35907d3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:17:44.555 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":5493,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.3722892680000003,"mdsmap_epoch":20,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":9.8899976949999999} 2020-11-20T12:17:44.556 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 5493, "mdsmap_epoch": 20, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 4.372289268, "state": "up:active", "uptime": 9.889997695, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:17:44.556 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:17:44.557 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:17:44.557 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:17:44.558 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:17:44.558 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:17:44.558 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:17:44.559 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:17:44.559 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:17:44.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:44.560 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:17:44.576 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:17:44.577 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:17:44.577 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:17:44.577 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:44.578 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:17:44.578 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:44.578 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:44.578 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:44.578 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:44.579 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:44.579 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:44.579 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:44.579 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:17:44.580 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:44.580 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:17:44.580 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:44.580 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:17:44.580 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:44.581 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:17:44.581 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:44.581 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:17:44.581 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:44.582 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:17:44.582 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:44.582 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:17:44.582 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:44.582 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:17:44.583 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:44.583 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:44.583 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:17:44.583 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:17:44.583 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:44.584 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:44.584 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:17:44.584 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:17:44.584 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:44.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:44.585 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:17:44.650 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:17:44.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:44.654 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:17:44.664 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:17:44.665 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:17:44.665 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:17:44.665 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:17:44.665 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:17:44.666 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:17:44.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:44.667 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:17:44.668 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:17:44.668 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:17:44.668 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:17:44.669 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:17:44.669 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:17:44.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:44.805 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:17:44.821 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:17:44.822 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:17:44.823 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:17:44.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:44.827 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2020-11-20T12:17:44.858 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.068+0000 7fd568ccd700 1 Processor -- start 2020-11-20T12:17:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.072+0000 7fd568ccd700 1 -- start start 2020-11-20T12:17:45.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.072+0000 7fd568ccd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 0x7fd564152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:45.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.072+0000 7fd568ccd700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd564152a30 con 0x7fd5640612f0 2020-11-20T12:17:45.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.072+0000 7fd568ccd700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5640aac10 con 0x7fd5640aa790 2020-11-20T12:17:45.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.072+0000 7fd568ccd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640adce0 con 0x7fd56413aee0 2020-11-20T12:17:45.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.072+0000 7fd563fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd5640612f0 0x7fd5640a7ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55210/0 (socket says 172.21.15.116:55210) 2020-11-20T12:17:45.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.072+0000 7fd563fff700 1 -- 172.21.15.116:0/135415471 learned_addr learned my addr 172.21.15.116:0/135415471 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:45.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5637fe700 1 --2- 172.21.15.116:0/135415471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 0x7fd564152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:45.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5627fc700 1 -- 172.21.15.116:0/135415471 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887898845 0 0) 0x7fd564152a30 con 0x7fd5640612f0 2020-11-20T12:17:45.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5627fc700 1 -- 172.21.15.116:0/135415471 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd550003240 con 0x7fd5640612f0 2020-11-20T12:17:45.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5627fc700 1 -- 172.21.15.116:0/135415471 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574113923 0 0) 0x7fd5640aac10 con 0x7fd5640aa790 2020-11-20T12:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5627fc700 1 -- 172.21.15.116:0/135415471 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd564152a30 con 0x7fd5640aa790 2020-11-20T12:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5637fe700 1 -- 172.21.15.116:0/135415471 >> v1:172.21.15.173:6789/0 conn(0x7fd5640aa790 legacy=0x7fd5640ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5637fe700 1 -- 172.21.15.116:0/135415471 >> v1:172.21.15.116:6789/0 conn(0x7fd5640612f0 legacy=0x7fd5640a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:45.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5637fe700 1 -- 172.21.15.116:0/135415471 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5640aaeb0 con 0x7fd56413aee0 2020-11-20T12:17:45.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5637fe700 1 --2- 172.21.15.116:0/135415471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 0x7fd564152540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd55400b9d0 tx=0x7fd554006df0).ready entity=mon.2 client_cookie=611a167a6b0c991c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:45.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5627fc700 1 -- 172.21.15.116:0/135415471 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554009680 con 0x7fd56413aee0 2020-11-20T12:17:45.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5627fc700 1 -- 172.21.15.116:0/135415471 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5540097e0 con 0x7fd56413aee0 2020-11-20T12:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd5627fc700 1 -- 172.21.15.116:0/135415471 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd554009b60 con 0x7fd56413aee0 2020-11-20T12:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd568ccd700 1 -- 172.21.15.116:0/135415471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 msgr2=0x7fd564152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd568ccd700 1 --2- 172.21.15.116:0/135415471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 0x7fd564152540 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fd55400b9d0 tx=0x7fd554006df0).stop 2020-11-20T12:17:45.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd568ccd700 1 -- 172.21.15.116:0/135415471 shutdown_connections 2020-11-20T12:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd568ccd700 1 --2- 172.21.15.116:0/135415471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 0x7fd564152540 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd568ccd700 1 -- 172.21.15.116:0/135415471 >> 172.21.15.116:0/135415471 conn(0x7fd5640678a0 msgr2=0x7fd564067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd568ccd700 1 -- 172.21.15.116:0/135415471 shutdown_connections 2020-11-20T12:17:45.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.076+0000 7fd568ccd700 1 -- 172.21.15.116:0/135415471 wait complete. 2020-11-20T12:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.080+0000 7fd568ccd700 1 Processor -- start 2020-11-20T12:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.080+0000 7fd568ccd700 1 -- start start 2020-11-20T12:17:45.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd568ccd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5640612f0 0x7fd564164e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:45.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd568ccd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5640aa790 0x7fd56409fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:45.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd568ccd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 0x7fd564150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:45.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd568ccd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd564165360 con 0x7fd5640aa790 2020-11-20T12:17:45.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd568ccd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640c5100 con 0x7fd5640612f0 2020-11-20T12:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd568ccd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5640c41e0 con 0x7fd56413aee0 2020-11-20T12:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd562ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5640aa790 0x7fd56409fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd562ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5640aa790 0x7fd56409fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45778/0 (socket says 172.21.15.116:45778) 2020-11-20T12:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd562ffd700 1 -- 172.21.15.116:0/3854087356 learned_addr learned my addr 172.21.15.116:0/3854087356 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:45.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd563fff700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 0x7fd564150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:45.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd5637fe700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5640612f0 0x7fd564164e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:45.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd563fff700 1 -- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5640612f0 msgr2=0x7fd564164e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:45.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd563fff700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5640612f0 0x7fd564164e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:45.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd563fff700 1 -- 172.21.15.116:0/3854087356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5640aa790 msgr2=0x7fd56409fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:45.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd563fff700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5640aa790 0x7fd56409fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:45.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd563fff700 1 -- 172.21.15.116:0/3854087356 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5640c4480 con 0x7fd56413aee0 2020-11-20T12:17:45.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd562ffd700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5640aa790 0x7fd56409fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:45.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd563fff700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 0x7fd564150940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd55c001420 tx=0x7fd55c007d80).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:45.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd5437fe700 1 -- 172.21.15.116:0/3854087356 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c005b80 con 0x7fd56413aee0 2020-11-20T12:17:45.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd5437fe700 1 -- 172.21.15.116:0/3854087356 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd55c005ce0 con 0x7fd56413aee0 2020-11-20T12:17:45.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd568ccd700 1 -- 172.21.15.116:0/3854087356 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5640c4720 con 0x7fd56413aee0 2020-11-20T12:17:45.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd5437fe700 1 -- 172.21.15.116:0/3854087356 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd55c007360 con 0x7fd56413aee0 2020-11-20T12:17:45.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.084+0000 7fd568ccd700 1 -- 172.21.15.116:0/3854087356 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5640c4c60 con 0x7fd56413aee0 2020-11-20T12:17:45.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.088+0000 7fd568ccd700 1 -- 172.21.15.116:0/3854087356 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5640c3f80 con 0x7fd56413aee0 2020-11-20T12:17:45.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.088+0000 7fd5437fe700 1 -- 172.21.15.116:0/3854087356 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd55c010030 con 0x7fd56413aee0 2020-11-20T12:17:45.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.088+0000 7fd5437fe700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd53c0557c0 0x7fd53c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:45.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.088+0000 7fd5437fe700 1 -- 172.21.15.116:0/3854087356 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd55c06e620 con 0x7fd56413aee0 2020-11-20T12:17:45.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.092+0000 7fd5637fe700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd53c0557c0 0x7fd53c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:45.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.092+0000 7fd5637fe700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd53c0557c0 0x7fd53c057c80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fd55400b150 tx=0x7fd554015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:45.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.096+0000 7fd5437fe700 1 -- 172.21.15.116:0/3854087356 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd55c044540 con 0x7fd56413aee0 2020-11-20T12:17:45.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.260+0000 7fd568ccd700 1 -- 172.21.15.116:0/3854087356 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5640c4df0 con 0x7fd56413aee0 2020-11-20T12:17:45.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.264+0000 7fd5437fe700 1 -- 172.21.15.116:0/3854087356 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fd55c034020 con 0x7fd56413aee0 2020-11-20T12:17:45.263 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:45.263 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":45,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:17:41.167389+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:17:38.002878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-20T12:17:38.995456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:45.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.264+0000 7fd568ccd700 1 -- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd53c0557c0 msgr2=0x7fd53c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:45.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.264+0000 7fd568ccd700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd53c0557c0 0x7fd53c057c80 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fd55400b150 tx=0x7fd554015000).stop 2020-11-20T12:17:45.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.268+0000 7fd568ccd700 1 -- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 msgr2=0x7fd564150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:45.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.268+0000 7fd568ccd700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 0x7fd564150940 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd55c001420 tx=0x7fd55c007d80).stop 2020-11-20T12:17:45.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.268+0000 7fd568ccd700 1 -- 172.21.15.116:0/3854087356 shutdown_connections 2020-11-20T12:17:45.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.268+0000 7fd568ccd700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd53c0557c0 0x7fd53c057c80 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:45.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.268+0000 7fd568ccd700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd56413aee0 0x7fd564150940 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:45.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.268+0000 7fd568ccd700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5640612f0 0x7fd564164e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:45.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.268+0000 7fd568ccd700 1 --2- 172.21.15.116:0/3854087356 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5640aa790 0x7fd56409fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:45.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.268+0000 7fd568ccd700 1 -- 172.21.15.116:0/3854087356 >> 172.21.15.116:0/3854087356 conn(0x7fd5640678a0 msgr2=0x7fd5640a8a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:45.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.268+0000 7fd568ccd700 1 -- 172.21.15.116:0/3854087356 shutdown_connections 2020-11-20T12:17:45.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.268+0000 7fd568ccd700 1 -- 172.21.15.116:0/3854087356 wait complete. 2020-11-20T12:17:45.292 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:17:38.002878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:17:38.995456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:17:45.293 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:45.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.512+0000 7fa26087b700 1 Processor -- start 2020-11-20T12:17:46.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.512+0000 7fa26087b700 1 -- start start 2020-11-20T12:17:47.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.512+0000 7fa26087b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa25c066b70 0x7fa25c0a5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.512+0000 7fa26087b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c066ff0 0x7fa25c0a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.512+0000 7fa26087b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25c14fc40 con 0x7fa25c14f860 2020-11-20T12:17:47.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.512+0000 7fa26087b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa25c0c0590 con 0x7fa25c066b70 2020-11-20T12:17:47.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.512+0000 7fa26087b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0c0830 con 0x7fa25c066ff0 2020-11-20T12:17:47.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25a59c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa25c14f860 0x7fa25c0c14b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55222/0 (socket says 172.21.15.116:55222) 2020-11-20T12:17:47.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25a59c700 1 -- 172.21.15.116:0/3082781511 learned_addr learned my addr 172.21.15.116:0/3082781511 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:47.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa259d9b700 1 --2- 172.21.15.116:0/3082781511 >> v2:172.21.15.173:3300/0 conn(0x7fa25c066b70 0x7fa25c0a5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25ad9d700 1 --2- 172.21.15.116:0/3082781511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c066ff0 0x7fa25c0a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25959a700 1 -- 172.21.15.116:0/3082781511 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989096251 0 0) 0x7fa24c0010d0 con 0x7fa25c14f860 2020-11-20T12:17:47.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25959a700 1 -- 172.21.15.116:0/3082781511 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa240003240 con 0x7fa25c14f860 2020-11-20T12:17:47.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25ad9d700 1 -- 172.21.15.116:0/3082781511 >> v2:172.21.15.173:3300/0 conn(0x7fa25c066b70 msgr2=0x7fa25c0a5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:47.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25ad9d700 1 --2- 172.21.15.116:0/3082781511 >> v2:172.21.15.173:3300/0 conn(0x7fa25c066b70 0x7fa25c0a5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25ad9d700 1 -- 172.21.15.116:0/3082781511 >> v1:172.21.15.116:6789/0 conn(0x7fa25c14f860 legacy=0x7fa25c0c14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25ad9d700 1 -- 172.21.15.116:0/3082781511 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c0c0ad0 con 0x7fa25c066ff0 2020-11-20T12:17:47.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25ad9d700 1 --2- 172.21.15.116:0/3082781511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c066ff0 0x7fa25c0a64a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa254009c90 tx=0x7fa2540091d0).ready entity=mon.2 client_cookie=bf491f5b3631b45a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25959a700 1 -- 172.21.15.116:0/3082781511 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400b9c0 con 0x7fa25c066ff0 2020-11-20T12:17:47.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25959a700 1 -- 172.21.15.116:0/3082781511 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25400bb20 con 0x7fa25c066ff0 2020-11-20T12:17:47.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.516+0000 7fa25959a700 1 -- 172.21.15.116:0/3082781511 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25400bea0 con 0x7fa25c066ff0 2020-11-20T12:17:47.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.520+0000 7fa26087b700 1 -- 172.21.15.116:0/3082781511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c066ff0 msgr2=0x7fa25c0a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.520+0000 7fa26087b700 1 --2- 172.21.15.116:0/3082781511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c066ff0 0x7fa25c0a64a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fa254009c90 tx=0x7fa2540091d0).stop 2020-11-20T12:17:47.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.520+0000 7fa26087b700 1 -- 172.21.15.116:0/3082781511 shutdown_connections 2020-11-20T12:17:47.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.520+0000 7fa26087b700 1 --2- 172.21.15.116:0/3082781511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c066ff0 0x7fa25c0a64a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.520+0000 7fa26087b700 1 --2- 172.21.15.116:0/3082781511 >> v2:172.21.15.173:3300/0 conn(0x7fa25c066b70 0x7fa25c0a5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.520+0000 7fa26087b700 1 -- 172.21.15.116:0/3082781511 >> 172.21.15.116:0/3082781511 conn(0x7fa25c060170 msgr2=0x7fa25c060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:47.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.520+0000 7fa26087b700 1 -- 172.21.15.116:0/3082781511 shutdown_connections 2020-11-20T12:17:47.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.520+0000 7fa26087b700 1 -- 172.21.15.116:0/3082781511 wait complete. 2020-11-20T12:17:47.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.520+0000 7fa26087b700 1 Processor -- start 2020-11-20T12:17:47.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa26087b700 1 -- start start 2020-11-20T12:17:47.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa26087b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa25c066b70 0x7fa25c1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa26087b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c066ff0 0x7fa25c1501e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa26087b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25c14f860 0x7fa25c14c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa26087b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0a3390 con 0x7fa25c14f860 2020-11-20T12:17:47.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa26087b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0a3630 con 0x7fa25c066b70 2020-11-20T12:17:47.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa26087b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa25c0a38d0 con 0x7fa25c066ff0 2020-11-20T12:17:47.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa25ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25c14f860 0x7fa25c14c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa25ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25c14f860 0x7fa25c14c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45796/0 (socket says 172.21.15.116:45796) 2020-11-20T12:17:47.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa25ad9d700 1 -- 172.21.15.116:0/1163864287 learned_addr learned my addr 172.21.15.116:0/1163864287 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:47.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa25a59c700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa25c066b70 0x7fa25c1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa259d9b700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c066ff0 0x7fa25c1501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa25ad9d700 1 -- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c066ff0 msgr2=0x7fa25c1501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa25ad9d700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c066ff0 0x7fa25c1501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa25ad9d700 1 -- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa25c066b70 msgr2=0x7fa25c1511b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa25ad9d700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa25c066b70 0x7fa25c1511b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa25ad9d700 1 -- 172.21.15.116:0/1163864287 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa25c0a3b70 con 0x7fa25c14f860 2020-11-20T12:17:47.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.524+0000 7fa25ad9d700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25c14f860 0x7fa25c14c030 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fa25400b6c0 tx=0x7fa254016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.528+0000 7fa24a7fc700 1 -- 172.21.15.116:0/1163864287 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25401c6c0 con 0x7fa25c14f860 2020-11-20T12:17:47.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.528+0000 7fa24a7fc700 1 -- 172.21.15.116:0/1163864287 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa25401c820 con 0x7fa25c14f860 2020-11-20T12:17:47.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.528+0000 7fa24a7fc700 1 -- 172.21.15.116:0/1163864287 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa25401cbc0 con 0x7fa25c14f860 2020-11-20T12:17:47.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.528+0000 7fa26087b700 1 -- 172.21.15.116:0/1163864287 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25c0a3e10 con 0x7fa25c14f860 2020-11-20T12:17:47.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.528+0000 7fa26087b700 1 -- 172.21.15.116:0/1163864287 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25c0a42b0 con 0x7fa25c14f860 2020-11-20T12:17:47.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.528+0000 7fa26087b700 1 -- 172.21.15.116:0/1163864287 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25c05ca30 con 0x7fa25c14f860 2020-11-20T12:17:47.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.528+0000 7fa24a7fc700 1 -- 172.21.15.116:0/1163864287 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa254017070 con 0x7fa25c14f860 2020-11-20T12:17:47.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.532+0000 7fa24a7fc700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2300557c0 0x7fa230057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.532+0000 7fa24a7fc700 1 -- 172.21.15.116:0/1163864287 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa254070b10 con 0x7fa25c14f860 2020-11-20T12:17:47.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.532+0000 7fa25a59c700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2300557c0 0x7fa230057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.532+0000 7fa25a59c700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2300557c0 0x7fa230057c80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fa24c001230 tx=0x7fa24c0078c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.536+0000 7fa24a7fc700 1 -- 172.21.15.116:0/1163864287 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa2540468a0 con 0x7fa25c14f860 2020-11-20T12:17:47.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.700+0000 7fa26087b700 1 -- 172.21.15.116:0/1163864287 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa25c061e80 con 0x7fa25c14f860 2020-11-20T12:17:47.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.700+0000 7fa24a7fc700 1 -- 172.21.15.116:0/1163864287 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa254015030 con 0x7fa25c14f860 2020-11-20T12:17:47.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 -- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2300557c0 msgr2=0x7fa230057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2300557c0 0x7fa230057c80 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7fa24c001230 tx=0x7fa24c0078c0).stop 2020-11-20T12:17:47.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 -- 172.21.15.116:0/1163864287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25c14f860 msgr2=0x7fa25c14c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25c14f860 0x7fa25c14c030 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fa25400b6c0 tx=0x7fa254016000).stop 2020-11-20T12:17:47.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 -- 172.21.15.116:0/1163864287 shutdown_connections 2020-11-20T12:17:47.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2300557c0 0x7fa230057c80 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa25c14f860 0x7fa25c14c030 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa25c066b70 0x7fa25c1511b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 --2- 172.21.15.116:0/1163864287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa25c066ff0 0x7fa25c1501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 -- 172.21.15.116:0/1163864287 >> 172.21.15.116:0/1163864287 conn(0x7fa25c060170 msgr2=0x7fa25c0c26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:47.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 -- 172.21.15.116:0/1163864287 shutdown_connections 2020-11-20T12:17:47.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:45.704+0000 7fa26087b700 1 -- 172.21.15.116:0/1163864287 wait complete. 2020-11-20T12:17:47.016 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:47.017 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:47.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.192+0000 7fe192056700 1 Processor -- start 2020-11-20T12:17:47.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.192+0000 7fe192056700 1 -- start start 2020-11-20T12:17:47.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.192+0000 7fe192056700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18c1502e0 0x7fe18c0ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.192+0000 7fe192056700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe18c159820 0x7fe18c159c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.192+0000 7fe192056700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe18c066250 con 0x7fe18c0badd0 2020-11-20T12:17:47.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.192+0000 7fe192056700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe18c067170 con 0x7fe18c159820 2020-11-20T12:17:47.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.192+0000 7fe192056700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c0664f0 con 0x7fe18c1502e0 2020-11-20T12:17:47.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.192+0000 7fe18affd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe18c0badd0 0x7fe18c0beb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55234/0 (socket says 172.21.15.116:55234) 2020-11-20T12:17:47.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.192+0000 7fe18affd700 1 -- 172.21.15.116:0/268258189 learned_addr learned my addr 172.21.15.116:0/268258189 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:47.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18bfff700 1 --2- 172.21.15.116:0/268258189 >> v2:172.21.15.173:3300/0 conn(0x7fe18c159820 0x7fe18c159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18b7fe700 1 --2- 172.21.15.116:0/268258189 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18c1502e0 0x7fe18c0ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18a7fc700 1 -- 172.21.15.116:0/268258189 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 250609028 0 0) 0x7fe18c066250 con 0x7fe18c0badd0 2020-11-20T12:17:47.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18a7fc700 1 -- 172.21.15.116:0/268258189 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe16c003240 con 0x7fe18c0badd0 2020-11-20T12:17:47.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18bfff700 1 -- 172.21.15.116:0/268258189 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18c1502e0 msgr2=0x7fe18c0ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:47.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18a7fc700 1 -- 172.21.15.116:0/268258189 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 174963487 0 0) 0x7fe16c003240 con 0x7fe18c0badd0 2020-11-20T12:17:47.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18bfff700 1 --2- 172.21.15.116:0/268258189 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18c1502e0 0x7fe18c0ba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18bfff700 1 -- 172.21.15.116:0/268258189 >> v1:172.21.15.116:6789/0 conn(0x7fe18c0badd0 legacy=0x7fe18c0beb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18bfff700 1 -- 172.21.15.116:0/268258189 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c066790 con 0x7fe18c159820 2020-11-20T12:17:47.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18bfff700 1 --2- 172.21.15.116:0/268258189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe18c159820 0x7fe18c159c00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fe184009c90 tx=0x7fe1840091d0).ready entity=mon.1 client_cookie=a15ca4058cb8eccc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18a7fc700 1 -- 172.21.15.116:0/268258189 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1740014e0 con 0x7fe18c159820 2020-11-20T12:17:47.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18a7fc700 1 -- 172.21.15.116:0/268258189 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18400b740 con 0x7fe18c159820 2020-11-20T12:17:47.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.196+0000 7fe18a7fc700 1 -- 172.21.15.116:0/268258189 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18400bac0 con 0x7fe18c159820 2020-11-20T12:17:47.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.200+0000 7fe192056700 1 -- 172.21.15.116:0/268258189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe18c159820 msgr2=0x7fe18c159c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.200+0000 7fe192056700 1 --2- 172.21.15.116:0/268258189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe18c159820 0x7fe18c159c00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fe184009c90 tx=0x7fe1840091d0).stop 2020-11-20T12:17:47.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.200+0000 7fe192056700 1 -- 172.21.15.116:0/268258189 shutdown_connections 2020-11-20T12:17:47.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.200+0000 7fe192056700 1 --2- 172.21.15.116:0/268258189 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe18c159820 0x7fe18c159c00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.200+0000 7fe192056700 1 --2- 172.21.15.116:0/268258189 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18c1502e0 0x7fe18c0ba8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.200+0000 7fe192056700 1 -- 172.21.15.116:0/268258189 >> 172.21.15.116:0/268258189 conn(0x7fe18c060170 msgr2=0x7fe18c060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:47.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.200+0000 7fe192056700 1 -- 172.21.15.116:0/268258189 shutdown_connections 2020-11-20T12:17:47.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.200+0000 7fe192056700 1 -- 172.21.15.116:0/268258189 wait complete. 2020-11-20T12:17:47.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.200+0000 7fe192056700 1 Processor -- start 2020-11-20T12:17:47.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe192056700 1 -- start start 2020-11-20T12:17:47.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe192056700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe18c1502e0 0x7fe18c0bd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe192056700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18c159820 0x7fe18c14d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe192056700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe18c150960 0x7fe18c150dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe192056700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c152d30 con 0x7fe18c150960 2020-11-20T12:17:47.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe192056700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c152f60 con 0x7fe18c1502e0 2020-11-20T12:17:47.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe192056700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe18c153200 con 0x7fe18c159820 2020-11-20T12:17:47.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe18c150960 0x7fe18c150dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe18c150960 0x7fe18c150dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45810/0 (socket says 172.21.15.116:45810) 2020-11-20T12:17:47.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18bfff700 1 -- 172.21.15.116:0/2016592788 learned_addr learned my addr 172.21.15.116:0/2016592788 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:47.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18b7fe700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe18c1502e0 0x7fe18c0bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18affd700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18c159820 0x7fe18c14d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18bfff700 1 -- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18c159820 msgr2=0x7fe18c14d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18bfff700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18c159820 0x7fe18c14d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18bfff700 1 -- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe18c1502e0 msgr2=0x7fe18c0bd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18bfff700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe18c1502e0 0x7fe18c0bd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18bfff700 1 -- 172.21.15.116:0/2016592788 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe18c1534a0 con 0x7fe18c150960 2020-11-20T12:17:47.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18b7fe700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe18c1502e0 0x7fe18c0bd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:47.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe18bfff700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe18c150960 0x7fe18c150dc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7fe18c065f50 tx=0x7fe184016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe1837fe700 1 -- 172.21.15.116:0/2016592788 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18400bec0 con 0x7fe18c150960 2020-11-20T12:17:47.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe1837fe700 1 -- 172.21.15.116:0/2016592788 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe18401d660 con 0x7fe18c150960 2020-11-20T12:17:47.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe1837fe700 1 -- 172.21.15.116:0/2016592788 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe18401d9e0 con 0x7fe18c150960 2020-11-20T12:17:47.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.204+0000 7fe192056700 1 -- 172.21.15.116:0/2016592788 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe18c14dcd0 con 0x7fe18c150960 2020-11-20T12:17:47.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.208+0000 7fe192056700 1 -- 172.21.15.116:0/2016592788 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe18c1514d0 con 0x7fe18c150960 2020-11-20T12:17:47.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.208+0000 7fe192056700 1 -- 172.21.15.116:0/2016592788 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe18c05ca30 con 0x7fe18c150960 2020-11-20T12:17:47.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.212+0000 7fe1837fe700 1 -- 172.21.15.116:0/2016592788 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe184017040 con 0x7fe18c150960 2020-11-20T12:17:47.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.212+0000 7fe1837fe700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1680557c0 0x7fe168057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.212+0000 7fe1837fe700 1 -- 172.21.15.116:0/2016592788 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe184071580 con 0x7fe18c150960 2020-11-20T12:17:47.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.212+0000 7fe18b7fe700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1680557c0 0x7fe168057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.212+0000 7fe18b7fe700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1680557c0 0x7fe168057c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fe17c006fd0 tx=0x7fe17c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.216+0000 7fe1837fe700 1 -- 172.21.15.116:0/2016592788 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe18404aed0 con 0x7fe18c150960 2020-11-20T12:17:47.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.376+0000 7fe192056700 1 -- 172.21.15.116:0/2016592788 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe18c067e60 con 0x7fe18c150960 2020-11-20T12:17:47.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.376+0000 7fe1837fe700 1 -- 172.21.15.116:0/2016592788 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe18404aae0 con 0x7fe18c150960 2020-11-20T12:17:47.377 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:47.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 -- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1680557c0 msgr2=0x7fe168057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1680557c0 0x7fe168057c80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7fe17c006fd0 tx=0x7fe17c008040).stop 2020-11-20T12:17:47.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 -- 172.21.15.116:0/2016592788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe18c150960 msgr2=0x7fe18c150dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe18c150960 0x7fe18c150dc0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7fe18c065f50 tx=0x7fe184016000).stop 2020-11-20T12:17:47.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 -- 172.21.15.116:0/2016592788 shutdown_connections 2020-11-20T12:17:47.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1680557c0 0x7fe168057c80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe18c150960 0x7fe18c150dc0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe18c1502e0 0x7fe18c0bd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 --2- 172.21.15.116:0/2016592788 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe18c159820 0x7fe18c14d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 -- 172.21.15.116:0/2016592788 >> 172.21.15.116:0/2016592788 conn(0x7fe18c060170 msgr2=0x7fe18c14e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:47.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 -- 172.21.15.116:0/2016592788 shutdown_connections 2020-11-20T12:17:47.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.380+0000 7fe192056700 1 -- 172.21.15.116:0/2016592788 wait complete. 2020-11-20T12:17:47.402 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:17:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.580+0000 7fa8f6060700 1 Processor -- start 2020-11-20T12:17:47.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.580+0000 7fa8f6060700 1 -- start start 2020-11-20T12:17:47.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.580+0000 7fa8f6060700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa8f00c1bb0 0x7fa8f0156940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.580+0000 7fa8f6060700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f00a38c0 con 0x7fa8f00a28c0 2020-11-20T12:17:47.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.580+0000 7fa8f6060700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f0156e30 con 0x7fa8f00c03f0 2020-11-20T12:17:47.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.580+0000 7fa8f6060700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f00a2e40 con 0x7fa8f00c1bb0 2020-11-20T12:17:47.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.580+0000 7fa8f585f700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa8f00a28c0 0x7fa8f0066570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55248/0 (socket says 172.21.15.116:55248) 2020-11-20T12:17:47.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.580+0000 7fa8f585f700 1 -- 172.21.15.116:0/3659214861 learned_addr learned my addr 172.21.15.116:0/3659214861 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:47.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.580+0000 7fa8f485d700 1 --2- 172.21.15.116:0/3659214861 >> v2:172.21.15.116:3300/0 conn(0x7fa8f00c1bb0 0x7fa8f0156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1214137934 0 0) 0x7fa8f00a38c0 con 0x7fa8f00a28c0 2020-11-20T12:17:47.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8d8003240 con 0x7fa8f00a28c0 2020-11-20T12:17:47.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1564406095 0 0) 0x7fa8d8003240 con 0x7fa8f00a28c0 2020-11-20T12:17:47.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 >> v2:172.21.15.116:3300/0 conn(0x7fa8f00c1bb0 msgr2=0x7fa8f0156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:47.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 --2- 172.21.15.116:0/3659214861 >> v2:172.21.15.116:3300/0 conn(0x7fa8f00c1bb0 0x7fa8f0156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 >> v1:172.21.15.173:6789/0 conn(0x7fa8f00c03f0 legacy=0x7fa8f00677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f00a30e0 con 0x7fa8f00a28c0 2020-11-20T12:17:47.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fa8e8001860 con 0x7fa8f00a28c0 2020-11-20T12:17:47.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 >> v1:172.21.15.116:6789/0 conn(0x7fa8f00a28c0 legacy=0x7fa8f0066570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 --2- 172.21.15.116:0/3659214861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d8003730 0x7fa8d8003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 --2- 172.21.15.116:0/3659214861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d80044e0 0x7fa8d8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 --2- 172.21.15.116:0/3659214861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d8004e50 0x7fa8d800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f00a2e40 con 0x7fa8d80044e0 2020-11-20T12:17:47.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d8004260 con 0x7fa8d8004e50 2020-11-20T12:17:47.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d800bc20 con 0x7fa8d8003730 2020-11-20T12:17:47.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8f485d700 1 --2- 172.21.15.116:0/3659214861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d80044e0 0x7fa8d8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8f505e700 1 --2- 172.21.15.116:0/3659214861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d8003730 0x7fa8d8003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8f585f700 1 --2- 172.21.15.116:0/3659214861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d8004e50 0x7fa8d800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8f485d700 1 -- 172.21.15.116:0/3659214861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d8003730 msgr2=0x7fa8d8003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:47.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8f485d700 1 --2- 172.21.15.116:0/3659214861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d8003730 0x7fa8d8003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8f485d700 1 -- 172.21.15.116:0/3659214861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d8004e50 msgr2=0x7fa8d800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:47.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8f485d700 1 --2- 172.21.15.116:0/3659214861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8d8004e50 0x7fa8d800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8f485d700 1 -- 172.21.15.116:0/3659214861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d8003240 con 0x7fa8d80044e0 2020-11-20T12:17:47.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8f585f700 1 -- 172.21.15.116:0/3659214861 reap_dead start 2020-11-20T12:17:47.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8f485d700 1 --2- 172.21.15.116:0/3659214861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d80044e0 0x7fa8d8004960 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fa8e000bc00 tx=0x7fa8e0006df0).ready entity=mon.0 client_cookie=67ef530b82ae96ae server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000b310 con 0x7fa8d80044e0 2020-11-20T12:17:47.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e000ea90 con 0x7fa8d80044e0 2020-11-20T12:17:47.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.584+0000 7fa8dffff700 1 -- 172.21.15.116:0/3659214861 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000ee10 con 0x7fa8d80044e0 2020-11-20T12:17:47.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.588+0000 7fa8f6060700 1 -- 172.21.15.116:0/3659214861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d80044e0 msgr2=0x7fa8d8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.588+0000 7fa8f6060700 1 --2- 172.21.15.116:0/3659214861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d80044e0 0x7fa8d8004960 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fa8e000bc00 tx=0x7fa8e0006df0).stop 2020-11-20T12:17:47.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.588+0000 7fa8f6060700 1 -- 172.21.15.116:0/3659214861 shutdown_connections 2020-11-20T12:17:47.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.588+0000 7fa8f6060700 1 --2- 172.21.15.116:0/3659214861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8d80044e0 0x7fa8d8004960 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.588+0000 7fa8f6060700 1 -- 172.21.15.116:0/3659214861 >> 172.21.15.116:0/3659214861 conn(0x7fa8f0060c30 msgr2=0x7fa8f00c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:47.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.588+0000 7fa8f6060700 1 -- 172.21.15.116:0/3659214861 shutdown_connections 2020-11-20T12:17:47.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.588+0000 7fa8f6060700 1 -- 172.21.15.116:0/3659214861 wait complete. 2020-11-20T12:17:47.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.588+0000 7fa8f6060700 1 Processor -- start 2020-11-20T12:17:47.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f6060700 1 -- start start 2020-11-20T12:17:47.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f6060700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d80044e0 0x7fa8f00a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f6060700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8f0165320 0x7fa8f00a8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f6060700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f00a8620 0x7fa8f008ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f6060700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0165b70 con 0x7fa8f0165320 2020-11-20T12:17:47.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f6060700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0165e10 con 0x7fa8f00a8620 2020-11-20T12:17:47.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f6060700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f01660b0 con 0x7fa8d80044e0 2020-11-20T12:17:47.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f485d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8f0165320 0x7fa8f00a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f485d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8f0165320 0x7fa8f00a8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45828/0 (socket says 172.21.15.116:45828) 2020-11-20T12:17:47.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f485d700 1 -- 172.21.15.116:0/1631964661 learned_addr learned my addr 172.21.15.116:0/1631964661 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:47.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f505e700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d80044e0 0x7fa8f00a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f585f700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f00a8620 0x7fa8f008ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f485d700 1 -- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d80044e0 msgr2=0x7fa8f00a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f485d700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d80044e0 0x7fa8f00a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f485d700 1 -- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f00a8620 msgr2=0x7fa8f008ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f485d700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f00a8620 0x7fa8f008ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f485d700 1 -- 172.21.15.116:0/1631964661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f00a39a0 con 0x7fa8f0165320 2020-11-20T12:17:47.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f505e700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d80044e0 0x7fa8f00a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:47.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8f485d700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8f0165320 0x7fa8f00a8130 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fa8d8005420 tx=0x7fa8e0019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.592+0000 7fa8dd7fa700 1 -- 172.21.15.116:0/1631964661 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e000b390 con 0x7fa8f0165320 2020-11-20T12:17:47.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.596+0000 7fa8dd7fa700 1 -- 172.21.15.116:0/1631964661 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e001d620 con 0x7fa8f0165320 2020-11-20T12:17:47.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.596+0000 7fa8f6060700 1 -- 172.21.15.116:0/1631964661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f0165780 con 0x7fa8f0165320 2020-11-20T12:17:47.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.596+0000 7fa8dd7fa700 1 -- 172.21.15.116:0/1631964661 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e001d7b0 con 0x7fa8f0165320 2020-11-20T12:17:47.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.596+0000 7fa8f6060700 1 -- 172.21.15.116:0/1631964661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f008f140 con 0x7fa8f0165320 2020-11-20T12:17:47.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.596+0000 7fa8f6060700 1 -- 172.21.15.116:0/1631964661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f008f330 con 0x7fa8f0165320 2020-11-20T12:17:47.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.596+0000 7fa8dd7fa700 1 -- 172.21.15.116:0/1631964661 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa8e002d070 con 0x7fa8f0165320 2020-11-20T12:17:47.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.596+0000 7fa8dd7fa700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8c80557c0 0x7fa8c8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.596+0000 7fa8dd7fa700 1 -- 172.21.15.116:0/1631964661 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa8e0070690 con 0x7fa8f0165320 2020-11-20T12:17:47.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.596+0000 7fa8f505e700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8c80557c0 0x7fa8c8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.600+0000 7fa8f505e700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8c80557c0 0x7fa8c8057c80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fa8ec000bc0 tx=0x7fa8ec009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.604+0000 7fa8dd7fa700 1 -- 172.21.15.116:0/1631964661 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa8f008f330 con 0x7fa8f0165320 2020-11-20T12:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.760+0000 7fa8f6060700 1 -- 172.21.15.116:0/1631964661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8f008f330 con 0x7fa8f0165320 2020-11-20T12:17:47.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.764+0000 7fa8dd7fa700 1 -- 172.21.15.116:0/1631964661 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa8f008f330 con 0x7fa8f0165320 2020-11-20T12:17:47.763 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:47.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 -- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8c80557c0 msgr2=0x7fa8c8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8c80557c0 0x7fa8c8057c80 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7fa8ec000bc0 tx=0x7fa8ec009040).stop 2020-11-20T12:17:47.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 -- 172.21.15.116:0/1631964661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8f0165320 msgr2=0x7fa8f00a8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8f0165320 0x7fa8f00a8130 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7fa8d8005420 tx=0x7fa8e0019000).stop 2020-11-20T12:17:47.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 -- 172.21.15.116:0/1631964661 shutdown_connections 2020-11-20T12:17:47.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8c80557c0 0x7fa8c8057c80 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa8f00a8620 0x7fa8f008ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa8d80044e0 0x7fa8f00a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 --2- 172.21.15.116:0/1631964661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa8f0165320 0x7fa8f00a8130 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 -- 172.21.15.116:0/1631964661 >> 172.21.15.116:0/1631964661 conn(0x7fa8f0060c30 msgr2=0x7fa8f0067960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 -- 172.21.15.116:0/1631964661 shutdown_connections 2020-11-20T12:17:47.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.768+0000 7fa8f6060700 1 -- 172.21.15.116:0/1631964661 wait complete. 2020-11-20T12:17:47.792 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:17:47.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.964+0000 7f0ded64f700 1 Processor -- start 2020-11-20T12:17:47.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.968+0000 7f0ded64f700 1 -- start start 2020-11-20T12:17:47.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.968+0000 7f0ded64f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0de8067910 0x7f0de8067cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.968+0000 7f0ded64f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8151b90 0x7f0de8060bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.968+0000 7f0ded64f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de8152560 con 0x7f0de80610b0 2020-11-20T12:17:47.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.968+0000 7f0ded64f700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de8152800 con 0x7f0de8067910 2020-11-20T12:17:47.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.968+0000 7f0ded64f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de8152aa0 con 0x7f0de8151b90 2020-11-20T12:17:47.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de67fc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8151b90 0x7f0de8060bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de7fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0de80610b0 0x7f0de8065040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55272/0 (socket says 172.21.15.116:55272) 2020-11-20T12:17:47.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de7fff700 1 -- 172.21.15.116:0/477817786 learned_addr learned my addr 172.21.15.116:0/477817786 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:47.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de6ffd700 1 --2- 172.21.15.116:0/477817786 >> v2:172.21.15.173:3300/0 conn(0x7f0de8067910 0x7f0de8067cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de67fc700 1 -- 172.21.15.116:0/477817786 >> v2:172.21.15.173:3300/0 conn(0x7f0de8067910 msgr2=0x7f0de8067cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:47.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de67fc700 1 --2- 172.21.15.116:0/477817786 >> v2:172.21.15.173:3300/0 conn(0x7f0de8067910 0x7f0de8067cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de67fc700 1 -- 172.21.15.116:0/477817786 >> v1:172.21.15.116:6789/0 conn(0x7f0de80610b0 legacy=0x7f0de8065040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de67fc700 1 -- 172.21.15.116:0/477817786 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de8060480 con 0x7f0de8151b90 2020-11-20T12:17:47.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de67fc700 1 --2- 172.21.15.116:0/477817786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8151b90 0x7f0de8060bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f0ddc00b9a0 tx=0x7f0ddc006df0).ready entity=mon.2 client_cookie=31c7b5c695bfd53d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de5ffb700 1 -- 172.21.15.116:0/477817786 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc0097b0 con 0x7f0de8151b90 2020-11-20T12:17:47.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de5ffb700 1 -- 172.21.15.116:0/477817786 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc009910 con 0x7f0de8151b90 2020-11-20T12:17:47.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0de5ffb700 1 -- 172.21.15.116:0/477817786 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc009c90 con 0x7f0de8151b90 2020-11-20T12:17:47.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0ded64f700 1 -- 172.21.15.116:0/477817786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8151b90 msgr2=0x7f0de8060bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.972+0000 7f0ded64f700 1 --2- 172.21.15.116:0/477817786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8151b90 0x7f0de8060bc0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f0ddc00b9a0 tx=0x7f0ddc006df0).stop 2020-11-20T12:17:47.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.976+0000 7f0ded64f700 1 -- 172.21.15.116:0/477817786 shutdown_connections 2020-11-20T12:17:47.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.976+0000 7f0ded64f700 1 --2- 172.21.15.116:0/477817786 >> v2:172.21.15.173:3300/0 conn(0x7f0de8067910 0x7f0de8067cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.976+0000 7f0ded64f700 1 --2- 172.21.15.116:0/477817786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8151b90 0x7f0de8060bc0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.976+0000 7f0ded64f700 1 -- 172.21.15.116:0/477817786 >> 172.21.15.116:0/477817786 conn(0x7f0de80c9a90 msgr2=0x7f0de80c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:47.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.976+0000 7f0ded64f700 1 -- 172.21.15.116:0/477817786 shutdown_connections 2020-11-20T12:17:47.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.976+0000 7f0ded64f700 1 -- 172.21.15.116:0/477817786 wait complete. 2020-11-20T12:17:47.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.976+0000 7f0ded64f700 1 Processor -- start 2020-11-20T12:17:47.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.976+0000 7f0ded64f700 1 -- start start 2020-11-20T12:17:47.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0ded64f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de80610b0 0x7f0de815e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0ded64f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8067910 0x7f0de80c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0ded64f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de8151b90 0x7f0de80c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0ded64f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81647b0 con 0x7f0de80610b0 2020-11-20T12:17:47.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0ded64f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81649e0 con 0x7f0de8151b90 2020-11-20T12:17:47.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0ded64f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de8164c80 con 0x7f0de8067910 2020-11-20T12:17:47.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de80610b0 0x7f0de815e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de80610b0 0x7f0de815e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45840/0 (socket says 172.21.15.116:45840) 2020-11-20T12:17:47.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de6ffd700 1 -- 172.21.15.116:0/827730648 learned_addr learned my addr 172.21.15.116:0/827730648 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:47.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de7fff700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de8151b90 0x7f0de80c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de67fc700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8067910 0x7f0de80c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de6ffd700 1 -- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8067910 msgr2=0x7f0de80c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de6ffd700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8067910 0x7f0de80c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de6ffd700 1 -- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de8151b90 msgr2=0x7f0de80c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:47.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de6ffd700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de8151b90 0x7f0de80c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:47.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de6ffd700 1 -- 172.21.15.116:0/827730648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de8164f20 con 0x7f0de80610b0 2020-11-20T12:17:47.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de7fff700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de8151b90 0x7f0de80c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:47.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de67fc700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8067910 0x7f0de80c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:47.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0de6ffd700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de80610b0 0x7f0de815e6d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f0dd800eba0 tx=0x7f0dd8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0dceffd700 1 -- 172.21.15.116:0/827730648 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800ca60 con 0x7f0de80610b0 2020-11-20T12:17:47.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0dceffd700 1 -- 172.21.15.116:0/827730648 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd800cbc0 con 0x7f0de80610b0 2020-11-20T12:17:47.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0ded64f700 1 -- 172.21.15.116:0/827730648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de8165660 con 0x7f0de80610b0 2020-11-20T12:17:47.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0dceffd700 1 -- 172.21.15.116:0/827730648 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80123f0 con 0x7f0de80610b0 2020-11-20T12:17:47.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.980+0000 7f0ded64f700 1 -- 172.21.15.116:0/827730648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de80c12c0 con 0x7f0de80610b0 2020-11-20T12:17:47.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.984+0000 7f0ded64f700 1 -- 172.21.15.116:0/827730648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de80c6310 con 0x7f0de80610b0 2020-11-20T12:17:47.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.984+0000 7f0dceffd700 1 -- 172.21.15.116:0/827730648 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0dd80127a0 con 0x7f0de80610b0 2020-11-20T12:17:47.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.984+0000 7f0dceffd700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dc40557c0 0x7f0dc4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:47.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.984+0000 7f0dceffd700 1 -- 172.21.15.116:0/827730648 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0dd806eba0 con 0x7f0de80610b0 2020-11-20T12:17:47.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.984+0000 7f0de67fc700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dc40557c0 0x7f0dc4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:47.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.988+0000 7f0de67fc700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dc40557c0 0x7f0dc4057c80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f0de80c5ca0 tx=0x7f0ddc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:47.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:47.992+0000 7f0dceffd700 1 -- 172.21.15.116:0/827730648 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0de80c6310 con 0x7f0de80610b0 2020-11-20T12:17:48.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.148+0000 7f0ded64f700 1 -- 172.21.15.116:0/827730648 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0de80c6310 con 0x7f0de80610b0 2020-11-20T12:17:48.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.148+0000 7f0dceffd700 1 -- 172.21.15.116:0/827730648 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0de80c6310 con 0x7f0de80610b0 2020-11-20T12:17:48.148 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:48.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 -- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dc40557c0 msgr2=0x7f0dc4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:48.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dc40557c0 0x7f0dc4057c80 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f0de80c5ca0 tx=0x7f0ddc015000).stop 2020-11-20T12:17:48.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 -- 172.21.15.116:0/827730648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de80610b0 msgr2=0x7f0de815e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:48.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de80610b0 0x7f0de815e6d0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f0dd800eba0 tx=0x7f0dd8006df0).stop 2020-11-20T12:17:48.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 -- 172.21.15.116:0/827730648 shutdown_connections 2020-11-20T12:17:48.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dc40557c0 0x7f0dc4057c80 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de8151b90 0x7f0de80c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de80610b0 0x7f0de815e6d0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 --2- 172.21.15.116:0/827730648 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de8067910 0x7f0de80c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 -- 172.21.15.116:0/827730648 >> 172.21.15.116:0/827730648 conn(0x7f0de80c9a90 msgr2=0x7f0de8153070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:48.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 -- 172.21.15.116:0/827730648 shutdown_connections 2020-11-20T12:17:48.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.152+0000 7f0ded64f700 1 -- 172.21.15.116:0/827730648 wait complete. 2020-11-20T12:17:48.176 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:17:48.176 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:48.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.344+0000 7fc63bd29700 1 Processor -- start 2020-11-20T12:17:48.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.344+0000 7fc63bd29700 1 -- start start 2020-11-20T12:17:48.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.344+0000 7fc63bd29700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6340aa570 0x7fc6340aa950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:48.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.344+0000 7fc63bd29700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc6340a81a0 0x7fc634151c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:48.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.344+0000 7fc63bd29700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc634152120 con 0x7fc6340ad270 2020-11-20T12:17:48.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.344+0000 7fc63bd29700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6340ab930 con 0x7fc6340a81a0 2020-11-20T12:17:48.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.344+0000 7fc63bd29700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6341523b0 con 0x7fc6340aa570 2020-11-20T12:17:48.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc63a2c6700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc6340ad270 0x7fc6340ad6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55282/0 (socket says 172.21.15.116:55282) 2020-11-20T12:17:48.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc63a2c6700 1 -- 172.21.15.116:0/3208796043 learned_addr learned my addr 172.21.15.116:0/3208796043 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:48.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc639ac5700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6340aa570 0x7fc6340aa950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:48.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc6392c4700 1 --2- 172.21.15.116:0/3208796043 >> v2:172.21.15.173:3300/0 conn(0x7fc6340a81a0 0x7fc634151c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:48.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461143336 0 0) 0x7fc634152120 con 0x7fc6340ad270 2020-11-20T12:17:48.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc61c003240 con 0x7fc6340ad270 2020-11-20T12:17:48.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1660802831 0 0) 0x7fc61c003240 con 0x7fc6340ad270 2020-11-20T12:17:48.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6340aa570 msgr2=0x7fc6340aa950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:48.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6340aa570 0x7fc6340aa950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 >> v2:172.21.15.173:3300/0 conn(0x7fc6340a81a0 msgr2=0x7fc634151c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:48.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 --2- 172.21.15.116:0/3208796043 >> v2:172.21.15.173:3300/0 conn(0x7fc6340a81a0 0x7fc634151c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc634152650 con 0x7fc6340ad270 2020-11-20T12:17:48.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc639ac5700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6340aa570 0x7fc6340aa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:48.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fc62c001860 con 0x7fc6340ad270 2020-11-20T12:17:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 >> v1:172.21.15.116:6789/0 conn(0x7fc6340ad270 legacy=0x7fc6340ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc61c0037b0 0x7fc61c003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc61c0044e0 0x7fc61c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6340aa570 0x7fc61c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:48.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6340ab930 con 0x7fc61c0037b0 2020-11-20T12:17:48.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6341523b0 con 0x7fc61c0044e0 2020-11-20T12:17:48.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61c005340 con 0x7fc6340aa570 2020-11-20T12:17:48.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.348+0000 7fc639ac5700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc61c0037b0 0x7fc61c003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:48.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc63a2c6700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6340aa570 0x7fc61c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:48.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc6392c4700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc61c0044e0 0x7fc61c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:48.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc639ac5700 1 -- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6340aa570 msgr2=0x7fc61c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:48.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc639ac5700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6340aa570 0x7fc61c004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc639ac5700 1 -- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc61c0044e0 msgr2=0x7fc61c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:48.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc639ac5700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc61c0044e0 0x7fc61c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc639ac5700 1 -- 172.21.15.116:0/3208796043 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c003240 con 0x7fc61c0037b0 2020-11-20T12:17:48.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc639ac5700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc61c0037b0 0x7fc61c003bd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fc63000be80 tx=0x7fc630006df0).ready entity=mon.0 client_cookie=b288349ace4d4a20 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:48.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63000b590 con 0x7fc61c0037b0 2020-11-20T12:17:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63000ec40 con 0x7fc61c0037b0 2020-11-20T12:17:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc638ac3700 1 -- 172.21.15.116:0/3208796043 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6300123f0 con 0x7fc61c0037b0 2020-11-20T12:17:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc63bd29700 1 -- 172.21.15.116:0/3208796043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc61c0037b0 msgr2=0x7fc61c003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc63bd29700 1 --2- 172.21.15.116:0/3208796043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc61c0037b0 0x7fc61c003bd0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fc63000be80 tx=0x7fc630006df0).stop 2020-11-20T12:17:48.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc63a2c6700 1 -- 172.21.15.116:0/3208796043 reap_dead start 2020-11-20T12:17:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc63bd29700 1 -- 172.21.15.116:0/3208796043 shutdown_connections 2020-11-20T12:17:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc63bd29700 1 -- 172.21.15.116:0/3208796043 >> 172.21.15.116:0/3208796043 conn(0x7fc6340678a0 msgr2=0x7fc634067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc63bd29700 1 -- 172.21.15.116:0/3208796043 shutdown_connections 2020-11-20T12:17:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.352+0000 7fc63bd29700 1 -- 172.21.15.116:0/3208796043 wait complete. 2020-11-20T12:17:48.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc63bd29700 1 Processor -- start 2020-11-20T12:17:48.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc63bd29700 1 -- start start 2020-11-20T12:17:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc63bd29700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc634163a30 0x7fc6341618e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc63bd29700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc634163e10 0x7fc634161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc63bd29700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc63414bb90 0x7fc6340c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:48.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc63bd29700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6340a29f0 con 0x7fc634163a30 2020-11-20T12:17:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc63bd29700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63414c430 con 0x7fc634163e10 2020-11-20T12:17:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc63bd29700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc63414c6d0 con 0x7fc63414bb90 2020-11-20T12:17:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc639ac5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc634163a30 0x7fc6341618e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:48.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc639ac5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc634163a30 0x7fc6341618e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45860/0 (socket says 172.21.15.116:45860) 2020-11-20T12:17:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc639ac5700 1 -- 172.21.15.116:0/3473563714 learned_addr learned my addr 172.21.15.116:0/3473563714 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc6392c4700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc634163e10 0x7fc634161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc63a2c6700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc63414bb90 0x7fc6340c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc639ac5700 1 -- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc63414bb90 msgr2=0x7fc6340c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:48.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc639ac5700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc63414bb90 0x7fc6340c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc639ac5700 1 -- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc634163e10 msgr2=0x7fc634161ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc639ac5700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc634163e10 0x7fc634161ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc639ac5700 1 -- 172.21.15.116:0/3473563714 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63414c970 con 0x7fc634163a30 2020-11-20T12:17:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.356+0000 7fc639ac5700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc634163a30 0x7fc6341618e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fc630013000 tx=0x7fc630019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:48.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.360+0000 7fc621ffb700 1 -- 172.21.15.116:0/3473563714 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc630012e30 con 0x7fc634163a30 2020-11-20T12:17:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.360+0000 7fc621ffb700 1 -- 172.21.15.116:0/3473563714 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc63001b600 con 0x7fc634163a30 2020-11-20T12:17:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.360+0000 7fc63bd29700 1 -- 172.21.15.116:0/3473563714 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6340a2c50 con 0x7fc634163a30 2020-11-20T12:17:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.360+0000 7fc621ffb700 1 -- 172.21.15.116:0/3473563714 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc63001b980 con 0x7fc634163a30 2020-11-20T12:17:48.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.360+0000 7fc63bd29700 1 -- 172.21.15.116:0/3473563714 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63414bff0 con 0x7fc634163a30 2020-11-20T12:17:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.360+0000 7fc63bd29700 1 -- 172.21.15.116:0/3473563714 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6340a2de0 con 0x7fc634163a30 2020-11-20T12:17:48.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.360+0000 7fc621ffb700 1 -- 172.21.15.116:0/3473563714 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc63002c020 con 0x7fc634163a30 2020-11-20T12:17:48.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.360+0000 7fc621ffb700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc60c0557c0 0x7fc60c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:48.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.364+0000 7fc621ffb700 1 -- 172.21.15.116:0/3473563714 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc63006fb20 con 0x7fc634163a30 2020-11-20T12:17:48.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.364+0000 7fc6392c4700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc60c0557c0 0x7fc60c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:48.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.364+0000 7fc6392c4700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc60c0557c0 0x7fc60c057c80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fc624006860 tx=0x7fc624008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:48.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.368+0000 7fc621ffb700 1 -- 172.21.15.116:0/3473563714 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc6300459c0 con 0x7fc634163a30 2020-11-20T12:17:48.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.576+0000 7fc63bd29700 1 -- 172.21.15.116:0/3473563714 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc63414cb00 con 0x7fc634163a30 2020-11-20T12:17:48.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.576+0000 7fc621ffb700 1 -- 172.21.15.116:0/3473563714 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+2849 (secure 0 0 0) 0x7fc630048fe0 con 0x7fc634163a30 2020-11-20T12:17:48.576 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:48.576 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1096724826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1096724826},{"type":"v1","addr":"172.21.15.173:6837","nonce":1096724826}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/1589443098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1589443098},{"type":"v1","addr":"172.21.15.173:6835","nonce":1589443098}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/966300328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":966300328},{"type":"v1","addr":"172.21.15.116:6833","nonce":966300328}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:17:40.163408+0000","modified":"2020-11-20T12:17:42.171423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5493":{"gid":5493,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/2908037349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2908037349},{"type":"v1","addr":"172.21.15.116:6835","nonce":2908037349}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-20T12:17:48.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 -- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc60c0557c0 msgr2=0x7fc60c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc60c0557c0 0x7fc60c057c80 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7fc624006860 tx=0x7fc624008040).stop 2020-11-20T12:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 -- 172.21.15.116:0/3473563714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc634163a30 msgr2=0x7fc6341618e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc634163a30 0x7fc6341618e0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fc630013000 tx=0x7fc630019000).stop 2020-11-20T12:17:48.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 -- 172.21.15.116:0/3473563714 shutdown_connections 2020-11-20T12:17:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc60c0557c0 0x7fc60c057c80 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc63414bb90 0x7fc6340c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc634163a30 0x7fc6341618e0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 --2- 172.21.15.116:0/3473563714 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc634163e10 0x7fc634161ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:48.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 -- 172.21.15.116:0/3473563714 >> 172.21.15.116:0/3473563714 conn(0x7fc6340678a0 msgr2=0x7fc634150a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 -- 172.21.15.116:0/3473563714 shutdown_connections 2020-11-20T12:17:48.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:48.580+0000 7fc63bd29700 1 -- 172.21.15.116:0/3473563714 wait complete. 2020-11-20T12:17:48.582 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 20 2020-11-20T12:17:49.006 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:17:40.163408+0000', 'modified': '2020-11-20T12:17:42.171423+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5493': {'gid': 5493, 'name': 'a', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/2908037349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2908037349}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 2908037349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:17:49.007 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:17:49.007 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:17:49.201 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":5493,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":9.0186388869999998,"mdsmap_epoch":20,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":14.536280146999999} 2020-11-20T12:17:49.201 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 5493, "mdsmap_epoch": 20, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 9.018638887, "state": "up:active", "uptime": 14.536280147, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:17:49.202 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:17:49.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:49.203 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-20T12:17:49.220 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:17:49.220 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:49.283 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:49.284 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:17:49.308 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:49.308 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:17:49.314 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:17:49.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:49.315 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-20T12:17:49.343 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:17:49.343 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:17:49.344 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-20T12:17:49.344 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-20T12:17:49.344 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:17:49.345 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:17:49.345 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:17:49.346 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:17:49.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:49.346 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:17:49.361 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:17:49.362 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:17:49.362 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:17:49.362 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:49.363 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:17:49.363 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:49.363 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:49.363 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:49.363 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:49.364 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:49.364 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:49.364 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:49.364 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:17:49.364 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:49.365 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:17:49.365 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:49.365 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:17:49.365 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:49.366 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:17:49.366 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:49.366 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:17:49.366 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:49.366 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:17:49.367 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:49.367 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:17:49.367 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:49.367 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:17:49.367 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:49.368 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:49.368 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:17:49.368 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:17:49.368 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:49.368 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:49.369 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:17:49.369 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:17:49.369 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:49.370 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:49.371 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:17:49.445 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:17:49.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:49.449 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:17:49.462 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:17:49.462 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:17:49.462 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:17:49.463 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:17:49.463 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:17:49.463 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:17:49.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:49.464 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:17:49.465 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:17:49.465 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:17:49.466 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:17:49.466 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:17:49.466 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:17:49.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:49.558 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:17:49.573 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:17:49.573 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:17:49.575 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:17:49.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:49.576 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2020-11-20T12:17:49.606 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:49.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.780+0000 7f104184c700 1 Processor -- start 2020-11-20T12:17:49.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.780+0000 7f104184c700 1 -- start start 2020-11-20T12:17:49.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f104184c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f103c13f1d0 0x7f103c0a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:49.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f104184c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f103c0a40c0 0x7f103c0c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:49.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f104184c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f103c13f5b0 0x7f103c0c44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:49.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f104184c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103c0a5680 con 0x7f103c0a40c0 2020-11-20T12:17:49.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f104184c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103c0a58e0 con 0x7f103c13f1d0 2020-11-20T12:17:49.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f104184c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103c0a5b80 con 0x7f103c13f5b0 2020-11-20T12:17:49.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103bfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f103c0a40c0 0x7f103c0c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:49.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103bfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f103c0a40c0 0x7f103c0c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45874/0 (socket says 172.21.15.116:45874) 2020-11-20T12:17:49.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103bfff700 1 -- 172.21.15.116:0/615318061 learned_addr learned my addr 172.21.15.116:0/615318061 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:49.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f104104b700 1 --2- 172.21.15.116:0/615318061 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f103c13f5b0 0x7f103c0c44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:49.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f104084a700 1 --2- 172.21.15.116:0/615318061 >> v2:172.21.15.173:3300/0 conn(0x7f103c13f1d0 0x7f103c0a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:49.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103bfff700 1 -- 172.21.15.116:0/615318061 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f103c13f5b0 msgr2=0x7f103c0c44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:49.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103bfff700 1 --2- 172.21.15.116:0/615318061 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f103c13f5b0 0x7f103c0c44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:49.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103bfff700 1 -- 172.21.15.116:0/615318061 >> v2:172.21.15.173:3300/0 conn(0x7f103c13f1d0 msgr2=0x7f103c0a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:49.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103bfff700 1 --2- 172.21.15.116:0/615318061 >> v2:172.21.15.173:3300/0 conn(0x7f103c13f1d0 0x7f103c0a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:49.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103bfff700 1 -- 172.21.15.116:0/615318061 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c0a5e20 con 0x7f103c0a40c0 2020-11-20T12:17:49.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f104104b700 1 --2- 172.21.15.116:0/615318061 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f103c13f5b0 0x7f103c0c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:49.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f104084a700 1 --2- 172.21.15.116:0/615318061 >> v2:172.21.15.173:3300/0 conn(0x7f103c13f1d0 0x7f103c0a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:49.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103bfff700 1 --2- 172.21.15.116:0/615318061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f103c0a40c0 0x7f103c0c3fb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f103c065960 tx=0x7f1024006eb0).ready entity=mon.0 client_cookie=3e1c671ffcfbf242 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:49.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103b7fe700 1 -- 172.21.15.116:0/615318061 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10240098d0 con 0x7f103c0a40c0 2020-11-20T12:17:49.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103b7fe700 1 -- 172.21.15.116:0/615318061 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1024009a30 con 0x7f103c0a40c0 2020-11-20T12:17:49.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.784+0000 7f103b7fe700 1 -- 172.21.15.116:0/615318061 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024009db0 con 0x7f103c0a40c0 2020-11-20T12:17:49.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.788+0000 7f104184c700 1 -- 172.21.15.116:0/615318061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f103c0a40c0 msgr2=0x7f103c0c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:49.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.788+0000 7f104184c700 1 --2- 172.21.15.116:0/615318061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f103c0a40c0 0x7f103c0c3fb0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f103c065960 tx=0x7f1024006eb0).stop 2020-11-20T12:17:49.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.788+0000 7f104184c700 1 -- 172.21.15.116:0/615318061 shutdown_connections 2020-11-20T12:17:49.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.788+0000 7f104184c700 1 --2- 172.21.15.116:0/615318061 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f103c13f5b0 0x7f103c0c44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:49.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.788+0000 7f104184c700 1 --2- 172.21.15.116:0/615318061 >> v2:172.21.15.173:3300/0 conn(0x7f103c13f1d0 0x7f103c0a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:49.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.788+0000 7f104184c700 1 --2- 172.21.15.116:0/615318061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f103c0a40c0 0x7f103c0c3fb0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:49.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.788+0000 7f104184c700 1 -- 172.21.15.116:0/615318061 >> 172.21.15.116:0/615318061 conn(0x7f103c062520 msgr2=0x7f103c062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:49.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.788+0000 7f104184c700 1 -- 172.21.15.116:0/615318061 shutdown_connections 2020-11-20T12:17:49.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.788+0000 7f104184c700 1 -- 172.21.15.116:0/615318061 wait complete. 2020-11-20T12:17:49.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.788+0000 7f104184c700 1 Processor -- start 2020-11-20T12:17:49.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.788+0000 7f104184c700 1 -- start start 2020-11-20T12:17:49.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f104184c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f103c13f1d0 0x7f103c1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:49.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f104184c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f103c13f5b0 0x7f103c164ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:49.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f104184c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f103c1654e0 0x7f103c09dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:49.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f104184c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c165ac0 con 0x7f103c13f5b0 2020-11-20T12:17:49.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f104184c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c165d60 con 0x7f103c13f1d0 2020-11-20T12:17:49.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f104184c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103c09d5a0 con 0x7f103c1654e0 2020-11-20T12:17:49.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f103bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f103c13f5b0 0x7f103c164ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:49.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f103bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f103c13f5b0 0x7f103c164ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45878/0 (socket says 172.21.15.116:45878) 2020-11-20T12:17:49.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f103bfff700 1 -- 172.21.15.116:0/627450051 learned_addr learned my addr 172.21.15.116:0/627450051 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:49.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f104104b700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f103c1654e0 0x7f103c09dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:49.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f104084a700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f103c13f1d0 0x7f103c1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:49.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f103bfff700 1 -- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f103c1654e0 msgr2=0x7f103c09dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:49.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f103bfff700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f103c1654e0 0x7f103c09dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:49.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f103bfff700 1 -- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f103c13f1d0 msgr2=0x7f103c1555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:49.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f103bfff700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f103c13f1d0 0x7f103c1555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:49.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f103bfff700 1 -- 172.21.15.116:0/627450051 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c09d840 con 0x7f103c13f5b0 2020-11-20T12:17:49.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f103bfff700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f103c13f5b0 0x7f103c164ff0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f103c0c1ad0 tx=0x7f1024015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:49.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f1038ff9700 1 -- 172.21.15.116:0/627450051 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024011dd0 con 0x7f103c13f5b0 2020-11-20T12:17:49.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f1038ff9700 1 -- 172.21.15.116:0/627450051 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102401e460 con 0x7f103c13f5b0 2020-11-20T12:17:49.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f104184c700 1 -- 172.21.15.116:0/627450051 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f103c0c1620 con 0x7f103c13f5b0 2020-11-20T12:17:49.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f1038ff9700 1 -- 172.21.15.116:0/627450051 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f102401e800 con 0x7f103c13f5b0 2020-11-20T12:17:49.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.792+0000 7f104184c700 1 -- 172.21.15.116:0/627450051 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103c09e1d0 con 0x7f103c13f5b0 2020-11-20T12:17:49.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.796+0000 7f1038ff9700 1 -- 172.21.15.116:0/627450051 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1024025030 con 0x7f103c13f5b0 2020-11-20T12:17:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.796+0000 7f104184c700 1 -- 172.21.15.116:0/627450051 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103c09d9d0 con 0x7f103c13f5b0 2020-11-20T12:17:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.796+0000 7f1038ff9700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10180557c0 0x7f1018057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.796+0000 7f1038ff9700 1 -- 172.21.15.116:0/627450051 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f102406fee0 con 0x7f103c13f5b0 2020-11-20T12:17:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.796+0000 7f104084a700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10180557c0 0x7f1018057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:49.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.796+0000 7f104084a700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10180557c0 0x7f1018057c80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f1034006f50 tx=0x7f1034008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:49.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.800+0000 7f1038ff9700 1 -- 172.21.15.116:0/627450051 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1024045d20 con 0x7f103c13f5b0 2020-11-20T12:17:49.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.964+0000 7f104184c700 1 -- 172.21.15.116:0/627450051 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f103c165f20 con 0x7f103c13f5b0 2020-11-20T12:17:49.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.964+0000 7f1038ff9700 1 -- 172.21.15.116:0/627450051 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f1024031200 con 0x7f103c13f5b0 2020-11-20T12:17:49.966 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:49.966 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":45,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:17:41.167389+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:17:38.002878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-20T12:17:38.995456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:49.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 -- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10180557c0 msgr2=0x7f1018057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:49.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10180557c0 0x7f1018057c80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f1034006f50 tx=0x7f1034008040).stop 2020-11-20T12:17:49.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 -- 172.21.15.116:0/627450051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f103c13f5b0 msgr2=0x7f103c164ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:49.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f103c13f5b0 0x7f103c164ff0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f103c0c1ad0 tx=0x7f1024015000).stop 2020-11-20T12:17:49.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 -- 172.21.15.116:0/627450051 shutdown_connections 2020-11-20T12:17:49.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10180557c0 0x7f1018057c80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:49.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f103c1654e0 0x7f103c09dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:49.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f103c13f1d0 0x7f103c1555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:49.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 --2- 172.21.15.116:0/627450051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f103c13f5b0 0x7f103c164ff0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:49.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 -- 172.21.15.116:0/627450051 >> 172.21.15.116:0/627450051 conn(0x7f103c062520 msgr2=0x7f103c155f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:49.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 -- 172.21.15.116:0/627450051 shutdown_connections 2020-11-20T12:17:49.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:49.968+0000 7f104184c700 1 -- 172.21.15.116:0/627450051 wait complete. 2020-11-20T12:17:49.995 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:17:38.002878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:17:38.995456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:17:49.996 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:50.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.176+0000 7fe38171c700 1 Processor -- start 2020-11-20T12:17:50.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.176+0000 7fe38171c700 1 -- start start 2020-11-20T12:17:50.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe37c067910 0x7fe37c067cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe37c0c1460 0x7fe37c0c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe37c0ba5e0 con 0x7fe37c14f660 2020-11-20T12:17:50.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c062300 con 0x7fe37c0c1460 2020-11-20T12:17:50.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe37c0c1de0 con 0x7fe37c067910 2020-11-20T12:17:50.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe37bfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe37c0c1460 0x7fe37c0c18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe37bfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe37c0c1460 0x7fe37c0c18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45886/0 (socket says 172.21.15.116:45886) 2020-11-20T12:17:50.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe37bfff700 1 -- 172.21.15.116:0/1591789002 learned_addr learned my addr 172.21.15.116:0/1591789002 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:50.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe37bfff700 1 -- 172.21.15.116:0/1591789002 >> v2:172.21.15.173:3300/0 conn(0x7fe37c067910 msgr2=0x7fe37c067cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:50.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe37bfff700 1 --2- 172.21.15.116:0/1591789002 >> v2:172.21.15.173:3300/0 conn(0x7fe37c067910 0x7fe37c067cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:50.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe37bfff700 1 -- 172.21.15.116:0/1591789002 >> v1:172.21.15.173:6789/0 conn(0x7fe37c14f660 legacy=0x7fe37c061bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:50.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe37bfff700 1 -- 172.21.15.116:0/1591789002 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0bc4a0 con 0x7fe37c0c1460 2020-11-20T12:17:50.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe37bfff700 1 --2- 172.21.15.116:0/1591789002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0c1460 0x7fe37c0c18f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fe37c064d40 tx=0x7fe374009290).ready entity=mon.0 client_cookie=5b2f6f79a1c30b66 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:50.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe379ffb700 1 -- 172.21.15.116:0/1591789002 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400bc10 con 0x7fe37c0c1460 2020-11-20T12:17:50.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe379ffb700 1 -- 172.21.15.116:0/1591789002 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37400bd70 con 0x7fe37c0c1460 2020-11-20T12:17:50.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe379ffb700 1 -- 172.21.15.116:0/1591789002 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe374011610 con 0x7fe37c0c1460 2020-11-20T12:17:50.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 -- 172.21.15.116:0/1591789002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0c1460 msgr2=0x7fe37c0c18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 --2- 172.21.15.116:0/1591789002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0c1460 0x7fe37c0c18f0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fe37c064d40 tx=0x7fe374009290).stop 2020-11-20T12:17:50.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 -- 172.21.15.116:0/1591789002 shutdown_connections 2020-11-20T12:17:50.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 --2- 172.21.15.116:0/1591789002 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c0c1460 0x7fe37c0c18f0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 --2- 172.21.15.116:0/1591789002 >> v2:172.21.15.173:3300/0 conn(0x7fe37c067910 0x7fe37c067cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:50.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 -- 172.21.15.116:0/1591789002 >> 172.21.15.116:0/1591789002 conn(0x7fe37c0c5680 msgr2=0x7fe37c0c5a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:50.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 -- 172.21.15.116:0/1591789002 shutdown_connections 2020-11-20T12:17:50.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.180+0000 7fe38171c700 1 -- 172.21.15.116:0/1591789002 wait complete. 2020-11-20T12:17:50.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.184+0000 7fe38171c700 1 Processor -- start 2020-11-20T12:17:50.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.184+0000 7fe38171c700 1 -- start start 2020-11-20T12:17:50.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.184+0000 7fe38171c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe37c067910 0x7fe37c0a7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.184+0000 7fe38171c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe37c0c1460 0x7fe37c161a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.184+0000 7fe38171c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c14f660 0x7fe37c145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe38171c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0a8190 con 0x7fe37c14f660 2020-11-20T12:17:50.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe38171c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c1349f0 con 0x7fe37c0c1460 2020-11-20T12:17:50.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe38171c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe37c0a9c80 con 0x7fe37c067910 2020-11-20T12:17:50.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe37bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c14f660 0x7fe37c145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe37bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c14f660 0x7fe37c145a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45896/0 (socket says 172.21.15.116:45896) 2020-11-20T12:17:50.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe37bfff700 1 -- 172.21.15.116:0/3026326837 learned_addr learned my addr 172.21.15.116:0/3026326837 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:50.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe37a7fc700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe37c0c1460 0x7fe37c161a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe37affd700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe37c067910 0x7fe37c0a7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe37bfff700 1 -- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe37c067910 msgr2=0x7fe37c0a7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe37bfff700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe37c067910 0x7fe37c0a7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe37bfff700 1 -- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe37c0c1460 msgr2=0x7fe37c161a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe37bfff700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe37c0c1460 0x7fe37c161a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe37bfff700 1 -- 172.21.15.116:0/3026326837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe37c0a9f20 con 0x7fe37c14f660 2020-11-20T12:17:50.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe37bfff700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c14f660 0x7fe37c145a30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fe3740092d0 tx=0x7fe374016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:50.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe362ffd700 1 -- 172.21.15.116:0/3026326837 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37400bee0 con 0x7fe37c14f660 2020-11-20T12:17:50.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe362ffd700 1 -- 172.21.15.116:0/3026326837 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe374010b90 con 0x7fe37c14f660 2020-11-20T12:17:50.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe38171c700 1 -- 172.21.15.116:0/3026326837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe37c0aa1c0 con 0x7fe37c14f660 2020-11-20T12:17:50.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe362ffd700 1 -- 172.21.15.116:0/3026326837 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37401c470 con 0x7fe37c14f660 2020-11-20T12:17:50.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.188+0000 7fe38171c700 1 -- 172.21.15.116:0/3026326837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe37c161f10 con 0x7fe37c14f660 2020-11-20T12:17:50.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.192+0000 7fe38171c700 1 -- 172.21.15.116:0/3026326837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe37c0c54a0 con 0x7fe37c14f660 2020-11-20T12:17:50.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.192+0000 7fe362ffd700 1 -- 172.21.15.116:0/3026326837 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe374017070 con 0x7fe37c14f660 2020-11-20T12:17:50.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.192+0000 7fe362ffd700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3580557c0 0x7fe358057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.192+0000 7fe362ffd700 1 -- 172.21.15.116:0/3026326837 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe374071670 con 0x7fe37c14f660 2020-11-20T12:17:50.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.196+0000 7fe37affd700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3580557c0 0x7fe358057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.196+0000 7fe37affd700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3580557c0 0x7fe358057c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fe36c00b770 tx=0x7fe36c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:50.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.196+0000 7fe362ffd700 1 -- 172.21.15.116:0/3026326837 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe374047480 con 0x7fe37c14f660 2020-11-20T12:17:50.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 -- 172.21.15.116:0/3026326837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe37c05ca30 con 0x7fe37c14f660 2020-11-20T12:17:50.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe362ffd700 1 -- 172.21.15.116:0/3026326837 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe374015070 con 0x7fe37c14f660 2020-11-20T12:17:50.359 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 -- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3580557c0 msgr2=0x7fe358057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3580557c0 0x7fe358057c80 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fe36c00b770 tx=0x7fe36c008040).stop 2020-11-20T12:17:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 -- 172.21.15.116:0/3026326837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c14f660 msgr2=0x7fe37c145a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c14f660 0x7fe37c145a30 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7fe3740092d0 tx=0x7fe374016000).stop 2020-11-20T12:17:50.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 -- 172.21.15.116:0/3026326837 shutdown_connections 2020-11-20T12:17:50.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3580557c0 0x7fe358057c80 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe37c14f660 0x7fe37c145a30 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe37c067910 0x7fe37c0a7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 --2- 172.21.15.116:0/3026326837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe37c0c1460 0x7fe37c161a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 -- 172.21.15.116:0/3026326837 >> 172.21.15.116:0/3026326837 conn(0x7fe37c0c5680 msgr2=0x7fe37c151e50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:50.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.360+0000 7fe38171c700 1 -- 172.21.15.116:0/3026326837 shutdown_connections 2020-11-20T12:17:50.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.364+0000 7fe38171c700 1 -- 172.21.15.116:0/3026326837 wait complete. 2020-11-20T12:17:50.385 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:50.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.552+0000 7fa2a4bd5700 1 Processor -- start 2020-11-20T12:17:50.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.552+0000 7fa2a4bd5700 1 -- start start 2020-11-20T12:17:50.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa2a4bd5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a013ad30 0x7fa2a00a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa2a4bd5700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a0152a80 con 0x7fa2a013b110 2020-11-20T12:17:50.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa2a4bd5700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a0152540 con 0x7fa2a0061250 2020-11-20T12:17:50.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa2a4bd5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a01527e0 con 0x7fa2a013ad30 2020-11-20T12:17:50.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29effd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa2a013b110 0x7fa2a0151ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55336/0 (socket says 172.21.15.116:55336) 2020-11-20T12:17:50.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29effd700 1 -- 172.21.15.116:0/1817668867 learned_addr learned my addr 172.21.15.116:0/1817668867 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:50.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29f7fe700 1 --2- 172.21.15.116:0/1817668867 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a013ad30 0x7fa2a00a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447217019 0 0) 0x7fa2a0152a80 con 0x7fa2a013b110 2020-11-20T12:17:50.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa28c003240 con 0x7fa2a013b110 2020-11-20T12:17:50.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 321671139 0 0) 0x7fa28c003240 con 0x7fa2a013b110 2020-11-20T12:17:50.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a013ad30 msgr2=0x7fa2a00a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:50.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 --2- 172.21.15.116:0/1817668867 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a013ad30 0x7fa2a00a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 >> v1:172.21.15.173:6789/0 conn(0x7fa2a0061250 legacy=0x7fa2a00ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a00ac910 con 0x7fa2a013b110 2020-11-20T12:17:50.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fa2940014e0 con 0x7fa2a013b110 2020-11-20T12:17:50.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 >> v1:172.21.15.116:6789/0 conn(0x7fa2a013b110 legacy=0x7fa2a0151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 --2- 172.21.15.116:0/1817668867 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a013ad30 0x7fa28c003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 --2- 172.21.15.116:0/1817668867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c003c70 0x7fa28c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 --2- 172.21.15.116:0/1817668867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa28c0049d0 0x7fa28c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a01527e0 con 0x7fa28c003c70 2020-11-20T12:17:50.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c004200 con 0x7fa28c0049d0 2020-11-20T12:17:50.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.556+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c00b730 con 0x7fa2a013ad30 2020-11-20T12:17:50.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa29effd700 1 --2- 172.21.15.116:0/1817668867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c003c70 0x7fa28c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa29effd700 1 -- 172.21.15.116:0/1817668867 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a013ad30 msgr2=0x7fa28c003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:50.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa29effd700 1 --2- 172.21.15.116:0/1817668867 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a013ad30 0x7fa28c003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:50.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa29effd700 1 -- 172.21.15.116:0/1817668867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa28c0049d0 msgr2=0x7fa28c004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:50.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa29effd700 1 --2- 172.21.15.116:0/1817668867 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa28c0049d0 0x7fa28c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:17:50.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa29effd700 1 -- 172.21.15.116:0/1817668867 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c003240 con 0x7fa28c003c70 2020-11-20T12:17:50.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa29effd700 1 --2- 172.21.15.116:0/1817668867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c003c70 0x7fa28c0044e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fa2a00c5e80 tx=0x7fa2940077d0).ready entity=mon.0 client_cookie=91653a40d4147a79 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:50.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2940070d0 con 0x7fa28c003c70 2020-11-20T12:17:50.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa294007230 con 0x7fa28c003c70 2020-11-20T12:17:50.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa29e7fc700 1 -- 172.21.15.116:0/1817668867 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2940055f0 con 0x7fa28c003c70 2020-11-20T12:17:50.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/1817668867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c003c70 msgr2=0x7fa28c0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa2a4bd5700 1 --2- 172.21.15.116:0/1817668867 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa28c003c70 0x7fa28c0044e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7fa2a00c5e80 tx=0x7fa2940077d0).stop 2020-11-20T12:17:50.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa29ffff700 1 -- 172.21.15.116:0/1817668867 reap_dead start 2020-11-20T12:17:50.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/1817668867 shutdown_connections 2020-11-20T12:17:50.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/1817668867 >> 172.21.15.116:0/1817668867 conn(0x7fa2a00678a0 msgr2=0x7fa2a0067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:50.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/1817668867 shutdown_connections 2020-11-20T12:17:50.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.560+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/1817668867 wait complete. 2020-11-20T12:17:50.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.564+0000 7fa2a4bd5700 1 Processor -- start 2020-11-20T12:17:50.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.564+0000 7fa2a4bd5700 1 -- start start 2020-11-20T12:17:50.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.564+0000 7fa2a4bd5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2a00a68c0 0x7fa2a0164e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.564+0000 7fa2a4bd5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a00a1390 0x7fa2a01616b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.564+0000 7fa2a4bd5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a00a9ef0 0x7fa2a00aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.564+0000 7fa2a4bd5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a0138940 con 0x7fa2a00a68c0 2020-11-20T12:17:50.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.564+0000 7fa2a4bd5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a01652f0 con 0x7fa2a00a1390 2020-11-20T12:17:50.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.564+0000 7fa2a4bd5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00a1910 con 0x7fa2a00a9ef0 2020-11-20T12:17:50.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2a00a68c0 0x7fa2a0164e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2a00a68c0 0x7fa2a0164e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45912/0 (socket says 172.21.15.116:45912) 2020-11-20T12:17:50.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29f7fe700 1 -- 172.21.15.116:0/724481056 learned_addr learned my addr 172.21.15.116:0/724481056 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:50.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29effd700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a00a1390 0x7fa2a01616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29ffff700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a00a9ef0 0x7fa2a00aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29f7fe700 1 -- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a00a9ef0 msgr2=0x7fa2a00aa370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29f7fe700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a00a9ef0 0x7fa2a00aa370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29f7fe700 1 -- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a00a1390 msgr2=0x7fa2a01616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29f7fe700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a00a1390 0x7fa2a01616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29f7fe700 1 -- 172.21.15.116:0/724481056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a00a1bb0 con 0x7fa2a00a68c0 2020-11-20T12:17:50.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29ffff700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a00a9ef0 0x7fa2a00aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:50.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa29f7fe700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2a00a68c0 0x7fa2a0164e00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fa290006d00 tx=0x7fa290006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:50.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa27f7fe700 1 -- 172.21.15.116:0/724481056 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290016070 con 0x7fa2a00a68c0 2020-11-20T12:17:50.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa27f7fe700 1 -- 172.21.15.116:0/724481056 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29000c8b0 con 0x7fa2a00a68c0 2020-11-20T12:17:50.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/724481056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a00a1e50 con 0x7fa2a00a68c0 2020-11-20T12:17:50.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa27f7fe700 1 -- 172.21.15.116:0/724481056 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29000cc30 con 0x7fa2a00a68c0 2020-11-20T12:17:50.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/724481056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a0161210 con 0x7fa2a00a68c0 2020-11-20T12:17:50.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/724481056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a00aa860 con 0x7fa2a00a68c0 2020-11-20T12:17:50.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.568+0000 7fa27f7fe700 1 -- 172.21.15.116:0/724481056 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa290012600 con 0x7fa2a00a68c0 2020-11-20T12:17:50.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.572+0000 7fa27f7fe700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2780557c0 0x7fa278057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.572+0000 7fa27f7fe700 1 -- 172.21.15.116:0/724481056 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa29006e8d0 con 0x7fa2a00a68c0 2020-11-20T12:17:50.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.572+0000 7fa29effd700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2780557c0 0x7fa278057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.572+0000 7fa29effd700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2780557c0 0x7fa278057c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fa2a005f2d0 tx=0x7fa294015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:50.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.576+0000 7fa27f7fe700 1 -- 172.21.15.116:0/724481056 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa2900446a0 con 0x7fa2a00a68c0 2020-11-20T12:17:50.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.736+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/724481056 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa2a0044f60 con 0x7fa2a00a68c0 2020-11-20T12:17:50.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.736+0000 7fa27f7fe700 1 -- 172.21.15.116:0/724481056 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa290047cc0 con 0x7fa2a00a68c0 2020-11-20T12:17:50.738 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:50.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2780557c0 msgr2=0x7fa278057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2780557c0 0x7fa278057c80 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fa2a005f2d0 tx=0x7fa294015000).stop 2020-11-20T12:17:50.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/724481056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2a00a68c0 msgr2=0x7fa2a0164e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2a00a68c0 0x7fa2a0164e00 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fa290006d00 tx=0x7fa290006df0).stop 2020-11-20T12:17:50.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/724481056 shutdown_connections 2020-11-20T12:17:50.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2780557c0 0x7fa278057c80 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2a00a9ef0 0x7fa2a00aa370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2a00a68c0 0x7fa2a0164e00 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 --2- 172.21.15.116:0/724481056 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2a00a1390 0x7fa2a01616b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/724481056 >> 172.21.15.116:0/724481056 conn(0x7fa2a00678a0 msgr2=0x7fa2a00c7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:50.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/724481056 shutdown_connections 2020-11-20T12:17:50.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.740+0000 7fa2a4bd5700 1 -- 172.21.15.116:0/724481056 wait complete. 2020-11-20T12:17:50.766 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:17:50.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.952+0000 7fe3b06dc700 1 Processor -- start 2020-11-20T12:17:50.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.952+0000 7fe3b06dc700 1 -- start start 2020-11-20T12:17:50.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3b06dc700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe3a80c0fe0 0x7fe3a80c2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3b06dc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe3a80c2af0 0x7fe3a80611a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3b06dc700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a80a2490 con 0x7fe3a80c0c00 2020-11-20T12:17:50.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3b06dc700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a8060280 con 0x7fe3a80c2af0 2020-11-20T12:17:50.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3b06dc700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a8060520 con 0x7fe3a80c0fe0 2020-11-20T12:17:50.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3aec79700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe3a80c2af0 0x7fe3a80611a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3adc77700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe3a80c0fe0 0x7fe3a80c2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3adc77700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe3a80c0fe0 0x7fe3a80c2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:44974/0 (socket says 172.21.15.116:44974) 2020-11-20T12:17:50.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3adc77700 1 -- 172.21.15.116:0/2819733300 learned_addr learned my addr 172.21.15.116:0/2819733300 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:50.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3aec79700 1 -- 172.21.15.116:0/2819733300 >> v2:172.21.15.173:3300/0 conn(0x7fe3a80c0fe0 msgr2=0x7fe3a80c2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:50.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3aec79700 1 --2- 172.21.15.116:0/2819733300 >> v2:172.21.15.173:3300/0 conn(0x7fe3a80c0fe0 0x7fe3a80c2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3aec79700 1 -- 172.21.15.116:0/2819733300 >> v1:172.21.15.173:6789/0 conn(0x7fe3a80c0c00 legacy=0x7fe3a80a1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3aec79700 1 -- 172.21.15.116:0/2819733300 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a80607c0 con 0x7fe3a80c2af0 2020-11-20T12:17:50.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3aec79700 1 --2- 172.21.15.116:0/2819733300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3a80c2af0 0x7fe3a80611a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fe3a8067e70 tx=0x7fe3a00091d0).ready entity=mon.0 client_cookie=114cd9db53b29f59 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:50.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3ad476700 1 -- 172.21.15.116:0/2819733300 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a000ba70 con 0x7fe3a80c2af0 2020-11-20T12:17:50.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3ad476700 1 -- 172.21.15.116:0/2819733300 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a000bbd0 con 0x7fe3a80c2af0 2020-11-20T12:17:50.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.956+0000 7fe3ad476700 1 -- 172.21.15.116:0/2819733300 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0011400 con 0x7fe3a80c2af0 2020-11-20T12:17:50.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.960+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2819733300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3a80c2af0 msgr2=0x7fe3a80611a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.960+0000 7fe3b06dc700 1 --2- 172.21.15.116:0/2819733300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3a80c2af0 0x7fe3a80611a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fe3a8067e70 tx=0x7fe3a00091d0).stop 2020-11-20T12:17:50.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.960+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2819733300 shutdown_connections 2020-11-20T12:17:50.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.960+0000 7fe3b06dc700 1 --2- 172.21.15.116:0/2819733300 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3a80c2af0 0x7fe3a80611a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.960+0000 7fe3b06dc700 1 --2- 172.21.15.116:0/2819733300 >> v2:172.21.15.173:3300/0 conn(0x7fe3a80c0fe0 0x7fe3a80c2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.960+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2819733300 >> 172.21.15.116:0/2819733300 conn(0x7fe3a80c14a0 msgr2=0x7fe3a80c18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:50.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.960+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2819733300 shutdown_connections 2020-11-20T12:17:50.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.960+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2819733300 wait complete. 2020-11-20T12:17:50.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3b06dc700 1 Processor -- start 2020-11-20T12:17:50.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3b06dc700 1 -- start start 2020-11-20T12:17:50.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3b06dc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3a80c0c00 0x7fe3a80a8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3b06dc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3a80c0fe0 0x7fe3a80aa430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3b06dc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3a80c2af0 0x7fe3a80a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3b06dc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8135f20 con 0x7fe3a80c0fe0 2020-11-20T12:17:50.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3b06dc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a81361c0 con 0x7fe3a80c2af0 2020-11-20T12:17:50.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3b06dc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8136460 con 0x7fe3a80c0c00 2020-11-20T12:17:50.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3adc77700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3a80c0fe0 0x7fe3a80aa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3adc77700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3a80c0fe0 0x7fe3a80aa430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45928/0 (socket says 172.21.15.116:45928) 2020-11-20T12:17:50.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3adc77700 1 -- 172.21.15.116:0/2398593976 learned_addr learned my addr 172.21.15.116:0/2398593976 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:50.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3aec79700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3a80c2af0 0x7fe3a80a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3ae478700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3a80c0c00 0x7fe3a80a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3adc77700 1 -- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3a80c0c00 msgr2=0x7fe3a80a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3adc77700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3a80c0c00 0x7fe3a80a8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3adc77700 1 -- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3a80c2af0 msgr2=0x7fe3a80a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:50.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3adc77700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3a80c2af0 0x7fe3a80a4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:50.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3adc77700 1 -- 172.21.15.116:0/2398593976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a8136700 con 0x7fe3a80c0fe0 2020-11-20T12:17:50.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.964+0000 7fe3ae478700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3a80c0c00 0x7fe3a80a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:50.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.968+0000 7fe3adc77700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3a80c0fe0 0x7fe3a80aa430 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7fe39800f840 tx=0x7fe398008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:50.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.968+0000 7fe3967fc700 1 -- 172.21.15.116:0/2398593976 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398014670 con 0x7fe3a80c0fe0 2020-11-20T12:17:50.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.968+0000 7fe3967fc700 1 -- 172.21.15.116:0/2398593976 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3980147d0 con 0x7fe3a80c0fe0 2020-11-20T12:17:50.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.968+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2398593976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a8137980 con 0x7fe3a80c0fe0 2020-11-20T12:17:50.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.968+0000 7fe3967fc700 1 -- 172.21.15.116:0/2398593976 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe398014b50 con 0x7fe3a80c0fe0 2020-11-20T12:17:50.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.968+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2398593976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a80a5400 con 0x7fe3a80c0fe0 2020-11-20T12:17:50.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.968+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2398593976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a805ca30 con 0x7fe3a80c0fe0 2020-11-20T12:17:50.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.968+0000 7fe3967fc700 1 -- 172.21.15.116:0/2398593976 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe398024070 con 0x7fe3a80c0fe0 2020-11-20T12:17:50.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.972+0000 7fe3967fc700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3800557c0 0x7fe380057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:50.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.972+0000 7fe3967fc700 1 -- 172.21.15.116:0/2398593976 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe39806ebf0 con 0x7fe3a80c0fe0 2020-11-20T12:17:50.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.972+0000 7fe3ae478700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3800557c0 0x7fe380057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:50.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.972+0000 7fe3ae478700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3800557c0 0x7fe380057c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fe3a8138e90 tx=0x7fe3a4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:50.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:50.976+0000 7fe3967fc700 1 -- 172.21.15.116:0/2398593976 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe398044a90 con 0x7fe3a80c0fe0 2020-11-20T12:17:51.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.140+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2398593976 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe3a8068120 con 0x7fe3a80c0fe0 2020-11-20T12:17:51.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.140+0000 7fe3967fc700 1 -- 172.21.15.116:0/2398593976 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe398018070 con 0x7fe3a80c0fe0 2020-11-20T12:17:51.140 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:51.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3800557c0 msgr2=0x7fe380057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3800557c0 0x7fe380057c80 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fe3a8138e90 tx=0x7fe3a4009040).stop 2020-11-20T12:17:51.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2398593976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3a80c0fe0 msgr2=0x7fe3a80aa430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3a80c0fe0 0x7fe3a80aa430 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7fe39800f840 tx=0x7fe398008040).stop 2020-11-20T12:17:51.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2398593976 shutdown_connections 2020-11-20T12:17:51.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe3800557c0 0x7fe380057c80 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe3a80c2af0 0x7fe3a80a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe3a80c0c00 0x7fe3a80a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 --2- 172.21.15.116:0/2398593976 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe3a80c0fe0 0x7fe3a80aa430 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2398593976 >> 172.21.15.116:0/2398593976 conn(0x7fe3a80c14a0 msgr2=0x7fe3a8060f00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:51.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2398593976 shutdown_connections 2020-11-20T12:17:51.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.144+0000 7fe3b06dc700 1 -- 172.21.15.116:0/2398593976 wait complete. 2020-11-20T12:17:51.166 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:17:51.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.336+0000 7f795a94e700 1 Processor -- start 2020-11-20T12:17:51.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.336+0000 7f795a94e700 1 -- start start 2020-11-20T12:17:51.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795a94e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f79540a0f00 0x7f79540a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795a94e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f79540c2df0 0x7f79540c3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795a94e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79540a50a0 con 0x7f79540a12e0 2020-11-20T12:17:51.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795a94e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79540c3770 con 0x7f79540c2df0 2020-11-20T12:17:51.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795a94e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79540a5340 con 0x7f79540a0f00 2020-11-20T12:17:51.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795914b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f79540a12e0 0x7f7954150370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55370/0 (socket says 172.21.15.116:55370) 2020-11-20T12:17:51.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795914b700 1 -- 172.21.15.116:0/1464313236 learned_addr learned my addr 172.21.15.116:0/1464313236 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:51.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795a14d700 1 --2- 172.21.15.116:0/1464313236 >> v2:172.21.15.116:3300/0 conn(0x7f79540c2df0 0x7f79540c3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795994c700 1 --2- 172.21.15.116:0/1464313236 >> v2:172.21.15.173:3300/0 conn(0x7f79540a0f00 0x7f79540a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795894a700 1 -- 172.21.15.116:0/1464313236 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2566838519 0 0) 0x7f79540a50a0 con 0x7f79540a12e0 2020-11-20T12:17:51.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795894a700 1 -- 172.21.15.116:0/1464313236 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f793c003240 con 0x7f79540a12e0 2020-11-20T12:17:51.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795a14d700 1 -- 172.21.15.116:0/1464313236 >> v2:172.21.15.173:3300/0 conn(0x7f79540a0f00 msgr2=0x7f79540a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:51.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795a14d700 1 --2- 172.21.15.116:0/1464313236 >> v2:172.21.15.173:3300/0 conn(0x7f79540a0f00 0x7f79540a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795a14d700 1 -- 172.21.15.116:0/1464313236 >> v1:172.21.15.116:6789/0 conn(0x7f79540a12e0 legacy=0x7f7954150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795894a700 1 -- 172.21.15.116:0/1464313236 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2602922888 0 0) 0x7f793c003240 con 0x7f79540a12e0 2020-11-20T12:17:51.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795a14d700 1 -- 172.21.15.116:0/1464313236 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79540a55e0 con 0x7f79540c2df0 2020-11-20T12:17:51.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.340+0000 7f795a14d700 1 --2- 172.21.15.116:0/1464313236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79540c2df0 0x7f79540c3280 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f79540c1af0 tx=0x7f794c0091d0).ready entity=mon.0 client_cookie=a62c9cfcdd3d4e21 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:51.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.344+0000 7f795894a700 1 -- 172.21.15.116:0/1464313236 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79440014e0 con 0x7f79540c2df0 2020-11-20T12:17:51.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.344+0000 7f795894a700 1 -- 172.21.15.116:0/1464313236 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c00b6b0 con 0x7f79540c2df0 2020-11-20T12:17:51.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.344+0000 7f795894a700 1 -- 172.21.15.116:0/1464313236 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c00ba30 con 0x7f79540c2df0 2020-11-20T12:17:51.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.344+0000 7f795a94e700 1 -- 172.21.15.116:0/1464313236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79540c2df0 msgr2=0x7f79540c3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.344+0000 7f795a94e700 1 --2- 172.21.15.116:0/1464313236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79540c2df0 0x7f79540c3280 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f79540c1af0 tx=0x7f794c0091d0).stop 2020-11-20T12:17:51.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.344+0000 7f795a94e700 1 -- 172.21.15.116:0/1464313236 shutdown_connections 2020-11-20T12:17:51.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.344+0000 7f795a94e700 1 --2- 172.21.15.116:0/1464313236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79540c2df0 0x7f79540c3280 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.344+0000 7f795a94e700 1 --2- 172.21.15.116:0/1464313236 >> v2:172.21.15.173:3300/0 conn(0x7f79540a0f00 0x7f79540a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.344+0000 7f795a94e700 1 -- 172.21.15.116:0/1464313236 >> 172.21.15.116:0/1464313236 conn(0x7f7954060c30 msgr2=0x7f79540c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.344+0000 7f795a94e700 1 -- 172.21.15.116:0/1464313236 shutdown_connections 2020-11-20T12:17:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.344+0000 7f795a94e700 1 -- 172.21.15.116:0/1464313236 wait complete. 2020-11-20T12:17:51.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795a94e700 1 Processor -- start 2020-11-20T12:17:51.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795a94e700 1 -- start start 2020-11-20T12:17:51.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795a94e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79540a0f00 0x7f79540bf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795a94e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79540a12e0 0x7f7954062b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795a94e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79540c2df0 0x7f7954063060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795a94e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79540bf580 con 0x7f79540a0f00 2020-11-20T12:17:51.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795a94e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795414d3c0 con 0x7f79540a12e0 2020-11-20T12:17:51.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795a94e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795414d660 con 0x7f79540c2df0 2020-11-20T12:17:51.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795994c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79540a0f00 0x7f79540bf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795994c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79540a0f00 0x7f79540bf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45940/0 (socket says 172.21.15.116:45940) 2020-11-20T12:17:51.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795994c700 1 -- 172.21.15.116:0/1124894231 learned_addr learned my addr 172.21.15.116:0/1124894231 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:51.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795914b700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79540a12e0 0x7f7954062b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795a14d700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79540c2df0 0x7f7954063060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795994c700 1 -- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79540c2df0 msgr2=0x7f7954063060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795994c700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79540c2df0 0x7f7954063060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795994c700 1 -- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79540a12e0 msgr2=0x7f7954062b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795994c700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79540a12e0 0x7f7954062b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795994c700 1 -- 172.21.15.116:0/1124894231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795414d900 con 0x7f79540a0f00 2020-11-20T12:17:51.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.348+0000 7f795994c700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79540a0f00 0x7f79540bf090 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f795000eba0 tx=0x7f7950006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:51.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.352+0000 7f7941ffb700 1 -- 172.21.15.116:0/1124894231 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795000c9b0 con 0x7f79540a0f00 2020-11-20T12:17:51.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.352+0000 7f7941ffb700 1 -- 172.21.15.116:0/1124894231 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795000cb10 con 0x7f79540a0f00 2020-11-20T12:17:51.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.352+0000 7f7941ffb700 1 -- 172.21.15.116:0/1124894231 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795000ce90 con 0x7f79540a0f00 2020-11-20T12:17:51.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.352+0000 7f795a94e700 1 -- 172.21.15.116:0/1124894231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795414dba0 con 0x7f79540a0f00 2020-11-20T12:17:51.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.352+0000 7f795a94e700 1 -- 172.21.15.116:0/1124894231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795414e1e0 con 0x7f79540a0f00 2020-11-20T12:17:51.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.352+0000 7f795a94e700 1 -- 172.21.15.116:0/1124894231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7954044f60 con 0x7f79540a0f00 2020-11-20T12:17:51.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.352+0000 7f7941ffb700 1 -- 172.21.15.116:0/1124894231 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7950012600 con 0x7f79540a0f00 2020-11-20T12:17:51.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.356+0000 7f7941ffb700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f792c0557c0 0x7f792c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.356+0000 7f7941ffb700 1 -- 172.21.15.116:0/1124894231 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f795006eaa0 con 0x7f79540a0f00 2020-11-20T12:17:51.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.356+0000 7f795914b700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f792c0557c0 0x7f792c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.356+0000 7f795914b700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f792c0557c0 0x7f792c057c80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f795414f270 tx=0x7f7944001280).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:51.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.360+0000 7f7941ffb700 1 -- 172.21.15.116:0/1124894231 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7950044940 con 0x7f79540a0f00 2020-11-20T12:17:51.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.520+0000 7f795a94e700 1 -- 172.21.15.116:0/1124894231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f795414f2a0 con 0x7f79540a0f00 2020-11-20T12:17:51.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.520+0000 7f7941ffb700 1 -- 172.21.15.116:0/1124894231 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7950047f60 con 0x7f79540a0f00 2020-11-20T12:17:51.520 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:51.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 -- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f792c0557c0 msgr2=0x7f792c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f792c0557c0 0x7f792c057c80 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f795414f270 tx=0x7f7944001280).stop 2020-11-20T12:17:51.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 -- 172.21.15.116:0/1124894231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79540a0f00 msgr2=0x7f79540bf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79540a0f00 0x7f79540bf090 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f795000eba0 tx=0x7f7950006df0).stop 2020-11-20T12:17:51.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 -- 172.21.15.116:0/1124894231 shutdown_connections 2020-11-20T12:17:51.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f792c0557c0 0x7f792c057c80 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79540c2df0 0x7f7954063060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79540a0f00 0x7f79540bf090 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 --2- 172.21.15.116:0/1124894231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79540a12e0 0x7f7954062b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 -- 172.21.15.116:0/1124894231 >> 172.21.15.116:0/1124894231 conn(0x7f7954060c30 msgr2=0x7f79540a2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:51.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 -- 172.21.15.116:0/1124894231 shutdown_connections 2020-11-20T12:17:51.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.524+0000 7f795a94e700 1 -- 172.21.15.116:0/1124894231 wait complete. 2020-11-20T12:17:51.544 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:17:51.545 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:51.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.724+0000 7fe73b59e700 1 Processor -- start 2020-11-20T12:17:51.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe73b59e700 1 -- start start 2020-11-20T12:17:51.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe73b59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe73c0aa790 0x7fe73c152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe73b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe73c0612f0 0x7fe73c0617a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe73b59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe73c152a30 con 0x7fe73c13aee0 2020-11-20T12:17:51.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe73b59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe73c0adce0 con 0x7fe73c0aa790 2020-11-20T12:17:51.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe73b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0aac10 con 0x7fe73c0612f0 2020-11-20T12:17:51.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe73a59c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe73c13aee0 0x7fe73c0ad5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55384/0 (socket says 172.21.15.116:55384) 2020-11-20T12:17:51.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe73a59c700 1 -- 172.21.15.116:0/1042653035 learned_addr learned my addr 172.21.15.116:0/1042653035 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:51.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe731d9b700 1 --2- 172.21.15.116:0/1042653035 >> v2:172.21.15.173:3300/0 conn(0x7fe73c0aa790 0x7fe73c152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe73ad9d700 1 --2- 172.21.15.116:0/1042653035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe73c0612f0 0x7fe73c0617a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe739d9b700 1 -- 172.21.15.116:0/1042653035 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 471966288 0 0) 0x7fe73c152a30 con 0x7fe73c13aee0 2020-11-20T12:17:51.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe739d9b700 1 -- 172.21.15.116:0/1042653035 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe71c003240 con 0x7fe73c13aee0 2020-11-20T12:17:51.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe731d9b700 1 -- 172.21.15.116:0/1042653035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe73c0612f0 msgr2=0x7fe73c0617a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:51.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe731d9b700 1 --2- 172.21.15.116:0/1042653035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe73c0612f0 0x7fe73c0617a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe731d9b700 1 -- 172.21.15.116:0/1042653035 >> v1:172.21.15.116:6789/0 conn(0x7fe73c13aee0 legacy=0x7fe73c0ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.728+0000 7fe731d9b700 1 -- 172.21.15.116:0/1042653035 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c0aaeb0 con 0x7fe73c0aa790 2020-11-20T12:17:51.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe731d9b700 1 --2- 172.21.15.116:0/1042653035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe73c0aa790 0x7fe73c152540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fe72400b9a0 tx=0x7fe724006df0).ready entity=mon.1 client_cookie=f7b2b38e8b8da2e7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:51.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe739d9b700 1 -- 172.21.15.116:0/1042653035 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724009680 con 0x7fe73c0aa790 2020-11-20T12:17:51.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe739d9b700 1 -- 172.21.15.116:0/1042653035 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7240097e0 con 0x7fe73c0aa790 2020-11-20T12:17:51.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe739d9b700 1 -- 172.21.15.116:0/1042653035 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724009b60 con 0x7fe73c0aa790 2020-11-20T12:17:51.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe73b59e700 1 -- 172.21.15.116:0/1042653035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe73c0aa790 msgr2=0x7fe73c152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe73b59e700 1 --2- 172.21.15.116:0/1042653035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe73c0aa790 0x7fe73c152540 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fe72400b9a0 tx=0x7fe724006df0).stop 2020-11-20T12:17:51.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe73b59e700 1 -- 172.21.15.116:0/1042653035 shutdown_connections 2020-11-20T12:17:51.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe73b59e700 1 --2- 172.21.15.116:0/1042653035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe73c0612f0 0x7fe73c0617a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe73b59e700 1 --2- 172.21.15.116:0/1042653035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe73c0aa790 0x7fe73c152540 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe73b59e700 1 -- 172.21.15.116:0/1042653035 >> 172.21.15.116:0/1042653035 conn(0x7fe73c0678a0 msgr2=0x7fe73c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:51.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe73b59e700 1 -- 172.21.15.116:0/1042653035 shutdown_connections 2020-11-20T12:17:51.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.732+0000 7fe73b59e700 1 -- 172.21.15.116:0/1042653035 wait complete. 2020-11-20T12:17:51.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73b59e700 1 Processor -- start 2020-11-20T12:17:51.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73b59e700 1 -- start start 2020-11-20T12:17:51.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe73c0612f0 0x7fe73c150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe73c0aa790 0x7fe73c0a24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe73c13aee0 0x7fe73c0a7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c1527c0 con 0x7fe73c0aa790 2020-11-20T12:17:51.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c152a60 con 0x7fe73c13aee0 2020-11-20T12:17:51.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe73c0a1d60 con 0x7fe73c0612f0 2020-11-20T12:17:51.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe731d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe73c0aa790 0x7fe73c0a24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73a59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe73c0612f0 0x7fe73c150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73ad9d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe73c13aee0 0x7fe73c0a7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73a59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe73c0612f0 0x7fe73c150d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:54904/0 (socket says 172.21.15.116:54904) 2020-11-20T12:17:51.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.736+0000 7fe73a59c700 1 -- 172.21.15.116:0/1917168253 learned_addr learned my addr 172.21.15.116:0/1917168253 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:51.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe731d9b700 1 -- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe73c0612f0 msgr2=0x7fe73c150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe731d9b700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe73c0612f0 0x7fe73c150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe731d9b700 1 -- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe73c13aee0 msgr2=0x7fe73c0a7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe731d9b700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe73c13aee0 0x7fe73c0a7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe731d9b700 1 -- 172.21.15.116:0/1917168253 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe73c0a2000 con 0x7fe73c0aa790 2020-11-20T12:17:51.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe731d9b700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe73c0aa790 0x7fe73c0a24a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fe724012000 tx=0x7fe724015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:51.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe732ffd700 1 -- 172.21.15.116:0/1917168253 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe724011cc0 con 0x7fe73c0aa790 2020-11-20T12:17:51.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe732ffd700 1 -- 172.21.15.116:0/1917168253 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe724011e20 con 0x7fe73c0aa790 2020-11-20T12:17:51.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe73b59e700 1 -- 172.21.15.116:0/1917168253 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe73c151200 con 0x7fe73c0aa790 2020-11-20T12:17:51.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe732ffd700 1 -- 172.21.15.116:0/1917168253 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72401e670 con 0x7fe73c0aa790 2020-11-20T12:17:51.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe73b59e700 1 -- 172.21.15.116:0/1917168253 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe73c0a2990 con 0x7fe73c0aa790 2020-11-20T12:17:51.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.740+0000 7fe73b59e700 1 -- 172.21.15.116:0/1917168253 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe73c0acb30 con 0x7fe73c0aa790 2020-11-20T12:17:51.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.744+0000 7fe732ffd700 1 -- 172.21.15.116:0/1917168253 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe724025030 con 0x7fe73c0aa790 2020-11-20T12:17:51.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.744+0000 7fe732ffd700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7180557f0 0x7fe718057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:51.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.744+0000 7fe732ffd700 1 -- 172.21.15.116:0/1917168253 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe72406fcd0 con 0x7fe73c0aa790 2020-11-20T12:17:51.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.744+0000 7fe73a59c700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7180557f0 0x7fe718057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:51.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.744+0000 7fe73a59c700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7180557f0 0x7fe718057cb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fe72c007fd0 tx=0x7fe72c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:51.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.748+0000 7fe732ffd700 1 -- 172.21.15.116:0/1917168253 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe724045b70 con 0x7fe73c0aa790 2020-11-20T12:17:51.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.952+0000 7fe73b59e700 1 -- 172.21.15.116:0/1917168253 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe73c0a2190 con 0x7fe73c0aa790 2020-11-20T12:17:51.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe732ffd700 1 -- 172.21.15.116:0/1917168253 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+2849 (secure 0 0 0) 0x7fe724023020 con 0x7fe73c0aa790 2020-11-20T12:17:51.955 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:51.955 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1096724826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1096724826},{"type":"v1","addr":"172.21.15.173:6837","nonce":1096724826}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/1589443098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1589443098},{"type":"v1","addr":"172.21.15.173:6835","nonce":1589443098}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/966300328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":966300328},{"type":"v1","addr":"172.21.15.116:6833","nonce":966300328}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:17:40.163408+0000","modified":"2020-11-20T12:17:42.171423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5493":{"gid":5493,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/2908037349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2908037349},{"type":"v1","addr":"172.21.15.116:6835","nonce":2908037349}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-20T12:17:51.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 -- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7180557f0 msgr2=0x7fe718057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7180557f0 0x7fe718057cb0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fe72c007fd0 tx=0x7fe72c009040).stop 2020-11-20T12:17:51.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 -- 172.21.15.116:0/1917168253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe73c0aa790 msgr2=0x7fe73c0a24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:51.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe73c0aa790 0x7fe73c0a24a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7fe724012000 tx=0x7fe724015000).stop 2020-11-20T12:17:51.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 -- 172.21.15.116:0/1917168253 shutdown_connections 2020-11-20T12:17:51.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7180557f0 0x7fe718057cb0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe73c13aee0 0x7fe73c0a7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe73c0612f0 0x7fe73c150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 --2- 172.21.15.116:0/1917168253 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe73c0aa790 0x7fe73c0a24a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:51.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 -- 172.21.15.116:0/1917168253 >> 172.21.15.116:0/1917168253 conn(0x7fe73c0678a0 msgr2=0x7fe73c139b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:51.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 -- 172.21.15.116:0/1917168253 shutdown_connections 2020-11-20T12:17:51.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:51.956+0000 7fe73b59e700 1 -- 172.21.15.116:0/1917168253 wait complete. 2020-11-20T12:17:51.961 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 20 2020-11-20T12:17:52.385 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:17:40.163408+0000', 'modified': '2020-11-20T12:17:42.171423+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5493': {'gid': 5493, 'name': 'a', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/2908037349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2908037349}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 2908037349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:17:52.386 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:17:52.386 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:17:52.582 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":5493,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":12.398460889000001,"mdsmap_epoch":20,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":17.916105267999999} 2020-11-20T12:17:52.582 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 5493, "mdsmap_epoch": 20, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 12.398460889, "state": "up:active", "uptime": 17.916105268, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:17:52.584 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:17:52.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:52.585 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-20T12:17:52.604 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:17:52.604 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:52.668 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:52.668 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:17:52.692 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:52.692 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:17:52.697 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:17:52.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:52.698 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-20T12:17:52.723 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:17:52.724 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:17:52.724 INFO:tasks.cephfs.mount:self.client.name = client.2 2020-11-20T12:17:52.724 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2020-11-20T12:17:52.725 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:17:52.725 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:17:52.725 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:17:52.726 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:17:52.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:52.727 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:17:52.745 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:17:52.745 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:17:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:17:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:17:52.746 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:52.747 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:52.747 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:52.747 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:52.748 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:52.748 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:52.748 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:17:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:17:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:52.749 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:17:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:52.750 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:17:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:17:52.750 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:52.751 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:17:52.751 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:17:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:17:52.752 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:52.753 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:52.753 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:17:52.753 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:17:52.753 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:52.753 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:52.754 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:17:52.754 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:17:52.754 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:52.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:52.755 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:17:52.824 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:17:52.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:52.828 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:17:52.848 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:17:52.849 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:17:52.849 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:17:52.849 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:17:52.849 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:17:52.850 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:17:52.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:52.851 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:17:52.852 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:17:52.852 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:17:52.852 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:17:52.853 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:17:52.853 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:17:52.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:52.988 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:17:53.010 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:17:53.011 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:17:53.012 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:17:53.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:53.013 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2020-11-20T12:17:53.089 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:53.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.272+0000 7fbed9b16700 1 Processor -- start 2020-11-20T12:17:53.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbed9b16700 1 -- start start 2020-11-20T12:17:53.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbed9b16700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbed40c41c0 0x7fbed40c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbed9b16700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbed40a5a10 0x7fbed40a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbed9b16700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbed40a9180 con 0x7fbed4138670 2020-11-20T12:17:53.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbed9b16700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed40a6380 con 0x7fbed40a5a10 2020-11-20T12:17:53.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbed9b16700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbed40a8c80 con 0x7fbed40c41c0 2020-11-20T12:17:53.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbecbfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbed40a5a10 0x7fbed40a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbecbfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbed40a5a10 0x7fbed40a5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45964/0 (socket says 172.21.15.116:45964) 2020-11-20T12:17:53.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbecbfff700 1 -- 172.21.15.116:0/3278007080 learned_addr learned my addr 172.21.15.116:0/3278007080 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:53.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbed8b14700 1 --2- 172.21.15.116:0/3278007080 >> v2:172.21.15.173:3300/0 conn(0x7fbed40c41c0 0x7fbed40c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbecbfff700 1 -- 172.21.15.116:0/3278007080 >> v2:172.21.15.173:3300/0 conn(0x7fbed40c41c0 msgr2=0x7fbed40c45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:53.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbecbfff700 1 --2- 172.21.15.116:0/3278007080 >> v2:172.21.15.173:3300/0 conn(0x7fbed40c41c0 0x7fbed40c45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbecbfff700 1 -- 172.21.15.116:0/3278007080 >> v1:172.21.15.173:6789/0 conn(0x7fbed4138670 legacy=0x7fbed4138b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:53.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbecbfff700 1 -- 172.21.15.116:0/3278007080 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed40a9420 con 0x7fbed40a5a10 2020-11-20T12:17:53.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.276+0000 7fbecbfff700 1 --2- 172.21.15.116:0/3278007080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40a5a10 0x7fbed40a5e90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fbed4065960 tx=0x7fbec4006eb0).ready entity=mon.0 client_cookie=4fbc2493d31ba72b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbecb7fe700 1 -- 172.21.15.116:0/3278007080 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4009810 con 0x7fbed40a5a10 2020-11-20T12:17:53.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbecb7fe700 1 -- 172.21.15.116:0/3278007080 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec4009970 con 0x7fbed40a5a10 2020-11-20T12:17:53.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbecb7fe700 1 -- 172.21.15.116:0/3278007080 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4009cf0 con 0x7fbed40a5a10 2020-11-20T12:17:53.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbed9b16700 1 -- 172.21.15.116:0/3278007080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40a5a10 msgr2=0x7fbed40a5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbed9b16700 1 --2- 172.21.15.116:0/3278007080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40a5a10 0x7fbed40a5e90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fbed4065960 tx=0x7fbec4006eb0).stop 2020-11-20T12:17:53.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbed9b16700 1 -- 172.21.15.116:0/3278007080 shutdown_connections 2020-11-20T12:17:53.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbed9b16700 1 --2- 172.21.15.116:0/3278007080 >> v2:172.21.15.173:3300/0 conn(0x7fbed40c41c0 0x7fbed40c45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbed9b16700 1 --2- 172.21.15.116:0/3278007080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40a5a10 0x7fbed40a5e90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbed9b16700 1 -- 172.21.15.116:0/3278007080 >> 172.21.15.116:0/3278007080 conn(0x7fbed4062520 msgr2=0x7fbed4062930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbed9b16700 1 -- 172.21.15.116:0/3278007080 shutdown_connections 2020-11-20T12:17:53.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbed9b16700 1 -- 172.21.15.116:0/3278007080 wait complete. 2020-11-20T12:17:53.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbed9b16700 1 Processor -- start 2020-11-20T12:17:53.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.280+0000 7fbed9b16700 1 -- start start 2020-11-20T12:17:53.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbed9b16700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed40a5a10 0x7fbed4137aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbed9b16700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40c41c0 0x7fbed4156f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbed9b16700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed4138670 0x7fbed40c3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbed9b16700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed4157910 con 0x7fbed40c41c0 2020-11-20T12:17:53.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbed9b16700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbed4157bb0 con 0x7fbed40a5a10 2020-11-20T12:17:53.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbed9b16700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbed4157e50 con 0x7fbed4138670 2020-11-20T12:17:53.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbecbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40c41c0 0x7fbed4156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbecbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40c41c0 0x7fbed4156f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45972/0 (socket says 172.21.15.116:45972) 2020-11-20T12:17:53.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbecbfff700 1 -- 172.21.15.116:0/588956971 learned_addr learned my addr 172.21.15.116:0/588956971 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:53.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbed8b14700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed40a5a10 0x7fbed4137aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbed9315700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed4138670 0x7fbed40c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbecbfff700 1 -- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed4138670 msgr2=0x7fbed40c3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbecbfff700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed4138670 0x7fbed40c3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbecbfff700 1 -- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed40a5a10 msgr2=0x7fbed4137aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbecbfff700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed40a5a10 0x7fbed4137aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbecbfff700 1 -- 172.21.15.116:0/588956971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbed41580f0 con 0x7fbed40c41c0 2020-11-20T12:17:53.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbecbfff700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40c41c0 0x7fbed4156f20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fbec4006c40 tx=0x7fbec4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbec8ff9700 1 -- 172.21.15.116:0/588956971 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec4011d10 con 0x7fbed40c41c0 2020-11-20T12:17:53.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbec8ff9700 1 -- 172.21.15.116:0/588956971 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbec4011e70 con 0x7fbed40c41c0 2020-11-20T12:17:53.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbed9b16700 1 -- 172.21.15.116:0/588956971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbed4158390 con 0x7fbed40c41c0 2020-11-20T12:17:53.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbec8ff9700 1 -- 172.21.15.116:0/588956971 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbec401e670 con 0x7fbed40c41c0 2020-11-20T12:17:53.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.284+0000 7fbed9b16700 1 -- 172.21.15.116:0/588956971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbed4137f90 con 0x7fbed40c41c0 2020-11-20T12:17:53.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.288+0000 7fbed9b16700 1 -- 172.21.15.116:0/588956971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbed4158550 con 0x7fbed40c41c0 2020-11-20T12:17:53.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.288+0000 7fbec8ff9700 1 -- 172.21.15.116:0/588956971 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fbec401a040 con 0x7fbed40c41c0 2020-11-20T12:17:53.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.288+0000 7fbec8ff9700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbeac0557f0 0x7fbeac057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.288+0000 7fbec8ff9700 1 -- 172.21.15.116:0/588956971 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbec406f950 con 0x7fbed40c41c0 2020-11-20T12:17:53.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.288+0000 7fbed8b14700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbeac0557f0 0x7fbeac057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.292+0000 7fbed8b14700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbeac0557f0 0x7fbeac057cb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fbed0006fd0 tx=0x7fbed0006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.296+0000 7fbec8ff9700 1 -- 172.21.15.116:0/588956971 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbec4045840 con 0x7fbed40c41c0 2020-11-20T12:17:53.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.452+0000 7fbed9b16700 1 -- 172.21.15.116:0/588956971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbed4044f60 con 0x7fbed40c41c0 2020-11-20T12:17:53.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.456+0000 7fbec8ff9700 1 -- 172.21.15.116:0/588956971 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fbec4023070 con 0x7fbed40c41c0 2020-11-20T12:17:53.457 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:53.458 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":45,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:17:41.167389+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:17:38.002878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-20T12:17:38.995456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:53.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.456+0000 7fbed9b16700 1 -- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbeac0557f0 msgr2=0x7fbeac057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.456+0000 7fbed9b16700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbeac0557f0 0x7fbeac057cb0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fbed0006fd0 tx=0x7fbed0006df0).stop 2020-11-20T12:17:53.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.460+0000 7fbed9b16700 1 -- 172.21.15.116:0/588956971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40c41c0 msgr2=0x7fbed4156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.460+0000 7fbed9b16700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40c41c0 0x7fbed4156f20 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7fbec4006c40 tx=0x7fbec4015000).stop 2020-11-20T12:17:53.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.460+0000 7fbed9b16700 1 -- 172.21.15.116:0/588956971 shutdown_connections 2020-11-20T12:17:53.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.460+0000 7fbed9b16700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbeac0557f0 0x7fbeac057cb0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.460+0000 7fbed9b16700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbed4138670 0x7fbed40c3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.460+0000 7fbed9b16700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbed40a5a10 0x7fbed4137aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.460+0000 7fbed9b16700 1 --2- 172.21.15.116:0/588956971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbed40c41c0 0x7fbed4156f20 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.460+0000 7fbed9b16700 1 -- 172.21.15.116:0/588956971 >> 172.21.15.116:0/588956971 conn(0x7fbed4062520 msgr2=0x7fbed4151f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.460+0000 7fbed9b16700 1 -- 172.21.15.116:0/588956971 shutdown_connections 2020-11-20T12:17:53.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.460+0000 7fbed9b16700 1 -- 172.21.15.116:0/588956971 wait complete. 2020-11-20T12:17:53.485 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:17:38.002878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:17:38.995456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:17:53.487 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:53.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f904d05d700 1 Processor -- start 2020-11-20T12:17:53.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f904d05d700 1 -- start start 2020-11-20T12:17:53.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f904d05d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f904813ad30 0x7f90480a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f904d05d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9048061250 0x7f90480616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f904d05d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9048152a80 con 0x7f904813b110 2020-11-20T12:17:53.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f904d05d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9048152540 con 0x7f904813ad30 2020-11-20T12:17:53.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f904d05d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90481527e0 con 0x7f9048061250 2020-11-20T12:17:53.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f9047fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f904813ad30 0x7f90480a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f90477fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f904813b110 0x7f9048151ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55412/0 (socket says 172.21.15.116:55412) 2020-11-20T12:17:53.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f9047fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f904813ad30 0x7f90480a7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45976/0 (socket says 172.21.15.116:45976) 2020-11-20T12:17:53.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f90477fe700 1 -- 172.21.15.116:0/3925388039 learned_addr learned my addr 172.21.15.116:0/3925388039 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:53.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.672+0000 7f904c85c700 1 --2- 172.21.15.116:0/3925388039 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9048061250 0x7f90480616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f9047fff700 1 -- 172.21.15.116:0/3925388039 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9048061250 msgr2=0x7f90480616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:53.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f9047fff700 1 --2- 172.21.15.116:0/3925388039 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9048061250 0x7f90480616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f9047fff700 1 -- 172.21.15.116:0/3925388039 >> v1:172.21.15.116:6789/0 conn(0x7f904813b110 legacy=0x7f9048151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f9047fff700 1 -- 172.21.15.116:0/3925388039 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90480ac910 con 0x7f904813ad30 2020-11-20T12:17:53.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f9047fff700 1 --2- 172.21.15.116:0/3925388039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f904813ad30 0x7f90480a7fb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f903800b9a0 tx=0x7f9038006df0).ready entity=mon.0 client_cookie=bf88fd11f71c2edc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f9046ffd700 1 -- 172.21.15.116:0/3925388039 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038009840 con 0x7f904813ad30 2020-11-20T12:17:53.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f9046ffd700 1 -- 172.21.15.116:0/3925388039 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90380099a0 con 0x7f904813ad30 2020-11-20T12:17:53.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f9046ffd700 1 -- 172.21.15.116:0/3925388039 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9038009d20 con 0x7f904813ad30 2020-11-20T12:17:53.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f904d05d700 1 -- 172.21.15.116:0/3925388039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f904813ad30 msgr2=0x7f90480a7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f904d05d700 1 --2- 172.21.15.116:0/3925388039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f904813ad30 0x7f90480a7fb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f903800b9a0 tx=0x7f9038006df0).stop 2020-11-20T12:17:53.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f904d05d700 1 -- 172.21.15.116:0/3925388039 shutdown_connections 2020-11-20T12:17:53.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f904d05d700 1 --2- 172.21.15.116:0/3925388039 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f904813ad30 0x7f90480a7fb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f904d05d700 1 --2- 172.21.15.116:0/3925388039 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9048061250 0x7f90480616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f904d05d700 1 -- 172.21.15.116:0/3925388039 >> 172.21.15.116:0/3925388039 conn(0x7f90480678a0 msgr2=0x7f9048067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f904d05d700 1 -- 172.21.15.116:0/3925388039 shutdown_connections 2020-11-20T12:17:53.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f904d05d700 1 -- 172.21.15.116:0/3925388039 wait complete. 2020-11-20T12:17:53.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.676+0000 7f904d05d700 1 Processor -- start 2020-11-20T12:17:53.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904d05d700 1 -- start start 2020-11-20T12:17:53.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904d05d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9048061250 0x7f90480a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904d05d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f904813b110 0x7f90480c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904d05d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90480a3860 0x7f90480c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904d05d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9048164640 con 0x7f90480a3860 2020-11-20T12:17:53.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904d05d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90481648e0 con 0x7f904813b110 2020-11-20T12:17:53.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904d05d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9048164b80 con 0x7f9048061250 2020-11-20T12:17:53.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904c85c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90480a3860 0x7f90480c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904c85c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90480a3860 0x7f90480c6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45986/0 (socket says 172.21.15.116:45986) 2020-11-20T12:17:53.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904c85c700 1 -- 172.21.15.116:0/4137563975 learned_addr learned my addr 172.21.15.116:0/4137563975 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:53.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f90477fe700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f904813b110 0x7f90480c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f9047fff700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9048061250 0x7f90480a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904c85c700 1 -- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9048061250 msgr2=0x7f90480a3370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904c85c700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9048061250 0x7f90480a3370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904c85c700 1 -- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f904813b110 msgr2=0x7f90480c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904c85c700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f904813b110 0x7f90480c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904c85c700 1 -- 172.21.15.116:0/4137563975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9048164e20 con 0x7f90480a3860 2020-11-20T12:17:53.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904c85c700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90480a3860 0x7f90480c6690 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f904000eb70 tx=0x7f9040008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f9027fff700 1 -- 172.21.15.116:0/4137563975 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904000c670 con 0x7f90480a3860 2020-11-20T12:17:53.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f9027fff700 1 -- 172.21.15.116:0/4137563975 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904000c7d0 con 0x7f90480a3860 2020-11-20T12:17:53.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904d05d700 1 -- 172.21.15.116:0/4137563975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90481650c0 con 0x7f90480a3860 2020-11-20T12:17:53.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f9027fff700 1 -- 172.21.15.116:0/4137563975 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904000cb70 con 0x7f90480a3860 2020-11-20T12:17:53.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.680+0000 7f904d05d700 1 -- 172.21.15.116:0/4137563975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90480a2a60 con 0x7f90480a3860 2020-11-20T12:17:53.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.684+0000 7f904d05d700 1 -- 172.21.15.116:0/4137563975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9048044f60 con 0x7f90480a3860 2020-11-20T12:17:53.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.684+0000 7f9027fff700 1 -- 172.21.15.116:0/4137563975 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9040013430 con 0x7f90480a3860 2020-11-20T12:17:53.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.684+0000 7f9027fff700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90200557c0 0x7f9020057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:53.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.688+0000 7f9027fff700 1 -- 172.21.15.116:0/4137563975 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f904006e240 con 0x7f90480a3860 2020-11-20T12:17:53.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.688+0000 7f9047fff700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90200557c0 0x7f9020057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:53.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.688+0000 7f9047fff700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90200557c0 0x7f9020057c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f90480ab090 tx=0x7f9038015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:53.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.692+0000 7f9027fff700 1 -- 172.21.15.116:0/4137563975 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9040044080 con 0x7f90480a3860 2020-11-20T12:17:53.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.852+0000 7f904d05d700 1 -- 172.21.15.116:0/4137563975 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9048069320 con 0x7f90480a3860 2020-11-20T12:17:53.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.852+0000 7f9027fff700 1 -- 172.21.15.116:0/4137563975 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7f90400476a0 con 0x7f90480a3860 2020-11-20T12:17:53.853 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:53.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 -- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90200557c0 msgr2=0x7f9020057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90200557c0 0x7f9020057c80 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f90480ab090 tx=0x7f9038015000).stop 2020-11-20T12:17:53.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 -- 172.21.15.116:0/4137563975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90480a3860 msgr2=0x7f90480c6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:53.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90480a3860 0x7f90480c6690 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f904000eb70 tx=0x7f9040008040).stop 2020-11-20T12:17:53.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 -- 172.21.15.116:0/4137563975 shutdown_connections 2020-11-20T12:17:53.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90200557c0 0x7f9020057c80 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90480a3860 0x7f90480c6690 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9048061250 0x7f90480a3370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 --2- 172.21.15.116:0/4137563975 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f904813b110 0x7f90480c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:53.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 -- 172.21.15.116:0/4137563975 >> 172.21.15.116:0/4137563975 conn(0x7f90480678a0 msgr2=0x7f90480ad290 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:53.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 -- 172.21.15.116:0/4137563975 shutdown_connections 2020-11-20T12:17:53.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:53.856+0000 7f904d05d700 1 -- 172.21.15.116:0/4137563975 wait complete. 2020-11-20T12:17:53.881 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:54.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.076+0000 7f0040958700 1 Processor -- start 2020-11-20T12:17:54.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.076+0000 7f0040958700 1 -- start start 2020-11-20T12:17:54.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f0040958700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003c0c0cc0 0x7f003c0a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f0040958700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f003c149cf0 0x7f003c0aa4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f0040958700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f003c0a8f10 con 0x7f003c149910 2020-11-20T12:17:54.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f0040958700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f003c0a9e30 con 0x7f003c149cf0 2020-11-20T12:17:54.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f0040958700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c0a91b0 con 0x7f003c0c0cc0 2020-11-20T12:17:54.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f003ad9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f003c149cf0 0x7f003c0aa4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f003ad9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f003c149cf0 0x7f003c0aa4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45994/0 (socket says 172.21.15.116:45994) 2020-11-20T12:17:54.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f003ad9d700 1 -- 172.21.15.116:0/3360712030 learned_addr learned my addr 172.21.15.116:0/3360712030 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:54.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f0033fff700 1 --2- 172.21.15.116:0/3360712030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003c0c0cc0 0x7f003c0a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f0039d9b700 1 -- 172.21.15.116:0/3360712030 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398007142 0 0) 0x7f003c0a8f10 con 0x7f003c149910 2020-11-20T12:17:54.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f003ad9d700 1 -- 172.21.15.116:0/3360712030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003c0c0cc0 msgr2=0x7f003c0a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:54.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f003ad9d700 1 --2- 172.21.15.116:0/3360712030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003c0c0cc0 0x7f003c0a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f003ad9d700 1 -- 172.21.15.116:0/3360712030 >> v1:172.21.15.173:6789/0 conn(0x7f003c149910 legacy=0x7f003c150520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f003ad9d700 1 -- 172.21.15.116:0/3360712030 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c0a9450 con 0x7f003c149cf0 2020-11-20T12:17:54.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f003ad9d700 1 --2- 172.21.15.116:0/3360712030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f003c149cf0 0x7f003c0aa4d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f0034009d50 tx=0x7f0034009290).ready entity=mon.0 client_cookie=a1178cec9dee744b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:54.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f0039d9b700 1 -- 172.21.15.116:0/3360712030 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003400bc50 con 0x7f003c149cf0 2020-11-20T12:17:54.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f0039d9b700 1 -- 172.21.15.116:0/3360712030 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003400bdb0 con 0x7f003c149cf0 2020-11-20T12:17:54.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.080+0000 7f0039d9b700 1 -- 172.21.15.116:0/3360712030 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0034011610 con 0x7f003c149cf0 2020-11-20T12:17:54.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.084+0000 7f0040958700 1 -- 172.21.15.116:0/3360712030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f003c149cf0 msgr2=0x7f003c0aa4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.084+0000 7f0040958700 1 --2- 172.21.15.116:0/3360712030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f003c149cf0 0x7f003c0aa4d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f0034009d50 tx=0x7f0034009290).stop 2020-11-20T12:17:54.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.084+0000 7f0040958700 1 -- 172.21.15.116:0/3360712030 shutdown_connections 2020-11-20T12:17:54.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.084+0000 7f0040958700 1 --2- 172.21.15.116:0/3360712030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f003c149cf0 0x7f003c0aa4d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.084+0000 7f0040958700 1 --2- 172.21.15.116:0/3360712030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003c0c0cc0 0x7f003c0a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.084+0000 7f0040958700 1 -- 172.21.15.116:0/3360712030 >> 172.21.15.116:0/3360712030 conn(0x7f003c0c14a0 msgr2=0x7f003c0c18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:54.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.084+0000 7f0040958700 1 -- 172.21.15.116:0/3360712030 shutdown_connections 2020-11-20T12:17:54.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.084+0000 7f0040958700 1 -- 172.21.15.116:0/3360712030 wait complete. 2020-11-20T12:17:54.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f0040958700 1 Processor -- start 2020-11-20T12:17:54.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f0040958700 1 -- start start 2020-11-20T12:17:54.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f0040958700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f003c0c0cc0 0x7f003c168910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f0040958700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003c149910 0x7f003c151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f0040958700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f003c149cf0 0x7f003c0a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f0040958700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c14ae50 con 0x7f003c0c0cc0 2020-11-20T12:17:54.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f0040958700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f003c14b0f0 con 0x7f003c149cf0 2020-11-20T12:17:54.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f0040958700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f003c14b390 con 0x7f003c149910 2020-11-20T12:17:54.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f003a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f003c0c0cc0 0x7f003c168910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f003a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f003c0c0cc0 0x7f003c168910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:45998/0 (socket says 172.21.15.116:45998) 2020-11-20T12:17:54.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f003ad9d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f003c149cf0 0x7f003c0a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f0033fff700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003c149910 0x7f003c151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.088+0000 7f003a59c700 1 -- 172.21.15.116:0/451944129 learned_addr learned my addr 172.21.15.116:0/451944129 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:54.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f003a59c700 1 -- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003c149910 msgr2=0x7f003c151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f003a59c700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003c149910 0x7f003c151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f003a59c700 1 -- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f003c149cf0 msgr2=0x7f003c0a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f003a59c700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f003c149cf0 0x7f003c0a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f003a59c700 1 -- 172.21.15.116:0/451944129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c14b630 con 0x7f003c0c0cc0 2020-11-20T12:17:54.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f003a59c700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f003c0c0cc0 0x7f003c168910 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f002c007fd0 tx=0x7f002c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:54.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f00327fc700 1 -- 172.21.15.116:0/451944129 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c00e490 con 0x7f003c0c0cc0 2020-11-20T12:17:54.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f00327fc700 1 -- 172.21.15.116:0/451944129 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002c00e610 con 0x7f003c0c0cc0 2020-11-20T12:17:54.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f0040958700 1 -- 172.21.15.116:0/451944129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f003c064660 con 0x7f003c0c0cc0 2020-11-20T12:17:54.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f00327fc700 1 -- 172.21.15.116:0/451944129 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002c00e9d0 con 0x7f003c0c0cc0 2020-11-20T12:17:54.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f0040958700 1 -- 172.21.15.116:0/451944129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f003c168fa0 con 0x7f003c0c0cc0 2020-11-20T12:17:54.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.092+0000 7f0040958700 1 -- 172.21.15.116:0/451944129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f003c05ca30 con 0x7f003c0c0cc0 2020-11-20T12:17:54.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.096+0000 7f00327fc700 1 -- 172.21.15.116:0/451944129 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f002c019030 con 0x7f003c0c0cc0 2020-11-20T12:17:54.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.096+0000 7f00327fc700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00180557c0 0x7f0018057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.096+0000 7f00327fc700 1 -- 172.21.15.116:0/451944129 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f002c06da90 con 0x7f003c0c0cc0 2020-11-20T12:17:54.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.096+0000 7f0033fff700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00180557c0 0x7f0018057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.096+0000 7f0033fff700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00180557c0 0x7f0018057c80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f003c0a2e30 tx=0x7f0024006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:54.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.100+0000 7f00327fc700 1 -- 172.21.15.116:0/451944129 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f002c0439e0 con 0x7f003c0c0cc0 2020-11-20T12:17:54.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.260+0000 7f0040958700 1 -- 172.21.15.116:0/451944129 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f003c068120 con 0x7f003c0c0cc0 2020-11-20T12:17:54.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.260+0000 7f00327fc700 1 -- 172.21.15.116:0/451944129 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7f002c047000 con 0x7f003c0c0cc0 2020-11-20T12:17:54.261 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:54.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 -- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00180557c0 msgr2=0x7f0018057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00180557c0 0x7f0018057c80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f003c0a2e30 tx=0x7f0024006df0).stop 2020-11-20T12:17:54.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 -- 172.21.15.116:0/451944129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f003c0c0cc0 msgr2=0x7f003c168910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f003c0c0cc0 0x7f003c168910 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f002c007fd0 tx=0x7f002c007ac0).stop 2020-11-20T12:17:54.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 -- 172.21.15.116:0/451944129 shutdown_connections 2020-11-20T12:17:54.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00180557c0 0x7f0018057c80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f003c149cf0 0x7f003c0a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f003c0c0cc0 0x7f003c168910 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 --2- 172.21.15.116:0/451944129 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f003c149910 0x7f003c151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 -- 172.21.15.116:0/451944129 >> 172.21.15.116:0/451944129 conn(0x7f003c0c14a0 msgr2=0x7f003c14fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:54.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 -- 172.21.15.116:0/451944129 shutdown_connections 2020-11-20T12:17:54.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.264+0000 7f0040958700 1 -- 172.21.15.116:0/451944129 wait complete. 2020-11-20T12:17:54.286 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:17:54.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.464+0000 7fe22958e700 1 Processor -- start 2020-11-20T12:17:54.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.464+0000 7fe22958e700 1 -- start start 2020-11-20T12:17:54.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe22958e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240bffa0 0x7fe2240c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe22958e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2240c0920 con 0x7fe2240a3700 2020-11-20T12:17:54.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe22958e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22414dee0 con 0x7fe22414db00 2020-11-20T12:17:54.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe22958e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240a3e80 con 0x7fe2240bffa0 2020-11-20T12:17:54.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe2237fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240bffa0 0x7fe2240c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe222ffd700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fe22414db00 0x7fe224061180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:51668/0 (socket says 172.21.15.116:51668) 2020-11-20T12:17:54.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe222ffd700 1 -- 172.21.15.116:0/3330760864 learned_addr learned my addr 172.21.15.116:0/3330760864 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:54.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe2227fc700 1 --1- 172.21.15.116:0/3330760864 >> v1:172.21.15.116:6789/0 conn(0x7fe2240a3700 0x7fe2240bf890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55442/0 (socket says 172.21.15.116:55442) 2020-11-20T12:17:54.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe221ffb700 1 -- 172.21.15.116:0/3330760864 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614260168 0 0) 0x7fe22414dee0 con 0x7fe22414db00 2020-11-20T12:17:54.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe221ffb700 1 -- 172.21.15.116:0/3330760864 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe210003240 con 0x7fe22414db00 2020-11-20T12:17:54.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe221ffb700 1 -- 172.21.15.116:0/3330760864 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255653468 0 0) 0x7fe2240c0920 con 0x7fe2240a3700 2020-11-20T12:17:54.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe2237fe700 1 -- 172.21.15.116:0/3330760864 >> v1:172.21.15.173:6789/0 conn(0x7fe22414db00 legacy=0x7fe224061180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe2237fe700 1 -- 172.21.15.116:0/3330760864 >> v1:172.21.15.116:6789/0 conn(0x7fe2240a3700 legacy=0x7fe2240bf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe2237fe700 1 -- 172.21.15.116:0/3330760864 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2240a4120 con 0x7fe2240bffa0 2020-11-20T12:17:54.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.468+0000 7fe2237fe700 1 --2- 172.21.15.116:0/3330760864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240bffa0 0x7fe2240c0430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fe21c009cc0 tx=0x7fe21c0091d0).ready entity=mon.2 client_cookie=9addcc7443bf2193 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:54.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.472+0000 7fe221ffb700 1 -- 172.21.15.116:0/3330760864 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c00bb60 con 0x7fe2240bffa0 2020-11-20T12:17:54.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.472+0000 7fe221ffb700 1 -- 172.21.15.116:0/3330760864 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21c00bcc0 con 0x7fe2240bffa0 2020-11-20T12:17:54.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.472+0000 7fe221ffb700 1 -- 172.21.15.116:0/3330760864 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c011400 con 0x7fe2240bffa0 2020-11-20T12:17:54.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.472+0000 7fe22958e700 1 -- 172.21.15.116:0/3330760864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240bffa0 msgr2=0x7fe2240c0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.472+0000 7fe22958e700 1 --2- 172.21.15.116:0/3330760864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240bffa0 0x7fe2240c0430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fe21c009cc0 tx=0x7fe21c0091d0).stop 2020-11-20T12:17:54.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.472+0000 7fe22958e700 1 -- 172.21.15.116:0/3330760864 shutdown_connections 2020-11-20T12:17:54.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.472+0000 7fe22958e700 1 --2- 172.21.15.116:0/3330760864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240bffa0 0x7fe2240c0430 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.472+0000 7fe22958e700 1 -- 172.21.15.116:0/3330760864 >> 172.21.15.116:0/3330760864 conn(0x7fe2240c1e60 msgr2=0x7fe224066a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:54.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.472+0000 7fe22958e700 1 -- 172.21.15.116:0/3330760864 shutdown_connections 2020-11-20T12:17:54.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.472+0000 7fe22958e700 1 -- 172.21.15.116:0/3330760864 wait complete. 2020-11-20T12:17:54.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.472+0000 7fe22958e700 1 Processor -- start 2020-11-20T12:17:54.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe22958e700 1 -- start start 2020-11-20T12:17:54.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe22958e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240a3700 0x7fe2240a8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe22958e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2240bffa0 0x7fe2241401c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe22958e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe22414db00 0x7fe2241470a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe22958e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240a44a0 con 0x7fe22414db00 2020-11-20T12:17:54.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe22958e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240a4740 con 0x7fe2240bffa0 2020-11-20T12:17:54.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe22958e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2240a49e0 con 0x7fe2240a3700 2020-11-20T12:17:54.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe2237fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe22414db00 0x7fe2241470a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe2237fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe22414db00 0x7fe2241470a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46014/0 (socket says 172.21.15.116:46014) 2020-11-20T12:17:54.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe2237fe700 1 -- 172.21.15.116:0/1139264312 learned_addr learned my addr 172.21.15.116:0/1139264312 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:54.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe222ffd700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240a3700 0x7fe2240a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe2227fc700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2240bffa0 0x7fe2241401c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe2237fe700 1 -- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240a3700 msgr2=0x7fe2240a8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe2237fe700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240a3700 0x7fe2240a8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe2237fe700 1 -- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2240bffa0 msgr2=0x7fe2241401c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe2237fe700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2240bffa0 0x7fe2241401c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe2237fe700 1 -- 172.21.15.116:0/1139264312 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2240a4c80 con 0x7fe22414db00 2020-11-20T12:17:54.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe222ffd700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240a3700 0x7fe2240a8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:54.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe2237fe700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe22414db00 0x7fe2241470a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fe21c009c10 tx=0x7fe21c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:54.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe20affd700 1 -- 172.21.15.116:0/1139264312 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c010800 con 0x7fe22414db00 2020-11-20T12:17:54.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe20affd700 1 -- 172.21.15.116:0/1139264312 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe21c010960 con 0x7fe22414db00 2020-11-20T12:17:54.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe22958e700 1 -- 172.21.15.116:0/1139264312 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2240a8730 con 0x7fe22414db00 2020-11-20T12:17:54.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe20affd700 1 -- 172.21.15.116:0/1139264312 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe21c010ce0 con 0x7fe22414db00 2020-11-20T12:17:54.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.476+0000 7fe22958e700 1 -- 172.21.15.116:0/1139264312 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2241406b0 con 0x7fe22414db00 2020-11-20T12:17:54.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.480+0000 7fe22958e700 1 -- 172.21.15.116:0/1139264312 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe224044f60 con 0x7fe22414db00 2020-11-20T12:17:54.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.480+0000 7fe20affd700 1 -- 172.21.15.116:0/1139264312 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe21c024030 con 0x7fe22414db00 2020-11-20T12:17:54.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.480+0000 7fe20affd700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2000557c0 0x7fe200057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.480+0000 7fe20affd700 1 -- 172.21.15.116:0/1139264312 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe21c070360 con 0x7fe22414db00 2020-11-20T12:17:54.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.480+0000 7fe222ffd700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2000557c0 0x7fe200057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.484+0000 7fe222ffd700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2000557c0 0x7fe200057c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fe2140055a0 tx=0x7fe214009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:54.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.484+0000 7fe20affd700 1 -- 172.21.15.116:0/1139264312 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe21c046330 con 0x7fe22414db00 2020-11-20T12:17:54.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.644+0000 7fe22958e700 1 -- 172.21.15.116:0/1139264312 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2241408a0 con 0x7fe22414db00 2020-11-20T12:17:54.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.644+0000 7fe20affd700 1 -- 172.21.15.116:0/1139264312 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe21c0220a0 con 0x7fe22414db00 2020-11-20T12:17:54.646 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:54.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 -- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2000557c0 msgr2=0x7fe200057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2000557c0 0x7fe200057c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7fe2140055a0 tx=0x7fe214009040).stop 2020-11-20T12:17:54.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 -- 172.21.15.116:0/1139264312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe22414db00 msgr2=0x7fe2241470a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe22414db00 0x7fe2241470a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fe21c009c10 tx=0x7fe21c016000).stop 2020-11-20T12:17:54.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 -- 172.21.15.116:0/1139264312 shutdown_connections 2020-11-20T12:17:54.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2000557c0 0x7fe200057c80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe22414db00 0x7fe2241470a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2240a3700 0x7fe2240a8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 --2- 172.21.15.116:0/1139264312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2240bffa0 0x7fe2241401c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 -- 172.21.15.116:0/1139264312 >> 172.21.15.116:0/1139264312 conn(0x7fe2240c1e60 msgr2=0x7fe224066a10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:54.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 -- 172.21.15.116:0/1139264312 shutdown_connections 2020-11-20T12:17:54.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.648+0000 7fe22958e700 1 -- 172.21.15.116:0/1139264312 wait complete. 2020-11-20T12:17:54.674 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:17:54.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.872+0000 7ff5eb59e700 1 Processor -- start 2020-11-20T12:17:54.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.872+0000 7ff5eb59e700 1 -- start start 2020-11-20T12:17:54.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.872+0000 7ff5eb59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff5ec13ad30 0x7ff5ec0a7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.872+0000 7ff5eb59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5ec13b110 0x7ff5ec151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.872+0000 7ff5eb59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5ec151fa0 0x7ff5ec061250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.872+0000 7ff5eb59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ec152a80 con 0x7ff5ec151fa0 2020-11-20T12:17:54.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.872+0000 7ff5eb59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ec152540 con 0x7ff5ec13ad30 2020-11-20T12:17:54.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.872+0000 7ff5eb59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec1527e0 con 0x7ff5ec13b110 2020-11-20T12:17:54.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5ead9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5ec151fa0 0x7ff5ec061250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5ead9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5ec151fa0 0x7ff5ec061250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46022/0 (socket says 172.21.15.116:46022) 2020-11-20T12:17:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5ead9d700 1 -- 172.21.15.116:0/3550906346 learned_addr learned my addr 172.21.15.116:0/3550906346 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:54.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5ea59c700 1 --2- 172.21.15.116:0/3550906346 >> v2:172.21.15.173:3300/0 conn(0x7ff5ec13ad30 0x7ff5ec0a7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5e9d9b700 1 --2- 172.21.15.116:0/3550906346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5ec13b110 0x7ff5ec151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5ead9d700 1 -- 172.21.15.116:0/3550906346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5ec13b110 msgr2=0x7ff5ec151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:54.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5ead9d700 1 --2- 172.21.15.116:0/3550906346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5ec13b110 0x7ff5ec151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5ead9d700 1 -- 172.21.15.116:0/3550906346 >> v2:172.21.15.173:3300/0 conn(0x7ff5ec13ad30 msgr2=0x7ff5ec0a7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:54.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5ead9d700 1 --2- 172.21.15.116:0/3550906346 >> v2:172.21.15.173:3300/0 conn(0x7ff5ec13ad30 0x7ff5ec0a7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5ead9d700 1 -- 172.21.15.116:0/3550906346 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec0ac910 con 0x7ff5ec151fa0 2020-11-20T12:17:54.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5ead9d700 1 --2- 172.21.15.116:0/3550906346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5ec151fa0 0x7ff5ec061250 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7ff5ec081330 tx=0x7ff5e4009290).ready entity=mon.0 client_cookie=1cf76ef87d4f9cb2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:54.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5e959a700 1 -- 172.21.15.116:0/3550906346 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e400bba0 con 0x7ff5ec151fa0 2020-11-20T12:17:54.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5e959a700 1 -- 172.21.15.116:0/3550906346 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e400bd00 con 0x7ff5ec151fa0 2020-11-20T12:17:54.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5e959a700 1 -- 172.21.15.116:0/3550906346 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e4011610 con 0x7ff5ec151fa0 2020-11-20T12:17:54.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5eb59e700 1 -- 172.21.15.116:0/3550906346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5ec151fa0 msgr2=0x7ff5ec061250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.876+0000 7ff5eb59e700 1 --2- 172.21.15.116:0/3550906346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5ec151fa0 0x7ff5ec061250 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7ff5ec081330 tx=0x7ff5e4009290).stop 2020-11-20T12:17:54.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.880+0000 7ff5eb59e700 1 -- 172.21.15.116:0/3550906346 shutdown_connections 2020-11-20T12:17:54.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.880+0000 7ff5eb59e700 1 --2- 172.21.15.116:0/3550906346 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5ec151fa0 0x7ff5ec061250 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.880+0000 7ff5eb59e700 1 --2- 172.21.15.116:0/3550906346 >> v2:172.21.15.173:3300/0 conn(0x7ff5ec13ad30 0x7ff5ec0a7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.880+0000 7ff5eb59e700 1 --2- 172.21.15.116:0/3550906346 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5ec13b110 0x7ff5ec151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.880+0000 7ff5eb59e700 1 -- 172.21.15.116:0/3550906346 >> 172.21.15.116:0/3550906346 conn(0x7ff5ec0678a0 msgr2=0x7ff5ec067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:54.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.880+0000 7ff5eb59e700 1 -- 172.21.15.116:0/3550906346 shutdown_connections 2020-11-20T12:17:54.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.880+0000 7ff5eb59e700 1 -- 172.21.15.116:0/3550906346 wait complete. 2020-11-20T12:17:54.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.880+0000 7ff5eb59e700 1 Processor -- start 2020-11-20T12:17:54.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.880+0000 7ff5eb59e700 1 -- start start 2020-11-20T12:17:54.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5eb59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5ec13ad30 0x7ff5ec0c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5eb59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5ec13b110 0x7ff5ec066e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5eb59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5ec151fa0 0x7ff5ec064fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5eb59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec168330 con 0x7ff5ec13b110 2020-11-20T12:17:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5eb59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec168560 con 0x7ff5ec13ad30 2020-11-20T12:17:54.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5eb59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec168800 con 0x7ff5ec151fa0 2020-11-20T12:17:54.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5e9d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5ec13b110 0x7ff5ec066e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5e9d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5ec13b110 0x7ff5ec066e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46024/0 (socket says 172.21.15.116:46024) 2020-11-20T12:17:54.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5e9d9b700 1 -- 172.21.15.116:0/830866089 learned_addr learned my addr 172.21.15.116:0/830866089 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:54.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5ead9d700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5ec151fa0 0x7ff5ec064fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5ea59c700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5ec13ad30 0x7ff5ec0c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5e9d9b700 1 -- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5ec151fa0 msgr2=0x7ff5ec064fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5e9d9b700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5ec151fa0 0x7ff5ec064fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5e9d9b700 1 -- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5ec13ad30 msgr2=0x7ff5ec0c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:54.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5e9d9b700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5ec13ad30 0x7ff5ec0c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:54.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5e9d9b700 1 -- 172.21.15.116:0/830866089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec168aa0 con 0x7ff5ec13b110 2020-11-20T12:17:54.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5ead9d700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5ec151fa0 0x7ff5ec064fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:54.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5e9d9b700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5ec13b110 0x7ff5ec066e20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7ff5e000ebd0 tx=0x7ff5e0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:54.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5da7fc700 1 -- 172.21.15.116:0/830866089 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000c9b0 con 0x7ff5ec13b110 2020-11-20T12:17:54.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5da7fc700 1 -- 172.21.15.116:0/830866089 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e000cb10 con 0x7ff5ec13b110 2020-11-20T12:17:54.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5eb59e700 1 -- 172.21.15.116:0/830866089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ec168d40 con 0x7ff5ec13b110 2020-11-20T12:17:54.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5da7fc700 1 -- 172.21.15.116:0/830866089 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e000ce90 con 0x7ff5ec13b110 2020-11-20T12:17:54.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.884+0000 7ff5eb59e700 1 -- 172.21.15.116:0/830866089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ec064840 con 0x7ff5ec13b110 2020-11-20T12:17:54.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.888+0000 7ff5eb59e700 1 -- 172.21.15.116:0/830866089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ec044f60 con 0x7ff5ec13b110 2020-11-20T12:17:54.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.888+0000 7ff5da7fc700 1 -- 172.21.15.116:0/830866089 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff5e0012600 con 0x7ff5ec13b110 2020-11-20T12:17:54.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.888+0000 7ff5da7fc700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5c00557c0 0x7ff5c0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:54.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.888+0000 7ff5da7fc700 1 -- 172.21.15.116:0/830866089 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff5e006eb70 con 0x7ff5ec13b110 2020-11-20T12:17:54.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.888+0000 7ff5ea59c700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5c00557c0 0x7ff5c0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:54.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.892+0000 7ff5ea59c700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5c00557c0 0x7ff5c0057c80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7ff5ec0c7c60 tx=0x7ff5dc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:54.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:54.896+0000 7ff5da7fc700 1 -- 172.21.15.116:0/830866089 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff5e0044a10 con 0x7ff5ec13b110 2020-11-20T12:17:55.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.056+0000 7ff5eb59e700 1 -- 172.21.15.116:0/830866089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5ec069320 con 0x7ff5ec13b110 2020-11-20T12:17:55.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.056+0000 7ff5da7fc700 1 -- 172.21.15.116:0/830866089 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7ff5e0048030 con 0x7ff5ec13b110 2020-11-20T12:17:55.056 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:55.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.056+0000 7ff5eb59e700 1 -- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5c00557c0 msgr2=0x7ff5c0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:55.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.056+0000 7ff5eb59e700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5c00557c0 0x7ff5c0057c80 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7ff5ec0c7c60 tx=0x7ff5dc008040).stop 2020-11-20T12:17:55.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.056+0000 7ff5eb59e700 1 -- 172.21.15.116:0/830866089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5ec13b110 msgr2=0x7ff5ec066e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:55.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.056+0000 7ff5eb59e700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5ec13b110 0x7ff5ec066e20 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7ff5e000ebd0 tx=0x7ff5e0006df0).stop 2020-11-20T12:17:55.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.056+0000 7ff5eb59e700 1 -- 172.21.15.116:0/830866089 shutdown_connections 2020-11-20T12:17:55.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.056+0000 7ff5eb59e700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5c00557c0 0x7ff5c0057c80 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.056+0000 7ff5eb59e700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5ec151fa0 0x7ff5ec064fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.056+0000 7ff5eb59e700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5ec13ad30 0x7ff5ec0c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.056+0000 7ff5eb59e700 1 --2- 172.21.15.116:0/830866089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5ec13b110 0x7ff5ec066e20 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.060+0000 7ff5eb59e700 1 -- 172.21.15.116:0/830866089 >> 172.21.15.116:0/830866089 conn(0x7ff5ec0678a0 msgr2=0x7ff5ec0adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:55.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.060+0000 7ff5eb59e700 1 -- 172.21.15.116:0/830866089 shutdown_connections 2020-11-20T12:17:55.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.060+0000 7ff5eb59e700 1 -- 172.21.15.116:0/830866089 wait complete. 2020-11-20T12:17:55.080 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:17:55.081 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:17:55.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.260+0000 7f1ab2d14700 1 Processor -- start 2020-11-20T12:17:55.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.260+0000 7f1ab2d14700 1 -- start start 2020-11-20T12:17:55.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2d14700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1aac0aa790 0x7f1aac152540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:55.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2d14700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1aac0612f0 0x7f1aac061760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:55.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2d14700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aac152a30 con 0x7f1aac13aee0 2020-11-20T12:17:55.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2d14700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aac0adce0 con 0x7f1aac0aa790 2020-11-20T12:17:55.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2d14700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aac061c50 con 0x7f1aac0612f0 2020-11-20T12:17:55.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab1511700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1aac0aa790 0x7f1aac152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:55.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab1511700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1aac0aa790 0x7f1aac152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46034/0 (socket says 172.21.15.116:46034) 2020-11-20T12:17:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab1511700 1 -- 172.21.15.116:0/586947488 learned_addr learned my addr 172.21.15.116:0/586947488 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:55.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2513700 1 --2- 172.21.15.116:0/586947488 >> v2:172.21.15.173:3300/0 conn(0x7f1aac0612f0 0x7f1aac061760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab1511700 1 -- 172.21.15.116:0/586947488 >> v2:172.21.15.173:3300/0 conn(0x7f1aac0612f0 msgr2=0x7f1aac061760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab1511700 1 --2- 172.21.15.116:0/586947488 >> v2:172.21.15.173:3300/0 conn(0x7f1aac0612f0 0x7f1aac061760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab1511700 1 -- 172.21.15.116:0/586947488 >> v1:172.21.15.116:6789/0 conn(0x7f1aac13aee0 legacy=0x7f1aac0ad5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab1511700 1 -- 172.21.15.116:0/586947488 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac0aac10 con 0x7f1aac0aa790 2020-11-20T12:17:55.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab1511700 1 --2- 172.21.15.116:0/586947488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1aac0aa790 0x7f1aac152540 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f1aac081760 tx=0x7f1a9c006eb0).ready entity=mon.0 client_cookie=3442e0ab544d6cfb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab0d10700 1 -- 172.21.15.116:0/586947488 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c0098d0 con 0x7f1aac0aa790 2020-11-20T12:17:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab0d10700 1 -- 172.21.15.116:0/586947488 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c009a30 con 0x7f1aac0aa790 2020-11-20T12:17:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab0d10700 1 -- 172.21.15.116:0/586947488 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c009db0 con 0x7f1aac0aa790 2020-11-20T12:17:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2d14700 1 -- 172.21.15.116:0/586947488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1aac0aa790 msgr2=0x7f1aac152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:55.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2d14700 1 --2- 172.21.15.116:0/586947488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1aac0aa790 0x7f1aac152540 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f1aac081760 tx=0x7f1a9c006eb0).stop 2020-11-20T12:17:55.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2d14700 1 -- 172.21.15.116:0/586947488 shutdown_connections 2020-11-20T12:17:55.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2d14700 1 --2- 172.21.15.116:0/586947488 >> v2:172.21.15.173:3300/0 conn(0x7f1aac0612f0 0x7f1aac061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2d14700 1 --2- 172.21.15.116:0/586947488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1aac0aa790 0x7f1aac152540 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.264+0000 7f1ab2d14700 1 -- 172.21.15.116:0/586947488 >> 172.21.15.116:0/586947488 conn(0x7f1aac0678a0 msgr2=0x7f1aac067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:55.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.268+0000 7f1ab2d14700 1 -- 172.21.15.116:0/586947488 shutdown_connections 2020-11-20T12:17:55.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.268+0000 7f1ab2d14700 1 -- 172.21.15.116:0/586947488 wait complete. 2020-11-20T12:17:55.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.268+0000 7f1ab2d14700 1 Processor -- start 2020-11-20T12:17:55.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.268+0000 7f1ab2d14700 1 -- start start 2020-11-20T12:17:55.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab2d14700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1aac0612f0 0x7f1aac161a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:55.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab2d14700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1aac0aa790 0x7f1aac0a7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:55.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab2d14700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac13aee0 0x7f1aac0c1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:55.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab2d14700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac160e30 con 0x7f1aac0aa790 2020-11-20T12:17:55.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab2d14700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac1651f0 con 0x7f1aac13aee0 2020-11-20T12:17:55.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab2d14700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aac161090 con 0x7f1aac0612f0 2020-11-20T12:17:55.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab1511700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1aac0aa790 0x7f1aac0a7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:55.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab1511700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1aac0aa790 0x7f1aac0a7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46040/0 (socket says 172.21.15.116:46040) 2020-11-20T12:17:55.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab1511700 1 -- 172.21.15.116:0/2198886169 learned_addr learned my addr 172.21.15.116:0/2198886169 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:55.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab1d12700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1aac0612f0 0x7f1aac161a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:55.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab2513700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac13aee0 0x7f1aac0c1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:55.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab1511700 1 -- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1aac0612f0 msgr2=0x7f1aac161a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:55.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab1511700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1aac0612f0 0x7f1aac161a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab1511700 1 -- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac13aee0 msgr2=0x7f1aac0c1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:55.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab1511700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac13aee0 0x7f1aac0c1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab1511700 1 -- 172.21.15.116:0/2198886169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aac161330 con 0x7f1aac0aa790 2020-11-20T12:17:55.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab1511700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1aac0aa790 0x7f1aac0a7e80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f1aac0c5ef0 tx=0x7f1a9c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:55.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1a99ffb700 1 -- 172.21.15.116:0/2198886169 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c011c80 con 0x7f1aac0aa790 2020-11-20T12:17:55.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1a99ffb700 1 -- 172.21.15.116:0/2198886169 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a9c011de0 con 0x7f1aac0aa790 2020-11-20T12:17:55.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab2d14700 1 -- 172.21.15.116:0/2198886169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aac1615d0 con 0x7f1aac0aa790 2020-11-20T12:17:55.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1a99ffb700 1 -- 172.21.15.116:0/2198886169 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a9c01e670 con 0x7f1aac0aa790 2020-11-20T12:17:55.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.272+0000 7f1ab2d14700 1 -- 172.21.15.116:0/2198886169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aac0a8590 con 0x7f1aac0aa790 2020-11-20T12:17:55.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.276+0000 7f1ab2d14700 1 -- 172.21.15.116:0/2198886169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aac044f60 con 0x7f1aac0aa790 2020-11-20T12:17:55.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.276+0000 7f1a99ffb700 1 -- 172.21.15.116:0/2198886169 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1a9c025030 con 0x7f1aac0aa790 2020-11-20T12:17:55.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.276+0000 7f1a99ffb700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a840557c0 0x7f1a84057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:55.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.276+0000 7f1a99ffb700 1 -- 172.21.15.116:0/2198886169 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1a9c06fd70 con 0x7f1aac0aa790 2020-11-20T12:17:55.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.276+0000 7f1ab1d12700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a840557c0 0x7f1a84057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:55.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.276+0000 7f1ab1d12700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a840557c0 0x7f1a84057c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f1aa8003670 tx=0x7f1aa8007b90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:55.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.280+0000 7f1a99ffb700 1 -- 172.21.15.116:0/2198886169 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1a9c045bb0 con 0x7f1aac0aa790 2020-11-20T12:17:55.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.488+0000 7f1ab2d14700 1 -- 172.21.15.116:0/2198886169 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1aac060500 con 0x7f1aac0aa790 2020-11-20T12:17:55.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1a99ffb700 1 -- 172.21.15.116:0/2198886169 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+2849 (secure 0 0 0) 0x7f1a9c023070 con 0x7f1aac0aa790 2020-11-20T12:17:55.491 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:55.491 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1096724826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1096724826},{"type":"v1","addr":"172.21.15.173:6837","nonce":1096724826}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/1589443098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1589443098},{"type":"v1","addr":"172.21.15.173:6835","nonce":1589443098}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/966300328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":966300328},{"type":"v1","addr":"172.21.15.116:6833","nonce":966300328}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:17:40.163408+0000","modified":"2020-11-20T12:17:42.171423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5493":{"gid":5493,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/2908037349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2908037349},{"type":"v1","addr":"172.21.15.116:6835","nonce":2908037349}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-20T12:17:55.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 -- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a840557c0 msgr2=0x7f1a84057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:55.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a840557c0 0x7f1a84057c80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f1aa8003670 tx=0x7f1aa8007b90).stop 2020-11-20T12:17:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 -- 172.21.15.116:0/2198886169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1aac0aa790 msgr2=0x7f1aac0a7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1aac0aa790 0x7f1aac0a7e80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f1aac0c5ef0 tx=0x7f1a9c015000).stop 2020-11-20T12:17:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 -- 172.21.15.116:0/2198886169 shutdown_connections 2020-11-20T12:17:55.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a840557c0 0x7f1a84057c80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1aac13aee0 0x7f1aac0c1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1aac0612f0 0x7f1aac161a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 --2- 172.21.15.116:0/2198886169 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1aac0aa790 0x7f1aac0a7e80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:55.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 -- 172.21.15.116:0/2198886169 >> 172.21.15.116:0/2198886169 conn(0x7f1aac0678a0 msgr2=0x7f1aac0c5670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:55.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 -- 172.21.15.116:0/2198886169 shutdown_connections 2020-11-20T12:17:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:55.492+0000 7f1ab2d14700 1 -- 172.21.15.116:0/2198886169 wait complete. 2020-11-20T12:17:55.499 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 20 2020-11-20T12:17:55.921 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:17:40.163408+0000', 'modified': '2020-11-20T12:17:42.171423+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5493': {'gid': 5493, 'name': 'a', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/2908037349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2908037349}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 2908037349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:17:55.921 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:17:55.922 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:17:56.116 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":5493,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.932298668,"mdsmap_epoch":20,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":21.449995489999999} 2020-11-20T12:17:57.694 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 5493, "mdsmap_epoch": 20, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 15.932298668, "state": "up:active", "uptime": 21.44999549, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:17:57.695 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:17:57.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:57.697 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.2 2020-11-20T12:17:57.719 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:17:57.720 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:57.795 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:57.797 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:17:57.820 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:17:57.821 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:17:57.828 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:17:57.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:57.829 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2020-11-20T12:17:57.859 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:17:57.859 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:17:57.860 INFO:tasks.cephfs.mount:self.client.name = client.3 2020-11-20T12:17:57.860 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2020-11-20T12:17:57.861 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:17:57.862 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:17:57.862 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:17:57.863 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:17:57.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:57.864 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:17:57.885 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:17:57.886 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:17:57.886 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:17:57.886 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:57.887 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:17:57.887 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:57.887 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:57.887 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:57.888 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:57.888 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:57.888 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:17:57.888 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:57.888 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:17:57.889 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:57.889 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:17:57.889 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:57.889 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:17:57.890 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:57.890 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:17:57.890 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:57.890 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:17:57.890 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:57.891 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:17:57.891 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:17:57.891 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:17:57.891 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:57.891 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:17:57.892 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:57.892 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:57.892 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:17:57.892 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:17:57.893 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:57.893 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:17:57.893 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:17:57.893 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:17:57.893 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:17:57.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:57.895 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:17:57.933 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:17:57.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:57.937 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:17:58.006 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:17:58.007 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:17:58.007 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:17:58.007 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:17:58.007 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:17:58.008 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:17:58.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:58.009 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:17:58.010 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:17:58.010 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:17:58.010 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:17:58.011 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:17:58.011 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:17:58.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:58.108 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:17:58.127 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:17:58.128 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:17:58.129 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:17:58.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:17:58.130 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2020-11-20T12:17:58.160 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:17:58.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2f10a3700 1 Processor -- start 2020-11-20T12:17:58.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2f10a3700 1 -- start start 2020-11-20T12:17:58.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2f10a3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe2ec13ac60 0x7fe2ec1524e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2f10a3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2ec155060 0x7fe2ec1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2f10a3700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2ec1529d0 con 0x7fe2ec060170 2020-11-20T12:17:58.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2f10a3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ec155b70 con 0x7fe2ec155060 2020-11-20T12:17:58.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2f10a3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2ec155e10 con 0x7fe2ec13ac60 2020-11-20T12:17:58.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2eb7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2ec155060 0x7fe2ec1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2eb7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2ec155060 0x7fe2ec1554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46050/0 (socket says 172.21.15.116:46050) 2020-11-20T12:17:58.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2eb7fe700 1 -- 172.21.15.116:0/203891071 learned_addr learned my addr 172.21.15.116:0/203891071 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:58.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2eb7fe700 1 -- 172.21.15.116:0/203891071 >> v2:172.21.15.173:3300/0 conn(0x7fe2ec13ac60 msgr2=0x7fe2ec1524e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:58.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.344+0000 7fe2ebfff700 1 --2- 172.21.15.116:0/203891071 >> v2:172.21.15.173:3300/0 conn(0x7fe2ec13ac60 0x7fe2ec1524e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2eb7fe700 1 --2- 172.21.15.116:0/203891071 >> v2:172.21.15.173:3300/0 conn(0x7fe2ec13ac60 0x7fe2ec1524e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2eb7fe700 1 -- 172.21.15.116:0/203891071 >> v1:172.21.15.173:6789/0 conn(0x7fe2ec060170 legacy=0x7fe2ec0bf7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:58.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2eb7fe700 1 -- 172.21.15.116:0/203891071 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec151da0 con 0x7fe2ec155060 2020-11-20T12:17:58.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2ebfff700 1 --2- 172.21.15.116:0/203891071 >> v2:172.21.15.173:3300/0 conn(0x7fe2ec13ac60 0x7fe2ec1524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:58.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2eb7fe700 1 --2- 172.21.15.116:0/203891071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2ec155060 0x7fe2ec1554e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fe2ec066330 tx=0x7fe2e0006eb0).ready entity=mon.0 client_cookie=61af9a7218cca766 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:58.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2eaffd700 1 -- 172.21.15.116:0/203891071 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0009920 con 0x7fe2ec155060 2020-11-20T12:17:58.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2eaffd700 1 -- 172.21.15.116:0/203891071 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e0009a80 con 0x7fe2ec155060 2020-11-20T12:17:58.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2eaffd700 1 -- 172.21.15.116:0/203891071 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0009e00 con 0x7fe2ec155060 2020-11-20T12:17:58.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2f10a3700 1 -- 172.21.15.116:0/203891071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2ec155060 msgr2=0x7fe2ec1554e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:58.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2f10a3700 1 --2- 172.21.15.116:0/203891071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2ec155060 0x7fe2ec1554e0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fe2ec066330 tx=0x7fe2e0006eb0).stop 2020-11-20T12:17:58.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2f10a3700 1 -- 172.21.15.116:0/203891071 shutdown_connections 2020-11-20T12:17:58.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2f10a3700 1 --2- 172.21.15.116:0/203891071 >> v2:172.21.15.173:3300/0 conn(0x7fe2ec13ac60 0x7fe2ec1524e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2f10a3700 1 --2- 172.21.15.116:0/203891071 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2ec155060 0x7fe2ec1554e0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2f10a3700 1 -- 172.21.15.116:0/203891071 >> 172.21.15.116:0/203891071 conn(0x7fe2ec0c51a0 msgr2=0x7fe2ec0c5580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:58.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2f10a3700 1 -- 172.21.15.116:0/203891071 shutdown_connections 2020-11-20T12:17:58.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.348+0000 7fe2f10a3700 1 -- 172.21.15.116:0/203891071 wait complete. 2020-11-20T12:17:58.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2f10a3700 1 Processor -- start 2020-11-20T12:17:58.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2f10a3700 1 -- start start 2020-11-20T12:17:58.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2f10a3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2ec060170 0x7fe2ec0aaa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2f10a3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ec13ac60 0x7fe2ec161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2f10a3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2ec155060 0x7fe2ec0a8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2f10a3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec1562a0 con 0x7fe2ec060170 2020-11-20T12:17:58.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2f10a3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec156540 con 0x7fe2ec13ac60 2020-11-20T12:17:58.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2f10a3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2ec1567e0 con 0x7fe2ec155060 2020-11-20T12:17:58.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2ebfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2ec060170 0x7fe2ec0aaa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2ebfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2ec060170 0x7fe2ec0aaa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46056/0 (socket says 172.21.15.116:46056) 2020-11-20T12:17:58.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2ebfff700 1 -- 172.21.15.116:0/3950494121 learned_addr learned my addr 172.21.15.116:0/3950494121 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:58.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2eb7fe700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ec13ac60 0x7fe2ec161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2f08a2700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2ec155060 0x7fe2ec0a8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2ebfff700 1 -- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2ec155060 msgr2=0x7fe2ec0a8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:58.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2ebfff700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2ec155060 0x7fe2ec0a8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2ebfff700 1 -- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ec13ac60 msgr2=0x7fe2ec161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:58.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.352+0000 7fe2ebfff700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ec13ac60 0x7fe2ec161f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.356+0000 7fe2ebfff700 1 -- 172.21.15.116:0/3950494121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec156a80 con 0x7fe2ec060170 2020-11-20T12:17:58.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.356+0000 7fe2eb7fe700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ec13ac60 0x7fe2ec161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:58.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.356+0000 7fe2f08a2700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2ec155060 0x7fe2ec0a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:17:58.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.356+0000 7fe2ebfff700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2ec060170 0x7fe2ec0aaa70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fe2dc00cb50 tx=0x7fe2dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:58.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.356+0000 7fe2cbfff700 1 -- 172.21.15.116:0/3950494121 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc015070 con 0x7fe2ec060170 2020-11-20T12:17:58.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.356+0000 7fe2cbfff700 1 -- 172.21.15.116:0/3950494121 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc006020 con 0x7fe2ec060170 2020-11-20T12:17:58.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.356+0000 7fe2f10a3700 1 -- 172.21.15.116:0/3950494121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2ec0aaf60 con 0x7fe2ec060170 2020-11-20T12:17:58.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.356+0000 7fe2cbfff700 1 -- 172.21.15.116:0/3950494121 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0063a0 con 0x7fe2ec060170 2020-11-20T12:17:58.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.356+0000 7fe2f10a3700 1 -- 172.21.15.116:0/3950494121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2ec0a8640 con 0x7fe2ec060170 2020-11-20T12:17:58.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.356+0000 7fe2f10a3700 1 -- 172.21.15.116:0/3950494121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2ec156c10 con 0x7fe2ec060170 2020-11-20T12:17:58.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.360+0000 7fe2cbfff700 1 -- 172.21.15.116:0/3950494121 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe2dc0079e0 con 0x7fe2ec060170 2020-11-20T12:17:58.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.360+0000 7fe2cbfff700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2c40557c0 0x7fe2c4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.360+0000 7fe2cbfff700 1 -- 172.21.15.116:0/3950494121 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe2dc06ef70 con 0x7fe2ec060170 2020-11-20T12:17:58.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.360+0000 7fe2eb7fe700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2c40557c0 0x7fe2c4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.360+0000 7fe2eb7fe700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2c40557c0 0x7fe2c4057c80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fe2ec0ab450 tx=0x7fe2e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:58.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.364+0000 7fe2cbfff700 1 -- 172.21.15.116:0/3950494121 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe2dc044e30 con 0x7fe2ec060170 2020-11-20T12:17:58.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2f10a3700 1 -- 172.21.15.116:0/3950494121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2ec044f60 con 0x7fe2ec060170 2020-11-20T12:17:58.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2cbfff700 1 -- 172.21.15.116:0/3950494121 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fe2dc048450 con 0x7fe2ec060170 2020-11-20T12:17:58.524 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:17:58.524 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":45,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:17:41.167389+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:17:38.002878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-20T12:17:38.995456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:17:58.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2f10a3700 1 -- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2c40557c0 msgr2=0x7fe2c4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:58.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2f10a3700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2c40557c0 0x7fe2c4057c80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7fe2ec0ab450 tx=0x7fe2e0015000).stop 2020-11-20T12:17:58.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2f10a3700 1 -- 172.21.15.116:0/3950494121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2ec060170 msgr2=0x7fe2ec0aaa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:58.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2f10a3700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2ec060170 0x7fe2ec0aaa70 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7fe2dc00cb50 tx=0x7fe2dc008040).stop 2020-11-20T12:17:58.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2f10a3700 1 -- 172.21.15.116:0/3950494121 shutdown_connections 2020-11-20T12:17:58.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2f10a3700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2c40557c0 0x7fe2c4057c80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2f10a3700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2ec155060 0x7fe2ec0a8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2f10a3700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2ec060170 0x7fe2ec0aaa70 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2f10a3700 1 --2- 172.21.15.116:0/3950494121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2ec13ac60 0x7fe2ec161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.524+0000 7fe2f10a3700 1 -- 172.21.15.116:0/3950494121 >> 172.21.15.116:0/3950494121 conn(0x7fe2ec0c51a0 msgr2=0x7fe2ec065570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:58.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.528+0000 7fe2f10a3700 1 -- 172.21.15.116:0/3950494121 shutdown_connections 2020-11-20T12:17:58.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.528+0000 7fe2f10a3700 1 -- 172.21.15.116:0/3950494121 wait complete. 2020-11-20T12:17:58.554 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:17:38.002878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:17:38.995456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:17:58.555 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:17:58.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.736+0000 7fe28f59e700 1 Processor -- start 2020-11-20T12:17:58.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.740+0000 7fe28f59e700 1 -- start start 2020-11-20T12:17:58.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.740+0000 7fe28f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2900644e0 0x7fe2900c0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.740+0000 7fe28f59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe290137d60 0x7fe2900c3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.740+0000 7fe28f59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2900c1620 con 0x7fe2900648c0 2020-11-20T12:17:58.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.740+0000 7fe28f59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2900c2cd0 con 0x7fe290137d60 2020-11-20T12:17:58.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.740+0000 7fe28f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900c2f70 con 0x7fe2900644e0 2020-11-20T12:17:58.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28ed9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe290137d60 0x7fe2900c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28dd9b700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fe2900648c0 0x7fe2900670c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:51730/0 (socket says 172.21.15.116:51730) 2020-11-20T12:17:58.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28dd9b700 1 -- 172.21.15.116:0/530253030 learned_addr learned my addr 172.21.15.116:0/530253030 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:58.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28e59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2900644e0 0x7fe2900c0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28ed9d700 1 -- 172.21.15.116:0/530253030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2900644e0 msgr2=0x7fe2900c0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:58.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28ed9d700 1 --2- 172.21.15.116:0/530253030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2900644e0 0x7fe2900c0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28ed9d700 1 -- 172.21.15.116:0/530253030 >> v1:172.21.15.173:6789/0 conn(0x7fe2900648c0 legacy=0x7fe2900670c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:58.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28ed9d700 1 -- 172.21.15.116:0/530253030 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900c3210 con 0x7fe290137d60 2020-11-20T12:17:58.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28ed9d700 1 --2- 172.21.15.116:0/530253030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe290137d60 0x7fe2900c3950 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fe288009c90 tx=0x7fe2880091d0).ready entity=mon.0 client_cookie=61aaf07534770d3f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:58.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28d59a700 1 -- 172.21.15.116:0/530253030 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28800bb00 con 0x7fe290137d60 2020-11-20T12:17:58.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28d59a700 1 -- 172.21.15.116:0/530253030 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28800bc60 con 0x7fe290137d60 2020-11-20T12:17:58.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28d59a700 1 -- 172.21.15.116:0/530253030 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe288011400 con 0x7fe290137d60 2020-11-20T12:17:58.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28f59e700 1 -- 172.21.15.116:0/530253030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe290137d60 msgr2=0x7fe2900c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:58.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28f59e700 1 --2- 172.21.15.116:0/530253030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe290137d60 0x7fe2900c3950 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7fe288009c90 tx=0x7fe2880091d0).stop 2020-11-20T12:17:58.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28f59e700 1 -- 172.21.15.116:0/530253030 shutdown_connections 2020-11-20T12:17:58.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28f59e700 1 --2- 172.21.15.116:0/530253030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe290137d60 0x7fe2900c3950 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28f59e700 1 --2- 172.21.15.116:0/530253030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2900644e0 0x7fe2900c0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28f59e700 1 -- 172.21.15.116:0/530253030 >> 172.21.15.116:0/530253030 conn(0x7fe290061d50 msgr2=0x7fe2900c50d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:58.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28f59e700 1 -- 172.21.15.116:0/530253030 shutdown_connections 2020-11-20T12:17:58.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.744+0000 7fe28f59e700 1 -- 172.21.15.116:0/530253030 wait complete. 2020-11-20T12:17:58.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28f59e700 1 Processor -- start 2020-11-20T12:17:58.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28f59e700 1 -- start start 2020-11-20T12:17:58.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28f59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900644e0 0x7fe290063520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28f59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900648c0 0x7fe2900bd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe290137d60 0x7fe2900bd550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28f59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900be0b0 con 0x7fe2900648c0 2020-11-20T12:17:58.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28f59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900a8190 con 0x7fe2900644e0 2020-11-20T12:17:58.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2900bdb70 con 0x7fe290137d60 2020-11-20T12:17:58.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28dd9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900648c0 0x7fe2900bd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28ed9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe290137d60 0x7fe2900bd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28e59c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900644e0 0x7fe290063520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28ed9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe290137d60 0x7fe2900bd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:55022/0 (socket says 172.21.15.116:55022) 2020-11-20T12:17:58.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28ed9d700 1 -- 172.21.15.116:0/95371229 learned_addr learned my addr 172.21.15.116:0/95371229 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:58.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28ed9d700 1 -- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900644e0 msgr2=0x7fe290063520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:58.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28ed9d700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900644e0 0x7fe290063520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28ed9d700 1 -- 172.21.15.116:0/95371229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900648c0 msgr2=0x7fe2900bd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:58.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28ed9d700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900648c0 0x7fe2900bd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.748+0000 7fe28ed9d700 1 -- 172.21.15.116:0/95371229 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2900bde10 con 0x7fe290137d60 2020-11-20T12:17:58.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.752+0000 7fe28ed9d700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe290137d60 0x7fe2900bd550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fe288006f00 tx=0x7fe288016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:58.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.752+0000 7fe27e7fc700 1 -- 172.21.15.116:0/95371229 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28801c6c0 con 0x7fe290137d60 2020-11-20T12:17:58.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.752+0000 7fe27e7fc700 1 -- 172.21.15.116:0/95371229 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe28801c820 con 0x7fe290137d60 2020-11-20T12:17:58.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.752+0000 7fe28f59e700 1 -- 172.21.15.116:0/95371229 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2900a83f0 con 0x7fe290137d60 2020-11-20T12:17:58.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.752+0000 7fe27e7fc700 1 -- 172.21.15.116:0/95371229 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe28801cbc0 con 0x7fe290137d60 2020-11-20T12:17:58.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.752+0000 7fe28f59e700 1 -- 172.21.15.116:0/95371229 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe290063a10 con 0x7fe290137d60 2020-11-20T12:17:58.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.752+0000 7fe28f59e700 1 -- 172.21.15.116:0/95371229 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe290063c30 con 0x7fe290137d60 2020-11-20T12:17:58.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.752+0000 7fe27e7fc700 1 -- 172.21.15.116:0/95371229 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe288017070 con 0x7fe290137d60 2020-11-20T12:17:58.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.756+0000 7fe27e7fc700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2640557c0 0x7fe264057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:58.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.756+0000 7fe27e7fc700 1 -- 172.21.15.116:0/95371229 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe288070c90 con 0x7fe290137d60 2020-11-20T12:17:58.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.756+0000 7fe28e59c700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2640557c0 0x7fe264057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:58.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.756+0000 7fe28e59c700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2640557c0 0x7fe264057c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fe280006fd0 tx=0x7fe280006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:58.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.760+0000 7fe27e7fc700 1 -- 172.21.15.116:0/95371229 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe288046aa0 con 0x7fe290137d60 2020-11-20T12:17:58.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.920+0000 7fe28f59e700 1 -- 172.21.15.116:0/95371229 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe290044f60 con 0x7fe290137d60 2020-11-20T12:17:58.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.920+0000 7fe27e7fc700 1 -- 172.21.15.116:0/95371229 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe288015070 con 0x7fe290137d60 2020-11-20T12:17:58.922 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:17:58.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 -- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2640557c0 msgr2=0x7fe264057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:58.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2640557c0 0x7fe264057c80 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fe280006fd0 tx=0x7fe280006df0).stop 2020-11-20T12:17:58.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 -- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe290137d60 msgr2=0x7fe2900bd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:58.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe290137d60 0x7fe2900bd550 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fe288006f00 tx=0x7fe288016000).stop 2020-11-20T12:17:58.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 -- 172.21.15.116:0/95371229 shutdown_connections 2020-11-20T12:17:58.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2640557c0 0x7fe264057c80 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe290137d60 0x7fe2900bd550 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2900644e0 0x7fe290063520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 --2- 172.21.15.116:0/95371229 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2900648c0 0x7fe2900bd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:58.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 -- 172.21.15.116:0/95371229 >> 172.21.15.116:0/95371229 conn(0x7fe290061d50 msgr2=0x7fe290060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:58.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 -- 172.21.15.116:0/95371229 shutdown_connections 2020-11-20T12:17:58.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:58.924+0000 7fe28f59e700 1 -- 172.21.15.116:0/95371229 wait complete. 2020-11-20T12:17:58.950 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:17:59.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.128+0000 7fc6c24b0700 1 Processor -- start 2020-11-20T12:17:59.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.128+0000 7fc6c24b0700 1 -- start start 2020-11-20T12:17:59.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.128+0000 7fc6c24b0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc6bc14bf70 0x7fc6bc0c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.128+0000 7fc6c24b0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6bc0c0240 0x7fc6bc14edd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.128+0000 7fc6c24b0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6bc1524b0 con 0x7fc6bc14c350 2020-11-20T12:17:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.128+0000 7fc6c24b0700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6bc0c10e0 con 0x7fc6bc14bf70 2020-11-20T12:17:59.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.128+0000 7fc6c24b0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc152750 con 0x7fc6bc0c0240 2020-11-20T12:17:59.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.128+0000 7fc6bbfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc6bc14bf70 0x7fc6bc0c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.128+0000 7fc6b3fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc6bc14c350 0x7fc6bc062c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55516/0 (socket says 172.21.15.116:55516) 2020-11-20T12:17:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.128+0000 7fc6b3fff700 1 -- 172.21.15.116:0/3696510115 learned_addr learned my addr 172.21.15.116:0/3696510115 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.128+0000 7fc6c0a4d700 1 --2- 172.21.15.116:0/3696510115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6bc0c0240 0x7fc6bc14edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6bbfff700 1 -- 172.21.15.116:0/3696510115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6bc0c0240 msgr2=0x7fc6bc14edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:59.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6bbfff700 1 --2- 172.21.15.116:0/3696510115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6bc0c0240 0x7fc6bc14edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6bbfff700 1 -- 172.21.15.116:0/3696510115 >> v1:172.21.15.116:6789/0 conn(0x7fc6bc14c350 legacy=0x7fc6bc062c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6bbfff700 1 -- 172.21.15.116:0/3696510115 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc1529f0 con 0x7fc6bc14bf70 2020-11-20T12:17:59.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6bb7fe700 1 -- 172.21.15.116:0/3696510115 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183068828 0 0) 0x7fc6bc1524b0 con 0x7fc6bc14c350 2020-11-20T12:17:59.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6bbfff700 1 --2- 172.21.15.116:0/3696510115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6bc14bf70 0x7fc6bc0c0bf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fc6ac00b9a0 tx=0x7fc6ac006df0).ready entity=mon.0 client_cookie=b8bd874a630705b9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:59.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6bb7fe700 1 -- 172.21.15.116:0/3696510115 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac009840 con 0x7fc6bc14bf70 2020-11-20T12:17:59.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6bb7fe700 1 -- 172.21.15.116:0/3696510115 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6ac0099a0 con 0x7fc6bc14bf70 2020-11-20T12:17:59.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6bb7fe700 1 -- 172.21.15.116:0/3696510115 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6ac009d20 con 0x7fc6bc14bf70 2020-11-20T12:17:59.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3696510115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6bc14bf70 msgr2=0x7fc6bc0c0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6c24b0700 1 --2- 172.21.15.116:0/3696510115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6bc14bf70 0x7fc6bc0c0bf0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fc6ac00b9a0 tx=0x7fc6ac006df0).stop 2020-11-20T12:17:59.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3696510115 shutdown_connections 2020-11-20T12:17:59.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6c24b0700 1 --2- 172.21.15.116:0/3696510115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6bc14bf70 0x7fc6bc0c0bf0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6c24b0700 1 --2- 172.21.15.116:0/3696510115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6bc0c0240 0x7fc6bc14edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3696510115 >> 172.21.15.116:0/3696510115 conn(0x7fc6bc060170 msgr2=0x7fc6bc060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:59.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3696510115 shutdown_connections 2020-11-20T12:17:59.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.132+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3696510115 wait complete. 2020-11-20T12:17:59.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6c24b0700 1 Processor -- start 2020-11-20T12:17:59.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6c24b0700 1 -- start start 2020-11-20T12:17:59.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6c24b0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6bc0c0240 0x7fc6bc146610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6c24b0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6bc14bf70 0x7fc6bc161e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6c24b0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6bc14c350 0x7fc6bc15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6c24b0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc14fbb0 con 0x7fc6bc14c350 2020-11-20T12:17:59.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6c24b0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc14fe50 con 0x7fc6bc0c0240 2020-11-20T12:17:59.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6c24b0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6bc1500f0 con 0x7fc6bc14bf70 2020-11-20T12:17:59.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6c0a4d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6bc14c350 0x7fc6bc15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6c0a4d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6bc14c350 0x7fc6bc15b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46086/0 (socket says 172.21.15.116:46086) 2020-11-20T12:17:59.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6c0a4d700 1 -- 172.21.15.116:0/3243589690 learned_addr learned my addr 172.21.15.116:0/3243589690 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:59.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6b3fff700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6bc14bf70 0x7fc6bc161e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.136+0000 7fc6bbfff700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6bc0c0240 0x7fc6bc146610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6c0a4d700 1 -- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6bc14bf70 msgr2=0x7fc6bc161e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6c0a4d700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6bc14bf70 0x7fc6bc161e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6c0a4d700 1 -- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6bc0c0240 msgr2=0x7fc6bc146610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6c0a4d700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6bc0c0240 0x7fc6bc146610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6c0a4d700 1 -- 172.21.15.116:0/3243589690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc1616e0 con 0x7fc6bc14c350 2020-11-20T12:17:59.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6c0a4d700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6bc14c350 0x7fc6bc15b270 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fc6b400ef50 tx=0x7fc6b40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:59.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6b8ff9700 1 -- 172.21.15.116:0/3243589690 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b400ccc0 con 0x7fc6bc14c350 2020-11-20T12:17:59.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6b8ff9700 1 -- 172.21.15.116:0/3243589690 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b400ce20 con 0x7fc6bc14c350 2020-11-20T12:17:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3243589690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6bc161980 con 0x7fc6bc14c350 2020-11-20T12:17:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6b8ff9700 1 -- 172.21.15.116:0/3243589690 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4012620 con 0x7fc6bc14c350 2020-11-20T12:17:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3243589690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6bc146b00 con 0x7fc6bc14c350 2020-11-20T12:17:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3243589690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6bc05ca30 con 0x7fc6bc14c350 2020-11-20T12:17:59.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.140+0000 7fc6b8ff9700 1 -- 172.21.15.116:0/3243589690 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc6b4022070 con 0x7fc6bc14c350 2020-11-20T12:17:59.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.144+0000 7fc6b8ff9700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6900557c0 0x7fc690057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.144+0000 7fc6b8ff9700 1 -- 172.21.15.116:0/3243589690 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc6b406efc0 con 0x7fc6bc14c350 2020-11-20T12:17:59.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.144+0000 7fc6bbfff700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6900557c0 0x7fc690057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.144+0000 7fc6bbfff700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6900557c0 0x7fc690057c80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fc6ac006e30 tx=0x7fc6ac015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:59.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.148+0000 7fc6b8ff9700 1 -- 172.21.15.116:0/3243589690 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc6b4044e60 con 0x7fc6bc14c350 2020-11-20T12:17:59.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.308+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3243589690 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc6bc0bfff0 con 0x7fc6bc14c350 2020-11-20T12:17:59.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.308+0000 7fc6b8ff9700 1 -- 172.21.15.116:0/3243589690 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc6b4016030 con 0x7fc6bc14c350 2020-11-20T12:17:59.308 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:59.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6900557c0 msgr2=0x7fc690057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6900557c0 0x7fc690057c80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fc6ac006e30 tx=0x7fc6ac015000).stop 2020-11-20T12:17:59.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3243589690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6bc14c350 msgr2=0x7fc6bc15b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6bc14c350 0x7fc6bc15b270 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fc6b400ef50 tx=0x7fc6b40091d0).stop 2020-11-20T12:17:59.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3243589690 shutdown_connections 2020-11-20T12:17:59.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6900557c0 0x7fc690057c80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6bc14c350 0x7fc6bc15b270 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6bc0c0240 0x7fc6bc146610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 --2- 172.21.15.116:0/3243589690 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6bc14bf70 0x7fc6bc161e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3243589690 >> 172.21.15.116:0/3243589690 conn(0x7fc6bc060170 msgr2=0x7fc6bc0a5fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:59.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3243589690 shutdown_connections 2020-11-20T12:17:59.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.312+0000 7fc6c24b0700 1 -- 172.21.15.116:0/3243589690 wait complete. 2020-11-20T12:17:59.335 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:17:59.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.512+0000 7f66a359e700 1 Processor -- start 2020-11-20T12:17:59.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.512+0000 7f66a359e700 1 -- start start 2020-11-20T12:17:59.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.516+0000 7f66a359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66a414db60 0x7f66a40c3120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.516+0000 7f66a359e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f66a40a1750 0x7f66a40a3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.516+0000 7f66a359e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a40a1cd0 con 0x7f66a414d780 2020-11-20T12:17:59.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.516+0000 7f66a359e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a40c2740 con 0x7f66a40a1750 2020-11-20T12:17:59.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.516+0000 7f66a359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a40c29e0 con 0x7f66a414db60 2020-11-20T12:17:59.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.516+0000 7f6699d9b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66a414db60 0x7f66a40c3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.516+0000 7f66a259c700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f66a414d780 0x7f66a40a31e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:51756/0 (socket says 172.21.15.116:51756) 2020-11-20T12:17:59.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.516+0000 7f66a259c700 1 -- 172.21.15.116:0/3130633751 learned_addr learned my addr 172.21.15.116:0/3130633751 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:59.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.516+0000 7f66a2d9d700 1 --2- 172.21.15.116:0/3130633751 >> v2:172.21.15.173:3300/0 conn(0x7f66a40a1750 0x7f66a40a3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.516+0000 7f66a1d9b700 1 -- 172.21.15.116:0/3130633751 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1762169950 0 0) 0x7f66a40a1cd0 con 0x7f66a414d780 2020-11-20T12:17:59.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.516+0000 7f66a1d9b700 1 -- 172.21.15.116:0/3130633751 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6684003240 con 0x7f66a414d780 2020-11-20T12:17:59.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f6699d9b700 1 -- 172.21.15.116:0/3130633751 >> v2:172.21.15.173:3300/0 conn(0x7f66a40a1750 msgr2=0x7f66a40a3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:17:59.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f6699d9b700 1 --2- 172.21.15.116:0/3130633751 >> v2:172.21.15.173:3300/0 conn(0x7f66a40a1750 0x7f66a40a3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f6699d9b700 1 -- 172.21.15.116:0/3130633751 >> v1:172.21.15.173:6789/0 conn(0x7f66a414d780 legacy=0x7f66a40a31e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f6699d9b700 1 -- 172.21.15.116:0/3130633751 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a40c2c80 con 0x7f66a414db60 2020-11-20T12:17:59.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a2d9d700 1 --2- 172.21.15.116:0/3130633751 >> v2:172.21.15.173:3300/0 conn(0x7f66a40a1750 0x7f66a40a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:59.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f6699d9b700 1 --2- 172.21.15.116:0/3130633751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66a414db60 0x7f66a40c3120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f668c00b9a0 tx=0x7f668c006df0).ready entity=mon.2 client_cookie=d240a32bda3193a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:59.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a1d9b700 1 -- 172.21.15.116:0/3130633751 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c009710 con 0x7f66a414db60 2020-11-20T12:17:59.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a1d9b700 1 -- 172.21.15.116:0/3130633751 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668c009870 con 0x7f66a414db60 2020-11-20T12:17:59.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a1d9b700 1 -- 172.21.15.116:0/3130633751 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668c009bf0 con 0x7f66a414db60 2020-11-20T12:17:59.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a359e700 1 -- 172.21.15.116:0/3130633751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66a414db60 msgr2=0x7f66a40c3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a359e700 1 --2- 172.21.15.116:0/3130633751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66a414db60 0x7f66a40c3120 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f668c00b9a0 tx=0x7f668c006df0).stop 2020-11-20T12:17:59.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a359e700 1 -- 172.21.15.116:0/3130633751 shutdown_connections 2020-11-20T12:17:59.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a359e700 1 --2- 172.21.15.116:0/3130633751 >> v2:172.21.15.173:3300/0 conn(0x7f66a40a1750 0x7f66a40a3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a359e700 1 --2- 172.21.15.116:0/3130633751 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66a414db60 0x7f66a40c3120 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a359e700 1 -- 172.21.15.116:0/3130633751 >> 172.21.15.116:0/3130633751 conn(0x7f66a4060c30 msgr2=0x7f66a40c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:59.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a359e700 1 -- 172.21.15.116:0/3130633751 shutdown_connections 2020-11-20T12:17:59.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.520+0000 7f66a359e700 1 -- 172.21.15.116:0/3130633751 wait complete. 2020-11-20T12:17:59.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a359e700 1 Processor -- start 2020-11-20T12:17:59.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a359e700 1 -- start start 2020-11-20T12:17:59.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a40a1750 0x7f66a40bef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a414d780 0x7f66a413aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66a414db60 0x7f66a4062d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a4138200 con 0x7f66a40a1750 2020-11-20T12:17:59.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a40bfad0 con 0x7f66a414d780 2020-11-20T12:17:59.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a40bfd70 con 0x7f66a414db60 2020-11-20T12:17:59.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a40a1750 0x7f66a40bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a40a1750 0x7f66a40bef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46098/0 (socket says 172.21.15.116:46098) 2020-11-20T12:17:59.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a259c700 1 -- 172.21.15.116:0/4033099493 learned_addr learned my addr 172.21.15.116:0/4033099493 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:59.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f6699d9b700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a414d780 0x7f66a413aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a2d9d700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66a414db60 0x7f66a4062d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a259c700 1 -- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66a414db60 msgr2=0x7f66a4062d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a259c700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66a414db60 0x7f66a4062d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a259c700 1 -- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a414d780 msgr2=0x7f66a413aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a259c700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a414d780 0x7f66a413aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a259c700 1 -- 172.21.15.116:0/4033099493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a40c0010 con 0x7f66a40a1750 2020-11-20T12:17:59.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.524+0000 7f66a259c700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a40a1750 0x7f66a40bef20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f669400c6f0 tx=0x7f6694007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:59.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.528+0000 7f669affd700 1 -- 172.21.15.116:0/4033099493 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66940072e0 con 0x7f66a40a1750 2020-11-20T12:17:59.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.528+0000 7f669affd700 1 -- 172.21.15.116:0/4033099493 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6694007440 con 0x7f66a40a1750 2020-11-20T12:17:59.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.528+0000 7f66a359e700 1 -- 172.21.15.116:0/4033099493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a40c02b0 con 0x7f66a40a1750 2020-11-20T12:17:59.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.528+0000 7f669affd700 1 -- 172.21.15.116:0/4033099493 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6694005800 con 0x7f66a40a1750 2020-11-20T12:17:59.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.528+0000 7f66a359e700 1 -- 172.21.15.116:0/4033099493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a40bf410 con 0x7f66a40a1750 2020-11-20T12:17:59.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.528+0000 7f66a359e700 1 -- 172.21.15.116:0/4033099493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a40c0440 con 0x7f66a40a1750 2020-11-20T12:17:59.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.528+0000 7f669affd700 1 -- 172.21.15.116:0/4033099493 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6694010070 con 0x7f66a40a1750 2020-11-20T12:17:59.531 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:17:59.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.532+0000 7f669affd700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66800557c0 0x7f6680057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.532+0000 7f669affd700 1 -- 172.21.15.116:0/4033099493 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f669406de70 con 0x7f66a40a1750 2020-11-20T12:17:59.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.532+0000 7f6699d9b700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66800557c0 0x7f6680057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.532+0000 7f6699d9b700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66800557c0 0x7f6680057c80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f668c006e30 tx=0x7f668c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:59.537 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:17:59.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.536+0000 7f669affd700 1 -- 172.21.15.116:0/4033099493 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6694043c00 con 0x7f66a40a1750 2020-11-20T12:17:59.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.696+0000 7f66a359e700 1 -- 172.21.15.116:0/4033099493 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f66a4044f60 con 0x7f66a40a1750 2020-11-20T12:17:59.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.696+0000 7f669affd700 1 -- 172.21.15.116:0/4033099493 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6694047220 con 0x7f66a40a1750 2020-11-20T12:17:59.696 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:17:59.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 -- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66800557c0 msgr2=0x7f6680057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66800557c0 0x7f6680057c80 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f668c006e30 tx=0x7f668c015000).stop 2020-11-20T12:17:59.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 -- 172.21.15.116:0/4033099493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a40a1750 msgr2=0x7f66a40bef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a40a1750 0x7f66a40bef20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f669400c6f0 tx=0x7f6694007ad0).stop 2020-11-20T12:17:59.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 -- 172.21.15.116:0/4033099493 shutdown_connections 2020-11-20T12:17:59.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66800557c0 0x7f6680057c80 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66a414db60 0x7f66a4062d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66a40a1750 0x7f66a40bef20 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 --2- 172.21.15.116:0/4033099493 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66a414d780 0x7f66a413aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 -- 172.21.15.116:0/4033099493 >> 172.21.15.116:0/4033099493 conn(0x7f66a4060c30 msgr2=0x7f66a4090a40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:59.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 -- 172.21.15.116:0/4033099493 shutdown_connections 2020-11-20T12:17:59.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.700+0000 7f66a359e700 1 -- 172.21.15.116:0/4033099493 wait complete. 2020-11-20T12:17:59.727 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:17:59.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.908+0000 7f27fc83a700 1 Processor -- start 2020-11-20T12:17:59.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.908+0000 7f27fc83a700 1 -- start start 2020-11-20T12:17:59.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.908+0000 7f27fc83a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f27f80a12e0 0x7f27f8149240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.908+0000 7f27fc83a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f80a50a0 con 0x7f27f80a5fc0 2020-11-20T12:17:59.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.908+0000 7f27fc83a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f80a5340 con 0x7f27f80a0f00 2020-11-20T12:17:59.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.908+0000 7f27fc83a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f80a55e0 con 0x7f27f80a12e0 2020-11-20T12:17:59.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f659c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f27f80a5fc0 0x7f27f80c2df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55542/0 (socket says 172.21.15.116:55542) 2020-11-20T12:17:59.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f659c700 1 -- 172.21.15.116:0/3239297405 learned_addr learned my addr 172.21.15.116:0/3239297405 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:17:59.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f759e700 1 --2- 172.21.15.116:0/3239297405 >> v2:172.21.15.173:3300/0 conn(0x7f27f80a12e0 0x7f27f8149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f5d9b700 1 -- 172.21.15.116:0/3239297405 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420186034 0 0) 0x7f27f80a50a0 con 0x7f27f80a5fc0 2020-11-20T12:17:59.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f5d9b700 1 -- 172.21.15.116:0/3239297405 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27e4003240 con 0x7f27f80a5fc0 2020-11-20T12:17:59.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f759e700 1 -- 172.21.15.116:0/3239297405 >> v1:172.21.15.173:6789/0 conn(0x7f27f80a0f00 legacy=0x7f27f8150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f759e700 1 -- 172.21.15.116:0/3239297405 >> v1:172.21.15.116:6789/0 conn(0x7f27f80a5fc0 legacy=0x7f27f80c2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f759e700 1 -- 172.21.15.116:0/3239297405 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f80a5880 con 0x7f27f80a12e0 2020-11-20T12:17:59.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f759e700 1 --2- 172.21.15.116:0/3239297405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27f80a12e0 0x7f27f8149240 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f27f0009c90 tx=0x7f27f00091d0).ready entity=mon.1 client_cookie=e613b5c717bb1ea9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:59.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f5d9b700 1 -- 172.21.15.116:0/3239297405 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f000ba70 con 0x7f27f80a12e0 2020-11-20T12:17:59.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f5d9b700 1 -- 172.21.15.116:0/3239297405 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f000bbd0 con 0x7f27f80a12e0 2020-11-20T12:17:59.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27f5d9b700 1 -- 172.21.15.116:0/3239297405 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0011400 con 0x7f27f80a12e0 2020-11-20T12:17:59.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27fc83a700 1 -- 172.21.15.116:0/3239297405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27f80a12e0 msgr2=0x7f27f8149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27fc83a700 1 --2- 172.21.15.116:0/3239297405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27f80a12e0 0x7f27f8149240 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f27f0009c90 tx=0x7f27f00091d0).stop 2020-11-20T12:17:59.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27fc83a700 1 -- 172.21.15.116:0/3239297405 shutdown_connections 2020-11-20T12:17:59.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27fc83a700 1 --2- 172.21.15.116:0/3239297405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27f80a12e0 0x7f27f8149240 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27fc83a700 1 -- 172.21.15.116:0/3239297405 >> 172.21.15.116:0/3239297405 conn(0x7f27f8060c30 msgr2=0x7f27f80c3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:17:59.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27fc83a700 1 -- 172.21.15.116:0/3239297405 shutdown_connections 2020-11-20T12:17:59.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.912+0000 7f27fc83a700 1 -- 172.21.15.116:0/3239297405 wait complete. 2020-11-20T12:17:59.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.916+0000 7f27fc83a700 1 Processor -- start 2020-11-20T12:17:59.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.916+0000 7f27fc83a700 1 -- start start 2020-11-20T12:17:59.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27fc83a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27f80a0f00 0x7f27f8090830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27fc83a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27f80a12e0 0x7f27f8067040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27fc83a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27f80a5fc0 0x7f27f8067530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27fc83a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f813d550 con 0x7f27f80a12e0 2020-11-20T12:17:59.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27fc83a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8090d20 con 0x7f27f80a5fc0 2020-11-20T12:17:59.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27fc83a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f8090fb0 con 0x7f27f80a0f00 2020-11-20T12:17:59.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27f80a12e0 0x7f27f8067040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27f80a12e0 0x7f27f8067040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46114/0 (socket says 172.21.15.116:46114) 2020-11-20T12:17:59.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f659c700 1 -- 172.21.15.116:0/1792089741 learned_addr learned my addr 172.21.15.116:0/1792089741 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:17:59.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f759e700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27f80a5fc0 0x7f27f8067530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f6d9d700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27f80a0f00 0x7f27f8090830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f659c700 1 -- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27f80a0f00 msgr2=0x7f27f8090830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f659c700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27f80a0f00 0x7f27f8090830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f659c700 1 -- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27f80a5fc0 msgr2=0x7f27f8067530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:17:59.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f659c700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27f80a5fc0 0x7f27f8067530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:17:59.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f659c700 1 -- 172.21.15.116:0/1792089741 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f8091250 con 0x7f27f80a12e0 2020-11-20T12:17:59.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f6d9d700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27f80a0f00 0x7f27f8090830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:17:59.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f759e700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27f80a5fc0 0x7f27f8067530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:17:59.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27f659c700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27f80a12e0 0x7f27f8067040 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f27ec00c710 tx=0x7f27ec007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:59.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27deffd700 1 -- 172.21.15.116:0/1792089741 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0072e0 con 0x7f27f80a12e0 2020-11-20T12:17:59.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27deffd700 1 -- 172.21.15.116:0/1792089741 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ec007440 con 0x7f27f80a12e0 2020-11-20T12:17:59.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27fc83a700 1 -- 172.21.15.116:0/1792089741 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f80914f0 con 0x7f27f80a12e0 2020-11-20T12:17:59.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27deffd700 1 -- 172.21.15.116:0/1792089741 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec005800 con 0x7f27f80a12e0 2020-11-20T12:17:59.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.920+0000 7f27fc83a700 1 -- 172.21.15.116:0/1792089741 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f80bd320 con 0x7f27f80a12e0 2020-11-20T12:17:59.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.924+0000 7f27fc83a700 1 -- 172.21.15.116:0/1792089741 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f8044f60 con 0x7f27f80a12e0 2020-11-20T12:17:59.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.924+0000 7f27deffd700 1 -- 172.21.15.116:0/1792089741 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f27ec010070 con 0x7f27f80a12e0 2020-11-20T12:17:59.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.924+0000 7f27deffd700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f27d40557c0 0x7f27d4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:17:59.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.924+0000 7f27deffd700 1 -- 172.21.15.116:0/1792089741 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f27ec06dd70 con 0x7f27f80a12e0 2020-11-20T12:17:59.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.924+0000 7f27f6d9d700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f27d40557c0 0x7f27d4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:17:59.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.928+0000 7f27f6d9d700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f27d40557c0 0x7f27d4057c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f27e8001570 tx=0x7f27e8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:17:59.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:17:59.928+0000 7f27deffd700 1 -- 172.21.15.116:0/1792089741 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f27ec043b00 con 0x7f27f80a12e0 2020-11-20T12:18:00.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.084+0000 7f27fc83a700 1 -- 172.21.15.116:0/1792089741 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27f8064080 con 0x7f27f80a12e0 2020-11-20T12:18:00.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27deffd700 1 -- 172.21.15.116:0/1792089741 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7f27ec047120 con 0x7f27f80a12e0 2020-11-20T12:18:00.087 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:00.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 -- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f27d40557c0 msgr2=0x7f27d4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f27d40557c0 0x7f27d4057c80 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f27e8001570 tx=0x7f27e8009040).stop 2020-11-20T12:18:00.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 -- 172.21.15.116:0/1792089741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27f80a12e0 msgr2=0x7f27f8067040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27f80a12e0 0x7f27f8067040 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f27ec00c710 tx=0x7f27ec007aa0).stop 2020-11-20T12:18:00.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 -- 172.21.15.116:0/1792089741 shutdown_connections 2020-11-20T12:18:00.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f27d40557c0 0x7f27d4057c80 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f27f80a5fc0 0x7f27f8067530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f27f80a0f00 0x7f27f8090830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 --2- 172.21.15.116:0/1792089741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f27f80a12e0 0x7f27f8067040 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 -- 172.21.15.116:0/1792089741 >> 172.21.15.116:0/1792089741 conn(0x7f27f8060c30 msgr2=0x7f27f814c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:00.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 -- 172.21.15.116:0/1792089741 shutdown_connections 2020-11-20T12:18:00.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.088+0000 7f27fc83a700 1 -- 172.21.15.116:0/1792089741 wait complete. 2020-11-20T12:18:00.111 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:18:00.111 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:18:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f5236a02700 1 Processor -- start 2020-11-20T12:18:00.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f5236a02700 1 -- start start 2020-11-20T12:18:00.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f5236a02700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f52300a7e40 0x7f5230064960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f5236a02700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5230062c50 0x7f52300a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f5236a02700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5230064e50 con 0x7f5230064580 2020-11-20T12:18:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f5236a02700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52301551f0 con 0x7f52300a7e40 2020-11-20T12:18:00.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f5236a02700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52300a8430 con 0x7f5230062c50 2020-11-20T12:18:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f52351ff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f52300a7e40 0x7f5230064960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:00.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f52351ff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f52300a7e40 0x7f5230064960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46124/0 (socket says 172.21.15.116:46124) 2020-11-20T12:18:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f52351ff700 1 -- 172.21.15.116:0/2269580862 learned_addr learned my addr 172.21.15.116:0/2269580862 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f5235a00700 1 --1- 172.21.15.116:0/2269580862 >> v1:172.21.15.173:6789/0 conn(0x7f5230064580 0x7f5230151c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:51784/0 (socket says 172.21.15.116:51784) 2020-11-20T12:18:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f5236201700 1 --2- 172.21.15.116:0/2269580862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5230062c50 0x7f52300a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f52351ff700 1 -- 172.21.15.116:0/2269580862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5230062c50 msgr2=0x7f52300a62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:00.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f52351ff700 1 --2- 172.21.15.116:0/2269580862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5230062c50 0x7f52300a62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f52351ff700 1 -- 172.21.15.116:0/2269580862 >> v1:172.21.15.173:6789/0 conn(0x7f5230064580 legacy=0x7f5230151c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.276+0000 7f52351ff700 1 -- 172.21.15.116:0/2269580862 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300a86d0 con 0x7f52300a7e40 2020-11-20T12:18:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f52351ff700 1 --2- 172.21.15.116:0/2269580862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52300a7e40 0x7f5230064960 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f5220006fd0 tx=0x7f5220006eb0).ready entity=mon.0 client_cookie=605d9e2542fe50c9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:00.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f52349fe700 1 -- 172.21.15.116:0/2269580862 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52200098a0 con 0x7f52300a7e40 2020-11-20T12:18:00.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f52349fe700 1 -- 172.21.15.116:0/2269580862 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5220009a00 con 0x7f52300a7e40 2020-11-20T12:18:00.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f52349fe700 1 -- 172.21.15.116:0/2269580862 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220009d80 con 0x7f52300a7e40 2020-11-20T12:18:00.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f5236a02700 1 -- 172.21.15.116:0/2269580862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52300a7e40 msgr2=0x7f5230064960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f5236a02700 1 --2- 172.21.15.116:0/2269580862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52300a7e40 0x7f5230064960 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f5220006fd0 tx=0x7f5220006eb0).stop 2020-11-20T12:18:00.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f5236a02700 1 -- 172.21.15.116:0/2269580862 shutdown_connections 2020-11-20T12:18:00.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f5236a02700 1 --2- 172.21.15.116:0/2269580862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5230062c50 0x7f52300a62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f5236a02700 1 --2- 172.21.15.116:0/2269580862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52300a7e40 0x7f5230064960 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f5236a02700 1 -- 172.21.15.116:0/2269580862 >> 172.21.15.116:0/2269580862 conn(0x7f5230060c40 msgr2=0x7f52300c62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:00.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f5236a02700 1 -- 172.21.15.116:0/2269580862 shutdown_connections 2020-11-20T12:18:00.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.280+0000 7f5236a02700 1 -- 172.21.15.116:0/2269580862 wait complete. 2020-11-20T12:18:00.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.284+0000 7f5236a02700 1 Processor -- start 2020-11-20T12:18:00.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.284+0000 7f5236a02700 1 -- start start 2020-11-20T12:18:00.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236a02700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5230062c50 0x7f5230150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236a02700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5230064580 0x7f52300c2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236a02700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52301513e0 0x7f52300c2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236a02700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5230157100 con 0x7f52301513e0 2020-11-20T12:18:00.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236a02700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523013a500 con 0x7f5230064580 2020-11-20T12:18:00.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236a02700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52300c3490 con 0x7f5230062c50 2020-11-20T12:18:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236201700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52301513e0 0x7f52300c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236201700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52301513e0 0x7f52300c2540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46130/0 (socket says 172.21.15.116:46130) 2020-11-20T12:18:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236201700 1 -- 172.21.15.116:0/836457971 learned_addr learned my addr 172.21.15.116:0/836457971 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5235a00700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5230062c50 0x7f5230150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:00.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f52351ff700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5230064580 0x7f52300c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236201700 1 -- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5230062c50 msgr2=0x7f5230150ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236201700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5230062c50 0x7f5230150ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236201700 1 -- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5230064580 msgr2=0x7f52300c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236201700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5230064580 0x7f52300c2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236201700 1 -- 172.21.15.116:0/836457971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300c2a30 con 0x7f52301513e0 2020-11-20T12:18:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236201700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52301513e0 0x7f52300c2540 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f522800ef50 tx=0x7f52280091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f521dffb700 1 -- 172.21.15.116:0/836457971 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522800ccc0 con 0x7f52301513e0 2020-11-20T12:18:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f521dffb700 1 -- 172.21.15.116:0/836457971 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522800ce20 con 0x7f52301513e0 2020-11-20T12:18:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236a02700 1 -- 172.21.15.116:0/836457971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52300c2cb0 con 0x7f52301513e0 2020-11-20T12:18:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f521dffb700 1 -- 172.21.15.116:0/836457971 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228012620 con 0x7f52301513e0 2020-11-20T12:18:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.288+0000 7f5236a02700 1 -- 172.21.15.116:0/836457971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52300c31f0 con 0x7f52301513e0 2020-11-20T12:18:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.292+0000 7f5236a02700 1 -- 172.21.15.116:0/836457971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52300ac180 con 0x7f52301513e0 2020-11-20T12:18:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.292+0000 7f521dffb700 1 -- 172.21.15.116:0/836457971 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5228022070 con 0x7f52301513e0 2020-11-20T12:18:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.292+0000 7f521dffb700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 0x7f5208057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.292+0000 7f521dffb700 1 -- 172.21.15.116:0/836457971 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f522806ef60 con 0x7f52301513e0 2020-11-20T12:18:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.292+0000 7f5235a00700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 0x7f5208057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:00.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.292+0000 7f5235a00700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 0x7f5208057c80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f522c007f70 tx=0x7f522c000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:00.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.300+0000 7f521dffb700 1 -- 172.21.15.116:0/836457971 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f52300ac180 con 0x7f52301513e0 2020-11-20T12:18:00.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.508+0000 7f5236a02700 1 -- 172.21.15.116:0/836457971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52300ac180 con 0x7f52301513e0 2020-11-20T12:18:00.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.508+0000 7f521dffb700 1 -- 172.21.15.116:0/836457971 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+2849 (secure 0 0 0) 0x7f52300ac180 con 0x7f52301513e0 2020-11-20T12:18:00.509 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:00.510 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1096724826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1096724826},{"type":"v1","addr":"172.21.15.173:6837","nonce":1096724826}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/1589443098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1589443098},{"type":"v1","addr":"172.21.15.173:6835","nonce":1589443098}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/966300328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":966300328},{"type":"v1","addr":"172.21.15.116:6833","nonce":966300328}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:17:40.163408+0000","modified":"2020-11-20T12:17:42.171423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5493":{"gid":5493,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/2908037349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2908037349},{"type":"v1","addr":"172.21.15.116:6835","nonce":2908037349}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-20T12:18:00.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 -- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 msgr2=0x7f5208057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 0x7f5208057c80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f522c007f70 tx=0x7f522c000b80).stop 2020-11-20T12:18:00.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 -- 172.21.15.116:0/836457971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52301513e0 msgr2=0x7f52300c2540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:00.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52301513e0 0x7f52300c2540 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f522800ef50 tx=0x7f52280091d0).stop 2020-11-20T12:18:00.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 -- 172.21.15.116:0/836457971 shutdown_connections 2020-11-20T12:18:00.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 0x7f5208057c80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52301513e0 0x7f52300c2540 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5230062c50 0x7f5230150ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 --2- 172.21.15.116:0/836457971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5230064580 0x7f52300c2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:00.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 -- 172.21.15.116:0/836457971 >> 172.21.15.116:0/836457971 conn(0x7f5230060c40 msgr2=0x7f52300c4210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:00.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 -- 172.21.15.116:0/836457971 shutdown_connections 2020-11-20T12:18:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:00.512+0000 7f5236a02700 1 -- 172.21.15.116:0/836457971 wait complete. 2020-11-20T12:18:00.516 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 20 2020-11-20T12:18:00.942 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:17:40.163408+0000', 'modified': '2020-11-20T12:17:42.171423+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5493': {'gid': 5493, 'name': 'a', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/2908037349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2908037349}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 2908037349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:18:00.942 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:18:00.943 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:18:01.137 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":5493,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.954669827,"mdsmap_epoch":20,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":26.472311982000001} 2020-11-20T12:18:01.137 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 5493, "mdsmap_epoch": 20, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 20.954669827, "state": "up:active", "uptime": 26.472311982, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:18:01.138 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:18:01.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:01.139 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.3 2020-11-20T12:18:01.156 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:18:01.157 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:01.229 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:01.229 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:18:01.254 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:01.255 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:18:01.260 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:18:01.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:01.261 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2020-11-20T12:18:01.289 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-20T12:18:01.289 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:18:01.290 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 2020-11-20T12:18:01.410 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:18:01.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:01.412 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T12:18:01.428 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.manila mds 'allow *' osd 'allow rw' mon 'allow *' 2020-11-20T12:18:01.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.624+0000 7eff6bc2b700 1 Processor -- start 2020-11-20T12:18:01.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.624+0000 7eff6bc2b700 1 -- start start 2020-11-20T12:18:01.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.628+0000 7eff6bc2b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7eff640a7770 0x7eff640a7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:01.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.628+0000 7eff6bc2b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff64152b00 0x7eff640c5290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:01.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.628+0000 7eff6bc2b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff64156270 con 0x7eff640c5780 2020-11-20T12:18:01.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.628+0000 7eff6bc2b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff640c5c10 con 0x7eff640a7770 2020-11-20T12:18:01.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.628+0000 7eff6bc2b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff64156510 con 0x7eff64152b00 2020-11-20T12:18:01.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.628+0000 7eff6a1c8700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7eff640c5780 0x7eff64155680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55572/0 (socket says 172.21.15.116:55572) 2020-11-20T12:18:01.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.628+0000 7eff6a1c8700 1 -- 172.21.15.116:0/2548828943 learned_addr learned my addr 172.21.15.116:0/2548828943 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:01.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.628+0000 7eff699c7700 1 --2- 172.21.15.116:0/2548828943 >> v2:172.21.15.173:3300/0 conn(0x7eff640a7770 0x7eff640a7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:01.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.628+0000 7eff691c6700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff64152b00 0x7eff640c5290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:01.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.628+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334578471 0 0) 0x7eff64156270 con 0x7eff640c5780 2020-11-20T12:18:01.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.628+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff4c003240 con 0x7eff640c5780 2020-11-20T12:18:01.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 132437895 0 0) 0x7eff4c003240 con 0x7eff640c5780 2020-11-20T12:18:01.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff64152b00 msgr2=0x7eff640c5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:01.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff64152b00 0x7eff640c5290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:01.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 >> v2:172.21.15.173:3300/0 conn(0x7eff640a7770 msgr2=0x7eff640a7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:01.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 --2- 172.21.15.116:0/2548828943 >> v2:172.21.15.173:3300/0 conn(0x7eff640a7770 0x7eff640a7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:01.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff641523c0 con 0x7eff640c5780 2020-11-20T12:18:01.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7eff5c001860 con 0x7eff640c5780 2020-11-20T12:18:01.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 >> v1:172.21.15.116:6789/0 conn(0x7eff640c5780 legacy=0x7eff64155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:01.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff4c0037b0 0x7eff4c003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:01.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff64152b00 0x7eff4c0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:01.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff4c0049d0 0x7eff4c004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:01.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff640c5c10 con 0x7eff4c0049d0 2020-11-20T12:18:01.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff64156510 con 0x7eff4c0037b0 2020-11-20T12:18:01.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff4c0042a0 con 0x7eff64152b00 2020-11-20T12:18:01.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff699c7700 1 --2- 172.21.15.116:0/2548828943 >> v2:172.21.15.173:3300/0 conn(0x7eff640a7770 0x7eff640a7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:01.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff6a1c8700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff4c0049d0 0x7eff4c004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:01.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff691c6700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff64152b00 0x7eff4c0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:01.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff699c7700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff4c0037b0 0x7eff4c003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:01.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff6a1c8700 1 -- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff64152b00 msgr2=0x7eff4c0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:01.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff6a1c8700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff64152b00 0x7eff4c0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:01.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff6a1c8700 1 -- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff4c0037b0 msgr2=0x7eff4c003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:01.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff6a1c8700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff4c0037b0 0x7eff4c003bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff6a1c8700 1 -- 172.21.15.116:0/2548828943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff4c003240 con 0x7eff4c0049d0 2020-11-20T12:18:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff699c7700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff4c0037b0 0x7eff4c003bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff6a1c8700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff4c0049d0 0x7eff4c004e50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7eff5c00c600 tx=0x7eff5c007ad0).ready entity=mon.0 client_cookie=b2568744d43e2d11 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:01.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c005b80 con 0x7eff4c0049d0 2020-11-20T12:18:01.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c005ce0 con 0x7eff4c0049d0 2020-11-20T12:18:01.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.632+0000 7eff689c5700 1 -- 172.21.15.116:0/2548828943 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c007320 con 0x7eff4c0049d0 2020-11-20T12:18:01.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.636+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2548828943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff4c0049d0 msgr2=0x7eff4c004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:01.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.636+0000 7eff6bc2b700 1 --2- 172.21.15.116:0/2548828943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff4c0049d0 0x7eff4c004e50 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7eff5c00c600 tx=0x7eff5c007ad0).stop 2020-11-20T12:18:01.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.636+0000 7eff6a1c8700 1 -- 172.21.15.116:0/2548828943 reap_dead start 2020-11-20T12:18:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.636+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2548828943 shutdown_connections 2020-11-20T12:18:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.636+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2548828943 >> 172.21.15.116:0/2548828943 conn(0x7eff640678a0 msgr2=0x7eff64067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:01.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.636+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2548828943 shutdown_connections 2020-11-20T12:18:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.636+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2548828943 wait complete. 2020-11-20T12:18:01.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.636+0000 7eff6bc2b700 1 Processor -- start 2020-11-20T12:18:01.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.636+0000 7eff6bc2b700 1 -- start start 2020-11-20T12:18:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff6bc2b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff64153080 0x7eff64153460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:01.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff6bc2b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff64161350 0x7eff640c0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff6bc2b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff640c7420 0x7eff640c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff6bc2b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff64161820 con 0x7eff64161350 2020-11-20T12:18:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff6bc2b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff64161ac0 con 0x7eff64153080 2020-11-20T12:18:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff6bc2b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff64161d60 con 0x7eff640c7420 2020-11-20T12:18:01.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff691c6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff64161350 0x7eff640c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff691c6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff64161350 0x7eff640c0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46148/0 (socket says 172.21.15.116:46148) 2020-11-20T12:18:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff691c6700 1 -- 172.21.15.116:0/2923715556 learned_addr learned my addr 172.21.15.116:0/2923715556 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff6a1c8700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff640c7420 0x7eff640c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:01.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff699c7700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff64153080 0x7eff64153460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff691c6700 1 -- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff640c7420 msgr2=0x7eff640c0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff691c6700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff640c7420 0x7eff640c0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff691c6700 1 -- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff64153080 msgr2=0x7eff64153460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff691c6700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff64153080 0x7eff64153460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:01.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff691c6700 1 -- 172.21.15.116:0/2923715556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff64162000 con 0x7eff64161350 2020-11-20T12:18:01.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff6a1c8700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff640c7420 0x7eff640c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:01.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff691c6700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff64161350 0x7eff640c0150 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7eff54002f00 tx=0x7eff54008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:01.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff51ffb700 1 -- 172.21.15.116:0/2923715556 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54011990 con 0x7eff64161350 2020-11-20T12:18:01.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff51ffb700 1 -- 172.21.15.116:0/2923715556 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff54011af0 con 0x7eff64161350 2020-11-20T12:18:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2923715556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff641622a0 con 0x7eff64161350 2020-11-20T12:18:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.640+0000 7eff51ffb700 1 -- 172.21.15.116:0/2923715556 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54011e70 con 0x7eff64161350 2020-11-20T12:18:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.644+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2923715556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff640c7a20 con 0x7eff64161350 2020-11-20T12:18:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.644+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2923715556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff640be090 con 0x7eff64161350 2020-11-20T12:18:01.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.644+0000 7eff51ffb700 1 -- 172.21.15.116:0/2923715556 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7eff54024070 con 0x7eff64161350 2020-11-20T12:18:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.648+0000 7eff51ffb700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff3c0557c0 0x7eff3c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.648+0000 7eff51ffb700 1 -- 172.21.15.116:0/2923715556 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7eff5406f190 con 0x7eff64161350 2020-11-20T12:18:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.648+0000 7eff699c7700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff3c0557c0 0x7eff3c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:01.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.648+0000 7eff699c7700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff3c0557c0 0x7eff3c057c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7eff60006dc0 tx=0x7eff60008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:01.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.652+0000 7eff51ffb700 1 -- 172.21.15.116:0/2923715556 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7eff54044f20 con 0x7eff64161350 2020-11-20T12:18:01.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.868+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2923715556 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]} v 0) v1 -- 0x7eff6405ca30 con 0x7eff64161350 2020-11-20T12:18:01.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.876+0000 7eff51ffb700 1 -- 172.21.15.116:0/2923715556 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]}]=0 v11) v1 ==== 158+0+64 (secure 0 0 0) 0x7eff54018070 con 0x7eff64161350 2020-11-20T12:18:01.875 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:01.875 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQD5s7dfn0HnMxAAXUDbTOIUNzTn4X3w7Zfkpw== 2020-11-20T12:18:01.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff3c0557c0 msgr2=0x7eff3c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:01.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff3c0557c0 0x7eff3c057c80 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7eff60006dc0 tx=0x7eff60008040).stop 2020-11-20T12:18:01.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2923715556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff64161350 msgr2=0x7eff640c0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:01.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff64161350 0x7eff640c0150 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7eff54002f00 tx=0x7eff54008040).stop 2020-11-20T12:18:01.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2923715556 shutdown_connections 2020-11-20T12:18:01.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff3c0557c0 0x7eff3c057c80 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:01.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff640c7420 0x7eff640c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:01.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff64153080 0x7eff64153460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:01.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 --2- 172.21.15.116:0/2923715556 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff64161350 0x7eff640c0150 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:01.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2923715556 >> 172.21.15.116:0/2923715556 conn(0x7eff640678a0 msgr2=0x7eff64154b60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:01.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2923715556 shutdown_connections 2020-11-20T12:18:01.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:01.880+0000 7eff6bc2b700 1 -- 172.21.15.116:0/2923715556 wait complete. 2020-11-20T12:18:01.903 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:18:01.904 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:01.916 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:01.931 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:18:01.931 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:18:01.931 INFO:teuthology.orchestra.run.smithi173.stderr:64 bytes copied, 0.000287527 s, 223 kB/s 2020-11-20T12:18:01.933 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:01.940 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:02.002 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:02.002 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:02.002 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:02.003 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:02.003 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.003 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:02.003 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.003 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:02.004 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.004 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:02.004 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:02.004 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.004 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:02.005 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:02.005 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.005 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:02.005 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:02.006 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.006 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:02.006 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:02.006 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.006 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:02.007 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.007 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:02.007 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:02.007 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:02.008 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:02.008 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:02.008 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:02.008 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:02.008 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:02.009 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:02.009 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:02.009 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.009 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:02.009 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.010 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:02.011 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.011 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:02.011 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.011 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:02.012 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:02.012 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:02.012 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:02.012 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:02.012 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:02.013 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.013 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:02.013 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:02.013 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.013 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:02.014 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:02.014 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.014 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:02.014 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:02.015 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.015 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:02.015 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:02.015 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:02.015 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.016 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:02.016 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:02.016 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.016 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:02.016 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.017 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:02.017 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.017 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:02.017 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.017 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:02.018 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:02.018 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.018 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:02.018 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:02.018 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:02.019 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:02.019 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:02.019 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:02.019 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:02.019 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:02.020 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:02.020 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:02.020 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:02.020 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:02.020 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:02.021 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:02.021 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:02.021 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:02.021 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:02.022 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:02.022 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:02.022 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:02.022 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.022 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:02.023 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:02.023 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:02.023 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:02.023 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:02.023 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:02.024 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:02.024 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:02.024 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.024 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:02.024 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:02.025 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:02.025 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:02.025 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:02.025 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:02.025 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:02.026 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:02.026 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:02.026 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:02.026 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:02.026 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:02.027 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:02.027 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:02.027 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:02.027 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:02.027 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:02.028 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:02.029 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:02.029 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:02.029 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:02.029 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:02.030 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:02.030 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:02.030 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:02.030 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:02.030 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:02.031 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:02.031 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:02.031 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:02.031 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:02.031 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:02.032 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:02.032 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:02.032 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:02.032 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:02.032 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:02.033 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:02.033 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:02.033 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:02.033 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:02.033 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:02.034 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:02.034 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:02.034 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:02.035 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:02.035 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:02.035 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:02.035 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:02.054 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:02.055 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:02.055 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:02.055 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:02.055 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.055 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:02.056 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.056 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:02.056 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.056 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:02.057 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:02.057 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.057 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:02.057 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:02.057 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.058 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:02.058 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:02.058 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.058 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:02.059 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:02.059 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.059 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:02.059 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.059 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:02.060 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:02.061 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:02.061 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:02.061 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:02.062 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:02.063 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:02.063 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:02.064 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:02.064 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:02.064 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.064 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:02.064 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.065 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:02.065 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.065 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:02.065 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.065 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:02.066 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:02.066 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:02.066 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:02.066 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:02.066 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:02.067 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.067 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:02.067 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:02.067 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.067 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:02.068 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:02.068 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.068 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:02.068 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:02.068 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.069 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:02.069 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:02.069 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:02.069 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.070 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:02.070 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:02.070 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.070 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:02.070 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.071 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:02.071 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.071 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:02.071 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.071 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:02.072 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:02.072 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.072 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:02.072 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:02.072 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:02.073 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:02.073 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:02.073 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:02.073 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:02.073 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:02.074 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:02.074 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:02.074 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:02.074 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:02.074 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:02.075 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:02.075 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:02.075 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:02.075 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:02.075 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:02.076 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:02.076 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:02.076 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.076 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:02.076 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:02.077 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:02.077 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:02.077 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:02.077 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:02.078 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:02.078 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:02.078 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.078 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:02.079 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:02.079 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:02.079 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:02.079 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:02.079 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:02.080 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:02.080 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:02.080 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:02.080 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:02.080 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:02.081 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:02.081 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:02.081 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:02.081 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:02.082 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:02.082 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:02.082 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:02.082 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:02.082 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:02.083 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:02.083 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:02.083 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:02.083 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:02.083 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:02.084 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:02.084 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:02.084 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:02.084 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:02.085 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:02.085 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:02.085 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:02.085 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:02.085 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:02.086 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:02.086 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:02.086 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:02.086 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:02.086 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:02.087 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:02.087 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:02.087 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:02.087 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:02.087 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:02.088 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:02.088 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:02.088 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:02.088 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:02.088 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:02.090 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:18:02.090 INFO:teuthology.orchestra.run.smithi173:> from __future__ import print_function 2020-11-20T12:18:02.091 INFO:teuthology.orchestra.run.smithi173:> from ceph_volume_client import CephFSVolumeClient, VolumePath 2020-11-20T12:18:02.091 INFO:teuthology.orchestra.run.smithi173:> from sys import version_info as sys_version_info 2020-11-20T12:18:02.091 INFO:teuthology.orchestra.run.smithi173:> from rados import OSError as rados_OSError 2020-11-20T12:18:02.092 INFO:teuthology.orchestra.run.smithi173:> import logging 2020-11-20T12:18:02.092 INFO:teuthology.orchestra.run.smithi173:> log = logging.getLogger("ceph_volume_client") 2020-11-20T12:18:02.092 INFO:teuthology.orchestra.run.smithi173:> log.addHandler(logging.StreamHandler()) 2020-11-20T12:18:02.093 INFO:teuthology.orchestra.run.smithi173:> log.setLevel(logging.DEBUG) 2020-11-20T12:18:02.093 INFO:teuthology.orchestra.run.smithi173:> vc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None) 2020-11-20T12:18:02.093 INFO:teuthology.orchestra.run.smithi173:> vc.connect() 2020-11-20T12:18:02.094 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:18:02.094 INFO:teuthology.orchestra.run.smithi173:> vp = VolumePath("grpid", "volid") 2020-11-20T12:18:02.094 INFO:teuthology.orchestra.run.smithi173:> create_result = vc.create_volume(vp, 1024*1024*10) 2020-11-20T12:18:02.095 INFO:teuthology.orchestra.run.smithi173:> print(create_result['"'"'mount_path'"'"']) 2020-11-20T12:18:02.095 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:18:02.096 INFO:teuthology.orchestra.run.smithi173:> vc.disconnect() 2020-11-20T12:18:02.096 INFO:teuthology.orchestra.run.smithi173:> ' 2020-11-20T12:18:02.300 INFO:teuthology.orchestra.run.smithi173.stderr:Configuring to RADOS with config /etc/ceph/ceph.conf... 2020-11-20T12:18:02.310 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to RADOS... 2020-11-20T12:18:02.319 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to RADOS complete 2020-11-20T12:18:02.320 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to cephfs... 2020-11-20T12:18:02.320 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS initializing... 2020-11-20T12:18:02.326 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS mounting... 2020-11-20T12:18:02.337 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to cephfs complete 2020-11-20T12:18:02.338 INFO:teuthology.orchestra.run.smithi173.stderr:Recovering from partial auth updates (if any)... 2020-11-20T12:18:02.338 INFO:teuthology.orchestra.run.smithi173.stderr:Nothing to recover. No auth meta files. 2020-11-20T12:18:02.338 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: /volumes/grpid/volid 2020-11-20T12:18:02.361 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: grpid/volid, using rados namespace fsvolumens_volid to isolate data. 2020-11-20T12:18:02.370 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:18:02.370 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs... 2020-11-20T12:18:02.383 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs complete 2020-11-20T12:18:02.383 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados... 2020-11-20T12:18:02.386 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados complete 2020-11-20T12:18:02.387 INFO:teuthology.orchestra.run.smithi173.stdout:/volumes/grpid/volid 2020-11-20T12:18:02.391 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:18:02.597 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.21501 2020-11-20T12:18:02.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.776+0000 7f6370aa7700 1 Processor -- start 2020-11-20T12:18:02.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.776+0000 7f6370aa7700 1 -- start start 2020-11-20T12:18:02.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f6370aa7700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f636c0627c0 0x7f636c062ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f6370aa7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f636c0c14e0 0x7f636c066ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f6370aa7700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f636c151880 con 0x7f636c14d660 2020-11-20T12:18:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f6370aa7700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f636c151ae0 con 0x7f636c0627c0 2020-11-20T12:18:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f6370aa7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636c151d80 con 0x7f636c0c14e0 2020-11-20T12:18:02.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f636b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f636c0627c0 0x7f636c062ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:02.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f636b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f636c0627c0 0x7f636c062ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46154/0 (socket says 172.21.15.116:46154) 2020-11-20T12:18:02.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f636b7fe700 1 -- 172.21.15.116:0/755649508 learned_addr learned my addr 172.21.15.116:0/755649508 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:02.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f6363fff700 1 --2- 172.21.15.116:0/755649508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f636c0c14e0 0x7f636c066ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:02.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f636b7fe700 1 -- 172.21.15.116:0/755649508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f636c0c14e0 msgr2=0x7f636c066ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:02.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f636b7fe700 1 --2- 172.21.15.116:0/755649508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f636c0c14e0 0x7f636c066ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:02.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f636b7fe700 1 -- 172.21.15.116:0/755649508 >> v1:172.21.15.173:6789/0 conn(0x7f636c14d660 legacy=0x7f636c14dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:02.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f636b7fe700 1 -- 172.21.15.116:0/755649508 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c152020 con 0x7f636c0627c0 2020-11-20T12:18:02.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f636b7fe700 1 --2- 172.21.15.116:0/755649508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f636c0627c0 0x7f636c062ba0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f635c006fd0 tx=0x7f635c006eb0).ready entity=mon.0 client_cookie=f972b6facecf2790 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f636affd700 1 -- 172.21.15.116:0/755649508 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c009890 con 0x7f636c0627c0 2020-11-20T12:18:02.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f636affd700 1 -- 172.21.15.116:0/755649508 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c0099f0 con 0x7f636c0627c0 2020-11-20T12:18:02.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.780+0000 7f636affd700 1 -- 172.21.15.116:0/755649508 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c009d70 con 0x7f636c0627c0 2020-11-20T12:18:02.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.784+0000 7f6370aa7700 1 -- 172.21.15.116:0/755649508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f636c0627c0 msgr2=0x7f636c062ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:02.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.784+0000 7f6370aa7700 1 --2- 172.21.15.116:0/755649508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f636c0627c0 0x7f636c062ba0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f635c006fd0 tx=0x7f635c006eb0).stop 2020-11-20T12:18:02.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.784+0000 7f6370aa7700 1 -- 172.21.15.116:0/755649508 shutdown_connections 2020-11-20T12:18:02.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.784+0000 7f6370aa7700 1 --2- 172.21.15.116:0/755649508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f636c0627c0 0x7f636c062ba0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:02.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.784+0000 7f6370aa7700 1 --2- 172.21.15.116:0/755649508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f636c0c14e0 0x7f636c066ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:02.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.784+0000 7f6370aa7700 1 -- 172.21.15.116:0/755649508 >> 172.21.15.116:0/755649508 conn(0x7f636c060170 msgr2=0x7f636c060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:02.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.784+0000 7f6370aa7700 1 -- 172.21.15.116:0/755649508 shutdown_connections 2020-11-20T12:18:02.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.784+0000 7f6370aa7700 1 -- 172.21.15.116:0/755649508 wait complete. 2020-11-20T12:18:02.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f6370aa7700 1 Processor -- start 2020-11-20T12:18:02.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f6370aa7700 1 -- start start 2020-11-20T12:18:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f6370aa7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f636c0627c0 0x7f636c137700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f6370aa7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f636c0c14e0 0x7f636c0a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f6370aa7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f636c14d660 0x7f636c0a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f6370aa7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c140210 con 0x7f636c0627c0 2020-11-20T12:18:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f6370aa7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f636c163890 con 0x7f636c0c14e0 2020-11-20T12:18:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f6370aa7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f636c1413e0 con 0x7f636c14d660 2020-11-20T12:18:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f636c0627c0 0x7f636c137700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f636b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f636c0627c0 0x7f636c137700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46162/0 (socket says 172.21.15.116:46162) 2020-11-20T12:18:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f636b7fe700 1 -- 172.21.15.116:0/961316104 learned_addr learned my addr 172.21.15.116:0/961316104 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f6363fff700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f636c0c14e0 0x7f636c0a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f636bfff700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f636c14d660 0x7f636c0a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f636b7fe700 1 -- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f636c14d660 msgr2=0x7f636c0a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f636b7fe700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f636c14d660 0x7f636c0a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f636b7fe700 1 -- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f636c0c14e0 msgr2=0x7f636c0a3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f636b7fe700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f636c0c14e0 0x7f636c0a3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f636b7fe700 1 -- 172.21.15.116:0/961316104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c0a8470 con 0x7f636c0627c0 2020-11-20T12:18:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f636b7fe700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f636c0627c0 0x7f636c137700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f636c068630 tx=0x7f635c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f63637fe700 1 -- 172.21.15.116:0/961316104 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c011dd0 con 0x7f636c0627c0 2020-11-20T12:18:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f63637fe700 1 -- 172.21.15.116:0/961316104 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635c01e460 con 0x7f636c0627c0 2020-11-20T12:18:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f6370aa7700 1 -- 172.21.15.116:0/961316104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f636c163ab0 con 0x7f636c0627c0 2020-11-20T12:18:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.788+0000 7f63637fe700 1 -- 172.21.15.116:0/961316104 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635c01e7e0 con 0x7f636c0627c0 2020-11-20T12:18:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.792+0000 7f6370aa7700 1 -- 172.21.15.116:0/961316104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f636c163ff0 con 0x7f636c0627c0 2020-11-20T12:18:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.792+0000 7f6370aa7700 1 -- 172.21.15.116:0/961316104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636c044f60 con 0x7f636c0627c0 2020-11-20T12:18:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.792+0000 7f63637fe700 1 -- 172.21.15.116:0/961316104 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f635c025030 con 0x7f636c0627c0 2020-11-20T12:18:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.792+0000 7f63637fe700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63480557c0 0x7f6348057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.792+0000 7f63637fe700 1 -- 172.21.15.116:0/961316104 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f635c06fdf0 con 0x7f636c0627c0 2020-11-20T12:18:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.796+0000 7f6363fff700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63480557c0 0x7f6348057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.796+0000 7f6363fff700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63480557c0 0x7f6348057c80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f636c0a33c0 tx=0x7f6354008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:02.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:02.800+0000 7f63637fe700 1 -- 172.21.15.116:0/961316104 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f635c045d40 con 0x7f636c0627c0 2020-11-20T12:18:03.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.008+0000 7f6370aa7700 1 -- 172.21.15.116:0/961316104 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.21501"} v 0) v1 -- 0x7f636c068070 con 0x7f636c0627c0 2020-11-20T12:18:03.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.016+0000 7f63637fe700 1 -- 172.21.15.116:0/961316104 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.21501"}]=0 v12) v1 ==== 92+0+63 (secure 0 0 0) 0x7f635c023030 con 0x7f636c0627c0 2020-11-20T12:18:03.017 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:03.017 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQD7s7dfGvK8ABAAQrQSMNUDrPgMZ8IKp73f3Q== 2020-11-20T12:18:03.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 -- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63480557c0 msgr2=0x7f6348057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:03.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63480557c0 0x7f6348057c80 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f636c0a33c0 tx=0x7f6354008040).stop 2020-11-20T12:18:03.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 -- 172.21.15.116:0/961316104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f636c0627c0 msgr2=0x7f636c137700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:03.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f636c0627c0 0x7f636c137700 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f636c068630 tx=0x7f635c015000).stop 2020-11-20T12:18:03.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 -- 172.21.15.116:0/961316104 shutdown_connections 2020-11-20T12:18:03.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63480557c0 0x7f6348057c80 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:03.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f636c14d660 0x7f636c0a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:03.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f636c0627c0 0x7f636c137700 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:03.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 --2- 172.21.15.116:0/961316104 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f636c0c14e0 0x7f636c0a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:03.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 -- 172.21.15.116:0/961316104 >> 172.21.15.116:0/961316104 conn(0x7f636c060170 msgr2=0x7f636c0c3850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:03.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 -- 172.21.15.116:0/961316104 shutdown_connections 2020-11-20T12:18:03.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.020+0000 7f6370aa7700 1 -- 172.21.15.116:0/961316104 wait complete. 2020-11-20T12:18:03.042 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2020-11-20T12:18:03.043 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:18:03.043 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.2 2020-11-20T12:18:03.117 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:18:03.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:03.118 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T12:18:03.138 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:18:03.139 INFO:teuthology.orchestra.run.smithi173:> from __future__ import print_function 2020-11-20T12:18:03.139 INFO:teuthology.orchestra.run.smithi173:> from ceph_volume_client import CephFSVolumeClient, VolumePath 2020-11-20T12:18:03.139 INFO:teuthology.orchestra.run.smithi173:> from sys import version_info as sys_version_info 2020-11-20T12:18:03.140 INFO:teuthology.orchestra.run.smithi173:> from rados import OSError as rados_OSError 2020-11-20T12:18:03.140 INFO:teuthology.orchestra.run.smithi173:> import logging 2020-11-20T12:18:03.141 INFO:teuthology.orchestra.run.smithi173:> log = logging.getLogger("ceph_volume_client") 2020-11-20T12:18:03.141 INFO:teuthology.orchestra.run.smithi173:> log.addHandler(logging.StreamHandler()) 2020-11-20T12:18:03.141 INFO:teuthology.orchestra.run.smithi173:> log.setLevel(logging.DEBUG) 2020-11-20T12:18:03.142 INFO:teuthology.orchestra.run.smithi173:> vc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", "/volumes", None) 2020-11-20T12:18:03.142 INFO:teuthology.orchestra.run.smithi173:> vc.connect() 2020-11-20T12:18:03.143 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:18:03.143 INFO:teuthology.orchestra.run.smithi173:> vp = VolumePath("grpid", "volid") 2020-11-20T12:18:03.144 INFO:teuthology.orchestra.run.smithi173:> auth_result = vc.authorize(vp, "21501", readonly=False, 2020-11-20T12:18:03.144 INFO:teuthology.orchestra.run.smithi173:> tenant_id="None") 2020-11-20T12:18:03.145 INFO:teuthology.orchestra.run.smithi173:> print(auth_result['"'"'auth_key'"'"']) 2020-11-20T12:18:03.145 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:18:03.145 INFO:teuthology.orchestra.run.smithi173:> vc.disconnect() 2020-11-20T12:18:03.146 INFO:teuthology.orchestra.run.smithi173:> ' 2020-11-20T12:18:03.327 INFO:teuthology.orchestra.run.smithi173.stderr:Configuring to RADOS with config /etc/ceph/ceph.conf... 2020-11-20T12:18:03.331 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to RADOS... 2020-11-20T12:18:03.344 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to RADOS complete 2020-11-20T12:18:03.345 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to cephfs... 2020-11-20T12:18:03.345 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS initializing... 2020-11-20T12:18:03.349 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS mounting... 2020-11-20T12:18:03.360 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to cephfs complete 2020-11-20T12:18:03.361 INFO:teuthology.orchestra.run.smithi173.stderr:Recovering from partial auth updates (if any)... 2020-11-20T12:18:03.364 INFO:teuthology.orchestra.run.smithi173.stderr:Recovered from partial auth updates (if any). 2020-11-20T12:18:03.381 INFO:teuthology.orchestra.run.smithi173.stderr:Creating meta for ID 21501 with tenant None 2020-11-20T12:18:03.381 INFO:teuthology.orchestra.run.smithi173.stderr:Authorize: no existing meta 2020-11-20T12:18:03.399 INFO:teuthology.orchestra.run.smithi173.stderr:Authorizing Ceph id '21501' for path '/volumes/grpid/volid' 2020-11-20T12:18:03.457 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:18:03.457 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs... 2020-11-20T12:18:03.466 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs complete 2020-11-20T12:18:03.466 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados... 2020-11-20T12:18:03.469 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados complete 2020-11-20T12:18:03.470 INFO:teuthology.orchestra.run.smithi173.stdout:AQD7s7dfGvK8ABAAQrQSMNUDrPgMZ8IKp73f3Q== 2020-11-20T12:18:03.473 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:18:03.649 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-key client.21501 2020-11-20T12:18:03.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.832+0000 7f1866f0a700 1 Processor -- start 2020-11-20T12:18:03.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.832+0000 7f1866f0a700 1 -- start start 2020-11-20T12:18:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1866f0a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a4110 0x7f18600a44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:03.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1866f0a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1860067da0 0x7f1860068220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1866f0a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1860156460 con 0x7f18600a6110 2020-11-20T12:18:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1866f0a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18601570e0 con 0x7f1860067da0 2020-11-20T12:18:03.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1866f0a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1860156700 con 0x7f18600a4110 2020-11-20T12:18:03.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1865f08700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a4110 0x7f18600a44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:03.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1865f08700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a4110 0x7f18600a44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:55118/0 (socket says 172.21.15.116:55118) 2020-11-20T12:18:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1865f08700 1 -- 172.21.15.116:0/2816277734 learned_addr learned my addr 172.21.15.116:0/2816277734 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1865707700 1 --2- 172.21.15.116:0/2816277734 >> v2:172.21.15.173:3300/0 conn(0x7f1860067da0 0x7f1860068220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:03.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1865f08700 1 -- 172.21.15.116:0/2816277734 >> v2:172.21.15.173:3300/0 conn(0x7f1860067da0 msgr2=0x7f1860068220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:03.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1865f08700 1 --2- 172.21.15.116:0/2816277734 >> v2:172.21.15.173:3300/0 conn(0x7f1860067da0 0x7f1860068220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:03.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1865f08700 1 -- 172.21.15.116:0/2816277734 >> v1:172.21.15.173:6789/0 conn(0x7f18600a6110 legacy=0x7f18600a65c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:03.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.836+0000 7f1865f08700 1 -- 172.21.15.116:0/2816277734 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18601569a0 con 0x7f18600a4110 2020-11-20T12:18:03.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1865f08700 1 --2- 172.21.15.116:0/2816277734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a4110 0x7f18600a44f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f185c006fd0 tx=0x7f185c006eb0).ready entity=mon.2 client_cookie=79dd2c6167a67dee server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:03.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1864f06700 1 -- 172.21.15.116:0/2816277734 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c009810 con 0x7f18600a4110 2020-11-20T12:18:03.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1864f06700 1 -- 172.21.15.116:0/2816277734 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c009970 con 0x7f18600a4110 2020-11-20T12:18:03.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1864f06700 1 -- 172.21.15.116:0/2816277734 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c009cf0 con 0x7f18600a4110 2020-11-20T12:18:03.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1866f0a700 1 -- 172.21.15.116:0/2816277734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a4110 msgr2=0x7f18600a44f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:03.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1866f0a700 1 --2- 172.21.15.116:0/2816277734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a4110 0x7f18600a44f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f185c006fd0 tx=0x7f185c006eb0).stop 2020-11-20T12:18:03.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1866f0a700 1 -- 172.21.15.116:0/2816277734 shutdown_connections 2020-11-20T12:18:03.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1866f0a700 1 --2- 172.21.15.116:0/2816277734 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a4110 0x7f18600a44f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:03.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1866f0a700 1 --2- 172.21.15.116:0/2816277734 >> v2:172.21.15.173:3300/0 conn(0x7f1860067da0 0x7f1860068220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:03.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1866f0a700 1 -- 172.21.15.116:0/2816277734 >> 172.21.15.116:0/2816277734 conn(0x7f1860060170 msgr2=0x7f1860060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:03.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1866f0a700 1 -- 172.21.15.116:0/2816277734 shutdown_connections 2020-11-20T12:18:03.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.840+0000 7f1866f0a700 1 -- 172.21.15.116:0/2816277734 wait complete. 2020-11-20T12:18:03.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.844+0000 7f1866f0a700 1 Processor -- start 2020-11-20T12:18:03.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.844+0000 7f1866f0a700 1 -- start start 2020-11-20T12:18:03.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.844+0000 7f1866f0a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1860067da0 0x7f18600a2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:03.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.844+0000 7f1866f0a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18600a4110 0x7f186009e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:03.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.844+0000 7f1866f0a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a6110 0x7f18600a8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:03.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.844+0000 7f1866f0a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18600a3380 con 0x7f18600a4110 2020-11-20T12:18:03.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.844+0000 7f1866f0a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186009eb90 con 0x7f1860067da0 2020-11-20T12:18:03.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.844+0000 7f1866f0a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18600a8a30 con 0x7f18600a6110 2020-11-20T12:18:03.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1865707700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18600a4110 0x7f186009e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:03.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1865707700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18600a4110 0x7f186009e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46174/0 (socket says 172.21.15.116:46174) 2020-11-20T12:18:03.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1865707700 1 -- 172.21.15.116:0/3286256109 learned_addr learned my addr 172.21.15.116:0/3286256109 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:03.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1865f08700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1860067da0 0x7f18600a2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:03.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1866709700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a6110 0x7f18600a8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:03.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1865707700 1 -- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a6110 msgr2=0x7f18600a8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:03.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1865707700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a6110 0x7f18600a8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:03.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1865707700 1 -- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1860067da0 msgr2=0x7f18600a2e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:03.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1865707700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1860067da0 0x7f18600a2e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:03.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1865707700 1 -- 172.21.15.116:0/3286256109 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1860166270 con 0x7f18600a4110 2020-11-20T12:18:03.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1865707700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18600a4110 0x7f186009e6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f185000eba0 tx=0x7f1850006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:03.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f184dffb700 1 -- 172.21.15.116:0/3286256109 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185000c9b0 con 0x7f18600a4110 2020-11-20T12:18:03.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f184dffb700 1 -- 172.21.15.116:0/3286256109 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185000cb10 con 0x7f18600a4110 2020-11-20T12:18:03.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1866f0a700 1 -- 172.21.15.116:0/3286256109 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186009dcc0 con 0x7f18600a4110 2020-11-20T12:18:03.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f184dffb700 1 -- 172.21.15.116:0/3286256109 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185000ce90 con 0x7f18600a4110 2020-11-20T12:18:03.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1866f0a700 1 -- 172.21.15.116:0/3286256109 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186009e200 con 0x7f18600a4110 2020-11-20T12:18:03.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.848+0000 7f1866f0a700 1 -- 172.21.15.116:0/3286256109 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186009db00 con 0x7f18600a4110 2020-11-20T12:18:03.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.852+0000 7f184dffb700 1 -- 172.21.15.116:0/3286256109 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f185001a030 con 0x7f18600a4110 2020-11-20T12:18:03.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.852+0000 7f184dffb700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 0x7f1838057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:03.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.852+0000 7f184dffb700 1 -- 172.21.15.116:0/3286256109 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f185006e310 con 0x7f18600a4110 2020-11-20T12:18:03.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.852+0000 7f1865f08700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 0x7f1838057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:03.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.856+0000 7f1865f08700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 0x7f1838057c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f185c00b1b0 tx=0x7f185c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:03.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:03.860+0000 7f184dffb700 1 -- 172.21.15.116:0/3286256109 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1850044260 con 0x7f18600a4110 2020-11-20T12:18:04.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.068+0000 7f1866f0a700 1 -- 172.21.15.116:0/3286256109 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth get-key", "entity": "client.21501"} v 0) v1 -- 0x7f186009e390 con 0x7f18600a4110 2020-11-20T12:18:04.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.068+0000 7f184dffb700 1 -- 172.21.15.116:0/3286256109 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-key", "entity": "client.21501"}]=0 v13) v1 ==== 86+0+40 (secure 0 0 0) 0x7f1850047880 con 0x7f18600a4110 2020-11-20T12:18:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 -- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 msgr2=0x7f1838057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:04.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 0x7f1838057c80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f185c00b1b0 tx=0x7f185c015000).stop 2020-11-20T12:18:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 -- 172.21.15.116:0/3286256109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18600a4110 msgr2=0x7f186009e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18600a4110 0x7f186009e6a0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f185000eba0 tx=0x7f1850006df0).stop 2020-11-20T12:18:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 -- 172.21.15.116:0/3286256109 shutdown_connections 2020-11-20T12:18:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18380557c0 0x7f1838057c80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:04.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18600a6110 0x7f18600a8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:04.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1860067da0 0x7f18600a2e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:04.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 --2- 172.21.15.116:0/3286256109 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f18600a4110 0x7f186009e6a0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:04.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 -- 172.21.15.116:0/3286256109 >> 172.21.15.116:0/3286256109 conn(0x7f1860060170 msgr2=0x7f1860060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:04.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 -- 172.21.15.116:0/3286256109 shutdown_connections 2020-11-20T12:18:04.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.072+0000 7f1866f0a700 1 -- 172.21.15.116:0/3286256109 wait complete. 2020-11-20T12:18:04.098 INFO:teuthology.orchestra.run.smithi116.stdout:AQD7s7dfGvK8ABAAQrQSMNUDrPgMZ8IKp73f3Q== 2020-11-20T12:18:04.099 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-20T12:18:04.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.276+0000 7f2293571700 1 Processor -- start 2020-11-20T12:18:04.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.280+0000 7f2293571700 1 -- start start 2020-11-20T12:18:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.280+0000 7f2293571700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f228c0a73e0 0x7f228c0a6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.280+0000 7f2293571700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f228c0c4a70 0x7f228c067770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:04.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.280+0000 7f2293571700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228c159bf0 con 0x7f228c0a6710 2020-11-20T12:18:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.280+0000 7f2293571700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f228c159e90 con 0x7f228c0c4a70 2020-11-20T12:18:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.280+0000 7f2293571700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c15a130 con 0x7f228c0a73e0 2020-11-20T12:18:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f229130d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f228c0a6710 0x7f228c0a4120 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55616/0 (socket says 172.21.15.116:55616) 2020-11-20T12:18:04.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f229130d700 1 -- 172.21.15.116:0/3667268812 learned_addr learned my addr 172.21.15.116:0/3667268812 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2291b0e700 1 --2- 172.21.15.116:0/3667268812 >> v2:172.21.15.116:3300/0 conn(0x7f228c0c4a70 0x7f228c067770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:04.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2290b0c700 1 --2- 172.21.15.116:0/3667268812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f228c0a73e0 0x7f228c0a6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f227bfff700 1 -- 172.21.15.116:0/3667268812 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433398092 0 0) 0x7f228c159bf0 con 0x7f228c0a6710 2020-11-20T12:18:04.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f227bfff700 1 -- 172.21.15.116:0/3667268812 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2274003240 con 0x7f228c0a6710 2020-11-20T12:18:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2291b0e700 1 -- 172.21.15.116:0/3667268812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f228c0a73e0 msgr2=0x7f228c0a6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2291b0e700 1 --2- 172.21.15.116:0/3667268812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f228c0a73e0 0x7f228c0a6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2291b0e700 1 -- 172.21.15.116:0/3667268812 >> v1:172.21.15.116:6789/0 conn(0x7f228c0a6710 legacy=0x7f228c0a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:04.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2291b0e700 1 -- 172.21.15.116:0/3667268812 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c15a3d0 con 0x7f228c0c4a70 2020-11-20T12:18:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2291b0e700 1 --2- 172.21.15.116:0/3667268812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f228c0c4a70 0x7f228c067770 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f2284009c90 tx=0x7f22840091d0).ready entity=mon.0 client_cookie=23976cd5c6775e05 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f227bfff700 1 -- 172.21.15.116:0/3667268812 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228400b9c0 con 0x7f228c0c4a70 2020-11-20T12:18:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f227bfff700 1 -- 172.21.15.116:0/3667268812 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228400bb20 con 0x7f228c0c4a70 2020-11-20T12:18:04.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f227bfff700 1 -- 172.21.15.116:0/3667268812 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228400bea0 con 0x7f228c0c4a70 2020-11-20T12:18:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2293571700 1 -- 172.21.15.116:0/3667268812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f228c0c4a70 msgr2=0x7f228c067770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:04.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2293571700 1 --2- 172.21.15.116:0/3667268812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f228c0c4a70 0x7f228c067770 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f2284009c90 tx=0x7f22840091d0).stop 2020-11-20T12:18:04.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2293571700 1 -- 172.21.15.116:0/3667268812 shutdown_connections 2020-11-20T12:18:04.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2293571700 1 --2- 172.21.15.116:0/3667268812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f228c0c4a70 0x7f228c067770 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:04.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2293571700 1 --2- 172.21.15.116:0/3667268812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f228c0a73e0 0x7f228c0a6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:04.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.284+0000 7f2293571700 1 -- 172.21.15.116:0/3667268812 >> 172.21.15.116:0/3667268812 conn(0x7f228c063340 msgr2=0x7f228c0c4360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:04.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.288+0000 7f2293571700 1 -- 172.21.15.116:0/3667268812 shutdown_connections 2020-11-20T12:18:04.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.288+0000 7f2293571700 1 -- 172.21.15.116:0/3667268812 wait complete. 2020-11-20T12:18:04.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.288+0000 7f2293571700 1 Processor -- start 2020-11-20T12:18:04.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2293571700 1 -- start start 2020-11-20T12:18:04.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2293571700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f228c0a6710 0x7f228c1521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:04.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2293571700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f228c0a73e0 0x7f228c158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2293571700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f228c0c4a70 0x7f228c0c1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2293571700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c165120 con 0x7f228c0a73e0 2020-11-20T12:18:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2293571700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228c0a3410 con 0x7f228c0c4a70 2020-11-20T12:18:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2293571700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228c165300 con 0x7f228c0a6710 2020-11-20T12:18:04.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2290b0c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f228c0a73e0 0x7f228c158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2290b0c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f228c0a73e0 0x7f228c158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46190/0 (socket says 172.21.15.116:46190) 2020-11-20T12:18:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2290b0c700 1 -- 172.21.15.116:0/2099851345 learned_addr learned my addr 172.21.15.116:0/2099851345 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f229130d700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f228c0a6710 0x7f228c1521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:04.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2290b0c700 1 -- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f228c0a6710 msgr2=0x7f228c1521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2290b0c700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f228c0a6710 0x7f228c1521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2290b0c700 1 -- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f228c0c4a70 msgr2=0x7f228c0c1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2290b0c700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f228c0c4a70 0x7f228c0c1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2290b0c700 1 -- 172.21.15.116:0/2099851345 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f228c1526d0 con 0x7f228c0a73e0 2020-11-20T12:18:04.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2290b0c700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f228c0a73e0 0x7f228c158290 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f227c00ebd0 tx=0x7f227c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f22797fa700 1 -- 172.21.15.116:0/2099851345 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c00c9b0 con 0x7f228c0a73e0 2020-11-20T12:18:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f22797fa700 1 -- 172.21.15.116:0/2099851345 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c00cb10 con 0x7f228c0a73e0 2020-11-20T12:18:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f2293571700 1 -- 172.21.15.116:0/2099851345 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f228c159200 con 0x7f228c0a73e0 2020-11-20T12:18:04.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.292+0000 7f22797fa700 1 -- 172.21.15.116:0/2099851345 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c00ce90 con 0x7f228c0a73e0 2020-11-20T12:18:04.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.296+0000 7f2293571700 1 -- 172.21.15.116:0/2099851345 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228c165560 con 0x7f228c0a73e0 2020-11-20T12:18:04.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.296+0000 7f2293571700 1 -- 172.21.15.116:0/2099851345 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228c165720 con 0x7f228c0a73e0 2020-11-20T12:18:04.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.296+0000 7f22797fa700 1 -- 172.21.15.116:0/2099851345 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f227c012600 con 0x7f228c0a73e0 2020-11-20T12:18:04.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.296+0000 7f22797fa700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22640557c0 0x7f2264057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:04.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.296+0000 7f22797fa700 1 -- 172.21.15.116:0/2099851345 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f227c06ea50 con 0x7f228c0a73e0 2020-11-20T12:18:04.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.296+0000 7f229130d700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22640557c0 0x7f2264057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:04.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.300+0000 7f229130d700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22640557c0 0x7f2264057c80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f22880055a0 tx=0x7f2288009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:04.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.300+0000 7f22797fa700 1 -- 172.21.15.116:0/2099851345 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f227c0448f0 con 0x7f228c0a73e0 2020-11-20T12:18:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.516+0000 7f2293571700 1 -- 172.21.15.116:0/2099851345 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f228c044f60 con 0x7f228c0a73e0 2020-11-20T12:18:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.516+0000 7f22797fa700 1 -- 172.21.15.116:0/2099851345 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5730 (secure 0 0 0) 0x7f227c047f10 con 0x7f228c0a73e0 2020-11-20T12:18:04.517 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.517 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:18:04.517 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2020-11-20T12:18:04.518 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.518 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2020-11-20T12:18:04.518 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw==", 2020-11-20T12:18:04.518 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.519 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:04.519 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.519 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.519 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.519 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.520 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.520 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.520 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2020-11-20T12:18:04.520 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ==", 2020-11-20T12:18:04.521 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.521 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:04.532 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.532 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.533 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.533 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.533 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.534 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.534 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2020-11-20T12:18:04.534 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag==", 2020-11-20T12:18:04.535 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.535 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:04.535 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.536 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.536 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.536 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.537 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.537 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.537 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2020-11-20T12:18:04.538 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ==", 2020-11-20T12:18:04.538 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.538 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:04.538 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.539 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.539 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.539 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.540 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.540 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.540 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2020-11-20T12:18:04.541 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ==", 2020-11-20T12:18:04.541 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.541 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.542 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.542 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.542 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.542 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.543 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.543 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2020-11-20T12:18:04.543 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q==", 2020-11-20T12:18:04.544 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.544 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.544 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.545 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.545 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.545 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.546 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.546 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2020-11-20T12:18:04.546 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw==", 2020-11-20T12:18:04.546 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.547 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.548 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.548 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.548 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.549 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.549 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.549 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2020-11-20T12:18:04.550 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w==", 2020-11-20T12:18:04.550 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.550 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.550 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.551 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.551 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.551 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.552 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.552 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2020-11-20T12:18:04.552 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA==", 2020-11-20T12:18:04.553 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.553 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.553 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.554 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.554 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.554 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.555 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.555 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2020-11-20T12:18:04.555 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ==", 2020-11-20T12:18:04.556 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.556 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.556 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.556 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.557 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.557 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.557 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.558 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2020-11-20T12:18:04.558 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg==", 2020-11-20T12:18:04.558 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.559 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.559 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.559 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.560 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.560 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.560 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.561 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2020-11-20T12:18:04.561 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg==", 2020-11-20T12:18:04.561 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.562 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.562 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.562 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.562 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.563 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.563 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.563 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2020-11-20T12:18:04.564 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA==", 2020-11-20T12:18:04.564 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.564 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:04.565 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2020-11-20T12:18:04.565 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-20T12:18:04.565 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.566 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.566 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.566 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2020-11-20T12:18:04.566 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow==", 2020-11-20T12:18:04.567 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.567 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:04.567 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:18:04.568 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:18:04.568 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:18:04.568 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.569 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.569 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.569 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2020-11-20T12:18:04.570 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ==", 2020-11-20T12:18:04.570 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.570 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:04.571 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:18:04.571 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:18:04.571 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:18:04.571 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.572 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.572 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.572 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.21501", 2020-11-20T12:18:04.573 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQD7s7dfGvK8ABAAQrQSMNUDrPgMZ8IKp73f3Q==", 2020-11-20T12:18:04.573 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.573 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow rw path=/volumes/grpid/volid", 2020-11-20T12:18:04.574 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2020-11-20T12:18:04.574 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw pool=cephfs_data namespace=fsvolumens_volid" 2020-11-20T12:18:04.574 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.575 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.575 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.575 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2020-11-20T12:18:04.576 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ==", 2020-11-20T12:18:04.576 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.576 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:04.576 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:18:04.577 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:18:04.577 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:18:04.577 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.578 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.578 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.578 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2020-11-20T12:18:04.579 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBDs7dfh+KKOBAAZvcPXWly9/mxRE7NWuR85A==", 2020-11-20T12:18:04.579 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.579 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:18:04.580 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:04.580 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.580 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.580 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.581 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.581 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.581 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.manila", 2020-11-20T12:18:04.582 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQD5s7dfn0HnMxAAXUDbTOIUNzTn4X3w7Zfkpw==", 2020-11-20T12:18:04.582 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.582 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:18:04.583 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:04.583 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw" 2020-11-20T12:18:04.583 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.584 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.584 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.584 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2020-11-20T12:18:04.585 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A==", 2020-11-20T12:18:04.585 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.585 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:18:04.586 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:18:04.586 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.586 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.586 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:04.587 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:04.587 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2020-11-20T12:18:04.587 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA==", 2020-11-20T12:18:04.588 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:04.588 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:18:04.588 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:18:04.589 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:04.589 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.589 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:04.589 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:04.590 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:18:04.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 -- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22640557c0 msgr2=0x7f2264057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:04.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22640557c0 0x7f2264057c80 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f22880055a0 tx=0x7f2288009040).stop 2020-11-20T12:18:04.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 -- 172.21.15.116:0/2099851345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f228c0a73e0 msgr2=0x7f228c158290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:04.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f228c0a73e0 0x7f228c158290 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f227c00ebd0 tx=0x7f227c006df0).stop 2020-11-20T12:18:04.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 -- 172.21.15.116:0/2099851345 shutdown_connections 2020-11-20T12:18:04.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22640557c0 0x7f2264057c80 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:04.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f228c0c4a70 0x7f228c0c1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:04.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f228c0a6710 0x7f228c1521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:04.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 --2- 172.21.15.116:0/2099851345 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f228c0a73e0 0x7f228c158290 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:04.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 -- 172.21.15.116:0/2099851345 >> 172.21.15.116:0/2099851345 conn(0x7f228c063340 msgr2=0x7f228c1547c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:04.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 -- 172.21.15.116:0/2099851345 shutdown_connections 2020-11-20T12:18:04.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:04.520+0000 7f2293571700 1 -- 172.21.15.116:0/2099851345 wait complete. 2020-11-20T12:18:04.596 INFO:teuthology.orchestra.run.smithi116:> set -ex 2020-11-20T12:18:04.597 INFO:teuthology.orchestra.run.smithi116:> sudo dd of=/etc/ceph/ceph.client.21501.keyring 2020-11-20T12:18:04.612 INFO:teuthology.orchestra.run.smithi116.stderr:+ sudo dd of=/etc/ceph/ceph.client.21501.keyring 2020-11-20T12:18:04.627 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records in 2020-11-20T12:18:04.627 INFO:teuthology.orchestra.run.smithi116.stderr:0+1 records out 2020-11-20T12:18:04.627 INFO:teuthology.orchestra.run.smithi116.stderr:68 bytes copied, 0.000197118 s, 345 kB/s 2020-11-20T12:18:04.630 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:04.680 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:04.740 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:04.740 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:04.741 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:04.741 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:04.741 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.741 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:04.742 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.742 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:04.742 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.742 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:04.743 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:04.743 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.743 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:04.743 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:04.743 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.744 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:04.744 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:04.744 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.744 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:04.745 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:04.745 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.745 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:04.745 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.745 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:04.746 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:04.746 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:04.746 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:04.747 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:04.747 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:04.747 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:04.748 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:04.748 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:04.748 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:04.748 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.749 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:04.749 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.749 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:04.749 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.750 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:04.750 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.750 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:04.751 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:04.751 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:04.751 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:04.751 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:04.752 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:04.752 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.752 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:04.752 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:04.753 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.753 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:04.753 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:04.753 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.754 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:04.754 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:04.754 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.754 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:04.754 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:04.755 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:04.755 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.755 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:04.755 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:04.755 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.756 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:04.756 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.756 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:04.756 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.757 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:04.757 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.757 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:04.757 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:04.757 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.758 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:04.758 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:04.758 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:04.758 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:04.759 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:04.759 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:04.759 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:04.759 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:04.759 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:04.760 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:04.760 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:04.760 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:04.760 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:04.761 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:04.761 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:04.761 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:04.761 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:04.762 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:04.762 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:04.762 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:04.762 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.762 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:04.763 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:04.763 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:04.763 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:04.763 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:04.764 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:04.764 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:04.764 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:04.764 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.764 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:04.765 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:04.765 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:04.765 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:04.765 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:04.766 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:04.766 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:04.766 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:04.766 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:04.766 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:04.767 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:04.767 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:04.767 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.767 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:04.768 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:04.768 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:04.768 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:04.768 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:04.768 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:04.769 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:04.769 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:04.769 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:04.769 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:04.770 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:04.770 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:04.770 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:04.770 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:04.771 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:04.771 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:04.771 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:04.771 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:04.771 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:04.772 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:04.777 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:04.777 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:04.777 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:04.778 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:04.778 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:04.778 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:04.778 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:04.778 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:04.779 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:04.779 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:04.779 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:04.779 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:04.779 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:04.780 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:04.780 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:04.780 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:04.780 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:04.781 INFO:teuthology.orchestra.run.smithi116.stdout: client quota = True 2020-11-20T12:18:04.782 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:04.782 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:04.783 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.784 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:04.785 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:04.786 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.787 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:04.787 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:04.787 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.787 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:04.787 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:04.788 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.788 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:04.788 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.788 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:04.788 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:04.789 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:04.789 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:04.789 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:04.789 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:04.789 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:04.790 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:04.790 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:04.790 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:04.790 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.790 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:04.791 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.791 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:04.791 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.791 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:04.792 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.792 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:04.792 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:04.792 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:04.792 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:04.793 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:04.793 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:04.793 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.793 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:04.793 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:04.794 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.794 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:04.794 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:04.794 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.794 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:04.795 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:04.795 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.795 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:04.795 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:04.796 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:04.796 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.796 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:04.796 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:04.796 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.797 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:04.797 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.797 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:04.797 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.797 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:04.798 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.798 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:04.798 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:04.798 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.798 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:04.799 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:04.799 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:04.799 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:04.799 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:04.799 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:04.800 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:04.800 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:04.800 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:04.800 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:04.800 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:04.801 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:04.801 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:04.801 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:04.801 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:04.802 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:04.802 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:04.802 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:04.802 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:04.802 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:04.803 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.803 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:04.803 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:04.803 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:04.803 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:04.804 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:04.804 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:04.804 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:04.804 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:04.804 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.805 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:04.805 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:04.805 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:04.805 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:04.805 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:04.806 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:04.806 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:04.806 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:04.806 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:04.807 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:04.807 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:04.807 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:04.807 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.807 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:04.808 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:04.808 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:04.808 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:04.808 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:04.808 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:04.809 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:04.809 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:04.809 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:04.809 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:04.809 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:04.810 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:04.810 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:04.810 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:04.810 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:04.810 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:04.811 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:04.811 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:04.811 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:04.811 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:04.811 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:04.812 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:04.812 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:04.812 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:04.812 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:04.812 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:04.813 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:04.813 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:04.813 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:04.813 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:04.813 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:04.814 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:04.814 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:04.814 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:04.814 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:04.814 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:04.815 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:18:04.815 INFO:teuthology.orchestra.run.smithi173.stdout: client quota = True 2020-11-20T12:18:04.816 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:04.823 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:04.884 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:04.884 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:04.884 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:04.885 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:04.885 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.885 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:04.885 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.885 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:04.886 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.886 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:04.886 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:04.886 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.887 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:04.887 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:04.887 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.887 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:04.887 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:04.888 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.888 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:04.888 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:04.888 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.889 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:04.889 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.889 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:04.889 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:04.889 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:04.890 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:04.890 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:04.890 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:04.890 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:04.891 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:04.891 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:04.891 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:04.891 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.891 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:04.892 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.892 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:04.892 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.892 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:04.893 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.893 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:04.893 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:04.893 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:04.894 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:04.894 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:04.894 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:04.894 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.895 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:04.895 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:04.895 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.895 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:04.895 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:04.896 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.896 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:04.896 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:04.896 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.896 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:04.897 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:04.897 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:04.897 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.897 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:04.898 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:04.898 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.898 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:04.898 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.898 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:04.899 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.899 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:04.899 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.899 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:04.900 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:04.900 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.900 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:04.900 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:04.900 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:04.901 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:04.901 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:04.901 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:04.901 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:04.901 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:04.902 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:04.902 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:04.902 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:04.902 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:04.902 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:04.903 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:04.903 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:04.903 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:04.903 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:04.904 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:04.904 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:04.904 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:04.904 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.904 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:04.905 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:04.905 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:04.905 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:04.905 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:04.905 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:04.906 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:04.906 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:04.906 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.906 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:04.907 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:04.907 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:04.907 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:04.907 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:04.907 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:04.908 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:04.908 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:04.908 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:04.908 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:04.909 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:04.909 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:04.909 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:04.909 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:04.909 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:04.910 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:04.910 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:04.910 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:04.910 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:04.910 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:04.911 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:04.911 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:04.911 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:04.911 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:04.911 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:04.912 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:04.912 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:04.912 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:04.912 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:04.913 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:04.913 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:04.913 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:04.913 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:04.913 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:04.914 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:04.914 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:04.914 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:04.914 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:04.915 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:04.915 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:04.915 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:04.915 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:04.915 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:04.916 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:04.916 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:04.916 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:04.916 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:04.916 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:04.917 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:04.917 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:04.917 INFO:teuthology.orchestra.run.smithi116.stdout: client quota = True 2020-11-20T12:18:04.917 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:04.919 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:04.919 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:04.919 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:04.920 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:04.920 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.920 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:04.920 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.920 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:04.921 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.921 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:04.921 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:04.922 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.922 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:04.922 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:04.922 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.922 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:04.923 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:04.923 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.923 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:04.923 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:04.923 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.924 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:04.924 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.924 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:04.924 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:04.925 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:04.925 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:04.925 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:04.925 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:04.925 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:04.926 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:04.926 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:04.926 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:04.926 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.926 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:04.927 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.927 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:04.927 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.927 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:04.928 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.928 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:04.928 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:04.928 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:04.928 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:04.929 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:04.929 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:04.929 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.929 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:04.930 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:04.930 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.930 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:04.930 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:04.930 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.931 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:04.931 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:04.931 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.931 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:04.932 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:04.932 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:04.932 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.932 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:04.932 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:04.933 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.933 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:04.933 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.933 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:04.933 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.934 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:04.934 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.934 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:04.934 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:04.934 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.935 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:04.935 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:04.935 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:04.935 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:04.936 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:04.936 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:04.936 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:04.936 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:04.936 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:04.937 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:04.937 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:04.937 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:04.937 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:04.937 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:04.938 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:04.938 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:04.938 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:04.938 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:04.939 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:04.939 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:04.939 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.939 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:04.939 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:04.940 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:04.940 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:04.940 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:04.940 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:04.940 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:04.941 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:04.941 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.941 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:04.941 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:04.942 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:04.942 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:04.942 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:04.942 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:04.942 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:04.943 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:04.943 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:04.943 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:04.943 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:04.944 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:04.944 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:04.944 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:04.944 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:04.944 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:04.945 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:04.945 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:04.945 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:04.945 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:04.946 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:04.946 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:04.946 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:04.946 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:04.946 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:04.947 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:04.947 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:04.947 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:04.947 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:04.947 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:04.948 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:04.948 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:04.948 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:04.948 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:04.948 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:04.949 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:04.949 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:04.949 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:04.949 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:04.950 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:04.950 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:04.950 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:04.950 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:04.951 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:04.951 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:04.951 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:04.951 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:04.951 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:04.952 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:04.952 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:18:04.952 INFO:teuthology.orchestra.run.smithi173.stdout: client quota = True 2020-11-20T12:18:04.952 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:04.953 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:04.960 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:05.023 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:05.023 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:05.023 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:05.024 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:05.024 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.024 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:05.024 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.025 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:05.025 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.025 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:05.025 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:05.025 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.026 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:05.026 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:05.026 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.026 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:05.027 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:05.027 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.027 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:05.028 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:05.028 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.028 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:05.029 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.029 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:05.029 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:05.029 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:05.029 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:05.030 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:05.030 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:05.030 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:05.030 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:05.031 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:05.031 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:05.031 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.032 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:05.032 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.033 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:05.033 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.033 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:05.033 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.033 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:05.034 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:05.034 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:05.034 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:05.034 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:05.035 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:05.035 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.035 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:05.035 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:05.035 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.036 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:05.036 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:05.036 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.036 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:05.036 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:05.037 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.037 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:05.037 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:05.038 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:05.038 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.038 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:05.038 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:05.038 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.039 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:05.039 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.040 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:05.040 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.040 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:05.041 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.041 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:05.042 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:05.042 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.043 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:05.043 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:05.044 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:05.044 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:05.045 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:05.045 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:05.045 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:05.046 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:05.046 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:05.047 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:05.047 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:05.048 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:05.048 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:05.049 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:05.049 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:05.049 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:05.050 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:05.050 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:05.050 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:05.051 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:05.051 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.052 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:05.052 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:05.053 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:05.053 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:05.054 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:05.054 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:05.055 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:05.055 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:05.055 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.056 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:05.056 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:05.056 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:05.057 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:05.057 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:05.058 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:05.058 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:05.059 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:05.059 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:05.059 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:05.060 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:05.060 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:05.060 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.061 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:05.061 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:05.062 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:05.062 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:05.063 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:05.063 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:05.064 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:05.064 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:05.065 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:05.065 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:05.066 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:05.066 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:05.066 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:05.067 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:05.067 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:05.067 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:05.068 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:05.069 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:05.069 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:05.069 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:05.070 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:05.070 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:05.070 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:05.071 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:05.071 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:05.071 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:05.072 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:05.072 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:05.072 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:05.073 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:05.073 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:05.074 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:05.075 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:05.075 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:05.076 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:05.076 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:05.077 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:05.077 INFO:teuthology.orchestra.run.smithi116.stdout: client quota = True 2020-11-20T12:18:05.077 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:05.078 INFO:teuthology.orchestra.run.smithi116.stdout: debug objecter = 20 2020-11-20T12:18:05.080 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:05.081 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:05.081 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:05.082 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:05.082 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.082 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:05.083 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.083 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:05.083 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.084 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:05.084 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:05.084 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.085 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:05.085 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:05.086 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.086 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:05.086 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:05.087 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.087 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:05.087 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:05.088 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.088 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:05.088 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.090 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:05.090 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:05.090 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:05.091 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:05.091 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:05.091 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:05.092 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:05.092 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:05.093 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:05.093 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:05.093 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.094 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:05.094 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.094 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:05.095 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.095 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:05.096 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.096 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:05.096 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:05.097 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:05.097 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:05.098 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:05.098 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:05.098 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.099 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:05.099 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:05.100 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.100 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:05.101 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:05.101 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.101 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:05.102 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:05.102 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.102 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:05.103 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:05.103 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:05.103 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.103 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:05.104 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:05.104 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.104 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:05.104 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.104 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:05.105 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.105 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:05.105 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.105 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:05.106 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:05.106 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.106 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:05.106 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:05.106 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:05.107 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:05.107 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:05.107 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:05.107 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:05.107 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:05.108 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:05.108 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:05.108 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:05.108 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:05.108 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:05.109 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:05.109 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:05.109 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:05.109 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:05.109 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:05.110 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:05.110 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:05.110 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.111 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:05.111 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:05.111 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:05.111 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:05.112 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:05.112 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:05.112 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:05.112 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:05.112 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.113 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:05.113 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:05.113 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:05.113 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:05.113 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:05.114 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:05.114 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:05.114 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:05.114 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:05.114 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:05.115 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:05.115 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:05.115 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.115 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:05.115 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:05.116 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:05.116 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:05.116 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:05.116 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:05.117 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:05.117 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:05.117 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:05.117 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:05.118 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:05.118 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:05.118 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:05.118 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:05.118 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:05.119 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:05.119 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:05.119 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:05.119 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:05.119 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:05.120 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:05.120 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:05.120 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:05.121 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:05.121 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:05.121 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:05.121 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:05.122 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:05.122 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:05.122 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:05.122 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:05.122 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:05.123 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:05.123 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:05.123 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:05.123 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:05.123 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:18:05.124 INFO:teuthology.orchestra.run.smithi173.stdout: client quota = True 2020-11-20T12:18:05.124 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:05.124 INFO:teuthology.orchestra.run.smithi173.stdout: debug objecter = 20 2020-11-20T12:18:05.125 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:05.132 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:05.196 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:05.196 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:05.196 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:05.197 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:05.197 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.197 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:05.197 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.198 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:05.198 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.198 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:05.198 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:05.199 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.199 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:05.199 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:05.199 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.199 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:05.200 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:05.200 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.200 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:05.200 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:05.201 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.201 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:05.201 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.201 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:05.202 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:05.202 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:05.202 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:05.202 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:05.202 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:05.203 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:05.203 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:05.203 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:05.203 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:05.204 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.204 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:05.204 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.205 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:05.205 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.205 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:05.205 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.206 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:05.206 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:05.206 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:05.206 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:05.206 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:05.207 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:05.207 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.207 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:05.207 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:05.208 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.208 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:05.208 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:05.208 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.208 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:05.209 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:05.209 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.209 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:05.209 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:05.210 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:05.210 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.210 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:05.210 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:05.211 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.211 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:05.211 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.211 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:05.212 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.212 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:05.212 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.212 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:05.213 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:05.213 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.213 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:05.213 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:05.214 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:05.214 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:05.214 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:05.215 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:05.215 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:05.215 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:05.215 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:05.216 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:05.216 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:05.216 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:05.216 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:05.217 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:05.217 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:05.217 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:05.217 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:05.218 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:05.218 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:05.218 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:05.218 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.218 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:05.219 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:05.219 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:05.219 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:05.219 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:05.220 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:05.220 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:05.220 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:05.220 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.220 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:05.221 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:05.221 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:05.221 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:05.221 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:05.222 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:05.222 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:05.222 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:05.222 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:05.223 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:05.223 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:05.223 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:05.223 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.224 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:05.224 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:05.224 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:05.225 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:05.225 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:05.225 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:05.225 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:05.226 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:05.226 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:05.226 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:05.226 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:05.226 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:05.227 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:05.227 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:05.227 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:05.227 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:05.228 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:05.228 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:05.228 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:05.228 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:05.229 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:05.229 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:05.229 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:05.229 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:05.229 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:05.230 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:05.230 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:05.230 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:05.230 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:05.231 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:05.231 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:05.231 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:05.231 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:05.231 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:05.232 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:05.232 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:05.232 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:05.232 INFO:teuthology.orchestra.run.smithi116.stdout: client quota = True 2020-11-20T12:18:05.233 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:05.233 INFO:teuthology.orchestra.run.smithi116.stdout: debug objecter = 20 2020-11-20T12:18:05.233 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.21501.keyring 2020-11-20T12:18:05.235 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:05.235 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:05.236 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:05.236 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:05.236 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.236 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:05.237 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.237 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:05.237 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.237 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:05.238 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:05.238 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.238 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:05.238 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:05.238 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.239 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:05.239 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:05.239 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.239 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:05.240 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:05.240 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.240 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:05.240 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.240 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:05.241 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:05.241 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:05.241 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:05.241 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:05.242 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:05.242 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:05.242 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:05.242 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:05.242 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:05.243 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.243 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:05.243 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.243 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:05.244 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.244 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:05.244 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.244 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:05.244 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:05.245 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:05.245 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:05.245 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:05.245 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:05.246 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.246 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:05.246 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:05.246 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.247 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:05.247 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:05.247 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.247 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:05.248 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:05.248 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.248 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:05.248 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:05.248 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:05.249 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.249 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:05.249 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:05.249 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.250 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:05.250 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.250 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:05.251 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.251 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:05.251 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.251 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:05.252 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:05.252 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.252 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:05.252 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:05.252 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:05.253 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:05.253 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:05.253 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:05.253 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:05.254 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:05.254 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:05.254 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:05.254 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:05.254 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:05.255 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:05.255 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:05.255 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:05.255 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:05.256 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:05.256 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:05.256 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:05.256 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:05.257 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.257 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:05.257 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:05.257 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:05.257 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:05.258 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:05.258 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:05.258 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:05.258 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:05.259 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.259 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:05.259 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:05.259 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:05.259 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:05.260 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:05.260 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:05.260 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:05.260 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:05.261 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:05.261 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:05.261 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:05.261 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:05.262 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:05.262 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:05.262 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:05.262 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:05.262 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:05.263 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:05.263 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:05.263 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:05.263 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:05.264 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:05.264 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:05.264 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:05.264 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:05.264 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:05.265 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:05.265 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:05.265 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:05.265 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:05.266 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:05.266 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:05.266 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:05.266 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:05.267 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:05.267 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:05.267 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:05.267 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:05.267 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:05.268 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:05.268 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:05.268 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:05.268 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:05.269 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:05.269 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:05.269 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:05.269 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:05.270 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:05.270 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:05.270 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:18:05.270 INFO:teuthology.orchestra.run.smithi173.stdout: client quota = True 2020-11-20T12:18:05.271 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:05.271 INFO:teuthology.orchestra.run.smithi173.stdout: debug objecter = 20 2020-11-20T12:18:05.271 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.21501.keyring 2020-11-20T12:18:05.271 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:18:05.272 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:18:05.272 INFO:tasks.cephfs.mount:self.client.name = client.21501 2020-11-20T12:18:05.273 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2020-11-20T12:18:05.273 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:18:05.273 INFO:tasks.cephfs.mount:self.cephfs_mntpt = /volumes/grpid/volid 2020-11-20T12:18:05.274 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:18:05.274 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:18:05.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:05.275 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:18:05.289 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:18:05.290 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:18:05.290 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:18:05.290 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:05.290 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:18:05.291 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:05.291 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:05.291 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:05.291 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:05.292 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:05.292 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:05.292 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:05.292 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:18:05.292 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:05.293 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:18:05.293 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:05.293 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:18:05.293 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:05.293 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:18:05.294 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:05.294 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:18:05.294 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:05.294 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:18:05.294 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:05.295 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:18:05.295 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:05.295 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:18:05.295 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:05.296 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:18:05.296 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:18:05.296 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:18:05.296 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:05.296 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:18:05.297 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:18:05.297 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:18:05.297 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:05.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:05.298 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:18:05.363 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:18:05.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:05.367 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:18:05.380 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:18:05.381 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:18:05.381 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:18:05.381 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:18:05.381 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:18:05.382 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:18:05.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:05.383 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:18:05.384 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:18:05.384 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:18:05.385 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:18:05.385 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:18:05.385 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:18:05.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:05.484 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:18:05.500 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:18:05.501 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:18:05.502 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:18:05.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:05.503 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2020-11-20T12:18:05.581 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:05.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.760+0000 7f987f59e700 1 Processor -- start 2020-11-20T12:18:05.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.760+0000 7f987f59e700 1 -- start start 2020-11-20T12:18:05.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f988014a210 0x7f98801536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:05.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987f59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f98800a36d0 0x7f9880066880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:05.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987f59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f98800ba860 0x7f98800bad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:05.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987f59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98800c42c0 con 0x7f98800ba860 2020-11-20T12:18:05.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987f59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9880153be0 con 0x7f98800a36d0 2020-11-20T12:18:05.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98800bb200 con 0x7f988014a210 2020-11-20T12:18:05.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987ed9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f98800ba860 0x7f98800bad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:05.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987ed9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f98800ba860 0x7f98800bad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46200/0 (socket says 172.21.15.116:46200) 2020-11-20T12:18:05.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987ed9d700 1 -- 172.21.15.116:0/189934478 learned_addr learned my addr 172.21.15.116:0/189934478 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:05.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987dd9b700 1 --2- 172.21.15.116:0/189934478 >> v2:172.21.15.173:3300/0 conn(0x7f98800a36d0 0x7f9880066880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:05.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987e59c700 1 --2- 172.21.15.116:0/189934478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f988014a210 0x7f98801536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:05.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987ed9d700 1 -- 172.21.15.116:0/189934478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f988014a210 msgr2=0x7f98801536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:05.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987ed9d700 1 --2- 172.21.15.116:0/189934478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f988014a210 0x7f98801536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:05.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987ed9d700 1 -- 172.21.15.116:0/189934478 >> v2:172.21.15.173:3300/0 conn(0x7f98800a36d0 msgr2=0x7f9880066880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:05.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987ed9d700 1 --2- 172.21.15.116:0/189934478 >> v2:172.21.15.173:3300/0 conn(0x7f98800a36d0 0x7f9880066880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:05.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987ed9d700 1 -- 172.21.15.116:0/189934478 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9880153e00 con 0x7f98800ba860 2020-11-20T12:18:05.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987dd9b700 1 --2- 172.21.15.116:0/189934478 >> v2:172.21.15.173:3300/0 conn(0x7f98800a36d0 0x7f9880066880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:05.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.764+0000 7f987e59c700 1 --2- 172.21.15.116:0/189934478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f988014a210 0x7f98801536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:05.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987ed9d700 1 --2- 172.21.15.116:0/189934478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98800ba860 0x7f98800bad10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f988005f800 tx=0x7f9878006eb0).ready entity=mon.0 client_cookie=54d01600ff6a3ce2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:05.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987d59a700 1 -- 172.21.15.116:0/189934478 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780098d0 con 0x7f98800ba860 2020-11-20T12:18:05.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987d59a700 1 -- 172.21.15.116:0/189934478 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9878009a30 con 0x7f98800ba860 2020-11-20T12:18:05.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987d59a700 1 -- 172.21.15.116:0/189934478 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9878009db0 con 0x7f98800ba860 2020-11-20T12:18:05.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987f59e700 1 -- 172.21.15.116:0/189934478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98800ba860 msgr2=0x7f98800bad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:05.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987f59e700 1 --2- 172.21.15.116:0/189934478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98800ba860 0x7f98800bad10 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f988005f800 tx=0x7f9878006eb0).stop 2020-11-20T12:18:05.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987f59e700 1 -- 172.21.15.116:0/189934478 shutdown_connections 2020-11-20T12:18:05.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987f59e700 1 --2- 172.21.15.116:0/189934478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98800ba860 0x7f98800bad10 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:05.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987f59e700 1 --2- 172.21.15.116:0/189934478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f988014a210 0x7f98801536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:05.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987f59e700 1 --2- 172.21.15.116:0/189934478 >> v2:172.21.15.173:3300/0 conn(0x7f98800a36d0 0x7f9880066880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:05.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987f59e700 1 -- 172.21.15.116:0/189934478 >> 172.21.15.116:0/189934478 conn(0x7f9880060690 msgr2=0x7f98800c4440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:05.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987f59e700 1 -- 172.21.15.116:0/189934478 shutdown_connections 2020-11-20T12:18:05.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.768+0000 7f987f59e700 1 -- 172.21.15.116:0/189934478 wait complete. 2020-11-20T12:18:05.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.772+0000 7f987f59e700 1 Processor -- start 2020-11-20T12:18:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.772+0000 7f987f59e700 1 -- start start 2020-11-20T12:18:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.772+0000 7f987f59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98800a36d0 0x7f9880148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.772+0000 7f987f59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98800ba860 0x7f98800a67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:05.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.772+0000 7f987f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f988014a210 0x7f98800bfa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:05.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.772+0000 7f987f59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98800a73f0 con 0x7f98800a36d0 2020-11-20T12:18:05.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.772+0000 7f987f59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988014f630 con 0x7f98800ba860 2020-11-20T12:18:05.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.772+0000 7f987f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98800a7620 con 0x7f988014a210 2020-11-20T12:18:05.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98800a36d0 0x7f9880148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:05.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98800a36d0 0x7f9880148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46202/0 (socket says 172.21.15.116:46202) 2020-11-20T12:18:05.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987e59c700 1 -- 172.21.15.116:0/4056773201 learned_addr learned my addr 172.21.15.116:0/4056773201 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:05.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987ed9d700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f988014a210 0x7f98800bfa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:05.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987dd9b700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98800ba860 0x7f98800a67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:05.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987e59c700 1 -- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f988014a210 msgr2=0x7f98800bfa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:05.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987e59c700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f988014a210 0x7f98800bfa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:05.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987e59c700 1 -- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98800ba860 msgr2=0x7f98800a67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:05.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987e59c700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98800ba860 0x7f98800a67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:05.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987e59c700 1 -- 172.21.15.116:0/4056773201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98800a78c0 con 0x7f98800a36d0 2020-11-20T12:18:05.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987dd9b700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98800ba860 0x7f98800a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:05.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987e59c700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98800a36d0 0x7f9880148f00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f987000ec90 tx=0x7f9870008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:05.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f986e7fc700 1 -- 172.21.15.116:0/4056773201 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987000c9d0 con 0x7f98800a36d0 2020-11-20T12:18:05.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f986e7fc700 1 -- 172.21.15.116:0/4056773201 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987000cb30 con 0x7f98800a36d0 2020-11-20T12:18:05.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987f59e700 1 -- 172.21.15.116:0/4056773201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98800a7b60 con 0x7f98800a36d0 2020-11-20T12:18:05.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f986e7fc700 1 -- 172.21.15.116:0/4056773201 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9870013430 con 0x7f98800a36d0 2020-11-20T12:18:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987f59e700 1 -- 172.21.15.116:0/4056773201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98800a8540 con 0x7f98800a36d0 2020-11-20T12:18:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f987f59e700 1 -- 172.21.15.116:0/4056773201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98800a7eb0 con 0x7f98800a36d0 2020-11-20T12:18:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.776+0000 7f986e7fc700 1 -- 172.21.15.116:0/4056773201 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f98700137a0 con 0x7f98800a36d0 2020-11-20T12:18:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.780+0000 7f986e7fc700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98540557c0 0x7f9854057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:05.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.780+0000 7f986e7fc700 1 -- 172.21.15.116:0/4056773201 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f987006e850 con 0x7f98800a36d0 2020-11-20T12:18:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.780+0000 7f987dd9b700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98540557c0 0x7f9854057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.780+0000 7f987dd9b700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98540557c0 0x7f9854057c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f9874006bd0 tx=0x7f9874008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:05.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.784+0000 7f986e7fc700 1 -- 172.21.15.116:0/4056773201 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9870044690 con 0x7f98800a36d0 2020-11-20T12:18:05.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.940+0000 7f987f59e700 1 -- 172.21.15.116:0/4056773201 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f988005d340 con 0x7f98800a36d0 2020-11-20T12:18:05.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f986e7fc700 1 -- 172.21.15.116:0/4056773201 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f9870047cb0 con 0x7f98800a36d0 2020-11-20T12:18:05.942 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:05.943 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":45,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:17:41.167389+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:17:38.002878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-20T12:17:38.995456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:05.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 -- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98540557c0 msgr2=0x7f9854057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:05.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98540557c0 0x7f9854057c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f9874006bd0 tx=0x7f9874008040).stop 2020-11-20T12:18:05.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 -- 172.21.15.116:0/4056773201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98800a36d0 msgr2=0x7f9880148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:05.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98800a36d0 0x7f9880148f00 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f987000ec90 tx=0x7f9870008040).stop 2020-11-20T12:18:05.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 -- 172.21.15.116:0/4056773201 shutdown_connections 2020-11-20T12:18:05.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98540557c0 0x7f9854057c80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:05.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f988014a210 0x7f98800bfa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:05.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98800a36d0 0x7f9880148f00 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:05.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 --2- 172.21.15.116:0/4056773201 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98800ba860 0x7f98800a67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:05.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 -- 172.21.15.116:0/4056773201 >> 172.21.15.116:0/4056773201 conn(0x7f9880060690 msgr2=0x7f9880066540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:05.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 -- 172.21.15.116:0/4056773201 shutdown_connections 2020-11-20T12:18:05.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:05.944+0000 7f987f59e700 1 -- 172.21.15.116:0/4056773201 wait complete. 2020-11-20T12:18:05.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:17:38.002878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:17:38.995456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:18:05.967 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:18:06.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.152+0000 7f5bd88d3700 1 Processor -- start 2020-11-20T12:18:06.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.152+0000 7f5bd88d3700 1 -- start start 2020-11-20T12:18:06.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd88d3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5bd4152d00 0x7f5bd40aae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd88d3700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd4068120 con 0x7f5bd4155cd0 2020-11-20T12:18:06.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd88d3700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd40a8eb0 con 0x7f5bd4151910 2020-11-20T12:18:06.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd88d3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd40a9150 con 0x7f5bd4152d00 2020-11-20T12:18:06.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd3fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5bd4155cd0 0x7f5bd40c67c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55648/0 (socket says 172.21.15.116:55648) 2020-11-20T12:18:06.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd3fff700 1 -- 172.21.15.116:0/2027042021 learned_addr learned my addr 172.21.15.116:0/2027042021 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:06.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd2ffd700 1 --2- 172.21.15.116:0/2027042021 >> v2:172.21.15.173:3300/0 conn(0x7f5bd4152d00 0x7f5bd40aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136498394 0 0) 0x7f5bd4068120 con 0x7f5bd4155cd0 2020-11-20T12:18:06.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bc0003240 con 0x7f5bd4155cd0 2020-11-20T12:18:06.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1667210328 0 0) 0x7f5bc0003240 con 0x7f5bd4155cd0 2020-11-20T12:18:06.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 >> v2:172.21.15.173:3300/0 conn(0x7f5bd4152d00 msgr2=0x7f5bd40aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:06.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 --2- 172.21.15.116:0/2027042021 >> v2:172.21.15.173:3300/0 conn(0x7f5bd4152d00 0x7f5bd40aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 >> v1:172.21.15.173:6789/0 conn(0x7f5bd4151910 legacy=0x7f5bd41555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd40a93f0 con 0x7f5bd4155cd0 2020-11-20T12:18:06.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f5bcc001860 con 0x7f5bd4155cd0 2020-11-20T12:18:06.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 >> v1:172.21.15.116:6789/0 conn(0x7f5bd4155cd0 legacy=0x7f5bd40c67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 --2- 172.21.15.116:0/2027042021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bc0003730 0x7f5bc0003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 --2- 172.21.15.116:0/2027042021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bc00044e0 0x7f5bc0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 --2- 172.21.15.116:0/2027042021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bc0004e50 0x7f5bc000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd40a9150 con 0x7f5bc0004e50 2020-11-20T12:18:06.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc0004260 con 0x7f5bc0003730 2020-11-20T12:18:06.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.156+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc000bc20 con 0x7f5bc00044e0 2020-11-20T12:18:06.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd2ffd700 1 --2- 172.21.15.116:0/2027042021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bc00044e0 0x7f5bc0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd37fe700 1 --2- 172.21.15.116:0/2027042021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bc0003730 0x7f5bc0003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd3fff700 1 --2- 172.21.15.116:0/2027042021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bc0004e50 0x7f5bc000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd2ffd700 1 -- 172.21.15.116:0/2027042021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bc0003730 msgr2=0x7f5bc0003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:06.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd2ffd700 1 --2- 172.21.15.116:0/2027042021 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bc0003730 0x7f5bc0003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd2ffd700 1 -- 172.21.15.116:0/2027042021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bc0004e50 msgr2=0x7f5bc000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:06.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd2ffd700 1 --2- 172.21.15.116:0/2027042021 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bc0004e50 0x7f5bc000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd2ffd700 1 -- 172.21.15.116:0/2027042021 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc0003240 con 0x7f5bc00044e0 2020-11-20T12:18:06.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd3fff700 1 -- 172.21.15.116:0/2027042021 reap_dead start 2020-11-20T12:18:06.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd2ffd700 1 --2- 172.21.15.116:0/2027042021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bc00044e0 0x7f5bc0004960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f5bc800bc00 tx=0x7f5bc8006df0).ready entity=mon.2 client_cookie=617eec36a191d781 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:06.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc800b310 con 0x7f5bc00044e0 2020-11-20T12:18:06.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc800ea90 con 0x7f5bc00044e0 2020-11-20T12:18:06.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd27fc700 1 -- 172.21.15.116:0/2027042021 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc800ee10 con 0x7f5bc00044e0 2020-11-20T12:18:06.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd88d3700 1 -- 172.21.15.116:0/2027042021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bc00044e0 msgr2=0x7f5bc0004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd88d3700 1 --2- 172.21.15.116:0/2027042021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bc00044e0 0x7f5bc0004960 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f5bc800bc00 tx=0x7f5bc8006df0).stop 2020-11-20T12:18:06.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd88d3700 1 -- 172.21.15.116:0/2027042021 shutdown_connections 2020-11-20T12:18:06.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd88d3700 1 --2- 172.21.15.116:0/2027042021 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bc00044e0 0x7f5bc0004960 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd88d3700 1 -- 172.21.15.116:0/2027042021 >> 172.21.15.116:0/2027042021 conn(0x7f5bd4067ae0 msgr2=0x7f5bd40c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:06.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd88d3700 1 -- 172.21.15.116:0/2027042021 shutdown_connections 2020-11-20T12:18:06.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.160+0000 7f5bd88d3700 1 -- 172.21.15.116:0/2027042021 wait complete. 2020-11-20T12:18:06.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd88d3700 1 Processor -- start 2020-11-20T12:18:06.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd88d3700 1 -- start start 2020-11-20T12:18:06.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd88d3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4160ab0 0x7f5bd4093100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd88d3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd4094cf0 0x7f5bd40ad050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd88d3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bd4095170 0x7f5bd40ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd88d3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4093dd0 con 0x7f5bd4095170 2020-11-20T12:18:06.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd88d3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4094070 con 0x7f5bd4094cf0 2020-11-20T12:18:06.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd88d3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd4094310 con 0x7f5bd4160ab0 2020-11-20T12:18:06.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bd4095170 0x7f5bd40ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bd4095170 0x7f5bd40ad540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46226/0 (socket says 172.21.15.116:46226) 2020-11-20T12:18:06.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd3fff700 1 -- 172.21.15.116:0/3966282852 learned_addr learned my addr 172.21.15.116:0/3966282852 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:06.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd2ffd700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd4094cf0 0x7f5bd40ad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd37fe700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4160ab0 0x7f5bd4093100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd3fff700 1 -- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4160ab0 msgr2=0x7f5bd4093100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd3fff700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4160ab0 0x7f5bd4093100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd3fff700 1 -- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd4094cf0 msgr2=0x7f5bd40ad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd3fff700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd4094cf0 0x7f5bd40ad050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd3fff700 1 -- 172.21.15.116:0/3966282852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd40945b0 con 0x7f5bd4095170 2020-11-20T12:18:06.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd37fe700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4160ab0 0x7f5bd4093100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:06.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.164+0000 7f5bd3fff700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bd4095170 0x7f5bd40ad540 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f5bd40450c0 tx=0x7f5bd40c5280).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:06.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.168+0000 7f5bb37fe700 1 -- 172.21.15.116:0/3966282852 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc00c9a0 con 0x7f5bd4095170 2020-11-20T12:18:06.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.168+0000 7f5bb37fe700 1 -- 172.21.15.116:0/3966282852 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc00cb00 con 0x7f5bd4095170 2020-11-20T12:18:06.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.168+0000 7f5bd88d3700 1 -- 172.21.15.116:0/3966282852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd4094850 con 0x7f5bd4095170 2020-11-20T12:18:06.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.168+0000 7f5bb37fe700 1 -- 172.21.15.116:0/3966282852 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc00ce80 con 0x7f5bd4095170 2020-11-20T12:18:06.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.168+0000 7f5bd88d3700 1 -- 172.21.15.116:0/3966282852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd40935f0 con 0x7f5bd4095170 2020-11-20T12:18:06.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.168+0000 7f5bd88d3700 1 -- 172.21.15.116:0/3966282852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd405d340 con 0x7f5bd4095170 2020-11-20T12:18:06.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.168+0000 7f5bb37fe700 1 -- 172.21.15.116:0/3966282852 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5bcc014640 con 0x7f5bd4095170 2020-11-20T12:18:06.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.168+0000 7f5bb37fe700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5bac0557c0 0x7f5bac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.168+0000 7f5bb37fe700 1 -- 172.21.15.116:0/3966282852 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5bcc06e4b0 con 0x7f5bd4095170 2020-11-20T12:18:06.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.172+0000 7f5bd37fe700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5bac0557c0 0x7f5bac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.172+0000 7f5bd37fe700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5bac0557c0 0x7f5bac057c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f5bc4000bc0 tx=0x7f5bc4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:06.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.176+0000 7f5bb37fe700 1 -- 172.21.15.116:0/3966282852 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5bcc0442f0 con 0x7f5bd4095170 2020-11-20T12:18:06.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bd88d3700 1 -- 172.21.15.116:0/3966282852 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5bd4061cc0 con 0x7f5bd4095170 2020-11-20T12:18:06.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bb37fe700 1 -- 172.21.15.116:0/3966282852 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5bcc047910 con 0x7f5bd4095170 2020-11-20T12:18:06.335 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:18:06.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bd88d3700 1 -- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5bac0557c0 msgr2=0x7f5bac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bd88d3700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5bac0557c0 0x7f5bac057c80 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f5bc4000bc0 tx=0x7f5bc4009040).stop 2020-11-20T12:18:06.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bd88d3700 1 -- 172.21.15.116:0/3966282852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bd4095170 msgr2=0x7f5bd40ad540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bd88d3700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bd4095170 0x7f5bd40ad540 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f5bd40450c0 tx=0x7f5bd40c5280).stop 2020-11-20T12:18:06.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bd88d3700 1 -- 172.21.15.116:0/3966282852 shutdown_connections 2020-11-20T12:18:06.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bd88d3700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5bac0557c0 0x7f5bac057c80 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bd88d3700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5bd4095170 0x7f5bd40ad540 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bd88d3700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5bd4160ab0 0x7f5bd4093100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bd88d3700 1 --2- 172.21.15.116:0/3966282852 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5bd4094cf0 0x7f5bd40ad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.336+0000 7f5bd88d3700 1 -- 172.21.15.116:0/3966282852 >> 172.21.15.116:0/3966282852 conn(0x7f5bd4067ae0 msgr2=0x7f5bd4060850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:06.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.340+0000 7f5bd88d3700 1 -- 172.21.15.116:0/3966282852 shutdown_connections 2020-11-20T12:18:06.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.340+0000 7f5bd88d3700 1 -- 172.21.15.116:0/3966282852 wait complete. 2020-11-20T12:18:06.367 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:18:06.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.548+0000 7faa79969700 1 Processor -- start 2020-11-20T12:18:06.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.548+0000 7faa79969700 1 -- start start 2020-11-20T12:18:06.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa79969700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7faa74151990 0x7faa740ad020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa79969700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa74151d70 0x7faa740c8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa79969700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa740a7760 0x7faa740a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa79969700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa740ad610 con 0x7faa740a7760 2020-11-20T12:18:06.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa79969700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa740ad8b0 con 0x7faa74151990 2020-11-20T12:18:06.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa79969700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa740adb50 con 0x7faa74151d70 2020-11-20T12:18:06.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa737fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa740a7760 0x7faa740a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa737fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa740a7760 0x7faa740a7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46234/0 (socket says 172.21.15.116:46234) 2020-11-20T12:18:06.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa737fe700 1 -- 172.21.15.116:0/3821458289 learned_addr learned my addr 172.21.15.116:0/3821458289 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:06.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa72ffd700 1 --2- 172.21.15.116:0/3821458289 >> v2:172.21.15.173:3300/0 conn(0x7faa74151990 0x7faa740ad020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa727fc700 1 --2- 172.21.15.116:0/3821458289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa74151d70 0x7faa740c8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa737fe700 1 -- 172.21.15.116:0/3821458289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa74151d70 msgr2=0x7faa740c8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:06.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa737fe700 1 --2- 172.21.15.116:0/3821458289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa74151d70 0x7faa740c8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa737fe700 1 -- 172.21.15.116:0/3821458289 >> v2:172.21.15.173:3300/0 conn(0x7faa74151990 msgr2=0x7faa740ad020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:06.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa737fe700 1 --2- 172.21.15.116:0/3821458289 >> v2:172.21.15.173:3300/0 conn(0x7faa74151990 0x7faa740ad020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa737fe700 1 -- 172.21.15.116:0/3821458289 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa740addf0 con 0x7faa740a7760 2020-11-20T12:18:06.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.552+0000 7faa737fe700 1 --2- 172.21.15.116:0/3821458289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740a7760 0x7faa740a7c10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7faa74081d30 tx=0x7faa6c009290).ready entity=mon.0 client_cookie=ce2586e5cae4146e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:06.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa71ffb700 1 -- 172.21.15.116:0/3821458289 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c00bb10 con 0x7faa740a7760 2020-11-20T12:18:06.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa71ffb700 1 -- 172.21.15.116:0/3821458289 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6c00bc70 con 0x7faa740a7760 2020-11-20T12:18:06.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa71ffb700 1 -- 172.21.15.116:0/3821458289 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6c011400 con 0x7faa740a7760 2020-11-20T12:18:06.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa79969700 1 -- 172.21.15.116:0/3821458289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740a7760 msgr2=0x7faa740a7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa79969700 1 --2- 172.21.15.116:0/3821458289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740a7760 0x7faa740a7c10 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7faa74081d30 tx=0x7faa6c009290).stop 2020-11-20T12:18:06.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa79969700 1 -- 172.21.15.116:0/3821458289 shutdown_connections 2020-11-20T12:18:06.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa79969700 1 --2- 172.21.15.116:0/3821458289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa740a7760 0x7faa740a7c10 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa79969700 1 --2- 172.21.15.116:0/3821458289 >> v2:172.21.15.173:3300/0 conn(0x7faa74151990 0x7faa740ad020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa79969700 1 --2- 172.21.15.116:0/3821458289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa74151d70 0x7faa740c8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa79969700 1 -- 172.21.15.116:0/3821458289 >> 172.21.15.116:0/3821458289 conn(0x7faa74067dd0 msgr2=0x7faa740c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:06.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa79969700 1 -- 172.21.15.116:0/3821458289 shutdown_connections 2020-11-20T12:18:06.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa79969700 1 -- 172.21.15.116:0/3821458289 wait complete. 2020-11-20T12:18:06.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa79969700 1 Processor -- start 2020-11-20T12:18:06.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.556+0000 7faa79969700 1 -- start start 2020-11-20T12:18:06.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa79969700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa740a7760 0x7faa740aa640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa79969700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa74151990 0x7faa7414df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa79969700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa740aab30 0x7faa7414e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa79969700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa741699e0 con 0x7faa74151990 2020-11-20T12:18:06.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa79969700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa74165690 con 0x7faa740a7760 2020-11-20T12:18:06.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa79969700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa74165930 con 0x7faa740aab30 2020-11-20T12:18:06.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa727fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa74151990 0x7faa7414df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa727fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa74151990 0x7faa7414df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46238/0 (socket says 172.21.15.116:46238) 2020-11-20T12:18:06.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa727fc700 1 -- 172.21.15.116:0/3634195409 learned_addr learned my addr 172.21.15.116:0/3634195409 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:06.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa737fe700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa740aab30 0x7faa7414e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa72ffd700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa740a7760 0x7faa740aa640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa727fc700 1 -- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa740aab30 msgr2=0x7faa7414e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa727fc700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa740aab30 0x7faa7414e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa727fc700 1 -- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa740a7760 msgr2=0x7faa740aa640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa727fc700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa740a7760 0x7faa740aa640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa727fc700 1 -- 172.21.15.116:0/3634195409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa74165bd0 con 0x7faa74151990 2020-11-20T12:18:06.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa727fc700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa74151990 0x7faa7414df20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7faa6800eba0 tx=0x7faa68006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:06.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa5affd700 1 -- 172.21.15.116:0/3634195409 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6800b960 con 0x7faa74151990 2020-11-20T12:18:06.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa5affd700 1 -- 172.21.15.116:0/3634195409 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6800bac0 con 0x7faa74151990 2020-11-20T12:18:06.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa79969700 1 -- 172.21.15.116:0/3634195409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa74165e70 con 0x7faa74151990 2020-11-20T12:18:06.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa5affd700 1 -- 172.21.15.116:0/3634195409 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa6800be40 con 0x7faa74151990 2020-11-20T12:18:06.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.560+0000 7faa79969700 1 -- 172.21.15.116:0/3634195409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa741664b0 con 0x7faa74151990 2020-11-20T12:18:06.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.564+0000 7faa79969700 1 -- 172.21.15.116:0/3634195409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa74166000 con 0x7faa74151990 2020-11-20T12:18:06.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.564+0000 7faa5affd700 1 -- 172.21.15.116:0/3634195409 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7faa68012600 con 0x7faa74151990 2020-11-20T12:18:06.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.564+0000 7faa5affd700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa500557c0 0x7faa50057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.564+0000 7faa5affd700 1 -- 172.21.15.116:0/3634195409 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faa6806ea00 con 0x7faa74151990 2020-11-20T12:18:06.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.564+0000 7faa72ffd700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa500557c0 0x7faa50057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.568+0000 7faa72ffd700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa500557c0 0x7faa50057c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7faa740c8110 tx=0x7faa64008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:06.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.572+0000 7faa5affd700 1 -- 172.21.15.116:0/3634195409 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7faa680448a0 con 0x7faa74151990 2020-11-20T12:18:06.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.728+0000 7faa79969700 1 -- 172.21.15.116:0/3634195409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faa7405d340 con 0x7faa74151990 2020-11-20T12:18:06.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa5affd700 1 -- 172.21.15.116:0/3634195409 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7faa68047ec0 con 0x7faa74151990 2020-11-20T12:18:06.730 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:06.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 -- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa500557c0 msgr2=0x7faa50057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa500557c0 0x7faa50057c80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7faa740c8110 tx=0x7faa64008040).stop 2020-11-20T12:18:06.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 -- 172.21.15.116:0/3634195409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa74151990 msgr2=0x7faa7414df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa74151990 0x7faa7414df20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7faa6800eba0 tx=0x7faa68006df0).stop 2020-11-20T12:18:06.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 -- 172.21.15.116:0/3634195409 shutdown_connections 2020-11-20T12:18:06.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa500557c0 0x7faa50057c80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa740aab30 0x7faa7414e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa740a7760 0x7faa740aa640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 --2- 172.21.15.116:0/3634195409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa74151990 0x7faa7414df20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 -- 172.21.15.116:0/3634195409 >> 172.21.15.116:0/3634195409 conn(0x7faa74067dd0 msgr2=0x7faa7414edf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:06.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 -- 172.21.15.116:0/3634195409 shutdown_connections 2020-11-20T12:18:06.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.732+0000 7faa79969700 1 -- 172.21.15.116:0/3634195409 wait complete. 2020-11-20T12:18:06.755 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:18:06.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.936+0000 7f105e398700 1 Processor -- start 2020-11-20T12:18:06.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.936+0000 7f105e398700 1 -- start start 2020-11-20T12:18:06.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.940+0000 7f105e398700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f10580a36d0 0x7f10581536f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.940+0000 7f105e398700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10580ba860 0x7f10580bacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.940+0000 7f105e398700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1058066f90 con 0x7f105814a210 2020-11-20T12:18:06.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.940+0000 7f105e398700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10580c42c0 con 0x7f10580ba860 2020-11-20T12:18:06.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.940+0000 7f105e398700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1058153be0 con 0x7f10580a36d0 2020-11-20T12:18:06.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.940+0000 7f105db97700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10580ba860 0x7f10580bacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.940+0000 7f105d396700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f105814a210 0x7f1058066880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:51908/0 (socket says 172.21.15.116:51908) 2020-11-20T12:18:06.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.940+0000 7f105d396700 1 -- 172.21.15.116:0/3118465522 learned_addr learned my addr 172.21.15.116:0/3118465522 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:06.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.940+0000 7f105cb95700 1 --2- 172.21.15.116:0/3118465522 >> v2:172.21.15.173:3300/0 conn(0x7f10580a36d0 0x7f10581536f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.940+0000 7f1047fff700 1 -- 172.21.15.116:0/3118465522 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3825913032 0 0) 0x7f1058066f90 con 0x7f105814a210 2020-11-20T12:18:06.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f1047fff700 1 -- 172.21.15.116:0/3118465522 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1040003240 con 0x7f105814a210 2020-11-20T12:18:06.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105db97700 1 -- 172.21.15.116:0/3118465522 >> v2:172.21.15.173:3300/0 conn(0x7f10580a36d0 msgr2=0x7f10581536f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:06.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105db97700 1 --2- 172.21.15.116:0/3118465522 >> v2:172.21.15.173:3300/0 conn(0x7f10580a36d0 0x7f10581536f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105db97700 1 -- 172.21.15.116:0/3118465522 >> v1:172.21.15.173:6789/0 conn(0x7f105814a210 legacy=0x7f1058066880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105db97700 1 -- 172.21.15.116:0/3118465522 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1058153e00 con 0x7f10580ba860 2020-11-20T12:18:06.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105db97700 1 --2- 172.21.15.116:0/3118465522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10580ba860 0x7f10580bacb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f10580c4e50 tx=0x7f1050006df0).ready entity=mon.0 client_cookie=c8bead0b99ae04a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:06.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f1047fff700 1 -- 172.21.15.116:0/3118465522 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050009640 con 0x7f10580ba860 2020-11-20T12:18:06.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f1047fff700 1 -- 172.21.15.116:0/3118465522 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10500097a0 con 0x7f10580ba860 2020-11-20T12:18:06.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f1047fff700 1 -- 172.21.15.116:0/3118465522 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1050009b20 con 0x7f10580ba860 2020-11-20T12:18:06.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105e398700 1 -- 172.21.15.116:0/3118465522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10580ba860 msgr2=0x7f10580bacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105e398700 1 --2- 172.21.15.116:0/3118465522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10580ba860 0x7f10580bacb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f10580c4e50 tx=0x7f1050006df0).stop 2020-11-20T12:18:06.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105e398700 1 -- 172.21.15.116:0/3118465522 shutdown_connections 2020-11-20T12:18:06.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105e398700 1 --2- 172.21.15.116:0/3118465522 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10580ba860 0x7f10580bacb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105e398700 1 --2- 172.21.15.116:0/3118465522 >> v2:172.21.15.173:3300/0 conn(0x7f10580a36d0 0x7f10581536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:06.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105e398700 1 -- 172.21.15.116:0/3118465522 >> 172.21.15.116:0/3118465522 conn(0x7f1058060690 msgr2=0x7f10580c4440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:06.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105e398700 1 -- 172.21.15.116:0/3118465522 shutdown_connections 2020-11-20T12:18:06.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.944+0000 7f105e398700 1 -- 172.21.15.116:0/3118465522 wait complete. 2020-11-20T12:18:06.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.948+0000 7f105e398700 1 Processor -- start 2020-11-20T12:18:06.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.948+0000 7f105e398700 1 -- start start 2020-11-20T12:18:06.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105e398700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10580a36d0 0x7f10580a4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105e398700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10580ba860 0x7f10580bb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105e398700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f105814a210 0x7f10580c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105e398700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1058063030 con 0x7f10580ba860 2020-11-20T12:18:06.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105e398700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10580bbf10 con 0x7f10580a36d0 2020-11-20T12:18:06.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105e398700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10580bb9e0 con 0x7f105814a210 2020-11-20T12:18:06.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105cb95700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10580ba860 0x7f10580bb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105cb95700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10580ba860 0x7f10580bb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46252/0 (socket says 172.21.15.116:46252) 2020-11-20T12:18:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105cb95700 1 -- 172.21.15.116:0/4281442089 learned_addr learned my addr 172.21.15.116:0/4281442089 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105cb95700 1 -- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f105814a210 msgr2=0x7f10580c04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105cb95700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f105814a210 0x7f10580c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105cb95700 1 -- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10580a36d0 msgr2=0x7f10580a4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:06.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105cb95700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10580a36d0 0x7f10580a4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105cb95700 1 -- 172.21.15.116:0/4281442089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10580bbc70 con 0x7f10580ba860 2020-11-20T12:18:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105cb95700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10580ba860 0x7f10580bb4f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f104800ebd0 tx=0x7f1048006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f10457fa700 1 -- 172.21.15.116:0/4281442089 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104800b9a0 con 0x7f10580ba860 2020-11-20T12:18:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f10457fa700 1 -- 172.21.15.116:0/4281442089 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104800bb00 con 0x7f10580ba860 2020-11-20T12:18:06.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105e398700 1 -- 172.21.15.116:0/4281442089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1058062650 con 0x7f10580ba860 2020-11-20T12:18:06.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f10457fa700 1 -- 172.21.15.116:0/4281442089 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104800be80 con 0x7f10580ba860 2020-11-20T12:18:06.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.952+0000 7f105e398700 1 -- 172.21.15.116:0/4281442089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1058062b90 con 0x7f10580ba860 2020-11-20T12:18:06.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.956+0000 7f105e398700 1 -- 172.21.15.116:0/4281442089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105805d340 con 0x7f10580ba860 2020-11-20T12:18:06.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.956+0000 7f10457fa700 1 -- 172.21.15.116:0/4281442089 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1048012600 con 0x7f10580ba860 2020-11-20T12:18:06.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.956+0000 7f10457fa700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10300557f0 0x7f1030057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.956+0000 7f10457fa700 1 -- 172.21.15.116:0/4281442089 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f104806eab0 con 0x7f10580ba860 2020-11-20T12:18:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.956+0000 7f105d396700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10300557f0 0x7f1030057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:06.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.960+0000 7f105d396700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10300557f0 0x7f1030057cb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f1054005920 tx=0x7f1054007a20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:06.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:06.960+0000 7f10457fa700 1 -- 172.21.15.116:0/4281442089 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f10480448d0 con 0x7f10580ba860 2020-11-20T12:18:07.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.120+0000 7f105e398700 1 -- 172.21.15.116:0/4281442089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1058060be0 con 0x7f10580ba860 2020-11-20T12:18:07.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.120+0000 7f10457fa700 1 -- 172.21.15.116:0/4281442089 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1048047ef0 con 0x7f10580ba860 2020-11-20T12:18:07.120 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:07.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 -- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10300557f0 msgr2=0x7f1030057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10300557f0 0x7f1030057cb0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f1054005920 tx=0x7f1054007a20).stop 2020-11-20T12:18:07.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 -- 172.21.15.116:0/4281442089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10580ba860 msgr2=0x7f10580bb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10580ba860 0x7f10580bb4f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f104800ebd0 tx=0x7f1048006df0).stop 2020-11-20T12:18:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 -- 172.21.15.116:0/4281442089 shutdown_connections 2020-11-20T12:18:07.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10300557f0 0x7f1030057cb0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f105814a210 0x7f10580c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f10580a36d0 0x7f10580a4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 --2- 172.21.15.116:0/4281442089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10580ba860 0x7f10580bb4f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 -- 172.21.15.116:0/4281442089 >> 172.21.15.116:0/4281442089 conn(0x7f1058060690 msgr2=0x7f1058061510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:07.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 -- 172.21.15.116:0/4281442089 shutdown_connections 2020-11-20T12:18:07.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.124+0000 7f105e398700 1 -- 172.21.15.116:0/4281442089 wait complete. 2020-11-20T12:18:07.147 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:18:07.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.348+0000 7f5acdf51700 1 Processor -- start 2020-11-20T12:18:07.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.348+0000 7f5acdf51700 1 -- start start 2020-11-20T12:18:07.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.348+0000 7f5acdf51700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac80a79f0 0x7f5ac80a7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.348+0000 7f5acdf51700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5ac80bda80 0x7f5ac80bdf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.348+0000 7f5acdf51700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac8155950 con 0x7f5ac8151d00 2020-11-20T12:18:07.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.348+0000 7f5acdf51700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac8155b80 con 0x7f5ac80bda80 2020-11-20T12:18:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.348+0000 7f5acdf51700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac8155e20 con 0x7f5ac80a79f0 2020-11-20T12:18:07.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.352+0000 7f5ac7fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5ac80bda80 0x7f5ac80bdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.352+0000 7f5ac7fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5ac80bda80 0x7f5ac80bdf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:45314/0 (socket says 172.21.15.116:45314) 2020-11-20T12:18:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.352+0000 7f5ac7fff700 1 -- 172.21.15.116:0/2839181579 learned_addr learned my addr 172.21.15.116:0/2839181579 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:07.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.352+0000 7f5ac77fe700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f5ac8151d00 0x7f5ac80c3ea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:51922/0 (socket says 172.21.15.116:51922) 2020-11-20T12:18:07.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.352+0000 7f5ac6ffd700 1 --2- 172.21.15.116:0/2839181579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac80a79f0 0x7f5ac80a7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.352+0000 7f5ac7fff700 1 -- 172.21.15.116:0/2839181579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac80a79f0 msgr2=0x7f5ac80a7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:07.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.352+0000 7f5ac7fff700 1 --2- 172.21.15.116:0/2839181579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac80a79f0 0x7f5ac80a7e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.352+0000 7f5ac7fff700 1 -- 172.21.15.116:0/2839181579 >> v1:172.21.15.173:6789/0 conn(0x7f5ac8151d00 legacy=0x7f5ac80c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.352+0000 7f5ac7fff700 1 -- 172.21.15.116:0/2839181579 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac81560c0 con 0x7f5ac80bda80 2020-11-20T12:18:07.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.352+0000 7f5ac6ffd700 1 --2- 172.21.15.116:0/2839181579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac80a79f0 0x7f5ac80a7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:07.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.352+0000 7f5ac7fff700 1 --2- 172.21.15.116:0/2839181579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac80bda80 0x7f5ac80bdf10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f5ac0009d50 tx=0x7f5ac0009290).ready entity=mon.1 client_cookie=e428d83a5a77eaef server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:07.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.360+0000 7f5ac67fc700 1 -- 172.21.15.116:0/2839181579 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac000bbc0 con 0x7f5ac80bda80 2020-11-20T12:18:07.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.360+0000 7f5ac67fc700 1 -- 172.21.15.116:0/2839181579 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ac000bd20 con 0x7f5ac80bda80 2020-11-20T12:18:07.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.360+0000 7f5ac67fc700 1 -- 172.21.15.116:0/2839181579 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0011610 con 0x7f5ac80bda80 2020-11-20T12:18:07.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.360+0000 7f5acdf51700 1 -- 172.21.15.116:0/2839181579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac80bda80 msgr2=0x7f5ac80bdf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.360+0000 7f5acdf51700 1 --2- 172.21.15.116:0/2839181579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac80bda80 0x7f5ac80bdf10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f5ac0009d50 tx=0x7f5ac0009290).stop 2020-11-20T12:18:07.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.360+0000 7f5acdf51700 1 -- 172.21.15.116:0/2839181579 shutdown_connections 2020-11-20T12:18:07.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.360+0000 7f5acdf51700 1 --2- 172.21.15.116:0/2839181579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac80bda80 0x7f5ac80bdf10 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.360+0000 7f5acdf51700 1 --2- 172.21.15.116:0/2839181579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac80a79f0 0x7f5ac80a7e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.360+0000 7f5acdf51700 1 -- 172.21.15.116:0/2839181579 >> 172.21.15.116:0/2839181579 conn(0x7f5ac80c9510 msgr2=0x7f5ac80c54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:07.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.360+0000 7f5acdf51700 1 -- 172.21.15.116:0/2839181579 shutdown_connections 2020-11-20T12:18:07.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.360+0000 7f5acdf51700 1 -- 172.21.15.116:0/2839181579 wait complete. 2020-11-20T12:18:07.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.364+0000 7f5acdf51700 1 Processor -- start 2020-11-20T12:18:07.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.364+0000 7f5acdf51700 1 -- start start 2020-11-20T12:18:07.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.364+0000 7f5acdf51700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac80a79f0 0x7f5ac80ab9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.364+0000 7f5acdf51700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac80bda80 0x7f5ac81520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.364+0000 7f5acdf51700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ac8151d00 0x7f5ac814e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.364+0000 7f5acdf51700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac8152a60 con 0x7f5ac8151d00 2020-11-20T12:18:07.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.364+0000 7f5acdf51700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac8152d00 con 0x7f5ac80bda80 2020-11-20T12:18:07.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.364+0000 7f5acdf51700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac8152fa0 con 0x7f5ac80a79f0 2020-11-20T12:18:07.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ac8151d00 0x7f5ac814e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ac8151d00 0x7f5ac814e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46268/0 (socket says 172.21.15.116:46268) 2020-11-20T12:18:07.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac7fff700 1 -- 172.21.15.116:0/460517054 learned_addr learned my addr 172.21.15.116:0/460517054 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:07.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac6ffd700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac80bda80 0x7f5ac81520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac77fe700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac80a79f0 0x7f5ac80ab9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac6ffd700 1 -- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac80a79f0 msgr2=0x7f5ac80ab9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac6ffd700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac80a79f0 0x7f5ac80ab9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac6ffd700 1 -- 172.21.15.116:0/460517054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ac8151d00 msgr2=0x7f5ac814e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac6ffd700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ac8151d00 0x7f5ac814e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac6ffd700 1 -- 172.21.15.116:0/460517054 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac8153240 con 0x7f5ac80bda80 2020-11-20T12:18:07.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac7fff700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ac8151d00 0x7f5ac814e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:07.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5ac6ffd700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac80bda80 0x7f5ac81520e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f5abc00ec70 tx=0x7f5abc006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:07.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5aa77fe700 1 -- 172.21.15.116:0/460517054 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc00ca20 con 0x7f5ac80bda80 2020-11-20T12:18:07.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5aa77fe700 1 -- 172.21.15.116:0/460517054 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5abc00cb80 con 0x7f5ac80bda80 2020-11-20T12:18:07.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5aa77fe700 1 -- 172.21.15.116:0/460517054 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5abc0123f0 con 0x7f5ac80bda80 2020-11-20T12:18:07.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5acdf51700 1 -- 172.21.15.116:0/460517054 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac81534e0 con 0x7f5ac80bda80 2020-11-20T12:18:07.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.368+0000 7f5acdf51700 1 -- 172.21.15.116:0/460517054 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac80ab160 con 0x7f5ac80bda80 2020-11-20T12:18:07.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.372+0000 7f5acdf51700 1 -- 172.21.15.116:0/460517054 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac8153670 con 0x7f5ac80bda80 2020-11-20T12:18:07.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.372+0000 7f5aa77fe700 1 -- 172.21.15.116:0/460517054 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5abc0127a0 con 0x7f5ac80bda80 2020-11-20T12:18:07.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.372+0000 7f5aa77fe700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5aa00557c0 0x7f5aa0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.372+0000 7f5aa77fe700 1 -- 172.21.15.116:0/460517054 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5abc06ec20 con 0x7f5ac80bda80 2020-11-20T12:18:07.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.372+0000 7f5ac77fe700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5aa00557c0 0x7f5aa0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.372+0000 7f5ac77fe700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5aa00557c0 0x7f5aa0057c80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f5ab80045d0 tx=0x7f5ab8007b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:07.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.376+0000 7f5aa77fe700 1 -- 172.21.15.116:0/460517054 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5abc044a40 con 0x7f5ac80bda80 2020-11-20T12:18:07.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.536+0000 7f5acdf51700 1 -- 172.21.15.116:0/460517054 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ac8068d50 con 0x7f5ac80bda80 2020-11-20T12:18:07.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.536+0000 7f5aa77fe700 1 -- 172.21.15.116:0/460517054 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5abc048060 con 0x7f5ac80bda80 2020-11-20T12:18:07.540 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:07.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 -- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5aa00557c0 msgr2=0x7f5aa0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5aa00557c0 0x7f5aa0057c80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f5ab80045d0 tx=0x7f5ab8007b80).stop 2020-11-20T12:18:07.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 -- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac80bda80 msgr2=0x7f5ac81520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac80bda80 0x7f5ac81520e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f5abc00ec70 tx=0x7f5abc006df0).stop 2020-11-20T12:18:07.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 -- 172.21.15.116:0/460517054 shutdown_connections 2020-11-20T12:18:07.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5aa00557c0 0x7f5aa0057c80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5ac8151d00 0x7f5ac814e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5ac80a79f0 0x7f5ac80ab9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 --2- 172.21.15.116:0/460517054 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5ac80bda80 0x7f5ac81520e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 -- 172.21.15.116:0/460517054 >> 172.21.15.116:0/460517054 conn(0x7f5ac80c9510 msgr2=0x7f5ac80bf2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:07.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 -- 172.21.15.116:0/460517054 shutdown_connections 2020-11-20T12:18:07.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.540+0000 7f5acdf51700 1 -- 172.21.15.116:0/460517054 wait complete. 2020-11-20T12:18:07.574 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:18:07.582 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:18:07.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.764+0000 7fc445c10700 1 Processor -- start 2020-11-20T12:18:07.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.768+0000 7fc445c10700 1 -- start start 2020-11-20T12:18:07.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.768+0000 7fc445c10700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc44014e540 0x7fc44014e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.768+0000 7fc445c10700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc4400a7d50 0x7fc44005bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.768+0000 7fc445c10700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4400a5af0 con 0x7fc4400a6db0 2020-11-20T12:18:07.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.768+0000 7fc445c10700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc44005c260 con 0x7fc44014e540 2020-11-20T12:18:07.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.768+0000 7fc445c10700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4400a8380 con 0x7fc4400a7d50 2020-11-20T12:18:07.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.768+0000 7fc44540f700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc4400a6db0 0x7fc440152400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55708/0 (socket says 172.21.15.116:55708) 2020-11-20T12:18:07.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.768+0000 7fc444c0e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc44014e540 0x7fc44014e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.768+0000 7fc44540f700 1 -- 172.21.15.116:0/2773082966 learned_addr learned my addr 172.21.15.116:0/2773082966 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:07.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.768+0000 7fc437fff700 1 --2- 172.21.15.116:0/2773082966 >> v2:172.21.15.173:3300/0 conn(0x7fc4400a7d50 0x7fc44005bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc4377fe700 1 -- 172.21.15.116:0/2773082966 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844536507 0 0) 0x7fc4400a5af0 con 0x7fc4400a6db0 2020-11-20T12:18:07.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc4377fe700 1 -- 172.21.15.116:0/2773082966 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc42c003240 con 0x7fc4400a6db0 2020-11-20T12:18:07.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc444c0e700 1 -- 172.21.15.116:0/2773082966 >> v2:172.21.15.173:3300/0 conn(0x7fc4400a7d50 msgr2=0x7fc44005bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:07.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc444c0e700 1 --2- 172.21.15.116:0/2773082966 >> v2:172.21.15.173:3300/0 conn(0x7fc4400a7d50 0x7fc44005bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc444c0e700 1 -- 172.21.15.116:0/2773082966 >> v1:172.21.15.116:6789/0 conn(0x7fc4400a6db0 legacy=0x7fc440152400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc444c0e700 1 -- 172.21.15.116:0/2773082966 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4400a5310 con 0x7fc44014e540 2020-11-20T12:18:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc444c0e700 1 --2- 172.21.15.116:0/2773082966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc44014e540 0x7fc44014e920 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fc440063c00 tx=0x7fc43c006df0).ready entity=mon.0 client_cookie=398601b6339da08f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc4377fe700 1 -- 172.21.15.116:0/2773082966 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c009640 con 0x7fc44014e540 2020-11-20T12:18:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc4377fe700 1 -- 172.21.15.116:0/2773082966 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43c0097a0 con 0x7fc44014e540 2020-11-20T12:18:07.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc4377fe700 1 -- 172.21.15.116:0/2773082966 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43c009b20 con 0x7fc44014e540 2020-11-20T12:18:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc445c10700 1 -- 172.21.15.116:0/2773082966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc44014e540 msgr2=0x7fc44014e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc445c10700 1 --2- 172.21.15.116:0/2773082966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc44014e540 0x7fc44014e920 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7fc440063c00 tx=0x7fc43c006df0).stop 2020-11-20T12:18:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc445c10700 1 -- 172.21.15.116:0/2773082966 shutdown_connections 2020-11-20T12:18:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc445c10700 1 --2- 172.21.15.116:0/2773082966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc44014e540 0x7fc44014e920 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc445c10700 1 --2- 172.21.15.116:0/2773082966 >> v2:172.21.15.173:3300/0 conn(0x7fc4400a7d50 0x7fc44005bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc445c10700 1 -- 172.21.15.116:0/2773082966 >> 172.21.15.116:0/2773082966 conn(0x7fc4400c2200 msgr2=0x7fc44005fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc445c10700 1 -- 172.21.15.116:0/2773082966 shutdown_connections 2020-11-20T12:18:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.772+0000 7fc445c10700 1 -- 172.21.15.116:0/2773082966 wait complete. 2020-11-20T12:18:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc445c10700 1 Processor -- start 2020-11-20T12:18:07.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc445c10700 1 -- start start 2020-11-20T12:18:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc445c10700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4400a6db0 0x7fc440152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc445c10700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4400a7d50 0x7fc440151430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc445c10700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc44014e540 0x7fc4400c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc445c10700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4401614d0 con 0x7fc4400a7d50 2020-11-20T12:18:07.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc445c10700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc440161770 con 0x7fc44014e540 2020-11-20T12:18:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc445c10700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc440161a10 con 0x7fc4400a6db0 2020-11-20T12:18:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc437fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4400a7d50 0x7fc440151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc437fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4400a7d50 0x7fc440151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46282/0 (socket says 172.21.15.116:46282) 2020-11-20T12:18:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc437fff700 1 -- 172.21.15.116:0/3680703154 learned_addr learned my addr 172.21.15.116:0/3680703154 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:07.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc444c0e700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4400a6db0 0x7fc440152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc44540f700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc44014e540 0x7fc4400c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc437fff700 1 -- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4400a6db0 msgr2=0x7fc440152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc437fff700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4400a6db0 0x7fc440152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc437fff700 1 -- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc44014e540 msgr2=0x7fc4400c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:07.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc437fff700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc44014e540 0x7fc4400c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:07.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc437fff700 1 -- 172.21.15.116:0/3680703154 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc440161cb0 con 0x7fc4400a7d50 2020-11-20T12:18:07.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.776+0000 7fc437fff700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4400a7d50 0x7fc440151430 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fc43000eba0 tx=0x7fc430006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:07.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.780+0000 7fc434ff9700 1 -- 172.21.15.116:0/3680703154 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000c8e0 con 0x7fc4400a7d50 2020-11-20T12:18:07.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.780+0000 7fc434ff9700 1 -- 172.21.15.116:0/3680703154 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43000ca40 con 0x7fc4400a7d50 2020-11-20T12:18:07.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.780+0000 7fc445c10700 1 -- 172.21.15.116:0/3680703154 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc440151920 con 0x7fc4400a7d50 2020-11-20T12:18:07.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.780+0000 7fc434ff9700 1 -- 172.21.15.116:0/3680703154 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000cdc0 con 0x7fc4400a7d50 2020-11-20T12:18:07.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.780+0000 7fc445c10700 1 -- 172.21.15.116:0/3680703154 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4400c4650 con 0x7fc4400a7d50 2020-11-20T12:18:07.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.780+0000 7fc445c10700 1 -- 172.21.15.116:0/3680703154 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4400c48d0 con 0x7fc4400a7d50 2020-11-20T12:18:07.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.780+0000 7fc434ff9700 1 -- 172.21.15.116:0/3680703154 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc430012600 con 0x7fc4400a7d50 2020-11-20T12:18:07.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.780+0000 7fc434ff9700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4180557f0 0x7fc418057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:07.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.780+0000 7fc434ff9700 1 -- 172.21.15.116:0/3680703154 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc43006e990 con 0x7fc4400a7d50 2020-11-20T12:18:07.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.784+0000 7fc444c0e700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4180557f0 0x7fc418057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:07.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.784+0000 7fc444c0e700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4180557f0 0x7fc418057cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fc43c006e30 tx=0x7fc43c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:07.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.788+0000 7fc434ff9700 1 -- 172.21.15.116:0/3680703154 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc4300447d0 con 0x7fc4400a7d50 2020-11-20T12:18:07.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.996+0000 7fc445c10700 1 -- 172.21.15.116:0/3680703154 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc440161e40 con 0x7fc4400a7d50 2020-11-20T12:18:07.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:07.996+0000 7fc434ff9700 1 -- 172.21.15.116:0/3680703154 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+2849 (secure 0 0 0) 0x7fc430047df0 con 0x7fc4400a7d50 2020-11-20T12:18:07.997 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:07.998 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1096724826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1096724826},{"type":"v1","addr":"172.21.15.173:6837","nonce":1096724826}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/1589443098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1589443098},{"type":"v1","addr":"172.21.15.173:6835","nonce":1589443098}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/966300328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":966300328},{"type":"v1","addr":"172.21.15.116:6833","nonce":966300328}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:17:40.163408+0000","modified":"2020-11-20T12:17:42.171423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5493":{"gid":5493,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/2908037349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2908037349},{"type":"v1","addr":"172.21.15.116:6835","nonce":2908037349}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-20T12:18:08.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 -- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4180557f0 msgr2=0x7fc418057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4180557f0 0x7fc418057cb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7fc43c006e30 tx=0x7fc43c015000).stop 2020-11-20T12:18:08.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 -- 172.21.15.116:0/3680703154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4400a7d50 msgr2=0x7fc440151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:08.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4400a7d50 0x7fc440151430 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7fc43000eba0 tx=0x7fc430006df0).stop 2020-11-20T12:18:08.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 -- 172.21.15.116:0/3680703154 shutdown_connections 2020-11-20T12:18:08.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4180557f0 0x7fc418057cb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc44014e540 0x7fc4400c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4400a6db0 0x7fc440152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 --2- 172.21.15.116:0/3680703154 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4400a7d50 0x7fc440151430 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:08.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 -- 172.21.15.116:0/3680703154 >> 172.21.15.116:0/3680703154 conn(0x7fc4400c2200 msgr2=0x7fc440060a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:08.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 -- 172.21.15.116:0/3680703154 shutdown_connections 2020-11-20T12:18:08.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:08.000+0000 7fc445c10700 1 -- 172.21.15.116:0/3680703154 wait complete. 2020-11-20T12:18:08.006 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 20 2020-11-20T12:18:08.443 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:17:40.163408+0000', 'modified': '2020-11-20T12:17:42.171423+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5493}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5493': {'gid': 5493, 'name': 'a', 'rank': 0, 'incarnation': 18, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/2908037349', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 2908037349}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 2908037349}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:18:08.444 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:18:08.444 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:18:08.639 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":5493,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":28.456602139000001,"mdsmap_epoch":20,"osdmap_epoch":45,"osdmap_epoch_barrier":45,"uptime":33.974252309999997} 2020-11-20T12:18:08.639 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 5493, "mdsmap_epoch": 20, "osdmap_epoch": 45, "osdmap_epoch_barrier": 45, "rank_uptime": 28.456602139, "state": "up:active", "uptime": 33.97425231, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:18:08.640 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:18:08.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:08.641 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.2 2020-11-20T12:18:08.659 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:18:08.660 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/volumes/grpid/volid /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=21501,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:08.729 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=21501,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:08.730 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=21501,norbytes" will pass to kernel. 2020-11-20T12:18:08.757 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=21501,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:08.758 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=21501,norbytes" will pass to kernel. 2020-11-20T12:18:08.764 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:18:08.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:08.765 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2020-11-20T12:18:08.801 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:18:08.802 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:18:08.842 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:18:08.887 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:18:08.887 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-20T12:18:08.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:18:08.888 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-20T12:18:08.970 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:18:08.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:08.971 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:18:08.992 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.21501... 2020-11-20T12:18:08.993 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:18:08.993 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.2 2020-11-20T12:18:09.078 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:18:09.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:09.079 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T12:18:09.097 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2020-11-20T12:18:09.098 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:18:09.098 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.3 2020-11-20T12:18:09.175 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:18:09.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:09.176 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T12:18:09.196 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:09.203 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:09.266 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:09.266 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:09.266 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:09.267 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:09.267 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.267 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:09.267 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.268 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:09.268 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.268 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:09.268 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:09.268 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.269 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:09.269 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:09.270 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.270 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:09.270 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:09.271 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.271 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:09.271 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:09.271 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.272 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:09.272 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.272 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:09.272 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:09.272 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:09.273 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:09.273 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:09.273 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:09.273 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:09.274 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:09.274 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:09.274 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:09.274 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.274 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:09.275 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.275 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:09.275 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.275 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:09.276 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.276 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:09.276 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:09.280 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:09.280 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:09.280 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:09.280 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:09.281 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.281 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:09.281 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:09.281 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.282 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:09.282 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:09.282 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.283 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:09.283 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:09.283 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.283 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:09.284 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:09.284 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:09.284 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.284 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:09.285 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:09.285 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.285 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:09.285 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.286 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:09.286 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.286 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:09.286 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.287 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:09.287 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:09.287 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.287 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:09.288 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:09.288 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:09.288 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:09.288 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:09.288 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:09.289 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:09.289 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:09.289 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:09.289 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:09.290 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:09.290 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:09.290 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:09.290 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:09.290 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:09.291 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:09.291 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:09.291 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:09.291 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:09.291 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:09.292 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.292 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:09.292 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.292 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:09.292 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:09.293 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:09.293 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.293 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.293 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:09.294 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.294 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:09.294 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.294 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:09.294 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:09.295 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:09.295 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:09.295 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:09.295 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.295 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:09.296 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:09.296 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.296 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:09.296 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.296 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:09.297 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:09.298 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:09.298 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:09.299 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:09.299 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:09.299 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:09.299 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:09.299 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.300 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:09.300 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.300 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.300 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:09.301 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:09.301 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:09.301 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:09.301 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:09.301 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:09.302 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:09.302 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:09.302 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:09.302 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.303 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:09.303 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:09.303 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:09.304 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:09.304 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:09.304 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:09.304 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:09.304 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:09.305 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:09.305 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:09.305 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.306 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.306 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:09.306 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:09.306 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:09.306 INFO:teuthology.orchestra.run.smithi116.stdout: client quota = True 2020-11-20T12:18:09.307 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:09.307 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.21501.keyring 2020-11-20T12:18:09.309 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:09.309 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:09.309 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:09.310 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:09.310 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.310 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:09.310 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.310 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:09.311 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.311 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:09.311 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:09.311 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.312 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:09.312 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:09.312 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.312 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:09.312 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:09.313 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.313 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:09.313 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:09.313 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.313 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:09.314 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.314 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:09.314 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:09.316 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:09.316 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:09.316 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:09.317 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:09.317 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:09.317 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:09.318 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:09.318 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:09.318 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.318 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:09.319 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.319 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:09.319 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.320 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:09.320 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.320 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:09.320 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:09.321 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:09.322 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.322 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:09.322 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:09.323 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:09.324 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:09.325 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:09.326 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.327 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.328 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:09.329 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:09.330 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:09.331 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:09.332 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:09.333 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:09.334 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:09.335 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.336 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:09.336 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:09.337 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:09.338 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.339 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.339 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:09.340 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:09.341 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:09.342 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.343 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:09.343 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:09.344 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:09.345 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.346 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.346 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:09.346 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:09.347 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:09.347 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:09.347 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:09.348 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:09.348 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:09.348 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:09.348 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:09.349 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:09.350 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:09.351 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:09.351 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:09.351 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:09.352 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.353 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:09.353 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:09.353 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:18:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: client quota = True 2020-11-20T12:18:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:09.354 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.21501.keyring 2020-11-20T12:18:09.356 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:09.363 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:09.431 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:09.431 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:09.431 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:09.431 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:09.432 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.432 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:09.432 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.432 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:09.432 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.433 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:09.433 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:09.433 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.433 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:09.434 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:09.434 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.434 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:09.434 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:09.434 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.435 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:09.435 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:09.435 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.435 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:09.435 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.436 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:09.436 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:09.436 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:09.436 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:09.436 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:09.437 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:09.437 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:09.437 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:09.437 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:09.437 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:09.438 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.438 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:09.438 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.438 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:09.439 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:09.439 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:09.439 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:09.440 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:09.440 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:09.440 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:09.440 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:09.440 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.441 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:09.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:09.441 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:09.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:09.442 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.442 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:09.442 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:09.442 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.443 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:09.445 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:09.445 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:09.445 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.446 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:09.446 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:09.446 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.446 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:09.446 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.447 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:09.447 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.447 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:09.447 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.448 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:09.448 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:09.448 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.448 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:09.448 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:09.449 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:09.449 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:09.449 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:09.450 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:09.450 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:09.450 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:09.450 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:09.450 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:09.451 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:09.451 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:09.451 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:09.451 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:09.451 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:09.452 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:09.452 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:09.452 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:09.452 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:09.453 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:09.453 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.453 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:09.453 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.454 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:09.454 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:09.454 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:09.454 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.454 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.455 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:09.455 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.455 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:09.455 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.455 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:09.456 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:09.456 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:09.456 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:09.456 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:09.456 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.457 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:09.457 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:09.457 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.457 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:09.458 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.458 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:09.458 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:09.458 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:09.458 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:09.459 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:09.459 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:09.459 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:09.459 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:09.459 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.460 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:09.460 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.460 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.460 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:09.460 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:09.461 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:09.461 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:09.461 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:09.461 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:09.461 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:09.462 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:09.462 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:09.462 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.462 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:09.462 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:09.463 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:09.463 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:09.463 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:09.463 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:09.463 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:09.464 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:09.464 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:09.464 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:09.464 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.464 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.465 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:09.465 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:09.465 INFO:teuthology.orchestra.run.smithi116.stdout: client quota = True 2020-11-20T12:18:09.465 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:09.465 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.21501.keyring 2020-11-20T12:18:09.469 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:09.469 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:09.470 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:09.470 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:09.470 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.470 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:09.470 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.471 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:09.471 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.471 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:09.471 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:09.472 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.472 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:09.472 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:09.472 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.472 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:09.473 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:09.473 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.473 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:09.473 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:09.473 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.474 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:09.474 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.474 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:09.474 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:09.474 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:09.475 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:09.475 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:09.475 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:09.475 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:09.475 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:09.476 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:09.476 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:09.476 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.476 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:09.476 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.477 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:09.477 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.478 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:09.478 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.478 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:09.478 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:09.478 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:09.479 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:09.479 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:09.479 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:09.479 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.479 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:09.480 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:09.480 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.480 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:09.480 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:09.480 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.480 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:09.481 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:09.481 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.481 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:09.481 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:09.482 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:09.482 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.482 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:09.482 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:09.482 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.483 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:09.483 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.483 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:09.483 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.483 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:09.484 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.484 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:09.484 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:09.484 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.484 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:09.485 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:09.485 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:09.485 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:09.485 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:09.485 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:09.486 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:09.486 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:09.486 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:09.486 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:09.486 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:09.487 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:09.487 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:09.487 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:09.487 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:09.487 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:09.488 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:09.488 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:09.488 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:09.488 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:09.488 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.489 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:09.489 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.489 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:09.489 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:09.489 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:09.490 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.490 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:09.491 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.491 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:09.491 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:09.491 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:09.491 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:09.492 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:09.492 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.492 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:09.492 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:09.492 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.493 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:09.493 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.493 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:09.493 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:09.493 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:09.494 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:09.494 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:09.494 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:09.494 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:09.495 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:09.495 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.495 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:09.495 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.495 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.495 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:09.496 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:09.496 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:09.496 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:09.496 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:09.496 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:09.497 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:09.497 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:09.497 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:09.497 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.497 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:09.498 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:09.498 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:09.498 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:09.498 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:09.499 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:09.499 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:09.499 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:09.499 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:09.499 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:09.500 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.500 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.500 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:09.500 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:18:09.500 INFO:teuthology.orchestra.run.smithi173.stdout: client quota = True 2020-11-20T12:18:09.501 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:09.501 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.21501.keyring 2020-11-20T12:18:09.502 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:09.509 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:09.573 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:09.573 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:09.574 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:09.574 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:09.574 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.574 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:09.574 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.575 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:09.575 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.575 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:09.575 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:09.575 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.576 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:09.576 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:09.576 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.576 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:09.577 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:09.577 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.577 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:09.577 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:09.577 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.578 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:09.578 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.578 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:09.578 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:09.579 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:09.579 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:09.579 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:09.579 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:09.580 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:09.580 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:09.580 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:09.580 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:09.580 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.581 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:09.581 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.581 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:09.581 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.582 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:09.582 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.582 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:09.582 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:09.583 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:09.583 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:09.583 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:09.583 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:09.583 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.584 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:09.584 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:09.584 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.584 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:09.584 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:09.585 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.585 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:09.585 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:09.585 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.585 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:09.586 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:09.586 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:09.586 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.586 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:09.587 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:09.587 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.587 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:09.587 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.587 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:09.588 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.588 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:09.588 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.588 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:09.588 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:09.589 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.589 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:09.589 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:09.589 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:09.590 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:09.590 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:09.590 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:09.590 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:09.590 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:09.591 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:09.591 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:09.591 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:09.591 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:09.591 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:09.592 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:09.592 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:09.592 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:09.592 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:09.592 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:09.593 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:09.593 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:09.593 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.593 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:09.593 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.594 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:09.594 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:09.594 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:09.594 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.594 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.595 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:09.595 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.595 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:09.595 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.595 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:09.596 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:09.596 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:09.596 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:09.596 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:09.596 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.597 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:09.597 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:09.597 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.597 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:09.598 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.598 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:09.598 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:09.598 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:09.598 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:09.599 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:09.599 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:09.599 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:09.599 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:09.599 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.600 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:09.600 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.600 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.600 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:09.600 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:09.601 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:09.601 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:09.601 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:09.601 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:09.601 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:09.602 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:09.602 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:09.602 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.602 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:09.602 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:09.603 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:09.603 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:09.603 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:09.603 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:09.603 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:09.604 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:09.604 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:09.604 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:09.604 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.605 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.605 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:09.605 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:09.605 INFO:teuthology.orchestra.run.smithi116.stdout: client quota = True 2020-11-20T12:18:09.606 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:09.607 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:09.607 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:09.608 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:09.608 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:09.608 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.608 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:09.608 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.609 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:09.609 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.609 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:09.609 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:09.610 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.610 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:09.610 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:09.610 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.610 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:09.611 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:09.611 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.611 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:09.611 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:09.611 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.612 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:09.612 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.612 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:09.612 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:09.612 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:09.613 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:09.613 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:09.613 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:09.613 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:09.613 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:09.614 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:09.614 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:09.614 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.614 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:09.614 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.615 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:09.615 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.615 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:09.615 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.615 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:09.616 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:09.616 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:09.616 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:09.616 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:09.617 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:09.617 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.617 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:09.617 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:09.617 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.618 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:09.618 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:09.618 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.618 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:09.619 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:09.619 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.619 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:09.619 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:09.619 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:09.620 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.620 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:09.620 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:09.620 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.620 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:09.621 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.621 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:09.621 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.621 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:09.622 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.622 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:09.622 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:09.622 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.622 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:09.623 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:09.623 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:09.623 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:09.623 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:09.623 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:09.624 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:09.624 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:09.624 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:09.624 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:09.624 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:09.625 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:09.625 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:09.625 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:09.625 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:09.625 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:09.626 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:09.626 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:09.626 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:09.626 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:09.626 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.627 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:09.627 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.627 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:09.627 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:09.627 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:09.628 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.628 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.628 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:09.628 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.628 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:09.629 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.629 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:09.629 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:09.629 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:09.629 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:09.630 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:09.630 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.630 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:09.630 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:09.630 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.631 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:09.631 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.631 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:09.631 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:09.632 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:09.632 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:09.633 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:09.633 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:09.633 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:09.633 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:09.633 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.634 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:09.634 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.634 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.634 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:09.634 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:09.635 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:09.635 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:09.635 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:09.635 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:09.635 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:09.636 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:09.636 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:09.636 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.636 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:09.636 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:09.637 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:09.637 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:09.637 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:09.637 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:09.638 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:09.638 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:09.638 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:09.638 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:09.638 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.639 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.639 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:09.639 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:18:09.639 INFO:teuthology.orchestra.run.smithi173.stdout: client quota = True 2020-11-20T12:18:09.639 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:09.641 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:09.647 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:09.710 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:09.711 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:09.711 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:09.711 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:09.711 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.712 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:09.712 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.712 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:09.712 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.712 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:09.713 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:09.713 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.713 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:09.713 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:09.715 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.715 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:09.715 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:09.716 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.716 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:09.716 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:09.716 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.717 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:09.717 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.717 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:09.717 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:09.717 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:09.718 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:09.718 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:09.718 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:09.718 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:09.718 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:09.719 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:09.719 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:09.719 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.719 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:09.719 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.720 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:09.720 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.720 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:09.720 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.720 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:09.721 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:09.721 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:09.721 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:09.721 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:09.722 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:09.722 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.722 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:09.722 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:09.722 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.723 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:09.723 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:09.723 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.723 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:09.723 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:09.724 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.724 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:09.724 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:09.724 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:09.724 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.728 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:09.728 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:09.729 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.729 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:09.729 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.730 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:09.730 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.730 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:09.730 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.730 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:09.731 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:09.731 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.731 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:09.731 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:09.732 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:09.733 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:09.733 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:09.734 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:09.734 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:09.734 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:09.735 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:09.735 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:09.735 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:09.735 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:09.736 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:09.737 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:09.738 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:09.738 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:09.738 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:09.738 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:09.739 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:09.739 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:09.739 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.740 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:09.740 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.740 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:09.740 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:09.740 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:09.741 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.741 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.742 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:09.742 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.743 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:09.743 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.743 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:09.744 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:09.744 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:09.745 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:09.745 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:09.745 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.745 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:09.746 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:09.746 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.746 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:09.746 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.747 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:09.747 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:09.747 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:09.747 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:09.748 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:09.748 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:09.748 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:09.748 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:09.750 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.750 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:09.750 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.751 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.751 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:09.751 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:09.751 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:09.752 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:09.752 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:09.752 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:09.752 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:09.752 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:09.753 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:09.753 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.753 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:09.753 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:09.753 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:09.754 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:09.754 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:09.754 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:09.754 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:09.755 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:09.755 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:09.755 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:09.755 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.755 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.756 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:09.756 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:09.756 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:09.763 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:09.763 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:09.764 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:09.764 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:09.764 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.764 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:09.765 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.765 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:09.765 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.765 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:09.765 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:09.766 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.766 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:09.766 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:09.766 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.767 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:09.767 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:09.767 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.767 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:09.767 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:09.768 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.768 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:09.768 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.768 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:09.768 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:09.769 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:09.769 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:09.769 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:09.769 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:09.769 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:09.770 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:09.770 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:09.770 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:09.770 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.770 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:09.771 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.771 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:09.771 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.771 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:09.771 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.772 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:09.772 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:09.772 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:09.772 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:09.773 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:09.773 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:09.773 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.773 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:09.773 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:09.774 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.774 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:09.774 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:09.774 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.774 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:09.775 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:09.775 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.775 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:09.775 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:09.775 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:09.776 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.776 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:09.776 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:09.776 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.776 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:09.777 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.777 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:09.777 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.777 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:09.777 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.778 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:09.778 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:09.778 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.778 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:09.779 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:09.779 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:09.779 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:09.779 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:09.779 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:09.780 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:09.780 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:09.780 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:09.780 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:09.780 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:09.781 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:09.781 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:09.781 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:09.781 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:09.781 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:09.782 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:09.782 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:09.782 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:09.782 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:09.782 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.783 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:09.783 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.783 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:09.783 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:09.783 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:09.784 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.784 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.784 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:09.784 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.784 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:09.785 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.785 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:09.785 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:09.785 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:09.785 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:09.786 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:09.786 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.786 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:09.786 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:09.786 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.787 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:09.787 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.787 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:09.787 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:09.788 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:09.788 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:09.788 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:09.788 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:09.788 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:09.789 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:09.789 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.789 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:09.789 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.789 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.790 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:09.790 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:09.790 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:09.790 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:09.790 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:09.791 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:09.791 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:09.791 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:09.791 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:09.791 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.792 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:09.792 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:09.792 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:09.792 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:09.793 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:09.793 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:09.793 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:09.794 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:09.795 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:09.795 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:09.795 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.795 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.795 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:09.796 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:18:09.796 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:09.797 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:09.804 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:09.867 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:09.868 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:09.868 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:09.868 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:09.868 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.869 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:09.869 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.869 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:09.869 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.869 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:09.870 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:09.870 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.870 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:09.870 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:09.870 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.871 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:09.871 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:09.871 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.871 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:09.871 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:09.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.872 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:09.872 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.872 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:09.872 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:09.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:09.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:09.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:09.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:09.873 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:09.874 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:09.874 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:09.874 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:09.874 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.874 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:09.875 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.875 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:09.875 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.875 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:09.875 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.876 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:09.876 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:09.876 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:09.876 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:09.876 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:09.877 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:09.877 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.877 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:09.877 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:09.877 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.878 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:09.878 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:09.878 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.878 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:09.878 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:09.879 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.879 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:09.879 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:09.879 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:09.879 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.880 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:09.880 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:09.880 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.881 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:09.881 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.881 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:09.881 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.881 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:09.882 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.882 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:09.882 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:09.882 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.882 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:09.883 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:09.883 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:09.883 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:09.883 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:09.883 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:09.884 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:09.884 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:09.884 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:09.885 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:09.885 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:09.885 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:09.886 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:09.886 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:09.886 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:09.886 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:09.886 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:09.887 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:09.887 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:09.887 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:09.887 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.887 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:09.888 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.888 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:09.888 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:09.888 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:09.888 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.889 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.889 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:09.889 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.889 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:09.889 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.890 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:09.890 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:09.890 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:09.890 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:09.890 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:09.891 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.891 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:09.891 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:09.891 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.891 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:09.892 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:09.892 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:09.892 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:09.892 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:09.892 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:09.893 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:09.893 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:09.893 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:09.893 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:09.893 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.894 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:09.894 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.894 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.894 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:09.894 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:09.895 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:09.895 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:09.895 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:09.895 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:09.895 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:09.896 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:09.896 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:09.896 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:09.896 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:09.896 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:09.897 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:09.897 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:09.897 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:09.897 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:09.897 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:09.898 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:09.898 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:09.898 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:09.898 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.898 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.899 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:09.899 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:09.900 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:09.900 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:09.900 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:09.901 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:09.901 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.901 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:09.901 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.902 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:09.902 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.902 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:09.902 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:09.902 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.903 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:09.903 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:09.903 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.903 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:09.903 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:09.904 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.904 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:09.904 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:09.904 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.904 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:09.905 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.905 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:09.905 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:09.905 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:09.905 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:09.906 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:09.906 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:09.906 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:09.906 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:09.906 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:09.907 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:09.907 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.907 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:09.907 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.907 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:09.908 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.908 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:09.908 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.908 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:09.908 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:09.909 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:09.909 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:09.909 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:09.909 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:09.909 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.910 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:09.910 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:09.910 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.910 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:09.910 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:09.911 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.911 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:09.911 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:09.911 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.911 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:09.912 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:09.912 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:09.912 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.912 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:09.912 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:09.913 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.913 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:09.913 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.913 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:09.913 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.914 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:09.914 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.914 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:09.914 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:09.914 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.915 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:09.915 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:09.915 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:09.915 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:09.915 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:09.916 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:09.916 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:09.916 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:09.916 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:09.916 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:09.917 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:09.917 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:09.917 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:09.917 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:09.918 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:09.918 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:09.918 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:09.918 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:09.918 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:09.918 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:09.919 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.919 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:09.919 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.919 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:09.919 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:09.920 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:09.920 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.920 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.920 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:09.920 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.921 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:09.921 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.921 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:09.921 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:09.921 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:09.922 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:09.922 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:09.922 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:09.922 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:09.922 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:09.923 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:09.923 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:09.923 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:09.923 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:09.923 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:09.924 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:09.924 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:09.924 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:09.924 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:09.924 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:09.925 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:09.925 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.925 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:09.925 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.925 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.926 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:09.926 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:09.926 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:09.926 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:09.926 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:09.927 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:09.927 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:09.927 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:09.927 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:09.927 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:09.928 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:09.929 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:09.930 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:09.930 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:09.930 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:09.930 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:18:09.931 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-20T12:18:10.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.112+0000 7fd3697b0700 1 Processor -- start 2020-11-20T12:18:10.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.112+0000 7fd3697b0700 1 -- start start 2020-11-20T12:18:10.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd3697b0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd3640c15a0 0x7fd364152ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd3697b0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd364153090 con 0x7fd364148af0 2020-11-20T12:18:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd3697b0700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd36414ee00 con 0x7fd3640bfc50 2020-11-20T12:18:10.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd3697b0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd36414ebe0 con 0x7fd3640c15a0 2020-11-20T12:18:10.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd362ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd364148af0 0x7fd3640ba7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55724/0 (socket says 172.21.15.116:55724) 2020-11-20T12:18:10.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd362ffd700 1 -- 172.21.15.116:0/3105656270 learned_addr learned my addr 172.21.15.116:0/3105656270 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd3627fc700 1 --2- 172.21.15.116:0/3105656270 >> v2:172.21.15.173:3300/0 conn(0x7fd3640c15a0 0x7fd364152ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3127537330 0 0) 0x7fd364153090 con 0x7fd364148af0 2020-11-20T12:18:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd350003240 con 0x7fd364148af0 2020-11-20T12:18:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2288373515 0 0) 0x7fd36414ee00 con 0x7fd3640bfc50 2020-11-20T12:18:10.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd364153090 con 0x7fd3640bfc50 2020-11-20T12:18:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1379174352 0 0) 0x7fd350003240 con 0x7fd364148af0 2020-11-20T12:18:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 >> v2:172.21.15.173:3300/0 conn(0x7fd3640c15a0 msgr2=0x7fd364152ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:10.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 --2- 172.21.15.116:0/3105656270 >> v2:172.21.15.173:3300/0 conn(0x7fd3640c15a0 0x7fd364152ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 >> v1:172.21.15.173:6789/0 conn(0x7fd3640bfc50 legacy=0x7fd3640c00a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:10.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd36414f0a0 con 0x7fd364148af0 2020-11-20T12:18:10.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fd3540014e0 con 0x7fd364148af0 2020-11-20T12:18:10.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 >> v1:172.21.15.116:6789/0 conn(0x7fd364148af0 legacy=0x7fd3640ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:10.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 --2- 172.21.15.116:0/3105656270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd350003910 0x7fd350003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 --2- 172.21.15.116:0/3105656270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3500044e0 0x7fd350004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 --2- 172.21.15.116:0/3105656270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd350004ea0 0x7fd35000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd36414ebe0 con 0x7fd350004ea0 2020-11-20T12:18:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35000bc50 con 0x7fd350003910 2020-11-20T12:18:10.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.116+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd35000be10 con 0x7fd3500044e0 2020-11-20T12:18:10.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3637fe700 1 --2- 172.21.15.116:0/3105656270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd350004ea0 0x7fd35000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3627fc700 1 --2- 172.21.15.116:0/3105656270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3500044e0 0x7fd350004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd362ffd700 1 --2- 172.21.15.116:0/3105656270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd350003910 0x7fd350003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3637fe700 1 -- 172.21.15.116:0/3105656270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3500044e0 msgr2=0x7fd350004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:10.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3637fe700 1 --2- 172.21.15.116:0/3105656270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd3500044e0 0x7fd350004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3637fe700 1 -- 172.21.15.116:0/3105656270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd350003910 msgr2=0x7fd350003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:10.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3637fe700 1 --2- 172.21.15.116:0/3105656270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd350003910 0x7fd350003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3637fe700 1 -- 172.21.15.116:0/3105656270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd350003240 con 0x7fd350004ea0 2020-11-20T12:18:10.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3637fe700 1 -- 172.21.15.116:0/3105656270 reap_dead start 2020-11-20T12:18:10.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3637fe700 1 --2- 172.21.15.116:0/3105656270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd350004ea0 0x7fd35000b730 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fd35c002010 tx=0x7fd35c007f10).ready entity=mon.0 client_cookie=7d79d97c1f855c71 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:10.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c0016e0 con 0x7fd350004ea0 2020-11-20T12:18:10.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35c00cc80 con 0x7fd350004ea0 2020-11-20T12:18:10.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd361ffb700 1 -- 172.21.15.116:0/3105656270 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35c006870 con 0x7fd350004ea0 2020-11-20T12:18:10.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3697b0700 1 -- 172.21.15.116:0/3105656270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd350004ea0 msgr2=0x7fd35000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:10.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3697b0700 1 --2- 172.21.15.116:0/3105656270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd350004ea0 0x7fd35000b730 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fd35c002010 tx=0x7fd35c007f10).stop 2020-11-20T12:18:10.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3697b0700 1 -- 172.21.15.116:0/3105656270 shutdown_connections 2020-11-20T12:18:10.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3697b0700 1 --2- 172.21.15.116:0/3105656270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd350004ea0 0x7fd35000b730 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3697b0700 1 -- 172.21.15.116:0/3105656270 >> 172.21.15.116:0/3105656270 conn(0x7fd3640c4ec0 msgr2=0x7fd3640c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:10.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3697b0700 1 -- 172.21.15.116:0/3105656270 shutdown_connections 2020-11-20T12:18:10.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.120+0000 7fd3697b0700 1 -- 172.21.15.116:0/3105656270 wait complete. 2020-11-20T12:18:10.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.124+0000 7fd3697b0700 1 Processor -- start 2020-11-20T12:18:10.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.124+0000 7fd3697b0700 1 -- start start 2020-11-20T12:18:10.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.124+0000 7fd3697b0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd350004ea0 0x7fd3640a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.124+0000 7fd3697b0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3640a73a0 0x7fd3640a7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.124+0000 7fd3697b0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd364164780 0x7fd364164c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.124+0000 7fd3697b0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd364165580 con 0x7fd3640a73a0 2020-11-20T12:18:10.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.124+0000 7fd3697b0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd364153cb0 con 0x7fd350004ea0 2020-11-20T12:18:10.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3697b0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd364153f50 con 0x7fd364164780 2020-11-20T12:18:10.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3627fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3640a73a0 0x7fd3640a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3627fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3640a73a0 0x7fd3640a7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46304/0 (socket says 172.21.15.116:46304) 2020-11-20T12:18:10.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3627fc700 1 -- 172.21.15.116:0/2852230566 learned_addr learned my addr 172.21.15.116:0/2852230566 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:10.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3637fe700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd364164780 0x7fd364164c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd362ffd700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd350004ea0 0x7fd3640a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3627fc700 1 -- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd364164780 msgr2=0x7fd364164c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:10.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3627fc700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd364164780 0x7fd364164c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3627fc700 1 -- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd350004ea0 msgr2=0x7fd3640a3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:10.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3627fc700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd350004ea0 0x7fd3640a3870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3627fc700 1 -- 172.21.15.116:0/2852230566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3641541f0 con 0x7fd3640a73a0 2020-11-20T12:18:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd362ffd700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd350004ea0 0x7fd3640a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3627fc700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3640a73a0 0x7fd3640a7820 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fd3580109f0 tx=0x7fd358008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:10.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd34affd700 1 -- 172.21.15.116:0/2852230566 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800e930 con 0x7fd3640a73a0 2020-11-20T12:18:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd34affd700 1 -- 172.21.15.116:0/2852230566 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd35800ea90 con 0x7fd3640a73a0 2020-11-20T12:18:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3697b0700 1 -- 172.21.15.116:0/2852230566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd364154490 con 0x7fd3640a73a0 2020-11-20T12:18:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd34affd700 1 -- 172.21.15.116:0/2852230566 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd35800ee10 con 0x7fd3640a73a0 2020-11-20T12:18:10.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3697b0700 1 -- 172.21.15.116:0/2852230566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3641549d0 con 0x7fd3640a73a0 2020-11-20T12:18:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.128+0000 7fd3697b0700 1 -- 172.21.15.116:0/2852230566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd364154b60 con 0x7fd3640a73a0 2020-11-20T12:18:10.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.132+0000 7fd34affd700 1 -- 172.21.15.116:0/2852230566 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd358014680 con 0x7fd3640a73a0 2020-11-20T12:18:10.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.132+0000 7fd34affd700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3400557c0 0x7fd340057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.132+0000 7fd34affd700 1 -- 172.21.15.116:0/2852230566 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd35806eb30 con 0x7fd3640a73a0 2020-11-20T12:18:10.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.132+0000 7fd362ffd700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3400557c0 0x7fd340057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.136+0000 7fd362ffd700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3400557c0 0x7fd340057c80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fd3540057a0 tx=0x7fd354009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:10.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.136+0000 7fd34affd700 1 -- 172.21.15.116:0/2852230566 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd3580449d0 con 0x7fd3640a73a0 2020-11-20T12:18:10.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.300+0000 7fd3697b0700 1 -- 172.21.15.116:0/2852230566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd36405cf30 con 0x7fd3640a73a0 2020-11-20T12:18:10.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.308+0000 7fd34affd700 1 -- 172.21.15.116:0/2852230566 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd358047ff0 con 0x7fd3640a73a0 2020-11-20T12:18:10.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 -- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3400557c0 msgr2=0x7fd340057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:10.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3400557c0 0x7fd340057c80 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7fd3540057a0 tx=0x7fd354009040).stop 2020-11-20T12:18:10.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 -- 172.21.15.116:0/2852230566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3640a73a0 msgr2=0x7fd3640a7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:10.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3640a73a0 0x7fd3640a7820 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fd3580109f0 tx=0x7fd358008040).stop 2020-11-20T12:18:10.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 -- 172.21.15.116:0/2852230566 shutdown_connections 2020-11-20T12:18:10.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3400557c0 0x7fd340057c80 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd364164780 0x7fd364164c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd350004ea0 0x7fd3640a3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 --2- 172.21.15.116:0/2852230566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3640a73a0 0x7fd3640a7820 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 -- 172.21.15.116:0/2852230566 >> 172.21.15.116:0/2852230566 conn(0x7fd3640c4ec0 msgr2=0x7fd36414d710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:10.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 -- 172.21.15.116:0/2852230566 shutdown_connections 2020-11-20T12:18:10.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.312+0000 7fd3697b0700 1 -- 172.21.15.116:0/2852230566 wait complete. 2020-11-20T12:18:10.336 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_21501' 2020-11-20T12:18:10.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.508+0000 7f55f7ef4700 1 Processor -- start 2020-11-20T12:18:10.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.508+0000 7f55f7ef4700 1 -- start start 2020-11-20T12:18:10.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.508+0000 7f55f7ef4700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f55f0150680 0x7f55f0066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.508+0000 7f55f7ef4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f55f00b9700 0x7f55f00b9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.508+0000 7f55f7ef4700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f01573d0 con 0x7f55f0156f20 2020-11-20T12:18:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.508+0000 7f55f7ef4700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f0150a60 con 0x7f55f00b9700 2020-11-20T12:18:10.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.508+0000 7f55f7ef4700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f0066af0 con 0x7f55f0150680 2020-11-20T12:18:10.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f6491700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f55f00b9700 0x7f55f00b9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f5c90700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f55f0150680 0x7f55f0066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f6491700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f55f00b9700 0x7f55f00b9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46314/0 (socket says 172.21.15.116:46314) 2020-11-20T12:18:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f6491700 1 -- 172.21.15.116:0/1758719817 learned_addr learned my addr 172.21.15.116:0/1758719817 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:10.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f5c90700 1 -- 172.21.15.116:0/1758719817 >> v2:172.21.15.116:3300/0 conn(0x7f55f00b9700 msgr2=0x7f55f00b9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f5c90700 1 --2- 172.21.15.116:0/1758719817 >> v2:172.21.15.116:3300/0 conn(0x7f55f00b9700 0x7f55f00b9b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f5c90700 1 -- 172.21.15.116:0/1758719817 >> v1:172.21.15.173:6789/0 conn(0x7f55f0156f20 legacy=0x7f55f00bf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f5c90700 1 -- 172.21.15.116:0/1758719817 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0066d90 con 0x7f55f0150680 2020-11-20T12:18:10.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f5c90700 1 --2- 172.21.15.116:0/1758719817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55f0150680 0x7f55f0066450 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f55f005c830 tx=0x7f55ec006df0).ready entity=mon.1 client_cookie=8aae1e8faffa9dc6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:10.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f4c8e700 1 -- 172.21.15.116:0/1758719817 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec0097b0 con 0x7f55f0150680 2020-11-20T12:18:10.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f4c8e700 1 -- 172.21.15.116:0/1758719817 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ec009910 con 0x7f55f0150680 2020-11-20T12:18:10.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f4c8e700 1 -- 172.21.15.116:0/1758719817 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec009c90 con 0x7f55f0150680 2020-11-20T12:18:10.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f7ef4700 1 -- 172.21.15.116:0/1758719817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55f0150680 msgr2=0x7f55f0066450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:10.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f7ef4700 1 --2- 172.21.15.116:0/1758719817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55f0150680 0x7f55f0066450 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f55f005c830 tx=0x7f55ec006df0).stop 2020-11-20T12:18:10.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f7ef4700 1 -- 172.21.15.116:0/1758719817 shutdown_connections 2020-11-20T12:18:10.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f7ef4700 1 --2- 172.21.15.116:0/1758719817 >> v2:172.21.15.116:3300/0 conn(0x7f55f00b9700 0x7f55f00b9b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f7ef4700 1 --2- 172.21.15.116:0/1758719817 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55f0150680 0x7f55f0066450 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f7ef4700 1 -- 172.21.15.116:0/1758719817 >> 172.21.15.116:0/1758719817 conn(0x7f55f00c4f80 msgr2=0x7f55f00c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:10.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.512+0000 7f55f7ef4700 1 -- 172.21.15.116:0/1758719817 shutdown_connections 2020-11-20T12:18:10.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.516+0000 7f55f7ef4700 1 -- 172.21.15.116:0/1758719817 wait complete. 2020-11-20T12:18:10.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.516+0000 7f55f7ef4700 1 Processor -- start 2020-11-20T12:18:10.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.516+0000 7f55f7ef4700 1 -- start start 2020-11-20T12:18:10.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.516+0000 7f55f7ef4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55f00b9700 0x7f55f00a4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.516+0000 7f55f7ef4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55f0150680 0x7f55f00a6590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.516+0000 7f55f7ef4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f0156f20 0x7f55f0154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.516+0000 7f55f7ef4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00a6fa0 con 0x7f55f0156f20 2020-11-20T12:18:10.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.516+0000 7f55f7ef4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00a6a80 con 0x7f55f0150680 2020-11-20T12:18:10.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.516+0000 7f55f7ef4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f00a6d00 con 0x7f55f00b9700 2020-11-20T12:18:10.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f6491700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f0156f20 0x7f55f0154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f6491700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f0156f20 0x7f55f0154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46318/0 (socket says 172.21.15.116:46318) 2020-11-20T12:18:10.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f6491700 1 -- 172.21.15.116:0/3668458087 learned_addr learned my addr 172.21.15.116:0/3668458087 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f5c90700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55f00b9700 0x7f55f00a4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f548f700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55f0150680 0x7f55f00a6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f6491700 1 -- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55f00b9700 msgr2=0x7f55f00a4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:10.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f6491700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55f00b9700 0x7f55f00a4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f6491700 1 -- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55f0150680 msgr2=0x7f55f00a6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:10.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f6491700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55f0150680 0x7f55f00a6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:10.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f6491700 1 -- 172.21.15.116:0/3668458087 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f00a7580 con 0x7f55f0156f20 2020-11-20T12:18:10.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f548f700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55f0150680 0x7f55f00a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:10.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f6491700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f0156f20 0x7f55f0154da0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f55e800ec50 tx=0x7f55e8009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:10.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55ddffb700 1 -- 172.21.15.116:0/3668458087 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e800ca60 con 0x7f55f0156f20 2020-11-20T12:18:10.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55ddffb700 1 -- 172.21.15.116:0/3668458087 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55e800cbc0 con 0x7f55f0156f20 2020-11-20T12:18:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f7ef4700 1 -- 172.21.15.116:0/3668458087 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f00a7820 con 0x7f55f0156f20 2020-11-20T12:18:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55ddffb700 1 -- 172.21.15.116:0/3668458087 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55e80123f0 con 0x7f55f0156f20 2020-11-20T12:18:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f7ef4700 1 -- 172.21.15.116:0/3668458087 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f0155290 con 0x7f55f0156f20 2020-11-20T12:18:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.520+0000 7f55f7ef4700 1 -- 172.21.15.116:0/3668458087 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f00a7cc0 con 0x7f55f0156f20 2020-11-20T12:18:10.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.524+0000 7f55ddffb700 1 -- 172.21.15.116:0/3668458087 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f55e8022070 con 0x7f55f0156f20 2020-11-20T12:18:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.524+0000 7f55ddffb700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55c80557f0 0x7f55c8057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.524+0000 7f55ddffb700 1 -- 172.21.15.116:0/3668458087 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f55e806ed60 con 0x7f55f0156f20 2020-11-20T12:18:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.524+0000 7f55f5c90700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55c80557f0 0x7f55c8057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:10.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.524+0000 7f55f5c90700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55c80557f0 0x7f55c8057cb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f55ec006e30 tx=0x7f55ec015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:10.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.528+0000 7f55ddffb700 1 -- 172.21.15.116:0/3668458087 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f55f00a7cc0 con 0x7f55f0156f20 2020-11-20T12:18:10.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:10.740+0000 7f55f7ef4700 1 -- 172.21.15.116:0/3668458087 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_21501"]} v 0) v1 -- 0x7f55f00a7cc0 con 0x7f55f0156f20 2020-11-20T12:18:11.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.380+0000 7f55ddffb700 1 -- 172.21.15.116:0/3668458087 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_21501"]}]=0 v136) v1 ==== 138+0+0 (secure 0 0 0) 0x7f55f00a7cc0 con 0x7f55f0156f20 2020-11-20T12:18:11.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 -- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55c80557f0 msgr2=0x7f55c8057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:11.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55c80557f0 0x7f55c8057cb0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f55ec006e30 tx=0x7f55ec015000).stop 2020-11-20T12:18:11.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 -- 172.21.15.116:0/3668458087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f0156f20 msgr2=0x7f55f0154da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:11.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f0156f20 0x7f55f0154da0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f55e800ec50 tx=0x7f55e8009290).stop 2020-11-20T12:18:11.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 -- 172.21.15.116:0/3668458087 shutdown_connections 2020-11-20T12:18:11.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55c80557f0 0x7f55c8057cb0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:11.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f55f0156f20 0x7f55f0154da0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:11.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f55f00b9700 0x7f55f00a4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:11.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 --2- 172.21.15.116:0/3668458087 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f55f0150680 0x7f55f00a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:11.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 -- 172.21.15.116:0/3668458087 >> 172.21.15.116:0/3668458087 conn(0x7f55f00c4f80 msgr2=0x7f55f00c22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:11.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 -- 172.21.15.116:0/3668458087 shutdown_connections 2020-11-20T12:18:11.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.384+0000 7f55f7ef4700 1 -- 172.21.15.116:0/3668458087 wait complete. 2020-11-20T12:18:11.411 INFO:tasks.cephfs_test_runner:test_21501 (tasks.cephfs.test_volume_client.TestVolumeClient) ... ok 2020-11-20T12:18:11.411 INFO:tasks.cephfs_test_runner:Starting test: test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient) 2020-11-20T12:18:11.412 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_data_isolated' 2020-11-20T12:18:11.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.584+0000 7f9289c06700 1 Processor -- start 2020-11-20T12:18:11.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.584+0000 7f9289c06700 1 -- start start 2020-11-20T12:18:11.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9289c06700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f928414bfb0 0x7f9284061490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:11.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9289c06700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9284064a60 0x7f9284064ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:11.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9289c06700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9284156250 con 0x7f928414bbd0 2020-11-20T12:18:11.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9289c06700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92840a70e0 con 0x7f9284064a60 2020-11-20T12:18:11.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9289c06700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9284151d70 con 0x7f928414bfb0 2020-11-20T12:18:11.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9288c04700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f928414bbd0 0x7f92840a8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55758/0 (socket says 172.21.15.116:55758) 2020-11-20T12:18:11.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9288c04700 1 -- 172.21.15.116:0/3224798974 learned_addr learned my addr 172.21.15.116:0/3224798974 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:11.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9289405700 1 --2- 172.21.15.116:0/3224798974 >> v2:172.21.15.116:3300/0 conn(0x7f9284064a60 0x7f9284064ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:11.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9283fff700 1 --2- 172.21.15.116:0/3224798974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f928414bfb0 0x7f9284061490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:11.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f92837fe700 1 -- 172.21.15.116:0/3224798974 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476293928 0 0) 0x7f9284156250 con 0x7f928414bbd0 2020-11-20T12:18:11.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f92837fe700 1 -- 172.21.15.116:0/3224798974 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9268003240 con 0x7f928414bbd0 2020-11-20T12:18:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9283fff700 1 -- 172.21.15.116:0/3224798974 >> v2:172.21.15.116:3300/0 conn(0x7f9284064a60 msgr2=0x7f9284064ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9283fff700 1 --2- 172.21.15.116:0/3224798974 >> v2:172.21.15.116:3300/0 conn(0x7f9284064a60 0x7f9284064ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f92837fe700 1 -- 172.21.15.116:0/3224798974 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 344746171 0 0) 0x7f9268003240 con 0x7f928414bbd0 2020-11-20T12:18:11.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9283fff700 1 -- 172.21.15.116:0/3224798974 >> v1:172.21.15.116:6789/0 conn(0x7f928414bbd0 legacy=0x7f92840a8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:11.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9283fff700 1 -- 172.21.15.116:0/3224798974 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92841557d0 con 0x7f928414bfb0 2020-11-20T12:18:11.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.588+0000 7f9283fff700 1 --2- 172.21.15.116:0/3224798974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f928414bfb0 0x7f9284061490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f927000b9a0 tx=0x7f9270006df0).ready entity=mon.2 client_cookie=6b2df7f2812c79c8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:11.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.592+0000 7f92837fe700 1 -- 172.21.15.116:0/3224798974 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270009680 con 0x7f928414bfb0 2020-11-20T12:18:11.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.592+0000 7f92837fe700 1 -- 172.21.15.116:0/3224798974 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92700097e0 con 0x7f928414bfb0 2020-11-20T12:18:11.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.592+0000 7f92837fe700 1 -- 172.21.15.116:0/3224798974 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270009b60 con 0x7f928414bfb0 2020-11-20T12:18:11.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.592+0000 7f9289c06700 1 -- 172.21.15.116:0/3224798974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f928414bfb0 msgr2=0x7f9284061490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:11.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.592+0000 7f9289c06700 1 --2- 172.21.15.116:0/3224798974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f928414bfb0 0x7f9284061490 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f927000b9a0 tx=0x7f9270006df0).stop 2020-11-20T12:18:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.592+0000 7f9289c06700 1 -- 172.21.15.116:0/3224798974 shutdown_connections 2020-11-20T12:18:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.592+0000 7f9289c06700 1 --2- 172.21.15.116:0/3224798974 >> v2:172.21.15.116:3300/0 conn(0x7f9284064a60 0x7f9284064ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.592+0000 7f9289c06700 1 --2- 172.21.15.116:0/3224798974 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f928414bfb0 0x7f9284061490 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.592+0000 7f9289c06700 1 -- 172.21.15.116:0/3224798974 >> 172.21.15.116:0/3224798974 conn(0x7f92840c6560 msgr2=0x7f92840c6940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.592+0000 7f9289c06700 1 -- 172.21.15.116:0/3224798974 shutdown_connections 2020-11-20T12:18:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.592+0000 7f9289c06700 1 -- 172.21.15.116:0/3224798974 wait complete. 2020-11-20T12:18:11.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.596+0000 7f9289c06700 1 Processor -- start 2020-11-20T12:18:11.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.596+0000 7f9289c06700 1 -- start start 2020-11-20T12:18:11.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.596+0000 7f9289c06700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9284064a60 0x7f92840abf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:11.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.596+0000 7f9289c06700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f928414bbd0 0x7f9284147310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:11.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.596+0000 7f9289c06700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f928414bfb0 0x7f9284168670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:11.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.596+0000 7f9289c06700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92840a9860 con 0x7f928414bbd0 2020-11-20T12:18:11.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.596+0000 7f9289c06700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928414f2a0 con 0x7f928414bfb0 2020-11-20T12:18:11.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.596+0000 7f9289c06700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9284164fa0 con 0x7f9284064a60 2020-11-20T12:18:11.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9283fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f928414bbd0 0x7f9284147310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:11.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9283fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f928414bbd0 0x7f9284147310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46330/0 (socket says 172.21.15.116:46330) 2020-11-20T12:18:11.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9283fff700 1 -- 172.21.15.116:0/3069619025 learned_addr learned my addr 172.21.15.116:0/3069619025 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:11.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9289405700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f928414bfb0 0x7f9284168670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:11.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9288c04700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9284064a60 0x7f92840abf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:11.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9283fff700 1 -- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9284064a60 msgr2=0x7f92840abf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:11.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9283fff700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9284064a60 0x7f92840abf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:11.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9283fff700 1 -- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f928414bfb0 msgr2=0x7f9284168670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:11.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9283fff700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f928414bfb0 0x7f9284168670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:11.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9283fff700 1 -- 172.21.15.116:0/3069619025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928414ed60 con 0x7f928414bbd0 2020-11-20T12:18:11.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9283fff700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f928414bbd0 0x7f9284147310 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f9270006e30 tx=0x7f9270015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:11.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9280ff9700 1 -- 172.21.15.116:0/3069619025 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270011d70 con 0x7f928414bbd0 2020-11-20T12:18:11.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9280ff9700 1 -- 172.21.15.116:0/3069619025 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f927001d450 con 0x7f928414bbd0 2020-11-20T12:18:11.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9280ff9700 1 -- 172.21.15.116:0/3069619025 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927001d810 con 0x7f928414bbd0 2020-11-20T12:18:11.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9289c06700 1 -- 172.21.15.116:0/3069619025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928414f000 con 0x7f928414bbd0 2020-11-20T12:18:11.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9289c06700 1 -- 172.21.15.116:0/3069619025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92840ac470 con 0x7f928414bbd0 2020-11-20T12:18:11.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9289c06700 1 -- 172.21.15.116:0/3069619025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f928405cf30 con 0x7f928414bbd0 2020-11-20T12:18:11.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.600+0000 7f9280ff9700 1 -- 172.21.15.116:0/3069619025 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f927000f070 con 0x7f928414bbd0 2020-11-20T12:18:11.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.604+0000 7f9280ff9700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f925c0557c0 0x7f925c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:11.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.604+0000 7f9280ff9700 1 -- 172.21.15.116:0/3069619025 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f927006f7a0 con 0x7f928414bbd0 2020-11-20T12:18:11.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.604+0000 7f9288c04700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f925c0557c0 0x7f925c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:11.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.604+0000 7f9288c04700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f925c0557c0 0x7f925c057c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f92780014b0 tx=0x7f9278001610).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:11.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.608+0000 7f9280ff9700 1 -- 172.21.15.116:0/3069619025 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f92700456f0 con 0x7f928414bbd0 2020-11-20T12:18:11.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:11.824+0000 7f9289c06700 1 -- 172.21.15.116:0/3069619025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_data_isolated"]} v 0) v1 -- 0x7f9284063490 con 0x7f928414bbd0 2020-11-20T12:18:12.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.384+0000 7f9280ff9700 1 -- 172.21.15.116:0/3069619025 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_data_isolated"]}]=0 v137) v1 ==== 149+0+0 (secure 0 0 0) 0x7f9270022020 con 0x7f928414bbd0 2020-11-20T12:18:12.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.388+0000 7f9289c06700 1 -- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f925c0557c0 msgr2=0x7f925c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:12.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.388+0000 7f9289c06700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f925c0557c0 0x7f925c057c80 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f92780014b0 tx=0x7f9278001610).stop 2020-11-20T12:18:12.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.388+0000 7f9289c06700 1 -- 172.21.15.116:0/3069619025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f928414bbd0 msgr2=0x7f9284147310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:12.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.388+0000 7f9289c06700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f928414bbd0 0x7f9284147310 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f9270006e30 tx=0x7f9270015000).stop 2020-11-20T12:18:12.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.388+0000 7f9289c06700 1 -- 172.21.15.116:0/3069619025 shutdown_connections 2020-11-20T12:18:12.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.388+0000 7f9289c06700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f925c0557c0 0x7f925c057c80 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.388+0000 7f9289c06700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f928414bfb0 0x7f9284168670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.388+0000 7f9289c06700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9284064a60 0x7f92840abf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.388+0000 7f9289c06700 1 --2- 172.21.15.116:0/3069619025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f928414bbd0 0x7f9284147310 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.388+0000 7f9289c06700 1 -- 172.21.15.116:0/3069619025 >> 172.21.15.116:0/3069619025 conn(0x7f92840c6560 msgr2=0x7f92840c46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:12.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.392+0000 7f9289c06700 1 -- 172.21.15.116:0/3069619025 shutdown_connections 2020-11-20T12:18:12.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.392+0000 7f9289c06700 1 -- 172.21.15.116:0/3069619025 wait complete. 2020-11-20T12:18:12.418 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-20T12:18:12.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.604+0000 7fdf331bc700 1 Processor -- start 2020-11-20T12:18:12.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.604+0000 7fdf331bc700 1 -- start start 2020-11-20T12:18:12.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf331bc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdf2c153860 0x7fdf2c064910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:12.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf331bc700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c066f40 con 0x7fdf2c0c6280 2020-11-20T12:18:12.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf331bc700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c159c70 con 0x7fdf2c061280 2020-11-20T12:18:12.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf331bc700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c159f10 con 0x7fdf2c153860 2020-11-20T12:18:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf31759700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fdf2c0c6280 0x7fdf2c066830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55776/0 (socket says 172.21.15.116:55776) 2020-11-20T12:18:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf31759700 1 -- 172.21.15.116:0/759342508 learned_addr learned my addr 172.21.15.116:0/759342508 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf30f58700 1 --2- 172.21.15.116:0/759342508 >> v2:172.21.15.116:3300/0 conn(0x7fdf2c153860 0x7fdf2c064910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:12.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 740402483 0 0) 0x7fdf2c066f40 con 0x7fdf2c0c6280 2020-11-20T12:18:12.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf10003240 con 0x7fdf2c0c6280 2020-11-20T12:18:12.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 830430302 0 0) 0x7fdf10003240 con 0x7fdf2c0c6280 2020-11-20T12:18:12.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 >> v2:172.21.15.116:3300/0 conn(0x7fdf2c153860 msgr2=0x7fdf2c064910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:12.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 --2- 172.21.15.116:0/759342508 >> v2:172.21.15.116:3300/0 conn(0x7fdf2c153860 0x7fdf2c064910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 >> v1:172.21.15.173:6789/0 conn(0x7fdf2c061280 legacy=0x7fdf2c066120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:12.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c15a1b0 con 0x7fdf2c0c6280 2020-11-20T12:18:12.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fdf24001860 con 0x7fdf2c0c6280 2020-11-20T12:18:12.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 >> v1:172.21.15.116:6789/0 conn(0x7fdf2c0c6280 legacy=0x7fdf2c066830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:12.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 --2- 172.21.15.116:0/759342508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf10003730 0x7fdf10003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:12.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 --2- 172.21.15.116:0/759342508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf100044e0 0x7fdf10004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:12.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 --2- 172.21.15.116:0/759342508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf10004e50 0x7fdf1000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:12.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c159f10 con 0x7fdf100044e0 2020-11-20T12:18:12.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf10005330 con 0x7fdf10004e50 2020-11-20T12:18:12.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1000bc20 con 0x7fdf10003730 2020-11-20T12:18:12.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2bfff700 1 --2- 172.21.15.116:0/759342508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf100044e0 0x7fdf10004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:12.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf31759700 1 --2- 172.21.15.116:0/759342508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf10004e50 0x7fdf1000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:12.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2bfff700 1 -- 172.21.15.116:0/759342508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf10003730 msgr2=0x7fdf10003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:12.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2bfff700 1 --2- 172.21.15.116:0/759342508 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf10003730 0x7fdf10003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:12.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2bfff700 1 -- 172.21.15.116:0/759342508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf10004e50 msgr2=0x7fdf1000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:12.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2bfff700 1 --2- 172.21.15.116:0/759342508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf10004e50 0x7fdf1000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2bfff700 1 -- 172.21.15.116:0/759342508 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf10003240 con 0x7fdf100044e0 2020-11-20T12:18:12.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf31759700 1 --2- 172.21.15.116:0/759342508 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf10004e50 0x7fdf1000b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:12.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf31759700 1 -- 172.21.15.116:0/759342508 reap_dead start 2020-11-20T12:18:12.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.608+0000 7fdf2bfff700 1 --2- 172.21.15.116:0/759342508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf100044e0 0x7fdf10004960 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fdf18001a00 tx=0x7fdf18007b70).ready entity=mon.0 client_cookie=4d33f2dde64d23f6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:12.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.612+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800c830 con 0x7fdf100044e0 2020-11-20T12:18:12.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.612+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1800c990 con 0x7fdf100044e0 2020-11-20T12:18:12.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.612+0000 7fdf2b7fe700 1 -- 172.21.15.116:0/759342508 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800cd50 con 0x7fdf100044e0 2020-11-20T12:18:12.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.612+0000 7fdf331bc700 1 -- 172.21.15.116:0/759342508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf100044e0 msgr2=0x7fdf10004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:12.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.612+0000 7fdf331bc700 1 --2- 172.21.15.116:0/759342508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf100044e0 0x7fdf10004960 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fdf18001a00 tx=0x7fdf18007b70).stop 2020-11-20T12:18:12.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.612+0000 7fdf331bc700 1 -- 172.21.15.116:0/759342508 shutdown_connections 2020-11-20T12:18:12.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.612+0000 7fdf331bc700 1 --2- 172.21.15.116:0/759342508 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf100044e0 0x7fdf10004960 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.612+0000 7fdf331bc700 1 -- 172.21.15.116:0/759342508 >> 172.21.15.116:0/759342508 conn(0x7fdf2c0c71f0 msgr2=0x7fdf2c0c3200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:12.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.612+0000 7fdf331bc700 1 -- 172.21.15.116:0/759342508 shutdown_connections 2020-11-20T12:18:12.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.612+0000 7fdf331bc700 1 -- 172.21.15.116:0/759342508 wait complete. 2020-11-20T12:18:12.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.616+0000 7fdf331bc700 1 Processor -- start 2020-11-20T12:18:12.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.616+0000 7fdf331bc700 1 -- start start 2020-11-20T12:18:12.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf331bc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf100044e0 0x7fdf2c164f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:12.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf331bc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf2c1517e0 0x7fdf2c1505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:12.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf331bc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf2c150ac0 0x7fdf2c168bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:12.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf331bc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c168530 con 0x7fdf2c1517e0 2020-11-20T12:18:12.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf331bc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c1657e0 con 0x7fdf2c150ac0 2020-11-20T12:18:12.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf331bc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c165a80 con 0x7fdf100044e0 2020-11-20T12:18:12.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf2bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf2c1517e0 0x7fdf2c1505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:12.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf2bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf2c1517e0 0x7fdf2c1505d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46352/0 (socket says 172.21.15.116:46352) 2020-11-20T12:18:12.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf2bfff700 1 -- 172.21.15.116:0/3560784032 learned_addr learned my addr 172.21.15.116:0/3560784032 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:12.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf30f58700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf100044e0 0x7fdf2c164f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:12.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf31759700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf2c150ac0 0x7fdf2c168bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:12.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf2bfff700 1 -- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf100044e0 msgr2=0x7fdf2c164f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:12.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf2bfff700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf100044e0 0x7fdf2c164f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf2bfff700 1 -- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf2c150ac0 msgr2=0x7fdf2c168bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:12.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf2bfff700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf2c150ac0 0x7fdf2c168bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf2bfff700 1 -- 172.21.15.116:0/3560784032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c165d20 con 0x7fdf2c1517e0 2020-11-20T12:18:12.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf30f58700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf100044e0 0x7fdf2c164f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:12.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf31759700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf2c150ac0 0x7fdf2c168bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:12.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf2bfff700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf2c1517e0 0x7fdf2c1505d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fdf10005300 tx=0x7fdf18016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:12.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf28ff9700 1 -- 172.21.15.116:0/3560784032 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18007240 con 0x7fdf2c1517e0 2020-11-20T12:18:12.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf28ff9700 1 -- 172.21.15.116:0/3560784032 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf180073a0 con 0x7fdf2c1517e0 2020-11-20T12:18:12.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf331bc700 1 -- 172.21.15.116:0/3560784032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c168720 con 0x7fdf2c1517e0 2020-11-20T12:18:12.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf28ff9700 1 -- 172.21.15.116:0/3560784032 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf18012710 con 0x7fdf2c1517e0 2020-11-20T12:18:12.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.620+0000 7fdf331bc700 1 -- 172.21.15.116:0/3560784032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c1690b0 con 0x7fdf2c1517e0 2020-11-20T12:18:12.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.624+0000 7fdf331bc700 1 -- 172.21.15.116:0/3560784032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c165eb0 con 0x7fdf2c1517e0 2020-11-20T12:18:12.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.624+0000 7fdf28ff9700 1 -- 172.21.15.116:0/3560784032 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fdf18010030 con 0x7fdf2c1517e0 2020-11-20T12:18:12.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.624+0000 7fdf28ff9700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf040557c0 0x7fdf04057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:12.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.624+0000 7fdf28ff9700 1 -- 172.21.15.116:0/3560784032 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdf1806fd60 con 0x7fdf2c1517e0 2020-11-20T12:18:12.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.624+0000 7fdf30f58700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf040557c0 0x7fdf04057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:12.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.628+0000 7fdf30f58700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf040557c0 0x7fdf04057c80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fdf20006a90 tx=0x7fdf20008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:12.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.632+0000 7fdf28ff9700 1 -- 172.21.15.116:0/3560784032 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fdf18045c00 con 0x7fdf2c1517e0 2020-11-20T12:18:12.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.792+0000 7fdf331bc700 1 -- 172.21.15.116:0/3560784032 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fdf2c1688b0 con 0x7fdf2c1517e0 2020-11-20T12:18:12.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.800+0000 7fdf28ff9700 1 -- 172.21.15.116:0/3560784032 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdf18049220 con 0x7fdf2c1517e0 2020-11-20T12:18:12.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.804+0000 7fdf331bc700 1 -- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf040557c0 msgr2=0x7fdf04057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:12.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.804+0000 7fdf331bc700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf040557c0 0x7fdf04057c80 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7fdf20006a90 tx=0x7fdf20008040).stop 2020-11-20T12:18:12.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.804+0000 7fdf331bc700 1 -- 172.21.15.116:0/3560784032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf2c1517e0 msgr2=0x7fdf2c1505d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:12.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.804+0000 7fdf331bc700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf2c1517e0 0x7fdf2c1505d0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7fdf10005300 tx=0x7fdf18016000).stop 2020-11-20T12:18:12.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.804+0000 7fdf331bc700 1 -- 172.21.15.116:0/3560784032 shutdown_connections 2020-11-20T12:18:12.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.804+0000 7fdf331bc700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf040557c0 0x7fdf04057c80 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.804+0000 7fdf331bc700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf2c150ac0 0x7fdf2c168bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.804+0000 7fdf331bc700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf100044e0 0x7fdf2c164f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.804+0000 7fdf331bc700 1 --2- 172.21.15.116:0/3560784032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf2c1517e0 0x7fdf2c1505d0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:12.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.804+0000 7fdf331bc700 1 -- 172.21.15.116:0/3560784032 >> 172.21.15.116:0/3560784032 conn(0x7fdf2c0c71f0 msgr2=0x7fdf2c1526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:12.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.808+0000 7fdf331bc700 1 -- 172.21.15.116:0/3560784032 shutdown_connections 2020-11-20T12:18:12.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:12.808+0000 7fdf331bc700 1 -- 172.21.15.116:0/3560784032 wait complete. 2020-11-20T12:18:12.830 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:18:12.831 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:18:12.870 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:18:12.910 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:18:12.911 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2020-11-20T12:18:12.911 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:18:13.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.096+0000 7f081dda4700 1 Processor -- start 2020-11-20T12:18:13.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.096+0000 7f081dda4700 1 -- start start 2020-11-20T12:18:13.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.096+0000 7f081dda4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08180a97a0 0x7f08180ad440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.096+0000 7f081dda4700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08180ada00 con 0x7f081813afa0 2020-11-20T12:18:13.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.096+0000 7f081dda4700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08180adca0 con 0x7f081813b380 2020-11-20T12:18:13.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.096+0000 7f081dda4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08180adf40 con 0x7f08180a97a0 2020-11-20T12:18:13.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.096+0000 7f081cda2700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f081813afa0 0x7f08180aa5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55792/0 (socket says 172.21.15.116:55792) 2020-11-20T12:18:13.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.096+0000 7f081cda2700 1 -- 172.21.15.116:0/1161158669 learned_addr learned my addr 172.21.15.116:0/1161158669 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:13.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.096+0000 7f081d5a3700 1 --2- 172.21.15.116:0/1161158669 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08180a97a0 0x7f08180ad440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.096+0000 7f08177fe700 1 -- 172.21.15.116:0/1161158669 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4166349427 0 0) 0x7f08180ada00 con 0x7f081813afa0 2020-11-20T12:18:13.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.096+0000 7f08177fe700 1 -- 172.21.15.116:0/1161158669 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07fc003240 con 0x7f081813afa0 2020-11-20T12:18:13.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f08177fe700 1 -- 172.21.15.116:0/1161158669 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3961829816 0 0) 0x7f08180adca0 con 0x7f081813b380 2020-11-20T12:18:13.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f081d5a3700 1 -- 172.21.15.116:0/1161158669 >> v1:172.21.15.173:6789/0 conn(0x7f081813b380 legacy=0x7f08180acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f081d5a3700 1 -- 172.21.15.116:0/1161158669 >> v1:172.21.15.116:6789/0 conn(0x7f081813afa0 legacy=0x7f08180aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f081d5a3700 1 -- 172.21.15.116:0/1161158669 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08180ae1e0 con 0x7f08180a97a0 2020-11-20T12:18:13.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f081d5a3700 1 --2- 172.21.15.116:0/1161158669 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08180a97a0 0x7f08180ad440 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f0810009cc0 tx=0x7f08100091d0).ready entity=mon.2 client_cookie=40df5fd68e9dc0a5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:13.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f08177fe700 1 -- 172.21.15.116:0/1161158669 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081000bad0 con 0x7f08180a97a0 2020-11-20T12:18:13.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f08177fe700 1 -- 172.21.15.116:0/1161158669 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081000bc30 con 0x7f08180a97a0 2020-11-20T12:18:13.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f08177fe700 1 -- 172.21.15.116:0/1161158669 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0810011400 con 0x7f08180a97a0 2020-11-20T12:18:13.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f081dda4700 1 -- 172.21.15.116:0/1161158669 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08180a97a0 msgr2=0x7f08180ad440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f081dda4700 1 --2- 172.21.15.116:0/1161158669 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08180a97a0 0x7f08180ad440 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f0810009cc0 tx=0x7f08100091d0).stop 2020-11-20T12:18:13.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f081dda4700 1 -- 172.21.15.116:0/1161158669 shutdown_connections 2020-11-20T12:18:13.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f081dda4700 1 --2- 172.21.15.116:0/1161158669 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08180a97a0 0x7f08180ad440 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f081dda4700 1 -- 172.21.15.116:0/1161158669 >> 172.21.15.116:0/1161158669 conn(0x7f08180678a0 msgr2=0x7f0818067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:13.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f081dda4700 1 -- 172.21.15.116:0/1161158669 shutdown_connections 2020-11-20T12:18:13.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.100+0000 7f081dda4700 1 -- 172.21.15.116:0/1161158669 wait complete. 2020-11-20T12:18:13.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081dda4700 1 Processor -- start 2020-11-20T12:18:13.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081dda4700 1 -- start start 2020-11-20T12:18:13.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081dda4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08180a97a0 0x7f0818151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081dda4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f081813afa0 0x7f0818137ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081dda4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f081813b380 0x7f08180c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081dda4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08180a0700 con 0x7f08180a97a0 2020-11-20T12:18:13.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081dda4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0818136130 con 0x7f081813b380 2020-11-20T12:18:13.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081dda4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0818137080 con 0x7f081813afa0 2020-11-20T12:18:13.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081cda2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08180a97a0 0x7f0818151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081cda2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08180a97a0 0x7f0818151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46364/0 (socket says 172.21.15.116:46364) 2020-11-20T12:18:13.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081cda2700 1 -- 172.21.15.116:0/620220897 learned_addr learned my addr 172.21.15.116:0/620220897 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:13.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081d5a3700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f081813b380 0x7f08180c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f0817fff700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f081813afa0 0x7f0818137ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081cda2700 1 -- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f081813afa0 msgr2=0x7f0818137ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081cda2700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f081813afa0 0x7f0818137ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081cda2700 1 -- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f081813b380 msgr2=0x7f08180c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081cda2700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f081813b380 0x7f08180c7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081cda2700 1 -- 172.21.15.116:0/620220897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818137320 con 0x7f08180a97a0 2020-11-20T12:18:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.104+0000 7f081cda2700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08180a97a0 0x7f0818151190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f080c00c6a0 tx=0x7f080c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.108+0000 7f0814ff9700 1 -- 172.21.15.116:0/620220897 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c0072e0 con 0x7f08180a97a0 2020-11-20T12:18:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.108+0000 7f0814ff9700 1 -- 172.21.15.116:0/620220897 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c007440 con 0x7f08180a97a0 2020-11-20T12:18:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.108+0000 7f0814ff9700 1 -- 172.21.15.116:0/620220897 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c005820 con 0x7f08180a97a0 2020-11-20T12:18:13.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.108+0000 7f081dda4700 1 -- 172.21.15.116:0/620220897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08180a0960 con 0x7f08180a97a0 2020-11-20T12:18:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.108+0000 7f081dda4700 1 -- 172.21.15.116:0/620220897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0818151680 con 0x7f08180a97a0 2020-11-20T12:18:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.108+0000 7f081dda4700 1 -- 172.21.15.116:0/620220897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f081805cf30 con 0x7f08180a97a0 2020-11-20T12:18:13.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.108+0000 7f0814ff9700 1 -- 172.21.15.116:0/620220897 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f080c010070 con 0x7f08180a97a0 2020-11-20T12:18:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.108+0000 7f0814ff9700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07f00557c0 0x7f07f0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.108+0000 7f0814ff9700 1 -- 172.21.15.116:0/620220897 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f080c06dd50 con 0x7f08180a97a0 2020-11-20T12:18:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.108+0000 7f0817fff700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07f00557c0 0x7f07f0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.112+0000 7f0817fff700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07f00557c0 0x7f07f0057c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f08180a77d0 tx=0x7f0804009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:13.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.112+0000 7f0814ff9700 1 -- 172.21.15.116:0/620220897 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f080c043bf0 con 0x7f08180a97a0 2020-11-20T12:18:13.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f081dda4700 1 -- 172.21.15.116:0/620220897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0818060170 con 0x7f08180a97a0 2020-11-20T12:18:13.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f0814ff9700 1 -- 172.21.15.116:0/620220897 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+2849 (secure 0 0 0) 0x7f080c018b40 con 0x7f08180a97a0 2020-11-20T12:18:13.323 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:13.323 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1096724826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1096724826},{"type":"v1","addr":"172.21.15.173:6837","nonce":1096724826}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/1589443098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1589443098},{"type":"v1","addr":"172.21.15.173:6835","nonce":1589443098}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/966300328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":966300328},{"type":"v1","addr":"172.21.15.116:6833","nonce":966300328}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:17:40.163408+0000","modified":"2020-11-20T12:17:42.171423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5493":{"gid":5493,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/2908037349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2908037349},{"type":"v1","addr":"172.21.15.116:6835","nonce":2908037349}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-20T12:18:13.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f081dda4700 1 -- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07f00557c0 msgr2=0x7f07f0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f081dda4700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07f00557c0 0x7f07f0057c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f08180a77d0 tx=0x7f0804009040).stop 2020-11-20T12:18:13.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f081dda4700 1 -- 172.21.15.116:0/620220897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08180a97a0 msgr2=0x7f0818151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f081dda4700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08180a97a0 0x7f0818151190 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f080c00c6a0 tx=0x7f080c007ac0).stop 2020-11-20T12:18:13.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f081dda4700 1 -- 172.21.15.116:0/620220897 shutdown_connections 2020-11-20T12:18:13.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f081dda4700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07f00557c0 0x7f07f0057c80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f081dda4700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f081813b380 0x7f08180c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f081dda4700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08180a97a0 0x7f0818151190 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f081dda4700 1 --2- 172.21.15.116:0/620220897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f081813afa0 0x7f0818137ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.324+0000 7f081dda4700 1 -- 172.21.15.116:0/620220897 >> 172.21.15.116:0/620220897 conn(0x7f08180678a0 msgr2=0x7f08180c8df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:13.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.328+0000 7f081dda4700 1 -- 172.21.15.116:0/620220897 shutdown_connections 2020-11-20T12:18:13.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.328+0000 7f081dda4700 1 -- 172.21.15.116:0/620220897 wait complete. 2020-11-20T12:18:13.328 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 20 2020-11-20T12:18:13.349 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:13.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.528+0000 7fb3305d7700 1 Processor -- start 2020-11-20T12:18:13.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.528+0000 7fb3305d7700 1 -- start start 2020-11-20T12:18:13.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.528+0000 7fb3305d7700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb3280c3780 0x7fb3280c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.528+0000 7fb3305d7700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3280a6950 con 0x7fb3280a6280 2020-11-20T12:18:13.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.528+0000 7fb3305d7700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3280a6bf0 con 0x7fb3280c1970 2020-11-20T12:18:13.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.528+0000 7fb3305d7700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32814fed0 con 0x7fb3280c3780 2020-11-20T12:18:13.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.528+0000 7fb32db72700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb3280a6280 0x7fb3280c22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55808/0 (socket says 172.21.15.116:55808) 2020-11-20T12:18:13.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32db72700 1 -- 172.21.15.116:0/3206767071 learned_addr learned my addr 172.21.15.116:0/3206767071 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:13.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32e373700 1 --2- 172.21.15.116:0/3206767071 >> v2:172.21.15.173:3300/0 conn(0x7fb3280c3780 0x7fb3280c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32d371700 1 -- 172.21.15.116:0/3206767071 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3782221025 0 0) 0x7fb3280a6950 con 0x7fb3280a6280 2020-11-20T12:18:13.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32d371700 1 -- 172.21.15.116:0/3206767071 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb310003240 con 0x7fb3280a6280 2020-11-20T12:18:13.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32d371700 1 -- 172.21.15.116:0/3206767071 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 238299801 0 0) 0x7fb3280a6bf0 con 0x7fb3280c1970 2020-11-20T12:18:13.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32d371700 1 -- 172.21.15.116:0/3206767071 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3280a6950 con 0x7fb3280c1970 2020-11-20T12:18:13.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32e373700 1 -- 172.21.15.116:0/3206767071 >> v1:172.21.15.173:6789/0 conn(0x7fb3280c1970 legacy=0x7fb328150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32d371700 1 -- 172.21.15.116:0/3206767071 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4020313515 0 0) 0x7fb310003240 con 0x7fb3280a6280 2020-11-20T12:18:13.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32e373700 1 -- 172.21.15.116:0/3206767071 >> v1:172.21.15.116:6789/0 conn(0x7fb3280a6280 legacy=0x7fb3280c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32e373700 1 -- 172.21.15.116:0/3206767071 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3280a6e90 con 0x7fb3280c3780 2020-11-20T12:18:13.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32e373700 1 --2- 172.21.15.116:0/3206767071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3280c3780 0x7fb3280c3b60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fb32400b9a0 tx=0x7fb324006df0).ready entity=mon.1 client_cookie=95da245f71adc75c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:13.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32d371700 1 -- 172.21.15.116:0/3206767071 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3180014e0 con 0x7fb3280c3780 2020-11-20T12:18:13.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32d371700 1 -- 172.21.15.116:0/3206767071 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb324011400 con 0x7fb3280c3780 2020-11-20T12:18:13.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.532+0000 7fb32d371700 1 -- 172.21.15.116:0/3206767071 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324011780 con 0x7fb3280c3780 2020-11-20T12:18:13.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.536+0000 7fb3305d7700 1 -- 172.21.15.116:0/3206767071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3280c3780 msgr2=0x7fb3280c3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.536+0000 7fb3305d7700 1 --2- 172.21.15.116:0/3206767071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3280c3780 0x7fb3280c3b60 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7fb32400b9a0 tx=0x7fb324006df0).stop 2020-11-20T12:18:13.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.536+0000 7fb3305d7700 1 -- 172.21.15.116:0/3206767071 shutdown_connections 2020-11-20T12:18:13.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.536+0000 7fb3305d7700 1 --2- 172.21.15.116:0/3206767071 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3280c3780 0x7fb3280c3b60 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.536+0000 7fb3305d7700 1 -- 172.21.15.116:0/3206767071 >> 172.21.15.116:0/3206767071 conn(0x7fb3280c4f80 msgr2=0x7fb3280c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:13.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.536+0000 7fb3305d7700 1 -- 172.21.15.116:0/3206767071 shutdown_connections 2020-11-20T12:18:13.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.536+0000 7fb3305d7700 1 -- 172.21.15.116:0/3206767071 wait complete. 2020-11-20T12:18:13.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.536+0000 7fb3305d7700 1 Processor -- start 2020-11-20T12:18:13.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3305d7700 1 -- start start 2020-11-20T12:18:13.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3305d7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3280a6280 0x7fb32814d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3305d7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3280c1970 0x7fb3280bb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3305d7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3280c3780 0x7fb3280a9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3305d7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3280a01d0 con 0x7fb3280c1970 2020-11-20T12:18:13.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3305d7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3280a03c0 con 0x7fb3280c3780 2020-11-20T12:18:13.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3305d7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3280a0620 con 0x7fb3280a6280 2020-11-20T12:18:13.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32db72700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3280c1970 0x7fb3280bb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32db72700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3280c1970 0x7fb3280bb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46380/0 (socket says 172.21.15.116:46380) 2020-11-20T12:18:13.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32db72700 1 -- 172.21.15.116:0/759663888 learned_addr learned my addr 172.21.15.116:0/759663888 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:13.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32eb74700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3280c3780 0x7fb3280a9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32e373700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3280a6280 0x7fb32814d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32db72700 1 -- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3280a6280 msgr2=0x7fb32814d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32db72700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3280a6280 0x7fb32814d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32db72700 1 -- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3280c3780 msgr2=0x7fb3280a9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32db72700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3280c3780 0x7fb3280a9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32db72700 1 -- 172.21.15.116:0/759663888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3280a08c0 con 0x7fb3280c1970 2020-11-20T12:18:13.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32eb74700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3280c3780 0x7fb3280a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb32db72700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3280c1970 0x7fb3280bb7d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fb31800c860 tx=0x7fb3180019a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3167fc700 1 -- 172.21.15.116:0/759663888 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318007170 con 0x7fb3280c1970 2020-11-20T12:18:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3167fc700 1 -- 172.21.15.116:0/759663888 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3180072d0 con 0x7fb3280c1970 2020-11-20T12:18:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3305d7700 1 -- 172.21.15.116:0/759663888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3280a0b60 con 0x7fb3280c1970 2020-11-20T12:18:13.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3167fc700 1 -- 172.21.15.116:0/759663888 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318004960 con 0x7fb3280c1970 2020-11-20T12:18:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.540+0000 7fb3305d7700 1 -- 172.21.15.116:0/759663888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32814d810 con 0x7fb3280c1970 2020-11-20T12:18:13.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.544+0000 7fb3305d7700 1 -- 172.21.15.116:0/759663888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32805d2a0 con 0x7fb3280c1970 2020-11-20T12:18:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.544+0000 7fb3167fc700 1 -- 172.21.15.116:0/759663888 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb31801c030 con 0x7fb3280c1970 2020-11-20T12:18:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.544+0000 7fb3167fc700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3000557c0 0x7fb300057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.544+0000 7fb3167fc700 1 -- 172.21.15.116:0/759663888 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb31806e710 con 0x7fb3280c1970 2020-11-20T12:18:13.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.548+0000 7fb32e373700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3000557c0 0x7fb300057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.548+0000 7fb32e373700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3000557c0 0x7fb300057c80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fb32400b150 tx=0x7fb324013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:13.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.552+0000 7fb3167fc700 1 -- 172.21.15.116:0/759663888 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb318044550 con 0x7fb3280c1970 2020-11-20T12:18:13.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.712+0000 7fb3305d7700 1 -- 172.21.15.116:0/759663888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb3280649f0 con 0x7fb3280c1970 2020-11-20T12:18:13.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.712+0000 7fb3167fc700 1 -- 172.21.15.116:0/759663888 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16874 (secure 0 0 0) 0x7fb318047b70 con 0x7fb3280c1970 2020-11-20T12:18:13.713 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:13.714 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":45,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:17:41.167389+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:17:38.002878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-20T12:17:38.995456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:13.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 -- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3000557c0 msgr2=0x7fb300057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3000557c0 0x7fb300057c80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7fb32400b150 tx=0x7fb324013000).stop 2020-11-20T12:18:13.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 -- 172.21.15.116:0/759663888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3280c1970 msgr2=0x7fb3280bb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3280c1970 0x7fb3280bb7d0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fb31800c860 tx=0x7fb3180019a0).stop 2020-11-20T12:18:13.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 -- 172.21.15.116:0/759663888 shutdown_connections 2020-11-20T12:18:13.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3000557c0 0x7fb300057c80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb3280c3780 0x7fb3280a9c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3280a6280 0x7fb32814d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 --2- 172.21.15.116:0/759663888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb3280c1970 0x7fb3280bb7d0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 -- 172.21.15.116:0/759663888 >> 172.21.15.116:0/759663888 conn(0x7fb3280c4f80 msgr2=0x7fb3280bfff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:13.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 -- 172.21.15.116:0/759663888 shutdown_connections 2020-11-20T12:18:13.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.716+0000 7fb3305d7700 1 -- 172.21.15.116:0/759663888 wait complete. 2020-11-20T12:18:13.744 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:17:38.002878+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:17:38.995456+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:18:13.745 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:18:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.936+0000 7ff85e8ce700 1 Processor -- start 2020-11-20T12:18:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.936+0000 7ff85e8ce700 1 -- start start 2020-11-20T12:18:13.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85e8ce700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff8580bca90 0x7ff8580bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85e8ce700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8580c1eb0 con 0x7ff858066180 2020-11-20T12:18:13.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85e8ce700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85814fee0 con 0x7ff8580c3610 2020-11-20T12:18:13.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85e8ce700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff85814fcf0 con 0x7ff8580bca90 2020-11-20T12:18:13.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85d0cb700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff8580bca90 0x7ff8580bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85e0cd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff858066180 0x7ff8580665b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55824/0 (socket says 172.21.15.116:55824) 2020-11-20T12:18:13.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85e0cd700 1 -- 172.21.15.116:0/2763256176 learned_addr learned my addr 172.21.15.116:0/2763256176 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:13.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85c8ca700 1 -- 172.21.15.116:0/2763256176 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717494456 0 0) 0x7ff8580c1eb0 con 0x7ff858066180 2020-11-20T12:18:13.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85c8ca700 1 -- 172.21.15.116:0/2763256176 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff840003240 con 0x7ff858066180 2020-11-20T12:18:13.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85d0cb700 1 -- 172.21.15.116:0/2763256176 >> v1:172.21.15.173:6789/0 conn(0x7ff8580c3610 legacy=0x7ff858065120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85d0cb700 1 -- 172.21.15.116:0/2763256176 >> v1:172.21.15.116:6789/0 conn(0x7ff858066180 legacy=0x7ff8580665b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85d0cb700 1 -- 172.21.15.116:0/2763256176 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff858150180 con 0x7ff8580bca90 2020-11-20T12:18:13.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85d0cb700 1 --2- 172.21.15.116:0/2763256176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8580bca90 0x7ff8580bcef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7ff84800b9a0 tx=0x7ff848006df0).ready entity=mon.1 client_cookie=27118a38c4cd8934 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:13.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85c8ca700 1 -- 172.21.15.116:0/2763256176 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8480097b0 con 0x7ff8580bca90 2020-11-20T12:18:13.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85c8ca700 1 -- 172.21.15.116:0/2763256176 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff848009910 con 0x7ff8580bca90 2020-11-20T12:18:13.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.940+0000 7ff85c8ca700 1 -- 172.21.15.116:0/2763256176 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848009c90 con 0x7ff8580bca90 2020-11-20T12:18:13.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.944+0000 7ff85e8ce700 1 -- 172.21.15.116:0/2763256176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8580bca90 msgr2=0x7ff8580bcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.944+0000 7ff85e8ce700 1 --2- 172.21.15.116:0/2763256176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8580bca90 0x7ff8580bcef0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7ff84800b9a0 tx=0x7ff848006df0).stop 2020-11-20T12:18:13.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.944+0000 7ff85e8ce700 1 -- 172.21.15.116:0/2763256176 shutdown_connections 2020-11-20T12:18:13.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.944+0000 7ff85e8ce700 1 --2- 172.21.15.116:0/2763256176 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8580bca90 0x7ff8580bcef0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.944+0000 7ff85e8ce700 1 -- 172.21.15.116:0/2763256176 >> 172.21.15.116:0/2763256176 conn(0x7ff85807ff70 msgr2=0x7ff858080350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:13.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.944+0000 7ff85e8ce700 1 -- 172.21.15.116:0/2763256176 shutdown_connections 2020-11-20T12:18:13.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.944+0000 7ff85e8ce700 1 -- 172.21.15.116:0/2763256176 wait complete. 2020-11-20T12:18:13.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e8ce700 1 Processor -- start 2020-11-20T12:18:13.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e8ce700 1 -- start start 2020-11-20T12:18:13.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e8ce700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff858066180 0x7ff858149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e8ce700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8580bca90 0x7ff858153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e8ce700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8580c3610 0x7ff858153710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e8ce700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff858168bd0 con 0x7ff8580c3610 2020-11-20T12:18:13.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e8ce700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff858169a80 con 0x7ff858066180 2020-11-20T12:18:13.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e8ce700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff858168e00 con 0x7ff8580bca90 2020-11-20T12:18:13.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e0cd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8580c3610 0x7ff858153710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e0cd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8580c3610 0x7ff858153710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46396/0 (socket says 172.21.15.116:46396) 2020-11-20T12:18:13.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e0cd700 1 -- 172.21.15.116:0/3950973937 learned_addr learned my addr 172.21.15.116:0/3950973937 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:13.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85d8cc700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff858066180 0x7ff858149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85d0cb700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8580bca90 0x7ff858153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e0cd700 1 -- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8580bca90 msgr2=0x7ff858153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e0cd700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8580bca90 0x7ff858153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e0cd700 1 -- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff858066180 msgr2=0x7ff858149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e0cd700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff858066180 0x7ff858149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e0cd700 1 -- 172.21.15.116:0/3950973937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8581690a0 con 0x7ff8580c3610 2020-11-20T12:18:13.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff85e0cd700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8580c3610 0x7ff858153710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7ff850001420 tx=0x7ff850007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:13.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff845ffb700 1 -- 172.21.15.116:0/3950973937 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850005b80 con 0x7ff8580c3610 2020-11-20T12:18:13.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.952+0000 7ff85e8ce700 1 -- 172.21.15.116:0/3950973937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff858169340 con 0x7ff8580c3610 2020-11-20T12:18:13.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.948+0000 7ff845ffb700 1 -- 172.21.15.116:0/3950973937 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff850005ce0 con 0x7ff8580c3610 2020-11-20T12:18:13.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.952+0000 7ff845ffb700 1 -- 172.21.15.116:0/3950973937 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff850007320 con 0x7ff8580c3610 2020-11-20T12:18:13.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.952+0000 7ff85e8ce700 1 -- 172.21.15.116:0/3950973937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff858149540 con 0x7ff8580c3610 2020-11-20T12:18:13.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.952+0000 7ff85e8ce700 1 -- 172.21.15.116:0/3950973937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8581497c0 con 0x7ff8580c3610 2020-11-20T12:18:13.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.952+0000 7ff845ffb700 1 -- 172.21.15.116:0/3950973937 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff850010030 con 0x7ff8580c3610 2020-11-20T12:18:13.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.956+0000 7ff845ffb700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8300557c0 0x7ff830057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:13.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.956+0000 7ff845ffb700 1 -- 172.21.15.116:0/3950973937 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff85006e780 con 0x7ff8580c3610 2020-11-20T12:18:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.956+0000 7ff85d8cc700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8300557c0 0x7ff830057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.956+0000 7ff85d8cc700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8300557c0 0x7ff830057c80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7ff858168b30 tx=0x7ff854009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:13.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:13.960+0000 7ff845ffb700 1 -- 172.21.15.116:0/3950973937 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff850044510 con 0x7ff8580c3610 2020-11-20T12:18:14.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.124+0000 7ff85e8ce700 1 -- 172.21.15.116:0/3950973937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff85805cf30 con 0x7ff8580c3610 2020-11-20T12:18:14.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.124+0000 7ff845ffb700 1 -- 172.21.15.116:0/3950973937 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v45) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff850047b30 con 0x7ff8580c3610 2020-11-20T12:18:14.123 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:18:14.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 -- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8300557c0 msgr2=0x7ff830057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8300557c0 0x7ff830057c80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7ff858168b30 tx=0x7ff854009040).stop 2020-11-20T12:18:14.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 -- 172.21.15.116:0/3950973937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8580c3610 msgr2=0x7ff858153710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8580c3610 0x7ff858153710 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7ff850001420 tx=0x7ff850007e20).stop 2020-11-20T12:18:14.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 -- 172.21.15.116:0/3950973937 shutdown_connections 2020-11-20T12:18:14.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8300557c0 0x7ff830057c80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8580c3610 0x7ff858153710 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff858066180 0x7ff858149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 --2- 172.21.15.116:0/3950973937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8580bca90 0x7ff858153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 -- 172.21.15.116:0/3950973937 >> 172.21.15.116:0/3950973937 conn(0x7ff85807ff70 msgr2=0x7ff8580c39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:14.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 -- 172.21.15.116:0/3950973937 shutdown_connections 2020-11-20T12:18:14.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.128+0000 7ff85e8ce700 1 -- 172.21.15.116:0/3950973937 wait complete. 2020-11-20T12:18:14.150 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:18:14.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.328+0000 7fda6243e700 1 Processor -- start 2020-11-20T12:18:14.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.332+0000 7fda6243e700 1 -- start start 2020-11-20T12:18:14.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.332+0000 7fda6243e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fda5c1518f0 0x7fda5c0bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.332+0000 7fda6243e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fda5c0b93c0 0x7fda5c0c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.332+0000 7fda6243e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5c0bf510 con 0x7fda5c151cd0 2020-11-20T12:18:14.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.332+0000 7fda6243e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0bf7b0 con 0x7fda5c1518f0 2020-11-20T12:18:14.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.332+0000 7fda6243e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0bfa50 con 0x7fda5c0b93c0 2020-11-20T12:18:14.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.332+0000 7fda6143c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fda5c1518f0 0x7fda5c0bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda61c3d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fda5c0b93c0 0x7fda5c0c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda61c3d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fda5c0b93c0 0x7fda5c0c0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:45456/0 (socket says 172.21.15.116:45456) 2020-11-20T12:18:14.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda61c3d700 1 -- 172.21.15.116:0/1746746622 learned_addr learned my addr 172.21.15.116:0/1746746622 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:14.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda6143c700 1 -- 172.21.15.116:0/1746746622 >> v2:172.21.15.173:3300/0 conn(0x7fda5c0b93c0 msgr2=0x7fda5c0c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:14.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda6143c700 1 --2- 172.21.15.116:0/1746746622 >> v2:172.21.15.173:3300/0 conn(0x7fda5c0b93c0 0x7fda5c0c0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda6143c700 1 -- 172.21.15.116:0/1746746622 >> v1:172.21.15.173:6789/0 conn(0x7fda5c151cd0 legacy=0x7fda5c066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda6143c700 1 -- 172.21.15.116:0/1746746622 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0bfcf0 con 0x7fda5c1518f0 2020-11-20T12:18:14.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda61c3d700 1 --2- 172.21.15.116:0/1746746622 >> v2:172.21.15.173:3300/0 conn(0x7fda5c0b93c0 0x7fda5c0c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:14.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda6143c700 1 --2- 172.21.15.116:0/1746746622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda5c1518f0 0x7fda5c0bed20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fda5c0c1f60 tx=0x7fda58006df0).ready entity=mon.0 client_cookie=81711a394b1f0fcb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:14.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda4bfff700 1 -- 172.21.15.116:0/1746746622 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda580097b0 con 0x7fda5c1518f0 2020-11-20T12:18:14.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda4bfff700 1 -- 172.21.15.116:0/1746746622 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda58009910 con 0x7fda5c1518f0 2020-11-20T12:18:14.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda4bfff700 1 -- 172.21.15.116:0/1746746622 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda58009c90 con 0x7fda5c1518f0 2020-11-20T12:18:14.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda6243e700 1 -- 172.21.15.116:0/1746746622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda5c1518f0 msgr2=0x7fda5c0bed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda6243e700 1 --2- 172.21.15.116:0/1746746622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda5c1518f0 0x7fda5c0bed20 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fda5c0c1f60 tx=0x7fda58006df0).stop 2020-11-20T12:18:14.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda6243e700 1 -- 172.21.15.116:0/1746746622 shutdown_connections 2020-11-20T12:18:14.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda6243e700 1 --2- 172.21.15.116:0/1746746622 >> v2:172.21.15.173:3300/0 conn(0x7fda5c0b93c0 0x7fda5c0c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda6243e700 1 --2- 172.21.15.116:0/1746746622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda5c1518f0 0x7fda5c0bed20 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.336+0000 7fda6243e700 1 -- 172.21.15.116:0/1746746622 >> 172.21.15.116:0/1746746622 conn(0x7fda5c068350 msgr2=0x7fda5c0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:14.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.340+0000 7fda6243e700 1 -- 172.21.15.116:0/1746746622 shutdown_connections 2020-11-20T12:18:14.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.340+0000 7fda6243e700 1 -- 172.21.15.116:0/1746746622 wait complete. 2020-11-20T12:18:14.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.340+0000 7fda6243e700 1 Processor -- start 2020-11-20T12:18:14.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.340+0000 7fda6243e700 1 -- start start 2020-11-20T12:18:14.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda6243e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda5c0b93c0 0x7fda5c162790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda6243e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda5c151cd0 0x7fda5c162050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda6243e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda5c0a9430 0x7fda5c0c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda6243e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0aa430 con 0x7fda5c0a9430 2020-11-20T12:18:14.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda6243e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c162540 con 0x7fda5c0b93c0 2020-11-20T12:18:14.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda6243e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0a7770 con 0x7fda5c151cd0 2020-11-20T12:18:14.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda61c3d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda5c0a9430 0x7fda5c0c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda61c3d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda5c0a9430 0x7fda5c0c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46410/0 (socket says 172.21.15.116:46410) 2020-11-20T12:18:14.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda61c3d700 1 -- 172.21.15.116:0/1713127969 learned_addr learned my addr 172.21.15.116:0/1713127969 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:14.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda60c3b700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda5c151cd0 0x7fda5c162050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda6143c700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda5c0b93c0 0x7fda5c162790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda61c3d700 1 -- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda5c151cd0 msgr2=0x7fda5c162050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda61c3d700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda5c151cd0 0x7fda5c162050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda61c3d700 1 -- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda5c0b93c0 msgr2=0x7fda5c162790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda61c3d700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda5c0b93c0 0x7fda5c162790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda61c3d700 1 -- 172.21.15.116:0/1713127969 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0a99b0 con 0x7fda5c0a9430 2020-11-20T12:18:14.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda6143c700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda5c0b93c0 0x7fda5c162790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:14.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda60c3b700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda5c151cd0 0x7fda5c162050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:14.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda61c3d700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda5c0a9430 0x7fda5c0c4510 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fda5400ef80 tx=0x7fda54009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:14.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda497fa700 1 -- 172.21.15.116:0/1713127969 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5400ce50 con 0x7fda5c0a9430 2020-11-20T12:18:14.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda497fa700 1 -- 172.21.15.116:0/1713127969 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda540123f0 con 0x7fda5c0a9430 2020-11-20T12:18:14.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda497fa700 1 -- 172.21.15.116:0/1713127969 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda54012770 con 0x7fda5c0a9430 2020-11-20T12:18:14.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.344+0000 7fda6243e700 1 -- 172.21.15.116:0/1713127969 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c0a9c50 con 0x7fda5c0a9430 2020-11-20T12:18:14.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.348+0000 7fda6243e700 1 -- 172.21.15.116:0/1713127969 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c0aa190 con 0x7fda5c0a9430 2020-11-20T12:18:14.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.348+0000 7fda6243e700 1 -- 172.21.15.116:0/1713127969 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c164be0 con 0x7fda5c0a9430 2020-11-20T12:18:14.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.348+0000 7fda497fa700 1 -- 172.21.15.116:0/1713127969 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fda54022070 con 0x7fda5c0a9430 2020-11-20T12:18:14.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.348+0000 7fda497fa700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda340557f0 0x7fda34057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.348+0000 7fda497fa700 1 -- 172.21.15.116:0/1713127969 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fda5406f1f0 con 0x7fda5c0a9430 2020-11-20T12:18:14.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.352+0000 7fda6143c700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda340557f0 0x7fda34057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.352+0000 7fda6143c700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda340557f0 0x7fda34057cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fda5c0651c0 tx=0x7fda58015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:14.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.356+0000 7fda497fa700 1 -- 172.21.15.116:0/1713127969 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fda5c164be0 con 0x7fda5c0a9430 2020-11-20T12:18:14.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.516+0000 7fda6243e700 1 -- 172.21.15.116:0/1713127969 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fda5c164be0 con 0x7fda5c0a9430 2020-11-20T12:18:14.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.516+0000 7fda497fa700 1 -- 172.21.15.116:0/1713127969 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v45) v1 ==== 92+0+10 (secure 0 0 0) 0x7fda5c164be0 con 0x7fda5c0a9430 2020-11-20T12:18:14.515 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:14.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 -- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda340557f0 msgr2=0x7fda34057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda340557f0 0x7fda34057cb0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7fda5c0651c0 tx=0x7fda58015000).stop 2020-11-20T12:18:14.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 -- 172.21.15.116:0/1713127969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda5c0a9430 msgr2=0x7fda5c0c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda5c0a9430 0x7fda5c0c4510 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fda5400ef80 tx=0x7fda54009290).stop 2020-11-20T12:18:14.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 -- 172.21.15.116:0/1713127969 shutdown_connections 2020-11-20T12:18:14.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fda340557f0 0x7fda34057cb0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fda5c0a9430 0x7fda5c0c4510 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fda5c0b93c0 0x7fda5c162790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 --2- 172.21.15.116:0/1713127969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fda5c151cd0 0x7fda5c162050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 -- 172.21.15.116:0/1713127969 >> 172.21.15.116:0/1713127969 conn(0x7fda5c068350 msgr2=0x7fda5c0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:14.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 -- 172.21.15.116:0/1713127969 shutdown_connections 2020-11-20T12:18:14.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.520+0000 7fda6243e700 1 -- 172.21.15.116:0/1713127969 wait complete. 2020-11-20T12:18:14.545 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:18:14.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f6a61700 1 Processor -- start 2020-11-20T12:18:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f6a61700 1 -- start start 2020-11-20T12:18:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f6a61700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47f00a8400 0x7f47f00aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f6a61700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f47f00ad540 0x7f47f00ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f6a61700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f00ace40 con 0x7f47f00aa7d0 2020-11-20T12:18:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f6a61700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f01526c0 con 0x7f47f00ad540 2020-11-20T12:18:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f6a61700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0152960 con 0x7f47f00a8400 2020-11-20T12:18:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f5a5f700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f47f00aa7d0 0x7f47f0151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55848/0 (socket says 172.21.15.116:55848) 2020-11-20T12:18:14.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f5a5f700 1 -- 172.21.15.116:0/2491726390 learned_addr learned my addr 172.21.15.116:0/2491726390 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f525e700 1 --2- 172.21.15.116:0/2491726390 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47f00a8400 0x7f47f00aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f6260700 1 --2- 172.21.15.116:0/2491726390 >> v2:172.21.15.173:3300/0 conn(0x7f47f00ad540 0x7f47f00ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f4a5d700 1 -- 172.21.15.116:0/2491726390 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3153147587 0 0) 0x7f47f00ace40 con 0x7f47f00aa7d0 2020-11-20T12:18:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f4a5d700 1 -- 172.21.15.116:0/2491726390 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d8003240 con 0x7f47f00aa7d0 2020-11-20T12:18:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f525e700 1 -- 172.21.15.116:0/2491726390 >> v2:172.21.15.173:3300/0 conn(0x7f47f00ad540 msgr2=0x7f47f00ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:14.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f525e700 1 --2- 172.21.15.116:0/2491726390 >> v2:172.21.15.173:3300/0 conn(0x7f47f00ad540 0x7f47f00ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.724+0000 7f47f525e700 1 -- 172.21.15.116:0/2491726390 >> v1:172.21.15.116:6789/0 conn(0x7f47f00aa7d0 legacy=0x7f47f0151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f525e700 1 -- 172.21.15.116:0/2491726390 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f0152c00 con 0x7f47f00a8400 2020-11-20T12:18:14.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f525e700 1 --2- 172.21.15.116:0/2491726390 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47f00a8400 0x7f47f00aabb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f47e000b9a0 tx=0x7f47e0006df0).ready entity=mon.2 client_cookie=c4ad5b320fca4caf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:14.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f4a5d700 1 -- 172.21.15.116:0/2491726390 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0009680 con 0x7f47f00a8400 2020-11-20T12:18:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f4a5d700 1 -- 172.21.15.116:0/2491726390 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e00097e0 con 0x7f47f00a8400 2020-11-20T12:18:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f4a5d700 1 -- 172.21.15.116:0/2491726390 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e0009b60 con 0x7f47f00a8400 2020-11-20T12:18:14.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f6a61700 1 -- 172.21.15.116:0/2491726390 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47f00a8400 msgr2=0x7f47f00aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f6a61700 1 --2- 172.21.15.116:0/2491726390 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47f00a8400 0x7f47f00aabb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f47e000b9a0 tx=0x7f47e0006df0).stop 2020-11-20T12:18:14.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f6a61700 1 -- 172.21.15.116:0/2491726390 shutdown_connections 2020-11-20T12:18:14.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f6a61700 1 --2- 172.21.15.116:0/2491726390 >> v2:172.21.15.173:3300/0 conn(0x7f47f00ad540 0x7f47f00ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f6a61700 1 --2- 172.21.15.116:0/2491726390 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47f00a8400 0x7f47f00aabb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f6a61700 1 -- 172.21.15.116:0/2491726390 >> 172.21.15.116:0/2491726390 conn(0x7f47f00678a0 msgr2=0x7f47f0067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:14.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f6a61700 1 -- 172.21.15.116:0/2491726390 shutdown_connections 2020-11-20T12:18:14.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.728+0000 7f47f6a61700 1 -- 172.21.15.116:0/2491726390 wait complete. 2020-11-20T12:18:14.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f6a61700 1 Processor -- start 2020-11-20T12:18:14.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f6a61700 1 -- start start 2020-11-20T12:18:14.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f6a61700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47f00a8400 0x7f47f014c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f6a61700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47f00aa7d0 0x7f47f014aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f6a61700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47f00ad540 0x7f47f014f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f6a61700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0165400 con 0x7f47f00a8400 2020-11-20T12:18:14.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f6a61700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0164980 con 0x7f47f00aa7d0 2020-11-20T12:18:14.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f6a61700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f0164c20 con 0x7f47f00ad540 2020-11-20T12:18:14.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f5a5f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47f00a8400 0x7f47f014c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f5a5f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47f00a8400 0x7f47f014c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46420/0 (socket says 172.21.15.116:46420) 2020-11-20T12:18:14.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f5a5f700 1 -- 172.21.15.116:0/915168366 learned_addr learned my addr 172.21.15.116:0/915168366 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:14.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f6260700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47f00ad540 0x7f47f014f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.732+0000 7f47f525e700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47f00aa7d0 0x7f47f014aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47f5a5f700 1 -- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47f00ad540 msgr2=0x7f47f014f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47f5a5f700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47f00ad540 0x7f47f014f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47f5a5f700 1 -- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47f00aa7d0 msgr2=0x7f47f014aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47f5a5f700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47f00aa7d0 0x7f47f014aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47f5a5f700 1 -- 172.21.15.116:0/915168366 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f0164ec0 con 0x7f47f00a8400 2020-11-20T12:18:14.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47f5a5f700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47f00a8400 0x7f47f014c820 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f47ec00c6f0 tx=0x7f47ec007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:14.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47ddffb700 1 -- 172.21.15.116:0/915168366 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec007320 con 0x7f47f00a8400 2020-11-20T12:18:14.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47ddffb700 1 -- 172.21.15.116:0/915168366 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47ec007480 con 0x7f47f00a8400 2020-11-20T12:18:14.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47f6a61700 1 -- 172.21.15.116:0/915168366 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f0165160 con 0x7f47f00a8400 2020-11-20T12:18:14.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47ddffb700 1 -- 172.21.15.116:0/915168366 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47ec005800 con 0x7f47f00a8400 2020-11-20T12:18:14.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47f6a61700 1 -- 172.21.15.116:0/915168366 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f0164520 con 0x7f47f00a8400 2020-11-20T12:18:14.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47f6a61700 1 -- 172.21.15.116:0/915168366 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f005cf30 con 0x7f47f00a8400 2020-11-20T12:18:14.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.736+0000 7f47ddffb700 1 -- 172.21.15.116:0/915168366 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f47ec010030 con 0x7f47f00a8400 2020-11-20T12:18:14.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.740+0000 7f47ddffb700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47c80557c0 0x7f47c8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:14.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.740+0000 7f47ddffb700 1 -- 172.21.15.116:0/915168366 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f47ec06e470 con 0x7f47f00a8400 2020-11-20T12:18:14.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.740+0000 7f47f525e700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47c80557c0 0x7f47c8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:14.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.740+0000 7f47f525e700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47c80557c0 0x7f47c8057c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f47e000b9a0 tx=0x7f47e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:14.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.744+0000 7f47ddffb700 1 -- 172.21.15.116:0/915168366 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f47ec0442b0 con 0x7f47f00a8400 2020-11-20T12:18:14.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.904+0000 7f47f6a61700 1 -- 172.21.15.116:0/915168366 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f47f014edf0 con 0x7f47f00a8400 2020-11-20T12:18:14.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.904+0000 7f47ddffb700 1 -- 172.21.15.116:0/915168366 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+10 (secure 0 0 0) 0x7f47ec0478d0 con 0x7f47f00a8400 2020-11-20T12:18:14.905 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 -- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47c80557c0 msgr2=0x7f47c8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47c80557c0 0x7f47c8057c80 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f47e000b9a0 tx=0x7f47e0015000).stop 2020-11-20T12:18:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 -- 172.21.15.116:0/915168366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47f00a8400 msgr2=0x7f47f014c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:14.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47f00a8400 0x7f47f014c820 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f47ec00c6f0 tx=0x7f47ec007ad0).stop 2020-11-20T12:18:14.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 -- 172.21.15.116:0/915168366 shutdown_connections 2020-11-20T12:18:14.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47c80557c0 0x7f47c8057c80 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47f00ad540 0x7f47f014f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47f00a8400 0x7f47f014c820 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 --2- 172.21.15.116:0/915168366 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47f00aa7d0 0x7f47f014aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:14.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 -- 172.21.15.116:0/915168366 >> 172.21.15.116:0/915168366 conn(0x7f47f00678a0 msgr2=0x7f47f0153cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:14.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 -- 172.21.15.116:0/915168366 shutdown_connections 2020-11-20T12:18:14.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:14.908+0000 7f47f6a61700 1 -- 172.21.15.116:0/915168366 wait complete. 2020-11-20T12:18:14.929 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:18:15.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.128+0000 7fa63e36e700 1 Processor -- start 2020-11-20T12:18:15.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.128+0000 7fa63e36e700 1 -- start start 2020-11-20T12:18:15.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.128+0000 7fa63e36e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa6380aa1a0 0x7fa6380c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.128+0000 7fa63e36e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6380c2de0 0x7fa6380a8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.128+0000 7fa63e36e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa638151cf0 con 0x7fa6380aa580 2020-11-20T12:18:15.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.128+0000 7fa63e36e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6380c7e00 con 0x7fa6380aa1a0 2020-11-20T12:18:15.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.128+0000 7fa63e36e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6380c80a0 con 0x7fa6380c2de0 2020-11-20T12:18:15.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.128+0000 7fa63d36c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa6380aa1a0 0x7fa6380c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.128+0000 7fa63d36c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa6380aa1a0 0x7fa6380c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46428/0 (socket says 172.21.15.116:46428) 2020-11-20T12:18:15.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.128+0000 7fa63d36c700 1 -- 172.21.15.116:0/3326158123 learned_addr learned my addr 172.21.15.116:0/3326158123 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:15.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.128+0000 7fa63db6d700 1 --2- 172.21.15.116:0/3326158123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6380c2de0 0x7fa6380a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63d36c700 1 -- 172.21.15.116:0/3326158123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6380c2de0 msgr2=0x7fa6380a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:15.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63d36c700 1 --2- 172.21.15.116:0/3326158123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6380c2de0 0x7fa6380a8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63d36c700 1 -- 172.21.15.116:0/3326158123 >> v1:172.21.15.116:6789/0 conn(0x7fa6380aa580 legacy=0x7fa638061460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:15.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63d36c700 1 -- 172.21.15.116:0/3326158123 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6380c34f0 con 0x7fa6380aa1a0 2020-11-20T12:18:15.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63d36c700 1 --2- 172.21.15.116:0/3326158123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6380aa1a0 0x7fa6380c37d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fa634006fd0 tx=0x7fa634006eb0).ready entity=mon.0 client_cookie=2f08ab0d880a0df2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:15.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa627fff700 1 -- 172.21.15.116:0/3326158123 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6340098d0 con 0x7fa6380aa1a0 2020-11-20T12:18:15.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa627fff700 1 -- 172.21.15.116:0/3326158123 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa634009a30 con 0x7fa6380aa1a0 2020-11-20T12:18:15.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa627fff700 1 -- 172.21.15.116:0/3326158123 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa634009db0 con 0x7fa6380aa1a0 2020-11-20T12:18:15.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63e36e700 1 -- 172.21.15.116:0/3326158123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6380aa1a0 msgr2=0x7fa6380c37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63e36e700 1 --2- 172.21.15.116:0/3326158123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6380aa1a0 0x7fa6380c37d0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fa634006fd0 tx=0x7fa634006eb0).stop 2020-11-20T12:18:15.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63e36e700 1 -- 172.21.15.116:0/3326158123 shutdown_connections 2020-11-20T12:18:15.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63e36e700 1 --2- 172.21.15.116:0/3326158123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6380aa1a0 0x7fa6380c37d0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63e36e700 1 --2- 172.21.15.116:0/3326158123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6380c2de0 0x7fa6380a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63e36e700 1 -- 172.21.15.116:0/3326158123 >> 172.21.15.116:0/3326158123 conn(0x7fa638068370 msgr2=0x7fa6380c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:15.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63e36e700 1 -- 172.21.15.116:0/3326158123 shutdown_connections 2020-11-20T12:18:15.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.132+0000 7fa63e36e700 1 -- 172.21.15.116:0/3326158123 wait complete. 2020-11-20T12:18:15.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.136+0000 7fa63e36e700 1 Processor -- start 2020-11-20T12:18:15.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.136+0000 7fa63e36e700 1 -- start start 2020-11-20T12:18:15.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.136+0000 7fa63e36e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6380aa1a0 0x7fa6381612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.136+0000 7fa63e36e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6380aa580 0x7fa638165260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.136+0000 7fa63e36e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6380c2de0 0x7fa638164050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.136+0000 7fa63e36e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6381622d0 con 0x7fa6380c2de0 2020-11-20T12:18:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.136+0000 7fa63e36e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa638161850 con 0x7fa6380aa580 2020-11-20T12:18:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.136+0000 7fa63e36e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa638161af0 con 0x7fa6380aa1a0 2020-11-20T12:18:15.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63db6d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6380c2de0 0x7fa638164050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63db6d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6380c2de0 0x7fa638164050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46438/0 (socket says 172.21.15.116:46438) 2020-11-20T12:18:15.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63db6d700 1 -- 172.21.15.116:0/4105569349 learned_addr learned my addr 172.21.15.116:0/4105569349 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:15.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63cb6b700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6380aa580 0x7fa638165260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63d36c700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6380aa1a0 0x7fa6381612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63db6d700 1 -- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6380aa1a0 msgr2=0x7fa6381612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63db6d700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6380aa1a0 0x7fa6381612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63db6d700 1 -- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6380aa580 msgr2=0x7fa638165260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63db6d700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6380aa580 0x7fa638165260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63db6d700 1 -- 172.21.15.116:0/4105569349 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa638161d90 con 0x7fa6380c2de0 2020-11-20T12:18:15.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63db6d700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6380c2de0 0x7fa638164050 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fa63000db40 tx=0x7fa630008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:15.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa6257fa700 1 -- 172.21.15.116:0/4105569349 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6300099b0 con 0x7fa6380c2de0 2020-11-20T12:18:15.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa6257fa700 1 -- 172.21.15.116:0/4105569349 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa630009b10 con 0x7fa6380c2de0 2020-11-20T12:18:15.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63e36e700 1 -- 172.21.15.116:0/4105569349 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa638162030 con 0x7fa6380c2de0 2020-11-20T12:18:15.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa6257fa700 1 -- 172.21.15.116:0/4105569349 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa630013480 con 0x7fa6380c2de0 2020-11-20T12:18:15.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.140+0000 7fa63e36e700 1 -- 172.21.15.116:0/4105569349 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa638164540 con 0x7fa6380c2de0 2020-11-20T12:18:15.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.144+0000 7fa63e36e700 1 -- 172.21.15.116:0/4105569349 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6381624f0 con 0x7fa6380c2de0 2020-11-20T12:18:15.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.144+0000 7fa6257fa700 1 -- 172.21.15.116:0/4105569349 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa6300137f0 con 0x7fa6380c2de0 2020-11-20T12:18:15.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.144+0000 7fa6257fa700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6100557c0 0x7fa610057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.144+0000 7fa6257fa700 1 -- 172.21.15.116:0/4105569349 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa63006ed10 con 0x7fa6380c2de0 2020-11-20T12:18:15.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.144+0000 7fa63d36c700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6100557c0 0x7fa610057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.144+0000 7fa63d36c700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6100557c0 0x7fa610057c80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fa63400c010 tx=0x7fa634015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:15.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.148+0000 7fa6257fa700 1 -- 172.21.15.116:0/4105569349 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa6381624f0 con 0x7fa6380c2de0 2020-11-20T12:18:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.308+0000 7fa63e36e700 1 -- 172.21.15.116:0/4105569349 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa6381624f0 con 0x7fa6380c2de0 2020-11-20T12:18:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.308+0000 7fa6257fa700 1 -- 172.21.15.116:0/4105569349 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa6381624f0 con 0x7fa6380c2de0 2020-11-20T12:18:15.307 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:15.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.308+0000 7fa63e36e700 1 -- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6100557c0 msgr2=0x7fa610057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.308+0000 7fa63e36e700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6100557c0 0x7fa610057c80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fa63400c010 tx=0x7fa634015000).stop 2020-11-20T12:18:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.308+0000 7fa63e36e700 1 -- 172.21.15.116:0/4105569349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6380c2de0 msgr2=0x7fa638164050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.308+0000 7fa63e36e700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6380c2de0 0x7fa638164050 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fa63000db40 tx=0x7fa630008040).stop 2020-11-20T12:18:15.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.312+0000 7fa63e36e700 1 -- 172.21.15.116:0/4105569349 shutdown_connections 2020-11-20T12:18:15.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.312+0000 7fa63e36e700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6100557c0 0x7fa610057c80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.312+0000 7fa63e36e700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa6380c2de0 0x7fa638164050 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.312+0000 7fa63e36e700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa6380aa1a0 0x7fa6381612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.312+0000 7fa63e36e700 1 --2- 172.21.15.116:0/4105569349 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa6380aa580 0x7fa638165260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.312+0000 7fa63e36e700 1 -- 172.21.15.116:0/4105569349 >> 172.21.15.116:0/4105569349 conn(0x7fa638068370 msgr2=0x7fa6380ad7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.312+0000 7fa63e36e700 1 -- 172.21.15.116:0/4105569349 shutdown_connections 2020-11-20T12:18:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.312+0000 7fa63e36e700 1 -- 172.21.15.116:0/4105569349 wait complete. 2020-11-20T12:18:15.332 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:18:15.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.500+0000 7f0b0cdb9700 1 Processor -- start 2020-11-20T12:18:15.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.504+0000 7f0b0cdb9700 1 -- start start 2020-11-20T12:18:15.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.504+0000 7f0b0cdb9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0b080bac70 0x7f0b080c0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.504+0000 7f0b0cdb9700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b08066140 con 0x7f0b080c2230 2020-11-20T12:18:15.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.504+0000 7f0b0cdb9700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b081526c0 con 0x7f0b080ba890 2020-11-20T12:18:15.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.504+0000 7f0b0cdb9700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b080c1d80 con 0x7f0b080bac70 2020-11-20T12:18:15.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.504+0000 7f0afffff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0b080c2230 0x7f0b08148a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55876/0 (socket says 172.21.15.116:55876) 2020-11-20T12:18:15.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.504+0000 7f0afffff700 1 -- 172.21.15.116:0/2364531685 learned_addr learned my addr 172.21.15.116:0/2364531685 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:15.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.504+0000 7f0b06d9d700 1 --2- 172.21.15.116:0/2364531685 >> v2:172.21.15.116:3300/0 conn(0x7f0b080bac70 0x7f0b080c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.504+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3173382479 0 0) 0x7f0b08066140 con 0x7f0b080c2230 2020-11-20T12:18:15.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.504+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aec003240 con 0x7f0b080c2230 2020-11-20T12:18:15.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 966400617 0 0) 0x7f0aec003240 con 0x7f0b080c2230 2020-11-20T12:18:15.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 >> v2:172.21.15.116:3300/0 conn(0x7f0b080bac70 msgr2=0x7f0b080c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:15.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 --2- 172.21.15.116:0/2364531685 >> v2:172.21.15.116:3300/0 conn(0x7f0b080bac70 0x7f0b080c0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 >> v1:172.21.15.173:6789/0 conn(0x7f0b080ba890 legacy=0x7f0b080c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b08152960 con 0x7f0b080c2230 2020-11-20T12:18:15.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f0af00014e0 con 0x7f0b080c2230 2020-11-20T12:18:15.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 >> v1:172.21.15.116:6789/0 conn(0x7f0b080c2230 legacy=0x7f0b08148a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 --2- 172.21.15.116:0/2364531685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aec003730 0x7f0aec003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 --2- 172.21.15.116:0/2364531685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 0x7f0aec004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 --2- 172.21.15.116:0/2364531685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0aec004e50 0x7f0aec00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b080c1d80 con 0x7f0aec0044e0 2020-11-20T12:18:15.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec004260 con 0x7f0aec004e50 2020-11-20T12:18:15.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aec00bc20 con 0x7f0aec003730 2020-11-20T12:18:15.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0afffff700 1 --2- 172.21.15.116:0/2364531685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 0x7f0aec004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b0659c700 1 --2- 172.21.15.116:0/2364531685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aec003730 0x7f0aec003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0afffff700 1 -- 172.21.15.116:0/2364531685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aec003730 msgr2=0x7f0aec003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:15.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0afffff700 1 --2- 172.21.15.116:0/2364531685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aec003730 0x7f0aec003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0afffff700 1 -- 172.21.15.116:0/2364531685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0aec004e50 msgr2=0x7f0aec00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:15.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0afffff700 1 --2- 172.21.15.116:0/2364531685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0aec004e50 0x7f0aec00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:15.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0afffff700 1 -- 172.21.15.116:0/2364531685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aec003240 con 0x7f0aec0044e0 2020-11-20T12:18:15.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b0659c700 1 --2- 172.21.15.116:0/2364531685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0aec003730 0x7f0aec003b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:15.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0afffff700 1 --2- 172.21.15.116:0/2364531685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 0x7f0aec004960 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f0af000c410 tx=0x7f0b080ba560).ready entity=mon.0 client_cookie=c588a2378484a5a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:15.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b06d9d700 1 -- 172.21.15.116:0/2364531685 reap_dead start 2020-11-20T12:18:15.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af000ce80 con 0x7f0aec0044e0 2020-11-20T12:18:15.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af00072e0 con 0x7f0aec0044e0 2020-11-20T12:18:15.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b05d9b700 1 -- 172.21.15.116:0/2364531685 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0005800 con 0x7f0aec0044e0 2020-11-20T12:18:15.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b0cdb9700 1 -- 172.21.15.116:0/2364531685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 msgr2=0x7f0aec004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b0cdb9700 1 --2- 172.21.15.116:0/2364531685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 0x7f0aec004960 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f0af000c410 tx=0x7f0b080ba560).stop 2020-11-20T12:18:15.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b0cdb9700 1 -- 172.21.15.116:0/2364531685 shutdown_connections 2020-11-20T12:18:15.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b0cdb9700 1 --2- 172.21.15.116:0/2364531685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 0x7f0aec004960 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b0cdb9700 1 -- 172.21.15.116:0/2364531685 >> 172.21.15.116:0/2364531685 conn(0x7f0b080c4f80 msgr2=0x7f0b080c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:15.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b0cdb9700 1 -- 172.21.15.116:0/2364531685 shutdown_connections 2020-11-20T12:18:15.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.508+0000 7f0b0cdb9700 1 -- 172.21.15.116:0/2364531685 wait complete. 2020-11-20T12:18:15.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0cdb9700 1 Processor -- start 2020-11-20T12:18:15.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0cdb9700 1 -- start start 2020-11-20T12:18:15.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0cdb9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 0x7f0b080a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0cdb9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0b080631d0 0x7f0b08148990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0cdb9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b08148e80 0x7f0b080a8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0cdb9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b08136700 con 0x7f0aec0044e0 2020-11-20T12:18:15.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0cdb9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b081369a0 con 0x7f0b08148e80 2020-11-20T12:18:15.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0cdb9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b08136c40 con 0x7f0b080631d0 2020-11-20T12:18:15.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 0x7f0b080a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 0x7f0b080a9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46454/0 (socket says 172.21.15.116:46454) 2020-11-20T12:18:15.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0659c700 1 -- 172.21.15.116:0/897592015 learned_addr learned my addr 172.21.15.116:0/897592015 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:15.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0afffff700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0b080631d0 0x7f0b08148990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b06d9d700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b08148e80 0x7f0b080a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0659c700 1 -- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0b080631d0 msgr2=0x7f0b08148990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.512+0000 7f0b0659c700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0b080631d0 0x7f0b08148990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0b0659c700 1 -- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b08148e80 msgr2=0x7f0b080a8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0b0659c700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b08148e80 0x7f0b080a8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0b0659c700 1 -- 172.21.15.116:0/897592015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b08136ee0 con 0x7f0aec0044e0 2020-11-20T12:18:15.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0b06d9d700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b08148e80 0x7f0b080a8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:15.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0b0659c700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 0x7f0b080a9eb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f0af8000bc0 tx=0x7f0af8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:15.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0afe7fc700 1 -- 172.21.15.116:0/897592015 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af800b630 con 0x7f0aec0044e0 2020-11-20T12:18:15.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0afe7fc700 1 -- 172.21.15.116:0/897592015 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0af800b790 con 0x7f0aec0044e0 2020-11-20T12:18:15.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0b0cdb9700 1 -- 172.21.15.116:0/897592015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b08137180 con 0x7f0aec0044e0 2020-11-20T12:18:15.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0afe7fc700 1 -- 172.21.15.116:0/897592015 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af800bb10 con 0x7f0aec0044e0 2020-11-20T12:18:15.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0b0cdb9700 1 -- 172.21.15.116:0/897592015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b080a8730 con 0x7f0aec0044e0 2020-11-20T12:18:15.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0af7fff700 1 -- 172.21.15.116:0/897592015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0adc004f90 con 0x7f0aec0044e0 2020-11-20T12:18:15.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.516+0000 7f0afe7fc700 1 -- 172.21.15.116:0/897592015 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0af801c460 con 0x7f0aec0044e0 2020-11-20T12:18:15.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.520+0000 7f0afe7fc700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ad80557c0 0x7f0ad8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.520+0000 7f0afe7fc700 1 -- 172.21.15.116:0/897592015 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0af806e930 con 0x7f0aec0044e0 2020-11-20T12:18:15.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.520+0000 7f0afffff700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ad80557c0 0x7f0ad8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.520+0000 7f0afffff700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ad80557c0 0x7f0ad8057c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f0af0008fd0 tx=0x7f0af0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:15.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.524+0000 7f0afe7fc700 1 -- 172.21.15.116:0/897592015 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0af8044660 con 0x7f0aec0044e0 2020-11-20T12:18:15.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 -- 172.21.15.116:0/897592015 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0adc001660 con 0x7f0aec0044e0 2020-11-20T12:18:15.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0afe7fc700 1 -- 172.21.15.116:0/897592015 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+2849 (secure 0 0 0) 0x7f0af8047c80 con 0x7f0aec0044e0 2020-11-20T12:18:15.731 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:15.731 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5051,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1096724826","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":1096724826},{"type":"v1","addr":"172.21.15.173:6837","nonce":1096724826}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5482,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/1589443098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1589443098},{"type":"v1","addr":"172.21.15.173:6835","nonce":1589443098}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17},{"gid":5490,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/966300328","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":966300328},{"type":"v1","addr":"172.21.15.116:6833","nonce":966300328}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":17}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:17:40.163408+0000","modified":"2020-11-20T12:17:42.171423+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5493},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5493":{"gid":5493,"name":"a","rank":0,"incarnation":18,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/2908037349","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":2908037349},{"type":"v1","addr":"172.21.15.116:6835","nonce":2908037349}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2020-11-20T12:18:15.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 -- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ad80557c0 msgr2=0x7f0ad8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ad80557c0 0x7f0ad8057c80 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f0af0008fd0 tx=0x7f0af0015000).stop 2020-11-20T12:18:15.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 -- 172.21.15.116:0/897592015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 msgr2=0x7f0b080a9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 0x7f0b080a9eb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f0af8000bc0 tx=0x7f0af8009040).stop 2020-11-20T12:18:15.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 -- 172.21.15.116:0/897592015 shutdown_connections 2020-11-20T12:18:15.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ad80557c0 0x7f0ad8057c80 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0b08148e80 0x7f0b080a8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0aec0044e0 0x7f0b080a9eb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 --2- 172.21.15.116:0/897592015 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0b080631d0 0x7f0b08148990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 -- 172.21.15.116:0/897592015 >> 172.21.15.116:0/897592015 conn(0x7f0b080c4f80 msgr2=0x7f0b080638a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:15.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.732+0000 7f0af7fff700 1 -- 172.21.15.116:0/897592015 shutdown_connections 2020-11-20T12:18:15.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.736+0000 7f0af7fff700 1 -- 172.21.15.116:0/897592015 wait complete. 2020-11-20T12:18:15.736 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 20 2020-11-20T12:18:15.757 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:15.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.956+0000 7f783283d700 1 Processor -- start 2020-11-20T12:18:15.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.956+0000 7f783283d700 1 -- start start 2020-11-20T12:18:15.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.956+0000 7f783283d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f782c155df0 0x7f782c065d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.956+0000 7f783283d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c0c0530 con 0x7f782c062c60 2020-11-20T12:18:15.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.956+0000 7f783283d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f782c0c07d0 con 0x7f782c062880 2020-11-20T12:18:15.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.956+0000 7f783283d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f782c0c0a70 con 0x7f782c155df0 2020-11-20T12:18:15.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.956+0000 7f783103a700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f782c062c60 0x7f782c1556e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:55898/0 (socket says 172.21.15.116:55898) 2020-11-20T12:18:15.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.956+0000 7f783103a700 1 -- 172.21.15.116:0/3737466597 learned_addr learned my addr 172.21.15.116:0/3737466597 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:15.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f783203c700 1 --2- 172.21.15.116:0/3737466597 >> v2:172.21.15.116:3300/0 conn(0x7f782c155df0 0x7f782c065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f7830839700 1 -- 172.21.15.116:0/3737466597 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262665575 0 0) 0x7f782c0c0530 con 0x7f782c062c60 2020-11-20T12:18:15.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f7830839700 1 -- 172.21.15.116:0/3737466597 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7814003240 con 0x7f782c062c60 2020-11-20T12:18:15.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f7830839700 1 -- 172.21.15.116:0/3737466597 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525980332 0 0) 0x7f782c0c07d0 con 0x7f782c062880 2020-11-20T12:18:15.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f7830839700 1 -- 172.21.15.116:0/3737466597 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f782c0c0530 con 0x7f782c062880 2020-11-20T12:18:15.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f7830839700 1 -- 172.21.15.116:0/3737466597 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1660426049 0 0) 0x7f7814003240 con 0x7f782c062c60 2020-11-20T12:18:15.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f783203c700 1 -- 172.21.15.116:0/3737466597 >> v1:172.21.15.173:6789/0 conn(0x7f782c062880 legacy=0x7f782c0677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f783203c700 1 -- 172.21.15.116:0/3737466597 >> v1:172.21.15.116:6789/0 conn(0x7f782c062c60 legacy=0x7f782c1556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f783203c700 1 -- 172.21.15.116:0/3737466597 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c1599b0 con 0x7f782c155df0 2020-11-20T12:18:15.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f783203c700 1 --2- 172.21.15.116:0/3737466597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782c155df0 0x7f782c065d40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f7824009c90 tx=0x7f78240091d0).ready entity=mon.0 client_cookie=7ca93ffe24ad832f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:15.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f7830839700 1 -- 172.21.15.116:0/3737466597 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c0014e0 con 0x7f782c155df0 2020-11-20T12:18:15.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f7830839700 1 -- 172.21.15.116:0/3737466597 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782400b6b0 con 0x7f782c155df0 2020-11-20T12:18:15.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f7830839700 1 -- 172.21.15.116:0/3737466597 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782400ba30 con 0x7f782c155df0 2020-11-20T12:18:15.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f783283d700 1 -- 172.21.15.116:0/3737466597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782c155df0 msgr2=0x7f782c065d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f783283d700 1 --2- 172.21.15.116:0/3737466597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782c155df0 0x7f782c065d40 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f7824009c90 tx=0x7f78240091d0).stop 2020-11-20T12:18:15.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f783283d700 1 -- 172.21.15.116:0/3737466597 shutdown_connections 2020-11-20T12:18:15.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f783283d700 1 --2- 172.21.15.116:0/3737466597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782c155df0 0x7f782c065d40 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.960+0000 7f783283d700 1 -- 172.21.15.116:0/3737466597 >> 172.21.15.116:0/3737466597 conn(0x7f782c060c40 msgr2=0x7f782c0c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:15.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.964+0000 7f783283d700 1 -- 172.21.15.116:0/3737466597 shutdown_connections 2020-11-20T12:18:15.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.964+0000 7f783283d700 1 -- 172.21.15.116:0/3737466597 wait complete. 2020-11-20T12:18:15.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.964+0000 7f783283d700 1 Processor -- start 2020-11-20T12:18:15.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.964+0000 7f783283d700 1 -- start start 2020-11-20T12:18:15.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783283d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f782c062880 0x7f782c157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783283d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782c062c60 0x7f782c142110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783283d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f782c155df0 0x7f782c0a9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783283d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c1587e0 con 0x7f782c062c60 2020-11-20T12:18:15.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783283d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782c158a80 con 0x7f782c062880 2020-11-20T12:18:15.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783283d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f782c158d20 con 0x7f782c155df0 2020-11-20T12:18:15.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783103a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782c062c60 0x7f782c142110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783103a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782c062c60 0x7f782c142110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46470/0 (socket says 172.21.15.116:46470) 2020-11-20T12:18:15.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783103a700 1 -- 172.21.15.116:0/326622686 learned_addr learned my addr 172.21.15.116:0/326622686 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:15.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783183b700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f782c062880 0x7f782c157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783203c700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f782c155df0 0x7f782c0a9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783103a700 1 -- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f782c155df0 msgr2=0x7f782c0a9ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783103a700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f782c155df0 0x7f782c0a9ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783103a700 1 -- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f782c062880 msgr2=0x7f782c157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:15.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783103a700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f782c062880 0x7f782c157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:15.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783103a700 1 -- 172.21.15.116:0/326622686 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c158fc0 con 0x7f782c062c60 2020-11-20T12:18:15.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783103a700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782c062c60 0x7f782c142110 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f781c0013f0 tx=0x7f781c001af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:15.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f7819ffb700 1 -- 172.21.15.116:0/326622686 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c007170 con 0x7f782c062c60 2020-11-20T12:18:15.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f7819ffb700 1 -- 172.21.15.116:0/326622686 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f781c0072f0 con 0x7f782c062c60 2020-11-20T12:18:15.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783283d700 1 -- 172.21.15.116:0/326622686 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f782c1584d0 con 0x7f782c062c60 2020-11-20T12:18:15.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f7819ffb700 1 -- 172.21.15.116:0/326622686 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781c00aa80 con 0x7f782c062c60 2020-11-20T12:18:15.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.968+0000 7f783283d700 1 -- 172.21.15.116:0/326622686 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f782c142600 con 0x7f782c062c60 2020-11-20T12:18:15.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.972+0000 7f783283d700 1 -- 172.21.15.116:0/326622686 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782c159150 con 0x7f782c062c60 2020-11-20T12:18:15.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.972+0000 7f7819ffb700 1 -- 172.21.15.116:0/326622686 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f781c01b060 con 0x7f782c062c60 2020-11-20T12:18:15.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.972+0000 7f7819ffb700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78040557c0 0x7f7804057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:15.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.972+0000 7f7819ffb700 1 -- 172.21.15.116:0/326622686 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f781c06e420 con 0x7f782c062c60 2020-11-20T12:18:15.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.972+0000 7f783183b700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78040557c0 0x7f7804057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:15.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.972+0000 7f783183b700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78040557c0 0x7f7804057c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f7828003500 tx=0x7f7828009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:15.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:15.976+0000 7f7819ffb700 1 -- 172.21.15.116:0/326622686 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f781c0441f0 con 0x7f782c062c60 2020-11-20T12:18:16.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.136+0000 7f783283d700 1 -- 172.21.15.116:0/326622686 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f782c05cf30 con 0x7f782c062c60 2020-11-20T12:18:16.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.136+0000 7f7819ffb700 1 -- 172.21.15.116:0/326622686 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v45) v1 ==== 74+0+16874 (secure 0 0 0) 0x7f781c019030 con 0x7f782c062c60 2020-11-20T12:18:16.135 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:16.136 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":45,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:17:41.167389+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:17:38.002878+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2020-11-20T12:17:38.995456+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:16.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 -- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78040557c0 msgr2=0x7f7804057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:16.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78040557c0 0x7f7804057c80 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f7828003500 tx=0x7f7828009040).stop 2020-11-20T12:18:16.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 -- 172.21.15.116:0/326622686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782c062c60 msgr2=0x7f782c142110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:16.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782c062c60 0x7f782c142110 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7f781c0013f0 tx=0x7f781c001af0).stop 2020-11-20T12:18:16.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 -- 172.21.15.116:0/326622686 shutdown_connections 2020-11-20T12:18:16.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78040557c0 0x7f7804057c80 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:16.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f782c155df0 0x7f782c0a9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:16.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f782c062880 0x7f782c157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:16.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 --2- 172.21.15.116:0/326622686 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f782c062c60 0x7f782c142110 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:16.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 -- 172.21.15.116:0/326622686 >> 172.21.15.116:0/326622686 conn(0x7f782c060c40 msgr2=0x7f782c0c24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:16.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 -- 172.21.15.116:0/326622686 shutdown_connections 2020-11-20T12:18:16.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.140+0000 7f783283d700 1 -- 172.21.15.116:0/326622686 wait complete. 2020-11-20T12:18:16.161 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-20T12:18:16.161 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-20T12:18:16.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.336+0000 7ff4e6813700 1 Processor -- start 2020-11-20T12:18:16.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.336+0000 7ff4e6813700 1 -- start start 2020-11-20T12:18:16.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e6813700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff4e01559d0 0x7ff4e0155db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:16.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e6813700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e00a3f30 0x7ff4e00a43c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:16.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e6813700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4e01562a0 con 0x7ff4e00a67f0 2020-11-20T12:18:16.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e6813700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e01564d0 con 0x7ff4e01559d0 2020-11-20T12:18:16.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e6813700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e0156770 con 0x7ff4e00a3f30 2020-11-20T12:18:16.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e5811700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff4e01559d0 0x7ff4e0155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:16.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e6012700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e00a3f30 0x7ff4e00a43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:16.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e5811700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff4e01559d0 0x7ff4e0155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:45528/0 (socket says 172.21.15.116:45528) 2020-11-20T12:18:16.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e5811700 1 -- 172.21.15.116:0/2030668955 learned_addr learned my addr 172.21.15.116:0/2030668955 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:16.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e5010700 1 --1- 172.21.15.116:0/2030668955 >> v1:172.21.15.173:6789/0 conn(0x7ff4e00a67f0 0x7ff4e0062750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:52142/0 (socket says 172.21.15.116:52142) 2020-11-20T12:18:16.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e6012700 1 -- 172.21.15.116:0/2030668955 >> v2:172.21.15.173:3300/0 conn(0x7ff4e01559d0 msgr2=0x7ff4e0155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:16.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e6012700 1 --2- 172.21.15.116:0/2030668955 >> v2:172.21.15.173:3300/0 conn(0x7ff4e01559d0 0x7ff4e0155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:16.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e6012700 1 -- 172.21.15.116:0/2030668955 >> v1:172.21.15.173:6789/0 conn(0x7ff4e00a67f0 legacy=0x7ff4e0062750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:16.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e6012700 1 -- 172.21.15.116:0/2030668955 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e0156a10 con 0x7ff4e00a3f30 2020-11-20T12:18:16.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e5811700 1 --2- 172.21.15.116:0/2030668955 >> v2:172.21.15.173:3300/0 conn(0x7ff4e01559d0 0x7ff4e0155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:16.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e480f700 1 -- 172.21.15.116:0/2030668955 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823179457 0 0) 0x7ff4e01562a0 con 0x7ff4e00a67f0 2020-11-20T12:18:16.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e6012700 1 --2- 172.21.15.116:0/2030668955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e00a3f30 0x7ff4e00a43c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7ff4d8009c90 tx=0x7ff4d80091d0).ready entity=mon.2 client_cookie=f96dda72d08dd03b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:16.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e480f700 1 -- 172.21.15.116:0/2030668955 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d800ba70 con 0x7ff4e00a3f30 2020-11-20T12:18:16.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e480f700 1 -- 172.21.15.116:0/2030668955 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d800bbd0 con 0x7ff4e00a3f30 2020-11-20T12:18:16.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.340+0000 7ff4e480f700 1 -- 172.21.15.116:0/2030668955 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d8011400 con 0x7ff4e00a3f30 2020-11-20T12:18:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.344+0000 7ff4e6813700 1 -- 172.21.15.116:0/2030668955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e00a3f30 msgr2=0x7ff4e00a43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.344+0000 7ff4e6813700 1 --2- 172.21.15.116:0/2030668955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e00a3f30 0x7ff4e00a43c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7ff4d8009c90 tx=0x7ff4d80091d0).stop 2020-11-20T12:18:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.344+0000 7ff4e6813700 1 -- 172.21.15.116:0/2030668955 shutdown_connections 2020-11-20T12:18:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.344+0000 7ff4e6813700 1 --2- 172.21.15.116:0/2030668955 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e00a3f30 0x7ff4e00a43c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:16.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.344+0000 7ff4e6813700 1 --2- 172.21.15.116:0/2030668955 >> v2:172.21.15.173:3300/0 conn(0x7ff4e01559d0 0x7ff4e0155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:16.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.344+0000 7ff4e6813700 1 -- 172.21.15.116:0/2030668955 >> 172.21.15.116:0/2030668955 conn(0x7ff4e0060170 msgr2=0x7ff4e0060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:16.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.344+0000 7ff4e6813700 1 -- 172.21.15.116:0/2030668955 shutdown_connections 2020-11-20T12:18:16.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.344+0000 7ff4e6813700 1 -- 172.21.15.116:0/2030668955 wait complete. 2020-11-20T12:18:16.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.344+0000 7ff4e6813700 1 Processor -- start 2020-11-20T12:18:16.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6813700 1 -- start start 2020-11-20T12:18:16.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6813700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e00a3f30 0x7ff4e0165960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:16.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6813700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4e00a67f0 0x7ff4e014f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:16.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6813700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e01559d0 0x7ff4e00a8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:16.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6813700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e0165e50 con 0x7ff4e01559d0 2020-11-20T12:18:16.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6813700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e00a75d0 con 0x7ff4e00a67f0 2020-11-20T12:18:16.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6813700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e00a7870 con 0x7ff4e00a3f30 2020-11-20T12:18:16.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6012700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e01559d0 0x7ff4e00a8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:16.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6012700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e01559d0 0x7ff4e00a8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46486/0 (socket says 172.21.15.116:46486) 2020-11-20T12:18:16.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6012700 1 -- 172.21.15.116:0/3555599125 learned_addr learned my addr 172.21.15.116:0/3555599125 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:16.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e5010700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4e00a67f0 0x7ff4e014f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:16.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e5811700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e00a3f30 0x7ff4e0165960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:16.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6012700 1 -- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e00a3f30 msgr2=0x7ff4e0165960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:16.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6012700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e00a3f30 0x7ff4e0165960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:16.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6012700 1 -- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4e00a67f0 msgr2=0x7ff4e014f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:16.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6012700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4e00a67f0 0x7ff4e014f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:16.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6012700 1 -- 172.21.15.116:0/3555599125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e00a7b10 con 0x7ff4e01559d0 2020-11-20T12:18:16.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.348+0000 7ff4e6012700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e01559d0 0x7ff4e00a8cb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7ff4d8011f40 tx=0x7ff4d8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:16.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.352+0000 7ff4cdffb700 1 -- 172.21.15.116:0/3555599125 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d801c5f0 con 0x7ff4e01559d0 2020-11-20T12:18:16.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.352+0000 7ff4cdffb700 1 -- 172.21.15.116:0/3555599125 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d801c750 con 0x7ff4e01559d0 2020-11-20T12:18:16.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.352+0000 7ff4e6813700 1 -- 172.21.15.116:0/3555599125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e00a7db0 con 0x7ff4e01559d0 2020-11-20T12:18:16.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.352+0000 7ff4cdffb700 1 -- 172.21.15.116:0/3555599125 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d801cb10 con 0x7ff4e01559d0 2020-11-20T12:18:16.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.352+0000 7ff4e6813700 1 -- 172.21.15.116:0/3555599125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e00a84f0 con 0x7ff4e01559d0 2020-11-20T12:18:16.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.352+0000 7ff4e6813700 1 -- 172.21.15.116:0/3555599125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e005cd40 con 0x7ff4e01559d0 2020-11-20T12:18:16.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.356+0000 7ff4cdffb700 1 -- 172.21.15.116:0/3555599125 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff4d8017070 con 0x7ff4e01559d0 2020-11-20T12:18:16.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.356+0000 7ff4cdffb700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4b80557c0 0x7ff4b8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:16.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.356+0000 7ff4cdffb700 1 -- 172.21.15.116:0/3555599125 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff4d8070670 con 0x7ff4e01559d0 2020-11-20T12:18:16.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.356+0000 7ff4e5811700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4b80557c0 0x7ff4b8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:16.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.356+0000 7ff4e5811700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4b80557c0 0x7ff4b8057c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7ff4e00a8900 tx=0x7ff4dc006eb0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:16.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.360+0000 7ff4cdffb700 1 -- 172.21.15.116:0/3555599125 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff4d8046510 con 0x7ff4e01559d0 2020-11-20T12:18:16.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:16.572+0000 7ff4e6813700 1 -- 172.21.15.116:0/3555599125 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7ff4e0061180 con 0x7ff4e01559d0 2020-11-20T12:18:17.414 INFO:tasks.ceph.mon.a.smithi116.stderr:2020-11-20T12:18:17.416+0000 7fc111d1b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-20T12:18:17.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.432+0000 7ff4cdffb700 1 -- 172.21.15.116:0/3555599125 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) v1 ==== 161+0+0 (secure 0 0 0) 0x7ff4d8015070 con 0x7ff4e01559d0 2020-11-20T12:18:17.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 -- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4b80557c0 msgr2=0x7ff4b8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:17.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4b80557c0 0x7ff4b8057c80 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7ff4e00a8900 tx=0x7ff4dc006eb0).stop 2020-11-20T12:18:17.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 -- 172.21.15.116:0/3555599125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e01559d0 msgr2=0x7ff4e00a8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:17.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e01559d0 0x7ff4e00a8cb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7ff4d8011f40 tx=0x7ff4d8016000).stop 2020-11-20T12:18:17.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 -- 172.21.15.116:0/3555599125 shutdown_connections 2020-11-20T12:18:17.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4b80557c0 0x7ff4b8057c80 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:17.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4e01559d0 0x7ff4e00a8cb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:17.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4e00a3f30 0x7ff4e0165960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:17.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 --2- 172.21.15.116:0/3555599125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4e00a67f0 0x7ff4e014f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:17.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 -- 172.21.15.116:0/3555599125 >> 172.21.15.116:0/3555599125 conn(0x7ff4e0060170 msgr2=0x7ff4e0060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:17.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 -- 172.21.15.116:0/3555599125 shutdown_connections 2020-11-20T12:18:17.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.436+0000 7ff4e6813700 1 -- 172.21.15.116:0/3555599125 wait complete. 2020-11-20T12:18:17.437 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-20T12:18:17.444 INFO:tasks.ceph.mds.a.smithi116.stderr: -6795> 2020-11-20T12:17:44.544+0000 7f35907d3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:18:17.459 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-20T12:18:17.493 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:18:17.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.628+0000 7f76632a5700 1 Processor -- start 2020-11-20T12:18:17.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.632+0000 7f76632a5700 1 -- start start 2020-11-20T12:18:17.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.632+0000 7f76632a5700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f765c062ce0 0x7f765c13a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:17.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.632+0000 7f76632a5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f765c155480 0x7f765c152630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:17.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.632+0000 7f76632a5700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f765c152b20 0x7f765c0c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:17.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.632+0000 7f76632a5700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c13ae90 con 0x7f765c062ce0 2020-11-20T12:18:17.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.632+0000 7f76632a5700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765c155c70 con 0x7f765c152b20 2020-11-20T12:18:17.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.632+0000 7f76632a5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c155f10 con 0x7f765c155480 2020-11-20T12:18:17.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76622a3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f765c062ce0 0x7f765c13a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:17.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76622a3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f765c062ce0 0x7f765c13a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46502/0 (socket says 172.21.15.116:46502) 2020-11-20T12:18:17.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76622a3700 1 -- 172.21.15.116:0/1470266526 learned_addr learned my addr 172.21.15.116:0/1470266526 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:17.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f7662aa4700 1 --2- 172.21.15.116:0/1470266526 >> v2:172.21.15.173:3300/0 conn(0x7f765c152b20 0x7f765c0c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:17.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f7661aa2700 1 --2- 172.21.15.116:0/1470266526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f765c155480 0x7f765c152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:17.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76622a3700 1 -- 172.21.15.116:0/1470266526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f765c155480 msgr2=0x7f765c152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:17.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76622a3700 1 --2- 172.21.15.116:0/1470266526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f765c155480 0x7f765c152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:17.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76622a3700 1 -- 172.21.15.116:0/1470266526 >> v2:172.21.15.173:3300/0 conn(0x7f765c152b20 msgr2=0x7f765c0c2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:17.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76622a3700 1 --2- 172.21.15.116:0/1470266526 >> v2:172.21.15.173:3300/0 conn(0x7f765c152b20 0x7f765c0c2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:17.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76622a3700 1 -- 172.21.15.116:0/1470266526 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c1561b0 con 0x7f765c062ce0 2020-11-20T12:18:17.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76622a3700 1 --2- 172.21.15.116:0/1470266526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f765c062ce0 0x7f765c13a9a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f7658006fd0 tx=0x7f7658006eb0).ready entity=mon.0 client_cookie=55e2f1a0f5409a3c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:17.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76612a1700 1 -- 172.21.15.116:0/1470266526 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658009890 con 0x7f765c062ce0 2020-11-20T12:18:17.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76612a1700 1 -- 172.21.15.116:0/1470266526 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76580099f0 con 0x7f765c062ce0 2020-11-20T12:18:17.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76612a1700 1 -- 172.21.15.116:0/1470266526 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7658009d70 con 0x7f765c062ce0 2020-11-20T12:18:17.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76632a5700 1 -- 172.21.15.116:0/1470266526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f765c062ce0 msgr2=0x7f765c13a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:17.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76632a5700 1 --2- 172.21.15.116:0/1470266526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f765c062ce0 0x7f765c13a9a0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f7658006fd0 tx=0x7f7658006eb0).stop 2020-11-20T12:18:17.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76632a5700 1 -- 172.21.15.116:0/1470266526 shutdown_connections 2020-11-20T12:18:17.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76632a5700 1 --2- 172.21.15.116:0/1470266526 >> v2:172.21.15.173:3300/0 conn(0x7f765c152b20 0x7f765c0c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:17.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76632a5700 1 --2- 172.21.15.116:0/1470266526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f765c062ce0 0x7f765c13a9a0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:17.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76632a5700 1 --2- 172.21.15.116:0/1470266526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f765c155480 0x7f765c152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:17.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.636+0000 7f76632a5700 1 -- 172.21.15.116:0/1470266526 >> 172.21.15.116:0/1470266526 conn(0x7f765c07ff70 msgr2=0x7f765c080380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:17.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.640+0000 7f76632a5700 1 -- 172.21.15.116:0/1470266526 shutdown_connections 2020-11-20T12:18:17.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.640+0000 7f76632a5700 1 -- 172.21.15.116:0/1470266526 wait complete. 2020-11-20T12:18:17.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.640+0000 7f76632a5700 1 Processor -- start 2020-11-20T12:18:17.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.640+0000 7f76632a5700 1 -- start start 2020-11-20T12:18:17.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f76632a5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f765c062ce0 0x7f765c0be930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:17.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f76632a5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f765c152b20 0x7f765c164940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:17.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f76632a5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f765c155480 0x7f765c0bbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:17.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f76632a5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c0bc000 con 0x7f765c152b20 2020-11-20T12:18:17.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f76632a5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765c164fe0 con 0x7f765c062ce0 2020-11-20T12:18:17.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f76632a5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765c165280 con 0x7f765c155480 2020-11-20T12:18:17.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f7661aa2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f765c152b20 0x7f765c164940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:17.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f7661aa2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f765c152b20 0x7f765c164940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46512/0 (socket says 172.21.15.116:46512) 2020-11-20T12:18:17.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f7661aa2700 1 -- 172.21.15.116:0/1721347616 learned_addr learned my addr 172.21.15.116:0/1721347616 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:17.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f7662aa4700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f765c155480 0x7f765c0bbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:17.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f76622a3700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f765c062ce0 0x7f765c0be930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:17.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f7661aa2700 1 -- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f765c155480 msgr2=0x7f765c0bbb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:17.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f7661aa2700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f765c155480 0x7f765c0bbb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:17.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f7661aa2700 1 -- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f765c062ce0 msgr2=0x7f765c0be930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:17.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f7661aa2700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f765c062ce0 0x7f765c0be930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:17.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f7661aa2700 1 -- 172.21.15.116:0/1721347616 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c165520 con 0x7f765c152b20 2020-11-20T12:18:17.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f7661aa2700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f765c152b20 0x7f765c164940 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f764c00db60 tx=0x7f764c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:17.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f764a7fc700 1 -- 172.21.15.116:0/1721347616 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00be30 con 0x7f765c152b20 2020-11-20T12:18:17.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f76632a5700 1 -- 172.21.15.116:0/1721347616 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f765c1657c0 con 0x7f765c152b20 2020-11-20T12:18:17.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f764a7fc700 1 -- 172.21.15.116:0/1721347616 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c00b170 con 0x7f765c152b20 2020-11-20T12:18:17.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f764a7fc700 1 -- 172.21.15.116:0/1721347616 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c0099e0 con 0x7f765c152b20 2020-11-20T12:18:17.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.644+0000 7f76632a5700 1 -- 172.21.15.116:0/1721347616 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f765c0bc1f0 con 0x7f765c152b20 2020-11-20T12:18:17.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.648+0000 7f76632a5700 1 -- 172.21.15.116:0/1721347616 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f765c05cf30 con 0x7f765c152b20 2020-11-20T12:18:17.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.648+0000 7f764a7fc700 1 -- 172.21.15.116:0/1721347616 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f764c009d50 con 0x7f765c152b20 2020-11-20T12:18:17.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.652+0000 7f764a7fc700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76340557c0 0x7f7634057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:17.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.652+0000 7f764a7fc700 1 -- 172.21.15.116:0/1721347616 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f764c06ecd0 con 0x7f765c152b20 2020-11-20T12:18:17.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.652+0000 7f76622a3700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76340557c0 0x7f7634057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:17.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.652+0000 7f76622a3700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76340557c0 0x7f7634057c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f765800b1b0 tx=0x7f7658015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:17.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.656+0000 7f764a7fc700 1 -- 172.21.15.116:0/1721347616 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f764c0449b0 con 0x7f765c152b20 2020-11-20T12:18:17.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:17.860+0000 7f76632a5700 1 -- 172.21.15.116:0/1721347616 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f765c0bdf40 con 0x7f765c152b20 2020-11-20T12:18:18.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.436+0000 7f764a7fc700 1 -- 172.21.15.116:0/1721347616 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 104+0+0 (secure 0 0 0) 0x7f764c047fd0 con 0x7f765c152b20 2020-11-20T12:18:18.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 -- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76340557c0 msgr2=0x7f7634057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:18.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76340557c0 0x7f7634057c80 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f765800b1b0 tx=0x7f7658015000).stop 2020-11-20T12:18:18.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 -- 172.21.15.116:0/1721347616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f765c152b20 msgr2=0x7f765c164940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:18.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f765c152b20 0x7f765c164940 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f764c00db60 tx=0x7f764c008040).stop 2020-11-20T12:18:18.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 -- 172.21.15.116:0/1721347616 shutdown_connections 2020-11-20T12:18:18.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f76340557c0 0x7f7634057c80 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:18.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f765c155480 0x7f765c0bbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:18.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f765c062ce0 0x7f765c0be930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:18.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 --2- 172.21.15.116:0/1721347616 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f765c152b20 0x7f765c164940 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:18.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 -- 172.21.15.116:0/1721347616 >> 172.21.15.116:0/1721347616 conn(0x7f765c07ff70 msgr2=0x7f765c0c1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:18.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 -- 172.21.15.116:0/1721347616 shutdown_connections 2020-11-20T12:18:18.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.440+0000 7f76632a5700 1 -- 172.21.15.116:0/1721347616 wait complete. 2020-11-20T12:18:18.464 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-20T12:18:18.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.660+0000 7f2c729f3700 1 Processor -- start 2020-11-20T12:18:18.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.660+0000 7f2c729f3700 1 -- start start 2020-11-20T12:18:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c729f3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c6c14f450 0x7f2c6c064880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c729f3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2c6c0c2150 0x7f2c6c0bf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c729f3700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c6c152ff0 con 0x7f2c6c14f830 2020-11-20T12:18:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c729f3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c6c0babf0 con 0x7f2c6c0c2150 2020-11-20T12:18:18.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c729f3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c152af0 con 0x7f2c6c14f450 2020-11-20T12:18:18.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c70f90700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2c6c0c2150 0x7f2c6c0bf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:18.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c70f90700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2c6c0c2150 0x7f2c6c0bf4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46518/0 (socket says 172.21.15.116:46518) 2020-11-20T12:18:18.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c70f90700 1 -- 172.21.15.116:0/4079812744 learned_addr learned my addr 172.21.15.116:0/4079812744 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:18.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c6bfff700 1 --2- 172.21.15.116:0/4079812744 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c6c14f450 0x7f2c6c064880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:18.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c70f90700 1 -- 172.21.15.116:0/4079812744 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c6c14f450 msgr2=0x7f2c6c064880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c70f90700 1 --2- 172.21.15.116:0/4079812744 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c6c14f450 0x7f2c6c064880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c70f90700 1 -- 172.21.15.116:0/4079812744 >> v1:172.21.15.173:6789/0 conn(0x7f2c6c14f830 legacy=0x7f2c6c0b9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c70f90700 1 -- 172.21.15.116:0/4079812744 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6c152d50 con 0x7f2c6c0c2150 2020-11-20T12:18:18.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c70f90700 1 --2- 172.21.15.116:0/4079812744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c6c0c2150 0x7f2c6c0bf4b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f2c64009d50 tx=0x7f2c64009290).ready entity=mon.0 client_cookie=f35d2d24870c85d1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c6affd700 1 -- 172.21.15.116:0/4079812744 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400bb80 con 0x7f2c6c0c2150 2020-11-20T12:18:18.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c6affd700 1 -- 172.21.15.116:0/4079812744 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6400bce0 con 0x7f2c6c0c2150 2020-11-20T12:18:18.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c6affd700 1 -- 172.21.15.116:0/4079812744 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64011610 con 0x7f2c6c0c2150 2020-11-20T12:18:18.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c729f3700 1 -- 172.21.15.116:0/4079812744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c6c0c2150 msgr2=0x7f2c6c0bf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:18.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.664+0000 7f2c729f3700 1 --2- 172.21.15.116:0/4079812744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c6c0c2150 0x7f2c6c0bf4b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f2c64009d50 tx=0x7f2c64009290).stop 2020-11-20T12:18:18.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.668+0000 7f2c729f3700 1 -- 172.21.15.116:0/4079812744 shutdown_connections 2020-11-20T12:18:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.668+0000 7f2c729f3700 1 --2- 172.21.15.116:0/4079812744 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c6c0c2150 0x7f2c6c0bf4b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.668+0000 7f2c729f3700 1 --2- 172.21.15.116:0/4079812744 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c6c14f450 0x7f2c6c064880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.668+0000 7f2c729f3700 1 -- 172.21.15.116:0/4079812744 >> 172.21.15.116:0/4079812744 conn(0x7f2c6c0c57c0 msgr2=0x7f2c6c0c1790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.668+0000 7f2c729f3700 1 -- 172.21.15.116:0/4079812744 shutdown_connections 2020-11-20T12:18:18.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.668+0000 7f2c729f3700 1 -- 172.21.15.116:0/4079812744 wait complete. 2020-11-20T12:18:18.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.668+0000 7f2c729f3700 1 Processor -- start 2020-11-20T12:18:18.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.668+0000 7f2c729f3700 1 -- start start 2020-11-20T12:18:18.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c729f3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c6c0c2150 0x7f2c6c141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:18.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c729f3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c6c14f450 0x7f2c6c14e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:18.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c729f3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c6c14e670 0x7f2c6c0c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:18.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c729f3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0bd180 con 0x7f2c6c14e670 2020-11-20T12:18:18.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c729f3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0bd3e0 con 0x7f2c6c14f450 2020-11-20T12:18:18.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c729f3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c0bd680 con 0x7f2c6c0c2150 2020-11-20T12:18:18.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c70f90700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c6c14e670 0x7f2c6c0c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:18.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c70f90700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c6c14e670 0x7f2c6c0c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46526/0 (socket says 172.21.15.116:46526) 2020-11-20T12:18:18.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c70f90700 1 -- 172.21.15.116:0/1318263057 learned_addr learned my addr 172.21.15.116:0/1318263057 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:18.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c6bfff700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c6c0c2150 0x7f2c6c141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:18.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c6b7fe700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c6c14f450 0x7f2c6c14e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:18.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c70f90700 1 -- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c6c0c2150 msgr2=0x7f2c6c141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:18.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c70f90700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c6c0c2150 0x7f2c6c141710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:18.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c70f90700 1 -- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c6c14f450 msgr2=0x7f2c6c14e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:18.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c70f90700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c6c14f450 0x7f2c6c14e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:18.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c70f90700 1 -- 172.21.15.116:0/1318263057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c6c0bd920 con 0x7f2c6c14e670 2020-11-20T12:18:18.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c6b7fe700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c6c14f450 0x7f2c6c14e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:18.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c70f90700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c6c14e670 0x7f2c6c0c4030 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f2c640092d0 tx=0x7f2c64016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:18.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c4bfff700 1 -- 172.21.15.116:0/1318263057 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c6400bb80 con 0x7f2c6c14e670 2020-11-20T12:18:18.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c4bfff700 1 -- 172.21.15.116:0/1318263057 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c6400be50 con 0x7f2c6c14e670 2020-11-20T12:18:18.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c729f3700 1 -- 172.21.15.116:0/1318263057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c0bdbc0 con 0x7f2c6c14e670 2020-11-20T12:18:18.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c4bfff700 1 -- 172.21.15.116:0/1318263057 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64010e30 con 0x7f2c6c14e670 2020-11-20T12:18:18.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.672+0000 7f2c729f3700 1 -- 172.21.15.116:0/1318263057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c0be300 con 0x7f2c6c14e670 2020-11-20T12:18:18.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.676+0000 7f2c729f3700 1 -- 172.21.15.116:0/1318263057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c05d2a0 con 0x7f2c6c14e670 2020-11-20T12:18:18.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.676+0000 7f2c4bfff700 1 -- 172.21.15.116:0/1318263057 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2c64017070 con 0x7f2c6c14e670 2020-11-20T12:18:18.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.676+0000 7f2c4bfff700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c440557c0 0x7f2c44057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:18.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.680+0000 7f2c4bfff700 1 -- 172.21.15.116:0/1318263057 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2c64070a70 con 0x7f2c6c14e670 2020-11-20T12:18:18.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.680+0000 7f2c6bfff700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c440557c0 0x7f2c44057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:18.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.680+0000 7f2c6bfff700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c440557c0 0x7f2c44057c80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f2c5c006fd0 tx=0x7f2c5c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:18.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.684+0000 7f2c4bfff700 1 -- 172.21.15.116:0/1318263057 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2c64046940 con 0x7f2c6c14e670 2020-11-20T12:18:18.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:18.844+0000 7f2c729f3700 1 -- 172.21.15.116:0/1318263057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c6c14ed50 con 0x7f2c6c14e670 2020-11-20T12:18:19.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.444+0000 7f2c4bfff700 1 -- 172.21.15.116:0/1318263057 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v47) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2c64070e60 con 0x7f2c6c14e670 2020-11-20T12:18:19.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 -- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c440557c0 msgr2=0x7f2c44057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:19.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c440557c0 0x7f2c44057c80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f2c5c006fd0 tx=0x7f2c5c008040).stop 2020-11-20T12:18:19.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 -- 172.21.15.116:0/1318263057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c6c14e670 msgr2=0x7f2c6c0c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:19.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c6c14e670 0x7f2c6c0c4030 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f2c640092d0 tx=0x7f2c64016000).stop 2020-11-20T12:18:19.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 -- 172.21.15.116:0/1318263057 shutdown_connections 2020-11-20T12:18:19.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2c440557c0 0x7f2c44057c80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:19.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2c6c14e670 0x7f2c6c0c4030 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:19.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2c6c0c2150 0x7f2c6c141710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:19.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 --2- 172.21.15.116:0/1318263057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2c6c14f450 0x7f2c6c14e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:19.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 -- 172.21.15.116:0/1318263057 >> 172.21.15.116:0/1318263057 conn(0x7f2c6c0c57c0 msgr2=0x7f2c6c067970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:19.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 -- 172.21.15.116:0/1318263057 shutdown_connections 2020-11-20T12:18:19.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.448+0000 7f2c729f3700 1 -- 172.21.15.116:0/1318263057 wait complete. 2020-11-20T12:18:19.452 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2020-11-20T12:18:19.476 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-20T12:18:19.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.660+0000 7f9fb96c3700 1 Processor -- start 2020-11-20T12:18:19.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.660+0000 7f9fb96c3700 1 -- start start 2020-11-20T12:18:19.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.660+0000 7f9fb96c3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40c4e60 0x7f9fb4063110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.660+0000 7f9fb96c3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9fb4063600 0x7f9fb40677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.660+0000 7f9fb96c3700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb40a4b60 con 0x7f9fb40a9e00 2020-11-20T12:18:19.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.660+0000 7f9fb96c3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb40a4e00 con 0x7f9fb4063600 2020-11-20T12:18:19.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.660+0000 7f9fb96c3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4062730 con 0x7f9fb40c4e60 2020-11-20T12:18:19.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb37fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9fb4063600 0x7f9fb40677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:19.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb2ffd700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f9fb40a9e00 0x7f9fb40a4370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:52196/0 (socket says 172.21.15.116:52196) 2020-11-20T12:18:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb27fc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40c4e60 0x7f9fb4063110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb2ffd700 1 -- 172.21.15.116:0/217718317 learned_addr learned my addr 172.21.15.116:0/217718317 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:19.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb27fc700 1 -- 172.21.15.116:0/217718317 >> v2:172.21.15.173:3300/0 conn(0x7f9fb4063600 msgr2=0x7f9fb40677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb27fc700 1 --2- 172.21.15.116:0/217718317 >> v2:172.21.15.173:3300/0 conn(0x7f9fb4063600 0x7f9fb40677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb27fc700 1 -- 172.21.15.116:0/217718317 >> v1:172.21.15.173:6789/0 conn(0x7f9fb40a9e00 legacy=0x7f9fb40a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb27fc700 1 -- 172.21.15.116:0/217718317 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40629d0 con 0x7f9fb40c4e60 2020-11-20T12:18:19.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb27fc700 1 --2- 172.21.15.116:0/217718317 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40c4e60 0x7f9fb4063110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f9fa800b9a0 tx=0x7f9fa8006df0).ready entity=mon.2 client_cookie=3f9228da2b2ab21 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:19.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb1ffb700 1 -- 172.21.15.116:0/217718317 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa80097b0 con 0x7f9fb40c4e60 2020-11-20T12:18:19.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb1ffb700 1 -- 172.21.15.116:0/217718317 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa8009910 con 0x7f9fb40c4e60 2020-11-20T12:18:19.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb1ffb700 1 -- 172.21.15.116:0/217718317 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8009c90 con 0x7f9fb40c4e60 2020-11-20T12:18:19.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb96c3700 1 -- 172.21.15.116:0/217718317 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40c4e60 msgr2=0x7f9fb4063110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:19.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb96c3700 1 --2- 172.21.15.116:0/217718317 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40c4e60 0x7f9fb4063110 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f9fa800b9a0 tx=0x7f9fa8006df0).stop 2020-11-20T12:18:19.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb96c3700 1 -- 172.21.15.116:0/217718317 shutdown_connections 2020-11-20T12:18:19.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb96c3700 1 --2- 172.21.15.116:0/217718317 >> v2:172.21.15.173:3300/0 conn(0x7f9fb4063600 0x7f9fb40677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:19.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb96c3700 1 --2- 172.21.15.116:0/217718317 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40c4e60 0x7f9fb4063110 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:19.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.664+0000 7f9fb96c3700 1 -- 172.21.15.116:0/217718317 >> 172.21.15.116:0/217718317 conn(0x7f9fb4060c40 msgr2=0x7f9fb40c4380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:19.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.668+0000 7f9fb96c3700 1 -- 172.21.15.116:0/217718317 shutdown_connections 2020-11-20T12:18:19.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.668+0000 7f9fb96c3700 1 -- 172.21.15.116:0/217718317 wait complete. 2020-11-20T12:18:19.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.668+0000 7f9fb96c3700 1 Processor -- start 2020-11-20T12:18:19.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb96c3700 1 -- start start 2020-11-20T12:18:19.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb96c3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb4063600 0x7f9fb40a06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:19.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb96c3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40a9e00 0x7f9fb409d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:19.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb96c3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb40c4e60 0x7f9fb409d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:19.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb96c3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb41533d0 con 0x7f9fb4063600 2020-11-20T12:18:19.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb96c3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40a2080 con 0x7f9fb40c4e60 2020-11-20T12:18:19.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb96c3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb4165f30 con 0x7f9fb40a9e00 2020-11-20T12:18:19.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb4063600 0x7f9fb40a06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:19.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb27fc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40a9e00 0x7f9fb409d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:19.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb4063600 0x7f9fb40a06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46536/0 (socket says 172.21.15.116:46536) 2020-11-20T12:18:19.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb2ffd700 1 -- 172.21.15.116:0/663443735 learned_addr learned my addr 172.21.15.116:0/663443735 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:19.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb37fe700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb40c4e60 0x7f9fb409d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:19.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb2ffd700 1 -- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40a9e00 msgr2=0x7f9fb409d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:19.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb2ffd700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40a9e00 0x7f9fb409d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:19.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb2ffd700 1 -- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb40c4e60 msgr2=0x7f9fb409d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:19.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb2ffd700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb40c4e60 0x7f9fb409d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:19.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.672+0000 7f9fb2ffd700 1 -- 172.21.15.116:0/663443735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb41661d0 con 0x7f9fb4063600 2020-11-20T12:18:19.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.676+0000 7f9fb2ffd700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb4063600 0x7f9fb40a06c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f9fa400c710 tx=0x7f9fa4007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:19.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.676+0000 7f9f9affd700 1 -- 172.21.15.116:0/663443735 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa40072e0 con 0x7f9fb4063600 2020-11-20T12:18:19.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.676+0000 7f9f9affd700 1 -- 172.21.15.116:0/663443735 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa4007440 con 0x7f9fb4063600 2020-11-20T12:18:19.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.676+0000 7f9fb96c3700 1 -- 172.21.15.116:0/663443735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb40a22e0 con 0x7f9fb4063600 2020-11-20T12:18:19.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.676+0000 7f9f9affd700 1 -- 172.21.15.116:0/663443735 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa4005800 con 0x7f9fb4063600 2020-11-20T12:18:19.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.676+0000 7f9fb96c3700 1 -- 172.21.15.116:0/663443735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb40a8310 con 0x7f9fb4063600 2020-11-20T12:18:19.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.676+0000 7f9fb96c3700 1 -- 172.21.15.116:0/663443735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb405cf30 con 0x7f9fb4063600 2020-11-20T12:18:19.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.676+0000 7f9f9affd700 1 -- 172.21.15.116:0/663443735 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9fa401a060 con 0x7f9fb4063600 2020-11-20T12:18:19.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.680+0000 7f9f9affd700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 0x7f9f90057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:19.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.680+0000 7f9f9affd700 1 -- 172.21.15.116:0/663443735 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f9fa406dc20 con 0x7f9fb4063600 2020-11-20T12:18:19.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.680+0000 7f9fb27fc700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 0x7f9f90057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:19.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.680+0000 7f9fb27fc700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 0x7f9f90057c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9fa8006e30 tx=0x7f9fa8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:19.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.684+0000 7f9f9affd700 1 -- 172.21.15.116:0/663443735 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9fa4043c20 con 0x7f9fb4063600 2020-11-20T12:18:19.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:19.840+0000 7f9fb96c3700 1 -- 172.21.15.116:0/663443735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9fb4061860 con 0x7f9fb4063600 2020-11-20T12:18:20.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.456+0000 7f9f9affd700 1 -- 172.21.15.116:0/663443735 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v48) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9fa4018070 con 0x7f9fb4063600 2020-11-20T12:18:20.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 -- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 msgr2=0x7f9f90057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:20.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 0x7f9f90057c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9fa8006e30 tx=0x7f9fa8015000).stop 2020-11-20T12:18:20.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 -- 172.21.15.116:0/663443735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb4063600 msgr2=0x7f9fb40a06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:20.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb4063600 0x7f9fb40a06c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f9fa400c710 tx=0x7f9fa4007c40).stop 2020-11-20T12:18:20.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 -- 172.21.15.116:0/663443735 shutdown_connections 2020-11-20T12:18:20.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 0x7f9f90057c80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:20.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb40c4e60 0x7f9fb409d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:20.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb4063600 0x7f9fb40a06c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:20.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 --2- 172.21.15.116:0/663443735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb40a9e00 0x7f9fb409d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:20.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 -- 172.21.15.116:0/663443735 >> 172.21.15.116:0/663443735 conn(0x7f9fb4060c40 msgr2=0x7f9fb4153ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:20.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 -- 172.21.15.116:0/663443735 shutdown_connections 2020-11-20T12:18:20.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.460+0000 7f9fb96c3700 1 -- 172.21.15.116:0/663443735 wait complete. 2020-11-20T12:18:20.462 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2020-11-20T12:18:20.484 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T12:18:20.484 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-20T12:18:20.484 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-20T12:18:20.485 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:18:20.485 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-20T12:18:20.485 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-20T12:18:20.486 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-20T12:18:20.486 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:18:20.486 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T12:18:20.486 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-20T12:18:20.487 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-20T12:18:20.487 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:18:20.487 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-20T12:18:20.487 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-20T12:18:20.488 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-20T12:18:20.488 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:18:20.506 INFO:tasks.ceph.mds.a:Stopped 2020-11-20T12:18:20.506 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T12:18:20.507 INFO:tasks.ceph.mds.c:Stopped 2020-11-20T12:18:20.508 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-20T12:18:20.515 INFO:tasks.ceph.mds.b:Stopped 2020-11-20T12:18:20.515 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T12:18:20.516 INFO:tasks.ceph.mds.d:Stopped 2020-11-20T12:18:20.516 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-20T12:18:20.520 INFO:tasks.ceph.mds.b:Started 2020-11-20T12:18:20.521 DEBUG:teuthology.parallel:result is None 2020-11-20T12:18:20.523 INFO:tasks.ceph.mds.d:Started 2020-11-20T12:18:20.523 DEBUG:teuthology.parallel:result is None 2020-11-20T12:18:20.558 INFO:tasks.ceph.mds.a:Started 2020-11-20T12:18:20.558 DEBUG:teuthology.parallel:result is None 2020-11-20T12:18:20.563 INFO:tasks.ceph.mds.c:Started 2020-11-20T12:18:20.564 DEBUG:teuthology.parallel:result is None 2020-11-20T12:18:20.564 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-20T12:18:20.621 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2020-11-20T12:18:20.623 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2020-11-20T12:18:20.656 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:18:20.669 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:18:20.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2020-11-20T12:18:20.720 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2020-11-20T12:18:20.721 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2020-11-20T12:18:20.721 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2020-11-20T12:18:20.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.740+0000 7fb291732700 1 Processor -- start 2020-11-20T12:18:20.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.740+0000 7fb291732700 1 -- start start 2020-11-20T12:18:20.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb291732700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb28c0be9c0 0x7fb28c0c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:20.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb291732700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb28c155580 con 0x7fb28c0c5420 2020-11-20T12:18:20.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb291732700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb28c1557b0 con 0x7fb28c0c3f90 2020-11-20T12:18:20.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb291732700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb28c155a50 con 0x7fb28c0be9c0 2020-11-20T12:18:20.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb28b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb28c0be9c0 0x7fb28c0c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:20.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb28b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb28c0be9c0 0x7fb28c0c5800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46572/0 (socket says 172.21.15.116:46572) 2020-11-20T12:18:20.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb28b7fe700 1 -- 172.21.15.116:0/1990997370 learned_addr learned my addr 172.21.15.116:0/1990997370 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:20.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb28b7fe700 1 -- 172.21.15.116:0/1990997370 >> v1:172.21.15.173:6789/0 conn(0x7fb28c0c3f90 legacy=0x7fb28c151440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb28b7fe700 1 -- 172.21.15.116:0/1990997370 >> v1:172.21.15.116:6789/0 conn(0x7fb28c0c5420 legacy=0x7fb28c0c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb28a7fc700 1 -- 172.21.15.116:0/1990997370 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1077507886 0 0) 0x7fb28c155580 con 0x7fb28c0c5420 2020-11-20T12:18:20.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb28b7fe700 1 -- 172.21.15.116:0/1990997370 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c155cf0 con 0x7fb28c0be9c0 2020-11-20T12:18:20.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb28b7fe700 1 --2- 172.21.15.116:0/1990997370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb28c0be9c0 0x7fb28c0c5800 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fb284009d50 tx=0x7fb284009290).ready entity=mon.0 client_cookie=c3d020a31ae03176 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:20.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb28a7fc700 1 -- 172.21.15.116:0/1990997370 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28400bc80 con 0x7fb28c0be9c0 2020-11-20T12:18:20.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb28a7fc700 1 -- 172.21.15.116:0/1990997370 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb28400bde0 con 0x7fb28c0be9c0 2020-11-20T12:18:20.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb28a7fc700 1 -- 172.21.15.116:0/1990997370 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb284011610 con 0x7fb28c0be9c0 2020-11-20T12:18:20.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb291732700 1 -- 172.21.15.116:0/1990997370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb28c0be9c0 msgr2=0x7fb28c0c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:20.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.744+0000 7fb291732700 1 --2- 172.21.15.116:0/1990997370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb28c0be9c0 0x7fb28c0c5800 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fb284009d50 tx=0x7fb284009290).stop 2020-11-20T12:18:20.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.748+0000 7fb291732700 1 -- 172.21.15.116:0/1990997370 shutdown_connections 2020-11-20T12:18:20.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.748+0000 7fb291732700 1 --2- 172.21.15.116:0/1990997370 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb28c0be9c0 0x7fb28c0c5800 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:20.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.748+0000 7fb291732700 1 -- 172.21.15.116:0/1990997370 >> 172.21.15.116:0/1990997370 conn(0x7fb28c0678a0 msgr2=0x7fb28c067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:20.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.748+0000 7fb291732700 1 -- 172.21.15.116:0/1990997370 shutdown_connections 2020-11-20T12:18:20.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.748+0000 7fb291732700 1 -- 172.21.15.116:0/1990997370 wait complete. 2020-11-20T12:18:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.748+0000 7fb291732700 1 Processor -- start 2020-11-20T12:18:20.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb291732700 1 -- start start 2020-11-20T12:18:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb291732700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb28c0be9c0 0x7fb28c0c0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb291732700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb28c0c3f90 0x7fb28c0ab800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:20.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb291732700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb28c0c5420 0x7fb28c152710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb291732700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c154290 con 0x7fb28c0c5420 2020-11-20T12:18:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb291732700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c152010 con 0x7fb28c0be9c0 2020-11-20T12:18:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb291732700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28c1544f0 con 0x7fb28c0c3f90 2020-11-20T12:18:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb28b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb28c0c5420 0x7fb28c152710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:20.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb28b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb28c0c5420 0x7fb28c152710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46578/0 (socket says 172.21.15.116:46578) 2020-11-20T12:18:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb28b7fe700 1 -- 172.21.15.116:0/2257119562 learned_addr learned my addr 172.21.15.116:0/2257119562 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:20.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb28affd700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb28c0be9c0 0x7fb28c0c0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb2827fc700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb28c0c3f90 0x7fb28c0ab800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:20.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb28b7fe700 1 -- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb28c0c3f90 msgr2=0x7fb28c0ab800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb28b7fe700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb28c0c3f90 0x7fb28c0ab800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb28b7fe700 1 -- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb28c0be9c0 msgr2=0x7fb28c0c0030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.752+0000 7fb28b7fe700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb28c0be9c0 0x7fb28c0c0030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:20.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.756+0000 7fb28b7fe700 1 -- 172.21.15.116:0/2257119562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb28c154790 con 0x7fb28c0c5420 2020-11-20T12:18:20.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.756+0000 7fb2827fc700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb28c0c3f90 0x7fb28c0ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:20.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.756+0000 7fb28b7fe700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb28c0c5420 0x7fb28c152710 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fb2840092d0 tx=0x7fb284016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:20.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.756+0000 7fb2837fe700 1 -- 172.21.15.116:0/2257119562 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb284011e30 con 0x7fb28c0c5420 2020-11-20T12:18:20.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.756+0000 7fb2837fe700 1 -- 172.21.15.116:0/2257119562 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb284010b90 con 0x7fb28c0c5420 2020-11-20T12:18:20.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.756+0000 7fb291732700 1 -- 172.21.15.116:0/2257119562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb28c154a30 con 0x7fb28c0c5420 2020-11-20T12:18:20.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.756+0000 7fb2837fe700 1 -- 172.21.15.116:0/2257119562 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28401c470 con 0x7fb28c0c5420 2020-11-20T12:18:20.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.756+0000 7fb291732700 1 -- 172.21.15.116:0/2257119562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb28c0c0520 con 0x7fb28c0c5420 2020-11-20T12:18:20.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.756+0000 7fb291732700 1 -- 172.21.15.116:0/2257119562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb28c155ed0 con 0x7fb28c0c5420 2020-11-20T12:18:20.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.760+0000 7fb2837fe700 1 -- 172.21.15.116:0/2257119562 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb284017070 con 0x7fb28c0c5420 2020-11-20T12:18:20.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.760+0000 7fb2837fe700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2680557c0 0x7fb268057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:20.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.760+0000 7fb2837fe700 1 -- 172.21.15.116:0/2257119562 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fb284070190 con 0x7fb28c0c5420 2020-11-20T12:18:20.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.760+0000 7fb28affd700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2680557c0 0x7fb268057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:20.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.760+0000 7fb28affd700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2680557c0 0x7fb268057c80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fb27c003680 tx=0x7fb27c007b70).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.764+0000 7fb2837fe700 1 -- 172.21.15.116:0/2257119562 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb284046390 con 0x7fb28c0c5420 2020-11-20T12:18:20.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:20.940+0000 7fb291732700 1 -- 172.21.15.116:0/2257119562 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb28c05d2a0 con 0x7fb28c0c5420 2020-11-20T12:18:21.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.704+0000 7fb2837fe700 1 -- 172.21.15.116:0/2257119562 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb284015070 con 0x7fb28c0c5420 2020-11-20T12:18:21.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 -- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2680557c0 msgr2=0x7fb268057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:21.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2680557c0 0x7fb268057c80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7fb27c003680 tx=0x7fb27c007b70).stop 2020-11-20T12:18:21.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 -- 172.21.15.116:0/2257119562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb28c0c5420 msgr2=0x7fb28c152710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:21.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb28c0c5420 0x7fb28c152710 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fb2840092d0 tx=0x7fb284016000).stop 2020-11-20T12:18:21.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 -- 172.21.15.116:0/2257119562 shutdown_connections 2020-11-20T12:18:21.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2680557c0 0x7fb268057c80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:21.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb28c0c5420 0x7fb28c152710 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:21.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb28c0be9c0 0x7fb28c0c0030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:21.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 --2- 172.21.15.116:0/2257119562 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb28c0c3f90 0x7fb28c0ab800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:21.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 -- 172.21.15.116:0/2257119562 >> 172.21.15.116:0/2257119562 conn(0x7fb28c0678a0 msgr2=0x7fb28c0a9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:21.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 -- 172.21.15.116:0/2257119562 shutdown_connections 2020-11-20T12:18:21.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.708+0000 7fb291732700 1 -- 172.21.15.116:0/2257119562 wait complete. 2020-11-20T12:18:21.709 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2020-11-20T12:18:21.733 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-20T12:18:21.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.904+0000 7fb2361d3700 1 Processor -- start 2020-11-20T12:18:21.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.904+0000 7fb2361d3700 1 -- start start 2020-11-20T12:18:21.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.904+0000 7fb2361d3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb230153450 0x7fb230062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:21.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.904+0000 7fb2361d3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb2300677a0 0x7fb2300b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:21.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.904+0000 7fb2361d3700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2301543a0 con 0x7fb230062880 2020-11-20T12:18:21.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2361d3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb230153920 con 0x7fb230153450 2020-11-20T12:18:21.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2361d3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb230153bc0 con 0x7fb2300677a0 2020-11-20T12:18:21.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2349d0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb230153450 0x7fb230062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:21.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2351d1700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb230062880 0x7fb2300c4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56020/0 (socket says 172.21.15.116:56020) 2020-11-20T12:18:21.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2351d1700 1 -- 172.21.15.116:0/4123029215 learned_addr learned my addr 172.21.15.116:0/4123029215 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:21.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2359d2700 1 --2- 172.21.15.116:0/4123029215 >> v2:172.21.15.173:3300/0 conn(0x7fb2300677a0 0x7fb2300b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:21.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb21ffff700 1 -- 172.21.15.116:0/4123029215 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696900205 0 0) 0x7fb2301543a0 con 0x7fb230062880 2020-11-20T12:18:21.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb21ffff700 1 -- 172.21.15.116:0/4123029215 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb218003240 con 0x7fb230062880 2020-11-20T12:18:21.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2349d0700 1 -- 172.21.15.116:0/4123029215 >> v2:172.21.15.173:3300/0 conn(0x7fb2300677a0 msgr2=0x7fb2300b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:21.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2349d0700 1 --2- 172.21.15.116:0/4123029215 >> v2:172.21.15.173:3300/0 conn(0x7fb2300677a0 0x7fb2300b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:21.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2349d0700 1 -- 172.21.15.116:0/4123029215 >> v1:172.21.15.116:6789/0 conn(0x7fb230062880 legacy=0x7fb2300c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:21.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2349d0700 1 -- 172.21.15.116:0/4123029215 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb230067c30 con 0x7fb230153450 2020-11-20T12:18:21.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2349d0700 1 --2- 172.21.15.116:0/4123029215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb230153450 0x7fb230062c60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fb2300a3f20 tx=0x7fb220006df0).ready entity=mon.0 client_cookie=b3fea47355c569b5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:21.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb21ffff700 1 -- 172.21.15.116:0/4123029215 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2200097b0 con 0x7fb230153450 2020-11-20T12:18:21.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb21ffff700 1 -- 172.21.15.116:0/4123029215 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb220009910 con 0x7fb230153450 2020-11-20T12:18:21.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb21ffff700 1 -- 172.21.15.116:0/4123029215 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb220009c90 con 0x7fb230153450 2020-11-20T12:18:21.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2361d3700 1 -- 172.21.15.116:0/4123029215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb230153450 msgr2=0x7fb230062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:21.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2361d3700 1 --2- 172.21.15.116:0/4123029215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb230153450 0x7fb230062c60 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fb2300a3f20 tx=0x7fb220006df0).stop 2020-11-20T12:18:21.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2361d3700 1 -- 172.21.15.116:0/4123029215 shutdown_connections 2020-11-20T12:18:21.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2361d3700 1 --2- 172.21.15.116:0/4123029215 >> v2:172.21.15.173:3300/0 conn(0x7fb2300677a0 0x7fb2300b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:21.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2361d3700 1 --2- 172.21.15.116:0/4123029215 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb230153450 0x7fb230062c60 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:21.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.908+0000 7fb2361d3700 1 -- 172.21.15.116:0/4123029215 >> 172.21.15.116:0/4123029215 conn(0x7fb230060c40 msgr2=0x7fb2300c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:21.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.912+0000 7fb2361d3700 1 -- 172.21.15.116:0/4123029215 shutdown_connections 2020-11-20T12:18:21.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.912+0000 7fb2361d3700 1 -- 172.21.15.116:0/4123029215 wait complete. 2020-11-20T12:18:21.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.912+0000 7fb2361d3700 1 Processor -- start 2020-11-20T12:18:21.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2361d3700 1 -- start start 2020-11-20T12:18:21.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2361d3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb230062880 0x7fb2301610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:21.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2361d3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2300677a0 0x7fb230149610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:21.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2361d3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb230153450 0x7fb230164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:21.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2361d3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb230161940 con 0x7fb230153450 2020-11-20T12:18:21.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2361d3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb230161be0 con 0x7fb2300677a0 2020-11-20T12:18:21.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2361d3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb230161e80 con 0x7fb230062880 2020-11-20T12:18:21.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2359d2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb230153450 0x7fb230164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:21.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2359d2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb230153450 0x7fb230164670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46590/0 (socket says 172.21.15.116:46590) 2020-11-20T12:18:21.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2359d2700 1 -- 172.21.15.116:0/2331158121 learned_addr learned my addr 172.21.15.116:0/2331158121 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:21.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2349d0700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2300677a0 0x7fb230149610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:21.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2359d2700 1 -- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb230062880 msgr2=0x7fb2301610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:21.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2359d2700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb230062880 0x7fb2301610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:21.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2359d2700 1 -- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2300677a0 msgr2=0x7fb230149610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:21.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2359d2700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2300677a0 0x7fb230149610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:21.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2359d2700 1 -- 172.21.15.116:0/2331158121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2300c3940 con 0x7fb230153450 2020-11-20T12:18:21.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2349d0700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2300677a0 0x7fb230149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:21.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2359d2700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb230153450 0x7fb230164670 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fb22800eb70 tx=0x7fb228008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:21.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb21d7fa700 1 -- 172.21.15.116:0/2331158121 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800c830 con 0x7fb230153450 2020-11-20T12:18:21.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb21d7fa700 1 -- 172.21.15.116:0/2331158121 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22800c990 con 0x7fb230153450 2020-11-20T12:18:21.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb21d7fa700 1 -- 172.21.15.116:0/2331158121 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22800cd10 con 0x7fb230153450 2020-11-20T12:18:21.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2361d3700 1 -- 172.21.15.116:0/2331158121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2301615d0 con 0x7fb230153450 2020-11-20T12:18:21.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.916+0000 7fb2361d3700 1 -- 172.21.15.116:0/2331158121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb230149ca0 con 0x7fb230153450 2020-11-20T12:18:21.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.920+0000 7fb2361d3700 1 -- 172.21.15.116:0/2331158121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2300c37a0 con 0x7fb230153450 2020-11-20T12:18:21.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.920+0000 7fb21d7fa700 1 -- 172.21.15.116:0/2331158121 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb228013640 con 0x7fb230153450 2020-11-20T12:18:21.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.920+0000 7fb21d7fa700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2080557c0 0x7fb208057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:21.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.920+0000 7fb21d7fa700 1 -- 172.21.15.116:0/2331158121 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb22806e5c0 con 0x7fb230153450 2020-11-20T12:18:21.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.924+0000 7fb2351d1700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2080557c0 0x7fb208057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:21.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.924+0000 7fb2351d1700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2080557c0 0x7fb208057c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fb22c0055a0 tx=0x7fb22c0076a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:21.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:21.924+0000 7fb21d7fa700 1 -- 172.21.15.116:0/2331158121 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb2300c37a0 con 0x7fb230153450 2020-11-20T12:18:22.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.144+0000 7fb2361d3700 1 -- 172.21.15.116:0/2331158121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb2300c37a0 con 0x7fb230153450 2020-11-20T12:18:22.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.144+0000 7fb21d7fa700 1 -- 172.21.15.116:0/2331158121 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5730 (secure 0 0 0) 0x7fb2300c37a0 con 0x7fb230153450 2020-11-20T12:18:22.145 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:22.145 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:18:22.146 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2020-11-20T12:18:22.146 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.146 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2020-11-20T12:18:22.146 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw==", 2020-11-20T12:18:22.146 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.147 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:22.148 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.148 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.148 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.148 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.149 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.149 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.149 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2020-11-20T12:18:22.149 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ==", 2020-11-20T12:18:22.150 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.150 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:22.150 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.150 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.150 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.151 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.151 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.151 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.151 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2020-11-20T12:18:22.151 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag==", 2020-11-20T12:18:22.152 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.152 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:22.152 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.152 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.152 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.153 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.153 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.153 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.153 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2020-11-20T12:18:22.154 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ==", 2020-11-20T12:18:22.154 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.154 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:22.154 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.155 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.155 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.155 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.155 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.155 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.156 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2020-11-20T12:18:22.156 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ==", 2020-11-20T12:18:22.156 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.156 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.156 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.157 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.157 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.157 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.157 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.157 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2020-11-20T12:18:22.158 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q==", 2020-11-20T12:18:22.158 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.158 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.158 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.159 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.159 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.159 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.159 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.159 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2020-11-20T12:18:22.160 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw==", 2020-11-20T12:18:22.160 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.160 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.160 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.160 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.161 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.161 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.161 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.161 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2020-11-20T12:18:22.161 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w==", 2020-11-20T12:18:22.162 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.162 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.162 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.162 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.162 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.163 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.163 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.163 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2020-11-20T12:18:22.163 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA==", 2020-11-20T12:18:22.163 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.164 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.164 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.164 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.164 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.165 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.165 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.165 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2020-11-20T12:18:22.165 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ==", 2020-11-20T12:18:22.165 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.166 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.166 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.166 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.166 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.167 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.167 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.167 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2020-11-20T12:18:22.167 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg==", 2020-11-20T12:18:22.167 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.168 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.168 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.168 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.168 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.168 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.169 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.169 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2020-11-20T12:18:22.169 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg==", 2020-11-20T12:18:22.169 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.169 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.170 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.170 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.170 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.170 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.171 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.171 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2020-11-20T12:18:22.171 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA==", 2020-11-20T12:18:22.171 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.171 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:22.172 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2020-11-20T12:18:22.172 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-20T12:18:22.172 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.172 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.172 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.173 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2020-11-20T12:18:22.173 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow==", 2020-11-20T12:18:22.173 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.173 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:22.173 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:18:22.174 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:18:22.174 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:18:22.174 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.174 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.174 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.175 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2020-11-20T12:18:22.175 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ==", 2020-11-20T12:18:22.175 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.175 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:22.176 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:18:22.176 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:18:22.176 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:18:22.176 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.177 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.177 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.177 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.21501", 2020-11-20T12:18:22.177 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQD7s7dfGvK8ABAAQrQSMNUDrPgMZ8IKp73f3Q==", 2020-11-20T12:18:22.178 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.178 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow rw path=/volumes/grpid/volid", 2020-11-20T12:18:22.178 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2020-11-20T12:18:22.178 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw pool=cephfs_data namespace=fsvolumens_volid" 2020-11-20T12:18:22.179 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.179 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.179 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.179 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2020-11-20T12:18:22.179 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ==", 2020-11-20T12:18:22.180 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.180 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:18:22.180 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:18:22.180 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:18:22.180 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:18:22.181 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.181 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.181 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.181 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2020-11-20T12:18:22.181 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBDs7dfh+KKOBAAZvcPXWly9/mxRE7NWuR85A==", 2020-11-20T12:18:22.182 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.182 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:18:22.182 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:18:22.182 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.182 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.183 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.183 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.183 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.183 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.manila", 2020-11-20T12:18:22.184 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQD5s7dfn0HnMxAAXUDbTOIUNzTn4X3w7Zfkpw==", 2020-11-20T12:18:22.184 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.184 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:18:22.184 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:18:22.184 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw" 2020-11-20T12:18:22.185 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.185 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.185 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.185 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2020-11-20T12:18:22.185 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A==", 2020-11-20T12:18:22.186 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.186 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:18:22.186 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:18:22.186 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.186 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.187 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:22.187 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:22.187 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2020-11-20T12:18:22.187 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA==", 2020-11-20T12:18:22.187 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:18:22.188 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:18:22.188 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:18:22.188 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:18:22.188 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.188 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:22.189 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:22.189 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:18:22.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 -- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2080557c0 msgr2=0x7fb208057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2080557c0 0x7fb208057c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fb22c0055a0 tx=0x7fb22c0076a0).stop 2020-11-20T12:18:22.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 -- 172.21.15.116:0/2331158121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb230153450 msgr2=0x7fb230164670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb230153450 0x7fb230164670 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fb22800eb70 tx=0x7fb228008040).stop 2020-11-20T12:18:22.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 -- 172.21.15.116:0/2331158121 shutdown_connections 2020-11-20T12:18:22.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb2080557c0 0x7fb208057c80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb230153450 0x7fb230164670 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb230062880 0x7fb2301610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:22.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 --2- 172.21.15.116:0/2331158121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb2300677a0 0x7fb230149610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 -- 172.21.15.116:0/2331158121 >> 172.21.15.116:0/2331158121 conn(0x7fb230060c40 msgr2=0x7fb2300a53f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:22.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 -- 172.21.15.116:0/2331158121 shutdown_connections 2020-11-20T12:18:22.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.148+0000 7fb2361d3700 1 -- 172.21.15.116:0/2331158121 wait complete. 2020-11-20T12:18:22.193 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.21501 2020-11-20T12:18:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.372+0000 7f5cc930a700 1 Processor -- start 2020-11-20T12:18:22.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.372+0000 7f5cc930a700 1 -- start start 2020-11-20T12:18:22.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.372+0000 7f5cc930a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6c20 0x7f5cc414fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:22.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.372+0000 7f5cc930a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc4150340 con 0x7f5cc4066cb0 2020-11-20T12:18:22.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.372+0000 7f5cc930a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cc40677e0 con 0x7f5cc40a6840 2020-11-20T12:18:22.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.372+0000 7f5cc930a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4067a80 con 0x7f5cc40a6c20 2020-11-20T12:18:22.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc37fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5cc4066cb0 0x7f5cc4153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56036/0 (socket says 172.21.15.116:56036) 2020-11-20T12:18:22.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc37fe700 1 -- 172.21.15.116:0/2586160757 learned_addr learned my addr 172.21.15.116:0/2586160757 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:22.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc27fc700 1 --2- 172.21.15.116:0/2586160757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6c20 0x7f5cc414fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:22.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc1ffb700 1 -- 172.21.15.116:0/2586160757 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472190112 0 0) 0x7f5cc4150340 con 0x7f5cc4066cb0 2020-11-20T12:18:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc1ffb700 1 -- 172.21.15.116:0/2586160757 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cb0003240 con 0x7f5cc4066cb0 2020-11-20T12:18:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc27fc700 1 -- 172.21.15.116:0/2586160757 >> v1:172.21.15.173:6789/0 conn(0x7f5cc40a6840 legacy=0x7f5cc40bff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc1ffb700 1 -- 172.21.15.116:0/2586160757 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290162064 0 0) 0x7f5cc40677e0 con 0x7f5cc40a6840 2020-11-20T12:18:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc27fc700 1 -- 172.21.15.116:0/2586160757 >> v1:172.21.15.116:6789/0 conn(0x7f5cc4066cb0 legacy=0x7f5cc4153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc27fc700 1 -- 172.21.15.116:0/2586160757 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc4067d20 con 0x7f5cc40a6c20 2020-11-20T12:18:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc27fc700 1 --2- 172.21.15.116:0/2586160757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6c20 0x7f5cc414fe50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f5cb800b9d0 tx=0x7f5cb8006df0).ready entity=mon.2 client_cookie=d8393273455d1e2b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc1ffb700 1 -- 172.21.15.116:0/2586160757 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb80096b0 con 0x7f5cc40a6c20 2020-11-20T12:18:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc1ffb700 1 -- 172.21.15.116:0/2586160757 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb8009810 con 0x7f5cc40a6c20 2020-11-20T12:18:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc1ffb700 1 -- 172.21.15.116:0/2586160757 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8009b90 con 0x7f5cc40a6c20 2020-11-20T12:18:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc930a700 1 -- 172.21.15.116:0/2586160757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6c20 msgr2=0x7f5cc414fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc930a700 1 --2- 172.21.15.116:0/2586160757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6c20 0x7f5cc414fe50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f5cb800b9d0 tx=0x7f5cb8006df0).stop 2020-11-20T12:18:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc930a700 1 -- 172.21.15.116:0/2586160757 shutdown_connections 2020-11-20T12:18:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc930a700 1 --2- 172.21.15.116:0/2586160757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6c20 0x7f5cc414fe50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc930a700 1 -- 172.21.15.116:0/2586160757 >> 172.21.15.116:0/2586160757 conn(0x7f5cc4060170 msgr2=0x7f5cc4060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc930a700 1 -- 172.21.15.116:0/2586160757 shutdown_connections 2020-11-20T12:18:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.376+0000 7f5cc930a700 1 -- 172.21.15.116:0/2586160757 wait complete. 2020-11-20T12:18:22.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.380+0000 7f5cc930a700 1 Processor -- start 2020-11-20T12:18:22.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.380+0000 7f5cc930a700 1 -- start start 2020-11-20T12:18:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc930a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cc4066cb0 0x7f5cc414a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc930a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6840 0x7f5cc4148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc930a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cc40a6c20 0x7f5cc41493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc930a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc41622a0 con 0x7f5cc4066cb0 2020-11-20T12:18:22.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc930a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc4163150 con 0x7f5cc40a6c20 2020-11-20T12:18:22.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc930a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc408e260 con 0x7f5cc40a6840 2020-11-20T12:18:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cc4066cb0 0x7f5cc414a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cc4066cb0 0x7f5cc414a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46606/0 (socket says 172.21.15.116:46606) 2020-11-20T12:18:22.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc2ffd700 1 -- 172.21.15.116:0/1041707413 learned_addr learned my addr 172.21.15.116:0/1041707413 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc37fe700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cc40a6c20 0x7f5cc41493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc27fc700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6840 0x7f5cc4148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc2ffd700 1 -- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6840 msgr2=0x7f5cc4148ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc2ffd700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6840 0x7f5cc4148ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc2ffd700 1 -- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cc40a6c20 msgr2=0x7f5cc41493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc2ffd700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cc40a6c20 0x7f5cc41493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc2ffd700 1 -- 172.21.15.116:0/1041707413 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc41624d0 con 0x7f5cc4066cb0 2020-11-20T12:18:22.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc27fc700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6840 0x7f5cc4148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:22.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc2ffd700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cc4066cb0 0x7f5cc414a230 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f5cb4007fd0 tx=0x7f5cb4007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:22.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5caaffd700 1 -- 172.21.15.116:0/1041707413 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb400e560 con 0x7f5cc4066cb0 2020-11-20T12:18:22.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5caaffd700 1 -- 172.21.15.116:0/1041707413 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb400e6e0 con 0x7f5cc4066cb0 2020-11-20T12:18:22.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5caaffd700 1 -- 172.21.15.116:0/1041707413 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb400eaa0 con 0x7f5cc4066cb0 2020-11-20T12:18:22.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc930a700 1 -- 172.21.15.116:0/1041707413 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc4162770 con 0x7f5cc4066cb0 2020-11-20T12:18:22.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.384+0000 7f5cc930a700 1 -- 172.21.15.116:0/1041707413 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc4162cb0 con 0x7f5cc4066cb0 2020-11-20T12:18:22.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.388+0000 7f5cc930a700 1 -- 172.21.15.116:0/1041707413 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc4162eb0 con 0x7f5cc4066cb0 2020-11-20T12:18:22.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.388+0000 7f5caaffd700 1 -- 172.21.15.116:0/1041707413 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5cb4019030 con 0x7f5cc4066cb0 2020-11-20T12:18:22.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.388+0000 7f5caaffd700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ca00557c0 0x7f5ca0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:22.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.388+0000 7f5caaffd700 1 -- 172.21.15.116:0/1041707413 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f5cb406d7b0 con 0x7f5cc4066cb0 2020-11-20T12:18:22.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.388+0000 7f5cc27fc700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ca00557c0 0x7f5ca0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:22.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.392+0000 7f5cc27fc700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ca00557c0 0x7f5ca0057c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f5cb8006e30 tx=0x7f5cb8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:22.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.392+0000 7f5caaffd700 1 -- 172.21.15.116:0/1041707413 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5cc4162eb0 con 0x7f5cc4066cb0 2020-11-20T12:18:22.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.604+0000 7f5cc930a700 1 -- 172.21.15.116:0/1041707413 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.21501"} v 0) v1 -- 0x7f5cc405d2a0 con 0x7f5cc4066cb0 2020-11-20T12:18:22.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.616+0000 7f5caaffd700 1 -- 172.21.15.116:0/1041707413 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.21501"}]=0 updated v14) v1 ==== 89+0+0 (secure 0 0 0) 0x7f5cb4043a30 con 0x7f5cc4066cb0 2020-11-20T12:18:22.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 -- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ca00557c0 msgr2=0x7f5ca0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ca00557c0 0x7f5ca0057c80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f5cb8006e30 tx=0x7f5cb8015000).stop 2020-11-20T12:18:22.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 -- 172.21.15.116:0/1041707413 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cc4066cb0 msgr2=0x7f5cc414a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cc4066cb0 0x7f5cc414a230 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f5cb4007fd0 tx=0x7f5cb4007ac0).stop 2020-11-20T12:18:22.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 -- 172.21.15.116:0/1041707413 shutdown_connections 2020-11-20T12:18:22.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5ca00557c0 0x7f5ca0057c80 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5cc40a6c20 0x7f5cc41493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5cc4066cb0 0x7f5cc414a230 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 --2- 172.21.15.116:0/1041707413 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5cc40a6840 0x7f5cc4148ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 -- 172.21.15.116:0/1041707413 >> 172.21.15.116:0/1041707413 conn(0x7f5cc4060170 msgr2=0x7f5cc40a73d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:22.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 -- 172.21.15.116:0/1041707413 shutdown_connections 2020-11-20T12:18:22.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.620+0000 7f5cc930a700 1 -- 172.21.15.116:0/1041707413 wait complete. 2020-11-20T12:18:22.621 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:18:22.644 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.manila 2020-11-20T12:18:22.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.816+0000 7f060181e700 1 Processor -- start 2020-11-20T12:18:22.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.816+0000 7f060181e700 1 -- start start 2020-11-20T12:18:22.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f060181e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f05fc14bef0 0x7f05fc154700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:22.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f060181e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc14f9f0 con 0x7f05fc0c3db0 2020-11-20T12:18:22.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f060181e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc1530d0 con 0x7f05fc14bb10 2020-11-20T12:18:22.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f060181e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05fc153370 con 0x7f05fc14bef0 2020-11-20T12:18:22.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f05fa7fc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f05fc0c3db0 0x7f05fc153ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56046/0 (socket says 172.21.15.116:56046) 2020-11-20T12:18:22.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f05fa7fc700 1 -- 172.21.15.116:0/1678780121 learned_addr learned my addr 172.21.15.116:0/1678780121 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:22.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f05fb7fe700 1 --2- 172.21.15.116:0/1678780121 >> v2:172.21.15.173:3300/0 conn(0x7f05fc14bef0 0x7f05fc154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:22.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f05f9ffb700 1 -- 172.21.15.116:0/1678780121 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 596969321 0 0) 0x7f05fc14f9f0 con 0x7f05fc0c3db0 2020-11-20T12:18:22.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f05f9ffb700 1 -- 172.21.15.116:0/1678780121 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05e8003240 con 0x7f05fc0c3db0 2020-11-20T12:18:22.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f05fb7fe700 1 -- 172.21.15.116:0/1678780121 >> v1:172.21.15.173:6789/0 conn(0x7f05fc14bb10 legacy=0x7f05fc0c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f05fb7fe700 1 -- 172.21.15.116:0/1678780121 >> v1:172.21.15.116:6789/0 conn(0x7f05fc0c3db0 legacy=0x7f05fc153ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f05fb7fe700 1 -- 172.21.15.116:0/1678780121 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc153610 con 0x7f05fc14bef0 2020-11-20T12:18:22.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.820+0000 7f05fb7fe700 1 --2- 172.21.15.116:0/1678780121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05fc14bef0 0x7f05fc154700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f05f4009c90 tx=0x7f05f40091d0).ready entity=mon.1 client_cookie=7ad821901ba86b86 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:22.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f05f9ffb700 1 -- 172.21.15.116:0/1678780121 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f400ba70 con 0x7f05fc14bef0 2020-11-20T12:18:22.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f05f9ffb700 1 -- 172.21.15.116:0/1678780121 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f400bbd0 con 0x7f05fc14bef0 2020-11-20T12:18:22.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f05f9ffb700 1 -- 172.21.15.116:0/1678780121 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4011400 con 0x7f05fc14bef0 2020-11-20T12:18:22.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f060181e700 1 -- 172.21.15.116:0/1678780121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05fc14bef0 msgr2=0x7f05fc154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f060181e700 1 --2- 172.21.15.116:0/1678780121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05fc14bef0 0x7f05fc154700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f05f4009c90 tx=0x7f05f40091d0).stop 2020-11-20T12:18:22.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f060181e700 1 -- 172.21.15.116:0/1678780121 shutdown_connections 2020-11-20T12:18:22.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f060181e700 1 --2- 172.21.15.116:0/1678780121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05fc14bef0 0x7f05fc154700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f060181e700 1 -- 172.21.15.116:0/1678780121 >> 172.21.15.116:0/1678780121 conn(0x7f05fc0615b0 msgr2=0x7f05fc061990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:22.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f060181e700 1 -- 172.21.15.116:0/1678780121 shutdown_connections 2020-11-20T12:18:22.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f060181e700 1 -- 172.21.15.116:0/1678780121 wait complete. 2020-11-20T12:18:22.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f060181e700 1 Processor -- start 2020-11-20T12:18:22.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.824+0000 7f060181e700 1 -- start start 2020-11-20T12:18:22.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f060181e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05fc0c3db0 0x7f05fc161f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:22.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f060181e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05fc14bb10 0x7f05fc137650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:22.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f060181e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05fc14bef0 0x7f05fc14aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:22.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f060181e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0acfd0 con 0x7f05fc14bef0 2020-11-20T12:18:22.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f060181e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1623f0 con 0x7f05fc14bb10 2020-11-20T12:18:22.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f060181e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc165070 con 0x7f05fc0c3db0 2020-11-20T12:18:22.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05fb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05fc14bef0 0x7f05fc14aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:22.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05fb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05fc14bef0 0x7f05fc14aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46620/0 (socket says 172.21.15.116:46620) 2020-11-20T12:18:22.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05fb7fe700 1 -- 172.21.15.116:0/917399774 learned_addr learned my addr 172.21.15.116:0/917399774 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:22.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05fa7fc700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05fc14bb10 0x7f05fc137650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:22.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05faffd700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05fc0c3db0 0x7f05fc161f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:22.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05fb7fe700 1 -- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05fc0c3db0 msgr2=0x7f05fc161f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05fb7fe700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05fc0c3db0 0x7f05fc161f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05fb7fe700 1 -- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05fc14bb10 msgr2=0x7f05fc137650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:22.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05fb7fe700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05fc14bb10 0x7f05fc137650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:22.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05fb7fe700 1 -- 172.21.15.116:0/917399774 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0ad230 con 0x7f05fc14bef0 2020-11-20T12:18:22.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05faffd700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05fc0c3db0 0x7f05fc161f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:22.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05fb7fe700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05fc14bef0 0x7f05fc14aac0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f05fc061180 tx=0x7f05f4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:22.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05e2ffd700 1 -- 172.21.15.116:0/917399774 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4010700 con 0x7f05fc14bef0 2020-11-20T12:18:22.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05e2ffd700 1 -- 172.21.15.116:0/917399774 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f4010860 con 0x7f05fc14bef0 2020-11-20T12:18:22.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f060181e700 1 -- 172.21.15.116:0/917399774 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc0ad4d0 con 0x7f05fc14bef0 2020-11-20T12:18:22.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f05e2ffd700 1 -- 172.21.15.116:0/917399774 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f4010be0 con 0x7f05fc14bef0 2020-11-20T12:18:22.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.828+0000 7f060181e700 1 -- 172.21.15.116:0/917399774 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc137b40 con 0x7f05fc14bef0 2020-11-20T12:18:22.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.832+0000 7f060181e700 1 -- 172.21.15.116:0/917399774 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc05cd40 con 0x7f05fc14bef0 2020-11-20T12:18:22.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.832+0000 7f05e2ffd700 1 -- 172.21.15.116:0/917399774 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f05f4024030 con 0x7f05fc14bef0 2020-11-20T12:18:22.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.832+0000 7f05e2ffd700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05d80557c0 0x7f05d8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:22.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.832+0000 7f05e2ffd700 1 -- 172.21.15.116:0/917399774 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f05f406ffe0 con 0x7f05fc14bef0 2020-11-20T12:18:22.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.832+0000 7f05faffd700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05d80557c0 0x7f05d8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:22.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.836+0000 7f05faffd700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05d80557c0 0x7f05d8057c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f05ec0014d0 tx=0x7f05ec009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:22.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:22.840+0000 7f05e2ffd700 1 -- 172.21.15.116:0/917399774 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f05f4046280 con 0x7f05fc14bef0 2020-11-20T12:18:23.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.048+0000 7f060181e700 1 -- 172.21.15.116:0/917399774 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.manila"} v 0) v1 -- 0x7f05fc060d70 con 0x7f05fc14bef0 2020-11-20T12:18:23.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f05e2ffd700 1 -- 172.21.15.116:0/917399774 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.manila"}]=0 updated v15) v1 ==== 90+0+0 (secure 0 0 0) 0x7f05f4022070 con 0x7f05fc14bef0 2020-11-20T12:18:23.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 -- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05d80557c0 msgr2=0x7f05d8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:23.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05d80557c0 0x7f05d8057c80 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f05ec0014d0 tx=0x7f05ec009040).stop 2020-11-20T12:18:23.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 -- 172.21.15.116:0/917399774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05fc14bef0 msgr2=0x7f05fc14aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:23.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05fc14bef0 0x7f05fc14aac0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f05fc061180 tx=0x7f05f4016000).stop 2020-11-20T12:18:23.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 -- 172.21.15.116:0/917399774 shutdown_connections 2020-11-20T12:18:23.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05d80557c0 0x7f05d8057c80 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05fc14bef0 0x7f05fc14aac0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05fc0c3db0 0x7f05fc161f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 --2- 172.21.15.116:0/917399774 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05fc14bb10 0x7f05fc137650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 -- 172.21.15.116:0/917399774 >> 172.21.15.116:0/917399774 conn(0x7f05fc0615b0 msgr2=0x7f05fc0c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:23.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 -- 172.21.15.116:0/917399774 shutdown_connections 2020-11-20T12:18:23.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.056+0000 7f060181e700 1 -- 172.21.15.116:0/917399774 wait complete. 2020-11-20T12:18:23.060 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:18:23.081 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:23.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.244+0000 7f162ae87700 1 Processor -- start 2020-11-20T12:18:23.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.244+0000 7f162ae87700 1 -- start start 2020-11-20T12:18:23.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.244+0000 7f162ae87700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f162413afa0 0x7f16240aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:23.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.244+0000 7f162ae87700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162413b380 0x7f16240aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:23.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.244+0000 7f162ae87700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16240ada00 con 0x7f16240acd30 2020-11-20T12:18:23.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.244+0000 7f162ae87700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16240adca0 con 0x7f162413afa0 2020-11-20T12:18:23.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.244+0000 7f162ae87700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16240adf40 con 0x7f162413b380 2020-11-20T12:18:23.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1629e85700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f162413afa0 0x7f16240aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:23.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1629e85700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f162413afa0 0x7f16240aa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46630/0 (socket says 172.21.15.116:46630) 2020-11-20T12:18:23.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1629684700 1 --2- 172.21.15.116:0/4007385854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162413b380 0x7f16240aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:23.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1629e85700 1 -- 172.21.15.116:0/4007385854 learned_addr learned my addr 172.21.15.116:0/4007385854 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:23.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f162a686700 1 --1- 172.21.15.116:0/4007385854 >> v1:172.21.15.173:6789/0 conn(0x7f16240acd30 0x7f16240ad1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:52292/0 (socket says 172.21.15.116:52292) 2020-11-20T12:18:23.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1629e85700 1 -- 172.21.15.116:0/4007385854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162413b380 msgr2=0x7f16240aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:23.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1629e85700 1 --2- 172.21.15.116:0/4007385854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162413b380 0x7f16240aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1629e85700 1 -- 172.21.15.116:0/4007385854 >> v1:172.21.15.173:6789/0 conn(0x7f16240acd30 legacy=0x7f16240ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:23.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1629e85700 1 -- 172.21.15.116:0/4007385854 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16240ae1e0 con 0x7f162413afa0 2020-11-20T12:18:23.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1629e85700 1 --2- 172.21.15.116:0/4007385854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f162413afa0 0x7f16240aa5b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f1620006fd0 tx=0x7f1620006eb0).ready entity=mon.0 client_cookie=92f2c6cf35ec08c2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:23.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1628e83700 1 -- 172.21.15.116:0/4007385854 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620009810 con 0x7f162413afa0 2020-11-20T12:18:23.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1628e83700 1 -- 172.21.15.116:0/4007385854 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1620009970 con 0x7f162413afa0 2020-11-20T12:18:23.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f1628e83700 1 -- 172.21.15.116:0/4007385854 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1620009cf0 con 0x7f162413afa0 2020-11-20T12:18:23.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f162ae87700 1 -- 172.21.15.116:0/4007385854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f162413afa0 msgr2=0x7f16240aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:23.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f162ae87700 1 --2- 172.21.15.116:0/4007385854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f162413afa0 0x7f16240aa5b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f1620006fd0 tx=0x7f1620006eb0).stop 2020-11-20T12:18:23.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f162ae87700 1 -- 172.21.15.116:0/4007385854 shutdown_connections 2020-11-20T12:18:23.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f162ae87700 1 --2- 172.21.15.116:0/4007385854 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f162413afa0 0x7f16240aa5b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f162ae87700 1 --2- 172.21.15.116:0/4007385854 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f162413b380 0x7f16240aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f162ae87700 1 -- 172.21.15.116:0/4007385854 >> 172.21.15.116:0/4007385854 conn(0x7f16240678a0 msgr2=0x7f1624067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:23.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f162ae87700 1 -- 172.21.15.116:0/4007385854 shutdown_connections 2020-11-20T12:18:23.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.248+0000 7f162ae87700 1 -- 172.21.15.116:0/4007385854 wait complete. 2020-11-20T12:18:23.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.252+0000 7f162ae87700 1 Processor -- start 2020-11-20T12:18:23.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.252+0000 7f162ae87700 1 -- start start 2020-11-20T12:18:23.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f162ae87700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16240acd30 0x7f16240a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:23.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f162ae87700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f162413b380 0x7f16240a2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:23.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f162ae87700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16240a3300 0x7f1624151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:23.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f162ae87700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16240a24e0 con 0x7f162413b380 2020-11-20T12:18:23.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f162ae87700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16240a2780 con 0x7f16240a3300 2020-11-20T12:18:23.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f162ae87700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162409f3c0 con 0x7f16240acd30 2020-11-20T12:18:23.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1629684700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f162413b380 0x7f16240a2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:23.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1629e85700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16240acd30 0x7f16240a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:23.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1629e85700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16240acd30 0x7f16240a1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:55582/0 (socket says 172.21.15.116:55582) 2020-11-20T12:18:23.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1629e85700 1 -- 172.21.15.116:0/2324116603 learned_addr learned my addr 172.21.15.116:0/2324116603 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:23.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1629684700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f162413b380 0x7f16240a2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46636/0 (socket says 172.21.15.116:46636) 2020-11-20T12:18:23.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f162a686700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16240a3300 0x7f1624151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:23.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1629684700 1 -- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16240acd30 msgr2=0x7f16240a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:23.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1629684700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16240acd30 0x7f16240a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1629684700 1 -- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16240a3300 msgr2=0x7f1624151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:23.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1629684700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16240a3300 0x7f1624151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1629684700 1 -- 172.21.15.116:0/2324116603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162409f660 con 0x7f162413b380 2020-11-20T12:18:23.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1629684700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f162413b380 0x7f16240a2e10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f161400eba0 tx=0x7f1614006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:23.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1611ffb700 1 -- 172.21.15.116:0/2324116603 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400b8a0 con 0x7f162413b380 2020-11-20T12:18:23.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1611ffb700 1 -- 172.21.15.116:0/2324116603 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161400ba00 con 0x7f162413b380 2020-11-20T12:18:23.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f162ae87700 1 -- 172.21.15.116:0/2324116603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162409f900 con 0x7f162413b380 2020-11-20T12:18:23.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f1611ffb700 1 -- 172.21.15.116:0/2324116603 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161400bd80 con 0x7f162413b380 2020-11-20T12:18:23.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.256+0000 7f162ae87700 1 -- 172.21.15.116:0/2324116603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1624153dd0 con 0x7f162413b380 2020-11-20T12:18:23.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.260+0000 7f162ae87700 1 -- 172.21.15.116:0/2324116603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16241540b0 con 0x7f162413b380 2020-11-20T12:18:23.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.260+0000 7f1611ffb700 1 -- 172.21.15.116:0/2324116603 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1614012600 con 0x7f162413b380 2020-11-20T12:18:23.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.260+0000 7f1611ffb700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15fc0557c0 0x7f15fc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:23.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.260+0000 7f1611ffb700 1 -- 172.21.15.116:0/2324116603 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f161406e590 con 0x7f162413b380 2020-11-20T12:18:23.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.264+0000 7f1629e85700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15fc0557c0 0x7f15fc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:23.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.264+0000 7f1629e85700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15fc0557c0 0x7f15fc057c80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f1620006fa0 tx=0x7f1620015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:23.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.268+0000 7f1611ffb700 1 -- 172.21.15.116:0/2324116603 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1614044700 con 0x7f162413b380 2020-11-20T12:18:23.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.424+0000 7f162ae87700 1 -- 172.21.15.116:0/2324116603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f162405cf30 con 0x7f162413b380 2020-11-20T12:18:23.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f1611ffb700 1 -- 172.21.15.116:0/2324116603 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13723 (secure 0 0 0) 0x7f1614047d20 con 0x7f162413b380 2020-11-20T12:18:23.427 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:23.428 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":50,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:18:21.695424+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:23.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 -- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15fc0557c0 msgr2=0x7f15fc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:23.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15fc0557c0 0x7f15fc057c80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f1620006fa0 tx=0x7f1620015000).stop 2020-11-20T12:18:23.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 -- 172.21.15.116:0/2324116603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f162413b380 msgr2=0x7f16240a2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:23.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f162413b380 0x7f16240a2e10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f161400eba0 tx=0x7f1614006df0).stop 2020-11-20T12:18:23.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 -- 172.21.15.116:0/2324116603 shutdown_connections 2020-11-20T12:18:23.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15fc0557c0 0x7f15fc057c80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16240a3300 0x7f1624151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16240acd30 0x7f16240a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 --2- 172.21.15.116:0/2324116603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f162413b380 0x7f16240a2e10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 -- 172.21.15.116:0/2324116603 >> 172.21.15.116:0/2324116603 conn(0x7f16240678a0 msgr2=0x7f16240c7ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 -- 172.21.15.116:0/2324116603 shutdown_connections 2020-11-20T12:18:23.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:23.428+0000 7f162ae87700 1 -- 172.21.15.116:0/2324116603 wait complete. 2020-11-20T12:18:23.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T12:18:23.922 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:18:24.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.112+0000 7f74c32b0700 1 Processor -- start 2020-11-20T12:18:24.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.112+0000 7f74c32b0700 1 -- start start 2020-11-20T12:18:24.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74c32b0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f74bc0606f0 0x7f74bc0aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74c32b0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc0a67e0 con 0x7f74bc066d90 2020-11-20T12:18:24.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74c32b0700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc0a5fd0 con 0x7f74bc060310 2020-11-20T12:18:24.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74c32b0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74bc0a6270 con 0x7f74bc0606f0 2020-11-20T12:18:24.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74c184d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f74bc066d90 0x7f74bc159af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56078/0 (socket says 172.21.15.116:56078) 2020-11-20T12:18:24.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74c184d700 1 -- 172.21.15.116:0/3367940631 learned_addr learned my addr 172.21.15.116:0/3367940631 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:24.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74c084b700 1 --2- 172.21.15.116:0/3367940631 >> v2:172.21.15.173:3300/0 conn(0x7f74bc0606f0 0x7f74bc0aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 646089128 0 0) 0x7f74bc0a67e0 con 0x7f74bc066d90 2020-11-20T12:18:24.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a4003240 con 0x7f74bc066d90 2020-11-20T12:18:24.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600262774 0 0) 0x7f74bc0a5fd0 con 0x7f74bc060310 2020-11-20T12:18:24.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74bc0a67e0 con 0x7f74bc060310 2020-11-20T12:18:24.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3243070770 0 0) 0x7f74a4003240 con 0x7f74bc066d90 2020-11-20T12:18:24.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 >> v2:172.21.15.173:3300/0 conn(0x7f74bc0606f0 msgr2=0x7f74bc0aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:24.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 --2- 172.21.15.116:0/3367940631 >> v2:172.21.15.173:3300/0 conn(0x7f74bc0606f0 0x7f74bc0aa0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 >> v1:172.21.15.173:6789/0 conn(0x7f74bc060310 legacy=0x7f74bc066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc0a6a40 con 0x7f74bc066d90 2020-11-20T12:18:24.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f74b4001860 con 0x7f74bc066d90 2020-11-20T12:18:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 >> v1:172.21.15.116:6789/0 conn(0x7f74bc066d90 legacy=0x7f74bc159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74a40038c0 0x7f74a4003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 0x7f74a4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74a4004e50 0x7f74a400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc0a6270 con 0x7f74a4004e50 2020-11-20T12:18:24.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a400bc20 con 0x7f74a40038c0 2020-11-20T12:18:24.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a400bde0 con 0x7f74a40044e0 2020-11-20T12:18:24.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74c184d700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74a4004e50 0x7f74a400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74c084b700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 0x7f74a4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.116+0000 7f74c104c700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74a40038c0 0x7f74a4003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c084b700 1 -- 172.21.15.116:0/3367940631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74a40038c0 msgr2=0x7f74a4003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:24.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c084b700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74a40038c0 0x7f74a4003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c084b700 1 -- 172.21.15.116:0/3367940631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74a4004e50 msgr2=0x7f74a400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:24.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c084b700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74a4004e50 0x7f74a400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c084b700 1 -- 172.21.15.116:0/3367940631 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a4003240 con 0x7f74a40044e0 2020-11-20T12:18:24.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c184d700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74a4004e50 0x7f74a400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:24.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c184d700 1 -- 172.21.15.116:0/3367940631 reap_dead start 2020-11-20T12:18:24.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c084b700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 0x7f74a4004960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f74ac00bc00 tx=0x7f74ac006df0).ready entity=mon.2 client_cookie=365f4564f80d7944 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:24.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00b310 con 0x7f74a40044e0 2020-11-20T12:18:24.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac00ea30 con 0x7f74a40044e0 2020-11-20T12:18:24.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74abfff700 1 -- 172.21.15.116:0/3367940631 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00edb0 con 0x7f74a40044e0 2020-11-20T12:18:24.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c32b0700 1 -- 172.21.15.116:0/3367940631 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 msgr2=0x7f74a4004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c32b0700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 0x7f74a4004960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f74ac00bc00 tx=0x7f74ac006df0).stop 2020-11-20T12:18:24.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c32b0700 1 -- 172.21.15.116:0/3367940631 shutdown_connections 2020-11-20T12:18:24.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c32b0700 1 --2- 172.21.15.116:0/3367940631 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 0x7f74a4004960 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c32b0700 1 -- 172.21.15.116:0/3367940631 >> 172.21.15.116:0/3367940631 conn(0x7f74bc05ba80 msgr2=0x7f74bc0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:24.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c32b0700 1 -- 172.21.15.116:0/3367940631 shutdown_connections 2020-11-20T12:18:24.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.120+0000 7f74c32b0700 1 -- 172.21.15.116:0/3367940631 wait complete. 2020-11-20T12:18:24.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c32b0700 1 Processor -- start 2020-11-20T12:18:24.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c32b0700 1 -- start start 2020-11-20T12:18:24.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c32b0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 0x7f74bc0bbe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c32b0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74bc143e90 0x7f74bc0a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c32b0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74bc144310 0x7f74bc0a7d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c32b0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc1653f0 con 0x7f74bc144310 2020-11-20T12:18:24.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c32b0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc165620 con 0x7f74bc143e90 2020-11-20T12:18:24.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c32b0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc1658c0 con 0x7f74a40044e0 2020-11-20T12:18:24.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c184d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74bc144310 0x7f74bc0a7d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c184d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74bc144310 0x7f74bc0a7d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46656/0 (socket says 172.21.15.116:46656) 2020-11-20T12:18:24.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c184d700 1 -- 172.21.15.116:0/615079444 learned_addr learned my addr 172.21.15.116:0/615079444 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:24.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.124+0000 7f74c104c700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 0x7f74bc0bbe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74c084b700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74bc143e90 0x7f74bc0a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74c184d700 1 -- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 msgr2=0x7f74bc0bbe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74c184d700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 0x7f74bc0bbe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74c184d700 1 -- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74bc143e90 msgr2=0x7f74bc0a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74c184d700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74bc143e90 0x7f74bc0a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74c184d700 1 -- 172.21.15.116:0/615079444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc165b60 con 0x7f74bc144310 2020-11-20T12:18:24.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74c184d700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74bc144310 0x7f74bc0a7d80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f74bc067170 tx=0x7f74b4007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:24.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74a97fa700 1 -- 172.21.15.116:0/615079444 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b400fce0 con 0x7f74bc144310 2020-11-20T12:18:24.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74a97fa700 1 -- 172.21.15.116:0/615079444 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b400fe40 con 0x7f74bc144310 2020-11-20T12:18:24.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74c32b0700 1 -- 172.21.15.116:0/615079444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74bc165e00 con 0x7f74bc144310 2020-11-20T12:18:24.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74a97fa700 1 -- 172.21.15.116:0/615079444 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4012580 con 0x7f74bc144310 2020-11-20T12:18:24.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74c32b0700 1 -- 172.21.15.116:0/615079444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74bc0bc370 con 0x7f74bc144310 2020-11-20T12:18:24.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.128+0000 7f74c32b0700 1 -- 172.21.15.116:0/615079444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74bc05d2a0 con 0x7f74bc144310 2020-11-20T12:18:24.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.132+0000 7f74a97fa700 1 -- 172.21.15.116:0/615079444 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f74b4012930 con 0x7f74bc144310 2020-11-20T12:18:24.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.132+0000 7f74a97fa700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74940557c0 0x7f7494057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.132+0000 7f74a97fa700 1 -- 172.21.15.116:0/615079444 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f74b406de20 con 0x7f74bc144310 2020-11-20T12:18:24.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.132+0000 7f74c104c700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74940557c0 0x7f7494057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.132+0000 7f74c104c700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74940557c0 0x7f7494057c80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f74b8007fd0 tx=0x7f74b8007b90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:24.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.136+0000 7f74a97fa700 1 -- 172.21.15.116:0/615079444 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f74b4043ff0 con 0x7f74bc144310 2020-11-20T12:18:24.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.296+0000 7f74c32b0700 1 -- 172.21.15.116:0/615079444 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f74bc156140 con 0x7f74bc144310 2020-11-20T12:18:24.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74a97fa700 1 -- 172.21.15.116:0/615079444 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v50) v1 ==== 110+0+10 (secure 0 0 0) 0x7f74b4019030 con 0x7f74bc144310 2020-11-20T12:18:24.297 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:18:24.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 -- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74940557c0 msgr2=0x7f7494057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74940557c0 0x7f7494057c80 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f74b8007fd0 tx=0x7f74b8007b90).stop 2020-11-20T12:18:24.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 -- 172.21.15.116:0/615079444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74bc144310 msgr2=0x7f74bc0a7d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74bc144310 0x7f74bc0a7d80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f74bc067170 tx=0x7f74b4007ad0).stop 2020-11-20T12:18:24.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 -- 172.21.15.116:0/615079444 shutdown_connections 2020-11-20T12:18:24.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74940557c0 0x7f7494057c80 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74bc144310 0x7f74bc0a7d80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f74a40044e0 0x7f74bc0bbe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 --2- 172.21.15.116:0/615079444 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74bc143e90 0x7f74bc0a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 -- 172.21.15.116:0/615079444 >> 172.21.15.116:0/615079444 conn(0x7f74bc05ba80 msgr2=0x7f74bc0c2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:24.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 -- 172.21.15.116:0/615079444 shutdown_connections 2020-11-20T12:18:24.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.300+0000 7f74c32b0700 1 -- 172.21.15.116:0/615079444 wait complete. 2020-11-20T12:18:24.324 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:18:24.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.500+0000 7fd2d3557700 1 Processor -- start 2020-11-20T12:18:24.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.500+0000 7fd2d3557700 1 -- start start 2020-11-20T12:18:24.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d3557700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd2cc060310 0x7fd2cc0606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d3557700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0a8550 0x7fd2cc0c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d3557700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2cc0674b0 con 0x7fd2cc0c2cc0 2020-11-20T12:18:24.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d3557700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2cc0a56c0 con 0x7fd2cc060310 2020-11-20T12:18:24.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d3557700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0a5960 con 0x7fd2cc0a8550 2020-11-20T12:18:24.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d1d54700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0a8550 0x7fd2cc0c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d1d54700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0a8550 0x7fd2cc0c27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:55610/0 (socket says 172.21.15.116:55610) 2020-11-20T12:18:24.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d1d54700 1 -- 172.21.15.116:0/3894240318 learned_addr learned my addr 172.21.15.116:0/3894240318 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:24.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d2555700 1 --2- 172.21.15.116:0/3894240318 >> v2:172.21.15.173:3300/0 conn(0x7fd2cc060310 0x7fd2cc0606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d1d54700 1 -- 172.21.15.116:0/3894240318 >> v2:172.21.15.173:3300/0 conn(0x7fd2cc060310 msgr2=0x7fd2cc0606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:24.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d1d54700 1 --2- 172.21.15.116:0/3894240318 >> v2:172.21.15.173:3300/0 conn(0x7fd2cc060310 0x7fd2cc0606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d1d54700 1 -- 172.21.15.116:0/3894240318 >> v1:172.21.15.173:6789/0 conn(0x7fd2cc0c2cc0 legacy=0x7fd2cc0650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d1d54700 1 -- 172.21.15.116:0/3894240318 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc0a5c00 con 0x7fd2cc0a8550 2020-11-20T12:18:24.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.504+0000 7fd2d1d54700 1 --2- 172.21.15.116:0/3894240318 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0a8550 0x7fd2cc0c27d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd2bc006fd0 tx=0x7fd2bc006eb0).ready entity=mon.2 client_cookie=8ed8d999c20ad0f0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:24.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.508+0000 7fd2d1553700 1 -- 172.21.15.116:0/3894240318 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc0098d0 con 0x7fd2cc0a8550 2020-11-20T12:18:24.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.508+0000 7fd2d1553700 1 -- 172.21.15.116:0/3894240318 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2bc009a30 con 0x7fd2cc0a8550 2020-11-20T12:18:24.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.508+0000 7fd2d1553700 1 -- 172.21.15.116:0/3894240318 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc009db0 con 0x7fd2cc0a8550 2020-11-20T12:18:24.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.508+0000 7fd2d3557700 1 -- 172.21.15.116:0/3894240318 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0a8550 msgr2=0x7fd2cc0c27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.508+0000 7fd2d3557700 1 --2- 172.21.15.116:0/3894240318 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0a8550 0x7fd2cc0c27d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fd2bc006fd0 tx=0x7fd2bc006eb0).stop 2020-11-20T12:18:24.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.508+0000 7fd2d3557700 1 -- 172.21.15.116:0/3894240318 shutdown_connections 2020-11-20T12:18:24.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.508+0000 7fd2d3557700 1 --2- 172.21.15.116:0/3894240318 >> v2:172.21.15.173:3300/0 conn(0x7fd2cc060310 0x7fd2cc0606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.508+0000 7fd2d3557700 1 --2- 172.21.15.116:0/3894240318 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0a8550 0x7fd2cc0c27d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.508+0000 7fd2d3557700 1 -- 172.21.15.116:0/3894240318 >> 172.21.15.116:0/3894240318 conn(0x7fd2cc061d50 msgr2=0x7fd2cc0c1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:24.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.508+0000 7fd2d3557700 1 -- 172.21.15.116:0/3894240318 shutdown_connections 2020-11-20T12:18:24.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.508+0000 7fd2d3557700 1 -- 172.21.15.116:0/3894240318 wait complete. 2020-11-20T12:18:24.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.512+0000 7fd2d3557700 1 Processor -- start 2020-11-20T12:18:24.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.512+0000 7fd2d3557700 1 -- start start 2020-11-20T12:18:24.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d3557700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc060310 0x7fd2cc09f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d3557700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0c1da0 0x7fd2cc0a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d3557700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2cc0a1d70 0x7fd2cc168fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d3557700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc0a21d0 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d3557700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc09d550 con 0x7fd2cc060310 2020-11-20T12:18:24.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d3557700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2cc169ce0 con 0x7fd2cc0c1da0 2020-11-20T12:18:24.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d2d56700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2cc0a1d70 0x7fd2cc168fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d2d56700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2cc0a1d70 0x7fd2cc168fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46670/0 (socket says 172.21.15.116:46670) 2020-11-20T12:18:24.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d2d56700 1 -- 172.21.15.116:0/1928529217 learned_addr learned my addr 172.21.15.116:0/1928529217 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:24.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d2555700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc060310 0x7fd2cc09f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d1d54700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0c1da0 0x7fd2cc0a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d2d56700 1 -- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0c1da0 msgr2=0x7fd2cc0a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d2d56700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0c1da0 0x7fd2cc0a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d2d56700 1 -- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc060310 msgr2=0x7fd2cc09f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d2d56700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc060310 0x7fd2cc09f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d2d56700 1 -- 172.21.15.116:0/1928529217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc09f620 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d2d56700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2cc0a1d70 0x7fd2cc168fa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fd2c400cac0 tx=0x7fd2c4007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:24.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2ba7fc700 1 -- 172.21.15.116:0/1928529217 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400cea0 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2ba7fc700 1 -- 172.21.15.116:0/1928529217 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2c4001460 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2ba7fc700 1 -- 172.21.15.116:0/1928529217 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2c400a940 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d3557700 1 -- 172.21.15.116:0/1928529217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2cc09d7b0 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.516+0000 7fd2d3557700 1 -- 172.21.15.116:0/1928529217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2cc09dcf0 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.520+0000 7fd2d3557700 1 -- 172.21.15.116:0/1928529217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2cc09deb0 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.520+0000 7fd2ba7fc700 1 -- 172.21.15.116:0/1928529217 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd2c4010070 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.520+0000 7fd2ba7fc700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2a40557c0 0x7fd2a4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.520+0000 7fd2ba7fc700 1 -- 172.21.15.116:0/1928529217 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd2c406ded0 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.524+0000 7fd2d2555700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2a40557c0 0x7fd2a4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.524+0000 7fd2d2555700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2a40557c0 0x7fd2a4057c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fd2c800da50 tx=0x7fd2c8006f90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:24.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.528+0000 7fd2ba7fc700 1 -- 172.21.15.116:0/1928529217 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd2c40440a0 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.688+0000 7fd2d3557700 1 -- 172.21.15.116:0/1928529217 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd2cc05cf30 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.688+0000 7fd2ba7fc700 1 -- 172.21.15.116:0/1928529217 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v52) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd2c400acb0 con 0x7fd2cc0a1d70 2020-11-20T12:18:24.688 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:24.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.688+0000 7fd2d3557700 1 -- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2a40557c0 msgr2=0x7fd2a4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.688+0000 7fd2d3557700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2a40557c0 0x7fd2a4057c80 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fd2c800da50 tx=0x7fd2c8006f90).stop 2020-11-20T12:18:24.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.688+0000 7fd2d3557700 1 -- 172.21.15.116:0/1928529217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2cc0a1d70 msgr2=0x7fd2cc168fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.688+0000 7fd2d3557700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2cc0a1d70 0x7fd2cc168fa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7fd2c400cac0 tx=0x7fd2c4007f10).stop 2020-11-20T12:18:24.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.692+0000 7fd2d3557700 1 -- 172.21.15.116:0/1928529217 shutdown_connections 2020-11-20T12:18:24.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.692+0000 7fd2d3557700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2a40557c0 0x7fd2a4057c80 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.692+0000 7fd2d3557700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd2cc0a1d70 0x7fd2cc168fa0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.692+0000 7fd2d3557700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd2cc060310 0x7fd2cc09f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.692+0000 7fd2d3557700 1 --2- 172.21.15.116:0/1928529217 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd2cc0c1da0 0x7fd2cc0a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.692+0000 7fd2d3557700 1 -- 172.21.15.116:0/1928529217 >> 172.21.15.116:0/1928529217 conn(0x7fd2cc061d50 msgr2=0x7fd2cc1543f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:24.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.692+0000 7fd2d3557700 1 -- 172.21.15.116:0/1928529217 shutdown_connections 2020-11-20T12:18:24.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.692+0000 7fd2d3557700 1 -- 172.21.15.116:0/1928529217 wait complete. 2020-11-20T12:18:24.712 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-20T12:18:24.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.904+0000 7f6624ef3700 1 Processor -- start 2020-11-20T12:18:24.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.904+0000 7f6624ef3700 1 -- start start 2020-11-20T12:18:24.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.904+0000 7f6624ef3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6620062880 0x7f6620062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.904+0000 7f6624ef3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f66200b94e0 0x7f66200b9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.904+0000 7f6624ef3700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66200ba1b0 con 0x7f662014b990 2020-11-20T12:18:24.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.904+0000 7f6624ef3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6620152e60 con 0x7f66200b94e0 2020-11-20T12:18:24.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.904+0000 7f6624ef3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66201555a0 con 0x7f6620062880 2020-11-20T12:18:24.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.904+0000 7f6616ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f662014b990 0x7f662014be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56112/0 (socket says 172.21.15.116:56112) 2020-11-20T12:18:24.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.904+0000 7f6616ffd700 1 -- 172.21.15.116:0/3636208448 learned_addr learned my addr 172.21.15.116:0/3636208448 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:24.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.904+0000 7f661f7fe700 1 --2- 172.21.15.116:0/3636208448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6620062880 0x7f6620062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.904+0000 7f661ffff700 1 --2- 172.21.15.116:0/3636208448 >> v2:172.21.15.173:3300/0 conn(0x7f66200b94e0 0x7f66200b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f661effd700 1 -- 172.21.15.116:0/3636208448 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440707088 0 0) 0x7f66200ba1b0 con 0x7f662014b990 2020-11-20T12:18:24.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f661effd700 1 -- 172.21.15.116:0/3636208448 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6600003240 con 0x7f662014b990 2020-11-20T12:18:24.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f661f7fe700 1 -- 172.21.15.116:0/3636208448 >> v2:172.21.15.173:3300/0 conn(0x7f66200b94e0 msgr2=0x7f66200b9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:24.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f661f7fe700 1 --2- 172.21.15.116:0/3636208448 >> v2:172.21.15.173:3300/0 conn(0x7f66200b94e0 0x7f66200b9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f661f7fe700 1 -- 172.21.15.116:0/3636208448 >> v1:172.21.15.116:6789/0 conn(0x7f662014b990 legacy=0x7f662014be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f661f7fe700 1 -- 172.21.15.116:0/3636208448 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6620155840 con 0x7f6620062880 2020-11-20T12:18:24.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f661f7fe700 1 --2- 172.21.15.116:0/3636208448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6620062880 0x7f6620062c60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f661000b9a0 tx=0x7f6610006df0).ready entity=mon.2 client_cookie=ef2c8efaf00f8d06 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:24.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f661effd700 1 -- 172.21.15.116:0/3636208448 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610009680 con 0x7f6620062880 2020-11-20T12:18:24.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f661effd700 1 -- 172.21.15.116:0/3636208448 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66100097e0 con 0x7f6620062880 2020-11-20T12:18:24.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f661effd700 1 -- 172.21.15.116:0/3636208448 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6610009b60 con 0x7f6620062880 2020-11-20T12:18:24.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f6624ef3700 1 -- 172.21.15.116:0/3636208448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6620062880 msgr2=0x7f6620062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f6624ef3700 1 --2- 172.21.15.116:0/3636208448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6620062880 0x7f6620062c60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f661000b9a0 tx=0x7f6610006df0).stop 2020-11-20T12:18:24.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f6624ef3700 1 -- 172.21.15.116:0/3636208448 shutdown_connections 2020-11-20T12:18:24.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f6624ef3700 1 --2- 172.21.15.116:0/3636208448 >> v2:172.21.15.173:3300/0 conn(0x7f66200b94e0 0x7f66200b9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f6624ef3700 1 --2- 172.21.15.116:0/3636208448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6620062880 0x7f6620062c60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f6624ef3700 1 -- 172.21.15.116:0/3636208448 >> 172.21.15.116:0/3636208448 conn(0x7f6620060c40 msgr2=0x7f66200c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:24.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f6624ef3700 1 -- 172.21.15.116:0/3636208448 shutdown_connections 2020-11-20T12:18:24.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.908+0000 7f6624ef3700 1 -- 172.21.15.116:0/3636208448 wait complete. 2020-11-20T12:18:24.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.912+0000 7f6624ef3700 1 Processor -- start 2020-11-20T12:18:24.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.912+0000 7f6624ef3700 1 -- start start 2020-11-20T12:18:24.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6624ef3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6620062880 0x7f6620161440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6624ef3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66200b94e0 0x7f66200bce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6624ef3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f662014b990 0x7f66200bd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6624ef3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6620161ce0 con 0x7f66200b94e0 2020-11-20T12:18:24.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6624ef3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66200a74a0 con 0x7f662014b990 2020-11-20T12:18:24.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6624ef3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66200a7700 con 0x7f6620062880 2020-11-20T12:18:24.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6616ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66200b94e0 0x7f66200bce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6616ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66200b94e0 0x7f66200bce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46682/0 (socket says 172.21.15.116:46682) 2020-11-20T12:18:24.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6616ffd700 1 -- 172.21.15.116:0/1364321643 learned_addr learned my addr 172.21.15.116:0/1364321643 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f661ffff700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f662014b990 0x7f66200bd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f661f7fe700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6620062880 0x7f6620161440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6616ffd700 1 -- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6620062880 msgr2=0x7f6620161440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6616ffd700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6620062880 0x7f6620161440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6616ffd700 1 -- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f662014b990 msgr2=0x7f66200bd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:24.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6616ffd700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f662014b990 0x7f66200bd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:24.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6616ffd700 1 -- 172.21.15.116:0/1364321643 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66200a79a0 con 0x7f66200b94e0 2020-11-20T12:18:24.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6616ffd700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66200b94e0 0x7f66200bce20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f660800c710 tx=0x7f6608007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:24.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6617fff700 1 -- 172.21.15.116:0/1364321643 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66080072e0 con 0x7f66200b94e0 2020-11-20T12:18:24.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6617fff700 1 -- 172.21.15.116:0/1364321643 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6608007440 con 0x7f66200b94e0 2020-11-20T12:18:24.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6624ef3700 1 -- 172.21.15.116:0/1364321643 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66200a7c40 con 0x7f66200b94e0 2020-11-20T12:18:24.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6617fff700 1 -- 172.21.15.116:0/1364321643 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66080057a0 con 0x7f66200b94e0 2020-11-20T12:18:24.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.916+0000 7f6624ef3700 1 -- 172.21.15.116:0/1364321643 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66200bb3f0 con 0x7f66200b94e0 2020-11-20T12:18:24.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.920+0000 7f6624ef3700 1 -- 172.21.15.116:0/1364321643 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66200aa390 con 0x7f66200b94e0 2020-11-20T12:18:24.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.920+0000 7f6617fff700 1 -- 172.21.15.116:0/1364321643 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6608010070 con 0x7f66200b94e0 2020-11-20T12:18:24.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.920+0000 7f6617fff700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65fc0557c0 0x7f65fc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:24.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.920+0000 7f6617fff700 1 -- 172.21.15.116:0/1364321643 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f660806da70 con 0x7f66200b94e0 2020-11-20T12:18:24.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.920+0000 7f661f7fe700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65fc0557c0 0x7f65fc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:24.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.924+0000 7f661f7fe700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65fc0557c0 0x7f65fc057c80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f6610006e30 tx=0x7f6610015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:24.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:24.924+0000 7f6617fff700 1 -- 172.21.15.116:0/1364321643 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6608043b30 con 0x7f66200b94e0 2020-11-20T12:18:25.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.084+0000 7f6624ef3700 1 -- 172.21.15.116:0/1364321643 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f66200a7dd0 con 0x7f66200b94e0 2020-11-20T12:18:25.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6617fff700 1 -- 172.21.15.116:0/1364321643 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v52) v1 ==== 84+0+143 (secure 0 0 0) 0x7f6608047150 con 0x7f66200b94e0 2020-11-20T12:18:25.086 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:25.086 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2020-11-20T12:18:25.086 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:25.087 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2020-11-20T12:18:25.087 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "device_health_metrics" 2020-11-20T12:18:25.087 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:25.087 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:25.088 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 2, 2020-11-20T12:18:25.088 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "rbd" 2020-11-20T12:18:25.088 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:25.088 INFO:teuthology.orchestra.run.smithi116.stdout:] 2020-11-20T12:18:25.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 -- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65fc0557c0 msgr2=0x7f65fc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65fc0557c0 0x7f65fc057c80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f6610006e30 tx=0x7f6610015000).stop 2020-11-20T12:18:25.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 -- 172.21.15.116:0/1364321643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66200b94e0 msgr2=0x7f66200bce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66200b94e0 0x7f66200bce20 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f660800c710 tx=0x7f6608007aa0).stop 2020-11-20T12:18:25.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 -- 172.21.15.116:0/1364321643 shutdown_connections 2020-11-20T12:18:25.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65fc0557c0 0x7f65fc057c80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f662014b990 0x7f66200bd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6620062880 0x7f6620161440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 --2- 172.21.15.116:0/1364321643 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66200b94e0 0x7f66200bce20 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 -- 172.21.15.116:0/1364321643 >> 172.21.15.116:0/1364321643 conn(0x7f6620060c40 msgr2=0x7f66200669a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:25.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 -- 172.21.15.116:0/1364321643 shutdown_connections 2020-11-20T12:18:25.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.088+0000 7f6624ef3700 1 -- 172.21.15.116:0/1364321643 wait complete. 2020-11-20T12:18:25.113 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T12:18:25.114 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-20T12:18:25.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.292+0000 7f0c7579b700 1 Processor -- start 2020-11-20T12:18:25.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.292+0000 7f0c7579b700 1 -- start start 2020-11-20T12:18:25.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c7579b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 0x7f0c700a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c7579b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0c700aabb0 0x7f0c70151e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c7579b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c700ace40 con 0x7f0c700ad540 2020-11-20T12:18:25.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c7579b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c701526c0 con 0x7f0c700aabb0 2020-11-20T12:18:25.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c7579b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c70152960 con 0x7f0c700aa7d0 2020-11-20T12:18:25.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c6ffff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 0x7f0c700a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c6ffff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 0x7f0c700a8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:55638/0 (socket says 172.21.15.116:55638) 2020-11-20T12:18:25.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c6ffff700 1 -- 172.21.15.116:0/4065709354 learned_addr learned my addr 172.21.15.116:0/4065709354 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:25.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c74f9a700 1 --1- 172.21.15.116:0/4065709354 >> v1:172.21.15.173:6789/0 conn(0x7f0c700ad540 0x7f0c700ad9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:52354/0 (socket says 172.21.15.116:52354) 2020-11-20T12:18:25.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c6f7fe700 1 --2- 172.21.15.116:0/4065709354 >> v2:172.21.15.173:3300/0 conn(0x7f0c700aabb0 0x7f0c70151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c6effd700 1 -- 172.21.15.116:0/4065709354 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1886426443 0 0) 0x7f0c700ace40 con 0x7f0c700ad540 2020-11-20T12:18:25.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c6ffff700 1 -- 172.21.15.116:0/4065709354 >> v2:172.21.15.173:3300/0 conn(0x7f0c700aabb0 msgr2=0x7f0c70151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:25.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c6ffff700 1 --2- 172.21.15.116:0/4065709354 >> v2:172.21.15.173:3300/0 conn(0x7f0c700aabb0 0x7f0c70151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c6ffff700 1 -- 172.21.15.116:0/4065709354 >> v1:172.21.15.173:6789/0 conn(0x7f0c700ad540 legacy=0x7f0c700ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c6ffff700 1 -- 172.21.15.116:0/4065709354 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70152c00 con 0x7f0c700aa7d0 2020-11-20T12:18:25.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c6ffff700 1 --2- 172.21.15.116:0/4065709354 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 0x7f0c700a8400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f0c60006fd0 tx=0x7f0c60006eb0).ready entity=mon.2 client_cookie=8a7410c7ea258b60 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:25.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.296+0000 7f0c6effd700 1 -- 172.21.15.116:0/4065709354 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60009810 con 0x7f0c700aa7d0 2020-11-20T12:18:25.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.300+0000 7f0c6effd700 1 -- 172.21.15.116:0/4065709354 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c60009970 con 0x7f0c700aa7d0 2020-11-20T12:18:25.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.300+0000 7f0c6effd700 1 -- 172.21.15.116:0/4065709354 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60009cf0 con 0x7f0c700aa7d0 2020-11-20T12:18:25.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.300+0000 7f0c7579b700 1 -- 172.21.15.116:0/4065709354 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 msgr2=0x7f0c700a8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.300+0000 7f0c7579b700 1 --2- 172.21.15.116:0/4065709354 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 0x7f0c700a8400 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f0c60006fd0 tx=0x7f0c60006eb0).stop 2020-11-20T12:18:25.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.300+0000 7f0c7579b700 1 -- 172.21.15.116:0/4065709354 shutdown_connections 2020-11-20T12:18:25.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.300+0000 7f0c7579b700 1 --2- 172.21.15.116:0/4065709354 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 0x7f0c700a8400 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.300+0000 7f0c7579b700 1 --2- 172.21.15.116:0/4065709354 >> v2:172.21.15.173:3300/0 conn(0x7f0c700aabb0 0x7f0c70151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.300+0000 7f0c7579b700 1 -- 172.21.15.116:0/4065709354 >> 172.21.15.116:0/4065709354 conn(0x7f0c700678a0 msgr2=0x7f0c70067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:25.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.300+0000 7f0c7579b700 1 -- 172.21.15.116:0/4065709354 shutdown_connections 2020-11-20T12:18:25.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.300+0000 7f0c7579b700 1 -- 172.21.15.116:0/4065709354 wait complete. 2020-11-20T12:18:25.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.304+0000 7f0c7579b700 1 Processor -- start 2020-11-20T12:18:25.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.304+0000 7f0c7579b700 1 -- start start 2020-11-20T12:18:25.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.304+0000 7f0c7579b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 0x7f0c70150040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.304+0000 7f0c7579b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c700aabb0 0x7f0c700c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.304+0000 7f0c7579b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c700ad540 0x7f0c70149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.304+0000 7f0c7579b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c70150530 con 0x7f0c700aabb0 2020-11-20T12:18:25.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.304+0000 7f0c7579b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c70160fe0 con 0x7f0c700ad540 2020-11-20T12:18:25.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.304+0000 7f0c7579b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c70161280 con 0x7f0c700aa7d0 2020-11-20T12:18:25.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c6f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c700aabb0 0x7f0c700c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c6f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c700aabb0 0x7f0c700c6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46696/0 (socket says 172.21.15.116:46696) 2020-11-20T12:18:25.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c6f7fe700 1 -- 172.21.15.116:0/2391683557 learned_addr learned my addr 172.21.15.116:0/2391683557 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:25.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c74f9a700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c700ad540 0x7f0c70149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c6ffff700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 0x7f0c70150040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c6f7fe700 1 -- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 msgr2=0x7f0c70150040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c6f7fe700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 0x7f0c70150040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c6f7fe700 1 -- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c700ad540 msgr2=0x7f0c70149f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c6f7fe700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c700ad540 0x7f0c70149f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c6f7fe700 1 -- 172.21.15.116:0/2391683557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c70161520 con 0x7f0c700aabb0 2020-11-20T12:18:25.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c6f7fe700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c700aabb0 0x7f0c700c6e70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f0c6400eba0 tx=0x7f0c64006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:25.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c4ffff700 1 -- 172.21.15.116:0/2391683557 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400c8e0 con 0x7f0c700aabb0 2020-11-20T12:18:25.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c4ffff700 1 -- 172.21.15.116:0/2391683557 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6400ca40 con 0x7f0c700aabb0 2020-11-20T12:18:25.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c7579b700 1 -- 172.21.15.116:0/2391683557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c701617c0 con 0x7f0c700aabb0 2020-11-20T12:18:25.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c4ffff700 1 -- 172.21.15.116:0/2391683557 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6400cdc0 con 0x7f0c700aabb0 2020-11-20T12:18:25.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.308+0000 7f0c7579b700 1 -- 172.21.15.116:0/2391683557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c70161d00 con 0x7f0c700aabb0 2020-11-20T12:18:25.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.312+0000 7f0c7579b700 1 -- 172.21.15.116:0/2391683557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c7005cf30 con 0x7f0c700aabb0 2020-11-20T12:18:25.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.312+0000 7f0c4ffff700 1 -- 172.21.15.116:0/2391683557 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0c6401a030 con 0x7f0c700aabb0 2020-11-20T12:18:25.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.312+0000 7f0c4ffff700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c480557f0 0x7f0c48057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.312+0000 7f0c4ffff700 1 -- 172.21.15.116:0/2391683557 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f0c6406deb0 con 0x7f0c700aabb0 2020-11-20T12:18:25.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.312+0000 7f0c6ffff700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c480557f0 0x7f0c48057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.312+0000 7f0c6ffff700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c480557f0 0x7f0c48057cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f0c6000b1b0 tx=0x7f0c60015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:25.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.320+0000 7f0c4ffff700 1 -- 172.21.15.116:0/2391683557 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0c64044130 con 0x7f0c700aabb0 2020-11-20T12:18:25.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.476+0000 7f0c7579b700 1 -- 172.21.15.116:0/2391683557 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f0c700601b0 con 0x7f0c700aabb0 2020-11-20T12:18:25.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.684+0000 7f0c4ffff700 1 -- 172.21.15.116:0/2391683557 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v53) v1 ==== 133+0+0 (secure 0 0 0) 0x7f0c64047750 con 0x7f0c700aabb0 2020-11-20T12:18:25.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 -- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c480557f0 msgr2=0x7f0c48057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c480557f0 0x7f0c48057cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f0c6000b1b0 tx=0x7f0c60015000).stop 2020-11-20T12:18:25.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 -- 172.21.15.116:0/2391683557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c700aabb0 msgr2=0x7f0c700c6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c700aabb0 0x7f0c700c6e70 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f0c6400eba0 tx=0x7f0c64006df0).stop 2020-11-20T12:18:25.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 -- 172.21.15.116:0/2391683557 shutdown_connections 2020-11-20T12:18:25.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c480557f0 0x7f0c48057cb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c700ad540 0x7f0c70149f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c700aa7d0 0x7f0c70150040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 --2- 172.21.15.116:0/2391683557 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c700aabb0 0x7f0c700c6e70 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 -- 172.21.15.116:0/2391683557 >> 172.21.15.116:0/2391683557 conn(0x7f0c700678a0 msgr2=0x7f0c701534d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:25.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 -- 172.21.15.116:0/2391683557 shutdown_connections 2020-11-20T12:18:25.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.688+0000 7f0c7579b700 1 -- 172.21.15.116:0/2391683557 wait complete. 2020-11-20T12:18:25.692 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2020-11-20T12:18:25.714 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-20T12:18:25.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f262d1f3700 1 Processor -- start 2020-11-20T12:18:25.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f262d1f3700 1 -- start start 2020-11-20T12:18:25.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f262d1f3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2628155c80 0x7f2628062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f262d1f3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2628156060 0x7f26280c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f262d1f3700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26280b93c0 con 0x7f26280c12c0 2020-11-20T12:18:25.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f262d1f3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26280ba580 con 0x7f2628155c80 2020-11-20T12:18:25.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f262d1f3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26280b9660 con 0x7f2628156060 2020-11-20T12:18:25.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f2627fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2628155c80 0x7f2628062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f261f7fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2628156060 0x7f26280c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f262c9f2700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f26280c12c0 0x7f2628060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56142/0 (socket says 172.21.15.116:56142) 2020-11-20T12:18:25.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f261f7fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2628156060 0x7f26280c0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:45756/0 (socket says 172.21.15.116:45756) 2020-11-20T12:18:25.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.888+0000 7f262c9f2700 1 -- 172.21.15.116:0/2554788958 learned_addr learned my addr 172.21.15.116:0/2554788958 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:25.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f2627fff700 1 -- 172.21.15.116:0/2554788958 >> v2:172.21.15.173:3300/0 conn(0x7f2628156060 msgr2=0x7f26280c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:25.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f2627fff700 1 --2- 172.21.15.116:0/2554788958 >> v2:172.21.15.173:3300/0 conn(0x7f2628156060 0x7f26280c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f26277fe700 1 -- 172.21.15.116:0/2554788958 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316335162 0 0) 0x7f26280b93c0 con 0x7f26280c12c0 2020-11-20T12:18:25.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f2627fff700 1 -- 172.21.15.116:0/2554788958 >> v1:172.21.15.116:6789/0 conn(0x7f26280c12c0 legacy=0x7f2628060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f2627fff700 1 -- 172.21.15.116:0/2554788958 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26280b9900 con 0x7f2628155c80 2020-11-20T12:18:25.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f261f7fe700 1 --2- 172.21.15.116:0/2554788958 >> v2:172.21.15.173:3300/0 conn(0x7f2628156060 0x7f26280c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:25.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f2627fff700 1 --2- 172.21.15.116:0/2554788958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2628155c80 0x7f2628062ed0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f2628065d50 tx=0x7f2618006df0).ready entity=mon.0 client_cookie=d21b424d2ed1f943 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:25.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f26277fe700 1 -- 172.21.15.116:0/2554788958 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26180097b0 con 0x7f2628155c80 2020-11-20T12:18:25.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f26277fe700 1 -- 172.21.15.116:0/2554788958 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2618009910 con 0x7f2628155c80 2020-11-20T12:18:25.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f26277fe700 1 -- 172.21.15.116:0/2554788958 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618009c90 con 0x7f2628155c80 2020-11-20T12:18:25.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f262d1f3700 1 -- 172.21.15.116:0/2554788958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2628155c80 msgr2=0x7f2628062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f262d1f3700 1 --2- 172.21.15.116:0/2554788958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2628155c80 0x7f2628062ed0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f2628065d50 tx=0x7f2618006df0).stop 2020-11-20T12:18:25.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f262d1f3700 1 -- 172.21.15.116:0/2554788958 shutdown_connections 2020-11-20T12:18:25.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f262d1f3700 1 --2- 172.21.15.116:0/2554788958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2628155c80 0x7f2628062ed0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f262d1f3700 1 --2- 172.21.15.116:0/2554788958 >> v2:172.21.15.173:3300/0 conn(0x7f2628156060 0x7f26280c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f262d1f3700 1 -- 172.21.15.116:0/2554788958 >> 172.21.15.116:0/2554788958 conn(0x7f2628068350 msgr2=0x7f26280627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:25.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f262d1f3700 1 -- 172.21.15.116:0/2554788958 shutdown_connections 2020-11-20T12:18:25.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.892+0000 7f262d1f3700 1 -- 172.21.15.116:0/2554788958 wait complete. 2020-11-20T12:18:25.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.896+0000 7f262d1f3700 1 Processor -- start 2020-11-20T12:18:25.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.896+0000 7f262d1f3700 1 -- start start 2020-11-20T12:18:25.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f262d1f3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26280c12c0 0x7f26280a8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f262d1f3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2628156060 0x7f2628158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f262d1f3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26280a92f0 0x7f2628164d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f262d1f3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280a9790 con 0x7f26280c12c0 2020-11-20T12:18:25.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f262d1f3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280a9a30 con 0x7f2628156060 2020-11-20T12:18:25.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f262d1f3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26280a9cd0 con 0x7f26280a92f0 2020-11-20T12:18:25.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2627fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26280c12c0 0x7f26280a8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2627fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26280c12c0 0x7f26280a8a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46710/0 (socket says 172.21.15.116:46710) 2020-11-20T12:18:25.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2627fff700 1 -- 172.21.15.116:0/941123261 learned_addr learned my addr 172.21.15.116:0/941123261 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:25.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f261f7fe700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2628156060 0x7f2628158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f262c9f2700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26280a92f0 0x7f2628164d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2627fff700 1 -- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26280a92f0 msgr2=0x7f2628164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2627fff700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26280a92f0 0x7f2628164d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2627fff700 1 -- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2628156060 msgr2=0x7f2628158bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:25.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2627fff700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2628156060 0x7f2628158bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:25.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2627fff700 1 -- 172.21.15.116:0/941123261 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26280a9f70 con 0x7f26280c12c0 2020-11-20T12:18:25.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f262c9f2700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26280a92f0 0x7f2628164d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:25.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2627fff700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26280c12c0 0x7f26280a8a20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f26280c54a0 tx=0x7f2618015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:25.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2624ff9700 1 -- 172.21.15.116:0/941123261 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2618011e00 con 0x7f26280c12c0 2020-11-20T12:18:25.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2624ff9700 1 -- 172.21.15.116:0/941123261 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f261801e460 con 0x7f26280c12c0 2020-11-20T12:18:25.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f262d1f3700 1 -- 172.21.15.116:0/941123261 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26280aa210 con 0x7f26280c12c0 2020-11-20T12:18:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f2624ff9700 1 -- 172.21.15.116:0/941123261 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f261801e800 con 0x7f26280c12c0 2020-11-20T12:18:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f262d1f3700 1 -- 172.21.15.116:0/941123261 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26280a90b0 con 0x7f26280c12c0 2020-11-20T12:18:25.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.900+0000 7f262d1f3700 1 -- 172.21.15.116:0/941123261 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2628066260 con 0x7f26280c12c0 2020-11-20T12:18:25.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.904+0000 7f2624ff9700 1 -- 172.21.15.116:0/941123261 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2618025030 con 0x7f26280c12c0 2020-11-20T12:18:25.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.904+0000 7f2624ff9700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26040557f0 0x7f2604057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:25.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.904+0000 7f2624ff9700 1 -- 172.21.15.116:0/941123261 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f261806f740 con 0x7f26280c12c0 2020-11-20T12:18:25.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.904+0000 7f261f7fe700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26040557f0 0x7f2604057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:25.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.908+0000 7f261f7fe700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26040557f0 0x7f2604057cb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f26280be640 tx=0x7f2610008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:25.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:25.912+0000 7f2624ff9700 1 -- 172.21.15.116:0/941123261 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2628066260 con 0x7f26280c12c0 2020-11-20T12:18:26.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.072+0000 7f262d1f3700 1 -- 172.21.15.116:0/941123261 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f26280a90b0 con 0x7f26280c12c0 2020-11-20T12:18:26.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.688+0000 7f2624ff9700 1 -- 172.21.15.116:0/941123261 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v54) v1 ==== 125+0+0 (secure 0 0 0) 0x7f26280a90b0 con 0x7f26280c12c0 2020-11-20T12:18:26.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 -- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26040557f0 msgr2=0x7f2604057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:26.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26040557f0 0x7f2604057cb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f26280be640 tx=0x7f2610008040).stop 2020-11-20T12:18:26.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 -- 172.21.15.116:0/941123261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26280c12c0 msgr2=0x7f26280a8a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:26.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26280c12c0 0x7f26280a8a20 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f26280c54a0 tx=0x7f2618015000).stop 2020-11-20T12:18:26.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 -- 172.21.15.116:0/941123261 shutdown_connections 2020-11-20T12:18:26.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26040557f0 0x7f2604057cb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:26.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26280a92f0 0x7f2628164d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:26.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26280c12c0 0x7f26280a8a20 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:26.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 --2- 172.21.15.116:0/941123261 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2628156060 0x7f2628158bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:26.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 -- 172.21.15.116:0/941123261 >> 172.21.15.116:0/941123261 conn(0x7f2628068350 msgr2=0x7f26280c1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:26.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 -- 172.21.15.116:0/941123261 shutdown_connections 2020-11-20T12:18:26.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.696+0000 7f262d1f3700 1 -- 172.21.15.116:0/941123261 wait complete. 2020-11-20T12:18:26.701 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2020-11-20T12:18:26.721 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T12:18:26.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.900+0000 7f23b864c700 1 Processor -- start 2020-11-20T12:18:26.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.900+0000 7f23b864c700 1 -- start start 2020-11-20T12:18:26.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b864c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23b00a7830 0x7f23b00a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:26.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b864c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f23b00c4c50 0x7f23b00be110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:26.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b864c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b00c3d30 con 0x7f23b00be600 2020-11-20T12:18:26.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b864c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b00bea70 con 0x7f23b00c4c50 2020-11-20T12:18:26.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b864c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00c3fd0 con 0x7f23b00a7830 2020-11-20T12:18:26.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b6be9700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f23b00be600 0x7f23b01515f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56154/0 (socket says 172.21.15.116:56154) 2020-11-20T12:18:26.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b6be9700 1 -- 172.21.15.116:0/625664735 learned_addr learned my addr 172.21.15.116:0/625664735 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:26.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b5be7700 1 --2- 172.21.15.116:0/625664735 >> v2:172.21.15.173:3300/0 conn(0x7f23b00c4c50 0x7f23b00be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:26.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b63e8700 1 --2- 172.21.15.116:0/625664735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23b00a7830 0x7f23b00a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:26.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b53e6700 1 -- 172.21.15.116:0/625664735 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 35518090 0 0) 0x7f23b00c3d30 con 0x7f23b00be600 2020-11-20T12:18:26.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b53e6700 1 -- 172.21.15.116:0/625664735 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2398003240 con 0x7f23b00be600 2020-11-20T12:18:26.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b5be7700 1 -- 172.21.15.116:0/625664735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23b00a7830 msgr2=0x7f23b00a7c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:26.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b5be7700 1 --2- 172.21.15.116:0/625664735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23b00a7830 0x7f23b00a7c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:26.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b5be7700 1 -- 172.21.15.116:0/625664735 >> v1:172.21.15.116:6789/0 conn(0x7f23b00be600 legacy=0x7f23b01515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:26.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b5be7700 1 -- 172.21.15.116:0/625664735 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b00c4270 con 0x7f23b00c4c50 2020-11-20T12:18:26.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.904+0000 7f23b5be7700 1 --2- 172.21.15.116:0/625664735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23b00c4c50 0x7f23b00be110 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f23a000b9a0 tx=0x7f23a0006df0).ready entity=mon.1 client_cookie=8402ab8e06919400 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:26.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.908+0000 7f23b53e6700 1 -- 172.21.15.116:0/625664735 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a0009680 con 0x7f23b00c4c50 2020-11-20T12:18:26.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.908+0000 7f23b53e6700 1 -- 172.21.15.116:0/625664735 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a00097e0 con 0x7f23b00c4c50 2020-11-20T12:18:26.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.908+0000 7f23b53e6700 1 -- 172.21.15.116:0/625664735 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a0009b60 con 0x7f23b00c4c50 2020-11-20T12:18:26.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.908+0000 7f23b864c700 1 -- 172.21.15.116:0/625664735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23b00c4c50 msgr2=0x7f23b00be110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:26.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.908+0000 7f23b864c700 1 --2- 172.21.15.116:0/625664735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23b00c4c50 0x7f23b00be110 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f23a000b9a0 tx=0x7f23a0006df0).stop 2020-11-20T12:18:26.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.908+0000 7f23b864c700 1 -- 172.21.15.116:0/625664735 shutdown_connections 2020-11-20T12:18:26.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.908+0000 7f23b864c700 1 --2- 172.21.15.116:0/625664735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23b00a7830 0x7f23b00a7c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:26.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.908+0000 7f23b864c700 1 --2- 172.21.15.116:0/625664735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23b00c4c50 0x7f23b00be110 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:26.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.908+0000 7f23b864c700 1 -- 172.21.15.116:0/625664735 >> 172.21.15.116:0/625664735 conn(0x7f23b0068320 msgr2=0x7f23b00c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:26.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.908+0000 7f23b864c700 1 -- 172.21.15.116:0/625664735 shutdown_connections 2020-11-20T12:18:26.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.908+0000 7f23b864c700 1 -- 172.21.15.116:0/625664735 wait complete. 2020-11-20T12:18:26.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b864c700 1 Processor -- start 2020-11-20T12:18:26.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b864c700 1 -- start start 2020-11-20T12:18:26.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b864c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23b00a7830 0x7f23b00c1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:26.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b864c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23b00be600 0x7f23b00ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:26.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b864c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23b00c4c50 0x7f23b0160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:26.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b864c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b00acc00 con 0x7f23b00a7830 2020-11-20T12:18:26.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b864c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0164ec0 con 0x7f23b00be600 2020-11-20T12:18:26.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b864c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b0165160 con 0x7f23b00c4c50 2020-11-20T12:18:26.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b63e8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23b00a7830 0x7f23b00c1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:26.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b63e8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23b00a7830 0x7f23b00c1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46724/0 (socket says 172.21.15.116:46724) 2020-11-20T12:18:26.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b63e8700 1 -- 172.21.15.116:0/3025652239 learned_addr learned my addr 172.21.15.116:0/3025652239 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:26.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b6be9700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23b00c4c50 0x7f23b0160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:26.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b5be7700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23b00be600 0x7f23b00ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:26.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b63e8700 1 -- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23b00c4c50 msgr2=0x7f23b0160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:26.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b63e8700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23b00c4c50 0x7f23b0160a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:26.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b63e8700 1 -- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23b00be600 msgr2=0x7f23b00ae180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:26.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b63e8700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23b00be600 0x7f23b00ae180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:26.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.912+0000 7f23b63e8700 1 -- 172.21.15.116:0/3025652239 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b0165400 con 0x7f23b00a7830 2020-11-20T12:18:26.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.916+0000 7f23b63e8700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23b00a7830 0x7f23b00c1b00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f23ac00eba0 tx=0x7f23ac006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:26.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.916+0000 7f239e7fc700 1 -- 172.21.15.116:0/3025652239 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac00ca60 con 0x7f23b00a7830 2020-11-20T12:18:26.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.916+0000 7f239e7fc700 1 -- 172.21.15.116:0/3025652239 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23ac00cbc0 con 0x7f23b00a7830 2020-11-20T12:18:26.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.916+0000 7f23b864c700 1 -- 172.21.15.116:0/3025652239 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b01656a0 con 0x7f23b00a7830 2020-11-20T12:18:26.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.916+0000 7f239e7fc700 1 -- 172.21.15.116:0/3025652239 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23ac0123f0 con 0x7f23b00a7830 2020-11-20T12:18:26.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.916+0000 7f23b864c700 1 -- 172.21.15.116:0/3025652239 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b00c1ff0 con 0x7f23b00a7830 2020-11-20T12:18:26.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.916+0000 7f23b864c700 1 -- 172.21.15.116:0/3025652239 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b00c22a0 con 0x7f23b00a7830 2020-11-20T12:18:26.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.916+0000 7f239e7fc700 1 -- 172.21.15.116:0/3025652239 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f23ac0127a0 con 0x7f23b00a7830 2020-11-20T12:18:26.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.920+0000 7f239e7fc700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f23880557c0 0x7f2388057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:26.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.920+0000 7f239e7fc700 1 -- 172.21.15.116:0/3025652239 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f23ac06eb30 con 0x7f23b00a7830 2020-11-20T12:18:26.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.920+0000 7f23b5be7700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f23880557c0 0x7f2388057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:26.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.920+0000 7f23b5be7700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f23880557c0 0x7f2388057c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f23a000b9a0 tx=0x7f23a0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:26.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:26.928+0000 7f239e7fc700 1 -- 172.21.15.116:0/3025652239 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f23ac044a40 con 0x7f23b00a7830 2020-11-20T12:18:27.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.132+0000 7f23b864c700 1 -- 172.21.15.116:0/3025652239 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f23b01659f0 con 0x7f23b00a7830 2020-11-20T12:18:27.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.144+0000 7f239e7fc700 1 -- 172.21.15.116:0/3025652239 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 9 and data pool 10 v25) v1 ==== 188+0+0 (secure 0 0 0) 0x7f23ac048060 con 0x7f23b00a7830 2020-11-20T12:18:27.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 -- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f23880557c0 msgr2=0x7f2388057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f23880557c0 0x7f2388057c80 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f23a000b9a0 tx=0x7f23a0015000).stop 2020-11-20T12:18:27.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 -- 172.21.15.116:0/3025652239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23b00a7830 msgr2=0x7f23b00c1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23b00a7830 0x7f23b00c1b00 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f23ac00eba0 tx=0x7f23ac006df0).stop 2020-11-20T12:18:27.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 -- 172.21.15.116:0/3025652239 shutdown_connections 2020-11-20T12:18:27.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f23880557c0 0x7f2388057c80 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f23b00c4c50 0x7f23b0160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f23b00a7830 0x7f23b00c1b00 secure :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0x7f23ac00eba0 tx=0x7f23ac006df0).stop 2020-11-20T12:18:27.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 --2- 172.21.15.116:0/3025652239 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f23b00be600 0x7f23b00ae180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 -- 172.21.15.116:0/3025652239 >> 172.21.15.116:0/3025652239 conn(0x7f23b0068320 msgr2=0x7f23b00bde30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:27.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 -- 172.21.15.116:0/3025652239 shutdown_connections 2020-11-20T12:18:27.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.152+0000 7f23b864c700 1 -- 172.21.15.116:0/3025652239 wait complete. 2020-11-20T12:18:27.165 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 9 and data pool 10 2020-11-20T12:18:27.188 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f977149e700 1 Processor -- start 2020-11-20T12:18:27.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f977149e700 1 -- start start 2020-11-20T12:18:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f977149e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c062880 0x7f976c062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f977149e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f976c14b990 0x7f976c14be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f977149e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976c0ba1b0 con 0x7f976c0b94e0 2020-11-20T12:18:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f977149e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976c152e60 con 0x7f976c14b990 2020-11-20T12:18:27.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f977149e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c1555a0 con 0x7f976c062880 2020-11-20T12:18:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f9770c9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f976c0b94e0 0x7f976c0b9960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56180/0 (socket says 172.21.15.116:56180) 2020-11-20T12:18:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f9770c9d700 1 -- 172.21.15.116:0/3470538665 learned_addr learned my addr 172.21.15.116:0/3470538665 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f976bfff700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c062880 0x7f976c062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f976b7fe700 1 --2- 172.21.15.116:0/3470538665 >> v2:172.21.15.173:3300/0 conn(0x7f976c14b990 0x7f976c14be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912322028 0 0) 0x7f976c0ba1b0 con 0x7f976c0b94e0 2020-11-20T12:18:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.376+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9758003240 con 0x7f976c0b94e0 2020-11-20T12:18:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1492617925 0 0) 0x7f9758003240 con 0x7f976c0b94e0 2020-11-20T12:18:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c062880 msgr2=0x7f976c062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:27.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c062880 0x7f976c062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 >> v2:172.21.15.173:3300/0 conn(0x7f976c14b990 msgr2=0x7f976c14be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 --2- 172.21.15.116:0/3470538665 >> v2:172.21.15.173:3300/0 conn(0x7f976c14b990 0x7f976c14be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c155840 con 0x7f976c0b94e0 2020-11-20T12:18:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f9764001860 con 0x7f976c0b94e0 2020-11-20T12:18:27.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 >> v1:172.21.15.116:6789/0 conn(0x7f976c0b94e0 legacy=0x7f976c0b9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c062880 0x7f97580037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9758003d30 0x7f97580044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97580049d0 0x7f9758004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c152e60 con 0x7f9758003d30 2020-11-20T12:18:27.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c1555a0 con 0x7f97580049d0 2020-11-20T12:18:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97580042f0 con 0x7f976c062880 2020-11-20T12:18:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976b7fe700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9758003d30 0x7f97580044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f9770c9d700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97580049d0 0x7f9758004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976bfff700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c062880 0x7f97580037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976b7fe700 1 -- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c062880 msgr2=0x7f97580037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976b7fe700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c062880 0x7f97580037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976b7fe700 1 -- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97580049d0 msgr2=0x7f9758004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:27.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976b7fe700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97580049d0 0x7f9758004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976b7fe700 1 -- 172.21.15.116:0/3470538665 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9758003240 con 0x7f9758003d30 2020-11-20T12:18:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f9770c9d700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97580049d0 0x7f9758004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976b7fe700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9758003d30 0x7f97580044e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f976c063150 tx=0x7f9760006df0).ready entity=mon.0 client_cookie=bf04c464a7e34553 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:27.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976000b330 con 0x7f9758003d30 2020-11-20T12:18:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f976000ea80 con 0x7f9758003d30 2020-11-20T12:18:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f976affd700 1 -- 172.21.15.116:0/3470538665 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976000ee00 con 0x7f9758003d30 2020-11-20T12:18:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f977149e700 1 -- 172.21.15.116:0/3470538665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9758003d30 msgr2=0x7f97580044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f977149e700 1 --2- 172.21.15.116:0/3470538665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9758003d30 0x7f97580044e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f976c063150 tx=0x7f9760006df0).stop 2020-11-20T12:18:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f9770c9d700 1 -- 172.21.15.116:0/3470538665 reap_dead start 2020-11-20T12:18:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.380+0000 7f977149e700 1 -- 172.21.15.116:0/3470538665 shutdown_connections 2020-11-20T12:18:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.384+0000 7f977149e700 1 -- 172.21.15.116:0/3470538665 >> 172.21.15.116:0/3470538665 conn(0x7f976c060c40 msgr2=0x7f976c0c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:27.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.384+0000 7f977149e700 1 -- 172.21.15.116:0/3470538665 shutdown_connections 2020-11-20T12:18:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.384+0000 7f977149e700 1 -- 172.21.15.116:0/3470538665 wait complete. 2020-11-20T12:18:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.384+0000 7f977149e700 1 Processor -- start 2020-11-20T12:18:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.384+0000 7f977149e700 1 -- start start 2020-11-20T12:18:27.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f977149e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f976c0a7da0 0x7f976c0a8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f977149e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c0a9b20 0x7f976c164f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f977149e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f976c0a3e50 0x7f976c0c3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f977149e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c0aa110 con 0x7f976c0a7da0 2020-11-20T12:18:27.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f977149e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c0aa3b0 con 0x7f976c0a3e50 2020-11-20T12:18:27.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f977149e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c1645b0 con 0x7f976c0a9b20 2020-11-20T12:18:27.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f976bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f976c0a7da0 0x7f976c0a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f976bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f976c0a7da0 0x7f976c0a8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46754/0 (socket says 172.21.15.116:46754) 2020-11-20T12:18:27.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f976bfff700 1 -- 172.21.15.116:0/2341576700 learned_addr learned my addr 172.21.15.116:0/2341576700 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:27.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f976b7fe700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c0a9b20 0x7f976c164f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f9770c9d700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f976c0a3e50 0x7f976c0c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f976bfff700 1 -- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c0a9b20 msgr2=0x7f976c164f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f976bfff700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c0a9b20 0x7f976c164f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f976bfff700 1 -- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f976c0a3e50 msgr2=0x7f976c0c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f976bfff700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f976c0a3e50 0x7f976c0c3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f976bfff700 1 -- 172.21.15.116:0/2341576700 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c164850 con 0x7f976c0a7da0 2020-11-20T12:18:27.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f976bfff700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f976c0a7da0 0x7f976c0a8180 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f975c0067d0 tx=0x7f975c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:27.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f974bfff700 1 -- 172.21.15.116:0/2341576700 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00ea90 con 0x7f976c0a7da0 2020-11-20T12:18:27.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f974bfff700 1 -- 172.21.15.116:0/2341576700 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c00ebf0 con 0x7f976c0a7da0 2020-11-20T12:18:27.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f974bfff700 1 -- 172.21.15.116:0/2341576700 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c014470 con 0x7f976c0a7da0 2020-11-20T12:18:27.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f977149e700 1 -- 172.21.15.116:0/2341576700 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976c164af0 con 0x7f976c0a7da0 2020-11-20T12:18:27.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.388+0000 7f977149e700 1 -- 172.21.15.116:0/2341576700 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976c0c3870 con 0x7f976c0a7da0 2020-11-20T12:18:27.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.392+0000 7f977149e700 1 -- 172.21.15.116:0/2341576700 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976c05cf30 con 0x7f976c0a7da0 2020-11-20T12:18:27.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.392+0000 7f974bfff700 1 -- 172.21.15.116:0/2341576700 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f975c024070 con 0x7f976c0a7da0 2020-11-20T12:18:27.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.392+0000 7f974bfff700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97440557c0 0x7f9744057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.392+0000 7f974bfff700 1 -- 172.21.15.116:0/2341576700 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f975c06f1a0 con 0x7f976c0a7da0 2020-11-20T12:18:27.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.392+0000 7f976b7fe700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97440557c0 0x7f9744057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.396+0000 7f976b7fe700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97440557c0 0x7f9744057c80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f976c06a6e0 tx=0x7f9760018000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:27.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.396+0000 7f974bfff700 1 -- 172.21.15.116:0/2341576700 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f975c048e60 con 0x7f976c0a7da0 2020-11-20T12:18:27.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2020-11-20T12:18:27.438 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2020-11-20T12:18:27.439 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2020-11-20T12:18:27.439 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2020-11-20T12:18:27.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.556+0000 7f977149e700 1 -- 172.21.15.116:0/2341576700 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f976c061230 con 0x7f976c0a7da0 2020-11-20T12:18:27.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f974bfff700 1 -- 172.21.15.116:0/2341576700 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f975c018020 con 0x7f976c0a7da0 2020-11-20T12:18:27.558 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:27.558 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":55,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:18:27.137464+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:18:25.558441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-20T12:18:26.151785+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:27.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 -- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97440557c0 msgr2=0x7f9744057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97440557c0 0x7f9744057c80 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f976c06a6e0 tx=0x7f9760018000).stop 2020-11-20T12:18:27.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 -- 172.21.15.116:0/2341576700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f976c0a7da0 msgr2=0x7f976c0a8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f976c0a7da0 0x7f976c0a8180 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f975c0067d0 tx=0x7f975c008040).stop 2020-11-20T12:18:27.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 -- 172.21.15.116:0/2341576700 shutdown_connections 2020-11-20T12:18:27.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97440557c0 0x7f9744057c80 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f976c0a3e50 0x7f976c0c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f976c0a7da0 0x7f976c0a8180 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 --2- 172.21.15.116:0/2341576700 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f976c0a9b20 0x7f976c164f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 -- 172.21.15.116:0/2341576700 >> 172.21.15.116:0/2341576700 conn(0x7f976c060c40 msgr2=0x7f976c153780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:27.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 -- 172.21.15.116:0/2341576700 shutdown_connections 2020-11-20T12:18:27.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.560+0000 7f977149e700 1 -- 172.21.15.116:0/2341576700 wait complete. 2020-11-20T12:18:27.584 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:27.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.760+0000 7f0398c5b700 1 Processor -- start 2020-11-20T12:18:27.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.760+0000 7f0398c5b700 1 -- start start 2020-11-20T12:18:27.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.760+0000 7f0398c5b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0394066450 0x7f0394156f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.760+0000 7f0398c5b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f03940b9700 0x7f03940b9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.760+0000 7f0398c5b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0394066af0 con 0x7f0394150680 2020-11-20T12:18:27.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.760+0000 7f0398c5b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0394066900 con 0x7f03940b9700 2020-11-20T12:18:27.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.760+0000 7f0398c5b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0394150a60 con 0x7f0394066450 2020-11-20T12:18:27.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f039259c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0394150680 0x7f03940bf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56196/0 (socket says 172.21.15.116:56196) 2020-11-20T12:18:27.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f039259c700 1 -- 172.21.15.116:0/3488208989 learned_addr learned my addr 172.21.15.116:0/3488208989 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:27.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f038bfff700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0394066450 0x7f0394156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0392d9d700 1 --2- 172.21.15.116:0/3488208989 >> v2:172.21.15.173:3300/0 conn(0x7f03940b9700 0x7f03940b9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498216312 0 0) 0x7f0394066af0 con 0x7f0394150680 2020-11-20T12:18:27.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0374003240 con 0x7f0394150680 2020-11-20T12:18:27.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2761849907 0 0) 0x7f0374003240 con 0x7f0394150680 2020-11-20T12:18:27.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0394066450 msgr2=0x7f0394156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:27.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0394066450 0x7f0394156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 >> v2:172.21.15.173:3300/0 conn(0x7f03940b9700 msgr2=0x7f03940b9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:27.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 --2- 172.21.15.116:0/3488208989 >> v2:172.21.15.173:3300/0 conn(0x7f03940b9700 0x7f03940b9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0394066d90 con 0x7f0394150680 2020-11-20T12:18:27.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f03840014e0 con 0x7f0394150680 2020-11-20T12:18:27.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f038bfff700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0394066450 0x7f0394156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:27.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 >> v1:172.21.15.116:6789/0 conn(0x7f0394150680 legacy=0x7f03940bf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0374003730 0x7f0374003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0394066450 0x7f03740044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03740049d0 0x7f0374004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0394150a60 con 0x7f03740049d0 2020-11-20T12:18:27.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0374004190 con 0x7f0374003730 2020-11-20T12:18:27.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0374004350 con 0x7f0394066450 2020-11-20T12:18:27.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0392d9d700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03740049d0 0x7f0374004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f039259c700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0374003730 0x7f0374003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f038bfff700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0394066450 0x7f03740044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0392d9d700 1 -- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0394066450 msgr2=0x7f03740044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:27.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0392d9d700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0394066450 0x7f03740044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0392d9d700 1 -- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0374003730 msgr2=0x7f0374003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:27.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0392d9d700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0374003730 0x7f0374003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.764+0000 7f0392d9d700 1 -- 172.21.15.116:0/3488208989 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0374003240 con 0x7f03740049d0 2020-11-20T12:18:27.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.768+0000 7f0392d9d700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03740049d0 0x7f0374004e50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f038c009ed0 tx=0x7f038c0091d0).ready entity=mon.0 client_cookie=22bdb4cb9e5b0c42 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:27.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.768+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0095c0 con 0x7f03740049d0 2020-11-20T12:18:27.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.768+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c00ed10 con 0x7f03740049d0 2020-11-20T12:18:27.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.768+0000 7f0391d9b700 1 -- 172.21.15.116:0/3488208989 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c012600 con 0x7f03740049d0 2020-11-20T12:18:27.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.768+0000 7f0398c5b700 1 -- 172.21.15.116:0/3488208989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03740049d0 msgr2=0x7f0374004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.768+0000 7f0398c5b700 1 --2- 172.21.15.116:0/3488208989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f03740049d0 0x7f0374004e50 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f038c009ed0 tx=0x7f038c0091d0).stop 2020-11-20T12:18:27.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.768+0000 7f0392d9d700 1 -- 172.21.15.116:0/3488208989 reap_dead start 2020-11-20T12:18:27.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.768+0000 7f0398c5b700 1 -- 172.21.15.116:0/3488208989 shutdown_connections 2020-11-20T12:18:27.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.768+0000 7f0398c5b700 1 -- 172.21.15.116:0/3488208989 >> 172.21.15.116:0/3488208989 conn(0x7f03940c4f80 msgr2=0x7f03940c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:27.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.768+0000 7f0398c5b700 1 -- 172.21.15.116:0/3488208989 shutdown_connections 2020-11-20T12:18:27.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.768+0000 7f0398c5b700 1 -- 172.21.15.116:0/3488208989 wait complete. 2020-11-20T12:18:27.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0398c5b700 1 Processor -- start 2020-11-20T12:18:27.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0398c5b700 1 -- start start 2020-11-20T12:18:27.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0398c5b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03740049d0 0x7f03940bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0398c5b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03940a6970 0x7f03940a7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0398c5b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0394168ff0 0x7f0394169450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0398c5b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0394168920 con 0x7f0394168ff0 2020-11-20T12:18:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0398c5b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03940a6df0 con 0x7f03740049d0 2020-11-20T12:18:27.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0398c5b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03940bc4d0 con 0x7f03940a6970 2020-11-20T12:18:27.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0392d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0394168ff0 0x7f0394169450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0392d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0394168ff0 0x7f0394169450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46778/0 (socket says 172.21.15.116:46778) 2020-11-20T12:18:27.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0392d9d700 1 -- 172.21.15.116:0/1562764351 learned_addr learned my addr 172.21.15.116:0/1562764351 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:27.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f038bfff700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03940a6970 0x7f03940a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f039259c700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03740049d0 0x7f03940bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0392d9d700 1 -- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03940a6970 msgr2=0x7f03940a7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0392d9d700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03940a6970 0x7f03940a7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0392d9d700 1 -- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03740049d0 msgr2=0x7f03940bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0392d9d700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03740049d0 0x7f03940bbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0392d9d700 1 -- 172.21.15.116:0/1562764351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03940bc770 con 0x7f0394168ff0 2020-11-20T12:18:27.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f0392d9d700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0394168ff0 0x7f0394169450 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f038c009ea0 tx=0x7f038c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:27.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.772+0000 7f038a7fc700 1 -- 172.21.15.116:0/1562764351 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c0095c0 con 0x7f0394168ff0 2020-11-20T12:18:27.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.776+0000 7f038a7fc700 1 -- 172.21.15.116:0/1562764351 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c00ee80 con 0x7f0394168ff0 2020-11-20T12:18:27.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.776+0000 7f038a7fc700 1 -- 172.21.15.116:0/1562764351 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c01dd30 con 0x7f0394168ff0 2020-11-20T12:18:27.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.776+0000 7f0398c5b700 1 -- 172.21.15.116:0/1562764351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03940bca10 con 0x7f0394168ff0 2020-11-20T12:18:27.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.776+0000 7f0398c5b700 1 -- 172.21.15.116:0/1562764351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03940a7e70 con 0x7f0394168ff0 2020-11-20T12:18:27.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.776+0000 7f0398c5b700 1 -- 172.21.15.116:0/1562764351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039405d2a0 con 0x7f0394168ff0 2020-11-20T12:18:27.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.776+0000 7f038a7fc700 1 -- 172.21.15.116:0/1562764351 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f038c02f070 con 0x7f0394168ff0 2020-11-20T12:18:27.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.776+0000 7f038a7fc700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f03700557c0 0x7f0370057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:27.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.780+0000 7f038a7fc700 1 -- 172.21.15.116:0/1562764351 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f038c0704c0 con 0x7f0394168ff0 2020-11-20T12:18:27.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.780+0000 7f039259c700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f03700557c0 0x7f0370057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:27.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.780+0000 7f039259c700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f03700557c0 0x7f0370057c80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f03840077d0 tx=0x7f0384007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:27.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.784+0000 7f038a7fc700 1 -- 172.21.15.116:0/1562764351 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f038c046360 con 0x7f0394168ff0 2020-11-20T12:18:27.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.944+0000 7f0398c5b700 1 -- 172.21.15.116:0/1562764351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03940a3d80 con 0x7f0394168ff0 2020-11-20T12:18:27.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.944+0000 7f038a7fc700 1 -- 172.21.15.116:0/1562764351 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v55) v1 ==== 74+0+16889 (secure 0 0 0) 0x7f038c016070 con 0x7f0394168ff0 2020-11-20T12:18:27.943 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:27.944 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":55,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:18:27.137464+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:18:25.558441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-20T12:18:26.151785+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.944+0000 7f0398c5b700 1 -- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f03700557c0 msgr2=0x7f0370057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.944+0000 7f0398c5b700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f03700557c0 0x7f0370057c80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f03840077d0 tx=0x7f0384007800).stop 2020-11-20T12:18:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.944+0000 7f0398c5b700 1 -- 172.21.15.116:0/1562764351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0394168ff0 msgr2=0x7f0394169450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.944+0000 7f0398c5b700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0394168ff0 0x7f0394169450 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f038c009ea0 tx=0x7f038c01a000).stop 2020-11-20T12:18:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.948+0000 7f0398c5b700 1 -- 172.21.15.116:0/1562764351 shutdown_connections 2020-11-20T12:18:27.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.948+0000 7f0398c5b700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f03700557c0 0x7f0370057c80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.948+0000 7f0398c5b700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0394168ff0 0x7f0394169450 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.948+0000 7f0398c5b700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f03740049d0 0x7f03940bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.948+0000 7f0398c5b700 1 --2- 172.21.15.116:0/1562764351 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03940a6970 0x7f03940a7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:27.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.948+0000 7f0398c5b700 1 -- 172.21.15.116:0/1562764351 >> 172.21.15.116:0/1562764351 conn(0x7f03940c4f80 msgr2=0x7f03940c1e80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:27.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.948+0000 7f0398c5b700 1 -- 172.21.15.116:0/1562764351 shutdown_connections 2020-11-20T12:18:27.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:27.948+0000 7f0398c5b700 1 -- 172.21.15.116:0/1562764351 wait complete. 2020-11-20T12:18:27.968 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-20T12:18:28.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.140+0000 7fedbb246700 1 Processor -- start 2020-11-20T12:18:28.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.144+0000 7fedbb246700 1 -- start start 2020-11-20T12:18:28.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedbb246700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fedb40c8ed0 0x7fedb41528e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:28.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedbb246700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedb40614c0 0x7fedb4061950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:28.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedbb246700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb40ace20 con 0x7fedb414be10 2020-11-20T12:18:28.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedbb246700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb4152dd0 con 0x7fedb40c8ed0 2020-11-20T12:18:28.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedbb246700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40c9450 con 0x7fedb40614c0 2020-11-20T12:18:28.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedba244700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fedb414be10 0x7fedb40ada60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56216/0 (socket says 172.21.15.116:56216) 2020-11-20T12:18:28.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedba244700 1 -- 172.21.15.116:0/1062856449 learned_addr learned my addr 172.21.15.116:0/1062856449 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:28.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedb9a43700 1 --2- 172.21.15.116:0/1062856449 >> v2:172.21.15.116:3300/0 conn(0x7fedb40c8ed0 0x7fedb41528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:28.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedbaa45700 1 --2- 172.21.15.116:0/1062856449 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedb40614c0 0x7fedb4061950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:28.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedb9a43700 1 -- 172.21.15.116:0/1062856449 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedb40614c0 msgr2=0x7fedb4061950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:28.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedb9a43700 1 --2- 172.21.15.116:0/1062856449 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedb40614c0 0x7fedb4061950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:28.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedb9a43700 1 -- 172.21.15.116:0/1062856449 >> v1:172.21.15.116:6789/0 conn(0x7fedb414be10 legacy=0x7fedb40ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:28.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedb9a43700 1 -- 172.21.15.116:0/1062856449 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb40ad080 con 0x7fedb40c8ed0 2020-11-20T12:18:28.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedb9a43700 1 --2- 172.21.15.116:0/1062856449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedb40c8ed0 0x7fedb41528e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7feda400b9a0 tx=0x7feda4006df0).ready entity=mon.0 client_cookie=b47dcb7a8b7f473d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:28.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedb9242700 1 -- 172.21.15.116:0/1062856449 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda40097b0 con 0x7fedb40c8ed0 2020-11-20T12:18:28.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedb9242700 1 -- 172.21.15.116:0/1062856449 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda4009910 con 0x7fedb40c8ed0 2020-11-20T12:18:28.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.148+0000 7fedb9242700 1 -- 172.21.15.116:0/1062856449 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda4009c90 con 0x7fedb40c8ed0 2020-11-20T12:18:28.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.152+0000 7fedbb246700 1 -- 172.21.15.116:0/1062856449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedb40c8ed0 msgr2=0x7fedb41528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:28.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.152+0000 7fedbb246700 1 --2- 172.21.15.116:0/1062856449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedb40c8ed0 0x7fedb41528e0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7feda400b9a0 tx=0x7feda4006df0).stop 2020-11-20T12:18:28.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.152+0000 7fedbb246700 1 -- 172.21.15.116:0/1062856449 shutdown_connections 2020-11-20T12:18:28.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.152+0000 7fedbb246700 1 --2- 172.21.15.116:0/1062856449 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedb40614c0 0x7fedb4061950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:28.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.152+0000 7fedbb246700 1 --2- 172.21.15.116:0/1062856449 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedb40c8ed0 0x7fedb41528e0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:28.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.152+0000 7fedbb246700 1 -- 172.21.15.116:0/1062856449 >> 172.21.15.116:0/1062856449 conn(0x7fedb4068370 msgr2=0x7fedb40c87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:28.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.152+0000 7fedbb246700 1 -- 172.21.15.116:0/1062856449 shutdown_connections 2020-11-20T12:18:28.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.152+0000 7fedbb246700 1 -- 172.21.15.116:0/1062856449 wait complete. 2020-11-20T12:18:28.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.152+0000 7fedbb246700 1 Processor -- start 2020-11-20T12:18:28.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.152+0000 7fedbb246700 1 -- start start 2020-11-20T12:18:28.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedbb246700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedb40614c0 0x7fedb41390d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:28.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedbb246700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedb40c8ed0 0x7fedb414c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:28.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedbb246700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedb414be10 0x7fedb414b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:28.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedbb246700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40a8250 con 0x7fedb40614c0 2020-11-20T12:18:28.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedbb246700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40a84f0 con 0x7fedb40c8ed0 2020-11-20T12:18:28.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedbb246700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb40a8790 con 0x7fedb414be10 2020-11-20T12:18:28.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedba244700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedb40614c0 0x7fedb41390d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:28.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedba244700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedb40614c0 0x7fedb41390d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46788/0 (socket says 172.21.15.116:46788) 2020-11-20T12:18:28.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedba244700 1 -- 172.21.15.116:0/3947308123 learned_addr learned my addr 172.21.15.116:0/3947308123 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:28.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedb9a43700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedb40c8ed0 0x7fedb414c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:28.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedbaa45700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedb414be10 0x7fedb414b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:28.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedba244700 1 -- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedb414be10 msgr2=0x7fedb414b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:28.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedba244700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedb414be10 0x7fedb414b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:28.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedba244700 1 -- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedb40c8ed0 msgr2=0x7fedb414c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:28.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedba244700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedb40c8ed0 0x7fedb414c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:28.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedba244700 1 -- 172.21.15.116:0/3947308123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb40a8a30 con 0x7fedb40614c0 2020-11-20T12:18:28.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedba244700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedb40614c0 0x7fedb41390d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fedb000c710 tx=0x7fedb0007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:28.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7feda27fc700 1 -- 172.21.15.116:0/3947308123 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb00072e0 con 0x7fedb40614c0 2020-11-20T12:18:28.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedbb246700 1 -- 172.21.15.116:0/3947308123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb40a8cd0 con 0x7fedb40614c0 2020-11-20T12:18:28.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7feda27fc700 1 -- 172.21.15.116:0/3947308123 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedb0007440 con 0x7fedb40614c0 2020-11-20T12:18:28.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7fedbb246700 1 -- 172.21.15.116:0/3947308123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb4139760 con 0x7fedb40614c0 2020-11-20T12:18:28.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.156+0000 7feda27fc700 1 -- 172.21.15.116:0/3947308123 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedb0005800 con 0x7fedb40614c0 2020-11-20T12:18:28.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.160+0000 7fedbb246700 1 -- 172.21.15.116:0/3947308123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb405cf30 con 0x7fedb40614c0 2020-11-20T12:18:28.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.160+0000 7feda27fc700 1 -- 172.21.15.116:0/3947308123 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fedb0010070 con 0x7fedb40614c0 2020-11-20T12:18:28.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.160+0000 7feda27fc700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fed8c0557c0 0x7fed8c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:28.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.160+0000 7feda27fc700 1 -- 172.21.15.116:0/3947308123 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fedb006dc50 con 0x7fedb40614c0 2020-11-20T12:18:28.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.160+0000 7fedb9a43700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fed8c0557c0 0x7fed8c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:28.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.160+0000 7fedb9a43700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fed8c0557c0 0x7fed8c057c80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7feda400b150 tx=0x7feda4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:28.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.164+0000 7feda27fc700 1 -- 172.21.15.116:0/3947308123 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fedb0043af0 con 0x7fedb40614c0 2020-11-20T12:18:28.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:28.376+0000 7fedbb246700 1 -- 172.21.15.116:0/3947308123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fedb40601e0 con 0x7fedb40614c0 2020-11-20T12:18:29.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.168+0000 7feda27fc700 1 -- 172.21.15.116:0/3947308123 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v28) v1 ==== 118+0+0 (secure 0 0 0) 0x7fedb0007de0 con 0x7fedb40614c0 2020-11-20T12:18:29.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 -- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fed8c0557c0 msgr2=0x7fed8c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fed8c0557c0 0x7fed8c057c80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7feda400b150 tx=0x7feda4015000).stop 2020-11-20T12:18:29.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 -- 172.21.15.116:0/3947308123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedb40614c0 msgr2=0x7fedb41390d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedb40614c0 0x7fedb41390d0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fedb000c710 tx=0x7fedb0007ca0).stop 2020-11-20T12:18:29.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 -- 172.21.15.116:0/3947308123 shutdown_connections 2020-11-20T12:18:29.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fed8c0557c0 0x7fed8c057c80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedb414be10 0x7fedb414b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedb40614c0 0x7fedb41390d0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 --2- 172.21.15.116:0/3947308123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedb40c8ed0 0x7fedb414c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 -- 172.21.15.116:0/3947308123 >> 172.21.15.116:0/3947308123 conn(0x7fedb4068370 msgr2=0x7fedb40ad980 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:29.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 -- 172.21.15.116:0/3947308123 shutdown_connections 2020-11-20T12:18:29.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.172+0000 7fedbb246700 1 -- 172.21.15.116:0/3947308123 wait complete. 2020-11-20T12:18:29.194 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:18:29.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe044df7700 1 Processor -- start 2020-11-20T12:18:29.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe044df7700 1 -- start start 2020-11-20T12:18:29.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe044df7700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe040062880 0x7fe040062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe044df7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 0x7fe0400677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe044df7700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0400c4c10 0x7fe0400b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe044df7700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0401543a0 con 0x7fe0400c4c10 2020-11-20T12:18:29.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe044df7700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe040153920 con 0x7fe040062880 2020-11-20T12:18:29.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe044df7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe040153bc0 con 0x7fe040153450 2020-11-20T12:18:29.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe03ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0400c4c10 0x7fe0400b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe03effd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 0x7fe0400677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe03effd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 0x7fe0400677a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:55746/0 (socket says 172.21.15.116:55746) 2020-11-20T12:18:29.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe03effd700 1 -- 172.21.15.116:0/3028004658 learned_addr learned my addr 172.21.15.116:0/3028004658 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:29.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe03f7fe700 1 --2- 172.21.15.116:0/3028004658 >> v2:172.21.15.173:3300/0 conn(0x7fe040062880 0x7fe040062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe03effd700 1 -- 172.21.15.116:0/3028004658 >> v2:172.21.15.173:3300/0 conn(0x7fe040062880 msgr2=0x7fe040062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:29.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe03effd700 1 --2- 172.21.15.116:0/3028004658 >> v2:172.21.15.173:3300/0 conn(0x7fe040062880 0x7fe040062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.376+0000 7fe03effd700 1 -- 172.21.15.116:0/3028004658 >> v2:172.21.15.116:3300/0 conn(0x7fe0400c4c10 msgr2=0x7fe0400b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:29.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe03effd700 1 --2- 172.21.15.116:0/3028004658 >> v2:172.21.15.116:3300/0 conn(0x7fe0400c4c10 0x7fe0400b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe03effd700 1 -- 172.21.15.116:0/3028004658 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe040153e60 con 0x7fe040153450 2020-11-20T12:18:29.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe03effd700 1 --2- 172.21.15.116:0/3028004658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 0x7fe0400677a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fe0400c1e20 tx=0x7fe028006eb0).ready entity=mon.2 client_cookie=77842e542fec64c4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:29.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe03e7fc700 1 -- 172.21.15.116:0/3028004658 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028009720 con 0x7fe040153450 2020-11-20T12:18:29.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe03e7fc700 1 -- 172.21.15.116:0/3028004658 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe028009880 con 0x7fe040153450 2020-11-20T12:18:29.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe03e7fc700 1 -- 172.21.15.116:0/3028004658 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028009c00 con 0x7fe040153450 2020-11-20T12:18:29.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe044df7700 1 -- 172.21.15.116:0/3028004658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 msgr2=0x7fe0400677a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe044df7700 1 --2- 172.21.15.116:0/3028004658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 0x7fe0400677a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fe0400c1e20 tx=0x7fe028006eb0).stop 2020-11-20T12:18:29.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe044df7700 1 -- 172.21.15.116:0/3028004658 shutdown_connections 2020-11-20T12:18:29.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe044df7700 1 --2- 172.21.15.116:0/3028004658 >> v2:172.21.15.116:3300/0 conn(0x7fe0400c4c10 0x7fe0400b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe044df7700 1 --2- 172.21.15.116:0/3028004658 >> v2:172.21.15.173:3300/0 conn(0x7fe040062880 0x7fe040062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe044df7700 1 --2- 172.21.15.116:0/3028004658 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 0x7fe0400677a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe044df7700 1 -- 172.21.15.116:0/3028004658 >> 172.21.15.116:0/3028004658 conn(0x7fe040060c40 msgr2=0x7fe0400c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:29.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe044df7700 1 -- 172.21.15.116:0/3028004658 shutdown_connections 2020-11-20T12:18:29.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.380+0000 7fe044df7700 1 -- 172.21.15.116:0/3028004658 wait complete. 2020-11-20T12:18:29.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe044df7700 1 Processor -- start 2020-11-20T12:18:29.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe044df7700 1 -- start start 2020-11-20T12:18:29.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe044df7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe040062880 0x7fe0401503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe044df7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0400c4c10 0x7fe0401508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe044df7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 0x7fe040150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe044df7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04013a6f0 con 0x7fe040062880 2020-11-20T12:18:29.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe044df7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe040142640 con 0x7fe0400c4c10 2020-11-20T12:18:29.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe044df7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe040143530 con 0x7fe040153450 2020-11-20T12:18:29.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe040062880 0x7fe0401503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe040062880 0x7fe0401503b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46802/0 (socket says 172.21.15.116:46802) 2020-11-20T12:18:29.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03f7fe700 1 -- 172.21.15.116:0/1513848738 learned_addr learned my addr 172.21.15.116:0/1513848738 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:29.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03ffff700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 0x7fe040150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03effd700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0400c4c10 0x7fe0401508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03f7fe700 1 -- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 msgr2=0x7fe040150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03f7fe700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 0x7fe040150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03f7fe700 1 -- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0400c4c10 msgr2=0x7fe0401508a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03f7fe700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0400c4c10 0x7fe0401508a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03f7fe700 1 -- 172.21.15.116:0/1513848738 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0401428b0 con 0x7fe040062880 2020-11-20T12:18:29.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03effd700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0400c4c10 0x7fe0401508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:29.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe03f7fe700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe040062880 0x7fe0401503b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fe03000eba0 tx=0x7fe030006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:29.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe0377fe700 1 -- 172.21.15.116:0/1513848738 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe03000ca60 con 0x7fe040062880 2020-11-20T12:18:29.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe0377fe700 1 -- 172.21.15.116:0/1513848738 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe03000cbc0 con 0x7fe040062880 2020-11-20T12:18:29.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.384+0000 7fe044df7700 1 -- 172.21.15.116:0/1513848738 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe040142b50 con 0x7fe040062880 2020-11-20T12:18:29.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.388+0000 7fe0377fe700 1 -- 172.21.15.116:0/1513848738 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300123f0 con 0x7fe040062880 2020-11-20T12:18:29.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.388+0000 7fe044df7700 1 -- 172.21.15.116:0/1513848738 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0401444d0 con 0x7fe040062880 2020-11-20T12:18:29.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.388+0000 7fe044df7700 1 -- 172.21.15.116:0/1513848738 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04005cf30 con 0x7fe040062880 2020-11-20T12:18:29.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.388+0000 7fe0377fe700 1 -- 172.21.15.116:0/1513848738 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe0300127a0 con 0x7fe040062880 2020-11-20T12:18:29.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.392+0000 7fe0377fe700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe01c0557c0 0x7fe01c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.392+0000 7fe0377fe700 1 -- 172.21.15.116:0/1513848738 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe03006eb90 con 0x7fe040062880 2020-11-20T12:18:29.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.392+0000 7fe03effd700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe01c0557c0 0x7fe01c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.392+0000 7fe03effd700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe01c0557c0 0x7fe01c057c80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fe040153850 tx=0x7fe028015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:29.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.396+0000 7fe0377fe700 1 -- 172.21.15.116:0/1513848738 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe030044a30 con 0x7fe040062880 2020-11-20T12:18:29.587 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:18:29.594 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:18:29.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.604+0000 7fe044df7700 1 -- 172.21.15.116:0/1513848738 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe0400c1370 con 0x7fe040062880 2020-11-20T12:18:29.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe0377fe700 1 -- 172.21.15.116:0/1513848738 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2850 (secure 0 0 0) 0x7fe030048050 con 0x7fe040062880 2020-11-20T12:18:29.605 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:29.606 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/785878283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":785878283},{"type":"v1","addr":"172.21.15.173:6837","nonce":785878283}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5952,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1568769426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1568769426},{"type":"v1","addr":"172.21.15.173:6836","nonce":1568769426}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3552490451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3552490451},{"type":"v1","addr":"172.21.15.116:6833","nonce":3552490451}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:18:27.137821+0000","modified":"2020-11-20T12:18:29.161493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3370673707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3370673707},{"type":"v1","addr":"172.21.15.116:6835","nonce":3370673707}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-20T12:18:29.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 -- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe01c0557c0 msgr2=0x7fe01c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe01c0557c0 0x7fe01c057c80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fe040153850 tx=0x7fe028015000).stop 2020-11-20T12:18:29.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 -- 172.21.15.116:0/1513848738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe040062880 msgr2=0x7fe0401503b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe040062880 0x7fe0401503b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fe03000eba0 tx=0x7fe030006df0).stop 2020-11-20T12:18:29.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 -- 172.21.15.116:0/1513848738 shutdown_connections 2020-11-20T12:18:29.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe01c0557c0 0x7fe01c057c80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe040153450 0x7fe040150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe040062880 0x7fe0401503b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 --2- 172.21.15.116:0/1513848738 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0400c4c10 0x7fe0401508a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 -- 172.21.15.116:0/1513848738 >> 172.21.15.116:0/1513848738 conn(0x7fe040060c40 msgr2=0x7fe040067da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:29.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 -- 172.21.15.116:0/1513848738 shutdown_connections 2020-11-20T12:18:29.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.608+0000 7fe044df7700 1 -- 172.21.15.116:0/1513848738 wait complete. 2020-11-20T12:18:29.610 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 28 2020-11-20T12:18:29.631 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:29.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.876+0000 7efe1759e700 1 Processor -- start 2020-11-20T12:18:29.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.876+0000 7efe1759e700 1 -- start start 2020-11-20T12:18:29.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe1759e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efe18062880 0x7efe18062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe1759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe18153450 0x7efe180677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe1759e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe181543a0 con 0x7efe180c4c10 2020-11-20T12:18:29.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe1759e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe18153920 con 0x7efe18062880 2020-11-20T12:18:29.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe1759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe18153bc0 con 0x7efe18153450 2020-11-20T12:18:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe16d9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7efe180c4c10 0x7efe180b9710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56246/0 (socket says 172.21.15.116:56246) 2020-11-20T12:18:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe16d9d700 1 -- 172.21.15.116:0/533052938 learned_addr learned my addr 172.21.15.116:0/533052938 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe1659c700 1 --2- 172.21.15.116:0/533052938 >> v2:172.21.15.116:3300/0 conn(0x7efe18062880 0x7efe18062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe0dd9b700 1 --2- 172.21.15.116:0/533052938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe18153450 0x7efe180677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe15d9b700 1 -- 172.21.15.116:0/533052938 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047150445 0 0) 0x7efe181543a0 con 0x7efe180c4c10 2020-11-20T12:18:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe15d9b700 1 -- 172.21.15.116:0/533052938 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdf8003240 con 0x7efe180c4c10 2020-11-20T12:18:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe0dd9b700 1 -- 172.21.15.116:0/533052938 >> v2:172.21.15.116:3300/0 conn(0x7efe18062880 msgr2=0x7efe18062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe0dd9b700 1 --2- 172.21.15.116:0/533052938 >> v2:172.21.15.116:3300/0 conn(0x7efe18062880 0x7efe18062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe0dd9b700 1 -- 172.21.15.116:0/533052938 >> v1:172.21.15.116:6789/0 conn(0x7efe180c4c10 legacy=0x7efe180b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe0dd9b700 1 -- 172.21.15.116:0/533052938 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe18153e60 con 0x7efe18153450 2020-11-20T12:18:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe1659c700 1 --2- 172.21.15.116:0/533052938 >> v2:172.21.15.116:3300/0 conn(0x7efe18062880 0x7efe18062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.880+0000 7efe0dd9b700 1 --2- 172.21.15.116:0/533052938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe18153450 0x7efe180677a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7efe0000b9a0 tx=0x7efe00006df0).ready entity=mon.2 client_cookie=fb0eb5c21946e562 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.884+0000 7efe15d9b700 1 -- 172.21.15.116:0/533052938 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00009680 con 0x7efe18153450 2020-11-20T12:18:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.884+0000 7efe15d9b700 1 -- 172.21.15.116:0/533052938 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe000097e0 con 0x7efe18153450 2020-11-20T12:18:29.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.884+0000 7efe15d9b700 1 -- 172.21.15.116:0/533052938 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe00009b60 con 0x7efe18153450 2020-11-20T12:18:29.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.884+0000 7efe1759e700 1 -- 172.21.15.116:0/533052938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe18153450 msgr2=0x7efe180677a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.884+0000 7efe1759e700 1 --2- 172.21.15.116:0/533052938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe18153450 0x7efe180677a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7efe0000b9a0 tx=0x7efe00006df0).stop 2020-11-20T12:18:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.884+0000 7efe1759e700 1 -- 172.21.15.116:0/533052938 shutdown_connections 2020-11-20T12:18:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.884+0000 7efe1759e700 1 --2- 172.21.15.116:0/533052938 >> v2:172.21.15.116:3300/0 conn(0x7efe18062880 0x7efe18062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.884+0000 7efe1759e700 1 --2- 172.21.15.116:0/533052938 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe18153450 0x7efe180677a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.884+0000 7efe1759e700 1 -- 172.21.15.116:0/533052938 >> 172.21.15.116:0/533052938 conn(0x7efe18060c40 msgr2=0x7efe180c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.884+0000 7efe1759e700 1 -- 172.21.15.116:0/533052938 shutdown_connections 2020-11-20T12:18:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.884+0000 7efe1759e700 1 -- 172.21.15.116:0/533052938 wait complete. 2020-11-20T12:18:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe1759e700 1 Processor -- start 2020-11-20T12:18:29.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe1759e700 1 -- start start 2020-11-20T12:18:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe1759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe18062880 0x7efe1814a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe1759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c4c10 0x7efe1815f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe1759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe1815f7b0 0x7efe18160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe1759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1814c2e0 con 0x7efe1815f7b0 2020-11-20T12:18:29.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe1759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe18160230 con 0x7efe18062880 2020-11-20T12:18:29.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe1759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1815fcf0 con 0x7efe180c4c10 2020-11-20T12:18:29.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe16d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe1815f7b0 0x7efe18160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe16d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe1815f7b0 0x7efe18160430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46820/0 (socket says 172.21.15.116:46820) 2020-11-20T12:18:29.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe16d9d700 1 -- 172.21.15.116:0/1498263064 learned_addr learned my addr 172.21.15.116:0/1498263064 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:29.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe1659c700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe18062880 0x7efe1814a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.888+0000 7efe0dd9b700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c4c10 0x7efe1815f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe16d9d700 1 -- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c4c10 msgr2=0x7efe1815f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe16d9d700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c4c10 0x7efe1815f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe16d9d700 1 -- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe18062880 msgr2=0x7efe1814a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:29.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe16d9d700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe18062880 0x7efe1814a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:29.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe16d9d700 1 -- 172.21.15.116:0/1498263064 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe1815ff90 con 0x7efe1815f7b0 2020-11-20T12:18:29.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe1659c700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe18062880 0x7efe1814a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:29.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe16d9d700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe1815f7b0 0x7efe18160430 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7efe10001420 tx=0x7efe10007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:29.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe0effd700 1 -- 172.21.15.116:0/1498263064 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10005b80 con 0x7efe1815f7b0 2020-11-20T12:18:29.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe0effd700 1 -- 172.21.15.116:0/1498263064 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe10005ce0 con 0x7efe1815f7b0 2020-11-20T12:18:29.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe1759e700 1 -- 172.21.15.116:0/1498263064 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1814c4d0 con 0x7efe1815f7b0 2020-11-20T12:18:29.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe0effd700 1 -- 172.21.15.116:0/1498263064 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10007320 con 0x7efe1815f7b0 2020-11-20T12:18:29.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe1759e700 1 -- 172.21.15.116:0/1498263064 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe180676f0 con 0x7efe1815f7b0 2020-11-20T12:18:29.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe1759e700 1 -- 172.21.15.116:0/1498263064 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1814c970 con 0x7efe1815f7b0 2020-11-20T12:18:29.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.892+0000 7efe0effd700 1 -- 172.21.15.116:0/1498263064 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7efe10010030 con 0x7efe1815f7b0 2020-11-20T12:18:29.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.896+0000 7efe0effd700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 0x7efdf4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:29.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.896+0000 7efe0effd700 1 -- 172.21.15.116:0/1498263064 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efe1006e670 con 0x7efe1815f7b0 2020-11-20T12:18:29.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.896+0000 7efe1659c700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 0x7efdf4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:29.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.896+0000 7efe1659c700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 0x7efdf4057c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7efe181603e0 tx=0x7efe08008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:29.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:29.900+0000 7efe0effd700 1 -- 172.21.15.116:0/1498263064 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efe1814c970 con 0x7efe1815f7b0 2020-11-20T12:18:30.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.060+0000 7efe1759e700 1 -- 172.21.15.116:0/1498263064 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe1814c970 con 0x7efe1815f7b0 2020-11-20T12:18:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.060+0000 7efe0effd700 1 -- 172.21.15.116:0/1498263064 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16876 (secure 0 0 0) 0x7efe1814c970 con 0x7efe1815f7b0 2020-11-20T12:18:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 -- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 msgr2=0x7efdf4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 0x7efdf4057c80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7efe181603e0 tx=0x7efe08008040).stop 2020-11-20T12:18:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 -- 172.21.15.116:0/1498263064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe1815f7b0 msgr2=0x7efe18160430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe1815f7b0 0x7efe18160430 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7efe10001420 tx=0x7efe10007e20).stop 2020-11-20T12:18:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 -- 172.21.15.116:0/1498263064 shutdown_connections 2020-11-20T12:18:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efdf40557c0 0x7efdf4057c80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efe1815f7b0 0x7efe18160430 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efe18062880 0x7efe1814a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 --2- 172.21.15.116:0/1498263064 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efe180c4c10 0x7efe1815f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 -- 172.21.15.116:0/1498263064 >> 172.21.15.116:0/1498263064 conn(0x7efe18060c40 msgr2=0x7efe18149c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:30.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 -- 172.21.15.116:0/1498263064 shutdown_connections 2020-11-20T12:18:30.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.064+0000 7efe1759e700 1 -- 172.21.15.116:0/1498263064 wait complete. 2020-11-20T12:18:30.087 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:30.087 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":56,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:18:28.142253+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:18:25.558441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-20T12:18:26.151785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:30.090 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-20T12:18:30.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.280+0000 7f6c36e88700 1 Processor -- start 2020-11-20T12:18:30.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.280+0000 7f6c36e88700 1 -- start start 2020-11-20T12:18:30.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c36e88700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6c30154dd0 0x7f6c30156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c36e88700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6c300bd810 0x7f6c300be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c36e88700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c301551b0 0x7f6c300a6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c36e88700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c300bdfc0 con 0x7f6c30154dd0 2020-11-20T12:18:30.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c36e88700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c300bdd20 con 0x7f6c300bd810 2020-11-20T12:18:30.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c36e88700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c30155780 con 0x7f6c301551b0 2020-11-20T12:18:30.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c34c24700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6c30154dd0 0x7f6c30156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c34c24700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6c30154dd0 0x7f6c30156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46824/0 (socket says 172.21.15.116:46824) 2020-11-20T12:18:30.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c34c24700 1 -- 172.21.15.116:0/1624090555 learned_addr learned my addr 172.21.15.116:0/1624090555 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:30.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c2ffff700 1 --2- 172.21.15.116:0/1624090555 >> v2:172.21.15.173:3300/0 conn(0x7f6c300bd810 0x7f6c300be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c35425700 1 --2- 172.21.15.116:0/1624090555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c301551b0 0x7f6c300a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c34c24700 1 -- 172.21.15.116:0/1624090555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c301551b0 msgr2=0x7f6c300a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:30.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c34c24700 1 --2- 172.21.15.116:0/1624090555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c301551b0 0x7f6c300a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c34c24700 1 -- 172.21.15.116:0/1624090555 >> v2:172.21.15.173:3300/0 conn(0x7f6c300bd810 msgr2=0x7f6c300be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:30.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c34c24700 1 --2- 172.21.15.116:0/1624090555 >> v2:172.21.15.173:3300/0 conn(0x7f6c300bd810 0x7f6c300be340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c34c24700 1 -- 172.21.15.116:0/1624090555 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c30155a20 con 0x7f6c30154dd0 2020-11-20T12:18:30.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.284+0000 7f6c34c24700 1 --2- 172.21.15.116:0/1624090555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c30154dd0 0x7f6c30156160 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f6c30061310 tx=0x7f6c24006eb0).ready entity=mon.0 client_cookie=6a7a0979ba219f15 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:30.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c2f7fe700 1 -- 172.21.15.116:0/1624090555 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24009720 con 0x7f6c30154dd0 2020-11-20T12:18:30.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c2f7fe700 1 -- 172.21.15.116:0/1624090555 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c24009880 con 0x7f6c30154dd0 2020-11-20T12:18:30.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c2f7fe700 1 -- 172.21.15.116:0/1624090555 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24009c00 con 0x7f6c30154dd0 2020-11-20T12:18:30.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c36e88700 1 -- 172.21.15.116:0/1624090555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c30154dd0 msgr2=0x7f6c30156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c36e88700 1 --2- 172.21.15.116:0/1624090555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c30154dd0 0x7f6c30156160 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f6c30061310 tx=0x7f6c24006eb0).stop 2020-11-20T12:18:30.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c36e88700 1 -- 172.21.15.116:0/1624090555 shutdown_connections 2020-11-20T12:18:30.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c36e88700 1 --2- 172.21.15.116:0/1624090555 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c301551b0 0x7f6c300a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c36e88700 1 --2- 172.21.15.116:0/1624090555 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c30154dd0 0x7f6c30156160 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c36e88700 1 --2- 172.21.15.116:0/1624090555 >> v2:172.21.15.173:3300/0 conn(0x7f6c300bd810 0x7f6c300be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c36e88700 1 -- 172.21.15.116:0/1624090555 >> 172.21.15.116:0/1624090555 conn(0x7f6c3005ba80 msgr2=0x7f6c300c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:30.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c36e88700 1 -- 172.21.15.116:0/1624090555 shutdown_connections 2020-11-20T12:18:30.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.288+0000 7f6c36e88700 1 -- 172.21.15.116:0/1624090555 wait complete. 2020-11-20T12:18:30.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c36e88700 1 Processor -- start 2020-11-20T12:18:30.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c36e88700 1 -- start start 2020-11-20T12:18:30.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c36e88700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c300bd810 0x7f6c30154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c36e88700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c301551b0 0x7f6c30143f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c36e88700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c301613c0 0x7f6c30161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c36e88700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c300c32a0 con 0x7f6c300bd810 2020-11-20T12:18:30.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c36e88700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3009f610 con 0x7f6c301551b0 2020-11-20T12:18:30.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c36e88700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3009f890 con 0x7f6c301613c0 2020-11-20T12:18:30.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c34c24700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c300bd810 0x7f6c30154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c34c24700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c300bd810 0x7f6c30154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46830/0 (socket says 172.21.15.116:46830) 2020-11-20T12:18:30.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c34c24700 1 -- 172.21.15.116:0/3234563119 learned_addr learned my addr 172.21.15.116:0/3234563119 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:30.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c2ffff700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c301551b0 0x7f6c30143f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.292+0000 7f6c35425700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c301613c0 0x7f6c30161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c34c24700 1 -- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c301613c0 msgr2=0x7f6c30161840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c34c24700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c301613c0 0x7f6c30161840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c34c24700 1 -- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c301551b0 msgr2=0x7f6c30143f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c34c24700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c301551b0 0x7f6c30143f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c34c24700 1 -- 172.21.15.116:0/3234563119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c3009fb30 con 0x7f6c300bd810 2020-11-20T12:18:30.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c35425700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c301613c0 0x7f6c30161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:30.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c34c24700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c300bd810 0x7f6c30154110 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f6c2400b1b0 tx=0x7f6c24015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:30.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c2cff9700 1 -- 172.21.15.116:0/3234563119 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24011c20 con 0x7f6c300bd810 2020-11-20T12:18:30.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c2cff9700 1 -- 172.21.15.116:0/3234563119 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c24011d80 con 0x7f6c300bd810 2020-11-20T12:18:30.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c2cff9700 1 -- 172.21.15.116:0/3234563119 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2401e670 con 0x7f6c300bd810 2020-11-20T12:18:30.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c36e88700 1 -- 172.21.15.116:0/3234563119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c30151a50 con 0x7f6c300bd810 2020-11-20T12:18:30.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c36e88700 1 -- 172.21.15.116:0/3234563119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c30144680 con 0x7f6c300bd810 2020-11-20T12:18:30.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.296+0000 7f6c36e88700 1 -- 172.21.15.116:0/3234563119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3005d2a0 con 0x7f6c300bd810 2020-11-20T12:18:30.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.300+0000 7f6c2cff9700 1 -- 172.21.15.116:0/3234563119 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6c24025030 con 0x7f6c300bd810 2020-11-20T12:18:30.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.300+0000 7f6c2cff9700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6c080557c0 0x7f6c08057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.300+0000 7f6c2cff9700 1 -- 172.21.15.116:0/3234563119 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6c2406f9e0 con 0x7f6c300bd810 2020-11-20T12:18:30.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.300+0000 7f6c2ffff700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6c080557c0 0x7f6c08057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.300+0000 7f6c2ffff700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6c080557c0 0x7f6c08057c80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f6c300a9ea0 tx=0x7f6c1c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:30.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.304+0000 7f6c2cff9700 1 -- 172.21.15.116:0/3234563119 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6c24045930 con 0x7f6c300bd810 2020-11-20T12:18:30.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.524+0000 7f6c36e88700 1 -- 172.21.15.116:0/3234563119 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f6c30144680 con 0x7f6c300bd810 2020-11-20T12:18:30.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.532+0000 7f6c2cff9700 1 -- 172.21.15.116:0/3234563119 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7f6c30144680 con 0x7f6c300bd810 2020-11-20T12:18:30.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.532+0000 7f6c36e88700 1 -- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6c080557c0 msgr2=0x7f6c08057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.532+0000 7f6c36e88700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6c080557c0 0x7f6c08057c80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7f6c300a9ea0 tx=0x7f6c1c006df0).stop 2020-11-20T12:18:30.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.532+0000 7f6c36e88700 1 -- 172.21.15.116:0/3234563119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c300bd810 msgr2=0x7f6c30154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.532+0000 7f6c36e88700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c300bd810 0x7f6c30154110 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f6c2400b1b0 tx=0x7f6c24015000).stop 2020-11-20T12:18:30.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.532+0000 7f6c36e88700 1 -- 172.21.15.116:0/3234563119 shutdown_connections 2020-11-20T12:18:30.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.532+0000 7f6c36e88700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6c080557c0 0x7f6c08057c80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.532+0000 7f6c36e88700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6c301613c0 0x7f6c30161840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.532+0000 7f6c36e88700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6c300bd810 0x7f6c30154110 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.532+0000 7f6c36e88700 1 --2- 172.21.15.116:0/3234563119 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6c301551b0 0x7f6c30143f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.532+0000 7f6c36e88700 1 -- 172.21.15.116:0/3234563119 >> 172.21.15.116:0/3234563119 conn(0x7f6c3005ba80 msgr2=0x7f6c300a7a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:30.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.536+0000 7f6c36e88700 1 -- 172.21.15.116:0/3234563119 shutdown_connections 2020-11-20T12:18:30.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.536+0000 7f6c36e88700 1 -- 172.21.15.116:0/3234563119 wait complete. 2020-11-20T12:18:30.536 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2020-11-20T12:18:30.558 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:18:30.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58f59e700 1 Processor -- start 2020-11-20T12:18:30.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58f59e700 1 -- start start 2020-11-20T12:18:30.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58f59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc59013b380 0x7fc5900acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58f59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc5900ad220 0x7fc5900a8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58f59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5900ada00 con 0x7fc59013afa0 2020-11-20T12:18:30.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58f59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5900adca0 con 0x7fc5900ad220 2020-11-20T12:18:30.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58f59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5900adf40 con 0x7fc59013b380 2020-11-20T12:18:30.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58ed9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc5900ad220 0x7fc5900a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58ed9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc5900ad220 0x7fc5900a8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46842/0 (socket says 172.21.15.116:46842) 2020-11-20T12:18:30.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58ed9d700 1 -- 172.21.15.116:0/3678234991 learned_addr learned my addr 172.21.15.116:0/3678234991 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:30.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58dd9b700 1 --2- 172.21.15.116:0/3678234991 >> v2:172.21.15.173:3300/0 conn(0x7fc59013b380 0x7fc5900acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58ed9d700 1 -- 172.21.15.116:0/3678234991 >> v2:172.21.15.173:3300/0 conn(0x7fc59013b380 msgr2=0x7fc5900acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:30.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58ed9d700 1 --2- 172.21.15.116:0/3678234991 >> v2:172.21.15.173:3300/0 conn(0x7fc59013b380 0x7fc5900acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58ed9d700 1 -- 172.21.15.116:0/3678234991 >> v1:172.21.15.173:6789/0 conn(0x7fc59013afa0 legacy=0x7fc5900aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:30.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.736+0000 7fc58ed9d700 1 -- 172.21.15.116:0/3678234991 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5900ae1e0 con 0x7fc5900ad220 2020-11-20T12:18:30.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58ed9d700 1 --2- 172.21.15.116:0/3678234991 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5900ad220 0x7fc5900a8790 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7fc5900625b0 tx=0x7fc588009290).ready entity=mon.0 client_cookie=bea613ea18359209 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:30.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58d59a700 1 -- 172.21.15.116:0/3678234991 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58800bb80 con 0x7fc5900ad220 2020-11-20T12:18:30.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58d59a700 1 -- 172.21.15.116:0/3678234991 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58800bce0 con 0x7fc5900ad220 2020-11-20T12:18:30.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58d59a700 1 -- 172.21.15.116:0/3678234991 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588011610 con 0x7fc5900ad220 2020-11-20T12:18:30.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58f59e700 1 -- 172.21.15.116:0/3678234991 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5900ad220 msgr2=0x7fc5900a8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58f59e700 1 --2- 172.21.15.116:0/3678234991 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5900ad220 0x7fc5900a8790 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7fc5900625b0 tx=0x7fc588009290).stop 2020-11-20T12:18:30.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58f59e700 1 -- 172.21.15.116:0/3678234991 shutdown_connections 2020-11-20T12:18:30.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58f59e700 1 --2- 172.21.15.116:0/3678234991 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5900ad220 0x7fc5900a8790 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58f59e700 1 --2- 172.21.15.116:0/3678234991 >> v2:172.21.15.173:3300/0 conn(0x7fc59013b380 0x7fc5900acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58f59e700 1 -- 172.21.15.116:0/3678234991 >> 172.21.15.116:0/3678234991 conn(0x7fc5900678a0 msgr2=0x7fc590067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:30.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58f59e700 1 -- 172.21.15.116:0/3678234991 shutdown_connections 2020-11-20T12:18:30.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.740+0000 7fc58f59e700 1 -- 172.21.15.116:0/3678234991 wait complete. 2020-11-20T12:18:30.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58f59e700 1 Processor -- start 2020-11-20T12:18:30.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58f59e700 1 -- start start 2020-11-20T12:18:30.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58f59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5900ad220 0x7fc5900a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc59013afa0 0x7fc590164b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58f59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc59013b380 0x7fc5900a6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58f59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5900a5450 con 0x7fc5900ad220 2020-11-20T12:18:30.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58f59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc590164ff0 con 0x7fc59013b380 2020-11-20T12:18:30.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5900a56b0 con 0x7fc59013afa0 2020-11-20T12:18:30.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5900ad220 0x7fc5900a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5900ad220 0x7fc5900a4db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46848/0 (socket says 172.21.15.116:46848) 2020-11-20T12:18:30.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58e59c700 1 -- 172.21.15.116:0/465540478 learned_addr learned my addr 172.21.15.116:0/465540478 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:30.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58ed9d700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc59013b380 0x7fc5900a6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58dd9b700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc59013afa0 0x7fc590164b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58e59c700 1 -- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc59013afa0 msgr2=0x7fc590164b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58e59c700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc59013afa0 0x7fc590164b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58e59c700 1 -- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc59013b380 msgr2=0x7fc5900a6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58e59c700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc59013b380 0x7fc5900a6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58e59c700 1 -- 172.21.15.116:0/465540478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5900a5950 con 0x7fc5900ad220 2020-11-20T12:18:30.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58dd9b700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc59013afa0 0x7fc590164b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:30.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58e59c700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5900ad220 0x7fc5900a4db0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fc58000c730 tx=0x7fc580007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:30.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc57e7fc700 1 -- 172.21.15.116:0/465540478 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58000e560 con 0x7fc5900ad220 2020-11-20T12:18:30.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc57e7fc700 1 -- 172.21.15.116:0/465540478 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58000e6e0 con 0x7fc5900ad220 2020-11-20T12:18:30.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58f59e700 1 -- 172.21.15.116:0/465540478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5900a5bf0 con 0x7fc5900ad220 2020-11-20T12:18:30.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc57e7fc700 1 -- 172.21.15.116:0/465540478 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58000eaa0 con 0x7fc5900ad220 2020-11-20T12:18:30.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.744+0000 7fc58f59e700 1 -- 172.21.15.116:0/465540478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5900a6580 con 0x7fc5900ad220 2020-11-20T12:18:30.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.748+0000 7fc58f59e700 1 -- 172.21.15.116:0/465540478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59005cf30 con 0x7fc5900ad220 2020-11-20T12:18:30.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.748+0000 7fc57e7fc700 1 -- 172.21.15.116:0/465540478 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc580019030 con 0x7fc5900ad220 2020-11-20T12:18:30.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.748+0000 7fc57e7fc700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5640557f0 0x7fc564057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:30.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.748+0000 7fc57e7fc700 1 -- 172.21.15.116:0/465540478 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc58006dbd0 con 0x7fc5900ad220 2020-11-20T12:18:30.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.752+0000 7fc58dd9b700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5640557f0 0x7fc564057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:30.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.752+0000 7fc58dd9b700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5640557f0 0x7fc564057cb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fc5900c7850 tx=0x7fc584008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:30.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.756+0000 7fc57e7fc700 1 -- 172.21.15.116:0/465540478 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc580043a10 con 0x7fc5900ad220 2020-11-20T12:18:30.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc58f59e700 1 -- 172.21.15.116:0/465540478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc590060170 con 0x7fc5900ad220 2020-11-20T12:18:30.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc57e7fc700 1 -- 172.21.15.116:0/465540478 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2850 (secure 0 0 0) 0x7fc580047030 con 0x7fc5900ad220 2020-11-20T12:18:30.963 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:30.963 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/785878283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":785878283},{"type":"v1","addr":"172.21.15.173:6837","nonce":785878283}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5952,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1568769426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1568769426},{"type":"v1","addr":"172.21.15.173:6836","nonce":1568769426}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3552490451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3552490451},{"type":"v1","addr":"172.21.15.116:6833","nonce":3552490451}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:18:27.137821+0000","modified":"2020-11-20T12:18:29.161493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3370673707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3370673707},{"type":"v1","addr":"172.21.15.116:6835","nonce":3370673707}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-20T12:18:30.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc58f59e700 1 -- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5640557f0 msgr2=0x7fc564057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc58f59e700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5640557f0 0x7fc564057cb0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7fc5900c7850 tx=0x7fc584008040).stop 2020-11-20T12:18:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc58f59e700 1 -- 172.21.15.116:0/465540478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5900ad220 msgr2=0x7fc5900a4db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:30.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc58f59e700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5900ad220 0x7fc5900a4db0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fc58000c730 tx=0x7fc580007c30).stop 2020-11-20T12:18:30.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc58f59e700 1 -- 172.21.15.116:0/465540478 shutdown_connections 2020-11-20T12:18:30.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc58f59e700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5640557f0 0x7fc564057cb0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc58f59e700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc59013b380 0x7fc5900a6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc58f59e700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5900ad220 0x7fc5900a4db0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc58f59e700 1 --2- 172.21.15.116:0/465540478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc59013afa0 0x7fc590164b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:30.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.964+0000 7fc58f59e700 1 -- 172.21.15.116:0/465540478 >> 172.21.15.116:0/465540478 conn(0x7fc5900678a0 msgr2=0x7fc5900c82a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:30.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.968+0000 7fc58f59e700 1 -- 172.21.15.116:0/465540478 shutdown_connections 2020-11-20T12:18:30.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:30.968+0000 7fc58f59e700 1 -- 172.21.15.116:0/465540478 wait complete. 2020-11-20T12:18:30.968 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 28 2020-11-20T12:18:31.392 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:18:27.137821+0000', 'modified': '2020-11-20T12:18:29.161493+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5970': {'gid': 5970, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/3370673707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3370673707}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3370673707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:18:31.393 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:18:31.394 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:18:31.474 INFO:tasks.ceph.mds.c.smithi116.stderr:2020-11-20T12:18:31.476+0000 7fd4125d6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:18:31.593 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":5970,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.4344535860000001,"mdsmap_epoch":28,"osdmap_epoch":56,"osdmap_epoch_barrier":56,"uptime":10.901379377} 2020-11-20T12:18:31.593 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 5970, "mdsmap_epoch": 28, "osdmap_epoch": 56, "osdmap_epoch_barrier": 56, "rank_uptime": 4.434453586, "state": "up:active", "uptime": 10.901379377, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:18:31.594 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:18:31.594 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:18:31.595 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:18:31.595 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:18:31.595 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:18:31.596 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:18:31.596 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:18:31.597 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:18:31.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:31.598 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:18:31.620 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:18:31.621 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:18:31.622 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:18:31.622 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:31.623 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:18:31.623 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:31.624 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:31.624 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:31.624 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:31.626 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:31.626 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:31.626 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:31.627 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:18:31.627 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:31.627 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:18:31.628 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:31.628 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:18:31.628 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:31.629 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:18:31.629 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:31.629 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:18:31.629 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:31.629 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:18:31.630 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:31.630 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:18:31.630 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:31.630 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:18:31.631 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:31.631 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:18:31.631 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:18:31.632 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:18:31.632 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:31.632 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:18:31.632 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:18:31.633 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:18:31.633 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:31.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:31.634 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:18:31.681 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:18:31.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:31.687 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:18:31.748 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:18:31.748 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:18:31.749 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:18:31.749 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:18:31.749 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:18:31.749 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:18:31.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:31.751 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:18:31.751 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:18:31.752 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:18:31.752 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:18:31.753 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:18:31.753 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:18:31.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:31.858 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:18:31.873 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:18:31.873 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:18:31.874 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:18:31.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:31.875 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2020-11-20T12:18:31.902 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:32.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.128+0000 7f085d39d700 1 Processor -- start 2020-11-20T12:18:32.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.128+0000 7f085d39d700 1 -- start start 2020-11-20T12:18:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.128+0000 7f085d39d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0858061760 0x7f08580c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f085d39d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0858060840 con 0x7f0858151600 2020-11-20T12:18:32.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f085d39d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0858060ae0 con 0x7f08581519e0 2020-11-20T12:18:32.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f085d39d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0858060d80 con 0x7f0858061760 2020-11-20T12:18:32.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f084e7fc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0858061760 0x7f08580c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f084e7fc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0858061760 0x7f08580c4d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46854/0 (socket says 172.21.15.116:46854) 2020-11-20T12:18:32.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f084e7fc700 1 -- 172.21.15.116:0/4221053915 learned_addr learned my addr 172.21.15.116:0/4221053915 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:32.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f0856ffd700 1 --1- 172.21.15.116:0/4221053915 >> v1:172.21.15.116:6789/0 conn(0x7f0858151600 0x7f08580c44f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56286/0 (socket says 172.21.15.116:56286) 2020-11-20T12:18:32.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f084e7fc700 1 -- 172.21.15.116:0/4221053915 >> v1:172.21.15.173:6789/0 conn(0x7f08581519e0 legacy=0x7f08580becc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f084e7fc700 1 -- 172.21.15.116:0/4221053915 >> v1:172.21.15.116:6789/0 conn(0x7f0858151600 legacy=0x7f08580c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f084e7fc700 1 -- 172.21.15.116:0/4221053915 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0858061020 con 0x7f0858061760 2020-11-20T12:18:32.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f084e7fc700 1 --2- 172.21.15.116:0/4221053915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0858061760 0x7f08580c4d70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f0840006fd0 tx=0x7f0840006eb0).ready entity=mon.0 client_cookie=9428da39bb7d2649 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f08567fc700 1 -- 172.21.15.116:0/4221053915 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08400098d0 con 0x7f0858061760 2020-11-20T12:18:32.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f08567fc700 1 -- 172.21.15.116:0/4221053915 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0840009a30 con 0x7f0858061760 2020-11-20T12:18:32.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f08567fc700 1 -- 172.21.15.116:0/4221053915 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0840009db0 con 0x7f0858061760 2020-11-20T12:18:32.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f085d39d700 1 -- 172.21.15.116:0/4221053915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0858061760 msgr2=0x7f08580c4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f085d39d700 1 --2- 172.21.15.116:0/4221053915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0858061760 0x7f08580c4d70 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f0840006fd0 tx=0x7f0840006eb0).stop 2020-11-20T12:18:32.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f085d39d700 1 -- 172.21.15.116:0/4221053915 shutdown_connections 2020-11-20T12:18:32.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f085d39d700 1 --2- 172.21.15.116:0/4221053915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0858061760 0x7f08580c4d70 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f085d39d700 1 -- 172.21.15.116:0/4221053915 >> 172.21.15.116:0/4221053915 conn(0x7f08580ca830 msgr2=0x7f08580c6800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:32.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f085d39d700 1 -- 172.21.15.116:0/4221053915 shutdown_connections 2020-11-20T12:18:32.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.132+0000 7f085d39d700 1 -- 172.21.15.116:0/4221053915 wait complete. 2020-11-20T12:18:32.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.136+0000 7f085d39d700 1 Processor -- start 2020-11-20T12:18:32.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.136+0000 7f085d39d700 1 -- start start 2020-11-20T12:18:32.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.136+0000 7f085d39d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0858151600 0x7f085814ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.136+0000 7f085d39d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08581519e0 0x7f085814e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.136+0000 7f085d39d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f085814cf70 0x7f085814ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.136+0000 7f085d39d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08580aa980 con 0x7f085814cf70 2020-11-20T12:18:32.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.136+0000 7f085d39d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f085814f060 con 0x7f0858151600 2020-11-20T12:18:32.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.136+0000 7f085d39d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f085814f300 con 0x7f08581519e0 2020-11-20T12:18:32.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.136+0000 7f0857fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f085814cf70 0x7f085814ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f0857fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f085814cf70 0x7f085814ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46862/0 (socket says 172.21.15.116:46862) 2020-11-20T12:18:32.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f0857fff700 1 -- 172.21.15.116:0/1866516571 learned_addr learned my addr 172.21.15.116:0/1866516571 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:32.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f0856ffd700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0858151600 0x7f085814ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f084e7fc700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08581519e0 0x7f085814e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f0857fff700 1 -- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08581519e0 msgr2=0x7f085814e520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f0857fff700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08581519e0 0x7f085814e520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f0857fff700 1 -- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0858151600 msgr2=0x7f085814ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f0857fff700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0858151600 0x7f085814ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f0857fff700 1 -- 172.21.15.116:0/1866516571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085814f5a0 con 0x7f085814cf70 2020-11-20T12:18:32.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f0856ffd700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0858151600 0x7f085814ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:32.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f084e7fc700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08581519e0 0x7f085814e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:32.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f0857fff700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f085814cf70 0x7f085814ea10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f085000ca60 tx=0x7f0850007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f084f7fe700 1 -- 172.21.15.116:0/1866516571 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085000a8c0 con 0x7f085814cf70 2020-11-20T12:18:32.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f084f7fe700 1 -- 172.21.15.116:0/1866516571 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085000aa20 con 0x7f085814cf70 2020-11-20T12:18:32.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f084f7fe700 1 -- 172.21.15.116:0/1866516571 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085000ada0 con 0x7f085814cf70 2020-11-20T12:18:32.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f085d39d700 1 -- 172.21.15.116:0/1866516571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08580aabb0 con 0x7f085814cf70 2020-11-20T12:18:32.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f085d39d700 1 -- 172.21.15.116:0/1866516571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08581388e0 con 0x7f085814cf70 2020-11-20T12:18:32.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.140+0000 7f085d39d700 1 -- 172.21.15.116:0/1866516571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08580aad40 con 0x7f085814cf70 2020-11-20T12:18:32.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.144+0000 7f084f7fe700 1 -- 172.21.15.116:0/1866516571 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0850019030 con 0x7f085814cf70 2020-11-20T12:18:32.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.144+0000 7f084f7fe700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08340557f0 0x7f0834057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.144+0000 7f084f7fe700 1 -- 172.21.15.116:0/1866516571 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f085006e0a0 con 0x7f085814cf70 2020-11-20T12:18:32.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.144+0000 7f0856ffd700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08340557f0 0x7f0834057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.144+0000 7f0856ffd700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08340557f0 0x7f0834057cb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f0848007580 tx=0x7f0848000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.148+0000 7f084f7fe700 1 -- 172.21.15.116:0/1866516571 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0850043ee0 con 0x7f085814cf70 2020-11-20T12:18:32.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.312+0000 7f085d39d700 1 -- 172.21.15.116:0/1866516571 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f085805d2a0 con 0x7f085814cf70 2020-11-20T12:18:32.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.312+0000 7f084f7fe700 1 -- 172.21.15.116:0/1866516571 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f0850025200 con 0x7f085814cf70 2020-11-20T12:18:32.312 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:32.313 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":56,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:18:28.142253+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:18:25.558441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-20T12:18:26.151785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:32.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 -- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08340557f0 msgr2=0x7f0834057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08340557f0 0x7f0834057cb0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f0848007580 tx=0x7f0848000b80).stop 2020-11-20T12:18:32.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 -- 172.21.15.116:0/1866516571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f085814cf70 msgr2=0x7f085814ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f085814cf70 0x7f085814ea10 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f085000ca60 tx=0x7f0850007f40).stop 2020-11-20T12:18:32.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 -- 172.21.15.116:0/1866516571 shutdown_connections 2020-11-20T12:18:32.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08340557f0 0x7f0834057cb0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f085814cf70 0x7f085814ea10 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0858151600 0x7f085814ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 --2- 172.21.15.116:0/1866516571 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08581519e0 0x7f085814e520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 -- 172.21.15.116:0/1866516571 >> 172.21.15.116:0/1866516571 conn(0x7f08580ca830 msgr2=0x7f085814c100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:32.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 -- 172.21.15.116:0/1866516571 shutdown_connections 2020-11-20T12:18:32.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.316+0000 7f085d39d700 1 -- 172.21.15.116:0/1866516571 wait complete. 2020-11-20T12:18:32.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:18:25.558441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:18:26.151785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:18:32.339 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:18:32.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.520+0000 7f9580e6f700 1 Processor -- start 2020-11-20T12:18:32.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.520+0000 7f9580e6f700 1 -- start start 2020-11-20T12:18:32.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.524+0000 7f9580e6f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f957c063100 0x7f957c0bff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.524+0000 7f9580e6f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f957c0a66e0 0x7f957c0a6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.524+0000 7f9580e6f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f957c0c0400 con 0x7f957c14e590 2020-11-20T12:18:32.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.524+0000 7f9580e6f700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f957c0c0690 con 0x7f957c0a66e0 2020-11-20T12:18:32.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.524+0000 7f9580e6f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957c0c0930 con 0x7f957c063100 2020-11-20T12:18:32.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957b7fe700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f957c14e590 0x7f957c14f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:52530/0 (socket says 172.21.15.116:52530) 2020-11-20T12:18:32.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957b7fe700 1 -- 172.21.15.116:0/1431133950 learned_addr learned my addr 172.21.15.116:0/1431133950 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:32.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957affd700 1 --2- 172.21.15.116:0/1431133950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f957c063100 0x7f957c0bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957bfff700 1 --2- 172.21.15.116:0/1431133950 >> v2:172.21.15.173:3300/0 conn(0x7f957c0a66e0 0x7f957c0a6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957a7fc700 1 -- 172.21.15.116:0/1431133950 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2088450640 0 0) 0x7f957c0c0400 con 0x7f957c14e590 2020-11-20T12:18:32.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957a7fc700 1 -- 172.21.15.116:0/1431133950 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9568003240 con 0x7f957c14e590 2020-11-20T12:18:32.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957affd700 1 -- 172.21.15.116:0/1431133950 >> v2:172.21.15.173:3300/0 conn(0x7f957c0a66e0 msgr2=0x7f957c0a6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:32.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957affd700 1 --2- 172.21.15.116:0/1431133950 >> v2:172.21.15.173:3300/0 conn(0x7f957c0a66e0 0x7f957c0a6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957affd700 1 -- 172.21.15.116:0/1431133950 >> v1:172.21.15.173:6789/0 conn(0x7f957c14e590 legacy=0x7f957c14f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957affd700 1 -- 172.21.15.116:0/1431133950 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c0c0bd0 con 0x7f957c063100 2020-11-20T12:18:32.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957affd700 1 --2- 172.21.15.116:0/1431133950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f957c063100 0x7f957c0bff10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f957000b9a0 tx=0x7f9570006df0).ready entity=mon.2 client_cookie=7e44717885db4018 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957a7fc700 1 -- 172.21.15.116:0/1431133950 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570009680 con 0x7f957c063100 2020-11-20T12:18:32.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957a7fc700 1 -- 172.21.15.116:0/1431133950 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95700097e0 con 0x7f957c063100 2020-11-20T12:18:32.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f957a7fc700 1 -- 172.21.15.116:0/1431133950 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9570009b60 con 0x7f957c063100 2020-11-20T12:18:32.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f9580e6f700 1 -- 172.21.15.116:0/1431133950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f957c063100 msgr2=0x7f957c0bff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f9580e6f700 1 --2- 172.21.15.116:0/1431133950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f957c063100 0x7f957c0bff10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f957000b9a0 tx=0x7f9570006df0).stop 2020-11-20T12:18:32.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f9580e6f700 1 -- 172.21.15.116:0/1431133950 shutdown_connections 2020-11-20T12:18:32.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f9580e6f700 1 --2- 172.21.15.116:0/1431133950 >> v2:172.21.15.173:3300/0 conn(0x7f957c0a66e0 0x7f957c0a6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f9580e6f700 1 --2- 172.21.15.116:0/1431133950 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f957c063100 0x7f957c0bff10 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.528+0000 7f9580e6f700 1 -- 172.21.15.116:0/1431133950 >> 172.21.15.116:0/1431133950 conn(0x7f957c068350 msgr2=0x7f957c0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:32.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.532+0000 7f9580e6f700 1 -- 172.21.15.116:0/1431133950 shutdown_connections 2020-11-20T12:18:32.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.532+0000 7f9580e6f700 1 -- 172.21.15.116:0/1431133950 wait complete. 2020-11-20T12:18:32.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.532+0000 7f9580e6f700 1 Processor -- start 2020-11-20T12:18:32.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.532+0000 7f9580e6f700 1 -- start start 2020-11-20T12:18:32.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.532+0000 7f9580e6f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f957c063100 0x7f957c0a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.532+0000 7f9580e6f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f957c0a66e0 0x7f957c161f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.532+0000 7f9580e6f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c14e590 0x7f957c0c2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.532+0000 7f9580e6f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c14a850 con 0x7f957c063100 2020-11-20T12:18:32.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.532+0000 7f9580e6f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957c0a7ae0 con 0x7f957c14e590 2020-11-20T12:18:32.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.532+0000 7f9580e6f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957c0bc340 con 0x7f957c0a66e0 2020-11-20T12:18:32.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f957b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f957c063100 0x7f957c0a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f957affd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f957c0a66e0 0x7f957c161f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f957b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f957c063100 0x7f957c0a5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46872/0 (socket says 172.21.15.116:46872) 2020-11-20T12:18:32.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f957b7fe700 1 -- 172.21.15.116:0/665642207 learned_addr learned my addr 172.21.15.116:0/665642207 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:32.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f957bfff700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c14e590 0x7f957c0c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f957b7fe700 1 -- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f957c0a66e0 msgr2=0x7f957c161f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f957b7fe700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f957c0a66e0 0x7f957c161f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f957b7fe700 1 -- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c14e590 msgr2=0x7f957c0c2210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f957b7fe700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c14e590 0x7f957c0c2210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f957b7fe700 1 -- 172.21.15.116:0/665642207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f957c0bc0a0 con 0x7f957c063100 2020-11-20T12:18:32.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f957b7fe700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f957c063100 0x7f957c0a5f00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f956c00c6f0 tx=0x7f956c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f955b7fe700 1 -- 172.21.15.116:0/665642207 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c0072e0 con 0x7f957c063100 2020-11-20T12:18:32.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f955b7fe700 1 -- 172.21.15.116:0/665642207 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f956c007440 con 0x7f957c063100 2020-11-20T12:18:32.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f9580e6f700 1 -- 172.21.15.116:0/665642207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f957c161550 con 0x7f957c063100 2020-11-20T12:18:32.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f955b7fe700 1 -- 172.21.15.116:0/665642207 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f956c005800 con 0x7f957c063100 2020-11-20T12:18:32.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.536+0000 7f9580e6f700 1 -- 172.21.15.116:0/665642207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f957c161a90 con 0x7f957c063100 2020-11-20T12:18:32.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.540+0000 7f9580e6f700 1 -- 172.21.15.116:0/665642207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f957c05cf30 con 0x7f957c063100 2020-11-20T12:18:32.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.540+0000 7f955b7fe700 1 -- 172.21.15.116:0/665642207 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f956c01a060 con 0x7f957c063100 2020-11-20T12:18:32.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.540+0000 7f955b7fe700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95540557c0 0x7f9554057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.540+0000 7f955b7fe700 1 -- 172.21.15.116:0/665642207 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f956c06dfc0 con 0x7f957c063100 2020-11-20T12:18:32.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.540+0000 7f957affd700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95540557c0 0x7f9554057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.544+0000 7f957affd700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95540557c0 0x7f9554057c80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f957000b150 tx=0x7f9570015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.544+0000 7f955b7fe700 1 -- 172.21.15.116:0/665642207 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f956c043d50 con 0x7f957c063100 2020-11-20T12:18:32.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.704+0000 7f9580e6f700 1 -- 172.21.15.116:0/665642207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f957c066220 con 0x7f957c063100 2020-11-20T12:18:32.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.704+0000 7f955b7fe700 1 -- 172.21.15.116:0/665642207 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v56) v1 ==== 110+0+10 (secure 0 0 0) 0x7f956c018030 con 0x7f957c063100 2020-11-20T12:18:32.703 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:18:32.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 -- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95540557c0 msgr2=0x7f9554057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95540557c0 0x7f9554057c80 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f957000b150 tx=0x7f9570015000).stop 2020-11-20T12:18:32.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 -- 172.21.15.116:0/665642207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f957c063100 msgr2=0x7f957c0a5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f957c063100 0x7f957c0a5f00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f956c00c6f0 tx=0x7f956c007ad0).stop 2020-11-20T12:18:32.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 -- 172.21.15.116:0/665642207 shutdown_connections 2020-11-20T12:18:32.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f95540557c0 0x7f9554057c80 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f957c14e590 0x7f957c0c2210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f957c063100 0x7f957c0a5f00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 --2- 172.21.15.116:0/665642207 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f957c0a66e0 0x7f957c161f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 -- 172.21.15.116:0/665642207 >> 172.21.15.116:0/665642207 conn(0x7f957c068350 msgr2=0x7f957c14d490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:32.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 -- 172.21.15.116:0/665642207 shutdown_connections 2020-11-20T12:18:32.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.708+0000 7f9580e6f700 1 -- 172.21.15.116:0/665642207 wait complete. 2020-11-20T12:18:32.733 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:18:32.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.904+0000 7f7dcefc1700 1 Processor -- start 2020-11-20T12:18:32.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.904+0000 7f7dcefc1700 1 -- start start 2020-11-20T12:18:32.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.908+0000 7f7dcefc1700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7dc80aa7d0 0x7f7dc80a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.908+0000 7f7dcefc1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc80ad540 0x7f7dc80ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.908+0000 7f7dcefc1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7dc80ace40 con 0x7f7dc80aabb0 2020-11-20T12:18:32.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.908+0000 7f7dcefc1700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7dc81526c0 con 0x7f7dc80aa7d0 2020-11-20T12:18:32.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.908+0000 7f7dcefc1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8152960 con 0x7f7dc80ad540 2020-11-20T12:18:32.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.908+0000 7f7dcd7be700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7dc80aabb0 0x7f7dc8151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56316/0 (socket says 172.21.15.116:56316) 2020-11-20T12:18:32.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.908+0000 7f7dcd7be700 1 -- 172.21.15.116:0/3662860005 learned_addr learned my addr 172.21.15.116:0/3662860005 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:32.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dcdfbf700 1 --2- 172.21.15.116:0/3662860005 >> v2:172.21.15.173:3300/0 conn(0x7f7dc80aa7d0 0x7f7dc80a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.908+0000 7f7dce7c0700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc80ad540 0x7f7dc80ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058292991 0 0) 0x7f7dc80ace40 con 0x7f7dc80aabb0 2020-11-20T12:18:32.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7db0003240 con 0x7f7dc80aabb0 2020-11-20T12:18:32.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1629792763 0 0) 0x7f7db0003240 con 0x7f7dc80aabb0 2020-11-20T12:18:32.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc80ad540 msgr2=0x7f7dc80ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:32.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc80ad540 0x7f7dc80ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 >> v2:172.21.15.173:3300/0 conn(0x7f7dc80aa7d0 msgr2=0x7f7dc80a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:32.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 --2- 172.21.15.116:0/3662860005 >> v2:172.21.15.173:3300/0 conn(0x7f7dc80aa7d0 0x7f7dc80a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc8152c00 con 0x7f7dc80aabb0 2020-11-20T12:18:32.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f7db80014e0 con 0x7f7dc80aabb0 2020-11-20T12:18:32.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 >> v1:172.21.15.116:6789/0 conn(0x7f7dc80aabb0 legacy=0x7f7dc8151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00037b0 0x7f7db0003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc80ad540 0x7f7db00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db00049d0 0x7f7db0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc81526c0 con 0x7f7db00049d0 2020-11-20T12:18:32.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc8152960 con 0x7f7db00037b0 2020-11-20T12:18:32.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0004340 con 0x7f7dc80ad540 2020-11-20T12:18:32.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dce7c0700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db00049d0 0x7f7db0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dcdfbf700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00037b0 0x7f7db0003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dcd7be700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc80ad540 0x7f7db00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dce7c0700 1 -- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc80ad540 msgr2=0x7f7db00044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:32.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dce7c0700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc80ad540 0x7f7db00044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dce7c0700 1 -- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00037b0 msgr2=0x7f7db0003bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:32.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dce7c0700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00037b0 0x7f7db0003bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dce7c0700 1 -- 172.21.15.116:0/3662860005 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db0003240 con 0x7f7db00049d0 2020-11-20T12:18:32.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dcd7be700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc80ad540 0x7f7db00044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:32.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.912+0000 7f7dce7c0700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db00049d0 0x7f7db0004e50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f7dc0009ea0 tx=0x7f7dc00091d0).ready entity=mon.0 client_cookie=5098c752fa2a7043 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00095c0 con 0x7f7db00049d0 2020-11-20T12:18:32.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc000ecf0 con 0x7f7db00049d0 2020-11-20T12:18:32.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dccfbd700 1 -- 172.21.15.116:0/3662860005 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc0012600 con 0x7f7db00049d0 2020-11-20T12:18:32.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dcefc1700 1 -- 172.21.15.116:0/3662860005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db00049d0 msgr2=0x7f7db0004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dcefc1700 1 --2- 172.21.15.116:0/3662860005 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7db00049d0 0x7f7db0004e50 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f7dc0009ea0 tx=0x7f7dc00091d0).stop 2020-11-20T12:18:32.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dce7c0700 1 -- 172.21.15.116:0/3662860005 reap_dead start 2020-11-20T12:18:32.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dcefc1700 1 -- 172.21.15.116:0/3662860005 shutdown_connections 2020-11-20T12:18:32.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dcefc1700 1 -- 172.21.15.116:0/3662860005 >> 172.21.15.116:0/3662860005 conn(0x7f7dc80678a0 msgr2=0x7f7dc8067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:32.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dcefc1700 1 -- 172.21.15.116:0/3662860005 shutdown_connections 2020-11-20T12:18:32.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dcefc1700 1 -- 172.21.15.116:0/3662860005 wait complete. 2020-11-20T12:18:32.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dcefc1700 1 Processor -- start 2020-11-20T12:18:32.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.916+0000 7f7dcefc1700 1 -- start start 2020-11-20T12:18:32.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dcefc1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00049d0 0x7f7dc8140190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dcefc1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc809f390 0x7f7dc8160d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dcefc1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc809f810 0x7f7dc8161210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dcefc1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc809fd10 con 0x7f7dc809f810 2020-11-20T12:18:32.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dcefc1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc809ffb0 con 0x7f7db00049d0 2020-11-20T12:18:32.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dcefc1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7dc80a0250 con 0x7f7dc809f390 2020-11-20T12:18:32.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dce7c0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc809f810 0x7f7dc8161210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dce7c0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc809f810 0x7f7dc8161210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46896/0 (socket says 172.21.15.116:46896) 2020-11-20T12:18:32.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dce7c0700 1 -- 172.21.15.116:0/356615759 learned_addr learned my addr 172.21.15.116:0/356615759 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:32.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dcdfbf700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00049d0 0x7f7dc8140190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dcd7be700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc809f390 0x7f7dc8160d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dce7c0700 1 -- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc809f390 msgr2=0x7f7dc8160d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dce7c0700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc809f390 0x7f7dc8160d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dce7c0700 1 -- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00049d0 msgr2=0x7f7dc8140190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:32.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dce7c0700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00049d0 0x7f7dc8140190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:32.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dce7c0700 1 -- 172.21.15.116:0/356615759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dc80a04f0 con 0x7f7dc809f810 2020-11-20T12:18:32.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dcd7be700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc809f390 0x7f7dc8160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:32.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dce7c0700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc809f810 0x7f7dc8161210 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f7dc0009e70 tx=0x7f7dc001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7db5ffb700 1 -- 172.21.15.116:0/356615759 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc00095c0 con 0x7f7dc809f810 2020-11-20T12:18:32.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7db5ffb700 1 -- 172.21.15.116:0/356615759 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dc000ee60 con 0x7f7dc809f810 2020-11-20T12:18:32.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dcefc1700 1 -- 172.21.15.116:0/356615759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7dc80a0790 con 0x7f7dc809f810 2020-11-20T12:18:32.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7db5ffb700 1 -- 172.21.15.116:0/356615759 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dc001b660 con 0x7f7dc809f810 2020-11-20T12:18:32.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.920+0000 7f7dcefc1700 1 -- 172.21.15.116:0/356615759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7dc8150820 con 0x7f7dc809f810 2020-11-20T12:18:32.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.924+0000 7f7dcefc1700 1 -- 172.21.15.116:0/356615759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7dc805cf30 con 0x7f7dc809f810 2020-11-20T12:18:32.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.924+0000 7f7db5ffb700 1 -- 172.21.15.116:0/356615759 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7dc002d070 con 0x7f7dc809f810 2020-11-20T12:18:32.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.924+0000 7f7db5ffb700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7da00557c0 0x7f7da0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:32.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.924+0000 7f7db5ffb700 1 -- 172.21.15.116:0/356615759 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7dc0070a60 con 0x7f7dc809f810 2020-11-20T12:18:32.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.924+0000 7f7dcdfbf700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7da00557c0 0x7f7da0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:32.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.928+0000 7f7dcdfbf700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7da00557c0 0x7f7da0057c80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f7dc4006d00 tx=0x7f7dc4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:32.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:32.932+0000 7f7db5ffb700 1 -- 172.21.15.116:0/356615759 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7dc0046900 con 0x7f7dc809f810 2020-11-20T12:18:33.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.092+0000 7f7dcefc1700 1 -- 172.21.15.116:0/356615759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7dc80601b0 con 0x7f7dc809f810 2020-11-20T12:18:33.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.092+0000 7f7db5ffb700 1 -- 172.21.15.116:0/356615759 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7dc0016070 con 0x7f7dc809f810 2020-11-20T12:18:33.091 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:33.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.092+0000 7f7dcefc1700 1 -- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7da00557c0 msgr2=0x7f7da0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.092+0000 7f7dcefc1700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7da00557c0 0x7f7da0057c80 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f7dc4006d00 tx=0x7f7dc4008040).stop 2020-11-20T12:18:33.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.092+0000 7f7dcefc1700 1 -- 172.21.15.116:0/356615759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc809f810 msgr2=0x7f7dc8161210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.096+0000 7f7dcefc1700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc809f810 0x7f7dc8161210 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f7dc0009e70 tx=0x7f7dc001a000).stop 2020-11-20T12:18:33.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.096+0000 7f7dcefc1700 1 -- 172.21.15.116:0/356615759 shutdown_connections 2020-11-20T12:18:33.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.096+0000 7f7dcefc1700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7da00557c0 0x7f7da0057c80 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.096+0000 7f7dcefc1700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7dc809f810 0x7f7dc8161210 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.096+0000 7f7dcefc1700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7db00049d0 0x7f7dc8140190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.096+0000 7f7dcefc1700 1 --2- 172.21.15.116:0/356615759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7dc809f390 0x7f7dc8160d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.096+0000 7f7dcefc1700 1 -- 172.21.15.116:0/356615759 >> 172.21.15.116:0/356615759 conn(0x7f7dc80678a0 msgr2=0x7f7dc81547a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:33.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.096+0000 7f7dcefc1700 1 -- 172.21.15.116:0/356615759 shutdown_connections 2020-11-20T12:18:33.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.096+0000 7f7dcefc1700 1 -- 172.21.15.116:0/356615759 wait complete. 2020-11-20T12:18:33.116 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:18:33.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.288+0000 7f9fbcbcc700 1 Processor -- start 2020-11-20T12:18:33.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.288+0000 7f9fbcbcc700 1 -- start start 2020-11-20T12:18:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.292+0000 7f9fbcbcc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9fb814f880 0x7f9fb80b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.292+0000 7f9fbcbcc700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9fb80c0e90 0x7f9fb80c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.292+0000 7f9fbcbcc700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb80a8930 con 0x7f9fb80a81f0 2020-11-20T12:18:33.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.292+0000 7f9fbcbcc700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb80ba2a0 con 0x7f9fb814f880 2020-11-20T12:18:33.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.292+0000 7f9fbcbcc700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb80a8690 con 0x7f9fb80c0e90 2020-11-20T12:18:33.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.292+0000 7f9fb6ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9fb80a81f0 0x7f9fb8060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56334/0 (socket says 172.21.15.116:56334) 2020-11-20T12:18:33.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.292+0000 7f9fb6ffd700 1 -- 172.21.15.116:0/2167763582 learned_addr learned my addr 172.21.15.116:0/2167763582 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:33.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.292+0000 7f9fb77fe700 1 --2- 172.21.15.116:0/2167763582 >> v2:172.21.15.116:3300/0 conn(0x7f9fb814f880 0x7f9fb80b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.292+0000 7f9fb7fff700 1 --2- 172.21.15.116:0/2167763582 >> v2:172.21.15.173:3300/0 conn(0x7f9fb80c0e90 0x7f9fb80c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.292+0000 7f9fb67fc700 1 -- 172.21.15.116:0/2167763582 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1108925718 0 0) 0x7f9fb80a8930 con 0x7f9fb80a81f0 2020-11-20T12:18:33.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.292+0000 7f9fb67fc700 1 -- 172.21.15.116:0/2167763582 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fa4003240 con 0x7f9fb80a81f0 2020-11-20T12:18:33.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fb77fe700 1 -- 172.21.15.116:0/2167763582 >> v2:172.21.15.173:3300/0 conn(0x7f9fb80c0e90 msgr2=0x7f9fb80c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:33.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fb77fe700 1 --2- 172.21.15.116:0/2167763582 >> v2:172.21.15.173:3300/0 conn(0x7f9fb80c0e90 0x7f9fb80c12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fb77fe700 1 -- 172.21.15.116:0/2167763582 >> v1:172.21.15.116:6789/0 conn(0x7f9fb80a81f0 legacy=0x7f9fb8060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fb67fc700 1 -- 172.21.15.116:0/2167763582 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4273688668 0 0) 0x7f9fa4003240 con 0x7f9fb80a81f0 2020-11-20T12:18:33.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fb77fe700 1 -- 172.21.15.116:0/2167763582 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb80b94d0 con 0x7f9fb814f880 2020-11-20T12:18:33.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fb77fe700 1 --2- 172.21.15.116:0/2167763582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb814f880 0x7f9fb80b9db0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f9fb8063860 tx=0x7f9fa8006df0).ready entity=mon.0 client_cookie=31e96fcce1b0eec1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:33.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fb67fc700 1 -- 172.21.15.116:0/2167763582 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac0014e0 con 0x7f9fb814f880 2020-11-20T12:18:33.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fb67fc700 1 -- 172.21.15.116:0/2167763582 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fa8011400 con 0x7f9fb814f880 2020-11-20T12:18:33.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fb67fc700 1 -- 172.21.15.116:0/2167763582 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fa8011780 con 0x7f9fb814f880 2020-11-20T12:18:33.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/2167763582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb814f880 msgr2=0x7f9fb80b9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fbcbcc700 1 --2- 172.21.15.116:0/2167763582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb814f880 0x7f9fb80b9db0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f9fb8063860 tx=0x7f9fa8006df0).stop 2020-11-20T12:18:33.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/2167763582 shutdown_connections 2020-11-20T12:18:33.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fbcbcc700 1 --2- 172.21.15.116:0/2167763582 >> v2:172.21.15.173:3300/0 conn(0x7f9fb80c0e90 0x7f9fb80c12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fbcbcc700 1 --2- 172.21.15.116:0/2167763582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb814f880 0x7f9fb80b9db0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/2167763582 >> 172.21.15.116:0/2167763582 conn(0x7f9fb8068350 msgr2=0x7f9fb80627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:33.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/2167763582 shutdown_connections 2020-11-20T12:18:33.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.296+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/2167763582 wait complete. 2020-11-20T12:18:33.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.300+0000 7f9fbcbcc700 1 Processor -- start 2020-11-20T12:18:33.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.300+0000 7f9fbcbcc700 1 -- start start 2020-11-20T12:18:33.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.300+0000 7f9fbcbcc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb80a81f0 0x7f9fb81534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.300+0000 7f9fbcbcc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb80c0e90 0x7f9fb8151e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.300+0000 7f9fbcbcc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb814f880 0x7f9fb813cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.300+0000 7f9fbcbcc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80c3540 con 0x7f9fb814f880 2020-11-20T12:18:33.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.300+0000 7f9fbcbcc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80c37e0 con 0x7f9fb80a81f0 2020-11-20T12:18:33.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.300+0000 7f9fbcbcc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb80c3a80 con 0x7f9fb80c0e90 2020-11-20T12:18:33.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.300+0000 7f9fb7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb814f880 0x7f9fb813cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.300+0000 7f9fb7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb814f880 0x7f9fb813cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46910/0 (socket says 172.21.15.116:46910) 2020-11-20T12:18:33.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.300+0000 7f9fb7fff700 1 -- 172.21.15.116:0/1888312195 learned_addr learned my addr 172.21.15.116:0/1888312195 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:33.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fb6ffd700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb80c0e90 0x7f9fb8151e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fb77fe700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb80a81f0 0x7f9fb81534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fb7fff700 1 -- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb80c0e90 msgr2=0x7f9fb8151e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fb7fff700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb80c0e90 0x7f9fb8151e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fb7fff700 1 -- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb80a81f0 msgr2=0x7f9fb81534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fb7fff700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb80a81f0 0x7f9fb81534b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fb7fff700 1 -- 172.21.15.116:0/1888312195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb80c3d20 con 0x7f9fb814f880 2020-11-20T12:18:33.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fb6ffd700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb80c0e90 0x7f9fb8151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:33.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fb7fff700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb814f880 0x7f9fb813cb50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f9fb000ef50 tx=0x7f9fb00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:33.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9f977fe700 1 -- 172.21.15.116:0/1888312195 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb000ccc0 con 0x7f9fb814f880 2020-11-20T12:18:33.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9f977fe700 1 -- 172.21.15.116:0/1888312195 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fb000ce20 con 0x7f9fb814f880 2020-11-20T12:18:33.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9f977fe700 1 -- 172.21.15.116:0/1888312195 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fb0012620 con 0x7f9fb814f880 2020-11-20T12:18:33.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/1888312195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb80c3fc0 con 0x7f9fb814f880 2020-11-20T12:18:33.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/1888312195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb8152520 con 0x7f9fb814f880 2020-11-20T12:18:33.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.304+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/1888312195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb80bbac0 con 0x7f9fb814f880 2020-11-20T12:18:33.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.308+0000 7f9f977fe700 1 -- 172.21.15.116:0/1888312195 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9fb0022070 con 0x7f9fb814f880 2020-11-20T12:18:33.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.308+0000 7f9f977fe700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 0x7f9f90057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.308+0000 7f9f977fe700 1 -- 172.21.15.116:0/1888312195 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9fb006f020 con 0x7f9fb814f880 2020-11-20T12:18:33.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.308+0000 7f9fb77fe700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 0x7f9f90057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.308+0000 7f9fb77fe700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 0x7f9f90057c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f9fa800b150 tx=0x7f9fa8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:33.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.312+0000 7f9f977fe700 1 -- 172.21.15.116:0/1888312195 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9fb80bbac0 con 0x7f9fb814f880 2020-11-20T12:18:33.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.476+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/1888312195 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9fb80bbac0 con 0x7f9fb814f880 2020-11-20T12:18:33.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.476+0000 7f9f977fe700 1 -- 172.21.15.116:0/1888312195 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9fb80bbac0 con 0x7f9fb814f880 2020-11-20T12:18:33.475 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:33.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.476+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 msgr2=0x7f9f90057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.476+0000 7f9fbcbcc700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 0x7f9f90057c80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f9fa800b150 tx=0x7f9fa8015000).stop 2020-11-20T12:18:33.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.476+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/1888312195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb814f880 msgr2=0x7f9fb813cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.476+0000 7f9fbcbcc700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb814f880 0x7f9fb813cb50 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f9fb000ef50 tx=0x7f9fb00091d0).stop 2020-11-20T12:18:33.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.480+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/1888312195 shutdown_connections 2020-11-20T12:18:33.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.480+0000 7f9fbcbcc700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9f900557c0 0x7f9f90057c80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.480+0000 7f9fbcbcc700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9fb814f880 0x7f9fb813cb50 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.480+0000 7f9fbcbcc700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9fb80a81f0 0x7f9fb81534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.480+0000 7f9fbcbcc700 1 --2- 172.21.15.116:0/1888312195 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9fb80c0e90 0x7f9fb8151e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.480+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/1888312195 >> 172.21.15.116:0/1888312195 conn(0x7f9fb8068350 msgr2=0x7f9fb814e810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:33.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.480+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/1888312195 shutdown_connections 2020-11-20T12:18:33.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.480+0000 7f9fbcbcc700 1 -- 172.21.15.116:0/1888312195 wait complete. 2020-11-20T12:18:33.499 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:18:33.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.688+0000 7fd549c3d700 1 Processor -- start 2020-11-20T12:18:33.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.692+0000 7fd549c3d700 1 -- start start 2020-11-20T12:18:33.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.692+0000 7fd549c3d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414e590 0x7fd544063100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.692+0000 7fd549c3d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5440bff10 0x7fd54414f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.692+0000 7fd549c3d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5440635f0 con 0x7fd54414f960 2020-11-20T12:18:33.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.692+0000 7fd549c3d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5440c0690 con 0x7fd5440bff10 2020-11-20T12:18:33.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.692+0000 7fd549c3d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5440c0930 con 0x7fd54414e590 2020-11-20T12:18:33.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.692+0000 7fd543fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5440bff10 0x7fd54414f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd548c3b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414e590 0x7fd544063100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd548c3b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414e590 0x7fd544063100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:55864/0 (socket says 172.21.15.116:55864) 2020-11-20T12:18:33.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd543fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd5440bff10 0x7fd54414f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46916/0 (socket says 172.21.15.116:46916) 2020-11-20T12:18:33.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd548c3b700 1 -- 172.21.15.116:0/78633727 learned_addr learned my addr 172.21.15.116:0/78633727 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:33.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd54943c700 1 --1- 172.21.15.116:0/78633727 >> v1:172.21.15.173:6789/0 conn(0x7fd54414f960 0x7fd5440a66e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:52582/0 (socket says 172.21.15.116:52582) 2020-11-20T12:18:33.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd543fff700 1 -- 172.21.15.116:0/78633727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414e590 msgr2=0x7fd544063100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:33.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd543fff700 1 --2- 172.21.15.116:0/78633727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414e590 0x7fd544063100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd543fff700 1 -- 172.21.15.116:0/78633727 >> v1:172.21.15.173:6789/0 conn(0x7fd54414f960 legacy=0x7fd5440a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd543fff700 1 -- 172.21.15.116:0/78633727 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440c0bd0 con 0x7fd5440bff10 2020-11-20T12:18:33.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd548c3b700 1 --2- 172.21.15.116:0/78633727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414e590 0x7fd544063100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:33.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd543fff700 1 --2- 172.21.15.116:0/78633727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5440bff10 0x7fd54414f470 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fd53000b9a0 tx=0x7fd530006df0).ready entity=mon.0 client_cookie=2f8d61e297f5e70c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:33.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd5437fe700 1 -- 172.21.15.116:0/78633727 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5300097b0 con 0x7fd5440bff10 2020-11-20T12:18:33.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd5437fe700 1 -- 172.21.15.116:0/78633727 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd530009910 con 0x7fd5440bff10 2020-11-20T12:18:33.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd5437fe700 1 -- 172.21.15.116:0/78633727 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd530009c90 con 0x7fd5440bff10 2020-11-20T12:18:33.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd549c3d700 1 -- 172.21.15.116:0/78633727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5440bff10 msgr2=0x7fd54414f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd549c3d700 1 --2- 172.21.15.116:0/78633727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5440bff10 0x7fd54414f470 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7fd53000b9a0 tx=0x7fd530006df0).stop 2020-11-20T12:18:33.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd549c3d700 1 -- 172.21.15.116:0/78633727 shutdown_connections 2020-11-20T12:18:33.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd549c3d700 1 --2- 172.21.15.116:0/78633727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414e590 0x7fd544063100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd549c3d700 1 --2- 172.21.15.116:0/78633727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5440bff10 0x7fd54414f470 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd549c3d700 1 -- 172.21.15.116:0/78633727 >> 172.21.15.116:0/78633727 conn(0x7fd544068350 msgr2=0x7fd5440627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:33.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd549c3d700 1 -- 172.21.15.116:0/78633727 shutdown_connections 2020-11-20T12:18:33.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.696+0000 7fd549c3d700 1 -- 172.21.15.116:0/78633727 wait complete. 2020-11-20T12:18:33.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.700+0000 7fd549c3d700 1 Processor -- start 2020-11-20T12:18:33.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.700+0000 7fd549c3d700 1 -- start start 2020-11-20T12:18:33.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd549c3d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5440bff10 0x7fd5441622e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd549c3d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd54414e590 0x7fd5440bb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd549c3d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414f960 0x7fd54414d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd549c3d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5440be580 con 0x7fd5440bff10 2020-11-20T12:18:33.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd549c3d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5440bd660 con 0x7fd54414e590 2020-11-20T12:18:33.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd549c3d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5440bd900 con 0x7fd54414f960 2020-11-20T12:18:33.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd548c3b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5440bff10 0x7fd5441622e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd548c3b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5440bff10 0x7fd5441622e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46920/0 (socket says 172.21.15.116:46920) 2020-11-20T12:18:33.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd548c3b700 1 -- 172.21.15.116:0/2344191768 learned_addr learned my addr 172.21.15.116:0/2344191768 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:33.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd543fff700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd54414e590 0x7fd5440bb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd54943c700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414f960 0x7fd54414d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd548c3b700 1 -- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414f960 msgr2=0x7fd54414d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd548c3b700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414f960 0x7fd54414d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd548c3b700 1 -- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd54414e590 msgr2=0x7fd5440bb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd548c3b700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd54414e590 0x7fd5440bb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd548c3b700 1 -- 172.21.15.116:0/2344191768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440bdba0 con 0x7fd5440bff10 2020-11-20T12:18:33.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd548c3b700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5440bff10 0x7fd5441622e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fd53800ef20 tx=0x7fd538006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd540ff9700 1 -- 172.21.15.116:0/2344191768 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd53800cbc0 con 0x7fd5440bff10 2020-11-20T12:18:33.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd540ff9700 1 -- 172.21.15.116:0/2344191768 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd53800cd20 con 0x7fd5440bff10 2020-11-20T12:18:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd540ff9700 1 -- 172.21.15.116:0/2344191768 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd538012600 con 0x7fd5440bff10 2020-11-20T12:18:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd549c3d700 1 -- 172.21.15.116:0/2344191768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5440bde40 con 0x7fd5440bff10 2020-11-20T12:18:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.704+0000 7fd549c3d700 1 -- 172.21.15.116:0/2344191768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5440bbb30 con 0x7fd5440bff10 2020-11-20T12:18:33.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.708+0000 7fd549c3d700 1 -- 172.21.15.116:0/2344191768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54405cf30 con 0x7fd5440bff10 2020-11-20T12:18:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.708+0000 7fd540ff9700 1 -- 172.21.15.116:0/2344191768 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd538022070 con 0x7fd5440bff10 2020-11-20T12:18:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.708+0000 7fd540ff9700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd51c0557c0 0x7fd51c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:33.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.708+0000 7fd540ff9700 1 -- 172.21.15.116:0/2344191768 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd53806ed10 con 0x7fd5440bff10 2020-11-20T12:18:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.708+0000 7fd543fff700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd51c0557c0 0x7fd51c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:33.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.712+0000 7fd543fff700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd51c0557c0 0x7fd51c057c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fd53000b9a0 tx=0x7fd530017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:33.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.716+0000 7fd540ff9700 1 -- 172.21.15.116:0/2344191768 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd538044bb0 con 0x7fd5440bff10 2020-11-20T12:18:33.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.876+0000 7fd549c3d700 1 -- 172.21.15.116:0/2344191768 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd544066220 con 0x7fd5440bff10 2020-11-20T12:18:33.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.876+0000 7fd540ff9700 1 -- 172.21.15.116:0/2344191768 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+10 (secure 0 0 0) 0x7fd5380481d0 con 0x7fd5440bff10 2020-11-20T12:18:33.876 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:33.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 -- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd51c0557c0 msgr2=0x7fd51c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd51c0557c0 0x7fd51c057c80 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7fd53000b9a0 tx=0x7fd530017000).stop 2020-11-20T12:18:33.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 -- 172.21.15.116:0/2344191768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5440bff10 msgr2=0x7fd5441622e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:33.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5440bff10 0x7fd5441622e0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7fd53800ef20 tx=0x7fd538006eb0).stop 2020-11-20T12:18:33.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 -- 172.21.15.116:0/2344191768 shutdown_connections 2020-11-20T12:18:33.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd51c0557c0 0x7fd51c057c80 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd54414f960 0x7fd54414d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5440bff10 0x7fd5441622e0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 --2- 172.21.15.116:0/2344191768 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd54414e590 0x7fd5440bb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:33.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 -- 172.21.15.116:0/2344191768 >> 172.21.15.116:0/2344191768 conn(0x7fd544068350 msgr2=0x7fd544151160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:33.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 -- 172.21.15.116:0/2344191768 shutdown_connections 2020-11-20T12:18:33.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:33.880+0000 7fd549c3d700 1 -- 172.21.15.116:0/2344191768 wait complete. 2020-11-20T12:18:33.906 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:18:33.907 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:18:34.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.084+0000 7f590741f700 1 Processor -- start 2020-11-20T12:18:34.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.084+0000 7f590741f700 1 -- start start 2020-11-20T12:18:34.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590741f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5900062880 0x7f5900062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:34.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590741f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f59000b9bf0 0x7f590014be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:34.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590741f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59000ba1b0 con 0x7f590014b990 2020-11-20T12:18:34.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590741f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5900152e60 con 0x7f5900062880 2020-11-20T12:18:34.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590741f700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59001555a0 con 0x7f59000b9bf0 2020-11-20T12:18:34.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590641d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5900062880 0x7f5900062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:34.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590641d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5900062880 0x7f5900062c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46928/0 (socket says 172.21.15.116:46928) 2020-11-20T12:18:34.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590641d700 1 -- 172.21.15.116:0/1662450713 learned_addr learned my addr 172.21.15.116:0/1662450713 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:34.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590641d700 1 -- 172.21.15.116:0/1662450713 >> v2:172.21.15.173:3300/0 conn(0x7f59000b9bf0 msgr2=0x7f590014be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:34.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f5906c1e700 1 --2- 172.21.15.116:0/1662450713 >> v2:172.21.15.173:3300/0 conn(0x7f59000b9bf0 0x7f590014be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:34.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590641d700 1 --2- 172.21.15.116:0/1662450713 >> v2:172.21.15.173:3300/0 conn(0x7f59000b9bf0 0x7f590014be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:34.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590641d700 1 -- 172.21.15.116:0/1662450713 >> v1:172.21.15.173:6789/0 conn(0x7f590014b990 legacy=0x7f59000b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:34.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590641d700 1 -- 172.21.15.116:0/1662450713 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900155840 con 0x7f5900062880 2020-11-20T12:18:34.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f5906c1e700 1 --2- 172.21.15.116:0/1662450713 >> v2:172.21.15.173:3300/0 conn(0x7f59000b9bf0 0x7f590014be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:34.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.088+0000 7f590641d700 1 --2- 172.21.15.116:0/1662450713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5900062880 0x7f5900062c60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f5900152fa0 tx=0x7f58fc006eb0).ready entity=mon.0 client_cookie=e2763508735226bd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590541b700 1 -- 172.21.15.116:0/1662450713 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc0098d0 con 0x7f5900062880 2020-11-20T12:18:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590541b700 1 -- 172.21.15.116:0/1662450713 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58fc009a30 con 0x7f5900062880 2020-11-20T12:18:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590541b700 1 -- 172.21.15.116:0/1662450713 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58fc009db0 con 0x7f5900062880 2020-11-20T12:18:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590741f700 1 -- 172.21.15.116:0/1662450713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5900062880 msgr2=0x7f5900062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:34.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590741f700 1 --2- 172.21.15.116:0/1662450713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5900062880 0x7f5900062c60 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f5900152fa0 tx=0x7f58fc006eb0).stop 2020-11-20T12:18:34.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590741f700 1 -- 172.21.15.116:0/1662450713 shutdown_connections 2020-11-20T12:18:34.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590741f700 1 --2- 172.21.15.116:0/1662450713 >> v2:172.21.15.173:3300/0 conn(0x7f59000b9bf0 0x7f590014be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:34.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590741f700 1 --2- 172.21.15.116:0/1662450713 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5900062880 0x7f5900062c60 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:34.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590741f700 1 -- 172.21.15.116:0/1662450713 >> 172.21.15.116:0/1662450713 conn(0x7f5900060c40 msgr2=0x7f59000c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:34.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590741f700 1 -- 172.21.15.116:0/1662450713 shutdown_connections 2020-11-20T12:18:34.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590741f700 1 -- 172.21.15.116:0/1662450713 wait complete. 2020-11-20T12:18:34.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.092+0000 7f590741f700 1 Processor -- start 2020-11-20T12:18:34.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f590741f700 1 -- start start 2020-11-20T12:18:34.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f590741f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5900062880 0x7f59000c3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:34.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f590741f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59000b9bf0 0x7f59000a3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:34.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f590741f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f590014b990 0x7f5900149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:34.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f590741f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59000a4c20 con 0x7f59000b9bf0 2020-11-20T12:18:34.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f590741f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59000aa400 con 0x7f590014b990 2020-11-20T12:18:34.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f590741f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59000a4440 con 0x7f5900062880 2020-11-20T12:18:34.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f5905c1c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59000b9bf0 0x7f59000a3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:34.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f5905c1c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59000b9bf0 0x7f59000a3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46934/0 (socket says 172.21.15.116:46934) 2020-11-20T12:18:34.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f5905c1c700 1 -- 172.21.15.116:0/3948065773 learned_addr learned my addr 172.21.15.116:0/3948065773 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:34.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f5906c1e700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f590014b990 0x7f5900149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:34.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f590641d700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5900062880 0x7f59000c3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:34.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f5905c1c700 1 -- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5900062880 msgr2=0x7f59000c3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:34.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f5905c1c700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5900062880 0x7f59000c3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:34.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f5905c1c700 1 -- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f590014b990 msgr2=0x7f5900149dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:34.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f5905c1c700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f590014b990 0x7f5900149dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:34.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f5905c1c700 1 -- 172.21.15.116:0/3948065773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59000a46e0 con 0x7f59000b9bf0 2020-11-20T12:18:34.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f5905c1c700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59000b9bf0 0x7f59000a3e10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f58f000c6e0 tx=0x7f58f0007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:34.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f58ee7fc700 1 -- 172.21.15.116:0/3948065773 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0007020 con 0x7f59000b9bf0 2020-11-20T12:18:34.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f58ee7fc700 1 -- 172.21.15.116:0/3948065773 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f0007180 con 0x7f59000b9bf0 2020-11-20T12:18:34.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f590741f700 1 -- 172.21.15.116:0/3948065773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59000a4980 con 0x7f59000b9bf0 2020-11-20T12:18:34.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.096+0000 7f58ee7fc700 1 -- 172.21.15.116:0/3948065773 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f000e560 con 0x7f59000b9bf0 2020-11-20T12:18:34.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.100+0000 7f590741f700 1 -- 172.21.15.116:0/3948065773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590014a2c0 con 0x7f59000b9bf0 2020-11-20T12:18:34.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.100+0000 7f590741f700 1 -- 172.21.15.116:0/3948065773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5900062ff0 con 0x7f59000b9bf0 2020-11-20T12:18:34.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.100+0000 7f58ee7fc700 1 -- 172.21.15.116:0/3948065773 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f58f0018020 con 0x7f59000b9bf0 2020-11-20T12:18:34.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.100+0000 7f58ee7fc700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58d80557c0 0x7f58d8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:34.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.100+0000 7f58ee7fc700 1 -- 172.21.15.116:0/3948065773 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f58f006dff0 con 0x7f59000b9bf0 2020-11-20T12:18:34.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.100+0000 7f590641d700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58d80557c0 0x7f58d8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:34.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.104+0000 7f590641d700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58d80557c0 0x7f58d8057c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f5900152fa0 tx=0x7f58fc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:34.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.108+0000 7f58ee7fc700 1 -- 172.21.15.116:0/3948065773 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f58f0043e90 con 0x7f59000b9bf0 2020-11-20T12:18:34.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.320+0000 7f590741f700 1 -- 172.21.15.116:0/3948065773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f590005cf30 con 0x7f59000b9bf0 2020-11-20T12:18:34.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.320+0000 7f58ee7fc700 1 -- 172.21.15.116:0/3948065773 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2850 (secure 0 0 0) 0x7f58f00474b0 con 0x7f59000b9bf0 2020-11-20T12:18:34.321 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:34.321 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/785878283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":785878283},{"type":"v1","addr":"172.21.15.173:6837","nonce":785878283}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5952,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1568769426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1568769426},{"type":"v1","addr":"172.21.15.173:6836","nonce":1568769426}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3552490451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3552490451},{"type":"v1","addr":"172.21.15.116:6833","nonce":3552490451}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:18:27.137821+0000","modified":"2020-11-20T12:18:29.161493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3370673707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3370673707},{"type":"v1","addr":"172.21.15.116:6835","nonce":3370673707}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-20T12:18:34.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 -- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58d80557c0 msgr2=0x7f58d8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:34.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58d80557c0 0x7f58d8057c80 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f5900152fa0 tx=0x7f58fc015000).stop 2020-11-20T12:18:34.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 -- 172.21.15.116:0/3948065773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59000b9bf0 msgr2=0x7f59000a3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:34.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59000b9bf0 0x7f59000a3e10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7f58f000c6e0 tx=0x7f58f0007c50).stop 2020-11-20T12:18:34.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 -- 172.21.15.116:0/3948065773 shutdown_connections 2020-11-20T12:18:34.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58d80557c0 0x7f58d8057c80 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:34.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f590014b990 0x7f5900149dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:34.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5900062880 0x7f59000c3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:34.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 --2- 172.21.15.116:0/3948065773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59000b9bf0 0x7f59000a3e10 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:34.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 -- 172.21.15.116:0/3948065773 >> 172.21.15.116:0/3948065773 conn(0x7f5900060c40 msgr2=0x7f590014be20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:34.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 -- 172.21.15.116:0/3948065773 shutdown_connections 2020-11-20T12:18:34.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:34.324+0000 7f590741f700 1 -- 172.21.15.116:0/3948065773 wait complete. 2020-11-20T12:18:34.326 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 28 2020-11-20T12:18:34.747 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:18:27.137821+0000', 'modified': '2020-11-20T12:18:29.161493+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5970': {'gid': 5970, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/3370673707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3370673707}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3370673707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:18:34.748 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:18:34.748 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:18:34.943 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":5970,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.7857841170000004,"mdsmap_epoch":28,"osdmap_epoch":56,"osdmap_epoch_barrier":56,"uptime":14.252686019} 2020-11-20T12:18:34.944 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 5970, "mdsmap_epoch": 28, "osdmap_epoch": 56, "osdmap_epoch_barrier": 56, "rank_uptime": 7.785784117, "state": "up:active", "uptime": 14.252686019, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:18:34.945 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:18:34.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:34.945 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-20T12:18:34.963 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:18:34.963 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:35.036 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:35.036 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:18:35.064 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:35.064 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:18:35.070 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:18:35.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:35.071 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-20T12:18:35.099 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:18:35.100 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:18:35.100 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-20T12:18:35.100 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-20T12:18:35.101 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:18:35.101 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:18:35.101 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:18:35.102 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:18:35.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:35.103 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:18:35.121 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:18:35.121 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:18:35.122 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:18:35.122 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:35.122 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:18:35.122 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:35.123 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:35.123 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:35.123 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:35.123 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:35.124 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:35.124 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:35.124 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:18:35.124 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:35.125 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:18:35.125 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:35.125 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:18:35.125 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:35.125 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:18:35.126 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:35.126 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:18:35.126 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:35.126 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:18:35.127 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:35.127 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:18:35.127 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:35.127 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:18:35.128 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:35.128 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:18:35.128 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:18:35.128 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:18:35.129 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:35.129 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:18:35.129 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:18:35.129 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:18:35.130 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:35.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:35.131 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:18:35.162 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:18:35.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:35.168 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:18:35.226 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:18:35.227 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:18:35.227 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:18:35.227 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:18:35.227 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:18:35.227 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:18:35.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:35.229 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:18:35.229 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:18:35.230 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:18:35.230 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:18:35.231 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:18:35.231 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:18:35.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:35.325 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:18:35.341 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:18:35.341 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:18:35.342 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:18:35.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:35.343 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2020-11-20T12:18:35.416 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:35.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.592+0000 7f0d709f9700 1 Processor -- start 2020-11-20T12:18:35.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.592+0000 7f0d709f9700 1 -- start start 2020-11-20T12:18:35.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d709f9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0d6c0ba890 0x7f0d6c0bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:35.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d709f9700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0d6c0c2230 0x7f0d6c0c1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:35.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d709f9700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6c066140 con 0x7f0d6c148a20 2020-11-20T12:18:35.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d709f9700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6c1526c0 con 0x7f0d6c0ba890 2020-11-20T12:18:35.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d709f9700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6c152960 con 0x7f0d6c0c2230 2020-11-20T12:18:35.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d6a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0d6c0ba890 0x7f0d6c0bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:35.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d63fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0d6c0c2230 0x7f0d6c0c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:35.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d63fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0d6c0c2230 0x7f0d6c0c1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:45998/0 (socket says 172.21.15.116:45998) 2020-11-20T12:18:35.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d63fff700 1 -- 172.21.15.116:0/3435411098 learned_addr learned my addr 172.21.15.116:0/3435411098 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d6a59c700 1 -- 172.21.15.116:0/3435411098 >> v2:172.21.15.173:3300/0 conn(0x7f0d6c0c2230 msgr2=0x7f0d6c0c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d6a59c700 1 --2- 172.21.15.116:0/3435411098 >> v2:172.21.15.173:3300/0 conn(0x7f0d6c0c2230 0x7f0d6c0c1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d6a59c700 1 -- 172.21.15.116:0/3435411098 >> v1:172.21.15.173:6789/0 conn(0x7f0d6c148a20 legacy=0x7f0d6c0c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d6a59c700 1 -- 172.21.15.116:0/3435411098 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c152c00 con 0x7f0d6c0ba890 2020-11-20T12:18:35.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d63fff700 1 --2- 172.21.15.116:0/3435411098 >> v2:172.21.15.173:3300/0 conn(0x7f0d6c0c2230 0x7f0d6c0c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:35.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.596+0000 7f0d6a59c700 1 --2- 172.21.15.116:0/3435411098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d6c0ba890 0x7f0d6c0bac70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f0d6c067910 tx=0x7f0d5c006df0).ready entity=mon.0 client_cookie=fc2748a8fd8e15a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:35.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d69d9b700 1 -- 172.21.15.116:0/3435411098 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c0097b0 con 0x7f0d6c0ba890 2020-11-20T12:18:35.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d69d9b700 1 -- 172.21.15.116:0/3435411098 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c009910 con 0x7f0d6c0ba890 2020-11-20T12:18:35.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d69d9b700 1 -- 172.21.15.116:0/3435411098 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c009c90 con 0x7f0d6c0ba890 2020-11-20T12:18:35.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d709f9700 1 -- 172.21.15.116:0/3435411098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d6c0ba890 msgr2=0x7f0d6c0bac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:35.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d709f9700 1 --2- 172.21.15.116:0/3435411098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d6c0ba890 0x7f0d6c0bac70 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f0d6c067910 tx=0x7f0d5c006df0).stop 2020-11-20T12:18:35.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d709f9700 1 -- 172.21.15.116:0/3435411098 shutdown_connections 2020-11-20T12:18:35.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d709f9700 1 --2- 172.21.15.116:0/3435411098 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d6c0ba890 0x7f0d6c0bac70 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:35.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d709f9700 1 --2- 172.21.15.116:0/3435411098 >> v2:172.21.15.173:3300/0 conn(0x7f0d6c0c2230 0x7f0d6c0c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:35.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d709f9700 1 -- 172.21.15.116:0/3435411098 >> 172.21.15.116:0/3435411098 conn(0x7f0d6c0c4f80 msgr2=0x7f0d6c0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:35.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d709f9700 1 -- 172.21.15.116:0/3435411098 shutdown_connections 2020-11-20T12:18:35.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d709f9700 1 -- 172.21.15.116:0/3435411098 wait complete. 2020-11-20T12:18:35.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d709f9700 1 Processor -- start 2020-11-20T12:18:35.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.600+0000 7f0d709f9700 1 -- start start 2020-11-20T12:18:35.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d709f9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d6c0ba890 0x7f0d6c0c0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:35.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d709f9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d6c0c2230 0x7f0d6c150930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:35.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d709f9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d6c148a20 0x7f0d6c147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:35.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d709f9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c161430 con 0x7f0d6c148a20 2020-11-20T12:18:35.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d709f9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c161660 con 0x7f0d6c0c2230 2020-11-20T12:18:35.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d709f9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c161900 con 0x7f0d6c0ba890 2020-11-20T12:18:35.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d6ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d6c148a20 0x7f0d6c147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d6ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d6c148a20 0x7f0d6c147460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46956/0 (socket says 172.21.15.116:46956) 2020-11-20T12:18:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d6ad9d700 1 -- 172.21.15.116:0/3337751334 learned_addr learned my addr 172.21.15.116:0/3337751334 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d63fff700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d6c0c2230 0x7f0d6c150930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d6a59c700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d6c0ba890 0x7f0d6c0c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:35.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d6ad9d700 1 -- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d6c0ba890 msgr2=0x7f0d6c0c0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:35.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d6ad9d700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d6c0ba890 0x7f0d6c0c0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:35.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d6ad9d700 1 -- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d6c0c2230 msgr2=0x7f0d6c150930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:35.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d6ad9d700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d6c0c2230 0x7f0d6c150930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:35.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d6ad9d700 1 -- 172.21.15.116:0/3337751334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c161ba0 con 0x7f0d6c148a20 2020-11-20T12:18:35.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d6ad9d700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d6c148a20 0x7f0d6c147460 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f0d6400ca90 tx=0x7f0d64007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:35.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d627fc700 1 -- 172.21.15.116:0/3337751334 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6400ce60 con 0x7f0d6c148a20 2020-11-20T12:18:35.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d627fc700 1 -- 172.21.15.116:0/3337751334 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6400a730 con 0x7f0d6c148a20 2020-11-20T12:18:35.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d709f9700 1 -- 172.21.15.116:0/3337751334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6c161e40 con 0x7f0d6c148a20 2020-11-20T12:18:35.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d627fc700 1 -- 172.21.15.116:0/3337751334 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6400aad0 con 0x7f0d6c148a20 2020-11-20T12:18:35.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.604+0000 7f0d709f9700 1 -- 172.21.15.116:0/3337751334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6c0aa150 con 0x7f0d6c148a20 2020-11-20T12:18:35.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.608+0000 7f0d709f9700 1 -- 172.21.15.116:0/3337751334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6c05d2a0 con 0x7f0d6c148a20 2020-11-20T12:18:35.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.608+0000 7f0d627fc700 1 -- 172.21.15.116:0/3337751334 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0d64019030 con 0x7f0d6c148a20 2020-11-20T12:18:35.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.608+0000 7f0d627fc700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0d480557c0 0x7f0d48057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:35.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.608+0000 7f0d627fc700 1 -- 172.21.15.116:0/3337751334 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0d6406dbf0 con 0x7f0d6c148a20 2020-11-20T12:18:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.608+0000 7f0d6a59c700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0d480557c0 0x7f0d48057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:35.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.612+0000 7f0d6a59c700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0d480557c0 0x7f0d48057c80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f0d5c006e30 tx=0x7f0d5c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:35.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.616+0000 7f0d627fc700 1 -- 172.21.15.116:0/3337751334 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0d64043a90 con 0x7f0d6c148a20 2020-11-20T12:18:35.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.776+0000 7f0d709f9700 1 -- 172.21.15.116:0/3337751334 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d6c061380 con 0x7f0d6c148a20 2020-11-20T12:18:36.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d627fc700 1 -- 172.21.15.116:0/3337751334 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f0d640470b0 con 0x7f0d6c148a20 2020-11-20T12:18:36.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 -- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0d480557c0 msgr2=0x7f0d48057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:36.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0d480557c0 0x7f0d48057c80 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f0d5c006e30 tx=0x7f0d5c015000).stop 2020-11-20T12:18:36.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 -- 172.21.15.116:0/3337751334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d6c148a20 msgr2=0x7f0d6c147460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:36.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d6c148a20 0x7f0d6c147460 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f0d6400ca90 tx=0x7f0d64007f50).stop 2020-11-20T12:18:36.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 -- 172.21.15.116:0/3337751334 shutdown_connections 2020-11-20T12:18:36.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0d480557c0 0x7f0d48057c80 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0d6c148a20 0x7f0d6c147460 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0d6c0ba890 0x7f0d6c0c0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 --2- 172.21.15.116:0/3337751334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0d6c0c2230 0x7f0d6c150930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 -- 172.21.15.116:0/3337751334 >> 172.21.15.116:0/3337751334 conn(0x7f0d6c0c4f80 msgr2=0x7f0d6c062e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:36.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 -- 172.21.15.116:0/3337751334 shutdown_connections 2020-11-20T12:18:36.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:35.780+0000 7f0d709f9700 1 -- 172.21.15.116:0/3337751334 wait complete. 2020-11-20T12:18:36.285 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:36.285 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":56,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:18:28.142253+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:18:25.558441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-20T12:18:26.151785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:36.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:18:25.558441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:18:26.151785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:18:36.287 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:18:36.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.472+0000 7f30db57a700 1 Processor -- start 2020-11-20T12:18:36.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.472+0000 7f30db57a700 1 -- start start 2020-11-20T12:18:36.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30db57a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d414f880 0x7f30d40b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30db57a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f30d40a81f0 0x7f30d4060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30db57a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d40a8930 con 0x7f30d4060660 2020-11-20T12:18:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30db57a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d40ba2a0 con 0x7f30d40a81f0 2020-11-20T12:18:36.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30db57a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40a8690 con 0x7f30d414f880 2020-11-20T12:18:36.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30dad79700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f30d4060660 0x7f30d40a8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56396/0 (socket says 172.21.15.116:56396) 2020-11-20T12:18:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30dad79700 1 -- 172.21.15.116:0/3595699748 learned_addr learned my addr 172.21.15.116:0/3595699748 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9d77700 1 --2- 172.21.15.116:0/3595699748 >> v2:172.21.15.116:3300/0 conn(0x7f30d40a81f0 0x7f30d4060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30da578700 1 --2- 172.21.15.116:0/3595699748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d414f880 0x7f30d40b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425945045 0 0) 0x7f30d40a8930 con 0x7f30d4060660 2020-11-20T12:18:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30bc003240 con 0x7f30d4060660 2020-11-20T12:18:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1342979645 0 0) 0x7f30bc003240 con 0x7f30d4060660 2020-11-20T12:18:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d414f880 msgr2=0x7f30d40b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:36.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 --2- 172.21.15.116:0/3595699748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d414f880 0x7f30d40b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 >> v2:172.21.15.116:3300/0 conn(0x7f30d40a81f0 msgr2=0x7f30d4060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:36.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 --2- 172.21.15.116:0/3595699748 >> v2:172.21.15.116:3300/0 conn(0x7f30d40a81f0 0x7f30d4060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d40b94d0 con 0x7f30d4060660 2020-11-20T12:18:36.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f30cc001860 con 0x7f30d4060660 2020-11-20T12:18:36.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9d77700 1 --2- 172.21.15.116:0/3595699748 >> v2:172.21.15.116:3300/0 conn(0x7f30d40a81f0 0x7f30d4060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:36.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 >> v1:172.21.15.116:6789/0 conn(0x7f30d4060660 legacy=0x7f30d40a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:36.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 --2- 172.21.15.116:0/3595699748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30bc003730 0x7f30bc003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 --2- 172.21.15.116:0/3595699748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d414f880 0x7f30bc0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 --2- 172.21.15.116:0/3595699748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30bc0049d0 0x7f30bc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40ba2a0 con 0x7f30bc003730 2020-11-20T12:18:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.476+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40a8690 con 0x7f30bc0049d0 2020-11-20T12:18:36.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30bc004250 con 0x7f30d414f880 2020-11-20T12:18:36.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30da578700 1 --2- 172.21.15.116:0/3595699748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30bc003730 0x7f30bc003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30da578700 1 -- 172.21.15.116:0/3595699748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d414f880 msgr2=0x7f30bc0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:36.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30da578700 1 --2- 172.21.15.116:0/3595699748 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d414f880 0x7f30bc0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:36.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30da578700 1 -- 172.21.15.116:0/3595699748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30bc0049d0 msgr2=0x7f30bc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:36.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30da578700 1 --2- 172.21.15.116:0/3595699748 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30bc0049d0 0x7f30bc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:36.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30da578700 1 -- 172.21.15.116:0/3595699748 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30bc003240 con 0x7f30bc003730 2020-11-20T12:18:36.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30da578700 1 --2- 172.21.15.116:0/3595699748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30bc003730 0x7f30bc003b50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f30d000bc00 tx=0x7f30d0006df0).ready entity=mon.0 client_cookie=ab8a00beb4486fb6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:36.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d000b310 con 0x7f30bc003730 2020-11-20T12:18:36.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d000eb40 con 0x7f30bc003730 2020-11-20T12:18:36.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30d9576700 1 -- 172.21.15.116:0/3595699748 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d000eec0 con 0x7f30bc003730 2020-11-20T12:18:36.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30db57a700 1 -- 172.21.15.116:0/3595699748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30bc003730 msgr2=0x7f30bc003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:36.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30db57a700 1 --2- 172.21.15.116:0/3595699748 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30bc003730 0x7f30bc003b50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f30d000bc00 tx=0x7f30d0006df0).stop 2020-11-20T12:18:36.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30dad79700 1 -- 172.21.15.116:0/3595699748 reap_dead start 2020-11-20T12:18:36.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30db57a700 1 -- 172.21.15.116:0/3595699748 shutdown_connections 2020-11-20T12:18:36.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30db57a700 1 -- 172.21.15.116:0/3595699748 >> 172.21.15.116:0/3595699748 conn(0x7f30d4068350 msgr2=0x7f30d40627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:36.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30db57a700 1 -- 172.21.15.116:0/3595699748 shutdown_connections 2020-11-20T12:18:36.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.480+0000 7f30db57a700 1 -- 172.21.15.116:0/3595699748 wait complete. 2020-11-20T12:18:36.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30db57a700 1 Processor -- start 2020-11-20T12:18:36.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30db57a700 1 -- start start 2020-11-20T12:18:36.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30db57a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30bc003730 0x7f30d415f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30db57a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d40c1fe0 0x7f30d4154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30db57a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d4154850 0x7f30d40c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30db57a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d415fd60 con 0x7f30d4154850 2020-11-20T12:18:36.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30db57a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40c10c0 con 0x7f30bc003730 2020-11-20T12:18:36.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30db57a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d40c1360 con 0x7f30d40c1fe0 2020-11-20T12:18:36.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30dad79700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d4154850 0x7f30d40c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30dad79700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d4154850 0x7f30d40c2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46976/0 (socket says 172.21.15.116:46976) 2020-11-20T12:18:36.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30dad79700 1 -- 172.21.15.116:0/2823731685 learned_addr learned my addr 172.21.15.116:0/2823731685 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:36.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30d9d77700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d40c1fe0 0x7f30d4154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.484+0000 7f30da578700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30bc003730 0x7f30d415f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30dad79700 1 -- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d40c1fe0 msgr2=0x7f30d4154360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:36.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30dad79700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d40c1fe0 0x7f30d4154360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30dad79700 1 -- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30bc003730 msgr2=0x7f30d415f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:36.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30dad79700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30bc003730 0x7f30d415f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30dad79700 1 -- 172.21.15.116:0/2823731685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d40c1600 con 0x7f30d4154850 2020-11-20T12:18:36.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30d9d77700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d40c1fe0 0x7f30d4154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:36.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30dad79700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d4154850 0x7f30d40c2ab0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f30d40b9cb0 tx=0x7f30cc001e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:36.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30c27fc700 1 -- 172.21.15.116:0/2823731685 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc00ab50 con 0x7f30d4154850 2020-11-20T12:18:36.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30c27fc700 1 -- 172.21.15.116:0/2823731685 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc00acf0 con 0x7f30d4154850 2020-11-20T12:18:36.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30db57a700 1 -- 172.21.15.116:0/2823731685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d40c18a0 con 0x7f30d4154850 2020-11-20T12:18:36.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30c27fc700 1 -- 172.21.15.116:0/2823731685 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc004ad0 con 0x7f30d4154850 2020-11-20T12:18:36.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30db57a700 1 -- 172.21.15.116:0/2823731685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d40c0db0 con 0x7f30d4154850 2020-11-20T12:18:36.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30db57a700 1 -- 172.21.15.116:0/2823731685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d405cf30 con 0x7f30d4154850 2020-11-20T12:18:36.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.488+0000 7f30c27fc700 1 -- 172.21.15.116:0/2823731685 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f30cc01d070 con 0x7f30d4154850 2020-11-20T12:18:36.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.492+0000 7f30c27fc700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30ac0557c0 0x7f30ac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.492+0000 7f30c27fc700 1 -- 172.21.15.116:0/2823731685 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f30cc06dd80 con 0x7f30d4154850 2020-11-20T12:18:36.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.492+0000 7f30da578700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30ac0557c0 0x7f30ac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.492+0000 7f30da578700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30ac0557c0 0x7f30ac057c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f30d0006d00 tx=0x7f30d0019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:36.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.496+0000 7f30c27fc700 1 -- 172.21.15.116:0/2823731685 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f30cc043b10 con 0x7f30d4154850 2020-11-20T12:18:36.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.652+0000 7f30db57a700 1 -- 172.21.15.116:0/2823731685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f30d40aa4f0 con 0x7f30d4154850 2020-11-20T12:18:36.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30c27fc700 1 -- 172.21.15.116:0/2823731685 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v56) v1 ==== 110+0+10 (secure 0 0 0) 0x7f30cc01b020 con 0x7f30d4154850 2020-11-20T12:18:36.653 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:18:36.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 -- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30ac0557c0 msgr2=0x7f30ac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:36.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30ac0557c0 0x7f30ac057c80 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f30d0006d00 tx=0x7f30d0019000).stop 2020-11-20T12:18:36.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 -- 172.21.15.116:0/2823731685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d4154850 msgr2=0x7f30d40c2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:36.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d4154850 0x7f30d40c2ab0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f30d40b9cb0 tx=0x7f30cc001e00).stop 2020-11-20T12:18:36.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 -- 172.21.15.116:0/2823731685 shutdown_connections 2020-11-20T12:18:36.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30ac0557c0 0x7f30ac057c80 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30d4154850 0x7f30d40c2ab0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30bc003730 0x7f30d415f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 --2- 172.21.15.116:0/2823731685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30d40c1fe0 0x7f30d4154360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 -- 172.21.15.116:0/2823731685 >> 172.21.15.116:0/2823731685 conn(0x7f30d4068350 msgr2=0x7f30d40a6830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:36.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 -- 172.21.15.116:0/2823731685 shutdown_connections 2020-11-20T12:18:36.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.656+0000 7f30db57a700 1 -- 172.21.15.116:0/2823731685 wait complete. 2020-11-20T12:18:36.679 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:18:36.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.860+0000 7fb54afda700 1 Processor -- start 2020-11-20T12:18:36.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.860+0000 7fb54afda700 1 -- start start 2020-11-20T12:18:36.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb54afda700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb54414e330 0x7fb544152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb54afda700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54414e710 0x7fb5440c2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb54afda700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb5440a4760 0x7fb5440650a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb54afda700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5441520c0 con 0x7fb5440a4760 2020-11-20T12:18:36.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb54afda700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5440a4c70 con 0x7fb54414e330 2020-11-20T12:18:36.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb54afda700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441522f0 con 0x7fb54414e710 2020-11-20T12:18:36.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb5497d7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54414e710 0x7fb5440c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb549fd8700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb54414e330 0x7fb544152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb5497d7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54414e710 0x7fb5440c2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:55930/0 (socket says 172.21.15.116:55930) 2020-11-20T12:18:36.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb5497d7700 1 -- 172.21.15.116:0/802916388 learned_addr learned my addr 172.21.15.116:0/802916388 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:36.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb54a7d9700 1 --2- 172.21.15.116:0/802916388 >> v2:172.21.15.116:3300/0 conn(0x7fb5440a4760 0x7fb5440650a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb549fd8700 1 -- 172.21.15.116:0/802916388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54414e710 msgr2=0x7fb5440c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:36.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb549fd8700 1 --2- 172.21.15.116:0/802916388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54414e710 0x7fb5440c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb549fd8700 1 -- 172.21.15.116:0/802916388 >> v2:172.21.15.116:3300/0 conn(0x7fb5440a4760 msgr2=0x7fb5440650a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:36.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb549fd8700 1 --2- 172.21.15.116:0/802916388 >> v2:172.21.15.116:3300/0 conn(0x7fb5440a4760 0x7fb5440650a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb549fd8700 1 -- 172.21.15.116:0/802916388 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544152590 con 0x7fb54414e330 2020-11-20T12:18:36.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.864+0000 7fb5497d7700 1 --2- 172.21.15.116:0/802916388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54414e710 0x7fb5440c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:36.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb549fd8700 1 --2- 172.21.15.116:0/802916388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb54414e330 0x7fb544152cd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fb544066d80 tx=0x7fb540006df0).ready entity=mon.1 client_cookie=ea16f10567fdcee server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:36.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb548fd6700 1 -- 172.21.15.116:0/802916388 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540009640 con 0x7fb54414e330 2020-11-20T12:18:36.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb548fd6700 1 -- 172.21.15.116:0/802916388 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5400097a0 con 0x7fb54414e330 2020-11-20T12:18:36.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb548fd6700 1 -- 172.21.15.116:0/802916388 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540009b20 con 0x7fb54414e330 2020-11-20T12:18:36.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb54afda700 1 -- 172.21.15.116:0/802916388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb54414e330 msgr2=0x7fb544152cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:36.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb54afda700 1 --2- 172.21.15.116:0/802916388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb54414e330 0x7fb544152cd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fb544066d80 tx=0x7fb540006df0).stop 2020-11-20T12:18:36.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb54afda700 1 -- 172.21.15.116:0/802916388 shutdown_connections 2020-11-20T12:18:36.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb54afda700 1 --2- 172.21.15.116:0/802916388 >> v2:172.21.15.116:3300/0 conn(0x7fb5440a4760 0x7fb5440650a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb54afda700 1 --2- 172.21.15.116:0/802916388 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb54414e330 0x7fb544152cd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb54afda700 1 --2- 172.21.15.116:0/802916388 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb54414e710 0x7fb5440c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb54afda700 1 -- 172.21.15.116:0/802916388 >> 172.21.15.116:0/802916388 conn(0x7fb544061d50 msgr2=0x7fb5440c1f70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:36.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb54afda700 1 -- 172.21.15.116:0/802916388 shutdown_connections 2020-11-20T12:18:36.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.868+0000 7fb54afda700 1 -- 172.21.15.116:0/802916388 wait complete. 2020-11-20T12:18:36.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb54afda700 1 Processor -- start 2020-11-20T12:18:36.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb54afda700 1 -- start start 2020-11-20T12:18:36.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb54afda700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5440a4760 0x7fb54414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb54afda700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb54414e710 0x7fb544140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb54afda700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5440645a0 0x7fb5441414e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb54afda700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54413fdb0 con 0x7fb54414e710 2020-11-20T12:18:36.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb54afda700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb544140050 con 0x7fb5440645a0 2020-11-20T12:18:36.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb54afda700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5441402f0 con 0x7fb5440a4760 2020-11-20T12:18:36.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb5497d7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb54414e710 0x7fb544140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb5497d7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb54414e710 0x7fb544140ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46988/0 (socket says 172.21.15.116:46988) 2020-11-20T12:18:36.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb5497d7700 1 -- 172.21.15.116:0/1109777410 learned_addr learned my addr 172.21.15.116:0/1109777410 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:36.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb549fd8700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5440a4760 0x7fb54414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb54a7d9700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5440645a0 0x7fb5441414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb5497d7700 1 -- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5440a4760 msgr2=0x7fb54414fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:36.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb5497d7700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5440a4760 0x7fb54414fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb5497d7700 1 -- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5440645a0 msgr2=0x7fb5441414e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:36.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb5497d7700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5440645a0 0x7fb5441414e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:36.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb5497d7700 1 -- 172.21.15.116:0/1109777410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb544140590 con 0x7fb54414e710 2020-11-20T12:18:36.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.872+0000 7fb5497d7700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb54414e710 0x7fb544140ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fb53400ef50 tx=0x7fb534006eb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:36.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.876+0000 7fb531ffb700 1 -- 172.21.15.116:0/1109777410 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53400cc80 con 0x7fb54414e710 2020-11-20T12:18:36.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.876+0000 7fb531ffb700 1 -- 172.21.15.116:0/1109777410 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53400cde0 con 0x7fb54414e710 2020-11-20T12:18:36.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.876+0000 7fb54afda700 1 -- 172.21.15.116:0/1109777410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb544136c10 con 0x7fb54414e710 2020-11-20T12:18:36.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.876+0000 7fb531ffb700 1 -- 172.21.15.116:0/1109777410 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534012600 con 0x7fb54414e710 2020-11-20T12:18:36.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.876+0000 7fb54afda700 1 -- 172.21.15.116:0/1109777410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb544134050 con 0x7fb54414e710 2020-11-20T12:18:36.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.876+0000 7fb54afda700 1 -- 172.21.15.116:0/1109777410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb544136da0 con 0x7fb54414e710 2020-11-20T12:18:36.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.876+0000 7fb531ffb700 1 -- 172.21.15.116:0/1109777410 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb534022070 con 0x7fb54414e710 2020-11-20T12:18:36.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.880+0000 7fb531ffb700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb51c0557c0 0x7fb51c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:36.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.880+0000 7fb531ffb700 1 -- 172.21.15.116:0/1109777410 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb53406ee20 con 0x7fb54414e710 2020-11-20T12:18:36.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.880+0000 7fb549fd8700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb51c0557c0 0x7fb51c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:36.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.880+0000 7fb549fd8700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb51c0557c0 0x7fb51c057c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fb544064a10 tx=0x7fb540015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:36.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:36.884+0000 7fb531ffb700 1 -- 172.21.15.116:0/1109777410 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb534044cc0 con 0x7fb54414e710 2020-11-20T12:18:37.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.040+0000 7fb54afda700 1 -- 172.21.15.116:0/1109777410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb54405cf30 con 0x7fb54414e710 2020-11-20T12:18:37.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.040+0000 7fb531ffb700 1 -- 172.21.15.116:0/1109777410 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb5340482e0 con 0x7fb54414e710 2020-11-20T12:18:37.039 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:37.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.040+0000 7fb54afda700 1 -- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb51c0557c0 msgr2=0x7fb51c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.040+0000 7fb54afda700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb51c0557c0 0x7fb51c057c80 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7fb544064a10 tx=0x7fb540015000).stop 2020-11-20T12:18:37.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.040+0000 7fb54afda700 1 -- 172.21.15.116:0/1109777410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb54414e710 msgr2=0x7fb544140ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.040+0000 7fb54afda700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb54414e710 0x7fb544140ff0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fb53400ef50 tx=0x7fb534006eb0).stop 2020-11-20T12:18:37.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.044+0000 7fb54afda700 1 -- 172.21.15.116:0/1109777410 shutdown_connections 2020-11-20T12:18:37.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.044+0000 7fb54afda700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb51c0557c0 0x7fb51c057c80 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.044+0000 7fb54afda700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5440645a0 0x7fb5441414e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.044+0000 7fb54afda700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5440a4760 0x7fb54414fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.044+0000 7fb54afda700 1 --2- 172.21.15.116:0/1109777410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb54414e710 0x7fb544140ff0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.044+0000 7fb54afda700 1 -- 172.21.15.116:0/1109777410 >> 172.21.15.116:0/1109777410 conn(0x7fb544061d50 msgr2=0x7fb5440aaad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:37.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.044+0000 7fb54afda700 1 -- 172.21.15.116:0/1109777410 shutdown_connections 2020-11-20T12:18:37.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.044+0000 7fb54afda700 1 -- 172.21.15.116:0/1109777410 wait complete. 2020-11-20T12:18:37.065 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:18:37.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e8a5e700 1 Processor -- start 2020-11-20T12:18:37.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e8a5e700 1 -- start start 2020-11-20T12:18:37.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e8a5e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69e40a81f0 0x7f69e40b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:37.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e8a5e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e40a8930 con 0x7f69e40c0e90 2020-11-20T12:18:37.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e8a5e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e40ba2a0 con 0x7f69e414f880 2020-11-20T12:18:37.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e8a5e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e40a8690 con 0x7f69e40a81f0 2020-11-20T12:18:37.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69e40a81f0 0x7f69e40b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69e40a81f0 0x7f69e40b9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:46996/0 (socket says 172.21.15.116:46996) 2020-11-20T12:18:37.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e259c700 1 -- 172.21.15.116:0/581844876 learned_addr learned my addr 172.21.15.116:0/581844876 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:37.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e259c700 1 -- 172.21.15.116:0/581844876 >> v1:172.21.15.173:6789/0 conn(0x7f69e414f880 legacy=0x7f69e4060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:37.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e259c700 1 -- 172.21.15.116:0/581844876 >> v1:172.21.15.116:6789/0 conn(0x7f69e40c0e90 legacy=0x7f69e40c1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.240+0000 7f69e259c700 1 -- 172.21.15.116:0/581844876 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e40b94d0 con 0x7f69e40a81f0 2020-11-20T12:18:37.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.244+0000 7f69e259c700 1 --2- 172.21.15.116:0/581844876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e40a81f0 0x7f69e40b9db0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f69d8006fd0 tx=0x7f69d8006eb0).ready entity=mon.0 client_cookie=ff6af19d8e9189ce server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:37.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.244+0000 7f69e1d9b700 1 -- 172.21.15.116:0/581844876 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8009890 con 0x7f69e40a81f0 2020-11-20T12:18:37.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.244+0000 7f69e1d9b700 1 -- 172.21.15.116:0/581844876 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d80099f0 con 0x7f69e40a81f0 2020-11-20T12:18:37.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.244+0000 7f69e1d9b700 1 -- 172.21.15.116:0/581844876 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8009d70 con 0x7f69e40a81f0 2020-11-20T12:18:37.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.244+0000 7f69e8a5e700 1 -- 172.21.15.116:0/581844876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e40a81f0 msgr2=0x7f69e40b9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.244+0000 7f69e8a5e700 1 --2- 172.21.15.116:0/581844876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e40a81f0 0x7f69e40b9db0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f69d8006fd0 tx=0x7f69d8006eb0).stop 2020-11-20T12:18:37.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.244+0000 7f69e8a5e700 1 -- 172.21.15.116:0/581844876 shutdown_connections 2020-11-20T12:18:37.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.244+0000 7f69e8a5e700 1 --2- 172.21.15.116:0/581844876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e40a81f0 0x7f69e40b9db0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.244+0000 7f69e8a5e700 1 -- 172.21.15.116:0/581844876 >> 172.21.15.116:0/581844876 conn(0x7f69e4068350 msgr2=0x7f69e40627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:37.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.244+0000 7f69e8a5e700 1 -- 172.21.15.116:0/581844876 shutdown_connections 2020-11-20T12:18:37.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.244+0000 7f69e8a5e700 1 -- 172.21.15.116:0/581844876 wait complete. 2020-11-20T12:18:37.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.248+0000 7f69e8a5e700 1 Processor -- start 2020-11-20T12:18:37.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.248+0000 7f69e8a5e700 1 -- start start 2020-11-20T12:18:37.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e8a5e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e40a81f0 0x7f69e414cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:37.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e8a5e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69e40c0e90 0x7f69e40bc720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:37.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e8a5e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e414f880 0x7f69e40bcc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:37.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e8a5e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e40c0a10 con 0x7f69e414f880 2020-11-20T12:18:37.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e8a5e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e4161550 con 0x7f69e40a81f0 2020-11-20T12:18:37.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e8a5e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e41617f0 con 0x7f69e40c0e90 2020-11-20T12:18:37.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e414f880 0x7f69e40bcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e414f880 0x7f69e40bcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47004/0 (socket says 172.21.15.116:47004) 2020-11-20T12:18:37.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e359e700 1 -- 172.21.15.116:0/2522520131 learned_addr learned my addr 172.21.15.116:0/2522520131 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:37.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e259c700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69e40c0e90 0x7f69e40bc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e2d9d700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e40a81f0 0x7f69e414cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e359e700 1 -- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69e40c0e90 msgr2=0x7f69e40bc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e359e700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69e40c0e90 0x7f69e40bc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e359e700 1 -- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e40a81f0 msgr2=0x7f69e414cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e359e700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e40a81f0 0x7f69e414cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e359e700 1 -- 172.21.15.116:0/2522520131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e4161a90 con 0x7f69e414f880 2020-11-20T12:18:37.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e359e700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e414f880 0x7f69e40bcc10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f69dc00ca90 tx=0x7f69dc001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:37.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69caffd700 1 -- 172.21.15.116:0/2522520131 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc00a730 con 0x7f69e414f880 2020-11-20T12:18:37.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69caffd700 1 -- 172.21.15.116:0/2522520131 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69dc00a890 con 0x7f69e414f880 2020-11-20T12:18:37.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e8a5e700 1 -- 172.21.15.116:0/2522520131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e4161d30 con 0x7f69e414f880 2020-11-20T12:18:37.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69caffd700 1 -- 172.21.15.116:0/2522520131 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69dc00ac30 con 0x7f69e414f880 2020-11-20T12:18:37.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.252+0000 7f69e8a5e700 1 -- 172.21.15.116:0/2522520131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e41621d0 con 0x7f69e414f880 2020-11-20T12:18:37.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.256+0000 7f69e8a5e700 1 -- 172.21.15.116:0/2522520131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e4161ec0 con 0x7f69e414f880 2020-11-20T12:18:37.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.256+0000 7f69caffd700 1 -- 172.21.15.116:0/2522520131 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f69dc010070 con 0x7f69e414f880 2020-11-20T12:18:37.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.256+0000 7f69caffd700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69c00557f0 0x7f69c0057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:37.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.256+0000 7f69caffd700 1 -- 172.21.15.116:0/2522520131 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f69dc06e440 con 0x7f69e414f880 2020-11-20T12:18:37.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.256+0000 7f69e2d9d700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69c00557f0 0x7f69c0057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.256+0000 7f69e2d9d700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69c00557f0 0x7f69c0057cb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f69d4003500 tx=0x7f69d4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:37.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.264+0000 7f69caffd700 1 -- 172.21.15.116:0/2522520131 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f69dc0442e0 con 0x7f69e414f880 2020-11-20T12:18:37.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.420+0000 7f69e8a5e700 1 -- 172.21.15.116:0/2522520131 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69e405cf30 con 0x7f69e414f880 2020-11-20T12:18:37.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.420+0000 7f69caffd700 1 -- 172.21.15.116:0/2522520131 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+10 (secure 0 0 0) 0x7f69dc047900 con 0x7f69e414f880 2020-11-20T12:18:37.419 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 -- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69c00557f0 msgr2=0x7f69c0057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69c00557f0 0x7f69c0057cb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f69d4003500 tx=0x7f69d4009040).stop 2020-11-20T12:18:37.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 -- 172.21.15.116:0/2522520131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e414f880 msgr2=0x7f69e40bcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e414f880 0x7f69e40bcc10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f69dc00ca90 tx=0x7f69dc001360).stop 2020-11-20T12:18:37.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 -- 172.21.15.116:0/2522520131 shutdown_connections 2020-11-20T12:18:37.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69c00557f0 0x7f69c0057cb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69e414f880 0x7f69e40bcc10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69e40a81f0 0x7f69e414cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 --2- 172.21.15.116:0/2522520131 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69e40c0e90 0x7f69e40bc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 -- 172.21.15.116:0/2522520131 >> 172.21.15.116:0/2522520131 conn(0x7f69e4068350 msgr2=0x7f69e41533a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:37.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 -- 172.21.15.116:0/2522520131 shutdown_connections 2020-11-20T12:18:37.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.424+0000 7f69e8a5e700 1 -- 172.21.15.116:0/2522520131 wait complete. 2020-11-20T12:18:37.447 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:18:37.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.636+0000 7fe74caed700 1 Processor -- start 2020-11-20T12:18:37.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.636+0000 7fe74caed700 1 -- start start 2020-11-20T12:18:37.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe74caed700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe7480606f0 0x7fe7480bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:37.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe74caed700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480bfaa0 0x7fe7480c1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:37.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe74caed700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe748066b90 con 0x7fe748060310 2020-11-20T12:18:37.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe74caed700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe748066e00 con 0x7fe7480606f0 2020-11-20T12:18:37.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe74caed700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7480670a0 con 0x7fe7480bfaa0 2020-11-20T12:18:37.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe747fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480bfaa0 0x7fe7480c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe746ffd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe7480606f0 0x7fe7480bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe7477fe700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fe748060310 0x7fe748066480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:52672/0 (socket says 172.21.15.116:52672) 2020-11-20T12:18:37.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe7477fe700 1 -- 172.21.15.116:0/2345740523 learned_addr learned my addr 172.21.15.116:0/2345740523 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:37.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe746ffd700 1 -- 172.21.15.116:0/2345740523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480bfaa0 msgr2=0x7fe7480c1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:37.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe746ffd700 1 --2- 172.21.15.116:0/2345740523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480bfaa0 0x7fe7480c1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe746ffd700 1 -- 172.21.15.116:0/2345740523 >> v1:172.21.15.173:6789/0 conn(0x7fe748060310 legacy=0x7fe748066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe746ffd700 1 -- 172.21.15.116:0/2345740523 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe748067340 con 0x7fe7480606f0 2020-11-20T12:18:37.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe747fff700 1 --2- 172.21.15.116:0/2345740523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480bfaa0 0x7fe7480c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:37.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe7467fc700 1 -- 172.21.15.116:0/2345740523 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798043535 0 0) 0x7fe748066b90 con 0x7fe748060310 2020-11-20T12:18:37.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.640+0000 7fe746ffd700 1 --2- 172.21.15.116:0/2345740523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7480606f0 0x7fe7480bf5b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fe73c00b9a0 tx=0x7fe73c006df0).ready entity=mon.1 client_cookie=61437a9e2541f7af server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:37.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe7467fc700 1 -- 172.21.15.116:0/2345740523 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c0097b0 con 0x7fe7480606f0 2020-11-20T12:18:37.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe7467fc700 1 -- 172.21.15.116:0/2345740523 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe73c009910 con 0x7fe7480606f0 2020-11-20T12:18:37.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe7467fc700 1 -- 172.21.15.116:0/2345740523 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe73c009c90 con 0x7fe7480606f0 2020-11-20T12:18:37.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe74caed700 1 -- 172.21.15.116:0/2345740523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7480606f0 msgr2=0x7fe7480bf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe74caed700 1 --2- 172.21.15.116:0/2345740523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7480606f0 0x7fe7480bf5b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7fe73c00b9a0 tx=0x7fe73c006df0).stop 2020-11-20T12:18:37.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe74caed700 1 -- 172.21.15.116:0/2345740523 shutdown_connections 2020-11-20T12:18:37.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe74caed700 1 --2- 172.21.15.116:0/2345740523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480bfaa0 0x7fe7480c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe74caed700 1 --2- 172.21.15.116:0/2345740523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe7480606f0 0x7fe7480bf5b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe74caed700 1 -- 172.21.15.116:0/2345740523 >> 172.21.15.116:0/2345740523 conn(0x7fe748068350 msgr2=0x7fe7480627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:37.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe74caed700 1 -- 172.21.15.116:0/2345740523 shutdown_connections 2020-11-20T12:18:37.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe74caed700 1 -- 172.21.15.116:0/2345740523 wait complete. 2020-11-20T12:18:37.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe74caed700 1 Processor -- start 2020-11-20T12:18:37.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.644+0000 7fe74caed700 1 -- start start 2020-11-20T12:18:37.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe74caed700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe748060310 0x7fe748141160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:37.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe74caed700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480606f0 0x7fe748140650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:37.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe74caed700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7480bfaa0 0x7fe74813dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:37.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe74caed700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74813fce0 con 0x7fe7480bfaa0 2020-11-20T12:18:37.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe74caed700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74814f700 con 0x7fe748060310 2020-11-20T12:18:37.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe74caed700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74814f9a0 con 0x7fe7480606f0 2020-11-20T12:18:37.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe747fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7480bfaa0 0x7fe74813dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe747fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7480bfaa0 0x7fe74813dfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47014/0 (socket says 172.21.15.116:47014) 2020-11-20T12:18:37.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe747fff700 1 -- 172.21.15.116:0/1240548908 learned_addr learned my addr 172.21.15.116:0/1240548908 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:37.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe7477fe700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe748060310 0x7fe748141160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe746ffd700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480606f0 0x7fe748140650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe747fff700 1 -- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480606f0 msgr2=0x7fe748140650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe747fff700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480606f0 0x7fe748140650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe747fff700 1 -- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe748060310 msgr2=0x7fe748141160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe747fff700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe748060310 0x7fe748141160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe747fff700 1 -- 172.21.15.116:0/1240548908 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74813ff10 con 0x7fe7480bfaa0 2020-11-20T12:18:37.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe7477fe700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe748060310 0x7fe748141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:37.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe746ffd700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480606f0 0x7fe748140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:37.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe747fff700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7480bfaa0 0x7fe74813dfb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7fe74000ef30 tx=0x7fe7400091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:37.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe7277fe700 1 -- 172.21.15.116:0/1240548908 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74000cdf0 con 0x7fe7480bfaa0 2020-11-20T12:18:37.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe7277fe700 1 -- 172.21.15.116:0/1240548908 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7400123f0 con 0x7fe7480bfaa0 2020-11-20T12:18:37.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe74caed700 1 -- 172.21.15.116:0/1240548908 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7481401b0 con 0x7fe7480bfaa0 2020-11-20T12:18:37.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe7277fe700 1 -- 172.21.15.116:0/1240548908 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe740012790 con 0x7fe7480bfaa0 2020-11-20T12:18:37.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.648+0000 7fe74caed700 1 -- 172.21.15.116:0/1240548908 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe748141870 con 0x7fe7480bfaa0 2020-11-20T12:18:37.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.652+0000 7fe74caed700 1 -- 172.21.15.116:0/1240548908 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe748140340 con 0x7fe7480bfaa0 2020-11-20T12:18:37.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.652+0000 7fe7277fe700 1 -- 172.21.15.116:0/1240548908 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe740022070 con 0x7fe7480bfaa0 2020-11-20T12:18:37.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.652+0000 7fe7277fe700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7200557c0 0x7fe720057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:37.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.652+0000 7fe7277fe700 1 -- 172.21.15.116:0/1240548908 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe74006f0e0 con 0x7fe7480bfaa0 2020-11-20T12:18:37.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.652+0000 7fe7477fe700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7200557c0 0x7fe720057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:37.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.656+0000 7fe7477fe700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7200557c0 0x7fe720057c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fe738001530 tx=0x7fe738009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:37.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.660+0000 7fe7277fe700 1 -- 172.21.15.116:0/1240548908 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe740044f80 con 0x7fe7480bfaa0 2020-11-20T12:18:37.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.816+0000 7fe74caed700 1 -- 172.21.15.116:0/1240548908 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe74805cf30 con 0x7fe7480bfaa0 2020-11-20T12:18:37.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.820+0000 7fe7277fe700 1 -- 172.21.15.116:0/1240548908 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe740016020 con 0x7fe7480bfaa0 2020-11-20T12:18:37.817 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:37.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.820+0000 7fe74caed700 1 -- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7200557c0 msgr2=0x7fe720057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.820+0000 7fe74caed700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7200557c0 0x7fe720057c80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fe738001530 tx=0x7fe738009040).stop 2020-11-20T12:18:37.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.820+0000 7fe74caed700 1 -- 172.21.15.116:0/1240548908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7480bfaa0 msgr2=0x7fe74813dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:37.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.820+0000 7fe74caed700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7480bfaa0 0x7fe74813dfb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7fe74000ef30 tx=0x7fe7400091d0).stop 2020-11-20T12:18:37.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.824+0000 7fe74caed700 1 -- 172.21.15.116:0/1240548908 shutdown_connections 2020-11-20T12:18:37.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.824+0000 7fe74caed700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe7200557c0 0x7fe720057c80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.824+0000 7fe74caed700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe7480bfaa0 0x7fe74813dfb0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.824+0000 7fe74caed700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe748060310 0x7fe748141160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.824+0000 7fe74caed700 1 --2- 172.21.15.116:0/1240548908 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe7480606f0 0x7fe748140650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.824+0000 7fe74caed700 1 -- 172.21.15.116:0/1240548908 >> 172.21.15.116:0/1240548908 conn(0x7fe748068350 msgr2=0x7fe7480bd530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.824+0000 7fe74caed700 1 -- 172.21.15.116:0/1240548908 shutdown_connections 2020-11-20T12:18:37.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:37.824+0000 7fe74caed700 1 -- 172.21.15.116:0/1240548908 wait complete. 2020-11-20T12:18:37.848 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:18:37.849 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:18:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.028+0000 7fc701bd7700 1 Processor -- start 2020-11-20T12:18:38.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.028+0000 7fc701bd7700 1 -- start start 2020-11-20T12:18:38.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.032+0000 7fc701bd7700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc6fc13afa0 0x7fc6fc0aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:38.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.032+0000 7fc701bd7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13b380 0x7fc6fc0aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:38.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.032+0000 7fc701bd7700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6fc0ada00 con 0x7fc6fc0acd30 2020-11-20T12:18:38.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.032+0000 7fc701bd7700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6fc0adca0 con 0x7fc6fc13afa0 2020-11-20T12:18:38.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.032+0000 7fc701bd7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc0adf40 con 0x7fc6fc13b380 2020-11-20T12:18:38.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.032+0000 7fc6fbfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13b380 0x7fc6fc0aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:38.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.032+0000 7fc6fbfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13b380 0x7fc6fc0aaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:55972/0 (socket says 172.21.15.116:55972) 2020-11-20T12:18:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.032+0000 7fc6fbfff700 1 -- 172.21.15.116:0/2528303069 learned_addr learned my addr 172.21.15.116:0/2528303069 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.032+0000 7fc700bd5700 1 --2- 172.21.15.116:0/2528303069 >> v2:172.21.15.173:3300/0 conn(0x7fc6fc13afa0 0x7fc6fc0aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.032+0000 7fc6fbfff700 1 -- 172.21.15.116:0/2528303069 >> v2:172.21.15.173:3300/0 conn(0x7fc6fc13afa0 msgr2=0x7fc6fc0aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.032+0000 7fc6fbfff700 1 --2- 172.21.15.116:0/2528303069 >> v2:172.21.15.173:3300/0 conn(0x7fc6fc13afa0 0x7fc6fc0aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc6fbfff700 1 -- 172.21.15.116:0/2528303069 >> v1:172.21.15.173:6789/0 conn(0x7fc6fc0acd30 legacy=0x7fc6fc0ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc6fbfff700 1 -- 172.21.15.116:0/2528303069 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc0ae1e0 con 0x7fc6fc13b380 2020-11-20T12:18:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc6fbfff700 1 --2- 172.21.15.116:0/2528303069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13b380 0x7fc6fc0aaaa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc6e8006fd0 tx=0x7fc6e8006eb0).ready entity=mon.2 client_cookie=de341a6c64989d47 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc6fb7fe700 1 -- 172.21.15.116:0/2528303069 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e80097d0 con 0x7fc6fc13b380 2020-11-20T12:18:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc6fb7fe700 1 -- 172.21.15.116:0/2528303069 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e8009930 con 0x7fc6fc13b380 2020-11-20T12:18:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc6fb7fe700 1 -- 172.21.15.116:0/2528303069 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e8009cb0 con 0x7fc6fc13b380 2020-11-20T12:18:38.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc701bd7700 1 -- 172.21.15.116:0/2528303069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13b380 msgr2=0x7fc6fc0aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:38.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc701bd7700 1 --2- 172.21.15.116:0/2528303069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13b380 0x7fc6fc0aaaa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc6e8006fd0 tx=0x7fc6e8006eb0).stop 2020-11-20T12:18:38.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc701bd7700 1 -- 172.21.15.116:0/2528303069 shutdown_connections 2020-11-20T12:18:38.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc701bd7700 1 --2- 172.21.15.116:0/2528303069 >> v2:172.21.15.173:3300/0 conn(0x7fc6fc13afa0 0x7fc6fc0aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc701bd7700 1 --2- 172.21.15.116:0/2528303069 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13b380 0x7fc6fc0aaaa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc701bd7700 1 -- 172.21.15.116:0/2528303069 >> 172.21.15.116:0/2528303069 conn(0x7fc6fc0678a0 msgr2=0x7fc6fc067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc701bd7700 1 -- 172.21.15.116:0/2528303069 shutdown_connections 2020-11-20T12:18:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.036+0000 7fc701bd7700 1 -- 172.21.15.116:0/2528303069 wait complete. 2020-11-20T12:18:38.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.040+0000 7fc701bd7700 1 Processor -- start 2020-11-20T12:18:38.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.040+0000 7fc701bd7700 1 -- start start 2020-11-20T12:18:38.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc701bd7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc0acd30 0x7fc6fc163e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:38.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc701bd7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13afa0 0x7fc6fc160f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:38.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc701bd7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6fc13b380 0x7fc6fc09f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:38.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc701bd7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc164350 con 0x7fc6fc13b380 2020-11-20T12:18:38.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc701bd7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc161440 con 0x7fc6fc0acd30 2020-11-20T12:18:38.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc701bd7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6fc1540e0 con 0x7fc6fc13afa0 2020-11-20T12:18:38.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc7013d6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6fc13b380 0x7fc6fc09f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:38.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc700bd5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc0acd30 0x7fc6fc163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:38.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc7013d6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6fc13b380 0x7fc6fc09f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47032/0 (socket says 172.21.15.116:47032) 2020-11-20T12:18:38.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc7013d6700 1 -- 172.21.15.116:0/1651842428 learned_addr learned my addr 172.21.15.116:0/1651842428 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:38.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc6fbfff700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13afa0 0x7fc6fc160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:38.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc700bd5700 1 -- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13afa0 msgr2=0x7fc6fc160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:38.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc700bd5700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13afa0 0x7fc6fc160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:38.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc700bd5700 1 -- 172.21.15.116:0/1651842428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6fc13b380 msgr2=0x7fc6fc09f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:38.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc700bd5700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6fc13b380 0x7fc6fc09f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:38.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc700bd5700 1 -- 172.21.15.116:0/1651842428 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6fc154380 con 0x7fc6fc0acd30 2020-11-20T12:18:38.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc7013d6700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6fc13b380 0x7fc6fc09f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:38.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc700bd5700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc0acd30 0x7fc6fc163e60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fc6f000eba0 tx=0x7fc6f0006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:38.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc6f8ff9700 1 -- 172.21.15.116:0/1651842428 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f000c820 con 0x7fc6fc0acd30 2020-11-20T12:18:38.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc6f8ff9700 1 -- 172.21.15.116:0/1651842428 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6f000c980 con 0x7fc6fc0acd30 2020-11-20T12:18:38.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc701bd7700 1 -- 172.21.15.116:0/1651842428 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6fc154620 con 0x7fc6fc0acd30 2020-11-20T12:18:38.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc6f8ff9700 1 -- 172.21.15.116:0/1651842428 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6f000cd00 con 0x7fc6fc0acd30 2020-11-20T12:18:38.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.044+0000 7fc701bd7700 1 -- 172.21.15.116:0/1651842428 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6fc154b60 con 0x7fc6fc0acd30 2020-11-20T12:18:38.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.048+0000 7fc701bd7700 1 -- 172.21.15.116:0/1651842428 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6fc05cf30 con 0x7fc6fc0acd30 2020-11-20T12:18:38.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.048+0000 7fc6f8ff9700 1 -- 172.21.15.116:0/1651842428 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc6f0012600 con 0x7fc6fc0acd30 2020-11-20T12:18:38.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.048+0000 7fc6f8ff9700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6d40557c0 0x7fc6d4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:38.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.048+0000 7fc6f8ff9700 1 -- 172.21.15.116:0/1651842428 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc6f006ea90 con 0x7fc6fc0acd30 2020-11-20T12:18:38.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.048+0000 7fc6fbfff700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6d40557c0 0x7fc6d4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:38.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.052+0000 7fc6fbfff700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6d40557c0 0x7fc6d4057c80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fc6fc0a0f30 tx=0x7fc6e8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:38.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.056+0000 7fc6f8ff9700 1 -- 172.21.15.116:0/1651842428 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc6f0044820 con 0x7fc6fc0acd30 2020-11-20T12:18:38.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.268+0000 7fc701bd7700 1 -- 172.21.15.116:0/1651842428 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc6fc060170 con 0x7fc6fc0acd30 2020-11-20T12:18:38.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.268+0000 7fc6f8ff9700 1 -- 172.21.15.116:0/1651842428 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2850 (secure 0 0 0) 0x7fc6f000ce60 con 0x7fc6fc0acd30 2020-11-20T12:18:38.267 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:38.267 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/785878283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":785878283},{"type":"v1","addr":"172.21.15.173:6837","nonce":785878283}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5952,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1568769426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1568769426},{"type":"v1","addr":"172.21.15.173:6836","nonce":1568769426}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3552490451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3552490451},{"type":"v1","addr":"172.21.15.116:6833","nonce":3552490451}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:18:27.137821+0000","modified":"2020-11-20T12:18:29.161493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3370673707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3370673707},{"type":"v1","addr":"172.21.15.116:6835","nonce":3370673707}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-20T12:18:38.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 -- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6d40557c0 msgr2=0x7fc6d4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:38.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6d40557c0 0x7fc6d4057c80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fc6fc0a0f30 tx=0x7fc6e8015000).stop 2020-11-20T12:18:38.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 -- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc0acd30 msgr2=0x7fc6fc163e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:38.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc0acd30 0x7fc6fc163e60 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7fc6f000eba0 tx=0x7fc6f0006df0).stop 2020-11-20T12:18:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 -- 172.21.15.116:0/1651842428 shutdown_connections 2020-11-20T12:18:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc6d40557c0 0x7fc6d4057c80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc6fc13b380 0x7fc6fc09f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:38.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc6fc0acd30 0x7fc6fc163e60 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 --2- 172.21.15.116:0/1651842428 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc6fc13afa0 0x7fc6fc160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 -- 172.21.15.116:0/1651842428 >> 172.21.15.116:0/1651842428 conn(0x7fc6fc0678a0 msgr2=0x7fc6fc0c6530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 -- 172.21.15.116:0/1651842428 shutdown_connections 2020-11-20T12:18:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:38.272+0000 7fc701bd7700 1 -- 172.21.15.116:0/1651842428 wait complete. 2020-11-20T12:18:38.273 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 28 2020-11-20T12:18:38.698 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:18:27.137821+0000', 'modified': '2020-11-20T12:18:29.161493+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5970': {'gid': 5970, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/3370673707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3370673707}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3370673707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:18:38.698 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:18:38.699 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:18:38.892 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":5970,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.734598149,"mdsmap_epoch":28,"osdmap_epoch":56,"osdmap_epoch_barrier":56,"uptime":18.201489937000002} 2020-11-20T12:18:38.892 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 5970, "mdsmap_epoch": 28, "osdmap_epoch": 56, "osdmap_epoch_barrier": 56, "rank_uptime": 11.734598149, "state": "up:active", "uptime": 18.201489937, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:18:38.893 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:18:38.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:38.894 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-20T12:18:38.911 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:18:38.912 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:39.025 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:39.025 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:18:39.049 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:39.050 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:18:39.058 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:18:39.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:39.059 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-20T12:18:39.088 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:18:39.088 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:18:39.088 INFO:tasks.cephfs.mount:self.client.name = client.2 2020-11-20T12:18:39.089 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2020-11-20T12:18:39.089 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:18:39.090 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:18:39.090 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:18:39.090 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:18:39.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:39.091 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:18:39.110 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:18:39.110 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:18:39.110 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:18:39.111 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:39.111 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:18:39.111 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:39.111 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:39.111 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:39.112 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:39.112 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:39.112 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:39.112 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:39.112 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:18:39.113 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:39.113 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:18:39.113 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:39.113 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:18:39.114 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:39.114 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:18:39.114 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:39.114 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:18:39.114 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:39.115 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:18:39.115 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:39.115 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:18:39.115 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:39.115 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:18:39.116 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:39.116 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:18:39.116 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:18:39.116 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:18:39.116 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:39.117 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:18:39.117 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:18:39.117 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:18:39.117 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:39.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:39.119 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:18:39.197 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:18:39.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:39.201 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:18:39.219 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:18:39.219 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:18:39.219 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:18:39.219 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:18:39.220 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:18:39.220 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:18:39.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:39.221 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:18:39.222 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:18:39.222 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:18:39.223 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:18:39.223 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:18:39.223 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:18:39.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:39.318 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:18:39.335 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:18:39.335 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:18:39.336 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:18:39.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:39.337 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2020-11-20T12:18:39.366 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:39.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.540+0000 7fa368e8d700 1 Processor -- start 2020-11-20T12:18:39.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.540+0000 7fa368e8d700 1 -- start start 2020-11-20T12:18:39.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.540+0000 7fa368e8d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa364148040 0x7fa364148420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.540+0000 7fa368e8d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa364068420 0x7fa3640626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.540+0000 7fa368e8d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa364067190 con 0x7fa364062bd0 2020-11-20T12:18:39.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.540+0000 7fa368e8d700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3640677a0 con 0x7fa364068420 2020-11-20T12:18:39.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.540+0000 7fa368e8d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa364067a40 con 0x7fa364148040 2020-11-20T12:18:39.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa362ffd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa364068420 0x7fa3640626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa3637fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa364148040 0x7fa364148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa363fff700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fa364062bd0 0x7fa364066480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:52700/0 (socket says 172.21.15.116:52700) 2020-11-20T12:18:39.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa363fff700 1 -- 172.21.15.116:0/2204404512 learned_addr learned my addr 172.21.15.116:0/2204404512 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:39.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa362ffd700 1 -- 172.21.15.116:0/2204404512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa364148040 msgr2=0x7fa364148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:39.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa362ffd700 1 --2- 172.21.15.116:0/2204404512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa364148040 0x7fa364148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa362ffd700 1 -- 172.21.15.116:0/2204404512 >> v1:172.21.15.173:6789/0 conn(0x7fa364062bd0 legacy=0x7fa364066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:39.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa3627fc700 1 -- 172.21.15.116:0/2204404512 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223881819 0 0) 0x7fa364067190 con 0x7fa364062bd0 2020-11-20T12:18:39.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa362ffd700 1 -- 172.21.15.116:0/2204404512 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa364067ce0 con 0x7fa364068420 2020-11-20T12:18:39.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa362ffd700 1 --2- 172.21.15.116:0/2204404512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa364068420 0x7fa3640626e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fa35800dc30 tx=0x7fa358006e80).ready entity=mon.1 client_cookie=a0282fc9dc149e37 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:39.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa3627fc700 1 -- 172.21.15.116:0/2204404512 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35800bbb0 con 0x7fa364068420 2020-11-20T12:18:39.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa3627fc700 1 -- 172.21.15.116:0/2204404512 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35800bd10 con 0x7fa364068420 2020-11-20T12:18:39.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa3627fc700 1 -- 172.21.15.116:0/2204404512 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa358010680 con 0x7fa364068420 2020-11-20T12:18:39.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa368e8d700 1 -- 172.21.15.116:0/2204404512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa364068420 msgr2=0x7fa3640626e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:39.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa368e8d700 1 --2- 172.21.15.116:0/2204404512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa364068420 0x7fa3640626e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fa35800dc30 tx=0x7fa358006e80).stop 2020-11-20T12:18:39.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa368e8d700 1 -- 172.21.15.116:0/2204404512 shutdown_connections 2020-11-20T12:18:39.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa368e8d700 1 --2- 172.21.15.116:0/2204404512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa364148040 0x7fa364148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa368e8d700 1 --2- 172.21.15.116:0/2204404512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa364068420 0x7fa3640626e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.544+0000 7fa368e8d700 1 -- 172.21.15.116:0/2204404512 >> 172.21.15.116:0/2204404512 conn(0x7fa364060c40 msgr2=0x7fa3640c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:39.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.548+0000 7fa368e8d700 1 -- 172.21.15.116:0/2204404512 shutdown_connections 2020-11-20T12:18:39.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.548+0000 7fa368e8d700 1 -- 172.21.15.116:0/2204404512 wait complete. 2020-11-20T12:18:39.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.548+0000 7fa368e8d700 1 Processor -- start 2020-11-20T12:18:39.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.548+0000 7fa368e8d700 1 -- start start 2020-11-20T12:18:39.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa368e8d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa364062bd0 0x7fa3641650c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa368e8d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa364068420 0x7fa3641610e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa368e8d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa364148040 0x7fa3640a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa368e8d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3641655b0 con 0x7fa364068420 2020-11-20T12:18:39.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa368e8d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3641615d0 con 0x7fa364062bd0 2020-11-20T12:18:39.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa368e8d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3641617f0 con 0x7fa364148040 2020-11-20T12:18:39.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa362ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa364068420 0x7fa3641610e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa3637fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa364062bd0 0x7fa3641650c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa363fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa364148040 0x7fa3640a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa3637fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa364062bd0 0x7fa3641650c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:46094/0 (socket says 172.21.15.116:46094) 2020-11-20T12:18:39.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa3637fe700 1 -- 172.21.15.116:0/437797093 learned_addr learned my addr 172.21.15.116:0/437797093 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:39.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa363fff700 1 -- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa364062bd0 msgr2=0x7fa3641650c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:39.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa363fff700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa364062bd0 0x7fa3641650c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa363fff700 1 -- 172.21.15.116:0/437797093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa364068420 msgr2=0x7fa3641610e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa363fff700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa364068420 0x7fa3641610e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa363fff700 1 -- 172.21.15.116:0/437797093 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa364161a90 con 0x7fa364148040 2020-11-20T12:18:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa362ffd700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa364068420 0x7fa3641610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:39.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa363fff700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa364148040 0x7fa3640a65d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa35c00cb10 tx=0x7fa35c007d00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:39.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa3437fe700 1 -- 172.21.15.116:0/437797093 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00ce20 con 0x7fa364148040 2020-11-20T12:18:39.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa3437fe700 1 -- 172.21.15.116:0/437797093 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c005b80 con 0x7fa364148040 2020-11-20T12:18:39.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa368e8d700 1 -- 172.21.15.116:0/437797093 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa364161d30 con 0x7fa364148040 2020-11-20T12:18:39.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa3437fe700 1 -- 172.21.15.116:0/437797093 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c007110 con 0x7fa364148040 2020-11-20T12:18:39.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.552+0000 7fa368e8d700 1 -- 172.21.15.116:0/437797093 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36414e340 con 0x7fa364148040 2020-11-20T12:18:39.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.556+0000 7fa368e8d700 1 -- 172.21.15.116:0/437797093 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36405cf30 con 0x7fa364148040 2020-11-20T12:18:39.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.556+0000 7fa3437fe700 1 -- 172.21.15.116:0/437797093 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa35c010030 con 0x7fa364148040 2020-11-20T12:18:39.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.556+0000 7fa3437fe700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa33c0557c0 0x7fa33c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.560+0000 7fa3437fe700 1 -- 172.21.15.116:0/437797093 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa35c06e700 con 0x7fa364148040 2020-11-20T12:18:39.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.560+0000 7fa3637fe700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa33c0557c0 0x7fa33c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.560+0000 7fa3637fe700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa33c0557c0 0x7fa33c057c80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fa35400cba0 tx=0x7fa354006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:39.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.564+0000 7fa3437fe700 1 -- 172.21.15.116:0/437797093 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa35c044510 con 0x7fa364148040 2020-11-20T12:18:39.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.728+0000 7fa368e8d700 1 -- 172.21.15.116:0/437797093 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa364061230 con 0x7fa364148040 2020-11-20T12:18:39.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.728+0000 7fa3437fe700 1 -- 172.21.15.116:0/437797093 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16876 (secure 0 0 0) 0x7fa35c034020 con 0x7fa364148040 2020-11-20T12:18:39.729 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:39.730 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":56,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:18:28.142253+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:18:25.558441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-20T12:18:26.151785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:39.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 -- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa33c0557c0 msgr2=0x7fa33c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:39.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa33c0557c0 0x7fa33c057c80 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fa35400cba0 tx=0x7fa354006df0).stop 2020-11-20T12:18:39.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 -- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa364148040 msgr2=0x7fa3640a65d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:39.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa364148040 0x7fa3640a65d0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fa35c00cb10 tx=0x7fa35c007d00).stop 2020-11-20T12:18:39.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 -- 172.21.15.116:0/437797093 shutdown_connections 2020-11-20T12:18:39.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa33c0557c0 0x7fa33c057c80 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa364148040 0x7fa3640a65d0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa364062bd0 0x7fa3641650c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 --2- 172.21.15.116:0/437797093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa364068420 0x7fa3641610e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 -- 172.21.15.116:0/437797093 >> 172.21.15.116:0/437797093 conn(0x7fa364060c40 msgr2=0x7fa3640b9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:39.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 -- 172.21.15.116:0/437797093 shutdown_connections 2020-11-20T12:18:39.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.732+0000 7fa368e8d700 1 -- 172.21.15.116:0/437797093 wait complete. 2020-11-20T12:18:39.760 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:18:25.558441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:18:26.151785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:18:39.762 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:18:39.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.948+0000 7f67f7420700 1 Processor -- start 2020-11-20T12:18:39.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.948+0000 7f67f7420700 1 -- start start 2020-11-20T12:18:39.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.952+0000 7f67f7420700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f67f014f880 0x7f67f00b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.952+0000 7f67f7420700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f67f00c0e90 0x7f67f00c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.952+0000 7f67f7420700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67f00a8930 con 0x7f67f00a81f0 2020-11-20T12:18:39.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.952+0000 7f67f7420700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f00ba2a0 con 0x7f67f00c0e90 2020-11-20T12:18:39.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.952+0000 7f67f7420700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67f00a8690 con 0x7f67f014f880 2020-11-20T12:18:39.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.952+0000 7f67f5c1d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f67f00a81f0 0x7f67f0060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56486/0 (socket says 172.21.15.116:56486) 2020-11-20T12:18:39.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.952+0000 7f67f5c1d700 1 -- 172.21.15.116:0/2032054848 learned_addr learned my addr 172.21.15.116:0/2032054848 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:39.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.952+0000 7f67f641e700 1 --2- 172.21.15.116:0/2032054848 >> v2:172.21.15.173:3300/0 conn(0x7f67f014f880 0x7f67f00b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.952+0000 7f67f6c1f700 1 --2- 172.21.15.116:0/2032054848 >> v2:172.21.15.116:3300/0 conn(0x7f67f00c0e90 0x7f67f00c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.952+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135318072 0 0) 0x7f67f00a8930 con 0x7f67f00a81f0 2020-11-20T12:18:39.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67d8003240 con 0x7f67f00a81f0 2020-11-20T12:18:39.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1865250689 0 0) 0x7f67d8003240 con 0x7f67f00a81f0 2020-11-20T12:18:39.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 >> v2:172.21.15.173:3300/0 conn(0x7f67f014f880 msgr2=0x7f67f00b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:39.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 --2- 172.21.15.116:0/2032054848 >> v2:172.21.15.173:3300/0 conn(0x7f67f014f880 0x7f67f00b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 >> v2:172.21.15.116:3300/0 conn(0x7f67f00c0e90 msgr2=0x7f67f00c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:39.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 --2- 172.21.15.116:0/2032054848 >> v2:172.21.15.116:3300/0 conn(0x7f67f00c0e90 0x7f67f00c12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f00b94d0 con 0x7f67f00a81f0 2020-11-20T12:18:39.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f67e00014e0 con 0x7f67f00a81f0 2020-11-20T12:18:39.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 >> v1:172.21.15.116:6789/0 conn(0x7f67f00a81f0 legacy=0x7f67f0060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:39.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 --2- 172.21.15.116:0/2032054848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67d8003800 0x7f67d8003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 --2- 172.21.15.116:0/2032054848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67d80044e0 0x7f67d8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 --2- 172.21.15.116:0/2032054848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67d8004e50 0x7f67d800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00ba2a0 con 0x7f67d80044e0 2020-11-20T12:18:39.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f00a8690 con 0x7f67d8003800 2020-11-20T12:18:39.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d800bc20 con 0x7f67d8004e50 2020-11-20T12:18:39.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f5c1d700 1 --2- 172.21.15.116:0/2032054848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67d80044e0 0x7f67d8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f641e700 1 --2- 172.21.15.116:0/2032054848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67d8003800 0x7f67d8003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f6c1f700 1 --2- 172.21.15.116:0/2032054848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67d8004e50 0x7f67d800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f5c1d700 1 -- 172.21.15.116:0/2032054848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67d8004e50 msgr2=0x7f67d800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:39.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f5c1d700 1 --2- 172.21.15.116:0/2032054848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67d8004e50 0x7f67d800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f5c1d700 1 -- 172.21.15.116:0/2032054848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67d8003800 msgr2=0x7f67d8003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:39.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f5c1d700 1 --2- 172.21.15.116:0/2032054848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67d8003800 0x7f67d8003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f5c1d700 1 -- 172.21.15.116:0/2032054848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d8003240 con 0x7f67d80044e0 2020-11-20T12:18:39.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f6c1f700 1 -- 172.21.15.116:0/2032054848 reap_dead start 2020-11-20T12:18:39.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f5c1d700 1 --2- 172.21.15.116:0/2032054848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67d80044e0 0x7f67d8004960 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f67e000c240 tx=0x7f67e0007750).ready entity=mon.0 client_cookie=be8d4148dd400ac0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:39.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e00070d0 con 0x7f67d80044e0 2020-11-20T12:18:39.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e0007230 con 0x7f67d80044e0 2020-11-20T12:18:39.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.956+0000 7f67f541c700 1 -- 172.21.15.116:0/2032054848 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e00055f0 con 0x7f67d80044e0 2020-11-20T12:18:39.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.960+0000 7f67f7420700 1 -- 172.21.15.116:0/2032054848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67d80044e0 msgr2=0x7f67d8004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:39.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.960+0000 7f67f7420700 1 --2- 172.21.15.116:0/2032054848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67d80044e0 0x7f67d8004960 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f67e000c240 tx=0x7f67e0007750).stop 2020-11-20T12:18:39.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.960+0000 7f67f7420700 1 -- 172.21.15.116:0/2032054848 shutdown_connections 2020-11-20T12:18:39.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.960+0000 7f67f7420700 1 --2- 172.21.15.116:0/2032054848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67d80044e0 0x7f67d8004960 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.960+0000 7f67f7420700 1 -- 172.21.15.116:0/2032054848 >> 172.21.15.116:0/2032054848 conn(0x7f67f0068350 msgr2=0x7f67f00627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:39.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.960+0000 7f67f7420700 1 -- 172.21.15.116:0/2032054848 shutdown_connections 2020-11-20T12:18:39.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.960+0000 7f67f7420700 1 -- 172.21.15.116:0/2032054848 wait complete. 2020-11-20T12:18:39.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f7420700 1 Processor -- start 2020-11-20T12:18:39.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f7420700 1 -- start start 2020-11-20T12:18:39.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f7420700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67d80044e0 0x7f67f0154c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f7420700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67f0060170 0x7f67f00605f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f7420700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67f00c2b60 0x7f67f00c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f7420700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f0165e50 con 0x7f67f0060170 2020-11-20T12:18:39.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f7420700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f0154270 con 0x7f67f00c2b60 2020-11-20T12:18:39.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f7420700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67f0154510 con 0x7f67d80044e0 2020-11-20T12:18:39.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f5c1d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67f0060170 0x7f67f00605f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f5c1d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67f0060170 0x7f67f00605f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47064/0 (socket says 172.21.15.116:47064) 2020-11-20T12:18:39.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f5c1d700 1 -- 172.21.15.116:0/1352503732 learned_addr learned my addr 172.21.15.116:0/1352503732 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:39.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f6c1f700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67f00c2b60 0x7f67f00c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f641e700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67d80044e0 0x7f67f0154c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f5c1d700 1 -- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67d80044e0 msgr2=0x7f67f0154c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:39.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f5c1d700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67d80044e0 0x7f67f0154c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f5c1d700 1 -- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67f00c2b60 msgr2=0x7f67f00c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:39.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f5c1d700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67f00c2b60 0x7f67f00c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:39.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f5c1d700 1 -- 172.21.15.116:0/1352503732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f01547b0 con 0x7f67f0060170 2020-11-20T12:18:39.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f5c1d700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67f0060170 0x7f67f00605f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f67d8005400 tx=0x7f67e0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:39.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67de7fc700 1 -- 172.21.15.116:0/1352503732 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0005cc0 con 0x7f67f0060170 2020-11-20T12:18:39.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67de7fc700 1 -- 172.21.15.116:0/1352503732 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e0005e20 con 0x7f67f0060170 2020-11-20T12:18:39.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f7420700 1 -- 172.21.15.116:0/1352503732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f00c34d0 con 0x7f67f0060170 2020-11-20T12:18:39.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67de7fc700 1 -- 172.21.15.116:0/1352503732 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e00206d0 con 0x7f67f0060170 2020-11-20T12:18:39.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.964+0000 7f67f7420700 1 -- 172.21.15.116:0/1352503732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f0168750 con 0x7f67f0060170 2020-11-20T12:18:39.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.968+0000 7f67f7420700 1 -- 172.21.15.116:0/1352503732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f0060ae0 con 0x7f67f0060170 2020-11-20T12:18:39.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.968+0000 7f67de7fc700 1 -- 172.21.15.116:0/1352503732 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f67e0010070 con 0x7f67f0060170 2020-11-20T12:18:39.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.968+0000 7f67de7fc700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67c80557c0 0x7f67c8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:39.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.968+0000 7f67de7fc700 1 -- 172.21.15.116:0/1352503732 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f67e006fb30 con 0x7f67f0060170 2020-11-20T12:18:39.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.968+0000 7f67f641e700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67c80557c0 0x7f67c8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:39.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.972+0000 7f67f641e700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67c80557c0 0x7f67c8057c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f67ec006d00 tx=0x7f67ec008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:39.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:39.972+0000 7f67de7fc700 1 -- 172.21.15.116:0/1352503732 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f67e0045950 con 0x7f67f0060170 2020-11-20T12:18:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.128+0000 7f67f7420700 1 -- 172.21.15.116:0/1352503732 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f67f00c3660 con 0x7f67f0060170 2020-11-20T12:18:40.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67de7fc700 1 -- 172.21.15.116:0/1352503732 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v56) v1 ==== 110+0+10 (secure 0 0 0) 0x7f67f00c3660 con 0x7f67f0060170 2020-11-20T12:18:40.130 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:18:40.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 -- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67c80557c0 msgr2=0x7f67c8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67c80557c0 0x7f67c8057c80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f67ec006d00 tx=0x7f67ec008040).stop 2020-11-20T12:18:40.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 -- 172.21.15.116:0/1352503732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67f0060170 msgr2=0x7f67f00605f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67f0060170 0x7f67f00605f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f67d8005400 tx=0x7f67e0015000).stop 2020-11-20T12:18:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 -- 172.21.15.116:0/1352503732 shutdown_connections 2020-11-20T12:18:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67c80557c0 0x7f67c8057c80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67f00c2b60 0x7f67f00c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67d80044e0 0x7f67f0154c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 --2- 172.21.15.116:0/1352503732 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67f0060170 0x7f67f00605f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 -- 172.21.15.116:0/1352503732 >> 172.21.15.116:0/1352503732 conn(0x7f67f0068350 msgr2=0x7f67f00a66e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 -- 172.21.15.116:0/1352503732 shutdown_connections 2020-11-20T12:18:40.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.132+0000 7f67f7420700 1 -- 172.21.15.116:0/1352503732 wait complete. 2020-11-20T12:18:40.155 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:18:40.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.332+0000 7fd6e49fa700 1 Processor -- start 2020-11-20T12:18:40.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.332+0000 7fd6e49fa700 1 -- start start 2020-11-20T12:18:40.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.336+0000 7fd6e49fa700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd6e00aa980 0x7fd6e00aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.336+0000 7fd6e49fa700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e00ad250 con 0x7fd6e013b550 2020-11-20T12:18:40.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.336+0000 7fd6e49fa700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e0152fb0 con 0x7fd6e00adbf0 2020-11-20T12:18:40.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.336+0000 7fd6e49fa700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e00ad060 con 0x7fd6e00aa980 2020-11-20T12:18:40.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.336+0000 7fd6de59c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd6e013b550 0x7fd6e00a8630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56506/0 (socket says 172.21.15.116:56506) 2020-11-20T12:18:40.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.336+0000 7fd6de59c700 1 -- 172.21.15.116:0/4217269815 learned_addr learned my addr 172.21.15.116:0/4217269815 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:40.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.336+0000 7fd6ddd9b700 1 --2- 172.21.15.116:0/4217269815 >> v2:172.21.15.173:3300/0 conn(0x7fd6e00aa980 0x7fd6e00aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6ddd9b700 1 -- 172.21.15.116:0/4217269815 >> v1:172.21.15.173:6789/0 conn(0x7fd6e00adbf0 legacy=0x7fd6e0151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6ddd9b700 1 -- 172.21.15.116:0/4217269815 >> v1:172.21.15.116:6789/0 conn(0x7fd6e013b550 legacy=0x7fd6e00a8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6ddd9b700 1 -- 172.21.15.116:0/4217269815 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e0152d10 con 0x7fd6e00aa980 2020-11-20T12:18:40.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6ddd9b700 1 --2- 172.21.15.116:0/4217269815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6e00aa980 0x7fd6e00aae00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fd6d400b9a0 tx=0x7fd6d4006df0).ready entity=mon.1 client_cookie=c6e2504f4c870054 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:40.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6dd59a700 1 -- 172.21.15.116:0/4217269815 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40097b0 con 0x7fd6e00aa980 2020-11-20T12:18:40.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6dd59a700 1 -- 172.21.15.116:0/4217269815 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d4009910 con 0x7fd6e00aa980 2020-11-20T12:18:40.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6dd59a700 1 -- 172.21.15.116:0/4217269815 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4009c90 con 0x7fd6e00aa980 2020-11-20T12:18:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6e49fa700 1 -- 172.21.15.116:0/4217269815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6e00aa980 msgr2=0x7fd6e00aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6e49fa700 1 --2- 172.21.15.116:0/4217269815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6e00aa980 0x7fd6e00aae00 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fd6d400b9a0 tx=0x7fd6d4006df0).stop 2020-11-20T12:18:40.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6e49fa700 1 -- 172.21.15.116:0/4217269815 shutdown_connections 2020-11-20T12:18:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6e49fa700 1 --2- 172.21.15.116:0/4217269815 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6e00aa980 0x7fd6e00aae00 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.340+0000 7fd6e49fa700 1 -- 172.21.15.116:0/4217269815 >> 172.21.15.116:0/4217269815 conn(0x7fd6e00678a0 msgr2=0x7fd6e0067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.344+0000 7fd6e49fa700 1 -- 172.21.15.116:0/4217269815 shutdown_connections 2020-11-20T12:18:40.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.344+0000 7fd6e49fa700 1 -- 172.21.15.116:0/4217269815 wait complete. 2020-11-20T12:18:40.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.344+0000 7fd6e49fa700 1 Processor -- start 2020-11-20T12:18:40.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.344+0000 7fd6e49fa700 1 -- start start 2020-11-20T12:18:40.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6e49fa700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00aa980 0x7fd6e00becb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6e49fa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6e00adbf0 0x7fd6e0161450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6e49fa700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6e013b550 0x7fd6e014e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6e49fa700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e00bf1a0 con 0x7fd6e00aa980 2020-11-20T12:18:40.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6e49fa700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e0161940 con 0x7fd6e00adbf0 2020-11-20T12:18:40.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6e49fa700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e0161bd0 con 0x7fd6e013b550 2020-11-20T12:18:40.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6de59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00aa980 0x7fd6e00becb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6de59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00aa980 0x7fd6e00becb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47076/0 (socket says 172.21.15.116:47076) 2020-11-20T12:18:40.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6de59c700 1 -- 172.21.15.116:0/2174076668 learned_addr learned my addr 172.21.15.116:0/2174076668 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:40.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6ddd9b700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6e00adbf0 0x7fd6e0161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6ded9d700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6e013b550 0x7fd6e014e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6de59c700 1 -- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6e013b550 msgr2=0x7fd6e014e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6de59c700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6e013b550 0x7fd6e014e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6de59c700 1 -- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6e00adbf0 msgr2=0x7fd6e0161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6de59c700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6e00adbf0 0x7fd6e0161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6de59c700 1 -- 172.21.15.116:0/2174076668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e0161e70 con 0x7fd6e00aa980 2020-11-20T12:18:40.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6de59c700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00aa980 0x7fd6e00becb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fd6d000c710 tx=0x7fd6d0007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:40.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6ce7fc700 1 -- 172.21.15.116:0/2174076668 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00072e0 con 0x7fd6e00aa980 2020-11-20T12:18:40.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6ce7fc700 1 -- 172.21.15.116:0/2174076668 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d0007440 con 0x7fd6e00aa980 2020-11-20T12:18:40.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6ce7fc700 1 -- 172.21.15.116:0/2174076668 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0005800 con 0x7fd6e00aa980 2020-11-20T12:18:40.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6e49fa700 1 -- 172.21.15.116:0/2174076668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e0162110 con 0x7fd6e00aa980 2020-11-20T12:18:40.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.348+0000 7fd6e49fa700 1 -- 172.21.15.116:0/2174076668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e014c780 con 0x7fd6e00aa980 2020-11-20T12:18:40.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.352+0000 7fd6bbfff700 1 -- 172.21.15.116:0/2174076668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e005d2a0 con 0x7fd6e00aa980 2020-11-20T12:18:40.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.352+0000 7fd6ce7fc700 1 -- 172.21.15.116:0/2174076668 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd6d0010070 con 0x7fd6e00aa980 2020-11-20T12:18:40.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.352+0000 7fd6ce7fc700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6b40557c0 0x7fd6b4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.352+0000 7fd6ce7fc700 1 -- 172.21.15.116:0/2174076668 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd6d006dd80 con 0x7fd6e00aa980 2020-11-20T12:18:40.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.356+0000 7fd6ddd9b700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6b40557c0 0x7fd6b4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.356+0000 7fd6ddd9b700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6b40557c0 0x7fd6b4057c80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fd6e00bdcd0 tx=0x7fd6d4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:40.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.360+0000 7fd6ce7fc700 1 -- 172.21.15.116:0/2174076668 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd6d0043b10 con 0x7fd6e00aa980 2020-11-20T12:18:40.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.520+0000 7fd6bbfff700 1 -- 172.21.15.116:0/2174076668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd6e0060f00 con 0x7fd6e00aa980 2020-11-20T12:18:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6ce7fc700 1 -- 172.21.15.116:0/2174076668 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd6d0047130 con 0x7fd6e00aa980 2020-11-20T12:18:40.521 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:40.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 -- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6b40557c0 msgr2=0x7fd6b4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6b40557c0 0x7fd6b4057c80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7fd6e00bdcd0 tx=0x7fd6d4015000).stop 2020-11-20T12:18:40.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 -- 172.21.15.116:0/2174076668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00aa980 msgr2=0x7fd6e00becb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00aa980 0x7fd6e00becb0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fd6d000c710 tx=0x7fd6d0007c40).stop 2020-11-20T12:18:40.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 -- 172.21.15.116:0/2174076668 shutdown_connections 2020-11-20T12:18:40.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd6b40557c0 0x7fd6b4057c80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd6e013b550 0x7fd6e014e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd6e00aa980 0x7fd6e00becb0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 --2- 172.21.15.116:0/2174076668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd6e00adbf0 0x7fd6e0161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 -- 172.21.15.116:0/2174076668 >> 172.21.15.116:0/2174076668 conn(0x7fd6e00678a0 msgr2=0x7fd6e00c5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 -- 172.21.15.116:0/2174076668 shutdown_connections 2020-11-20T12:18:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.524+0000 7fd6bbfff700 1 -- 172.21.15.116:0/2174076668 wait complete. 2020-11-20T12:18:40.545 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:18:40.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.724+0000 7f18549de700 1 Processor -- start 2020-11-20T12:18:40.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.724+0000 7f18549de700 1 -- start start 2020-11-20T12:18:40.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f18549de700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1850149df0 0x7f1850153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f18549de700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f18500a9750 0x7f18500c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f18549de700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f185014a1d0 0x7f18500be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f18549de700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18500a9c20 con 0x7f1850149df0 2020-11-20T12:18:40.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f18549de700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18500a9ec0 con 0x7f18500a9750 2020-11-20T12:18:40.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f18549de700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1850153670 con 0x7f185014a1d0 2020-11-20T12:18:40.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184e59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1850149df0 0x7f1850153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184ed9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f185014a1d0 0x7f18500be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184e59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1850149df0 0x7f1850153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47088/0 (socket says 172.21.15.116:47088) 2020-11-20T12:18:40.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184e59c700 1 -- 172.21.15.116:0/1411033881 learned_addr learned my addr 172.21.15.116:0/1411033881 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:40.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184dd9b700 1 --2- 172.21.15.116:0/1411033881 >> v2:172.21.15.173:3300/0 conn(0x7f18500a9750 0x7f18500c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184ed9d700 1 -- 172.21.15.116:0/1411033881 >> v2:172.21.15.173:3300/0 conn(0x7f18500a9750 msgr2=0x7f18500c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:40.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184ed9d700 1 --2- 172.21.15.116:0/1411033881 >> v2:172.21.15.173:3300/0 conn(0x7f18500a9750 0x7f18500c0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184ed9d700 1 -- 172.21.15.116:0/1411033881 >> v2:172.21.15.116:3300/0 conn(0x7f1850149df0 msgr2=0x7f1850153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:40.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184ed9d700 1 --2- 172.21.15.116:0/1411033881 >> v2:172.21.15.116:3300/0 conn(0x7f1850149df0 0x7f1850153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184ed9d700 1 -- 172.21.15.116:0/1411033881 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1850153910 con 0x7f185014a1d0 2020-11-20T12:18:40.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184ed9d700 1 --2- 172.21.15.116:0/1411033881 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f185014a1d0 0x7f18500be240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f18500c4dc0 tx=0x7f18480091d0).ready entity=mon.2 client_cookie=be2f6c953e48b3fd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:40.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184d59a700 1 -- 172.21.15.116:0/1411033881 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184800b9c0 con 0x7f185014a1d0 2020-11-20T12:18:40.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184d59a700 1 -- 172.21.15.116:0/1411033881 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184800bb20 con 0x7f185014a1d0 2020-11-20T12:18:40.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f184d59a700 1 -- 172.21.15.116:0/1411033881 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184800bea0 con 0x7f185014a1d0 2020-11-20T12:18:40.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f18549de700 1 -- 172.21.15.116:0/1411033881 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f185014a1d0 msgr2=0x7f18500be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.728+0000 7f18549de700 1 --2- 172.21.15.116:0/1411033881 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f185014a1d0 0x7f18500be240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f18500c4dc0 tx=0x7f18480091d0).stop 2020-11-20T12:18:40.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.732+0000 7f18549de700 1 -- 172.21.15.116:0/1411033881 shutdown_connections 2020-11-20T12:18:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.732+0000 7f18549de700 1 --2- 172.21.15.116:0/1411033881 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f185014a1d0 0x7f18500be240 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.732+0000 7f18549de700 1 --2- 172.21.15.116:0/1411033881 >> v2:172.21.15.116:3300/0 conn(0x7f1850149df0 0x7f1850153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.732+0000 7f18549de700 1 --2- 172.21.15.116:0/1411033881 >> v2:172.21.15.173:3300/0 conn(0x7f18500a9750 0x7f18500c0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.732+0000 7f18549de700 1 -- 172.21.15.116:0/1411033881 >> 172.21.15.116:0/1411033881 conn(0x7f185005ba80 msgr2=0x7f18500c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:40.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.732+0000 7f18549de700 1 -- 172.21.15.116:0/1411033881 shutdown_connections 2020-11-20T12:18:40.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.732+0000 7f18549de700 1 -- 172.21.15.116:0/1411033881 wait complete. 2020-11-20T12:18:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.732+0000 7f18549de700 1 Processor -- start 2020-11-20T12:18:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.732+0000 7f18549de700 1 -- start start 2020-11-20T12:18:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f18549de700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18500a9750 0x7f185014f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f18549de700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1850149df0 0x7f185014c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f18549de700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f185014a1d0 0x7f18500be890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f18549de700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18501657c0 con 0x7f1850149df0 2020-11-20T12:18:40.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f18549de700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1850165a60 con 0x7f185014a1d0 2020-11-20T12:18:40.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f18549de700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1850165d00 con 0x7f18500a9750 2020-11-20T12:18:40.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184dd9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1850149df0 0x7f185014c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184dd9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1850149df0 0x7f185014c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47092/0 (socket says 172.21.15.116:47092) 2020-11-20T12:18:40.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184dd9b700 1 -- 172.21.15.116:0/2033738618 learned_addr learned my addr 172.21.15.116:0/2033738618 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:40.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184e59c700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18500a9750 0x7f185014f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184ed9d700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f185014a1d0 0x7f18500be890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184dd9b700 1 -- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18500a9750 msgr2=0x7f185014f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184dd9b700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18500a9750 0x7f185014f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184dd9b700 1 -- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f185014a1d0 msgr2=0x7f18500be890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184dd9b700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f185014a1d0 0x7f18500be890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184dd9b700 1 -- 172.21.15.116:0/2033738618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1850165fa0 con 0x7f1850149df0 2020-11-20T12:18:40.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184ed9d700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f185014a1d0 0x7f18500be890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:40.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.736+0000 7f184dd9b700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1850149df0 0x7f185014c860 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f184400ebd0 tx=0x7f1844006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:40.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.740+0000 7f183e7fc700 1 -- 172.21.15.116:0/2033738618 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184400b8d0 con 0x7f1850149df0 2020-11-20T12:18:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.740+0000 7f183e7fc700 1 -- 172.21.15.116:0/2033738618 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184400ba30 con 0x7f1850149df0 2020-11-20T12:18:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.740+0000 7f18549de700 1 -- 172.21.15.116:0/2033738618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1850166240 con 0x7f1850149df0 2020-11-20T12:18:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.740+0000 7f183e7fc700 1 -- 172.21.15.116:0/2033738618 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184400bdb0 con 0x7f1850149df0 2020-11-20T12:18:40.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.740+0000 7f18549de700 1 -- 172.21.15.116:0/2033738618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18500a6100 con 0x7f1850149df0 2020-11-20T12:18:40.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.740+0000 7f18549de700 1 -- 172.21.15.116:0/2033738618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185005d2a0 con 0x7f1850149df0 2020-11-20T12:18:40.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.740+0000 7f183e7fc700 1 -- 172.21.15.116:0/2033738618 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1844012600 con 0x7f1850149df0 2020-11-20T12:18:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.740+0000 7f183e7fc700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18240557c0 0x7f1824057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.740+0000 7f183e7fc700 1 -- 172.21.15.116:0/2033738618 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f184406e9a0 con 0x7f1850149df0 2020-11-20T12:18:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.744+0000 7f184e59c700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18240557c0 0x7f1824057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:40.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.744+0000 7f184e59c700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18240557c0 0x7f1824057c80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f1840006a10 tx=0x7f1840008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:40.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.748+0000 7f183e7fc700 1 -- 172.21.15.116:0/2033738618 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f18440447e0 con 0x7f1850149df0 2020-11-20T12:18:40.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.908+0000 7f18549de700 1 -- 172.21.15.116:0/2033738618 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1850061380 con 0x7f1850149df0 2020-11-20T12:18:40.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.908+0000 7f183e7fc700 1 -- 172.21.15.116:0/2033738618 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+10 (secure 0 0 0) 0x7f184406e070 con 0x7f1850149df0 2020-11-20T12:18:40.908 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:40.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 -- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18240557c0 msgr2=0x7f1824057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18240557c0 0x7f1824057c80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f1840006a10 tx=0x7f1840008040).stop 2020-11-20T12:18:40.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 -- 172.21.15.116:0/2033738618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1850149df0 msgr2=0x7f185014c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:40.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1850149df0 0x7f185014c860 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f184400ebd0 tx=0x7f1844006df0).stop 2020-11-20T12:18:40.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 -- 172.21.15.116:0/2033738618 shutdown_connections 2020-11-20T12:18:40.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f18240557c0 0x7f1824057c80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f185014a1d0 0x7f18500be890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f18500a9750 0x7f185014f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 --2- 172.21.15.116:0/2033738618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1850149df0 0x7f185014c860 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:40.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 -- 172.21.15.116:0/2033738618 >> 172.21.15.116:0/2033738618 conn(0x7f185005ba80 msgr2=0x7f18500c1300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:40.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 -- 172.21.15.116:0/2033738618 shutdown_connections 2020-11-20T12:18:40.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:40.912+0000 7f18549de700 1 -- 172.21.15.116:0/2033738618 wait complete. 2020-11-20T12:18:40.933 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:18:41.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.116+0000 7f09d3efe700 1 Processor -- start 2020-11-20T12:18:41.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.116+0000 7f09d3efe700 1 -- start start 2020-11-20T12:18:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.120+0000 7f09d3efe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f09cc0bd810 0x7f09cc156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.120+0000 7f09d3efe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09cc1551b0 0x7f09cc0a6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.120+0000 7f09d3efe700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09cc0bdfc0 con 0x7f09cc154dd0 2020-11-20T12:18:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.120+0000 7f09d3efe700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09cc0bdd20 con 0x7f09cc0bd810 2020-11-20T12:18:41.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.120+0000 7f09d3efe700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc155780 con 0x7f09cc1551b0 2020-11-20T12:18:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.120+0000 7f09d1499700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f09cc0bd810 0x7f09cc156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.120+0000 7f09d1499700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f09cc0bd810 0x7f09cc156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47102/0 (socket says 172.21.15.116:47102) 2020-11-20T12:18:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.120+0000 7f09d1499700 1 -- 172.21.15.116:0/3497482405 learned_addr learned my addr 172.21.15.116:0/3497482405 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.120+0000 7f09d249b700 1 --2- 172.21.15.116:0/3497482405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09cc1551b0 0x7f09cc0a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d1499700 1 -- 172.21.15.116:0/3497482405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09cc1551b0 msgr2=0x7f09cc0a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d1499700 1 --2- 172.21.15.116:0/3497482405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09cc1551b0 0x7f09cc0a6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d1499700 1 -- 172.21.15.116:0/3497482405 >> v1:172.21.15.173:6789/0 conn(0x7f09cc154dd0 legacy=0x7f09cc0be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:41.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d1499700 1 -- 172.21.15.116:0/3497482405 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc155a20 con 0x7f09cc0bd810 2020-11-20T12:18:41.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d1499700 1 --2- 172.21.15.116:0/3497482405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09cc0bd810 0x7f09cc156160 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f09bc006fd0 tx=0x7f09bc006eb0).ready entity=mon.0 client_cookie=5a2381aa399f1dd8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:41.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d0c98700 1 -- 172.21.15.116:0/3497482405 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc009810 con 0x7f09cc0bd810 2020-11-20T12:18:41.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d0c98700 1 -- 172.21.15.116:0/3497482405 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc009970 con 0x7f09cc0bd810 2020-11-20T12:18:41.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d0c98700 1 -- 172.21.15.116:0/3497482405 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc009cf0 con 0x7f09cc0bd810 2020-11-20T12:18:41.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d3efe700 1 -- 172.21.15.116:0/3497482405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09cc0bd810 msgr2=0x7f09cc156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:41.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d3efe700 1 --2- 172.21.15.116:0/3497482405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09cc0bd810 0x7f09cc156160 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f09bc006fd0 tx=0x7f09bc006eb0).stop 2020-11-20T12:18:41.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d3efe700 1 -- 172.21.15.116:0/3497482405 shutdown_connections 2020-11-20T12:18:41.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d3efe700 1 --2- 172.21.15.116:0/3497482405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09cc1551b0 0x7f09cc0a6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d3efe700 1 --2- 172.21.15.116:0/3497482405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09cc0bd810 0x7f09cc156160 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d3efe700 1 -- 172.21.15.116:0/3497482405 >> 172.21.15.116:0/3497482405 conn(0x7f09cc05ba80 msgr2=0x7f09cc0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:41.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d3efe700 1 -- 172.21.15.116:0/3497482405 shutdown_connections 2020-11-20T12:18:41.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.124+0000 7f09d3efe700 1 -- 172.21.15.116:0/3497482405 wait complete. 2020-11-20T12:18:41.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d3efe700 1 Processor -- start 2020-11-20T12:18:41.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d3efe700 1 -- start start 2020-11-20T12:18:41.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d3efe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09cc0bd810 0x7f09cc168d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d3efe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09cc154dd0 0x7f09cc156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d3efe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09cc1551b0 0x7f09cc0ba580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d3efe700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc156760 con 0x7f09cc154dd0 2020-11-20T12:18:41.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d3efe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc169310 con 0x7f09cc1551b0 2020-11-20T12:18:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d3efe700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09cc1695b0 con 0x7f09cc0bd810 2020-11-20T12:18:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d1499700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09cc154dd0 0x7f09cc156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d1499700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09cc154dd0 0x7f09cc156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47104/0 (socket says 172.21.15.116:47104) 2020-11-20T12:18:41.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d1499700 1 -- 172.21.15.116:0/884332783 learned_addr learned my addr 172.21.15.116:0/884332783 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:41.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d1c9a700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09cc0bd810 0x7f09cc168d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d1499700 1 -- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09cc0bd810 msgr2=0x7f09cc168d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.128+0000 7f09d249b700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09cc1551b0 0x7f09cc0ba580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09d1499700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09cc0bd810 0x7f09cc168d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09d1499700 1 -- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09cc1551b0 msgr2=0x7f09cc0ba580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09d1499700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09cc1551b0 0x7f09cc0ba580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09d1499700 1 -- 172.21.15.116:0/884332783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09cc169850 con 0x7f09cc154dd0 2020-11-20T12:18:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09d1c9a700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09cc0bd810 0x7f09cc168d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:41.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09d249b700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09cc1551b0 0x7f09cc0ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09d1499700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09cc154dd0 0x7f09cc156a00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f09cc061790 tx=0x7f09bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09b9ffb700 1 -- 172.21.15.116:0/884332783 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc011c80 con 0x7f09cc154dd0 2020-11-20T12:18:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09b9ffb700 1 -- 172.21.15.116:0/884332783 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09bc011de0 con 0x7f09cc154dd0 2020-11-20T12:18:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09d3efe700 1 -- 172.21.15.116:0/884332783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09cc169af0 con 0x7f09cc154dd0 2020-11-20T12:18:41.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09b9ffb700 1 -- 172.21.15.116:0/884332783 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09bc01e690 con 0x7f09cc154dd0 2020-11-20T12:18:41.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09d3efe700 1 -- 172.21.15.116:0/884332783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09cc0baa70 con 0x7f09cc154dd0 2020-11-20T12:18:41.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09d3efe700 1 -- 172.21.15.116:0/884332783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09cc05d2a0 con 0x7f09cc154dd0 2020-11-20T12:18:41.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.132+0000 7f09b9ffb700 1 -- 172.21.15.116:0/884332783 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f09bc025030 con 0x7f09cc154dd0 2020-11-20T12:18:41.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.136+0000 7f09b9ffb700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a40557c0 0x7f09a4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.136+0000 7f09b9ffb700 1 -- 172.21.15.116:0/884332783 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f09bc06fe80 con 0x7f09cc154dd0 2020-11-20T12:18:41.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.136+0000 7f09d1c9a700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a40557c0 0x7f09a4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.136+0000 7f09d1c9a700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a40557c0 0x7f09a4057c80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f09cc0bd3a0 tx=0x7f09c8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:41.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.140+0000 7f09b9ffb700 1 -- 172.21.15.116:0/884332783 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f09bc045dd0 con 0x7f09cc154dd0 2020-11-20T12:18:41.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.304+0000 7f09d3efe700 1 -- 172.21.15.116:0/884332783 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f09cc061380 con 0x7f09cc154dd0 2020-11-20T12:18:41.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.304+0000 7f09b9ffb700 1 -- 172.21.15.116:0/884332783 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+10 (secure 0 0 0) 0x7f09bc023030 con 0x7f09cc154dd0 2020-11-20T12:18:41.304 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:41.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 -- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a40557c0 msgr2=0x7f09a4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:41.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a40557c0 0x7f09a4057c80 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f09cc0bd3a0 tx=0x7f09c8009040).stop 2020-11-20T12:18:41.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 -- 172.21.15.116:0/884332783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09cc154dd0 msgr2=0x7f09cc156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:41.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09cc154dd0 0x7f09cc156a00 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f09cc061790 tx=0x7f09bc015000).stop 2020-11-20T12:18:41.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 -- 172.21.15.116:0/884332783 shutdown_connections 2020-11-20T12:18:41.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f09a40557c0 0x7f09a4057c80 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f09cc1551b0 0x7f09cc0ba580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f09cc0bd810 0x7f09cc168d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 --2- 172.21.15.116:0/884332783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f09cc154dd0 0x7f09cc156a00 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 -- 172.21.15.116:0/884332783 >> 172.21.15.116:0/884332783 conn(0x7f09cc05ba80 msgr2=0x7f09cc0a9810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 -- 172.21.15.116:0/884332783 shutdown_connections 2020-11-20T12:18:41.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.308+0000 7f09d3efe700 1 -- 172.21.15.116:0/884332783 wait complete. 2020-11-20T12:18:41.327 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:18:41.328 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:18:41.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.504+0000 7fc2edb1a700 1 Processor -- start 2020-11-20T12:18:41.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.504+0000 7fc2edb1a700 1 -- start start 2020-11-20T12:18:41.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2edb1a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc2e80c38c0 0x7fc2e80c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2edb1a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc2e80a7d20 0x7fc2e81555d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2edb1a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2e80a8620 0x7fc2e80a8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2edb1a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e80a8f50 con 0x7fc2e80c38c0 2020-11-20T12:18:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2edb1a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e8155ac0 con 0x7fc2e80a7d20 2020-11-20T12:18:41.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2edb1a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e81564c0 con 0x7fc2e80a8620 2020-11-20T12:18:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2e77fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc2e80c38c0 0x7fc2e80c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2e77fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc2e80c38c0 0x7fc2e80c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47114/0 (socket says 172.21.15.116:47114) 2020-11-20T12:18:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2e77fe700 1 -- 172.21.15.116:0/3263663675 learned_addr learned my addr 172.21.15.116:0/3263663675 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:41.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2e6ffd700 1 --2- 172.21.15.116:0/3263663675 >> v2:172.21.15.173:3300/0 conn(0x7fc2e80a7d20 0x7fc2e81555d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2e7fff700 1 --2- 172.21.15.116:0/3263663675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2e80a8620 0x7fc2e80a8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2e77fe700 1 -- 172.21.15.116:0/3263663675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2e80a8620 msgr2=0x7fc2e80a8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:41.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.508+0000 7fc2e77fe700 1 --2- 172.21.15.116:0/3263663675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2e80a8620 0x7fc2e80a8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2e77fe700 1 -- 172.21.15.116:0/3263663675 >> v2:172.21.15.173:3300/0 conn(0x7fc2e80a7d20 msgr2=0x7fc2e81555d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:41.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2e77fe700 1 --2- 172.21.15.116:0/3263663675 >> v2:172.21.15.173:3300/0 conn(0x7fc2e80a7d20 0x7fc2e81555d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2e77fe700 1 -- 172.21.15.116:0/3263663675 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e80a81a0 con 0x7fc2e80c38c0 2020-11-20T12:18:41.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2e6ffd700 1 --2- 172.21.15.116:0/3263663675 >> v2:172.21.15.173:3300/0 conn(0x7fc2e80a7d20 0x7fc2e81555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:41.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2e7fff700 1 --2- 172.21.15.116:0/3263663675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2e80a8620 0x7fc2e80a8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:41.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2e77fe700 1 --2- 172.21.15.116:0/3263663675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2e80c38c0 0x7fc2e80c3ca0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fc2e8061690 tx=0x7fc2d8006e50).ready entity=mon.0 client_cookie=930389124e2c7931 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:41.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2e67fc700 1 -- 172.21.15.116:0/3263663675 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8006030 con 0x7fc2e80c38c0 2020-11-20T12:18:41.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2e67fc700 1 -- 172.21.15.116:0/3263663675 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d8006190 con 0x7fc2e80c38c0 2020-11-20T12:18:41.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2e67fc700 1 -- 172.21.15.116:0/3263663675 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8006510 con 0x7fc2e80c38c0 2020-11-20T12:18:41.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2edb1a700 1 -- 172.21.15.116:0/3263663675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2e80c38c0 msgr2=0x7fc2e80c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:41.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2edb1a700 1 --2- 172.21.15.116:0/3263663675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2e80c38c0 0x7fc2e80c3ca0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7fc2e8061690 tx=0x7fc2d8006e50).stop 2020-11-20T12:18:41.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2edb1a700 1 -- 172.21.15.116:0/3263663675 shutdown_connections 2020-11-20T12:18:41.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2edb1a700 1 --2- 172.21.15.116:0/3263663675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2e80a8620 0x7fc2e80a8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2edb1a700 1 --2- 172.21.15.116:0/3263663675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2e80c38c0 0x7fc2e80c3ca0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2edb1a700 1 --2- 172.21.15.116:0/3263663675 >> v2:172.21.15.173:3300/0 conn(0x7fc2e80a7d20 0x7fc2e81555d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2edb1a700 1 -- 172.21.15.116:0/3263663675 >> 172.21.15.116:0/3263663675 conn(0x7fc2e80678a0 msgr2=0x7fc2e8067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2edb1a700 1 -- 172.21.15.116:0/3263663675 shutdown_connections 2020-11-20T12:18:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.512+0000 7fc2edb1a700 1 -- 172.21.15.116:0/3263663675 wait complete. 2020-11-20T12:18:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2edb1a700 1 Processor -- start 2020-11-20T12:18:41.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2edb1a700 1 -- start start 2020-11-20T12:18:41.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2edb1a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2e80a7d20 0x7fc2e80ae320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2edb1a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2e80a8620 0x7fc2e80ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2edb1a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2e80c38c0 0x7fc2e8160b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2edb1a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80c7260 con 0x7fc2e80a8620 2020-11-20T12:18:41.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2edb1a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80ad160 con 0x7fc2e80a7d20 2020-11-20T12:18:41.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2edb1a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e80ac720 con 0x7fc2e80c38c0 2020-11-20T12:18:41.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2e6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2e80a8620 0x7fc2e80ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2e6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2e80a8620 0x7fc2e80ac230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47120/0 (socket says 172.21.15.116:47120) 2020-11-20T12:18:41.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2e6ffd700 1 -- 172.21.15.116:0/1277572503 learned_addr learned my addr 172.21.15.116:0/1277572503 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:41.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2e77fe700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2e80a7d20 0x7fc2e80ae320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2e7fff700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2e80c38c0 0x7fc2e8160b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.516+0000 7fc2e6ffd700 1 -- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2e80c38c0 msgr2=0x7fc2e8160b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:41.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2e6ffd700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2e80c38c0 0x7fc2e8160b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2e6ffd700 1 -- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2e80a7d20 msgr2=0x7fc2e80ae320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:41.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2e6ffd700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2e80a7d20 0x7fc2e80ae320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2e6ffd700 1 -- 172.21.15.116:0/1277572503 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e80ac980 con 0x7fc2e80a8620 2020-11-20T12:18:41.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2e6ffd700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2e80a8620 0x7fc2e80ac230 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fc2dc00db90 tx=0x7fc2dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:41.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2c77fe700 1 -- 172.21.15.116:0/1277572503 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc0098e0 con 0x7fc2e80a8620 2020-11-20T12:18:41.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2c77fe700 1 -- 172.21.15.116:0/1277572503 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc009a40 con 0x7fc2e80a8620 2020-11-20T12:18:41.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2edb1a700 1 -- 172.21.15.116:0/1277572503 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e80acc20 con 0x7fc2e80a8620 2020-11-20T12:18:41.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2c77fe700 1 -- 172.21.15.116:0/1277572503 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc009de0 con 0x7fc2e80a8620 2020-11-20T12:18:41.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2edb1a700 1 -- 172.21.15.116:0/1277572503 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e80c7450 con 0x7fc2e80a8620 2020-11-20T12:18:41.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2edb1a700 1 -- 172.21.15.116:0/1277572503 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2e80c7610 con 0x7fc2e80a8620 2020-11-20T12:18:41.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.520+0000 7fc2c77fe700 1 -- 172.21.15.116:0/1277572503 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc2dc013690 con 0x7fc2e80a8620 2020-11-20T12:18:41.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.524+0000 7fc2c77fe700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2c00557c0 0x7fc2c0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:41.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.524+0000 7fc2c77fe700 1 -- 172.21.15.116:0/1277572503 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc2dc06e690 con 0x7fc2e80a8620 2020-11-20T12:18:41.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.524+0000 7fc2e77fe700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2c00557c0 0x7fc2c0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:41.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.524+0000 7fc2e77fe700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2c00557c0 0x7fc2c0057c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fc2d800b1b0 tx=0x7fc2d8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:41.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.528+0000 7fc2c77fe700 1 -- 172.21.15.116:0/1277572503 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc2dc0444d0 con 0x7fc2e80a8620 2020-11-20T12:18:41.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.732+0000 7fc2edb1a700 1 -- 172.21.15.116:0/1277572503 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2e8069950 con 0x7fc2e80a8620 2020-11-20T12:18:41.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2c77fe700 1 -- 172.21.15.116:0/1277572503 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2850 (secure 0 0 0) 0x7fc2dc06ce60 con 0x7fc2e80a8620 2020-11-20T12:18:41.734 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:41.734 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/785878283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":785878283},{"type":"v1","addr":"172.21.15.173:6837","nonce":785878283}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5952,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1568769426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1568769426},{"type":"v1","addr":"172.21.15.173:6836","nonce":1568769426}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3552490451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3552490451},{"type":"v1","addr":"172.21.15.116:6833","nonce":3552490451}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:18:27.137821+0000","modified":"2020-11-20T12:18:29.161493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3370673707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3370673707},{"type":"v1","addr":"172.21.15.116:6835","nonce":3370673707}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-20T12:18:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 -- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2c00557c0 msgr2=0x7fc2c0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:41.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2c00557c0 0x7fc2c0057c80 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fc2d800b1b0 tx=0x7fc2d8015000).stop 2020-11-20T12:18:41.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 -- 172.21.15.116:0/1277572503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2e80a8620 msgr2=0x7fc2e80ac230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:41.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2e80a8620 0x7fc2e80ac230 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fc2dc00db90 tx=0x7fc2dc008040).stop 2020-11-20T12:18:41.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 -- 172.21.15.116:0/1277572503 shutdown_connections 2020-11-20T12:18:41.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2c00557c0 0x7fc2c0057c80 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc2e80c38c0 0x7fc2e8160b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc2e80a7d20 0x7fc2e80ae320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 --2- 172.21.15.116:0/1277572503 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc2e80a8620 0x7fc2e80ac230 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 -- 172.21.15.116:0/1277572503 >> 172.21.15.116:0/1277572503 conn(0x7fc2e80678a0 msgr2=0x7fc2e80a90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 -- 172.21.15.116:0/1277572503 shutdown_connections 2020-11-20T12:18:41.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:41.736+0000 7fc2edb1a700 1 -- 172.21.15.116:0/1277572503 wait complete. 2020-11-20T12:18:41.739 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 28 2020-11-20T12:18:42.164 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:18:27.137821+0000', 'modified': '2020-11-20T12:18:29.161493+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5970': {'gid': 5970, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/3370673707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3370673707}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3370673707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:18:42.164 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:18:42.165 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:18:42.362 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":5970,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.204472595,"mdsmap_epoch":28,"osdmap_epoch":56,"osdmap_epoch_barrier":56,"uptime":21.671378232999999} 2020-11-20T12:18:42.362 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 5970, "mdsmap_epoch": 28, "osdmap_epoch": 56, "osdmap_epoch_barrier": 56, "rank_uptime": 15.204472595, "state": "up:active", "uptime": 21.671378233, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:18:42.363 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:18:42.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:42.364 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.2 2020-11-20T12:18:42.383 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:18:42.383 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:42.452 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:42.452 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:18:42.473 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:42.474 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:18:42.479 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:18:42.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:42.480 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2020-11-20T12:18:42.509 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:18:42.510 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:18:42.510 INFO:tasks.cephfs.mount:self.client.name = client.3 2020-11-20T12:18:42.510 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2020-11-20T12:18:42.511 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:18:42.511 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:18:42.511 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:18:42.512 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:18:42.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:42.513 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:18:42.529 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:18:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:18:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:18:42.529 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:18:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:42.530 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:42.530 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:42.531 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:42.531 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:42.531 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:18:42.531 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:42.531 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:18:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:18:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:18:42.532 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:42.533 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:18:42.533 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:42.533 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:18:42.533 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:42.533 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:18:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:18:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:18:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:18:42.534 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:42.535 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:18:42.535 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:18:42.535 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:18:42.535 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:42.535 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:18:42.536 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:18:42.536 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:18:42.536 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:18:42.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:42.538 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:18:42.619 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:18:42.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:42.624 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:18:42.641 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:18:42.642 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:18:42.642 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:18:42.642 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:18:42.643 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:18:42.643 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:18:42.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:42.644 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:18:42.645 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:18:42.645 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:18:42.646 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:18:42.646 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:18:42.646 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:18:42.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:42.789 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:18:42.805 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:18:42.805 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:18:42.805 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:18:42.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:42.807 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2020-11-20T12:18:42.836 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:43.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.012+0000 7f7cb2c44700 1 Processor -- start 2020-11-20T12:18:43.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.012+0000 7f7cb2c44700 1 -- start start 2020-11-20T12:18:43.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.012+0000 7f7cb2c44700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7cac062c60 0x7f7cac065d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.012+0000 7f7cb2c44700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cac0bfe20 0x7f7cac1556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.012+0000 7f7cb2c44700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cac0c02b0 con 0x7f7cac062880 2020-11-20T12:18:43.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.012+0000 7f7cb2c44700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cac0c0530 con 0x7f7cac062c60 2020-11-20T12:18:43.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.012+0000 7f7cb2c44700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0c07d0 con 0x7f7cac0bfe20 2020-11-20T12:18:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb1c42700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7cac062880 0x7f7cac0677a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56568/0 (socket says 172.21.15.116:56568) 2020-11-20T12:18:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb1c42700 1 -- 172.21.15.116:0/583087693 learned_addr learned my addr 172.21.15.116:0/583087693 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb1441700 1 --2- 172.21.15.116:0/583087693 >> v2:172.21.15.173:3300/0 conn(0x7f7cac062c60 0x7f7cac065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb2443700 1 --2- 172.21.15.116:0/583087693 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cac0bfe20 0x7f7cac1556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb1441700 1 -- 172.21.15.116:0/583087693 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cac0bfe20 msgr2=0x7f7cac1556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:43.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb1441700 1 --2- 172.21.15.116:0/583087693 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cac0bfe20 0x7f7cac1556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb1441700 1 -- 172.21.15.116:0/583087693 >> v1:172.21.15.116:6789/0 conn(0x7f7cac062880 legacy=0x7f7cac0677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb0c40700 1 -- 172.21.15.116:0/583087693 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181260548 0 0) 0x7f7cac0c02b0 con 0x7f7cac062880 2020-11-20T12:18:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb1441700 1 -- 172.21.15.116:0/583087693 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac0c0a70 con 0x7f7cac062c60 2020-11-20T12:18:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb1441700 1 --2- 172.21.15.116:0/583087693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cac062c60 0x7f7cac065d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f7c9c00b9a0 tx=0x7f7c9c006df0).ready entity=mon.1 client_cookie=9109611539662f62 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:43.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb0c40700 1 -- 172.21.15.116:0/583087693 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c0097b0 con 0x7f7cac062c60 2020-11-20T12:18:43.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb0c40700 1 -- 172.21.15.116:0/583087693 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c9c009910 con 0x7f7cac062c60 2020-11-20T12:18:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb0c40700 1 -- 172.21.15.116:0/583087693 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9c009c90 con 0x7f7cac062c60 2020-11-20T12:18:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb2c44700 1 -- 172.21.15.116:0/583087693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cac062c60 msgr2=0x7f7cac065d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb2c44700 1 --2- 172.21.15.116:0/583087693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cac062c60 0x7f7cac065d40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f7c9c00b9a0 tx=0x7f7c9c006df0).stop 2020-11-20T12:18:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb2c44700 1 -- 172.21.15.116:0/583087693 shutdown_connections 2020-11-20T12:18:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb2c44700 1 --2- 172.21.15.116:0/583087693 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cac0bfe20 0x7f7cac1556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb2c44700 1 --2- 172.21.15.116:0/583087693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cac062c60 0x7f7cac065d40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.016+0000 7f7cb2c44700 1 -- 172.21.15.116:0/583087693 >> 172.21.15.116:0/583087693 conn(0x7f7cac060c40 msgr2=0x7f7cac0c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.020+0000 7f7cb2c44700 1 -- 172.21.15.116:0/583087693 shutdown_connections 2020-11-20T12:18:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.020+0000 7f7cb2c44700 1 -- 172.21.15.116:0/583087693 wait complete. 2020-11-20T12:18:43.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.020+0000 7f7cb2c44700 1 Processor -- start 2020-11-20T12:18:43.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.020+0000 7f7cb2c44700 1 -- start start 2020-11-20T12:18:43.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2c44700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cac062880 0x7f7cac1521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2c44700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cac062c60 0x7f7cac157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2c44700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cac0bfe20 0x7f7cac1581d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2c44700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0a8490 con 0x7f7cac0bfe20 2020-11-20T12:18:43.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2c44700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0a8730 con 0x7f7cac062c60 2020-11-20T12:18:43.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2c44700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cac0a89d0 con 0x7f7cac062880 2020-11-20T12:18:43.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2443700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cac0bfe20 0x7f7cac1581d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2443700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cac0bfe20 0x7f7cac1581d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47140/0 (socket says 172.21.15.116:47140) 2020-11-20T12:18:43.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2443700 1 -- 172.21.15.116:0/3443686931 learned_addr learned my addr 172.21.15.116:0/3443686931 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:43.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb1c42700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cac062880 0x7f7cac1521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb1441700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cac062c60 0x7f7cac157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2443700 1 -- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cac062880 msgr2=0x7f7cac1521f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2443700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cac062880 0x7f7cac1521f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2443700 1 -- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cac062c60 msgr2=0x7f7cac157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2443700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cac062c60 0x7f7cac157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2443700 1 -- 172.21.15.116:0/3443686931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac0a8c70 con 0x7f7cac0bfe20 2020-11-20T12:18:43.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.024+0000 7f7cb2443700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cac0bfe20 0x7f7cac1581d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f7ca400d860 tx=0x7f7ca4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:43.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.028+0000 7f7c99ffb700 1 -- 172.21.15.116:0/3443686931 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca40125f0 con 0x7f7cac0bfe20 2020-11-20T12:18:43.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.028+0000 7f7cb2c44700 1 -- 172.21.15.116:0/3443686931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cac0a8f10 con 0x7f7cac0bfe20 2020-11-20T12:18:43.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.028+0000 7f7c99ffb700 1 -- 172.21.15.116:0/3443686931 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca4012750 con 0x7f7cac0bfe20 2020-11-20T12:18:43.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.028+0000 7f7c99ffb700 1 -- 172.21.15.116:0/3443686931 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4012af0 con 0x7f7cac0bfe20 2020-11-20T12:18:43.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.028+0000 7f7cb2c44700 1 -- 172.21.15.116:0/3443686931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cac0a7830 con 0x7f7cac0bfe20 2020-11-20T12:18:43.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.028+0000 7f7cb2c44700 1 -- 172.21.15.116:0/3443686931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cac05cf30 con 0x7f7cac0bfe20 2020-11-20T12:18:43.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.028+0000 7f7c99ffb700 1 -- 172.21.15.116:0/3443686931 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7ca4013480 con 0x7f7cac0bfe20 2020-11-20T12:18:43.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.028+0000 7f7c99ffb700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c840557c0 0x7f7c84057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.028+0000 7f7c99ffb700 1 -- 172.21.15.116:0/3443686931 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7ca406e9d0 con 0x7f7cac0bfe20 2020-11-20T12:18:43.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.032+0000 7f7cb1c42700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c840557c0 0x7f7c84057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.032+0000 7f7cb1c42700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c840557c0 0x7f7c84057c80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f7ca8007240 tx=0x7f7ca8001320).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:43.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.036+0000 7f7c99ffb700 1 -- 172.21.15.116:0/3443686931 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7ca4044810 con 0x7f7cac0bfe20 2020-11-20T12:18:43.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.196+0000 7f7cb2c44700 1 -- 172.21.15.116:0/3443686931 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7cac061230 con 0x7f7cac0bfe20 2020-11-20T12:18:43.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.196+0000 7f7c99ffb700 1 -- 172.21.15.116:0/3443686931 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+16876 (secure 0 0 0) 0x7f7ca4012e60 con 0x7f7cac0bfe20 2020-11-20T12:18:43.197 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:43.197 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":56,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:18:28.142253+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:18:25.558441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-20T12:18:26.151785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:43.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 -- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c840557c0 msgr2=0x7f7c84057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c840557c0 0x7f7c84057c80 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7f7ca8007240 tx=0x7f7ca8001320).stop 2020-11-20T12:18:43.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 -- 172.21.15.116:0/3443686931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cac0bfe20 msgr2=0x7f7cac1581d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cac0bfe20 0x7f7cac1581d0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f7ca400d860 tx=0x7f7ca4008040).stop 2020-11-20T12:18:43.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 -- 172.21.15.116:0/3443686931 shutdown_connections 2020-11-20T12:18:43.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c840557c0 0x7f7c84057c80 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7cac0bfe20 0x7f7cac1581d0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7cac062880 0x7f7cac1521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 --2- 172.21.15.116:0/3443686931 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7cac062c60 0x7f7cac157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 -- 172.21.15.116:0/3443686931 >> 172.21.15.116:0/3443686931 conn(0x7f7cac060c40 msgr2=0x7f7cac0663a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:43.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 -- 172.21.15.116:0/3443686931 shutdown_connections 2020-11-20T12:18:43.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.200+0000 7f7cb2c44700 1 -- 172.21.15.116:0/3443686931 wait complete. 2020-11-20T12:18:43.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:18:25.558441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:18:26.151785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:18:43.225 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:18:43.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.408+0000 7fcc68bae700 1 Processor -- start 2020-11-20T12:18:43.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.408+0000 7fcc68bae700 1 -- start start 2020-11-20T12:18:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.408+0000 7fcc68bae700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fcc64151cd0 0x7fcc64066480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.408+0000 7fcc68bae700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc640b93c0 0x7fcc64066970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.408+0000 7fcc68bae700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc640bf510 con 0x7fcc641518f0 2020-11-20T12:18:43.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.408+0000 7fcc68bae700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc640bf7b0 con 0x7fcc64151cd0 2020-11-20T12:18:43.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.408+0000 7fcc68bae700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640bfa50 con 0x7fcc640b93c0 2020-11-20T12:18:43.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc637fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fcc641518f0 0x7fcc640bed20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56582/0 (socket says 172.21.15.116:56582) 2020-11-20T12:18:43.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc637fe700 1 -- 172.21.15.116:0/3700439618 learned_addr learned my addr 172.21.15.116:0/3700439618 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:43.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc63fff700 1 --2- 172.21.15.116:0/3700439618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc640b93c0 0x7fcc64066970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc62ffd700 1 --2- 172.21.15.116:0/3700439618 >> v2:172.21.15.173:3300/0 conn(0x7fcc64151cd0 0x7fcc64066480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc627fc700 1 -- 172.21.15.116:0/3700439618 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3800254409 0 0) 0x7fcc640bf510 con 0x7fcc641518f0 2020-11-20T12:18:43.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc627fc700 1 -- 172.21.15.116:0/3700439618 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc50003240 con 0x7fcc641518f0 2020-11-20T12:18:43.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc63fff700 1 -- 172.21.15.116:0/3700439618 >> v2:172.21.15.173:3300/0 conn(0x7fcc64151cd0 msgr2=0x7fcc64066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:43.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc63fff700 1 --2- 172.21.15.116:0/3700439618 >> v2:172.21.15.173:3300/0 conn(0x7fcc64151cd0 0x7fcc64066480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc627fc700 1 -- 172.21.15.116:0/3700439618 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1213230972 0 0) 0x7fcc50003240 con 0x7fcc641518f0 2020-11-20T12:18:43.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc63fff700 1 -- 172.21.15.116:0/3700439618 >> v1:172.21.15.116:6789/0 conn(0x7fcc641518f0 legacy=0x7fcc640bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc63fff700 1 -- 172.21.15.116:0/3700439618 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640bfcf0 con 0x7fcc640b93c0 2020-11-20T12:18:43.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc63fff700 1 --2- 172.21.15.116:0/3700439618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc640b93c0 0x7fcc64066970 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fcc5c00b9a0 tx=0x7fcc5c006df0).ready entity=mon.2 client_cookie=41c74b297f134091 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:43.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc627fc700 1 -- 172.21.15.116:0/3700439618 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54001860 con 0x7fcc640b93c0 2020-11-20T12:18:43.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc627fc700 1 -- 172.21.15.116:0/3700439618 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c011400 con 0x7fcc640b93c0 2020-11-20T12:18:43.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.412+0000 7fcc627fc700 1 -- 172.21.15.116:0/3700439618 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c011780 con 0x7fcc640b93c0 2020-11-20T12:18:43.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.416+0000 7fcc68bae700 1 -- 172.21.15.116:0/3700439618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc640b93c0 msgr2=0x7fcc64066970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.416+0000 7fcc68bae700 1 --2- 172.21.15.116:0/3700439618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc640b93c0 0x7fcc64066970 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7fcc5c00b9a0 tx=0x7fcc5c006df0).stop 2020-11-20T12:18:43.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.416+0000 7fcc68bae700 1 -- 172.21.15.116:0/3700439618 shutdown_connections 2020-11-20T12:18:43.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.416+0000 7fcc68bae700 1 --2- 172.21.15.116:0/3700439618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc640b93c0 0x7fcc64066970 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.416+0000 7fcc68bae700 1 --2- 172.21.15.116:0/3700439618 >> v2:172.21.15.173:3300/0 conn(0x7fcc64151cd0 0x7fcc64066480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.416+0000 7fcc68bae700 1 -- 172.21.15.116:0/3700439618 >> 172.21.15.116:0/3700439618 conn(0x7fcc64068350 msgr2=0x7fcc640627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:43.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.416+0000 7fcc68bae700 1 -- 172.21.15.116:0/3700439618 shutdown_connections 2020-11-20T12:18:43.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.416+0000 7fcc68bae700 1 -- 172.21.15.116:0/3700439618 wait complete. 2020-11-20T12:18:43.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc68bae700 1 Processor -- start 2020-11-20T12:18:43.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc68bae700 1 -- start start 2020-11-20T12:18:43.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc68bae700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc640b93c0 0x7fcc640bdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc68bae700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc641518f0 0x7fcc64161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc68bae700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc64151cd0 0x7fcc640bb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc68bae700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc641535b0 con 0x7fcc64151cd0 2020-11-20T12:18:43.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc68bae700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc641537e0 con 0x7fcc640b93c0 2020-11-20T12:18:43.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc68bae700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64153a80 con 0x7fcc641518f0 2020-11-20T12:18:43.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc63fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc64151cd0 0x7fcc640bb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc63fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc64151cd0 0x7fcc640bb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47154/0 (socket says 172.21.15.116:47154) 2020-11-20T12:18:43.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc63fff700 1 -- 172.21.15.116:0/3243767906 learned_addr learned my addr 172.21.15.116:0/3243767906 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:43.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc637fe700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc640b93c0 0x7fcc640bdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.420+0000 7fcc62ffd700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc641518f0 0x7fcc64161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc63fff700 1 -- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc641518f0 msgr2=0x7fcc64161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc63fff700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc641518f0 0x7fcc64161ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc63fff700 1 -- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc640b93c0 msgr2=0x7fcc640bdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc63fff700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc640b93c0 0x7fcc640bdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc63fff700 1 -- 172.21.15.116:0/3243767906 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc64153d20 con 0x7fcc64151cd0 2020-11-20T12:18:43.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc63fff700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc64151cd0 0x7fcc640bb6c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7fcc5c006fd0 tx=0x7fcc5c013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:43.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc437fe700 1 -- 172.21.15.116:0/3243767906 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c010be0 con 0x7fcc64151cd0 2020-11-20T12:18:43.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc437fe700 1 -- 172.21.15.116:0/3243767906 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c010d40 con 0x7fcc64151cd0 2020-11-20T12:18:43.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc68bae700 1 -- 172.21.15.116:0/3243767906 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc64153fc0 con 0x7fcc64151cd0 2020-11-20T12:18:43.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc437fe700 1 -- 172.21.15.116:0/3243767906 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c01e670 con 0x7fcc64151cd0 2020-11-20T12:18:43.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc68bae700 1 -- 172.21.15.116:0/3243767906 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc640be2e0 con 0x7fcc64151cd0 2020-11-20T12:18:43.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc68bae700 1 -- 172.21.15.116:0/3243767906 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc64154150 con 0x7fcc64151cd0 2020-11-20T12:18:43.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.424+0000 7fcc437fe700 1 -- 172.21.15.116:0/3243767906 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fcc5c025030 con 0x7fcc64151cd0 2020-11-20T12:18:43.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.428+0000 7fcc437fe700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcc3c0557f0 0x7fcc3c057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.428+0000 7fcc437fe700 1 -- 172.21.15.116:0/3243767906 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcc5c06f830 con 0x7fcc64151cd0 2020-11-20T12:18:43.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.428+0000 7fcc637fe700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcc3c0557f0 0x7fcc3c057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.428+0000 7fcc637fe700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcc3c0557f0 0x7fcc3c057cb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fcc54001520 tx=0x7fcc54002110).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:43.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.432+0000 7fcc437fe700 1 -- 172.21.15.116:0/3243767906 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcc5c045780 con 0x7fcc64151cd0 2020-11-20T12:18:43.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.592+0000 7fcc68bae700 1 -- 172.21.15.116:0/3243767906 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcc640be2e0 con 0x7fcc64151cd0 2020-11-20T12:18:43.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc437fe700 1 -- 172.21.15.116:0/3243767906 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v56) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcc640be2e0 con 0x7fcc64151cd0 2020-11-20T12:18:43.594 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:18:43.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 -- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcc3c0557f0 msgr2=0x7fcc3c057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcc3c0557f0 0x7fcc3c057cb0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fcc54001520 tx=0x7fcc54002110).stop 2020-11-20T12:18:43.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 -- 172.21.15.116:0/3243767906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc64151cd0 msgr2=0x7fcc640bb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc64151cd0 0x7fcc640bb6c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7fcc5c006fd0 tx=0x7fcc5c013000).stop 2020-11-20T12:18:43.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 -- 172.21.15.116:0/3243767906 shutdown_connections 2020-11-20T12:18:43.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcc3c0557f0 0x7fcc3c057cb0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcc64151cd0 0x7fcc640bb6c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcc640b93c0 0x7fcc640bdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 --2- 172.21.15.116:0/3243767906 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcc641518f0 0x7fcc64161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 -- 172.21.15.116:0/3243767906 >> 172.21.15.116:0/3243767906 conn(0x7fcc64068350 msgr2=0x7fcc64066330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:43.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 -- 172.21.15.116:0/3243767906 shutdown_connections 2020-11-20T12:18:43.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.596+0000 7fcc68bae700 1 -- 172.21.15.116:0/3243767906 wait complete. 2020-11-20T12:18:43.620 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:18:43.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.804+0000 7f661a448700 1 Processor -- start 2020-11-20T12:18:43.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.804+0000 7f661a448700 1 -- start start 2020-11-20T12:18:43.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.804+0000 7f661a448700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6614151940 0x7f66140a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.804+0000 7f661a448700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6614151d20 0x7f6614062f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.804+0000 7f661a448700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140c7030 0x7f6614064a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.804+0000 7f661a448700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66140a7220 con 0x7f6614151940 2020-11-20T12:18:43.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.804+0000 7f661a448700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6614062530 con 0x7f6614151d20 2020-11-20T12:18:43.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.804+0000 7f661a448700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66140627d0 con 0x7f66140c7030 2020-11-20T12:18:43.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.804+0000 7f6613fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6614151940 0x7f66140a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f6613fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6614151940 0x7f66140a6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47162/0 (socket says 172.21.15.116:47162) 2020-11-20T12:18:43.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f66137fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6614151d20 0x7f6614062f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f6613fff700 1 -- 172.21.15.116:0/1936447263 learned_addr learned my addr 172.21.15.116:0/1936447263 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:43.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f66189e5700 1 --2- 172.21.15.116:0/1936447263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140c7030 0x7f6614064a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f6613fff700 1 -- 172.21.15.116:0/1936447263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140c7030 msgr2=0x7f6614064a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:43.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f6613fff700 1 --2- 172.21.15.116:0/1936447263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140c7030 0x7f6614064a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f6613fff700 1 -- 172.21.15.116:0/1936447263 >> v2:172.21.15.173:3300/0 conn(0x7f6614151d20 msgr2=0x7f6614062f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:43.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f6613fff700 1 --2- 172.21.15.116:0/1936447263 >> v2:172.21.15.173:3300/0 conn(0x7f6614151d20 0x7f6614062f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f6613fff700 1 -- 172.21.15.116:0/1936447263 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6614062a70 con 0x7f6614151940 2020-11-20T12:18:43.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f6613fff700 1 --2- 172.21.15.116:0/1936447263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6614151940 0x7f66140a6b80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f661405c7c0 tx=0x7f6604006eb0).ready entity=mon.0 client_cookie=551db522ce48a306 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:43.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f6612ffd700 1 -- 172.21.15.116:0/1936447263 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604009810 con 0x7f6614151940 2020-11-20T12:18:43.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f6612ffd700 1 -- 172.21.15.116:0/1936447263 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6604009970 con 0x7f6614151940 2020-11-20T12:18:43.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f6612ffd700 1 -- 172.21.15.116:0/1936447263 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6604009cf0 con 0x7f6614151940 2020-11-20T12:18:43.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f661a448700 1 -- 172.21.15.116:0/1936447263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6614151940 msgr2=0x7f66140a6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f661a448700 1 --2- 172.21.15.116:0/1936447263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6614151940 0x7f66140a6b80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f661405c7c0 tx=0x7f6604006eb0).stop 2020-11-20T12:18:43.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f661a448700 1 -- 172.21.15.116:0/1936447263 shutdown_connections 2020-11-20T12:18:43.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f661a448700 1 --2- 172.21.15.116:0/1936447263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140c7030 0x7f6614064a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f661a448700 1 --2- 172.21.15.116:0/1936447263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6614151940 0x7f66140a6b80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f661a448700 1 --2- 172.21.15.116:0/1936447263 >> v2:172.21.15.173:3300/0 conn(0x7f6614151d20 0x7f6614062f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f661a448700 1 -- 172.21.15.116:0/1936447263 >> 172.21.15.116:0/1936447263 conn(0x7f6614060c00 msgr2=0x7f66140c66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:43.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f661a448700 1 -- 172.21.15.116:0/1936447263 shutdown_connections 2020-11-20T12:18:43.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.808+0000 7f661a448700 1 -- 172.21.15.116:0/1936447263 wait complete. 2020-11-20T12:18:43.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.812+0000 7f661a448700 1 Processor -- start 2020-11-20T12:18:43.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.812+0000 7f661a448700 1 -- start start 2020-11-20T12:18:43.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f661a448700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140c7030 0x7f6614150810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f661a448700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6614151940 0x7f6614155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f661a448700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6614151d20 0x7f661409d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f661a448700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66140a96a0 con 0x7f6614151d20 2020-11-20T12:18:43.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f661a448700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66140a9940 con 0x7f6614151940 2020-11-20T12:18:43.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f661a448700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66140a9be0 con 0x7f66140c7030 2020-11-20T12:18:43.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f66189e5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6614151d20 0x7f661409d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f66189e5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6614151d20 0x7f661409d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47166/0 (socket says 172.21.15.116:47166) 2020-11-20T12:18:43.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f66189e5700 1 -- 172.21.15.116:0/3209050822 learned_addr learned my addr 172.21.15.116:0/3209050822 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:43.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f6613fff700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140c7030 0x7f6614150810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f66137fe700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6614151940 0x7f6614155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f66189e5700 1 -- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140c7030 msgr2=0x7f6614150810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f66189e5700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140c7030 0x7f6614150810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f66189e5700 1 -- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6614151940 msgr2=0x7f6614155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f66189e5700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6614151940 0x7f6614155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f66189e5700 1 -- 172.21.15.116:0/3209050822 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66140a9e80 con 0x7f6614151d20 2020-11-20T12:18:43.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f66189e5700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6614151d20 0x7f661409d440 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f660c00eb70 tx=0x7f660c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f65f3fff700 1 -- 172.21.15.116:0/3209050822 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c00c670 con 0x7f6614151d20 2020-11-20T12:18:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f65f3fff700 1 -- 172.21.15.116:0/3209050822 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660c00c7d0 con 0x7f6614151d20 2020-11-20T12:18:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f661a448700 1 -- 172.21.15.116:0/3209050822 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6614155700 con 0x7f6614151d20 2020-11-20T12:18:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f65f3fff700 1 -- 172.21.15.116:0/3209050822 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660c00cb70 con 0x7f6614151d20 2020-11-20T12:18:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.816+0000 7f661a448700 1 -- 172.21.15.116:0/3209050822 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6614150d80 con 0x7f6614151d20 2020-11-20T12:18:43.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.820+0000 7f661a448700 1 -- 172.21.15.116:0/3209050822 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f661405d2a0 con 0x7f6614151d20 2020-11-20T12:18:43.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.820+0000 7f65f3fff700 1 -- 172.21.15.116:0/3209050822 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f660c01d460 con 0x7f6614151d20 2020-11-20T12:18:43.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.820+0000 7f65f3fff700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65ec0557c0 0x7f65ec057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:43.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.820+0000 7f65f3fff700 1 -- 172.21.15.116:0/3209050822 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f660c06e670 con 0x7f6614151d20 2020-11-20T12:18:43.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.824+0000 7f6613fff700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65ec0557c0 0x7f65ec057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:43.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.824+0000 7f6613fff700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65ec0557c0 0x7f65ec057c80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f660400b1b0 tx=0x7f6604015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:43.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.828+0000 7f65f3fff700 1 -- 172.21.15.116:0/3209050822 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f660c0444b0 con 0x7f6614151d20 2020-11-20T12:18:43.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.988+0000 7f661a448700 1 -- 172.21.15.116:0/3209050822 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f66140635b0 con 0x7f6614151d20 2020-11-20T12:18:43.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.988+0000 7f65f3fff700 1 -- 172.21.15.116:0/3209050822 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v56) v1 ==== 92+0+10 (secure 0 0 0) 0x7f660c047ad0 con 0x7f6614151d20 2020-11-20T12:18:43.987 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:43.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 -- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65ec0557c0 msgr2=0x7f65ec057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65ec0557c0 0x7f65ec057c80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f660400b1b0 tx=0x7f6604015000).stop 2020-11-20T12:18:43.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 -- 172.21.15.116:0/3209050822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6614151d20 msgr2=0x7f661409d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:43.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6614151d20 0x7f661409d440 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f660c00eb70 tx=0x7f660c008040).stop 2020-11-20T12:18:43.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 -- 172.21.15.116:0/3209050822 shutdown_connections 2020-11-20T12:18:43.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f65ec0557c0 0x7f65ec057c80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6614151d20 0x7f661409d440 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66140c7030 0x7f6614150810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 --2- 172.21.15.116:0/3209050822 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6614151940 0x7f6614155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:43.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 -- 172.21.15.116:0/3209050822 >> 172.21.15.116:0/3209050822 conn(0x7f6614060c00 msgr2=0x7f6614061c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:43.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 -- 172.21.15.116:0/3209050822 shutdown_connections 2020-11-20T12:18:43.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:43.992+0000 7f661a448700 1 -- 172.21.15.116:0/3209050822 wait complete. 2020-11-20T12:18:44.014 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:18:44.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.188+0000 7f1b1523b700 1 Processor -- start 2020-11-20T12:18:44.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.188+0000 7f1b1523b700 1 -- start start 2020-11-20T12:18:44.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.192+0000 7f1b1523b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1b10154dd0 0x7f1b10156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.192+0000 7f1b1523b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b100bdfc0 con 0x7f1b101551b0 2020-11-20T12:18:44.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.192+0000 7f1b1523b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b100bdd20 con 0x7f1b100bd810 2020-11-20T12:18:44.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.192+0000 7f1b1523b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b10155780 con 0x7f1b10154dd0 2020-11-20T12:18:44.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.192+0000 7f1b0ed9d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1b10154dd0 0x7f1b10156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.192+0000 7f1b0ed9d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1b10154dd0 0x7f1b10156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:46224/0 (socket says 172.21.15.116:46224) 2020-11-20T12:18:44.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.192+0000 7f1b0ed9d700 1 -- 172.21.15.116:0/628700125 learned_addr learned my addr 172.21.15.116:0/628700125 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:44.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.192+0000 7f1b0f59e700 1 --1- 172.21.15.116:0/628700125 >> v1:172.21.15.116:6789/0 conn(0x7f1b101551b0 0x7f1b100a6780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56610/0 (socket says 172.21.15.116:56610) 2020-11-20T12:18:44.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.192+0000 7f1b07fff700 1 --1- 172.21.15.116:0/628700125 >> v1:172.21.15.173:6789/0 conn(0x7f1b100bd810 0x7f1b100be340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:52836/0 (socket says 172.21.15.116:52836) 2020-11-20T12:18:44.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.192+0000 7f1b0ed9d700 1 -- 172.21.15.116:0/628700125 >> v1:172.21.15.173:6789/0 conn(0x7f1b100bd810 legacy=0x7f1b100be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.192+0000 7f1b0ed9d700 1 -- 172.21.15.116:0/628700125 >> v1:172.21.15.116:6789/0 conn(0x7f1b101551b0 legacy=0x7f1b100a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b0ed9d700 1 -- 172.21.15.116:0/628700125 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b10155a20 con 0x7f1b10154dd0 2020-11-20T12:18:44.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b0ed9d700 1 --2- 172.21.15.116:0/628700125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b10154dd0 0x7f1b10156160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f1b00006fd0 tx=0x7f1b00006eb0).ready entity=mon.1 client_cookie=dde12ab45bf429c2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b0e59c700 1 -- 172.21.15.116:0/628700125 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b000098d0 con 0x7f1b10154dd0 2020-11-20T12:18:44.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b0e59c700 1 -- 172.21.15.116:0/628700125 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b00009a30 con 0x7f1b10154dd0 2020-11-20T12:18:44.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b0e59c700 1 -- 172.21.15.116:0/628700125 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00009db0 con 0x7f1b10154dd0 2020-11-20T12:18:44.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b1523b700 1 -- 172.21.15.116:0/628700125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b10154dd0 msgr2=0x7f1b10156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b1523b700 1 --2- 172.21.15.116:0/628700125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b10154dd0 0x7f1b10156160 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f1b00006fd0 tx=0x7f1b00006eb0).stop 2020-11-20T12:18:44.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b1523b700 1 -- 172.21.15.116:0/628700125 shutdown_connections 2020-11-20T12:18:44.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b1523b700 1 --2- 172.21.15.116:0/628700125 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b10154dd0 0x7f1b10156160 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b1523b700 1 -- 172.21.15.116:0/628700125 >> 172.21.15.116:0/628700125 conn(0x7f1b1005ba80 msgr2=0x7f1b100c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b1523b700 1 -- 172.21.15.116:0/628700125 shutdown_connections 2020-11-20T12:18:44.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.196+0000 7f1b1523b700 1 -- 172.21.15.116:0/628700125 wait complete. 2020-11-20T12:18:44.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.200+0000 7f1b1523b700 1 Processor -- start 2020-11-20T12:18:44.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.200+0000 7f1b1523b700 1 -- start start 2020-11-20T12:18:44.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.200+0000 7f1b1523b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b100bd810 0x7f1b10164a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.200+0000 7f1b1523b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b10154dd0 0x7f1b101516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.200+0000 7f1b1523b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b101551b0 0x7f1b1009d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.200+0000 7f1b1523b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b10151bd0 con 0x7f1b10154dd0 2020-11-20T12:18:44.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.200+0000 7f1b1523b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1009de10 con 0x7f1b100bd810 2020-11-20T12:18:44.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.200+0000 7f1b1523b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b10165060 con 0x7f1b101551b0 2020-11-20T12:18:44.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b07fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b10154dd0 0x7f1b101516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b07fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b10154dd0 0x7f1b101516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47182/0 (socket says 172.21.15.116:47182) 2020-11-20T12:18:44.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b07fff700 1 -- 172.21.15.116:0/2058641057 learned_addr learned my addr 172.21.15.116:0/2058641057 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b0ed9d700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b100bd810 0x7f1b10164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b0f59e700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b101551b0 0x7f1b1009d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b07fff700 1 -- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b101551b0 msgr2=0x7f1b1009d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b07fff700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b101551b0 0x7f1b1009d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b07fff700 1 -- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b100bd810 msgr2=0x7f1b10164a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b07fff700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b100bd810 0x7f1b10164a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b07fff700 1 -- 172.21.15.116:0/2058641057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b10165300 con 0x7f1b10154dd0 2020-11-20T12:18:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b0ed9d700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b100bd810 0x7f1b10164a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b07fff700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b10154dd0 0x7f1b101516e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f1af800c6e0 tx=0x7f1af8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b06ffd700 1 -- 172.21.15.116:0/2058641057 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af80072e0 con 0x7f1b10154dd0 2020-11-20T12:18:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b06ffd700 1 -- 172.21.15.116:0/2058641057 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1af8005530 con 0x7f1b10154dd0 2020-11-20T12:18:44.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b1523b700 1 -- 172.21.15.116:0/2058641057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b101655a0 con 0x7f1b10154dd0 2020-11-20T12:18:44.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b06ffd700 1 -- 172.21.15.116:0/2058641057 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af80058b0 con 0x7f1b10154dd0 2020-11-20T12:18:44.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.204+0000 7f1b1523b700 1 -- 172.21.15.116:0/2058641057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b101658c0 con 0x7f1b10154dd0 2020-11-20T12:18:44.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.208+0000 7f1b1523b700 1 -- 172.21.15.116:0/2058641057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b1005d2a0 con 0x7f1b10154dd0 2020-11-20T12:18:44.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.208+0000 7f1b06ffd700 1 -- 172.21.15.116:0/2058641057 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1af8010070 con 0x7f1b10154dd0 2020-11-20T12:18:44.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.208+0000 7f1b06ffd700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1aec0557c0 0x7f1aec057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.208+0000 7f1b06ffd700 1 -- 172.21.15.116:0/2058641057 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1af806dd70 con 0x7f1b10154dd0 2020-11-20T12:18:44.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.212+0000 7f1b0ed9d700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1aec0557c0 0x7f1aec057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.212+0000 7f1b0ed9d700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1aec0557c0 0x7f1aec057c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f1b0000b1b0 tx=0x7f1b00015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.216+0000 7f1b06ffd700 1 -- 172.21.15.116:0/2058641057 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1af8043c10 con 0x7f1b10154dd0 2020-11-20T12:18:44.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.380+0000 7f1b1523b700 1 -- 172.21.15.116:0/2058641057 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1b10061380 con 0x7f1b10154dd0 2020-11-20T12:18:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.380+0000 7f1b06ffd700 1 -- 172.21.15.116:0/2058641057 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1af8047230 con 0x7f1b10154dd0 2020-11-20T12:18:44.381 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 -- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1aec0557c0 msgr2=0x7f1aec057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1aec0557c0 0x7f1aec057c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f1b0000b1b0 tx=0x7f1b00015000).stop 2020-11-20T12:18:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 -- 172.21.15.116:0/2058641057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b10154dd0 msgr2=0x7f1b101516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b10154dd0 0x7f1b101516e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f1af800c6e0 tx=0x7f1af8007c10).stop 2020-11-20T12:18:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 -- 172.21.15.116:0/2058641057 shutdown_connections 2020-11-20T12:18:44.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1aec0557c0 0x7f1aec057c80 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1b101551b0 0x7f1b1009d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1b100bd810 0x7f1b10164a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 --2- 172.21.15.116:0/2058641057 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1b10154dd0 0x7f1b101516e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 -- 172.21.15.116:0/2058641057 >> 172.21.15.116:0/2058641057 conn(0x7f1b1005ba80 msgr2=0x7f1b100a9880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 -- 172.21.15.116:0/2058641057 shutdown_connections 2020-11-20T12:18:44.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.384+0000 7f1b1523b700 1 -- 172.21.15.116:0/2058641057 wait complete. 2020-11-20T12:18:44.406 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:18:44.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.580+0000 7f1280f5c700 1 Processor -- start 2020-11-20T12:18:44.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.580+0000 7f1280f5c700 1 -- start start 2020-11-20T12:18:44.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f1280f5c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f127c066c10 0x7f127c153060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f1280f5c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f127c061390 con 0x7f127c14bf40 2020-11-20T12:18:44.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f1280f5c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f127c061630 con 0x7f127c066090 2020-11-20T12:18:44.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f1280f5c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f127c0618d0 con 0x7f127c066c10 2020-11-20T12:18:44.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127affd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f127c066c10 0x7f127c153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127b7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f127c14bf40 0x7f127c061c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56620/0 (socket says 172.21.15.116:56620) 2020-11-20T12:18:44.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127b7fe700 1 -- 172.21.15.116:0/3521285843 learned_addr learned my addr 172.21.15.116:0/3521285843 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127bfff700 1 --1- 172.21.15.116:0/3521285843 >> v1:172.21.15.173:6789/0 conn(0x7f127c066090 0x7f127c0bbce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:52852/0 (socket says 172.21.15.116:52852) 2020-11-20T12:18:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127a7fc700 1 -- 172.21.15.116:0/3521285843 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821874237 0 0) 0x7f127c061390 con 0x7f127c14bf40 2020-11-20T12:18:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127affd700 1 -- 172.21.15.116:0/3521285843 >> v1:172.21.15.173:6789/0 conn(0x7f127c066090 legacy=0x7f127c0bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127affd700 1 -- 172.21.15.116:0/3521285843 >> v1:172.21.15.116:6789/0 conn(0x7f127c14bf40 legacy=0x7f127c061c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127affd700 1 -- 172.21.15.116:0/3521285843 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c0bb5a0 con 0x7f127c066c10 2020-11-20T12:18:44.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127affd700 1 --2- 172.21.15.116:0/3521285843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f127c066c10 0x7f127c153060 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f127000b9a0 tx=0x7f1270006df0).ready entity=mon.0 client_cookie=26f572975c970fe8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127a7fc700 1 -- 172.21.15.116:0/3521285843 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12700097b0 con 0x7f127c066c10 2020-11-20T12:18:44.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127a7fc700 1 -- 172.21.15.116:0/3521285843 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1270009910 con 0x7f127c066c10 2020-11-20T12:18:44.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.584+0000 7f127a7fc700 1 -- 172.21.15.116:0/3521285843 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1270009c90 con 0x7f127c066c10 2020-11-20T12:18:44.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.588+0000 7f1280f5c700 1 -- 172.21.15.116:0/3521285843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f127c066c10 msgr2=0x7f127c153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.588+0000 7f1280f5c700 1 --2- 172.21.15.116:0/3521285843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f127c066c10 0x7f127c153060 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f127000b9a0 tx=0x7f1270006df0).stop 2020-11-20T12:18:44.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.588+0000 7f1280f5c700 1 -- 172.21.15.116:0/3521285843 shutdown_connections 2020-11-20T12:18:44.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.588+0000 7f1280f5c700 1 --2- 172.21.15.116:0/3521285843 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f127c066c10 0x7f127c153060 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.588+0000 7f1280f5c700 1 -- 172.21.15.116:0/3521285843 >> 172.21.15.116:0/3521285843 conn(0x7f127c060170 msgr2=0x7f127c060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.588+0000 7f1280f5c700 1 -- 172.21.15.116:0/3521285843 shutdown_connections 2020-11-20T12:18:44.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.588+0000 7f1280f5c700 1 -- 172.21.15.116:0/3521285843 wait complete. 2020-11-20T12:18:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.588+0000 7f1280f5c700 1 Processor -- start 2020-11-20T12:18:44.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f1280f5c700 1 -- start start 2020-11-20T12:18:44.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f1280f5c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f127c066090 0x7f127c0a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f1280f5c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f127c14bf40 0x7f127c0aace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f1280f5c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f127c0ab1d0 0x7f127c160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f1280f5c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0c4640 con 0x7f127c066090 2020-11-20T12:18:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f1280f5c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0a9f00 con 0x7f127c14bf40 2020-11-20T12:18:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f1280f5c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127c0aa1a0 con 0x7f127c0ab1d0 2020-11-20T12:18:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f127b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f127c066090 0x7f127c0a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f127b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f127c066090 0x7f127c0a95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47192/0 (socket says 172.21.15.116:47192) 2020-11-20T12:18:44.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f127b7fe700 1 -- 172.21.15.116:0/969703829 learned_addr learned my addr 172.21.15.116:0/969703829 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:44.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f127affd700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f127c14bf40 0x7f127c0aace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f127bfff700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f127c0ab1d0 0x7f127c160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f127b7fe700 1 -- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f127c0ab1d0 msgr2=0x7f127c160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f127b7fe700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f127c0ab1d0 0x7f127c160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f127b7fe700 1 -- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f127c14bf40 msgr2=0x7f127c0aace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f127b7fe700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f127c14bf40 0x7f127c0aace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f127b7fe700 1 -- 172.21.15.116:0/969703829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f127c0aa440 con 0x7f127c066090 2020-11-20T12:18:44.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f127b7fe700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f127c066090 0x7f127c0a95f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f126c00c760 tx=0x7f126c007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.592+0000 7f125b7fe700 1 -- 172.21.15.116:0/969703829 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c0072e0 con 0x7f127c066090 2020-11-20T12:18:44.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.596+0000 7f1280f5c700 1 -- 172.21.15.116:0/969703829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f127c0aa6e0 con 0x7f127c066090 2020-11-20T12:18:44.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.596+0000 7f125b7fe700 1 -- 172.21.15.116:0/969703829 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f126c007440 con 0x7f127c066090 2020-11-20T12:18:44.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.596+0000 7f125b7fe700 1 -- 172.21.15.116:0/969703829 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f126c005840 con 0x7f127c066090 2020-11-20T12:18:44.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.596+0000 7f1280f5c700 1 -- 172.21.15.116:0/969703829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f127c161f50 con 0x7f127c066090 2020-11-20T12:18:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.596+0000 7f1280f5c700 1 -- 172.21.15.116:0/969703829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f127c05cf30 con 0x7f127c066090 2020-11-20T12:18:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.596+0000 7f125b7fe700 1 -- 172.21.15.116:0/969703829 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f126c010070 con 0x7f127c066090 2020-11-20T12:18:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.596+0000 7f125b7fe700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12540557c0 0x7f1254057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.596+0000 7f125b7fe700 1 -- 172.21.15.116:0/969703829 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f126c06dea0 con 0x7f127c066090 2020-11-20T12:18:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.596+0000 7f127affd700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12540557c0 0x7f1254057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.600+0000 7f127affd700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12540557c0 0x7f1254057c80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f127c067050 tx=0x7f1270015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.604+0000 7f125b7fe700 1 -- 172.21.15.116:0/969703829 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f126c043dc0 con 0x7f127c066090 2020-11-20T12:18:44.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.764+0000 7f1280f5c700 1 -- 172.21.15.116:0/969703829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f127c14b800 con 0x7f127c066090 2020-11-20T12:18:44.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.764+0000 7f125b7fe700 1 -- 172.21.15.116:0/969703829 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+10 (secure 0 0 0) 0x7f126c0473e0 con 0x7f127c066090 2020-11-20T12:18:44.763 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:18:44.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 -- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12540557c0 msgr2=0x7f1254057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12540557c0 0x7f1254057c80 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f127c067050 tx=0x7f1270015000).stop 2020-11-20T12:18:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 -- 172.21.15.116:0/969703829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f127c066090 msgr2=0x7f127c0a95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f127c066090 0x7f127c0a95f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f126c00c760 tx=0x7f126c007c90).stop 2020-11-20T12:18:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 -- 172.21.15.116:0/969703829 shutdown_connections 2020-11-20T12:18:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12540557c0 0x7f1254057c80 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f127c0ab1d0 0x7f127c160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f127c066090 0x7f127c0a95f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 --2- 172.21.15.116:0/969703829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f127c14bf40 0x7f127c0aace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 -- 172.21.15.116:0/969703829 >> 172.21.15.116:0/969703829 conn(0x7f127c060170 msgr2=0x7f127c154540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 -- 172.21.15.116:0/969703829 shutdown_connections 2020-11-20T12:18:44.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.768+0000 7f1280f5c700 1 -- 172.21.15.116:0/969703829 wait complete. 2020-11-20T12:18:44.791 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:18:44.792 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:18:44.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa2e94f700 1 Processor -- start 2020-11-20T12:18:44.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa2e94f700 1 -- start start 2020-11-20T12:18:44.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa2e94f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa28151d60 0x7ffa280b9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa2e94f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffa280bac10 0x7ffa280c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa2e94f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa280c23b0 con 0x7ffa28151980 2020-11-20T12:18:44.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa2e94f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa280ba850 con 0x7ffa280bac10 2020-11-20T12:18:44.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa2e94f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa280c2650 con 0x7ffa28151d60 2020-11-20T12:18:44.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa27fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ffa28151980 0x7ffa280bf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56634/0 (socket says 172.21.15.116:56634) 2020-11-20T12:18:44.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa27fff700 1 -- 172.21.15.116:0/192879451 learned_addr learned my addr 172.21.15.116:0/192879451 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:44.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa2ceec700 1 --2- 172.21.15.116:0/192879451 >> v2:172.21.15.116:3300/0 conn(0x7ffa280bac10 0x7ffa280c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa26ffd700 1 -- 172.21.15.116:0/192879451 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515776468 0 0) 0x7ffa280c23b0 con 0x7ffa28151980 2020-11-20T12:18:44.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa26ffd700 1 -- 172.21.15.116:0/192879451 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa14003240 con 0x7ffa28151980 2020-11-20T12:18:44.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.964+0000 7ffa277fe700 1 --2- 172.21.15.116:0/192879451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa28151d60 0x7ffa280b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa26ffd700 1 -- 172.21.15.116:0/192879451 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 251969342 0 0) 0x7ffa14003240 con 0x7ffa28151980 2020-11-20T12:18:44.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2ceec700 1 -- 172.21.15.116:0/192879451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa28151d60 msgr2=0x7ffa280b9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:44.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2ceec700 1 --2- 172.21.15.116:0/192879451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa28151d60 0x7ffa280b9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2ceec700 1 -- 172.21.15.116:0/192879451 >> v1:172.21.15.116:6789/0 conn(0x7ffa28151980 legacy=0x7ffa280bf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2ceec700 1 -- 172.21.15.116:0/192879451 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa280c28f0 con 0x7ffa280bac10 2020-11-20T12:18:44.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2ceec700 1 --2- 172.21.15.116:0/192879451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa280bac10 0x7ffa280c3030 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7ffa20009c90 tx=0x7ffa200091d0).ready entity=mon.0 client_cookie=e875a902e7832473 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa26ffd700 1 -- 172.21.15.116:0/192879451 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa180014e0 con 0x7ffa280bac10 2020-11-20T12:18:44.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa26ffd700 1 -- 172.21.15.116:0/192879451 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2000b760 con 0x7ffa280bac10 2020-11-20T12:18:44.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa26ffd700 1 -- 172.21.15.116:0/192879451 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2000bae0 con 0x7ffa280bac10 2020-11-20T12:18:44.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2e94f700 1 -- 172.21.15.116:0/192879451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa280bac10 msgr2=0x7ffa280c3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2e94f700 1 --2- 172.21.15.116:0/192879451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa280bac10 0x7ffa280c3030 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7ffa20009c90 tx=0x7ffa200091d0).stop 2020-11-20T12:18:44.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2e94f700 1 -- 172.21.15.116:0/192879451 shutdown_connections 2020-11-20T12:18:44.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2e94f700 1 --2- 172.21.15.116:0/192879451 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa280bac10 0x7ffa280c3030 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2e94f700 1 --2- 172.21.15.116:0/192879451 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa28151d60 0x7ffa280b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2e94f700 1 -- 172.21.15.116:0/192879451 >> 172.21.15.116:0/192879451 conn(0x7ffa280c4f80 msgr2=0x7ffa280c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:44.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2e94f700 1 -- 172.21.15.116:0/192879451 shutdown_connections 2020-11-20T12:18:44.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.968+0000 7ffa2e94f700 1 -- 172.21.15.116:0/192879451 wait complete. 2020-11-20T12:18:44.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.972+0000 7ffa2e94f700 1 Processor -- start 2020-11-20T12:18:44.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.972+0000 7ffa2e94f700 1 -- start start 2020-11-20T12:18:44.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa2e94f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa280bac10 0x7ffa280bd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa2e94f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa28151980 0x7ffa281645f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa2e94f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa28161870 0x7ffa280bed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa2e94f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa28164f30 con 0x7ffa28151980 2020-11-20T12:18:44.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa2e94f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa281651d0 con 0x7ffa28161870 2020-11-20T12:18:44.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa2e94f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa28165470 con 0x7ffa280bac10 2020-11-20T12:18:44.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa277fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa28151980 0x7ffa281645f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa277fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa28151980 0x7ffa281645f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47208/0 (socket says 172.21.15.116:47208) 2020-11-20T12:18:44.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa277fe700 1 -- 172.21.15.116:0/4117173312 learned_addr learned my addr 172.21.15.116:0/4117173312 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:44.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa27fff700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa280bac10 0x7ffa280bd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa2ceec700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa28161870 0x7ffa280bed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa277fe700 1 -- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa280bac10 msgr2=0x7ffa280bd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa277fe700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa280bac10 0x7ffa280bd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa277fe700 1 -- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa28161870 msgr2=0x7ffa280bed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:44.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa277fe700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa28161870 0x7ffa280bed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:44.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa277fe700 1 -- 172.21.15.116:0/4117173312 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa28161130 con 0x7ffa28151980 2020-11-20T12:18:44.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa2ceec700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa28161870 0x7ffa280bed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:44.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa277fe700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa28151980 0x7ffa281645f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7ffa1c00d8e0 tx=0x7ffa1c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa07fff700 1 -- 172.21.15.116:0/4117173312 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c009650 con 0x7ffa28151980 2020-11-20T12:18:44.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa2e94f700 1 -- 172.21.15.116:0/4117173312 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa281613d0 con 0x7ffa28151980 2020-11-20T12:18:44.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa07fff700 1 -- 172.21.15.116:0/4117173312 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1c0097b0 con 0x7ffa28151980 2020-11-20T12:18:44.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa07fff700 1 -- 172.21.15.116:0/4117173312 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1c009b50 con 0x7ffa28151980 2020-11-20T12:18:44.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.976+0000 7ffa2e94f700 1 -- 172.21.15.116:0/4117173312 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa28164ae0 con 0x7ffa28151980 2020-11-20T12:18:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.980+0000 7ffa2e94f700 1 -- 172.21.15.116:0/4117173312 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa28161560 con 0x7ffa28151980 2020-11-20T12:18:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.980+0000 7ffa07fff700 1 -- 172.21.15.116:0/4117173312 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ffa1c013480 con 0x7ffa28151980 2020-11-20T12:18:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.980+0000 7ffa07fff700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa000557c0 0x7ffa00057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:44.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.980+0000 7ffa07fff700 1 -- 172.21.15.116:0/4117173312 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffa1c06ea00 con 0x7ffa28151980 2020-11-20T12:18:44.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.980+0000 7ffa27fff700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa000557c0 0x7ffa00057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:44.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.984+0000 7ffa27fff700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa000557c0 0x7ffa00057c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7ffa180011a0 tx=0x7ffa18001280).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:44.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:44.988+0000 7ffa07fff700 1 -- 172.21.15.116:0/4117173312 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ffa1c044840 con 0x7ffa28151980 2020-11-20T12:18:45.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa2e94f700 1 -- 172.21.15.116:0/4117173312 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa2805d2a0 con 0x7ffa28151980 2020-11-20T12:18:45.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa07fff700 1 -- 172.21.15.116:0/4117173312 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+2850 (secure 0 0 0) 0x7ffa1c047e60 con 0x7ffa28151980 2020-11-20T12:18:45.194 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:45.195 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":28,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/785878283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":785878283},{"type":"v1","addr":"172.21.15.173:6837","nonce":785878283}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5952,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1568769426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1568769426},{"type":"v1","addr":"172.21.15.173:6836","nonce":1568769426}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3552490451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3552490451},{"type":"v1","addr":"172.21.15.116:6833","nonce":3552490451}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:18:27.137821+0000","modified":"2020-11-20T12:18:29.161493+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3370673707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3370673707},{"type":"v1","addr":"172.21.15.116:6835","nonce":3370673707}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-20T12:18:45.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa2e94f700 1 -- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa000557c0 msgr2=0x7ffa00057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:45.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa2e94f700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa000557c0 0x7ffa00057c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7ffa180011a0 tx=0x7ffa18001280).stop 2020-11-20T12:18:45.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa2e94f700 1 -- 172.21.15.116:0/4117173312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa28151980 msgr2=0x7ffa281645f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:45.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa2e94f700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa28151980 0x7ffa281645f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7ffa1c00d8e0 tx=0x7ffa1c008040).stop 2020-11-20T12:18:45.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa2e94f700 1 -- 172.21.15.116:0/4117173312 shutdown_connections 2020-11-20T12:18:45.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa2e94f700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa000557c0 0x7ffa00057c80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa2e94f700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa28161870 0x7ffa280bed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa2e94f700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa280bac10 0x7ffa280bd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa2e94f700 1 --2- 172.21.15.116:0/4117173312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa28151980 0x7ffa281645f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:45.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.196+0000 7ffa2e94f700 1 -- 172.21.15.116:0/4117173312 >> 172.21.15.116:0/4117173312 conn(0x7ffa280c4f80 msgr2=0x7ffa280c1f00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:45.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.200+0000 7ffa2e94f700 1 -- 172.21.15.116:0/4117173312 shutdown_connections 2020-11-20T12:18:45.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:45.200+0000 7ffa2e94f700 1 -- 172.21.15.116:0/4117173312 wait complete. 2020-11-20T12:18:45.200 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 28 2020-11-20T12:18:45.627 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:18:27.137821+0000', 'modified': '2020-11-20T12:18:29.161493+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5970}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5970': {'gid': 5970, 'name': 'c', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/3370673707', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 3370673707}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 3370673707}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:18:45.628 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:18:45.629 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:18:45.820 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":5970,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.662301297999999,"mdsmap_epoch":28,"osdmap_epoch":56,"osdmap_epoch_barrier":56,"uptime":25.129195615} 2020-11-20T12:18:45.821 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 5970, "mdsmap_epoch": 28, "osdmap_epoch": 56, "osdmap_epoch_barrier": 56, "rank_uptime": 18.662301298, "state": "up:active", "uptime": 25.129195615, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:18:45.821 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:18:45.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:45.822 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.3 2020-11-20T12:18:45.839 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:18:45.840 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:45.906 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:45.907 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:18:45.931 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:18:45.931 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:18:45.937 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:18:45.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:45.938 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2020-11-20T12:18:45.964 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-20T12:18:45.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:18:45.965 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 2020-11-20T12:18:46.045 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:18:46.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:46.046 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T12:18:46.062 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.manila mds 'allow *' osd 'allow rw' mon 'allow *' 2020-11-20T12:18:46.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.256+0000 7f744b59e700 1 Processor -- start 2020-11-20T12:18:46.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.256+0000 7f744b59e700 1 -- start start 2020-11-20T12:18:46.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f744b59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f744c0c3610 0x7f744c065120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:46.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f744b59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744c0c1eb0 con 0x7f744c0bca90 2020-11-20T12:18:46.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f744b59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f744c0bc3d0 con 0x7f744c0bcef0 2020-11-20T12:18:46.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f744b59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f744c14fcf0 con 0x7f744c0c3610 2020-11-20T12:18:46.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f744a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f744c0c3610 0x7f744c065120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:46.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f7449d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f744c0bca90 0x7f744c0bb5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56652/0 (socket says 172.21.15.116:56652) 2020-11-20T12:18:46.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f7449d9b700 1 -- 172.21.15.116:0/955008584 learned_addr learned my addr 172.21.15.116:0/955008584 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:46.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f744959a700 1 -- 172.21.15.116:0/955008584 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292472502 0 0) 0x7f744c0c1eb0 con 0x7f744c0bca90 2020-11-20T12:18:46.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f744959a700 1 -- 172.21.15.116:0/955008584 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7430003240 con 0x7f744c0bca90 2020-11-20T12:18:46.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f744a59c700 1 -- 172.21.15.116:0/955008584 >> v1:172.21.15.173:6789/0 conn(0x7f744c0bcef0 legacy=0x7f744c0bbcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f744a59c700 1 -- 172.21.15.116:0/955008584 >> v1:172.21.15.116:6789/0 conn(0x7f744c0bca90 legacy=0x7f744c0bb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f744a59c700 1 -- 172.21.15.116:0/955008584 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c14fee0 con 0x7f744c0c3610 2020-11-20T12:18:46.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.260+0000 7f744a59c700 1 --2- 172.21.15.116:0/955008584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f744c0c3610 0x7f744c065120 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f743c00b9a0 tx=0x7f743c006df0).ready entity=mon.0 client_cookie=c02db856f28ee21e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:46.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.264+0000 7f744959a700 1 -- 172.21.15.116:0/955008584 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c0097b0 con 0x7f744c0c3610 2020-11-20T12:18:46.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.264+0000 7f744959a700 1 -- 172.21.15.116:0/955008584 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f743c009910 con 0x7f744c0c3610 2020-11-20T12:18:46.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.264+0000 7f744959a700 1 -- 172.21.15.116:0/955008584 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743c009c90 con 0x7f744c0c3610 2020-11-20T12:18:46.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.264+0000 7f744b59e700 1 -- 172.21.15.116:0/955008584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f744c0c3610 msgr2=0x7f744c065120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.264+0000 7f744b59e700 1 --2- 172.21.15.116:0/955008584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f744c0c3610 0x7f744c065120 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f743c00b9a0 tx=0x7f743c006df0).stop 2020-11-20T12:18:46.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.264+0000 7f744b59e700 1 -- 172.21.15.116:0/955008584 shutdown_connections 2020-11-20T12:18:46.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.264+0000 7f744b59e700 1 --2- 172.21.15.116:0/955008584 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f744c0c3610 0x7f744c065120 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:46.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.264+0000 7f744b59e700 1 -- 172.21.15.116:0/955008584 >> 172.21.15.116:0/955008584 conn(0x7f744c07ff70 msgr2=0x7f744c080350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:46.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.264+0000 7f744b59e700 1 -- 172.21.15.116:0/955008584 shutdown_connections 2020-11-20T12:18:46.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.264+0000 7f744b59e700 1 -- 172.21.15.116:0/955008584 wait complete. 2020-11-20T12:18:46.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.268+0000 7f744b59e700 1 Processor -- start 2020-11-20T12:18:46.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.268+0000 7f744b59e700 1 -- start start 2020-11-20T12:18:46.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.268+0000 7f744b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f744c0bca90 0x7f744c163070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:46.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.268+0000 7f744b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f744c0bcef0 0x7f744c14ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:46.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.268+0000 7f744b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f744c0c3610 0x7f744c0a5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:46.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.268+0000 7f744b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c163560 con 0x7f744c0bcef0 2020-11-20T12:18:46.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.268+0000 7f744b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f744c0a78f0 con 0x7f744c0c3610 2020-11-20T12:18:46.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.268+0000 7f744b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f744c15f650 con 0x7f744c0bca90 2020-11-20T12:18:46.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.268+0000 7f7449d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f744c0bcef0 0x7f744c14ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:46.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f7449d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f744c0bcef0 0x7f744c14ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47222/0 (socket says 172.21.15.116:47222) 2020-11-20T12:18:46.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f7449d9b700 1 -- 172.21.15.116:0/2649711242 learned_addr learned my addr 172.21.15.116:0/2649711242 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:46.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f744a59c700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f744c0bca90 0x7f744c163070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:46.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f744ad9d700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f744c0c3610 0x7f744c0a5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:46.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f7449d9b700 1 -- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f744c0bca90 msgr2=0x7f744c163070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f7449d9b700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f744c0bca90 0x7f744c163070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:46.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f7449d9b700 1 -- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f744c0c3610 msgr2=0x7f744c0a5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f7449d9b700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f744c0c3610 0x7f744c0a5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:46.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f7449d9b700 1 -- 172.21.15.116:0/2649711242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c15f8f0 con 0x7f744c0bcef0 2020-11-20T12:18:46.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f744ad9d700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f744c0c3610 0x7f744c0a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:46.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f7449d9b700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f744c0bcef0 0x7f744c14ce70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f744000c710 tx=0x7f7440007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:46.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f743a7fc700 1 -- 172.21.15.116:0/2649711242 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74400072e0 con 0x7f744c0bcef0 2020-11-20T12:18:46.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f743a7fc700 1 -- 172.21.15.116:0/2649711242 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7440007440 con 0x7f744c0bcef0 2020-11-20T12:18:46.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f744b59e700 1 -- 172.21.15.116:0/2649711242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744c15fb90 con 0x7f744c0bcef0 2020-11-20T12:18:46.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f743a7fc700 1 -- 172.21.15.116:0/2649711242 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7440005800 con 0x7f744c0bcef0 2020-11-20T12:18:46.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f744b59e700 1 -- 172.21.15.116:0/2649711242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744c14d360 con 0x7f744c0bcef0 2020-11-20T12:18:46.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.272+0000 7f744b59e700 1 -- 172.21.15.116:0/2649711242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f744c14d5e0 con 0x7f744c0bcef0 2020-11-20T12:18:46.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.276+0000 7f743a7fc700 1 -- 172.21.15.116:0/2649711242 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7440010070 con 0x7f744c0bcef0 2020-11-20T12:18:46.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.276+0000 7f743a7fc700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74200557c0 0x7f7420057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.276+0000 7f743a7fc700 1 -- 172.21.15.116:0/2649711242 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f744006ddd0 con 0x7f744c0bcef0 2020-11-20T12:18:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.276+0000 7f744a59c700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74200557c0 0x7f7420057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.276+0000 7f744a59c700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74200557c0 0x7f7420057c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f743c00be60 tx=0x7f743c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:46.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.280+0000 7f743a7fc700 1 -- 172.21.15.116:0/2649711242 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7440043b60 con 0x7f744c0bcef0 2020-11-20T12:18:46.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.492+0000 7f744b59e700 1 -- 172.21.15.116:0/2649711242 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]} v 0) v1 -- 0x7f744c05cf30 con 0x7f744c0bcef0 2020-11-20T12:18:46.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f743a7fc700 1 -- 172.21.15.116:0/2649711242 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]}]=0 v17) v1 ==== 158+0+64 (secure 0 0 0) 0x7f7440018030 con 0x7f744c0bcef0 2020-11-20T12:18:46.497 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:46.498 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAmtLdfoY10HRAAsLJqwS8fhRHZJFZ2QO74lw== 2020-11-20T12:18:46.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 -- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74200557c0 msgr2=0x7f7420057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74200557c0 0x7f7420057c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f743c00be60 tx=0x7f743c015000).stop 2020-11-20T12:18:46.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 -- 172.21.15.116:0/2649711242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f744c0bcef0 msgr2=0x7f744c14ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f744c0bcef0 0x7f744c14ce70 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f744000c710 tx=0x7f7440007ad0).stop 2020-11-20T12:18:46.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 -- 172.21.15.116:0/2649711242 shutdown_connections 2020-11-20T12:18:46.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74200557c0 0x7f7420057c80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:46.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f744c0c3610 0x7f744c0a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:46.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f744c0bca90 0x7f744c163070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:46.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 --2- 172.21.15.116:0/2649711242 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f744c0bcef0 0x7f744c14ce70 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:46.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 -- 172.21.15.116:0/2649711242 >> 172.21.15.116:0/2649711242 conn(0x7f744c07ff70 msgr2=0x7f744c080560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:46.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 -- 172.21.15.116:0/2649711242 shutdown_connections 2020-11-20T12:18:46.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.500+0000 7f744b59e700 1 -- 172.21.15.116:0/2649711242 wait complete. 2020-11-20T12:18:46.525 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:18:46.526 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:46.542 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:46.556 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:18:46.556 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:18:46.556 INFO:teuthology.orchestra.run.smithi173.stderr:64 bytes copied, 0.000338997 s, 189 kB/s 2020-11-20T12:18:46.560 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:46.569 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:46.635 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:46.636 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:46.636 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:46.636 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:46.636 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.637 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:46.637 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.637 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:46.637 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.638 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:46.638 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:46.638 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.638 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:46.638 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:46.639 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.639 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:46.639 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:46.639 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.639 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:46.640 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:46.640 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.640 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:46.640 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.640 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:46.641 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:46.641 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:46.641 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:46.641 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:46.641 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:46.642 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:46.642 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:46.642 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:46.642 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:46.643 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.643 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:46.643 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.643 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:46.643 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.644 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:46.644 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.644 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:46.644 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:46.644 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:46.645 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:46.645 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:46.645 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:46.645 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.645 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:46.646 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:46.646 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.646 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:46.646 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:46.646 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.647 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:46.647 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:46.647 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.647 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:46.647 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:46.648 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:46.648 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.648 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:46.648 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:46.648 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.649 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:46.649 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.649 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:46.649 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.650 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:46.650 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.650 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:46.650 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:46.650 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.651 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:46.651 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:46.651 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:46.651 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:46.651 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:46.652 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:46.652 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:46.652 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:46.652 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:46.652 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:46.653 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:46.653 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:46.653 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:46.653 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:46.653 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:46.654 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:46.654 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:46.654 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:46.654 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:46.654 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:46.655 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.655 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:46.655 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:46.655 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:46.655 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:46.656 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:46.656 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:46.656 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:46.656 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:46.656 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.657 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:46.657 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:46.657 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:46.657 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:46.657 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:46.658 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:46.658 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:46.658 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:46.658 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:46.658 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:46.659 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:46.659 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:46.659 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:46.659 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:46.659 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:46.660 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:46.660 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:46.660 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:46.660 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:46.661 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:46.661 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:46.661 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:46.661 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:46.661 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:46.662 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:46.662 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:46.662 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:46.662 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:46.662 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:46.663 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:46.663 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:46.663 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:46.663 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:46.663 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:46.664 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:46.664 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:46.664 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:46.664 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:46.664 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:46.665 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:46.665 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:46.665 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:46.665 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:46.665 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:46.666 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:46.666 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:46.666 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:46.666 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:46.666 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:46.667 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:46.668 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:46.668 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:46.668 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:46.669 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:46.669 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.669 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:46.669 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.669 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:46.670 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.670 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:46.670 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:46.670 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.671 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:46.671 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:46.671 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.671 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:46.671 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:46.672 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.672 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:46.672 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:46.672 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.672 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:46.672 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.673 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:46.673 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:46.673 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:46.673 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:46.674 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:46.674 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:46.674 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:46.674 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:46.674 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:46.675 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:46.675 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.675 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:46.675 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.675 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:46.676 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.676 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:46.676 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.676 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:46.676 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:46.677 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:46.677 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:46.677 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:46.677 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:46.677 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.678 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:46.678 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:46.678 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.678 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:46.678 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:46.679 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.679 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:46.679 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:46.679 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.679 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:46.680 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:46.680 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:46.680 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.680 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:46.680 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:46.681 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.681 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:46.681 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.681 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:46.681 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.682 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:46.682 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.682 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:46.682 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:46.682 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.683 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:46.683 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:46.683 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:46.683 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:46.683 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:46.684 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:46.684 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:46.684 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:46.684 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:46.684 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:46.685 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:46.685 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:46.685 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:46.685 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:46.686 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:46.686 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:46.686 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:46.686 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:46.686 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:46.687 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:46.687 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.687 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:46.687 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:46.687 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:46.688 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:46.688 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:46.688 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:46.688 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:46.688 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:46.689 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.689 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:46.689 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:46.689 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:46.689 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:46.690 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:46.690 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:46.690 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:46.690 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:46.690 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:46.691 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:46.691 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:46.691 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:46.691 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:46.691 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:46.691 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:46.692 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:46.692 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:46.692 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:46.692 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:46.693 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:46.693 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:46.693 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:46.693 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:46.693 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:46.694 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:46.694 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:46.694 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:46.694 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:46.694 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:46.695 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:46.695 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:46.695 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:46.695 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:46.695 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:46.696 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:46.696 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:46.696 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:46.696 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:46.696 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:46.697 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:46.697 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:46.697 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:46.697 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:46.697 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:46.698 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:46.698 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:46.698 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:46.698 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:46.698 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:18:46.699 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:18:46.699 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2020-11-20T12:18:46.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.876+0000 7f4503657700 1 Processor -- start 2020-11-20T12:18:46.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.876+0000 7f4503657700 1 -- start start 2020-11-20T12:18:46.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.876+0000 7f4503657700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f44fc0c3dd0 0x7f44fc153390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:46.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.876+0000 7f4503657700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44fc153980 con 0x7f44fc0c39f0 2020-11-20T12:18:46.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.876+0000 7f4503657700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44fc153c20 con 0x7f44fc0a95d0 2020-11-20T12:18:46.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.876+0000 7f4503657700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44fc153ec0 con 0x7f44fc0c3dd0 2020-11-20T12:18:46.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.876+0000 7f45013f3700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f44fc0c39f0 0x7f44fc066d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56662/0 (socket says 172.21.15.116:56662) 2020-11-20T12:18:46.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.876+0000 7f45013f3700 1 -- 172.21.15.116:0/740562647 learned_addr learned my addr 172.21.15.116:0/740562647 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:46.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.876+0000 7f4500bf2700 1 --2- 172.21.15.116:0/740562647 >> v2:172.21.15.173:3300/0 conn(0x7f44fc0c3dd0 0x7f44fc153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:46.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f44ebfff700 1 -- 172.21.15.116:0/740562647 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1963661531 0 0) 0x7f44fc153980 con 0x7f44fc0c39f0 2020-11-20T12:18:46.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f44ebfff700 1 -- 172.21.15.116:0/740562647 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44e4003240 con 0x7f44fc0c39f0 2020-11-20T12:18:46.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f4500bf2700 1 -- 172.21.15.116:0/740562647 >> v1:172.21.15.173:6789/0 conn(0x7f44fc0a95d0 legacy=0x7f44fc0a5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f4500bf2700 1 -- 172.21.15.116:0/740562647 >> v1:172.21.15.116:6789/0 conn(0x7f44fc0c39f0 legacy=0x7f44fc066d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f44ebfff700 1 -- 172.21.15.116:0/740562647 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 762274172 0 0) 0x7f44e4003240 con 0x7f44fc0c39f0 2020-11-20T12:18:46.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f4500bf2700 1 -- 172.21.15.116:0/740562647 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc154160 con 0x7f44fc0c3dd0 2020-11-20T12:18:46.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f4500bf2700 1 --2- 172.21.15.116:0/740562647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44fc0c3dd0 0x7f44fc153390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f44ec00b9a0 tx=0x7f44ec006df0).ready entity=mon.1 client_cookie=3764dc9fa00e20b9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:46.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f44ebfff700 1 -- 172.21.15.116:0/740562647 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec009840 con 0x7f44fc0c3dd0 2020-11-20T12:18:46.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f44ebfff700 1 -- 172.21.15.116:0/740562647 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44ec0099a0 con 0x7f44fc0c3dd0 2020-11-20T12:18:46.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f44ebfff700 1 -- 172.21.15.116:0/740562647 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44ec009d20 con 0x7f44fc0c3dd0 2020-11-20T12:18:46.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f4503657700 1 -- 172.21.15.116:0/740562647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44fc0c3dd0 msgr2=0x7f44fc153390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f4503657700 1 --2- 172.21.15.116:0/740562647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44fc0c3dd0 0x7f44fc153390 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f44ec00b9a0 tx=0x7f44ec006df0).stop 2020-11-20T12:18:46.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f4503657700 1 -- 172.21.15.116:0/740562647 shutdown_connections 2020-11-20T12:18:46.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f4503657700 1 --2- 172.21.15.116:0/740562647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44fc0c3dd0 0x7f44fc153390 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:46.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.880+0000 7f4503657700 1 -- 172.21.15.116:0/740562647 >> 172.21.15.116:0/740562647 conn(0x7f44fc0c0de0 msgr2=0x7f44fc0c11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:46.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.884+0000 7f4503657700 1 -- 172.21.15.116:0/740562647 shutdown_connections 2020-11-20T12:18:46.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.884+0000 7f4503657700 1 -- 172.21.15.116:0/740562647 wait complete. 2020-11-20T12:18:46.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.884+0000 7f4503657700 1 Processor -- start 2020-11-20T12:18:46.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.884+0000 7f4503657700 1 -- start start 2020-11-20T12:18:46.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4503657700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44fc0a95d0 0x7f44fc0941d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:46.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4503657700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44fc0c39f0 0x7f44fc141350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:46.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4503657700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44fc0c3dd0 0x7f44fc141840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:46.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4503657700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc13ed70 con 0x7f44fc0a95d0 2020-11-20T12:18:46.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4503657700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc13f010 con 0x7f44fc0c39f0 2020-11-20T12:18:46.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4503657700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44fc13f2b0 con 0x7f44fc0c3dd0 2020-11-20T12:18:46.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f45013f3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44fc0a95d0 0x7f44fc0941d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:46.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f45013f3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44fc0a95d0 0x7f44fc0941d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47234/0 (socket says 172.21.15.116:47234) 2020-11-20T12:18:46.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f45013f3700 1 -- 172.21.15.116:0/2106318570 learned_addr learned my addr 172.21.15.116:0/2106318570 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:46.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4500bf2700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44fc0c39f0 0x7f44fc141350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:46.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4501bf4700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44fc0c3dd0 0x7f44fc141840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:46.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f45013f3700 1 -- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44fc0c3dd0 msgr2=0x7f44fc141840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f45013f3700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44fc0c3dd0 0x7f44fc141840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:46.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f45013f3700 1 -- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44fc0c39f0 msgr2=0x7f44fc141350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:46.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f45013f3700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44fc0c39f0 0x7f44fc141350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:46.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f45013f3700 1 -- 172.21.15.116:0/2106318570 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44fc13f550 con 0x7f44fc0a95d0 2020-11-20T12:18:46.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4500bf2700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44fc0c39f0 0x7f44fc141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:46.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f45013f3700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44fc0a95d0 0x7f44fc0941d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f44f800c690 tx=0x7f44f80017f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:46.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f44e97fa700 1 -- 172.21.15.116:0/2106318570 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8007170 con 0x7f44fc0a95d0 2020-11-20T12:18:46.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f44e97fa700 1 -- 172.21.15.116:0/2106318570 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44f80072d0 con 0x7f44fc0a95d0 2020-11-20T12:18:46.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4503657700 1 -- 172.21.15.116:0/2106318570 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44fc13f7f0 con 0x7f44fc0a95d0 2020-11-20T12:18:46.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f44e97fa700 1 -- 172.21.15.116:0/2106318570 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44f8004920 con 0x7f44fc0a95d0 2020-11-20T12:18:46.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4503657700 1 -- 172.21.15.116:0/2106318570 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44fc0946c0 con 0x7f44fc0a95d0 2020-11-20T12:18:46.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.888+0000 7f4503657700 1 -- 172.21.15.116:0/2106318570 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44fc13f980 con 0x7f44fc0a95d0 2020-11-20T12:18:46.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.892+0000 7f44e97fa700 1 -- 172.21.15.116:0/2106318570 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f44f801b060 con 0x7f44fc0a95d0 2020-11-20T12:18:46.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.892+0000 7f44e97fa700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44d40557c0 0x7f44d4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:46.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.892+0000 7f44e97fa700 1 -- 172.21.15.116:0/2106318570 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f44f806e140 con 0x7f44fc0a95d0 2020-11-20T12:18:46.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.892+0000 7f4500bf2700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44d40557c0 0x7f44d4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:46.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.892+0000 7f4500bf2700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44d40557c0 0x7f44d4057c80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f44fc0934f0 tx=0x7f44ec015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:46.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:46.896+0000 7f44e97fa700 1 -- 172.21.15.116:0/2106318570 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f44f8043fe0 con 0x7f44fc0a95d0 2020-11-20T12:18:47.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.052+0000 7f4503657700 1 -- 172.21.15.116:0/2106318570 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f44fc044f60 con 0x7f44fc0a95d0 2020-11-20T12:18:47.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.052+0000 7f44e97fa700 1 -- 172.21.15.116:0/2106318570 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) v1 ==== 81+0+34768 (secure 0 0 0) 0x7f44f8019030 con 0x7f44fc0a95d0 2020-11-20T12:18:47.053 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:47.054 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:18:47.054 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 56, 2020-11-20T12:18:47.054 INFO:teuthology.orchestra.run.smithi116.stdout: "fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", 2020-11-20T12:18:47.054 INFO:teuthology.orchestra.run.smithi116.stdout: "created": "2020-11-20T12:15:41.360400+0000", 2020-11-20T12:18:47.054 INFO:teuthology.orchestra.run.smithi116.stdout: "modified": "2020-11-20T12:18:28.142253+0000", 2020-11-20T12:18:47.055 INFO:teuthology.orchestra.run.smithi116.stdout: "last_up_change": "2020-11-20T12:15:50.549644+0000", 2020-11-20T12:18:47.055 INFO:teuthology.orchestra.run.smithi116.stdout: "last_in_change": "2020-11-20T12:15:50.549644+0000", 2020-11-20T12:18:47.055 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-20T12:18:47.055 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_num": 5799936, 2020-11-20T12:18:47.056 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_set": [ 2020-11-20T12:18:47.056 INFO:teuthology.orchestra.run.smithi116.stdout: "pglog_hardlimit", 2020-11-20T12:18:47.056 INFO:teuthology.orchestra.run.smithi116.stdout: "purged_snapdirs", 2020-11-20T12:18:47.056 INFO:teuthology.orchestra.run.smithi116.stdout: "recovery_deletes", 2020-11-20T12:18:47.056 INFO:teuthology.orchestra.run.smithi116.stdout: "sortbitwise" 2020-11-20T12:18:47.057 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2020-11-20T12:18:47.057 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_version": 4, 2020-11-20T12:18:47.057 INFO:teuthology.orchestra.run.smithi116.stdout: "full_ratio": 0.94999998807907104, 2020-11-20T12:18:47.057 INFO:teuthology.orchestra.run.smithi116.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-20T12:18:47.057 INFO:teuthology.orchestra.run.smithi116.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-20T12:18:47.058 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_snapshot": "", 2020-11-20T12:18:47.058 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_max": 10, 2020-11-20T12:18:47.058 INFO:teuthology.orchestra.run.smithi116.stdout: "max_osd": 8, 2020-11-20T12:18:47.058 INFO:teuthology.orchestra.run.smithi116.stdout: "require_min_compat_client": "luminous", 2020-11-20T12:18:47.058 INFO:teuthology.orchestra.run.smithi116.stdout: "min_compat_client": "jewel", 2020-11-20T12:18:47.059 INFO:teuthology.orchestra.run.smithi116.stdout: "require_osd_release": "pacific", 2020-11-20T12:18:47.059 INFO:teuthology.orchestra.run.smithi116.stdout: "pools": [ 2020-11-20T12:18:47.059 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.059 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 1, 2020-11-20T12:18:47.059 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": "device_health_metrics", 2020-11-20T12:18:47.060 INFO:teuthology.orchestra.run.smithi116.stdout: "create_time": "2020-11-20T12:15:46.607758+0000", 2020-11-20T12:18:47.060 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2020-11-20T12:18:47.060 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_names": "hashpspool", 2020-11-20T12:18:47.060 INFO:teuthology.orchestra.run.smithi116.stdout: "type": 1, 2020-11-20T12:18:47.060 INFO:teuthology.orchestra.run.smithi116.stdout: "size": 2, 2020-11-20T12:18:47.061 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": 1, 2020-11-20T12:18:47.061 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_rule": 0, 2020-11-20T12:18:47.061 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_count": 0, 2020-11-20T12:18:47.061 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_target": 0, 2020-11-20T12:18:47.061 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_barrier": 0, 2020-11-20T12:18:47.062 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-20T12:18:47.062 INFO:teuthology.orchestra.run.smithi116.stdout: "object_hash": 2, 2020-11-20T12:18:47.062 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscale_mode": "off", 2020-11-20T12:18:47.062 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": 1, 2020-11-20T12:18:47.062 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num": 1, 2020-11-20T12:18:47.063 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num_target": 1, 2020-11-20T12:18:47.063 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_target": 1, 2020-11-20T12:18:47.063 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_pending": 1, 2020-11-20T12:18:47.063 INFO:teuthology.orchestra.run.smithi116.stdout: "last_pg_merge_meta": { 2020-11-20T12:18:47.063 INFO:teuthology.orchestra.run.smithi116.stdout: "source_pgid": "0.0", 2020-11-20T12:18:47.064 INFO:teuthology.orchestra.run.smithi116.stdout: "ready_epoch": 0, 2020-11-20T12:18:47.064 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_started": 0, 2020-11-20T12:18:47.064 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_clean": 0, 2020-11-20T12:18:47.064 INFO:teuthology.orchestra.run.smithi116.stdout: "source_version": "0'0", 2020-11-20T12:18:47.064 INFO:teuthology.orchestra.run.smithi116.stdout: "target_version": "0'0" 2020-11-20T12:18:47.065 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.065 INFO:teuthology.orchestra.run.smithi116.stdout: "last_change": "16", 2020-11-20T12:18:47.065 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend": "0", 2020-11-20T12:18:47.065 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T12:18:47.066 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T12:18:47.066 INFO:teuthology.orchestra.run.smithi116.stdout: "auid": 0, 2020-11-20T12:18:47.066 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_mode": "selfmanaged", 2020-11-20T12:18:47.067 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_seq": 0, 2020-11-20T12:18:47.067 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_epoch": 0, 2020-11-20T12:18:47.068 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_snaps": [], 2020-11-20T12:18:47.068 INFO:teuthology.orchestra.run.smithi116.stdout: "removed_snaps": "[]", 2020-11-20T12:18:47.068 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_bytes": 0, 2020-11-20T12:18:47.068 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_objects": 0, 2020-11-20T12:18:47.068 INFO:teuthology.orchestra.run.smithi116.stdout: "tiers": [], 2020-11-20T12:18:47.069 INFO:teuthology.orchestra.run.smithi116.stdout: "tier_of": -1, 2020-11-20T12:18:47.069 INFO:teuthology.orchestra.run.smithi116.stdout: "read_tier": -1, 2020-11-20T12:18:47.069 INFO:teuthology.orchestra.run.smithi116.stdout: "write_tier": -1, 2020-11-20T12:18:47.069 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_mode": "none", 2020-11-20T12:18:47.069 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_bytes": 0, 2020-11-20T12:18:47.070 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_objects": 0, 2020-11-20T12:18:47.070 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T12:18:47.070 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T12:18:47.070 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T12:18:47.070 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_flush_age": 0, 2020-11-20T12:18:47.071 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_evict_age": 0, 2020-11-20T12:18:47.071 INFO:teuthology.orchestra.run.smithi116.stdout: "erasure_code_profile": "", 2020-11-20T12:18:47.071 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_params": { 2020-11-20T12:18:47.071 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "none" 2020-11-20T12:18:47.071 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.072 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_period": 0, 2020-11-20T12:18:47.072 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_count": 0, 2020-11-20T12:18:47.072 INFO:teuthology.orchestra.run.smithi116.stdout: "use_gmt_hitset": true, 2020-11-20T12:18:47.072 INFO:teuthology.orchestra.run.smithi116.stdout: "min_read_recency_for_promote": 0, 2020-11-20T12:18:47.072 INFO:teuthology.orchestra.run.smithi116.stdout: "min_write_recency_for_promote": 0, 2020-11-20T12:18:47.073 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T12:18:47.073 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_search_last_n": 0, 2020-11-20T12:18:47.073 INFO:teuthology.orchestra.run.smithi116.stdout: "grade_table": [], 2020-11-20T12:18:47.073 INFO:teuthology.orchestra.run.smithi116.stdout: "stripe_width": 0, 2020-11-20T12:18:47.074 INFO:teuthology.orchestra.run.smithi116.stdout: "expected_num_objects": 0, 2020-11-20T12:18:47.074 INFO:teuthology.orchestra.run.smithi116.stdout: "fast_read": false, 2020-11-20T12:18:47.074 INFO:teuthology.orchestra.run.smithi116.stdout: "options": { 2020-11-20T12:18:47.074 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_min": 1 2020-11-20T12:18:47.074 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.075 INFO:teuthology.orchestra.run.smithi116.stdout: "application_metadata": { 2020-11-20T12:18:47.075 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr_devicehealth": {} 2020-11-20T12:18:47.075 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.075 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.075 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.075 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 2, 2020-11-20T12:18:47.076 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": "rbd", 2020-11-20T12:18:47.076 INFO:teuthology.orchestra.run.smithi116.stdout: "create_time": "2020-11-20T12:15:55.044020+0000", 2020-11-20T12:18:47.076 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2020-11-20T12:18:47.076 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_names": "hashpspool", 2020-11-20T12:18:47.076 INFO:teuthology.orchestra.run.smithi116.stdout: "type": 1, 2020-11-20T12:18:47.077 INFO:teuthology.orchestra.run.smithi116.stdout: "size": 2, 2020-11-20T12:18:47.077 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": 1, 2020-11-20T12:18:47.077 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_rule": 0, 2020-11-20T12:18:47.077 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_count": 0, 2020-11-20T12:18:47.078 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_target": 0, 2020-11-20T12:18:47.078 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_barrier": 0, 2020-11-20T12:18:47.078 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-20T12:18:47.078 INFO:teuthology.orchestra.run.smithi116.stdout: "object_hash": 2, 2020-11-20T12:18:47.078 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscale_mode": "off", 2020-11-20T12:18:47.079 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": 8, 2020-11-20T12:18:47.079 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num": 8, 2020-11-20T12:18:47.079 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num_target": 8, 2020-11-20T12:18:47.079 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_target": 8, 2020-11-20T12:18:47.079 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_pending": 8, 2020-11-20T12:18:47.080 INFO:teuthology.orchestra.run.smithi116.stdout: "last_pg_merge_meta": { 2020-11-20T12:18:47.080 INFO:teuthology.orchestra.run.smithi116.stdout: "source_pgid": "0.0", 2020-11-20T12:18:47.080 INFO:teuthology.orchestra.run.smithi116.stdout: "ready_epoch": 0, 2020-11-20T12:18:47.080 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_started": 0, 2020-11-20T12:18:47.080 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_clean": 0, 2020-11-20T12:18:47.081 INFO:teuthology.orchestra.run.smithi116.stdout: "source_version": "0'0", 2020-11-20T12:18:47.081 INFO:teuthology.orchestra.run.smithi116.stdout: "target_version": "0'0" 2020-11-20T12:18:47.081 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.081 INFO:teuthology.orchestra.run.smithi116.stdout: "last_change": "19", 2020-11-20T12:18:47.081 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend": "0", 2020-11-20T12:18:47.082 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T12:18:47.082 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T12:18:47.082 INFO:teuthology.orchestra.run.smithi116.stdout: "auid": 0, 2020-11-20T12:18:47.082 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_mode": "selfmanaged", 2020-11-20T12:18:47.082 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_seq": 0, 2020-11-20T12:18:47.083 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_epoch": 0, 2020-11-20T12:18:47.083 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_snaps": [], 2020-11-20T12:18:47.083 INFO:teuthology.orchestra.run.smithi116.stdout: "removed_snaps": "[]", 2020-11-20T12:18:47.083 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_bytes": 0, 2020-11-20T12:18:47.083 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_objects": 0, 2020-11-20T12:18:47.084 INFO:teuthology.orchestra.run.smithi116.stdout: "tiers": [], 2020-11-20T12:18:47.084 INFO:teuthology.orchestra.run.smithi116.stdout: "tier_of": -1, 2020-11-20T12:18:47.084 INFO:teuthology.orchestra.run.smithi116.stdout: "read_tier": -1, 2020-11-20T12:18:47.084 INFO:teuthology.orchestra.run.smithi116.stdout: "write_tier": -1, 2020-11-20T12:18:47.084 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_mode": "none", 2020-11-20T12:18:47.085 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_bytes": 0, 2020-11-20T12:18:47.085 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_objects": 0, 2020-11-20T12:18:47.085 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T12:18:47.085 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T12:18:47.085 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T12:18:47.086 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_flush_age": 0, 2020-11-20T12:18:47.086 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_evict_age": 0, 2020-11-20T12:18:47.086 INFO:teuthology.orchestra.run.smithi116.stdout: "erasure_code_profile": "", 2020-11-20T12:18:47.086 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_params": { 2020-11-20T12:18:47.086 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "none" 2020-11-20T12:18:47.087 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.087 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_period": 0, 2020-11-20T12:18:47.087 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_count": 0, 2020-11-20T12:18:47.087 INFO:teuthology.orchestra.run.smithi116.stdout: "use_gmt_hitset": true, 2020-11-20T12:18:47.087 INFO:teuthology.orchestra.run.smithi116.stdout: "min_read_recency_for_promote": 0, 2020-11-20T12:18:47.088 INFO:teuthology.orchestra.run.smithi116.stdout: "min_write_recency_for_promote": 0, 2020-11-20T12:18:47.088 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T12:18:47.088 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_search_last_n": 0, 2020-11-20T12:18:47.088 INFO:teuthology.orchestra.run.smithi116.stdout: "grade_table": [], 2020-11-20T12:18:47.088 INFO:teuthology.orchestra.run.smithi116.stdout: "stripe_width": 0, 2020-11-20T12:18:47.089 INFO:teuthology.orchestra.run.smithi116.stdout: "expected_num_objects": 0, 2020-11-20T12:18:47.089 INFO:teuthology.orchestra.run.smithi116.stdout: "fast_read": false, 2020-11-20T12:18:47.089 INFO:teuthology.orchestra.run.smithi116.stdout: "options": {}, 2020-11-20T12:18:47.089 INFO:teuthology.orchestra.run.smithi116.stdout: "application_metadata": { 2020-11-20T12:18:47.089 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd": {} 2020-11-20T12:18:47.090 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.090 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.090 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.090 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 9, 2020-11-20T12:18:47.091 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": "cephfs_metadata", 2020-11-20T12:18:47.091 INFO:teuthology.orchestra.run.smithi116.stdout: "create_time": "2020-11-20T12:18:25.558441+0000", 2020-11-20T12:18:47.091 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2020-11-20T12:18:47.091 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_names": "hashpspool", 2020-11-20T12:18:47.091 INFO:teuthology.orchestra.run.smithi116.stdout: "type": 1, 2020-11-20T12:18:47.092 INFO:teuthology.orchestra.run.smithi116.stdout: "size": 2, 2020-11-20T12:18:47.092 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": 1, 2020-11-20T12:18:47.092 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_rule": 0, 2020-11-20T12:18:47.092 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_count": 0, 2020-11-20T12:18:47.092 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_target": 0, 2020-11-20T12:18:47.093 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_barrier": 0, 2020-11-20T12:18:47.093 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-20T12:18:47.093 INFO:teuthology.orchestra.run.smithi116.stdout: "object_hash": 2, 2020-11-20T12:18:47.093 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscale_mode": "off", 2020-11-20T12:18:47.093 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": 8, 2020-11-20T12:18:47.094 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num": 8, 2020-11-20T12:18:47.094 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num_target": 8, 2020-11-20T12:18:47.094 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_target": 8, 2020-11-20T12:18:47.094 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_pending": 8, 2020-11-20T12:18:47.094 INFO:teuthology.orchestra.run.smithi116.stdout: "last_pg_merge_meta": { 2020-11-20T12:18:47.095 INFO:teuthology.orchestra.run.smithi116.stdout: "source_pgid": "0.0", 2020-11-20T12:18:47.095 INFO:teuthology.orchestra.run.smithi116.stdout: "ready_epoch": 0, 2020-11-20T12:18:47.095 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_started": 0, 2020-11-20T12:18:47.095 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_clean": 0, 2020-11-20T12:18:47.095 INFO:teuthology.orchestra.run.smithi116.stdout: "source_version": "0'0", 2020-11-20T12:18:47.096 INFO:teuthology.orchestra.run.smithi116.stdout: "target_version": "0'0" 2020-11-20T12:18:47.096 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.096 INFO:teuthology.orchestra.run.smithi116.stdout: "last_change": "55", 2020-11-20T12:18:47.096 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend": "0", 2020-11-20T12:18:47.096 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T12:18:47.097 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T12:18:47.097 INFO:teuthology.orchestra.run.smithi116.stdout: "auid": 0, 2020-11-20T12:18:47.097 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_mode": "selfmanaged", 2020-11-20T12:18:47.097 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_seq": 0, 2020-11-20T12:18:47.097 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_epoch": 0, 2020-11-20T12:18:47.098 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_snaps": [], 2020-11-20T12:18:47.098 INFO:teuthology.orchestra.run.smithi116.stdout: "removed_snaps": "[]", 2020-11-20T12:18:47.098 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_bytes": 0, 2020-11-20T12:18:47.098 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_objects": 0, 2020-11-20T12:18:47.098 INFO:teuthology.orchestra.run.smithi116.stdout: "tiers": [], 2020-11-20T12:18:47.099 INFO:teuthology.orchestra.run.smithi116.stdout: "tier_of": -1, 2020-11-20T12:18:47.099 INFO:teuthology.orchestra.run.smithi116.stdout: "read_tier": -1, 2020-11-20T12:18:47.099 INFO:teuthology.orchestra.run.smithi116.stdout: "write_tier": -1, 2020-11-20T12:18:47.099 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_mode": "none", 2020-11-20T12:18:47.099 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_bytes": 0, 2020-11-20T12:18:47.100 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_objects": 0, 2020-11-20T12:18:47.100 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T12:18:47.100 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T12:18:47.100 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T12:18:47.100 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_flush_age": 0, 2020-11-20T12:18:47.101 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_evict_age": 0, 2020-11-20T12:18:47.101 INFO:teuthology.orchestra.run.smithi116.stdout: "erasure_code_profile": "", 2020-11-20T12:18:47.101 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_params": { 2020-11-20T12:18:47.101 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "none" 2020-11-20T12:18:47.101 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.102 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_period": 0, 2020-11-20T12:18:47.102 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_count": 0, 2020-11-20T12:18:47.102 INFO:teuthology.orchestra.run.smithi116.stdout: "use_gmt_hitset": true, 2020-11-20T12:18:47.102 INFO:teuthology.orchestra.run.smithi116.stdout: "min_read_recency_for_promote": 0, 2020-11-20T12:18:47.102 INFO:teuthology.orchestra.run.smithi116.stdout: "min_write_recency_for_promote": 0, 2020-11-20T12:18:47.103 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T12:18:47.103 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_search_last_n": 0, 2020-11-20T12:18:47.103 INFO:teuthology.orchestra.run.smithi116.stdout: "grade_table": [], 2020-11-20T12:18:47.103 INFO:teuthology.orchestra.run.smithi116.stdout: "stripe_width": 0, 2020-11-20T12:18:47.103 INFO:teuthology.orchestra.run.smithi116.stdout: "expected_num_objects": 0, 2020-11-20T12:18:47.104 INFO:teuthology.orchestra.run.smithi116.stdout: "fast_read": false, 2020-11-20T12:18:47.104 INFO:teuthology.orchestra.run.smithi116.stdout: "options": { 2020-11-20T12:18:47.104 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscale_bias": 4, 2020-11-20T12:18:47.104 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_min": 16, 2020-11-20T12:18:47.104 INFO:teuthology.orchestra.run.smithi116.stdout: "recovery_priority": 5 2020-11-20T12:18:47.105 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.105 INFO:teuthology.orchestra.run.smithi116.stdout: "application_metadata": { 2020-11-20T12:18:47.105 INFO:teuthology.orchestra.run.smithi116.stdout: "cephfs": { 2020-11-20T12:18:47.105 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": "cephfs" 2020-11-20T12:18:47.105 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.106 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.106 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.106 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.106 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 10, 2020-11-20T12:18:47.106 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": "cephfs_data", 2020-11-20T12:18:47.107 INFO:teuthology.orchestra.run.smithi116.stdout: "create_time": "2020-11-20T12:18:26.151785+0000", 2020-11-20T12:18:47.107 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2020-11-20T12:18:47.107 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_names": "hashpspool", 2020-11-20T12:18:47.107 INFO:teuthology.orchestra.run.smithi116.stdout: "type": 1, 2020-11-20T12:18:47.107 INFO:teuthology.orchestra.run.smithi116.stdout: "size": 2, 2020-11-20T12:18:47.108 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": 1, 2020-11-20T12:18:47.108 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_rule": 0, 2020-11-20T12:18:47.108 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_count": 0, 2020-11-20T12:18:47.108 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_target": 0, 2020-11-20T12:18:47.108 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_barrier": 0, 2020-11-20T12:18:47.109 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-20T12:18:47.109 INFO:teuthology.orchestra.run.smithi116.stdout: "object_hash": 2, 2020-11-20T12:18:47.109 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscale_mode": "off", 2020-11-20T12:18:47.109 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": 8, 2020-11-20T12:18:47.109 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num": 8, 2020-11-20T12:18:47.110 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num_target": 8, 2020-11-20T12:18:47.110 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_target": 8, 2020-11-20T12:18:47.110 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_pending": 8, 2020-11-20T12:18:47.110 INFO:teuthology.orchestra.run.smithi116.stdout: "last_pg_merge_meta": { 2020-11-20T12:18:47.110 INFO:teuthology.orchestra.run.smithi116.stdout: "source_pgid": "0.0", 2020-11-20T12:18:47.111 INFO:teuthology.orchestra.run.smithi116.stdout: "ready_epoch": 0, 2020-11-20T12:18:47.111 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_started": 0, 2020-11-20T12:18:47.111 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_clean": 0, 2020-11-20T12:18:47.111 INFO:teuthology.orchestra.run.smithi116.stdout: "source_version": "0'0", 2020-11-20T12:18:47.111 INFO:teuthology.orchestra.run.smithi116.stdout: "target_version": "0'0" 2020-11-20T12:18:47.112 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.112 INFO:teuthology.orchestra.run.smithi116.stdout: "last_change": "56", 2020-11-20T12:18:47.112 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend": "0", 2020-11-20T12:18:47.112 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T12:18:47.112 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T12:18:47.113 INFO:teuthology.orchestra.run.smithi116.stdout: "auid": 0, 2020-11-20T12:18:47.113 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_mode": "selfmanaged", 2020-11-20T12:18:47.113 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_seq": 0, 2020-11-20T12:18:47.113 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_epoch": 0, 2020-11-20T12:18:47.113 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_snaps": [], 2020-11-20T12:18:47.114 INFO:teuthology.orchestra.run.smithi116.stdout: "removed_snaps": "[]", 2020-11-20T12:18:47.114 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_bytes": 0, 2020-11-20T12:18:47.114 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_objects": 0, 2020-11-20T12:18:47.114 INFO:teuthology.orchestra.run.smithi116.stdout: "tiers": [], 2020-11-20T12:18:47.114 INFO:teuthology.orchestra.run.smithi116.stdout: "tier_of": -1, 2020-11-20T12:18:47.115 INFO:teuthology.orchestra.run.smithi116.stdout: "read_tier": -1, 2020-11-20T12:18:47.115 INFO:teuthology.orchestra.run.smithi116.stdout: "write_tier": -1, 2020-11-20T12:18:47.115 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_mode": "none", 2020-11-20T12:18:47.115 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_bytes": 0, 2020-11-20T12:18:47.115 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_objects": 0, 2020-11-20T12:18:47.116 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T12:18:47.116 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T12:18:47.116 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T12:18:47.116 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_flush_age": 0, 2020-11-20T12:18:47.116 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_evict_age": 0, 2020-11-20T12:18:47.116 INFO:teuthology.orchestra.run.smithi116.stdout: "erasure_code_profile": "", 2020-11-20T12:18:47.117 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_params": { 2020-11-20T12:18:47.117 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "none" 2020-11-20T12:18:47.117 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.117 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_period": 0, 2020-11-20T12:18:47.118 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_count": 0, 2020-11-20T12:18:47.118 INFO:teuthology.orchestra.run.smithi116.stdout: "use_gmt_hitset": true, 2020-11-20T12:18:47.118 INFO:teuthology.orchestra.run.smithi116.stdout: "min_read_recency_for_promote": 0, 2020-11-20T12:18:47.118 INFO:teuthology.orchestra.run.smithi116.stdout: "min_write_recency_for_promote": 0, 2020-11-20T12:18:47.118 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T12:18:47.119 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_search_last_n": 0, 2020-11-20T12:18:47.119 INFO:teuthology.orchestra.run.smithi116.stdout: "grade_table": [], 2020-11-20T12:18:47.119 INFO:teuthology.orchestra.run.smithi116.stdout: "stripe_width": 0, 2020-11-20T12:18:47.119 INFO:teuthology.orchestra.run.smithi116.stdout: "expected_num_objects": 0, 2020-11-20T12:18:47.119 INFO:teuthology.orchestra.run.smithi116.stdout: "fast_read": false, 2020-11-20T12:18:47.120 INFO:teuthology.orchestra.run.smithi116.stdout: "options": {}, 2020-11-20T12:18:47.120 INFO:teuthology.orchestra.run.smithi116.stdout: "application_metadata": { 2020-11-20T12:18:47.120 INFO:teuthology.orchestra.run.smithi116.stdout: "cephfs": { 2020-11-20T12:18:47.120 INFO:teuthology.orchestra.run.smithi116.stdout: "data": "cephfs" 2020-11-20T12:18:47.120 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.120 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.121 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.121 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2020-11-20T12:18:47.121 INFO:teuthology.orchestra.run.smithi116.stdout: "osds": [ 2020-11-20T12:18:47.121 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.122 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 0, 2020-11-20T12:18:47.122 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "b46f3e7a-a64d-4724-8c3e-ea252ab40ce2", 2020-11-20T12:18:47.122 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:47.122 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:47.122 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:47.122 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:47.123 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:47.123 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:47.123 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:47.123 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 54, 2020-11-20T12:18:47.123 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:47.124 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:47.124 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:47.124 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.124 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.124 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.125 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6800", 2020-11-20T12:18:47.125 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:47.125 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.125 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.125 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.126 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6801", 2020-11-20T12:18:47.126 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:47.126 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.126 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.126 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.127 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:47.127 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.127 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.127 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.127 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6803", 2020-11-20T12:18:47.128 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:47.128 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.128 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.128 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.128 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6805", 2020-11-20T12:18:47.129 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:47.129 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.129 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.129 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.129 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:47.130 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.130 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.130 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.130 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6810", 2020-11-20T12:18:47.131 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:47.131 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.131 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.131 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.131 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6812", 2020-11-20T12:18:47.132 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:47.132 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.132 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.132 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.132 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:47.133 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.133 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.133 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.133 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6806", 2020-11-20T12:18:47.133 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:47.134 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.134 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.134 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.134 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6808", 2020-11-20T12:18:47.134 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:47.135 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.135 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.135 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.135 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.116:6801/14053", 2020-11-20T12:18:47.135 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.116:6805/14053", 2020-11-20T12:18:47.136 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.116:6812/14053", 2020-11-20T12:18:47.136 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.116:6808/14053", 2020-11-20T12:18:47.136 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:47.136 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:47.136 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:47.137 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.137 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.137 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.137 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 1, 2020-11-20T12:18:47.137 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "90b5585c-eff0-40f1-8574-01d58ad5d7d0", 2020-11-20T12:18:47.138 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:47.138 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:47.138 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:47.138 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:47.138 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:47.139 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:47.139 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:47.139 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 54, 2020-11-20T12:18:47.139 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:47.139 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:47.140 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:47.140 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.140 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.140 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.140 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6824", 2020-11-20T12:18:47.141 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:47.141 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.141 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.141 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.141 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6825", 2020-11-20T12:18:47.142 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:47.142 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.142 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.142 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.142 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:47.143 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.143 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.143 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.143 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6826", 2020-11-20T12:18:47.143 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:47.144 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.144 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.144 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.144 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6827", 2020-11-20T12:18:47.144 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:47.145 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.145 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.145 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.145 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:47.145 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.146 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.146 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.146 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6830", 2020-11-20T12:18:47.146 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:47.146 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.147 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.147 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.147 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6831", 2020-11-20T12:18:47.147 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:47.147 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.148 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.148 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.148 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:47.148 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.148 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.149 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.149 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6828", 2020-11-20T12:18:47.149 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:47.149 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.149 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.150 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.150 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6829", 2020-11-20T12:18:47.150 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:47.150 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.150 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.151 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.151 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.116:6825/14069", 2020-11-20T12:18:47.151 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.116:6827/14069", 2020-11-20T12:18:47.151 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.116:6831/14069", 2020-11-20T12:18:47.151 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.116:6829/14069", 2020-11-20T12:18:47.151 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:47.152 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:47.152 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:47.152 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.152 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.153 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.153 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 2, 2020-11-20T12:18:47.153 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "063ebbab-aa94-46f2-a8c3-774fd7d8c39c", 2020-11-20T12:18:47.153 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:47.153 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:47.154 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:47.154 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:47.154 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:47.154 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:47.154 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:47.155 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 54, 2020-11-20T12:18:47.155 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:47.155 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:47.155 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:47.155 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.156 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.156 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.156 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6814", 2020-11-20T12:18:47.156 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:47.156 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.157 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.157 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.157 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6816", 2020-11-20T12:18:47.157 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:47.157 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.158 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.158 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.158 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:47.158 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.158 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.159 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.159 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6818", 2020-11-20T12:18:47.159 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:47.159 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.159 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.160 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.160 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6819", 2020-11-20T12:18:47.160 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:47.160 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.160 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.161 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.161 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:47.161 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.161 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.161 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.162 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6822", 2020-11-20T12:18:47.162 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:47.162 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.162 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.162 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.163 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6823", 2020-11-20T12:18:47.163 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:47.163 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.163 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.163 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.164 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:47.164 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.164 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.164 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.164 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6820", 2020-11-20T12:18:47.165 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:47.165 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.165 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.165 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.165 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6821", 2020-11-20T12:18:47.166 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:47.166 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.166 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.166 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.166 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.116:6816/14054", 2020-11-20T12:18:47.167 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.116:6819/14054", 2020-11-20T12:18:47.167 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.116:6823/14054", 2020-11-20T12:18:47.167 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.116:6821/14054", 2020-11-20T12:18:47.167 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:47.167 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:47.168 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:47.168 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.168 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.168 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.168 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 3, 2020-11-20T12:18:47.169 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "5a9021b2-8618-45a1-a93d-d000a17ef73d", 2020-11-20T12:18:47.169 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:47.169 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:47.169 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:47.169 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:47.170 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:47.170 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:47.170 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:47.170 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 53, 2020-11-20T12:18:47.170 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:47.171 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:47.171 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:47.171 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.171 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.171 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.172 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6802", 2020-11-20T12:18:47.172 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:47.172 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.172 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.172 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.173 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6804", 2020-11-20T12:18:47.173 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:47.173 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.173 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.173 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.174 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:47.174 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.174 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.174 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.174 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6807", 2020-11-20T12:18:47.175 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:47.175 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.175 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.175 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.175 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6809", 2020-11-20T12:18:47.176 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:47.176 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.176 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.176 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.176 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:47.177 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.177 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.177 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.177 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6815", 2020-11-20T12:18:47.177 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:47.178 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.178 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.178 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.178 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6817", 2020-11-20T12:18:47.178 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:47.179 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.179 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.179 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.179 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:47.179 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.180 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.180 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.180 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6811", 2020-11-20T12:18:47.180 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:47.180 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.181 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.181 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.181 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6813", 2020-11-20T12:18:47.181 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:47.181 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.182 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.182 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.182 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.116:6804/14055", 2020-11-20T12:18:47.182 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.116:6809/14055", 2020-11-20T12:18:47.182 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.116:6817/14055", 2020-11-20T12:18:47.183 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.116:6813/14055", 2020-11-20T12:18:47.183 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:47.183 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:47.183 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:47.183 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.184 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.184 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.184 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 4, 2020-11-20T12:18:47.184 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "b3238680-8ed8-4a81-aa6a-20548bb44de2", 2020-11-20T12:18:47.184 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:47.184 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:47.185 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:47.185 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:47.185 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:47.185 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:47.186 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:47.186 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 54, 2020-11-20T12:18:47.186 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:47.186 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:47.186 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:47.186 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.187 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.187 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.187 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6802", 2020-11-20T12:18:47.187 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:47.187 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.188 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.188 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.188 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6803", 2020-11-20T12:18:47.188 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:47.188 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.189 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.189 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.189 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:47.189 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.190 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.190 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.190 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6804", 2020-11-20T12:18:47.190 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:47.190 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.190 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.191 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.191 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6805", 2020-11-20T12:18:47.191 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:47.191 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.192 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.192 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.192 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:47.192 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.192 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.193 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.193 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6808", 2020-11-20T12:18:47.193 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:47.193 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.193 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.194 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.194 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6809", 2020-11-20T12:18:47.194 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:47.194 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.194 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.195 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.195 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:47.195 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.195 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.195 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.196 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6806", 2020-11-20T12:18:47.196 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:47.196 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.196 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.196 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.197 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6807", 2020-11-20T12:18:47.197 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:47.197 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.197 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.197 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.198 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.173:6803/14206", 2020-11-20T12:18:47.198 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.173:6805/14206", 2020-11-20T12:18:47.198 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.173:6809/14206", 2020-11-20T12:18:47.198 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.173:6807/14206", 2020-11-20T12:18:47.198 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:47.199 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:47.199 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:47.199 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.199 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.199 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.200 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 5, 2020-11-20T12:18:47.200 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "38264535-e8e9-409b-bbc5-46f51fcfcd75", 2020-11-20T12:18:47.200 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:47.200 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:47.200 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:47.201 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:47.201 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:47.201 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:47.201 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:47.201 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 54, 2020-11-20T12:18:47.202 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:47.202 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:47.202 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:47.202 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.203 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.203 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.203 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6818", 2020-11-20T12:18:47.203 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:47.203 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.203 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.204 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.204 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6821", 2020-11-20T12:18:47.204 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:47.204 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.205 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.205 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.205 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:47.205 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.205 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.206 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.206 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6824", 2020-11-20T12:18:47.206 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:47.206 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.206 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.207 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.207 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6826", 2020-11-20T12:18:47.207 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:47.207 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.207 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.208 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.208 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:47.208 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.208 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.208 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.209 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6831", 2020-11-20T12:18:47.209 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:47.209 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.209 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.209 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.210 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6833", 2020-11-20T12:18:47.210 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:47.210 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.210 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.210 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.211 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:47.211 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.211 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.211 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.211 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6827", 2020-11-20T12:18:47.212 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:47.212 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.212 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.212 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.212 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6829", 2020-11-20T12:18:47.213 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:47.213 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.213 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.213 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.213 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.173:6821/14215", 2020-11-20T12:18:47.214 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.173:6826/14215", 2020-11-20T12:18:47.214 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.173:6833/14215", 2020-11-20T12:18:47.214 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.173:6829/14215", 2020-11-20T12:18:47.214 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:47.214 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:47.215 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:47.215 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.215 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.215 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.215 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 6, 2020-11-20T12:18:47.216 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "be530a7d-5f7c-4107-9171-15f4a6a4c468", 2020-11-20T12:18:47.216 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:47.216 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:47.216 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:47.216 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:47.217 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:47.217 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:47.217 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:47.217 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 54, 2020-11-20T12:18:47.217 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:47.218 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:47.218 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:47.218 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.218 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.218 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.219 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6810", 2020-11-20T12:18:47.219 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:47.219 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.219 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.219 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.220 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6811", 2020-11-20T12:18:47.220 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:47.220 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.220 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.220 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.221 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:47.221 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.221 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.221 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.221 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6812", 2020-11-20T12:18:47.222 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:47.222 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.222 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.222 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.222 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6813", 2020-11-20T12:18:47.223 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:47.223 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.223 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.223 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.223 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:47.224 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.224 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.224 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.224 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6820", 2020-11-20T12:18:47.224 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:47.225 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.225 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.225 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.225 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6823", 2020-11-20T12:18:47.225 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:47.226 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.226 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.226 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.226 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:47.226 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.227 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.227 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.227 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6814", 2020-11-20T12:18:47.227 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:47.227 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.228 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.228 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.228 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6816", 2020-11-20T12:18:47.228 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:47.228 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.229 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.229 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.229 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.173:6811/14207", 2020-11-20T12:18:47.229 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.173:6813/14207", 2020-11-20T12:18:47.229 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.173:6823/14207", 2020-11-20T12:18:47.230 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.173:6816/14207", 2020-11-20T12:18:47.230 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:47.230 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:47.230 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:47.230 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.231 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.231 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.231 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 7, 2020-11-20T12:18:47.231 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "949fed52-858c-459b-8fcc-317b259ed305", 2020-11-20T12:18:47.231 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:47.232 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:47.232 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:47.232 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:47.232 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:47.232 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:47.233 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:47.233 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 53, 2020-11-20T12:18:47.233 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:47.233 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:47.233 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:47.234 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.234 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.234 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.234 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6815", 2020-11-20T12:18:47.234 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:47.235 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.235 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.235 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.235 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6817", 2020-11-20T12:18:47.235 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:47.236 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.236 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.236 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.236 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:47.236 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.237 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.237 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.237 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6819", 2020-11-20T12:18:47.237 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:47.237 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.238 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.238 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.238 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6822", 2020-11-20T12:18:47.238 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:47.238 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.239 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.239 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.239 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:47.239 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.239 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.240 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.240 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6830", 2020-11-20T12:18:47.240 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:47.240 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.240 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.241 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.241 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6832", 2020-11-20T12:18:47.241 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:47.241 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.242 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.242 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.242 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:47.242 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:47.242 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.243 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:47.243 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6825", 2020-11-20T12:18:47.243 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:47.243 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.243 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.243 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:47.244 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6828", 2020-11-20T12:18:47.244 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:47.244 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.244 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.244 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.245 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.173:6817/14231", 2020-11-20T12:18:47.245 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.173:6822/14231", 2020-11-20T12:18:47.245 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.173:6832/14231", 2020-11-20T12:18:47.245 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.173:6828/14231", 2020-11-20T12:18:47.245 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:47.246 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:47.246 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:47.246 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:47.246 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.246 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2020-11-20T12:18:47.247 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_xinfo": [ 2020-11-20T12:18:47.247 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.247 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 0, 2020-11-20T12:18:47.247 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:47.247 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:47.248 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:47.248 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:47.248 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:47.248 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.290082+0000", 2020-11-20T12:18:47.248 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:47.249 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.249 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.249 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 1, 2020-11-20T12:18:47.249 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:47.249 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:47.250 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:47.250 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:47.250 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:47.250 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.346769+0000", 2020-11-20T12:18:47.250 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:47.251 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.251 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.251 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 2, 2020-11-20T12:18:47.251 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:47.251 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:47.252 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:47.252 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:47.252 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:47.252 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.335366+0000", 2020-11-20T12:18:47.252 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:47.253 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.253 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.253 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 3, 2020-11-20T12:18:47.253 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:47.253 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:47.254 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:47.254 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:47.254 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:47.254 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.261673+0000", 2020-11-20T12:18:47.254 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:47.255 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.255 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.255 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 4, 2020-11-20T12:18:47.255 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:47.255 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:47.256 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:47.256 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:47.256 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:47.256 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.289965+0000", 2020-11-20T12:18:47.256 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:47.257 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.257 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.257 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 5, 2020-11-20T12:18:47.257 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:47.257 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:47.258 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:47.258 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:47.258 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:47.258 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.300182+0000", 2020-11-20T12:18:47.258 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:47.259 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.259 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.259 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 6, 2020-11-20T12:18:47.259 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:47.259 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:47.260 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:47.260 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:47.260 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:47.260 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.376325+0000", 2020-11-20T12:18:47.260 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:47.261 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.261 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:47.261 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 7, 2020-11-20T12:18:47.261 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:47.261 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:47.262 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:47.262 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:47.262 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:47.262 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.431267+0000", 2020-11-20T12:18:47.262 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:47.263 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.263 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2020-11-20T12:18:47.263 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_upmap": [], 2020-11-20T12:18:47.263 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_upmap_items": [], 2020-11-20T12:18:47.263 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_temp": [], 2020-11-20T12:18:47.264 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_temp": [], 2020-11-20T12:18:47.264 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklist": {}, 2020-11-20T12:18:47.264 INFO:teuthology.orchestra.run.smithi116.stdout: "erasure_code_profiles": { 2020-11-20T12:18:47.264 INFO:teuthology.orchestra.run.smithi116.stdout: "default": { 2020-11-20T12:18:47.264 INFO:teuthology.orchestra.run.smithi116.stdout: "crush-failure-domain": "osd", 2020-11-20T12:18:47.265 INFO:teuthology.orchestra.run.smithi116.stdout: "k": "2", 2020-11-20T12:18:47.265 INFO:teuthology.orchestra.run.smithi116.stdout: "m": "1", 2020-11-20T12:18:47.265 INFO:teuthology.orchestra.run.smithi116.stdout: "plugin": "jerasure", 2020-11-20T12:18:47.265 INFO:teuthology.orchestra.run.smithi116.stdout: "ruleset-failure-domain": "osd", 2020-11-20T12:18:47.265 INFO:teuthology.orchestra.run.smithi116.stdout: "technique": "reed_sol_van" 2020-11-20T12:18:47.266 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.266 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:47.266 INFO:teuthology.orchestra.run.smithi116.stdout: "removed_snaps_queue": [], 2020-11-20T12:18:47.266 INFO:teuthology.orchestra.run.smithi116.stdout: "new_removed_snaps": [], 2020-11-20T12:18:47.266 INFO:teuthology.orchestra.run.smithi116.stdout: "new_purged_snaps": [], 2020-11-20T12:18:47.267 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_node_flags": {}, 2020-11-20T12:18:47.267 INFO:teuthology.orchestra.run.smithi116.stdout: "device_class_flags": {}, 2020-11-20T12:18:47.267 INFO:teuthology.orchestra.run.smithi116.stdout: "stretch_mode": { 2020-11-20T12:18:47.267 INFO:teuthology.orchestra.run.smithi116.stdout: "stretch_mode_enabled": false, 2020-11-20T12:18:47.267 INFO:teuthology.orchestra.run.smithi116.stdout: "stretch_bucket_count": 0, 2020-11-20T12:18:47.268 INFO:teuthology.orchestra.run.smithi116.stdout: "degraded_stretch_mode": 0, 2020-11-20T12:18:47.268 INFO:teuthology.orchestra.run.smithi116.stdout: "recovering_stretch_mode": 0, 2020-11-20T12:18:47.268 INFO:teuthology.orchestra.run.smithi116.stdout: "stretch_mode_bucket": 0 2020-11-20T12:18:47.268 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:47.268 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:18:47.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 -- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44d40557c0 msgr2=0x7f44d4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:47.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44d40557c0 0x7f44d4057c80 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f44fc0934f0 tx=0x7f44ec015000).stop 2020-11-20T12:18:47.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 -- 172.21.15.116:0/2106318570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44fc0a95d0 msgr2=0x7f44fc0941d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:47.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44fc0a95d0 0x7f44fc0941d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f44f800c690 tx=0x7f44f80017f0).stop 2020-11-20T12:18:47.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 -- 172.21.15.116:0/2106318570 shutdown_connections 2020-11-20T12:18:47.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44d40557c0 0x7f44d4057c80 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:47.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44fc0c3dd0 0x7f44fc141840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:47.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44fc0a95d0 0x7f44fc0941d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:47.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 --2- 172.21.15.116:0/2106318570 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f44fc0c39f0 0x7f44fc141350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:47.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 -- 172.21.15.116:0/2106318570 >> 172.21.15.116:0/2106318570 conn(0x7f44fc0c0de0 msgr2=0x7f44fc14f250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:47.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 -- 172.21.15.116:0/2106318570 shutdown_connections 2020-11-20T12:18:47.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:47.056+0000 7f4503657700 1 -- 172.21.15.116:0/2106318570 wait complete. 2020-11-20T12:18:47.273 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:18:47.274 INFO:teuthology.orchestra.run.smithi173:> from __future__ import print_function 2020-11-20T12:18:47.274 INFO:teuthology.orchestra.run.smithi173:> from ceph_volume_client import CephFSVolumeClient, VolumePath 2020-11-20T12:18:47.274 INFO:teuthology.orchestra.run.smithi173:> from sys import version_info as sys_version_info 2020-11-20T12:18:47.275 INFO:teuthology.orchestra.run.smithi173:> from rados import OSError as rados_OSError 2020-11-20T12:18:47.275 INFO:teuthology.orchestra.run.smithi173:> import logging 2020-11-20T12:18:47.276 INFO:teuthology.orchestra.run.smithi173:> log = logging.getLogger("ceph_volume_client") 2020-11-20T12:18:47.276 INFO:teuthology.orchestra.run.smithi173:> log.addHandler(logging.StreamHandler()) 2020-11-20T12:18:47.276 INFO:teuthology.orchestra.run.smithi173:> log.setLevel(logging.DEBUG) 2020-11-20T12:18:47.277 INFO:teuthology.orchestra.run.smithi173:> vc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None) 2020-11-20T12:18:47.277 INFO:teuthology.orchestra.run.smithi173:> vc.connect() 2020-11-20T12:18:47.277 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:18:47.278 INFO:teuthology.orchestra.run.smithi173:> vp = VolumePath("grpid", "volid") 2020-11-20T12:18:47.278 INFO:teuthology.orchestra.run.smithi173:> vc.create_volume(vp, data_isolated=True) 2020-11-20T12:18:47.279 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:18:47.279 INFO:teuthology.orchestra.run.smithi173:> vc.disconnect() 2020-11-20T12:18:47.279 INFO:teuthology.orchestra.run.smithi173:> ' 2020-11-20T12:18:47.461 INFO:teuthology.orchestra.run.smithi173.stderr:Configuring to RADOS with config /etc/ceph/ceph.conf... 2020-11-20T12:18:47.465 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to RADOS... 2020-11-20T12:18:47.478 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to RADOS complete 2020-11-20T12:18:47.478 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to cephfs... 2020-11-20T12:18:47.479 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS initializing... 2020-11-20T12:18:47.484 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS mounting... 2020-11-20T12:18:47.496 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to cephfs complete 2020-11-20T12:18:47.496 INFO:teuthology.orchestra.run.smithi173.stderr:Recovering from partial auth updates (if any)... 2020-11-20T12:18:47.497 INFO:teuthology.orchestra.run.smithi173.stderr:Nothing to recover. No auth meta files. 2020-11-20T12:18:47.497 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: /volumes/grpid/volid 2020-11-20T12:18:47.509 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: grpid/volid, create pool fsvolume_volid as data_isolated =True. 2020-11-20T12:18:53.551 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: grpid/volid, using rados namespace fsvolumens_volid to isolate data. 2020-11-20T12:18:53.560 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:18:53.561 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs... 2020-11-20T12:18:53.580 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs complete 2020-11-20T12:18:53.581 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados... 2020-11-20T12:18:53.584 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados complete 2020-11-20T12:18:53.589 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:18:53.797 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2020-11-20T12:18:53.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.984+0000 7fabfe8bc700 1 Processor -- start 2020-11-20T12:18:53.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.984+0000 7fabfe8bc700 1 -- start start 2020-11-20T12:18:53.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.984+0000 7fabfe8bc700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fabf80a91f0 0x7fabf814c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:53.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.984+0000 7fabfe8bc700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf80aa1f0 con 0x7fabf80a6360 2020-11-20T12:18:53.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.984+0000 7fabfe8bc700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf8068770 con 0x7fabf80a6740 2020-11-20T12:18:53.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.984+0000 7fabfe8bc700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf80a9770 con 0x7fabf80a91f0 2020-11-20T12:18:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.984+0000 7fabf7fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fabf80a6360 0x7fabf814b900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56676/0 (socket says 172.21.15.116:56676) 2020-11-20T12:18:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.984+0000 7fabf7fff700 1 -- 172.21.15.116:0/3181868589 learned_addr learned my addr 172.21.15.116:0/3181868589 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.984+0000 7fabeffff700 1 --2- 172.21.15.116:0/3181868589 >> v2:172.21.15.173:3300/0 conn(0x7fabf80a91f0 0x7fabf814c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:53.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.984+0000 7fabf77fe700 1 -- 172.21.15.116:0/3181868589 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2003736730 0 0) 0x7fabf80aa1f0 con 0x7fabf80a6360 2020-11-20T12:18:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.984+0000 7fabf77fe700 1 -- 172.21.15.116:0/3181868589 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabd8003240 con 0x7fabf80a6360 2020-11-20T12:18:53.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabeffff700 1 -- 172.21.15.116:0/3181868589 >> v1:172.21.15.173:6789/0 conn(0x7fabf80a6740 legacy=0x7fabf8066670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:53.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabeffff700 1 -- 172.21.15.116:0/3181868589 >> v1:172.21.15.116:6789/0 conn(0x7fabf80a6360 legacy=0x7fabf814b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:53.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabf77fe700 1 -- 172.21.15.116:0/3181868589 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3734288496 0 0) 0x7fabd8003240 con 0x7fabf80a6360 2020-11-20T12:18:53.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabeffff700 1 -- 172.21.15.116:0/3181868589 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf80a9a10 con 0x7fabf80a91f0 2020-11-20T12:18:53.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabeffff700 1 --2- 172.21.15.116:0/3181868589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf80a91f0 0x7fabf814c010 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fabe000b9a0 tx=0x7fabe0006df0).ready entity=mon.1 client_cookie=a85aaa188f426e76 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:53.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabf77fe700 1 -- 172.21.15.116:0/3181868589 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe80014e0 con 0x7fabf80a91f0 2020-11-20T12:18:53.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabf77fe700 1 -- 172.21.15.116:0/3181868589 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe0009530 con 0x7fabf80a91f0 2020-11-20T12:18:53.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabf77fe700 1 -- 172.21.15.116:0/3181868589 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe00098b0 con 0x7fabf80a91f0 2020-11-20T12:18:53.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabfe8bc700 1 -- 172.21.15.116:0/3181868589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf80a91f0 msgr2=0x7fabf814c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:53.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabfe8bc700 1 --2- 172.21.15.116:0/3181868589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf80a91f0 0x7fabf814c010 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7fabe000b9a0 tx=0x7fabe0006df0).stop 2020-11-20T12:18:53.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabfe8bc700 1 -- 172.21.15.116:0/3181868589 shutdown_connections 2020-11-20T12:18:53.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabfe8bc700 1 --2- 172.21.15.116:0/3181868589 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf80a91f0 0x7fabf814c010 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:53.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabfe8bc700 1 -- 172.21.15.116:0/3181868589 >> 172.21.15.116:0/3181868589 conn(0x7fabf80c0de0 msgr2=0x7fabf80c11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:53.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabfe8bc700 1 -- 172.21.15.116:0/3181868589 shutdown_connections 2020-11-20T12:18:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.988+0000 7fabfe8bc700 1 -- 172.21.15.116:0/3181868589 wait complete. 2020-11-20T12:18:53.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfe8bc700 1 Processor -- start 2020-11-20T12:18:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfe8bc700 1 -- start start 2020-11-20T12:18:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfe8bc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf80a6360 0x7fabf80bf430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:53.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfe8bc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf80a6740 0x7fabf814edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfe8bc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf80a91f0 0x7fabf8150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfe8bc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8161270 con 0x7fabf80a91f0 2020-11-20T12:18:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfe8bc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8164d30 con 0x7fabf80a6360 2020-11-20T12:18:53.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfe8bc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf8164fd0 con 0x7fabf80a6740 2020-11-20T12:18:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfce59700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf80a91f0 0x7fabf8150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabeffff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf80a6740 0x7fabf814edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfce59700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf80a91f0 0x7fabf8150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47252/0 (socket says 172.21.15.116:47252) 2020-11-20T12:18:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfce59700 1 -- 172.21.15.116:0/1851968676 learned_addr learned my addr 172.21.15.116:0/1851968676 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:53.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabf7fff700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf80a6360 0x7fabf80bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:53.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfce59700 1 -- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf80a6740 msgr2=0x7fabf814edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:53.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfce59700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf80a6740 0x7fabf814edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:53.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.992+0000 7fabfce59700 1 -- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf80a6360 msgr2=0x7fabf80bf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:53.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.996+0000 7fabfce59700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf80a6360 0x7fabf80bf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:53.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.996+0000 7fabfce59700 1 -- 172.21.15.116:0/1851968676 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf8165270 con 0x7fabf80a91f0 2020-11-20T12:18:53.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.996+0000 7fabfce59700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf80a91f0 0x7fabf8150f30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fabf000ca60 tx=0x7fabf0007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:53.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.996+0000 7fabf4ff9700 1 -- 172.21.15.116:0/1851968676 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf000a770 con 0x7fabf80a91f0 2020-11-20T12:18:53.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.996+0000 7fabf4ff9700 1 -- 172.21.15.116:0/1851968676 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabf000a8d0 con 0x7fabf80a91f0 2020-11-20T12:18:53.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.996+0000 7fabfe8bc700 1 -- 172.21.15.116:0/1851968676 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf8165510 con 0x7fabf80a91f0 2020-11-20T12:18:53.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.996+0000 7fabf4ff9700 1 -- 172.21.15.116:0/1851968676 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabf000ac50 con 0x7fabf80a91f0 2020-11-20T12:18:53.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.996+0000 7fabfe8bc700 1 -- 172.21.15.116:0/1851968676 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf814f2e0 con 0x7fabf80a91f0 2020-11-20T12:18:53.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.996+0000 7fabfe8bc700 1 -- 172.21.15.116:0/1851968676 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf80660d0 con 0x7fabf80a91f0 2020-11-20T12:18:53.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:53.996+0000 7fabf4ff9700 1 -- 172.21.15.116:0/1851968676 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fabf0019030 con 0x7fabf80a91f0 2020-11-20T12:18:54.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.000+0000 7fabf4ff9700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd40557c0 0x7fabd4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:54.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.000+0000 7fabf4ff9700 1 -- 172.21.15.116:0/1851968676 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fabf006dfd0 con 0x7fabf80a91f0 2020-11-20T12:18:54.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.000+0000 7fabf7fff700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd40557c0 0x7fabd4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:54.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.000+0000 7fabf7fff700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd40557c0 0x7fabd4057c80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fabe8001e90 tx=0x7fabe8001d30).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:54.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.008+0000 7fabf4ff9700 1 -- 172.21.15.116:0/1851968676 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fabf80660d0 con 0x7fabf80a91f0 2020-11-20T12:18:54.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.168+0000 7fabfe8bc700 1 -- 172.21.15.116:0/1851968676 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fabf80660d0 con 0x7fabf80a91f0 2020-11-20T12:18:54.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.172+0000 7fabf4ff9700 1 -- 172.21.15.116:0/1851968676 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v59) v1 ==== 81+0+37399 (secure 0 0 0) 0x7fabf002a230 con 0x7fabf80a91f0 2020-11-20T12:18:54.171 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.171 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:18:54.171 INFO:teuthology.orchestra.run.smithi116.stdout: "epoch": 59, 2020-11-20T12:18:54.172 INFO:teuthology.orchestra.run.smithi116.stdout: "fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", 2020-11-20T12:18:54.172 INFO:teuthology.orchestra.run.smithi116.stdout: "created": "2020-11-20T12:15:41.360400+0000", 2020-11-20T12:18:54.172 INFO:teuthology.orchestra.run.smithi116.stdout: "modified": "2020-11-20T12:18:49.533460+0000", 2020-11-20T12:18:54.173 INFO:teuthology.orchestra.run.smithi116.stdout: "last_up_change": "2020-11-20T12:15:50.549644+0000", 2020-11-20T12:18:54.173 INFO:teuthology.orchestra.run.smithi116.stdout: "last_in_change": "2020-11-20T12:15:50.549644+0000", 2020-11-20T12:18:54.174 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2020-11-20T12:18:54.174 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_num": 5799936, 2020-11-20T12:18:54.174 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_set": [ 2020-11-20T12:18:54.175 INFO:teuthology.orchestra.run.smithi116.stdout: "pglog_hardlimit", 2020-11-20T12:18:54.175 INFO:teuthology.orchestra.run.smithi116.stdout: "purged_snapdirs", 2020-11-20T12:18:54.176 INFO:teuthology.orchestra.run.smithi116.stdout: "recovery_deletes", 2020-11-20T12:18:54.176 INFO:teuthology.orchestra.run.smithi116.stdout: "sortbitwise" 2020-11-20T12:18:54.177 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2020-11-20T12:18:54.177 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_version": 4, 2020-11-20T12:18:54.177 INFO:teuthology.orchestra.run.smithi116.stdout: "full_ratio": 0.94999998807907104, 2020-11-20T12:18:54.178 INFO:teuthology.orchestra.run.smithi116.stdout: "backfillfull_ratio": 0.89999997615814209, 2020-11-20T12:18:54.178 INFO:teuthology.orchestra.run.smithi116.stdout: "nearfull_ratio": 0.85000002384185791, 2020-11-20T12:18:54.178 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_snapshot": "", 2020-11-20T12:18:54.179 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_max": 11, 2020-11-20T12:18:54.179 INFO:teuthology.orchestra.run.smithi116.stdout: "max_osd": 8, 2020-11-20T12:18:54.179 INFO:teuthology.orchestra.run.smithi116.stdout: "require_min_compat_client": "luminous", 2020-11-20T12:18:54.180 INFO:teuthology.orchestra.run.smithi116.stdout: "min_compat_client": "jewel", 2020-11-20T12:18:54.180 INFO:teuthology.orchestra.run.smithi116.stdout: "require_osd_release": "pacific", 2020-11-20T12:18:54.180 INFO:teuthology.orchestra.run.smithi116.stdout: "pools": [ 2020-11-20T12:18:54.181 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.181 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 1, 2020-11-20T12:18:54.181 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": "device_health_metrics", 2020-11-20T12:18:54.182 INFO:teuthology.orchestra.run.smithi116.stdout: "create_time": "2020-11-20T12:15:46.607758+0000", 2020-11-20T12:18:54.182 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2020-11-20T12:18:54.182 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_names": "hashpspool", 2020-11-20T12:18:54.183 INFO:teuthology.orchestra.run.smithi116.stdout: "type": 1, 2020-11-20T12:18:54.183 INFO:teuthology.orchestra.run.smithi116.stdout: "size": 2, 2020-11-20T12:18:54.184 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": 1, 2020-11-20T12:18:54.184 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_rule": 0, 2020-11-20T12:18:54.184 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_count": 0, 2020-11-20T12:18:54.185 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_target": 0, 2020-11-20T12:18:54.186 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_barrier": 0, 2020-11-20T12:18:54.186 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-20T12:18:54.187 INFO:teuthology.orchestra.run.smithi116.stdout: "object_hash": 2, 2020-11-20T12:18:54.188 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscale_mode": "off", 2020-11-20T12:18:54.188 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": 1, 2020-11-20T12:18:54.189 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num": 1, 2020-11-20T12:18:54.189 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num_target": 1, 2020-11-20T12:18:54.189 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_target": 1, 2020-11-20T12:18:54.190 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_pending": 1, 2020-11-20T12:18:54.190 INFO:teuthology.orchestra.run.smithi116.stdout: "last_pg_merge_meta": { 2020-11-20T12:18:54.191 INFO:teuthology.orchestra.run.smithi116.stdout: "source_pgid": "0.0", 2020-11-20T12:18:54.192 INFO:teuthology.orchestra.run.smithi116.stdout: "ready_epoch": 0, 2020-11-20T12:18:54.192 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_started": 0, 2020-11-20T12:18:54.192 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_clean": 0, 2020-11-20T12:18:54.193 INFO:teuthology.orchestra.run.smithi116.stdout: "source_version": "0'0", 2020-11-20T12:18:54.193 INFO:teuthology.orchestra.run.smithi116.stdout: "target_version": "0'0" 2020-11-20T12:18:54.194 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.195 INFO:teuthology.orchestra.run.smithi116.stdout: "last_change": "16", 2020-11-20T12:18:54.195 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend": "0", 2020-11-20T12:18:54.195 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T12:18:54.195 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T12:18:54.196 INFO:teuthology.orchestra.run.smithi116.stdout: "auid": 0, 2020-11-20T12:18:54.196 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_mode": "selfmanaged", 2020-11-20T12:18:54.196 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_seq": 0, 2020-11-20T12:18:54.196 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_epoch": 0, 2020-11-20T12:18:54.196 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_snaps": [], 2020-11-20T12:18:54.197 INFO:teuthology.orchestra.run.smithi116.stdout: "removed_snaps": "[]", 2020-11-20T12:18:54.197 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_bytes": 0, 2020-11-20T12:18:54.197 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_objects": 0, 2020-11-20T12:18:54.197 INFO:teuthology.orchestra.run.smithi116.stdout: "tiers": [], 2020-11-20T12:18:54.198 INFO:teuthology.orchestra.run.smithi116.stdout: "tier_of": -1, 2020-11-20T12:18:54.198 INFO:teuthology.orchestra.run.smithi116.stdout: "read_tier": -1, 2020-11-20T12:18:54.198 INFO:teuthology.orchestra.run.smithi116.stdout: "write_tier": -1, 2020-11-20T12:18:54.198 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_mode": "none", 2020-11-20T12:18:54.198 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_bytes": 0, 2020-11-20T12:18:54.199 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_objects": 0, 2020-11-20T12:18:54.199 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T12:18:54.199 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T12:18:54.199 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T12:18:54.199 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_flush_age": 0, 2020-11-20T12:18:54.200 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_evict_age": 0, 2020-11-20T12:18:54.200 INFO:teuthology.orchestra.run.smithi116.stdout: "erasure_code_profile": "", 2020-11-20T12:18:54.200 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_params": { 2020-11-20T12:18:54.200 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "none" 2020-11-20T12:18:54.200 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.201 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_period": 0, 2020-11-20T12:18:54.201 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_count": 0, 2020-11-20T12:18:54.201 INFO:teuthology.orchestra.run.smithi116.stdout: "use_gmt_hitset": true, 2020-11-20T12:18:54.201 INFO:teuthology.orchestra.run.smithi116.stdout: "min_read_recency_for_promote": 0, 2020-11-20T12:18:54.202 INFO:teuthology.orchestra.run.smithi116.stdout: "min_write_recency_for_promote": 0, 2020-11-20T12:18:54.202 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T12:18:54.202 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_search_last_n": 0, 2020-11-20T12:18:54.202 INFO:teuthology.orchestra.run.smithi116.stdout: "grade_table": [], 2020-11-20T12:18:54.202 INFO:teuthology.orchestra.run.smithi116.stdout: "stripe_width": 0, 2020-11-20T12:18:54.203 INFO:teuthology.orchestra.run.smithi116.stdout: "expected_num_objects": 0, 2020-11-20T12:18:54.203 INFO:teuthology.orchestra.run.smithi116.stdout: "fast_read": false, 2020-11-20T12:18:54.203 INFO:teuthology.orchestra.run.smithi116.stdout: "options": { 2020-11-20T12:18:54.203 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_min": 1 2020-11-20T12:18:54.204 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.204 INFO:teuthology.orchestra.run.smithi116.stdout: "application_metadata": { 2020-11-20T12:18:54.204 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr_devicehealth": {} 2020-11-20T12:18:54.204 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.204 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.205 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.205 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 2, 2020-11-20T12:18:54.205 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": "rbd", 2020-11-20T12:18:54.205 INFO:teuthology.orchestra.run.smithi116.stdout: "create_time": "2020-11-20T12:15:55.044020+0000", 2020-11-20T12:18:54.206 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2020-11-20T12:18:54.206 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_names": "hashpspool", 2020-11-20T12:18:54.206 INFO:teuthology.orchestra.run.smithi116.stdout: "type": 1, 2020-11-20T12:18:54.206 INFO:teuthology.orchestra.run.smithi116.stdout: "size": 2, 2020-11-20T12:18:54.206 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": 1, 2020-11-20T12:18:54.207 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_rule": 0, 2020-11-20T12:18:54.207 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_count": 0, 2020-11-20T12:18:54.207 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_target": 0, 2020-11-20T12:18:54.207 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_barrier": 0, 2020-11-20T12:18:54.207 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-20T12:18:54.208 INFO:teuthology.orchestra.run.smithi116.stdout: "object_hash": 2, 2020-11-20T12:18:54.208 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscale_mode": "off", 2020-11-20T12:18:54.208 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": 8, 2020-11-20T12:18:54.208 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num": 8, 2020-11-20T12:18:54.208 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num_target": 8, 2020-11-20T12:18:54.209 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_target": 8, 2020-11-20T12:18:54.209 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_pending": 8, 2020-11-20T12:18:54.209 INFO:teuthology.orchestra.run.smithi116.stdout: "last_pg_merge_meta": { 2020-11-20T12:18:54.209 INFO:teuthology.orchestra.run.smithi116.stdout: "source_pgid": "0.0", 2020-11-20T12:18:54.210 INFO:teuthology.orchestra.run.smithi116.stdout: "ready_epoch": 0, 2020-11-20T12:18:54.210 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_started": 0, 2020-11-20T12:18:54.210 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_clean": 0, 2020-11-20T12:18:54.210 INFO:teuthology.orchestra.run.smithi116.stdout: "source_version": "0'0", 2020-11-20T12:18:54.210 INFO:teuthology.orchestra.run.smithi116.stdout: "target_version": "0'0" 2020-11-20T12:18:54.211 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.211 INFO:teuthology.orchestra.run.smithi116.stdout: "last_change": "19", 2020-11-20T12:18:54.211 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend": "0", 2020-11-20T12:18:54.211 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T12:18:54.211 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T12:18:54.212 INFO:teuthology.orchestra.run.smithi116.stdout: "auid": 0, 2020-11-20T12:18:54.212 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_mode": "selfmanaged", 2020-11-20T12:18:54.212 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_seq": 0, 2020-11-20T12:18:54.212 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_epoch": 0, 2020-11-20T12:18:54.212 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_snaps": [], 2020-11-20T12:18:54.213 INFO:teuthology.orchestra.run.smithi116.stdout: "removed_snaps": "[]", 2020-11-20T12:18:54.213 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_bytes": 0, 2020-11-20T12:18:54.213 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_objects": 0, 2020-11-20T12:18:54.213 INFO:teuthology.orchestra.run.smithi116.stdout: "tiers": [], 2020-11-20T12:18:54.214 INFO:teuthology.orchestra.run.smithi116.stdout: "tier_of": -1, 2020-11-20T12:18:54.214 INFO:teuthology.orchestra.run.smithi116.stdout: "read_tier": -1, 2020-11-20T12:18:54.214 INFO:teuthology.orchestra.run.smithi116.stdout: "write_tier": -1, 2020-11-20T12:18:54.214 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_mode": "none", 2020-11-20T12:18:54.214 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_bytes": 0, 2020-11-20T12:18:54.215 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_objects": 0, 2020-11-20T12:18:54.215 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T12:18:54.215 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T12:18:54.215 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T12:18:54.215 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_flush_age": 0, 2020-11-20T12:18:54.216 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_evict_age": 0, 2020-11-20T12:18:54.216 INFO:teuthology.orchestra.run.smithi116.stdout: "erasure_code_profile": "", 2020-11-20T12:18:54.216 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_params": { 2020-11-20T12:18:54.216 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "none" 2020-11-20T12:18:54.216 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.217 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_period": 0, 2020-11-20T12:18:54.217 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_count": 0, 2020-11-20T12:18:54.217 INFO:teuthology.orchestra.run.smithi116.stdout: "use_gmt_hitset": true, 2020-11-20T12:18:54.217 INFO:teuthology.orchestra.run.smithi116.stdout: "min_read_recency_for_promote": 0, 2020-11-20T12:18:54.218 INFO:teuthology.orchestra.run.smithi116.stdout: "min_write_recency_for_promote": 0, 2020-11-20T12:18:54.218 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T12:18:54.218 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_search_last_n": 0, 2020-11-20T12:18:54.218 INFO:teuthology.orchestra.run.smithi116.stdout: "grade_table": [], 2020-11-20T12:18:54.218 INFO:teuthology.orchestra.run.smithi116.stdout: "stripe_width": 0, 2020-11-20T12:18:54.219 INFO:teuthology.orchestra.run.smithi116.stdout: "expected_num_objects": 0, 2020-11-20T12:18:54.219 INFO:teuthology.orchestra.run.smithi116.stdout: "fast_read": false, 2020-11-20T12:18:54.219 INFO:teuthology.orchestra.run.smithi116.stdout: "options": {}, 2020-11-20T12:18:54.219 INFO:teuthology.orchestra.run.smithi116.stdout: "application_metadata": { 2020-11-20T12:18:54.219 INFO:teuthology.orchestra.run.smithi116.stdout: "rbd": {} 2020-11-20T12:18:54.220 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.220 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.220 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.220 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 9, 2020-11-20T12:18:54.220 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": "cephfs_metadata", 2020-11-20T12:18:54.229 INFO:teuthology.orchestra.run.smithi116.stdout: "create_time": "2020-11-20T12:18:25.558441+0000", 2020-11-20T12:18:54.229 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2020-11-20T12:18:54.229 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_names": "hashpspool", 2020-11-20T12:18:54.230 INFO:teuthology.orchestra.run.smithi116.stdout: "type": 1, 2020-11-20T12:18:54.230 INFO:teuthology.orchestra.run.smithi116.stdout: "size": 2, 2020-11-20T12:18:54.230 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": 1, 2020-11-20T12:18:54.230 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_rule": 0, 2020-11-20T12:18:54.230 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_count": 0, 2020-11-20T12:18:54.231 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_target": 0, 2020-11-20T12:18:54.231 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_barrier": 0, 2020-11-20T12:18:54.231 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-20T12:18:54.231 INFO:teuthology.orchestra.run.smithi116.stdout: "object_hash": 2, 2020-11-20T12:18:54.232 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscale_mode": "off", 2020-11-20T12:18:54.232 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": 8, 2020-11-20T12:18:54.232 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num": 8, 2020-11-20T12:18:54.232 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num_target": 8, 2020-11-20T12:18:54.232 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_target": 8, 2020-11-20T12:18:54.233 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_pending": 8, 2020-11-20T12:18:54.233 INFO:teuthology.orchestra.run.smithi116.stdout: "last_pg_merge_meta": { 2020-11-20T12:18:54.233 INFO:teuthology.orchestra.run.smithi116.stdout: "source_pgid": "0.0", 2020-11-20T12:18:54.233 INFO:teuthology.orchestra.run.smithi116.stdout: "ready_epoch": 0, 2020-11-20T12:18:54.233 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_started": 0, 2020-11-20T12:18:54.234 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_clean": 0, 2020-11-20T12:18:54.234 INFO:teuthology.orchestra.run.smithi116.stdout: "source_version": "0'0", 2020-11-20T12:18:54.234 INFO:teuthology.orchestra.run.smithi116.stdout: "target_version": "0'0" 2020-11-20T12:18:54.234 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.234 INFO:teuthology.orchestra.run.smithi116.stdout: "last_change": "55", 2020-11-20T12:18:54.235 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend": "0", 2020-11-20T12:18:54.235 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T12:18:54.235 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T12:18:54.235 INFO:teuthology.orchestra.run.smithi116.stdout: "auid": 0, 2020-11-20T12:18:54.235 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_mode": "selfmanaged", 2020-11-20T12:18:54.236 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_seq": 0, 2020-11-20T12:18:54.236 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_epoch": 0, 2020-11-20T12:18:54.236 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_snaps": [], 2020-11-20T12:18:54.236 INFO:teuthology.orchestra.run.smithi116.stdout: "removed_snaps": "[]", 2020-11-20T12:18:54.236 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_bytes": 0, 2020-11-20T12:18:54.237 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_objects": 0, 2020-11-20T12:18:54.237 INFO:teuthology.orchestra.run.smithi116.stdout: "tiers": [], 2020-11-20T12:18:54.237 INFO:teuthology.orchestra.run.smithi116.stdout: "tier_of": -1, 2020-11-20T12:18:54.237 INFO:teuthology.orchestra.run.smithi116.stdout: "read_tier": -1, 2020-11-20T12:18:54.237 INFO:teuthology.orchestra.run.smithi116.stdout: "write_tier": -1, 2020-11-20T12:18:54.238 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_mode": "none", 2020-11-20T12:18:54.238 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_bytes": 0, 2020-11-20T12:18:54.238 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_objects": 0, 2020-11-20T12:18:54.238 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T12:18:54.238 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T12:18:54.239 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T12:18:54.239 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_flush_age": 0, 2020-11-20T12:18:54.239 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_evict_age": 0, 2020-11-20T12:18:54.239 INFO:teuthology.orchestra.run.smithi116.stdout: "erasure_code_profile": "", 2020-11-20T12:18:54.239 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_params": { 2020-11-20T12:18:54.240 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "none" 2020-11-20T12:18:54.240 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.240 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_period": 0, 2020-11-20T12:18:54.240 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_count": 0, 2020-11-20T12:18:54.241 INFO:teuthology.orchestra.run.smithi116.stdout: "use_gmt_hitset": true, 2020-11-20T12:18:54.241 INFO:teuthology.orchestra.run.smithi116.stdout: "min_read_recency_for_promote": 0, 2020-11-20T12:18:54.241 INFO:teuthology.orchestra.run.smithi116.stdout: "min_write_recency_for_promote": 0, 2020-11-20T12:18:54.241 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T12:18:54.241 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_search_last_n": 0, 2020-11-20T12:18:54.242 INFO:teuthology.orchestra.run.smithi116.stdout: "grade_table": [], 2020-11-20T12:18:54.242 INFO:teuthology.orchestra.run.smithi116.stdout: "stripe_width": 0, 2020-11-20T12:18:54.242 INFO:teuthology.orchestra.run.smithi116.stdout: "expected_num_objects": 0, 2020-11-20T12:18:54.242 INFO:teuthology.orchestra.run.smithi116.stdout: "fast_read": false, 2020-11-20T12:18:54.242 INFO:teuthology.orchestra.run.smithi116.stdout: "options": { 2020-11-20T12:18:54.243 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscale_bias": 4, 2020-11-20T12:18:54.243 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_min": 16, 2020-11-20T12:18:54.243 INFO:teuthology.orchestra.run.smithi116.stdout: "recovery_priority": 5 2020-11-20T12:18:54.243 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.243 INFO:teuthology.orchestra.run.smithi116.stdout: "application_metadata": { 2020-11-20T12:18:54.244 INFO:teuthology.orchestra.run.smithi116.stdout: "cephfs": { 2020-11-20T12:18:54.244 INFO:teuthology.orchestra.run.smithi116.stdout: "metadata": "cephfs" 2020-11-20T12:18:54.244 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.244 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.244 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.245 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.245 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 10, 2020-11-20T12:18:54.245 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": "cephfs_data", 2020-11-20T12:18:54.245 INFO:teuthology.orchestra.run.smithi116.stdout: "create_time": "2020-11-20T12:18:26.151785+0000", 2020-11-20T12:18:54.245 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2020-11-20T12:18:54.246 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_names": "hashpspool", 2020-11-20T12:18:54.246 INFO:teuthology.orchestra.run.smithi116.stdout: "type": 1, 2020-11-20T12:18:54.246 INFO:teuthology.orchestra.run.smithi116.stdout: "size": 2, 2020-11-20T12:18:54.246 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": 1, 2020-11-20T12:18:54.246 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_rule": 0, 2020-11-20T12:18:54.247 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_count": 0, 2020-11-20T12:18:54.247 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_target": 0, 2020-11-20T12:18:54.247 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_barrier": 0, 2020-11-20T12:18:54.247 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-20T12:18:54.247 INFO:teuthology.orchestra.run.smithi116.stdout: "object_hash": 2, 2020-11-20T12:18:54.248 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscale_mode": "off", 2020-11-20T12:18:54.248 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": 8, 2020-11-20T12:18:54.248 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num": 8, 2020-11-20T12:18:54.248 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num_target": 8, 2020-11-20T12:18:54.248 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_target": 8, 2020-11-20T12:18:54.249 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_pending": 8, 2020-11-20T12:18:54.249 INFO:teuthology.orchestra.run.smithi116.stdout: "last_pg_merge_meta": { 2020-11-20T12:18:54.249 INFO:teuthology.orchestra.run.smithi116.stdout: "source_pgid": "0.0", 2020-11-20T12:18:54.249 INFO:teuthology.orchestra.run.smithi116.stdout: "ready_epoch": 0, 2020-11-20T12:18:54.249 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_started": 0, 2020-11-20T12:18:54.250 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_clean": 0, 2020-11-20T12:18:54.250 INFO:teuthology.orchestra.run.smithi116.stdout: "source_version": "0'0", 2020-11-20T12:18:54.250 INFO:teuthology.orchestra.run.smithi116.stdout: "target_version": "0'0" 2020-11-20T12:18:54.250 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.250 INFO:teuthology.orchestra.run.smithi116.stdout: "last_change": "56", 2020-11-20T12:18:54.251 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend": "0", 2020-11-20T12:18:54.251 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T12:18:54.251 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T12:18:54.251 INFO:teuthology.orchestra.run.smithi116.stdout: "auid": 0, 2020-11-20T12:18:54.251 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_mode": "selfmanaged", 2020-11-20T12:18:54.252 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_seq": 0, 2020-11-20T12:18:54.252 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_epoch": 0, 2020-11-20T12:18:54.252 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_snaps": [], 2020-11-20T12:18:54.252 INFO:teuthology.orchestra.run.smithi116.stdout: "removed_snaps": "[]", 2020-11-20T12:18:54.252 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_bytes": 0, 2020-11-20T12:18:54.253 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_objects": 0, 2020-11-20T12:18:54.253 INFO:teuthology.orchestra.run.smithi116.stdout: "tiers": [], 2020-11-20T12:18:54.253 INFO:teuthology.orchestra.run.smithi116.stdout: "tier_of": -1, 2020-11-20T12:18:54.253 INFO:teuthology.orchestra.run.smithi116.stdout: "read_tier": -1, 2020-11-20T12:18:54.253 INFO:teuthology.orchestra.run.smithi116.stdout: "write_tier": -1, 2020-11-20T12:18:54.254 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_mode": "none", 2020-11-20T12:18:54.254 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_bytes": 0, 2020-11-20T12:18:54.254 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_objects": 0, 2020-11-20T12:18:54.254 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T12:18:54.254 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T12:18:54.255 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T12:18:54.255 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_flush_age": 0, 2020-11-20T12:18:54.255 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_evict_age": 0, 2020-11-20T12:18:54.255 INFO:teuthology.orchestra.run.smithi116.stdout: "erasure_code_profile": "", 2020-11-20T12:18:54.255 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_params": { 2020-11-20T12:18:54.256 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "none" 2020-11-20T12:18:54.256 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.256 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_period": 0, 2020-11-20T12:18:54.256 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_count": 0, 2020-11-20T12:18:54.256 INFO:teuthology.orchestra.run.smithi116.stdout: "use_gmt_hitset": true, 2020-11-20T12:18:54.257 INFO:teuthology.orchestra.run.smithi116.stdout: "min_read_recency_for_promote": 0, 2020-11-20T12:18:54.257 INFO:teuthology.orchestra.run.smithi116.stdout: "min_write_recency_for_promote": 0, 2020-11-20T12:18:54.257 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T12:18:54.257 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_search_last_n": 0, 2020-11-20T12:18:54.258 INFO:teuthology.orchestra.run.smithi116.stdout: "grade_table": [], 2020-11-20T12:18:54.258 INFO:teuthology.orchestra.run.smithi116.stdout: "stripe_width": 0, 2020-11-20T12:18:54.258 INFO:teuthology.orchestra.run.smithi116.stdout: "expected_num_objects": 0, 2020-11-20T12:18:54.258 INFO:teuthology.orchestra.run.smithi116.stdout: "fast_read": false, 2020-11-20T12:18:54.258 INFO:teuthology.orchestra.run.smithi116.stdout: "options": {}, 2020-11-20T12:18:54.259 INFO:teuthology.orchestra.run.smithi116.stdout: "application_metadata": { 2020-11-20T12:18:54.259 INFO:teuthology.orchestra.run.smithi116.stdout: "cephfs": { 2020-11-20T12:18:54.259 INFO:teuthology.orchestra.run.smithi116.stdout: "data": "cephfs" 2020-11-20T12:18:54.259 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.259 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.260 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.260 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.260 INFO:teuthology.orchestra.run.smithi116.stdout: "pool": 11, 2020-11-20T12:18:54.260 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_name": "fsvolume_volid", 2020-11-20T12:18:54.260 INFO:teuthology.orchestra.run.smithi116.stdout: "create_time": "2020-11-20T12:18:47.595737+0000", 2020-11-20T12:18:54.261 INFO:teuthology.orchestra.run.smithi116.stdout: "flags": 1, 2020-11-20T12:18:54.261 INFO:teuthology.orchestra.run.smithi116.stdout: "flags_names": "hashpspool", 2020-11-20T12:18:54.261 INFO:teuthology.orchestra.run.smithi116.stdout: "type": 1, 2020-11-20T12:18:54.261 INFO:teuthology.orchestra.run.smithi116.stdout: "size": 2, 2020-11-20T12:18:54.261 INFO:teuthology.orchestra.run.smithi116.stdout: "min_size": 1, 2020-11-20T12:18:54.262 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_rule": 0, 2020-11-20T12:18:54.262 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_count": 0, 2020-11-20T12:18:54.262 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_target": 0, 2020-11-20T12:18:54.262 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_barrier": 0, 2020-11-20T12:18:54.262 INFO:teuthology.orchestra.run.smithi116.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2020-11-20T12:18:54.263 INFO:teuthology.orchestra.run.smithi116.stdout: "object_hash": 2, 2020-11-20T12:18:54.263 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_autoscale_mode": "off", 2020-11-20T12:18:54.263 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num": 32, 2020-11-20T12:18:54.263 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num": 32, 2020-11-20T12:18:54.263 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_placement_num_target": 32, 2020-11-20T12:18:54.264 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_target": 32, 2020-11-20T12:18:54.264 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_num_pending": 32, 2020-11-20T12:18:54.264 INFO:teuthology.orchestra.run.smithi116.stdout: "last_pg_merge_meta": { 2020-11-20T12:18:54.264 INFO:teuthology.orchestra.run.smithi116.stdout: "source_pgid": "0.0", 2020-11-20T12:18:54.264 INFO:teuthology.orchestra.run.smithi116.stdout: "ready_epoch": 0, 2020-11-20T12:18:54.265 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_started": 0, 2020-11-20T12:18:54.265 INFO:teuthology.orchestra.run.smithi116.stdout: "last_epoch_clean": 0, 2020-11-20T12:18:54.265 INFO:teuthology.orchestra.run.smithi116.stdout: "source_version": "0'0", 2020-11-20T12:18:54.265 INFO:teuthology.orchestra.run.smithi116.stdout: "target_version": "0'0" 2020-11-20T12:18:54.265 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.266 INFO:teuthology.orchestra.run.smithi116.stdout: "last_change": "59", 2020-11-20T12:18:54.266 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend": "0", 2020-11-20T12:18:54.266 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_prenautilus": "0", 2020-11-20T12:18:54.266 INFO:teuthology.orchestra.run.smithi116.stdout: "last_force_op_resend_preluminous": "0", 2020-11-20T12:18:54.266 INFO:teuthology.orchestra.run.smithi116.stdout: "auid": 0, 2020-11-20T12:18:54.267 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_mode": "selfmanaged", 2020-11-20T12:18:54.267 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_seq": 0, 2020-11-20T12:18:54.267 INFO:teuthology.orchestra.run.smithi116.stdout: "snap_epoch": 0, 2020-11-20T12:18:54.267 INFO:teuthology.orchestra.run.smithi116.stdout: "pool_snaps": [], 2020-11-20T12:18:54.267 INFO:teuthology.orchestra.run.smithi116.stdout: "removed_snaps": "[]", 2020-11-20T12:18:54.268 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_bytes": 0, 2020-11-20T12:18:54.268 INFO:teuthology.orchestra.run.smithi116.stdout: "quota_max_objects": 0, 2020-11-20T12:18:54.268 INFO:teuthology.orchestra.run.smithi116.stdout: "tiers": [], 2020-11-20T12:18:54.268 INFO:teuthology.orchestra.run.smithi116.stdout: "tier_of": -1, 2020-11-20T12:18:54.268 INFO:teuthology.orchestra.run.smithi116.stdout: "read_tier": -1, 2020-11-20T12:18:54.269 INFO:teuthology.orchestra.run.smithi116.stdout: "write_tier": -1, 2020-11-20T12:18:54.269 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_mode": "none", 2020-11-20T12:18:54.269 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_bytes": 0, 2020-11-20T12:18:54.269 INFO:teuthology.orchestra.run.smithi116.stdout: "target_max_objects": 0, 2020-11-20T12:18:54.269 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_ratio_micro": 400000, 2020-11-20T12:18:54.270 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2020-11-20T12:18:54.270 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_target_full_ratio_micro": 800000, 2020-11-20T12:18:54.270 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_flush_age": 0, 2020-11-20T12:18:54.270 INFO:teuthology.orchestra.run.smithi116.stdout: "cache_min_evict_age": 0, 2020-11-20T12:18:54.270 INFO:teuthology.orchestra.run.smithi116.stdout: "erasure_code_profile": "", 2020-11-20T12:18:54.271 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_params": { 2020-11-20T12:18:54.271 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "none" 2020-11-20T12:18:54.271 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.271 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_period": 0, 2020-11-20T12:18:54.271 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_count": 0, 2020-11-20T12:18:54.272 INFO:teuthology.orchestra.run.smithi116.stdout: "use_gmt_hitset": true, 2020-11-20T12:18:54.272 INFO:teuthology.orchestra.run.smithi116.stdout: "min_read_recency_for_promote": 0, 2020-11-20T12:18:54.272 INFO:teuthology.orchestra.run.smithi116.stdout: "min_write_recency_for_promote": 0, 2020-11-20T12:18:54.272 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_grade_decay_rate": 0, 2020-11-20T12:18:54.272 INFO:teuthology.orchestra.run.smithi116.stdout: "hit_set_search_last_n": 0, 2020-11-20T12:18:54.273 INFO:teuthology.orchestra.run.smithi116.stdout: "grade_table": [], 2020-11-20T12:18:54.273 INFO:teuthology.orchestra.run.smithi116.stdout: "stripe_width": 0, 2020-11-20T12:18:54.273 INFO:teuthology.orchestra.run.smithi116.stdout: "expected_num_objects": 0, 2020-11-20T12:18:54.273 INFO:teuthology.orchestra.run.smithi116.stdout: "fast_read": false, 2020-11-20T12:18:54.273 INFO:teuthology.orchestra.run.smithi116.stdout: "options": {}, 2020-11-20T12:18:54.274 INFO:teuthology.orchestra.run.smithi116.stdout: "application_metadata": { 2020-11-20T12:18:54.274 INFO:teuthology.orchestra.run.smithi116.stdout: "cephfs": { 2020-11-20T12:18:54.274 INFO:teuthology.orchestra.run.smithi116.stdout: "data": "cephfs" 2020-11-20T12:18:54.274 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.274 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.275 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.275 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2020-11-20T12:18:54.275 INFO:teuthology.orchestra.run.smithi116.stdout: "osds": [ 2020-11-20T12:18:54.275 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.275 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 0, 2020-11-20T12:18:54.276 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "b46f3e7a-a64d-4724-8c3e-ea252ab40ce2", 2020-11-20T12:18:54.276 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:54.276 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:54.276 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:54.276 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:54.277 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:54.277 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:54.277 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:54.277 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 57, 2020-11-20T12:18:54.278 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:54.278 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:54.278 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:54.278 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.278 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.279 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.279 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6800", 2020-11-20T12:18:54.279 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:54.279 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.279 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.280 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.280 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6801", 2020-11-20T12:18:54.280 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:54.281 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.282 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.282 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.282 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:54.282 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.283 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.283 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.283 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6803", 2020-11-20T12:18:54.283 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:54.283 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.284 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.284 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.284 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6805", 2020-11-20T12:18:54.284 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:54.284 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.285 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.285 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.285 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:54.285 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.285 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.286 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.286 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6810", 2020-11-20T12:18:54.286 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:54.286 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.286 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.287 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.287 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6812", 2020-11-20T12:18:54.287 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:54.287 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.288 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.288 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.288 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:54.288 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.288 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.289 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.289 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6806", 2020-11-20T12:18:54.289 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:54.289 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.289 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.290 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.290 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6808", 2020-11-20T12:18:54.290 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14053 2020-11-20T12:18:54.290 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.291 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.291 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.291 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.116:6801/14053", 2020-11-20T12:18:54.291 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.116:6805/14053", 2020-11-20T12:18:54.292 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.116:6812/14053", 2020-11-20T12:18:54.292 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.116:6808/14053", 2020-11-20T12:18:54.292 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:54.292 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:54.292 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:54.293 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.293 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.293 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.293 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 1, 2020-11-20T12:18:54.293 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "90b5585c-eff0-40f1-8574-01d58ad5d7d0", 2020-11-20T12:18:54.294 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:54.294 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:54.294 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:54.294 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:54.294 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:54.295 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:54.295 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:54.295 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 57, 2020-11-20T12:18:54.295 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:54.295 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:54.296 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:54.296 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.296 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.296 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.296 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6824", 2020-11-20T12:18:54.297 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:54.297 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.297 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.297 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.297 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6825", 2020-11-20T12:18:54.298 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:54.298 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.298 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.298 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.298 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:54.299 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.299 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.299 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.299 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6826", 2020-11-20T12:18:54.299 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:54.300 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.300 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.300 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.300 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6827", 2020-11-20T12:18:54.300 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:54.301 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.301 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.301 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.301 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:54.301 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.302 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.302 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.302 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6830", 2020-11-20T12:18:54.302 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:54.302 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.303 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.303 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.303 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6831", 2020-11-20T12:18:54.303 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:54.303 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.304 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.304 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.304 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:54.304 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.304 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.305 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.305 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6828", 2020-11-20T12:18:54.305 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:54.305 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.305 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.306 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.306 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6829", 2020-11-20T12:18:54.306 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14069 2020-11-20T12:18:54.306 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.306 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.307 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.307 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.116:6825/14069", 2020-11-20T12:18:54.307 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.116:6827/14069", 2020-11-20T12:18:54.307 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.116:6831/14069", 2020-11-20T12:18:54.307 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.116:6829/14069", 2020-11-20T12:18:54.308 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:54.308 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:54.308 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:54.308 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.308 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.309 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.309 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 2, 2020-11-20T12:18:54.309 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "063ebbab-aa94-46f2-a8c3-774fd7d8c39c", 2020-11-20T12:18:54.309 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:54.309 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:54.310 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:54.310 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:54.310 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:54.310 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:54.310 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:54.311 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 57, 2020-11-20T12:18:54.311 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:54.311 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:54.311 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:54.311 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.312 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.312 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.312 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6814", 2020-11-20T12:18:54.312 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:54.312 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.313 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.313 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.313 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6816", 2020-11-20T12:18:54.315 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:54.315 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.315 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.315 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.315 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:54.316 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.316 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.316 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.316 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6818", 2020-11-20T12:18:54.317 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:54.317 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.317 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.317 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.317 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6819", 2020-11-20T12:18:54.318 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:54.318 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.318 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.318 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.318 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:54.319 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.319 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.319 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.319 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6822", 2020-11-20T12:18:54.319 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:54.320 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.320 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.320 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.320 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6823", 2020-11-20T12:18:54.320 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:54.321 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.321 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.321 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.321 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:54.321 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.322 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.322 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.322 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6820", 2020-11-20T12:18:54.322 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:54.322 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.323 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.323 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.323 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6821", 2020-11-20T12:18:54.323 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14054 2020-11-20T12:18:54.323 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.324 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.324 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.324 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.116:6816/14054", 2020-11-20T12:18:54.324 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.116:6819/14054", 2020-11-20T12:18:54.324 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.116:6823/14054", 2020-11-20T12:18:54.325 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.116:6821/14054", 2020-11-20T12:18:54.325 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:54.325 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:54.325 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:54.325 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.326 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.326 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.326 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 3, 2020-11-20T12:18:54.326 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "5a9021b2-8618-45a1-a93d-d000a17ef73d", 2020-11-20T12:18:54.326 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:54.327 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:54.327 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:54.327 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:54.327 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:54.327 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:54.328 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:54.328 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 57, 2020-11-20T12:18:54.328 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:54.328 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:54.328 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:54.329 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.329 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.329 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.329 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6802", 2020-11-20T12:18:54.329 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:54.330 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.330 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.330 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.330 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6804", 2020-11-20T12:18:54.330 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:54.331 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.331 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.331 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.331 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:54.331 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.332 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.332 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.332 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6807", 2020-11-20T12:18:54.332 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:54.332 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.333 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.333 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.333 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6809", 2020-11-20T12:18:54.333 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:54.334 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.334 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.334 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.334 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:54.334 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.335 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.335 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.335 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6815", 2020-11-20T12:18:54.335 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:54.335 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.336 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.336 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.336 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6817", 2020-11-20T12:18:54.336 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:54.336 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.337 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.337 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.337 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:54.337 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.337 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.338 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.338 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6811", 2020-11-20T12:18:54.338 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:54.338 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.338 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.339 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.339 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.116:6813", 2020-11-20T12:18:54.339 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14055 2020-11-20T12:18:54.339 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.339 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.340 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.340 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.116:6804/14055", 2020-11-20T12:18:54.340 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.116:6809/14055", 2020-11-20T12:18:54.340 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.116:6817/14055", 2020-11-20T12:18:54.340 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.116:6813/14055", 2020-11-20T12:18:54.341 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:54.341 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:54.341 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:54.341 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.341 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.342 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.342 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 4, 2020-11-20T12:18:54.342 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "b3238680-8ed8-4a81-aa6a-20548bb44de2", 2020-11-20T12:18:54.342 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:54.342 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:54.343 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:54.343 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:54.343 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:54.343 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:54.343 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:54.344 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 57, 2020-11-20T12:18:54.344 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:54.344 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:54.344 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:54.344 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.345 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.345 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.345 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6802", 2020-11-20T12:18:54.345 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:54.346 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.346 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.346 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.346 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6803", 2020-11-20T12:18:54.346 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:54.347 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.347 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.347 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.347 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:54.347 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.348 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.348 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.348 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6804", 2020-11-20T12:18:54.348 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:54.348 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.349 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.349 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.349 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6805", 2020-11-20T12:18:54.349 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:54.349 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.350 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.350 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.350 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:54.350 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.350 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.351 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.351 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6808", 2020-11-20T12:18:54.351 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:54.351 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.351 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.352 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.352 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6809", 2020-11-20T12:18:54.352 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:54.352 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.352 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.353 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.353 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:54.353 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.353 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.353 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.354 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6806", 2020-11-20T12:18:54.354 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:54.354 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.354 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.354 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.355 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6807", 2020-11-20T12:18:54.355 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14206 2020-11-20T12:18:54.355 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.355 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.355 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.356 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.173:6803/14206", 2020-11-20T12:18:54.356 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.173:6805/14206", 2020-11-20T12:18:54.356 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.173:6809/14206", 2020-11-20T12:18:54.356 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.173:6807/14206", 2020-11-20T12:18:54.356 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:54.357 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:54.357 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:54.357 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.357 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.357 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.358 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 5, 2020-11-20T12:18:54.358 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "38264535-e8e9-409b-bbc5-46f51fcfcd75", 2020-11-20T12:18:54.358 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:54.358 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:54.358 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:54.359 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:54.359 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:54.359 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:54.359 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:54.359 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 57, 2020-11-20T12:18:54.360 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:54.360 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:54.360 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:54.360 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.360 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.361 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.361 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6818", 2020-11-20T12:18:54.361 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:54.361 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.361 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.362 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.362 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6821", 2020-11-20T12:18:54.362 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:54.362 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.362 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.363 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.363 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:54.363 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.363 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.363 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.364 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6824", 2020-11-20T12:18:54.364 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:54.364 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.364 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.364 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.365 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6826", 2020-11-20T12:18:54.365 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:54.365 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.365 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.365 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.366 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:54.366 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.366 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.366 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.366 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6831", 2020-11-20T12:18:54.367 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:54.367 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.367 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.367 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.367 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6833", 2020-11-20T12:18:54.368 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:54.368 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.368 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.368 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.368 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:54.369 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.369 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.369 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.369 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6827", 2020-11-20T12:18:54.370 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:54.370 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.370 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.370 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.370 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6829", 2020-11-20T12:18:54.371 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14215 2020-11-20T12:18:54.371 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.371 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.371 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.371 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.173:6821/14215", 2020-11-20T12:18:54.372 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.173:6826/14215", 2020-11-20T12:18:54.372 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.173:6833/14215", 2020-11-20T12:18:54.372 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.173:6829/14215", 2020-11-20T12:18:54.372 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:54.372 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:54.373 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:54.373 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.373 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.373 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.373 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 6, 2020-11-20T12:18:54.374 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "be530a7d-5f7c-4107-9171-15f4a6a4c468", 2020-11-20T12:18:54.374 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:54.374 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:54.374 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:54.374 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:54.375 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:54.375 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:54.375 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:54.375 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 57, 2020-11-20T12:18:54.375 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:54.376 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:54.376 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:54.376 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.376 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.376 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.377 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6810", 2020-11-20T12:18:54.377 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:54.377 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.377 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.377 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.378 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6811", 2020-11-20T12:18:54.378 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:54.378 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.378 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.378 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.379 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:54.379 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.379 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.379 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.379 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6812", 2020-11-20T12:18:54.380 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:54.380 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.380 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.380 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.380 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6813", 2020-11-20T12:18:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.381 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:54.382 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.382 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.382 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.382 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6820", 2020-11-20T12:18:54.382 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:54.382 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.383 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.383 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.383 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6823", 2020-11-20T12:18:54.383 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:54.383 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.384 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.384 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.384 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:54.384 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.384 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.385 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.385 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6814", 2020-11-20T12:18:54.385 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:54.385 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.386 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.386 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.386 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6816", 2020-11-20T12:18:54.386 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14207 2020-11-20T12:18:54.386 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.387 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.387 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.387 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.173:6811/14207", 2020-11-20T12:18:54.387 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.173:6813/14207", 2020-11-20T12:18:54.387 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.173:6823/14207", 2020-11-20T12:18:54.387 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.173:6816/14207", 2020-11-20T12:18:54.388 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:54.388 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:54.388 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:54.388 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.389 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.389 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.389 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 7, 2020-11-20T12:18:54.389 INFO:teuthology.orchestra.run.smithi116.stdout: "uuid": "949fed52-858c-459b-8fcc-317b259ed305", 2020-11-20T12:18:54.389 INFO:teuthology.orchestra.run.smithi116.stdout: "up": 1, 2020-11-20T12:18:54.390 INFO:teuthology.orchestra.run.smithi116.stdout: "in": 1, 2020-11-20T12:18:54.390 INFO:teuthology.orchestra.run.smithi116.stdout: "weight": 1, 2020-11-20T12:18:54.390 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_affinity": 1, 2020-11-20T12:18:54.390 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_begin": 0, 2020-11-20T12:18:54.390 INFO:teuthology.orchestra.run.smithi116.stdout: "last_clean_end": 0, 2020-11-20T12:18:54.391 INFO:teuthology.orchestra.run.smithi116.stdout: "up_from": 14, 2020-11-20T12:18:54.391 INFO:teuthology.orchestra.run.smithi116.stdout: "up_thru": 57, 2020-11-20T12:18:54.391 INFO:teuthology.orchestra.run.smithi116.stdout: "down_at": 0, 2020-11-20T12:18:54.391 INFO:teuthology.orchestra.run.smithi116.stdout: "lost_at": 0, 2020-11-20T12:18:54.391 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addrs": { 2020-11-20T12:18:54.392 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.392 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.392 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.392 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6815", 2020-11-20T12:18:54.392 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:54.393 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.393 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.393 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.393 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6817", 2020-11-20T12:18:54.393 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:54.394 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.394 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.394 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.394 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addrs": { 2020-11-20T12:18:54.394 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.395 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.395 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.395 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6819", 2020-11-20T12:18:54.395 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:54.395 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.396 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.396 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.396 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6822", 2020-11-20T12:18:54.396 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:54.396 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.397 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.397 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.397 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addrs": { 2020-11-20T12:18:54.397 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.397 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.398 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.398 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6830", 2020-11-20T12:18:54.398 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:54.398 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.398 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.399 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.399 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6832", 2020-11-20T12:18:54.399 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:54.399 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.399 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.400 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.400 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addrs": { 2020-11-20T12:18:54.400 INFO:teuthology.orchestra.run.smithi116.stdout: "addrvec": [ 2020-11-20T12:18:54.400 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.400 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v2", 2020-11-20T12:18:54.401 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6825", 2020-11-20T12:18:54.401 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:54.401 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.401 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.401 INFO:teuthology.orchestra.run.smithi116.stdout: "type": "v1", 2020-11-20T12:18:54.402 INFO:teuthology.orchestra.run.smithi116.stdout: "addr": "172.21.15.173:6828", 2020-11-20T12:18:54.402 INFO:teuthology.orchestra.run.smithi116.stdout: "nonce": 14231 2020-11-20T12:18:54.402 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.402 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.402 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.403 INFO:teuthology.orchestra.run.smithi116.stdout: "public_addr": "172.21.15.173:6817/14231", 2020-11-20T12:18:54.403 INFO:teuthology.orchestra.run.smithi116.stdout: "cluster_addr": "172.21.15.173:6822/14231", 2020-11-20T12:18:54.403 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_back_addr": "172.21.15.173:6832/14231", 2020-11-20T12:18:54.403 INFO:teuthology.orchestra.run.smithi116.stdout: "heartbeat_front_addr": "172.21.15.173:6828/14231", 2020-11-20T12:18:54.403 INFO:teuthology.orchestra.run.smithi116.stdout: "state": [ 2020-11-20T12:18:54.404 INFO:teuthology.orchestra.run.smithi116.stdout: "exists", 2020-11-20T12:18:54.404 INFO:teuthology.orchestra.run.smithi116.stdout: "up" 2020-11-20T12:18:54.404 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:18:54.404 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.404 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2020-11-20T12:18:54.405 INFO:teuthology.orchestra.run.smithi116.stdout: "osd_xinfo": [ 2020-11-20T12:18:54.405 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.405 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 0, 2020-11-20T12:18:54.405 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:54.405 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:54.406 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:54.406 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:54.406 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:54.406 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.290082+0000", 2020-11-20T12:18:54.406 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:54.407 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.407 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.407 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 1, 2020-11-20T12:18:54.407 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:54.407 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:54.408 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:54.408 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:54.408 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:54.408 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.346769+0000", 2020-11-20T12:18:54.408 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:54.409 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.409 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.409 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 2, 2020-11-20T12:18:54.409 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:54.409 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:54.410 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:54.410 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:54.410 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:54.410 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.335366+0000", 2020-11-20T12:18:54.410 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:54.411 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.411 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.411 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 3, 2020-11-20T12:18:54.411 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:54.412 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:54.412 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:54.412 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:54.412 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:54.412 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.261673+0000", 2020-11-20T12:18:54.413 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:54.413 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.413 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.413 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 4, 2020-11-20T12:18:54.413 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:54.414 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:54.414 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:54.414 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:54.414 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:54.414 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.289965+0000", 2020-11-20T12:18:54.415 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:54.415 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.415 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.415 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 5, 2020-11-20T12:18:54.415 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:54.416 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:54.416 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:54.416 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:54.416 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:54.416 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.300182+0000", 2020-11-20T12:18:54.417 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:54.417 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.417 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.417 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 6, 2020-11-20T12:18:54.417 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:54.418 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:54.418 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:54.418 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:54.418 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:54.418 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.376325+0000", 2020-11-20T12:18:54.419 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:54.419 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.419 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:18:54.419 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": 7, 2020-11-20T12:18:54.419 INFO:teuthology.orchestra.run.smithi116.stdout: "down_stamp": "0.000000", 2020-11-20T12:18:54.420 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_probability": 0, 2020-11-20T12:18:54.420 INFO:teuthology.orchestra.run.smithi116.stdout: "laggy_interval": 0, 2020-11-20T12:18:54.420 INFO:teuthology.orchestra.run.smithi116.stdout: "features": 4540138297136906239, 2020-11-20T12:18:54.420 INFO:teuthology.orchestra.run.smithi116.stdout: "old_weight": 0, 2020-11-20T12:18:54.420 INFO:teuthology.orchestra.run.smithi116.stdout: "last_purged_snaps_scrub": "2020-11-20T12:15:49.431267+0000", 2020-11-20T12:18:54.421 INFO:teuthology.orchestra.run.smithi116.stdout: "dead_epoch": 0 2020-11-20T12:18:54.421 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.421 INFO:teuthology.orchestra.run.smithi116.stdout: ], 2020-11-20T12:18:54.421 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_upmap": [], 2020-11-20T12:18:54.421 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_upmap_items": [], 2020-11-20T12:18:54.422 INFO:teuthology.orchestra.run.smithi116.stdout: "pg_temp": [], 2020-11-20T12:18:54.422 INFO:teuthology.orchestra.run.smithi116.stdout: "primary_temp": [], 2020-11-20T12:18:54.422 INFO:teuthology.orchestra.run.smithi116.stdout: "blocklist": {}, 2020-11-20T12:18:54.422 INFO:teuthology.orchestra.run.smithi116.stdout: "erasure_code_profiles": { 2020-11-20T12:18:54.422 INFO:teuthology.orchestra.run.smithi116.stdout: "default": { 2020-11-20T12:18:54.423 INFO:teuthology.orchestra.run.smithi116.stdout: "crush-failure-domain": "osd", 2020-11-20T12:18:54.423 INFO:teuthology.orchestra.run.smithi116.stdout: "k": "2", 2020-11-20T12:18:54.423 INFO:teuthology.orchestra.run.smithi116.stdout: "m": "1", 2020-11-20T12:18:54.423 INFO:teuthology.orchestra.run.smithi116.stdout: "plugin": "jerasure", 2020-11-20T12:18:54.423 INFO:teuthology.orchestra.run.smithi116.stdout: "ruleset-failure-domain": "osd", 2020-11-20T12:18:54.424 INFO:teuthology.orchestra.run.smithi116.stdout: "technique": "reed_sol_van" 2020-11-20T12:18:54.424 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.424 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:18:54.424 INFO:teuthology.orchestra.run.smithi116.stdout: "removed_snaps_queue": [], 2020-11-20T12:18:54.424 INFO:teuthology.orchestra.run.smithi116.stdout: "new_removed_snaps": [], 2020-11-20T12:18:54.425 INFO:teuthology.orchestra.run.smithi116.stdout: "new_purged_snaps": [], 2020-11-20T12:18:54.425 INFO:teuthology.orchestra.run.smithi116.stdout: "crush_node_flags": {}, 2020-11-20T12:18:54.425 INFO:teuthology.orchestra.run.smithi116.stdout: "device_class_flags": {}, 2020-11-20T12:18:54.425 INFO:teuthology.orchestra.run.smithi116.stdout: "stretch_mode": { 2020-11-20T12:18:54.425 INFO:teuthology.orchestra.run.smithi116.stdout: "stretch_mode_enabled": false, 2020-11-20T12:18:54.426 INFO:teuthology.orchestra.run.smithi116.stdout: "stretch_bucket_count": 0, 2020-11-20T12:18:54.426 INFO:teuthology.orchestra.run.smithi116.stdout: "degraded_stretch_mode": 0, 2020-11-20T12:18:54.426 INFO:teuthology.orchestra.run.smithi116.stdout: "recovering_stretch_mode": 0, 2020-11-20T12:18:54.426 INFO:teuthology.orchestra.run.smithi116.stdout: "stretch_mode_bucket": 0 2020-11-20T12:18:54.426 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:18:54.427 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:18:54.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.172+0000 7fabfe8bc700 1 -- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd40557c0 msgr2=0x7fabd4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:54.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.172+0000 7fabfe8bc700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd40557c0 0x7fabd4057c80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7fabe8001e90 tx=0x7fabe8001d30).stop 2020-11-20T12:18:54.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.172+0000 7fabfe8bc700 1 -- 172.21.15.116:0/1851968676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf80a91f0 msgr2=0x7fabf8150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:54.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.172+0000 7fabfe8bc700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf80a91f0 0x7fabf8150f30 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fabf000ca60 tx=0x7fabf0007f40).stop 2020-11-20T12:18:54.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.176+0000 7fabfe8bc700 1 -- 172.21.15.116:0/1851968676 shutdown_connections 2020-11-20T12:18:54.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.176+0000 7fabfe8bc700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabd40557c0 0x7fabd4057c80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:54.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.176+0000 7fabfe8bc700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabf80a91f0 0x7fabf8150f30 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:54.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.176+0000 7fabfe8bc700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabf80a6360 0x7fabf80bf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:54.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.176+0000 7fabfe8bc700 1 --2- 172.21.15.116:0/1851968676 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabf80a6740 0x7fabf814edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:54.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.176+0000 7fabfe8bc700 1 -- 172.21.15.116:0/1851968676 >> 172.21.15.116:0/1851968676 conn(0x7fabf80c0de0 msgr2=0x7fabf80c11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:54.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.176+0000 7fabfe8bc700 1 -- 172.21.15.116:0/1851968676 shutdown_connections 2020-11-20T12:18:54.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:54.176+0000 7fabfe8bc700 1 -- 172.21.15.116:0/1851968676 wait complete. 2020-11-20T12:18:54.431 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:18:54.432 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:18:54.473 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:18:54.512 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:18:54.512 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-20T12:18:54.513 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:18:54.513 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-20T12:18:54.596 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:18:54.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:54.598 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:18:54.615 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2020-11-20T12:18:54.616 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:18:54.616 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.2 2020-11-20T12:18:54.725 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:18:54.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:54.726 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T12:18:54.746 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2020-11-20T12:18:54.747 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:18:54.747 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.3 2020-11-20T12:18:54.826 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:18:54.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:18:54.827 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T12:18:54.846 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:54.853 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:18:54.920 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:18:54.921 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:18:54.921 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:54.921 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:18:54.921 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.921 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:18:54.922 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.922 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:54.922 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.922 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:54.923 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:18:54.923 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.923 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:18:54.923 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:18:54.923 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.924 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:54.924 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:54.924 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.924 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:54.924 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:54.925 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.925 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:18:54.925 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.925 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:54.925 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:18:54.926 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:54.926 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:54.926 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:54.926 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:54.926 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:18:54.927 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:54.927 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:54.927 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:54.927 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.928 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:54.928 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.928 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:18:54.928 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.929 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:18:54.929 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.929 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:18:54.929 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:18:54.930 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:18:54.930 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:54.930 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:18:54.930 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:54.931 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.931 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:18:54.931 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:18:54.931 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.931 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:54.932 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:18:54.932 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.932 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:18:54.932 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:54.932 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.933 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:18:54.933 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:18:54.933 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:54.933 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.933 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:18:54.934 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:18:54.934 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.934 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:54.934 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.934 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:18:54.935 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.935 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:18:54.935 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.935 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:54.935 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:54.936 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.936 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:18:54.936 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:18:54.936 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:18:54.937 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:18:54.937 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:18:54.937 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:18:54.937 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:18:54.937 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:18:54.938 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:18:54.938 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:18:54.938 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:18:54.938 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:18:54.938 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:18:54.939 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:18:54.939 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:54.939 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:54.939 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:54.939 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:54.940 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:18:54.940 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:54.940 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.940 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:18:54.940 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:54.941 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:18:54.941 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:54.941 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:54.941 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:54.942 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:54.942 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:54.942 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.942 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:18:54.943 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:54.943 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:18:54.943 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:18:54.943 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:18:54.943 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:18:54.944 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:54.944 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:54.944 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:54.944 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:18:54.945 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:54.945 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:18:54.945 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:54.945 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:18:54.945 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:18:54.946 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:18:54.946 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:18:54.946 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:54.946 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:54.946 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:18:54.947 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:18:54.947 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:54.947 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:18:54.947 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:54.947 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:54.948 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:18:54.948 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:18:54.948 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:18:54.948 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:18:54.948 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:18:54.949 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:18:54.949 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:18:54.949 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:18:54.949 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:18:54.949 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:18:54.950 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:18:54.950 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:54.950 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:18:54.950 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:18:54.950 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:18:54.951 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:18:54.951 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:18:54.951 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:18:54.951 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:18:54.951 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:18:54.952 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:18:54.952 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:18:54.952 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:18:54.952 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:18:54.964 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:18:54.964 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:18:54.964 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:18:54.965 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:18:54.965 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.965 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:18:54.965 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.966 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:18:54.966 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.966 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:18:54.966 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:18:54.967 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.967 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:18:54.967 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:18:54.967 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.968 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:18:54.968 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:18:54.968 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.968 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:18:54.969 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:18:54.969 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.969 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:18:54.969 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.970 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:18:54.970 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:18:54.970 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:18:54.970 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:18:54.970 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:18:54.971 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:18:54.971 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:18:54.971 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:18:54.971 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:18:54.971 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:18:54.972 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.972 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:18:54.972 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.972 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:18:54.973 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.973 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:18:54.973 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.973 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:18:54.973 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:18:54.974 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:18:54.974 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:18:54.974 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:18:54.974 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:18:54.974 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.975 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:18:54.975 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:18:54.975 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.975 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:18:54.975 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:18:54.976 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.976 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:18:54.976 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:18:54.976 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.976 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:18:54.977 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:18:54.977 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:18:54.977 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.977 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:18:54.977 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:18:54.978 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.978 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:18:54.978 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.978 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:18:54.978 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.979 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:18:54.979 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.979 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:18:54.979 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:18:54.979 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.980 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:18:54.980 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:18:54.980 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:18:54.980 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:18:54.981 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:18:54.981 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:18:54.981 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:18:54.981 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:18:54.982 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:18:54.982 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:18:54.982 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:18:54.982 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:18:54.982 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:18:54.983 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:18:54.983 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:18:54.983 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:18:54.983 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:18:54.984 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:18:54.984 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:18:54.984 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:54.984 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.984 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:18:54.985 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:54.985 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:18:54.985 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:54.985 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:54.985 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:54.986 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:54.986 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:18:54.986 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.986 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:18:54.986 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:54.987 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:18:54.987 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:18:54.987 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:18:54.987 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:18:54.987 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:18:54.988 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:18:54.988 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:18:54.988 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:18:54.988 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:18:54.988 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:18:54.989 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:18:54.989 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:18:54.989 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:18:54.989 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:18:54.989 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:18:54.990 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:18:54.990 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:18:54.990 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:18:54.990 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:18:54.990 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:54.991 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:18:54.991 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:54.991 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:54.991 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:18:54.991 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:18:54.992 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:18:54.992 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:18:54.992 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:18:54.992 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:18:54.993 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:18:54.993 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:18:54.993 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:18:54.993 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:18:54.994 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:18:54.994 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:18:54.994 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:18:54.994 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:18:54.995 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:18:54.995 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:18:54.995 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:18:54.995 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:18:54.995 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:18:54.996 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:18:54.996 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:18:54.996 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:18:54.996 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:18:54.996 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:18:54.997 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-20T12:18:55.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.180+0000 7f2e40b49700 1 Processor -- start 2020-11-20T12:18:55.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.180+0000 7f2e40b49700 1 -- start start 2020-11-20T12:18:55.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e40b49700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2e3c154dd0 0x7f2e3c156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e40b49700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e3c0bd810 0x7f2e3c0be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e40b49700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e3c0bdfc0 con 0x7f2e3c1551b0 2020-11-20T12:18:55.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e40b49700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c0bdd20 con 0x7f2e3c0bd810 2020-11-20T12:18:55.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e40b49700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e3c155780 con 0x7f2e3c154dd0 2020-11-20T12:18:55.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e39d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e3c0bd810 0x7f2e3c0be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e3ad9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2e3c1551b0 0x7f2e3c0a6780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56694/0 (socket says 172.21.15.116:56694) 2020-11-20T12:18:55.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e3ad9d700 1 -- 172.21.15.116:0/277168910 learned_addr learned my addr 172.21.15.116:0/277168910 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:55.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e3a59c700 1 --2- 172.21.15.116:0/277168910 >> v2:172.21.15.173:3300/0 conn(0x7f2e3c154dd0 0x7f2e3c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e39d9b700 1 -- 172.21.15.116:0/277168910 >> v2:172.21.15.173:3300/0 conn(0x7f2e3c154dd0 msgr2=0x7f2e3c156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:55.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e39d9b700 1 --2- 172.21.15.116:0/277168910 >> v2:172.21.15.173:3300/0 conn(0x7f2e3c154dd0 0x7f2e3c156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e39d9b700 1 -- 172.21.15.116:0/277168910 >> v1:172.21.15.116:6789/0 conn(0x7f2e3c1551b0 legacy=0x7f2e3c0a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e39d9b700 1 -- 172.21.15.116:0/277168910 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c155a20 con 0x7f2e3c0bd810 2020-11-20T12:18:55.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.184+0000 7f2e39d9b700 1 --2- 172.21.15.116:0/277168910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c0bd810 0x7f2e3c0be340 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f2e3c066290 tx=0x7f2e24006df0).ready entity=mon.0 client_cookie=8d5bc4c7523b54b6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:55.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.188+0000 7f2e3959a700 1 -- 172.21.15.116:0/277168910 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e240097b0 con 0x7f2e3c0bd810 2020-11-20T12:18:55.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.188+0000 7f2e3959a700 1 -- 172.21.15.116:0/277168910 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e24009910 con 0x7f2e3c0bd810 2020-11-20T12:18:55.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.188+0000 7f2e3959a700 1 -- 172.21.15.116:0/277168910 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e24009c90 con 0x7f2e3c0bd810 2020-11-20T12:18:55.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.188+0000 7f2e40b49700 1 -- 172.21.15.116:0/277168910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c0bd810 msgr2=0x7f2e3c0be340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.188+0000 7f2e40b49700 1 --2- 172.21.15.116:0/277168910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c0bd810 0x7f2e3c0be340 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f2e3c066290 tx=0x7f2e24006df0).stop 2020-11-20T12:18:55.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.188+0000 7f2e40b49700 1 -- 172.21.15.116:0/277168910 shutdown_connections 2020-11-20T12:18:55.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.188+0000 7f2e40b49700 1 --2- 172.21.15.116:0/277168910 >> v2:172.21.15.173:3300/0 conn(0x7f2e3c154dd0 0x7f2e3c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.188+0000 7f2e40b49700 1 --2- 172.21.15.116:0/277168910 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c0bd810 0x7f2e3c0be340 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.188+0000 7f2e40b49700 1 -- 172.21.15.116:0/277168910 >> 172.21.15.116:0/277168910 conn(0x7f2e3c05ba80 msgr2=0x7f2e3c0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:55.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.188+0000 7f2e40b49700 1 -- 172.21.15.116:0/277168910 shutdown_connections 2020-11-20T12:18:55.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.188+0000 7f2e40b49700 1 -- 172.21.15.116:0/277168910 wait complete. 2020-11-20T12:18:55.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e40b49700 1 Processor -- start 2020-11-20T12:18:55.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e40b49700 1 -- start start 2020-11-20T12:18:55.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e40b49700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0bd810 0x7f2e3c151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e40b49700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e3c154dd0 0x7f2e3c066480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e40b49700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c1551b0 0x7f2e3c066970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e40b49700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c1513f0 con 0x7f2e3c1551b0 2020-11-20T12:18:55.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e40b49700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c1625e0 con 0x7f2e3c154dd0 2020-11-20T12:18:55.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e40b49700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3c152280 con 0x7f2e3c0bd810 2020-11-20T12:18:55.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e3ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c1551b0 0x7f2e3c066970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e3ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c1551b0 0x7f2e3c066970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47264/0 (socket says 172.21.15.116:47264) 2020-11-20T12:18:55.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e3ad9d700 1 -- 172.21.15.116:0/1642759625 learned_addr learned my addr 172.21.15.116:0/1642759625 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:55.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e39d9b700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e3c154dd0 0x7f2e3c066480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e3ad9d700 1 -- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0bd810 msgr2=0x7f2e3c151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e3a59c700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0bd810 0x7f2e3c151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e3ad9d700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0bd810 0x7f2e3c151d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e3ad9d700 1 -- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e3c154dd0 msgr2=0x7f2e3c066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e3ad9d700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e3c154dd0 0x7f2e3c066480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e3ad9d700 1 -- 172.21.15.116:0/1642759625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e3c162340 con 0x7f2e3c1551b0 2020-11-20T12:18:55.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e39d9b700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e3c154dd0 0x7f2e3c066480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:55.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.192+0000 7f2e3ad9d700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c1551b0 0x7f2e3c066970 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f2e3400cac0 tx=0x7f2e34007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:55.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.196+0000 7f2e327fc700 1 -- 172.21.15.116:0/1642759625 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34005b80 con 0x7f2e3c1551b0 2020-11-20T12:18:55.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.196+0000 7f2e327fc700 1 -- 172.21.15.116:0/1642759625 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e34005ce0 con 0x7f2e3c1551b0 2020-11-20T12:18:55.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.196+0000 7f2e40b49700 1 -- 172.21.15.116:0/1642759625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e3c151650 con 0x7f2e3c1551b0 2020-11-20T12:18:55.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.196+0000 7f2e327fc700 1 -- 172.21.15.116:0/1642759625 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e34007320 con 0x7f2e3c1551b0 2020-11-20T12:18:55.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.196+0000 7f2e40b49700 1 -- 172.21.15.116:0/1642759625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e3c09f400 con 0x7f2e3c1551b0 2020-11-20T12:18:55.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.196+0000 7f2e40b49700 1 -- 172.21.15.116:0/1642759625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e3c05d2a0 con 0x7f2e3c1551b0 2020-11-20T12:18:55.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.196+0000 7f2e327fc700 1 -- 172.21.15.116:0/1642759625 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2e34010030 con 0x7f2e3c1551b0 2020-11-20T12:18:55.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.196+0000 7f2e327fc700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e180557c0 0x7f2e18057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.196+0000 7f2e327fc700 1 -- 172.21.15.116:0/1642759625 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f2e3406e810 con 0x7f2e3c1551b0 2020-11-20T12:18:55.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.200+0000 7f2e3a59c700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e180557c0 0x7f2e18057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.200+0000 7f2e3a59c700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e180557c0 0x7f2e18057c80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f2e2c006bd0 tx=0x7f2e2c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:55.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.204+0000 7f2e327fc700 1 -- 172.21.15.116:0/1642759625 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2e34044420 con 0x7f2e3c1551b0 2020-11-20T12:18:55.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.360+0000 7f2e40b49700 1 -- 172.21.15.116:0/1642759625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2e3c061380 con 0x7f2e3c1551b0 2020-11-20T12:18:55.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.364+0000 7f2e327fc700 1 -- 172.21.15.116:0/1642759625 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2e34047a40 con 0x7f2e3c1551b0 2020-11-20T12:18:55.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 -- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e180557c0 msgr2=0x7f2e18057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e180557c0 0x7f2e18057c80 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f2e2c006bd0 tx=0x7f2e2c008040).stop 2020-11-20T12:18:55.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 -- 172.21.15.116:0/1642759625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c1551b0 msgr2=0x7f2e3c066970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c1551b0 0x7f2e3c066970 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f2e3400cac0 tx=0x7f2e34007f50).stop 2020-11-20T12:18:55.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 -- 172.21.15.116:0/1642759625 shutdown_connections 2020-11-20T12:18:55.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e180557c0 0x7f2e18057c80 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e3c1551b0 0x7f2e3c066970 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e3c0bd810 0x7f2e3c151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 --2- 172.21.15.116:0/1642759625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e3c154dd0 0x7f2e3c066480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 -- 172.21.15.116:0/1642759625 >> 172.21.15.116:0/1642759625 conn(0x7f2e3c05ba80 msgr2=0x7f2e3c14e1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:55.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 -- 172.21.15.116:0/1642759625 shutdown_connections 2020-11-20T12:18:55.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.368+0000 7f2e40b49700 1 -- 172.21.15.116:0/1642759625 wait complete. 2020-11-20T12:18:55.391 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_data_isolated' 2020-11-20T12:18:55.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.556+0000 7f013700a700 1 Processor -- start 2020-11-20T12:18:55.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.556+0000 7f013700a700 1 -- start start 2020-11-20T12:18:55.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.560+0000 7f013700a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f013013afa0 0x7f01300aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.560+0000 7f013700a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f013013b380 0x7f01300aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.560+0000 7f013700a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01300ada00 con 0x7f01300acd30 2020-11-20T12:18:55.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.560+0000 7f013700a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01300adca0 con 0x7f013013afa0 2020-11-20T12:18:55.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.560+0000 7f013700a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01300adf40 con 0x7f013013b380 2020-11-20T12:18:55.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.560+0000 7f0136008700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f013013afa0 0x7f01300aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.560+0000 7f0136008700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f013013afa0 0x7f01300aa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47270/0 (socket says 172.21.15.116:47270) 2020-11-20T12:18:55.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.560+0000 7f0136008700 1 -- 172.21.15.116:0/2767695091 learned_addr learned my addr 172.21.15.116:0/2767695091 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:55.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f0135807700 1 --2- 172.21.15.116:0/2767695091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f013013b380 0x7f01300aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f0136008700 1 -- 172.21.15.116:0/2767695091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f013013b380 msgr2=0x7f01300aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:55.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f0136008700 1 --2- 172.21.15.116:0/2767695091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f013013b380 0x7f01300aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f0136008700 1 -- 172.21.15.116:0/2767695091 >> v1:172.21.15.116:6789/0 conn(0x7f01300acd30 legacy=0x7f01300ad1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f0136008700 1 -- 172.21.15.116:0/2767695091 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300ae1e0 con 0x7f013013afa0 2020-11-20T12:18:55.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f0135807700 1 --2- 172.21.15.116:0/2767695091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f013013b380 0x7f01300aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:55.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f0136008700 1 --2- 172.21.15.116:0/2767695091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013013afa0 0x7f01300aa5b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f012c006fd0 tx=0x7f012c006eb0).ready entity=mon.0 client_cookie=75298ecc22176b64 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:55.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f0135006700 1 -- 172.21.15.116:0/2767695091 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c0097d0 con 0x7f013013afa0 2020-11-20T12:18:55.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f0135006700 1 -- 172.21.15.116:0/2767695091 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c009930 con 0x7f013013afa0 2020-11-20T12:18:55.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f0135006700 1 -- 172.21.15.116:0/2767695091 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c009cb0 con 0x7f013013afa0 2020-11-20T12:18:55.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f013700a700 1 -- 172.21.15.116:0/2767695091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013013afa0 msgr2=0x7f01300aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f013700a700 1 --2- 172.21.15.116:0/2767695091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013013afa0 0x7f01300aa5b0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f012c006fd0 tx=0x7f012c006eb0).stop 2020-11-20T12:18:55.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f013700a700 1 -- 172.21.15.116:0/2767695091 shutdown_connections 2020-11-20T12:18:55.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f013700a700 1 --2- 172.21.15.116:0/2767695091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f013013afa0 0x7f01300aa5b0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f013700a700 1 --2- 172.21.15.116:0/2767695091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f013013b380 0x7f01300aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f013700a700 1 -- 172.21.15.116:0/2767695091 >> 172.21.15.116:0/2767695091 conn(0x7f01300678a0 msgr2=0x7f0130067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:55.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f013700a700 1 -- 172.21.15.116:0/2767695091 shutdown_connections 2020-11-20T12:18:55.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.564+0000 7f013700a700 1 -- 172.21.15.116:0/2767695091 wait complete. 2020-11-20T12:18:55.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.568+0000 7f013700a700 1 Processor -- start 2020-11-20T12:18:55.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.568+0000 7f013700a700 1 -- start start 2020-11-20T12:18:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f013700a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01300acd30 0x7f0130161850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f013700a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f013013afa0 0x7f0130151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f013700a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f013013b380 0x7f0130164560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f013700a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0130160f30 con 0x7f01300acd30 2020-11-20T12:18:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f013700a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0130164fc0 con 0x7f013013b380 2020-11-20T12:18:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f013700a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0130165260 con 0x7f013013afa0 2020-11-20T12:18:55.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f0136008700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01300acd30 0x7f0130161850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f0136008700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01300acd30 0x7f0130161850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47276/0 (socket says 172.21.15.116:47276) 2020-11-20T12:18:55.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f0136008700 1 -- 172.21.15.116:0/1655393536 learned_addr learned my addr 172.21.15.116:0/1655393536 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:55.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f0135807700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f013013afa0 0x7f0130151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f0136809700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f013013b380 0x7f0130164560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f0136008700 1 -- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f013013afa0 msgr2=0x7f0130151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f0136008700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f013013afa0 0x7f0130151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f0136008700 1 -- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f013013b380 msgr2=0x7f0130164560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:55.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f0136008700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f013013b380 0x7f0130164560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:55.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f0136008700 1 -- 172.21.15.116:0/1655393536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0130161110 con 0x7f01300acd30 2020-11-20T12:18:55.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f0136008700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01300acd30 0x7f0130161850 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f012c00c010 tx=0x7f012c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:55.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f011e7fc700 1 -- 172.21.15.116:0/1655393536 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c011c80 con 0x7f01300acd30 2020-11-20T12:18:55.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f011e7fc700 1 -- 172.21.15.116:0/1655393536 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012c011de0 con 0x7f01300acd30 2020-11-20T12:18:55.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f013700a700 1 -- 172.21.15.116:0/1655393536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01301613b0 con 0x7f01300acd30 2020-11-20T12:18:55.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f011e7fc700 1 -- 172.21.15.116:0/1655393536 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c01e690 con 0x7f01300acd30 2020-11-20T12:18:55.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.572+0000 7f013700a700 1 -- 172.21.15.116:0/1655393536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0130161d40 con 0x7f01300acd30 2020-11-20T12:18:55.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.576+0000 7f013700a700 1 -- 172.21.15.116:0/1655393536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f013013ac30 con 0x7f01300acd30 2020-11-20T12:18:55.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.576+0000 7f011e7fc700 1 -- 172.21.15.116:0/1655393536 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f012c025030 con 0x7f01300acd30 2020-11-20T12:18:55.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.576+0000 7f011e7fc700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01080557c0 0x7f0108057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.576+0000 7f011e7fc700 1 -- 172.21.15.116:0/1655393536 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f012c06ff80 con 0x7f01300acd30 2020-11-20T12:18:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.580+0000 7f0135807700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01080557c0 0x7f0108057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:55.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.580+0000 7f0135807700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01080557c0 0x7f0108057c80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f0130136060 tx=0x7f0120008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:55.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.584+0000 7f011e7fc700 1 -- 172.21.15.116:0/1655393536 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f013013ac30 con 0x7f01300acd30 2020-11-20T12:18:55.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:55.804+0000 7f013700a700 1 -- 172.21.15.116:0/1655393536 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_data_isolated"]} v 0) v1 -- 0x7f0130165420 con 0x7f01300acd30 2020-11-20T12:18:56.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.192+0000 7f011e7fc700 1 -- 172.21.15.116:0/1655393536 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_data_isolated"]}]=0 v177) v1 ==== 146+0+0 (secure 0 0 0) 0x7f0130165420 con 0x7f01300acd30 2020-11-20T12:18:56.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 -- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01080557c0 msgr2=0x7f0108057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01080557c0 0x7f0108057c80 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7f0130136060 tx=0x7f0120008040).stop 2020-11-20T12:18:56.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 -- 172.21.15.116:0/1655393536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01300acd30 msgr2=0x7f0130161850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01300acd30 0x7f0130161850 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f012c00c010 tx=0x7f012c015000).stop 2020-11-20T12:18:56.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 -- 172.21.15.116:0/1655393536 shutdown_connections 2020-11-20T12:18:56.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01080557c0 0x7f0108057c80 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f013013b380 0x7f0130164560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01300acd30 0x7f0130161850 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 --2- 172.21.15.116:0/1655393536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f013013afa0 0x7f0130151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 -- 172.21.15.116:0/1655393536 >> 172.21.15.116:0/1655393536 conn(0x7f01300678a0 msgr2=0x7f0130067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:56.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 -- 172.21.15.116:0/1655393536 shutdown_connections 2020-11-20T12:18:56.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.196+0000 7f013700a700 1 -- 172.21.15.116:0/1655393536 wait complete. 2020-11-20T12:18:56.218 INFO:tasks.cephfs_test_runner:test_data_isolated (tasks.cephfs.test_volume_client.TestVolumeClient) ... ok 2020-11-20T12:18:56.219 INFO:tasks.cephfs_test_runner:Starting test: test_default_prefix (tasks.cephfs.test_volume_client.TestVolumeClient) 2020-11-20T12:18:56.219 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_default_prefix' 2020-11-20T12:18:56.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.396+0000 7fb17adae700 1 Processor -- start 2020-11-20T12:18:56.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.400+0000 7fb17adae700 1 -- start start 2020-11-20T12:18:56.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.400+0000 7fb17adae700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 0x7fb1740a7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.400+0000 7fb17adae700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb1741555d0 0x7fb1740c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.400+0000 7fb17adae700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1741564c0 con 0x7fb1740c38c0 2020-11-20T12:18:56.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.400+0000 7fb17adae700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1740a8210 con 0x7fb1741555d0 2020-11-20T12:18:56.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.400+0000 7fb17adae700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb174155ce0 con 0x7fb1740c3ca0 2020-11-20T12:18:56.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb173fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 0x7fb1740a7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb173fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 0x7fb1740a7d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:56236/0 (socket says 172.21.15.116:56236) 2020-11-20T12:18:56.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb173fff700 1 -- 172.21.15.116:0/1097840287 learned_addr learned my addr 172.21.15.116:0/1097840287 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:56.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb17934b700 1 --2- 172.21.15.116:0/1097840287 >> v2:172.21.15.173:3300/0 conn(0x7fb1741555d0 0x7fb1740c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb173fff700 1 -- 172.21.15.116:0/1097840287 >> v2:172.21.15.173:3300/0 conn(0x7fb1741555d0 msgr2=0x7fb1740c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:56.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb173fff700 1 --2- 172.21.15.116:0/1097840287 >> v2:172.21.15.173:3300/0 conn(0x7fb1741555d0 0x7fb1740c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb173fff700 1 -- 172.21.15.116:0/1097840287 >> v1:172.21.15.173:6789/0 conn(0x7fb1740c38c0 legacy=0x7fb1740a8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb173fff700 1 -- 172.21.15.116:0/1097840287 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174155f80 con 0x7fb1740c3ca0 2020-11-20T12:18:56.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb173fff700 1 --2- 172.21.15.116:0/1097840287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 0x7fb1740a7d20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fb160006fd0 tx=0x7fb160006eb0).ready entity=mon.2 client_cookie=eccdd505c7852920 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb1737fe700 1 -- 172.21.15.116:0/1097840287 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160009810 con 0x7fb1740c3ca0 2020-11-20T12:18:56.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb1737fe700 1 -- 172.21.15.116:0/1097840287 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb160009970 con 0x7fb1740c3ca0 2020-11-20T12:18:56.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb1737fe700 1 -- 172.21.15.116:0/1097840287 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb160009cf0 con 0x7fb1740c3ca0 2020-11-20T12:18:56.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb17adae700 1 -- 172.21.15.116:0/1097840287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 msgr2=0x7fb1740a7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.404+0000 7fb17adae700 1 --2- 172.21.15.116:0/1097840287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 0x7fb1740a7d20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fb160006fd0 tx=0x7fb160006eb0).stop 2020-11-20T12:18:56.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.408+0000 7fb17adae700 1 -- 172.21.15.116:0/1097840287 shutdown_connections 2020-11-20T12:18:56.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.408+0000 7fb17adae700 1 --2- 172.21.15.116:0/1097840287 >> v2:172.21.15.173:3300/0 conn(0x7fb1741555d0 0x7fb1740c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.408+0000 7fb17adae700 1 --2- 172.21.15.116:0/1097840287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 0x7fb1740a7d20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.408+0000 7fb17adae700 1 -- 172.21.15.116:0/1097840287 >> 172.21.15.116:0/1097840287 conn(0x7fb1740678a0 msgr2=0x7fb174067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:56.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.408+0000 7fb17adae700 1 -- 172.21.15.116:0/1097840287 shutdown_connections 2020-11-20T12:18:56.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.408+0000 7fb17adae700 1 -- 172.21.15.116:0/1097840287 wait complete. 2020-11-20T12:18:56.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.408+0000 7fb17adae700 1 Processor -- start 2020-11-20T12:18:56.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.408+0000 7fb17adae700 1 -- start start 2020-11-20T12:18:56.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb17adae700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1740c38c0 0x7fb174161df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb17adae700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 0x7fb174162720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb17adae700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1741555d0 0x7fb1740a7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb17adae700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb174139ab0 con 0x7fb1740c38c0 2020-11-20T12:18:56.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb17adae700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb174156860 con 0x7fb1741555d0 2020-11-20T12:18:56.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb17adae700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb174156b00 con 0x7fb1740c3ca0 2020-11-20T12:18:56.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb178b4a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1740c38c0 0x7fb174161df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb173fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 0x7fb174162720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb178b4a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1740c38c0 0x7fb174161df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47290/0 (socket says 172.21.15.116:47290) 2020-11-20T12:18:56.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb178b4a700 1 -- 172.21.15.116:0/571471401 learned_addr learned my addr 172.21.15.116:0/571471401 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:56.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb17934b700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1741555d0 0x7fb1740a7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb178b4a700 1 -- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 msgr2=0x7fb174162720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb178b4a700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 0x7fb174162720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb178b4a700 1 -- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1741555d0 msgr2=0x7fb1740a7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:56.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb178b4a700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1741555d0 0x7fb1740a7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:56.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb178b4a700 1 -- 172.21.15.116:0/571471401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb174156da0 con 0x7fb1740c38c0 2020-11-20T12:18:56.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb17934b700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1741555d0 0x7fb1740a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:56.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb178b4a700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1740c38c0 0x7fb174161df0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fb16800c710 tx=0x7fb168007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb170ff9700 1 -- 172.21.15.116:0/571471401 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168001ea0 con 0x7fb1740c38c0 2020-11-20T12:18:56.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb17adae700 1 -- 172.21.15.116:0/571471401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb174157040 con 0x7fb1740c38c0 2020-11-20T12:18:56.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb170ff9700 1 -- 172.21.15.116:0/571471401 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16800e560 con 0x7fb1740c38c0 2020-11-20T12:18:56.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb170ff9700 1 -- 172.21.15.116:0/571471401 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16800e900 con 0x7fb1740c38c0 2020-11-20T12:18:56.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.412+0000 7fb17adae700 1 -- 172.21.15.116:0/571471401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb174162c10 con 0x7fb1740c38c0 2020-11-20T12:18:56.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.416+0000 7fb17adae700 1 -- 172.21.15.116:0/571471401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1741571d0 con 0x7fb1740c38c0 2020-11-20T12:18:56.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.416+0000 7fb170ff9700 1 -- 172.21.15.116:0/571471401 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb168010030 con 0x7fb1740c38c0 2020-11-20T12:18:56.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.420+0000 7fb170ff9700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb14c0557c0 0x7fb14c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:56.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.420+0000 7fb170ff9700 1 -- 172.21.15.116:0/571471401 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fb16806eba0 con 0x7fb1740c38c0 2020-11-20T12:18:56.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.420+0000 7fb173fff700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb14c0557c0 0x7fb14c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:56.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.420+0000 7fb173fff700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb14c0557c0 0x7fb14c057c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fb16000b1b0 tx=0x7fb160015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:56.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.424+0000 7fb170ff9700 1 -- 172.21.15.116:0/571471401 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb1680447f0 con 0x7fb1740c38c0 2020-11-20T12:18:56.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:56.640+0000 7fb17adae700 1 -- 172.21.15.116:0/571471401 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_default_prefix"]} v 0) v1 -- 0x7fb174069950 con 0x7fb1740c38c0 2020-11-20T12:18:57.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.208+0000 7fb170ff9700 1 -- 172.21.15.116:0/571471401 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_default_prefix"]}]=0 v178) v1 ==== 150+0+0 (secure 0 0 0) 0x7fb16806e020 con 0x7fb1740c38c0 2020-11-20T12:18:57.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.208+0000 7fb17adae700 1 -- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb14c0557c0 msgr2=0x7fb14c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.208+0000 7fb17adae700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb14c0557c0 0x7fb14c057c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fb16000b1b0 tx=0x7fb160015000).stop 2020-11-20T12:18:57.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.208+0000 7fb17adae700 1 -- 172.21.15.116:0/571471401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1740c38c0 msgr2=0x7fb174161df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.208+0000 7fb17adae700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1740c38c0 0x7fb174161df0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7fb16800c710 tx=0x7fb168007c00).stop 2020-11-20T12:18:57.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.208+0000 7fb17adae700 1 -- 172.21.15.116:0/571471401 shutdown_connections 2020-11-20T12:18:57.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.208+0000 7fb17adae700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb14c0557c0 0x7fb14c057c80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.208+0000 7fb17adae700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1741555d0 0x7fb1740a7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.208+0000 7fb17adae700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1740c38c0 0x7fb174161df0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.208+0000 7fb17adae700 1 --2- 172.21.15.116:0/571471401 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1740c3ca0 0x7fb174162720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.208+0000 7fb17adae700 1 -- 172.21.15.116:0/571471401 >> 172.21.15.116:0/571471401 conn(0x7fb1740678a0 msgr2=0x7fb174152540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:57.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.212+0000 7fb17adae700 1 -- 172.21.15.116:0/571471401 shutdown_connections 2020-11-20T12:18:57.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.212+0000 7fb17adae700 1 -- 172.21.15.116:0/571471401 wait complete. 2020-11-20T12:18:57.236 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-20T12:18:57.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.416+0000 7fd11499d700 1 Processor -- start 2020-11-20T12:18:57.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.416+0000 7fd11499d700 1 -- start start 2020-11-20T12:18:57.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.416+0000 7fd11499d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd1100c37d0 0x7fd1100c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:57.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd11499d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd110151cf0 con 0x7fd1100aa1a0 2020-11-20T12:18:57.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd11499d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1100c34f0 con 0x7fd1100aa580 2020-11-20T12:18:57.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd11499d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1100c32d0 con 0x7fd1100c37d0 2020-11-20T12:18:57.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd10ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd1100c37d0 0x7fd1100c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:57.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd10ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd1100c37d0 0x7fd1100c2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47302/0 (socket says 172.21.15.116:47302) 2020-11-20T12:18:57.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd10ffff700 1 -- 172.21.15.116:0/2332705458 learned_addr learned my addr 172.21.15.116:0/2332705458 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:57.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd10ffff700 1 -- 172.21.15.116:0/2332705458 >> v1:172.21.15.173:6789/0 conn(0x7fd1100aa580 legacy=0x7fd1100a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:57.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd10ffff700 1 -- 172.21.15.116:0/2332705458 >> v1:172.21.15.116:6789/0 conn(0x7fd1100aa1a0 legacy=0x7fd110061460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:57.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd10ffff700 1 -- 172.21.15.116:0/2332705458 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1100c7e00 con 0x7fd1100c37d0 2020-11-20T12:18:57.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd10ffff700 1 --2- 172.21.15.116:0/2332705458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1100c37d0 0x7fd1100c2de0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fd108009d50 tx=0x7fd108009290).ready entity=mon.0 client_cookie=ec1e15ffa5f7deb0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:57.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd10e7fc700 1 -- 172.21.15.116:0/2332705458 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10800bc00 con 0x7fd1100c37d0 2020-11-20T12:18:57.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd10e7fc700 1 -- 172.21.15.116:0/2332705458 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10800bd60 con 0x7fd1100c37d0 2020-11-20T12:18:57.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd10e7fc700 1 -- 172.21.15.116:0/2332705458 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108011610 con 0x7fd1100c37d0 2020-11-20T12:18:57.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd11499d700 1 -- 172.21.15.116:0/2332705458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1100c37d0 msgr2=0x7fd1100c2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd11499d700 1 --2- 172.21.15.116:0/2332705458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1100c37d0 0x7fd1100c2de0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fd108009d50 tx=0x7fd108009290).stop 2020-11-20T12:18:57.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd11499d700 1 -- 172.21.15.116:0/2332705458 shutdown_connections 2020-11-20T12:18:57.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd11499d700 1 --2- 172.21.15.116:0/2332705458 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1100c37d0 0x7fd1100c2de0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd11499d700 1 -- 172.21.15.116:0/2332705458 >> 172.21.15.116:0/2332705458 conn(0x7fd110068370 msgr2=0x7fd1100c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:57.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd11499d700 1 -- 172.21.15.116:0/2332705458 shutdown_connections 2020-11-20T12:18:57.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.420+0000 7fd11499d700 1 -- 172.21.15.116:0/2332705458 wait complete. 2020-11-20T12:18:57.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.424+0000 7fd11499d700 1 Processor -- start 2020-11-20T12:18:57.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.424+0000 7fd11499d700 1 -- start start 2020-11-20T12:18:57.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd11499d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1100aa1a0 0x7fd1101693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:57.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd11499d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1100aa580 0x7fd110157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:57.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd11499d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1100c37d0 0x7fd1100ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:57.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd11499d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd110165880 con 0x7fd1100aa1a0 2020-11-20T12:18:57.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd11499d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd110165b20 con 0x7fd1100aa580 2020-11-20T12:18:57.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd11499d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd110165dc0 con 0x7fd1100c37d0 2020-11-20T12:18:57.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1100aa1a0 0x7fd1101693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:57.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1100aa1a0 0x7fd1101693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47304/0 (socket says 172.21.15.116:47304) 2020-11-20T12:18:57.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10f7fe700 1 -- 172.21.15.116:0/4056522083 learned_addr learned my addr 172.21.15.116:0/4056522083 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:57.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10ffff700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1100c37d0 0x7fd1100ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:57.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10effd700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1100aa580 0x7fd110157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:57.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10f7fe700 1 -- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1100c37d0 msgr2=0x7fd1100ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10f7fe700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1100c37d0 0x7fd1100ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10f7fe700 1 -- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1100aa580 msgr2=0x7fd110157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10f7fe700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1100aa580 0x7fd110157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10f7fe700 1 -- 172.21.15.116:0/4056522083 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd110166060 con 0x7fd1100aa1a0 2020-11-20T12:18:57.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10ffff700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1100c37d0 0x7fd1100ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:18:57.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd10f7fe700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1100aa1a0 0x7fd1101693c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fd10000c730 tx=0x7fd100007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:57.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd0ef7fe700 1 -- 172.21.15.116:0/4056522083 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10000e560 con 0x7fd1100aa1a0 2020-11-20T12:18:57.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd0ef7fe700 1 -- 172.21.15.116:0/4056522083 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd10000e6e0 con 0x7fd1100aa1a0 2020-11-20T12:18:57.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd0ef7fe700 1 -- 172.21.15.116:0/4056522083 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10000eaa0 con 0x7fd1100aa1a0 2020-11-20T12:18:57.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd11499d700 1 -- 172.21.15.116:0/4056522083 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1101698b0 con 0x7fd1100aa1a0 2020-11-20T12:18:57.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.428+0000 7fd11499d700 1 -- 172.21.15.116:0/4056522083 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd110166300 con 0x7fd1100aa1a0 2020-11-20T12:18:57.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.432+0000 7fd11499d700 1 -- 172.21.15.116:0/4056522083 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11005cf30 con 0x7fd1100aa1a0 2020-11-20T12:18:57.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.432+0000 7fd0ef7fe700 1 -- 172.21.15.116:0/4056522083 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd100019030 con 0x7fd1100aa1a0 2020-11-20T12:18:57.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.432+0000 7fd0ef7fe700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0e80557c0 0x7fd0e8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:57.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.432+0000 7fd0ef7fe700 1 -- 172.21.15.116:0/4056522083 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fd10006dc50 con 0x7fd1100aa1a0 2020-11-20T12:18:57.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.436+0000 7fd10effd700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0e80557c0 0x7fd0e8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:57.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.436+0000 7fd10effd700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0e80557c0 0x7fd0e8057c80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fd1100ae960 tx=0x7fd104007b70).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:57.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.440+0000 7fd0ef7fe700 1 -- 172.21.15.116:0/4056522083 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd100043a20 con 0x7fd1100aa1a0 2020-11-20T12:18:57.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.596+0000 7fd11499d700 1 -- 172.21.15.116:0/4056522083 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7fd110060f40 con 0x7fd1100aa1a0 2020-11-20T12:18:57.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.604+0000 7fd0ef7fe700 1 -- 172.21.15.116:0/4056522083 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd100047040 con 0x7fd1100aa1a0 2020-11-20T12:18:57.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 -- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0e80557c0 msgr2=0x7fd0e8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0e80557c0 0x7fd0e8057c80 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fd1100ae960 tx=0x7fd104007b70).stop 2020-11-20T12:18:57.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 -- 172.21.15.116:0/4056522083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1100aa1a0 msgr2=0x7fd1101693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1100aa1a0 0x7fd1101693c0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7fd10000c730 tx=0x7fd100007c30).stop 2020-11-20T12:18:57.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 -- 172.21.15.116:0/4056522083 shutdown_connections 2020-11-20T12:18:57.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0e80557c0 0x7fd0e8057c80 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd1100c37d0 0x7fd1100ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd1100aa1a0 0x7fd1101693c0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 --2- 172.21.15.116:0/4056522083 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd1100aa580 0x7fd110157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 -- 172.21.15.116:0/4056522083 >> 172.21.15.116:0/4056522083 conn(0x7fd110068370 msgr2=0x7fd1100a96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:57.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 -- 172.21.15.116:0/4056522083 shutdown_connections 2020-11-20T12:18:57.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.608+0000 7fd11499d700 1 -- 172.21.15.116:0/4056522083 wait complete. 2020-11-20T12:18:57.633 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:18:57.634 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:18:57.670 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:18:57.708 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:18:57.708 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2020-11-20T12:18:57.709 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:18:57.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.888+0000 7fbe8359e700 1 Processor -- start 2020-11-20T12:18:57.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.888+0000 7fbe8359e700 1 -- start start 2020-11-20T12:18:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.888+0000 7fbe8359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe84148040 0x7fbe84148420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:57.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.888+0000 7fbe8359e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fbe84066480 0x7fbe84066930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:57.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.888+0000 7fbe8359e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe84067190 con 0x7fbe84068420 2020-11-20T12:18:57.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.888+0000 7fbe8359e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe840677a0 con 0x7fbe84066480 2020-11-20T12:18:57.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.888+0000 7fbe8359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe84067a40 con 0x7fbe84148040 2020-11-20T12:18:57.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe79d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbe84068420 0x7fbe840626e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56746/0 (socket says 172.21.15.116:56746) 2020-11-20T12:18:57.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe79d9b700 1 -- 172.21.15.116:0/1229111992 learned_addr learned my addr 172.21.15.116:0/1229111992 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:18:57.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8259c700 1 --2- 172.21.15.116:0/1229111992 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe84148040 0x7fbe84148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:57.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe82d9d700 1 --2- 172.21.15.116:0/1229111992 >> v2:172.21.15.173:3300/0 conn(0x7fbe84066480 0x7fbe84066930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:57.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe81d9b700 1 -- 172.21.15.116:0/1229111992 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462791283 0 0) 0x7fbe84067190 con 0x7fbe84068420 2020-11-20T12:18:57.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe81d9b700 1 -- 172.21.15.116:0/1229111992 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe64003240 con 0x7fbe84068420 2020-11-20T12:18:57.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8259c700 1 -- 172.21.15.116:0/1229111992 >> v2:172.21.15.173:3300/0 conn(0x7fbe84066480 msgr2=0x7fbe84066930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:57.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8259c700 1 --2- 172.21.15.116:0/1229111992 >> v2:172.21.15.173:3300/0 conn(0x7fbe84066480 0x7fbe84066930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8259c700 1 -- 172.21.15.116:0/1229111992 >> v1:172.21.15.116:6789/0 conn(0x7fbe84068420 legacy=0x7fbe840626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8259c700 1 -- 172.21.15.116:0/1229111992 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe84067ce0 con 0x7fbe84148040 2020-11-20T12:18:57.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8259c700 1 --2- 172.21.15.116:0/1229111992 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe84148040 0x7fbe84148420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fbe74009c90 tx=0x7fbe740091d0).ready entity=mon.2 client_cookie=11d3125e2969bd1b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:57.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe81d9b700 1 -- 172.21.15.116:0/1229111992 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7400b9c0 con 0x7fbe84148040 2020-11-20T12:18:57.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe81d9b700 1 -- 172.21.15.116:0/1229111992 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7400bb20 con 0x7fbe84148040 2020-11-20T12:18:57.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe81d9b700 1 -- 172.21.15.116:0/1229111992 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7400bea0 con 0x7fbe84148040 2020-11-20T12:18:57.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8359e700 1 -- 172.21.15.116:0/1229111992 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe84148040 msgr2=0x7fbe84148420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8359e700 1 --2- 172.21.15.116:0/1229111992 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe84148040 0x7fbe84148420 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fbe74009c90 tx=0x7fbe740091d0).stop 2020-11-20T12:18:57.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8359e700 1 -- 172.21.15.116:0/1229111992 shutdown_connections 2020-11-20T12:18:57.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8359e700 1 --2- 172.21.15.116:0/1229111992 >> v2:172.21.15.173:3300/0 conn(0x7fbe84066480 0x7fbe84066930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8359e700 1 --2- 172.21.15.116:0/1229111992 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe84148040 0x7fbe84148420 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.892+0000 7fbe8359e700 1 -- 172.21.15.116:0/1229111992 >> 172.21.15.116:0/1229111992 conn(0x7fbe84060c40 msgr2=0x7fbe840c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:57.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.896+0000 7fbe8359e700 1 -- 172.21.15.116:0/1229111992 shutdown_connections 2020-11-20T12:18:57.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.896+0000 7fbe8359e700 1 -- 172.21.15.116:0/1229111992 wait complete. 2020-11-20T12:18:57.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.896+0000 7fbe8359e700 1 Processor -- start 2020-11-20T12:18:57.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.896+0000 7fbe8359e700 1 -- start start 2020-11-20T12:18:57.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe84066480 0x7fbe84149ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:57.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe84068420 0x7fbe840a2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe84148040 0x7fbe840a5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe840a8930 con 0x7fbe84066480 2020-11-20T12:18:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe84161a30 con 0x7fbe84068420 2020-11-20T12:18:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe84161cd0 con 0x7fbe84148040 2020-11-20T12:18:57.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe84066480 0x7fbe84149ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:57.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe84066480 0x7fbe84149ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47318/0 (socket says 172.21.15.116:47318) 2020-11-20T12:18:57.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8259c700 1 -- 172.21.15.116:0/2042069035 learned_addr learned my addr 172.21.15.116:0/2042069035 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:57.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe79d9b700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe84068420 0x7fbe840a2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:57.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe82d9d700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe84148040 0x7fbe840a5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:57.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8259c700 1 -- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe84148040 msgr2=0x7fbe840a5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8259c700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe84148040 0x7fbe840a5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8259c700 1 -- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe84068420 msgr2=0x7fbe840a2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:57.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8259c700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe84068420 0x7fbe840a2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:57.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8259c700 1 -- 172.21.15.116:0/2042069035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe841494c0 con 0x7fbe84066480 2020-11-20T12:18:57.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8259c700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe84066480 0x7fbe84149ea0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fbe74001150 tx=0x7fbe74016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:57.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe7affd700 1 -- 172.21.15.116:0/2042069035 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7401d480 con 0x7fbe84066480 2020-11-20T12:18:57.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe7affd700 1 -- 172.21.15.116:0/2042069035 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe7401d5e0 con 0x7fbe84066480 2020-11-20T12:18:57.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8359e700 1 -- 172.21.15.116:0/2042069035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe84149760 con 0x7fbe84066480 2020-11-20T12:18:57.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe7affd700 1 -- 172.21.15.116:0/2042069035 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7401d960 con 0x7fbe84066480 2020-11-20T12:18:57.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.900+0000 7fbe8359e700 1 -- 172.21.15.116:0/2042069035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe8414a390 con 0x7fbe84066480 2020-11-20T12:18:57.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.904+0000 7fbe8359e700 1 -- 172.21.15.116:0/2042069035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe8405cf30 con 0x7fbe84066480 2020-11-20T12:18:57.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.904+0000 7fbe7affd700 1 -- 172.21.15.116:0/2042069035 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fbe74017070 con 0x7fbe84066480 2020-11-20T12:18:57.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.904+0000 7fbe7affd700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe600557f0 0x7fbe60057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:57.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.904+0000 7fbe7affd700 1 -- 172.21.15.116:0/2042069035 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fbe74070bb0 con 0x7fbe84066480 2020-11-20T12:18:57.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.904+0000 7fbe79d9b700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe600557f0 0x7fbe60057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:57.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.908+0000 7fbe79d9b700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe600557f0 0x7fbe60057cb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fbe8414bd10 tx=0x7fbe6c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:57.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:57.912+0000 7fbe7affd700 1 -- 172.21.15.116:0/2042069035 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbe74077080 con 0x7fbe84066480 2020-11-20T12:18:58.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.120+0000 7fbe8359e700 1 -- 172.21.15.116:0/2042069035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe84061230 con 0x7fbe84066480 2020-11-20T12:18:59.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.120+0000 7fbe7affd700 1 -- 172.21.15.116:0/2042069035 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2853 (secure 0 0 0) 0x7fbe74015030 con 0x7fbe84066480 2020-11-20T12:18:59.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 -- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe600557f0 msgr2=0x7fbe60057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe600557f0 0x7fbe60057cb0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fbe8414bd10 tx=0x7fbe6c009040).stop 2020-11-20T12:18:59.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 -- 172.21.15.116:0/2042069035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe84066480 msgr2=0x7fbe84149ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe84066480 0x7fbe84149ea0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fbe74001150 tx=0x7fbe74016000).stop 2020-11-20T12:18:59.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 -- 172.21.15.116:0/2042069035 shutdown_connections 2020-11-20T12:18:59.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbe600557f0 0x7fbe60057cb0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbe84148040 0x7fbe840a5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbe84066480 0x7fbe84149ea0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 --2- 172.21.15.116:0/2042069035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbe84068420 0x7fbe840a2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 -- 172.21.15.116:0/2042069035 >> 172.21.15.116:0/2042069035 conn(0x7fbe84060c40 msgr2=0x7fbe840bf150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:59.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 -- 172.21.15.116:0/2042069035 shutdown_connections 2020-11-20T12:18:59.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:58.124+0000 7fbe8359e700 1 -- 172.21.15.116:0/2042069035 wait complete. 2020-11-20T12:18:59.363 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 29 2020-11-20T12:18:59.364 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:59.364 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/785878283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":785878283},{"type":"v1","addr":"172.21.15.173:6837","nonce":785878283}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5952,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1568769426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1568769426},{"type":"v1","addr":"172.21.15.173:6836","nonce":1568769426}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3552490451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3552490451},{"type":"v1","addr":"172.21.15.116:6833","nonce":3552490451}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:18:27.137821+0000","modified":"2020-11-20T12:18:48.530378+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3370673707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3370673707},{"type":"v1","addr":"172.21.15.116:6835","nonce":3370673707}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-20T12:18:59.365 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:18:59.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.552+0000 7f84be517700 1 Processor -- start 2020-11-20T12:18:59.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.552+0000 7f84be517700 1 -- start start 2020-11-20T12:18:59.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84be517700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b80b92d0 0x7f84b8062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84be517700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f84b814de90 0x7f84b8147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84be517700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84b814e480 con 0x7f84b8062880 2020-11-20T12:18:59.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84be517700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84b814e720 con 0x7f84b814de90 2020-11-20T12:18:59.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84be517700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b814e9c0 con 0x7f84b80b92d0 2020-11-20T12:18:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84bcd14700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b80b92d0 0x7f84b8062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84bcd14700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b80b92d0 0x7f84b8062c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:56276/0 (socket says 172.21.15.116:56276) 2020-11-20T12:18:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84bcd14700 1 -- 172.21.15.116:0/3844634054 learned_addr learned my addr 172.21.15.116:0/3844634054 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84bdd16700 1 --2- 172.21.15.116:0/3844634054 >> v2:172.21.15.173:3300/0 conn(0x7f84b814de90 0x7f84b8147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84bcd14700 1 -- 172.21.15.116:0/3844634054 >> v2:172.21.15.173:3300/0 conn(0x7f84b814de90 msgr2=0x7f84b8147ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84bcd14700 1 --2- 172.21.15.116:0/3844634054 >> v2:172.21.15.173:3300/0 conn(0x7f84b814de90 0x7f84b8147ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84bcd14700 1 -- 172.21.15.116:0/3844634054 >> v1:172.21.15.173:6789/0 conn(0x7f84b8062880 legacy=0x7f84b8151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84bcd14700 1 -- 172.21.15.116:0/3844634054 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b814ec60 con 0x7f84b80b92d0 2020-11-20T12:18:59.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.556+0000 7f84bcd14700 1 --2- 172.21.15.116:0/3844634054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b80b92d0 0x7f84b8062c60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f84a8006fd0 tx=0x7f84a8006eb0).ready entity=mon.2 client_cookie=7b6235afebde5bae server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:59.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.560+0000 7f84a7fff700 1 -- 172.21.15.116:0/3844634054 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8009810 con 0x7f84b80b92d0 2020-11-20T12:18:59.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.560+0000 7f84a7fff700 1 -- 172.21.15.116:0/3844634054 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a8009970 con 0x7f84b80b92d0 2020-11-20T12:18:59.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.560+0000 7f84a7fff700 1 -- 172.21.15.116:0/3844634054 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8009cf0 con 0x7f84b80b92d0 2020-11-20T12:18:59.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.560+0000 7f84be517700 1 -- 172.21.15.116:0/3844634054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b80b92d0 msgr2=0x7f84b8062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.560+0000 7f84be517700 1 --2- 172.21.15.116:0/3844634054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b80b92d0 0x7f84b8062c60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f84a8006fd0 tx=0x7f84a8006eb0).stop 2020-11-20T12:18:59.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.560+0000 7f84be517700 1 -- 172.21.15.116:0/3844634054 shutdown_connections 2020-11-20T12:18:59.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.560+0000 7f84be517700 1 --2- 172.21.15.116:0/3844634054 >> v2:172.21.15.173:3300/0 conn(0x7f84b814de90 0x7f84b8147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.560+0000 7f84be517700 1 --2- 172.21.15.116:0/3844634054 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b80b92d0 0x7f84b8062c60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.560+0000 7f84be517700 1 -- 172.21.15.116:0/3844634054 >> 172.21.15.116:0/3844634054 conn(0x7f84b8060c40 msgr2=0x7f84b80c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:59.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.560+0000 7f84be517700 1 -- 172.21.15.116:0/3844634054 shutdown_connections 2020-11-20T12:18:59.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.560+0000 7f84be517700 1 -- 172.21.15.116:0/3844634054 wait complete. 2020-11-20T12:18:59.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84be517700 1 Processor -- start 2020-11-20T12:18:59.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84be517700 1 -- start start 2020-11-20T12:18:59.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84be517700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b8062880 0x7f84b8166200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84be517700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b80b92d0 0x7f84b81441b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84be517700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b814de90 0x7f84b81452c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84be517700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b8068310 con 0x7f84b80b92d0 2020-11-20T12:18:59.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84be517700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b81666f0 con 0x7f84b8062880 2020-11-20T12:18:59.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84be517700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84b8066b50 con 0x7f84b814de90 2020-11-20T12:18:59.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84bcd14700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b80b92d0 0x7f84b81441b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:59.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84bd515700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b8062880 0x7f84b8166200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84bcd14700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b80b92d0 0x7f84b81441b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47334/0 (socket says 172.21.15.116:47334) 2020-11-20T12:18:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84bd515700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b8062880 0x7f84b8166200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:46384/0 (socket says 172.21.15.116:46384) 2020-11-20T12:18:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84bcd14700 1 -- 172.21.15.116:0/455324317 learned_addr learned my addr 172.21.15.116:0/455324317 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.564+0000 7f84bdd16700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b814de90 0x7f84b81452c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84bcd14700 1 -- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b814de90 msgr2=0x7f84b81452c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84bcd14700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b814de90 0x7f84b81452c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84bcd14700 1 -- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b8062880 msgr2=0x7f84b8166200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84bcd14700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b8062880 0x7f84b8166200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84bcd14700 1 -- 172.21.15.116:0/455324317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b8066df0 con 0x7f84b80b92d0 2020-11-20T12:18:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84bd515700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b8062880 0x7f84b8166200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:59.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84bdd16700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b814de90 0x7f84b81452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:18:59.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84bcd14700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b80b92d0 0x7f84b81441b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f84a8010000 tx=0x7f84a8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:59.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84a57fa700 1 -- 172.21.15.116:0/455324317 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a8011d50 con 0x7f84b80b92d0 2020-11-20T12:18:59.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84a57fa700 1 -- 172.21.15.116:0/455324317 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84a801e460 con 0x7f84b80b92d0 2020-11-20T12:18:59.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84be517700 1 -- 172.21.15.116:0/455324317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b8067090 con 0x7f84b80b92d0 2020-11-20T12:18:59.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84a57fa700 1 -- 172.21.15.116:0/455324317 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84a801e820 con 0x7f84b80b92d0 2020-11-20T12:18:59.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84be517700 1 -- 172.21.15.116:0/455324317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b80675d0 con 0x7f84b80b92d0 2020-11-20T12:18:59.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84be517700 1 -- 172.21.15.116:0/455324317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b8067760 con 0x7f84b80b92d0 2020-11-20T12:18:59.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.568+0000 7f84a57fa700 1 -- 172.21.15.116:0/455324317 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f84a8025030 con 0x7f84b80b92d0 2020-11-20T12:18:59.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.572+0000 7f84a57fa700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84900557c0 0x7f8490057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.572+0000 7f84a57fa700 1 -- 172.21.15.116:0/455324317 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f84a806ffe0 con 0x7f84b80b92d0 2020-11-20T12:18:59.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.572+0000 7f84bd515700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84900557c0 0x7f8490057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:59.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.572+0000 7f84bd515700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84900557c0 0x7f8490057c80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f84b4000bc0 tx=0x7f84b4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:59.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.576+0000 7f84a57fa700 1 -- 172.21.15.116:0/455324317 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f84a8045d80 con 0x7f84b80b92d0 2020-11-20T12:18:59.659 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:18:59.668 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:18:59.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.740+0000 7f84be517700 1 -- 172.21.15.116:0/455324317 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f84b805cf30 con 0x7f84b80b92d0 2020-11-20T12:18:59.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.740+0000 7f84a57fa700 1 -- 172.21.15.116:0/455324317 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18427 (secure 0 0 0) 0x7f84a80493a0 con 0x7f84b80b92d0 2020-11-20T12:18:59.740 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:18:59.740 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":59,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:18:49.533460+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:18:25.558441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-20T12:18:26.151785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":11,"pool_name":"fsvolume_volid","create_time":"2020-11-20T12:18:47.595737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:18:59.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 -- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84900557c0 msgr2=0x7f8490057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84900557c0 0x7f8490057c80 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f84b4000bc0 tx=0x7f84b4009040).stop 2020-11-20T12:18:59.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 -- 172.21.15.116:0/455324317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b80b92d0 msgr2=0x7f84b81441b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b80b92d0 0x7f84b81441b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f84a8010000 tx=0x7f84a8015000).stop 2020-11-20T12:18:59.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 -- 172.21.15.116:0/455324317 shutdown_connections 2020-11-20T12:18:59.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84900557c0 0x7f8490057c80 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84b814de90 0x7f84b81452c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84b8062880 0x7f84b8166200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 --2- 172.21.15.116:0/455324317 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84b80b92d0 0x7f84b81441b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 -- 172.21.15.116:0/455324317 >> 172.21.15.116:0/455324317 conn(0x7f84b8060c40 msgr2=0x7f84b814bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:59.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 -- 172.21.15.116:0/455324317 shutdown_connections 2020-11-20T12:18:59.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.744+0000 7f84be517700 1 -- 172.21.15.116:0/455324317 wait complete. 2020-11-20T12:18:59.765 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:18:25.558441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:18:26.151785+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 11, 'pool_name': 'fsvolume_volid', 'create_time': '2020-11-20T12:18:47.595737+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:18:59.766 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:18:59.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.948+0000 7f7ea9fb0700 1 Processor -- start 2020-11-20T12:18:59.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.948+0000 7f7ea9fb0700 1 -- start start 2020-11-20T12:18:59.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.948+0000 7f7ea9fb0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7ea40a81f0 0x7f7ea40b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.948+0000 7f7ea9fb0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7ea40c0e90 0x7f7ea40c1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.948+0000 7f7ea9fb0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea40a8930 con 0x7f7ea414f880 2020-11-20T12:18:59.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.948+0000 7f7ea9fb0700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea40ba2a0 con 0x7f7ea40c0e90 2020-11-20T12:18:59.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.948+0000 7f7ea9fb0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea40a8690 con 0x7f7ea40a81f0 2020-11-20T12:18:59.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea97af700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7ea40c0e90 0x7f7ea40c1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:59.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea97af700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7ea40c0e90 0x7f7ea40c1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47342/0 (socket says 172.21.15.116:47342) 2020-11-20T12:18:59.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea97af700 1 -- 172.21.15.116:0/3988408821 learned_addr learned my addr 172.21.15.116:0/3988408821 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:59.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea97af700 1 -- 172.21.15.116:0/3988408821 >> v2:172.21.15.173:3300/0 conn(0x7f7ea40a81f0 msgr2=0x7f7ea40b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:18:59.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea97af700 1 --2- 172.21.15.116:0/3988408821 >> v2:172.21.15.173:3300/0 conn(0x7f7ea40a81f0 0x7f7ea40b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:59.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea97af700 1 -- 172.21.15.116:0/3988408821 >> v1:172.21.15.116:6789/0 conn(0x7f7ea414f880 legacy=0x7f7ea4060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea97af700 1 -- 172.21.15.116:0/3988408821 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea40b94d0 con 0x7f7ea40c0e90 2020-11-20T12:18:59.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea97af700 1 --2- 172.21.15.116:0/3988408821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ea40c0e90 0x7f7ea40c1320 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f7ea40624e0 tx=0x7f7e9c009290).ready entity=mon.0 client_cookie=9ff5a168ee9bd2f9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:59.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea37fe700 1 -- 172.21.15.116:0/3988408821 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00bbc0 con 0x7f7ea40c0e90 2020-11-20T12:18:59.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea37fe700 1 -- 172.21.15.116:0/3988408821 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e9c00bd20 con 0x7f7ea40c0e90 2020-11-20T12:18:59.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea37fe700 1 -- 172.21.15.116:0/3988408821 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c011610 con 0x7f7ea40c0e90 2020-11-20T12:18:59.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/3988408821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ea40c0e90 msgr2=0x7f7ea40c1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea9fb0700 1 --2- 172.21.15.116:0/3988408821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ea40c0e90 0x7f7ea40c1320 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f7ea40624e0 tx=0x7f7e9c009290).stop 2020-11-20T12:18:59.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/3988408821 shutdown_connections 2020-11-20T12:18:59.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea9fb0700 1 --2- 172.21.15.116:0/3988408821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ea40c0e90 0x7f7ea40c1320 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea9fb0700 1 --2- 172.21.15.116:0/3988408821 >> v2:172.21.15.173:3300/0 conn(0x7f7ea40a81f0 0x7f7ea40b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:18:59.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/3988408821 >> 172.21.15.116:0/3988408821 conn(0x7f7ea4068350 msgr2=0x7f7ea40627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:18:59.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/3988408821 shutdown_connections 2020-11-20T12:18:59.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.952+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/3988408821 wait complete. 2020-11-20T12:18:59.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea9fb0700 1 Processor -- start 2020-11-20T12:18:59.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea9fb0700 1 -- start start 2020-11-20T12:18:59.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea9fb0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ea40a81f0 0x7f7ea414d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea9fb0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ea40c0e90 0x7f7ea40bd4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea9fb0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ea414f880 0x7f7ea40bd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea9fb0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea414fc60 con 0x7f7ea40c0e90 2020-11-20T12:18:59.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea9fb0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea414dde0 con 0x7f7ea40a81f0 2020-11-20T12:18:59.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea9fb0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea414fe90 con 0x7f7ea414f880 2020-11-20T12:18:59.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ea40c0e90 0x7f7ea40bd4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:59.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ea40c0e90 0x7f7ea40bd4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47346/0 (socket says 172.21.15.116:47346) 2020-11-20T12:18:59.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea3fff700 1 -- 172.21.15.116:0/2369593550 learned_addr learned my addr 172.21.15.116:0/2369593550 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:18:59.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea97af700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ea414f880 0x7f7ea40bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:59.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea3fff700 1 -- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ea414f880 msgr2=0x7f7ea40bd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea8fae700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ea40a81f0 0x7f7ea414d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:59.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea3fff700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ea414f880 0x7f7ea40bd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea3fff700 1 -- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ea40a81f0 msgr2=0x7f7ea414d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:18:59.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea3fff700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ea40a81f0 0x7f7ea414d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:18:59.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea3fff700 1 -- 172.21.15.116:0/2369593550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea4150130 con 0x7f7ea40c0e90 2020-11-20T12:18:59.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea97af700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ea414f880 0x7f7ea40bd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:59.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea8fae700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ea40a81f0 0x7f7ea414d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:18:59.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.956+0000 7f7ea3fff700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ea40c0e90 0x7f7ea40bd4a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f7e900096e0 tx=0x7f7e90006e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:59.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.960+0000 7f7ea0ff9700 1 -- 172.21.15.116:0/2369593550 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9000d5c0 con 0x7f7ea40c0e90 2020-11-20T12:18:59.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.960+0000 7f7ea0ff9700 1 -- 172.21.15.116:0/2369593550 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e90000e90 con 0x7f7ea40c0e90 2020-11-20T12:18:59.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.960+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/2369593550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea41503d0 con 0x7f7ea40c0e90 2020-11-20T12:18:59.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.960+0000 7f7ea0ff9700 1 -- 172.21.15.116:0/2369593550 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e9000d970 con 0x7f7ea40c0e90 2020-11-20T12:18:59.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.960+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/2369593550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea4150b10 con 0x7f7ea40c0e90 2020-11-20T12:18:59.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.960+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/2369593550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea405cf30 con 0x7f7ea40c0e90 2020-11-20T12:18:59.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.960+0000 7f7ea0ff9700 1 -- 172.21.15.116:0/2369593550 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7e9000dce0 con 0x7f7ea40c0e90 2020-11-20T12:18:59.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.964+0000 7f7ea0ff9700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e7c0557c0 0x7f7e7c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:18:59.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.964+0000 7f7ea0ff9700 1 -- 172.21.15.116:0/2369593550 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f7e9006e740 con 0x7f7ea40c0e90 2020-11-20T12:18:59.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.964+0000 7f7ea8fae700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e7c0557c0 0x7f7e7c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:18:59.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.964+0000 7f7ea8fae700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e7c0557c0 0x7f7e7c057c80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f7e98001430 tx=0x7f7e98009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:18:59.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:18:59.968+0000 7f7ea0ff9700 1 -- 172.21.15.116:0/2369593550 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7e90044380 con 0x7f7ea40c0e90 2020-11-20T12:19:00.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.128+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/2369593550 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7ea4152d30 con 0x7f7ea40c0e90 2020-11-20T12:19:00.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.128+0000 7f7ea0ff9700 1 -- 172.21.15.116:0/2369593550 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v59) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7e900479a0 con 0x7f7ea40c0e90 2020-11-20T12:19:00.128 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:19:00.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e7c0557c0 msgr2=0x7f7e7c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e7c0557c0 0x7f7e7c057c80 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f7e98001430 tx=0x7f7e98009040).stop 2020-11-20T12:19:00.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/2369593550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ea40c0e90 msgr2=0x7f7ea40bd4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ea40c0e90 0x7f7ea40bd4a0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f7e900096e0 tx=0x7f7e90006e20).stop 2020-11-20T12:19:00.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/2369593550 shutdown_connections 2020-11-20T12:19:00.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e7c0557c0 0x7f7e7c057c80 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7ea414f880 0x7f7ea40bd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7ea40a81f0 0x7f7ea414d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 --2- 172.21.15.116:0/2369593550 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7ea40c0e90 0x7f7ea40bd4a0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/2369593550 >> 172.21.15.116:0/2369593550 conn(0x7f7ea4068350 msgr2=0x7f7ea414ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:00.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/2369593550 shutdown_connections 2020-11-20T12:19:00.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.132+0000 7f7ea9fb0700 1 -- 172.21.15.116:0/2369593550 wait complete. 2020-11-20T12:19:00.155 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:19:00.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.324+0000 7feadfefb700 1 Processor -- start 2020-11-20T12:19:00.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.324+0000 7feadfefb700 1 -- start start 2020-11-20T12:19:00.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.328+0000 7feadfefb700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fead8149df0 0x7fead8153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.328+0000 7feadfefb700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fead814a1d0 0x7fead80be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.328+0000 7feadfefb700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead80a9c20 con 0x7fead80a9750 2020-11-20T12:19:00.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.328+0000 7feadfefb700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead80a9ec0 con 0x7fead8149df0 2020-11-20T12:19:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.328+0000 7feadfefb700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead8153670 con 0x7fead814a1d0 2020-11-20T12:19:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.328+0000 7feaddc97700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fead8149df0 0x7fead8153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.328+0000 7feaddc97700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fead8149df0 0x7fead8153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47354/0 (socket says 172.21.15.116:47354) 2020-11-20T12:19:00.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.328+0000 7feaddc97700 1 -- 172.21.15.116:0/826389899 learned_addr learned my addr 172.21.15.116:0/826389899 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.328+0000 7feade498700 1 --2- 172.21.15.116:0/826389899 >> v2:172.21.15.173:3300/0 conn(0x7fead814a1d0 0x7fead80be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feaddc97700 1 -- 172.21.15.116:0/826389899 >> v2:172.21.15.173:3300/0 conn(0x7fead814a1d0 msgr2=0x7fead80be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feaddc97700 1 --2- 172.21.15.116:0/826389899 >> v2:172.21.15.173:3300/0 conn(0x7fead814a1d0 0x7fead80be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feaddc97700 1 -- 172.21.15.116:0/826389899 >> v1:172.21.15.116:6789/0 conn(0x7fead80a9750 legacy=0x7fead80c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feaddc97700 1 -- 172.21.15.116:0/826389899 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8153910 con 0x7fead8149df0 2020-11-20T12:19:00.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feaddc97700 1 --2- 172.21.15.116:0/826389899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fead8149df0 0x7fead8153db0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fead80c2370 tx=0x7fead4006eb0).ready entity=mon.0 client_cookie=8c4a6a6521f6cba4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feadcc95700 1 -- 172.21.15.116:0/826389899 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4009810 con 0x7fead8149df0 2020-11-20T12:19:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feadcc95700 1 -- 172.21.15.116:0/826389899 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead4009970 con 0x7fead8149df0 2020-11-20T12:19:00.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feadcc95700 1 -- 172.21.15.116:0/826389899 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4009cf0 con 0x7fead8149df0 2020-11-20T12:19:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feadfefb700 1 -- 172.21.15.116:0/826389899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fead8149df0 msgr2=0x7fead8153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feadfefb700 1 --2- 172.21.15.116:0/826389899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fead8149df0 0x7fead8153db0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7fead80c2370 tx=0x7fead4006eb0).stop 2020-11-20T12:19:00.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feadfefb700 1 -- 172.21.15.116:0/826389899 shutdown_connections 2020-11-20T12:19:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feadfefb700 1 --2- 172.21.15.116:0/826389899 >> v2:172.21.15.173:3300/0 conn(0x7fead814a1d0 0x7fead80be240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feadfefb700 1 --2- 172.21.15.116:0/826389899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fead8149df0 0x7fead8153db0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feadfefb700 1 -- 172.21.15.116:0/826389899 >> 172.21.15.116:0/826389899 conn(0x7fead805ba80 msgr2=0x7fead80c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feadfefb700 1 -- 172.21.15.116:0/826389899 shutdown_connections 2020-11-20T12:19:00.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.332+0000 7feadfefb700 1 -- 172.21.15.116:0/826389899 wait complete. 2020-11-20T12:19:00.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feadfefb700 1 Processor -- start 2020-11-20T12:19:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feadfefb700 1 -- start start 2020-11-20T12:19:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feadfefb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fead80a9750 0x7fead81477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feadfefb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fead8149df0 0x7fead8165420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feadfefb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fead814a1d0 0x7fead814c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feadfefb700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead814d5e0 con 0x7fead814a1d0 2020-11-20T12:19:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feadfefb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead814d0a0 con 0x7fead8149df0 2020-11-20T12:19:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feadfefb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead814d340 con 0x7fead80a9750 2020-11-20T12:19:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feade498700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fead814a1d0 0x7fead814c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feade498700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fead814a1d0 0x7fead814c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47364/0 (socket says 172.21.15.116:47364) 2020-11-20T12:19:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feade498700 1 -- 172.21.15.116:0/197929426 learned_addr learned my addr 172.21.15.116:0/197929426 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feadd496700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fead8149df0 0x7fead8165420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.336+0000 7feaddc97700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fead80a9750 0x7fead81477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feade498700 1 -- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fead80a9750 msgr2=0x7fead81477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feade498700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fead80a9750 0x7fead81477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feade498700 1 -- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fead8149df0 msgr2=0x7fead8165420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feade498700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fead8149df0 0x7fead8165420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feade498700 1 -- 172.21.15.116:0/197929426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8164ce0 con 0x7fead814a1d0 2020-11-20T12:19:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feade498700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fead814a1d0 0x7fead814c720 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fead000ec30 tx=0x7fead00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:00.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feac5ffb700 1 -- 172.21.15.116:0/197929426 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead000cac0 con 0x7fead814a1d0 2020-11-20T12:19:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feac5ffb700 1 -- 172.21.15.116:0/197929426 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead000cc20 con 0x7fead814a1d0 2020-11-20T12:19:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feac5ffb700 1 -- 172.21.15.116:0/197929426 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead00123f0 con 0x7fead814a1d0 2020-11-20T12:19:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feadfefb700 1 -- 172.21.15.116:0/197929426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead8164f80 con 0x7fead814a1d0 2020-11-20T12:19:00.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feadfefb700 1 -- 172.21.15.116:0/197929426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead814cc10 con 0x7fead814a1d0 2020-11-20T12:19:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.340+0000 7feadfefb700 1 -- 172.21.15.116:0/197929426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead8165110 con 0x7fead814a1d0 2020-11-20T12:19:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.344+0000 7feac5ffb700 1 -- 172.21.15.116:0/197929426 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fead0022070 con 0x7fead814a1d0 2020-11-20T12:19:00.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.344+0000 7feac5ffb700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feab00557c0 0x7feab0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.344+0000 7feac5ffb700 1 -- 172.21.15.116:0/197929426 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fead006efa0 con 0x7fead814a1d0 2020-11-20T12:19:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.344+0000 7feaddc97700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feab00557c0 0x7feab0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.344+0000 7feaddc97700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feab00557c0 0x7feab0057c80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fead4006c40 tx=0x7fead4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:00.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.348+0000 7feac5ffb700 1 -- 172.21.15.116:0/197929426 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fead0044cc0 con 0x7fead814a1d0 2020-11-20T12:19:00.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.512+0000 7feadfefb700 1 -- 172.21.15.116:0/197929426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fead805d2a0 con 0x7fead814a1d0 2020-11-20T12:19:00.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.512+0000 7feac5ffb700 1 -- 172.21.15.116:0/197929426 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v59) v1 ==== 92+0+10 (secure 0 0 0) 0x7fead0016070 con 0x7fead814a1d0 2020-11-20T12:19:00.510 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:00.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 -- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feab00557c0 msgr2=0x7feab0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feab00557c0 0x7feab0057c80 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7fead4006c40 tx=0x7fead4015000).stop 2020-11-20T12:19:00.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 -- 172.21.15.116:0/197929426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fead814a1d0 msgr2=0x7fead814c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fead814a1d0 0x7fead814c720 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7fead000ec30 tx=0x7fead00091d0).stop 2020-11-20T12:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 -- 172.21.15.116:0/197929426 shutdown_connections 2020-11-20T12:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feab00557c0 0x7feab0057c80 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fead814a1d0 0x7fead814c720 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fead80a9750 0x7fead81477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 --2- 172.21.15.116:0/197929426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fead8149df0 0x7fead8165420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 -- 172.21.15.116:0/197929426 >> 172.21.15.116:0/197929426 conn(0x7fead805ba80 msgr2=0x7fead8062880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:00.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 -- 172.21.15.116:0/197929426 shutdown_connections 2020-11-20T12:19:00.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.516+0000 7feadfefb700 1 -- 172.21.15.116:0/197929426 wait complete. 2020-11-20T12:19:00.536 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:19:00.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.724+0000 7f01e0ab0700 1 Processor -- start 2020-11-20T12:19:00.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.724+0000 7f01e0ab0700 1 -- start start 2020-11-20T12:19:00.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.724+0000 7f01e0ab0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f01dc148cf0 0x7f01dc0a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.724+0000 7f01e0ab0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f01dc0c5060 0x7f01dc14f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.724+0000 7f01e0ab0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01dc0a8fa0 con 0x7f01dc0c4bb0 2020-11-20T12:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.724+0000 7f01e0ab0700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01dc152710 con 0x7f01dc0c5060 2020-11-20T12:19:00.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.724+0000 7f01e0ab0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01dc1529b0 con 0x7f01dc148cf0 2020-11-20T12:19:00.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.724+0000 7f01d9d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f01dc0c4bb0 0x7f01dc0a66e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56802/0 (socket says 172.21.15.116:56802) 2020-11-20T12:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.724+0000 7f01d9d9b700 1 -- 172.21.15.116:0/3468910746 learned_addr learned my addr 172.21.15.116:0/3468910746 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.724+0000 7f01dad9d700 1 --2- 172.21.15.116:0/3468910746 >> v2:172.21.15.116:3300/0 conn(0x7f01dc0c5060 0x7f01dc14f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01da59c700 1 --2- 172.21.15.116:0/3468910746 >> v2:172.21.15.173:3300/0 conn(0x7f01dc148cf0 0x7f01dc0a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01d959a700 1 -- 172.21.15.116:0/3468910746 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3019858636 0 0) 0x7f01d00010d0 con 0x7f01dc0c4bb0 2020-11-20T12:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01d959a700 1 -- 172.21.15.116:0/3468910746 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01c0003240 con 0x7f01dc0c4bb0 2020-11-20T12:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01dad9d700 1 -- 172.21.15.116:0/3468910746 >> v2:172.21.15.173:3300/0 conn(0x7f01dc148cf0 msgr2=0x7f01dc0a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01dad9d700 1 --2- 172.21.15.116:0/3468910746 >> v2:172.21.15.173:3300/0 conn(0x7f01dc148cf0 0x7f01dc0a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01dad9d700 1 -- 172.21.15.116:0/3468910746 >> v1:172.21.15.116:6789/0 conn(0x7f01dc0c4bb0 legacy=0x7f01dc0a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01d959a700 1 -- 172.21.15.116:0/3468910746 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3700950421 0 0) 0x7f01c0003240 con 0x7f01dc0c4bb0 2020-11-20T12:19:00.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01dad9d700 1 -- 172.21.15.116:0/3468910746 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc152c50 con 0x7f01dc0c5060 2020-11-20T12:19:00.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01dad9d700 1 --2- 172.21.15.116:0/3468910746 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc0c5060 0x7f01dc14f030 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f01dc066290 tx=0x7f01d40091d0).ready entity=mon.0 client_cookie=35965b876e4d85ef server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:00.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01d959a700 1 -- 172.21.15.116:0/3468910746 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0001670 con 0x7f01dc0c5060 2020-11-20T12:19:00.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01d959a700 1 -- 172.21.15.116:0/3468910746 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d400b6b0 con 0x7f01dc0c5060 2020-11-20T12:19:00.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01d959a700 1 -- 172.21.15.116:0/3468910746 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d400ba30 con 0x7f01dc0c5060 2020-11-20T12:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01e0ab0700 1 -- 172.21.15.116:0/3468910746 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc0c5060 msgr2=0x7f01dc14f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01e0ab0700 1 --2- 172.21.15.116:0/3468910746 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc0c5060 0x7f01dc14f030 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f01dc066290 tx=0x7f01d40091d0).stop 2020-11-20T12:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01e0ab0700 1 -- 172.21.15.116:0/3468910746 shutdown_connections 2020-11-20T12:19:00.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01e0ab0700 1 --2- 172.21.15.116:0/3468910746 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc0c5060 0x7f01dc14f030 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01e0ab0700 1 --2- 172.21.15.116:0/3468910746 >> v2:172.21.15.173:3300/0 conn(0x7f01dc148cf0 0x7f01dc0a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01e0ab0700 1 -- 172.21.15.116:0/3468910746 >> 172.21.15.116:0/3468910746 conn(0x7f01dc05ba80 msgr2=0x7f01dc0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:00.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01e0ab0700 1 -- 172.21.15.116:0/3468910746 shutdown_connections 2020-11-20T12:19:00.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.728+0000 7f01e0ab0700 1 -- 172.21.15.116:0/3468910746 wait complete. 2020-11-20T12:19:00.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.732+0000 7f01e0ab0700 1 Processor -- start 2020-11-20T12:19:00.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.732+0000 7f01e0ab0700 1 -- start start 2020-11-20T12:19:00.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.732+0000 7f01e0ab0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0c4bb0 0x7f01dc147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.732+0000 7f01e0ab0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc0c5060 0x7f01dc145dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.732+0000 7f01e0ab0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc148cf0 0x7f01dc161330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.732+0000 7f01e0ab0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc162330 con 0x7f01dc0c5060 2020-11-20T12:19:00.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.732+0000 7f01e0ab0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc161b50 con 0x7f01dc148cf0 2020-11-20T12:19:00.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.732+0000 7f01e0ab0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc161df0 con 0x7f01dc0c4bb0 2020-11-20T12:19:00.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.732+0000 7f01d9d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc0c5060 0x7f01dc145dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.732+0000 7f01d9d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc0c5060 0x7f01dc145dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47376/0 (socket says 172.21.15.116:47376) 2020-11-20T12:19:00.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.732+0000 7f01d9d9b700 1 -- 172.21.15.116:0/1164312599 learned_addr learned my addr 172.21.15.116:0/1164312599 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:00.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01da59c700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0c4bb0 0x7f01dc147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01dad9d700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc148cf0 0x7f01dc161330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01d9d9b700 1 -- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0c4bb0 msgr2=0x7f01dc147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01d9d9b700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0c4bb0 0x7f01dc147550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01d9d9b700 1 -- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc148cf0 msgr2=0x7f01dc161330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01d9d9b700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc148cf0 0x7f01dc161330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01d9d9b700 1 -- 172.21.15.116:0/1164312599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01dc162090 con 0x7f01dc0c5060 2020-11-20T12:19:00.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01da59c700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0c4bb0 0x7f01dc147550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:00.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01d9d9b700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc0c5060 0x7f01dc145dc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f01d0001580 tx=0x7f01d0000fa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:00.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01ca7fc700 1 -- 172.21.15.116:0/1164312599 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0007170 con 0x7f01dc0c5060 2020-11-20T12:19:00.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01ca7fc700 1 -- 172.21.15.116:0/1164312599 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d00072d0 con 0x7f01dc0c5060 2020-11-20T12:19:00.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01e0ab0700 1 -- 172.21.15.116:0/1164312599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01dc1491e0 con 0x7f01dc0c5060 2020-11-20T12:19:00.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01ca7fc700 1 -- 172.21.15.116:0/1164312599 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d000aa00 con 0x7f01dc0c5060 2020-11-20T12:19:00.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01e0ab0700 1 -- 172.21.15.116:0/1164312599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01dc14acb0 con 0x7f01dc0c5060 2020-11-20T12:19:00.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.736+0000 7f01e0ab0700 1 -- 172.21.15.116:0/1164312599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01dc067490 con 0x7f01dc0c5060 2020-11-20T12:19:00.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.740+0000 7f01ca7fc700 1 -- 172.21.15.116:0/1164312599 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f01d0010070 con 0x7f01dc0c5060 2020-11-20T12:19:00.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.740+0000 7f01ca7fc700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01b00557c0 0x7f01b0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:00.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.740+0000 7f01ca7fc700 1 -- 172.21.15.116:0/1164312599 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f01d006e220 con 0x7f01dc0c5060 2020-11-20T12:19:00.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.740+0000 7f01da59c700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01b00557c0 0x7f01b0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:00.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.740+0000 7f01da59c700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01b00557c0 0x7f01b0057c80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f01cc006fd0 tx=0x7f01cc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:00.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.744+0000 7f01ca7fc700 1 -- 172.21.15.116:0/1164312599 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f01dc067490 con 0x7f01dc0c5060 2020-11-20T12:19:00.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.904+0000 7f01e0ab0700 1 -- 172.21.15.116:0/1164312599 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f01dc067490 con 0x7f01dc0c5060 2020-11-20T12:19:00.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.904+0000 7f01ca7fc700 1 -- 172.21.15.116:0/1164312599 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v59) v1 ==== 104+0+10 (secure 0 0 0) 0x7f01dc067490 con 0x7f01dc0c5060 2020-11-20T12:19:00.904 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:00.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 -- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01b00557c0 msgr2=0x7f01b0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01b00557c0 0x7f01b0057c80 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f01cc006fd0 tx=0x7f01cc008040).stop 2020-11-20T12:19:00.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 -- 172.21.15.116:0/1164312599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc0c5060 msgr2=0x7f01dc145dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:00.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc0c5060 0x7f01dc145dc0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f01d0001580 tx=0x7f01d0000fa0).stop 2020-11-20T12:19:00.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 -- 172.21.15.116:0/1164312599 shutdown_connections 2020-11-20T12:19:00.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01b00557c0 0x7f01b0057c80 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01dc148cf0 0x7f01dc161330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01dc0c4bb0 0x7f01dc147550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 --2- 172.21.15.116:0/1164312599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01dc0c5060 0x7f01dc145dc0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:00.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 -- 172.21.15.116:0/1164312599 >> 172.21.15.116:0/1164312599 conn(0x7f01dc05ba80 msgr2=0x7f01dc0c42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:00.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 -- 172.21.15.116:0/1164312599 shutdown_connections 2020-11-20T12:19:00.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:00.908+0000 7f01e0ab0700 1 -- 172.21.15.116:0/1164312599 wait complete. 2020-11-20T12:19:00.928 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:19:01.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.108+0000 7f7f71a11700 1 Processor -- start 2020-11-20T12:19:01.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.108+0000 7f7f71a11700 1 -- start start 2020-11-20T12:19:01.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f71a11700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f6c155620 0x7f7f6c155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f71a11700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7f6c060170 0x7f7f6c060600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f71a11700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6c0ba670 con 0x7f7f6c14c210 2020-11-20T12:19:01.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f71a11700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f6c14c5f0 con 0x7f7f6c060170 2020-11-20T12:19:01.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f71a11700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0ba420 con 0x7f7f6c155620 2020-11-20T12:19:01.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7f6c060170 0x7f7f6c060600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7f6c060170 0x7f7f6c060600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47384/0 (socket says 172.21.15.116:47384) 2020-11-20T12:19:01.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6b7fe700 1 -- 172.21.15.116:0/2194647729 learned_addr learned my addr 172.21.15.116:0/2194647729 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:01.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f63fff700 1 --2- 172.21.15.116:0/2194647729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f6c155620 0x7f7f6c155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6b7fe700 1 -- 172.21.15.116:0/2194647729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f6c155620 msgr2=0x7f7f6c155aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:01.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6b7fe700 1 --2- 172.21.15.116:0/2194647729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f6c155620 0x7f7f6c155aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6b7fe700 1 -- 172.21.15.116:0/2194647729 >> v1:172.21.15.116:6789/0 conn(0x7f7f6c14c210 legacy=0x7f7f6c0b9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6b7fe700 1 -- 172.21.15.116:0/2194647729 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0c02c0 con 0x7f7f6c060170 2020-11-20T12:19:01.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6a7fc700 1 -- 172.21.15.116:0/2194647729 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2108083832 0 0) 0x7f7f6c0ba670 con 0x7f7f6c14c210 2020-11-20T12:19:01.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6b7fe700 1 --2- 172.21.15.116:0/2194647729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f6c060170 0x7f7f6c060600 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f7f64009d80 tx=0x7f7f64009290).ready entity=mon.0 client_cookie=f000ac84387d3279 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:01.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6a7fc700 1 -- 172.21.15.116:0/2194647729 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6400bb80 con 0x7f7f6c060170 2020-11-20T12:19:01.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6a7fc700 1 -- 172.21.15.116:0/2194647729 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6400bce0 con 0x7f7f6c060170 2020-11-20T12:19:01.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.112+0000 7f7f6a7fc700 1 -- 172.21.15.116:0/2194647729 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f64011610 con 0x7f7f6c060170 2020-11-20T12:19:01.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.116+0000 7f7f71a11700 1 -- 172.21.15.116:0/2194647729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f6c060170 msgr2=0x7f7f6c060600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.116+0000 7f7f71a11700 1 --2- 172.21.15.116:0/2194647729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f6c060170 0x7f7f6c060600 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f7f64009d80 tx=0x7f7f64009290).stop 2020-11-20T12:19:01.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.116+0000 7f7f71a11700 1 -- 172.21.15.116:0/2194647729 shutdown_connections 2020-11-20T12:19:01.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.116+0000 7f7f71a11700 1 --2- 172.21.15.116:0/2194647729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f6c060170 0x7f7f6c060600 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.116+0000 7f7f71a11700 1 --2- 172.21.15.116:0/2194647729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f6c155620 0x7f7f6c155aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.116+0000 7f7f71a11700 1 -- 172.21.15.116:0/2194647729 >> 172.21.15.116:0/2194647729 conn(0x7f7f6c0c5880 msgr2=0x7f7f6c0c1850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:01.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.116+0000 7f7f71a11700 1 -- 172.21.15.116:0/2194647729 shutdown_connections 2020-11-20T12:19:01.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.116+0000 7f7f71a11700 1 -- 172.21.15.116:0/2194647729 wait complete. 2020-11-20T12:19:01.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.116+0000 7f7f71a11700 1 Processor -- start 2020-11-20T12:19:01.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.116+0000 7f7f71a11700 1 -- start start 2020-11-20T12:19:01.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f71a11700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f6c060170 0x7f7f6c139670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f71a11700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f6c14c210 0x7f7f6c14b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f71a11700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f6c155620 0x7f7f6c14b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f71a11700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0a9b90 con 0x7f7f6c060170 2020-11-20T12:19:01.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f71a11700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0aad10 con 0x7f7f6c14c210 2020-11-20T12:19:01.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f71a11700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c0a9df0 con 0x7f7f6c155620 2020-11-20T12:19:01.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f6affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f6c060170 0x7f7f6c139670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f6affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f6c060170 0x7f7f6c139670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47388/0 (socket says 172.21.15.116:47388) 2020-11-20T12:19:01.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f6affd700 1 -- 172.21.15.116:0/2887396238 learned_addr learned my addr 172.21.15.116:0/2887396238 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:01.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f6b7fe700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f6c155620 0x7f7f6c14b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f63fff700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f6c14c210 0x7f7f6c14b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f6affd700 1 -- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f6c155620 msgr2=0x7f7f6c14b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f6affd700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f6c155620 0x7f7f6c14b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f6affd700 1 -- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f6c14c210 msgr2=0x7f7f6c14b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f6affd700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f6c14c210 0x7f7f6c14b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f6affd700 1 -- 172.21.15.116:0/2887396238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f6c0aa090 con 0x7f7f6c060170 2020-11-20T12:19:01.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.120+0000 7f7f6affd700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f6c060170 0x7f7f6c139670 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f7f5c007fd0 tx=0x7f7f5c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:01.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.124+0000 7f7f62ffd700 1 -- 172.21.15.116:0/2887396238 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00e560 con 0x7f7f6c060170 2020-11-20T12:19:01.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.124+0000 7f7f62ffd700 1 -- 172.21.15.116:0/2887396238 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f5c00e6e0 con 0x7f7f6c060170 2020-11-20T12:19:01.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.124+0000 7f7f62ffd700 1 -- 172.21.15.116:0/2887396238 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5c00eaa0 con 0x7f7f6c060170 2020-11-20T12:19:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.124+0000 7f7f71a11700 1 -- 172.21.15.116:0/2887396238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c0aa330 con 0x7f7f6c060170 2020-11-20T12:19:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.124+0000 7f7f71a11700 1 -- 172.21.15.116:0/2887396238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c0aa870 con 0x7f7f6c060170 2020-11-20T12:19:01.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.124+0000 7f7f71a11700 1 -- 172.21.15.116:0/2887396238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c149a40 con 0x7f7f6c060170 2020-11-20T12:19:01.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.124+0000 7f7f62ffd700 1 -- 172.21.15.116:0/2887396238 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7f5c019030 con 0x7f7f6c060170 2020-11-20T12:19:01.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.124+0000 7f7f62ffd700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f480557c0 0x7f7f48057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.128+0000 7f7f62ffd700 1 -- 172.21.15.116:0/2887396238 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f7f5c06de60 con 0x7f7f6c060170 2020-11-20T12:19:01.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.128+0000 7f7f63fff700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f480557c0 0x7f7f48057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.128+0000 7f7f63fff700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f480557c0 0x7f7f48057c80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f7f6c0a99a0 tx=0x7f7f54006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:01.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.132+0000 7f7f62ffd700 1 -- 172.21.15.116:0/2887396238 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7f6c149a40 con 0x7f7f6c060170 2020-11-20T12:19:01.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.300+0000 7f7f71a11700 1 -- 172.21.15.116:0/2887396238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f6c149a40 con 0x7f7f6c060170 2020-11-20T12:19:01.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.300+0000 7f7f62ffd700 1 -- 172.21.15.116:0/2887396238 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v59) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7f6c149a40 con 0x7f7f6c060170 2020-11-20T12:19:01.299 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:01.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 -- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f480557c0 msgr2=0x7f7f48057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f480557c0 0x7f7f48057c80 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f7f6c0a99a0 tx=0x7f7f54006df0).stop 2020-11-20T12:19:01.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 -- 172.21.15.116:0/2887396238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f6c060170 msgr2=0x7f7f6c139670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f6c060170 0x7f7f6c139670 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f7f5c007fd0 tx=0x7f7f5c007ac0).stop 2020-11-20T12:19:01.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 -- 172.21.15.116:0/2887396238 shutdown_connections 2020-11-20T12:19:01.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f480557c0 0x7f7f48057c80 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f6c155620 0x7f7f6c14b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f6c060170 0x7f7f6c139670 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 --2- 172.21.15.116:0/2887396238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f6c14c210 0x7f7f6c14b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 -- 172.21.15.116:0/2887396238 >> 172.21.15.116:0/2887396238 conn(0x7f7f6c0c5880 msgr2=0x7f7f6c063ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:01.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 -- 172.21.15.116:0/2887396238 shutdown_connections 2020-11-20T12:19:01.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.304+0000 7f7f71a11700 1 -- 172.21.15.116:0/2887396238 wait complete. 2020-11-20T12:19:01.326 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fsvolume_volid pg_num 2020-11-20T12:19:01.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.504+0000 7fc82954a700 1 Processor -- start 2020-11-20T12:19:01.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.504+0000 7fc82954a700 1 -- start start 2020-11-20T12:19:01.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc82954a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8241518c0 0x7fc824151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc82954a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc8240c9010 0x7fc824061550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc82954a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8240a92f0 con 0x7fc824061a40 2020-11-20T12:19:01.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc82954a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8240c9450 con 0x7fc8240c9010 2020-11-20T12:19:01.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc82954a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240a9590 con 0x7fc8241518c0 2020-11-20T12:19:01.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc81bfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc8240c9010 0x7fc824061550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc81bfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc8240c9010 0x7fc824061550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47398/0 (socket says 172.21.15.116:47398) 2020-11-20T12:19:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc81bfff700 1 -- 172.21.15.116:0/3766075726 learned_addr learned my addr 172.21.15.116:0/3766075726 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc822ffd700 1 --2- 172.21.15.116:0/3766075726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8241518c0 0x7fc824151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc81bfff700 1 -- 172.21.15.116:0/3766075726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8241518c0 msgr2=0x7fc824151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:01.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc81bfff700 1 --2- 172.21.15.116:0/3766075726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8241518c0 0x7fc824151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc81bfff700 1 -- 172.21.15.116:0/3766075726 >> v1:172.21.15.116:6789/0 conn(0x7fc824061a40 legacy=0x7fc8240ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc81bfff700 1 -- 172.21.15.116:0/3766075726 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240a9830 con 0x7fc8240c9010 2020-11-20T12:19:01.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc822ffd700 1 --2- 172.21.15.116:0/3766075726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8241518c0 0x7fc824151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:01.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc81bfff700 1 --2- 172.21.15.116:0/3766075726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8240c9010 0x7fc824061550 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fc80c006fd0 tx=0x7fc80c006eb0).ready entity=mon.0 client_cookie=e45dd99b897f35d5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:01.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.508+0000 7fc8227fc700 1 -- 172.21.15.116:0/3766075726 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c0098d0 con 0x7fc8240c9010 2020-11-20T12:19:01.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.512+0000 7fc8227fc700 1 -- 172.21.15.116:0/3766075726 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c009a30 con 0x7fc8240c9010 2020-11-20T12:19:01.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.512+0000 7fc8227fc700 1 -- 172.21.15.116:0/3766075726 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c009db0 con 0x7fc8240c9010 2020-11-20T12:19:01.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.512+0000 7fc82954a700 1 -- 172.21.15.116:0/3766075726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8240c9010 msgr2=0x7fc824061550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.512+0000 7fc82954a700 1 --2- 172.21.15.116:0/3766075726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8240c9010 0x7fc824061550 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7fc80c006fd0 tx=0x7fc80c006eb0).stop 2020-11-20T12:19:01.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.512+0000 7fc82954a700 1 -- 172.21.15.116:0/3766075726 shutdown_connections 2020-11-20T12:19:01.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.512+0000 7fc82954a700 1 --2- 172.21.15.116:0/3766075726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8241518c0 0x7fc824151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.512+0000 7fc82954a700 1 --2- 172.21.15.116:0/3766075726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8240c9010 0x7fc824061550 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.512+0000 7fc82954a700 1 -- 172.21.15.116:0/3766075726 >> 172.21.15.116:0/3766075726 conn(0x7fc824068370 msgr2=0x7fc8240c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:01.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.512+0000 7fc82954a700 1 -- 172.21.15.116:0/3766075726 shutdown_connections 2020-11-20T12:19:01.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.512+0000 7fc82954a700 1 -- 172.21.15.116:0/3766075726 wait complete. 2020-11-20T12:19:01.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc82954a700 1 Processor -- start 2020-11-20T12:19:01.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc82954a700 1 -- start start 2020-11-20T12:19:01.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc82954a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc824061a40 0x7fc824161fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc82954a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8240c9010 0x7fc8240c19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc82954a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8241518c0 0x7fc8240a7d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc82954a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82409f430 con 0x7fc8240c9010 2020-11-20T12:19:01.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc82954a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240c21b0 con 0x7fc824061a40 2020-11-20T12:19:01.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc82954a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240c2450 con 0x7fc8241518c0 2020-11-20T12:19:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc81bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8240c9010 0x7fc8240c19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc81bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8240c9010 0x7fc8240c19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47406/0 (socket says 172.21.15.116:47406) 2020-11-20T12:19:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc81bfff700 1 -- 172.21.15.116:0/1496106548 learned_addr learned my addr 172.21.15.116:0/1496106548 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc8237fe700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8241518c0 0x7fc8240a7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc822ffd700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc824061a40 0x7fc824161fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc81bfff700 1 -- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8241518c0 msgr2=0x7fc8240a7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc81bfff700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8241518c0 0x7fc8240a7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc81bfff700 1 -- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc824061a40 msgr2=0x7fc824161fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc81bfff700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc824061a40 0x7fc824161fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.516+0000 7fc81bfff700 1 -- 172.21.15.116:0/1496106548 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240c26f0 con 0x7fc8240c9010 2020-11-20T12:19:01.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.520+0000 7fc81bfff700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8240c9010 0x7fc8240c19c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fc80c010000 tx=0x7fc80c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:01.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.520+0000 7fc81affd700 1 -- 172.21.15.116:0/1496106548 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c006780 con 0x7fc8240c9010 2020-11-20T12:19:01.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.520+0000 7fc81affd700 1 -- 172.21.15.116:0/1496106548 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc80c011d50 con 0x7fc8240c9010 2020-11-20T12:19:01.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.520+0000 7fc82954a700 1 -- 172.21.15.116:0/1496106548 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8240c2990 con 0x7fc8240c9010 2020-11-20T12:19:01.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.520+0000 7fc81affd700 1 -- 172.21.15.116:0/1496106548 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c01e6b0 con 0x7fc8240c9010 2020-11-20T12:19:01.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.520+0000 7fc82954a700 1 -- 172.21.15.116:0/1496106548 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8240a8220 con 0x7fc8240c9010 2020-11-20T12:19:01.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.520+0000 7fc82954a700 1 -- 172.21.15.116:0/1496106548 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82405d4b0 con 0x7fc8240c9010 2020-11-20T12:19:01.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.520+0000 7fc81affd700 1 -- 172.21.15.116:0/1496106548 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc80c025030 con 0x7fc8240c9010 2020-11-20T12:19:01.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.524+0000 7fc81affd700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8000557c0 0x7fc800057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.524+0000 7fc81affd700 1 -- 172.21.15.116:0/1496106548 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fc80c06fd10 con 0x7fc8240c9010 2020-11-20T12:19:01.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.524+0000 7fc822ffd700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8000557c0 0x7fc800057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.524+0000 7fc822ffd700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8000557c0 0x7fc800057c80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fc81400aa10 tx=0x7fc814008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:01.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.528+0000 7fc81affd700 1 -- 172.21.15.116:0/1496106548 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc80c045ab0 con 0x7fc8240c9010 2020-11-20T12:19:01.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.688+0000 7fc82954a700 1 -- 172.21.15.116:0/1496106548 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fsvolume_volid", "var": "pg_num"} v 0) v1 -- 0x7fc824069950 con 0x7fc8240c9010 2020-11-20T12:19:01.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.688+0000 7fc81affd700 1 -- 172.21.15.116:0/1496106548 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fsvolume_volid", "var": "pg_num"}]=0 v59) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc80c023070 con 0x7fc8240c9010 2020-11-20T12:19:01.688 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2020-11-20T12:19:01.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 -- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8000557c0 msgr2=0x7fc800057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8000557c0 0x7fc800057c80 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fc81400aa10 tx=0x7fc814008040).stop 2020-11-20T12:19:01.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 -- 172.21.15.116:0/1496106548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8240c9010 msgr2=0x7fc8240c19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8240c9010 0x7fc8240c19c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7fc80c010000 tx=0x7fc80c015000).stop 2020-11-20T12:19:01.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 -- 172.21.15.116:0/1496106548 shutdown_connections 2020-11-20T12:19:01.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8000557c0 0x7fc800057c80 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8241518c0 0x7fc8240a7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc824061a40 0x7fc824161fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 --2- 172.21.15.116:0/1496106548 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8240c9010 0x7fc8240c19c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 -- 172.21.15.116:0/1496106548 >> 172.21.15.116:0/1496106548 conn(0x7fc824068370 msgr2=0x7fc824150750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:01.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 -- 172.21.15.116:0/1496106548 shutdown_connections 2020-11-20T12:19:01.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.692+0000 7fc82954a700 1 -- 172.21.15.116:0/1496106548 wait complete. 2020-11-20T12:19:01.716 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:19:01.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.892+0000 7ffac53da700 1 Processor -- start 2020-11-20T12:19:01.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.892+0000 7ffac53da700 1 -- start start 2020-11-20T12:19:01.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffac53da700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ffac0151940 0x7ffac00c4380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffac53da700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac00615d0 0x7ffac0065310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffac53da700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffac0066070 0x7ffac0065800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffac53da700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac0060950 con 0x7ffac0066070 2020-11-20T12:19:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffac53da700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffac0060bf0 con 0x7ffac0151940 2020-11-20T12:19:01.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffac53da700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac0060e90 con 0x7ffac00615d0 2020-11-20T12:19:01.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffabffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffac0066070 0x7ffac0065800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffabffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffac0066070 0x7ffac0065800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47412/0 (socket says 172.21.15.116:47412) 2020-11-20T12:19:01.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffabffff700 1 -- 172.21.15.116:0/3524620295 learned_addr learned my addr 172.21.15.116:0/3524620295 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:01.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffabeffd700 1 --2- 172.21.15.116:0/3524620295 >> v2:172.21.15.173:3300/0 conn(0x7ffac0151940 0x7ffac00c4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffabffff700 1 -- 172.21.15.116:0/3524620295 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac00615d0 msgr2=0x7ffac0065310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffabe7fc700 1 --2- 172.21.15.116:0/3524620295 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac00615d0 0x7ffac0065310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.896+0000 7ffabffff700 1 --2- 172.21.15.116:0/3524620295 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac00615d0 0x7ffac0065310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffabffff700 1 -- 172.21.15.116:0/3524620295 >> v2:172.21.15.173:3300/0 conn(0x7ffac0151940 msgr2=0x7ffac00c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:01.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffabffff700 1 --2- 172.21.15.116:0/3524620295 >> v2:172.21.15.173:3300/0 conn(0x7ffac0151940 0x7ffac00c4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffabffff700 1 -- 172.21.15.116:0/3524620295 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac0061130 con 0x7ffac0066070 2020-11-20T12:19:01.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffabeffd700 1 --2- 172.21.15.116:0/3524620295 >> v2:172.21.15.173:3300/0 conn(0x7ffac0151940 0x7ffac00c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:01.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffabe7fc700 1 --2- 172.21.15.116:0/3524620295 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac00615d0 0x7ffac0065310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:01.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffabffff700 1 --2- 172.21.15.116:0/3524620295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffac0066070 0x7ffac0065800 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7ffac0066b00 tx=0x7ffab8009290).ready entity=mon.0 client_cookie=49517539e089e292 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:01.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffabdffb700 1 -- 172.21.15.116:0/3524620295 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab800bbc0 con 0x7ffac0066070 2020-11-20T12:19:01.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffabdffb700 1 -- 172.21.15.116:0/3524620295 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab800bd20 con 0x7ffac0066070 2020-11-20T12:19:01.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffabdffb700 1 -- 172.21.15.116:0/3524620295 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab8011610 con 0x7ffac0066070 2020-11-20T12:19:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffac53da700 1 -- 172.21.15.116:0/3524620295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffac0066070 msgr2=0x7ffac0065800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffac53da700 1 --2- 172.21.15.116:0/3524620295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffac0066070 0x7ffac0065800 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7ffac0066b00 tx=0x7ffab8009290).stop 2020-11-20T12:19:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffac53da700 1 -- 172.21.15.116:0/3524620295 shutdown_connections 2020-11-20T12:19:01.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffac53da700 1 --2- 172.21.15.116:0/3524620295 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffac0066070 0x7ffac0065800 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffac53da700 1 --2- 172.21.15.116:0/3524620295 >> v2:172.21.15.173:3300/0 conn(0x7ffac0151940 0x7ffac00c4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffac53da700 1 --2- 172.21.15.116:0/3524620295 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac00615d0 0x7ffac0065310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffac53da700 1 -- 172.21.15.116:0/3524620295 >> 172.21.15.116:0/3524620295 conn(0x7ffac00ca850 msgr2=0x7ffac00c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffac53da700 1 -- 172.21.15.116:0/3524620295 shutdown_connections 2020-11-20T12:19:01.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffac53da700 1 -- 172.21.15.116:0/3524620295 wait complete. 2020-11-20T12:19:01.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffac53da700 1 Processor -- start 2020-11-20T12:19:01.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.900+0000 7ffac53da700 1 -- start start 2020-11-20T12:19:01.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffac53da700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac00615d0 0x7ffac00ac3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffac53da700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffac0066070 0x7ffac00af3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffac53da700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffac0151940 0x7ffac00af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffac53da700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac00ac8c0 con 0x7ffac0066070 2020-11-20T12:19:01.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffac53da700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac00afdd0 con 0x7ffac0151940 2020-11-20T12:19:01.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffac53da700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac00aa6c0 con 0x7ffac00615d0 2020-11-20T12:19:01.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffabe7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffac0066070 0x7ffac00af3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffabe7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffac0066070 0x7ffac00af3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47420/0 (socket says 172.21.15.116:47420) 2020-11-20T12:19:01.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffabe7fc700 1 -- 172.21.15.116:0/2902994376 learned_addr learned my addr 172.21.15.116:0/2902994376 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:01.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffabffff700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffac0151940 0x7ffac00af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffabeffd700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac00615d0 0x7ffac00ac3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffabe7fc700 1 -- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac00615d0 msgr2=0x7ffac00ac3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffabe7fc700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac00615d0 0x7ffac00ac3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffabe7fc700 1 -- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffac0151940 msgr2=0x7ffac00af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffabe7fc700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffac0151940 0x7ffac00af8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.904+0000 7ffabe7fc700 1 -- 172.21.15.116:0/2902994376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac0164460 con 0x7ffac0066070 2020-11-20T12:19:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.908+0000 7ffabe7fc700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffac0066070 0x7ffac00af3f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7ffab400cb50 tx=0x7ffab4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:01.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.908+0000 7ffaa6ffd700 1 -- 172.21.15.116:0/2902994376 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab40077a0 con 0x7ffac0066070 2020-11-20T12:19:01.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.908+0000 7ffaa6ffd700 1 -- 172.21.15.116:0/2902994376 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffab4007900 con 0x7ffac0066070 2020-11-20T12:19:01.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.908+0000 7ffac53da700 1 -- 172.21.15.116:0/2902994376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffac0164700 con 0x7ffac0066070 2020-11-20T12:19:01.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.908+0000 7ffaa6ffd700 1 -- 172.21.15.116:0/2902994376 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffab4007c80 con 0x7ffac0066070 2020-11-20T12:19:01.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.908+0000 7ffac53da700 1 -- 172.21.15.116:0/2902994376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffac0164c40 con 0x7ffac0066070 2020-11-20T12:19:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.908+0000 7ffac53da700 1 -- 172.21.15.116:0/2902994376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffac00c4b10 con 0x7ffac0066070 2020-11-20T12:19:01.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.908+0000 7ffaa6ffd700 1 -- 172.21.15.116:0/2902994376 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ffab4011620 con 0x7ffac0066070 2020-11-20T12:19:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.912+0000 7ffaa6ffd700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa9c0557c0 0x7ffa9c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.912+0000 7ffaa6ffd700 1 -- 172.21.15.116:0/2902994376 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7ffab406e9b0 con 0x7ffac0066070 2020-11-20T12:19:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.912+0000 7ffabeffd700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa9c0557c0 0x7ffa9c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:01.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.912+0000 7ffabeffd700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa9c0557c0 0x7ffa9c057c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7ffab000ba80 tx=0x7ffab0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:01.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:01.916+0000 7ffaa6ffd700 1 -- 172.21.15.116:0/2902994376 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ffab40446d0 con 0x7ffac0066070 2020-11-20T12:19:02.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.124+0000 7ffac53da700 1 -- 172.21.15.116:0/2902994376 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffac005d2a0 con 0x7ffac0066070 2020-11-20T12:19:02.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffaa6ffd700 1 -- 172.21.15.116:0/2902994376 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+2853 (secure 0 0 0) 0x7ffab4047cf0 con 0x7ffac0066070 2020-11-20T12:19:02.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 -- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa9c0557c0 msgr2=0x7ffa9c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:02.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa9c0557c0 0x7ffa9c057c80 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7ffab000ba80 tx=0x7ffab0008040).stop 2020-11-20T12:19:02.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 -- 172.21.15.116:0/2902994376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffac0066070 msgr2=0x7ffac00af3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:02.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffac0066070 0x7ffac00af3f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7ffab400cb50 tx=0x7ffab4008040).stop 2020-11-20T12:19:02.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 -- 172.21.15.116:0/2902994376 shutdown_connections 2020-11-20T12:19:02.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa9c0557c0 0x7ffa9c057c80 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffac0151940 0x7ffac00af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffac00615d0 0x7ffac00ac3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 --2- 172.21.15.116:0/2902994376 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffac0066070 0x7ffac00af3f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 -- 172.21.15.116:0/2902994376 >> 172.21.15.116:0/2902994376 conn(0x7ffac00ca850 msgr2=0x7ffac0153100 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:02.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 -- 172.21.15.116:0/2902994376 shutdown_connections 2020-11-20T12:19:02.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.128+0000 7ffac53da700 1 -- 172.21.15.116:0/2902994376 wait complete. 2020-11-20T12:19:02.130 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:02.130 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":29,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5387,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/785878283","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":785878283},{"type":"v1","addr":"172.21.15.173:6837","nonce":785878283}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5952,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/1568769426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":1568769426},{"type":"v1","addr":"172.21.15.173:6836","nonce":1568769426}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24},{"gid":5964,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3552490451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3552490451},{"type":"v1","addr":"172.21.15.116:6833","nonce":3552490451}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:18:27.137821+0000","modified":"2020-11-20T12:18:48.530378+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5970},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5970":{"gid":5970,"name":"c","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/3370673707","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3370673707},{"type":"v1","addr":"172.21.15.116:6835","nonce":3370673707}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[10,11],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2020-11-20T12:19:02.131 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 29 2020-11-20T12:19:02.151 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:02.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.332+0000 7fc879564700 1 Processor -- start 2020-11-20T12:19:02.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc879564700 1 -- start start 2020-11-20T12:19:02.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc879564700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc87414f800 0x7fc874065330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc879564700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc8740aa570 0x7fc874065820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc879564700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc8740a7320 0x7fc8740a7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc879564700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc87414fbe0 con 0x7fc8740a7320 2020-11-20T12:19:02.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc879564700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8740a98f0 con 0x7fc8740aa570 2020-11-20T12:19:02.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc879564700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740a9b90 con 0x7fc87414f800 2020-11-20T12:19:02.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc873fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc8740a7320 0x7fc8740a7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc873fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc8740a7320 0x7fc8740a7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47426/0 (socket says 172.21.15.116:47426) 2020-11-20T12:19:02.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc873fff700 1 -- 172.21.15.116:0/2731352587 learned_addr learned my addr 172.21.15.116:0/2731352587 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:02.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc872ffd700 1 --2- 172.21.15.116:0/2731352587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc87414f800 0x7fc874065330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc86bfff700 1 --2- 172.21.15.116:0/2731352587 >> v2:172.21.15.173:3300/0 conn(0x7fc8740aa570 0x7fc874065820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc873fff700 1 -- 172.21.15.116:0/2731352587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc87414f800 msgr2=0x7fc874065330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc873fff700 1 --2- 172.21.15.116:0/2731352587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc87414f800 0x7fc874065330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc873fff700 1 -- 172.21.15.116:0/2731352587 >> v2:172.21.15.173:3300/0 conn(0x7fc8740aa570 msgr2=0x7fc874065820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:02.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.336+0000 7fc873fff700 1 --2- 172.21.15.116:0/2731352587 >> v2:172.21.15.173:3300/0 conn(0x7fc8740aa570 0x7fc874065820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc873fff700 1 -- 172.21.15.116:0/2731352587 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740a9e30 con 0x7fc8740a7320 2020-11-20T12:19:02.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc873fff700 1 --2- 172.21.15.116:0/2731352587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8740a7320 0x7fc8740a7770 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fc87405c4a0 tx=0x7fc86c009290).ready entity=mon.0 client_cookie=12c1de33a0beea47 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:02.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc8727fc700 1 -- 172.21.15.116:0/2731352587 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c00bb10 con 0x7fc8740a7320 2020-11-20T12:19:02.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc8727fc700 1 -- 172.21.15.116:0/2731352587 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c00bc70 con 0x7fc8740a7320 2020-11-20T12:19:02.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc8727fc700 1 -- 172.21.15.116:0/2731352587 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c011400 con 0x7fc8740a7320 2020-11-20T12:19:02.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc879564700 1 -- 172.21.15.116:0/2731352587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8740a7320 msgr2=0x7fc8740a7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:02.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc879564700 1 --2- 172.21.15.116:0/2731352587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8740a7320 0x7fc8740a7770 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fc87405c4a0 tx=0x7fc86c009290).stop 2020-11-20T12:19:02.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc879564700 1 -- 172.21.15.116:0/2731352587 shutdown_connections 2020-11-20T12:19:02.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc879564700 1 --2- 172.21.15.116:0/2731352587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc8740a7320 0x7fc8740a7770 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc879564700 1 --2- 172.21.15.116:0/2731352587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc87414f800 0x7fc874065330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc879564700 1 --2- 172.21.15.116:0/2731352587 >> v2:172.21.15.173:3300/0 conn(0x7fc8740aa570 0x7fc874065820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc879564700 1 -- 172.21.15.116:0/2731352587 >> 172.21.15.116:0/2731352587 conn(0x7fc8740c63f0 msgr2=0x7fc8740c23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:02.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc879564700 1 -- 172.21.15.116:0/2731352587 shutdown_connections 2020-11-20T12:19:02.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.340+0000 7fc879564700 1 -- 172.21.15.116:0/2731352587 wait complete. 2020-11-20T12:19:02.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc879564700 1 Processor -- start 2020-11-20T12:19:02.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc879564700 1 -- start start 2020-11-20T12:19:02.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc879564700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8740a7320 0x7fc87413aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc879564700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8740aa570 0x7fc87413afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc879564700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc87414f800 0x7fc87413b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc879564700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874152810 con 0x7fc87414f800 2020-11-20T12:19:02.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc879564700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8741522d0 con 0x7fc8740a7320 2020-11-20T12:19:02.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc879564700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc874152570 con 0x7fc8740aa570 2020-11-20T12:19:02.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc873fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc87414f800 0x7fc87413b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc872ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8740a7320 0x7fc87413aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc872ffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8740a7320 0x7fc87413aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:46482/0 (socket says 172.21.15.116:46482) 2020-11-20T12:19:02.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc873fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc87414f800 0x7fc87413b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47434/0 (socket says 172.21.15.116:47434) 2020-11-20T12:19:02.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc872ffd700 1 -- 172.21.15.116:0/1038372665 learned_addr learned my addr 172.21.15.116:0/1038372665 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:02.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.344+0000 7fc86bfff700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8740aa570 0x7fc87413afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc872ffd700 1 -- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8740aa570 msgr2=0x7fc87413afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:02.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc872ffd700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8740aa570 0x7fc87413afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc872ffd700 1 -- 172.21.15.116:0/1038372665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc87414f800 msgr2=0x7fc87413b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:02.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc872ffd700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc87414f800 0x7fc87413b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc872ffd700 1 -- 172.21.15.116:0/1038372665 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc874142980 con 0x7fc8740a7320 2020-11-20T12:19:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc872ffd700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8740a7320 0x7fc87413aad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fc86400ebd0 tx=0x7fc864006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc86affd700 1 -- 172.21.15.116:0/1038372665 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400b3b0 con 0x7fc8740a7320 2020-11-20T12:19:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc879564700 1 -- 172.21.15.116:0/1038372665 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc874142c20 con 0x7fc8740a7320 2020-11-20T12:19:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc86affd700 1 -- 172.21.15.116:0/1038372665 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86400c8e0 con 0x7fc8740a7320 2020-11-20T12:19:02.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc86affd700 1 -- 172.21.15.116:0/1038372665 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86400cc80 con 0x7fc8740a7320 2020-11-20T12:19:02.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc879564700 1 -- 172.21.15.116:0/1038372665 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87413cf90 con 0x7fc8740a7320 2020-11-20T12:19:02.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc879564700 1 -- 172.21.15.116:0/1038372665 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87405d2a0 con 0x7fc8740a7320 2020-11-20T12:19:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.348+0000 7fc86affd700 1 -- 172.21.15.116:0/1038372665 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc864033750 con 0x7fc8740a7320 2020-11-20T12:19:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.352+0000 7fc86affd700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8500557f0 0x7fc850057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.352+0000 7fc86affd700 1 -- 172.21.15.116:0/1038372665 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fc86406ed50 con 0x7fc8740a7320 2020-11-20T12:19:02.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.352+0000 7fc86bfff700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8500557f0 0x7fc850057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.352+0000 7fc86bfff700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8500557f0 0x7fc850057cb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7fc8740c4fe0 tx=0x7fc85c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:02.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.360+0000 7fc86affd700 1 -- 172.21.15.116:0/1038372665 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc864044a70 con 0x7fc8740a7320 2020-11-20T12:19:02.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.520+0000 7fc879564700 1 -- 172.21.15.116:0/1038372665 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8740a7100 con 0x7fc8740a7320 2020-11-20T12:19:02.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc86affd700 1 -- 172.21.15.116:0/1038372665 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+18427 (secure 0 0 0) 0x7fc864048090 con 0x7fc8740a7320 2020-11-20T12:19:02.521 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:02.521 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":59,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:18:49.533460+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:18:25.558441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2020-11-20T12:18:26.151785+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":11,"pool_name":"fsvolume_volid","create_time":"2020-11-20T12:18:47.595737+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:02.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 -- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8500557f0 msgr2=0x7fc850057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:02.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8500557f0 0x7fc850057cb0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7fc8740c4fe0 tx=0x7fc85c006df0).stop 2020-11-20T12:19:02.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 -- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8740a7320 msgr2=0x7fc87413aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:02.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8740a7320 0x7fc87413aad0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7fc86400ebd0 tx=0x7fc864006df0).stop 2020-11-20T12:19:02.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 -- 172.21.15.116:0/1038372665 shutdown_connections 2020-11-20T12:19:02.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8500557f0 0x7fc850057cb0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc87414f800 0x7fc87413b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc8740a7320 0x7fc87413aad0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 --2- 172.21.15.116:0/1038372665 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc8740aa570 0x7fc87413afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 -- 172.21.15.116:0/1038372665 >> 172.21.15.116:0/1038372665 conn(0x7fc8740c63f0 msgr2=0x7fc8740c39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 -- 172.21.15.116:0/1038372665 shutdown_connections 2020-11-20T12:19:02.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.524+0000 7fc879564700 1 -- 172.21.15.116:0/1038372665 wait complete. 2020-11-20T12:19:02.547 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-20T12:19:02.548 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-20T12:19:02.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.764+0000 7f9c9d08e700 1 Processor -- start 2020-11-20T12:19:02.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.764+0000 7f9c9d08e700 1 -- start start 2020-11-20T12:19:02.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.764+0000 7f9c9d08e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9c980a6840 0x7f9c9814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.764+0000 7f9c9d08e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a6c20 0x7f9c980bff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.764+0000 7f9c9d08e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c98150340 con 0x7f9c980c0440 2020-11-20T12:19:02.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.764+0000 7f9c9d08e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c980677e0 con 0x7f9c980a6840 2020-11-20T12:19:02.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.764+0000 7f9c9d08e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98067a80 con 0x7f9c980a6c20 2020-11-20T12:19:02.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.764+0000 7f9c9759e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9c980c0440 0x7f9c98066cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56876/0 (socket says 172.21.15.116:56876) 2020-11-20T12:19:02.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.764+0000 7f9c9759e700 1 -- 172.21.15.116:0/3222750540 learned_addr learned my addr 172.21.15.116:0/3222750540 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:02.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c8e59c700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a6c20 0x7f9c980bff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c96d9d700 1 --2- 172.21.15.116:0/3222750540 >> v2:172.21.15.173:3300/0 conn(0x7f9c980a6840 0x7f9c9814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577161819 0 0) 0x7f9c98150340 con 0x7f9c980c0440 2020-11-20T12:19:02.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c78003240 con 0x7f9c980c0440 2020-11-20T12:19:02.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3191575484 0 0) 0x7f9c78003240 con 0x7f9c980c0440 2020-11-20T12:19:02.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a6c20 msgr2=0x7f9c980bff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:02.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a6c20 0x7f9c980bff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 >> v2:172.21.15.173:3300/0 conn(0x7f9c980a6840 msgr2=0x7f9c9814fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:02.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 --2- 172.21.15.116:0/3222750540 >> v2:172.21.15.173:3300/0 conn(0x7f9c980a6840 0x7f9c9814fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c98067d20 con 0x7f9c980c0440 2020-11-20T12:19:02.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f9c90001860 con 0x7f9c980c0440 2020-11-20T12:19:02.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 >> v1:172.21.15.116:6789/0 conn(0x7f9c980c0440 legacy=0x7f9c98066cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:02.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a6c20 0x7f9c78003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c78003c40 0x7f9c780044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c780049d0 0x7f9c78004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c980677e0 con 0x7f9c78003c40 2020-11-20T12:19:02.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98067a80 con 0x7f9c780049d0 2020-11-20T12:19:02.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c78004200 con 0x7f9c980a6c20 2020-11-20T12:19:02.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c8e59c700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c78003c40 0x7f9c780044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c96d9d700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a6c20 0x7f9c78003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c8e59c700 1 -- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a6c20 msgr2=0x7f9c78003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:02.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c8e59c700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a6c20 0x7f9c78003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c8e59c700 1 -- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c780049d0 msgr2=0x7f9c78004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:02.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9759e700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c780049d0 0x7f9c78004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c8e59c700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c780049d0 0x7f9c78004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c8e59c700 1 -- 172.21.15.116:0/3222750540 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c78003240 con 0x7f9c78003c40 2020-11-20T12:19:02.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c96d9d700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a6c20 0x7f9c78003730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:02.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9759e700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c780049d0 0x7f9c78004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:02.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c8e59c700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c78003c40 0x7f9c780044e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f9c8000bc00 tx=0x7f9c80006df0).ready entity=mon.0 client_cookie=9a60a79495b3d852 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:02.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8000b310 con 0x7f9c78003c40 2020-11-20T12:19:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c8000eae0 con 0x7f9c78003c40 2020-11-20T12:19:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9659c700 1 -- 172.21.15.116:0/3222750540 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c8000ee60 con 0x7f9c78003c40 2020-11-20T12:19:02.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9d08e700 1 -- 172.21.15.116:0/3222750540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c78003c40 msgr2=0x7f9c780044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:02.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9d08e700 1 --2- 172.21.15.116:0/3222750540 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c78003c40 0x7f9c780044e0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f9c8000bc00 tx=0x7f9c80006df0).stop 2020-11-20T12:19:02.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.768+0000 7f9c9759e700 1 -- 172.21.15.116:0/3222750540 reap_dead start 2020-11-20T12:19:02.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.772+0000 7f9c9d08e700 1 -- 172.21.15.116:0/3222750540 shutdown_connections 2020-11-20T12:19:02.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.772+0000 7f9c9d08e700 1 -- 172.21.15.116:0/3222750540 >> 172.21.15.116:0/3222750540 conn(0x7f9c98060170 msgr2=0x7f9c98060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:02.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.772+0000 7f9c9d08e700 1 -- 172.21.15.116:0/3222750540 shutdown_connections 2020-11-20T12:19:02.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.772+0000 7f9c9d08e700 1 -- 172.21.15.116:0/3222750540 wait complete. 2020-11-20T12:19:02.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.772+0000 7f9c9d08e700 1 Processor -- start 2020-11-20T12:19:02.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.772+0000 7f9c9d08e700 1 -- start start 2020-11-20T12:19:02.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c9d08e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c98148b70 0x7f9c98148f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c9d08e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a9980 0x7f9c98094d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c9d08e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c9808e260 0x7f9c98095210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c9d08e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9814b100 con 0x7f9c9808e260 2020-11-20T12:19:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c9d08e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98095700 con 0x7f9c98148b70 2020-11-20T12:19:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c9d08e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9814a1e0 con 0x7f9c980a9980 2020-11-20T12:19:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c9759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c9808e260 0x7f9c98095210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c96d9d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c98148b70 0x7f9c98148f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c9759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c9808e260 0x7f9c98095210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47454/0 (socket says 172.21.15.116:47454) 2020-11-20T12:19:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c9759e700 1 -- 172.21.15.116:0/1711195470 learned_addr learned my addr 172.21.15.116:0/1711195470 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c8e59c700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a9980 0x7f9c98094d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c96d9d700 1 -- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a9980 msgr2=0x7f9c98094d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c96d9d700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a9980 0x7f9c98094d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c96d9d700 1 -- 172.21.15.116:0/1711195470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c9808e260 msgr2=0x7f9c98095210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c96d9d700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c9808e260 0x7f9c98095210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:02.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c96d9d700 1 -- 172.21.15.116:0/1711195470 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c9814a480 con 0x7f9c98148b70 2020-11-20T12:19:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c96d9d700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c98148b70 0x7f9c98148f50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f9c8800f830 tx=0x7f9c88008040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c8f7fe700 1 -- 172.21.15.116:0/1711195470 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c88017070 con 0x7f9c98148b70 2020-11-20T12:19:02.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c8f7fe700 1 -- 172.21.15.116:0/1711195470 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c88011660 con 0x7f9c98148b70 2020-11-20T12:19:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c9d08e700 1 -- 172.21.15.116:0/1711195470 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9814a720 con 0x7f9c98148b70 2020-11-20T12:19:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c8f7fe700 1 -- 172.21.15.116:0/1711195470 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c880119e0 con 0x7f9c98148b70 2020-11-20T12:19:02.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.776+0000 7f9c9d08e700 1 -- 172.21.15.116:0/1711195470 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9814ac60 con 0x7f9c98148b70 2020-11-20T12:19:02.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.780+0000 7f9c9d08e700 1 -- 172.21.15.116:0/1711195470 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c9814adf0 con 0x7f9c98148b70 2020-11-20T12:19:02.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.780+0000 7f9c8f7fe700 1 -- 172.21.15.116:0/1711195470 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9c8801c410 con 0x7f9c98148b70 2020-11-20T12:19:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.780+0000 7f9c8f7fe700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c740557c0 0x7f9c74057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.780+0000 7f9c8f7fe700 1 -- 172.21.15.116:0/1711195470 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f9c8806e7f0 con 0x7f9c98148b70 2020-11-20T12:19:02.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.780+0000 7f9c8e59c700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c740557c0 0x7f9c74057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.784+0000 7f9c8e59c700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c740557c0 0x7f9c74057c80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f9c9805e250 tx=0x7f9c80019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:02.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.788+0000 7f9c8f7fe700 1 -- 172.21.15.116:0/1711195470 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9c880444b0 con 0x7f9c98148b70 2020-11-20T12:19:02.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:02.996+0000 7f9c9d08e700 1 -- 172.21.15.116:0/1711195470 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9c9805d2a0 con 0x7f9c98148b70 2020-11-20T12:19:03.233 INFO:tasks.ceph.mon.a.smithi116.stderr:2020-11-20T12:19:03.236+0000 7fc111d1b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-20T12:19:03.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.248+0000 7f9c8f7fe700 1 -- 172.21.15.116:0/1711195470 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9c88047ad0 con 0x7f9c98148b70 2020-11-20T12:19:03.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 -- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c740557c0 msgr2=0x7f9c74057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:03.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c740557c0 0x7f9c74057c80 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f9c9805e250 tx=0x7f9c80019000).stop 2020-11-20T12:19:03.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 -- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c98148b70 msgr2=0x7f9c98148f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:03.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c98148b70 0x7f9c98148f50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f9c8800f830 tx=0x7f9c88008040).stop 2020-11-20T12:19:03.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 -- 172.21.15.116:0/1711195470 shutdown_connections 2020-11-20T12:19:03.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c740557c0 0x7f9c74057c80 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c9808e260 0x7f9c98095210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c98148b70 0x7f9c98148f50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 --2- 172.21.15.116:0/1711195470 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c980a9980 0x7f9c98094d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 -- 172.21.15.116:0/1711195470 >> 172.21.15.116:0/1711195470 conn(0x7f9c98060170 msgr2=0x7f9c98066c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:03.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 -- 172.21.15.116:0/1711195470 shutdown_connections 2020-11-20T12:19:03.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.252+0000 7f9c9d08e700 1 -- 172.21.15.116:0/1711195470 wait complete. 2020-11-20T12:19:03.254 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-20T12:19:03.260 INFO:tasks.ceph.mds.c.smithi116.stderr: -3841> 2020-11-20T12:18:31.476+0000 7fd4125d6700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:19:03.277 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-20T12:19:03.305 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:19:03.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.444+0000 7f9ac9730700 1 Processor -- start 2020-11-20T12:19:03.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.444+0000 7f9ac9730700 1 -- start start 2020-11-20T12:19:03.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.448+0000 7f9ac9730700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac4150cb0 0x7f9ac4065be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.448+0000 7f9ac9730700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9ac40c1f60 0x7f9ac414f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.448+0000 7f9ac9730700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac4150070 con 0x7f9ac4065800 2020-11-20T12:19:03.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.448+0000 7f9ac9730700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac41502d0 con 0x7f9ac40c1f60 2020-11-20T12:19:03.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.448+0000 7f9ac9730700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4150570 con 0x7f9ac4150cb0 2020-11-20T12:19:03.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.448+0000 7f9ac3fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9ac40c1f60 0x7f9ac414f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:03.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.448+0000 7f9ac3fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9ac40c1f60 0x7f9ac414f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47474/0 (socket says 172.21.15.116:47474) 2020-11-20T12:19:03.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.448+0000 7f9ac3fff700 1 -- 172.21.15.116:0/1144101545 learned_addr learned my addr 172.21.15.116:0/1144101545 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:03.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.448+0000 7f9ac27fc700 1 --2- 172.21.15.116:0/1144101545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac4150cb0 0x7f9ac4065be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:03.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac3fff700 1 -- 172.21.15.116:0/1144101545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac4150cb0 msgr2=0x7f9ac4065be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:03.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac3fff700 1 --2- 172.21.15.116:0/1144101545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac4150cb0 0x7f9ac4065be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:03.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac3fff700 1 -- 172.21.15.116:0/1144101545 >> v1:172.21.15.173:6789/0 conn(0x7f9ac4065800 legacy=0x7f9ac40c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:03.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac3fff700 1 -- 172.21.15.116:0/1144101545 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4150810 con 0x7f9ac40c1f60 2020-11-20T12:19:03.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac3fff700 1 --2- 172.21.15.116:0/1144101545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ac40c1f60 0x7f9ac414f490 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f9abc009d50 tx=0x7f9abc009290).ready entity=mon.0 client_cookie=ea197214b0bf39b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:03.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac1ffb700 1 -- 172.21.15.116:0/1144101545 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc00bbc0 con 0x7f9ac40c1f60 2020-11-20T12:19:03.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac1ffb700 1 -- 172.21.15.116:0/1144101545 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9abc00bd20 con 0x7f9ac40c1f60 2020-11-20T12:19:03.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac1ffb700 1 -- 172.21.15.116:0/1144101545 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9abc011610 con 0x7f9ac40c1f60 2020-11-20T12:19:03.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac9730700 1 -- 172.21.15.116:0/1144101545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ac40c1f60 msgr2=0x7f9ac414f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:03.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac9730700 1 --2- 172.21.15.116:0/1144101545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ac40c1f60 0x7f9ac414f490 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f9abc009d50 tx=0x7f9abc009290).stop 2020-11-20T12:19:03.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac9730700 1 -- 172.21.15.116:0/1144101545 shutdown_connections 2020-11-20T12:19:03.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac9730700 1 --2- 172.21.15.116:0/1144101545 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ac40c1f60 0x7f9ac414f490 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:03.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac9730700 1 --2- 172.21.15.116:0/1144101545 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac4150cb0 0x7f9ac4065be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:03.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac9730700 1 -- 172.21.15.116:0/1144101545 >> 172.21.15.116:0/1144101545 conn(0x7f9ac40c6480 msgr2=0x7f9ac40c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac9730700 1 -- 172.21.15.116:0/1144101545 shutdown_connections 2020-11-20T12:19:03.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.452+0000 7f9ac9730700 1 -- 172.21.15.116:0/1144101545 wait complete. 2020-11-20T12:19:03.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac9730700 1 Processor -- start 2020-11-20T12:19:03.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac9730700 1 -- start start 2020-11-20T12:19:03.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac9730700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ac4065800 0x7f9ac40c13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac9730700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac40c1f60 0x7f9ac40bf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac9730700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac4150cb0 0x7f9ac4164960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac9730700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac4060d00 con 0x7f9ac4065800 2020-11-20T12:19:03.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac9730700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40bfc00 con 0x7f9ac4150cb0 2020-11-20T12:19:03.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac9730700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac40bfe90 con 0x7f9ac40c1f60 2020-11-20T12:19:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ac4065800 0x7f9ac40c13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ac4065800 0x7f9ac40c13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47476/0 (socket says 172.21.15.116:47476) 2020-11-20T12:19:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac2ffd700 1 -- 172.21.15.116:0/4114315329 learned_addr learned my addr 172.21.15.116:0/4114315329 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:03.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac3fff700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac4150cb0 0x7f9ac4164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:03.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac27fc700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac40c1f60 0x7f9ac40bf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:03.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac2ffd700 1 -- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac40c1f60 msgr2=0x7f9ac40bf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:03.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac2ffd700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac40c1f60 0x7f9ac40bf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:03.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac2ffd700 1 -- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac4150cb0 msgr2=0x7f9ac4164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:03.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac2ffd700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac4150cb0 0x7f9ac4164960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:03.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac2ffd700 1 -- 172.21.15.116:0/4114315329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4060f60 con 0x7f9ac4065800 2020-11-20T12:19:03.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.456+0000 7f9ac2ffd700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ac4065800 0x7f9ac40c13a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f9aac001e70 tx=0x7f9aac007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:03.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.460+0000 7f9abaffd700 1 -- 172.21.15.116:0/4114315329 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac007020 con 0x7f9ac4065800 2020-11-20T12:19:03.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.460+0000 7f9abaffd700 1 -- 172.21.15.116:0/4114315329 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac007180 con 0x7f9ac4065800 2020-11-20T12:19:03.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.460+0000 7f9ac9730700 1 -- 172.21.15.116:0/4114315329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac4061200 con 0x7f9ac4065800 2020-11-20T12:19:03.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.460+0000 7f9abaffd700 1 -- 172.21.15.116:0/4114315329 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0064f0 con 0x7f9ac4065800 2020-11-20T12:19:03.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.460+0000 7f9ac9730700 1 -- 172.21.15.116:0/4114315329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac40619c0 con 0x7f9ac4065800 2020-11-20T12:19:03.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.460+0000 7f9ac9730700 1 -- 172.21.15.116:0/4114315329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac405d2a0 con 0x7f9ac4065800 2020-11-20T12:19:03.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.460+0000 7f9abaffd700 1 -- 172.21.15.116:0/4114315329 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9aac010070 con 0x7f9ac4065800 2020-11-20T12:19:03.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.460+0000 7f9abaffd700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9aa00557f0 0x7f9aa0057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:03.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.460+0000 7f9abaffd700 1 -- 172.21.15.116:0/4114315329 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6544+0+0 (secure 0 0 0) 0x7f9aac06db00 con 0x7f9ac4065800 2020-11-20T12:19:03.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.460+0000 7f9ac27fc700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9aa00557f0 0x7f9aa0057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:03.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.464+0000 7f9ac27fc700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9aa00557f0 0x7f9aa0057cb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f9ac40a6ce0 tx=0x7f9ab4006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:03.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.464+0000 7f9abaffd700 1 -- 172.21.15.116:0/4114315329 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9aac042fd0 con 0x7f9ac4065800 2020-11-20T12:19:03.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:03.668+0000 7f9ac9730700 1 -- 172.21.15.116:0/4114315329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f9ac40c3350 con 0x7f9ac4065800 2020-11-20T12:19:04.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.260+0000 7f9abaffd700 1 -- 172.21.15.116:0/4114315329 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9aac0437d0 con 0x7f9ac4065800 2020-11-20T12:19:04.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 -- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9aa00557f0 msgr2=0x7f9aa0057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:04.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9aa00557f0 0x7f9aa0057cb0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f9ac40a6ce0 tx=0x7f9ab4006df0).stop 2020-11-20T12:19:04.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 -- 172.21.15.116:0/4114315329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ac4065800 msgr2=0x7f9ac40c13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:04.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ac4065800 0x7f9ac40c13a0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f9aac001e70 tx=0x7f9aac007c50).stop 2020-11-20T12:19:04.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 -- 172.21.15.116:0/4114315329 shutdown_connections 2020-11-20T12:19:04.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9aa00557f0 0x7f9aa0057cb0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:04.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9ac4150cb0 0x7f9ac4164960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:04.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9ac4065800 0x7f9ac40c13a0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:04.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 --2- 172.21.15.116:0/4114315329 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9ac40c1f60 0x7f9ac40bf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:04.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 -- 172.21.15.116:0/4114315329 >> 172.21.15.116:0/4114315329 conn(0x7f9ac40c6480 msgr2=0x7f9ac41526b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:04.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 -- 172.21.15.116:0/4114315329 shutdown_connections 2020-11-20T12:19:04.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.264+0000 7f9ac9730700 1 -- 172.21.15.116:0/4114315329 wait complete. 2020-11-20T12:19:04.287 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-20T12:19:04.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.492+0000 7fb39623e700 1 Processor -- start 2020-11-20T12:19:04.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.492+0000 7fb39623e700 1 -- start start 2020-11-20T12:19:04.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb39623e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb390062020 0x7fb3900c3750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:04.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb39623e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3900c0f40 con 0x7fb390066120 2020-11-20T12:19:04.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb39623e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3900bc800 con 0x7fb3900c3370 2020-11-20T12:19:04.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb39623e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3900bcaa0 con 0x7fb390062020 2020-11-20T12:19:04.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38ffff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb390066120 0x7fb3900c18e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56922/0 (socket says 172.21.15.116:56922) 2020-11-20T12:19:04.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38ffff700 1 -- 172.21.15.116:0/2226324935 learned_addr learned my addr 172.21.15.116:0/2226324935 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:04.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38effd700 1 --2- 172.21.15.116:0/2226324935 >> v2:172.21.15.173:3300/0 conn(0x7fb390062020 0x7fb3900c3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:04.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 28285947 0 0) 0x7fb3900c0f40 con 0x7fb390066120 2020-11-20T12:19:04.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb37c003240 con 0x7fb390066120 2020-11-20T12:19:04.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2977709729 0 0) 0x7fb37c003240 con 0x7fb390066120 2020-11-20T12:19:04.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 >> v2:172.21.15.173:3300/0 conn(0x7fb390062020 msgr2=0x7fb3900c3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:04.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 --2- 172.21.15.116:0/2226324935 >> v2:172.21.15.173:3300/0 conn(0x7fb390062020 0x7fb3900c3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:04.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 >> v1:172.21.15.173:6789/0 conn(0x7fb3900c3370 legacy=0x7fb3900bbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:04.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3900c11a0 con 0x7fb390066120 2020-11-20T12:19:04.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fb388001860 con 0x7fb390066120 2020-11-20T12:19:04.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38effd700 1 --2- 172.21.15.116:0/2226324935 >> v2:172.21.15.173:3300/0 conn(0x7fb390062020 0x7fb3900c3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:04.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 >> v1:172.21.15.116:6789/0 conn(0x7fb390066120 legacy=0x7fb3900c18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:04.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 --2- 172.21.15.116:0/2226324935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb37c003730 0x7fb37c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:04.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 --2- 172.21.15.116:0/2226324935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb37c0044e0 0x7fb37c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:04.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 --2- 172.21.15.116:0/2226324935 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb37c004e50 0x7fb37c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:04.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900bcaa0 con 0x7fb37c004e50 2020-11-20T12:19:04.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c004210 con 0x7fb37c003730 2020-11-20T12:19:04.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.496+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37c00bc20 con 0x7fb37c0044e0 2020-11-20T12:19:04.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38ffff700 1 --2- 172.21.15.116:0/2226324935 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb37c004e50 0x7fb37c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:04.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38effd700 1 --2- 172.21.15.116:0/2226324935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb37c0044e0 0x7fb37c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:04.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38f7fe700 1 --2- 172.21.15.116:0/2226324935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb37c003730 0x7fb37c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:04.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38ffff700 1 -- 172.21.15.116:0/2226324935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb37c0044e0 msgr2=0x7fb37c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:04.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38ffff700 1 --2- 172.21.15.116:0/2226324935 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb37c0044e0 0x7fb37c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:04.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38ffff700 1 -- 172.21.15.116:0/2226324935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb37c003730 msgr2=0x7fb37c003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:04.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38ffff700 1 --2- 172.21.15.116:0/2226324935 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb37c003730 0x7fb37c003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:04.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38ffff700 1 -- 172.21.15.116:0/2226324935 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb37c003240 con 0x7fb37c004e50 2020-11-20T12:19:04.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38ffff700 1 -- 172.21.15.116:0/2226324935 reap_dead start 2020-11-20T12:19:04.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38ffff700 1 --2- 172.21.15.116:0/2226324935 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb37c004e50 0x7fb37c00b730 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fb38800c600 tx=0x7fb3900c1330).ready entity=mon.0 client_cookie=bac4ca873088ae70 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:04.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388005b80 con 0x7fb37c004e50 2020-11-20T12:19:04.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb388005ce0 con 0x7fb37c004e50 2020-11-20T12:19:04.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb38e7fc700 1 -- 172.21.15.116:0/2226324935 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388007320 con 0x7fb37c004e50 2020-11-20T12:19:04.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb39623e700 1 -- 172.21.15.116:0/2226324935 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb37c004e50 msgr2=0x7fb37c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:04.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb39623e700 1 --2- 172.21.15.116:0/2226324935 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb37c004e50 0x7fb37c00b730 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7fb38800c600 tx=0x7fb3900c1330).stop 2020-11-20T12:19:04.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb39623e700 1 -- 172.21.15.116:0/2226324935 shutdown_connections 2020-11-20T12:19:04.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb39623e700 1 --2- 172.21.15.116:0/2226324935 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb37c004e50 0x7fb37c00b730 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:04.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb39623e700 1 -- 172.21.15.116:0/2226324935 >> 172.21.15.116:0/2226324935 conn(0x7fb390060170 msgr2=0x7fb390060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:04.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb39623e700 1 -- 172.21.15.116:0/2226324935 shutdown_connections 2020-11-20T12:19:04.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.500+0000 7fb39623e700 1 -- 172.21.15.116:0/2226324935 wait complete. 2020-11-20T12:19:04.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.504+0000 7fb39623e700 1 Processor -- start 2020-11-20T12:19:04.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.504+0000 7fb39623e700 1 -- start start 2020-11-20T12:19:04.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.504+0000 7fb39623e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb37c004e50 0x7fb390162090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:04.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.504+0000 7fb39623e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3900bfcd0 0x7fb390157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:04.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.504+0000 7fb39623e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb39014f840 0x7fb390060550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:04.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.504+0000 7fb39623e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3900c0440 con 0x7fb39014f840 2020-11-20T12:19:04.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.504+0000 7fb39623e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3901569e0 con 0x7fb37c004e50 2020-11-20T12:19:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.504+0000 7fb39623e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb390156c80 con 0x7fb3900bfcd0 2020-11-20T12:19:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb38ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb39014f840 0x7fb390060550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb38ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb39014f840 0x7fb390060550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47500/0 (socket says 172.21.15.116:47500) 2020-11-20T12:19:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb38ffff700 1 -- 172.21.15.116:0/2669401418 learned_addr learned my addr 172.21.15.116:0/2669401418 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:04.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb38f7fe700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb37c004e50 0x7fb390162090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:04.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb38effd700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3900bfcd0 0x7fb390157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:04.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb38ffff700 1 -- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3900bfcd0 msgr2=0x7fb390157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:04.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb38ffff700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3900bfcd0 0x7fb390157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:04.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb38ffff700 1 -- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb37c004e50 msgr2=0x7fb390162090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb38ffff700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb37c004e50 0x7fb390162090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb38ffff700 1 -- 172.21.15.116:0/2669401418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb390156f20 con 0x7fb39014f840 2020-11-20T12:19:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb38ffff700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb39014f840 0x7fb390060550 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fb3900681b0 tx=0x7fb388015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:04.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb36f7fe700 1 -- 172.21.15.116:0/2669401418 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3880078a0 con 0x7fb39014f840 2020-11-20T12:19:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb36f7fe700 1 -- 172.21.15.116:0/2669401418 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3880148c0 con 0x7fb39014f840 2020-11-20T12:19:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb39623e700 1 -- 172.21.15.116:0/2669401418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3901571c0 con 0x7fb39014f840 2020-11-20T12:19:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb36f7fe700 1 -- 172.21.15.116:0/2669401418 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb388014c40 con 0x7fb39014f840 2020-11-20T12:19:04.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb39623e700 1 -- 172.21.15.116:0/2669401418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb390157b50 con 0x7fb39014f840 2020-11-20T12:19:04.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.508+0000 7fb39623e700 1 -- 172.21.15.116:0/2669401418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb39005d2a0 con 0x7fb39014f840 2020-11-20T12:19:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.512+0000 7fb36f7fe700 1 -- 172.21.15.116:0/2669401418 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb38801d070 con 0x7fb39014f840 2020-11-20T12:19:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.512+0000 7fb36f7fe700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3680557c0 0x7fb368057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.512+0000 7fb36f7fe700 1 -- 172.21.15.116:0/2669401418 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6544+0+0 (secure 0 0 0) 0x7fb38806fd40 con 0x7fb39014f840 2020-11-20T12:19:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.512+0000 7fb38f7fe700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3680557c0 0x7fb368057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:04.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.512+0000 7fb38f7fe700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3680557c0 0x7fb368057c80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fb380000bc0 tx=0x7fb380009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:04.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.516+0000 7fb36f7fe700 1 -- 172.21.15.116:0/2669401418 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb3880459d0 con 0x7fb39014f840 2020-11-20T12:19:04.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:04.684+0000 7fb39623e700 1 -- 172.21.15.116:0/2669401418 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fb390068760 con 0x7fb39014f840 2020-11-20T12:19:05.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.272+0000 7fb36f7fe700 1 -- 172.21.15.116:0/2669401418 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v61) v1 ==== 181+0+0 (secure 0 0 0) 0x7fb38801a0d0 con 0x7fb39014f840 2020-11-20T12:19:05.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 -- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3680557c0 msgr2=0x7fb368057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:05.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3680557c0 0x7fb368057c80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fb380000bc0 tx=0x7fb380009040).stop 2020-11-20T12:19:05.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 -- 172.21.15.116:0/2669401418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb39014f840 msgr2=0x7fb390060550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:05.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb39014f840 0x7fb390060550 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fb3900681b0 tx=0x7fb388015000).stop 2020-11-20T12:19:05.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 -- 172.21.15.116:0/2669401418 shutdown_connections 2020-11-20T12:19:05.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb3680557c0 0x7fb368057c80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb39014f840 0x7fb390060550 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb37c004e50 0x7fb390162090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 --2- 172.21.15.116:0/2669401418 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb3900bfcd0 0x7fb390157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 -- 172.21.15.116:0/2669401418 >> 172.21.15.116:0/2669401418 conn(0x7fb390060170 msgr2=0x7fb3900c4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:05.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 -- 172.21.15.116:0/2669401418 shutdown_connections 2020-11-20T12:19:05.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.276+0000 7fb39623e700 1 -- 172.21.15.116:0/2669401418 wait complete. 2020-11-20T12:19:05.278 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2020-11-20T12:19:05.300 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-20T12:19:05.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.484+0000 7f9869386700 1 Processor -- start 2020-11-20T12:19:05.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.484+0000 7f9869386700 1 -- start start 2020-11-20T12:19:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f9869386700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f98641553f0 0x7f9864151e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f9869386700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f98640aa5a0 0x7f98640aaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:05.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f9869386700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9864152d80 con 0x7f986414f2c0 2020-11-20T12:19:05.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f9869386700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98640aba60 con 0x7f98640aa5a0 2020-11-20T12:19:05.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f9869386700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98640acd60 con 0x7f98641553f0 2020-11-20T12:19:05.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f98637fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f98640aa5a0 0x7f98640aaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:05.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f9862ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f986414f2c0 0x7f98640a82e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56938/0 (socket says 172.21.15.116:56938) 2020-11-20T12:19:05.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f9862ffd700 1 -- 172.21.15.116:0/1852125292 learned_addr learned my addr 172.21.15.116:0/1852125292 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:05.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f98627fc700 1 --2- 172.21.15.116:0/1852125292 >> v2:172.21.15.173:3300/0 conn(0x7f98641553f0 0x7f9864151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:05.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f98637fe700 1 -- 172.21.15.116:0/1852125292 >> v2:172.21.15.173:3300/0 conn(0x7f98641553f0 msgr2=0x7f9864151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:05.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f98637fe700 1 --2- 172.21.15.116:0/1852125292 >> v2:172.21.15.173:3300/0 conn(0x7f98641553f0 0x7f9864151e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f98637fe700 1 -- 172.21.15.116:0/1852125292 >> v1:172.21.15.116:6789/0 conn(0x7f986414f2c0 legacy=0x7f98640a82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:05.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f98637fe700 1 -- 172.21.15.116:0/1852125292 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9864152310 con 0x7f98640aa5a0 2020-11-20T12:19:05.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f98637fe700 1 --2- 172.21.15.116:0/1852125292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98640aa5a0 0x7f98640aaa50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f98640a89f0 tx=0x7f985c0091d0).ready entity=mon.0 client_cookie=36bcedde80565943 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:05.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f9861ffb700 1 -- 172.21.15.116:0/1852125292 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c00bbc0 con 0x7f98640aa5a0 2020-11-20T12:19:05.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f9861ffb700 1 -- 172.21.15.116:0/1852125292 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c00bd20 con 0x7f98640aa5a0 2020-11-20T12:19:05.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.488+0000 7f9861ffb700 1 -- 172.21.15.116:0/1852125292 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c011610 con 0x7f98640aa5a0 2020-11-20T12:19:05.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.492+0000 7f9869386700 1 -- 172.21.15.116:0/1852125292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98640aa5a0 msgr2=0x7f98640aaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:05.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.492+0000 7f9869386700 1 --2- 172.21.15.116:0/1852125292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98640aa5a0 0x7f98640aaa50 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f98640a89f0 tx=0x7f985c0091d0).stop 2020-11-20T12:19:05.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.492+0000 7f9869386700 1 -- 172.21.15.116:0/1852125292 shutdown_connections 2020-11-20T12:19:05.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.492+0000 7f9869386700 1 --2- 172.21.15.116:0/1852125292 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98640aa5a0 0x7f98640aaa50 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.492+0000 7f9869386700 1 --2- 172.21.15.116:0/1852125292 >> v2:172.21.15.173:3300/0 conn(0x7f98641553f0 0x7f9864151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.492+0000 7f9869386700 1 -- 172.21.15.116:0/1852125292 >> 172.21.15.116:0/1852125292 conn(0x7f98640678a0 msgr2=0x7f9864067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:05.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.492+0000 7f9869386700 1 -- 172.21.15.116:0/1852125292 shutdown_connections 2020-11-20T12:19:05.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.492+0000 7f9869386700 1 -- 172.21.15.116:0/1852125292 wait complete. 2020-11-20T12:19:05.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.496+0000 7f9869386700 1 Processor -- start 2020-11-20T12:19:05.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.496+0000 7f9869386700 1 -- start start 2020-11-20T12:19:05.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.496+0000 7f9869386700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98640aa5a0 0x7f98641565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.496+0000 7f9869386700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f986414f2c0 0x7f9864157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.496+0000 7f9869386700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98641553f0 0x7f9864164470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.496+0000 7f9869386700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98640a1980 con 0x7f98641553f0 2020-11-20T12:19:05.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.496+0000 7f9869386700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9864164c60 con 0x7f98640aa5a0 2020-11-20T12:19:05.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.496+0000 7f9869386700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9864164f00 con 0x7f986414f2c0 2020-11-20T12:19:05.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.496+0000 7f98637fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98641553f0 0x7f9864164470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:05.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f98637fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98641553f0 0x7f9864164470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47514/0 (socket says 172.21.15.116:47514) 2020-11-20T12:19:05.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f98637fe700 1 -- 172.21.15.116:0/2509417722 learned_addr learned my addr 172.21.15.116:0/2509417722 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:05.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f9862ffd700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98640aa5a0 0x7f98641565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f98627fc700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f986414f2c0 0x7f9864157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f98637fe700 1 -- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f986414f2c0 msgr2=0x7f9864157c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f98637fe700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f986414f2c0 0x7f9864157c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f98637fe700 1 -- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98640aa5a0 msgr2=0x7f98641565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:05.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f98637fe700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98640aa5a0 0x7f98641565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f98637fe700 1 -- 172.21.15.116:0/2509417722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98641651a0 con 0x7f98641553f0 2020-11-20T12:19:05.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f98637fe700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98641553f0 0x7f9864164470 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f9864080e40 tx=0x7f985c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:05.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f984affd700 1 -- 172.21.15.116:0/2509417722 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c00bbc0 con 0x7f98641553f0 2020-11-20T12:19:05.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f984affd700 1 -- 172.21.15.116:0/2509417722 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985c00be90 con 0x7f98641553f0 2020-11-20T12:19:05.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f9869386700 1 -- 172.21.15.116:0/2509417722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98640a1be0 con 0x7f98641553f0 2020-11-20T12:19:05.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f984affd700 1 -- 172.21.15.116:0/2509417722 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985c010cf0 con 0x7f98641553f0 2020-11-20T12:19:05.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.500+0000 7f9869386700 1 -- 172.21.15.116:0/2509417722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9864164960 con 0x7f98641553f0 2020-11-20T12:19:05.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.504+0000 7f9869386700 1 -- 172.21.15.116:0/2509417722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98640a1da0 con 0x7f98641553f0 2020-11-20T12:19:05.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.504+0000 7f984affd700 1 -- 172.21.15.116:0/2509417722 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f985c024030 con 0x7f98641553f0 2020-11-20T12:19:05.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.504+0000 7f984affd700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98400557c0 0x7f9840057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:05.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.504+0000 7f984affd700 1 -- 172.21.15.116:0/2509417722 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6106+0+0 (secure 0 0 0) 0x7f985c070220 con 0x7f98641553f0 2020-11-20T12:19:05.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.504+0000 7f9862ffd700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98400557c0 0x7f9840057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:05.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.504+0000 7f9862ffd700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98400557c0 0x7f9840057c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f98640c8650 tx=0x7f9854007b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:05.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.508+0000 7f984affd700 1 -- 172.21.15.116:0/2509417722 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f985c0461d0 con 0x7f98641553f0 2020-11-20T12:19:05.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:05.668+0000 7f9869386700 1 -- 172.21.15.116:0/2509417722 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f986405cf30 con 0x7f98641553f0 2020-11-20T12:19:06.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.284+0000 7f984affd700 1 -- 172.21.15.116:0/2509417722 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v62) v1 ==== 169+0+0 (secure 0 0 0) 0x7f985c022020 con 0x7f98641553f0 2020-11-20T12:19:06.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 -- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98400557c0 msgr2=0x7f9840057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:06.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98400557c0 0x7f9840057c80 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f98640c8650 tx=0x7f9854007b80).stop 2020-11-20T12:19:06.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 -- 172.21.15.116:0/2509417722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98641553f0 msgr2=0x7f9864164470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:06.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98641553f0 0x7f9864164470 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7f9864080e40 tx=0x7f985c016000).stop 2020-11-20T12:19:06.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 -- 172.21.15.116:0/2509417722 shutdown_connections 2020-11-20T12:19:06.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98400557c0 0x7f9840057c80 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:06.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98641553f0 0x7f9864164470 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:06.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98640aa5a0 0x7f98641565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:06.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 --2- 172.21.15.116:0/2509417722 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f986414f2c0 0x7f9864157c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:06.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 -- 172.21.15.116:0/2509417722 >> 172.21.15.116:0/2509417722 conn(0x7f98640678a0 msgr2=0x7f98640a9820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:06.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 -- 172.21.15.116:0/2509417722 shutdown_connections 2020-11-20T12:19:06.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.288+0000 7f9869386700 1 -- 172.21.15.116:0/2509417722 wait complete. 2020-11-20T12:19:06.289 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2020-11-20T12:19:06.311 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm fsvolume_volid fsvolume_volid --yes-i-really-really-mean-it 2020-11-20T12:19:06.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.500+0000 7f117e6e1700 1 Processor -- start 2020-11-20T12:19:06.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.500+0000 7f117e6e1700 1 -- start start 2020-11-20T12:19:06.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117e6e1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f117814f270 0x7f11780aab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:06.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117e6e1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1178155db0 0x7f11780acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:06.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117e6e1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11780ab150 con 0x7f1178152ac0 2020-11-20T12:19:06.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117e6e1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1178156260 con 0x7f1178155db0 2020-11-20T12:19:06.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117e6e1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11781520e0 con 0x7f117814f270 2020-11-20T12:19:06.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117dee0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1178155db0 0x7f11780acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:06.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117dee0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1178155db0 0x7f11780acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47522/0 (socket says 172.21.15.116:47522) 2020-11-20T12:19:06.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117dee0700 1 -- 172.21.15.116:0/2055048885 learned_addr learned my addr 172.21.15.116:0/2055048885 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:06.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117cede700 1 --1- 172.21.15.116:0/2055048885 >> v1:172.21.15.116:6789/0 conn(0x7f1178152ac0 0x7f11780a9e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56954/0 (socket says 172.21.15.116:56954) 2020-11-20T12:19:06.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117d6df700 1 --2- 172.21.15.116:0/2055048885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f117814f270 0x7f11780aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:06.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117dee0700 1 -- 172.21.15.116:0/2055048885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f117814f270 msgr2=0x7f11780aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:06.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117dee0700 1 --2- 172.21.15.116:0/2055048885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f117814f270 0x7f11780aab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:06.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117dee0700 1 -- 172.21.15.116:0/2055048885 >> v1:172.21.15.116:6789/0 conn(0x7f1178152ac0 legacy=0x7f11780a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:06.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f1167fff700 1 -- 172.21.15.116:0/2055048885 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1694740755 0 0) 0x7f11780ab150 con 0x7f1178152ac0 2020-11-20T12:19:06.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117dee0700 1 -- 172.21.15.116:0/2055048885 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1178152380 con 0x7f1178155db0 2020-11-20T12:19:06.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.504+0000 7f117dee0700 1 --2- 172.21.15.116:0/2055048885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1178155db0 0x7f11780acd60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f1170009d50 tx=0x7f1170009290).ready entity=mon.0 client_cookie=64e0995297633911 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:06.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.508+0000 7f1167fff700 1 -- 172.21.15.116:0/2055048885 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117000bc80 con 0x7f1178155db0 2020-11-20T12:19:06.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.508+0000 7f1167fff700 1 -- 172.21.15.116:0/2055048885 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f117000bde0 con 0x7f1178155db0 2020-11-20T12:19:06.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.508+0000 7f1167fff700 1 -- 172.21.15.116:0/2055048885 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1170011610 con 0x7f1178155db0 2020-11-20T12:19:06.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.508+0000 7f117e6e1700 1 -- 172.21.15.116:0/2055048885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1178155db0 msgr2=0x7f11780acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:06.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.508+0000 7f117e6e1700 1 --2- 172.21.15.116:0/2055048885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1178155db0 0x7f11780acd60 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f1170009d50 tx=0x7f1170009290).stop 2020-11-20T12:19:06.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.508+0000 7f117e6e1700 1 -- 172.21.15.116:0/2055048885 shutdown_connections 2020-11-20T12:19:06.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.508+0000 7f117e6e1700 1 --2- 172.21.15.116:0/2055048885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1178155db0 0x7f11780acd60 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:06.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.508+0000 7f117e6e1700 1 --2- 172.21.15.116:0/2055048885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f117814f270 0x7f11780aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:06.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.508+0000 7f117e6e1700 1 -- 172.21.15.116:0/2055048885 >> 172.21.15.116:0/2055048885 conn(0x7f11780678a0 msgr2=0x7f1178067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.508+0000 7f117e6e1700 1 -- 172.21.15.116:0/2055048885 shutdown_connections 2020-11-20T12:19:06.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.508+0000 7f117e6e1700 1 -- 172.21.15.116:0/2055048885 wait complete. 2020-11-20T12:19:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117e6e1700 1 Processor -- start 2020-11-20T12:19:06.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117e6e1700 1 -- start start 2020-11-20T12:19:06.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117e6e1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f117814f270 0x7f1178153e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:06.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117e6e1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1178152ac0 0x7f11781619a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:06.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117e6e1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1178155db0 0x7f11780c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:06.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117e6e1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1178154360 con 0x7f1178155db0 2020-11-20T12:19:06.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117e6e1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1178161e90 con 0x7f117814f270 2020-11-20T12:19:06.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117e6e1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1178154580 con 0x7f1178152ac0 2020-11-20T12:19:06.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117dee0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1178155db0 0x7f11780c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:06.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117dee0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1178155db0 0x7f11780c04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47528/0 (socket says 172.21.15.116:47528) 2020-11-20T12:19:06.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117dee0700 1 -- 172.21.15.116:0/2613273027 learned_addr learned my addr 172.21.15.116:0/2613273027 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:06.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117d6df700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f117814f270 0x7f1178153e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:06.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.512+0000 7f117cede700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1178152ac0 0x7f11781619a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:06.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f117dee0700 1 -- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1178152ac0 msgr2=0x7f11781619a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:06.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f117dee0700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1178152ac0 0x7f11781619a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:06.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f117dee0700 1 -- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f117814f270 msgr2=0x7f1178153e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:06.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f117dee0700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f117814f270 0x7f1178153e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:06.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f117dee0700 1 -- 172.21.15.116:0/2613273027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1178154820 con 0x7f1178155db0 2020-11-20T12:19:06.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f117cede700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1178152ac0 0x7f11781619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:06.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f117d6df700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f117814f270 0x7f1178153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:06.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f117dee0700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1178155db0 0x7f11780c04d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f117000c010 tx=0x7f1170016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:06.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f11657fa700 1 -- 172.21.15.116:0/2613273027 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f117000b750 con 0x7f1178155db0 2020-11-20T12:19:06.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f11657fa700 1 -- 172.21.15.116:0/2613273027 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1170006c10 con 0x7f1178155db0 2020-11-20T12:19:06.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f117e6e1700 1 -- 172.21.15.116:0/2613273027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1178154ac0 con 0x7f1178155db0 2020-11-20T12:19:06.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f11657fa700 1 -- 172.21.15.116:0/2613273027 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1170010d90 con 0x7f1178155db0 2020-11-20T12:19:06.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f117e6e1700 1 -- 172.21.15.116:0/2613273027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1178155000 con 0x7f1178155db0 2020-11-20T12:19:06.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f117e6e1700 1 -- 172.21.15.116:0/2613273027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f117805cf30 con 0x7f1178155db0 2020-11-20T12:19:06.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.516+0000 7f11657fa700 1 -- 172.21.15.116:0/2613273027 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1170017070 con 0x7f1178155db0 2020-11-20T12:19:06.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.520+0000 7f11657fa700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11500557c0 0x7f1150057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:06.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.520+0000 7f11657fa700 1 -- 172.21.15.116:0/2613273027 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5724+0+0 (secure 0 0 0) 0x7f11700707c0 con 0x7f1178155db0 2020-11-20T12:19:06.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.520+0000 7f117d6df700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11500557c0 0x7f1150057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:06.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.520+0000 7f117d6df700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11500557c0 0x7f1150057c80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f117400d930 tx=0x7f1174008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:06.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.524+0000 7f11657fa700 1 -- 172.21.15.116:0/2613273027 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f11700467c0 con 0x7f1178155db0 2020-11-20T12:19:06.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:06.684+0000 7f117e6e1700 1 -- 172.21.15.116:0/2613273027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "fsvolume_volid", "pool2": "fsvolume_volid", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1178060f00 con 0x7f1178155db0 2020-11-20T12:19:07.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.296+0000 7f11657fa700 1 -- 172.21.15.116:0/2613273027 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "fsvolume_volid", "pool2": "fsvolume_volid", "yes_i_really_really_mean_it": true}]=0 pool 'fsvolume_volid' removed v63) v1 ==== 178+0+0 (secure 0 0 0) 0x7f1170015070 con 0x7f1178155db0 2020-11-20T12:19:07.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 -- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11500557c0 msgr2=0x7f1150057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:07.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11500557c0 0x7f1150057c80 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f117400d930 tx=0x7f1174008040).stop 2020-11-20T12:19:07.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 -- 172.21.15.116:0/2613273027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1178155db0 msgr2=0x7f11780c04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:07.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1178155db0 0x7f11780c04d0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f117000c010 tx=0x7f1170016000).stop 2020-11-20T12:19:07.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 -- 172.21.15.116:0/2613273027 shutdown_connections 2020-11-20T12:19:07.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f11500557c0 0x7f1150057c80 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:07.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1178155db0 0x7f11780c04d0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:07.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f117814f270 0x7f1178153e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:07.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 --2- 172.21.15.116:0/2613273027 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1178152ac0 0x7f11781619a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:07.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 -- 172.21.15.116:0/2613273027 >> 172.21.15.116:0/2613273027 conn(0x7f11780678a0 msgr2=0x7f1178061bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:07.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 -- 172.21.15.116:0/2613273027 shutdown_connections 2020-11-20T12:19:07.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.300+0000 7f117e6e1700 1 -- 172.21.15.116:0/2613273027 wait complete. 2020-11-20T12:19:07.302 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'fsvolume_volid' removed 2020-11-20T12:19:07.323 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T12:19:07.324 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-20T12:19:07.324 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-20T12:19:07.324 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:19:07.325 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-20T12:19:07.325 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-20T12:19:07.325 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-20T12:19:07.325 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:19:07.326 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T12:19:07.326 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-20T12:19:07.326 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-20T12:19:07.327 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:19:07.327 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-20T12:19:07.327 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-20T12:19:07.328 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-20T12:19:07.328 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:19:07.341 INFO:tasks.ceph.mds.a:Stopped 2020-11-20T12:19:07.341 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T12:19:07.345 INFO:tasks.ceph.mds.b:Stopped 2020-11-20T12:19:07.345 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T12:19:07.347 INFO:tasks.ceph.mds.c:Stopped 2020-11-20T12:19:07.347 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-20T12:19:07.350 INFO:tasks.ceph.mds.b:Started 2020-11-20T12:19:07.351 DEBUG:teuthology.parallel:result is None 2020-11-20T12:19:07.352 INFO:tasks.ceph.mds.d:Stopped 2020-11-20T12:19:07.352 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-20T12:19:07.356 INFO:tasks.ceph.mds.d:Started 2020-11-20T12:19:07.356 DEBUG:teuthology.parallel:result is None 2020-11-20T12:19:07.393 INFO:tasks.ceph.mds.a:Started 2020-11-20T12:19:07.394 DEBUG:teuthology.parallel:result is None 2020-11-20T12:19:07.397 INFO:tasks.ceph.mds.c:Started 2020-11-20T12:19:07.398 DEBUG:teuthology.parallel:result is None 2020-11-20T12:19:07.399 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-20T12:19:07.454 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2020-11-20T12:19:07.457 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2020-11-20T12:19:07.500 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:19:07.500 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:19:07.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.580+0000 7fa9b9521700 1 Processor -- start 2020-11-20T12:19:07.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.580+0000 7fa9b9521700 1 -- start start 2020-11-20T12:19:07.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b9521700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b4151990 0x7fa9b4066e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:07.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b9521700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa9b4151d70 0x7fa9b40bedc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:07.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b9521700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b4068670 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b9521700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b40683d0 con 0x7fa9b4151d70 2020-11-20T12:19:07.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b9521700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b4066480 con 0x7fa9b4151990 2020-11-20T12:19:07.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b37fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa9b40bf2b0 0x7fa9b40677e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:56998/0 (socket says 172.21.15.116:56998) 2020-11-20T12:19:07.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b37fe700 1 -- 172.21.15.116:0/306093024 learned_addr learned my addr 172.21.15.116:0/306093024 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:07.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9abfff700 1 --2- 172.21.15.116:0/306093024 >> v2:172.21.15.173:3300/0 conn(0x7fa9b4151d70 0x7fa9b40bedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:07.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b2ffd700 1 --2- 172.21.15.116:0/306093024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b4151990 0x7fa9b4066e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:07.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b27fc700 1 -- 172.21.15.116:0/306093024 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2452019358 0 0) 0x7fa9b4068670 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b27fc700 1 -- 172.21.15.116:0/306093024 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa994003240 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b2ffd700 1 -- 172.21.15.116:0/306093024 >> v2:172.21.15.173:3300/0 conn(0x7fa9b4151d70 msgr2=0x7fa9b40bedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b2ffd700 1 --2- 172.21.15.116:0/306093024 >> v2:172.21.15.173:3300/0 conn(0x7fa9b4151d70 0x7fa9b40bedc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b2ffd700 1 -- 172.21.15.116:0/306093024 >> v1:172.21.15.116:6789/0 conn(0x7fa9b40bf2b0 legacy=0x7fa9b40677e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b27fc700 1 -- 172.21.15.116:0/306093024 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1055642060 0 0) 0x7fa994003240 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b2ffd700 1 -- 172.21.15.116:0/306093024 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b4066720 con 0x7fa9b4151990 2020-11-20T12:19:07.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b2ffd700 1 --2- 172.21.15.116:0/306093024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b4151990 0x7fa9b4066e60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fa9a400b9a0 tx=0x7fa9a4006df0).ready entity=mon.2 client_cookie=64160120b0f1a3cd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:07.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.584+0000 7fa9b27fc700 1 -- 172.21.15.116:0/306093024 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4009680 con 0x7fa9b4151990 2020-11-20T12:19:07.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b27fc700 1 -- 172.21.15.116:0/306093024 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a40097e0 con 0x7fa9b4151990 2020-11-20T12:19:07.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b27fc700 1 -- 172.21.15.116:0/306093024 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4009b60 con 0x7fa9b4151990 2020-11-20T12:19:07.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b9521700 1 -- 172.21.15.116:0/306093024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b4151990 msgr2=0x7fa9b4066e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:07.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b9521700 1 --2- 172.21.15.116:0/306093024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b4151990 0x7fa9b4066e60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fa9a400b9a0 tx=0x7fa9a4006df0).stop 2020-11-20T12:19:07.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b9521700 1 -- 172.21.15.116:0/306093024 shutdown_connections 2020-11-20T12:19:07.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b9521700 1 --2- 172.21.15.116:0/306093024 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b4151990 0x7fa9b4066e60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:07.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b9521700 1 --2- 172.21.15.116:0/306093024 >> v2:172.21.15.173:3300/0 conn(0x7fa9b4151d70 0x7fa9b40bedc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:07.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b9521700 1 -- 172.21.15.116:0/306093024 >> 172.21.15.116:0/306093024 conn(0x7fa9b4060170 msgr2=0x7fa9b4060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:07.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b9521700 1 -- 172.21.15.116:0/306093024 shutdown_connections 2020-11-20T12:19:07.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b9521700 1 -- 172.21.15.116:0/306093024 wait complete. 2020-11-20T12:19:07.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b9521700 1 Processor -- start 2020-11-20T12:19:07.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.588+0000 7fa9b9521700 1 -- start start 2020-11-20T12:19:07.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b9521700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b40bf2b0 0x7fa9b4162560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:07.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b9521700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b4151990 0x7fa9b4164960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:07.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b9521700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9b4151d70 0x7fa9b4164e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:07.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b9521700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b4153140 con 0x7fa9b4151990 2020-11-20T12:19:07.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b9521700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b41533e0 con 0x7fa9b4151d70 2020-11-20T12:19:07.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b9521700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b4153680 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9abfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b4151990 0x7fa9b4164960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:07.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b2ffd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b40bf2b0 0x7fa9b4162560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:07.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9abfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b4151990 0x7fa9b4164960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47570/0 (socket says 172.21.15.116:47570) 2020-11-20T12:19:07.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9abfff700 1 -- 172.21.15.116:0/950685711 learned_addr learned my addr 172.21.15.116:0/950685711 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:07.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b37fe700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9b4151d70 0x7fa9b4164e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:07.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b2ffd700 1 -- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9b4151d70 msgr2=0x7fa9b4164e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b2ffd700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9b4151d70 0x7fa9b4164e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b2ffd700 1 -- 172.21.15.116:0/950685711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b4151990 msgr2=0x7fa9b4164960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b2ffd700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b4151990 0x7fa9b4164960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:07.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b2ffd700 1 -- 172.21.15.116:0/950685711 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b4153920 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9abfff700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b4151990 0x7fa9b4164960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b2ffd700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b40bf2b0 0x7fa9b4162560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fa9a4012000 tx=0x7fa9a4015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9aaffd700 1 -- 172.21.15.116:0/950685711 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a4011bc0 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9aaffd700 1 -- 172.21.15.116:0/950685711 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a4011d20 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b9521700 1 -- 172.21.15.116:0/950685711 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b4153bc0 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9aaffd700 1 -- 172.21.15.116:0/950685711 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a401e690 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.592+0000 7fa9b9521700 1 -- 172.21.15.116:0/950685711 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b4162a50 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.596+0000 7fa9b9521700 1 -- 172.21.15.116:0/950685711 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b4154060 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.596+0000 7fa9aaffd700 1 -- 172.21.15.116:0/950685711 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa9a4025030 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.596+0000 7fa9aaffd700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 0x7fa990057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.596+0000 7fa9aaffd700 1 -- 172.21.15.116:0/950685711 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5339+0+0 (secure 0 0 0) 0x7fa9a406fa20 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.600+0000 7fa9abfff700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 0x7fa990057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:07.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.600+0000 7fa9abfff700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 0x7fa990057cb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fa9b40c2fa0 tx=0x7fa99c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:07.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.604+0000 7fa9aaffd700 1 -- 172.21.15.116:0/950685711 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa9a4045b10 con 0x7fa9b40bf2b0 2020-11-20T12:19:07.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:07.780+0000 7fa9b9521700 1 -- 172.21.15.116:0/950685711 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fa9b405d2a0 con 0x7fa9b40bf2b0 2020-11-20T12:19:08.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.328+0000 7fa9aaffd700 1 -- 172.21.15.116:0/950685711 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) v1 ==== 97+0+0 (secure 0 0 0) 0x7fa9a4040080 con 0x7fa9b40bf2b0 2020-11-20T12:19:08.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 -- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 msgr2=0x7fa990057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:08.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 0x7fa990057cb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7fa9b40c2fa0 tx=0x7fa99c006df0).stop 2020-11-20T12:19:08.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 -- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b40bf2b0 msgr2=0x7fa9b4162560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:08.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b40bf2b0 0x7fa9b4162560 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fa9a4012000 tx=0x7fa9a4015000).stop 2020-11-20T12:19:08.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 -- 172.21.15.116:0/950685711 shutdown_connections 2020-11-20T12:19:08.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 0x7fa990057cb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9b4151d70 0x7fa9b4164e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b40bf2b0 0x7fa9b4162560 secure :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0x7fa9a4012000 tx=0x7fa9a4015000).stop 2020-11-20T12:19:08.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 --2- 172.21.15.116:0/950685711 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b4151990 0x7fa9b4164960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 -- 172.21.15.116:0/950685711 >> 172.21.15.116:0/950685711 conn(0x7fa9b4060170 msgr2=0x7fa9b4155eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:08.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 -- 172.21.15.116:0/950685711 shutdown_connections 2020-11-20T12:19:08.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.336+0000 7fa9b9521700 1 -- 172.21.15.116:0/950685711 wait complete. 2020-11-20T12:19:08.337 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2020-11-20T12:19:08.358 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-20T12:19:08.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.528+0000 7f909581e700 1 Processor -- start 2020-11-20T12:19:08.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.528+0000 7f909581e700 1 -- start start 2020-11-20T12:19:08.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.528+0000 7f909581e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f90901515d0 0x7f90901519b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:08.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.528+0000 7f909581e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90900bfcf0 con 0x7f9090065fc0 2020-11-20T12:19:08.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.528+0000 7f909581e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90900607c0 con 0x7f9090066420 2020-11-20T12:19:08.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.528+0000 7f909581e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90900605d0 con 0x7f90901515d0 2020-11-20T12:19:08.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.528+0000 7f908e7fc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9090065fc0 0x7f9090065310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57012/0 (socket says 172.21.15.116:57012) 2020-11-20T12:19:08.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.528+0000 7f908e7fc700 1 -- 172.21.15.116:0/1428522389 learned_addr learned my addr 172.21.15.116:0/1428522389 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:08.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908effd700 1 --2- 172.21.15.116:0/1428522389 >> v2:172.21.15.173:3300/0 conn(0x7f90901515d0 0x7f90901519b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533517878 0 0) 0x7f90900bfcf0 con 0x7f9090065fc0 2020-11-20T12:19:08.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f907c003240 con 0x7f9090065fc0 2020-11-20T12:19:08.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4097683921 0 0) 0x7f907c003240 con 0x7f9090065fc0 2020-11-20T12:19:08.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 >> v2:172.21.15.173:3300/0 conn(0x7f90901515d0 msgr2=0x7f90901519b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:08.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 --2- 172.21.15.116:0/1428522389 >> v2:172.21.15.173:3300/0 conn(0x7f90901515d0 0x7f90901519b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 >> v1:172.21.15.173:6789/0 conn(0x7f9090066420 legacy=0x7f90900a88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:08.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9090060a60 con 0x7f9090065fc0 2020-11-20T12:19:08.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f90840014e0 con 0x7f9090065fc0 2020-11-20T12:19:08.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 >> v1:172.21.15.116:6789/0 conn(0x7f9090065fc0 legacy=0x7f9090065310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:08.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f907c003730 0x7f907c003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:08.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907c0044e0 0x7f907c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:08.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f907c004e50 0x7f907c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:08.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c005330 con 0x7f907c003730 2020-11-20T12:19:08.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c00bc20 con 0x7f907c0044e0 2020-11-20T12:19:08.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907c00bd90 con 0x7f907c004e50 2020-11-20T12:19:08.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908effd700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f907c003730 0x7f907c003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908e7fc700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907c0044e0 0x7f907c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908ffff700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f907c004e50 0x7f907c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908effd700 1 -- 172.21.15.116:0/1428522389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f907c004e50 msgr2=0x7f907c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:08.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908effd700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f907c004e50 0x7f907c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908effd700 1 -- 172.21.15.116:0/1428522389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907c0044e0 msgr2=0x7f907c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:08.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908effd700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907c0044e0 0x7f907c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908effd700 1 -- 172.21.15.116:0/1428522389 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c003240 con 0x7f907c003730 2020-11-20T12:19:08.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908ffff700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f907c004e50 0x7f907c00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:08.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908ffff700 1 -- 172.21.15.116:0/1428522389 reap_dead start 2020-11-20T12:19:08.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908effd700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f907c003730 0x7f907c003b50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f908000bc00 tx=0x7f9080006df0).ready entity=mon.0 client_cookie=d66d3a7fe085d570 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:08.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908000b310 con 0x7f907c003730 2020-11-20T12:19:08.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908000ea90 con 0x7f907c003730 2020-11-20T12:19:08.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f908dffb700 1 -- 172.21.15.116:0/1428522389 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908000ee10 con 0x7f907c003730 2020-11-20T12:19:08.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f909581e700 1 -- 172.21.15.116:0/1428522389 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f907c003730 msgr2=0x7f907c003b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:08.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f909581e700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f907c003730 0x7f907c003b50 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f908000bc00 tx=0x7f9080006df0).stop 2020-11-20T12:19:08.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f909581e700 1 -- 172.21.15.116:0/1428522389 shutdown_connections 2020-11-20T12:19:08.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.532+0000 7f909581e700 1 --2- 172.21.15.116:0/1428522389 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f907c003730 0x7f907c003b50 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f909581e700 1 -- 172.21.15.116:0/1428522389 >> 172.21.15.116:0/1428522389 conn(0x7f90900ca850 msgr2=0x7f90900c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:08.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f909581e700 1 -- 172.21.15.116:0/1428522389 shutdown_connections 2020-11-20T12:19:08.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f909581e700 1 -- 172.21.15.116:0/1428522389 wait complete. 2020-11-20T12:19:08.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f909581e700 1 Processor -- start 2020-11-20T12:19:08.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f909581e700 1 -- start start 2020-11-20T12:19:08.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f909581e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907c003730 0x7f90901646d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:08.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f909581e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90900c1ab0 0x7f90900c2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:08.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f909581e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f909014cb20 0x7f9090063d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:08.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f909581e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9090153ef0 con 0x7f909014cb20 2020-11-20T12:19:08.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f909581e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9090164bc0 con 0x7f907c003730 2020-11-20T12:19:08.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f909581e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90901655c0 con 0x7f90900c1ab0 2020-11-20T12:19:08.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.536+0000 7f908ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f909014cb20 0x7f9090063d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f908ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f909014cb20 0x7f9090063d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47592/0 (socket says 172.21.15.116:47592) 2020-11-20T12:19:08.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f908ffff700 1 -- 172.21.15.116:0/2679110981 learned_addr learned my addr 172.21.15.116:0/2679110981 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:08.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f908effd700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907c003730 0x7f90901646d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f908e7fc700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90900c1ab0 0x7f90900c2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f908ffff700 1 -- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90900c1ab0 msgr2=0x7f90900c2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:08.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f908ffff700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90900c1ab0 0x7f90900c2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f908ffff700 1 -- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907c003730 msgr2=0x7f90901646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:08.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f908ffff700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907c003730 0x7f90901646d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f908ffff700 1 -- 172.21.15.116:0/2679110981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9090164de0 con 0x7f909014cb20 2020-11-20T12:19:08.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f908ffff700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f909014cb20 0x7f9090063d10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f90900c6780 tx=0x7f9088009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:08.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f9076ffd700 1 -- 172.21.15.116:0/2679110981 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908800c750 con 0x7f909014cb20 2020-11-20T12:19:08.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f9076ffd700 1 -- 172.21.15.116:0/2679110981 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f908800c8b0 con 0x7f909014cb20 2020-11-20T12:19:08.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f909581e700 1 -- 172.21.15.116:0/2679110981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9090165080 con 0x7f909014cb20 2020-11-20T12:19:08.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f9076ffd700 1 -- 172.21.15.116:0/2679110981 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908800cc30 con 0x7f909014cb20 2020-11-20T12:19:08.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f909581e700 1 -- 172.21.15.116:0/2679110981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90901540e0 con 0x7f909014cb20 2020-11-20T12:19:08.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.540+0000 7f909581e700 1 -- 172.21.15.116:0/2679110981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90901542a0 con 0x7f909014cb20 2020-11-20T12:19:08.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.544+0000 7f9076ffd700 1 -- 172.21.15.116:0/2679110981 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f908801b030 con 0x7f909014cb20 2020-11-20T12:19:08.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.544+0000 7f9076ffd700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f906c0557c0 0x7f906c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.544+0000 7f9076ffd700 1 -- 172.21.15.116:0/2679110981 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f908806dfe0 con 0x7f909014cb20 2020-11-20T12:19:08.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.544+0000 7f908effd700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f906c0557c0 0x7f906c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.544+0000 7f908effd700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f906c0557c0 0x7f906c057c80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f9080006d00 tx=0x7f9080019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:08.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.548+0000 7f9076ffd700 1 -- 172.21.15.116:0/2679110981 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9088044260 con 0x7f909014cb20 2020-11-20T12:19:08.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.764+0000 7f909581e700 1 -- 172.21.15.116:0/2679110981 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9090069950 con 0x7f909014cb20 2020-11-20T12:19:08.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.764+0000 7f9076ffd700 1 -- 172.21.15.116:0/2679110981 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5399 (secure 0 0 0) 0x7f9088024530 con 0x7f909014cb20 2020-11-20T12:19:08.763 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:08.764 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:19:08.764 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2020-11-20T12:19:08.764 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.764 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2020-11-20T12:19:08.764 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw==", 2020-11-20T12:19:08.765 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.765 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:08.765 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.765 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.766 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.766 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.766 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.766 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.766 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2020-11-20T12:19:08.767 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ==", 2020-11-20T12:19:08.767 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.767 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:08.767 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.767 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.768 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.768 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.768 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.768 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.768 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2020-11-20T12:19:08.769 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag==", 2020-11-20T12:19:08.769 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.769 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:08.769 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.769 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.770 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.770 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.770 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.770 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.770 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2020-11-20T12:19:08.771 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ==", 2020-11-20T12:19:08.771 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.771 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:08.771 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.771 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.772 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.772 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.772 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.772 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.772 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2020-11-20T12:19:08.773 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ==", 2020-11-20T12:19:08.773 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.773 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.773 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.773 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.774 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.774 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.774 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.774 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2020-11-20T12:19:08.774 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q==", 2020-11-20T12:19:08.775 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.775 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.775 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.775 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.775 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.776 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.776 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.776 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2020-11-20T12:19:08.776 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw==", 2020-11-20T12:19:08.776 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.777 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.777 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.777 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.777 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.778 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.778 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.778 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2020-11-20T12:19:08.778 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w==", 2020-11-20T12:19:08.778 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.779 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.779 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.779 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.779 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.779 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.780 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.780 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2020-11-20T12:19:08.780 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA==", 2020-11-20T12:19:08.780 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.780 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.780 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.781 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.781 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.781 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.781 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.782 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2020-11-20T12:19:08.782 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ==", 2020-11-20T12:19:08.782 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.782 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.782 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.783 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.783 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.783 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.783 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.783 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2020-11-20T12:19:08.784 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg==", 2020-11-20T12:19:08.784 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.784 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.784 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.784 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.785 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.785 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.785 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.785 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2020-11-20T12:19:08.785 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg==", 2020-11-20T12:19:08.786 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.786 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.786 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.786 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.786 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.787 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.787 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.787 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2020-11-20T12:19:08.787 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA==", 2020-11-20T12:19:08.787 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.788 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:08.788 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2020-11-20T12:19:08.788 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-20T12:19:08.788 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.788 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.789 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.789 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2020-11-20T12:19:08.789 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow==", 2020-11-20T12:19:08.789 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.789 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:08.790 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:19:08.790 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:19:08.790 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:19:08.790 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.790 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.791 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.791 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2020-11-20T12:19:08.791 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ==", 2020-11-20T12:19:08.791 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.791 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:08.792 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:19:08.792 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:19:08.792 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:19:08.792 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.792 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.793 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.793 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2020-11-20T12:19:08.793 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ==", 2020-11-20T12:19:08.793 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.794 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:08.794 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:19:08.794 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:19:08.794 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:19:08.794 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.795 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.795 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.795 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2020-11-20T12:19:08.795 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBDs7dfh+KKOBAAZvcPXWly9/mxRE7NWuR85A==", 2020-11-20T12:19:08.795 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.796 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:19:08.796 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:08.796 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.796 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.796 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.796 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.797 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.797 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.manila", 2020-11-20T12:19:08.797 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQAmtLdfoY10HRAAsLJqwS8fhRHZJFZ2QO74lw==", 2020-11-20T12:19:08.797 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.798 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:19:08.798 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:08.798 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw" 2020-11-20T12:19:08.798 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.798 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.799 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.799 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2020-11-20T12:19:08.799 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A==", 2020-11-20T12:19:08.799 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.799 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:19:08.800 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:19:08.800 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.800 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.800 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:08.800 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:08.800 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2020-11-20T12:19:08.801 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA==", 2020-11-20T12:19:08.801 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:08.801 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:19:08.801 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:19:08.802 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:08.802 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.802 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:08.802 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:19:08.802 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:19:08.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 -- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f906c0557c0 msgr2=0x7f906c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:08.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f906c0557c0 0x7f906c057c80 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f9080006d00 tx=0x7f9080019000).stop 2020-11-20T12:19:08.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 -- 172.21.15.116:0/2679110981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f909014cb20 msgr2=0x7f9090063d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:08.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f909014cb20 0x7f9090063d10 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f90900c6780 tx=0x7f9088009040).stop 2020-11-20T12:19:08.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 -- 172.21.15.116:0/2679110981 shutdown_connections 2020-11-20T12:19:08.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f906c0557c0 0x7f906c057c80 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f909014cb20 0x7f9090063d10 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f907c003730 0x7f90901646d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 --2- 172.21.15.116:0/2679110981 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90900c1ab0 0x7f90900c2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:08.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 -- 172.21.15.116:0/2679110981 >> 172.21.15.116:0/2679110981 conn(0x7f90900ca850 msgr2=0x7f90900c4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:08.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 -- 172.21.15.116:0/2679110981 shutdown_connections 2020-11-20T12:19:08.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.768+0000 7f909581e700 1 -- 172.21.15.116:0/2679110981 wait complete. 2020-11-20T12:19:08.807 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.manila 2020-11-20T12:19:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.996+0000 7f580abd3700 1 Processor -- start 2020-11-20T12:19:08.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:08.996+0000 7f580abd3700 1 -- start start 2020-11-20T12:19:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f580abd3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5804152390 0x7f58041527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f580abd3700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58040a60a0 con 0x7f580414e530 2020-11-20T12:19:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f580abd3700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58040a4ff0 con 0x7f580414e910 2020-11-20T12:19:08.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f580abd3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5804152ff0 con 0x7f5804152390 2020-11-20T12:19:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f580896f700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f580414e530 0x7f58040a5990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57030/0 (socket says 172.21.15.116:57030) 2020-11-20T12:19:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f580896f700 1 -- 172.21.15.116:0/892865679 learned_addr learned my addr 172.21.15.116:0/892865679 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f5809170700 1 --2- 172.21.15.116:0/892865679 >> v2:172.21.15.116:3300/0 conn(0x7f5804152390 0x7f58041527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:08.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f5809170700 1 -- 172.21.15.116:0/892865679 >> v1:172.21.15.173:6789/0 conn(0x7f580414e910 legacy=0x7f58040a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f5809170700 1 -- 172.21.15.116:0/892865679 >> v1:172.21.15.116:6789/0 conn(0x7f580414e530 legacy=0x7f58040a5990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f5809170700 1 -- 172.21.15.116:0/892865679 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58040a5250 con 0x7f5804152390 2020-11-20T12:19:09.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f5809170700 1 --2- 172.21.15.116:0/892865679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5804152390 0x7f58041527e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f57fc009c90 tx=0x7f57fc0091d0).ready entity=mon.0 client_cookie=16ada184e7d8185c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:09.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f58037fe700 1 -- 172.21.15.116:0/892865679 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc00bbc0 con 0x7f5804152390 2020-11-20T12:19:09.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f58037fe700 1 -- 172.21.15.116:0/892865679 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57fc00bd20 con 0x7f5804152390 2020-11-20T12:19:09.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f58037fe700 1 -- 172.21.15.116:0/892865679 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57fc011610 con 0x7f5804152390 2020-11-20T12:19:09.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f580abd3700 1 -- 172.21.15.116:0/892865679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5804152390 msgr2=0x7f58041527e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f580abd3700 1 --2- 172.21.15.116:0/892865679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5804152390 0x7f58041527e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f57fc009c90 tx=0x7f57fc0091d0).stop 2020-11-20T12:19:09.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f580abd3700 1 -- 172.21.15.116:0/892865679 shutdown_connections 2020-11-20T12:19:09.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f580abd3700 1 --2- 172.21.15.116:0/892865679 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5804152390 0x7f58041527e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.000+0000 7f580abd3700 1 -- 172.21.15.116:0/892865679 >> 172.21.15.116:0/892865679 conn(0x7f5804060170 msgr2=0x7f5804060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:09.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.004+0000 7f580abd3700 1 -- 172.21.15.116:0/892865679 shutdown_connections 2020-11-20T12:19:09.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.004+0000 7f580abd3700 1 -- 172.21.15.116:0/892865679 wait complete. 2020-11-20T12:19:09.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.004+0000 7f580abd3700 1 Processor -- start 2020-11-20T12:19:09.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.004+0000 7f580abd3700 1 -- start start 2020-11-20T12:19:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f580abd3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f580414e530 0x7f58040a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f580abd3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f580414e910 0x7f58041648f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f580abd3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5804152390 0x7f5804150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f580abd3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5804150090 con 0x7f580414e910 2020-11-20T12:19:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f580abd3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5804150330 con 0x7f5804152390 2020-11-20T12:19:09.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f580abd3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58041505d0 con 0x7f580414e530 2020-11-20T12:19:09.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5803fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f580414e910 0x7f58041648f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5803fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f580414e910 0x7f58041648f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47604/0 (socket says 172.21.15.116:47604) 2020-11-20T12:19:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5803fff700 1 -- 172.21.15.116:0/2796188623 learned_addr learned my addr 172.21.15.116:0/2796188623 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f580896f700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f580414e530 0x7f58040a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5809170700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5804152390 0x7f5804150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5803fff700 1 -- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f580414e530 msgr2=0x7f58040a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5803fff700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f580414e530 0x7f58040a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5803fff700 1 -- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5804152390 msgr2=0x7f5804150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5803fff700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5804152390 0x7f5804150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5803fff700 1 -- 172.21.15.116:0/2796188623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5804150870 con 0x7f580414e910 2020-11-20T12:19:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5809170700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5804152390 0x7f5804150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5803fff700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f580414e910 0x7f58041648f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f57f000c710 tx=0x7f57f0007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5800ff9700 1 -- 172.21.15.116:0/2796188623 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f000e560 con 0x7f580414e910 2020-11-20T12:19:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5800ff9700 1 -- 172.21.15.116:0/2796188623 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f000e6c0 con 0x7f580414e910 2020-11-20T12:19:09.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f580abd3700 1 -- 172.21.15.116:0/2796188623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5804150b10 con 0x7f580414e910 2020-11-20T12:19:09.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f5800ff9700 1 -- 172.21.15.116:0/2796188623 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f000ea40 con 0x7f580414e910 2020-11-20T12:19:09.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.008+0000 7f580abd3700 1 -- 172.21.15.116:0/2796188623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58040a3220 con 0x7f580414e910 2020-11-20T12:19:09.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.012+0000 7f580abd3700 1 -- 172.21.15.116:0/2796188623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5804150ca0 con 0x7f580414e910 2020-11-20T12:19:09.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.012+0000 7f5800ff9700 1 -- 172.21.15.116:0/2796188623 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f57f0019030 con 0x7f580414e910 2020-11-20T12:19:09.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.012+0000 7f5800ff9700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57dc0557c0 0x7f57dc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.012+0000 7f5800ff9700 1 -- 172.21.15.116:0/2796188623 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f57f006d960 con 0x7f580414e910 2020-11-20T12:19:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.012+0000 7f580896f700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57dc0557c0 0x7f57dc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.016+0000 7f580896f700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57dc0557c0 0x7f57dc057c80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f5804091e80 tx=0x7f57f8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:09.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.020+0000 7f5800ff9700 1 -- 172.21.15.116:0/2796188623 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f57f0043b30 con 0x7f580414e910 2020-11-20T12:19:09.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.232+0000 7f580abd3700 1 -- 172.21.15.116:0/2796188623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.manila"} v 0) v1 -- 0x7f580405d2a0 con 0x7f580414e910 2020-11-20T12:19:09.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.240+0000 7f5800ff9700 1 -- 172.21.15.116:0/2796188623 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.manila"}]=0 updated v18) v1 ==== 90+0+0 (secure 0 0 0) 0x7f57f0017070 con 0x7f580414e910 2020-11-20T12:19:09.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 -- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57dc0557c0 msgr2=0x7f57dc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57dc0557c0 0x7f57dc057c80 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f5804091e80 tx=0x7f57f8009040).stop 2020-11-20T12:19:09.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 -- 172.21.15.116:0/2796188623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f580414e910 msgr2=0x7f58041648f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f580414e910 0x7f58041648f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f57f000c710 tx=0x7f57f0007c10).stop 2020-11-20T12:19:09.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 -- 172.21.15.116:0/2796188623 shutdown_connections 2020-11-20T12:19:09.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57dc0557c0 0x7f57dc057c80 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5804152390 0x7f5804150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f580414e530 0x7f58040a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 --2- 172.21.15.116:0/2796188623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f580414e910 0x7f58041648f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 -- 172.21.15.116:0/2796188623 >> 172.21.15.116:0/2796188623 conn(0x7f5804060170 msgr2=0x7f58040624d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 -- 172.21.15.116:0/2796188623 shutdown_connections 2020-11-20T12:19:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.244+0000 7f580abd3700 1 -- 172.21.15.116:0/2796188623 wait complete. 2020-11-20T12:19:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:19:09.268 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:09.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.448+0000 7f122f073700 1 Processor -- start 2020-11-20T12:19:09.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.448+0000 7f122f073700 1 -- start start 2020-11-20T12:19:09.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.452+0000 7f122f073700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1228151910 0x7f1228061e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.452+0000 7f122f073700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1228151cf0 0x7f12280a5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.452+0000 7f122f073700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12280a6130 con 0x7f1228068200 2020-11-20T12:19:09.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.452+0000 7f122f073700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12280a5f10 con 0x7f1228151cf0 2020-11-20T12:19:09.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.452+0000 7f122f073700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1228061420 con 0x7f1228151910 2020-11-20T12:19:09.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.452+0000 7f1227fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f1228068200 0x7f1228067560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57044/0 (socket says 172.21.15.116:57044) 2020-11-20T12:19:09.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.452+0000 7f1227fff700 1 -- 172.21.15.116:0/1371569666 learned_addr learned my addr 172.21.15.116:0/1371569666 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:09.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.452+0000 7f122d610700 1 --2- 172.21.15.116:0/1371569666 >> v2:172.21.15.116:3300/0 conn(0x7f1228151cf0 0x7f12280a5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.452+0000 7f122ce0f700 1 --2- 172.21.15.116:0/1371569666 >> v2:172.21.15.173:3300/0 conn(0x7f1228151910 0x7f1228061e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f12277fe700 1 -- 172.21.15.116:0/1371569666 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3422719573 0 0) 0x7f12280a6130 con 0x7f1228068200 2020-11-20T12:19:09.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f12277fe700 1 -- 172.21.15.116:0/1371569666 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f120c003240 con 0x7f1228068200 2020-11-20T12:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122d610700 1 -- 172.21.15.116:0/1371569666 >> v2:172.21.15.173:3300/0 conn(0x7f1228151910 msgr2=0x7f1228061e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122d610700 1 --2- 172.21.15.116:0/1371569666 >> v2:172.21.15.173:3300/0 conn(0x7f1228151910 0x7f1228061e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122d610700 1 -- 172.21.15.116:0/1371569666 >> v1:172.21.15.116:6789/0 conn(0x7f1228068200 legacy=0x7f1228067560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122d610700 1 -- 172.21.15.116:0/1371569666 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12280616c0 con 0x7f1228151cf0 2020-11-20T12:19:09.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122d610700 1 --2- 172.21.15.116:0/1371569666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1228151cf0 0x7f12280a5a20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f1220009c90 tx=0x7f12200091d0).ready entity=mon.0 client_cookie=3f27957e82f1b625 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:09.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f12277fe700 1 -- 172.21.15.116:0/1371569666 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122000b980 con 0x7f1228151cf0 2020-11-20T12:19:09.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f12277fe700 1 -- 172.21.15.116:0/1371569666 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f122000bae0 con 0x7f1228151cf0 2020-11-20T12:19:09.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f12277fe700 1 -- 172.21.15.116:0/1371569666 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f122000be60 con 0x7f1228151cf0 2020-11-20T12:19:09.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122f073700 1 -- 172.21.15.116:0/1371569666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1228151cf0 msgr2=0x7f12280a5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122f073700 1 --2- 172.21.15.116:0/1371569666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1228151cf0 0x7f12280a5a20 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f1220009c90 tx=0x7f12200091d0).stop 2020-11-20T12:19:09.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122f073700 1 -- 172.21.15.116:0/1371569666 shutdown_connections 2020-11-20T12:19:09.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122f073700 1 --2- 172.21.15.116:0/1371569666 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1228151cf0 0x7f12280a5a20 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122f073700 1 --2- 172.21.15.116:0/1371569666 >> v2:172.21.15.173:3300/0 conn(0x7f1228151910 0x7f1228061e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122f073700 1 -- 172.21.15.116:0/1371569666 >> 172.21.15.116:0/1371569666 conn(0x7f12280c6030 msgr2=0x7f12280652f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:09.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122f073700 1 -- 172.21.15.116:0/1371569666 shutdown_connections 2020-11-20T12:19:09.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.456+0000 7f122f073700 1 -- 172.21.15.116:0/1371569666 wait complete. 2020-11-20T12:19:09.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.460+0000 7f122f073700 1 Processor -- start 2020-11-20T12:19:09.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.460+0000 7f122f073700 1 -- start start 2020-11-20T12:19:09.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f122f073700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1228068200 0x7f12281644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f122f073700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1228151910 0x7f1228154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f122f073700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1228151cf0 0x7f1228153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f122f073700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1228164cd0 con 0x7f1228151910 2020-11-20T12:19:09.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f122f073700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1228164f70 con 0x7f1228068200 2020-11-20T12:19:09.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f122f073700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1228165210 con 0x7f1228151cf0 2020-11-20T12:19:09.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1227fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1228151910 0x7f1228154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1227fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1228151910 0x7f1228154a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47618/0 (socket says 172.21.15.116:47618) 2020-11-20T12:19:09.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1227fff700 1 -- 172.21.15.116:0/3892077035 learned_addr learned my addr 172.21.15.116:0/3892077035 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:09.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f122ce0f700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1228068200 0x7f12281644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f122d610700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1228151cf0 0x7f1228153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1227fff700 1 -- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1228151cf0 msgr2=0x7f1228153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1227fff700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1228151cf0 0x7f1228153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1227fff700 1 -- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1228068200 msgr2=0x7f12281644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1227fff700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1228068200 0x7f12281644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1227fff700 1 -- 172.21.15.116:0/3892077035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122809ff50 con 0x7f1228151910 2020-11-20T12:19:09.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f122d610700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1228151cf0 0x7f1228153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:09.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1227fff700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1228151910 0x7f1228154a40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f121400c710 tx=0x7f1214007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:09.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1224ff9700 1 -- 172.21.15.116:0/3892077035 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12140072e0 con 0x7f1228151910 2020-11-20T12:19:09.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1224ff9700 1 -- 172.21.15.116:0/3892077035 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1214007440 con 0x7f1228151910 2020-11-20T12:19:09.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f122f073700 1 -- 172.21.15.116:0/3892077035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122809fc80 con 0x7f1228151910 2020-11-20T12:19:09.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f1224ff9700 1 -- 172.21.15.116:0/3892077035 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1214005800 con 0x7f1228151910 2020-11-20T12:19:09.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.464+0000 7f122f073700 1 -- 172.21.15.116:0/3892077035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1228154f30 con 0x7f1228151910 2020-11-20T12:19:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.468+0000 7f122f073700 1 -- 172.21.15.116:0/3892077035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1228062250 con 0x7f1228151910 2020-11-20T12:19:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.468+0000 7f1224ff9700 1 -- 172.21.15.116:0/3892077035 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1214010070 con 0x7f1228151910 2020-11-20T12:19:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.468+0000 7f1224ff9700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12000557c0 0x7f1200057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.468+0000 7f1224ff9700 1 -- 172.21.15.116:0/3892077035 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f121406da40 con 0x7f1228151910 2020-11-20T12:19:09.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.468+0000 7f122ce0f700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12000557c0 0x7f1200057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.472+0000 7f122ce0f700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12000557c0 0x7f1200057c80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f121c006fd0 tx=0x7f121c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:09.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.476+0000 7f1224ff9700 1 -- 172.21.15.116:0/3892077035 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1214043c90 con 0x7f1228151910 2020-11-20T12:19:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.636+0000 7f122f073700 1 -- 172.21.15.116:0/3892077035 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12281653d0 con 0x7f1228151910 2020-11-20T12:19:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.636+0000 7f1224ff9700 1 -- 172.21.15.116:0/3892077035 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+13724 (secure 0 0 0) 0x7f1214018030 con 0x7f1228151910 2020-11-20T12:19:09.638 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:09.638 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":64,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:08.322701+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 -- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12000557c0 msgr2=0x7f1200057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12000557c0 0x7f1200057c80 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f121c006fd0 tx=0x7f121c006df0).stop 2020-11-20T12:19:09.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 -- 172.21.15.116:0/3892077035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1228151910 msgr2=0x7f1228154a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1228151910 0x7f1228154a40 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f121400c710 tx=0x7f1214007ad0).stop 2020-11-20T12:19:09.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 -- 172.21.15.116:0/3892077035 shutdown_connections 2020-11-20T12:19:09.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12000557c0 0x7f1200057c80 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1228151cf0 0x7f1228153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1228068200 0x7f12281644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 --2- 172.21.15.116:0/3892077035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1228151910 0x7f1228154a40 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 -- 172.21.15.116:0/3892077035 >> 172.21.15.116:0/3892077035 conn(0x7f12280c6030 msgr2=0x7f1228067560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 -- 172.21.15.116:0/3892077035 shutdown_connections 2020-11-20T12:19:09.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.640+0000 7f122f073700 1 -- 172.21.15.116:0/3892077035 wait complete. 2020-11-20T12:19:09.665 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T12:19:09.666 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:19:09.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.852+0000 7fbc8759e700 1 Processor -- start 2020-11-20T12:19:09.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.852+0000 7fbc8759e700 1 -- start start 2020-11-20T12:19:09.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc8759e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbc88155c80 0x7fbc88062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc8759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc88156060 0x7fbc880c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc8759e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc880b93c0 con 0x7fbc880c12c0 2020-11-20T12:19:09.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc8759e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc880ba580 con 0x7fbc88155c80 2020-11-20T12:19:09.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc8759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880b9660 con 0x7fbc88156060 2020-11-20T12:19:09.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc86d9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fbc880c12c0 0x7fbc88060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57062/0 (socket says 172.21.15.116:57062) 2020-11-20T12:19:09.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc86d9d700 1 -- 172.21.15.116:0/259776846 learned_addr learned my addr 172.21.15.116:0/259776846 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:09.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc8659c700 1 --2- 172.21.15.116:0/259776846 >> v2:172.21.15.116:3300/0 conn(0x7fbc88155c80 0x7fbc88062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc7dd9b700 1 --2- 172.21.15.116:0/259776846 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc88156060 0x7fbc880c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc85d9b700 1 -- 172.21.15.116:0/259776846 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 326317468 0 0) 0x7fbc880b93c0 con 0x7fbc880c12c0 2020-11-20T12:19:09.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc85d9b700 1 -- 172.21.15.116:0/259776846 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc68003240 con 0x7fbc880c12c0 2020-11-20T12:19:09.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc8659c700 1 -- 172.21.15.116:0/259776846 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc88156060 msgr2=0x7fbc880c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:09.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc8659c700 1 --2- 172.21.15.116:0/259776846 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc88156060 0x7fbc880c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc8659c700 1 -- 172.21.15.116:0/259776846 >> v1:172.21.15.116:6789/0 conn(0x7fbc880c12c0 legacy=0x7fbc88060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc8659c700 1 -- 172.21.15.116:0/259776846 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880b9900 con 0x7fbc88155c80 2020-11-20T12:19:09.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc7dd9b700 1 --2- 172.21.15.116:0/259776846 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc88156060 0x7fbc880c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:09.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.856+0000 7fbc8659c700 1 --2- 172.21.15.116:0/259776846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88155c80 0x7fbc88062ed0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7fbc7800b9a0 tx=0x7fbc78006df0).ready entity=mon.0 client_cookie=a6bf66f12503cce5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:09.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.860+0000 7fbc85d9b700 1 -- 172.21.15.116:0/259776846 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78009680 con 0x7fbc88155c80 2020-11-20T12:19:09.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.860+0000 7fbc85d9b700 1 -- 172.21.15.116:0/259776846 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc780097e0 con 0x7fbc88155c80 2020-11-20T12:19:09.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.860+0000 7fbc85d9b700 1 -- 172.21.15.116:0/259776846 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc78009b60 con 0x7fbc88155c80 2020-11-20T12:19:09.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.860+0000 7fbc8759e700 1 -- 172.21.15.116:0/259776846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88155c80 msgr2=0x7fbc88062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.860+0000 7fbc8759e700 1 --2- 172.21.15.116:0/259776846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88155c80 0x7fbc88062ed0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7fbc7800b9a0 tx=0x7fbc78006df0).stop 2020-11-20T12:19:09.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.860+0000 7fbc8759e700 1 -- 172.21.15.116:0/259776846 shutdown_connections 2020-11-20T12:19:09.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.860+0000 7fbc8759e700 1 --2- 172.21.15.116:0/259776846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88155c80 0x7fbc88062ed0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.860+0000 7fbc8759e700 1 --2- 172.21.15.116:0/259776846 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc88156060 0x7fbc880c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.860+0000 7fbc8759e700 1 -- 172.21.15.116:0/259776846 >> 172.21.15.116:0/259776846 conn(0x7fbc88068350 msgr2=0x7fbc880627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:09.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.860+0000 7fbc8759e700 1 -- 172.21.15.116:0/259776846 shutdown_connections 2020-11-20T12:19:09.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.860+0000 7fbc8759e700 1 -- 172.21.15.116:0/259776846 wait complete. 2020-11-20T12:19:09.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.864+0000 7fbc8759e700 1 Processor -- start 2020-11-20T12:19:09.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.864+0000 7fbc8759e700 1 -- start start 2020-11-20T12:19:09.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.864+0000 7fbc8759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc880c12c0 0x7fbc88153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.864+0000 7fbc8759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88155c80 0x7fbc8815bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.864+0000 7fbc8759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc88156060 0x7fbc88168e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.864+0000 7fbc8759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880aa090 con 0x7fbc88155c80 2020-11-20T12:19:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.864+0000 7fbc8759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880aa330 con 0x7fbc88156060 2020-11-20T12:19:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.864+0000 7fbc8759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc880bc590 con 0x7fbc880c12c0 2020-11-20T12:19:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.864+0000 7fbc7dd9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88155c80 0x7fbc8815bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.864+0000 7fbc7dd9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88155c80 0x7fbc8815bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47632/0 (socket says 172.21.15.116:47632) 2020-11-20T12:19:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.864+0000 7fbc7dd9b700 1 -- 172.21.15.116:0/4004168610 learned_addr learned my addr 172.21.15.116:0/4004168610 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc8659c700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc880c12c0 0x7fbc88153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc86d9d700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc88156060 0x7fbc88168e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc7dd9b700 1 -- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc880c12c0 msgr2=0x7fbc88153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc7dd9b700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc880c12c0 0x7fbc88153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc7dd9b700 1 -- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc88156060 msgr2=0x7fbc88168e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc7dd9b700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc88156060 0x7fbc88168e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc7dd9b700 1 -- 172.21.15.116:0/4004168610 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc880bc830 con 0x7fbc88155c80 2020-11-20T12:19:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc7dd9b700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88155c80 0x7fbc8815bae0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fbc7000ef50 tx=0x7fbc70006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:09.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc7effd700 1 -- 172.21.15.116:0/4004168610 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc7000cbf0 con 0x7fbc88155c80 2020-11-20T12:19:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc7effd700 1 -- 172.21.15.116:0/4004168610 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc7000cd50 con 0x7fbc88155c80 2020-11-20T12:19:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc8759e700 1 -- 172.21.15.116:0/4004168610 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc880bcad0 con 0x7fbc88155c80 2020-11-20T12:19:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc7effd700 1 -- 172.21.15.116:0/4004168610 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc70012600 con 0x7fbc88155c80 2020-11-20T12:19:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc8759e700 1 -- 172.21.15.116:0/4004168610 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc8815bfd0 con 0x7fbc88155c80 2020-11-20T12:19:09.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc8759e700 1 -- 172.21.15.116:0/4004168610 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc880bcc60 con 0x7fbc88155c80 2020-11-20T12:19:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.868+0000 7fbc7effd700 1 -- 172.21.15.116:0/4004168610 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fbc70012970 con 0x7fbc88155c80 2020-11-20T12:19:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.872+0000 7fbc7effd700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc640557c0 0x7fbc64057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.872+0000 7fbc7effd700 1 -- 172.21.15.116:0/4004168610 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fbc7006eaf0 con 0x7fbc88155c80 2020-11-20T12:19:09.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.872+0000 7fbc8659c700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc640557c0 0x7fbc64057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:09.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.872+0000 7fbc8659c700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc640557c0 0x7fbc64057c80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fbc7800b150 tx=0x7fbc78015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:09.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:09.876+0000 7fbc7effd700 1 -- 172.21.15.116:0/4004168610 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbc70044cc0 con 0x7fbc88155c80 2020-11-20T12:19:10.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.040+0000 7fbc8759e700 1 -- 172.21.15.116:0/4004168610 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fbc8805cf30 con 0x7fbc88155c80 2020-11-20T12:19:10.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.040+0000 7fbc7effd700 1 -- 172.21.15.116:0/4004168610 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v64) v1 ==== 110+0+10 (secure 0 0 0) 0x7fbc700482e0 con 0x7fbc88155c80 2020-11-20T12:19:10.040 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:19:10.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 -- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc640557c0 msgr2=0x7fbc64057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc640557c0 0x7fbc64057c80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fbc7800b150 tx=0x7fbc78015000).stop 2020-11-20T12:19:10.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 -- 172.21.15.116:0/4004168610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88155c80 msgr2=0x7fbc8815bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88155c80 0x7fbc8815bae0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fbc7000ef50 tx=0x7fbc70006df0).stop 2020-11-20T12:19:10.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 -- 172.21.15.116:0/4004168610 shutdown_connections 2020-11-20T12:19:10.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc640557c0 0x7fbc64057c80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc88156060 0x7fbc88168e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc880c12c0 0x7fbc88153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 --2- 172.21.15.116:0/4004168610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc88155c80 0x7fbc8815bae0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 -- 172.21.15.116:0/4004168610 >> 172.21.15.116:0/4004168610 conn(0x7fbc88068350 msgr2=0x7fbc880670f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:10.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 -- 172.21.15.116:0/4004168610 shutdown_connections 2020-11-20T12:19:10.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.044+0000 7fbc8759e700 1 -- 172.21.15.116:0/4004168610 wait complete. 2020-11-20T12:19:10.066 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:19:10.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.252+0000 7f2de7157700 1 Processor -- start 2020-11-20T12:19:10.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de7157700 1 -- start start 2020-11-20T12:19:10.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de7157700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2de00a8400 0x7f2de00aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de7157700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2de00ad540 0x7f2de00ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de7157700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2de00ace40 con 0x7f2de00aa7d0 2020-11-20T12:19:10.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de7157700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de01526c0 con 0x7f2de00ad540 2020-11-20T12:19:10.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de7157700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de0152960 con 0x7f2de00a8400 2020-11-20T12:19:10.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de6956700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2de00ad540 0x7f2de00ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de5954700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2de00a8400 0x7f2de00aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de5954700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2de00a8400 0x7f2de00aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:46690/0 (socket says 172.21.15.116:46690) 2020-11-20T12:19:10.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de5954700 1 -- 172.21.15.116:0/1181947978 learned_addr learned my addr 172.21.15.116:0/1181947978 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:10.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de5954700 1 -- 172.21.15.116:0/1181947978 >> v2:172.21.15.116:3300/0 conn(0x7f2de00ad540 msgr2=0x7f2de00ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:10.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de5954700 1 --2- 172.21.15.116:0/1181947978 >> v2:172.21.15.116:3300/0 conn(0x7f2de00ad540 0x7f2de00ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de5954700 1 -- 172.21.15.116:0/1181947978 >> v1:172.21.15.173:6789/0 conn(0x7f2de00aa7d0 legacy=0x7f2de0151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.256+0000 7f2de5954700 1 -- 172.21.15.116:0/1181947978 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de0152c00 con 0x7f2de00a8400 2020-11-20T12:19:10.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de5954700 1 --2- 172.21.15.116:0/1181947978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00a8400 0x7f2de00aabb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f2de0155390 tx=0x7f2dd0006eb0).ready entity=mon.1 client_cookie=351cb0f808dac977 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:10.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de5153700 1 -- 172.21.15.116:0/1181947978 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0009810 con 0x7f2de00a8400 2020-11-20T12:19:10.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de5153700 1 -- 172.21.15.116:0/1181947978 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd0009970 con 0x7f2de00a8400 2020-11-20T12:19:10.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de5153700 1 -- 172.21.15.116:0/1181947978 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0009cf0 con 0x7f2de00a8400 2020-11-20T12:19:10.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de7157700 1 -- 172.21.15.116:0/1181947978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00a8400 msgr2=0x7f2de00aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de7157700 1 --2- 172.21.15.116:0/1181947978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00a8400 0x7f2de00aabb0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f2de0155390 tx=0x7f2dd0006eb0).stop 2020-11-20T12:19:10.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de7157700 1 -- 172.21.15.116:0/1181947978 shutdown_connections 2020-11-20T12:19:10.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de7157700 1 --2- 172.21.15.116:0/1181947978 >> v2:172.21.15.116:3300/0 conn(0x7f2de00ad540 0x7f2de00ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de7157700 1 --2- 172.21.15.116:0/1181947978 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00a8400 0x7f2de00aabb0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de7157700 1 -- 172.21.15.116:0/1181947978 >> 172.21.15.116:0/1181947978 conn(0x7f2de00678a0 msgr2=0x7f2de0067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:10.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de7157700 1 -- 172.21.15.116:0/1181947978 shutdown_connections 2020-11-20T12:19:10.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.260+0000 7f2de7157700 1 -- 172.21.15.116:0/1181947978 wait complete. 2020-11-20T12:19:10.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de7157700 1 Processor -- start 2020-11-20T12:19:10.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de7157700 1 -- start start 2020-11-20T12:19:10.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de7157700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2de00a8400 0x7f2de0150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de7157700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2de00aa7d0 0x7f2de014c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de7157700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00ad540 0x7f2de01647a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de7157700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de014caa0 con 0x7f2de00aa7d0 2020-11-20T12:19:10.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de7157700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de014ccf0 con 0x7f2de00ad540 2020-11-20T12:19:10.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de7157700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de014cf90 con 0x7f2de00a8400 2020-11-20T12:19:10.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de5954700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2de00aa7d0 0x7f2de014c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de5954700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2de00aa7d0 0x7f2de014c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47644/0 (socket says 172.21.15.116:47644) 2020-11-20T12:19:10.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de5954700 1 -- 172.21.15.116:0/3055438668 learned_addr learned my addr 172.21.15.116:0/3055438668 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:10.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de6956700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00ad540 0x7f2de01647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de6155700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2de00a8400 0x7f2de0150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de5954700 1 -- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2de00a8400 msgr2=0x7f2de0150fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de5954700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2de00a8400 0x7f2de0150fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de5954700 1 -- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00ad540 msgr2=0x7f2de01647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de5954700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00ad540 0x7f2de01647a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.264+0000 7f2de5954700 1 -- 172.21.15.116:0/3055438668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de014d230 con 0x7f2de00aa7d0 2020-11-20T12:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.268+0000 7f2de5954700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2de00aa7d0 0x7f2de014c5b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f2dd0010000 tx=0x7f2dd0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.268+0000 7f2dce7fc700 1 -- 172.21.15.116:0/3055438668 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd0011e70 con 0x7f2de00aa7d0 2020-11-20T12:19:10.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.268+0000 7f2dce7fc700 1 -- 172.21.15.116:0/3055438668 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd001e460 con 0x7f2de00aa7d0 2020-11-20T12:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.268+0000 7f2de7157700 1 -- 172.21.15.116:0/3055438668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de014d4d0 con 0x7f2de00aa7d0 2020-11-20T12:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.268+0000 7f2dce7fc700 1 -- 172.21.15.116:0/3055438668 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd001e820 con 0x7f2de00aa7d0 2020-11-20T12:19:10.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.268+0000 7f2de7157700 1 -- 172.21.15.116:0/3055438668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de014dc10 con 0x7f2de00aa7d0 2020-11-20T12:19:10.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.268+0000 7f2de7157700 1 -- 172.21.15.116:0/3055438668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de005cf30 con 0x7f2de00aa7d0 2020-11-20T12:19:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.268+0000 7f2dce7fc700 1 -- 172.21.15.116:0/3055438668 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2dd0025030 con 0x7f2de00aa7d0 2020-11-20T12:19:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.268+0000 7f2dce7fc700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2db80557c0 0x7f2db8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.272+0000 7f2dce7fc700 1 -- 172.21.15.116:0/3055438668 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f2dd006fad0 con 0x7f2de00aa7d0 2020-11-20T12:19:10.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.272+0000 7f2de6155700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2db80557c0 0x7f2db8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.272+0000 7f2de6155700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2db80557c0 0x7f2db8057c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f2ddc0034f0 tx=0x7f2ddc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:10.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.276+0000 7f2dce7fc700 1 -- 172.21.15.116:0/3055438668 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2dd0045ca0 con 0x7f2de00aa7d0 2020-11-20T12:19:10.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.436+0000 7f2de7157700 1 -- 172.21.15.116:0/3055438668 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2de00601b0 con 0x7f2de00aa7d0 2020-11-20T12:19:10.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.436+0000 7f2dce7fc700 1 -- 172.21.15.116:0/3055438668 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v64) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2dd0023020 con 0x7f2de00aa7d0 2020-11-20T12:19:10.435 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:10.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 -- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2db80557c0 msgr2=0x7f2db8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2db80557c0 0x7f2db8057c80 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f2ddc0034f0 tx=0x7f2ddc009040).stop 2020-11-20T12:19:10.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 -- 172.21.15.116:0/3055438668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2de00aa7d0 msgr2=0x7f2de014c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2de00aa7d0 0x7f2de014c5b0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f2dd0010000 tx=0x7f2dd0015000).stop 2020-11-20T12:19:10.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 -- 172.21.15.116:0/3055438668 shutdown_connections 2020-11-20T12:19:10.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2db80557c0 0x7f2db8057c80 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0x7f2ddc0034f0 tx=0x7f2ddc009040).stop 2020-11-20T12:19:10.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2de00ad540 0x7f2de01647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2de00a8400 0x7f2de0150fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 --2- 172.21.15.116:0/3055438668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2de00aa7d0 0x7f2de014c5b0 secure :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0x7f2dd0010000 tx=0x7f2dd0015000).stop 2020-11-20T12:19:10.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 -- 172.21.15.116:0/3055438668 >> 172.21.15.116:0/3055438668 conn(0x7f2de00678a0 msgr2=0x7f2de0153af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:10.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 -- 172.21.15.116:0/3055438668 shutdown_connections 2020-11-20T12:19:10.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.440+0000 7f2de7157700 1 -- 172.21.15.116:0/3055438668 wait complete. 2020-11-20T12:19:10.464 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-20T12:19:10.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.640+0000 7f0c8759e700 1 Processor -- start 2020-11-20T12:19:10.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.640+0000 7f0c8759e700 1 -- start start 2020-11-20T12:19:10.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.640+0000 7f0c8759e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0c88153450 0x7f0c88062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.640+0000 7f0c8759e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0c880677a0 0x7f0c880b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.640+0000 7f0c8759e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c881543a0 con 0x7f0c88062880 2020-11-20T12:19:10.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.640+0000 7f0c8759e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c88153920 con 0x7f0c88153450 2020-11-20T12:19:10.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.640+0000 7f0c8759e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c88153bc0 con 0x7f0c880677a0 2020-11-20T12:19:10.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8659c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0c88062880 0x7f0c880c4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57086/0 (socket says 172.21.15.116:57086) 2020-11-20T12:19:10.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8659c700 1 -- 172.21.15.116:0/2108054043 learned_addr learned my addr 172.21.15.116:0/2108054043 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:10.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c85d9b700 1 --2- 172.21.15.116:0/2108054043 >> v2:172.21.15.116:3300/0 conn(0x7f0c88153450 0x7f0c88062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c86d9d700 1 --2- 172.21.15.116:0/2108054043 >> v2:172.21.15.173:3300/0 conn(0x7f0c880677a0 0x7f0c880b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8559a700 1 -- 172.21.15.116:0/2108054043 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3963729170 0 0) 0x7f0c881543a0 con 0x7f0c88062880 2020-11-20T12:19:10.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8559a700 1 -- 172.21.15.116:0/2108054043 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c6c003240 con 0x7f0c88062880 2020-11-20T12:19:10.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c85d9b700 1 -- 172.21.15.116:0/2108054043 >> v2:172.21.15.173:3300/0 conn(0x7f0c880677a0 msgr2=0x7f0c880b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:10.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c85d9b700 1 --2- 172.21.15.116:0/2108054043 >> v2:172.21.15.173:3300/0 conn(0x7f0c880677a0 0x7f0c880b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c85d9b700 1 -- 172.21.15.116:0/2108054043 >> v1:172.21.15.116:6789/0 conn(0x7f0c88062880 legacy=0x7f0c880c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c85d9b700 1 -- 172.21.15.116:0/2108054043 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c88067c30 con 0x7f0c88153450 2020-11-20T12:19:10.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c85d9b700 1 --2- 172.21.15.116:0/2108054043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c88153450 0x7f0c88062c60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f0c880a3f20 tx=0x7f0c7c006df0).ready entity=mon.0 client_cookie=50274105419ff4e8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:10.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8559a700 1 -- 172.21.15.116:0/2108054043 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c009640 con 0x7f0c88153450 2020-11-20T12:19:10.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8559a700 1 -- 172.21.15.116:0/2108054043 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c7c0097a0 con 0x7f0c88153450 2020-11-20T12:19:10.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8559a700 1 -- 172.21.15.116:0/2108054043 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c7c009b20 con 0x7f0c88153450 2020-11-20T12:19:10.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8759e700 1 -- 172.21.15.116:0/2108054043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c88153450 msgr2=0x7f0c88062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8759e700 1 --2- 172.21.15.116:0/2108054043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c88153450 0x7f0c88062c60 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f0c880a3f20 tx=0x7f0c7c006df0).stop 2020-11-20T12:19:10.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8759e700 1 -- 172.21.15.116:0/2108054043 shutdown_connections 2020-11-20T12:19:10.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8759e700 1 --2- 172.21.15.116:0/2108054043 >> v2:172.21.15.173:3300/0 conn(0x7f0c880677a0 0x7f0c880b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8759e700 1 --2- 172.21.15.116:0/2108054043 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c88153450 0x7f0c88062c60 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.644+0000 7f0c8759e700 1 -- 172.21.15.116:0/2108054043 >> 172.21.15.116:0/2108054043 conn(0x7f0c88060c40 msgr2=0x7f0c880c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:10.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.648+0000 7f0c8759e700 1 -- 172.21.15.116:0/2108054043 shutdown_connections 2020-11-20T12:19:10.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.648+0000 7f0c8759e700 1 -- 172.21.15.116:0/2108054043 wait complete. 2020-11-20T12:19:10.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.648+0000 7f0c8759e700 1 Processor -- start 2020-11-20T12:19:10.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.648+0000 7f0c8759e700 1 -- start start 2020-11-20T12:19:10.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.648+0000 7f0c8759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c88062880 0x7f0c880c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c8759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c880677a0 0x7f0c881611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c8759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c88153450 0x7f0c881552d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c8759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880a1d60 con 0x7f0c88153450 2020-11-20T12:19:10.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c8759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880a2000 con 0x7f0c880677a0 2020-11-20T12:19:10.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c8759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c880a22a0 con 0x7f0c88062880 2020-11-20T12:19:10.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c86d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c88153450 0x7f0c881552d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c8659c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c88062880 0x7f0c880c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c86d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c88153450 0x7f0c881552d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47662/0 (socket says 172.21.15.116:47662) 2020-11-20T12:19:10.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c86d9d700 1 -- 172.21.15.116:0/1738397060 learned_addr learned my addr 172.21.15.116:0/1738397060 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:10.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c85d9b700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c880677a0 0x7f0c881611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c86d9d700 1 -- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c88062880 msgr2=0x7f0c880c2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c86d9d700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c88062880 0x7f0c880c2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c86d9d700 1 -- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c880677a0 msgr2=0x7f0c881611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c86d9d700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c880677a0 0x7f0c881611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c86d9d700 1 -- 172.21.15.116:0/1738397060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c880a2540 con 0x7f0c88153450 2020-11-20T12:19:10.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c86d9d700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c88153450 0x7f0c881552d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f0c8000ef50 tx=0x7f0c800091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:10.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c767fc700 1 -- 172.21.15.116:0/1738397060 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c8000cb40 con 0x7f0c88153450 2020-11-20T12:19:10.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c767fc700 1 -- 172.21.15.116:0/1738397060 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c8000cca0 con 0x7f0c88153450 2020-11-20T12:19:10.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c767fc700 1 -- 172.21.15.116:0/1738397060 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c800123f0 con 0x7f0c88153450 2020-11-20T12:19:10.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c8759e700 1 -- 172.21.15.116:0/1738397060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c880c2e40 con 0x7f0c88153450 2020-11-20T12:19:10.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.652+0000 7f0c8759e700 1 -- 172.21.15.116:0/1738397060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c881616c0 con 0x7f0c88153450 2020-11-20T12:19:10.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.656+0000 7f0c8759e700 1 -- 172.21.15.116:0/1738397060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c8805cf30 con 0x7f0c88153450 2020-11-20T12:19:10.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.656+0000 7f0c767fc700 1 -- 172.21.15.116:0/1738397060 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0c80022070 con 0x7f0c88153450 2020-11-20T12:19:10.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.656+0000 7f0c767fc700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c5c0557c0 0x7f0c5c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:10.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.656+0000 7f0c767fc700 1 -- 172.21.15.116:0/1738397060 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f0c8006ec10 con 0x7f0c88153450 2020-11-20T12:19:10.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.656+0000 7f0c8659c700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c5c0557c0 0x7f0c5c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:10.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.660+0000 7f0c8659c700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c5c0557c0 0x7f0c5c057c80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f0c880c1cb0 tx=0x7f0c78009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:10.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.664+0000 7f0c767fc700 1 -- 172.21.15.116:0/1738397060 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0c80044de0 con 0x7f0c88153450 2020-11-20T12:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 -- 172.21.15.116:0/1738397060 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f0c880c0b80 con 0x7f0c88153450 2020-11-20T12:19:10.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c767fc700 1 -- 172.21.15.116:0/1738397060 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) v1 ==== 84+0+143 (secure 0 0 0) 0x7f0c80016030 con 0x7f0c88153450 2020-11-20T12:19:10.826 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:10.826 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2020-11-20T12:19:10.826 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:10.826 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2020-11-20T12:19:10.827 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "device_health_metrics" 2020-11-20T12:19:10.827 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:10.827 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:10.827 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 2, 2020-11-20T12:19:10.828 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "rbd" 2020-11-20T12:19:10.828 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:10.828 INFO:teuthology.orchestra.run.smithi116.stdout:] 2020-11-20T12:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 -- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c5c0557c0 msgr2=0x7f0c5c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c5c0557c0 0x7f0c5c057c80 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f0c880c1cb0 tx=0x7f0c78009040).stop 2020-11-20T12:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 -- 172.21.15.116:0/1738397060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c88153450 msgr2=0x7f0c881552d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c88153450 0x7f0c881552d0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f0c8000ef50 tx=0x7f0c800091d0).stop 2020-11-20T12:19:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 -- 172.21.15.116:0/1738397060 shutdown_connections 2020-11-20T12:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c5c0557c0 0x7f0c5c057c80 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c88153450 0x7f0c881552d0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c88062880 0x7f0c880c2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 --2- 172.21.15.116:0/1738397060 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c880677a0 0x7f0c881611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 -- 172.21.15.116:0/1738397060 >> 172.21.15.116:0/1738397060 conn(0x7f0c88060c40 msgr2=0x7f0c880a5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 -- 172.21.15.116:0/1738397060 shutdown_connections 2020-11-20T12:19:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:10.828+0000 7f0c8759e700 1 -- 172.21.15.116:0/1738397060 wait complete. 2020-11-20T12:19:10.851 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T12:19:10.851 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-20T12:19:11.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.040+0000 7f60062b4700 1 Processor -- start 2020-11-20T12:19:11.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.040+0000 7f60062b4700 1 -- start start 2020-11-20T12:19:11.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.040+0000 7f60062b4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6000155610 0x7f6000155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.040+0000 7f60062b4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6000060790 0x7f6000060c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.040+0000 7f60062b4700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60000a2250 con 0x7f600014bfe0 2020-11-20T12:19:11.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.040+0000 7f60062b4700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60000a1f60 con 0x7f6000155610 2020-11-20T12:19:11.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.040+0000 7f60062b4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60000a24f0 con 0x7f6000060790 2020-11-20T12:19:11.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5fff7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6000155610 0x7f6000155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5ffffff700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f600014bfe0 0x7f60000a1850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:53330/0 (socket says 172.21.15.116:53330) 2020-11-20T12:19:11.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5ffffff700 1 -- 172.21.15.116:0/49569284 learned_addr learned my addr 172.21.15.116:0/49569284 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:11.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f6004851700 1 --2- 172.21.15.116:0/49569284 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6000060790 0x7f6000060c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5ffeffd700 1 -- 172.21.15.116:0/49569284 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1275625026 0 0) 0x7f60000a2250 con 0x7f600014bfe0 2020-11-20T12:19:11.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5fff7fe700 1 -- 172.21.15.116:0/49569284 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6000060790 msgr2=0x7f6000060c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:11.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5fff7fe700 1 --2- 172.21.15.116:0/49569284 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6000060790 0x7f6000060c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5fff7fe700 1 -- 172.21.15.116:0/49569284 >> v1:172.21.15.173:6789/0 conn(0x7f600014bfe0 legacy=0x7f60000a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5fff7fe700 1 -- 172.21.15.116:0/49569284 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60000a2790 con 0x7f6000155610 2020-11-20T12:19:11.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5fff7fe700 1 --2- 172.21.15.116:0/49569284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000155610 0x7f6000155aa0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f5ff400b9a0 tx=0x7f5ff4006df0).ready entity=mon.0 client_cookie=c5722f50013af825 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5ffeffd700 1 -- 172.21.15.116:0/49569284 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff40097b0 con 0x7f6000155610 2020-11-20T12:19:11.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5ffeffd700 1 -- 172.21.15.116:0/49569284 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff4009910 con 0x7f6000155610 2020-11-20T12:19:11.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.044+0000 7f5ffeffd700 1 -- 172.21.15.116:0/49569284 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4009c90 con 0x7f6000155610 2020-11-20T12:19:11.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.048+0000 7f60062b4700 1 -- 172.21.15.116:0/49569284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000155610 msgr2=0x7f6000155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.048+0000 7f60062b4700 1 --2- 172.21.15.116:0/49569284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000155610 0x7f6000155aa0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7f5ff400b9a0 tx=0x7f5ff4006df0).stop 2020-11-20T12:19:11.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.048+0000 7f60062b4700 1 -- 172.21.15.116:0/49569284 shutdown_connections 2020-11-20T12:19:11.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.048+0000 7f60062b4700 1 --2- 172.21.15.116:0/49569284 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6000060790 0x7f6000060c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.048+0000 7f60062b4700 1 --2- 172.21.15.116:0/49569284 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000155610 0x7f6000155aa0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.048+0000 7f60062b4700 1 -- 172.21.15.116:0/49569284 >> 172.21.15.116:0/49569284 conn(0x7f60000c17c0 msgr2=0x7f60000c1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:11.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.048+0000 7f60062b4700 1 -- 172.21.15.116:0/49569284 shutdown_connections 2020-11-20T12:19:11.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.048+0000 7f60062b4700 1 -- 172.21.15.116:0/49569284 wait complete. 2020-11-20T12:19:11.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.048+0000 7f60062b4700 1 Processor -- start 2020-11-20T12:19:11.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f60062b4700 1 -- start start 2020-11-20T12:19:11.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f60062b4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000060790 0x7f60000a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f60062b4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f600014bfe0 0x7f600014dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f60062b4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6000155610 0x7f60000a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f60062b4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60000a9b60 con 0x7f6000060790 2020-11-20T12:19:11.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f60062b4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60000a9380 con 0x7f6000155610 2020-11-20T12:19:11.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f60062b4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60000a7cd0 con 0x7f600014bfe0 2020-11-20T12:19:11.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5ffffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000060790 0x7f60000a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5ffffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000060790 0x7f60000a8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47672/0 (socket says 172.21.15.116:47672) 2020-11-20T12:19:11.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5ffffff700 1 -- 172.21.15.116:0/2071059009 learned_addr learned my addr 172.21.15.116:0/2071059009 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:11.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f6004851700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6000155610 0x7f60000a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5ffffff700 1 -- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f600014bfe0 msgr2=0x7f600014dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5ffffff700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f600014bfe0 0x7f600014dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:11.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5ffffff700 1 -- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6000155610 msgr2=0x7f60000a5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5ffffff700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6000155610 0x7f60000a5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5ffffff700 1 -- 172.21.15.116:0/2071059009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60000a7f70 con 0x7f6000060790 2020-11-20T12:19:11.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f6004851700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6000155610 0x7f60000a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:11.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5ffffff700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000060790 0x7f60000a8e90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f5ff000c750 tx=0x7f5ff0007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5fdffff700 1 -- 172.21.15.116:0/2071059009 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0007320 con 0x7f6000060790 2020-11-20T12:19:11.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5fdffff700 1 -- 172.21.15.116:0/2071059009 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0007480 con 0x7f6000060790 2020-11-20T12:19:11.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f5fdffff700 1 -- 172.21.15.116:0/2071059009 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0005800 con 0x7f6000060790 2020-11-20T12:19:11.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f60062b4700 1 -- 172.21.15.116:0/2071059009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60000a8210 con 0x7f6000060790 2020-11-20T12:19:11.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.052+0000 7f60062b4700 1 -- 172.21.15.116:0/2071059009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60000a8750 con 0x7f6000060790 2020-11-20T12:19:11.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.056+0000 7f60062b4700 1 -- 172.21.15.116:0/2071059009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600005d2a0 con 0x7f6000060790 2020-11-20T12:19:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.056+0000 7f5fdffff700 1 -- 172.21.15.116:0/2071059009 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5ff0010030 con 0x7f6000060790 2020-11-20T12:19:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.056+0000 7f5fdffff700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5fd80557f0 0x7f5fd8057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.056+0000 7f5fdffff700 1 -- 172.21.15.116:0/2071059009 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f5ff006e0d0 con 0x7f6000060790 2020-11-20T12:19:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.056+0000 7f5fff7fe700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5fd80557f0 0x7f5fd8057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.060+0000 7f5fff7fe700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5fd80557f0 0x7f5fd8057cb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f60000a9740 tx=0x7f5ff4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.064+0000 7f5fdffff700 1 -- 172.21.15.116:0/2071059009 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5ff00442a0 con 0x7f6000060790 2020-11-20T12:19:11.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.224+0000 7f60062b4700 1 -- 172.21.15.116:0/2071059009 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f6000155010 con 0x7f6000060790 2020-11-20T12:19:11.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.496+0000 7f5fdffff700 1 -- 172.21.15.116:0/2071059009 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v65) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5ff00478c0 con 0x7f6000060790 2020-11-20T12:19:11.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.500+0000 7f60062b4700 1 -- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5fd80557f0 msgr2=0x7f5fd8057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.500+0000 7f60062b4700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5fd80557f0 0x7f5fd8057cb0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f60000a9740 tx=0x7f5ff4015000).stop 2020-11-20T12:19:11.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.500+0000 7f60062b4700 1 -- 172.21.15.116:0/2071059009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000060790 msgr2=0x7f60000a8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.500+0000 7f60062b4700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000060790 0x7f60000a8e90 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f5ff000c750 tx=0x7f5ff0007ac0).stop 2020-11-20T12:19:11.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.500+0000 7f60062b4700 1 -- 172.21.15.116:0/2071059009 shutdown_connections 2020-11-20T12:19:11.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.500+0000 7f60062b4700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5fd80557f0 0x7f5fd8057cb0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.500+0000 7f60062b4700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6000155610 0x7f60000a5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.500+0000 7f60062b4700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6000060790 0x7f60000a8e90 secure :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0x7f5ff000c750 tx=0x7f5ff0007ac0).stop 2020-11-20T12:19:11.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.500+0000 7f60062b4700 1 --2- 172.21.15.116:0/2071059009 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f600014bfe0 0x7f600014dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:11.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.500+0000 7f60062b4700 1 -- 172.21.15.116:0/2071059009 >> 172.21.15.116:0/2071059009 conn(0x7f60000c17c0 msgr2=0x7f6000067620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:11.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.504+0000 7f60062b4700 1 -- 172.21.15.116:0/2071059009 shutdown_connections 2020-11-20T12:19:11.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.504+0000 7f60062b4700 1 -- 172.21.15.116:0/2071059009 wait complete. 2020-11-20T12:19:11.505 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2020-11-20T12:19:11.527 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-20T12:19:11.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.700+0000 7f479c85e700 1 Processor -- start 2020-11-20T12:19:11.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.700+0000 7f479c85e700 1 -- start start 2020-11-20T12:19:11.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479c85e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f479814f880 0x7f47980b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479c85e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f47980c0e90 0x7f47980c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479c85e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47980a8930 con 0x7f47980a81f0 2020-11-20T12:19:11.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479c85e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47980ba2a0 con 0x7f479814f880 2020-11-20T12:19:11.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479c85e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47980a8690 con 0x7f47980c0e90 2020-11-20T12:19:11.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f4796d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f479814f880 0x7f47980b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479659c700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f47980a81f0 0x7f4798060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:53344/0 (socket says 172.21.15.116:53344) 2020-11-20T12:19:11.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479659c700 1 -- 172.21.15.116:0/2730426486 learned_addr learned my addr 172.21.15.116:0/2730426486 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:11.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479759e700 1 --2- 172.21.15.116:0/2730426486 >> v2:172.21.15.173:3300/0 conn(0x7f47980c0e90 0x7f47980c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f4795d9b700 1 -- 172.21.15.116:0/2730426486 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2908669879 0 0) 0x7f47980a8930 con 0x7f47980a81f0 2020-11-20T12:19:11.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f4795d9b700 1 -- 172.21.15.116:0/2730426486 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4784003240 con 0x7f47980a81f0 2020-11-20T12:19:11.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479759e700 1 -- 172.21.15.116:0/2730426486 >> v2:172.21.15.116:3300/0 conn(0x7f479814f880 msgr2=0x7f47980b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:11.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479759e700 1 --2- 172.21.15.116:0/2730426486 >> v2:172.21.15.116:3300/0 conn(0x7f479814f880 0x7f47980b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479759e700 1 -- 172.21.15.116:0/2730426486 >> v1:172.21.15.173:6789/0 conn(0x7f47980a81f0 legacy=0x7f4798060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479759e700 1 -- 172.21.15.116:0/2730426486 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980b94d0 con 0x7f47980c0e90 2020-11-20T12:19:11.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.704+0000 7f479759e700 1 --2- 172.21.15.116:0/2730426486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47980c0e90 0x7f47980c12c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f4798063860 tx=0x7f47900091d0).ready entity=mon.1 client_cookie=3db6b181d3a5982e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.708+0000 7f4795d9b700 1 -- 172.21.15.116:0/2730426486 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479000b9c0 con 0x7f47980c0e90 2020-11-20T12:19:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.708+0000 7f4795d9b700 1 -- 172.21.15.116:0/2730426486 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479000bb20 con 0x7f47980c0e90 2020-11-20T12:19:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.708+0000 7f4795d9b700 1 -- 172.21.15.116:0/2730426486 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479000bea0 con 0x7f47980c0e90 2020-11-20T12:19:11.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.708+0000 7f479c85e700 1 -- 172.21.15.116:0/2730426486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47980c0e90 msgr2=0x7f47980c12c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.708+0000 7f479c85e700 1 --2- 172.21.15.116:0/2730426486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47980c0e90 0x7f47980c12c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f4798063860 tx=0x7f47900091d0).stop 2020-11-20T12:19:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.708+0000 7f479c85e700 1 -- 172.21.15.116:0/2730426486 shutdown_connections 2020-11-20T12:19:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.708+0000 7f479c85e700 1 --2- 172.21.15.116:0/2730426486 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47980c0e90 0x7f47980c12c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.708+0000 7f479c85e700 1 --2- 172.21.15.116:0/2730426486 >> v2:172.21.15.116:3300/0 conn(0x7f479814f880 0x7f47980b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.708+0000 7f479c85e700 1 -- 172.21.15.116:0/2730426486 >> 172.21.15.116:0/2730426486 conn(0x7f4798068350 msgr2=0x7f47980627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:11.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.708+0000 7f479c85e700 1 -- 172.21.15.116:0/2730426486 shutdown_connections 2020-11-20T12:19:11.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.708+0000 7f479c85e700 1 -- 172.21.15.116:0/2730426486 wait complete. 2020-11-20T12:19:11.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479c85e700 1 Processor -- start 2020-11-20T12:19:11.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479c85e700 1 -- start start 2020-11-20T12:19:11.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479c85e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47980a81f0 0x7f47980c1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479c85e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47980c0e90 0x7f4798150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479c85e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f479814f880 0x7f4798160390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479c85e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47980c1810 con 0x7f479814f880 2020-11-20T12:19:11.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479c85e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47980bea90 con 0x7f47980c0e90 2020-11-20T12:19:11.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479c85e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47980bed30 con 0x7f47980a81f0 2020-11-20T12:19:11.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f479814f880 0x7f4798160390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f479814f880 0x7f4798160390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47688/0 (socket says 172.21.15.116:47688) 2020-11-20T12:19:11.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479759e700 1 -- 172.21.15.116:0/2373845339 learned_addr learned my addr 172.21.15.116:0/2373845339 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:11.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f4796d9d700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47980a81f0 0x7f47980c1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479659c700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47980c0e90 0x7f4798150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479759e700 1 -- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47980a81f0 msgr2=0x7f47980c1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479759e700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47980a81f0 0x7f47980c1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479759e700 1 -- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47980c0e90 msgr2=0x7f4798150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:11.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479759e700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47980c0e90 0x7f4798150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:11.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.712+0000 7f479759e700 1 -- 172.21.15.116:0/2373845339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980befd0 con 0x7f479814f880 2020-11-20T12:19:11.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.716+0000 7f479759e700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f479814f880 0x7f4798160390 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f4790009440 tx=0x7f4790016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.716+0000 7f477effd700 1 -- 172.21.15.116:0/2373845339 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479001c5f0 con 0x7f479814f880 2020-11-20T12:19:11.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.716+0000 7f477effd700 1 -- 172.21.15.116:0/2373845339 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f479001c750 con 0x7f479814f880 2020-11-20T12:19:11.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.716+0000 7f477effd700 1 -- 172.21.15.116:0/2373845339 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f479001caf0 con 0x7f479814f880 2020-11-20T12:19:11.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.716+0000 7f479c85e700 1 -- 172.21.15.116:0/2373845339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47980bf270 con 0x7f479814f880 2020-11-20T12:19:11.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.716+0000 7f479c85e700 1 -- 172.21.15.116:0/2373845339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4798160880 con 0x7f479814f880 2020-11-20T12:19:11.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.716+0000 7f479c85e700 1 -- 172.21.15.116:0/2373845339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47980bf400 con 0x7f479814f880 2020-11-20T12:19:11.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.716+0000 7f477effd700 1 -- 172.21.15.116:0/2373845339 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4790024030 con 0x7f479814f880 2020-11-20T12:19:11.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.720+0000 7f477effd700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47740557f0 0x7f4774057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:11.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.720+0000 7f477effd700 1 -- 172.21.15.116:0/2373845339 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f47900700c0 con 0x7f479814f880 2020-11-20T12:19:11.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.720+0000 7f4796d9d700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47740557f0 0x7f4774057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:11.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.720+0000 7f4796d9d700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47740557f0 0x7f4774057cb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f4788006fd0 tx=0x7f4788006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:11.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.724+0000 7f477effd700 1 -- 172.21.15.116:0/2373845339 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4790046130 con 0x7f479814f880 2020-11-20T12:19:11.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:11.880+0000 7f479c85e700 1 -- 172.21.15.116:0/2373845339 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f479805cf30 con 0x7f479814f880 2020-11-20T12:19:12.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.536+0000 7f477effd700 1 -- 172.21.15.116:0/2373845339 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v69) v1 ==== 125+0+0 (secure 0 0 0) 0x7f4790022020 con 0x7f479814f880 2020-11-20T12:19:12.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 -- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47740557f0 msgr2=0x7f4774057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47740557f0 0x7f4774057cb0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f4788006fd0 tx=0x7f4788006df0).stop 2020-11-20T12:19:12.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 -- 172.21.15.116:0/2373845339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f479814f880 msgr2=0x7f4798160390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f479814f880 0x7f4798160390 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f4790009440 tx=0x7f4790016000).stop 2020-11-20T12:19:12.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 -- 172.21.15.116:0/2373845339 shutdown_connections 2020-11-20T12:19:12.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47740557f0 0x7f4774057cb0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f479814f880 0x7f4798160390 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47980a81f0 0x7f47980c1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 --2- 172.21.15.116:0/2373845339 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47980c0e90 0x7f4798150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 -- 172.21.15.116:0/2373845339 >> 172.21.15.116:0/2373845339 conn(0x7f4798068350 msgr2=0x7f47980a75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:12.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 -- 172.21.15.116:0/2373845339 shutdown_connections 2020-11-20T12:19:12.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.540+0000 7f479c85e700 1 -- 172.21.15.116:0/2373845339 wait complete. 2020-11-20T12:19:12.540 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2020-11-20T12:19:12.566 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T12:19:12.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.748+0000 7fd832a7d700 1 Processor -- start 2020-11-20T12:19:12.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.748+0000 7fd832a7d700 1 -- start start 2020-11-20T12:19:12.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd832a7d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c155b90 0x7fd82c156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:12.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd832a7d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd82c1571b0 con 0x7fd82c060880 2020-11-20T12:19:12.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd832a7d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd82c0c0b50 con 0x7fd82c0c06b0 2020-11-20T12:19:12.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd832a7d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c159f40 con 0x7fd82c155b90 2020-11-20T12:19:12.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd83227c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd82c060880 0x7fd82c0c0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57134/0 (socket says 172.21.15.116:57134) 2020-11-20T12:19:12.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd83227c700 1 -- 172.21.15.116:0/29826010 learned_addr learned my addr 172.21.15.116:0/29826010 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:12.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd831a7b700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c155b90 0x7fd82c156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:12.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 727484028 0 0) 0x7fd82c1571b0 con 0x7fd82c060880 2020-11-20T12:19:12.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd814003240 con 0x7fd82c060880 2020-11-20T12:19:12.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 111061890 0 0) 0x7fd814003240 con 0x7fd82c060880 2020-11-20T12:19:12.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c155b90 msgr2=0x7fd82c156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:12.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c155b90 0x7fd82c156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 >> v1:172.21.15.173:6789/0 conn(0x7fd82c0c06b0 legacy=0x7fd82c159830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c0a66b0 con 0x7fd82c060880 2020-11-20T12:19:12.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fd824001860 con 0x7fd82c060880 2020-11-20T12:19:12.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 >> v1:172.21.15.116:6789/0 conn(0x7fd82c060880 legacy=0x7fd82c0c0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd814003760 0x7fd814003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:12.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8140044e0 0x7fd814004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:12.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c155b90 0x7fd814004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:12.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c159f40 con 0x7fd814003760 2020-11-20T12:19:12.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd814005390 con 0x7fd8140044e0 2020-11-20T12:19:12.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd814004320 con 0x7fd82c155b90 2020-11-20T12:19:12.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd831a7b700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd814003760 0x7fd814003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:12.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd83227c700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c155b90 0x7fd814004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:12.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd83127a700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8140044e0 0x7fd814004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:12.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd831a7b700 1 -- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c155b90 msgr2=0x7fd814004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:12.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd831a7b700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c155b90 0x7fd814004e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd831a7b700 1 -- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8140044e0 msgr2=0x7fd814004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:12.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd831a7b700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8140044e0 0x7fd814004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd831a7b700 1 -- 172.21.15.116:0/29826010 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd814003240 con 0x7fd814003760 2020-11-20T12:19:12.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.752+0000 7fd83127a700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd8140044e0 0x7fd814004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:12.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.756+0000 7fd831a7b700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd814003760 0x7fd814003b80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fd82800bc00 tx=0x7fd828006df0).ready entity=mon.0 client_cookie=479476b129a8322f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:12.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.756+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82800b310 con 0x7fd814003760 2020-11-20T12:19:12.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.756+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82800eb40 con 0x7fd814003760 2020-11-20T12:19:12.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.756+0000 7fd830a79700 1 -- 172.21.15.116:0/29826010 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82800eec0 con 0x7fd814003760 2020-11-20T12:19:12.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.756+0000 7fd832a7d700 1 -- 172.21.15.116:0/29826010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd814003760 msgr2=0x7fd814003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.756+0000 7fd832a7d700 1 --2- 172.21.15.116:0/29826010 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd814003760 0x7fd814003b80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fd82800bc00 tx=0x7fd828006df0).stop 2020-11-20T12:19:12.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.756+0000 7fd83227c700 1 -- 172.21.15.116:0/29826010 reap_dead start 2020-11-20T12:19:12.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.756+0000 7fd832a7d700 1 -- 172.21.15.116:0/29826010 shutdown_connections 2020-11-20T12:19:12.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.756+0000 7fd832a7d700 1 -- 172.21.15.116:0/29826010 >> 172.21.15.116:0/29826010 conn(0x7fd82c0682e0 msgr2=0x7fd82c060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:12.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.756+0000 7fd832a7d700 1 -- 172.21.15.116:0/29826010 shutdown_connections 2020-11-20T12:19:12.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.756+0000 7fd832a7d700 1 -- 172.21.15.116:0/29826010 wait complete. 2020-11-20T12:19:12.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd832a7d700 1 Processor -- start 2020-11-20T12:19:12.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd832a7d700 1 -- start start 2020-11-20T12:19:12.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd832a7d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c150770 0x7fd82c150b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:12.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd832a7d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c153650 0x7fd82c14fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:12.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd832a7d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd82c164d00 0x7fd82c165140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:12.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd832a7d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c153ad0 con 0x7fd82c153650 2020-11-20T12:19:12.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd832a7d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c1691d0 con 0x7fd82c164d00 2020-11-20T12:19:12.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd832a7d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd82c169470 con 0x7fd82c150770 2020-11-20T12:19:12.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd83127a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c153650 0x7fd82c14fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:12.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd83127a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c153650 0x7fd82c14fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47710/0 (socket says 172.21.15.116:47710) 2020-11-20T12:19:12.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd83127a700 1 -- 172.21.15.116:0/2973473808 learned_addr learned my addr 172.21.15.116:0/2973473808 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:12.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd831a7b700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c150770 0x7fd82c150b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:12.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd83227c700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd82c164d00 0x7fd82c165140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:12.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd83127a700 1 -- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c150770 msgr2=0x7fd82c150b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd83127a700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c150770 0x7fd82c150b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd83127a700 1 -- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd82c164d00 msgr2=0x7fd82c165140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd83127a700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd82c164d00 0x7fd82c165140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd83127a700 1 -- 172.21.15.116:0/2973473808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd82c169710 con 0x7fd82c153650 2020-11-20T12:19:12.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd83227c700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd82c164d00 0x7fd82c165140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:12.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd83127a700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c153650 0x7fd82c14fbe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fd81c00d760 tx=0x7fd81c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:12.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd819ffb700 1 -- 172.21.15.116:0/2973473808 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c00b640 con 0x7fd82c153650 2020-11-20T12:19:12.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd819ffb700 1 -- 172.21.15.116:0/2973473808 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81c00b7a0 con 0x7fd82c153650 2020-11-20T12:19:12.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.760+0000 7fd832a7d700 1 -- 172.21.15.116:0/2973473808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82c1699b0 con 0x7fd82c153650 2020-11-20T12:19:12.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.764+0000 7fd819ffb700 1 -- 172.21.15.116:0/2973473808 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81c00bb20 con 0x7fd82c153650 2020-11-20T12:19:12.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.764+0000 7fd832a7d700 1 -- 172.21.15.116:0/2973473808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82c165630 con 0x7fd82c153650 2020-11-20T12:19:12.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.764+0000 7fd832a7d700 1 -- 172.21.15.116:0/2973473808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd82c05cf30 con 0x7fd82c153650 2020-11-20T12:19:12.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.764+0000 7fd819ffb700 1 -- 172.21.15.116:0/2973473808 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd81c01c460 con 0x7fd82c153650 2020-11-20T12:19:12.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.764+0000 7fd819ffb700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8040557c0 0x7fd804057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:12.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.764+0000 7fd819ffb700 1 -- 172.21.15.116:0/2973473808 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fd81c06e890 con 0x7fd82c153650 2020-11-20T12:19:12.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.768+0000 7fd831a7b700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8040557c0 0x7fd804057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:12.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.768+0000 7fd831a7b700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8040557c0 0x7fd804057c80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fd82c05f2f0 tx=0x7fd828019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:12.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.772+0000 7fd819ffb700 1 -- 172.21.15.116:0/2973473808 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd81c044630 con 0x7fd82c153650 2020-11-20T12:19:12.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.976+0000 7fd832a7d700 1 -- 172.21.15.116:0/2973473808 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fd82c062300 con 0x7fd82c153650 2020-11-20T12:19:12.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.988+0000 7fd819ffb700 1 -- 172.21.15.116:0/2973473808 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 12 and data pool 13 v34) v1 ==== 189+0+0 (secure 0 0 0) 0x7fd81c047c50 con 0x7fd82c153650 2020-11-20T12:19:12.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.992+0000 7fd832a7d700 1 -- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8040557c0 msgr2=0x7fd804057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.992+0000 7fd832a7d700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8040557c0 0x7fd804057c80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7fd82c05f2f0 tx=0x7fd828019000).stop 2020-11-20T12:19:12.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.992+0000 7fd832a7d700 1 -- 172.21.15.116:0/2973473808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c153650 msgr2=0x7fd82c14fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:12.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.992+0000 7fd832a7d700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c153650 0x7fd82c14fbe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7fd81c00d760 tx=0x7fd81c009040).stop 2020-11-20T12:19:12.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.992+0000 7fd832a7d700 1 -- 172.21.15.116:0/2973473808 shutdown_connections 2020-11-20T12:19:12.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.992+0000 7fd832a7d700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8040557c0 0x7fd804057c80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.992+0000 7fd832a7d700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd82c164d00 0x7fd82c165140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.992+0000 7fd832a7d700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd82c150770 0x7fd82c150b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.992+0000 7fd832a7d700 1 --2- 172.21.15.116:0/2973473808 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd82c153650 0x7fd82c14fbe0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:12.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.992+0000 7fd832a7d700 1 -- 172.21.15.116:0/2973473808 >> 172.21.15.116:0/2973473808 conn(0x7fd82c0682e0 msgr2=0x7fd82c156280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:12.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.996+0000 7fd832a7d700 1 -- 172.21.15.116:0/2973473808 shutdown_connections 2020-11-20T12:19:12.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:12.996+0000 7fd832a7d700 1 -- 172.21.15.116:0/2973473808 wait complete. 2020-11-20T12:19:13.004 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 12 and data pool 13 2020-11-20T12:19:13.026 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:13.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.192+0000 7f6bc8cac700 1 Processor -- start 2020-11-20T12:19:13.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.192+0000 7f6bc8cac700 1 -- start start 2020-11-20T12:19:13.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.192+0000 7f6bc8cac700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6bc4062880 0x7f6bc4062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.192+0000 7f6bc8cac700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc4065d40 0x7f6bc40bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.192+0000 7f6bc8cac700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bc40c0530 con 0x7f6bc40677a0 2020-11-20T12:19:13.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.192+0000 7f6bc8cac700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bc40661a0 con 0x7f6bc4062880 2020-11-20T12:19:13.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.192+0000 7f6bc8cac700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc40c0310 con 0x7f6bc4065d40 2020-11-20T12:19:13.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bbbfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc4065d40 0x7f6bc40bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc37fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6bc4062880 0x7f6bc4062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bbbfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc4065d40 0x7f6bc40bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:56678/0 (socket says 172.21.15.116:56678) 2020-11-20T12:19:13.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bbbfff700 1 -- 172.21.15.116:0/2894825102 learned_addr learned my addr 172.21.15.116:0/2894825102 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:13.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc3fff700 1 --1- 172.21.15.116:0/2894825102 >> v1:172.21.15.173:6789/0 conn(0x7f6bc40677a0 0x7f6bc41556e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:53388/0 (socket says 172.21.15.116:53388) 2020-11-20T12:19:13.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bbbfff700 1 -- 172.21.15.116:0/2894825102 >> v2:172.21.15.173:3300/0 conn(0x7f6bc4062880 msgr2=0x7f6bc4062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:13.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bbbfff700 1 --2- 172.21.15.116:0/2894825102 >> v2:172.21.15.173:3300/0 conn(0x7f6bc4062880 0x7f6bc4062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bbbfff700 1 -- 172.21.15.116:0/2894825102 >> v1:172.21.15.173:6789/0 conn(0x7f6bc40677a0 legacy=0x7f6bc41556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bbbfff700 1 -- 172.21.15.116:0/2894825102 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc40c07d0 con 0x7f6bc4065d40 2020-11-20T12:19:13.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bbbfff700 1 --2- 172.21.15.116:0/2894825102 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc4065d40 0x7f6bc40bfe20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f6bac006fd0 tx=0x7f6bac006eb0).ready entity=mon.2 client_cookie=219cb34267073cda server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:13.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc2ffd700 1 -- 172.21.15.116:0/2894825102 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac009810 con 0x7f6bc4065d40 2020-11-20T12:19:13.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc2ffd700 1 -- 172.21.15.116:0/2894825102 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bac009970 con 0x7f6bc4065d40 2020-11-20T12:19:13.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc2ffd700 1 -- 172.21.15.116:0/2894825102 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bac009cf0 con 0x7f6bc4065d40 2020-11-20T12:19:13.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc8cac700 1 -- 172.21.15.116:0/2894825102 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc4065d40 msgr2=0x7f6bc40bfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc8cac700 1 --2- 172.21.15.116:0/2894825102 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc4065d40 0x7f6bc40bfe20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f6bac006fd0 tx=0x7f6bac006eb0).stop 2020-11-20T12:19:13.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc8cac700 1 -- 172.21.15.116:0/2894825102 shutdown_connections 2020-11-20T12:19:13.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc8cac700 1 --2- 172.21.15.116:0/2894825102 >> v2:172.21.15.173:3300/0 conn(0x7f6bc4062880 0x7f6bc4062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc8cac700 1 --2- 172.21.15.116:0/2894825102 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc4065d40 0x7f6bc40bfe20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc8cac700 1 -- 172.21.15.116:0/2894825102 >> 172.21.15.116:0/2894825102 conn(0x7f6bc4060c40 msgr2=0x7f6bc40c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:13.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc8cac700 1 -- 172.21.15.116:0/2894825102 shutdown_connections 2020-11-20T12:19:13.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.196+0000 7f6bc8cac700 1 -- 172.21.15.116:0/2894825102 wait complete. 2020-11-20T12:19:13.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.200+0000 7f6bc8cac700 1 Processor -- start 2020-11-20T12:19:13.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.200+0000 7f6bc8cac700 1 -- start start 2020-11-20T12:19:13.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.200+0000 7f6bc8cac700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4062880 0x7f6bc4169590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.200+0000 7f6bc8cac700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc40677a0 0x7f6bc4165490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.200+0000 7f6bc8cac700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bc40bdb10 0x7f6bc4152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.200+0000 7f6bc8cac700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc4165cc0 con 0x7f6bc40bdb10 2020-11-20T12:19:13.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.200+0000 7f6bc8cac700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc4165f60 con 0x7f6bc4062880 2020-11-20T12:19:13.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.200+0000 7f6bc8cac700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bc4166200 con 0x7f6bc40677a0 2020-11-20T12:19:13.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.200+0000 7f6bc3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bc40bdb10 0x7f6bc4152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bc37fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4062880 0x7f6bc4169590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bbbfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc40677a0 0x7f6bc4165490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bc3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bc40bdb10 0x7f6bc4152260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47730/0 (socket says 172.21.15.116:47730) 2020-11-20T12:19:13.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bc3fff700 1 -- 172.21.15.116:0/543038564 learned_addr learned my addr 172.21.15.116:0/543038564 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:13.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bc3fff700 1 -- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc40677a0 msgr2=0x7f6bc4165490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bc3fff700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc40677a0 0x7f6bc4165490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bc3fff700 1 -- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4062880 msgr2=0x7f6bc4169590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bc3fff700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4062880 0x7f6bc4169590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bc3fff700 1 -- 172.21.15.116:0/543038564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc40a74a0 con 0x7f6bc40bdb10 2020-11-20T12:19:13.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bbbfff700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc40677a0 0x7f6bc4165490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:13.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bc3fff700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bc40bdb10 0x7f6bc4152260 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f6bbc00ca60 tx=0x7f6bbc007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bbb7fe700 1 -- 172.21.15.116:0/543038564 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc005ac0 con 0x7f6bc40bdb10 2020-11-20T12:19:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bc8cac700 1 -- 172.21.15.116:0/543038564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bc4165980 con 0x7f6bc40bdb10 2020-11-20T12:19:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bbb7fe700 1 -- 172.21.15.116:0/543038564 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bbc005c20 con 0x7f6bc40bdb10 2020-11-20T12:19:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bbb7fe700 1 -- 172.21.15.116:0/543038564 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc007110 con 0x7f6bc40bdb10 2020-11-20T12:19:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.204+0000 7f6bc8cac700 1 -- 172.21.15.116:0/543038564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bc40be190 con 0x7f6bc40bdb10 2020-11-20T12:19:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.208+0000 7f6bc8cac700 1 -- 172.21.15.116:0/543038564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bc41663c0 con 0x7f6bc40bdb10 2020-11-20T12:19:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.208+0000 7f6bbb7fe700 1 -- 172.21.15.116:0/543038564 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6bbc010030 con 0x7f6bc40bdb10 2020-11-20T12:19:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.208+0000 7f6bbb7fe700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ba00557c0 0x7f6ba0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.208+0000 7f6bbb7fe700 1 -- 172.21.15.116:0/543038564 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6bbc06e850 con 0x7f6bc40bdb10 2020-11-20T12:19:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.208+0000 7f6bc37fe700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ba00557c0 0x7f6ba0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.212+0000 7f6bc37fe700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ba00557c0 0x7f6ba0057c80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f6bc40668b0 tx=0x7f6bb4006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.216+0000 7f6bbb7fe700 1 -- 172.21.15.116:0/543038564 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6bbc044660 con 0x7f6bc40bdb10 2020-11-20T12:19:13.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.380+0000 7f6bc8cac700 1 -- 172.21.15.116:0/543038564 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6bc40a7630 con 0x7f6bc40bdb10 2020-11-20T12:19:13.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.380+0000 7f6bbb7fe700 1 -- 172.21.15.116:0/543038564 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16890 (secure 0 0 0) 0x7f6bbc047c80 con 0x7f6bc40bdb10 2020-11-20T12:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:13.379 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":70,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:12.981350+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:11.306087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:11.966077+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:13.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 -- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ba00557c0 msgr2=0x7f6ba0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ba00557c0 0x7f6ba0057c80 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f6bc40668b0 tx=0x7f6bb4006df0).stop 2020-11-20T12:19:13.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 -- 172.21.15.116:0/543038564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bc40bdb10 msgr2=0x7f6bc4152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bc40bdb10 0x7f6bc4152260 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f6bbc00ca60 tx=0x7f6bbc007f40).stop 2020-11-20T12:19:13.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 -- 172.21.15.116:0/543038564 shutdown_connections 2020-11-20T12:19:13.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ba00557c0 0x7f6ba0057c80 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6bc40bdb10 0x7f6bc4152260 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6bc4062880 0x7f6bc4169590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 --2- 172.21.15.116:0/543038564 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6bc40677a0 0x7f6bc4165490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 -- 172.21.15.116:0/543038564 >> 172.21.15.116:0/543038564 conn(0x7f6bc4060c40 msgr2=0x7f6bc40c3190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:13.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 -- 172.21.15.116:0/543038564 shutdown_connections 2020-11-20T12:19:13.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.384+0000 7f6bc8cac700 1 -- 172.21.15.116:0/543038564 wait complete. 2020-11-20T12:19:13.405 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:13.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.580+0000 7fe89b59e700 1 Processor -- start 2020-11-20T12:19:13.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.580+0000 7fe89b59e700 1 -- start start 2020-11-20T12:19:13.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.580+0000 7fe89b59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe89c0618e0 0x7fe89c061cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.580+0000 7fe89b59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe89c0adac0 0x7fe89c0a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.580+0000 7fe89b59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89c0ace80 con 0x7fe89c0adf50 2020-11-20T12:19:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.580+0000 7fe89b59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89c152b20 con 0x7fe89c0618e0 2020-11-20T12:19:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.580+0000 7fe89b59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89c0ad0e0 con 0x7fe89c0adac0 2020-11-20T12:19:13.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe89c0618e0 0x7fe89c061cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe89c0618e0 0x7fe89c061cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47738/0 (socket says 172.21.15.116:47738) 2020-11-20T12:19:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe899d9b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe89c0adac0 0x7fe89c0a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89a59c700 1 -- 172.21.15.116:0/4175815750 learned_addr learned my addr 172.21.15.116:0/4175815750 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:13.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89a59c700 1 -- 172.21.15.116:0/4175815750 >> v2:172.21.15.173:3300/0 conn(0x7fe89c0adac0 msgr2=0x7fe89c0a83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89a59c700 1 --2- 172.21.15.116:0/4175815750 >> v2:172.21.15.173:3300/0 conn(0x7fe89c0adac0 0x7fe89c0a83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89a59c700 1 -- 172.21.15.116:0/4175815750 >> v1:172.21.15.173:6789/0 conn(0x7fe89c0adf50 legacy=0x7fe89c0aa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89a59c700 1 -- 172.21.15.116:0/4175815750 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c0ad380 con 0x7fe89c0618e0 2020-11-20T12:19:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89a59c700 1 --2- 172.21.15.116:0/4175815750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe89c0618e0 0x7fe89c061cc0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fe88c006fd0 tx=0x7fe88c006eb0).ready entity=mon.0 client_cookie=fd1473671efb2827 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:13.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89959a700 1 -- 172.21.15.116:0/4175815750 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c0097d0 con 0x7fe89c0618e0 2020-11-20T12:19:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89959a700 1 -- 172.21.15.116:0/4175815750 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe88c009930 con 0x7fe89c0618e0 2020-11-20T12:19:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89959a700 1 -- 172.21.15.116:0/4175815750 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe88c009cb0 con 0x7fe89c0618e0 2020-11-20T12:19:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89b59e700 1 -- 172.21.15.116:0/4175815750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe89c0618e0 msgr2=0x7fe89c061cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.584+0000 7fe89b59e700 1 --2- 172.21.15.116:0/4175815750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe89c0618e0 0x7fe89c061cc0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fe88c006fd0 tx=0x7fe88c006eb0).stop 2020-11-20T12:19:13.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.588+0000 7fe89b59e700 1 -- 172.21.15.116:0/4175815750 shutdown_connections 2020-11-20T12:19:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.588+0000 7fe89b59e700 1 --2- 172.21.15.116:0/4175815750 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe89c0618e0 0x7fe89c061cc0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.588+0000 7fe89b59e700 1 --2- 172.21.15.116:0/4175815750 >> v2:172.21.15.173:3300/0 conn(0x7fe89c0adac0 0x7fe89c0a83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.588+0000 7fe89b59e700 1 -- 172.21.15.116:0/4175815750 >> 172.21.15.116:0/4175815750 conn(0x7fe89c0678a0 msgr2=0x7fe89c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:13.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.588+0000 7fe89b59e700 1 -- 172.21.15.116:0/4175815750 shutdown_connections 2020-11-20T12:19:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.588+0000 7fe89b59e700 1 -- 172.21.15.116:0/4175815750 wait complete. 2020-11-20T12:19:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.588+0000 7fe89b59e700 1 Processor -- start 2020-11-20T12:19:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.588+0000 7fe89b59e700 1 -- start start 2020-11-20T12:19:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe89b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe89c0618e0 0x7fe89c161f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe89b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe89c0adac0 0x7fe89c14c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe89b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe89c0adf50 0x7fe89c14ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe89b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c162470 con 0x7fe89c0adac0 2020-11-20T12:19:13.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe89b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c14c9f0 con 0x7fe89c0618e0 2020-11-20T12:19:13.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe89b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0a6ec0 con 0x7fe89c0adf50 2020-11-20T12:19:13.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe899d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe89c0adac0 0x7fe89c14c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe899d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe89c0adac0 0x7fe89c14c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47746/0 (socket says 172.21.15.116:47746) 2020-11-20T12:19:13.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe899d9b700 1 -- 172.21.15.116:0/1968398692 learned_addr learned my addr 172.21.15.116:0/1968398692 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:13.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe89a59c700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe89c0618e0 0x7fe89c161f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe89ad9d700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe89c0adf50 0x7fe89c14ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe899d9b700 1 -- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe89c0adf50 msgr2=0x7fe89c14ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe899d9b700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe89c0adf50 0x7fe89c14ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe899d9b700 1 -- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe89c0618e0 msgr2=0x7fe89c161f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe899d9b700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe89c0618e0 0x7fe89c161f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe899d9b700 1 -- 172.21.15.116:0/1968398692 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c0c6a60 con 0x7fe89c0adac0 2020-11-20T12:19:13.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe89ad9d700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe89c0adf50 0x7fe89c14ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:13.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe899d9b700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe89c0adac0 0x7fe89c14c500 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fe89000eba0 tx=0x7fe890006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:13.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe88a7fc700 1 -- 172.21.15.116:0/1968398692 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89000c9b0 con 0x7fe89c0adac0 2020-11-20T12:19:13.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe88a7fc700 1 -- 172.21.15.116:0/1968398692 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89000cb10 con 0x7fe89c0adac0 2020-11-20T12:19:13.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe89b59e700 1 -- 172.21.15.116:0/1968398692 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe89c0c6d00 con 0x7fe89c0adac0 2020-11-20T12:19:13.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe88a7fc700 1 -- 172.21.15.116:0/1968398692 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89000ce90 con 0x7fe89c0adac0 2020-11-20T12:19:13.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.592+0000 7fe89b59e700 1 -- 172.21.15.116:0/1968398692 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c156450 con 0x7fe89c0adac0 2020-11-20T12:19:13.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.596+0000 7fe89b59e700 1 -- 172.21.15.116:0/1968398692 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89c05cf30 con 0x7fe89c0adac0 2020-11-20T12:19:13.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.596+0000 7fe88a7fc700 1 -- 172.21.15.116:0/1968398692 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe890012600 con 0x7fe89c0adac0 2020-11-20T12:19:13.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.596+0000 7fe88a7fc700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe8700557c0 0x7fe870057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:13.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.596+0000 7fe88a7fc700 1 -- 172.21.15.116:0/1968398692 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe89006eb50 con 0x7fe89c0adac0 2020-11-20T12:19:13.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.596+0000 7fe89a59c700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe8700557c0 0x7fe870057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:13.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.600+0000 7fe89a59c700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe8700557c0 0x7fe870057c80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fe88c00b1b0 tx=0x7fe88c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:13.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.604+0000 7fe88a7fc700 1 -- 172.21.15.116:0/1968398692 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe8900449f0 con 0x7fe89c0adac0 2020-11-20T12:19:13.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.768+0000 7fe89b59e700 1 -- 172.21.15.116:0/1968398692 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe89c0abac0 con 0x7fe89c0adac0 2020-11-20T12:19:13.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.768+0000 7fe88a7fc700 1 -- 172.21.15.116:0/1968398692 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) v1 ==== 74+0+16890 (secure 0 0 0) 0x7fe890048010 con 0x7fe89c0adac0 2020-11-20T12:19:13.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 -- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe8700557c0 msgr2=0x7fe870057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe8700557c0 0x7fe870057c80 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fe88c00b1b0 tx=0x7fe88c015000).stop 2020-11-20T12:19:13.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 -- 172.21.15.116:0/1968398692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe89c0adac0 msgr2=0x7fe89c14c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:13.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe89c0adac0 0x7fe89c14c500 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7fe89000eba0 tx=0x7fe890006df0).stop 2020-11-20T12:19:13.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 -- 172.21.15.116:0/1968398692 shutdown_connections 2020-11-20T12:19:13.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe8700557c0 0x7fe870057c80 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe89c0adf50 0x7fe89c14ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe89c0618e0 0x7fe89c161f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 --2- 172.21.15.116:0/1968398692 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe89c0adac0 0x7fe89c14c500 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:13.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 -- 172.21.15.116:0/1968398692 >> 172.21.15.116:0/1968398692 conn(0x7fe89c0678a0 msgr2=0x7fe89c14ddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:13.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 -- 172.21.15.116:0/1968398692 shutdown_connections 2020-11-20T12:19:13.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:13.772+0000 7fe89b59e700 1 -- 172.21.15.116:0/1968398692 wait complete. 2020-11-20T12:19:13.945 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:13.946 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":70,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:12.981350+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:11.306087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:11.966077+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:13.946 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-20T12:19:14.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.136+0000 7fd3b777e700 1 Processor -- start 2020-11-20T12:19:14.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.136+0000 7fd3b777e700 1 -- start start 2020-11-20T12:19:14.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd3b777e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd3b0063dc0 0x7fd3b00ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:14.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd3b777e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b0152aa0 con 0x7fd3b0064280 2020-11-20T12:19:14.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd3b777e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b0152d40 con 0x7fd3b014c1b0 2020-11-20T12:19:14.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd3b777e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b00ba370 con 0x7fd3b0063dc0 2020-11-20T12:19:14.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd3b5d1b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd3b0064280 0x7fd3b00bf5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57190/0 (socket says 172.21.15.116:57190) 2020-11-20T12:19:14.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd3b5d1b700 1 -- 172.21.15.116:0/73596869 learned_addr learned my addr 172.21.15.116:0/73596869 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:14.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd3b4d19700 1 --2- 172.21.15.116:0/73596869 >> v2:172.21.15.173:3300/0 conn(0x7fd3b0063dc0 0x7fd3b00ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:14.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266062017 0 0) 0x7fd3b0152aa0 con 0x7fd3b0064280 2020-11-20T12:19:14.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd398003240 con 0x7fd3b0064280 2020-11-20T12:19:14.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 237576055 0 0) 0x7fd398003240 con 0x7fd3b0064280 2020-11-20T12:19:14.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 >> v2:172.21.15.173:3300/0 conn(0x7fd3b0063dc0 msgr2=0x7fd3b00ba8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:14.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 --2- 172.21.15.116:0/73596869 >> v2:172.21.15.173:3300/0 conn(0x7fd3b0063dc0 0x7fd3b00ba8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:14.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 >> v1:172.21.15.173:6789/0 conn(0x7fd3b014c1b0 legacy=0x7fd3b00636b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:14.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b00b9e30 con 0x7fd3b0064280 2020-11-20T12:19:14.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fd3a8001860 con 0x7fd3b0064280 2020-11-20T12:19:14.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 >> v1:172.21.15.116:6789/0 conn(0x7fd3b0064280 legacy=0x7fd3b00bf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:14.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 --2- 172.21.15.116:0/73596869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd398003780 0x7fd398003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:14.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 --2- 172.21.15.116:0/73596869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3980044e0 0x7fd398004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:14.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 --2- 172.21.15.116:0/73596869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd398004e50 0x7fd39800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:14.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b00ba370 con 0x7fd398004e50 2020-11-20T12:19:14.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3980053b0 con 0x7fd3980044e0 2020-11-20T12:19:14.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd39800bca0 con 0x7fd398003780 2020-11-20T12:19:14.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd3b4d19700 1 --2- 172.21.15.116:0/73596869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3980044e0 0x7fd398004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:14.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd3b551a700 1 --2- 172.21.15.116:0/73596869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd398003780 0x7fd398003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:14.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.140+0000 7fd3b5d1b700 1 --2- 172.21.15.116:0/73596869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd398004e50 0x7fd39800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:14.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b551a700 1 -- 172.21.15.116:0/73596869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3980044e0 msgr2=0x7fd398004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:14.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b551a700 1 --2- 172.21.15.116:0/73596869 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3980044e0 0x7fd398004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:14.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b551a700 1 -- 172.21.15.116:0/73596869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd398004e50 msgr2=0x7fd39800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:14.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b551a700 1 --2- 172.21.15.116:0/73596869 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd398004e50 0x7fd39800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:14.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b551a700 1 -- 172.21.15.116:0/73596869 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd398003240 con 0x7fd398003780 2020-11-20T12:19:14.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b5d1b700 1 -- 172.21.15.116:0/73596869 reap_dead start 2020-11-20T12:19:14.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b551a700 1 --2- 172.21.15.116:0/73596869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd398003780 0x7fd398003ba0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fd3ac001920 tx=0x7fd3ac007b90).ready entity=mon.2 client_cookie=83eabc654044770d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:14.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac007020 con 0x7fd398003780 2020-11-20T12:19:14.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac007180 con 0x7fd398003780 2020-11-20T12:19:14.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd39ffff700 1 -- 172.21.15.116:0/73596869 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac00c830 con 0x7fd398003780 2020-11-20T12:19:14.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b777e700 1 -- 172.21.15.116:0/73596869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd398003780 msgr2=0x7fd398003ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:14.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b777e700 1 --2- 172.21.15.116:0/73596869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd398003780 0x7fd398003ba0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fd3ac001920 tx=0x7fd3ac007b90).stop 2020-11-20T12:19:14.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b777e700 1 -- 172.21.15.116:0/73596869 shutdown_connections 2020-11-20T12:19:14.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b777e700 1 --2- 172.21.15.116:0/73596869 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd398003780 0x7fd398003ba0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:14.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b777e700 1 -- 172.21.15.116:0/73596869 >> 172.21.15.116:0/73596869 conn(0x7fd3b00c5820 msgr2=0x7fd3b00c17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:14.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b777e700 1 -- 172.21.15.116:0/73596869 shutdown_connections 2020-11-20T12:19:14.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.144+0000 7fd3b777e700 1 -- 172.21.15.116:0/73596869 wait complete. 2020-11-20T12:19:14.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.148+0000 7fd3b777e700 1 Processor -- start 2020-11-20T12:19:14.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.148+0000 7fd3b777e700 1 -- start start 2020-11-20T12:19:14.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.148+0000 7fd3b777e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd398003780 0x7fd3b0161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:14.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.148+0000 7fd3b777e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b00bcda0 0x7fd3b014ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:14.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.148+0000 7fd3b777e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b00631d0 0x7fd3b0063650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:14.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.148+0000 7fd3b777e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b00be440 con 0x7fd3b00bcda0 2020-11-20T12:19:14.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.148+0000 7fd3b777e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b00bd520 con 0x7fd3b00631d0 2020-11-20T12:19:14.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.148+0000 7fd3b777e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b00bd7c0 con 0x7fd398003780 2020-11-20T12:19:14.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b4d19700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b00bcda0 0x7fd3b014ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:14.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b4d19700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b00bcda0 0x7fd3b014ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47766/0 (socket says 172.21.15.116:47766) 2020-11-20T12:19:14.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b4d19700 1 -- 172.21.15.116:0/863460249 learned_addr learned my addr 172.21.15.116:0/863460249 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:14.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b551a700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd398003780 0x7fd3b0161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:14.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b5d1b700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b00631d0 0x7fd3b0063650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:14.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b4d19700 1 -- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd398003780 msgr2=0x7fd3b0161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:14.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b4d19700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd398003780 0x7fd3b0161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:14.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b4d19700 1 -- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b00631d0 msgr2=0x7fd3b0063650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:14.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b4d19700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b00631d0 0x7fd3b0063650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:14.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b4d19700 1 -- 172.21.15.116:0/863460249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b00bda60 con 0x7fd3b00bcda0 2020-11-20T12:19:14.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b5d1b700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b00631d0 0x7fd3b0063650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:14.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b4d19700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b00bcda0 0x7fd3b014ab20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fd3a0011a10 tx=0x7fd3a0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:14.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd39d7fa700 1 -- 172.21.15.116:0/863460249 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000f910 con 0x7fd3b00bcda0 2020-11-20T12:19:14.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd39d7fa700 1 -- 172.21.15.116:0/863460249 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a000fa70 con 0x7fd3b00bcda0 2020-11-20T12:19:14.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b777e700 1 -- 172.21.15.116:0/863460249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3b00bdd00 con 0x7fd3b00bcda0 2020-11-20T12:19:14.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd39d7fa700 1 -- 172.21.15.116:0/863460249 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a000fdf0 con 0x7fd3b00bcda0 2020-11-20T12:19:14.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b777e700 1 -- 172.21.15.116:0/863460249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b014b010 con 0x7fd3b00bcda0 2020-11-20T12:19:14.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.152+0000 7fd3b777e700 1 -- 172.21.15.116:0/863460249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b0063b40 con 0x7fd3b00bcda0 2020-11-20T12:19:14.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.156+0000 7fd39d7fa700 1 -- 172.21.15.116:0/863460249 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd3a0014650 con 0x7fd3b00bcda0 2020-11-20T12:19:14.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.156+0000 7fd39d7fa700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 0x7fd388057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:14.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.156+0000 7fd39d7fa700 1 -- 172.21.15.116:0/863460249 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd3a006efe0 con 0x7fd3b00bcda0 2020-11-20T12:19:14.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.156+0000 7fd3b551a700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 0x7fd388057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:14.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.156+0000 7fd3b551a700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 0x7fd388057c80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fd3ac001920 tx=0x7fd3ac015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:14.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.160+0000 7fd39d7fa700 1 -- 172.21.15.116:0/863460249 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd3a0044db0 con 0x7fd3b00bcda0 2020-11-20T12:19:14.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:14.364+0000 7fd3b777e700 1 -- 172.21.15.116:0/863460249 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd3b0069950 con 0x7fd3b00bcda0 2020-11-20T12:19:15.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.004+0000 7fd39d7fa700 1 -- 172.21.15.116:0/863460249 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v37) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd3a0018070 con 0x7fd3b00bcda0 2020-11-20T12:19:15.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 -- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 msgr2=0x7fd388057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 0x7fd388057c80 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fd3ac001920 tx=0x7fd3ac015000).stop 2020-11-20T12:19:15.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 -- 172.21.15.116:0/863460249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b00bcda0 msgr2=0x7fd3b014ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b00bcda0 0x7fd3b014ab20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fd3a0011a10 tx=0x7fd3a0006df0).stop 2020-11-20T12:19:15.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 -- 172.21.15.116:0/863460249 shutdown_connections 2020-11-20T12:19:15.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd3880557c0 0x7fd388057c80 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd3b00631d0 0x7fd3b0063650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd398003780 0x7fd3b0161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 --2- 172.21.15.116:0/863460249 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd3b00bcda0 0x7fd3b014ab20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 -- 172.21.15.116:0/863460249 >> 172.21.15.116:0/863460249 conn(0x7fd3b00c5820 msgr2=0x7fd3b014b7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:15.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 -- 172.21.15.116:0/863460249 shutdown_connections 2020-11-20T12:19:15.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.008+0000 7fd3b777e700 1 -- 172.21.15.116:0/863460249 wait complete. 2020-11-20T12:19:15.032 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:19:15.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.204+0000 7fa58759e700 1 Processor -- start 2020-11-20T12:19:15.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.204+0000 7fa58759e700 1 -- start start 2020-11-20T12:19:15.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.204+0000 7fa58759e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa588062c60 0x7fa588065d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.204+0000 7fa58759e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5880c02b0 con 0x7fa5880bfe20 2020-11-20T12:19:15.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.204+0000 7fa58759e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5880c0530 con 0x7fa588062880 2020-11-20T12:19:15.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.204+0000 7fa58759e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5880c07d0 con 0x7fa588062c60 2020-11-20T12:19:15.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.204+0000 7fa586d9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa5880bfe20 0x7fa5881556e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57210/0 (socket says 172.21.15.116:57210) 2020-11-20T12:19:15.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.204+0000 7fa586d9d700 1 -- 172.21.15.116:0/817409383 learned_addr learned my addr 172.21.15.116:0/817409383 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:15.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.204+0000 7fa585d9b700 1 --2- 172.21.15.116:0/817409383 >> v2:172.21.15.173:3300/0 conn(0x7fa588062c60 0x7fa588065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.204+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750771076 0 0) 0x7fa5880c02b0 con 0x7fa5880bfe20 2020-11-20T12:19:15.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.204+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa56c003240 con 0x7fa5880bfe20 2020-11-20T12:19:15.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 691667757 0 0) 0x7fa56c003240 con 0x7fa5880bfe20 2020-11-20T12:19:15.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 >> v2:172.21.15.173:3300/0 conn(0x7fa588062c60 msgr2=0x7fa588065d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:15.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 --2- 172.21.15.116:0/817409383 >> v2:172.21.15.173:3300/0 conn(0x7fa588062c60 0x7fa588065d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 >> v1:172.21.15.173:6789/0 conn(0x7fa588062880 legacy=0x7fa5880677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5880c0a70 con 0x7fa5880bfe20 2020-11-20T12:19:15.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fa580001860 con 0x7fa5880bfe20 2020-11-20T12:19:15.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 >> v1:172.21.15.116:6789/0 conn(0x7fa5880bfe20 legacy=0x7fa5881556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa56c003730 0x7fa56c003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 0x7fa56c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa56c004e50 0x7fa56c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880c07d0 con 0x7fa56c0044e0 2020-11-20T12:19:15.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c0041d0 con 0x7fa56c003730 2020-11-20T12:19:15.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56c00bc20 con 0x7fa56c004e50 2020-11-20T12:19:15.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa585d9b700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 0x7fa56c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58659c700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa56c003730 0x7fa56c003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa586d9d700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa56c004e50 0x7fa56c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa585d9b700 1 -- 172.21.15.116:0/817409383 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa56c004e50 msgr2=0x7fa56c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:15.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa585d9b700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa56c004e50 0x7fa56c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa585d9b700 1 -- 172.21.15.116:0/817409383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa56c003730 msgr2=0x7fa56c003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:15.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa585d9b700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa56c003730 0x7fa56c003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa585d9b700 1 -- 172.21.15.116:0/817409383 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa56c003240 con 0x7fa56c0044e0 2020-11-20T12:19:15.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa586d9d700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa56c004e50 0x7fa56c00b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:15.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58659c700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa56c003730 0x7fa56c003b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:15.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa586d9d700 1 -- 172.21.15.116:0/817409383 reap_dead start 2020-11-20T12:19:15.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa585d9b700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 0x7fa56c004960 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fa57c00bc00 tx=0x7fa57c006df0).ready entity=mon.0 client_cookie=eec8c567c310cd7c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:15.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c00b310 con 0x7fa56c0044e0 2020-11-20T12:19:15.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57c00ea90 con 0x7fa56c0044e0 2020-11-20T12:19:15.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.208+0000 7fa58559a700 1 -- 172.21.15.116:0/817409383 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57c00ee10 con 0x7fa56c0044e0 2020-11-20T12:19:15.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.212+0000 7fa58759e700 1 -- 172.21.15.116:0/817409383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 msgr2=0x7fa56c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.212+0000 7fa58759e700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 0x7fa56c004960 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7fa57c00bc00 tx=0x7fa57c006df0).stop 2020-11-20T12:19:15.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.212+0000 7fa58759e700 1 -- 172.21.15.116:0/817409383 shutdown_connections 2020-11-20T12:19:15.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.212+0000 7fa58759e700 1 --2- 172.21.15.116:0/817409383 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 0x7fa56c004960 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.212+0000 7fa58759e700 1 -- 172.21.15.116:0/817409383 >> 172.21.15.116:0/817409383 conn(0x7fa588060c40 msgr2=0x7fa5880c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:15.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.212+0000 7fa58759e700 1 -- 172.21.15.116:0/817409383 shutdown_connections 2020-11-20T12:19:15.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.212+0000 7fa58759e700 1 -- 172.21.15.116:0/817409383 wait complete. 2020-11-20T12:19:15.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.212+0000 7fa58759e700 1 Processor -- start 2020-11-20T12:19:15.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.212+0000 7fa58759e700 1 -- start start 2020-11-20T12:19:15.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 0x7fa588146710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5880a7320 0x7fa588169090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5880a7fb0 0x7fa5880a8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880bb500 con 0x7fa56c0044e0 2020-11-20T12:19:15.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880bb7a0 con 0x7fa5880a7fb0 2020-11-20T12:19:15.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5880bba40 con 0x7fa5880a7320 2020-11-20T12:19:15.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 0x7fa588146710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 0x7fa588146710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47784/0 (socket says 172.21.15.116:47784) 2020-11-20T12:19:15.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58659c700 1 -- 172.21.15.116:0/3020458128 learned_addr learned my addr 172.21.15.116:0/3020458128 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:15.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa585d9b700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5880a7320 0x7fa588169090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa586d9d700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5880a7fb0 0x7fa5880a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58659c700 1 -- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5880a7320 msgr2=0x7fa588169090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58659c700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5880a7320 0x7fa588169090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58659c700 1 -- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5880a7fb0 msgr2=0x7fa5880a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58659c700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5880a7fb0 0x7fa5880a8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58659c700 1 -- 172.21.15.116:0/3020458128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5880bbce0 con 0x7fa56c0044e0 2020-11-20T12:19:15.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58659c700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 0x7fa588146710 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fa57800fb50 tx=0x7fa578009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:15.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa5767fc700 1 -- 172.21.15.116:0/3020458128 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57800d820 con 0x7fa56c0044e0 2020-11-20T12:19:15.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa5767fc700 1 -- 172.21.15.116:0/3020458128 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa57800d980 con 0x7fa56c0044e0 2020-11-20T12:19:15.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58759e700 1 -- 172.21.15.116:0/3020458128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa588146c00 con 0x7fa56c0044e0 2020-11-20T12:19:15.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa5767fc700 1 -- 172.21.15.116:0/3020458128 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57800dd00 con 0x7fa56c0044e0 2020-11-20T12:19:15.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.216+0000 7fa58759e700 1 -- 172.21.15.116:0/3020458128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa588165130 con 0x7fa56c0044e0 2020-11-20T12:19:15.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.220+0000 7fa58759e700 1 -- 172.21.15.116:0/3020458128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58805cf30 con 0x7fa56c0044e0 2020-11-20T12:19:15.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.220+0000 7fa5767fc700 1 -- 172.21.15.116:0/3020458128 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa57801a030 con 0x7fa56c0044e0 2020-11-20T12:19:15.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.220+0000 7fa5767fc700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa55c0557c0 0x7fa55c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.220+0000 7fa5767fc700 1 -- 172.21.15.116:0/3020458128 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa57806e000 con 0x7fa56c0044e0 2020-11-20T12:19:15.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.220+0000 7fa585d9b700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa55c0557c0 0x7fa55c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.224+0000 7fa585d9b700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa55c0557c0 0x7fa55c057c80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fa588168620 tx=0x7fa57c019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:15.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.224+0000 7fa5767fc700 1 -- 172.21.15.116:0/3020458128 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa578043f50 con 0x7fa56c0044e0 2020-11-20T12:19:15.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.432+0000 7fa58759e700 1 -- 172.21.15.116:0/3020458128 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa588061230 con 0x7fa56c0044e0 2020-11-20T12:19:15.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.432+0000 7fa5767fc700 1 -- 172.21.15.116:0/3020458128 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2854 (secure 0 0 0) 0x7fa578047570 con 0x7fa56c0044e0 2020-11-20T12:19:15.431 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:15.431 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3869489418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3869489418},{"type":"v1","addr":"172.21.15.173:6837","nonce":3869489418}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/2912073472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2912073472},{"type":"v1","addr":"172.21.15.173:6835","nonce":2912073472}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/2221764249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2221764249},{"type":"v1","addr":"172.21.15.116:6834","nonce":2221764249}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:12.981713+0000","modified":"2020-11-20T12:19:14.999541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1946956517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1946956517},{"type":"v1","addr":"172.21.15.116:6835","nonce":1946956517}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-20T12:19:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 -- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa55c0557c0 msgr2=0x7fa55c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa55c0557c0 0x7fa55c057c80 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fa588168620 tx=0x7fa57c019000).stop 2020-11-20T12:19:15.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 -- 172.21.15.116:0/3020458128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 msgr2=0x7fa588146710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 0x7fa588146710 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7fa57800fb50 tx=0x7fa578009040).stop 2020-11-20T12:19:15.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 -- 172.21.15.116:0/3020458128 shutdown_connections 2020-11-20T12:19:15.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa55c0557c0 0x7fa55c057c80 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5880a7fb0 0x7fa5880a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa56c0044e0 0x7fa588146710 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 --2- 172.21.15.116:0/3020458128 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5880a7320 0x7fa588169090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 -- 172.21.15.116:0/3020458128 >> 172.21.15.116:0/3020458128 conn(0x7fa588060c40 msgr2=0x7fa5880c4280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:15.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 -- 172.21.15.116:0/3020458128 shutdown_connections 2020-11-20T12:19:15.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.436+0000 7fa58759e700 1 -- 172.21.15.116:0/3020458128 wait complete. 2020-11-20T12:19:15.436 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 37 2020-11-20T12:19:15.456 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:15.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.644+0000 7f61c990f700 1 Processor -- start 2020-11-20T12:19:15.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.644+0000 7f61c990f700 1 -- start start 2020-11-20T12:19:15.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c990f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f61c414bfa0 0x7f61c40a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c990f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f61c40a6860 0x7f61c40a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c990f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c414f0b0 con 0x7f61c414eb30 2020-11-20T12:19:15.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c990f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c414f350 con 0x7f61c40a6860 2020-11-20T12:19:15.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c990f700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61c414f5f0 con 0x7f61c414bfa0 2020-11-20T12:19:15.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c3fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f61c40a6860 0x7f61c40a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c3fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f61c40a6860 0x7f61c40a8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47796/0 (socket says 172.21.15.116:47796) 2020-11-20T12:19:15.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c3fff700 1 -- 172.21.15.116:0/261002846 learned_addr learned my addr 172.21.15.116:0/261002846 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:15.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c2ffd700 1 --2- 172.21.15.116:0/261002846 >> v2:172.21.15.173:3300/0 conn(0x7f61c414bfa0 0x7f61c40a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c3fff700 1 -- 172.21.15.116:0/261002846 >> v2:172.21.15.173:3300/0 conn(0x7f61c414bfa0 msgr2=0x7f61c40a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:15.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c3fff700 1 --2- 172.21.15.116:0/261002846 >> v2:172.21.15.173:3300/0 conn(0x7f61c414bfa0 0x7f61c40a89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c3fff700 1 -- 172.21.15.116:0/261002846 >> v1:172.21.15.173:6789/0 conn(0x7f61c414eb30 legacy=0x7f61c4152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c3fff700 1 -- 172.21.15.116:0/261002846 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c414f890 con 0x7f61c40a6860 2020-11-20T12:19:15.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c3fff700 1 --2- 172.21.15.116:0/261002846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c40a6860 0x7f61c40a8e90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f61c4063e80 tx=0x7f61bc009290).ready entity=mon.0 client_cookie=9b09e910e3237c7d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:15.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c1ffb700 1 -- 172.21.15.116:0/261002846 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00bbc0 con 0x7f61c40a6860 2020-11-20T12:19:15.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c1ffb700 1 -- 172.21.15.116:0/261002846 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc00bd20 con 0x7f61c40a6860 2020-11-20T12:19:15.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c1ffb700 1 -- 172.21.15.116:0/261002846 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc011610 con 0x7f61c40a6860 2020-11-20T12:19:15.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c990f700 1 -- 172.21.15.116:0/261002846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c40a6860 msgr2=0x7f61c40a8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c990f700 1 --2- 172.21.15.116:0/261002846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c40a6860 0x7f61c40a8e90 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f61c4063e80 tx=0x7f61bc009290).stop 2020-11-20T12:19:15.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c990f700 1 -- 172.21.15.116:0/261002846 shutdown_connections 2020-11-20T12:19:15.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c990f700 1 --2- 172.21.15.116:0/261002846 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c40a6860 0x7f61c40a8e90 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c990f700 1 --2- 172.21.15.116:0/261002846 >> v2:172.21.15.173:3300/0 conn(0x7f61c414bfa0 0x7f61c40a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.648+0000 7f61c990f700 1 -- 172.21.15.116:0/261002846 >> 172.21.15.116:0/261002846 conn(0x7f61c40c63f0 msgr2=0x7f61c40c23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:15.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c990f700 1 -- 172.21.15.116:0/261002846 shutdown_connections 2020-11-20T12:19:15.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c990f700 1 -- 172.21.15.116:0/261002846 wait complete. 2020-11-20T12:19:15.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c990f700 1 Processor -- start 2020-11-20T12:19:15.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c990f700 1 -- start start 2020-11-20T12:19:15.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c990f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61c40a6860 0x7f61c4149250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c990f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c414bfa0 0x7f61c414fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c990f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c414eb30 0x7f61c415ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c990f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c41504d0 con 0x7f61c414eb30 2020-11-20T12:19:15.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c990f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4150770 con 0x7f61c414bfa0 2020-11-20T12:19:15.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c990f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4150a10 con 0x7f61c40a6860 2020-11-20T12:19:15.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c414eb30 0x7f61c415ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c414eb30 0x7f61c415ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47802/0 (socket says 172.21.15.116:47802) 2020-11-20T12:19:15.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.652+0000 7f61c3fff700 1 -- 172.21.15.116:0/2125144277 learned_addr learned my addr 172.21.15.116:0/2125144277 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:15.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61c27fc700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c414bfa0 0x7f61c414fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61c2ffd700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61c40a6860 0x7f61c4149250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61c3fff700 1 -- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61c40a6860 msgr2=0x7f61c4149250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61c3fff700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61c40a6860 0x7f61c4149250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61c3fff700 1 -- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c414bfa0 msgr2=0x7f61c414fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61c3fff700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c414bfa0 0x7f61c414fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61c3fff700 1 -- 172.21.15.116:0/2125144277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4151df0 con 0x7f61c414eb30 2020-11-20T12:19:15.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61c3fff700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c414eb30 0x7f61c415ef10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f61bc0092d0 tx=0x7f61bc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:15.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61aaffd700 1 -- 172.21.15.116:0/2125144277 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc00bbc0 con 0x7f61c414eb30 2020-11-20T12:19:15.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61aaffd700 1 -- 172.21.15.116:0/2125144277 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61bc00be90 con 0x7f61c414eb30 2020-11-20T12:19:15.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61c990f700 1 -- 172.21.15.116:0/2125144277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c4152090 con 0x7f61c414eb30 2020-11-20T12:19:15.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61aaffd700 1 -- 172.21.15.116:0/2125144277 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61bc010e70 con 0x7f61c414eb30 2020-11-20T12:19:15.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61c990f700 1 -- 172.21.15.116:0/2125144277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c4149740 con 0x7f61c414eb30 2020-11-20T12:19:15.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61c990f700 1 -- 172.21.15.116:0/2125144277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c40a82b0 con 0x7f61c414eb30 2020-11-20T12:19:15.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61aaffd700 1 -- 172.21.15.116:0/2125144277 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f61bc023030 con 0x7f61c414eb30 2020-11-20T12:19:15.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61aaffd700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557c0 0x7f61a0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:15.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.656+0000 7f61aaffd700 1 -- 172.21.15.116:0/2125144277 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f61bc06ffc0 con 0x7f61c414eb30 2020-11-20T12:19:15.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.660+0000 7f61c2ffd700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557c0 0x7f61a0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:15.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.660+0000 7f61c2ffd700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557c0 0x7f61a0057c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f61b4009650 tx=0x7f61b4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:15.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.664+0000 7f61aaffd700 1 -- 172.21.15.116:0/2125144277 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f61bc045e00 con 0x7f61c414eb30 2020-11-20T12:19:15.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.820+0000 7f61c990f700 1 -- 172.21.15.116:0/2125144277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61c4152220 con 0x7f61c414eb30 2020-11-20T12:19:15.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.820+0000 7f61aaffd700 1 -- 172.21.15.116:0/2125144277 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f61bc02f200 con 0x7f61c414eb30 2020-11-20T12:19:15.819 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:15.819 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":71,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:13.986922+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:11.306087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:11.966077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:15.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 -- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557c0 msgr2=0x7f61a0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557c0 0x7f61a0057c80 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f61b4009650 tx=0x7f61b4008040).stop 2020-11-20T12:19:15.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 -- 172.21.15.116:0/2125144277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c414eb30 msgr2=0x7f61c415ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:15.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c414eb30 0x7f61c415ef10 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f61bc0092d0 tx=0x7f61bc016000).stop 2020-11-20T12:19:15.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 -- 172.21.15.116:0/2125144277 shutdown_connections 2020-11-20T12:19:15.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f61a00557c0 0x7f61a0057c80 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f61c414eb30 0x7f61c415ef10 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f61c40a6860 0x7f61c4149250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 --2- 172.21.15.116:0/2125144277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f61c414bfa0 0x7f61c414fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:15.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 -- 172.21.15.116:0/2125144277 >> 172.21.15.116:0/2125144277 conn(0x7f61c40c63f0 msgr2=0x7f61c4060d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:15.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 -- 172.21.15.116:0/2125144277 shutdown_connections 2020-11-20T12:19:15.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:15.824+0000 7f61c990f700 1 -- 172.21.15.116:0/2125144277 wait complete. 2020-11-20T12:19:15.843 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-20T12:19:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.032+0000 7f7f6621d700 1 Processor -- start 2020-11-20T12:19:16.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.032+0000 7f7f6621d700 1 -- start start 2020-11-20T12:19:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.032+0000 7f7f6621d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f60155b00 0x7f7f60064690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.032+0000 7f7f6621d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7f60155ee0 0x7f7f60060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.032+0000 7f7f6621d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f6015a890 con 0x7f7f60063950 2020-11-20T12:19:16.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.032+0000 7f7f6621d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f60064b80 con 0x7f7f60155ee0 2020-11-20T12:19:16.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.032+0000 7f7f6621d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600ba140 con 0x7f7f60155b00 2020-11-20T12:19:16.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.032+0000 7f7f64a1a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7f60155ee0 0x7f7f60060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.032+0000 7f7f64a1a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7f60155ee0 0x7f7f60060170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47808/0 (socket says 172.21.15.116:47808) 2020-11-20T12:19:16.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.032+0000 7f7f64a1a700 1 -- 172.21.15.116:0/355436135 learned_addr learned my addr 172.21.15.116:0/355436135 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:16.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.032+0000 7f7f6521b700 1 --2- 172.21.15.116:0/355436135 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f60155b00 0x7f7f60064690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f64a1a700 1 -- 172.21.15.116:0/355436135 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f60155b00 msgr2=0x7f7f60064690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:16.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f64a1a700 1 --2- 172.21.15.116:0/355436135 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f60155b00 0x7f7f60064690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f64a1a700 1 -- 172.21.15.116:0/355436135 >> v1:172.21.15.173:6789/0 conn(0x7f7f60063950 legacy=0x7f7f601598e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:16.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f64a1a700 1 -- 172.21.15.116:0/355436135 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f600ba560 con 0x7f7f60155ee0 2020-11-20T12:19:16.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f64a1a700 1 --2- 172.21.15.116:0/355436135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f60155ee0 0x7f7f60060170 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f7f50006fd0 tx=0x7f7f50006eb0).ready entity=mon.0 client_cookie=272c1270602d4fda server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:16.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f4ffff700 1 -- 172.21.15.116:0/355436135 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50009810 con 0x7f7f60155ee0 2020-11-20T12:19:16.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f4ffff700 1 -- 172.21.15.116:0/355436135 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f50009970 con 0x7f7f60155ee0 2020-11-20T12:19:16.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f4ffff700 1 -- 172.21.15.116:0/355436135 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50009cf0 con 0x7f7f60155ee0 2020-11-20T12:19:16.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f6621d700 1 -- 172.21.15.116:0/355436135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f60155ee0 msgr2=0x7f7f60060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:16.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f6621d700 1 --2- 172.21.15.116:0/355436135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f60155ee0 0x7f7f60060170 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7f7f50006fd0 tx=0x7f7f50006eb0).stop 2020-11-20T12:19:16.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f6621d700 1 -- 172.21.15.116:0/355436135 shutdown_connections 2020-11-20T12:19:16.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f6621d700 1 --2- 172.21.15.116:0/355436135 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f60155b00 0x7f7f60064690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f6621d700 1 --2- 172.21.15.116:0/355436135 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f60155ee0 0x7f7f60060170 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f6621d700 1 -- 172.21.15.116:0/355436135 >> 172.21.15.116:0/355436135 conn(0x7f7f600c5700 msgr2=0x7f7f600c16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:16.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f6621d700 1 -- 172.21.15.116:0/355436135 shutdown_connections 2020-11-20T12:19:16.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.036+0000 7f7f6621d700 1 -- 172.21.15.116:0/355436135 wait complete. 2020-11-20T12:19:16.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.040+0000 7f7f6621d700 1 Processor -- start 2020-11-20T12:19:16.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.040+0000 7f7f6621d700 1 -- start start 2020-11-20T12:19:16.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f6621d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f60063950 0x7f7f600a72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f6621d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f60155b00 0x7f7f60154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f6621d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f60155ee0 0x7f7f600a6a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f6621d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f601541c0 con 0x7f7f60155b00 2020-11-20T12:19:16.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f6621d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600a6ef0 con 0x7f7f60063950 2020-11-20T12:19:16.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f6621d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f60154c20 con 0x7f7f60155ee0 2020-11-20T12:19:16.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f64a1a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f60155b00 0x7f7f60154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f64a1a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f60155b00 0x7f7f60154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47814/0 (socket says 172.21.15.116:47814) 2020-11-20T12:19:16.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f64a1a700 1 -- 172.21.15.116:0/1707725561 learned_addr learned my addr 172.21.15.116:0/1707725561 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:16.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f6521b700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f60063950 0x7f7f600a72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f65a1c700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f60155ee0 0x7f7f600a6a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f64a1a700 1 -- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f60155ee0 msgr2=0x7f7f600a6a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f64a1a700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f60155ee0 0x7f7f600a6a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f64a1a700 1 -- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f60063950 msgr2=0x7f7f600a72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f64a1a700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f60063950 0x7f7f600a72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f64a1a700 1 -- 172.21.15.116:0/1707725561 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f600aa490 con 0x7f7f60155b00 2020-11-20T12:19:16.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f65a1c700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f60155ee0 0x7f7f600a6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:16.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f64a1a700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f60155b00 0x7f7f60154580 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f7f50010000 tx=0x7f7f50015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:16.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f4d7fa700 1 -- 172.21.15.116:0/1707725561 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f50006760 con 0x7f7f60155b00 2020-11-20T12:19:16.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f4d7fa700 1 -- 172.21.15.116:0/1707725561 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f50011c80 con 0x7f7f60155b00 2020-11-20T12:19:16.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f4d7fa700 1 -- 172.21.15.116:0/1707725561 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5001e460 con 0x7f7f60155b00 2020-11-20T12:19:16.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f6621d700 1 -- 172.21.15.116:0/1707725561 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f600aa730 con 0x7f7f60155b00 2020-11-20T12:19:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.044+0000 7f7f6621d700 1 -- 172.21.15.116:0/1707725561 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6014df00 con 0x7f7f60155b00 2020-11-20T12:19:16.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.048+0000 7f7f6621d700 1 -- 172.21.15.116:0/1707725561 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6005cf30 con 0x7f7f60155b00 2020-11-20T12:19:16.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.048+0000 7f7f4d7fa700 1 -- 172.21.15.116:0/1707725561 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7f50025030 con 0x7f7f60155b00 2020-11-20T12:19:16.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.048+0000 7f7f4d7fa700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f380557c0 0x7f7f38057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.048+0000 7f7f4d7fa700 1 -- 172.21.15.116:0/1707725561 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7f5006fc20 con 0x7f7f60155b00 2020-11-20T12:19:16.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.052+0000 7f7f6521b700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f380557c0 0x7f7f38057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.052+0000 7f7f6521b700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f380557c0 0x7f7f38057c80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f7f5c006fd0 tx=0x7f7f5c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:16.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.056+0000 7f7f4d7fa700 1 -- 172.21.15.116:0/1707725561 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7f50045ac0 con 0x7f7f60155b00 2020-11-20T12:19:16.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.268+0000 7f7f6621d700 1 -- 172.21.15.116:0/1707725561 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f7f600628e0 con 0x7f7f60155b00 2020-11-20T12:19:16.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.276+0000 7f7f4d7fa700 1 -- 172.21.15.116:0/1707725561 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7f500490e0 con 0x7f7f60155b00 2020-11-20T12:19:16.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 -- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f380557c0 msgr2=0x7f7f38057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:16.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f380557c0 0x7f7f38057c80 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f7f5c006fd0 tx=0x7f7f5c006df0).stop 2020-11-20T12:19:16.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 -- 172.21.15.116:0/1707725561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f60155b00 msgr2=0x7f7f60154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:16.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f60155b00 0x7f7f60154580 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f7f50010000 tx=0x7f7f50015000).stop 2020-11-20T12:19:16.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 -- 172.21.15.116:0/1707725561 shutdown_connections 2020-11-20T12:19:16.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7f380557c0 0x7f7f38057c80 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f60155ee0 0x7f7f600a6a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f60063950 0x7f7f600a72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 --2- 172.21.15.116:0/1707725561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f60155b00 0x7f7f60154580 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 -- 172.21.15.116:0/1707725561 >> 172.21.15.116:0/1707725561 conn(0x7f7f600c5700 msgr2=0x7f7f6014ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:16.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 -- 172.21.15.116:0/1707725561 shutdown_connections 2020-11-20T12:19:16.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.280+0000 7f7f6621d700 1 -- 172.21.15.116:0/1707725561 wait complete. 2020-11-20T12:19:16.282 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2020-11-20T12:19:16.302 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:19:16.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f4295332700 1 Processor -- start 2020-11-20T12:19:16.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f4295332700 1 -- start start 2020-11-20T12:19:16.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f4295332700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f429014f800 0x7f4290065330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f4295332700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f42900aa570 0x7f4290065820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f4295332700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42900a7320 0x7f42900a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f4295332700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f429014fbe0 con 0x7f429014f800 2020-11-20T12:19:16.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f4295332700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42900a98f0 con 0x7f42900aa570 2020-11-20T12:19:16.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f4295332700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42900a9b90 con 0x7f42900a7320 2020-11-20T12:19:16.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f428effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f429014f800 0x7f4290065330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f428effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f429014f800 0x7f4290065330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47820/0 (socket says 172.21.15.116:47820) 2020-11-20T12:19:16.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f4287fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f42900aa570 0x7f4290065820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f428effd700 1 -- 172.21.15.116:0/2848240519 learned_addr learned my addr 172.21.15.116:0/2848240519 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:16.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.488+0000 7f428ffff700 1 --2- 172.21.15.116:0/2848240519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42900a7320 0x7f42900a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f428effd700 1 -- 172.21.15.116:0/2848240519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42900a7320 msgr2=0x7f42900a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:16.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f428effd700 1 --2- 172.21.15.116:0/2848240519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42900a7320 0x7f42900a77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f428effd700 1 -- 172.21.15.116:0/2848240519 >> v2:172.21.15.173:3300/0 conn(0x7f42900aa570 msgr2=0x7f4290065820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:16.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f428effd700 1 --2- 172.21.15.116:0/2848240519 >> v2:172.21.15.173:3300/0 conn(0x7f42900aa570 0x7f4290065820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f428effd700 1 -- 172.21.15.116:0/2848240519 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42900a9e30 con 0x7f429014f800 2020-11-20T12:19:16.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f428effd700 1 --2- 172.21.15.116:0/2848240519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f429014f800 0x7f4290065330 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f42900c5880 tx=0x7f4280006eb0).ready entity=mon.0 client_cookie=4953d5b698997c11 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:16.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f428e7fc700 1 -- 172.21.15.116:0/2848240519 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280009720 con 0x7f429014f800 2020-11-20T12:19:16.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f428e7fc700 1 -- 172.21.15.116:0/2848240519 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4280009880 con 0x7f429014f800 2020-11-20T12:19:16.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f428e7fc700 1 -- 172.21.15.116:0/2848240519 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4280009c00 con 0x7f429014f800 2020-11-20T12:19:16.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f4295332700 1 -- 172.21.15.116:0/2848240519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f429014f800 msgr2=0x7f4290065330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:16.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f4295332700 1 --2- 172.21.15.116:0/2848240519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f429014f800 0x7f4290065330 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f42900c5880 tx=0x7f4280006eb0).stop 2020-11-20T12:19:16.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f4295332700 1 -- 172.21.15.116:0/2848240519 shutdown_connections 2020-11-20T12:19:16.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f4295332700 1 --2- 172.21.15.116:0/2848240519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42900a7320 0x7f42900a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f4295332700 1 --2- 172.21.15.116:0/2848240519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f429014f800 0x7f4290065330 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f4295332700 1 --2- 172.21.15.116:0/2848240519 >> v2:172.21.15.173:3300/0 conn(0x7f42900aa570 0x7f4290065820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f4295332700 1 -- 172.21.15.116:0/2848240519 >> 172.21.15.116:0/2848240519 conn(0x7f42900c63f0 msgr2=0x7f42900c23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:16.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f4295332700 1 -- 172.21.15.116:0/2848240519 shutdown_connections 2020-11-20T12:19:16.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.492+0000 7f4295332700 1 -- 172.21.15.116:0/2848240519 wait complete. 2020-11-20T12:19:16.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4295332700 1 Processor -- start 2020-11-20T12:19:16.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4295332700 1 -- start start 2020-11-20T12:19:16.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4295332700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42900a7320 0x7f429013f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4295332700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42900aa570 0x7f42900a2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4295332700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f429014f800 0x7f42900a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4295332700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4290150d90 con 0x7f42900aa570 2020-11-20T12:19:16.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4295332700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4290164dd0 con 0x7f429014f800 2020-11-20T12:19:16.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4295332700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4290165070 con 0x7f42900a7320 2020-11-20T12:19:16.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4287fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42900aa570 0x7f42900a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4287fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42900aa570 0x7f42900a2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47828/0 (socket says 172.21.15.116:47828) 2020-11-20T12:19:16.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4287fff700 1 -- 172.21.15.116:0/1677458957 learned_addr learned my addr 172.21.15.116:0/1677458957 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:16.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f428effd700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42900a7320 0x7f429013f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f428ffff700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f429014f800 0x7f42900a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4287fff700 1 -- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42900a7320 msgr2=0x7f429013f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:16.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4287fff700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42900a7320 0x7f429013f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4287fff700 1 -- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f429014f800 msgr2=0x7f42900a2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:16.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4287fff700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f429014f800 0x7f42900a2b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4287fff700 1 -- 172.21.15.116:0/1677458957 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4290165310 con 0x7f42900aa570 2020-11-20T12:19:16.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f428effd700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42900a7320 0x7f429013f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:16.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4287fff700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42900aa570 0x7f42900a2690 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f427800ebd0 tx=0x7f4278006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:16.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4286ffd700 1 -- 172.21.15.116:0/1677458957 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427800b9a0 con 0x7f42900aa570 2020-11-20T12:19:16.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4286ffd700 1 -- 172.21.15.116:0/1677458957 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427800bb00 con 0x7f42900aa570 2020-11-20T12:19:16.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4286ffd700 1 -- 172.21.15.116:0/1677458957 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427800be80 con 0x7f42900aa570 2020-11-20T12:19:16.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4295332700 1 -- 172.21.15.116:0/1677458957 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42901655b0 con 0x7f42900aa570 2020-11-20T12:19:16.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.496+0000 7f4295332700 1 -- 172.21.15.116:0/1677458957 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42900a45c0 con 0x7f42900aa570 2020-11-20T12:19:16.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.500+0000 7f4295332700 1 -- 172.21.15.116:0/1677458957 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4290080220 con 0x7f42900aa570 2020-11-20T12:19:16.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.500+0000 7f4286ffd700 1 -- 172.21.15.116:0/1677458957 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4278012600 con 0x7f42900aa570 2020-11-20T12:19:16.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.500+0000 7f4286ffd700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f426c0557c0 0x7f426c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:16.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.500+0000 7f4286ffd700 1 -- 172.21.15.116:0/1677458957 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f427806ea60 con 0x7f42900aa570 2020-11-20T12:19:16.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.504+0000 7f428effd700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f426c0557c0 0x7f426c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:16.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.504+0000 7f428effd700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f426c0557c0 0x7f426c057c80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f4280006ef0 tx=0x7f4280015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:16.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.508+0000 7f4286ffd700 1 -- 172.21.15.116:0/1677458957 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4290080220 con 0x7f42900aa570 2020-11-20T12:19:16.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.712+0000 7f4295332700 1 -- 172.21.15.116:0/1677458957 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4290080220 con 0x7f42900aa570 2020-11-20T12:19:16.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4286ffd700 1 -- 172.21.15.116:0/1677458957 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2854 (secure 0 0 0) 0x7f4290080220 con 0x7f42900aa570 2020-11-20T12:19:16.713 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:16.714 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3869489418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3869489418},{"type":"v1","addr":"172.21.15.173:6837","nonce":3869489418}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/2912073472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2912073472},{"type":"v1","addr":"172.21.15.173:6835","nonce":2912073472}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/2221764249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2221764249},{"type":"v1","addr":"172.21.15.116:6834","nonce":2221764249}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:12.981713+0000","modified":"2020-11-20T12:19:14.999541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1946956517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1946956517},{"type":"v1","addr":"172.21.15.116:6835","nonce":1946956517}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-20T12:19:16.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 -- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f426c0557c0 msgr2=0x7f426c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:16.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f426c0557c0 0x7f426c057c80 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f4280006ef0 tx=0x7f4280015000).stop 2020-11-20T12:19:16.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 -- 172.21.15.116:0/1677458957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42900aa570 msgr2=0x7f42900a2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:16.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42900aa570 0x7f42900a2690 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f427800ebd0 tx=0x7f4278006df0).stop 2020-11-20T12:19:16.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 -- 172.21.15.116:0/1677458957 shutdown_connections 2020-11-20T12:19:16.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f426c0557c0 0x7f426c057c80 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f429014f800 0x7f42900a2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42900a7320 0x7f429013f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 --2- 172.21.15.116:0/1677458957 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42900aa570 0x7f42900a2690 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:16.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 -- 172.21.15.116:0/1677458957 >> 172.21.15.116:0/1677458957 conn(0x7f42900c63f0 msgr2=0x7f4290063f10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:16.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 -- 172.21.15.116:0/1677458957 shutdown_connections 2020-11-20T12:19:16.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:16.716+0000 7f4295332700 1 -- 172.21.15.116:0/1677458957 wait complete. 2020-11-20T12:19:16.718 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 37 2020-11-20T12:19:17.146 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:19:12.981713+0000', 'modified': '2020-11-20T12:19:14.999541+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6429}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6429': {'gid': 6429, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/1946956517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 1946956517}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 1946956517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:19:17.146 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:19:17.147 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:19:17.345 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":6429,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.337778353,"mdsmap_epoch":37,"osdmap_epoch":71,"osdmap_epoch_barrier":71,"uptime":9.8148783179999999} 2020-11-20T12:19:17.346 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 6429, "mdsmap_epoch": 37, "osdmap_epoch": 71, "osdmap_epoch_barrier": 71, "rank_uptime": 4.337778353, "state": "up:active", "uptime": 9.814878318, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:19:17.347 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:19:17.347 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:19:17.348 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:19:17.349 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:19:17.349 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:19:17.350 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:19:17.350 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:19:17.350 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:19:17.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:17.351 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:19:17.368 INFO:tasks.ceph.mds.a.smithi116.stderr:2020-11-20T12:19:17.368+0000 7ff8cf713700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:19:17.369 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:19:17.369 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:19:17.369 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:19:17.370 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:17.370 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:19:17.370 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:17.370 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:17.370 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:17.371 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:17.371 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:17.371 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:17.371 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:17.371 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:19:17.372 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:17.372 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:19:17.372 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:17.372 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:19:17.372 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:17.373 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:19:17.373 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:17.373 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:19:17.373 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:17.374 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:19:17.374 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:17.374 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:19:17.374 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:17.374 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:19:17.375 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:17.375 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:19:17.375 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:19:17.375 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:19:17.375 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:17.376 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:19:17.376 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:19:17.376 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:19:17.376 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:17.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:17.378 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:19:17.413 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:19:17.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:17.416 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:19:17.478 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:19:17.479 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:19:17.479 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:19:17.479 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:19:17.479 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:19:17.479 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:19:17.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:17.481 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:19:17.481 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:19:17.482 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:19:17.482 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:19:17.483 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:19:17.483 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:19:17.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:17.574 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:19:17.590 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:19:17.590 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:19:17.591 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:19:17.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:17.592 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2020-11-20T12:19:17.623 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:17.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.844+0000 7f8d3aaf0700 1 Processor -- start 2020-11-20T12:19:17.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.844+0000 7f8d3aaf0700 1 -- start start 2020-11-20T12:19:17.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.844+0000 7f8d3aaf0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8d34062880 0x7f8d34062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:17.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.844+0000 7f8d3aaf0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8d340b9bf0 0x7f8d3414be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:17.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d3aaf0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d340ba1b0 con 0x7f8d3414b990 2020-11-20T12:19:17.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d3aaf0700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d34152e60 con 0x7f8d34062880 2020-11-20T12:19:17.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d3aaf0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d341555a0 con 0x7f8d340b9bf0 2020-11-20T12:19:17.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d392ed700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8d3414b990 0x7f8d340b94e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57268/0 (socket says 172.21.15.116:57268) 2020-11-20T12:19:17.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d392ed700 1 -- 172.21.15.116:0/2290104089 learned_addr learned my addr 172.21.15.116:0/2290104089 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:17.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d39aee700 1 --2- 172.21.15.116:0/2290104089 >> v2:172.21.15.116:3300/0 conn(0x7f8d34062880 0x7f8d34062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:17.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3902746224 0 0) 0x7f8d340ba1b0 con 0x7f8d3414b990 2020-11-20T12:19:17.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d1c003240 con 0x7f8d3414b990 2020-11-20T12:19:17.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d3a2ef700 1 --2- 172.21.15.116:0/2290104089 >> v2:172.21.15.173:3300/0 conn(0x7f8d340b9bf0 0x7f8d3414be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:17.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2473347349 0 0) 0x7f8d1c003240 con 0x7f8d3414b990 2020-11-20T12:19:17.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 >> v2:172.21.15.173:3300/0 conn(0x7f8d340b9bf0 msgr2=0x7f8d3414be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:17.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 --2- 172.21.15.116:0/2290104089 >> v2:172.21.15.173:3300/0 conn(0x7f8d340b9bf0 0x7f8d3414be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:17.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 >> v2:172.21.15.116:3300/0 conn(0x7f8d34062880 msgr2=0x7f8d34062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:17.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 --2- 172.21.15.116:0/2290104089 >> v2:172.21.15.116:3300/0 conn(0x7f8d34062880 0x7f8d34062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:17.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d34155840 con 0x7f8d3414b990 2020-11-20T12:19:17.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f8d240014e0 con 0x7f8d3414b990 2020-11-20T12:19:17.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 >> v1:172.21.15.116:6789/0 conn(0x7f8d3414b990 legacy=0x7f8d340b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:17.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 --2- 172.21.15.116:0/2290104089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c003730 0x7f8d1c003b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:17.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 --2- 172.21.15.116:0/2290104089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d1c0044e0 0x7f8d1c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:17.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 --2- 172.21.15.116:0/2290104089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d1c004e50 0x7f8d1c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:17.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d34152e60 con 0x7f8d1c003730 2020-11-20T12:19:17.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d341555a0 con 0x7f8d1c0044e0 2020-11-20T12:19:17.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c00bc20 con 0x7f8d1c004e50 2020-11-20T12:19:17.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d39aee700 1 --2- 172.21.15.116:0/2290104089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c003730 0x7f8d1c003b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:17.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d39aee700 1 -- 172.21.15.116:0/2290104089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d1c004e50 msgr2=0x7f8d1c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:17.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d39aee700 1 --2- 172.21.15.116:0/2290104089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d1c004e50 0x7f8d1c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:17.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d39aee700 1 -- 172.21.15.116:0/2290104089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d1c0044e0 msgr2=0x7f8d1c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:17.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d39aee700 1 --2- 172.21.15.116:0/2290104089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d1c0044e0 0x7f8d1c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:17.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d39aee700 1 -- 172.21.15.116:0/2290104089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1c003240 con 0x7f8d1c003730 2020-11-20T12:19:17.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d3a2ef700 1 -- 172.21.15.116:0/2290104089 reap_dead start 2020-11-20T12:19:17.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d39aee700 1 --2- 172.21.15.116:0/2290104089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c003730 0x7f8d1c003b70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f8d3000bc00 tx=0x7f8d30006df0).ready entity=mon.0 client_cookie=60fd6c94b7d30c28 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:17.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3000b310 con 0x7f8d1c003730 2020-11-20T12:19:17.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3000eb40 con 0x7f8d1c003730 2020-11-20T12:19:17.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.848+0000 7f8d38aec700 1 -- 172.21.15.116:0/2290104089 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3000eec0 con 0x7f8d1c003730 2020-11-20T12:19:17.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.852+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/2290104089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c003730 msgr2=0x7f8d1c003b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.852+0000 7f8d3aaf0700 1 --2- 172.21.15.116:0/2290104089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c003730 0x7f8d1c003b70 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f8d3000bc00 tx=0x7f8d30006df0).stop 2020-11-20T12:19:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.852+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/2290104089 shutdown_connections 2020-11-20T12:19:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.852+0000 7f8d3aaf0700 1 --2- 172.21.15.116:0/2290104089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d1c003730 0x7f8d1c003b70 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.852+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/2290104089 >> 172.21.15.116:0/2290104089 conn(0x7f8d34060c40 msgr2=0x7f8d340c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:17.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.852+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/2290104089 shutdown_connections 2020-11-20T12:19:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.852+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/2290104089 wait complete. 2020-11-20T12:19:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.852+0000 7f8d3aaf0700 1 Processor -- start 2020-11-20T12:19:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d3aaf0700 1 -- start start 2020-11-20T12:19:17.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d3aaf0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d1c003730 0x7f8d340a4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:17.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d3aaf0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d340a6750 0x7f8d340c2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:17.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d3aaf0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d340a6bd0 0x7f8d340c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:17.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d3aaf0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340a8e40 con 0x7f8d340a6750 2020-11-20T12:19:17.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d3aaf0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340a90e0 con 0x7f8d340a6bd0 2020-11-20T12:19:17.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d3aaf0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d340a9380 con 0x7f8d1c003730 2020-11-20T12:19:17.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d392ed700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d340a6750 0x7f8d340c2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:17.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d392ed700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d340a6750 0x7f8d340c2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47848/0 (socket says 172.21.15.116:47848) 2020-11-20T12:19:17.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d392ed700 1 -- 172.21.15.116:0/1111961473 learned_addr learned my addr 172.21.15.116:0/1111961473 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:17.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d3a2ef700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d340a6bd0 0x7f8d340c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:17.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d39aee700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d1c003730 0x7f8d340a4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d392ed700 1 -- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d1c003730 msgr2=0x7f8d340a4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d392ed700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d1c003730 0x7f8d340a4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d392ed700 1 -- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d340a6bd0 msgr2=0x7f8d340c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d392ed700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d340a6bd0 0x7f8d340c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:17.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d392ed700 1 -- 172.21.15.116:0/1111961473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d340a9620 con 0x7f8d340a6750 2020-11-20T12:19:17.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d39aee700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d1c003730 0x7f8d340a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.856+0000 7f8d392ed700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d340a6750 0x7f8d340c2180 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f8d2400ca60 tx=0x7f8d340677a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.860+0000 7f8d21ffb700 1 -- 172.21.15.116:0/1111961473 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d24007170 con 0x7f8d340a6750 2020-11-20T12:19:17.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.860+0000 7f8d21ffb700 1 -- 172.21.15.116:0/1111961473 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d240072d0 con 0x7f8d340a6750 2020-11-20T12:19:17.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.860+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/1111961473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d340a98c0 con 0x7f8d340a6750 2020-11-20T12:19:17.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.860+0000 7f8d21ffb700 1 -- 172.21.15.116:0/1111961473 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400aa10 con 0x7f8d340a6750 2020-11-20T12:19:17.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.860+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/1111961473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d340a4860 con 0x7f8d340a6750 2020-11-20T12:19:17.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.860+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/1111961473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d340a9a50 con 0x7f8d340a6750 2020-11-20T12:19:17.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.860+0000 7f8d21ffb700 1 -- 172.21.15.116:0/1111961473 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8d2401b060 con 0x7f8d340a6750 2020-11-20T12:19:17.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.860+0000 7f8d21ffb700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8d0c0557c0 0x7f8d0c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:17.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.864+0000 7f8d21ffb700 1 -- 172.21.15.116:0/1111961473 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8d2406e120 con 0x7f8d340a6750 2020-11-20T12:19:17.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.864+0000 7f8d39aee700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8d0c0557c0 0x7f8d0c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:17.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.864+0000 7f8d39aee700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8d0c0557c0 0x7f8d0c057c80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f8d3000b4d0 tx=0x7f8d30019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:17.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:17.868+0000 7f8d21ffb700 1 -- 172.21.15.116:0/1111961473 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8d24043fc0 con 0x7f8d340a6750 2020-11-20T12:19:18.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.024+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/1111961473 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8d34061230 con 0x7f8d340a6750 2020-11-20T12:19:18.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.024+0000 7f8d21ffb700 1 -- 172.21.15.116:0/1111961473 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f8d24019030 con 0x7f8d340a6750 2020-11-20T12:19:18.025 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:18.025 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":71,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:13.986922+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:11.306087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:11.966077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:18.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8d0c0557c0 msgr2=0x7f8d0c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8d0c0557c0 0x7f8d0c057c80 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f8d3000b4d0 tx=0x7f8d30019000).stop 2020-11-20T12:19:18.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/1111961473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d340a6750 msgr2=0x7f8d340c2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d340a6750 0x7f8d340c2180 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f8d2400ca60 tx=0x7f8d340677a0).stop 2020-11-20T12:19:18.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/1111961473 shutdown_connections 2020-11-20T12:19:18.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8d0c0557c0 0x7f8d0c057c80 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8d340a6bd0 0x7f8d340c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8d1c003730 0x7f8d340a4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 --2- 172.21.15.116:0/1111961473 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8d340a6750 0x7f8d340c2180 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/1111961473 >> 172.21.15.116:0/1111961473 conn(0x7f8d34060c40 msgr2=0x7f8d34152200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:18.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/1111961473 shutdown_connections 2020-11-20T12:19:18.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.028+0000 7f8d3aaf0700 1 -- 172.21.15.116:0/1111961473 wait complete. 2020-11-20T12:19:18.052 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:19:11.306087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:19:11.966077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:19:18.053 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:19:18.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.228+0000 7ffa5f59e700 1 Processor -- start 2020-11-20T12:19:18.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.228+0000 7ffa5f59e700 1 -- start start 2020-11-20T12:19:18.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.228+0000 7ffa5f59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffa600bff10 0x7ffa60066480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:18.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.228+0000 7ffa5f59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa600c0690 con 0x7ffa6014e590 2020-11-20T12:19:18.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.228+0000 7ffa5f59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa600c0930 con 0x7ffa60063100 2020-11-20T12:19:18.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.228+0000 7ffa5f59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa600c0bd0 con 0x7ffa600bff10 2020-11-20T12:19:18.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.228+0000 7ffa5ed9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffa600bff10 0x7ffa60066480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.228+0000 7ffa5e59c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ffa6014e590 0x7ffa6014f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57290/0 (socket says 172.21.15.116:57290) 2020-11-20T12:19:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5e59c700 1 -- 172.21.15.116:0/2260921198 learned_addr learned my addr 172.21.15.116:0/2260921198 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5ed9d700 1 -- 172.21.15.116:0/2260921198 >> v1:172.21.15.173:6789/0 conn(0x7ffa60063100 legacy=0x7ffa600a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5ed9d700 1 -- 172.21.15.116:0/2260921198 >> v1:172.21.15.116:6789/0 conn(0x7ffa6014e590 legacy=0x7ffa6014f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5ed9d700 1 -- 172.21.15.116:0/2260921198 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6014ed30 con 0x7ffa600bff10 2020-11-20T12:19:18.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5ed9d700 1 --2- 172.21.15.116:0/2260921198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa600bff10 0x7ffa60066480 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7ffa58009c90 tx=0x7ffa580091d0).ready entity=mon.0 client_cookie=a36424ed79dae9f1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:18.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5dd9b700 1 -- 172.21.15.116:0/2260921198 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5800bbc0 con 0x7ffa600bff10 2020-11-20T12:19:18.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5dd9b700 1 -- 172.21.15.116:0/2260921198 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5800bd20 con 0x7ffa600bff10 2020-11-20T12:19:18.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5dd9b700 1 -- 172.21.15.116:0/2260921198 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58011610 con 0x7ffa600bff10 2020-11-20T12:19:18.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5f59e700 1 -- 172.21.15.116:0/2260921198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa600bff10 msgr2=0x7ffa60066480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5f59e700 1 --2- 172.21.15.116:0/2260921198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa600bff10 0x7ffa60066480 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7ffa58009c90 tx=0x7ffa580091d0).stop 2020-11-20T12:19:18.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5f59e700 1 -- 172.21.15.116:0/2260921198 shutdown_connections 2020-11-20T12:19:18.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5f59e700 1 --2- 172.21.15.116:0/2260921198 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa600bff10 0x7ffa60066480 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5f59e700 1 -- 172.21.15.116:0/2260921198 >> 172.21.15.116:0/2260921198 conn(0x7ffa60068350 msgr2=0x7ffa600627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:18.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5f59e700 1 -- 172.21.15.116:0/2260921198 shutdown_connections 2020-11-20T12:19:18.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.232+0000 7ffa5f59e700 1 -- 172.21.15.116:0/2260921198 wait complete. 2020-11-20T12:19:18.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa5f59e700 1 Processor -- start 2020-11-20T12:19:18.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa5f59e700 1 -- start start 2020-11-20T12:19:18.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa5f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa60063100 0x7ffa60139ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:18.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa5f59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa600bff10 0x7ffa600b9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:18.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa5f59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa6014e590 0x7ffa60138c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:18.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa5f59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600a6740 con 0x7ffa6014e590 2020-11-20T12:19:18.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa5f59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600a41b0 con 0x7ffa600bff10 2020-11-20T12:19:18.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa5f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa600a5cc0 con 0x7ffa60063100 2020-11-20T12:19:18.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa5ed9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa6014e590 0x7ffa60138c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:18.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa5ed9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa6014e590 0x7ffa60138c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47864/0 (socket says 172.21.15.116:47864) 2020-11-20T12:19:18.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa5ed9d700 1 -- 172.21.15.116:0/4162447544 learned_addr learned my addr 172.21.15.116:0/4162447544 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:18.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.236+0000 7ffa57fff700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa600bff10 0x7ffa600b9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:18.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa5e59c700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa60063100 0x7ffa60139ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:18.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa5ed9d700 1 -- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa60063100 msgr2=0x7ffa60139ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa5ed9d700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa60063100 0x7ffa60139ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa5ed9d700 1 -- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa600bff10 msgr2=0x7ffa600b9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa5ed9d700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa600bff10 0x7ffa600b9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa5ed9d700 1 -- 172.21.15.116:0/4162447544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa600a5f60 con 0x7ffa6014e590 2020-11-20T12:19:18.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa5ed9d700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa6014e590 0x7ffa60138c60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7ffa600c54a0 tx=0x7ffa58016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:18.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa567fc700 1 -- 172.21.15.116:0/4162447544 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58011e60 con 0x7ffa6014e590 2020-11-20T12:19:18.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa5f59e700 1 -- 172.21.15.116:0/4162447544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa600a6200 con 0x7ffa6014e590 2020-11-20T12:19:18.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa567fc700 1 -- 172.21.15.116:0/4162447544 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa5800be90 con 0x7ffa6014e590 2020-11-20T12:19:18.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa567fc700 1 -- 172.21.15.116:0/4162447544 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58010cb0 con 0x7ffa6014e590 2020-11-20T12:19:18.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa5f59e700 1 -- 172.21.15.116:0/4162447544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa600b9a50 con 0x7ffa6014e590 2020-11-20T12:19:18.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.240+0000 7ffa5f59e700 1 -- 172.21.15.116:0/4162447544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa600b9cd0 con 0x7ffa6014e590 2020-11-20T12:19:18.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.244+0000 7ffa567fc700 1 -- 172.21.15.116:0/4162447544 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ffa58017070 con 0x7ffa6014e590 2020-11-20T12:19:18.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.244+0000 7ffa567fc700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa3c0557c0 0x7ffa3c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:18.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.244+0000 7ffa567fc700 1 -- 172.21.15.116:0/4162447544 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ffa58071560 con 0x7ffa6014e590 2020-11-20T12:19:18.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.244+0000 7ffa5e59c700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa3c0557c0 0x7ffa3c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:18.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.244+0000 7ffa5e59c700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa3c0557c0 0x7ffa3c057c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7ffa5000d750 tx=0x7ffa50009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:18.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.248+0000 7ffa567fc700 1 -- 172.21.15.116:0/4162447544 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ffa5804ae30 con 0x7ffa6014e590 2020-11-20T12:19:18.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.408+0000 7ffa5f59e700 1 -- 172.21.15.116:0/4162447544 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ffa6005cf30 con 0x7ffa6014e590 2020-11-20T12:19:18.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.408+0000 7ffa567fc700 1 -- 172.21.15.116:0/4162447544 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v71) v1 ==== 110+0+10 (secure 0 0 0) 0x7ffa5804aa20 con 0x7ffa6014e590 2020-11-20T12:19:18.408 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:19:18.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 -- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa3c0557c0 msgr2=0x7ffa3c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa3c0557c0 0x7ffa3c057c80 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7ffa5000d750 tx=0x7ffa50009040).stop 2020-11-20T12:19:18.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 -- 172.21.15.116:0/4162447544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa6014e590 msgr2=0x7ffa60138c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa6014e590 0x7ffa60138c60 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7ffa600c54a0 tx=0x7ffa58016000).stop 2020-11-20T12:19:18.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 -- 172.21.15.116:0/4162447544 shutdown_connections 2020-11-20T12:19:18.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ffa3c0557c0 0x7ffa3c057c80 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa6014e590 0x7ffa60138c60 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa60063100 0x7ffa60139ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 --2- 172.21.15.116:0/4162447544 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa600bff10 0x7ffa600b9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 -- 172.21.15.116:0/4162447544 >> 172.21.15.116:0/4162447544 conn(0x7ffa60068350 msgr2=0x7ffa60066440 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:18.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 -- 172.21.15.116:0/4162447544 shutdown_connections 2020-11-20T12:19:18.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.412+0000 7ffa5f59e700 1 -- 172.21.15.116:0/4162447544 wait complete. 2020-11-20T12:19:18.434 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:19:18.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.616+0000 7fc4de78c700 1 Processor -- start 2020-11-20T12:19:18.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.616+0000 7fc4de78c700 1 -- start start 2020-11-20T12:19:18.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.616+0000 7fc4de78c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc4d8155c80 0x7fc4d8062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:18.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.616+0000 7fc4de78c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc4d80bed20 0x7fc4d80bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:18.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.616+0000 7fc4de78c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4d80b93c0 con 0x7fc4d8156060 2020-11-20T12:19:18.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.616+0000 7fc4de78c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d80ba580 con 0x7fc4d8155c80 2020-11-20T12:19:18.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.616+0000 7fc4de78c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4d80b9660 con 0x7fc4d80bed20 2020-11-20T12:19:18.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.616+0000 7fc4dd78a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc4d8155c80 0x7fc4d8062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:18.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.616+0000 7fc4dd78a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc4d8155c80 0x7fc4d8062ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47868/0 (socket says 172.21.15.116:47868) 2020-11-20T12:19:18.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.616+0000 7fc4dd78a700 1 -- 172.21.15.116:0/2003805161 learned_addr learned my addr 172.21.15.116:0/2003805161 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:18.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4dd78a700 1 -- 172.21.15.116:0/2003805161 >> v2:172.21.15.173:3300/0 conn(0x7fc4d80bed20 msgr2=0x7fc4d80bf1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:18.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4dd78a700 1 --2- 172.21.15.116:0/2003805161 >> v2:172.21.15.173:3300/0 conn(0x7fc4d80bed20 0x7fc4d80bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:18.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4dd78a700 1 -- 172.21.15.116:0/2003805161 >> v1:172.21.15.173:6789/0 conn(0x7fc4d8156060 legacy=0x7fc4d80c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:18.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4dd78a700 1 -- 172.21.15.116:0/2003805161 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d80b9900 con 0x7fc4d8155c80 2020-11-20T12:19:18.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4dd78a700 1 --2- 172.21.15.116:0/2003805161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d8155c80 0x7fc4d8062ed0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fc4d805c5c0 tx=0x7fc4d4006eb0).ready entity=mon.0 client_cookie=21d6d78938660e9c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:18.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4c7fff700 1 -- 172.21.15.116:0/2003805161 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4009890 con 0x7fc4d8155c80 2020-11-20T12:19:18.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4c7fff700 1 -- 172.21.15.116:0/2003805161 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d40099f0 con 0x7fc4d8155c80 2020-11-20T12:19:18.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4c7fff700 1 -- 172.21.15.116:0/2003805161 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4009d70 con 0x7fc4d8155c80 2020-11-20T12:19:18.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4de78c700 1 -- 172.21.15.116:0/2003805161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d8155c80 msgr2=0x7fc4d8062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4de78c700 1 --2- 172.21.15.116:0/2003805161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d8155c80 0x7fc4d8062ed0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7fc4d805c5c0 tx=0x7fc4d4006eb0).stop 2020-11-20T12:19:18.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4de78c700 1 -- 172.21.15.116:0/2003805161 shutdown_connections 2020-11-20T12:19:18.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4de78c700 1 --2- 172.21.15.116:0/2003805161 >> v2:172.21.15.173:3300/0 conn(0x7fc4d80bed20 0x7fc4d80bf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:18.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4de78c700 1 --2- 172.21.15.116:0/2003805161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d8155c80 0x7fc4d8062ed0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4de78c700 1 -- 172.21.15.116:0/2003805161 >> 172.21.15.116:0/2003805161 conn(0x7fc4d8068350 msgr2=0x7fc4d80627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:18.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4de78c700 1 -- 172.21.15.116:0/2003805161 shutdown_connections 2020-11-20T12:19:18.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.620+0000 7fc4de78c700 1 -- 172.21.15.116:0/2003805161 wait complete. 2020-11-20T12:19:18.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.624+0000 7fc4de78c700 1 Processor -- start 2020-11-20T12:19:18.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.624+0000 7fc4de78c700 1 -- start start 2020-11-20T12:19:18.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.624+0000 7fc4de78c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d80bed20 0x7fc4d8150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:18.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.624+0000 7fc4de78c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8155c80 0x7fc4d80bda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:18.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.624+0000 7fc4de78c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4d8156060 0x7fc4d80c2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:18.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.624+0000 7fc4de78c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80be4c0 con 0x7fc4d80bed20 2020-11-20T12:19:18.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.624+0000 7fc4de78c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80a92d0 con 0x7fc4d8155c80 2020-11-20T12:19:18.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.624+0000 7fc4de78c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4d80a9570 con 0x7fc4d8156060 2020-11-20T12:19:18.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4dd78a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d80bed20 0x7fc4d8150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:18.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4dd78a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d80bed20 0x7fc4d8150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47874/0 (socket says 172.21.15.116:47874) 2020-11-20T12:19:18.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4dd78a700 1 -- 172.21.15.116:0/2108435715 learned_addr learned my addr 172.21.15.116:0/2108435715 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:18.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4dcf89700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8155c80 0x7fc4d80bda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:18.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4ddf8b700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4d8156060 0x7fc4d80c2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:18.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4dd78a700 1 -- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4d8156060 msgr2=0x7fc4d80c2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4dd78a700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4d8156060 0x7fc4d80c2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4dd78a700 1 -- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8155c80 msgr2=0x7fc4d80bda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4dd78a700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8155c80 0x7fc4d80bda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4dd78a700 1 -- 172.21.15.116:0/2108435715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d80a9810 con 0x7fc4d80bed20 2020-11-20T12:19:18.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4dd78a700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d80bed20 0x7fc4d8150df0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7fc4d80c54a0 tx=0x7fc4d4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:18.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4c57fa700 1 -- 172.21.15.116:0/2108435715 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4011d50 con 0x7fc4d80bed20 2020-11-20T12:19:18.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4c57fa700 1 -- 172.21.15.116:0/2108435715 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d401e460 con 0x7fc4d80bed20 2020-11-20T12:19:18.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4c57fa700 1 -- 172.21.15.116:0/2108435715 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d401e820 con 0x7fc4d80bed20 2020-11-20T12:19:18.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4de78c700 1 -- 172.21.15.116:0/2108435715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4d80a9ab0 con 0x7fc4d80bed20 2020-11-20T12:19:18.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4de78c700 1 -- 172.21.15.116:0/2108435715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d8158800 con 0x7fc4d80bed20 2020-11-20T12:19:18.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.628+0000 7fc4de78c700 1 -- 172.21.15.116:0/2108435715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4d805cf30 con 0x7fc4d80bed20 2020-11-20T12:19:18.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.632+0000 7fc4c57fa700 1 -- 172.21.15.116:0/2108435715 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc4d4025030 con 0x7fc4d80bed20 2020-11-20T12:19:18.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.632+0000 7fc4c57fa700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4b00557c0 0x7fc4b0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:18.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.632+0000 7fc4c57fa700 1 -- 172.21.15.116:0/2108435715 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc4d406fe60 con 0x7fc4d80bed20 2020-11-20T12:19:18.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.632+0000 7fc4dcf89700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4b00557c0 0x7fc4b0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:18.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.636+0000 7fc4dcf89700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4b00557c0 0x7fc4b0057c80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fc4c8006fd0 tx=0x7fc4c8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:18.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.636+0000 7fc4c57fa700 1 -- 172.21.15.116:0/2108435715 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc4d4045c70 con 0x7fc4d80bed20 2020-11-20T12:19:18.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.796+0000 7fc4de78c700 1 -- 172.21.15.116:0/2108435715 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc4d80639c0 con 0x7fc4d80bed20 2020-11-20T12:19:18.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.796+0000 7fc4c57fa700 1 -- 172.21.15.116:0/2108435715 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v71) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc4d4023070 con 0x7fc4d80bed20 2020-11-20T12:19:18.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.796+0000 7fc4de78c700 1 -- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4b00557c0 msgr2=0x7fc4b0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.796+0000 7fc4de78c700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4b00557c0 0x7fc4b0057c80 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fc4c8006fd0 tx=0x7fc4c8008040).stop 2020-11-20T12:19:18.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.796+0000 7fc4de78c700 1 -- 172.21.15.116:0/2108435715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d80bed20 msgr2=0x7fc4d8150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:18.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.796+0000 7fc4de78c700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d80bed20 0x7fc4d8150df0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7fc4d80c54a0 tx=0x7fc4d4015000).stop 2020-11-20T12:19:18.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.800+0000 7fc4de78c700 1 -- 172.21.15.116:0/2108435715 shutdown_connections 2020-11-20T12:19:18.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.800+0000 7fc4de78c700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4b00557c0 0x7fc4b0057c80 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.800+0000 7fc4de78c700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4d8156060 0x7fc4d80c2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.800+0000 7fc4de78c700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4d80bed20 0x7fc4d8150df0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.800+0000 7fc4de78c700 1 --2- 172.21.15.116:0/2108435715 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4d8155c80 0x7fc4d80bda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:18.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.800+0000 7fc4de78c700 1 -- 172.21.15.116:0/2108435715 >> 172.21.15.116:0/2108435715 conn(0x7fc4d8068350 msgr2=0x7fc4d80a6560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:18.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.800+0000 7fc4de78c700 1 -- 172.21.15.116:0/2108435715 shutdown_connections 2020-11-20T12:19:18.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:18.800+0000 7fc4de78c700 1 -- 172.21.15.116:0/2108435715 wait complete. 2020-11-20T12:19:18.816 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:18.820 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:19:19.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.016+0000 7f76baa26700 1 Processor -- start 2020-11-20T12:19:19.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.016+0000 7f76baa26700 1 -- start start 2020-11-20T12:19:19.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76baa26700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f76b41518f0 0x7f76b40bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76baa26700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f76b4151cd0 0x7f76b4066480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76baa26700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76b40b93c0 0x7f76b4066970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76baa26700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b40bf510 con 0x7f76b41518f0 2020-11-20T12:19:19.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76baa26700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b40bf7b0 con 0x7f76b4151cd0 2020-11-20T12:19:19.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76baa26700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b40bfa50 con 0x7f76b40b93c0 2020-11-20T12:19:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b9a24700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f76b41518f0 0x7f76b40bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b9a24700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f76b41518f0 0x7f76b40bed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47882/0 (socket says 172.21.15.116:47882) 2020-11-20T12:19:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b9a24700 1 -- 172.21.15.116:0/2131435507 learned_addr learned my addr 172.21.15.116:0/2131435507 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:19.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b9223700 1 --2- 172.21.15.116:0/2131435507 >> v2:172.21.15.173:3300/0 conn(0x7f76b4151cd0 0x7f76b4066480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76ba225700 1 --2- 172.21.15.116:0/2131435507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76b40b93c0 0x7f76b4066970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b9223700 1 -- 172.21.15.116:0/2131435507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76b40b93c0 msgr2=0x7f76b4066970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:19.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b9223700 1 --2- 172.21.15.116:0/2131435507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76b40b93c0 0x7f76b4066970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b9223700 1 -- 172.21.15.116:0/2131435507 >> v2:172.21.15.116:3300/0 conn(0x7f76b41518f0 msgr2=0x7f76b40bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:19.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b9223700 1 --2- 172.21.15.116:0/2131435507 >> v2:172.21.15.116:3300/0 conn(0x7f76b41518f0 0x7f76b40bed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b9223700 1 -- 172.21.15.116:0/2131435507 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b40bfcf0 con 0x7f76b4151cd0 2020-11-20T12:19:19.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b9a24700 1 --2- 172.21.15.116:0/2131435507 >> v2:172.21.15.116:3300/0 conn(0x7f76b41518f0 0x7f76b40bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:19.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b9223700 1 --2- 172.21.15.116:0/2131435507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76b4151cd0 0x7f76b4066480 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f76b40624e0 tx=0x7f76a4006df0).ready entity=mon.1 client_cookie=1429e22f42da0691 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:19.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b8a22700 1 -- 172.21.15.116:0/2131435507 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a4009640 con 0x7f76b4151cd0 2020-11-20T12:19:19.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b8a22700 1 -- 172.21.15.116:0/2131435507 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a40097a0 con 0x7f76b4151cd0 2020-11-20T12:19:19.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.020+0000 7f76b8a22700 1 -- 172.21.15.116:0/2131435507 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a4009b20 con 0x7f76b4151cd0 2020-11-20T12:19:19.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.024+0000 7f76baa26700 1 -- 172.21.15.116:0/2131435507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76b4151cd0 msgr2=0x7f76b4066480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.024+0000 7f76baa26700 1 --2- 172.21.15.116:0/2131435507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76b4151cd0 0x7f76b4066480 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f76b40624e0 tx=0x7f76a4006df0).stop 2020-11-20T12:19:19.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.024+0000 7f76baa26700 1 -- 172.21.15.116:0/2131435507 shutdown_connections 2020-11-20T12:19:19.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.024+0000 7f76baa26700 1 --2- 172.21.15.116:0/2131435507 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76b40b93c0 0x7f76b4066970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.024+0000 7f76baa26700 1 --2- 172.21.15.116:0/2131435507 >> v2:172.21.15.116:3300/0 conn(0x7f76b41518f0 0x7f76b40bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.024+0000 7f76baa26700 1 --2- 172.21.15.116:0/2131435507 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76b4151cd0 0x7f76b4066480 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.024+0000 7f76baa26700 1 -- 172.21.15.116:0/2131435507 >> 172.21.15.116:0/2131435507 conn(0x7f76b4068350 msgr2=0x7f76b40627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:19.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.024+0000 7f76baa26700 1 -- 172.21.15.116:0/2131435507 shutdown_connections 2020-11-20T12:19:19.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.024+0000 7f76baa26700 1 -- 172.21.15.116:0/2131435507 wait complete. 2020-11-20T12:19:19.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.024+0000 7f76baa26700 1 Processor -- start 2020-11-20T12:19:19.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.024+0000 7f76baa26700 1 -- start start 2020-11-20T12:19:19.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76baa26700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76b40b93c0 0x7f76b4162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76baa26700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b41518f0 0x7f76b40a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76baa26700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76b4151cd0 0x7f76b4138570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76baa26700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b4164740 con 0x7f76b41518f0 2020-11-20T12:19:19.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76baa26700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b4153190 con 0x7f76b4151cd0 2020-11-20T12:19:19.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76baa26700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b4153430 con 0x7f76b40b93c0 2020-11-20T12:19:19.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76b9223700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b41518f0 0x7f76b40a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76b9a24700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76b40b93c0 0x7f76b4162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76ba225700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76b4151cd0 0x7f76b4138570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76b9223700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b41518f0 0x7f76b40a9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47892/0 (socket says 172.21.15.116:47892) 2020-11-20T12:19:19.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76b9223700 1 -- 172.21.15.116:0/1059693762 learned_addr learned my addr 172.21.15.116:0/1059693762 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:19.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76ba225700 1 -- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76b40b93c0 msgr2=0x7f76b4162500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76ba225700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76b40b93c0 0x7f76b4162500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76ba225700 1 -- 172.21.15.116:0/1059693762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b41518f0 msgr2=0x7f76b40a9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76ba225700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b41518f0 0x7f76b40a9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76ba225700 1 -- 172.21.15.116:0/1059693762 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b41536d0 con 0x7f76b4151cd0 2020-11-20T12:19:19.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76b9223700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b41518f0 0x7f76b40a9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:19.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.028+0000 7f76ba225700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76b4151cd0 0x7f76b4138570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f76ac00ec00 tx=0x7f76ac0091d0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:19.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.032+0000 7f76a1ffb700 1 -- 172.21.15.116:0/1059693762 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00c9b0 con 0x7f76b4151cd0 2020-11-20T12:19:19.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.032+0000 7f76a1ffb700 1 -- 172.21.15.116:0/1059693762 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac00cb10 con 0x7f76b4151cd0 2020-11-20T12:19:19.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.032+0000 7f76baa26700 1 -- 172.21.15.116:0/1059693762 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b4153970 con 0x7f76b4151cd0 2020-11-20T12:19:19.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.032+0000 7f76a1ffb700 1 -- 172.21.15.116:0/1059693762 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00ceb0 con 0x7f76b4151cd0 2020-11-20T12:19:19.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.032+0000 7f76baa26700 1 -- 172.21.15.116:0/1059693762 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b41629f0 con 0x7f76b4151cd0 2020-11-20T12:19:19.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.032+0000 7f76baa26700 1 -- 172.21.15.116:0/1059693762 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b405cf30 con 0x7f76b4151cd0 2020-11-20T12:19:19.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.032+0000 7f76a1ffb700 1 -- 172.21.15.116:0/1059693762 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f76ac022070 con 0x7f76b4151cd0 2020-11-20T12:19:19.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.036+0000 7f76a1ffb700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f768c0557c0 0x7f768c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.036+0000 7f76a1ffb700 1 -- 172.21.15.116:0/1059693762 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f76ac06ec10 con 0x7f76b4151cd0 2020-11-20T12:19:19.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.036+0000 7f76b9a24700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f768c0557c0 0x7f768c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.036+0000 7f76b9a24700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f768c0557c0 0x7f768c057c80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f76b0006fd0 tx=0x7f76b0006eb0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:19.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.044+0000 7f76a1ffb700 1 -- 172.21.15.116:0/1059693762 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f76ac044ab0 con 0x7f76b4151cd0 2020-11-20T12:19:19.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.200+0000 7f76baa26700 1 -- 172.21.15.116:0/1059693762 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f76b40639c0 con 0x7f76b4151cd0 2020-11-20T12:19:19.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.204+0000 7f76a1ffb700 1 -- 172.21.15.116:0/1059693762 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+10 (secure 0 0 0) 0x7f76ac016070 con 0x7f76b4151cd0 2020-11-20T12:19:19.201 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:19.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.204+0000 7f76baa26700 1 -- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f768c0557c0 msgr2=0x7f768c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.204+0000 7f76baa26700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f768c0557c0 0x7f768c057c80 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7f76b0006fd0 tx=0x7f76b0006eb0).stop 2020-11-20T12:19:19.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.204+0000 7f76baa26700 1 -- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76b4151cd0 msgr2=0x7f76b4138570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.204+0000 7f76baa26700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76b4151cd0 0x7f76b4138570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f76ac00ec00 tx=0x7f76ac0091d0).stop 2020-11-20T12:19:19.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.208+0000 7f76baa26700 1 -- 172.21.15.116:0/1059693762 shutdown_connections 2020-11-20T12:19:19.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.208+0000 7f76baa26700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f768c0557c0 0x7f768c057c80 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.208+0000 7f76baa26700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76b4151cd0 0x7f76b4138570 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.208+0000 7f76baa26700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76b40b93c0 0x7f76b4162500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.208+0000 7f76baa26700 1 --2- 172.21.15.116:0/1059693762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76b41518f0 0x7f76b40a9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.208+0000 7f76baa26700 1 -- 172.21.15.116:0/1059693762 >> 172.21.15.116:0/1059693762 conn(0x7f76b4068350 msgr2=0x7f76b414d620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:19.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.208+0000 7f76baa26700 1 -- 172.21.15.116:0/1059693762 shutdown_connections 2020-11-20T12:19:19.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.208+0000 7f76baa26700 1 -- 172.21.15.116:0/1059693762 wait complete. 2020-11-20T12:19:19.230 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:19:19.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.404+0000 7f9178def700 1 Processor -- start 2020-11-20T12:19:19.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.404+0000 7f9178def700 1 -- start start 2020-11-20T12:19:19.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9178def700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f91740aadb0 0x7f9174151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9178def700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91741523e0 0x7f917413aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9178def700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91740acfe0 con 0x7f91740aa9d0 2020-11-20T12:19:19.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9178def700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91741529c0 con 0x7f91740aadb0 2020-11-20T12:19:19.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9178def700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9174152c60 con 0x7f91741523e0 2020-11-20T12:19:19.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9171d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f91740aadb0 0x7f9174151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9171d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f91740aadb0 0x7f9174151ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47898/0 (socket says 172.21.15.116:47898) 2020-11-20T12:19:19.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9171d9b700 1 -- 172.21.15.116:0/666984269 learned_addr learned my addr 172.21.15.116:0/666984269 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:19.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9172d9d700 1 --2- 172.21.15.116:0/666984269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91741523e0 0x7f917413aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9171d9b700 1 -- 172.21.15.116:0/666984269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91741523e0 msgr2=0x7f917413aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:19.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9171d9b700 1 --2- 172.21.15.116:0/666984269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91741523e0 0x7f917413aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9171d9b700 1 -- 172.21.15.116:0/666984269 >> v1:172.21.15.173:6789/0 conn(0x7f91740aa9d0 legacy=0x7f91740a8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9171d9b700 1 -- 172.21.15.116:0/666984269 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91740ad240 con 0x7f91740aadb0 2020-11-20T12:19:19.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f9171d9b700 1 --2- 172.21.15.116:0/666984269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91740aadb0 0x7f9174151ef0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f9168006fd0 tx=0x7f9168006eb0).ready entity=mon.0 client_cookie=ee12e4ecfc9f39d9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:19.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f917159a700 1 -- 172.21.15.116:0/666984269 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91680098d0 con 0x7f91740aadb0 2020-11-20T12:19:19.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f917159a700 1 -- 172.21.15.116:0/666984269 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9168009a30 con 0x7f91740aadb0 2020-11-20T12:19:19.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.408+0000 7f917159a700 1 -- 172.21.15.116:0/666984269 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9168009db0 con 0x7f91740aadb0 2020-11-20T12:19:19.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.412+0000 7f9178def700 1 -- 172.21.15.116:0/666984269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91740aadb0 msgr2=0x7f9174151ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.412+0000 7f9178def700 1 --2- 172.21.15.116:0/666984269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91740aadb0 0x7f9174151ef0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7f9168006fd0 tx=0x7f9168006eb0).stop 2020-11-20T12:19:19.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.412+0000 7f9178def700 1 -- 172.21.15.116:0/666984269 shutdown_connections 2020-11-20T12:19:19.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.412+0000 7f9178def700 1 --2- 172.21.15.116:0/666984269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91741523e0 0x7f917413aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.412+0000 7f9178def700 1 --2- 172.21.15.116:0/666984269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91740aadb0 0x7f9174151ef0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.412+0000 7f9178def700 1 -- 172.21.15.116:0/666984269 >> 172.21.15.116:0/666984269 conn(0x7f91740678a0 msgr2=0x7f9174067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:19.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.412+0000 7f9178def700 1 -- 172.21.15.116:0/666984269 shutdown_connections 2020-11-20T12:19:19.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.412+0000 7f9178def700 1 -- 172.21.15.116:0/666984269 wait complete. 2020-11-20T12:19:19.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f9178def700 1 Processor -- start 2020-11-20T12:19:19.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f9178def700 1 -- start start 2020-11-20T12:19:19.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f9178def700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91740aa9d0 0x7f9174165270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f9178def700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91740aadb0 0x7f9174154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f9178def700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91741523e0 0x7f9174150460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f9178def700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91741615d0 con 0x7f91740aa9d0 2020-11-20T12:19:19.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f9178def700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9174161870 con 0x7f91740aadb0 2020-11-20T12:19:19.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f9178def700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9174161b10 con 0x7f91741523e0 2020-11-20T12:19:19.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f917259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91740aa9d0 0x7f9174165270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f917259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91740aa9d0 0x7f9174165270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47902/0 (socket says 172.21.15.116:47902) 2020-11-20T12:19:19.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f9171d9b700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91740aadb0 0x7f9174154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f917259c700 1 -- 172.21.15.116:0/4152726175 learned_addr learned my addr 172.21.15.116:0/4152726175 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:19.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.416+0000 7f9172d9d700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91741523e0 0x7f9174150460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f917259c700 1 -- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91741523e0 msgr2=0x7f9174150460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f917259c700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91741523e0 0x7f9174150460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f917259c700 1 -- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91740aadb0 msgr2=0x7f9174154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f917259c700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91740aadb0 0x7f9174154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f917259c700 1 -- 172.21.15.116:0/4152726175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9174161db0 con 0x7f91740aa9d0 2020-11-20T12:19:19.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f917259c700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91740aa9d0 0x7f9174165270 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f9164001e70 tx=0x7f9164007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:19.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f91627fc700 1 -- 172.21.15.116:0/4152726175 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9164007020 con 0x7f91740aa9d0 2020-11-20T12:19:19.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f91627fc700 1 -- 172.21.15.116:0/4152726175 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9164007180 con 0x7f91740aa9d0 2020-11-20T12:19:19.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f9178def700 1 -- 172.21.15.116:0/4152726175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9174162050 con 0x7f91740aa9d0 2020-11-20T12:19:19.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f91627fc700 1 -- 172.21.15.116:0/4152726175 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91640064f0 con 0x7f91740aa9d0 2020-11-20T12:19:19.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f9178def700 1 -- 172.21.15.116:0/4152726175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9174154df0 con 0x7f91740aa9d0 2020-11-20T12:19:19.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.420+0000 7f9178def700 1 -- 172.21.15.116:0/4152726175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91741621e0 con 0x7f91740aa9d0 2020-11-20T12:19:19.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.424+0000 7f91627fc700 1 -- 172.21.15.116:0/4152726175 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9164010070 con 0x7f91740aa9d0 2020-11-20T12:19:19.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.424+0000 7f91627fc700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91480557f0 0x7f9148057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.424+0000 7f91627fc700 1 -- 172.21.15.116:0/4152726175 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f916406da70 con 0x7f91740aa9d0 2020-11-20T12:19:19.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.428+0000 7f9171d9b700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91480557f0 0x7f9148057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.428+0000 7f9171d9b700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91480557f0 0x7f9148057cb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f91740bfde0 tx=0x7f9168015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:19.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.432+0000 7f91627fc700 1 -- 172.21.15.116:0/4152726175 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9164042fd0 con 0x7f91740aa9d0 2020-11-20T12:19:19.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f9178def700 1 -- 172.21.15.116:0/4152726175 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f917405d2a0 con 0x7f91740aa9d0 2020-11-20T12:19:19.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f91627fc700 1 -- 172.21.15.116:0/4152726175 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+10 (secure 0 0 0) 0x7f9164043800 con 0x7f91740aa9d0 2020-11-20T12:19:19.591 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:19.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f9178def700 1 -- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91480557f0 msgr2=0x7f9148057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f9178def700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91480557f0 0x7f9148057cb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f91740bfde0 tx=0x7f9168015000).stop 2020-11-20T12:19:19.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f9178def700 1 -- 172.21.15.116:0/4152726175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91740aa9d0 msgr2=0x7f9174165270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f9178def700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91740aa9d0 0x7f9174165270 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f9164001e70 tx=0x7f9164007c50).stop 2020-11-20T12:19:19.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f9178def700 1 -- 172.21.15.116:0/4152726175 shutdown_connections 2020-11-20T12:19:19.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f9178def700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91480557f0 0x7f9148057cb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f9178def700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91741523e0 0x7f9174150460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f9178def700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f91740aa9d0 0x7f9174165270 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f9178def700 1 --2- 172.21.15.116:0/4152726175 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f91740aadb0 0x7f9174154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.592+0000 7f9178def700 1 -- 172.21.15.116:0/4152726175 >> 172.21.15.116:0/4152726175 conn(0x7f91740678a0 msgr2=0x7f9174138e40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:19.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.596+0000 7f9178def700 1 -- 172.21.15.116:0/4152726175 shutdown_connections 2020-11-20T12:19:19.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.596+0000 7f9178def700 1 -- 172.21.15.116:0/4152726175 wait complete. 2020-11-20T12:19:19.617 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:19:19.618 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:19:19.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.796+0000 7fa4b2f59700 1 Processor -- start 2020-11-20T12:19:19.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.796+0000 7fa4b2f59700 1 -- start start 2020-11-20T12:19:19.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4b2f59700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac0c3780 0x7fa4ac0c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4b2f59700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ac0a6700 con 0x7fa4ac0a6280 2020-11-20T12:19:19.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4b2f59700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4ac0c1e20 con 0x7fa4ac0c1970 2020-11-20T12:19:19.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4b2f59700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac14fed0 con 0x7fa4ac0c3780 2020-11-20T12:19:19.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4abfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa4ac0a6280 0x7fa4ac0c22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57346/0 (socket says 172.21.15.116:57346) 2020-11-20T12:19:19.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4abfff700 1 -- 172.21.15.116:0/3267831432 learned_addr learned my addr 172.21.15.116:0/3267831432 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:19.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4b0cf5700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac0c3780 0x7fa4ac0c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 908529943 0 0) 0x7fa4ac0a6700 con 0x7fa4ac0a6280 2020-11-20T12:19:19.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa490003240 con 0x7fa4ac0a6280 2020-11-20T12:19:19.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3043959772 0 0) 0x7fa490003240 con 0x7fa4ac0a6280 2020-11-20T12:19:19.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac0c3780 msgr2=0x7fa4ac0c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:19.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4ab7fe700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac0c3780 0x7fa4ac0c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 >> v1:172.21.15.173:6789/0 conn(0x7fa4ac0c1970 legacy=0x7fa4ac150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac0a6950 con 0x7fa4ac0a6280 2020-11-20T12:19:19.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fa4980014e0 con 0x7fa4ac0a6280 2020-11-20T12:19:19.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 >> v1:172.21.15.116:6789/0 conn(0x7fa4ac0a6280 legacy=0x7fa4ac0c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.800+0000 7fa4ab7fe700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa490003760 0x7fa490003b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4ab7fe700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4900040a0 0x7fa490004580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4ab7fe700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac0c3780 0x7fa490004be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4900050f0 con 0x7fa490003760 2020-11-20T12:19:19.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa490005260 con 0x7fa4900040a0 2020-11-20T12:19:19.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49000fb40 con 0x7fa4ac0c3780 2020-11-20T12:19:19.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b0cf5700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa490003760 0x7fa490003b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4abfff700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4900040a0 0x7fa490004580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b0cf5700 1 -- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac0c3780 msgr2=0x7fa490004be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:19.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b14f6700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac0c3780 0x7fa490004be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b0cf5700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac0c3780 0x7fa490004be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b0cf5700 1 -- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4900040a0 msgr2=0x7fa490004580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:19.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b0cf5700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4900040a0 0x7fa490004580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b0cf5700 1 -- 172.21.15.116:0/3267831432 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa490003240 con 0x7fa490003760 2020-11-20T12:19:19.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b14f6700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac0c3780 0x7fa490004be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:19.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4abfff700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4900040a0 0x7fa490004580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:19.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b0cf5700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa490003760 0x7fa490003b40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7fa4a000bc00 tx=0x7fa4a0006df0).ready entity=mon.0 client_cookie=c6e8448fda3c9c7f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:19.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a000b310 con 0x7fa490003760 2020-11-20T12:19:19.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a000eb40 con 0x7fa490003760 2020-11-20T12:19:19.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4ab7fe700 1 -- 172.21.15.116:0/3267831432 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a000eec0 con 0x7fa490003760 2020-11-20T12:19:19.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b2f59700 1 -- 172.21.15.116:0/3267831432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa490003760 msgr2=0x7fa490003b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b2f59700 1 --2- 172.21.15.116:0/3267831432 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa490003760 0x7fa490003b40 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7fa4a000bc00 tx=0x7fa4a0006df0).stop 2020-11-20T12:19:19.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b14f6700 1 -- 172.21.15.116:0/3267831432 reap_dead start 2020-11-20T12:19:19.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b2f59700 1 -- 172.21.15.116:0/3267831432 shutdown_connections 2020-11-20T12:19:19.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b2f59700 1 -- 172.21.15.116:0/3267831432 >> 172.21.15.116:0/3267831432 conn(0x7fa4ac0c4f80 msgr2=0x7fa4ac0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:19.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b2f59700 1 -- 172.21.15.116:0/3267831432 shutdown_connections 2020-11-20T12:19:19.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.804+0000 7fa4b2f59700 1 -- 172.21.15.116:0/3267831432 wait complete. 2020-11-20T12:19:19.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b2f59700 1 Processor -- start 2020-11-20T12:19:19.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b2f59700 1 -- start start 2020-11-20T12:19:19.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b2f59700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4ac15d300 0x7fa4ac15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b2f59700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ac15dbd0 0x7fa4ac14b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b2f59700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac14bbc0 0x7fa4ac0bcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b2f59700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac15e460 con 0x7fa4ac15d300 2020-11-20T12:19:19.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b2f59700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac14c420 con 0x7fa4ac15dbd0 2020-11-20T12:19:19.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b2f59700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4ac0bc350 con 0x7fa4ac14bbc0 2020-11-20T12:19:19.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b0cf5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4ac15d300 0x7fa4ac15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b0cf5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4ac15d300 0x7fa4ac15d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47922/0 (socket says 172.21.15.116:47922) 2020-11-20T12:19:19.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b0cf5700 1 -- 172.21.15.116:0/2122042306 learned_addr learned my addr 172.21.15.116:0/2122042306 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:19.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b14f6700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac14bbc0 0x7fa4ac0bcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4abfff700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ac15dbd0 0x7fa4ac14b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b0cf5700 1 -- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac14bbc0 msgr2=0x7fa4ac0bcd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b0cf5700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac14bbc0 0x7fa4ac0bcd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b0cf5700 1 -- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ac15dbd0 msgr2=0x7fa4ac14b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:19.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b0cf5700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ac15dbd0 0x7fa4ac14b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:19.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b0cf5700 1 -- 172.21.15.116:0/2122042306 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac0bc5f0 con 0x7fa4ac15d300 2020-11-20T12:19:19.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4abfff700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ac15dbd0 0x7fa4ac14b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:19.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b0cf5700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4ac15d300 0x7fa4ac15d6e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fa4a0013000 tx=0x7fa4a0019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:19.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4a8ff9700 1 -- 172.21.15.116:0/2122042306 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a001d620 con 0x7fa4ac15d300 2020-11-20T12:19:19.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4a8ff9700 1 -- 172.21.15.116:0/2122042306 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4a0021070 con 0x7fa4ac15d300 2020-11-20T12:19:19.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b2f59700 1 -- 172.21.15.116:0/2122042306 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4ac0bc890 con 0x7fa4ac15d300 2020-11-20T12:19:19.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4a8ff9700 1 -- 172.21.15.116:0/2122042306 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4a001d780 con 0x7fa4ac15d300 2020-11-20T12:19:19.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.808+0000 7fa4b2f59700 1 -- 172.21.15.116:0/2122042306 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4ac0bbfd0 con 0x7fa4ac15d300 2020-11-20T12:19:19.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.812+0000 7fa4b2f59700 1 -- 172.21.15.116:0/2122042306 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4ac05d2a0 con 0x7fa4ac15d300 2020-11-20T12:19:19.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.812+0000 7fa4a8ff9700 1 -- 172.21.15.116:0/2122042306 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa4a001daf0 con 0x7fa4ac15d300 2020-11-20T12:19:19.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.812+0000 7fa4a8ff9700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa4840557f0 0x7fa484057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:19.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.812+0000 7fa4a8ff9700 1 -- 172.21.15.116:0/2122042306 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa4a0070600 con 0x7fa4ac15d300 2020-11-20T12:19:19.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.812+0000 7fa4abfff700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa4840557f0 0x7fa484057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:19.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.812+0000 7fa4abfff700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa4840557f0 0x7fa484057cb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fa498001610 tx=0x7fa4ac0bffa0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:19.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:19.816+0000 7fa4a8ff9700 1 -- 172.21.15.116:0/2122042306 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa4a0046520 con 0x7fa4ac15d300 2020-11-20T12:19:20.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.024+0000 7fa4b2f59700 1 -- 172.21.15.116:0/2122042306 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4ac060f40 con 0x7fa4ac15d300 2020-11-20T12:19:20.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.028+0000 7fa4a8ff9700 1 -- 172.21.15.116:0/2122042306 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2854 (secure 0 0 0) 0x7fa4a0049b40 con 0x7fa4ac15d300 2020-11-20T12:19:20.027 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:20.028 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3869489418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3869489418},{"type":"v1","addr":"172.21.15.173:6837","nonce":3869489418}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/2912073472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2912073472},{"type":"v1","addr":"172.21.15.173:6835","nonce":2912073472}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/2221764249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2221764249},{"type":"v1","addr":"172.21.15.116:6834","nonce":2221764249}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:12.981713+0000","modified":"2020-11-20T12:19:14.999541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1946956517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1946956517},{"type":"v1","addr":"172.21.15.116:6835","nonce":1946956517}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-20T12:19:20.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 -- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa4840557f0 msgr2=0x7fa484057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:20.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa4840557f0 0x7fa484057cb0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fa498001610 tx=0x7fa4ac0bffa0).stop 2020-11-20T12:19:20.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 -- 172.21.15.116:0/2122042306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4ac15d300 msgr2=0x7fa4ac15d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:20.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4ac15d300 0x7fa4ac15d6e0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7fa4a0013000 tx=0x7fa4a0019000).stop 2020-11-20T12:19:20.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 -- 172.21.15.116:0/2122042306 shutdown_connections 2020-11-20T12:19:20.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa4840557f0 0x7fa484057cb0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:20.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa4ac14bbc0 0x7fa4ac0bcd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:20.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa4ac15d300 0x7fa4ac15d6e0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:20.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 --2- 172.21.15.116:0/2122042306 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa4ac15dbd0 0x7fa4ac14b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:20.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 -- 172.21.15.116:0/2122042306 >> 172.21.15.116:0/2122042306 conn(0x7fa4ac0c4f80 msgr2=0x7fa4ac14da60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:20.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 -- 172.21.15.116:0/2122042306 shutdown_connections 2020-11-20T12:19:20.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:20.032+0000 7fa4b2f59700 1 -- 172.21.15.116:0/2122042306 wait complete. 2020-11-20T12:19:20.032 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 37 2020-11-20T12:19:20.461 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:19:12.981713+0000', 'modified': '2020-11-20T12:19:14.999541+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6429}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6429': {'gid': 6429, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/1946956517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 1946956517}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 1946956517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:19:20.462 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:19:20.462 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:19:20.665 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":6429,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.6596002990000001,"mdsmap_epoch":37,"osdmap_epoch":71,"osdmap_epoch_barrier":71,"uptime":13.136701888999999} 2020-11-20T12:19:20.666 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 6429, "mdsmap_epoch": 37, "osdmap_epoch": 71, "osdmap_epoch_barrier": 71, "rank_uptime": 7.659600299, "state": "up:active", "uptime": 13.136701889, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:19:20.666 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:19:20.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:20.667 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-20T12:19:20.686 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:19:20.687 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:20.804 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:20.805 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:19:20.832 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:20.832 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:19:20.838 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:19:20.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:20.839 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-20T12:19:20.872 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:19:20.873 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:19:20.874 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-20T12:19:20.874 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-20T12:19:20.875 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:19:20.876 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:19:20.876 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:19:20.877 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:19:20.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:20.878 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:19:20.911 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:19:20.911 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:19:20.911 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:19:20.912 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:20.912 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:19:20.912 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:20.912 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:20.913 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:20.913 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:20.914 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:20.914 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:20.914 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:20.915 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:19:20.915 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:20.916 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:19:20.916 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:20.916 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:19:20.917 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:20.917 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:19:20.918 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:20.918 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:19:20.920 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:20.920 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:19:20.920 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:20.921 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:19:20.921 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:20.922 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:19:20.922 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:20.922 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:19:20.923 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:19:20.923 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:19:20.923 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:20.924 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:19:20.924 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:19:20.924 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:19:20.925 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:20.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:20.927 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:19:21.011 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:19:21.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:21.014 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:19:21.093 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:19:21.094 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:19:21.094 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:19:21.094 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:19:21.094 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:19:21.095 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:19:21.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:21.096 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:19:21.097 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:19:21.097 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:19:21.098 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:19:21.098 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:19:21.098 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:19:21.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:21.243 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:19:21.260 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:19:21.260 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:19:21.261 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:19:21.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:21.262 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2020-11-20T12:19:21.342 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:21.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.520+0000 7f535cd62700 1 Processor -- start 2020-11-20T12:19:21.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f535cd62700 1 -- start start 2020-11-20T12:19:21.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f535cd62700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5358062880 0x7f5358062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:21.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f535cd62700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53581543a0 con 0x7f53580677a0 2020-11-20T12:19:21.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f535cd62700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5358067c10 con 0x7f5358153450 2020-11-20T12:19:21.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f535cd62700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5358153920 con 0x7f5358062880 2020-11-20T12:19:21.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f5357fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f53580677a0 0x7f53580b9710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57372/0 (socket says 172.21.15.116:57372) 2020-11-20T12:19:21.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f5357fff700 1 -- 172.21.15.116:0/4030703407 learned_addr learned my addr 172.21.15.116:0/4030703407 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:21.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f53577fe700 1 --2- 172.21.15.116:0/4030703407 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5358062880 0x7f5358062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:21.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f5356ffd700 1 -- 172.21.15.116:0/4030703407 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132630278 0 0) 0x7f53581543a0 con 0x7f53580677a0 2020-11-20T12:19:21.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f5356ffd700 1 -- 172.21.15.116:0/4030703407 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5338003240 con 0x7f53580677a0 2020-11-20T12:19:21.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f53577fe700 1 -- 172.21.15.116:0/4030703407 >> v1:172.21.15.173:6789/0 conn(0x7f5358153450 legacy=0x7f53580c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:21.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f53577fe700 1 -- 172.21.15.116:0/4030703407 >> v1:172.21.15.116:6789/0 conn(0x7f53580677a0 legacy=0x7f53580b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:21.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.524+0000 7f53577fe700 1 -- 172.21.15.116:0/4030703407 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5358153bc0 con 0x7f5358062880 2020-11-20T12:19:21.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.528+0000 7f53577fe700 1 --2- 172.21.15.116:0/4030703407 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5358062880 0x7f5358062c60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f534800b9d0 tx=0x7f5348006df0).ready entity=mon.2 client_cookie=d4b82259b6d166b4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:21.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.528+0000 7f5356ffd700 1 -- 172.21.15.116:0/4030703407 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53480097b0 con 0x7f5358062880 2020-11-20T12:19:21.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.528+0000 7f5356ffd700 1 -- 172.21.15.116:0/4030703407 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5348009910 con 0x7f5358062880 2020-11-20T12:19:21.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.528+0000 7f5356ffd700 1 -- 172.21.15.116:0/4030703407 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5348009c90 con 0x7f5358062880 2020-11-20T12:19:21.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.528+0000 7f535cd62700 1 -- 172.21.15.116:0/4030703407 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5358062880 msgr2=0x7f5358062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:21.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.528+0000 7f535cd62700 1 --2- 172.21.15.116:0/4030703407 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5358062880 0x7f5358062c60 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f534800b9d0 tx=0x7f5348006df0).stop 2020-11-20T12:19:21.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.528+0000 7f535cd62700 1 -- 172.21.15.116:0/4030703407 shutdown_connections 2020-11-20T12:19:21.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.528+0000 7f535cd62700 1 --2- 172.21.15.116:0/4030703407 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5358062880 0x7f5358062c60 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.528+0000 7f535cd62700 1 -- 172.21.15.116:0/4030703407 >> 172.21.15.116:0/4030703407 conn(0x7f5358060c40 msgr2=0x7f53580c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:21.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.528+0000 7f535cd62700 1 -- 172.21.15.116:0/4030703407 shutdown_connections 2020-11-20T12:19:21.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.528+0000 7f535cd62700 1 -- 172.21.15.116:0/4030703407 wait complete. 2020-11-20T12:19:21.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f535cd62700 1 Processor -- start 2020-11-20T12:19:21.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f535cd62700 1 -- start start 2020-11-20T12:19:21.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f535cd62700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53580677a0 0x7f5358151cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:21.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f535cd62700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5358153450 0x7f535814c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:21.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f535cd62700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5358149e80 0x7f5358164ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:21.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f535cd62700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53581521e0 con 0x7f5358149e80 2020-11-20T12:19:21.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f535cd62700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f535814c790 con 0x7f53580677a0 2020-11-20T12:19:21.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f535cd62700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53581523c0 con 0x7f5358153450 2020-11-20T12:19:21.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f5357fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5358149e80 0x7f5358164ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:21.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f53577fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53580677a0 0x7f5358151cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:21.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f5357fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5358149e80 0x7f5358164ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47944/0 (socket says 172.21.15.116:47944) 2020-11-20T12:19:21.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f5357fff700 1 -- 172.21.15.116:0/3165413605 learned_addr learned my addr 172.21.15.116:0/3165413605 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:21.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.532+0000 7f534ffff700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5358153450 0x7f535814c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:21.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f5357fff700 1 -- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5358153450 msgr2=0x7f535814c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:21.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f5357fff700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5358153450 0x7f535814c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f5357fff700 1 -- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53580677a0 msgr2=0x7f5358151cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:21.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f5357fff700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53580677a0 0x7f5358151cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f5357fff700 1 -- 172.21.15.116:0/3165413605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53581526a0 con 0x7f5358149e80 2020-11-20T12:19:21.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f53577fe700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53580677a0 0x7f5358151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:21.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f5357fff700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5358149e80 0x7f5358164ac0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f5350001420 tx=0x7f5350007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:21.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f534f7fe700 1 -- 172.21.15.116:0/3165413605 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350005b80 con 0x7f5358149e80 2020-11-20T12:19:21.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f534f7fe700 1 -- 172.21.15.116:0/3165413605 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350005ce0 con 0x7f5358149e80 2020-11-20T12:19:21.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f535cd62700 1 -- 172.21.15.116:0/3165413605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5358152940 con 0x7f5358149e80 2020-11-20T12:19:21.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f534f7fe700 1 -- 172.21.15.116:0/3165413605 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350007320 con 0x7f5358149e80 2020-11-20T12:19:21.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f535cd62700 1 -- 172.21.15.116:0/3165413605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535814a2e0 con 0x7f5358149e80 2020-11-20T12:19:21.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f535cd62700 1 -- 172.21.15.116:0/3165413605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5358152f90 con 0x7f5358149e80 2020-11-20T12:19:21.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.536+0000 7f534f7fe700 1 -- 172.21.15.116:0/3165413605 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5350010030 con 0x7f5358149e80 2020-11-20T12:19:21.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.540+0000 7f534f7fe700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53340557c0 0x7f5334057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:21.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.540+0000 7f534f7fe700 1 -- 172.21.15.116:0/3165413605 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f535006e500 con 0x7f5358149e80 2020-11-20T12:19:21.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.540+0000 7f53577fe700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53340557c0 0x7f5334057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:21.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.540+0000 7f53577fe700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53340557c0 0x7f5334057c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f5358065160 tx=0x7f5348015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:21.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.544+0000 7f534f7fe700 1 -- 172.21.15.116:0/3165413605 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f53500443a0 con 0x7f5358149e80 2020-11-20T12:19:21.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.708+0000 7f535cd62700 1 -- 172.21.15.116:0/3165413605 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53580a1900 con 0x7f5358149e80 2020-11-20T12:19:21.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f534f7fe700 1 -- 172.21.15.116:0/3165413605 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f53500479c0 con 0x7f5358149e80 2020-11-20T12:19:21.710 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:21.710 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":71,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:13.986922+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:11.306087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:11.966077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:21.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 -- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53340557c0 msgr2=0x7f5334057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:21.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53340557c0 0x7f5334057c80 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7f5358065160 tx=0x7f5348015000).stop 2020-11-20T12:19:21.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 -- 172.21.15.116:0/3165413605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5358149e80 msgr2=0x7f5358164ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:21.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5358149e80 0x7f5358164ac0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f5350001420 tx=0x7f5350007e20).stop 2020-11-20T12:19:21.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 -- 172.21.15.116:0/3165413605 shutdown_connections 2020-11-20T12:19:21.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53340557c0 0x7f5334057c80 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5358149e80 0x7f5358164ac0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53580677a0 0x7f5358151cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 --2- 172.21.15.116:0/3165413605 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5358153450 0x7f535814c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 -- 172.21.15.116:0/3165413605 >> 172.21.15.116:0/3165413605 conn(0x7f5358060c40 msgr2=0x7f53580a6630 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:21.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 -- 172.21.15.116:0/3165413605 shutdown_connections 2020-11-20T12:19:21.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.712+0000 7f535cd62700 1 -- 172.21.15.116:0/3165413605 wait complete. 2020-11-20T12:19:21.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:19:11.306087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:19:11.966077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:19:21.735 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:19:21.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.916+0000 7f9c883dc700 1 Processor -- start 2020-11-20T12:19:21.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.916+0000 7f9c883dc700 1 -- start start 2020-11-20T12:19:21.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.916+0000 7f9c883dc700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9c80060310 0x7f9c800606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:21.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c883dc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9c80066d90 0x7f9c80159af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:21.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c883dc700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c800a67e0 con 0x7f9c800aa0d0 2020-11-20T12:19:21.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c883dc700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c800a6270 con 0x7f9c80066d90 2020-11-20T12:19:21.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c883dc700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c800aa530 con 0x7f9c80060310 2020-11-20T12:19:21.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c85977700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9c800aa0d0 0x7f9c80066680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57382/0 (socket says 172.21.15.116:57382) 2020-11-20T12:19:21.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c85977700 1 -- 172.21.15.116:0/1153340203 learned_addr learned my addr 172.21.15.116:0/1153340203 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:21.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c86979700 1 --2- 172.21.15.116:0/1153340203 >> v2:172.21.15.116:3300/0 conn(0x7f9c80066d90 0x7f9c80159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:21.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c86178700 1 --2- 172.21.15.116:0/1153340203 >> v2:172.21.15.173:3300/0 conn(0x7f9c80060310 0x7f9c800606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:21.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c85176700 1 -- 172.21.15.116:0/1153340203 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535167799 0 0) 0x7f9c800a67e0 con 0x7f9c800aa0d0 2020-11-20T12:19:21.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c85176700 1 -- 172.21.15.116:0/1153340203 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c68003240 con 0x7f9c800aa0d0 2020-11-20T12:19:21.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c86979700 1 -- 172.21.15.116:0/1153340203 >> v2:172.21.15.173:3300/0 conn(0x7f9c80060310 msgr2=0x7f9c800606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:21.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c86979700 1 --2- 172.21.15.116:0/1153340203 >> v2:172.21.15.173:3300/0 conn(0x7f9c80060310 0x7f9c800606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c86979700 1 -- 172.21.15.116:0/1153340203 >> v1:172.21.15.116:6789/0 conn(0x7f9c800aa0d0 legacy=0x7f9c80066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:21.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c86979700 1 -- 172.21.15.116:0/1153340203 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c800a5fd0 con 0x7f9c80066d90 2020-11-20T12:19:21.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c86979700 1 --2- 172.21.15.116:0/1153340203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80066d90 0x7f9c80159af0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f9c80066290 tx=0x7f9c780091d0).ready entity=mon.0 client_cookie=2bd8bb6a148e1aea server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:21.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c85176700 1 -- 172.21.15.116:0/1153340203 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800b9c0 con 0x7f9c80066d90 2020-11-20T12:19:21.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c85176700 1 -- 172.21.15.116:0/1153340203 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c7800bb20 con 0x7f9c80066d90 2020-11-20T12:19:21.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.920+0000 7f9c85176700 1 -- 172.21.15.116:0/1153340203 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c7800bea0 con 0x7f9c80066d90 2020-11-20T12:19:21.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.924+0000 7f9c883dc700 1 -- 172.21.15.116:0/1153340203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80066d90 msgr2=0x7f9c80159af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:21.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.924+0000 7f9c883dc700 1 --2- 172.21.15.116:0/1153340203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80066d90 0x7f9c80159af0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7f9c80066290 tx=0x7f9c780091d0).stop 2020-11-20T12:19:21.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.924+0000 7f9c883dc700 1 -- 172.21.15.116:0/1153340203 shutdown_connections 2020-11-20T12:19:21.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.924+0000 7f9c883dc700 1 --2- 172.21.15.116:0/1153340203 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80066d90 0x7f9c80159af0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.924+0000 7f9c883dc700 1 --2- 172.21.15.116:0/1153340203 >> v2:172.21.15.173:3300/0 conn(0x7f9c80060310 0x7f9c800606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.924+0000 7f9c883dc700 1 -- 172.21.15.116:0/1153340203 >> 172.21.15.116:0/1153340203 conn(0x7f9c8005ba80 msgr2=0x7f9c800c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:21.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.924+0000 7f9c883dc700 1 -- 172.21.15.116:0/1153340203 shutdown_connections 2020-11-20T12:19:21.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.924+0000 7f9c883dc700 1 -- 172.21.15.116:0/1153340203 wait complete. 2020-11-20T12:19:21.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c883dc700 1 Processor -- start 2020-11-20T12:19:21.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c883dc700 1 -- start start 2020-11-20T12:19:21.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c883dc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c80060310 0x7f9c800bb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c883dc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80066d90 0x7f9c80159900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c883dc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800aa0d0 0x7f9c80140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c883dc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8015a0d0 con 0x7f9c80066d90 2020-11-20T12:19:21.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c883dc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c800bb890 con 0x7f9c80060310 2020-11-20T12:19:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c883dc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c8015a300 con 0x7f9c800aa0d0 2020-11-20T12:19:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c85977700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80066d90 0x7f9c80159900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c85977700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80066d90 0x7f9c80159900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47956/0 (socket says 172.21.15.116:47956) 2020-11-20T12:19:21.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c85977700 1 -- 172.21.15.116:0/3773540848 learned_addr learned my addr 172.21.15.116:0/3773540848 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c86178700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c80060310 0x7f9c800bb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:21.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c86979700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800aa0d0 0x7f9c80140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c85977700 1 -- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800aa0d0 msgr2=0x7f9c80140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c85977700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800aa0d0 0x7f9c80140de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c85977700 1 -- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c80060310 msgr2=0x7f9c800bb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:21.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c85977700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c80060310 0x7f9c800bb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:21.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c85977700 1 -- 172.21.15.116:0/3773540848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8015a5a0 con 0x7f9c80066d90 2020-11-20T12:19:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c86178700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c80060310 0x7f9c800bb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:21.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c86979700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800aa0d0 0x7f9c80140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c85977700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80066d90 0x7f9c80159900 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f9c7000c710 tx=0x7f9c70007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c6e7fc700 1 -- 172.21.15.116:0/3773540848 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c700072e0 con 0x7f9c80066d90 2020-11-20T12:19:21.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c6e7fc700 1 -- 172.21.15.116:0/3773540848 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c70007440 con 0x7f9c80066d90 2020-11-20T12:19:21.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c6e7fc700 1 -- 172.21.15.116:0/3773540848 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70005820 con 0x7f9c80066d90 2020-11-20T12:19:21.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c883dc700 1 -- 172.21.15.116:0/3773540848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c80159df0 con 0x7f9c80066d90 2020-11-20T12:19:21.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.928+0000 7f9c883dc700 1 -- 172.21.15.116:0/3773540848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c80153c00 con 0x7f9c80066d90 2020-11-20T12:19:21.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.932+0000 7f9c883dc700 1 -- 172.21.15.116:0/3773540848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c8005d2a0 con 0x7f9c80066d90 2020-11-20T12:19:21.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.932+0000 7f9c6e7fc700 1 -- 172.21.15.116:0/3773540848 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9c70010070 con 0x7f9c80066d90 2020-11-20T12:19:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.932+0000 7f9c6e7fc700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c580557c0 0x7f9c58057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.932+0000 7f9c6e7fc700 1 -- 172.21.15.116:0/3773540848 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9c7006dd90 con 0x7f9c80066d90 2020-11-20T12:19:21.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.932+0000 7f9c86178700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c580557c0 0x7f9c58057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.936+0000 7f9c86178700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c580557c0 0x7f9c58057c80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f9c7c006bd0 tx=0x7f9c7c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:21.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:21.940+0000 7f9c6e7fc700 1 -- 172.21.15.116:0/3773540848 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9c70043c30 con 0x7f9c80066d90 2020-11-20T12:19:22.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.100+0000 7f9c883dc700 1 -- 172.21.15.116:0/3773540848 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9c8005c2e0 con 0x7f9c80066d90 2020-11-20T12:19:22.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.100+0000 7f9c6e7fc700 1 -- 172.21.15.116:0/3773540848 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v71) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9c70018030 con 0x7f9c80066d90 2020-11-20T12:19:22.099 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:19:22.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 -- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c580557c0 msgr2=0x7f9c58057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c580557c0 0x7f9c58057c80 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f9c7c006bd0 tx=0x7f9c7c008040).stop 2020-11-20T12:19:22.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 -- 172.21.15.116:0/3773540848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80066d90 msgr2=0x7f9c80159900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80066d90 0x7f9c80159900 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f9c7000c710 tx=0x7f9c70007a60).stop 2020-11-20T12:19:22.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 -- 172.21.15.116:0/3773540848 shutdown_connections 2020-11-20T12:19:22.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c580557c0 0x7f9c58057c80 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c800aa0d0 0x7f9c80140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c80060310 0x7f9c800bb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 --2- 172.21.15.116:0/3773540848 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c80066d90 0x7f9c80159900 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 -- 172.21.15.116:0/3773540848 >> 172.21.15.116:0/3773540848 conn(0x7f9c8005ba80 msgr2=0x7f9c800a7470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:22.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 -- 172.21.15.116:0/3773540848 shutdown_connections 2020-11-20T12:19:22.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.104+0000 7f9c883dc700 1 -- 172.21.15.116:0/3773540848 wait complete. 2020-11-20T12:19:22.124 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:19:22.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.300+0000 7f82d9515700 1 Processor -- start 2020-11-20T12:19:22.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.300+0000 7f82d9515700 1 -- start start 2020-11-20T12:19:22.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.300+0000 7f82d9515700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 0x7f82d4151b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.300+0000 7f82d9515700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f82d4060530 0x7f82d40609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.300+0000 7f82d9515700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82d40c7640 con 0x7f82d40c4380 2020-11-20T12:19:22.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.300+0000 7f82d9515700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82d40bed80 con 0x7f82d4060530 2020-11-20T12:19:22.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.300+0000 7f82d9515700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40bf020 con 0x7f82d4151770 2020-11-20T12:19:22.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.300+0000 7f82d27fc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f82d40c4380 0x7f82d40653e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57398/0 (socket says 172.21.15.116:57398) 2020-11-20T12:19:22.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.300+0000 7f82d27fc700 1 -- 172.21.15.116:0/1946571674 learned_addr learned my addr 172.21.15.116:0/1946571674 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:22.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.300+0000 7f82d3fff700 1 --2- 172.21.15.116:0/1946571674 >> v2:172.21.15.173:3300/0 conn(0x7f82d4060530 0x7f82d40609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.300+0000 7f82d2ffd700 1 --2- 172.21.15.116:0/1946571674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 0x7f82d4151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d1ffb700 1 -- 172.21.15.116:0/1946571674 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1931190738 0 0) 0x7f82d40c7640 con 0x7f82d40c4380 2020-11-20T12:19:22.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d2ffd700 1 -- 172.21.15.116:0/1946571674 >> v2:172.21.15.173:3300/0 conn(0x7f82d4060530 msgr2=0x7f82d40609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:22.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d2ffd700 1 --2- 172.21.15.116:0/1946571674 >> v2:172.21.15.173:3300/0 conn(0x7f82d4060530 0x7f82d40609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d2ffd700 1 -- 172.21.15.116:0/1946571674 >> v1:172.21.15.116:6789/0 conn(0x7f82d40c4380 legacy=0x7f82d40653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d2ffd700 1 -- 172.21.15.116:0/1946571674 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d40bf2c0 con 0x7f82d4151770 2020-11-20T12:19:22.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d2ffd700 1 --2- 172.21.15.116:0/1946571674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 0x7f82d4151b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f82c400b9a0 tx=0x7f82c4006df0).ready entity=mon.2 client_cookie=5eea1faf3de55f30 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:22.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d1ffb700 1 -- 172.21.15.116:0/1946571674 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c40097b0 con 0x7f82d4151770 2020-11-20T12:19:22.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d1ffb700 1 -- 172.21.15.116:0/1946571674 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c4009910 con 0x7f82d4151770 2020-11-20T12:19:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d1ffb700 1 -- 172.21.15.116:0/1946571674 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4009c90 con 0x7f82d4151770 2020-11-20T12:19:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d9515700 1 -- 172.21.15.116:0/1946571674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 msgr2=0x7f82d4151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d9515700 1 --2- 172.21.15.116:0/1946571674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 0x7f82d4151b50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f82c400b9a0 tx=0x7f82c4006df0).stop 2020-11-20T12:19:22.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d9515700 1 -- 172.21.15.116:0/1946571674 shutdown_connections 2020-11-20T12:19:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d9515700 1 --2- 172.21.15.116:0/1946571674 >> v2:172.21.15.173:3300/0 conn(0x7f82d4060530 0x7f82d40609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d9515700 1 --2- 172.21.15.116:0/1946571674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 0x7f82d4151b50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d9515700 1 -- 172.21.15.116:0/1946571674 >> 172.21.15.116:0/1946571674 conn(0x7f82d40ca810 msgr2=0x7f82d40c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d9515700 1 -- 172.21.15.116:0/1946571674 shutdown_connections 2020-11-20T12:19:22.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.304+0000 7f82d9515700 1 -- 172.21.15.116:0/1946571674 wait complete. 2020-11-20T12:19:22.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.308+0000 7f82d9515700 1 Processor -- start 2020-11-20T12:19:22.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d9515700 1 -- start start 2020-11-20T12:19:22.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d9515700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82d4060530 0x7f82d40acf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d9515700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82d40c4380 0x7f82d41648e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d9515700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 0x7f82d414efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d9515700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40afa80 con 0x7f82d4060530 2020-11-20T12:19:22.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d9515700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40afd20 con 0x7f82d40c4380 2020-11-20T12:19:22.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d9515700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82d40ae330 con 0x7f82d4151770 2020-11-20T12:19:22.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82d4060530 0x7f82d40acf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d2ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82d4060530 0x7f82d40acf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47968/0 (socket says 172.21.15.116:47968) 2020-11-20T12:19:22.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d2ffd700 1 -- 172.21.15.116:0/1233547586 learned_addr learned my addr 172.21.15.116:0/1233547586 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:22.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d27fc700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82d40c4380 0x7f82d41648e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d3fff700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 0x7f82d414efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d2ffd700 1 -- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 msgr2=0x7f82d414efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d2ffd700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 0x7f82d414efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d2ffd700 1 -- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82d40c4380 msgr2=0x7f82d41648e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d2ffd700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82d40c4380 0x7f82d41648e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d2ffd700 1 -- 172.21.15.116:0/1233547586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82d40ae5d0 con 0x7f82d4060530 2020-11-20T12:19:22.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d3fff700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 0x7f82d414efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:22.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.312+0000 7f82d2ffd700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82d4060530 0x7f82d40acf70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f82c4006fd0 tx=0x7f82c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:22.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.316+0000 7f82baffd700 1 -- 172.21.15.116:0/1233547586 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c4011d50 con 0x7f82d4060530 2020-11-20T12:19:22.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.316+0000 7f82baffd700 1 -- 172.21.15.116:0/1233547586 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c401e460 con 0x7f82d4060530 2020-11-20T12:19:22.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.316+0000 7f82d9515700 1 -- 172.21.15.116:0/1233547586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82d40ae870 con 0x7f82d4060530 2020-11-20T12:19:22.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.316+0000 7f82baffd700 1 -- 172.21.15.116:0/1233547586 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c401e7e0 con 0x7f82d4060530 2020-11-20T12:19:22.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.316+0000 7f82d9515700 1 -- 172.21.15.116:0/1233547586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82d40ad460 con 0x7f82d4060530 2020-11-20T12:19:22.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.316+0000 7f82d9515700 1 -- 172.21.15.116:0/1233547586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82d40aea00 con 0x7f82d4060530 2020-11-20T12:19:22.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.316+0000 7f82baffd700 1 -- 172.21.15.116:0/1233547586 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f82c4025030 con 0x7f82d4060530 2020-11-20T12:19:22.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.316+0000 7f82baffd700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82b00557f0 0x7f82b0057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.320+0000 7f82baffd700 1 -- 172.21.15.116:0/1233547586 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f82c406fc00 con 0x7f82d4060530 2020-11-20T12:19:22.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.320+0000 7f82d27fc700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82b00557f0 0x7f82b0057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.320+0000 7f82d27fc700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82b00557f0 0x7f82b0057cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f82d40c9ce0 tx=0x7f82c8007c20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:22.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.324+0000 7f82baffd700 1 -- 172.21.15.116:0/1233547586 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f82c4075020 con 0x7f82d4060530 2020-11-20T12:19:22.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.484+0000 7f82d9515700 1 -- 172.21.15.116:0/1233547586 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f82d4069950 con 0x7f82d4060530 2020-11-20T12:19:22.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.484+0000 7f82baffd700 1 -- 172.21.15.116:0/1233547586 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v71) v1 ==== 92+0+10 (secure 0 0 0) 0x7f82c4023030 con 0x7f82d4060530 2020-11-20T12:19:22.484 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:22.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 -- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82b00557f0 msgr2=0x7f82b0057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82b00557f0 0x7f82b0057cb0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f82d40c9ce0 tx=0x7f82c8007c20).stop 2020-11-20T12:19:22.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 -- 172.21.15.116:0/1233547586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82d4060530 msgr2=0x7f82d40acf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82d4060530 0x7f82d40acf70 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f82c4006fd0 tx=0x7f82c4015000).stop 2020-11-20T12:19:22.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 -- 172.21.15.116:0/1233547586 shutdown_connections 2020-11-20T12:19:22.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82b00557f0 0x7f82b0057cb0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82d4151770 0x7f82d414efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82d4060530 0x7f82d40acf70 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 --2- 172.21.15.116:0/1233547586 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82d40c4380 0x7f82d41648e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 -- 172.21.15.116:0/1233547586 >> 172.21.15.116:0/1233547586 conn(0x7f82d40ca810 msgr2=0x7f82d40657c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:22.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 -- 172.21.15.116:0/1233547586 shutdown_connections 2020-11-20T12:19:22.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.488+0000 7f82d9515700 1 -- 172.21.15.116:0/1233547586 wait complete. 2020-11-20T12:19:22.509 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:19:22.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.692+0000 7f975e020700 1 Processor -- start 2020-11-20T12:19:22.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.692+0000 7f975e020700 1 -- start start 2020-11-20T12:19:22.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.692+0000 7f975e020700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9758156060 0x7f97580c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.692+0000 7f975e020700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97580c12c0 0x7f97580bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.692+0000 7f975e020700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97580b93c0 con 0x7f9758155c80 2020-11-20T12:19:22.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.692+0000 7f975e020700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97580ba580 con 0x7f9758156060 2020-11-20T12:19:22.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.692+0000 7f975e020700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97580635e0 con 0x7f97580c12c0 2020-11-20T12:19:22.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f975c81d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9758156060 0x7f97580c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f975d01e700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f9758155c80 0x7f9758062ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:53640/0 (socket says 172.21.15.116:53640) 2020-11-20T12:19:22.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f975d01e700 1 -- 172.21.15.116:0/4273908762 learned_addr learned my addr 172.21.15.116:0/4273908762 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:22.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f975d81f700 1 --2- 172.21.15.116:0/4273908762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97580c12c0 0x7f97580bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f9747fff700 1 -- 172.21.15.116:0/4273908762 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818642412 0 0) 0x7f97580b93c0 con 0x7f9758155c80 2020-11-20T12:19:22.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f9747fff700 1 -- 172.21.15.116:0/4273908762 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9740003240 con 0x7f9758155c80 2020-11-20T12:19:22.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f975c81d700 1 -- 172.21.15.116:0/4273908762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97580c12c0 msgr2=0x7f97580bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:22.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f975c81d700 1 --2- 172.21.15.116:0/4273908762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97580c12c0 0x7f97580bed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f975c81d700 1 -- 172.21.15.116:0/4273908762 >> v1:172.21.15.173:6789/0 conn(0x7f9758155c80 legacy=0x7f9758062ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f975c81d700 1 -- 172.21.15.116:0/4273908762 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580b9660 con 0x7f9758156060 2020-11-20T12:19:22.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f975d81f700 1 --2- 172.21.15.116:0/4273908762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97580c12c0 0x7f97580bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:22.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f975c81d700 1 --2- 172.21.15.116:0/4273908762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9758156060 0x7f97580c0dd0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f974800b9a0 tx=0x7f9748006df0).ready entity=mon.0 client_cookie=1346179e705944bd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:22.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f9747fff700 1 -- 172.21.15.116:0/4273908762 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748009680 con 0x7f9758156060 2020-11-20T12:19:22.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f9747fff700 1 -- 172.21.15.116:0/4273908762 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97480097e0 con 0x7f9758156060 2020-11-20T12:19:22.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.696+0000 7f9747fff700 1 -- 172.21.15.116:0/4273908762 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748009b60 con 0x7f9758156060 2020-11-20T12:19:22.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.700+0000 7f975e020700 1 -- 172.21.15.116:0/4273908762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9758156060 msgr2=0x7f97580c0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.700+0000 7f975e020700 1 --2- 172.21.15.116:0/4273908762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9758156060 0x7f97580c0dd0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f974800b9a0 tx=0x7f9748006df0).stop 2020-11-20T12:19:22.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.700+0000 7f975e020700 1 -- 172.21.15.116:0/4273908762 shutdown_connections 2020-11-20T12:19:22.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.700+0000 7f975e020700 1 --2- 172.21.15.116:0/4273908762 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f97580c12c0 0x7f97580bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.700+0000 7f975e020700 1 --2- 172.21.15.116:0/4273908762 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9758156060 0x7f97580c0dd0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.700+0000 7f975e020700 1 -- 172.21.15.116:0/4273908762 >> 172.21.15.116:0/4273908762 conn(0x7f9758068350 msgr2=0x7f97580627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:22.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.700+0000 7f975e020700 1 -- 172.21.15.116:0/4273908762 shutdown_connections 2020-11-20T12:19:22.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.700+0000 7f975e020700 1 -- 172.21.15.116:0/4273908762 wait complete. 2020-11-20T12:19:22.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.700+0000 7f975e020700 1 Processor -- start 2020-11-20T12:19:22.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.700+0000 7f975e020700 1 -- start start 2020-11-20T12:19:22.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975e020700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97580c12c0 0x7f97581661c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975e020700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9758155c80 0x7f97580a8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975e020700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97580a9e70 0x7f9758164ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975e020700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97580bb990 con 0x7f97580a9e70 2020-11-20T12:19:22.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975e020700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97581666b0 con 0x7f97580c12c0 2020-11-20T12:19:22.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975e020700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97580aa3a0 con 0x7f9758155c80 2020-11-20T12:19:22.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975d81f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97580a9e70 0x7f9758164ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975d81f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97580a9e70 0x7f9758164ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47986/0 (socket says 172.21.15.116:47986) 2020-11-20T12:19:22.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975d81f700 1 -- 172.21.15.116:0/3627523776 learned_addr learned my addr 172.21.15.116:0/3627523776 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:22.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975d01e700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97580c12c0 0x7f97581661c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975c81d700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9758155c80 0x7f97580a8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975d81f700 1 -- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9758155c80 msgr2=0x7f97580a8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975d81f700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9758155c80 0x7f97580a8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975d81f700 1 -- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97580c12c0 msgr2=0x7f97581661c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975d81f700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97580c12c0 0x7f97581661c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975d81f700 1 -- 172.21.15.116:0/3627523776 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97580bdee0 con 0x7f97580a9e70 2020-11-20T12:19:22.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975d81f700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97580a9e70 0x7f9758164ee0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f975000ef30 tx=0x7f97500091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:22.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f97457fa700 1 -- 172.21.15.116:0/3627523776 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975000cc90 con 0x7f97580a9e70 2020-11-20T12:19:22.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f97457fa700 1 -- 172.21.15.116:0/3627523776 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975000cdf0 con 0x7f97580a9e70 2020-11-20T12:19:22.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975e020700 1 -- 172.21.15.116:0/3627523776 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97580be180 con 0x7f97580a9e70 2020-11-20T12:19:22.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f97457fa700 1 -- 172.21.15.116:0/3627523776 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9750012620 con 0x7f97580a9e70 2020-11-20T12:19:22.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.704+0000 7f975e020700 1 -- 172.21.15.116:0/3627523776 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97580a8f80 con 0x7f97580a9e70 2020-11-20T12:19:22.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.708+0000 7f975e020700 1 -- 172.21.15.116:0/3627523776 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97580c3ff0 con 0x7f97580a9e70 2020-11-20T12:19:22.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.708+0000 7f97457fa700 1 -- 172.21.15.116:0/3627523776 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9750022070 con 0x7f97580a9e70 2020-11-20T12:19:22.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.708+0000 7f97457fa700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97300557c0 0x7f9730057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:22.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.708+0000 7f97457fa700 1 -- 172.21.15.116:0/3627523776 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f975006ef80 con 0x7f97580a9e70 2020-11-20T12:19:22.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.708+0000 7f975d01e700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97300557c0 0x7f9730057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:22.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.712+0000 7f975d01e700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97300557c0 0x7f9730057c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f9754007fd0 tx=0x7f9754009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:22.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.716+0000 7f97457fa700 1 -- 172.21.15.116:0/3627523776 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9750044e20 con 0x7f97580a9e70 2020-11-20T12:19:22.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.872+0000 7f975e020700 1 -- 172.21.15.116:0/3627523776 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f975805cf30 con 0x7f97580a9e70 2020-11-20T12:19:22.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.872+0000 7f97457fa700 1 -- 172.21.15.116:0/3627523776 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+10 (secure 0 0 0) 0x7f97500160a0 con 0x7f97580a9e70 2020-11-20T12:19:22.870 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:22.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.872+0000 7f975e020700 1 -- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97300557c0 msgr2=0x7f9730057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.872+0000 7f975e020700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97300557c0 0x7f9730057c80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f9754007fd0 tx=0x7f9754009040).stop 2020-11-20T12:19:22.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.872+0000 7f975e020700 1 -- 172.21.15.116:0/3627523776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97580a9e70 msgr2=0x7f9758164ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:22.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.872+0000 7f975e020700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97580a9e70 0x7f9758164ee0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f975000ef30 tx=0x7f97500091d0).stop 2020-11-20T12:19:22.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.876+0000 7f975e020700 1 -- 172.21.15.116:0/3627523776 shutdown_connections 2020-11-20T12:19:22.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.876+0000 7f975e020700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97300557c0 0x7f9730057c80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.876+0000 7f975e020700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97580a9e70 0x7f9758164ee0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.876+0000 7f975e020700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97580c12c0 0x7f97581661c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.876+0000 7f975e020700 1 --2- 172.21.15.116:0/3627523776 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9758155c80 0x7f97580a8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:22.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.876+0000 7f975e020700 1 -- 172.21.15.116:0/3627523776 >> 172.21.15.116:0/3627523776 conn(0x7f9758068350 msgr2=0x7f97580c0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:22.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.876+0000 7f975e020700 1 -- 172.21.15.116:0/3627523776 shutdown_connections 2020-11-20T12:19:22.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:22.876+0000 7f975e020700 1 -- 172.21.15.116:0/3627523776 wait complete. 2020-11-20T12:19:22.894 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:19:23.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.076+0000 7f6f60aa8700 1 Processor -- start 2020-11-20T12:19:23.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.076+0000 7f6f60aa8700 1 -- start start 2020-11-20T12:19:23.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.076+0000 7f6f60aa8700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6f5c0a9750 0x7f6f5c0c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.076+0000 7f6f60aa8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f5c14a1d0 0x7f6f5c0be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.076+0000 7f6f60aa8700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f5c0a9c20 con 0x7f6f5c149df0 2020-11-20T12:19:23.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.076+0000 7f6f60aa8700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f5c0a9ec0 con 0x7f6f5c0a9750 2020-11-20T12:19:23.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.076+0000 7f6f60aa8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c153670 con 0x7f6f5c14a1d0 2020-11-20T12:19:23.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.076+0000 7f6f59d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6f5c0a9750 0x7f6f5c0c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.076+0000 7f6f5a59c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6f5c149df0 0x7f6f5c153db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57424/0 (socket says 172.21.15.116:57424) 2020-11-20T12:19:23.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.076+0000 7f6f5a59c700 1 -- 172.21.15.116:0/1395511590 learned_addr learned my addr 172.21.15.116:0/1395511590 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:23.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f5ad9d700 1 --2- 172.21.15.116:0/1395511590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f5c14a1d0 0x7f6f5c0be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f59d9b700 1 -- 172.21.15.116:0/1395511590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f5c14a1d0 msgr2=0x7f6f5c0be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:23.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f59d9b700 1 --2- 172.21.15.116:0/1395511590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f5c14a1d0 0x7f6f5c0be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f59d9b700 1 -- 172.21.15.116:0/1395511590 >> v1:172.21.15.116:6789/0 conn(0x7f6f5c149df0 legacy=0x7f6f5c153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f5959a700 1 -- 172.21.15.116:0/1395511590 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1686024457 0 0) 0x7f6f5c0a9c20 con 0x7f6f5c149df0 2020-11-20T12:19:23.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f59d9b700 1 -- 172.21.15.116:0/1395511590 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c153910 con 0x7f6f5c0a9750 2020-11-20T12:19:23.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f59d9b700 1 --2- 172.21.15.116:0/1395511590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f5c0a9750 0x7f6f5c0c0700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f6f4400b9a0 tx=0x7f6f44006df0).ready entity=mon.0 client_cookie=6f23496c2ec73776 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:23.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f5959a700 1 -- 172.21.15.116:0/1395511590 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f440097b0 con 0x7f6f5c0a9750 2020-11-20T12:19:23.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f5959a700 1 -- 172.21.15.116:0/1395511590 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f44009910 con 0x7f6f5c0a9750 2020-11-20T12:19:23.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f5959a700 1 -- 172.21.15.116:0/1395511590 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f44009c90 con 0x7f6f5c0a9750 2020-11-20T12:19:23.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f60aa8700 1 -- 172.21.15.116:0/1395511590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f5c0a9750 msgr2=0x7f6f5c0c0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f60aa8700 1 --2- 172.21.15.116:0/1395511590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f5c0a9750 0x7f6f5c0c0700 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7f6f4400b9a0 tx=0x7f6f44006df0).stop 2020-11-20T12:19:23.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f60aa8700 1 -- 172.21.15.116:0/1395511590 shutdown_connections 2020-11-20T12:19:23.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f60aa8700 1 --2- 172.21.15.116:0/1395511590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f5c14a1d0 0x7f6f5c0be240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f60aa8700 1 --2- 172.21.15.116:0/1395511590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f5c0a9750 0x7f6f5c0c0700 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f60aa8700 1 -- 172.21.15.116:0/1395511590 >> 172.21.15.116:0/1395511590 conn(0x7f6f5c05ba80 msgr2=0x7f6f5c0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:23.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f60aa8700 1 -- 172.21.15.116:0/1395511590 shutdown_connections 2020-11-20T12:19:23.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.080+0000 7f6f60aa8700 1 -- 172.21.15.116:0/1395511590 wait complete. 2020-11-20T12:19:23.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.084+0000 7f6f60aa8700 1 Processor -- start 2020-11-20T12:19:23.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.084+0000 7f6f60aa8700 1 -- start start 2020-11-20T12:19:23.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.084+0000 7f6f60aa8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f5c0a9750 0x7f6f5c147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.084+0000 7f6f60aa8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f5c149df0 0x7f6f5c148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.084+0000 7f6f60aa8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f5c14a1d0 0x7f6f5c0be270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.084+0000 7f6f60aa8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c151030 con 0x7f6f5c0a9750 2020-11-20T12:19:23.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.084+0000 7f6f60aa8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c165490 con 0x7f6f5c14a1d0 2020-11-20T12:19:23.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.084+0000 7f6f60aa8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5c151260 con 0x7f6f5c149df0 2020-11-20T12:19:23.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.084+0000 7f6f5a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f5c0a9750 0x7f6f5c147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.084+0000 7f6f5a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f5c0a9750 0x7f6f5c147b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:47996/0 (socket says 172.21.15.116:47996) 2020-11-20T12:19:23.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.084+0000 7f6f5a59c700 1 -- 172.21.15.116:0/2742754445 learned_addr learned my addr 172.21.15.116:0/2742754445 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:23.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f59d9b700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f5c149df0 0x7f6f5c148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f5ad9d700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f5c14a1d0 0x7f6f5c0be270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f5a59c700 1 -- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f5c149df0 msgr2=0x7f6f5c148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f5a59c700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f5c149df0 0x7f6f5c148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f5a59c700 1 -- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f5c14a1d0 msgr2=0x7f6f5c0be270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f5a59c700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f5c14a1d0 0x7f6f5c0be270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f5a59c700 1 -- 172.21.15.116:0/2742754445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c151500 con 0x7f6f5c0a9750 2020-11-20T12:19:23.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f5a59c700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f5c0a9750 0x7f6f5c147b60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f6f4c00c720 tx=0x7f6f4c007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:23.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f527fc700 1 -- 172.21.15.116:0/2742754445 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c007320 con 0x7f6f5c0a9750 2020-11-20T12:19:23.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f527fc700 1 -- 172.21.15.116:0/2742754445 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f4c007480 con 0x7f6f5c0a9750 2020-11-20T12:19:23.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f60aa8700 1 -- 172.21.15.116:0/2742754445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f5c1517a0 con 0x7f6f5c0a9750 2020-11-20T12:19:23.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f527fc700 1 -- 172.21.15.116:0/2742754445 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f4c005800 con 0x7f6f5c0a9750 2020-11-20T12:19:23.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f60aa8700 1 -- 172.21.15.116:0/2742754445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f5c151e90 con 0x7f6f5c0a9750 2020-11-20T12:19:23.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.088+0000 7f6f60aa8700 1 -- 172.21.15.116:0/2742754445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f5c05d2a0 con 0x7f6f5c0a9750 2020-11-20T12:19:23.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.092+0000 7f6f527fc700 1 -- 172.21.15.116:0/2742754445 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6f4c010030 con 0x7f6f5c0a9750 2020-11-20T12:19:23.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.092+0000 7f6f527fc700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f380557c0 0x7f6f38057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.092+0000 7f6f527fc700 1 -- 172.21.15.116:0/2742754445 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6f4c06e520 con 0x7f6f5c0a9750 2020-11-20T12:19:23.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.092+0000 7f6f59d9b700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f380557c0 0x7f6f38057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.092+0000 7f6f59d9b700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f380557c0 0x7f6f38057c80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f6f4400b150 tx=0x7f6f44015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:23.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.096+0000 7f6f527fc700 1 -- 172.21.15.116:0/2742754445 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6f4c0443c0 con 0x7f6f5c0a9750 2020-11-20T12:19:23.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.260+0000 7f6f60aa8700 1 -- 172.21.15.116:0/2742754445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f5c061380 con 0x7f6f5c0a9750 2020-11-20T12:19:23.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.260+0000 7f6f527fc700 1 -- 172.21.15.116:0/2742754445 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6f4c0479e0 con 0x7f6f5c0a9750 2020-11-20T12:19:23.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 -- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f380557c0 msgr2=0x7f6f38057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f380557c0 0x7f6f38057c80 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f6f4400b150 tx=0x7f6f44015000).stop 2020-11-20T12:19:23.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 -- 172.21.15.116:0/2742754445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f5c0a9750 msgr2=0x7f6f5c147b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f5c0a9750 0x7f6f5c147b60 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7f6f4c00c720 tx=0x7f6f4c007ac0).stop 2020-11-20T12:19:23.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 -- 172.21.15.116:0/2742754445 shutdown_connections 2020-11-20T12:19:23.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f380557c0 0x7f6f38057c80 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f5c14a1d0 0x7f6f5c0be270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f5c0a9750 0x7f6f5c147b60 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 --2- 172.21.15.116:0/2742754445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f5c149df0 0x7f6f5c148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 -- 172.21.15.116:0/2742754445 >> 172.21.15.116:0/2742754445 conn(0x7f6f5c05ba80 msgr2=0x7f6f5c066a00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:23.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 -- 172.21.15.116:0/2742754445 shutdown_connections 2020-11-20T12:19:23.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.264+0000 7f6f60aa8700 1 -- 172.21.15.116:0/2742754445 wait complete. 2020-11-20T12:19:23.266 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:23.283 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:19:23.283 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:19:23.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.448+0000 7fcbf0972700 1 Processor -- start 2020-11-20T12:19:23.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.448+0000 7fcbf0972700 1 -- start start 2020-11-20T12:19:23.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbf0972700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec062880 0x7fcbec062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbf0972700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcbec0b94e0 0x7fcbec0b9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbf0972700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbec0ba1b0 con 0x7fcbec14b990 2020-11-20T12:19:23.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbf0972700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbec152e60 con 0x7fcbec0b94e0 2020-11-20T12:19:23.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbf0972700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec1555a0 con 0x7fcbec062880 2020-11-20T12:19:23.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbeaffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fcbec14b990 0x7fcbec14be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57440/0 (socket says 172.21.15.116:57440) 2020-11-20T12:19:23.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbeaffd700 1 -- 172.21.15.116:0/583830091 learned_addr learned my addr 172.21.15.116:0/583830091 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:23.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbebfff700 1 --2- 172.21.15.116:0/583830091 >> v2:172.21.15.116:3300/0 conn(0x7fcbec0b94e0 0x7fcbec0b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbeb7fe700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec062880 0x7fcbec062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3626028504 0 0) 0x7fcbec0ba1b0 con 0x7fcbec14b990 2020-11-20T12:19:23.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbd8003240 con 0x7fcbec14b990 2020-11-20T12:19:23.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4212478068 0 0) 0x7fcbd8003240 con 0x7fcbec14b990 2020-11-20T12:19:23.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec062880 msgr2=0x7fcbec062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:23.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec062880 0x7fcbec062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 >> v2:172.21.15.116:3300/0 conn(0x7fcbec0b94e0 msgr2=0x7fcbec0b9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:23.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 --2- 172.21.15.116:0/583830091 >> v2:172.21.15.116:3300/0 conn(0x7fcbec0b94e0 0x7fcbec0b9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec155840 con 0x7fcbec14b990 2020-11-20T12:19:23.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fcbe0001500 con 0x7fcbec14b990 2020-11-20T12:19:23.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 >> v1:172.21.15.116:6789/0 conn(0x7fcbec14b990 legacy=0x7fcbec14be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbd80037b0 0x7fcbd8003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbd80044e0 0x7fcbd8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec062880 0x7fcbd8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec152e60 con 0x7fcbd80037b0 2020-11-20T12:19:23.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec1555a0 con 0x7fcbd80044e0 2020-11-20T12:19:23.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd8005340 con 0x7fcbec062880 2020-11-20T12:19:23.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.452+0000 7fcbebfff700 1 --2- 172.21.15.116:0/583830091 >> v2:172.21.15.116:3300/0 conn(0x7fcbec0b94e0 0x7fcbec0b9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:23.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbeb7fe700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbd80037b0 0x7fcbd8003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbeaffd700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbd80044e0 0x7fcbd8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbebfff700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec062880 0x7fcbd8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbeb7fe700 1 -- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec062880 msgr2=0x7fcbd8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:23.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbeb7fe700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec062880 0x7fcbd8004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbeb7fe700 1 -- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbd80044e0 msgr2=0x7fcbd8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:23.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbeb7fe700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbd80044e0 0x7fcbd8004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbeb7fe700 1 -- 172.21.15.116:0/583830091 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd8003240 con 0x7fcbd80037b0 2020-11-20T12:19:23.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbebfff700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec062880 0x7fcbd8004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:23.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbeb7fe700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbd80037b0 0x7fcbd8003bd0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fcbdc009ea0 tx=0x7fcbdc0091d0).ready entity=mon.0 client_cookie=d67c6e8b2c8cc353 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:23.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0095c0 con 0x7fcbd80037b0 2020-11-20T12:19:23.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc00ecf0 con 0x7fcbd80037b0 2020-11-20T12:19:23.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbea7fc700 1 -- 172.21.15.116:0/583830091 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc012600 con 0x7fcbd80037b0 2020-11-20T12:19:23.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbf0972700 1 -- 172.21.15.116:0/583830091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbd80037b0 msgr2=0x7fcbd8003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbf0972700 1 --2- 172.21.15.116:0/583830091 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbd80037b0 0x7fcbd8003bd0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fcbdc009ea0 tx=0x7fcbdc0091d0).stop 2020-11-20T12:19:23.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbebfff700 1 -- 172.21.15.116:0/583830091 reap_dead start 2020-11-20T12:19:23.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbf0972700 1 -- 172.21.15.116:0/583830091 shutdown_connections 2020-11-20T12:19:23.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbf0972700 1 -- 172.21.15.116:0/583830091 >> 172.21.15.116:0/583830091 conn(0x7fcbec060c40 msgr2=0x7fcbec0c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:23.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbf0972700 1 -- 172.21.15.116:0/583830091 shutdown_connections 2020-11-20T12:19:23.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.456+0000 7fcbf0972700 1 -- 172.21.15.116:0/583830091 wait complete. 2020-11-20T12:19:23.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.460+0000 7fcbf0972700 1 Processor -- start 2020-11-20T12:19:23.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.460+0000 7fcbf0972700 1 -- start start 2020-11-20T12:19:23.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.460+0000 7fcbf0972700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbec164e70 0x7fcbec160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.460+0000 7fcbf0972700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec165250 0x7fcbec154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.460+0000 7fcbf0972700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbec1645c0 0x7fcbec154610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.460+0000 7fcbf0972700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec161770 con 0x7fcbec164e70 2020-11-20T12:19:23.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.460+0000 7fcbf0972700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec161a10 con 0x7fcbec1645c0 2020-11-20T12:19:23.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.460+0000 7fcbf0972700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbec161cb0 con 0x7fcbec165250 2020-11-20T12:19:23.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbeb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbec164e70 0x7fcbec160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbeb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbec164e70 0x7fcbec160fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48016/0 (socket says 172.21.15.116:48016) 2020-11-20T12:19:23.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbeb7fe700 1 -- 172.21.15.116:0/2564828593 learned_addr learned my addr 172.21.15.116:0/2564828593 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:23.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbebfff700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbec1645c0 0x7fcbec154610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbeaffd700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec165250 0x7fcbec154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbeb7fe700 1 -- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec165250 msgr2=0x7fcbec154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbeb7fe700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec165250 0x7fcbec154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbeb7fe700 1 -- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbec1645c0 msgr2=0x7fcbec154610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbeb7fe700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbec1645c0 0x7fcbec154610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbeb7fe700 1 -- 172.21.15.116:0/2564828593 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbec0a8390 con 0x7fcbec164e70 2020-11-20T12:19:23.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbeb7fe700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbec164e70 0x7fcbec160fb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fcbdc014000 tx=0x7fcbdc01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:23.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbcb7fe700 1 -- 172.21.15.116:0/2564828593 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc01b630 con 0x7fcbec164e70 2020-11-20T12:19:23.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbcb7fe700 1 -- 172.21.15.116:0/2564828593 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc01b790 con 0x7fcbec164e70 2020-11-20T12:19:23.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbcb7fe700 1 -- 172.21.15.116:0/2564828593 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc01bb10 con 0x7fcbec164e70 2020-11-20T12:19:23.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbf0972700 1 -- 172.21.15.116:0/2564828593 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbec1614a0 con 0x7fcbec164e70 2020-11-20T12:19:23.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.464+0000 7fcbf0972700 1 -- 172.21.15.116:0/2564828593 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbec0bb330 con 0x7fcbec164e70 2020-11-20T12:19:23.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.468+0000 7fcbf0972700 1 -- 172.21.15.116:0/2564828593 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbec0bb520 con 0x7fcbec164e70 2020-11-20T12:19:23.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.468+0000 7fcbcb7fe700 1 -- 172.21.15.116:0/2564828593 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fcbdc02a020 con 0x7fcbec164e70 2020-11-20T12:19:23.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.468+0000 7fcbcb7fe700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcbc40557c0 0x7fcbc4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:23.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.468+0000 7fcbcb7fe700 1 -- 172.21.15.116:0/2564828593 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcbdc0706d0 con 0x7fcbec164e70 2020-11-20T12:19:23.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.468+0000 7fcbeaffd700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcbc40557c0 0x7fcbc4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:23.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.468+0000 7fcbeaffd700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcbc40557c0 0x7fcbc4057c80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7fcbe0000dd0 tx=0x7fcbec0be970).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:23.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.476+0000 7fcbcb7fe700 1 -- 172.21.15.116:0/2564828593 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcbdc046520 con 0x7fcbec164e70 2020-11-20T12:19:23.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.684+0000 7fcbf0972700 1 -- 172.21.15.116:0/2564828593 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbec05cf30 con 0x7fcbec164e70 2020-11-20T12:19:23.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.684+0000 7fcbcb7fe700 1 -- 172.21.15.116:0/2564828593 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2854 (secure 0 0 0) 0x7fcbdc016b40 con 0x7fcbec164e70 2020-11-20T12:19:23.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 -- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcbc40557c0 msgr2=0x7fcbc4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcbc40557c0 0x7fcbc4057c80 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7fcbe0000dd0 tx=0x7fcbec0be970).stop 2020-11-20T12:19:23.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 -- 172.21.15.116:0/2564828593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbec164e70 msgr2=0x7fcbec160fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:23.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbec164e70 0x7fcbec160fb0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7fcbdc014000 tx=0x7fcbdc01a000).stop 2020-11-20T12:19:23.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 -- 172.21.15.116:0/2564828593 shutdown_connections 2020-11-20T12:19:23.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcbc40557c0 0x7fcbc4057c80 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcbec1645c0 0x7fcbec154610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcbec164e70 0x7fcbec160fb0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 --2- 172.21.15.116:0/2564828593 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcbec165250 0x7fcbec154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:23.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 -- 172.21.15.116:0/2564828593 >> 172.21.15.116:0/2564828593 conn(0x7fcbec060c40 msgr2=0x7fcbec14ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:23.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 -- 172.21.15.116:0/2564828593 shutdown_connections 2020-11-20T12:19:23.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:23.688+0000 7fcbf0972700 1 -- 172.21.15.116:0/2564828593 wait complete. 2020-11-20T12:19:23.725 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 37 2020-11-20T12:19:23.726 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:23.726 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3869489418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3869489418},{"type":"v1","addr":"172.21.15.173:6837","nonce":3869489418}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/2912073472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2912073472},{"type":"v1","addr":"172.21.15.173:6835","nonce":2912073472}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/2221764249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2221764249},{"type":"v1","addr":"172.21.15.116:6834","nonce":2221764249}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:12.981713+0000","modified":"2020-11-20T12:19:14.999541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1946956517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1946956517},{"type":"v1","addr":"172.21.15.116:6835","nonce":1946956517}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-20T12:19:24.130 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:19:12.981713+0000', 'modified': '2020-11-20T12:19:14.999541+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6429}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6429': {'gid': 6429, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/1946956517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 1946956517}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 1946956517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:19:24.131 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:19:24.132 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:19:24.332 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":6429,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.329766586,"mdsmap_epoch":37,"osdmap_epoch":71,"osdmap_epoch_barrier":71,"uptime":16.806873019000001} 2020-11-20T12:19:24.333 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 6429, "mdsmap_epoch": 37, "osdmap_epoch": 71, "osdmap_epoch_barrier": 71, "rank_uptime": 11.329766586, "state": "up:active", "uptime": 16.806873019, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:19:24.333 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:19:24.334 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:24.334 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-20T12:19:24.355 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:19:24.356 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:24.473 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:24.474 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:19:24.497 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:24.497 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:19:24.502 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:19:24.503 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:24.503 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-20T12:19:24.540 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:19:24.541 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:19:24.541 INFO:tasks.cephfs.mount:self.client.name = client.2 2020-11-20T12:19:24.541 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2020-11-20T12:19:24.542 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:19:24.542 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:19:24.542 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:19:24.543 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:19:24.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:24.544 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:19:24.568 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:19:24.568 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:19:24.569 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:19:24.569 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:24.569 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:19:24.570 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:24.570 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:24.570 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:24.570 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:24.571 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:24.571 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:24.571 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:24.571 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:19:24.571 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:24.572 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:19:24.572 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:24.572 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:19:24.572 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:24.572 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:19:24.578 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:24.579 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:19:24.579 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:24.579 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:19:24.579 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:24.579 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:19:24.580 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:24.580 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:19:24.580 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:24.580 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:19:24.580 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:19:24.581 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:19:24.581 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:24.581 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:19:24.581 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:19:24.581 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:19:24.582 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:24.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:24.583 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:19:24.615 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:19:24.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:24.623 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:19:24.686 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:19:24.687 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:19:24.687 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:19:24.687 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:19:24.687 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:19:24.688 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:19:24.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:24.689 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:19:24.690 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:19:24.690 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:19:24.690 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:19:24.691 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:19:24.691 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:19:24.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:24.836 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:19:24.857 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:19:24.858 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:19:24.859 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:19:24.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:24.859 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2020-11-20T12:19:24.938 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:25.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.156+0000 7efff7187700 1 Processor -- start 2020-11-20T12:19:25.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.156+0000 7efff7187700 1 -- start start 2020-11-20T12:19:25.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.160+0000 7efff7187700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efff014e220 0x7efff00bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.160+0000 7efff7187700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff00b9d20 con 0x7efff00bfa80 2020-11-20T12:19:25.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.160+0000 7efff7187700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efff00b9f80 con 0x7efff00bf100 2020-11-20T12:19:25.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.160+0000 7efff7187700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efff00ba220 con 0x7efff014e220 2020-11-20T12:19:25.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.160+0000 7efff5724700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efff014e220 0x7efff00bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.160+0000 7efff4f23700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7efff00bfa80 0x7efff0066e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57460/0 (socket says 172.21.15.116:57460) 2020-11-20T12:19:25.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.160+0000 7efff4f23700 1 -- 172.21.15.116:0/2230367343 learned_addr learned my addr 172.21.15.116:0/2230367343 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:25.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.160+0000 7efff5724700 1 -- 172.21.15.116:0/2230367343 >> v1:172.21.15.173:6789/0 conn(0x7efff00bf100 legacy=0x7efff014db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.160+0000 7efff5724700 1 -- 172.21.15.116:0/2230367343 >> v1:172.21.15.116:6789/0 conn(0x7efff00bfa80 legacy=0x7efff0066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.160+0000 7efff5724700 1 -- 172.21.15.116:0/2230367343 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff00ba4c0 con 0x7efff014e220 2020-11-20T12:19:25.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.160+0000 7effef7fe700 1 -- 172.21.15.116:0/2230367343 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509013864 0 0) 0x7efff00b9d20 con 0x7efff00bfa80 2020-11-20T12:19:25.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.164+0000 7efff5724700 1 --2- 172.21.15.116:0/2230367343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efff014e220 0x7efff00bfe60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7effe8009c90 tx=0x7effe80091d0).ready entity=mon.0 client_cookie=4fcc7a7233c51367 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:25.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.164+0000 7effef7fe700 1 -- 172.21.15.116:0/2230367343 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe800bbc0 con 0x7efff014e220 2020-11-20T12:19:25.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.164+0000 7effef7fe700 1 -- 172.21.15.116:0/2230367343 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe800bd20 con 0x7efff014e220 2020-11-20T12:19:25.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.164+0000 7effef7fe700 1 -- 172.21.15.116:0/2230367343 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8011610 con 0x7efff014e220 2020-11-20T12:19:25.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.164+0000 7efff7187700 1 -- 172.21.15.116:0/2230367343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efff014e220 msgr2=0x7efff00bfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.164+0000 7efff7187700 1 --2- 172.21.15.116:0/2230367343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efff014e220 0x7efff00bfe60 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7effe8009c90 tx=0x7effe80091d0).stop 2020-11-20T12:19:25.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.164+0000 7efff7187700 1 -- 172.21.15.116:0/2230367343 shutdown_connections 2020-11-20T12:19:25.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.164+0000 7efff7187700 1 --2- 172.21.15.116:0/2230367343 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efff014e220 0x7efff00bfe60 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.164+0000 7efff7187700 1 -- 172.21.15.116:0/2230367343 >> 172.21.15.116:0/2230367343 conn(0x7efff00c4f80 msgr2=0x7efff00c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:25.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.164+0000 7efff7187700 1 -- 172.21.15.116:0/2230367343 shutdown_connections 2020-11-20T12:19:25.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.164+0000 7efff7187700 1 -- 172.21.15.116:0/2230367343 wait complete. 2020-11-20T12:19:25.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff7187700 1 Processor -- start 2020-11-20T12:19:25.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff7187700 1 -- start start 2020-11-20T12:19:25.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff7187700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efff00bf100 0x7efff00a5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff7187700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efff00bfa80 0x7efff014c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff7187700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efff014e220 0x7efff0062040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff7187700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff00a63f0 con 0x7efff014e220 2020-11-20T12:19:25.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff7187700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff00a69d0 con 0x7efff00bfa80 2020-11-20T12:19:25.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff7187700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efff014e600 con 0x7efff00bf100 2020-11-20T12:19:25.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff5724700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efff014e220 0x7efff0062040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff5724700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efff014e220 0x7efff0062040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48032/0 (socket says 172.21.15.116:48032) 2020-11-20T12:19:25.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff5724700 1 -- 172.21.15.116:0/1141511971 learned_addr learned my addr 172.21.15.116:0/1141511971 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:25.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7efff4f23700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efff00bf100 0x7efff00a5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.168+0000 7effeffff700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efff00bfa80 0x7efff014c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7efff5724700 1 -- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efff00bf100 msgr2=0x7efff00a5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7efff5724700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efff00bf100 0x7efff00a5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7efff5724700 1 -- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efff00bfa80 msgr2=0x7efff014c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7efff5724700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efff00bfa80 0x7efff014c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7efff5724700 1 -- 172.21.15.116:0/1141511971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff014e8a0 con 0x7efff014e220 2020-11-20T12:19:25.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7efff5724700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efff014e220 0x7efff0062040 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7effe8009fd0 tx=0x7effe8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:25.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7effecff9700 1 -- 172.21.15.116:0/1141511971 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8011e30 con 0x7efff014e220 2020-11-20T12:19:25.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7efff7187700 1 -- 172.21.15.116:0/1141511971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff00bb5e0 con 0x7efff014e220 2020-11-20T12:19:25.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7effecff9700 1 -- 172.21.15.116:0/1141511971 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe8010ae0 con 0x7efff014e220 2020-11-20T12:19:25.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7effecff9700 1 -- 172.21.15.116:0/1141511971 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe8010e60 con 0x7efff014e220 2020-11-20T12:19:25.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7efff7187700 1 -- 172.21.15.116:0/1141511971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff014cc30 con 0x7efff014e220 2020-11-20T12:19:25.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7efff7187700 1 -- 172.21.15.116:0/1141511971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff00c03e0 con 0x7efff014e220 2020-11-20T12:19:25.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.172+0000 7effecff9700 1 -- 172.21.15.116:0/1141511971 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7effe8017070 con 0x7efff014e220 2020-11-20T12:19:25.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.176+0000 7effecff9700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7effc80557f0 0x7effc8057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.176+0000 7effecff9700 1 -- 172.21.15.116:0/1141511971 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7effe8070b90 con 0x7efff014e220 2020-11-20T12:19:25.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.176+0000 7efff4f23700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7effc80557f0 0x7effc8057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.176+0000 7efff4f23700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7effc80557f0 0x7effc8057cb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7effe40011a0 tx=0x7effe40079e0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:25.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.180+0000 7effecff9700 1 -- 172.21.15.116:0/1141511971 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efff00c03e0 con 0x7efff014e220 2020-11-20T12:19:25.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.340+0000 7efff7187700 1 -- 172.21.15.116:0/1141511971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efff00c03e0 con 0x7efff014e220 2020-11-20T12:19:25.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.340+0000 7effecff9700 1 -- 172.21.15.116:0/1141511971 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16877 (secure 0 0 0) 0x7efff00c03e0 con 0x7efff014e220 2020-11-20T12:19:25.340 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:25.340 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":71,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:13.986922+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:11.306087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:11.966077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:25.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 -- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7effc80557f0 msgr2=0x7effc8057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7effc80557f0 0x7effc8057cb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7effe40011a0 tx=0x7effe40079e0).stop 2020-11-20T12:19:25.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 -- 172.21.15.116:0/1141511971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efff014e220 msgr2=0x7efff0062040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efff014e220 0x7efff0062040 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7effe8009fd0 tx=0x7effe8016000).stop 2020-11-20T12:19:25.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 -- 172.21.15.116:0/1141511971 shutdown_connections 2020-11-20T12:19:25.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7effc80557f0 0x7effc8057cb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efff014e220 0x7efff0062040 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efff00bf100 0x7efff00a5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 --2- 172.21.15.116:0/1141511971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efff00bfa80 0x7efff014c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 -- 172.21.15.116:0/1141511971 >> 172.21.15.116:0/1141511971 conn(0x7efff00c4f80 msgr2=0x7efff014d020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:25.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 -- 172.21.15.116:0/1141511971 shutdown_connections 2020-11-20T12:19:25.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.344+0000 7efff7187700 1 -- 172.21.15.116:0/1141511971 wait complete. 2020-11-20T12:19:25.365 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:19:11.306087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:19:11.966077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:19:25.366 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:19:25.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.548+0000 7f94f18eb700 1 Processor -- start 2020-11-20T12:19:25.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.548+0000 7f94f18eb700 1 -- start start 2020-11-20T12:19:25.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.548+0000 7f94f18eb700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f94ec14a1d0 0x7f94ec0be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.548+0000 7f94f18eb700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec0a9c20 con 0x7f94ec0a9750 2020-11-20T12:19:25.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94f18eb700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94ec0a9ec0 con 0x7f94ec149df0 2020-11-20T12:19:25.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94f18eb700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94ec153670 con 0x7f94ec14a1d0 2020-11-20T12:19:25.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94ea7fc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f94ec0a9750 0x7f94ec0c0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57474/0 (socket says 172.21.15.116:57474) 2020-11-20T12:19:25.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94ea7fc700 1 -- 172.21.15.116:0/912484835 learned_addr learned my addr 172.21.15.116:0/912484835 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:25.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94eb7fe700 1 --2- 172.21.15.116:0/912484835 >> v2:172.21.15.173:3300/0 conn(0x7f94ec14a1d0 0x7f94ec0be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94e9ffb700 1 -- 172.21.15.116:0/912484835 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402258317 0 0) 0x7f94ec0a9c20 con 0x7f94ec0a9750 2020-11-20T12:19:25.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94e9ffb700 1 -- 172.21.15.116:0/912484835 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94d8003240 con 0x7f94ec0a9750 2020-11-20T12:19:25.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94eb7fe700 1 -- 172.21.15.116:0/912484835 >> v1:172.21.15.173:6789/0 conn(0x7f94ec149df0 legacy=0x7f94ec153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94eb7fe700 1 -- 172.21.15.116:0/912484835 >> v1:172.21.15.116:6789/0 conn(0x7f94ec0a9750 legacy=0x7f94ec0c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94eb7fe700 1 -- 172.21.15.116:0/912484835 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec153910 con 0x7f94ec14a1d0 2020-11-20T12:19:25.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94eb7fe700 1 --2- 172.21.15.116:0/912484835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94ec14a1d0 0x7f94ec0be240 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f94e4009c90 tx=0x7f94e40091d0).ready entity=mon.1 client_cookie=a4b3abb28cceaf3b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:25.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94e9ffb700 1 -- 172.21.15.116:0/912484835 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e400ba70 con 0x7f94ec14a1d0 2020-11-20T12:19:25.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94e9ffb700 1 -- 172.21.15.116:0/912484835 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e400bbd0 con 0x7f94ec14a1d0 2020-11-20T12:19:25.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.552+0000 7f94e9ffb700 1 -- 172.21.15.116:0/912484835 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e4011400 con 0x7f94ec14a1d0 2020-11-20T12:19:25.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.556+0000 7f94f18eb700 1 -- 172.21.15.116:0/912484835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94ec14a1d0 msgr2=0x7f94ec0be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.556+0000 7f94f18eb700 1 --2- 172.21.15.116:0/912484835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94ec14a1d0 0x7f94ec0be240 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f94e4009c90 tx=0x7f94e40091d0).stop 2020-11-20T12:19:25.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.556+0000 7f94f18eb700 1 -- 172.21.15.116:0/912484835 shutdown_connections 2020-11-20T12:19:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.556+0000 7f94f18eb700 1 --2- 172.21.15.116:0/912484835 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94ec14a1d0 0x7f94ec0be240 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.556+0000 7f94f18eb700 1 -- 172.21.15.116:0/912484835 >> 172.21.15.116:0/912484835 conn(0x7f94ec05ba80 msgr2=0x7f94ec0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.556+0000 7f94f18eb700 1 -- 172.21.15.116:0/912484835 shutdown_connections 2020-11-20T12:19:25.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.556+0000 7f94f18eb700 1 -- 172.21.15.116:0/912484835 wait complete. 2020-11-20T12:19:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94f18eb700 1 Processor -- start 2020-11-20T12:19:25.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94f18eb700 1 -- start start 2020-11-20T12:19:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94f18eb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94ec0a9750 0x7f94ec14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94f18eb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94ec149df0 0x7f94ec0becd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94f18eb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94ec14a1d0 0x7f94ec14d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94f18eb700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec14d790 con 0x7f94ec149df0 2020-11-20T12:19:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94f18eb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec14b120 con 0x7f94ec0a9750 2020-11-20T12:19:25.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94f18eb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ec0bf1c0 con 0x7f94ec14a1d0 2020-11-20T12:19:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94ea7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94ec149df0 0x7f94ec0becd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94ea7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94ec149df0 0x7f94ec0becd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48046/0 (socket says 172.21.15.116:48046) 2020-11-20T12:19:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94eaffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94ec0a9750 0x7f94ec14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94ea7fc700 1 -- 172.21.15.116:0/460903257 learned_addr learned my addr 172.21.15.116:0/460903257 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94eb7fe700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94ec14a1d0 0x7f94ec14d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94ea7fc700 1 -- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94ec14a1d0 msgr2=0x7f94ec14d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94ea7fc700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94ec14a1d0 0x7f94ec14d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94ea7fc700 1 -- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94ec0a9750 msgr2=0x7f94ec14ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94ea7fc700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94ec0a9750 0x7f94ec14ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94ea7fc700 1 -- 172.21.15.116:0/460903257 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ec14d550 con 0x7f94ec149df0 2020-11-20T12:19:25.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94ea7fc700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94ec149df0 0x7f94ec0becd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f94e000c6f0 tx=0x7f94e0007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94d2ffd700 1 -- 172.21.15.116:0/460903257 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e00072e0 con 0x7f94ec149df0 2020-11-20T12:19:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94d2ffd700 1 -- 172.21.15.116:0/460903257 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94e0007440 con 0x7f94ec149df0 2020-11-20T12:19:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94f18eb700 1 -- 172.21.15.116:0/460903257 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94ec14be40 con 0x7f94ec149df0 2020-11-20T12:19:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.560+0000 7f94d2ffd700 1 -- 172.21.15.116:0/460903257 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94e0005800 con 0x7f94ec149df0 2020-11-20T12:19:25.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.564+0000 7f94f18eb700 1 -- 172.21.15.116:0/460903257 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94ec14c380 con 0x7f94ec149df0 2020-11-20T12:19:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.564+0000 7f94f18eb700 1 -- 172.21.15.116:0/460903257 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94ec0a6780 con 0x7f94ec149df0 2020-11-20T12:19:25.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.564+0000 7f94d2ffd700 1 -- 172.21.15.116:0/460903257 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f94e0010070 con 0x7f94ec149df0 2020-11-20T12:19:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.568+0000 7f94d2ffd700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94c80557c0 0x7f94c8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.568+0000 7f94d2ffd700 1 -- 172.21.15.116:0/460903257 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f94e006dcb0 con 0x7f94ec149df0 2020-11-20T12:19:25.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.568+0000 7f94eaffd700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94c80557c0 0x7f94c8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.568+0000 7f94eaffd700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94c80557c0 0x7f94c8057c80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f94ec0677b0 tx=0x7f94dc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:25.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.572+0000 7f94d2ffd700 1 -- 172.21.15.116:0/460903257 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f94e0043b50 con 0x7f94ec149df0 2020-11-20T12:19:25.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.728+0000 7f94f18eb700 1 -- 172.21.15.116:0/460903257 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f94ec14d550 con 0x7f94ec149df0 2020-11-20T12:19:25.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.728+0000 7f94d2ffd700 1 -- 172.21.15.116:0/460903257 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v71) v1 ==== 110+0+10 (secure 0 0 0) 0x7f94ec14d550 con 0x7f94ec149df0 2020-11-20T12:19:25.728 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:19:25.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 -- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94c80557c0 msgr2=0x7f94c8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94c80557c0 0x7f94c8057c80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f94ec0677b0 tx=0x7f94dc009040).stop 2020-11-20T12:19:25.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 -- 172.21.15.116:0/460903257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94ec149df0 msgr2=0x7f94ec0becd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94ec149df0 0x7f94ec0becd0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7f94e000c6f0 tx=0x7f94e0007ad0).stop 2020-11-20T12:19:25.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 -- 172.21.15.116:0/460903257 shutdown_connections 2020-11-20T12:19:25.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94c80557c0 0x7f94c8057c80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94ec14a1d0 0x7f94ec14d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94ec0a9750 0x7f94ec14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 --2- 172.21.15.116:0/460903257 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94ec149df0 0x7f94ec0becd0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 -- 172.21.15.116:0/460903257 >> 172.21.15.116:0/460903257 conn(0x7f94ec05ba80 msgr2=0x7f94ec0c4430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:25.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 -- 172.21.15.116:0/460903257 shutdown_connections 2020-11-20T12:19:25.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.732+0000 7f94f18eb700 1 -- 172.21.15.116:0/460903257 wait complete. 2020-11-20T12:19:25.757 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:19:25.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.948+0000 7f6a3dbdc700 1 Processor -- start 2020-11-20T12:19:25.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.948+0000 7f6a3dbdc700 1 -- start start 2020-11-20T12:19:25.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.948+0000 7f6a3dbdc700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6a3814f880 0x7f6a380b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.948+0000 7f6a3dbdc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6a380a81f0 0x7f6a38060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.948+0000 7f6a3dbdc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a38060660 0x7f6a380a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.948+0000 7f6a3dbdc700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a380a8930 con 0x7f6a380a81f0 2020-11-20T12:19:25.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.948+0000 7f6a3dbdc700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a380ba2a0 con 0x7f6a3814f880 2020-11-20T12:19:25.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.948+0000 7f6a3dbdc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380a8690 con 0x7f6a38060660 2020-11-20T12:19:25.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a37fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6a380a81f0 0x7f6a38060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a3cbda700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6a3814f880 0x7f6a380b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a37fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6a380a81f0 0x7f6a38060170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48054/0 (socket says 172.21.15.116:48054) 2020-11-20T12:19:25.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a3cbda700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6a3814f880 0x7f6a380b9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:47104/0 (socket says 172.21.15.116:47104) 2020-11-20T12:19:25.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a37fff700 1 -- 172.21.15.116:0/3995097462 learned_addr learned my addr 172.21.15.116:0/3995097462 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:25.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a3d3db700 1 --2- 172.21.15.116:0/3995097462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a38060660 0x7f6a380a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a37fff700 1 -- 172.21.15.116:0/3995097462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a38060660 msgr2=0x7f6a380a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:25.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a37fff700 1 --2- 172.21.15.116:0/3995097462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a38060660 0x7f6a380a8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a37fff700 1 -- 172.21.15.116:0/3995097462 >> v2:172.21.15.173:3300/0 conn(0x7f6a3814f880 msgr2=0x7f6a380b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:25.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a37fff700 1 --2- 172.21.15.116:0/3995097462 >> v2:172.21.15.173:3300/0 conn(0x7f6a3814f880 0x7f6a380b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a37fff700 1 -- 172.21.15.116:0/3995097462 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a380b94d0 con 0x7f6a380a81f0 2020-11-20T12:19:25.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a37fff700 1 --2- 172.21.15.116:0/3995097462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a380a81f0 0x7f6a38060170 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f6a38065ab0 tx=0x7f6a28009290).ready entity=mon.0 client_cookie=9eaee872df84788e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:25.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a377fe700 1 -- 172.21.15.116:0/3995097462 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2800bb10 con 0x7f6a380a81f0 2020-11-20T12:19:25.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a377fe700 1 -- 172.21.15.116:0/3995097462 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2800bc70 con 0x7f6a380a81f0 2020-11-20T12:19:25.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.952+0000 7f6a377fe700 1 -- 172.21.15.116:0/3995097462 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a28011400 con 0x7f6a380a81f0 2020-11-20T12:19:25.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.956+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/3995097462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a380a81f0 msgr2=0x7f6a38060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.956+0000 7f6a3dbdc700 1 --2- 172.21.15.116:0/3995097462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a380a81f0 0x7f6a38060170 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f6a38065ab0 tx=0x7f6a28009290).stop 2020-11-20T12:19:25.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.956+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/3995097462 shutdown_connections 2020-11-20T12:19:25.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.956+0000 7f6a3dbdc700 1 --2- 172.21.15.116:0/3995097462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a38060660 0x7f6a380a8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.956+0000 7f6a3dbdc700 1 --2- 172.21.15.116:0/3995097462 >> v2:172.21.15.173:3300/0 conn(0x7f6a3814f880 0x7f6a380b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.956+0000 7f6a3dbdc700 1 --2- 172.21.15.116:0/3995097462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a380a81f0 0x7f6a38060170 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.956+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/3995097462 >> 172.21.15.116:0/3995097462 conn(0x7f6a38068350 msgr2=0x7f6a380627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:25.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.956+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/3995097462 shutdown_connections 2020-11-20T12:19:25.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.956+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/3995097462 wait complete. 2020-11-20T12:19:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.960+0000 7f6a3dbdc700 1 Processor -- start 2020-11-20T12:19:25.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.960+0000 7f6a3dbdc700 1 -- start start 2020-11-20T12:19:25.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a3dbdc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a38060660 0x7f6a38161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a3dbdc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a3814f880 0x7f6a380c2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a3dbdc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a38165540 0x7f6a380a6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a3dbdc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a380c3840 con 0x7f6a38060660 2020-11-20T12:19:25.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a3dbdc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a38161580 con 0x7f6a3814f880 2020-11-20T12:19:25.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a3dbdc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a38161820 con 0x7f6a38165540 2020-11-20T12:19:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a3cbda700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a38060660 0x7f6a38161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a3814f880 0x7f6a380c2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a37fff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a3814f880 0x7f6a380c2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:47112/0 (socket says 172.21.15.116:47112) 2020-11-20T12:19:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a37fff700 1 -- 172.21.15.116:0/851681289 learned_addr learned my addr 172.21.15.116:0/851681289 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:25.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a3d3db700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a38165540 0x7f6a380a6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a37fff700 1 -- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a38165540 msgr2=0x7f6a380a6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a37fff700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a38165540 0x7f6a380a6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a37fff700 1 -- 172.21.15.116:0/851681289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a38060660 msgr2=0x7f6a38161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:25.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a37fff700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a38060660 0x7f6a38161f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:25.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a37fff700 1 -- 172.21.15.116:0/851681289 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a38161ac0 con 0x7f6a3814f880 2020-11-20T12:19:25.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a37fff700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a3814f880 0x7f6a380c2ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f6a28011000 tx=0x7f6a28016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:25.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a34ff9700 1 -- 172.21.15.116:0/851681289 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2801c470 con 0x7f6a3814f880 2020-11-20T12:19:25.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a34ff9700 1 -- 172.21.15.116:0/851681289 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a2801c5d0 con 0x7f6a3814f880 2020-11-20T12:19:25.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/851681289 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a381612b0 con 0x7f6a3814f880 2020-11-20T12:19:25.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a34ff9700 1 -- 172.21.15.116:0/851681289 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a2801c970 con 0x7f6a3814f880 2020-11-20T12:19:25.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.964+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/851681289 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a380c35d0 con 0x7f6a3814f880 2020-11-20T12:19:25.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.968+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/851681289 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a38161c50 con 0x7f6a3814f880 2020-11-20T12:19:25.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.968+0000 7f6a34ff9700 1 -- 172.21.15.116:0/851681289 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6a28017070 con 0x7f6a3814f880 2020-11-20T12:19:25.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.968+0000 7f6a34ff9700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6a140557c0 0x7f6a14057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:25.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.968+0000 7f6a34ff9700 1 -- 172.21.15.116:0/851681289 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6a280705b0 con 0x7f6a3814f880 2020-11-20T12:19:25.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.968+0000 7f6a3cbda700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6a140557c0 0x7f6a14057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:25.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.972+0000 7f6a3cbda700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6a140557c0 0x7f6a14057c80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f6a20006fd0 tx=0x7f6a20006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:25.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:25.976+0000 7f6a34ff9700 1 -- 172.21.15.116:0/851681289 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6a28046970 con 0x7f6a3814f880 2020-11-20T12:19:26.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.132+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/851681289 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6a3805cf30 con 0x7f6a3814f880 2020-11-20T12:19:26.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a34ff9700 1 -- 172.21.15.116:0/851681289 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v71) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6a28015020 con 0x7f6a3814f880 2020-11-20T12:19:26.133 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:26.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6a140557c0 msgr2=0x7f6a14057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6a140557c0 0x7f6a14057c80 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f6a20006fd0 tx=0x7f6a20006df0).stop 2020-11-20T12:19:26.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a3814f880 msgr2=0x7f6a380c2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a3814f880 0x7f6a380c2ec0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f6a28011000 tx=0x7f6a28016000).stop 2020-11-20T12:19:26.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/851681289 shutdown_connections 2020-11-20T12:19:26.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6a140557c0 0x7f6a14057c80 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6a38165540 0x7f6a380a6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6a38060660 0x7f6a38161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 --2- 172.21.15.116:0/851681289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6a3814f880 0x7f6a380c2ec0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/851681289 >> 172.21.15.116:0/851681289 conn(0x7f6a38068350 msgr2=0x7f6a380bdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:26.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/851681289 shutdown_connections 2020-11-20T12:19:26.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.136+0000 7f6a3dbdc700 1 -- 172.21.15.116:0/851681289 wait complete. 2020-11-20T12:19:26.160 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:19:26.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.336+0000 7f1729835700 1 Processor -- start 2020-11-20T12:19:26.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.336+0000 7f1729835700 1 -- start start 2020-11-20T12:19:26.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1729835700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1724155c80 0x7f1724062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:26.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1729835700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1724156060 0x7f17240c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:26.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1729835700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17240b93c0 con 0x7f17240c12c0 2020-11-20T12:19:26.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1729835700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17240ba580 con 0x7f1724155c80 2020-11-20T12:19:26.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1729835700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17240b9660 con 0x7f1724156060 2020-11-20T12:19:26.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1728833700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1724155c80 0x7f1724062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:26.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1728833700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1724155c80 0x7f1724062ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48068/0 (socket says 172.21.15.116:48068) 2020-11-20T12:19:26.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1728833700 1 -- 172.21.15.116:0/3391616712 learned_addr learned my addr 172.21.15.116:0/3391616712 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:26.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1729034700 1 --1- 172.21.15.116:0/3391616712 >> v1:172.21.15.173:6789/0 conn(0x7f17240c12c0 0x7f1724060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:53734/0 (socket says 172.21.15.116:53734) 2020-11-20T12:19:26.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1723fff700 1 --2- 172.21.15.116:0/3391616712 >> v2:172.21.15.173:3300/0 conn(0x7f1724156060 0x7f17240c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:26.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1728833700 1 -- 172.21.15.116:0/3391616712 >> v2:172.21.15.173:3300/0 conn(0x7f1724156060 msgr2=0x7f17240c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:26.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1728833700 1 --2- 172.21.15.116:0/3391616712 >> v2:172.21.15.173:3300/0 conn(0x7f1724156060 0x7f17240c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1728833700 1 -- 172.21.15.116:0/3391616712 >> v1:172.21.15.173:6789/0 conn(0x7f17240c12c0 legacy=0x7f1724060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1728833700 1 -- 172.21.15.116:0/3391616712 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17240b9900 con 0x7f1724155c80 2020-11-20T12:19:26.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f1728833700 1 --2- 172.21.15.116:0/3391616712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1724155c80 0x7f1724062ed0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f1724065d50 tx=0x7f1714009290).ready entity=mon.0 client_cookie=3153f8afba886720 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f17237fe700 1 -- 172.21.15.116:0/3391616712 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171400bbc0 con 0x7f1724155c80 2020-11-20T12:19:26.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f17237fe700 1 -- 172.21.15.116:0/3391616712 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171400bd20 con 0x7f1724155c80 2020-11-20T12:19:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.340+0000 7f17237fe700 1 -- 172.21.15.116:0/3391616712 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714011610 con 0x7f1724155c80 2020-11-20T12:19:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.344+0000 7f1729835700 1 -- 172.21.15.116:0/3391616712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1724155c80 msgr2=0x7f1724062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.344+0000 7f1729835700 1 --2- 172.21.15.116:0/3391616712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1724155c80 0x7f1724062ed0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f1724065d50 tx=0x7f1714009290).stop 2020-11-20T12:19:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.344+0000 7f1729835700 1 -- 172.21.15.116:0/3391616712 shutdown_connections 2020-11-20T12:19:26.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.344+0000 7f1729835700 1 --2- 172.21.15.116:0/3391616712 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1724155c80 0x7f1724062ed0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.344+0000 7f1729835700 1 --2- 172.21.15.116:0/3391616712 >> v2:172.21.15.173:3300/0 conn(0x7f1724156060 0x7f17240c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.344+0000 7f1729835700 1 -- 172.21.15.116:0/3391616712 >> 172.21.15.116:0/3391616712 conn(0x7f1724068350 msgr2=0x7f17240627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.344+0000 7f1729835700 1 -- 172.21.15.116:0/3391616712 shutdown_connections 2020-11-20T12:19:26.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.344+0000 7f1729835700 1 -- 172.21.15.116:0/3391616712 wait complete. 2020-11-20T12:19:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.344+0000 7f1729835700 1 Processor -- start 2020-11-20T12:19:26.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1729835700 1 -- start start 2020-11-20T12:19:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1729835700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17240c12c0 0x7f17240bb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1729835700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1724155c80 0x7f1724150c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1729835700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1724156060 0x7f17240a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1729835700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1724159c30 con 0x7f17240c12c0 2020-11-20T12:19:26.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1729835700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17240bbac0 con 0x7f1724155c80 2020-11-20T12:19:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1729835700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17240bbce0 con 0x7f1724156060 2020-11-20T12:19:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1728833700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17240c12c0 0x7f17240bb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1728833700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17240c12c0 0x7f17240bb5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48072/0 (socket says 172.21.15.116:48072) 2020-11-20T12:19:26.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1728833700 1 -- 172.21.15.116:0/3028806488 learned_addr learned my addr 172.21.15.116:0/3028806488 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1723fff700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1724155c80 0x7f1724150c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1729034700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1724156060 0x7f17240a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1728833700 1 -- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1724156060 msgr2=0x7f17240a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1728833700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1724156060 0x7f17240a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1728833700 1 -- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1724155c80 msgr2=0x7f1724150c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1728833700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1724155c80 0x7f1724150c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1728833700 1 -- 172.21.15.116:0/3028806488 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1724157350 con 0x7f17240c12c0 2020-11-20T12:19:26.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1728833700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17240c12c0 0x7f17240bb5d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f1714001130 tx=0x7f1714016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:26.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1720ff9700 1 -- 172.21.15.116:0/3028806488 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171401b6a0 con 0x7f17240c12c0 2020-11-20T12:19:26.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1720ff9700 1 -- 172.21.15.116:0/3028806488 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171401b800 con 0x7f17240c12c0 2020-11-20T12:19:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1720ff9700 1 -- 172.21.15.116:0/3028806488 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171401bba0 con 0x7f17240c12c0 2020-11-20T12:19:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.348+0000 7f1729835700 1 -- 172.21.15.116:0/3028806488 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17240bde10 con 0x7f17240c12c0 2020-11-20T12:19:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.352+0000 7f1729835700 1 -- 172.21.15.116:0/3028806488 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17240be350 con 0x7f17240c12c0 2020-11-20T12:19:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.352+0000 7f1729835700 1 -- 172.21.15.116:0/3028806488 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17241574e0 con 0x7f17240c12c0 2020-11-20T12:19:26.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.352+0000 7f1720ff9700 1 -- 172.21.15.116:0/3028806488 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1714017070 con 0x7f17240c12c0 2020-11-20T12:19:26.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.352+0000 7f1720ff9700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16fc0557c0 0x7f16fc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:26.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.352+0000 7f1720ff9700 1 -- 172.21.15.116:0/3028806488 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1714070600 con 0x7f17240c12c0 2020-11-20T12:19:26.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.352+0000 7f1723fff700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16fc0557c0 0x7f16fc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:26.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.356+0000 7f1723fff700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16fc0557c0 0x7f16fc057c80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f1724158e20 tx=0x7f1718008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:26.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.360+0000 7f1720ff9700 1 -- 172.21.15.116:0/3028806488 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1714046520 con 0x7f17240c12c0 2020-11-20T12:19:26.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.516+0000 7f1729835700 1 -- 172.21.15.116:0/3028806488 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f172405cf30 con 0x7f17240c12c0 2020-11-20T12:19:26.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.516+0000 7f1720ff9700 1 -- 172.21.15.116:0/3028806488 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+10 (secure 0 0 0) 0x7f17140150a0 con 0x7f17240c12c0 2020-11-20T12:19:26.516 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:26.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 -- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16fc0557c0 msgr2=0x7f16fc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16fc0557c0 0x7f16fc057c80 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f1724158e20 tx=0x7f1718008040).stop 2020-11-20T12:19:26.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 -- 172.21.15.116:0/3028806488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17240c12c0 msgr2=0x7f17240bb5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17240c12c0 0x7f17240bb5d0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f1714001130 tx=0x7f1714016000).stop 2020-11-20T12:19:26.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 -- 172.21.15.116:0/3028806488 shutdown_connections 2020-11-20T12:19:26.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16fc0557c0 0x7f16fc057c80 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1724156060 0x7f17240a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f17240c12c0 0x7f17240bb5d0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 --2- 172.21.15.116:0/3028806488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1724155c80 0x7f1724150c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 -- 172.21.15.116:0/3028806488 >> 172.21.15.116:0/3028806488 conn(0x7f1724068350 msgr2=0x7f1724155000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:26.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 -- 172.21.15.116:0/3028806488 shutdown_connections 2020-11-20T12:19:26.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.520+0000 7f1729835700 1 -- 172.21.15.116:0/3028806488 wait complete. 2020-11-20T12:19:26.542 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:19:26.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.728+0000 7f574227c700 1 Processor -- start 2020-11-20T12:19:26.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.728+0000 7f574227c700 1 -- start start 2020-11-20T12:19:26.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.728+0000 7f574227c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f573c1551b0 0x7f573c156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:26.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.728+0000 7f574227c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c0bdfc0 con 0x7f573c0bd810 2020-11-20T12:19:26.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.728+0000 7f574227c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f573c0bdd20 con 0x7f573c154dd0 2020-11-20T12:19:26.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.728+0000 7f574227c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f573c155780 con 0x7f573c1551b0 2020-11-20T12:19:26.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f5740819700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f573c1551b0 0x7f573c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:26.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f573b7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f573c0bd810 0x7f573c0a6780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57514/0 (socket says 172.21.15.116:57514) 2020-11-20T12:19:26.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f573b7fe700 1 -- 172.21.15.116:0/2646553819 learned_addr learned my addr 172.21.15.116:0/2646553819 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:26.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f573affd700 1 -- 172.21.15.116:0/2646553819 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3964286765 0 0) 0x7f573c0bdfc0 con 0x7f573c0bd810 2020-11-20T12:19:26.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f573affd700 1 -- 172.21.15.116:0/2646553819 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5728003240 con 0x7f573c0bd810 2020-11-20T12:19:26.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f573affd700 1 -- 172.21.15.116:0/2646553819 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3777668588 0 0) 0x7f5728003240 con 0x7f573c0bd810 2020-11-20T12:19:26.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f5740819700 1 -- 172.21.15.116:0/2646553819 >> v1:172.21.15.173:6789/0 conn(0x7f573c154dd0 legacy=0x7f573c0be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:26.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f5740819700 1 -- 172.21.15.116:0/2646553819 >> v1:172.21.15.116:6789/0 conn(0x7f573c0bd810 legacy=0x7f573c0a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f5740819700 1 -- 172.21.15.116:0/2646553819 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c155a20 con 0x7f573c1551b0 2020-11-20T12:19:26.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f5740819700 1 --2- 172.21.15.116:0/2646553819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f573c1551b0 0x7f573c156160 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f5734009c90 tx=0x7f57340091d0).ready entity=mon.0 client_cookie=fde89a7f6c4fbdf0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:26.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f573affd700 1 -- 172.21.15.116:0/2646553819 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57300014e0 con 0x7f573c1551b0 2020-11-20T12:19:26.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f573affd700 1 -- 172.21.15.116:0/2646553819 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f573400b760 con 0x7f573c1551b0 2020-11-20T12:19:26.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f573affd700 1 -- 172.21.15.116:0/2646553819 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573400bae0 con 0x7f573c1551b0 2020-11-20T12:19:26.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f574227c700 1 -- 172.21.15.116:0/2646553819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f573c1551b0 msgr2=0x7f573c156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f574227c700 1 --2- 172.21.15.116:0/2646553819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f573c1551b0 0x7f573c156160 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7f5734009c90 tx=0x7f57340091d0).stop 2020-11-20T12:19:26.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f574227c700 1 -- 172.21.15.116:0/2646553819 shutdown_connections 2020-11-20T12:19:26.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f574227c700 1 --2- 172.21.15.116:0/2646553819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f573c1551b0 0x7f573c156160 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f574227c700 1 -- 172.21.15.116:0/2646553819 >> 172.21.15.116:0/2646553819 conn(0x7f573c05ba80 msgr2=0x7f573c0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:26.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.732+0000 7f574227c700 1 -- 172.21.15.116:0/2646553819 shutdown_connections 2020-11-20T12:19:26.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f574227c700 1 -- 172.21.15.116:0/2646553819 wait complete. 2020-11-20T12:19:26.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f574227c700 1 Processor -- start 2020-11-20T12:19:26.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f574227c700 1 -- start start 2020-11-20T12:19:26.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f574227c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f573c0bd810 0x7f573c164e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:26.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f574227c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f573c154dd0 0x7f573c13fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:26.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f574227c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f573c1551b0 0x7f573c13ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:26.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f574227c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c152c90 con 0x7f573c154dd0 2020-11-20T12:19:26.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f574227c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f573c152f30 con 0x7f573c0bd810 2020-11-20T12:19:26.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f574227c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f573c1531d0 con 0x7f573c1551b0 2020-11-20T12:19:26.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f573b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f573c154dd0 0x7f573c13fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:26.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f573b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f573c154dd0 0x7f573c13fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48086/0 (socket says 172.21.15.116:48086) 2020-11-20T12:19:26.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f573b7fe700 1 -- 172.21.15.116:0/1242015893 learned_addr learned my addr 172.21.15.116:0/1242015893 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:26.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.736+0000 7f5740819700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f573c1551b0 0x7f573c13ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:26.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f573bfff700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f573c0bd810 0x7f573c164e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:26.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f573b7fe700 1 -- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f573c1551b0 msgr2=0x7f573c13ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f573b7fe700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f573c1551b0 0x7f573c13ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f573b7fe700 1 -- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f573c0bd810 msgr2=0x7f573c164e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f573b7fe700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f573c0bd810 0x7f573c164e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f573b7fe700 1 -- 172.21.15.116:0/1242015893 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f573c153470 con 0x7f573c154dd0 2020-11-20T12:19:26.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f573bfff700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f573c0bd810 0x7f573c164e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:26.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f573b7fe700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f573c154dd0 0x7f573c13fa90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f573000cae0 tx=0x7f57300011f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:26.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f571bfff700 1 -- 172.21.15.116:0/1242015893 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5730007320 con 0x7f573c154dd0 2020-11-20T12:19:26.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f571bfff700 1 -- 172.21.15.116:0/1242015893 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5730007480 con 0x7f573c154dd0 2020-11-20T12:19:26.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f571bfff700 1 -- 172.21.15.116:0/1242015893 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573000a8c0 con 0x7f573c154dd0 2020-11-20T12:19:26.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f574227c700 1 -- 172.21.15.116:0/1242015893 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f573c0a7870 con 0x7f573c154dd0 2020-11-20T12:19:26.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f574227c700 1 -- 172.21.15.116:0/1242015893 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f573c151f70 con 0x7f573c154dd0 2020-11-20T12:19:26.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f5719ffb700 1 -- 172.21.15.116:0/1242015893 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570c004f90 con 0x7f573c154dd0 2020-11-20T12:19:26.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.740+0000 7f571bfff700 1 -- 172.21.15.116:0/1242015893 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5730010030 con 0x7f573c154dd0 2020-11-20T12:19:26.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.744+0000 7f571bfff700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57140557c0 0x7f5714057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:26.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.744+0000 7f571bfff700 1 -- 172.21.15.116:0/1242015893 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f573006e860 con 0x7f573c154dd0 2020-11-20T12:19:26.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.744+0000 7f573bfff700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57140557c0 0x7f5714057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:26.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.744+0000 7f573bfff700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57140557c0 0x7f5714057c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f572c002510 tx=0x7f572c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:26.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.748+0000 7f571bfff700 1 -- 172.21.15.116:0/1242015893 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5730044700 con 0x7f573c154dd0 2020-11-20T12:19:26.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f5719ffb700 1 -- 172.21.15.116:0/1242015893 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f570c003b00 con 0x7f573c154dd0 2020-11-20T12:19:26.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f571bfff700 1 -- 172.21.15.116:0/1242015893 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5730047d20 con 0x7f573c154dd0 2020-11-20T12:19:26.906 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:26.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f5719ffb700 1 -- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57140557c0 msgr2=0x7f5714057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f5719ffb700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57140557c0 0x7f5714057c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f572c002510 tx=0x7f572c009040).stop 2020-11-20T12:19:26.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f5719ffb700 1 -- 172.21.15.116:0/1242015893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f573c154dd0 msgr2=0x7f573c13fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:26.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f5719ffb700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f573c154dd0 0x7f573c13fa90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f573000cae0 tx=0x7f57300011f0).stop 2020-11-20T12:19:26.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f5719ffb700 1 -- 172.21.15.116:0/1242015893 shutdown_connections 2020-11-20T12:19:26.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f5719ffb700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57140557c0 0x7f5714057c80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f5719ffb700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f573c1551b0 0x7f573c13ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f5719ffb700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f573c0bd810 0x7f573c164e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f5719ffb700 1 --2- 172.21.15.116:0/1242015893 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f573c154dd0 0x7f573c13fa90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:26.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.908+0000 7f5719ffb700 1 -- 172.21.15.116:0/1242015893 >> 172.21.15.116:0/1242015893 conn(0x7f573c05ba80 msgr2=0x7f573c060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:26.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.912+0000 7f5719ffb700 1 -- 172.21.15.116:0/1242015893 shutdown_connections 2020-11-20T12:19:26.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:26.912+0000 7f5719ffb700 1 -- 172.21.15.116:0/1242015893 wait complete. 2020-11-20T12:19:26.933 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:19:26.934 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:19:27.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583f2f4700 1 Processor -- start 2020-11-20T12:19:27.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583f2f4700 1 -- start start 2020-11-20T12:19:27.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583f2f4700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5838153450 0x7f5838062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:27.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583f2f4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f58380677a0 0x7f58380b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:27.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583f2f4700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58381543a0 con 0x7f5838062880 2020-11-20T12:19:27.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583f2f4700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5838153920 con 0x7f58380677a0 2020-11-20T12:19:27.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583f2f4700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5838153bc0 con 0x7f5838153450 2020-11-20T12:19:27.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583e2f2700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5838062880 0x7f58380c4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57530/0 (socket says 172.21.15.116:57530) 2020-11-20T12:19:27.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583e2f2700 1 -- 172.21.15.116:0/4282277704 learned_addr learned my addr 172.21.15.116:0/4282277704 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:27.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583eaf3700 1 --2- 172.21.15.116:0/4282277704 >> v2:172.21.15.116:3300/0 conn(0x7f58380677a0 0x7f58380b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:27.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583daf1700 1 --2- 172.21.15.116:0/4282277704 >> v2:172.21.15.173:3300/0 conn(0x7f5838153450 0x7f5838062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:27.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.120+0000 7f583d2f0700 1 -- 172.21.15.116:0/4282277704 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009698218 0 0) 0x7f58381543a0 con 0x7f5838062880 2020-11-20T12:19:27.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583d2f0700 1 -- 172.21.15.116:0/4282277704 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5820003240 con 0x7f5838062880 2020-11-20T12:19:27.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583eaf3700 1 -- 172.21.15.116:0/4282277704 >> v2:172.21.15.173:3300/0 conn(0x7f5838153450 msgr2=0x7f5838062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:27.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583eaf3700 1 --2- 172.21.15.116:0/4282277704 >> v2:172.21.15.173:3300/0 conn(0x7f5838153450 0x7f5838062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:27.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583eaf3700 1 -- 172.21.15.116:0/4282277704 >> v1:172.21.15.116:6789/0 conn(0x7f5838062880 legacy=0x7f58380c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:27.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583eaf3700 1 -- 172.21.15.116:0/4282277704 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838067c30 con 0x7f58380677a0 2020-11-20T12:19:27.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583eaf3700 1 --2- 172.21.15.116:0/4282277704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58380677a0 0x7f58380b9710 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f5838064b70 tx=0x7f58300091d0).ready entity=mon.0 client_cookie=c1472701478a2008 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:27.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583d2f0700 1 -- 172.21.15.116:0/4282277704 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583000b9c0 con 0x7f58380677a0 2020-11-20T12:19:27.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583d2f0700 1 -- 172.21.15.116:0/4282277704 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f583000bb20 con 0x7f58380677a0 2020-11-20T12:19:27.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583d2f0700 1 -- 172.21.15.116:0/4282277704 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583000bea0 con 0x7f58380677a0 2020-11-20T12:19:27.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583f2f4700 1 -- 172.21.15.116:0/4282277704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58380677a0 msgr2=0x7f58380b9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:27.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583f2f4700 1 --2- 172.21.15.116:0/4282277704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58380677a0 0x7f58380b9710 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f5838064b70 tx=0x7f58300091d0).stop 2020-11-20T12:19:27.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583f2f4700 1 -- 172.21.15.116:0/4282277704 shutdown_connections 2020-11-20T12:19:27.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583f2f4700 1 --2- 172.21.15.116:0/4282277704 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f58380677a0 0x7f58380b9710 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:27.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583f2f4700 1 --2- 172.21.15.116:0/4282277704 >> v2:172.21.15.173:3300/0 conn(0x7f5838153450 0x7f5838062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:27.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583f2f4700 1 -- 172.21.15.116:0/4282277704 >> 172.21.15.116:0/4282277704 conn(0x7f5838060c40 msgr2=0x7f58380c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:27.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583f2f4700 1 -- 172.21.15.116:0/4282277704 shutdown_connections 2020-11-20T12:19:27.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.124+0000 7f583f2f4700 1 -- 172.21.15.116:0/4282277704 wait complete. 2020-11-20T12:19:27.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.128+0000 7f583f2f4700 1 Processor -- start 2020-11-20T12:19:27.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.128+0000 7f583f2f4700 1 -- start start 2020-11-20T12:19:27.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.128+0000 7f583f2f4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5838062880 0x7f58380c0f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:27.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.128+0000 7f583f2f4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58380677a0 0x7f58380a7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:27.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.128+0000 7f583f2f4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5838153450 0x7f58380a7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:27.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.128+0000 7f583f2f4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58380c47a0 con 0x7f5838062880 2020-11-20T12:19:27.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.128+0000 7f583f2f4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58380c4a40 con 0x7f5838153450 2020-11-20T12:19:27.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.128+0000 7f583f2f4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58380c4ce0 con 0x7f58380677a0 2020-11-20T12:19:27.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583e2f2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5838062880 0x7f58380c0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:27.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583e2f2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5838062880 0x7f58380c0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48100/0 (socket says 172.21.15.116:48100) 2020-11-20T12:19:27.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583e2f2700 1 -- 172.21.15.116:0/509180914 learned_addr learned my addr 172.21.15.116:0/509180914 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:27.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583daf1700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58380677a0 0x7f58380a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:27.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583eaf3700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5838153450 0x7f58380a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:27.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583e2f2700 1 -- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58380677a0 msgr2=0x7f58380a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:27.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583e2f2700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58380677a0 0x7f58380a7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:27.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583e2f2700 1 -- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5838153450 msgr2=0x7f58380a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:27.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583e2f2700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5838153450 0x7f58380a7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:27.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583e2f2700 1 -- 172.21.15.116:0/509180914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58380c4f80 con 0x7f5838062880 2020-11-20T12:19:27.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583eaf3700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5838153450 0x7f58380a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:27.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583e2f2700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5838062880 0x7f58380c0f40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f583400c6f0 tx=0x7f5834007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:27.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f58267fc700 1 -- 172.21.15.116:0/509180914 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58340072e0 con 0x7f5838062880 2020-11-20T12:19:27.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f58267fc700 1 -- 172.21.15.116:0/509180914 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5834007440 con 0x7f5838062880 2020-11-20T12:19:27.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583f2f4700 1 -- 172.21.15.116:0/509180914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5838135db0 con 0x7f5838062880 2020-11-20T12:19:27.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f58267fc700 1 -- 172.21.15.116:0/509180914 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5834005800 con 0x7f5838062880 2020-11-20T12:19:27.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583f2f4700 1 -- 172.21.15.116:0/509180914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58380c15d0 con 0x7f5838062880 2020-11-20T12:19:27.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.132+0000 7f583f2f4700 1 -- 172.21.15.116:0/509180914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58380aa4b0 con 0x7f5838062880 2020-11-20T12:19:27.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.136+0000 7f58267fc700 1 -- 172.21.15.116:0/509180914 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f583401a060 con 0x7f5838062880 2020-11-20T12:19:27.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.136+0000 7f58267fc700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58100557c0 0x7f5810057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:27.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.136+0000 7f58267fc700 1 -- 172.21.15.116:0/509180914 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f583406df00 con 0x7f5838062880 2020-11-20T12:19:27.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.136+0000 7f583daf1700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58100557c0 0x7f5810057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:27.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.140+0000 7f583daf1700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58100557c0 0x7f5810057c80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f583814ca00 tx=0x7f5828006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:27.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.144+0000 7f58267fc700 1 -- 172.21.15.116:0/509180914 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f58380aa4b0 con 0x7f5838062880 2020-11-20T12:19:27.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.348+0000 7f583f2f4700 1 -- 172.21.15.116:0/509180914 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58380aa4b0 con 0x7f5838062880 2020-11-20T12:19:27.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.348+0000 7f58267fc700 1 -- 172.21.15.116:0/509180914 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2854 (secure 0 0 0) 0x7f58380aa4b0 con 0x7f5838062880 2020-11-20T12:19:27.347 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:27.347 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3869489418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3869489418},{"type":"v1","addr":"172.21.15.173:6837","nonce":3869489418}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/2912073472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2912073472},{"type":"v1","addr":"172.21.15.173:6835","nonce":2912073472}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/2221764249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2221764249},{"type":"v1","addr":"172.21.15.116:6834","nonce":2221764249}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:12.981713+0000","modified":"2020-11-20T12:19:14.999541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1946956517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1946956517},{"type":"v1","addr":"172.21.15.116:6835","nonce":1946956517}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-20T12:19:27.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 -- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58100557c0 msgr2=0x7f5810057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:27.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58100557c0 0x7f5810057c80 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7f583814ca00 tx=0x7f5828006df0).stop 2020-11-20T12:19:27.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 -- 172.21.15.116:0/509180914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5838062880 msgr2=0x7f58380c0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:27.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5838062880 0x7f58380c0f40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f583400c6f0 tx=0x7f5834007ad0).stop 2020-11-20T12:19:27.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 -- 172.21.15.116:0/509180914 shutdown_connections 2020-11-20T12:19:27.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f58100557c0 0x7f5810057c80 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:27.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5838153450 0x7f58380a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:27.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5838062880 0x7f58380c0f40 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:27.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 --2- 172.21.15.116:0/509180914 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f58380677a0 0x7f58380a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:27.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 -- 172.21.15.116:0/509180914 >> 172.21.15.116:0/509180914 conn(0x7f5838060c40 msgr2=0x7f5838150120 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:27.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 -- 172.21.15.116:0/509180914 shutdown_connections 2020-11-20T12:19:27.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:27.352+0000 7f583f2f4700 1 -- 172.21.15.116:0/509180914 wait complete. 2020-11-20T12:19:27.352 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 37 2020-11-20T12:19:27.783 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:19:12.981713+0000', 'modified': '2020-11-20T12:19:14.999541+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6429}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6429': {'gid': 6429, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/1946956517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 1946956517}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 1946956517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:19:27.783 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:19:27.784 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:19:27.984 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":6429,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.984879363999999,"mdsmap_epoch":37,"osdmap_epoch":71,"osdmap_epoch_barrier":71,"uptime":20.461984480999998} 2020-11-20T12:19:27.984 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 6429, "mdsmap_epoch": 37, "osdmap_epoch": 71, "osdmap_epoch_barrier": 71, "rank_uptime": 14.984879364, "state": "up:active", "uptime": 20.461984481, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:19:27.985 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:19:27.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:27.986 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.2 2020-11-20T12:19:28.005 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:19:28.006 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:28.078 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:28.078 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:19:28.103 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:28.103 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:19:28.110 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:19:28.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:28.111 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2020-11-20T12:19:28.140 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:19:28.140 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:19:28.141 INFO:tasks.cephfs.mount:self.client.name = client.3 2020-11-20T12:19:28.141 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2020-11-20T12:19:28.142 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:19:28.142 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:19:28.142 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:19:28.143 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:19:28.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:28.144 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:19:28.160 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:19:28.161 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:19:28.161 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:19:28.161 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:28.162 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:19:28.162 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:28.162 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:28.162 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:28.163 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:28.163 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:28.163 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:19:28.163 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:28.163 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:19:28.164 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:28.164 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:19:28.164 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:28.165 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:19:28.165 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:28.165 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:19:28.165 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:28.166 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:19:28.166 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:28.167 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:19:28.167 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:19:28.167 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:19:28.167 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:28.167 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:19:28.168 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:28.168 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:19:28.168 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:19:28.168 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:19:28.169 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:28.169 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:19:28.169 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:19:28.169 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:19:28.169 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:19:28.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:28.171 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:19:28.203 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:19:28.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:28.208 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:19:28.269 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:19:28.270 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:19:28.270 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:19:28.270 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:19:28.270 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:19:28.271 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:19:28.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:28.272 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:19:28.272 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:19:28.272 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:19:28.273 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:19:28.273 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:19:28.274 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:19:28.378 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:28.379 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:19:28.395 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:19:28.395 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:19:28.396 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:19:28.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:28.398 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2020-11-20T12:19:28.431 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:28.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.612+0000 7fabdca20700 1 Processor -- start 2020-11-20T12:19:28.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.612+0000 7fabdca20700 1 -- start start 2020-11-20T12:19:28.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.616+0000 7fabdca20700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fabd80c3140 0x7fabd80c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:28.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.616+0000 7fabdca20700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabd80a63a0 0x7fabd80a6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:28.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.616+0000 7fabdca20700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd814ffc0 con 0x7fabd8149f70 2020-11-20T12:19:28.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.616+0000 7fabdca20700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd8150260 con 0x7fabd80c3140 2020-11-20T12:19:28.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.616+0000 7fabdca20700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd8150500 con 0x7fabd80a63a0 2020-11-20T12:19:28.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.616+0000 7fabd5d9b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fabd80c3140 0x7fabd80c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:28.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.616+0000 7fabd659c700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fabd8149f70 0x7fabd80c1770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:53776/0 (socket says 172.21.15.116:53776) 2020-11-20T12:19:28.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.616+0000 7fabd5d9b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fabd80c3140 0x7fabd80c35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:47166/0 (socket says 172.21.15.116:47166) 2020-11-20T12:19:28.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.616+0000 7fabd659c700 1 -- 172.21.15.116:0/1338597577 learned_addr learned my addr 172.21.15.116:0/1338597577 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:28.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.616+0000 7fabd6d9d700 1 --2- 172.21.15.116:0/1338597577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabd80a63a0 0x7fabd80a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:28.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabd5d9b700 1 -- 172.21.15.116:0/1338597577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabd80a63a0 msgr2=0x7fabd80a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:28.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabd5d9b700 1 --2- 172.21.15.116:0/1338597577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabd80a63a0 0x7fabd80a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:28.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabd5d9b700 1 -- 172.21.15.116:0/1338597577 >> v1:172.21.15.173:6789/0 conn(0x7fabd8149f70 legacy=0x7fabd80c1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:28.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabd5d9b700 1 -- 172.21.15.116:0/1338597577 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd81507a0 con 0x7fabd80c3140 2020-11-20T12:19:28.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabd5d9b700 1 --2- 172.21.15.116:0/1338597577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabd80c3140 0x7fabd80c35c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fabcc00b9a0 tx=0x7fabcc006df0).ready entity=mon.1 client_cookie=92cb0bd555c20e55 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:28.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabd559a700 1 -- 172.21.15.116:0/1338597577 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc0097b0 con 0x7fabd80c3140 2020-11-20T12:19:28.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabd559a700 1 -- 172.21.15.116:0/1338597577 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc009910 con 0x7fabd80c3140 2020-11-20T12:19:28.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabd559a700 1 -- 172.21.15.116:0/1338597577 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc009c90 con 0x7fabd80c3140 2020-11-20T12:19:28.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabdca20700 1 -- 172.21.15.116:0/1338597577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabd80c3140 msgr2=0x7fabd80c35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:28.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabdca20700 1 --2- 172.21.15.116:0/1338597577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabd80c3140 0x7fabd80c35c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7fabcc00b9a0 tx=0x7fabcc006df0).stop 2020-11-20T12:19:28.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabdca20700 1 -- 172.21.15.116:0/1338597577 shutdown_connections 2020-11-20T12:19:28.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabdca20700 1 --2- 172.21.15.116:0/1338597577 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabd80a63a0 0x7fabd80a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:28.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabdca20700 1 --2- 172.21.15.116:0/1338597577 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabd80c3140 0x7fabd80c35c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:28.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabdca20700 1 -- 172.21.15.116:0/1338597577 >> 172.21.15.116:0/1338597577 conn(0x7fabd8060170 msgr2=0x7fabd8060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:28.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabdca20700 1 -- 172.21.15.116:0/1338597577 shutdown_connections 2020-11-20T12:19:28.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.620+0000 7fabdca20700 1 -- 172.21.15.116:0/1338597577 wait complete. 2020-11-20T12:19:28.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.624+0000 7fabdca20700 1 Processor -- start 2020-11-20T12:19:28.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.624+0000 7fabdca20700 1 -- start start 2020-11-20T12:19:28.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabdca20700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabd80a63a0 0x7fabd8144620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:28.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabdca20700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabd80c3140 0x7fabd80c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:28.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabdca20700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabd8149f70 0x7fabd81470e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:28.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabdca20700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80c48f0 con 0x7fabd80a63a0 2020-11-20T12:19:28.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabdca20700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80c4b90 con 0x7fabd80c3140 2020-11-20T12:19:28.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabdca20700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80c4e30 con 0x7fabd8149f70 2020-11-20T12:19:28.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabd659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabd80a63a0 0x7fabd8144620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:28.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabd659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabd80a63a0 0x7fabd8144620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48120/0 (socket says 172.21.15.116:48120) 2020-11-20T12:19:28.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabd659c700 1 -- 172.21.15.116:0/411801111 learned_addr learned my addr 172.21.15.116:0/411801111 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:28.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabd5d9b700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabd80c3140 0x7fabd80c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:28.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabd659c700 1 -- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabd8149f70 msgr2=0x7fabd81470e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:28.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabd659c700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabd8149f70 0x7fabd81470e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:28.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabd659c700 1 -- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabd80c3140 msgr2=0x7fabd80c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:28.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabd659c700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabd80c3140 0x7fabd80c3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:28.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabd659c700 1 -- 172.21.15.116:0/411801111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8146080 con 0x7fabd80a63a0 2020-11-20T12:19:28.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabd5d9b700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabd80c3140 0x7fabd80c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:28.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabd659c700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabd80a63a0 0x7fabd8144620 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7fabc800c710 tx=0x7fabc8007ca0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:28.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabc67fc700 1 -- 172.21.15.116:0/411801111 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc80072e0 con 0x7fabd80a63a0 2020-11-20T12:19:28.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabc67fc700 1 -- 172.21.15.116:0/411801111 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc8007440 con 0x7fabd80a63a0 2020-11-20T12:19:28.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabdca20700 1 -- 172.21.15.116:0/411801111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd8145d00 con 0x7fabd80a63a0 2020-11-20T12:19:28.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabc67fc700 1 -- 172.21.15.116:0/411801111 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8005800 con 0x7fabd80a63a0 2020-11-20T12:19:28.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabdca20700 1 -- 172.21.15.116:0/411801111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd80c5400 con 0x7fabd80a63a0 2020-11-20T12:19:28.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.628+0000 7fabdca20700 1 -- 172.21.15.116:0/411801111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd8145ef0 con 0x7fabd80a63a0 2020-11-20T12:19:28.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.632+0000 7fabc67fc700 1 -- 172.21.15.116:0/411801111 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fabc801a060 con 0x7fabd80a63a0 2020-11-20T12:19:28.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.632+0000 7fabc67fc700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabac0557c0 0x7fabac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:28.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.632+0000 7fabc67fc700 1 -- 172.21.15.116:0/411801111 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fabc806de40 con 0x7fabd80a63a0 2020-11-20T12:19:28.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.632+0000 7fabd5d9b700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabac0557c0 0x7fabac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:28.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.632+0000 7fabd5d9b700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabac0557c0 0x7fabac057c80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fabcc00b150 tx=0x7fabcc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:28.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.640+0000 7fabc67fc700 1 -- 172.21.15.116:0/411801111 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fabc8043ce0 con 0x7fabd80a63a0 2020-11-20T12:19:28.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.800+0000 7fabdca20700 1 -- 172.21.15.116:0/411801111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabd8069950 con 0x7fabd80a63a0 2020-11-20T12:19:28.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.804+0000 7fabc67fc700 1 -- 172.21.15.116:0/411801111 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+16877 (secure 0 0 0) 0x7fabc8018030 con 0x7fabd80a63a0 2020-11-20T12:19:28.802 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:28.802 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":71,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:13.986922+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:11.306087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:11.966077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":68,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:28.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.804+0000 7fabdca20700 1 -- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabac0557c0 msgr2=0x7fabac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:28.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.804+0000 7fabdca20700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabac0557c0 0x7fabac057c80 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7fabcc00b150 tx=0x7fabcc015000).stop 2020-11-20T12:19:28.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.804+0000 7fabdca20700 1 -- 172.21.15.116:0/411801111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabd80a63a0 msgr2=0x7fabd8144620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:28.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.804+0000 7fabdca20700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabd80a63a0 0x7fabd8144620 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7fabc800c710 tx=0x7fabc8007ca0).stop 2020-11-20T12:19:28.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.804+0000 7fabdca20700 1 -- 172.21.15.116:0/411801111 shutdown_connections 2020-11-20T12:19:28.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.804+0000 7fabdca20700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fabac0557c0 0x7fabac057c80 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:28.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.804+0000 7fabdca20700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fabd8149f70 0x7fabd81470e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:28.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.804+0000 7fabdca20700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fabd80a63a0 0x7fabd8144620 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.804+0000 7fabdca20700 1 --2- 172.21.15.116:0/411801111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fabd80c3140 0x7fabd80c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.804+0000 7fabdca20700 1 -- 172.21.15.116:0/411801111 >> 172.21.15.116:0/411801111 conn(0x7fabd8060170 msgr2=0x7fabd81521e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.808+0000 7fabdca20700 1 -- 172.21.15.116:0/411801111 shutdown_connections 2020-11-20T12:19:28.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:28.808+0000 7fabdca20700 1 -- 172.21.15.116:0/411801111 wait complete. 2020-11-20T12:19:28.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:19:11.306087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:19:11.966077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:19:28.829 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:19:29.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.008+0000 7f87bdc42700 1 Processor -- start 2020-11-20T12:19:29.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.008+0000 7f87bdc42700 1 -- start start 2020-11-20T12:19:29.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87bdc42700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87b814bf40 0x7f87b8061c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87bdc42700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f87b8066c10 0x7f87b8153060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87bdc42700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b8061390 con 0x7f87b8066090 2020-11-20T12:19:29.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87bdc42700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b8061630 con 0x7f87b8066c10 2020-11-20T12:19:29.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87bdc42700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80618d0 con 0x7f87b814bf40 2020-11-20T12:19:29.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87b7fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f87b8066c10 0x7f87b8153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87b7fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f87b8066c10 0x7f87b8153060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48128/0 (socket says 172.21.15.116:48128) 2020-11-20T12:19:29.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87b7fff700 1 -- 172.21.15.116:0/3433090365 learned_addr learned my addr 172.21.15.116:0/3433090365 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:29.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87bcc40700 1 --2- 172.21.15.116:0/3433090365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87b814bf40 0x7f87b8061c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87b7fff700 1 -- 172.21.15.116:0/3433090365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87b814bf40 msgr2=0x7f87b8061c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:29.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87b7fff700 1 --2- 172.21.15.116:0/3433090365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87b814bf40 0x7f87b8061c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87b7fff700 1 -- 172.21.15.116:0/3433090365 >> v1:172.21.15.173:6789/0 conn(0x7f87b8066090 legacy=0x7f87b80bbce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87b7fff700 1 -- 172.21.15.116:0/3433090365 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b80bb5a0 con 0x7f87b8066c10 2020-11-20T12:19:29.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.012+0000 7f87b7fff700 1 --2- 172.21.15.116:0/3433090365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b8066c10 0x7f87b8153060 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f87a4006fd0 tx=0x7f87a4006eb0).ready entity=mon.0 client_cookie=249d0cb972e942fa server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:29.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87b77fe700 1 -- 172.21.15.116:0/3433090365 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4009810 con 0x7f87b8066c10 2020-11-20T12:19:29.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87b77fe700 1 -- 172.21.15.116:0/3433090365 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a4009970 con 0x7f87b8066c10 2020-11-20T12:19:29.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87b77fe700 1 -- 172.21.15.116:0/3433090365 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a4009cf0 con 0x7f87b8066c10 2020-11-20T12:19:29.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87bdc42700 1 -- 172.21.15.116:0/3433090365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b8066c10 msgr2=0x7f87b8153060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87bdc42700 1 --2- 172.21.15.116:0/3433090365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b8066c10 0x7f87b8153060 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f87a4006fd0 tx=0x7f87a4006eb0).stop 2020-11-20T12:19:29.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87bdc42700 1 -- 172.21.15.116:0/3433090365 shutdown_connections 2020-11-20T12:19:29.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87bdc42700 1 --2- 172.21.15.116:0/3433090365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87b814bf40 0x7f87b8061c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87bdc42700 1 --2- 172.21.15.116:0/3433090365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b8066c10 0x7f87b8153060 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87bdc42700 1 -- 172.21.15.116:0/3433090365 >> 172.21.15.116:0/3433090365 conn(0x7f87b8060170 msgr2=0x7f87b8060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:29.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87bdc42700 1 -- 172.21.15.116:0/3433090365 shutdown_connections 2020-11-20T12:19:29.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87bdc42700 1 -- 172.21.15.116:0/3433090365 wait complete. 2020-11-20T12:19:29.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87bdc42700 1 Processor -- start 2020-11-20T12:19:29.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.016+0000 7f87bdc42700 1 -- start start 2020-11-20T12:19:29.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bdc42700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b8066090 0x7f87b80a6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bdc42700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87b814bf40 0x7f87b8164f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bdc42700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87b814f1c0 0x7f87b81488b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bdc42700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80a69c0 con 0x7f87b8066090 2020-11-20T12:19:29.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bdc42700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80a6c60 con 0x7f87b814f1c0 2020-11-20T12:19:29.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bdc42700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b80a6f00 con 0x7f87b814bf40 2020-11-20T12:19:29.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bcc40700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b8066090 0x7f87b80a6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bcc40700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b8066090 0x7f87b80a6120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48132/0 (socket says 172.21.15.116:48132) 2020-11-20T12:19:29.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bcc40700 1 -- 172.21.15.116:0/3904157547 learned_addr learned my addr 172.21.15.116:0/3904157547 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:29.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87b7fff700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87b814bf40 0x7f87b8164f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bd441700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87b814f1c0 0x7f87b81488b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bcc40700 1 -- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87b814bf40 msgr2=0x7f87b8164f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bcc40700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87b814bf40 0x7f87b8164f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bcc40700 1 -- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87b814f1c0 msgr2=0x7f87b81488b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bcc40700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87b814f1c0 0x7f87b81488b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bcc40700 1 -- 172.21.15.116:0/3904157547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b8164820 con 0x7f87b8066090 2020-11-20T12:19:29.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bcc40700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b8066090 0x7f87b80a6120 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f87ac00eba0 tx=0x7f87ac006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:29.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87b4ff9700 1 -- 172.21.15.116:0/3904157547 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00b9a0 con 0x7f87b8066090 2020-11-20T12:19:29.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87b4ff9700 1 -- 172.21.15.116:0/3904157547 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87ac00bb00 con 0x7f87b8066090 2020-11-20T12:19:29.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bdc42700 1 -- 172.21.15.116:0/3904157547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b8164ac0 con 0x7f87b8066090 2020-11-20T12:19:29.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87b4ff9700 1 -- 172.21.15.116:0/3904157547 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ac00bea0 con 0x7f87b8066090 2020-11-20T12:19:29.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.020+0000 7f87bdc42700 1 -- 172.21.15.116:0/3904157547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b8165450 con 0x7f87b8066090 2020-11-20T12:19:29.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.024+0000 7f87bdc42700 1 -- 172.21.15.116:0/3904157547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b805cf30 con 0x7f87b8066090 2020-11-20T12:19:29.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.024+0000 7f87b4ff9700 1 -- 172.21.15.116:0/3904157547 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f87ac012600 con 0x7f87b8066090 2020-11-20T12:19:29.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.024+0000 7f87b4ff9700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f87900557c0 0x7f8790057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.024+0000 7f87b4ff9700 1 -- 172.21.15.116:0/3904157547 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f87ac06eac0 con 0x7f87b8066090 2020-11-20T12:19:29.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.024+0000 7f87b7fff700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f87900557c0 0x7f8790057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.028+0000 7f87b7fff700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f87900557c0 0x7f8790057c80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f87b80c3ea0 tx=0x7f87a4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:29.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.032+0000 7f87b4ff9700 1 -- 172.21.15.116:0/3904157547 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f87ac044960 con 0x7f87b8066090 2020-11-20T12:19:29.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87bdc42700 1 -- 172.21.15.116:0/3904157547 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f87b8165700 con 0x7f87b8066090 2020-11-20T12:19:29.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87b4ff9700 1 -- 172.21.15.116:0/3904157547 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v71) v1 ==== 110+0+10 (secure 0 0 0) 0x7f87ac047f80 con 0x7f87b8066090 2020-11-20T12:19:29.186 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:19:29.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87bdc42700 1 -- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f87900557c0 msgr2=0x7f8790057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87bdc42700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f87900557c0 0x7f8790057c80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f87b80c3ea0 tx=0x7f87a4015000).stop 2020-11-20T12:19:29.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87bdc42700 1 -- 172.21.15.116:0/3904157547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b8066090 msgr2=0x7f87b80a6120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87bdc42700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b8066090 0x7f87b80a6120 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f87ac00eba0 tx=0x7f87ac006df0).stop 2020-11-20T12:19:29.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87bdc42700 1 -- 172.21.15.116:0/3904157547 shutdown_connections 2020-11-20T12:19:29.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87bdc42700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f87900557c0 0x7f8790057c80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87bdc42700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f87b814f1c0 0x7f87b81488b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87bdc42700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f87b8066090 0x7f87b80a6120 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87bdc42700 1 --2- 172.21.15.116:0/3904157547 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f87b814bf40 0x7f87b8164f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.188+0000 7f87bdc42700 1 -- 172.21.15.116:0/3904157547 >> 172.21.15.116:0/3904157547 conn(0x7f87b8060170 msgr2=0x7f87b8153ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:29.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.192+0000 7f87bdc42700 1 -- 172.21.15.116:0/3904157547 shutdown_connections 2020-11-20T12:19:29.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.192+0000 7f87bdc42700 1 -- 172.21.15.116:0/3904157547 wait complete. 2020-11-20T12:19:29.213 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:19:29.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.392+0000 7fe9d1dee700 1 Processor -- start 2020-11-20T12:19:29.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.392+0000 7fe9d1dee700 1 -- start start 2020-11-20T12:19:29.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d1dee700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe9cc0a8400 0x7fe9cc0aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d1dee700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9cc0ad540 0x7fe9cc0ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d1dee700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9cc0ace40 con 0x7fe9cc0aa7d0 2020-11-20T12:19:29.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d1dee700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc1526c0 con 0x7fe9cc0ad540 2020-11-20T12:19:29.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d1dee700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc152960 con 0x7fe9cc0a8400 2020-11-20T12:19:29.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d15ed700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9cc0ad540 0x7fe9cc0ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d15ed700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe9cc0ad540 0x7fe9cc0ad9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48144/0 (socket says 172.21.15.116:48144) 2020-11-20T12:19:29.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d15ed700 1 -- 172.21.15.116:0/1233239366 learned_addr learned my addr 172.21.15.116:0/1233239366 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:29.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9cbfff700 1 --2- 172.21.15.116:0/1233239366 >> v2:172.21.15.173:3300/0 conn(0x7fe9cc0a8400 0x7fe9cc0aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d15ed700 1 -- 172.21.15.116:0/1233239366 >> v2:172.21.15.173:3300/0 conn(0x7fe9cc0a8400 msgr2=0x7fe9cc0aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:29.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d15ed700 1 --2- 172.21.15.116:0/1233239366 >> v2:172.21.15.173:3300/0 conn(0x7fe9cc0a8400 0x7fe9cc0aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d15ed700 1 -- 172.21.15.116:0/1233239366 >> v1:172.21.15.173:6789/0 conn(0x7fe9cc0aa7d0 legacy=0x7fe9cc151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d15ed700 1 -- 172.21.15.116:0/1233239366 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc152c00 con 0x7fe9cc0ad540 2020-11-20T12:19:29.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d15ed700 1 --2- 172.21.15.116:0/1233239366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9cc0ad540 0x7fe9cc0ad9d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7fe9cc155390 tx=0x7fe9c4009290).ready entity=mon.0 client_cookie=ff09b25c2f16bbfb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:29.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9cb7fe700 1 -- 172.21.15.116:0/1233239366 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c400bbc0 con 0x7fe9cc0ad540 2020-11-20T12:19:29.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9cb7fe700 1 -- 172.21.15.116:0/1233239366 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c400bd20 con 0x7fe9cc0ad540 2020-11-20T12:19:29.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9cb7fe700 1 -- 172.21.15.116:0/1233239366 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c4011610 con 0x7fe9cc0ad540 2020-11-20T12:19:29.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1233239366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9cc0ad540 msgr2=0x7fe9cc0ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d1dee700 1 --2- 172.21.15.116:0/1233239366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9cc0ad540 0x7fe9cc0ad9d0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7fe9cc155390 tx=0x7fe9c4009290).stop 2020-11-20T12:19:29.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.396+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1233239366 shutdown_connections 2020-11-20T12:19:29.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 --2- 172.21.15.116:0/1233239366 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9cc0ad540 0x7fe9cc0ad9d0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 --2- 172.21.15.116:0/1233239366 >> v2:172.21.15.173:3300/0 conn(0x7fe9cc0a8400 0x7fe9cc0aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1233239366 >> 172.21.15.116:0/1233239366 conn(0x7fe9cc0678a0 msgr2=0x7fe9cc067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:29.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1233239366 shutdown_connections 2020-11-20T12:19:29.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1233239366 wait complete. 2020-11-20T12:19:29.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 Processor -- start 2020-11-20T12:19:29.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 -- start start 2020-11-20T12:19:29.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9cc0a8400 0x7fe9cc14d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9cc0aa7d0 0x7fe9cc14c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9cc0ad540 0x7fe9cc158ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc1571a0 con 0x7fe9cc0a8400 2020-11-20T12:19:29.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc157440 con 0x7fe9cc0aa7d0 2020-11-20T12:19:29.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d1dee700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc1503d0 con 0x7fe9cc0ad540 2020-11-20T12:19:29.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.400+0000 7fe9d0dec700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9cc0a8400 0x7fe9cc14d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9cbfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9cc0aa7d0 0x7fe9cc14c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d0dec700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9cc0a8400 0x7fe9cc14d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48148/0 (socket says 172.21.15.116:48148) 2020-11-20T12:19:29.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9cbfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9cc0aa7d0 0x7fe9cc14c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:47198/0 (socket says 172.21.15.116:47198) 2020-11-20T12:19:29.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d0dec700 1 -- 172.21.15.116:0/1480863089 learned_addr learned my addr 172.21.15.116:0/1480863089 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:29.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d15ed700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9cc0ad540 0x7fe9cc158ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d0dec700 1 -- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9cc0ad540 msgr2=0x7fe9cc158ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d0dec700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9cc0ad540 0x7fe9cc158ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d0dec700 1 -- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9cc0aa7d0 msgr2=0x7fe9cc14c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d0dec700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9cc0aa7d0 0x7fe9cc14c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d0dec700 1 -- 172.21.15.116:0/1480863089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc150670 con 0x7fe9cc0a8400 2020-11-20T12:19:29.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d0dec700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9cc0a8400 0x7fe9cc14d470 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7fe9c0001e70 tx=0x7fe9c0007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:29.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9c8ff9700 1 -- 172.21.15.116:0/1480863089 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c0007020 con 0x7fe9cc0a8400 2020-11-20T12:19:29.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9c8ff9700 1 -- 172.21.15.116:0/1480863089 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9c0007180 con 0x7fe9cc0a8400 2020-11-20T12:19:29.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1480863089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9cc150910 con 0x7fe9cc0a8400 2020-11-20T12:19:29.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9c8ff9700 1 -- 172.21.15.116:0/1480863089 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9c000e560 con 0x7fe9cc0a8400 2020-11-20T12:19:29.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1480863089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc14ca60 con 0x7fe9cc0a8400 2020-11-20T12:19:29.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.404+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1480863089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc0c5bd0 con 0x7fe9cc0a8400 2020-11-20T12:19:29.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.408+0000 7fe9c8ff9700 1 -- 172.21.15.116:0/1480863089 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe9c0010070 con 0x7fe9cc0a8400 2020-11-20T12:19:29.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.408+0000 7fe9c8ff9700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9a40557c0 0x7fe9a4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.408+0000 7fe9c8ff9700 1 -- 172.21.15.116:0/1480863089 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe9c006d9d0 con 0x7fe9cc0a8400 2020-11-20T12:19:29.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.408+0000 7fe9cbfff700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9a40557c0 0x7fe9a4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.408+0000 7fe9cbfff700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9a40557c0 0x7fe9a4057c80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fe9cc0c95a0 tx=0x7fe9b8006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:29.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.412+0000 7fe9c8ff9700 1 -- 172.21.15.116:0/1480863089 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe9c0042fd0 con 0x7fe9cc0a8400 2020-11-20T12:19:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.572+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1480863089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe9cc05cf30 con 0x7fe9cc0a8400 2020-11-20T12:19:29.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.572+0000 7fe9c8ff9700 1 -- 172.21.15.116:0/1480863089 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v71) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe9c0043870 con 0x7fe9cc0a8400 2020-11-20T12:19:29.571 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:29.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.572+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9a40557c0 msgr2=0x7fe9a4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.572+0000 7fe9d1dee700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9a40557c0 0x7fe9a4057c80 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fe9cc0c95a0 tx=0x7fe9b8006df0).stop 2020-11-20T12:19:29.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.572+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1480863089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9cc0a8400 msgr2=0x7fe9cc14d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.572+0000 7fe9d1dee700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9cc0a8400 0x7fe9cc14d470 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7fe9c0001e70 tx=0x7fe9c0007c50).stop 2020-11-20T12:19:29.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.576+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1480863089 shutdown_connections 2020-11-20T12:19:29.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.576+0000 7fe9d1dee700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe9a40557c0 0x7fe9a4057c80 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.576+0000 7fe9d1dee700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe9cc0ad540 0x7fe9cc158ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.576+0000 7fe9d1dee700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe9cc0a8400 0x7fe9cc14d470 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.576+0000 7fe9d1dee700 1 --2- 172.21.15.116:0/1480863089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe9cc0aa7d0 0x7fe9cc14c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.576+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1480863089 >> 172.21.15.116:0/1480863089 conn(0x7fe9cc0678a0 msgr2=0x7fe9cc153780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.576+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1480863089 shutdown_connections 2020-11-20T12:19:29.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.576+0000 7fe9d1dee700 1 -- 172.21.15.116:0/1480863089 wait complete. 2020-11-20T12:19:29.599 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:19:29.738 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:19:29.752 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:19:29.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.796+0000 7f3ef2fd2700 1 Processor -- start 2020-11-20T12:19:29.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.796+0000 7f3ef2fd2700 1 -- start start 2020-11-20T12:19:29.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.800+0000 7f3ef2fd2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3eec0aa7d0 0x7f3eec0a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.800+0000 7f3ef2fd2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eec0aabb0 0x7f3eec151e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.800+0000 7f3ef2fd2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec0ace40 con 0x7f3eec0ad540 2020-11-20T12:19:29.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.800+0000 7f3ef2fd2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eec1526c0 con 0x7f3eec0aa7d0 2020-11-20T12:19:29.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.800+0000 7f3ef2fd2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec152960 con 0x7f3eec0aabb0 2020-11-20T12:19:29.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.800+0000 7f3ef1fd0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3eec0aa7d0 0x7f3eec0a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.800+0000 7f3ef1fd0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3eec0aa7d0 0x7f3eec0a8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48154/0 (socket says 172.21.15.116:48154) 2020-11-20T12:19:29.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.800+0000 7f3ef1fd0700 1 -- 172.21.15.116:0/1194911262 learned_addr learned my addr 172.21.15.116:0/1194911262 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:29.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.800+0000 7f3ef17cf700 1 --2- 172.21.15.116:0/1194911262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eec0aabb0 0x7f3eec151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef1fd0700 1 -- 172.21.15.116:0/1194911262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eec0aabb0 msgr2=0x7f3eec151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:29.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef1fd0700 1 --2- 172.21.15.116:0/1194911262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eec0aabb0 0x7f3eec151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef1fd0700 1 -- 172.21.15.116:0/1194911262 >> v1:172.21.15.173:6789/0 conn(0x7f3eec0ad540 legacy=0x7f3eec0ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef1fd0700 1 -- 172.21.15.116:0/1194911262 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec152c00 con 0x7f3eec0aa7d0 2020-11-20T12:19:29.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef1fd0700 1 --2- 172.21.15.116:0/1194911262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eec0aa7d0 0x7f3eec0a8400 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f3ee8006fd0 tx=0x7f3ee8006eb0).ready entity=mon.0 client_cookie=559576fd39197c2a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:29.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef0fce700 1 -- 172.21.15.116:0/1194911262 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8009810 con 0x7f3eec0aa7d0 2020-11-20T12:19:29.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef0fce700 1 -- 172.21.15.116:0/1194911262 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee8009970 con 0x7f3eec0aa7d0 2020-11-20T12:19:29.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef0fce700 1 -- 172.21.15.116:0/1194911262 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8009cf0 con 0x7f3eec0aa7d0 2020-11-20T12:19:29.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1194911262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eec0aa7d0 msgr2=0x7f3eec0a8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef2fd2700 1 --2- 172.21.15.116:0/1194911262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eec0aa7d0 0x7f3eec0a8400 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f3ee8006fd0 tx=0x7f3ee8006eb0).stop 2020-11-20T12:19:29.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1194911262 shutdown_connections 2020-11-20T12:19:29.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef2fd2700 1 --2- 172.21.15.116:0/1194911262 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eec0aa7d0 0x7f3eec0a8400 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef2fd2700 1 --2- 172.21.15.116:0/1194911262 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eec0aabb0 0x7f3eec151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1194911262 >> 172.21.15.116:0/1194911262 conn(0x7f3eec0678a0 msgr2=0x7f3eec067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:29.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1194911262 shutdown_connections 2020-11-20T12:19:29.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.804+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1194911262 wait complete. 2020-11-20T12:19:29.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef2fd2700 1 Processor -- start 2020-11-20T12:19:29.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef2fd2700 1 -- start start 2020-11-20T12:19:29.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef2fd2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eec0aabb0 0x7f3eec09fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef2fd2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eec0ad540 0x7f3eec0a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef2fd2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eec0a6fa0 0x7f3eec164800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef2fd2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec150c20 con 0x7f3eec0ad540 2020-11-20T12:19:29.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef2fd2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec150ec0 con 0x7f3eec0aabb0 2020-11-20T12:19:29.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef2fd2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec151160 con 0x7f3eec0a6fa0 2020-11-20T12:19:29.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef17cf700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eec0ad540 0x7f3eec0a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef17cf700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eec0ad540 0x7f3eec0a6ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48162/0 (socket says 172.21.15.116:48162) 2020-11-20T12:19:29.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef17cf700 1 -- 172.21.15.116:0/1303035360 learned_addr learned my addr 172.21.15.116:0/1303035360 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:29.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef1fd0700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eec0aabb0 0x7f3eec09fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef27d1700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eec0a6fa0 0x7f3eec164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef17cf700 1 -- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eec0a6fa0 msgr2=0x7f3eec164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef17cf700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eec0a6fa0 0x7f3eec164800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef17cf700 1 -- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eec0aabb0 msgr2=0x7f3eec09fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef17cf700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eec0aabb0 0x7f3eec09fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef17cf700 1 -- 172.21.15.116:0/1303035360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec151400 con 0x7f3eec0ad540 2020-11-20T12:19:29.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef27d1700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eec0a6fa0 0x7f3eec164800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:29.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ef17cf700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eec0ad540 0x7f3eec0a6ab0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f3edc00eba0 tx=0x7f3edc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:29.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.808+0000 7f3ed9ffb700 1 -- 172.21.15.116:0/1303035360 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc00ca60 con 0x7f3eec0ad540 2020-11-20T12:19:29.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.812+0000 7f3ed9ffb700 1 -- 172.21.15.116:0/1303035360 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc00cbc0 con 0x7f3eec0ad540 2020-11-20T12:19:29.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.812+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1303035360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eec0a0160 con 0x7f3eec0ad540 2020-11-20T12:19:29.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.812+0000 7f3ed9ffb700 1 -- 172.21.15.116:0/1303035360 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0123f0 con 0x7f3eec0ad540 2020-11-20T12:19:29.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.812+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1303035360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec0a0460 con 0x7f3eec0ad540 2020-11-20T12:19:29.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.812+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1303035360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec151590 con 0x7f3eec0ad540 2020-11-20T12:19:29.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.812+0000 7f3ed9ffb700 1 -- 172.21.15.116:0/1303035360 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3edc0127a0 con 0x7f3eec0ad540 2020-11-20T12:19:29.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.812+0000 7f3ed9ffb700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3ec40557f0 0x7f3ec4057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:29.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.812+0000 7f3ed9ffb700 1 -- 172.21.15.116:0/1303035360 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3edc06eb50 con 0x7f3eec0ad540 2020-11-20T12:19:29.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.816+0000 7f3ef1fd0700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3ec40557f0 0x7f3ec4057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:29.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.816+0000 7f3ef1fd0700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3ec40557f0 0x7f3ec4057cb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f3eec060600 tx=0x7f3ee8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:29.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.820+0000 7f3ed9ffb700 1 -- 172.21.15.116:0/1303035360 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3edc0449f0 con 0x7f3eec0ad540 2020-11-20T12:19:29.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.976+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1303035360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3eec05cf30 con 0x7f3eec0ad540 2020-11-20T12:19:29.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.976+0000 7f3ed9ffb700 1 -- 172.21.15.116:0/1303035360 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v71) v1 ==== 104+0+10 (secure 0 0 0) 0x7f3edc048010 con 0x7f3eec0ad540 2020-11-20T12:19:29.977 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:29.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3ec40557f0 msgr2=0x7f3ec4057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3ec40557f0 0x7f3ec4057cb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f3eec060600 tx=0x7f3ee8015000).stop 2020-11-20T12:19:29.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1303035360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eec0ad540 msgr2=0x7f3eec0a6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:29.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eec0ad540 0x7f3eec0a6ab0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f3edc00eba0 tx=0x7f3edc006df0).stop 2020-11-20T12:19:29.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1303035360 shutdown_connections 2020-11-20T12:19:29.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3ec40557f0 0x7f3ec4057cb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3eec0a6fa0 0x7f3eec164800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3eec0aabb0 0x7f3eec09fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 --2- 172.21.15.116:0/1303035360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3eec0ad540 0x7f3eec0a6ab0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:29.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1303035360 >> 172.21.15.116:0/1303035360 conn(0x7f3eec0678a0 msgr2=0x7f3eec14a760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:29.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1303035360 shutdown_connections 2020-11-20T12:19:29.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:29.980+0000 7f3ef2fd2700 1 -- 172.21.15.116:0/1303035360 wait complete. 2020-11-20T12:19:30.003 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:19:30.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.228+0000 7f6f450e6700 1 Processor -- start 2020-11-20T12:19:30.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.228+0000 7f6f450e6700 1 -- start start 2020-11-20T12:19:30.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.232+0000 7f6f450e6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6f4014f3d0 0x7f6f40061280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:30.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.232+0000 7f6f450e6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f40062b60 0x7f6f40064910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:30.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.232+0000 7f6f450e6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f40155880 con 0x7f6f40062fe0 2020-11-20T12:19:30.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.232+0000 7f6f450e6700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f40155b20 con 0x7f6f4014f3d0 2020-11-20T12:19:30.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.232+0000 7f6f450e6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f40155dc0 con 0x7f6f40062b60 2020-11-20T12:19:30.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.232+0000 7f6f3f59e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6f40062fe0 0x7f6f401521a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57606/0 (socket says 172.21.15.116:57606) 2020-11-20T12:19:30.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.232+0000 7f6f3f59e700 1 -- 172.21.15.116:0/2736314065 learned_addr learned my addr 172.21.15.116:0/2736314065 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:30.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.232+0000 7f6f3ed9d700 1 --2- 172.21.15.116:0/2736314065 >> v2:172.21.15.173:3300/0 conn(0x7f6f4014f3d0 0x7f6f40061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:30.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.232+0000 7f6f3659c700 1 --2- 172.21.15.116:0/2736314065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f40062b60 0x7f6f40064910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:30.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f3e59c700 1 -- 172.21.15.116:0/2736314065 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932539130 0 0) 0x7f6f40155880 con 0x7f6f40062fe0 2020-11-20T12:19:30.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f3e59c700 1 -- 172.21.15.116:0/2736314065 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f20003240 con 0x7f6f40062fe0 2020-11-20T12:19:30.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f3ed9d700 1 -- 172.21.15.116:0/2736314065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f40062b60 msgr2=0x7f6f40064910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:30.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f3ed9d700 1 --2- 172.21.15.116:0/2736314065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f40062b60 0x7f6f40064910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f3ed9d700 1 -- 172.21.15.116:0/2736314065 >> v1:172.21.15.116:6789/0 conn(0x7f6f40062fe0 legacy=0x7f6f401521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f3ed9d700 1 -- 172.21.15.116:0/2736314065 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f40156060 con 0x7f6f4014f3d0 2020-11-20T12:19:30.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f3ed9d700 1 --2- 172.21.15.116:0/2736314065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f4014f3d0 0x7f6f40061280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f6f3000b9a0 tx=0x7f6f30006df0).ready entity=mon.1 client_cookie=799f50cd2e6848e7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:30.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f3e59c700 1 -- 172.21.15.116:0/2736314065 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30009680 con 0x7f6f4014f3d0 2020-11-20T12:19:30.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f3e59c700 1 -- 172.21.15.116:0/2736314065 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f300097e0 con 0x7f6f4014f3d0 2020-11-20T12:19:30.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f3e59c700 1 -- 172.21.15.116:0/2736314065 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30009b60 con 0x7f6f4014f3d0 2020-11-20T12:19:30.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f450e6700 1 -- 172.21.15.116:0/2736314065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f4014f3d0 msgr2=0x7f6f40061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f450e6700 1 --2- 172.21.15.116:0/2736314065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f4014f3d0 0x7f6f40061280 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7f6f3000b9a0 tx=0x7f6f30006df0).stop 2020-11-20T12:19:30.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f450e6700 1 -- 172.21.15.116:0/2736314065 shutdown_connections 2020-11-20T12:19:30.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f450e6700 1 --2- 172.21.15.116:0/2736314065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f4014f3d0 0x7f6f40061280 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f450e6700 1 --2- 172.21.15.116:0/2736314065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f40062b60 0x7f6f40064910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f450e6700 1 -- 172.21.15.116:0/2736314065 >> 172.21.15.116:0/2736314065 conn(0x7f6f400c7230 msgr2=0x7f6f400c3200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:30.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f450e6700 1 -- 172.21.15.116:0/2736314065 shutdown_connections 2020-11-20T12:19:30.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.236+0000 7f6f450e6700 1 -- 172.21.15.116:0/2736314065 wait complete. 2020-11-20T12:19:30.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f450e6700 1 Processor -- start 2020-11-20T12:19:30.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f450e6700 1 -- start start 2020-11-20T12:19:30.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f450e6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f40062b60 0x7f6f40146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:30.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f450e6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f40062fe0 0x7f6f40145ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:30.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f450e6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f4014f3d0 0x7f6f401463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:30.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f450e6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f400aab20 con 0x7f6f40062fe0 2020-11-20T12:19:30.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f450e6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f40153050 con 0x7f6f40062b60 2020-11-20T12:19:30.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f450e6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f401468b0 con 0x7f6f4014f3d0 2020-11-20T12:19:30.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f3659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f40062fe0 0x7f6f40145ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:30.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f3659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f40062fe0 0x7f6f40145ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48176/0 (socket says 172.21.15.116:48176) 2020-11-20T12:19:30.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f3659c700 1 -- 172.21.15.116:0/1334534266 learned_addr learned my addr 172.21.15.116:0/1334534266 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:30.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f3ed9d700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f40062b60 0x7f6f40146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:30.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f3f59e700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f4014f3d0 0x7f6f401463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:30.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f3659c700 1 -- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f4014f3d0 msgr2=0x7f6f401463c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f3659c700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f4014f3d0 0x7f6f401463c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f3659c700 1 -- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f40062b60 msgr2=0x7f6f40146c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f3659c700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f40062b60 0x7f6f40146c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.240+0000 7f6f3659c700 1 -- 172.21.15.116:0/1334534266 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f40149a70 con 0x7f6f40062fe0 2020-11-20T12:19:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.244+0000 7f6f3659c700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f40062fe0 0x7f6f40145ed0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f6f2800ebd0 tx=0x7f6f28006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:30.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.244+0000 7f6f377fe700 1 -- 172.21.15.116:0/1334534266 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2800c9b0 con 0x7f6f40062fe0 2020-11-20T12:19:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.244+0000 7f6f377fe700 1 -- 172.21.15.116:0/1334534266 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2800cb10 con 0x7f6f40062fe0 2020-11-20T12:19:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.244+0000 7f6f450e6700 1 -- 172.21.15.116:0/1334534266 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f40149530 con 0x7f6f40062fe0 2020-11-20T12:19:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.244+0000 7f6f377fe700 1 -- 172.21.15.116:0/1334534266 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2800ce90 con 0x7f6f40062fe0 2020-11-20T12:19:30.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.244+0000 7f6f450e6700 1 -- 172.21.15.116:0/1334534266 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f40147170 con 0x7f6f40062fe0 2020-11-20T12:19:30.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.244+0000 7f6f450e6700 1 -- 172.21.15.116:0/1334534266 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f401473c0 con 0x7f6f40062fe0 2020-11-20T12:19:30.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.244+0000 7f6f377fe700 1 -- 172.21.15.116:0/1334534266 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6f28012600 con 0x7f6f40062fe0 2020-11-20T12:19:30.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.248+0000 7f6f377fe700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f1c0557c0 0x7f6f1c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:30.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.248+0000 7f6f377fe700 1 -- 172.21.15.116:0/1334534266 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6f2806ea70 con 0x7f6f40062fe0 2020-11-20T12:19:30.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.248+0000 7f6f3ed9d700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f1c0557c0 0x7f6f1c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:30.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.248+0000 7f6f3ed9d700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f1c0557c0 0x7f6f1c057c80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f6f3000b150 tx=0x7f6f30015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:30.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.252+0000 7f6f377fe700 1 -- 172.21.15.116:0/1334534266 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6f280448b0 con 0x7f6f40062fe0 2020-11-20T12:19:30.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.412+0000 7f6f450e6700 1 -- 172.21.15.116:0/1334534266 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6f40069950 con 0x7f6f40062fe0 2020-11-20T12:19:30.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.412+0000 7f6f377fe700 1 -- 172.21.15.116:0/1334534266 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v71) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6f2806ee60 con 0x7f6f40062fe0 2020-11-20T12:19:30.412 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:30.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 -- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f1c0557c0 msgr2=0x7f6f1c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f1c0557c0 0x7f6f1c057c80 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f6f3000b150 tx=0x7f6f30015000).stop 2020-11-20T12:19:30.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 -- 172.21.15.116:0/1334534266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f40062fe0 msgr2=0x7f6f40145ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f40062fe0 0x7f6f40145ed0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f6f2800ebd0 tx=0x7f6f28006df0).stop 2020-11-20T12:19:30.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 -- 172.21.15.116:0/1334534266 shutdown_connections 2020-11-20T12:19:30.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6f1c0557c0 0x7f6f1c057c80 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6f4014f3d0 0x7f6f401463c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6f40062b60 0x7f6f40146c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 --2- 172.21.15.116:0/1334534266 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6f40062fe0 0x7f6f40145ed0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 -- 172.21.15.116:0/1334534266 >> 172.21.15.116:0/1334534266 conn(0x7f6f400c7230 msgr2=0x7f6f40061880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:30.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 -- 172.21.15.116:0/1334534266 shutdown_connections 2020-11-20T12:19:30.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.416+0000 7f6f450e6700 1 -- 172.21.15.116:0/1334534266 wait complete. 2020-11-20T12:19:30.440 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:19:30.440 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:19:30.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.620+0000 7f8e86dc3700 1 Processor -- start 2020-11-20T12:19:30.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.620+0000 7f8e86dc3700 1 -- start start 2020-11-20T12:19:30.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.620+0000 7f8e86dc3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e80062880 0x7f8e80062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:30.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.620+0000 7f8e86dc3700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e800ba1b0 con 0x7f8e800b9bf0 2020-11-20T12:19:30.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.620+0000 7f8e86dc3700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e801555a0 con 0x7f8e8014b990 2020-11-20T12:19:30.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.620+0000 7f8e86dc3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e80155840 con 0x7f8e80062880 2020-11-20T12:19:30.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.620+0000 7f8e865c2700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8e800b9bf0 0x7f8e80151e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57620/0 (socket says 172.21.15.116:57620) 2020-11-20T12:19:30.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.620+0000 7f8e85dc1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e80062880 0x7f8e80062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.620+0000 7f8e865c2700 1 -- 172.21.15.116:0/2791385299 learned_addr learned my addr 172.21.15.116:0/2791385299 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e84dbf700 1 -- 172.21.15.116:0/2791385299 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067092410 0 0) 0x7f8e801555a0 con 0x7f8e8014b990 2020-11-20T12:19:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e84dbf700 1 -- 172.21.15.116:0/2791385299 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e68003240 con 0x7f8e8014b990 2020-11-20T12:19:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e85dc1700 1 -- 172.21.15.116:0/2791385299 >> v1:172.21.15.173:6789/0 conn(0x7f8e8014b990 legacy=0x7f8e800b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e85dc1700 1 -- 172.21.15.116:0/2791385299 >> v1:172.21.15.116:6789/0 conn(0x7f8e800b9bf0 legacy=0x7f8e80151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e85dc1700 1 -- 172.21.15.116:0/2791385299 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e80155ae0 con 0x7f8e80062880 2020-11-20T12:19:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e85dc1700 1 --2- 172.21.15.116:0/2791385299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e80062880 0x7f8e80062c60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f8e7c00b9a0 tx=0x7f8e7c006df0).ready entity=mon.0 client_cookie=1ae2ad4444308088 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:30.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e84dbf700 1 -- 172.21.15.116:0/2791385299 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c0096b0 con 0x7f8e80062880 2020-11-20T12:19:30.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e84dbf700 1 -- 172.21.15.116:0/2791385299 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e7c009810 con 0x7f8e80062880 2020-11-20T12:19:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e84dbf700 1 -- 172.21.15.116:0/2791385299 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e7c009b90 con 0x7f8e80062880 2020-11-20T12:19:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2791385299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e80062880 msgr2=0x7f8e80062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e86dc3700 1 --2- 172.21.15.116:0/2791385299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e80062880 0x7f8e80062c60 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f8e7c00b9a0 tx=0x7f8e7c006df0).stop 2020-11-20T12:19:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2791385299 shutdown_connections 2020-11-20T12:19:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e86dc3700 1 --2- 172.21.15.116:0/2791385299 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e80062880 0x7f8e80062c60 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2791385299 >> 172.21.15.116:0/2791385299 conn(0x7f8e80060c40 msgr2=0x7f8e800c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:30.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2791385299 shutdown_connections 2020-11-20T12:19:30.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.624+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2791385299 wait complete. 2020-11-20T12:19:30.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.628+0000 7f8e86dc3700 1 Processor -- start 2020-11-20T12:19:30.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.628+0000 7f8e86dc3700 1 -- start start 2020-11-20T12:19:30.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.628+0000 7f8e86dc3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e80062880 0x7f8e80135d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:30.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.628+0000 7f8e86dc3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e800b9bf0 0x7f8e800bc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:30.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.628+0000 7f8e86dc3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e8014b990 0x7f8e800bcd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:30.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.628+0000 7f8e86dc3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800bd520 con 0x7f8e8014b990 2020-11-20T12:19:30.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.628+0000 7f8e86dc3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800c2220 con 0x7f8e800b9bf0 2020-11-20T12:19:30.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.628+0000 7f8e86dc3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e800c24c0 con 0x7f8e80062880 2020-11-20T12:19:30.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.628+0000 7f8e865c2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e8014b990 0x7f8e800bcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:30.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.628+0000 7f8e865c2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e8014b990 0x7f8e800bcd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48192/0 (socket says 172.21.15.116:48192) 2020-11-20T12:19:30.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.628+0000 7f8e865c2700 1 -- 172.21.15.116:0/2001748772 learned_addr learned my addr 172.21.15.116:0/2001748772 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:30.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e855c0700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e800b9bf0 0x7f8e800bc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:30.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e85dc1700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e80062880 0x7f8e80135d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:30.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e865c2700 1 -- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e80062880 msgr2=0x7f8e80135d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e865c2700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e80062880 0x7f8e80135d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e865c2700 1 -- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e800b9bf0 msgr2=0x7f8e800bc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e865c2700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e800b9bf0 0x7f8e800bc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e865c2700 1 -- 172.21.15.116:0/2001748772 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e800c2760 con 0x7f8e8014b990 2020-11-20T12:19:30.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e865c2700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e8014b990 0x7f8e800bcd00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f8e7800cab0 tx=0x7f8e78007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:30.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e6dffb700 1 -- 172.21.15.116:0/2001748772 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78005c50 con 0x7f8e8014b990 2020-11-20T12:19:30.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e6dffb700 1 -- 172.21.15.116:0/2001748772 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e78005db0 con 0x7f8e8014b990 2020-11-20T12:19:30.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2001748772 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e800bd1f0 con 0x7f8e8014b990 2020-11-20T12:19:30.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e6dffb700 1 -- 172.21.15.116:0/2001748772 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e78007360 con 0x7f8e8014b990 2020-11-20T12:19:30.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2001748772 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e80136390 con 0x7f8e8014b990 2020-11-20T12:19:30.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.632+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2001748772 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e8005c880 con 0x7f8e8014b990 2020-11-20T12:19:30.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.636+0000 7f8e6dffb700 1 -- 172.21.15.116:0/2001748772 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8e78010030 con 0x7f8e8014b990 2020-11-20T12:19:30.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.636+0000 7f8e6dffb700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e580557c0 0x7f8e58057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:30.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.636+0000 7f8e6dffb700 1 -- 172.21.15.116:0/2001748772 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8e7806ea70 con 0x7f8e8014b990 2020-11-20T12:19:30.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.636+0000 7f8e85dc1700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e580557c0 0x7f8e58057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:30.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.636+0000 7f8e85dc1700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e580557c0 0x7f8e58057c80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f8e7c00b150 tx=0x7f8e7c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:30.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.640+0000 7f8e6dffb700 1 -- 172.21.15.116:0/2001748772 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8e8005c880 con 0x7f8e8014b990 2020-11-20T12:19:30.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.844+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2001748772 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e8005c880 con 0x7f8e8014b990 2020-11-20T12:19:30.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.848+0000 7f8e6dffb700 1 -- 172.21.15.116:0/2001748772 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) v1 ==== 94+0+2854 (secure 0 0 0) 0x7f8e8005c880 con 0x7f8e8014b990 2020-11-20T12:19:30.845 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:30.845 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":37,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3869489418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3869489418},{"type":"v1","addr":"172.21.15.173:6837","nonce":3869489418}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/2912073472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2912073472},{"type":"v1","addr":"172.21.15.173:6835","nonce":2912073472}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/2221764249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2221764249},{"type":"v1","addr":"172.21.15.116:6834","nonce":2221764249}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:12.981713+0000","modified":"2020-11-20T12:19:14.999541+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1946956517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1946956517},{"type":"v1","addr":"172.21.15.116:6835","nonce":1946956517}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-20T12:19:30.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e580557c0 msgr2=0x7f8e58057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e580557c0 0x7f8e58057c80 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f8e7c00b150 tx=0x7f8e7c015000).stop 2020-11-20T12:19:30.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2001748772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e8014b990 msgr2=0x7f8e800bcd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:30.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e8014b990 0x7f8e800bcd00 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f8e7800cab0 tx=0x7f8e78007e00).stop 2020-11-20T12:19:30.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2001748772 shutdown_connections 2020-11-20T12:19:30.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e580557c0 0x7f8e58057c80 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e8014b990 0x7f8e800bcd00 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e80062880 0x7f8e80135d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 --2- 172.21.15.116:0/2001748772 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e800b9bf0 0x7f8e800bc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:30.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2001748772 >> 172.21.15.116:0/2001748772 conn(0x7f8e80060c40 msgr2=0x7f8e80067810 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:30.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2001748772 shutdown_connections 2020-11-20T12:19:30.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:30.852+0000 7f8e86dc3700 1 -- 172.21.15.116:0/2001748772 wait complete. 2020-11-20T12:19:30.852 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 37 2020-11-20T12:19:31.276 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 37, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:19:12.981713+0000', 'modified': '2020-11-20T12:19:14.999541+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6429}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6429': {'gid': 6429, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/1946956517', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 1946956517}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 1946956517}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:19:31.277 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:19:31.278 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:19:31.479 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":6429,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.478270164000001,"mdsmap_epoch":37,"osdmap_epoch":71,"osdmap_epoch_barrier":71,"uptime":23.955379822000001} 2020-11-20T12:19:31.479 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 6429, "mdsmap_epoch": 37, "osdmap_epoch": 71, "osdmap_epoch_barrier": 71, "rank_uptime": 18.478270164, "state": "up:active", "uptime": 23.955379822, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:19:31.480 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:19:31.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:31.481 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.3 2020-11-20T12:19:31.497 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:19:31.498 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:31.614 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:31.615 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:19:31.643 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:19:31.644 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:19:31.650 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:19:31.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:31.652 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2020-11-20T12:19:31.688 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-20T12:19:31.689 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:19:31.689 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 2020-11-20T12:19:31.818 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:19:31.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:31.819 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T12:19:31.842 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.manila mds 'allow *' osd 'allow rw' mon 'allow *' 2020-11-20T12:19:32.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.032+0000 7fe4b4844700 1 Processor -- start 2020-11-20T12:19:32.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.032+0000 7fe4b4844700 1 -- start start 2020-11-20T12:19:32.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4b4844700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe4b0060310 0x7fe4b00606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4b4844700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b0066e00 con 0x7fe4b0066480 2020-11-20T12:19:32.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4b4844700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4b00670a0 con 0x7fe4b00668e0 2020-11-20T12:19:32.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4b4844700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4b0067340 con 0x7fe4b0060310 2020-11-20T12:19:32.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4aeffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe4b0066480 0x7fe4b00bf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57632/0 (socket says 172.21.15.116:57632) 2020-11-20T12:19:32.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4aeffd700 1 -- 172.21.15.116:0/3346327688 learned_addr learned my addr 172.21.15.116:0/3346327688 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:32.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4af7fe700 1 --2- 172.21.15.116:0/3346327688 >> v2:172.21.15.173:3300/0 conn(0x7fe4b0060310 0x7fe4b00606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2681718859 0 0) 0x7fe4b0066e00 con 0x7fe4b0066480 2020-11-20T12:19:32.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe49c004350 con 0x7fe4b0066480 2020-11-20T12:19:32.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560947006 0 0) 0x7fe4b00670a0 con 0x7fe4b00668e0 2020-11-20T12:19:32.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4b0066e00 con 0x7fe4b00668e0 2020-11-20T12:19:32.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3893305277 0 0) 0x7fe49c004350 con 0x7fe4b0066480 2020-11-20T12:19:32.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 >> v2:172.21.15.173:3300/0 conn(0x7fe4b0060310 msgr2=0x7fe4b00606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:32.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 --2- 172.21.15.116:0/3346327688 >> v2:172.21.15.173:3300/0 conn(0x7fe4b0060310 0x7fe4b00606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 >> v1:172.21.15.173:6789/0 conn(0x7fe4b00668e0 legacy=0x7fe4b00c1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b00c0d30 con 0x7fe4b0066480 2020-11-20T12:19:32.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fe4a40014e0 con 0x7fe4b0066480 2020-11-20T12:19:32.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 >> v1:172.21.15.116:6789/0 conn(0x7fe4b0066480 legacy=0x7fe4b00bf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 --2- 172.21.15.116:0/3346327688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c005a30 0x7fe49c005e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 --2- 172.21.15.116:0/3346327688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c004590 0x7fe49c004a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 --2- 172.21.15.116:0/3346327688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe49c004f00 0x7fe49c003240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b0067340 con 0x7fe49c004f00 2020-11-20T12:19:32.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c003730 con 0x7fe49c005a30 2020-11-20T12:19:32.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0038f0 con 0x7fe49c004590 2020-11-20T12:19:32.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.036+0000 7fe4affff700 1 --2- 172.21.15.116:0/3346327688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe49c004f00 0x7fe49c003240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4aeffd700 1 --2- 172.21.15.116:0/3346327688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c004590 0x7fe49c004a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4af7fe700 1 --2- 172.21.15.116:0/3346327688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c005a30 0x7fe49c005e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4affff700 1 -- 172.21.15.116:0/3346327688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c004590 msgr2=0x7fe49c004a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:32.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4affff700 1 --2- 172.21.15.116:0/3346327688 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c004590 0x7fe49c004a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4affff700 1 -- 172.21.15.116:0/3346327688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c005a30 msgr2=0x7fe49c005e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:32.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4affff700 1 --2- 172.21.15.116:0/3346327688 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe49c005a30 0x7fe49c005e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4affff700 1 -- 172.21.15.116:0/3346327688 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c004350 con 0x7fe49c004f00 2020-11-20T12:19:32.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4affff700 1 -- 172.21.15.116:0/3346327688 reap_dead start 2020-11-20T12:19:32.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4affff700 1 --2- 172.21.15.116:0/3346327688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe49c004f00 0x7fe49c003240 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7fe4a8001fa0 tx=0x7fe4a8007f10).ready entity=mon.0 client_cookie=9a3b69657c1dd196 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:32.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a80016a0 con 0x7fe49c004f00 2020-11-20T12:19:32.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a800cc80 con 0x7fe49c004f00 2020-11-20T12:19:32.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4ae7fc700 1 -- 172.21.15.116:0/3346327688 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8006870 con 0x7fe49c004f00 2020-11-20T12:19:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4b4844700 1 -- 172.21.15.116:0/3346327688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe49c004f00 msgr2=0x7fe49c003240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4b4844700 1 --2- 172.21.15.116:0/3346327688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe49c004f00 0x7fe49c003240 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7fe4a8001fa0 tx=0x7fe4a8007f10).stop 2020-11-20T12:19:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4b4844700 1 -- 172.21.15.116:0/3346327688 shutdown_connections 2020-11-20T12:19:32.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4b4844700 1 --2- 172.21.15.116:0/3346327688 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe49c004f00 0x7fe49c003240 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4b4844700 1 -- 172.21.15.116:0/3346327688 >> 172.21.15.116:0/3346327688 conn(0x7fe4b0068350 msgr2=0x7fe4b00627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:32.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4b4844700 1 -- 172.21.15.116:0/3346327688 shutdown_connections 2020-11-20T12:19:32.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.040+0000 7fe4b4844700 1 -- 172.21.15.116:0/3346327688 wait complete. 2020-11-20T12:19:32.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.044+0000 7fe4b4844700 1 Processor -- start 2020-11-20T12:19:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.044+0000 7fe4b4844700 1 -- start start 2020-11-20T12:19:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.044+0000 7fe4b4844700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c004f00 0x7fe4b013ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.044+0000 7fe4b4844700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4b0149e50 0x7fe4b00a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.044+0000 7fe4b4844700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4b014b2f0 0x7fe4b00a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.044+0000 7fe4b4844700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b014a3d0 con 0x7fe4b0149e50 2020-11-20T12:19:32.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.044+0000 7fe4b4844700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b014a670 con 0x7fe4b014b2f0 2020-11-20T12:19:32.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.044+0000 7fe4b4844700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b014a910 con 0x7fe49c004f00 2020-11-20T12:19:32.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.044+0000 7fe4aeffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4b0149e50 0x7fe4b00a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.044+0000 7fe4aeffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4b0149e50 0x7fe4b00a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48210/0 (socket says 172.21.15.116:48210) 2020-11-20T12:19:32.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.044+0000 7fe4aeffd700 1 -- 172.21.15.116:0/3949178062 learned_addr learned my addr 172.21.15.116:0/3949178062 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:32.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe4affff700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4b014b2f0 0x7fe4b00a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe4af7fe700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c004f00 0x7fe4b013ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe4aeffd700 1 -- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c004f00 msgr2=0x7fe4b013ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe4aeffd700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c004f00 0x7fe4b013ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe4aeffd700 1 -- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4b014b2f0 msgr2=0x7fe4b00a8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe4aeffd700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4b014b2f0 0x7fe4b00a8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe4aeffd700 1 -- 172.21.15.116:0/3949178062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b014abb0 con 0x7fe4b0149e50 2020-11-20T12:19:32.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe4aeffd700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4b0149e50 0x7fe4b00a7ff0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fe4a400e820 tx=0x7fe4a4001180).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe48f7fe700 1 -- 172.21.15.116:0/3949178062 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a400a5c0 con 0x7fe4b0149e50 2020-11-20T12:19:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe48f7fe700 1 -- 172.21.15.116:0/3949178062 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a4010040 con 0x7fe4b0149e50 2020-11-20T12:19:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe4b4844700 1 -- 172.21.15.116:0/3949178062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4b014ae50 con 0x7fe4b0149e50 2020-11-20T12:19:32.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe48f7fe700 1 -- 172.21.15.116:0/3949178062 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a400a950 con 0x7fe4b0149e50 2020-11-20T12:19:32.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe4b4844700 1 -- 172.21.15.116:0/3949178062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4b00a84e0 con 0x7fe4b0149e50 2020-11-20T12:19:32.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe4b4844700 1 -- 172.21.15.116:0/3949178062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4b005cf30 con 0x7fe4b0149e50 2020-11-20T12:19:32.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.048+0000 7fe48f7fe700 1 -- 172.21.15.116:0/3949178062 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe4a401b020 con 0x7fe4b0149e50 2020-11-20T12:19:32.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.052+0000 7fe48f7fe700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe4880557c0 0x7fe488057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:32.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.052+0000 7fe48f7fe700 1 -- 172.21.15.116:0/3949178062 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe4a406dcb0 con 0x7fe4b0149e50 2020-11-20T12:19:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.052+0000 7fe4af7fe700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe4880557c0 0x7fe488057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.052+0000 7fe4af7fe700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe4880557c0 0x7fe488057c80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fe4a0006a90 tx=0x7fe4a0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:32.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.056+0000 7fe48f7fe700 1 -- 172.21.15.116:0/3949178062 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe4a4043b50 con 0x7fe4b0149e50 2020-11-20T12:19:32.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.268+0000 7fe4b4844700 1 -- 172.21.15.116:0/3949178062 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]} v 0) v1 -- 0x7fe4b00befa0 con 0x7fe4b0149e50 2020-11-20T12:19:32.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe48f7fe700 1 -- 172.21.15.116:0/3949178062 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]}]=0 v20) v1 ==== 158+0+64 (secure 0 0 0) 0x7fe4a4047170 con 0x7fe4b0149e50 2020-11-20T12:19:32.278 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:19:32.278 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQBUtLdfv7FOEBAAcNA1BqusVps0OxH8B/ZsqA== 2020-11-20T12:19:32.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 -- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe4880557c0 msgr2=0x7fe488057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe4880557c0 0x7fe488057c80 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fe4a0006a90 tx=0x7fe4a0008040).stop 2020-11-20T12:19:32.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 -- 172.21.15.116:0/3949178062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4b0149e50 msgr2=0x7fe4b00a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:32.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4b0149e50 0x7fe4b00a7ff0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7fe4a400e820 tx=0x7fe4a4001180).stop 2020-11-20T12:19:32.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 -- 172.21.15.116:0/3949178062 shutdown_connections 2020-11-20T12:19:32.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe4880557c0 0x7fe488057c80 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe4b014b2f0 0x7fe4b00a8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe49c004f00 0x7fe4b013ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 --2- 172.21.15.116:0/3949178062 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe4b0149e50 0x7fe4b00a7ff0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:32.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 -- 172.21.15.116:0/3949178062 >> 172.21.15.116:0/3949178062 conn(0x7fe4b0068350 msgr2=0x7fe4b0150ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:32.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 -- 172.21.15.116:0/3949178062 shutdown_connections 2020-11-20T12:19:32.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:32.280+0000 7fe4b4844700 1 -- 172.21.15.116:0/3949178062 wait complete. 2020-11-20T12:19:32.304 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:19:32.305 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:19:32.324 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:19:32.337 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:19:32.338 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:19:32.338 INFO:teuthology.orchestra.run.smithi173.stderr:64 bytes copied, 0.000229352 s, 279 kB/s 2020-11-20T12:19:32.340 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:19:32.348 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:19:32.411 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:19:32.411 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:19:32.412 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:19:32.412 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:19:32.412 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.412 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:19:32.413 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.413 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:19:32.413 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.413 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:19:32.414 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:19:32.414 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.414 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:19:32.414 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:19:32.414 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.415 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:19:32.415 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:19:32.415 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.415 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:19:32.415 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:19:32.416 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.416 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:19:32.416 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.416 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:19:32.416 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:19:32.417 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:19:32.417 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:19:32.417 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:19:32.417 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:19:32.417 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:19:32.418 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:19:32.418 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:19:32.418 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:19:32.418 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.418 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:19:32.419 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.419 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:19:32.419 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.419 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:19:32.419 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.420 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:19:32.420 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:19:32.420 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:19:32.420 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:19:32.420 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:19:32.420 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:19:32.421 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.421 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:19:32.421 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:19:32.421 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.422 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:19:32.422 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:19:32.422 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.422 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:19:32.422 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:19:32.422 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.423 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:19:32.423 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:19:32.423 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:19:32.423 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.423 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:19:32.424 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:19:32.424 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.424 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:19:32.424 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.424 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:19:32.425 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.425 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:19:32.425 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.425 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:19:32.425 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:19:32.426 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.426 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:19:32.426 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:19:32.426 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:19:32.426 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:19:32.427 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:19:32.427 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:19:32.427 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:19:32.427 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:19:32.427 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:19:32.428 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:19:32.428 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:19:32.428 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:19:32.428 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:19:32.428 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:19:32.429 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:19:32.429 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:19:32.429 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:19:32.429 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:19:32.429 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:19:32.430 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:19:32.430 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.430 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:19:32.430 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:19:32.430 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:19:32.431 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:19:32.431 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:19:32.431 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:19:32.431 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:19:32.431 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:19:32.432 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.432 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:19:32.432 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:19:32.432 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:19:32.432 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:19:32.433 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:19:32.433 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:19:32.433 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:19:32.433 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:19:32.434 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:19:32.434 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:19:32.434 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:19:32.434 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:19:32.434 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:32.435 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:19:32.435 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:19:32.435 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:19:32.436 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:19:32.436 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:19:32.436 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:19:32.436 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:19:32.436 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:19:32.437 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:19:32.437 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:19:32.437 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:19:32.437 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:19:32.437 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:19:32.438 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:19:32.438 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:19:32.438 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:19:32.438 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:19:32.438 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:19:32.439 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:19:32.439 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:19:32.439 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:19:32.439 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:19:32.439 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:19:32.440 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:19:32.440 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:19:32.440 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:19:32.440 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:19:32.440 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:19:32.441 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:19:32.441 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:19:32.441 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:19:32.441 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:19:32.441 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:19:32.442 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:19:32.442 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:19:32.442 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:19:32.442 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:19:32.458 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:19:32.459 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:19:32.460 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:19:32.460 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:19:32.461 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.461 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:19:32.462 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.462 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:19:32.463 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.463 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:19:32.463 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:19:32.464 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.464 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:19:32.464 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:19:32.465 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.466 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:19:32.466 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:19:32.467 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.467 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:19:32.467 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:19:32.468 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.468 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:19:32.469 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.469 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:19:32.470 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:19:32.470 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:19:32.470 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:19:32.471 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:19:32.473 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:19:32.473 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:19:32.474 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:19:32.474 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:19:32.474 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:19:32.474 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.474 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:19:32.475 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.475 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:19:32.475 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.475 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:19:32.475 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.476 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:19:32.476 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:19:32.476 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:19:32.476 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:19:32.476 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:19:32.477 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:19:32.477 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.477 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:19:32.477 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:19:32.477 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.478 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:19:32.478 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:19:32.478 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.478 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:19:32.478 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:19:32.479 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.479 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:19:32.479 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:19:32.479 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:19:32.479 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.480 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:19:32.480 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:19:32.480 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.480 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:19:32.480 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.482 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:19:32.482 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.483 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:19:32.483 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.484 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:19:32.484 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:19:32.485 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.485 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:19:32.486 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:19:32.487 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:19:32.488 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:19:32.489 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:19:32.490 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:19:32.491 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:19:32.493 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:19:32.495 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:19:32.496 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:19:32.497 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:19:32.497 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:19:32.498 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:19:32.499 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:19:32.499 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:19:32.499 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:19:32.499 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:19:32.500 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:19:32.500 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:19:32.500 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:19:32.501 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.501 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:19:32.501 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:19:32.502 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:19:32.502 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:19:32.502 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:19:32.503 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:19:32.503 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:19:32.503 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:19:32.504 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.504 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:19:32.504 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:19:32.504 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:19:32.505 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:19:32.505 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:19:32.505 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:19:32.506 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:19:32.506 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:19:32.506 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:19:32.507 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:19:32.507 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:19:32.507 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:19:32.507 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:32.508 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:19:32.508 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:19:32.508 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:19:32.509 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:19:32.509 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:19:32.509 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:19:32.510 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:19:32.510 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:19:32.510 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:19:32.511 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:19:32.511 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:19:32.511 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:19:32.511 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:19:32.512 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:19:32.512 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:19:32.512 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:19:32.513 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:19:32.513 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:19:32.513 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:19:32.513 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:19:32.514 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:19:32.514 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:19:32.514 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:19:32.515 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:19:32.515 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:19:32.515 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:19:32.515 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:19:32.516 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:19:32.516 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:19:32.516 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:19:32.517 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:19:32.517 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:19:32.517 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:19:32.518 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:19:32.518 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:19:32.518 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:19:32.519 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:19:32.520 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:19:32.521 INFO:teuthology.orchestra.run.smithi173:> from __future__ import print_function 2020-11-20T12:19:32.521 INFO:teuthology.orchestra.run.smithi173:> from ceph_volume_client import CephFSVolumeClient, VolumePath 2020-11-20T12:19:32.522 INFO:teuthology.orchestra.run.smithi173:> from sys import version_info as sys_version_info 2020-11-20T12:19:32.522 INFO:teuthology.orchestra.run.smithi173:> from rados import OSError as rados_OSError 2020-11-20T12:19:32.523 INFO:teuthology.orchestra.run.smithi173:> import logging 2020-11-20T12:19:32.523 INFO:teuthology.orchestra.run.smithi173:> log = logging.getLogger("ceph_volume_client") 2020-11-20T12:19:32.524 INFO:teuthology.orchestra.run.smithi173:> log.addHandler(logging.StreamHandler()) 2020-11-20T12:19:32.524 INFO:teuthology.orchestra.run.smithi173:> log.setLevel(logging.DEBUG) 2020-11-20T12:19:32.524 INFO:teuthology.orchestra.run.smithi173:> vc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None) 2020-11-20T12:19:32.525 INFO:teuthology.orchestra.run.smithi173:> vc.connect() 2020-11-20T12:19:32.525 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:19:32.526 INFO:teuthology.orchestra.run.smithi173:> vp = VolumePath("grpid", "volid") 2020-11-20T12:19:32.526 INFO:teuthology.orchestra.run.smithi173:> vc.create_volume(vp, 10, data_isolated=True) 2020-11-20T12:19:32.527 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:19:32.527 INFO:teuthology.orchestra.run.smithi173:> vc.disconnect() 2020-11-20T12:19:32.528 INFO:teuthology.orchestra.run.smithi173:> ' 2020-11-20T12:19:32.710 INFO:teuthology.orchestra.run.smithi173.stderr:Configuring to RADOS with config /etc/ceph/ceph.conf... 2020-11-20T12:19:32.715 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to RADOS... 2020-11-20T12:19:32.727 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to RADOS complete 2020-11-20T12:19:32.728 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to cephfs... 2020-11-20T12:19:32.728 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS initializing... 2020-11-20T12:19:32.735 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS mounting... 2020-11-20T12:19:32.746 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to cephfs complete 2020-11-20T12:19:32.746 INFO:teuthology.orchestra.run.smithi173.stderr:Recovering from partial auth updates (if any)... 2020-11-20T12:19:32.746 INFO:teuthology.orchestra.run.smithi173.stderr:Nothing to recover. No auth meta files. 2020-11-20T12:19:32.746 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: /volumes/grpid/volid 2020-11-20T12:19:32.770 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: grpid/volid, create pool fsvolume_volid as data_isolated =True. 2020-11-20T12:19:38.445 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: grpid/volid, using rados namespace fsvolumens_volid to isolate data. 2020-11-20T12:19:38.458 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:19:38.458 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs... 2020-11-20T12:19:38.472 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs complete 2020-11-20T12:19:38.473 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados... 2020-11-20T12:19:38.476 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados complete 2020-11-20T12:19:38.480 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:19:38.639 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:19:38.640 INFO:teuthology.orchestra.run.smithi116:> import os 2020-11-20T12:19:38.640 INFO:teuthology.orchestra.run.smithi116:> import stat 2020-11-20T12:19:38.641 INFO:teuthology.orchestra.run.smithi116:> import json 2020-11-20T12:19:38.641 INFO:teuthology.orchestra.run.smithi116:> import sys 2020-11-20T12:19:38.642 INFO:teuthology.orchestra.run.smithi116:> 2020-11-20T12:19:38.642 INFO:teuthology.orchestra.run.smithi116:> try: 2020-11-20T12:19:38.643 INFO:teuthology.orchestra.run.smithi116:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/volumes/grpid/volid'"'"') 2020-11-20T12:19:38.643 INFO:teuthology.orchestra.run.smithi116:> except OSError as e: 2020-11-20T12:19:38.643 INFO:teuthology.orchestra.run.smithi116:> sys.exit(e.errno) 2020-11-20T12:19:38.644 INFO:teuthology.orchestra.run.smithi116:> 2020-11-20T12:19:38.645 INFO:teuthology.orchestra.run.smithi116:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2020-11-20T12:19:38.646 INFO:teuthology.orchestra.run.smithi116:> print(json.dumps( 2020-11-20T12:19:38.646 INFO:teuthology.orchestra.run.smithi116:> dict([(a, getattr(s, a)) for a in attrs]), 2020-11-20T12:19:38.647 INFO:teuthology.orchestra.run.smithi116:> indent=2)) 2020-11-20T12:19:38.648 INFO:teuthology.orchestra.run.smithi116:> ' 2020-11-20T12:19:38.777 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:19:38.777 INFO:teuthology.orchestra.run.smithi116.stdout: "st_mode": 16877, 2020-11-20T12:19:38.777 INFO:teuthology.orchestra.run.smithi116.stdout: "st_ino": 1099511627778, 2020-11-20T12:19:38.777 INFO:teuthology.orchestra.run.smithi116.stdout: "st_dev": 47, 2020-11-20T12:19:38.778 INFO:teuthology.orchestra.run.smithi116.stdout: "st_nlink": 2, 2020-11-20T12:19:38.778 INFO:teuthology.orchestra.run.smithi116.stdout: "st_uid": 0, 2020-11-20T12:19:38.778 INFO:teuthology.orchestra.run.smithi116.stdout: "st_gid": 0, 2020-11-20T12:19:38.778 INFO:teuthology.orchestra.run.smithi116.stdout: "st_size": 0, 2020-11-20T12:19:38.779 INFO:teuthology.orchestra.run.smithi116.stdout: "st_atime": 1605874772.7644105, 2020-11-20T12:19:38.779 INFO:teuthology.orchestra.run.smithi116.stdout: "st_mtime": 1605874772.7644105, 2020-11-20T12:19:38.779 INFO:teuthology.orchestra.run.smithi116.stdout: "st_ctime": 1605874778.4513178 2020-11-20T12:19:38.779 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:19:38.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:19:38.956 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/grpid/volid) 2020-11-20T12:19:39.005 INFO:teuthology.orchestra.run.smithi116.stdout:fsvolumens_volid 2020-11-20T12:19:39.006 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:19:39.006 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:19:39.092 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:19:39.135 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:19:39.136 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-20T12:19:39.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:19:39.137 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-20T12:19:39.217 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:19:39.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:39.218 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:19:39.236 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2020-11-20T12:19:39.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:19:39.237 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.2 2020-11-20T12:19:39.372 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:19:39.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:39.373 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T12:19:39.389 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2020-11-20T12:19:39.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:19:39.391 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.3 2020-11-20T12:19:39.471 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:19:39.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:19:39.472 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T12:19:39.490 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:19:39.495 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:19:39.559 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:19:39.560 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:19:39.560 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:19:39.560 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:19:39.560 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.560 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:19:39.561 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.561 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:19:39.561 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.561 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:19:39.562 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:19:39.562 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.562 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:19:39.562 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:19:39.562 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.563 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:19:39.563 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:19:39.563 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.563 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:19:39.564 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:19:39.564 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.564 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:19:39.564 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.564 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:19:39.565 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:19:39.565 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:19:39.565 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:19:39.565 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:19:39.565 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:19:39.566 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:19:39.566 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:19:39.566 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:19:39.566 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:19:39.567 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.567 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:19:39.567 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.567 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:19:39.567 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.568 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:19:39.568 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.568 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:19:39.568 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:19:39.568 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:19:39.569 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:19:39.569 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:19:39.569 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:19:39.569 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.569 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:19:39.570 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:19:39.570 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.570 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:19:39.570 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:19:39.570 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.571 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:19:39.571 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:19:39.571 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.571 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:19:39.572 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:19:39.572 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:19:39.572 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.572 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:19:39.572 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:19:39.573 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.573 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:19:39.573 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.573 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:19:39.573 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.574 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:19:39.574 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.574 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:19:39.574 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:19:39.574 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.575 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:19:39.575 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:19:39.575 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:19:39.575 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:19:39.575 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:19:39.576 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:19:39.576 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:19:39.576 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:19:39.576 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:19:39.576 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:19:39.577 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:19:39.577 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:19:39.577 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:19:39.577 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:19:39.578 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:19:39.578 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:19:39.578 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:19:39.578 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:19:39.578 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:19:39.579 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:19:39.579 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.579 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:19:39.579 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:19:39.579 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:19:39.580 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:19:39.580 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:19:39.580 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:19:39.580 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:19:39.581 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:19:39.581 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.581 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:19:39.581 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:19:39.581 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:19:39.582 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:19:39.582 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:19:39.582 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:19:39.582 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:19:39.582 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:19:39.583 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:19:39.583 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:19:39.583 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:19:39.583 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:19:39.583 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:39.584 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:19:39.584 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:19:39.584 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:19:39.584 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:19:39.584 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:19:39.585 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:19:39.585 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:19:39.585 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:19:39.585 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:19:39.586 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:19:39.586 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:19:39.586 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:19:39.586 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:19:39.586 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:19:39.587 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:19:39.587 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:19:39.587 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:19:39.587 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:19:39.587 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:19:39.588 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:19:39.588 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:19:39.588 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:19:39.588 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:19:39.588 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:19:39.589 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:19:39.589 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:19:39.589 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:19:39.589 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:19:39.589 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:19:39.590 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:19:39.590 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:19:39.590 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:19:39.590 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:19:39.590 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:19:39.591 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:19:39.591 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:19:39.592 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:19:39.593 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:19:39.593 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:19:39.593 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:19:39.593 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.593 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:19:39.594 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.594 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:19:39.594 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.594 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:19:39.594 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:19:39.595 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.595 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:19:39.595 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:19:39.595 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.596 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:19:39.596 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:19:39.596 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.596 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:19:39.596 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:19:39.597 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.597 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:19:39.597 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.597 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:19:39.597 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:19:39.598 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:19:39.598 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:19:39.598 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:19:39.598 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:19:39.598 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:19:39.599 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:19:39.599 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:19:39.599 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:19:39.599 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.599 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:19:39.600 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.600 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:19:39.600 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.600 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:19:39.600 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.601 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:19:39.601 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:19:39.601 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:19:39.601 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:19:39.601 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:19:39.602 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:19:39.602 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.602 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:19:39.602 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:19:39.602 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.603 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:19:39.603 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:19:39.603 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.603 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:19:39.603 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:19:39.604 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.604 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:19:39.604 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:19:39.604 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:19:39.605 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.605 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:19:39.605 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:19:39.605 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.605 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:19:39.606 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.606 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:19:39.606 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.606 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:19:39.606 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.607 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:19:39.607 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:19:39.607 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.607 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:19:39.607 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:19:39.608 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:19:39.608 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:19:39.608 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:19:39.608 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:19:39.608 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:19:39.609 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:19:39.609 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:19:39.609 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:19:39.609 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:19:39.609 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:19:39.610 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:19:39.610 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:19:39.610 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:19:39.610 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:19:39.610 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:19:39.611 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:19:39.611 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:19:39.611 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:19:39.611 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.611 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:19:39.612 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:19:39.612 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:19:39.612 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:19:39.612 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:19:39.612 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:19:39.613 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:19:39.613 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:19:39.613 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.613 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:19:39.613 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:19:39.614 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:19:39.614 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:19:39.614 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:19:39.614 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:19:39.614 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:19:39.615 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:19:39.615 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:19:39.615 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:19:39.615 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:19:39.615 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:19:39.616 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:19:39.616 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:19:39.616 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:19:39.616 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:19:39.616 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:19:39.617 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:19:39.617 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:19:39.617 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:19:39.617 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:19:39.617 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:19:39.618 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:19:39.618 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:19:39.618 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:19:39.618 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:19:39.618 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:19:39.619 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:19:39.619 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:19:39.619 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:19:39.619 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:19:39.619 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:19:39.620 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:19:39.620 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:19:39.620 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:19:39.620 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:19:39.620 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:19:39.621 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:19:39.621 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:19:39.621 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:19:39.621 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:19:39.622 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:19:39.622 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:19:39.622 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:19:39.622 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:19:39.622 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:19:39.623 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:19:39.623 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:19:39.623 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:19:39.624 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-20T12:19:39.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.812+0000 7f0e429da700 1 Processor -- start 2020-11-20T12:19:39.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.812+0000 7f0e429da700 1 -- start start 2020-11-20T12:19:39.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.812+0000 7f0e429da700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0e3c155800 0x7f0e3c0c0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.812+0000 7f0e429da700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c0bada0 0x7f0e3c0bb250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.812+0000 7f0e429da700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e3c156790 con 0x7f0e3c0c0cc0 2020-11-20T12:19:39.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.812+0000 7f0e429da700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e3c155d10 con 0x7f0e3c155800 2020-11-20T12:19:39.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.812+0000 7f0e429da700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c155fb0 con 0x7f0e3c0bada0 2020-11-20T12:19:39.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.812+0000 7f0e3bfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0e3c0c0cc0 0x7f0e3c0c1860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57650/0 (socket says 172.21.15.116:57650) 2020-11-20T12:19:39.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.812+0000 7f0e3bfff700 1 -- 172.21.15.116:0/3271652594 learned_addr learned my addr 172.21.15.116:0/3271652594 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:39.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.812+0000 7f0e3b7fe700 1 --2- 172.21.15.116:0/3271652594 >> v2:172.21.15.173:3300/0 conn(0x7f0e3c155800 0x7f0e3c0c0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.812+0000 7f0e40f77700 1 --2- 172.21.15.116:0/3271652594 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c0bada0 0x7f0e3c0bb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e3affd700 1 -- 172.21.15.116:0/3271652594 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632821754 0 0) 0x7f0e3c156790 con 0x7f0e3c0c0cc0 2020-11-20T12:19:39.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e3affd700 1 -- 172.21.15.116:0/3271652594 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e28003240 con 0x7f0e3c0c0cc0 2020-11-20T12:19:39.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e40f77700 1 -- 172.21.15.116:0/3271652594 >> v2:172.21.15.173:3300/0 conn(0x7f0e3c155800 msgr2=0x7f0e3c0c0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:39.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e40f77700 1 --2- 172.21.15.116:0/3271652594 >> v2:172.21.15.173:3300/0 conn(0x7f0e3c155800 0x7f0e3c0c0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e40f77700 1 -- 172.21.15.116:0/3271652594 >> v1:172.21.15.116:6789/0 conn(0x7f0e3c0c0cc0 legacy=0x7f0e3c0c1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e40f77700 1 -- 172.21.15.116:0/3271652594 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c156250 con 0x7f0e3c0bada0 2020-11-20T12:19:39.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e40f77700 1 --2- 172.21.15.116:0/3271652594 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c0bada0 0x7f0e3c0bb250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0e34009c90 tx=0x7f0e340091d0).ready entity=mon.2 client_cookie=9daf4ad8c7fb1b04 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e3affd700 1 -- 172.21.15.116:0/3271652594 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400b9c0 con 0x7f0e3c0bada0 2020-11-20T12:19:39.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e3affd700 1 -- 172.21.15.116:0/3271652594 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3400bb20 con 0x7f0e3c0bada0 2020-11-20T12:19:39.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e3affd700 1 -- 172.21.15.116:0/3271652594 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400bea0 con 0x7f0e3c0bada0 2020-11-20T12:19:39.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e429da700 1 -- 172.21.15.116:0/3271652594 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c0bada0 msgr2=0x7f0e3c0bb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e429da700 1 --2- 172.21.15.116:0/3271652594 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c0bada0 0x7f0e3c0bb250 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f0e34009c90 tx=0x7f0e340091d0).stop 2020-11-20T12:19:39.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e429da700 1 -- 172.21.15.116:0/3271652594 shutdown_connections 2020-11-20T12:19:39.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e429da700 1 --2- 172.21.15.116:0/3271652594 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c0bada0 0x7f0e3c0bb250 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e429da700 1 --2- 172.21.15.116:0/3271652594 >> v2:172.21.15.173:3300/0 conn(0x7f0e3c155800 0x7f0e3c0c0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e429da700 1 -- 172.21.15.116:0/3271652594 >> 172.21.15.116:0/3271652594 conn(0x7f0e3c0c2af0 msgr2=0x7f0e3c063920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:39.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e429da700 1 -- 172.21.15.116:0/3271652594 shutdown_connections 2020-11-20T12:19:39.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.816+0000 7f0e429da700 1 -- 172.21.15.116:0/3271652594 wait complete. 2020-11-20T12:19:39.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e429da700 1 Processor -- start 2020-11-20T12:19:39.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e429da700 1 -- start start 2020-11-20T12:19:39.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e429da700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e3c0bada0 0x7f0e3c0a3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e429da700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3c0c0cc0 0x7f0e3c0a3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e429da700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c155800 0x7f0e3c1625b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e429da700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c0a3bd0 con 0x7f0e3c0bada0 2020-11-20T12:19:39.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e429da700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c0a3960 con 0x7f0e3c0c0cc0 2020-11-20T12:19:39.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e429da700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c1525e0 con 0x7f0e3c155800 2020-11-20T12:19:39.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e3bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e3c0bada0 0x7f0e3c0a3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e3bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e3c0bada0 0x7f0e3c0a3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48224/0 (socket says 172.21.15.116:48224) 2020-11-20T12:19:39.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e3bfff700 1 -- 172.21.15.116:0/614671499 learned_addr learned my addr 172.21.15.116:0/614671499 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:39.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e40f77700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c155800 0x7f0e3c1625b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.820+0000 7f0e3b7fe700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3c0c0cc0 0x7f0e3c0a3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.824+0000 7f0e40f77700 1 -- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3c0c0cc0 msgr2=0x7f0e3c0a3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.824+0000 7f0e40f77700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3c0c0cc0 0x7f0e3c0a3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.824+0000 7f0e40f77700 1 -- 172.21.15.116:0/614671499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e3c0bada0 msgr2=0x7f0e3c0a3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:39.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.824+0000 7f0e40f77700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e3c0bada0 0x7f0e3c0a3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:39.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.824+0000 7f0e40f77700 1 -- 172.21.15.116:0/614671499 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e3c152880 con 0x7f0e3c155800 2020-11-20T12:19:39.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.824+0000 7f0e40f77700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c155800 0x7f0e3c1625b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0e34009c90 tx=0x7f0e34016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.824+0000 7f0e1bfff700 1 -- 172.21.15.116:0/614671499 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3400b3c0 con 0x7f0e3c155800 2020-11-20T12:19:39.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.824+0000 7f0e1bfff700 1 -- 172.21.15.116:0/614671499 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e3401c5f0 con 0x7f0e3c155800 2020-11-20T12:19:39.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.824+0000 7f0e429da700 1 -- 172.21.15.116:0/614671499 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c152b20 con 0x7f0e3c155800 2020-11-20T12:19:39.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.824+0000 7f0e1bfff700 1 -- 172.21.15.116:0/614671499 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3401c970 con 0x7f0e3c155800 2020-11-20T12:19:39.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.824+0000 7f0e429da700 1 -- 172.21.15.116:0/614671499 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c162aa0 con 0x7f0e3c155800 2020-11-20T12:19:39.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.828+0000 7f0e429da700 1 -- 172.21.15.116:0/614671499 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e3c05d2a0 con 0x7f0e3c155800 2020-11-20T12:19:39.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.828+0000 7f0e1bfff700 1 -- 172.21.15.116:0/614671499 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0e34024030 con 0x7f0e3c155800 2020-11-20T12:19:39.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.828+0000 7f0e1bfff700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0e140557c0 0x7f0e14057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:39.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.828+0000 7f0e1bfff700 1 -- 172.21.15.116:0/614671499 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f0e340703c0 con 0x7f0e3c155800 2020-11-20T12:19:39.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.828+0000 7f0e3bfff700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0e140557c0 0x7f0e14057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:39.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.832+0000 7f0e3bfff700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0e140557c0 0x7f0e14057c80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f0e2c0011a0 tx=0x7f0e2c000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:39.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.832+0000 7f0e1bfff700 1 -- 172.21.15.116:0/614671499 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0e34046050 con 0x7f0e3c155800 2020-11-20T12:19:39.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:39.988+0000 7f0e429da700 1 -- 172.21.15.116:0/614671499 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0e3c063450 con 0x7f0e3c155800 2020-11-20T12:19:39.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.000+0000 7f0e1bfff700 1 -- 172.21.15.116:0/614671499 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0e340150a0 con 0x7f0e3c155800 2020-11-20T12:19:40.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.004+0000 7f0e429da700 1 -- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0e140557c0 msgr2=0x7f0e14057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.004+0000 7f0e429da700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0e140557c0 0x7f0e14057c80 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f0e2c0011a0 tx=0x7f0e2c000b80).stop 2020-11-20T12:19:40.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.004+0000 7f0e429da700 1 -- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c155800 msgr2=0x7f0e3c1625b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.004+0000 7f0e429da700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c155800 0x7f0e3c1625b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f0e34009c90 tx=0x7f0e34016000).stop 2020-11-20T12:19:40.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.004+0000 7f0e429da700 1 -- 172.21.15.116:0/614671499 shutdown_connections 2020-11-20T12:19:40.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.004+0000 7f0e429da700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0e140557c0 0x7f0e14057c80 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.004+0000 7f0e429da700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0e3c155800 0x7f0e3c1625b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.004+0000 7f0e429da700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0e3c0bada0 0x7f0e3c0a3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.004+0000 7f0e429da700 1 --2- 172.21.15.116:0/614671499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0e3c0c0cc0 0x7f0e3c0a3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.004+0000 7f0e429da700 1 -- 172.21.15.116:0/614671499 >> 172.21.15.116:0/614671499 conn(0x7f0e3c0c2af0 msgr2=0x7f0e3c150330 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:40.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.008+0000 7f0e429da700 1 -- 172.21.15.116:0/614671499 shutdown_connections 2020-11-20T12:19:40.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.008+0000 7f0e429da700 1 -- 172.21.15.116:0/614671499 wait complete. 2020-11-20T12:19:40.026 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_default_prefix' 2020-11-20T12:19:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.196+0000 7ff25a80c700 1 Processor -- start 2020-11-20T12:19:40.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.200+0000 7ff25a80c700 1 -- start start 2020-11-20T12:19:40.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.200+0000 7ff25a80c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff25413b380 0x7ff2540acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.200+0000 7ff25a80c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2540ad220 0x7ff2540a8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.200+0000 7ff25a80c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2540ada00 con 0x7ff25413afa0 2020-11-20T12:19:40.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.200+0000 7ff25a80c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2540adca0 con 0x7ff25413b380 2020-11-20T12:19:40.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.200+0000 7ff25a80c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540adf40 con 0x7ff2540ad220 2020-11-20T12:19:40.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff259009700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff25413b380 0x7ff2540acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff25980a700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7ff25413afa0 0x7ff2540aa5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:53894/0 (socket says 172.21.15.116:53894) 2020-11-20T12:19:40.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff25980a700 1 -- 172.21.15.116:0/3747434495 learned_addr learned my addr 172.21.15.116:0/3747434495 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:40.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff25a00b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2540ad220 0x7ff2540a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff259009700 1 -- 172.21.15.116:0/3747434495 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2540ad220 msgr2=0x7ff2540a8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:40.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff258808700 1 -- 172.21.15.116:0/3747434495 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852488880 0 0) 0x7ff2540ada00 con 0x7ff25413afa0 2020-11-20T12:19:40.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff259009700 1 --2- 172.21.15.116:0/3747434495 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2540ad220 0x7ff2540a8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff259009700 1 -- 172.21.15.116:0/3747434495 >> v1:172.21.15.173:6789/0 conn(0x7ff25413afa0 legacy=0x7ff2540aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff259009700 1 -- 172.21.15.116:0/3747434495 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2540ae1e0 con 0x7ff25413b380 2020-11-20T12:19:40.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff259009700 1 --2- 172.21.15.116:0/3747434495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff25413b380 0x7ff2540acd30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7ff24400b9a0 tx=0x7ff244006df0).ready entity=mon.0 client_cookie=231c790b382c457e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:40.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff258808700 1 -- 172.21.15.116:0/3747434495 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2440097b0 con 0x7ff25413b380 2020-11-20T12:19:40.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff258808700 1 -- 172.21.15.116:0/3747434495 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff244009910 con 0x7ff25413b380 2020-11-20T12:19:40.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff258808700 1 -- 172.21.15.116:0/3747434495 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244009c90 con 0x7ff25413b380 2020-11-20T12:19:40.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff25a80c700 1 -- 172.21.15.116:0/3747434495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff25413b380 msgr2=0x7ff2540acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.204+0000 7ff25a80c700 1 --2- 172.21.15.116:0/3747434495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff25413b380 0x7ff2540acd30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7ff24400b9a0 tx=0x7ff244006df0).stop 2020-11-20T12:19:40.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.208+0000 7ff25a80c700 1 -- 172.21.15.116:0/3747434495 shutdown_connections 2020-11-20T12:19:40.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.208+0000 7ff25a80c700 1 --2- 172.21.15.116:0/3747434495 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2540ad220 0x7ff2540a8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.208+0000 7ff25a80c700 1 --2- 172.21.15.116:0/3747434495 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff25413b380 0x7ff2540acd30 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.208+0000 7ff25a80c700 1 -- 172.21.15.116:0/3747434495 >> 172.21.15.116:0/3747434495 conn(0x7ff2540678a0 msgr2=0x7ff254067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:40.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.208+0000 7ff25a80c700 1 -- 172.21.15.116:0/3747434495 shutdown_connections 2020-11-20T12:19:40.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.208+0000 7ff25a80c700 1 -- 172.21.15.116:0/3747434495 wait complete. 2020-11-20T12:19:40.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.208+0000 7ff25a80c700 1 Processor -- start 2020-11-20T12:19:40.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.208+0000 7ff25a80c700 1 -- start start 2020-11-20T12:19:40.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff25a80c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2540ad220 0x7ff254166220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff25a80c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff25413afa0 0x7ff254167db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff25a80c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff25413b380 0x7ff2540aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff25a80c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540a0470 con 0x7ff25413afa0 2020-11-20T12:19:40.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff25a80c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540a0710 con 0x7ff25413b380 2020-11-20T12:19:40.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff25a80c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2540a09b0 con 0x7ff2540ad220 2020-11-20T12:19:40.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff259009700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff25413afa0 0x7ff254167db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff259009700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff25413afa0 0x7ff254167db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48238/0 (socket says 172.21.15.116:48238) 2020-11-20T12:19:40.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff259009700 1 -- 172.21.15.116:0/2001439241 learned_addr learned my addr 172.21.15.116:0/2001439241 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:40.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff25980a700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2540ad220 0x7ff254166220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff259009700 1 -- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2540ad220 msgr2=0x7ff254166220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff259009700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2540ad220 0x7ff254166220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff259009700 1 -- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff25413b380 msgr2=0x7ff2540aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff259009700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff25413b380 0x7ff2540aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:40.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff259009700 1 -- 172.21.15.116:0/2001439241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2540a0c50 con 0x7ff25413afa0 2020-11-20T12:19:40.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff25980a700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2540ad220 0x7ff254166220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:40.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff259009700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff25413afa0 0x7ff254167db0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7ff244006e30 tx=0x7ff244015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:40.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff241ffb700 1 -- 172.21.15.116:0/2001439241 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff244011e50 con 0x7ff25413afa0 2020-11-20T12:19:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff241ffb700 1 -- 172.21.15.116:0/2001439241 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff24401e460 con 0x7ff25413afa0 2020-11-20T12:19:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff25a80c700 1 -- 172.21.15.116:0/2001439241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2540a0ef0 con 0x7ff25413afa0 2020-11-20T12:19:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.212+0000 7ff241ffb700 1 -- 172.21.15.116:0/2001439241 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff24401e7e0 con 0x7ff25413afa0 2020-11-20T12:19:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.216+0000 7ff25a80c700 1 -- 172.21.15.116:0/2001439241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2541684c0 con 0x7ff25413afa0 2020-11-20T12:19:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.216+0000 7ff25a80c700 1 -- 172.21.15.116:0/2001439241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff25405cf30 con 0x7ff25413afa0 2020-11-20T12:19:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.216+0000 7ff241ffb700 1 -- 172.21.15.116:0/2001439241 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff244025030 con 0x7ff25413afa0 2020-11-20T12:19:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.216+0000 7ff241ffb700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff22c0557c0 0x7ff22c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.216+0000 7ff241ffb700 1 -- 172.21.15.116:0/2001439241 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7ff2440700c0 con 0x7ff25413afa0 2020-11-20T12:19:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.220+0000 7ff25980a700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff22c0557c0 0x7ff22c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.220+0000 7ff25980a700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff22c0557c0 0x7ff22c057c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7ff250001550 tx=0x7ff250009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:40.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.224+0000 7ff241ffb700 1 -- 172.21.15.116:0/2001439241 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff244045d80 con 0x7ff25413afa0 2020-11-20T12:19:40.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.444+0000 7ff25a80c700 1 -- 172.21.15.116:0/2001439241 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_default_prefix"]} v 0) v1 -- 0x7ff254060170 con 0x7ff25413afa0 2020-11-20T12:19:40.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff241ffb700 1 -- 172.21.15.116:0/2001439241 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_default_prefix"]}]=0 v216) v1 ==== 147+0+0 (secure 0 0 0) 0x7ff244023070 con 0x7ff25413afa0 2020-11-20T12:19:40.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff25a80c700 1 -- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff22c0557c0 msgr2=0x7ff22c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff25a80c700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff22c0557c0 0x7ff22c057c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7ff250001550 tx=0x7ff250009040).stop 2020-11-20T12:19:40.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff25a80c700 1 -- 172.21.15.116:0/2001439241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff25413afa0 msgr2=0x7ff254167db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff25a80c700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff25413afa0 0x7ff254167db0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7ff244006e30 tx=0x7ff244015000).stop 2020-11-20T12:19:40.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff25a80c700 1 -- 172.21.15.116:0/2001439241 shutdown_connections 2020-11-20T12:19:40.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff25a80c700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff22c0557c0 0x7ff22c057c80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff25a80c700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff25413b380 0x7ff2540aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:40.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff25a80c700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff2540ad220 0x7ff254166220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff25a80c700 1 --2- 172.21.15.116:0/2001439241 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff25413afa0 0x7ff254167db0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff25a80c700 1 -- 172.21.15.116:0/2001439241 >> 172.21.15.116:0/2001439241 conn(0x7ff2540678a0 msgr2=0x7ff254139e70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:40.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.476+0000 7ff25a80c700 1 -- 172.21.15.116:0/2001439241 shutdown_connections 2020-11-20T12:19:40.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.480+0000 7ff25a80c700 1 -- 172.21.15.116:0/2001439241 wait complete. 2020-11-20T12:19:40.501 INFO:tasks.cephfs_test_runner:test_default_prefix (tasks.cephfs.test_volume_client.TestVolumeClient) ... ok 2020-11-20T12:19:40.501 INFO:tasks.cephfs_test_runner:Starting test: test_delete_object (tasks.cephfs.test_volume_client.TestVolumeClient) 2020-11-20T12:19:40.502 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_delete_object' 2020-11-20T12:19:40.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.672+0000 7f4a4d56d700 1 Processor -- start 2020-11-20T12:19:40.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.672+0000 7f4a4d56d700 1 -- start start 2020-11-20T12:19:40.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a4d56d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4a48151580 0x7f4a48155340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a4d56d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a480c8dc0 0x7f4a48061590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a4d56d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a480ac0e0 con 0x7f4a48151960 2020-11-20T12:19:40.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a4d56d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a481559e0 con 0x7f4a480c8dc0 2020-11-20T12:19:40.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a4d56d700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a48155c80 con 0x7f4a48151580 2020-11-20T12:19:40.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a477fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a480c8dc0 0x7f4a48061590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a46ffd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4a48151580 0x7f4a48155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a477fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4a480c8dc0 0x7f4a48061590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48248/0 (socket says 172.21.15.116:48248) 2020-11-20T12:19:40.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a477fe700 1 -- 172.21.15.116:0/1830136939 learned_addr learned my addr 172.21.15.116:0/1830136939 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:40.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a46ffd700 1 -- 172.21.15.116:0/1830136939 >> v2:172.21.15.116:3300/0 conn(0x7f4a480c8dc0 msgr2=0x7f4a48061590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:40.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a46ffd700 1 --2- 172.21.15.116:0/1830136939 >> v2:172.21.15.116:3300/0 conn(0x7f4a480c8dc0 0x7f4a48061590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a46ffd700 1 -- 172.21.15.116:0/1830136939 >> v1:172.21.15.173:6789/0 conn(0x7f4a48151960 legacy=0x7f4a480ab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a45ffb700 1 -- 172.21.15.116:0/1830136939 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195703441 0 0) 0x7f4a480ac0e0 con 0x7f4a48151960 2020-11-20T12:19:40.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.676+0000 7f4a46ffd700 1 -- 172.21.15.116:0/1830136939 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a48155f20 con 0x7f4a48151580 2020-11-20T12:19:40.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a46ffd700 1 --2- 172.21.15.116:0/1830136939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a48151580 0x7f4a48155340 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f4a480815a0 tx=0x7f4a38006df0).ready entity=mon.1 client_cookie=4b56396b664fef73 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:40.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a45ffb700 1 -- 172.21.15.116:0/1830136939 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a380097b0 con 0x7f4a48151580 2020-11-20T12:19:40.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a45ffb700 1 -- 172.21.15.116:0/1830136939 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a38009910 con 0x7f4a48151580 2020-11-20T12:19:40.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a45ffb700 1 -- 172.21.15.116:0/1830136939 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a38009c90 con 0x7f4a48151580 2020-11-20T12:19:40.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1830136939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a48151580 msgr2=0x7f4a48155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a4d56d700 1 --2- 172.21.15.116:0/1830136939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a48151580 0x7f4a48155340 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f4a480815a0 tx=0x7f4a38006df0).stop 2020-11-20T12:19:40.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1830136939 shutdown_connections 2020-11-20T12:19:40.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a4d56d700 1 --2- 172.21.15.116:0/1830136939 >> v2:172.21.15.116:3300/0 conn(0x7f4a480c8dc0 0x7f4a48061590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a4d56d700 1 --2- 172.21.15.116:0/1830136939 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a48151580 0x7f4a48155340 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1830136939 >> 172.21.15.116:0/1830136939 conn(0x7f4a48068330 msgr2=0x7f4a480c86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:40.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1830136939 shutdown_connections 2020-11-20T12:19:40.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.680+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1830136939 wait complete. 2020-11-20T12:19:40.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.684+0000 7f4a4d56d700 1 Processor -- start 2020-11-20T12:19:40.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.684+0000 7f4a4d56d700 1 -- start start 2020-11-20T12:19:40.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.684+0000 7f4a4d56d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a480c8dc0 0x7f4a4814b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.684+0000 7f4a4d56d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a48151580 0x7f4a480acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.684+0000 7f4a4d56d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a48151960 0x7f4a4813b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.684+0000 7f4a4d56d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48151da0 con 0x7f4a48151580 2020-11-20T12:19:40.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.684+0000 7f4a4d56d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48151fd0 con 0x7f4a480c8dc0 2020-11-20T12:19:40.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.684+0000 7f4a4d56d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a48152270 con 0x7f4a48151960 2020-11-20T12:19:40.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a467fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a48151580 0x7f4a480acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a467fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a48151580 0x7f4a480acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48252/0 (socket says 172.21.15.116:48252) 2020-11-20T12:19:40.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a467fc700 1 -- 172.21.15.116:0/1976747834 learned_addr learned my addr 172.21.15.116:0/1976747834 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:40.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a46ffd700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a480c8dc0 0x7f4a4814b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a477fe700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a48151960 0x7f4a4813b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a467fc700 1 -- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a48151960 msgr2=0x7f4a4813b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a467fc700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a48151960 0x7f4a4813b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a467fc700 1 -- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a480c8dc0 msgr2=0x7f4a4814b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:40.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a467fc700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a480c8dc0 0x7f4a4814b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:40.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a467fc700 1 -- 172.21.15.116:0/1976747834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a48152510 con 0x7f4a48151580 2020-11-20T12:19:40.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a467fc700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a48151580 0x7f4a480acfc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f4a3c00c730 tx=0x7f4a3c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:40.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a2effd700 1 -- 172.21.15.116:0/1976747834 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c007020 con 0x7f4a48151580 2020-11-20T12:19:40.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a2effd700 1 -- 172.21.15.116:0/1976747834 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c007180 con 0x7f4a48151580 2020-11-20T12:19:40.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1976747834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a481527b0 con 0x7f4a48151580 2020-11-20T12:19:40.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a2effd700 1 -- 172.21.15.116:0/1976747834 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0064f0 con 0x7f4a48151580 2020-11-20T12:19:40.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.688+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1976747834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a4814be30 con 0x7f4a48151580 2020-11-20T12:19:40.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.692+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1976747834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a4805cf30 con 0x7f4a48151580 2020-11-20T12:19:40.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.692+0000 7f4a2effd700 1 -- 172.21.15.116:0/1976747834 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4a3c010030 con 0x7f4a48151580 2020-11-20T12:19:40.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.692+0000 7f4a2effd700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a240557c0 0x7f4a24057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:40.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.692+0000 7f4a2effd700 1 -- 172.21.15.116:0/1976747834 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f4a3c06df80 con 0x7f4a48151580 2020-11-20T12:19:40.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.692+0000 7f4a46ffd700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a240557c0 0x7f4a24057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:40.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.692+0000 7f4a46ffd700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a240557c0 0x7f4a24057c80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f4a3800b150 tx=0x7f4a38015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:40.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.696+0000 7f4a2effd700 1 -- 172.21.15.116:0/1976747834 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4a3c043ca0 con 0x7f4a48151580 2020-11-20T12:19:40.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:40.908+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1976747834 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_delete_object"]} v 0) v1 -- 0x7f4a48150d50 con 0x7f4a48151580 2020-11-20T12:19:41.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.480+0000 7f4a2effd700 1 -- 172.21.15.116:0/1976747834 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_delete_object"]}]=0 v217) v1 ==== 149+0+0 (secure 0 0 0) 0x7f4a3c0472c0 con 0x7f4a48151580 2020-11-20T12:19:41.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a240557c0 msgr2=0x7f4a24057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:41.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a240557c0 0x7f4a24057c80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f4a3800b150 tx=0x7f4a38015000).stop 2020-11-20T12:19:41.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1976747834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a48151580 msgr2=0x7f4a480acfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:41.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a48151580 0x7f4a480acfc0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f4a3c00c730 tx=0x7f4a3c007c50).stop 2020-11-20T12:19:41.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1976747834 shutdown_connections 2020-11-20T12:19:41.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4a240557c0 0x7f4a24057c80 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a48151960 0x7f4a4813b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4a480c8dc0 0x7f4a4814b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 --2- 172.21.15.116:0/1976747834 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a48151580 0x7f4a480acfc0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1976747834 >> 172.21.15.116:0/1976747834 conn(0x7f4a48068330 msgr2=0x7f4a480aa020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:41.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1976747834 shutdown_connections 2020-11-20T12:19:41.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.484+0000 7f4a4d56d700 1 -- 172.21.15.116:0/1976747834 wait complete. 2020-11-20T12:19:41.506 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-20T12:19:41.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.692+0000 7f76a562c700 1 Processor -- start 2020-11-20T12:19:41.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.692+0000 7f76a562c700 1 -- start start 2020-11-20T12:19:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f76a562c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f76a013b0d0 0x7f76a00a8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f76a562c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76a0151ee0 0x7f76a0152370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f76a562c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a01557f0 con 0x7f76a013b4b0 2020-11-20T12:19:41.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f76a562c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76a0152b10 con 0x7f76a013b0d0 2020-11-20T12:19:41.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f76a562c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0152db0 con 0x7f76a0151ee0 2020-11-20T12:19:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f769e7fc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f76a013b4b0 0x7f76a00aa7e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57694/0 (socket says 172.21.15.116:57694) 2020-11-20T12:19:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f769e7fc700 1 -- 172.21.15.116:0/2507710325 learned_addr learned my addr 172.21.15.116:0/2507710325 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f769effd700 1 --2- 172.21.15.116:0/2507710325 >> v2:172.21.15.173:3300/0 conn(0x7f76a013b0d0 0x7f76a00a8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f769f7fe700 1 --2- 172.21.15.116:0/2507710325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76a0151ee0 0x7f76a0152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:41.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f769effd700 1 -- 172.21.15.116:0/2507710325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76a0151ee0 msgr2=0x7f76a0152370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f769effd700 1 --2- 172.21.15.116:0/2507710325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76a0151ee0 0x7f76a0152370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f769effd700 1 -- 172.21.15.116:0/2507710325 >> v1:172.21.15.116:6789/0 conn(0x7f76a013b4b0 legacy=0x7f76a00aa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f769effd700 1 -- 172.21.15.116:0/2507710325 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00ad170 con 0x7f76a013b0d0 2020-11-20T12:19:41.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f769f7fe700 1 --2- 172.21.15.116:0/2507710325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76a0151ee0 0x7f76a0152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:41.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.696+0000 7f769effd700 1 --2- 172.21.15.116:0/2507710325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76a013b0d0 0x7f76a00a8420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f769000b9a0 tx=0x7f7690006df0).ready entity=mon.1 client_cookie=d3bd2037fc2b61de server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f769dffb700 1 -- 172.21.15.116:0/2507710325 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76900097b0 con 0x7f76a013b0d0 2020-11-20T12:19:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f769dffb700 1 -- 172.21.15.116:0/2507710325 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7690009910 con 0x7f76a013b0d0 2020-11-20T12:19:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f769dffb700 1 -- 172.21.15.116:0/2507710325 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7690009c90 con 0x7f76a013b0d0 2020-11-20T12:19:41.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f76a562c700 1 -- 172.21.15.116:0/2507710325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76a013b0d0 msgr2=0x7f76a00a8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:41.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f76a562c700 1 --2- 172.21.15.116:0/2507710325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76a013b0d0 0x7f76a00a8420 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f769000b9a0 tx=0x7f7690006df0).stop 2020-11-20T12:19:41.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f76a562c700 1 -- 172.21.15.116:0/2507710325 shutdown_connections 2020-11-20T12:19:41.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f76a562c700 1 --2- 172.21.15.116:0/2507710325 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76a013b0d0 0x7f76a00a8420 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f76a562c700 1 --2- 172.21.15.116:0/2507710325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76a0151ee0 0x7f76a0152370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f76a562c700 1 -- 172.21.15.116:0/2507710325 >> 172.21.15.116:0/2507710325 conn(0x7f76a00678a0 msgr2=0x7f76a0067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:41.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f76a562c700 1 -- 172.21.15.116:0/2507710325 shutdown_connections 2020-11-20T12:19:41.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f76a562c700 1 -- 172.21.15.116:0/2507710325 wait complete. 2020-11-20T12:19:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f76a562c700 1 Processor -- start 2020-11-20T12:19:41.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.700+0000 7f76a562c700 1 -- start start 2020-11-20T12:19:41.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f76a562c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76a013b0d0 0x7f76a00a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:41.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f76a562c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76a013b4b0 0x7f76a00a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:41.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f76a562c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76a0151ee0 0x7f76a0151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:41.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f76a562c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a0164710 con 0x7f76a0151ee0 2020-11-20T12:19:41.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f76a562c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00a0f30 con 0x7f76a013b0d0 2020-11-20T12:19:41.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f76a562c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a00a11d0 con 0x7f76a013b4b0 2020-11-20T12:19:41.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f769f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76a0151ee0 0x7f76a0151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:41.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f769f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76a0151ee0 0x7f76a0151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48268/0 (socket says 172.21.15.116:48268) 2020-11-20T12:19:41.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f769f7fe700 1 -- 172.21.15.116:0/471981323 learned_addr learned my addr 172.21.15.116:0/471981323 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:41.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f769effd700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76a013b0d0 0x7f76a00a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:41.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f769e7fc700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76a013b4b0 0x7f76a00a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:41.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f769f7fe700 1 -- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76a013b4b0 msgr2=0x7f76a00a87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:41.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f769f7fe700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76a013b4b0 0x7f76a00a87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f769f7fe700 1 -- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76a013b0d0 msgr2=0x7f76a00a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:41.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f769f7fe700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76a013b0d0 0x7f76a00a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f769f7fe700 1 -- 172.21.15.116:0/471981323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a00a1470 con 0x7f76a0151ee0 2020-11-20T12:19:41.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f769f7fe700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76a0151ee0 0x7f76a0151180 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f769800db60 tx=0x7f7698008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:41.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f7686ffd700 1 -- 172.21.15.116:0/471981323 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769800b870 con 0x7f76a0151ee0 2020-11-20T12:19:41.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f7686ffd700 1 -- 172.21.15.116:0/471981323 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769800b9d0 con 0x7f76a0151ee0 2020-11-20T12:19:41.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f76a562c700 1 -- 172.21.15.116:0/471981323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a00a0c70 con 0x7f76a0151ee0 2020-11-20T12:19:41.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f7686ffd700 1 -- 172.21.15.116:0/471981323 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769800bd70 con 0x7f76a0151ee0 2020-11-20T12:19:41.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.704+0000 7f76a562c700 1 -- 172.21.15.116:0/471981323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a00a8cb0 con 0x7f76a0151ee0 2020-11-20T12:19:41.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.708+0000 7f76a562c700 1 -- 172.21.15.116:0/471981323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a00a1670 con 0x7f76a0151ee0 2020-11-20T12:19:41.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.708+0000 7f7686ffd700 1 -- 172.21.15.116:0/471981323 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7698012630 con 0x7f76a0151ee0 2020-11-20T12:19:41.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.708+0000 7f7686ffd700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f767c0557c0 0x7f767c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:41.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.708+0000 7f7686ffd700 1 -- 172.21.15.116:0/471981323 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f769806eac0 con 0x7f76a0151ee0 2020-11-20T12:19:41.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.712+0000 7f769effd700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f767c0557c0 0x7f767c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:41.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.712+0000 7f769effd700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f767c0557c0 0x7f767c057c80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f769000b150 tx=0x7f7690015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:41.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.716+0000 7f7686ffd700 1 -- 172.21.15.116:0/471981323 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f76a00a1670 con 0x7f76a0151ee0 2020-11-20T12:19:41.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.872+0000 7f76a562c700 1 -- 172.21.15.116:0/471981323 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f76a00a1670 con 0x7f76a0151ee0 2020-11-20T12:19:41.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f7686ffd700 1 -- 172.21.15.116:0/471981323 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f76a00a1670 con 0x7f76a0151ee0 2020-11-20T12:19:41.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 -- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f767c0557c0 msgr2=0x7f767c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:41.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f767c0557c0 0x7f767c057c80 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f769000b150 tx=0x7f7690015000).stop 2020-11-20T12:19:41.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 -- 172.21.15.116:0/471981323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76a0151ee0 msgr2=0x7f76a0151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:41.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76a0151ee0 0x7f76a0151180 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f769800db60 tx=0x7f7698008040).stop 2020-11-20T12:19:41.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 -- 172.21.15.116:0/471981323 shutdown_connections 2020-11-20T12:19:41.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f767c0557c0 0x7f767c057c80 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f76a0151ee0 0x7f76a0151180 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f76a013b0d0 0x7f76a00a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 --2- 172.21.15.116:0/471981323 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f76a013b4b0 0x7f76a00a87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:41.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 -- 172.21.15.116:0/471981323 >> 172.21.15.116:0/471981323 conn(0x7f76a00678a0 msgr2=0x7f76a00613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:41.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 -- 172.21.15.116:0/471981323 shutdown_connections 2020-11-20T12:19:41.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:41.884+0000 7f76a562c700 1 -- 172.21.15.116:0/471981323 wait complete. 2020-11-20T12:19:41.907 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:19:41.908 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:19:41.952 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:19:41.995 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:19:41.995 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2020-11-20T12:19:41.996 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:19:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.180+0000 7f3f8359e700 1 Processor -- start 2020-11-20T12:19:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.180+0000 7f3f8359e700 1 -- start start 2020-11-20T12:19:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.184+0000 7f3f8359e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3f8413b380 0x7f3f840acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.184+0000 7f3f8359e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f840ada00 con 0x7f3f8413afa0 2020-11-20T12:19:42.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.184+0000 7f3f8359e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f840adca0 con 0x7f3f840ad220 2020-11-20T12:19:42.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.184+0000 7f3f8359e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f840adf40 con 0x7f3f8413b380 2020-11-20T12:19:42.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.184+0000 7f3f8259c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3f8413afa0 0x7f3f840aa5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57706/0 (socket says 172.21.15.116:57706) 2020-11-20T12:19:42.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.184+0000 7f3f8259c700 1 -- 172.21.15.116:0/904970967 learned_addr learned my addr 172.21.15.116:0/904970967 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:42.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.184+0000 7f3f81d9b700 1 --2- 172.21.15.116:0/904970967 >> v2:172.21.15.116:3300/0 conn(0x7f3f8413b380 0x7f3f840acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.184+0000 7f3f8159a700 1 -- 172.21.15.116:0/904970967 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832882637 0 0) 0x7f3f840ada00 con 0x7f3f8413afa0 2020-11-20T12:19:42.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8159a700 1 -- 172.21.15.116:0/904970967 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f68003240 con 0x7f3f8413afa0 2020-11-20T12:19:42.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8159a700 1 -- 172.21.15.116:0/904970967 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3113412610 0 0) 0x7f3f840adca0 con 0x7f3f840ad220 2020-11-20T12:19:42.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8159a700 1 -- 172.21.15.116:0/904970967 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f840ada00 con 0x7f3f840ad220 2020-11-20T12:19:42.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f81d9b700 1 -- 172.21.15.116:0/904970967 >> v1:172.21.15.173:6789/0 conn(0x7f3f840ad220 legacy=0x7f3f840a8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f81d9b700 1 -- 172.21.15.116:0/904970967 >> v1:172.21.15.116:6789/0 conn(0x7f3f8413afa0 legacy=0x7f3f840aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f81d9b700 1 -- 172.21.15.116:0/904970967 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f840ae1e0 con 0x7f3f8413b380 2020-11-20T12:19:42.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f81d9b700 1 --2- 172.21.15.116:0/904970967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8413b380 0x7f3f840acd30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f3f7800b9a0 tx=0x7f3f78006df0).ready entity=mon.0 client_cookie=48829504c801b1c6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:42.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8159a700 1 -- 172.21.15.116:0/904970967 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78009680 con 0x7f3f8413b380 2020-11-20T12:19:42.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8159a700 1 -- 172.21.15.116:0/904970967 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f780097e0 con 0x7f3f8413b380 2020-11-20T12:19:42.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8159a700 1 -- 172.21.15.116:0/904970967 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78009b60 con 0x7f3f8413b380 2020-11-20T12:19:42.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8359e700 1 -- 172.21.15.116:0/904970967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8413b380 msgr2=0x7f3f840acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8359e700 1 --2- 172.21.15.116:0/904970967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8413b380 0x7f3f840acd30 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f3f7800b9a0 tx=0x7f3f78006df0).stop 2020-11-20T12:19:42.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8359e700 1 -- 172.21.15.116:0/904970967 shutdown_connections 2020-11-20T12:19:42.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8359e700 1 --2- 172.21.15.116:0/904970967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f8413b380 0x7f3f840acd30 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8359e700 1 -- 172.21.15.116:0/904970967 >> 172.21.15.116:0/904970967 conn(0x7f3f840678a0 msgr2=0x7f3f84067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:42.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8359e700 1 -- 172.21.15.116:0/904970967 shutdown_connections 2020-11-20T12:19:42.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.188+0000 7f3f8359e700 1 -- 172.21.15.116:0/904970967 wait complete. 2020-11-20T12:19:42.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8359e700 1 Processor -- start 2020-11-20T12:19:42.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8359e700 1 -- start start 2020-11-20T12:19:42.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f840ad220 0x7f3f84138c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f8413afa0 0x7f3f841615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f8413b380 0x7f3f84164300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f840a4780 con 0x7f3f840ad220 2020-11-20T12:19:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f841651c0 con 0x7f3f8413afa0 2020-11-20T12:19:42.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84164c80 con 0x7f3f8413b380 2020-11-20T12:19:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f840ad220 0x7f3f84138c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f840ad220 0x7f3f84138c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48280/0 (socket says 172.21.15.116:48280) 2020-11-20T12:19:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8259c700 1 -- 172.21.15.116:0/589090402 learned_addr learned my addr 172.21.15.116:0/589090402 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f81d9b700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f8413afa0 0x7f3f841615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f82d9d700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f8413b380 0x7f3f84164300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8259c700 1 -- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f8413b380 msgr2=0x7f3f84164300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8259c700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f8413b380 0x7f3f84164300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8259c700 1 -- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f8413afa0 msgr2=0x7f3f841615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8259c700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f8413afa0 0x7f3f841615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8259c700 1 -- 172.21.15.116:0/589090402 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f84164f20 con 0x7f3f840ad220 2020-11-20T12:19:42.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8259c700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f840ad220 0x7f3f84138c20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f3f7400c760 tx=0x7f3f74007b70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:42.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f727fc700 1 -- 172.21.15.116:0/589090402 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74007320 con 0x7f3f840ad220 2020-11-20T12:19:42.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f727fc700 1 -- 172.21.15.116:0/589090402 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f74007480 con 0x7f3f840ad220 2020-11-20T12:19:42.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f727fc700 1 -- 172.21.15.116:0/589090402 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74005800 con 0x7f3f840ad220 2020-11-20T12:19:42.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.192+0000 7f3f8359e700 1 -- 172.21.15.116:0/589090402 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f84160eb0 con 0x7f3f840ad220 2020-11-20T12:19:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.196+0000 7f3f8359e700 1 -- 172.21.15.116:0/589090402 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f84139110 con 0x7f3f840ad220 2020-11-20T12:19:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.196+0000 7f3f8359e700 1 -- 172.21.15.116:0/589090402 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f84160cf0 con 0x7f3f840ad220 2020-11-20T12:19:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.196+0000 7f3f727fc700 1 -- 172.21.15.116:0/589090402 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3f74010030 con 0x7f3f840ad220 2020-11-20T12:19:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.196+0000 7f3f727fc700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f580557c0 0x7f3f58057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.196+0000 7f3f727fc700 1 -- 172.21.15.116:0/589090402 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f3f7406e610 con 0x7f3f840ad220 2020-11-20T12:19:42.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.200+0000 7f3f81d9b700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f580557c0 0x7f3f58057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.200+0000 7f3f81d9b700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f580557c0 0x7f3f58057c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f3f8413b810 tx=0x7f3f78015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:42.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.204+0000 7f3f727fc700 1 -- 172.21.15.116:0/589090402 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3f74044330 con 0x7f3f840ad220 2020-11-20T12:19:42.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.412+0000 7f3f8359e700 1 -- 172.21.15.116:0/589090402 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f8405cf30 con 0x7f3f840ad220 2020-11-20T12:19:42.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.412+0000 7f3f727fc700 1 -- 172.21.15.116:0/589090402 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2857 (secure 0 0 0) 0x7f3f74047950 con 0x7f3f840ad220 2020-11-20T12:19:42.411 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:42.411 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3869489418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3869489418},{"type":"v1","addr":"172.21.15.173:6837","nonce":3869489418}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/2912073472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2912073472},{"type":"v1","addr":"172.21.15.173:6835","nonce":2912073472}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/2221764249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2221764249},{"type":"v1","addr":"172.21.15.116:6834","nonce":2221764249}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:12.981713+0000","modified":"2020-11-20T12:19:33.421473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1946956517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1946956517},{"type":"v1","addr":"172.21.15.116:6835","nonce":1946956517}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-20T12:19:42.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 -- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f580557c0 msgr2=0x7f3f58057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f580557c0 0x7f3f58057c80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f3f8413b810 tx=0x7f3f78015000).stop 2020-11-20T12:19:42.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 -- 172.21.15.116:0/589090402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f840ad220 msgr2=0x7f3f84138c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f840ad220 0x7f3f84138c20 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f3f7400c760 tx=0x7f3f74007b70).stop 2020-11-20T12:19:42.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 -- 172.21.15.116:0/589090402 shutdown_connections 2020-11-20T12:19:42.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3f580557c0 0x7f3f58057c80 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3f8413b380 0x7f3f84164300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3f840ad220 0x7f3f84138c20 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 --2- 172.21.15.116:0/589090402 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3f8413afa0 0x7f3f841615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 -- 172.21.15.116:0/589090402 >> 172.21.15.116:0/589090402 conn(0x7f3f840678a0 msgr2=0x7f3f84154cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:42.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 -- 172.21.15.116:0/589090402 shutdown_connections 2020-11-20T12:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.416+0000 7f3f8359e700 1 -- 172.21.15.116:0/589090402 wait complete. 2020-11-20T12:19:42.416 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 38 2020-11-20T12:19:42.435 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:42.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f374a173700 1 Processor -- start 2020-11-20T12:19:42.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f374a173700 1 -- start start 2020-11-20T12:19:42.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f374a173700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f37440a8300 0x7f3744062f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f374a173700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f374414e260 0x7f374414e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f374a173700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 0x7f37440a5740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f374a173700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37440a5c30 con 0x7f374414e260 2020-11-20T12:19:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f374a173700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37440a86e0 con 0x7f37440a8300 2020-11-20T12:19:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f374a173700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374414ed60 con 0x7f37440a52d0 2020-11-20T12:19:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f3748970700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f374414e260 0x7f374414e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f3748970700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f374414e260 0x7f374414e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48290/0 (socket says 172.21.15.116:48290) 2020-11-20T12:19:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f3748970700 1 -- 172.21.15.116:0/3627075316 learned_addr learned my addr 172.21.15.116:0/3627075316 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:42.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f3749171700 1 --2- 172.21.15.116:0/3627075316 >> v2:172.21.15.173:3300/0 conn(0x7f37440a8300 0x7f3744062f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f3749972700 1 --2- 172.21.15.116:0/3627075316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 0x7f37440a5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f3748970700 1 -- 172.21.15.116:0/3627075316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 msgr2=0x7f37440a5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:42.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f3748970700 1 --2- 172.21.15.116:0/3627075316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 0x7f37440a5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f3748970700 1 -- 172.21.15.116:0/3627075316 >> v2:172.21.15.173:3300/0 conn(0x7f37440a8300 msgr2=0x7f3744062f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f3748970700 1 --2- 172.21.15.116:0/3627075316 >> v2:172.21.15.173:3300/0 conn(0x7f37440a8300 0x7f3744062f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f3748970700 1 -- 172.21.15.116:0/3627075316 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374414f000 con 0x7f374414e260 2020-11-20T12:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.624+0000 7f3749972700 1 --2- 172.21.15.116:0/3627075316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 0x7f37440a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:42.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f3748970700 1 --2- 172.21.15.116:0/3627075316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f374414e260 0x7f374414e6e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f3744066190 tx=0x7f3734006eb0).ready entity=mon.0 client_cookie=9b2cac73372b5e9a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:42.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f3733fff700 1 -- 172.21.15.116:0/3627075316 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734009810 con 0x7f374414e260 2020-11-20T12:19:42.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f3733fff700 1 -- 172.21.15.116:0/3627075316 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3734009970 con 0x7f374414e260 2020-11-20T12:19:42.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f3733fff700 1 -- 172.21.15.116:0/3627075316 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3734009cf0 con 0x7f374414e260 2020-11-20T12:19:42.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f374a173700 1 -- 172.21.15.116:0/3627075316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f374414e260 msgr2=0x7f374414e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f374a173700 1 --2- 172.21.15.116:0/3627075316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f374414e260 0x7f374414e6e0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f3744066190 tx=0x7f3734006eb0).stop 2020-11-20T12:19:42.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f374a173700 1 -- 172.21.15.116:0/3627075316 shutdown_connections 2020-11-20T12:19:42.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f374a173700 1 --2- 172.21.15.116:0/3627075316 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 0x7f37440a5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f374a173700 1 --2- 172.21.15.116:0/3627075316 >> v2:172.21.15.173:3300/0 conn(0x7f37440a8300 0x7f3744062f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f374a173700 1 --2- 172.21.15.116:0/3627075316 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f374414e260 0x7f374414e6e0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f374a173700 1 -- 172.21.15.116:0/3627075316 >> 172.21.15.116:0/3627075316 conn(0x7f37440c5620 msgr2=0x7f37440c15f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:42.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f374a173700 1 -- 172.21.15.116:0/3627075316 shutdown_connections 2020-11-20T12:19:42.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.628+0000 7f374a173700 1 -- 172.21.15.116:0/3627075316 wait complete. 2020-11-20T12:19:42.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f374a173700 1 Processor -- start 2020-11-20T12:19:42.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f374a173700 1 -- start start 2020-11-20T12:19:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f374a173700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 0x7f374409de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f374a173700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37440a8300 0x7f3744169010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f374a173700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37440a3b50 0x7f374414fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f374a173700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374414f530 con 0x7f37440a3b50 2020-11-20T12:19:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f374a173700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37441696b0 con 0x7f37440a8300 2020-11-20T12:19:42.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f374a173700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3744169950 con 0x7f37440a52d0 2020-11-20T12:19:42.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f3749972700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37440a3b50 0x7f374414fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f3749972700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37440a3b50 0x7f374414fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48296/0 (socket says 172.21.15.116:48296) 2020-11-20T12:19:42.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f3749972700 1 -- 172.21.15.116:0/3231997236 learned_addr learned my addr 172.21.15.116:0/3231997236 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f3748970700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37440a8300 0x7f3744169010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.632+0000 7f3749171700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 0x7f374409de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f3749972700 1 -- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 msgr2=0x7f374409de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f3749972700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 0x7f374409de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f3749972700 1 -- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37440a8300 msgr2=0x7f3744169010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f3749972700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37440a8300 0x7f3744169010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f3749972700 1 -- 172.21.15.116:0/3231997236 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3744169bf0 con 0x7f37440a3b50 2020-11-20T12:19:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f3748970700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37440a8300 0x7f3744169010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f3749171700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 0x7f374409de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f3749972700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37440a3b50 0x7f374414fa20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f373c00db60 tx=0x7f373c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f37317fa700 1 -- 172.21.15.116:0/3231997236 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c00b9a0 con 0x7f37440a3b50 2020-11-20T12:19:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f37317fa700 1 -- 172.21.15.116:0/3231997236 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f373c00bb00 con 0x7f37440a3b50 2020-11-20T12:19:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f374a173700 1 -- 172.21.15.116:0/3231997236 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3744169e90 con 0x7f37440a3b50 2020-11-20T12:19:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f37317fa700 1 -- 172.21.15.116:0/3231997236 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f373c00bea0 con 0x7f37440a3b50 2020-11-20T12:19:42.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f374a173700 1 -- 172.21.15.116:0/3231997236 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37440a3f70 con 0x7f37440a3b50 2020-11-20T12:19:42.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.636+0000 7f374a173700 1 -- 172.21.15.116:0/3231997236 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37440a41f0 con 0x7f37440a3b50 2020-11-20T12:19:42.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.640+0000 7f37317fa700 1 -- 172.21.15.116:0/3231997236 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f373c012630 con 0x7f37440a3b50 2020-11-20T12:19:42.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.640+0000 7f37317fa700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 0x7f371c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:42.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.640+0000 7f37317fa700 1 -- 172.21.15.116:0/3231997236 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f373c06ee50 con 0x7f37440a3b50 2020-11-20T12:19:42.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.640+0000 7f3749171700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 0x7f371c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:42.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.640+0000 7f3749171700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 0x7f371c057c80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f37440c3990 tx=0x7f3740008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:42.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.644+0000 7f37317fa700 1 -- 172.21.15.116:0/3231997236 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f373c044b70 con 0x7f37440a3b50 2020-11-20T12:19:42.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.804+0000 7f374a173700 1 -- 172.21.15.116:0/3231997236 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f374405cf30 con 0x7f37440a3b50 2020-11-20T12:19:42.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.804+0000 7f37317fa700 1 -- 172.21.15.116:0/3231997236 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18428 (secure 0 0 0) 0x7f373c0102e0 con 0x7f37440a3b50 2020-11-20T12:19:42.802 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:42.802 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":74,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:34.425107+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:11.306087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:11.966077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":14,"pool_name":"fsvolume_volid","create_time":"2020-11-20T12:19:32.859361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:42.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.804+0000 7f374a173700 1 -- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 msgr2=0x7f371c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.804+0000 7f374a173700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 0x7f371c057c80 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f37440c3990 tx=0x7f3740008040).stop 2020-11-20T12:19:42.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.804+0000 7f374a173700 1 -- 172.21.15.116:0/3231997236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37440a3b50 msgr2=0x7f374414fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:42.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.808+0000 7f374a173700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37440a3b50 0x7f374414fa20 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f373c00db60 tx=0x7f373c008040).stop 2020-11-20T12:19:42.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.808+0000 7f374a173700 1 -- 172.21.15.116:0/3231997236 shutdown_connections 2020-11-20T12:19:42.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.808+0000 7f374a173700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 0x7f371c057c80 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.808+0000 7f374a173700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37440a3b50 0x7f374414fa20 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.808+0000 7f374a173700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37440a52d0 0x7f374409de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.808+0000 7f374a173700 1 --2- 172.21.15.116:0/3231997236 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f37440a8300 0x7f3744169010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.808+0000 7f374a173700 1 -- 172.21.15.116:0/3231997236 >> 172.21.15.116:0/3231997236 conn(0x7f37440c5620 msgr2=0x7f3744063f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.808+0000 7f374a173700 1 -- 172.21.15.116:0/3231997236 shutdown_connections 2020-11-20T12:19:42.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:42.808+0000 7f374a173700 1 -- 172.21.15.116:0/3231997236 wait complete. 2020-11-20T12:19:42.870 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:19:11.306087+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '70', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:19:11.966077+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}, {'pool': 14, 'pool_name': 'fsvolume_volid', 'create_time': '2020-11-20T12:19:32.859361+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '74', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:19:42.871 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:19:43.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.048+0000 7f8eea1b0700 1 Processor -- start 2020-11-20T12:19:43.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.048+0000 7f8eea1b0700 1 -- start start 2020-11-20T12:19:43.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8eea1b0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8ee40aa7d0 0x7f8ee40a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8eea1b0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40ad540 0x7f8ee40ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8eea1b0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee40ace40 con 0x7f8ee40aabb0 2020-11-20T12:19:43.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8eea1b0700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee41526c0 con 0x7f8ee40aa7d0 2020-11-20T12:19:43.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8eea1b0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee4152960 con 0x7f8ee40ad540 2020-11-20T12:19:43.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ee89ad700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8ee40aabb0 0x7f8ee4151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57736/0 (socket says 172.21.15.116:57736) 2020-11-20T12:19:43.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ee89ad700 1 -- 172.21.15.116:0/3306197680 learned_addr learned my addr 172.21.15.116:0/3306197680 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:43.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ee91ae700 1 --2- 172.21.15.116:0/3306197680 >> v2:172.21.15.116:3300/0 conn(0x7f8ee40aa7d0 0x7f8ee40a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ee99af700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40ad540 0x7f8ee40ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1005970248 0 0) 0x7f8ee40ace40 con 0x7f8ee40aabb0 2020-11-20T12:19:43.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ecc003240 con 0x7f8ee40aabb0 2020-11-20T12:19:43.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3696253787 0 0) 0x7f8ecc003240 con 0x7f8ee40aabb0 2020-11-20T12:19:43.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40ad540 msgr2=0x7f8ee40ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:43.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40ad540 0x7f8ee40ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 >> v2:172.21.15.116:3300/0 conn(0x7f8ee40aa7d0 msgr2=0x7f8ee40a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:43.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 --2- 172.21.15.116:0/3306197680 >> v2:172.21.15.116:3300/0 conn(0x7f8ee40aa7d0 0x7f8ee40a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee4152c00 con 0x7f8ee40aabb0 2020-11-20T12:19:43.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f8ed40014e0 con 0x7f8ee40aabb0 2020-11-20T12:19:43.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 >> v1:172.21.15.116:6789/0 conn(0x7f8ee40aabb0 legacy=0x7f8ee4151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40ad540 0x7f8ecc003750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ecc003c60 0x7f8ecc0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ecc0049d0 0x7f8ecc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee41526c0 con 0x7f8ecc0049d0 2020-11-20T12:19:43.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee4152960 con 0x7f8ecc003c60 2020-11-20T12:19:43.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc0042d0 con 0x7f8ee40ad540 2020-11-20T12:19:43.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.052+0000 7f8ee99af700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ecc0049d0 0x7f8ecc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ee89ad700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ecc003c60 0x7f8ecc0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ee99af700 1 -- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40ad540 msgr2=0x7f8ecc003750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:43.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ee91ae700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40ad540 0x7f8ecc003750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ee99af700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40ad540 0x7f8ecc003750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ee99af700 1 -- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ecc003c60 msgr2=0x7f8ecc0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ee99af700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ecc003c60 0x7f8ecc0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ee99af700 1 -- 172.21.15.116:0/3306197680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc003240 con 0x7f8ecc0049d0 2020-11-20T12:19:43.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ee89ad700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ecc003c60 0x7f8ecc0044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:43.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ee91ae700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40ad540 0x7f8ecc003750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:43.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ee99af700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ecc0049d0 0x7f8ecc004e50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f8edc009ea0 tx=0x7f8edc0091d0).ready entity=mon.0 client_cookie=fc683c174ce53a56 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc0095c0 con 0x7f8ecc0049d0 2020-11-20T12:19:43.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8edc00eda0 con 0x7f8ecc0049d0 2020-11-20T12:19:43.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ed3fff700 1 -- 172.21.15.116:0/3306197680 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc012600 con 0x7f8ecc0049d0 2020-11-20T12:19:43.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8eea1b0700 1 -- 172.21.15.116:0/3306197680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ecc0049d0 msgr2=0x7f8ecc004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8eea1b0700 1 --2- 172.21.15.116:0/3306197680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ecc0049d0 0x7f8ecc004e50 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7f8edc009ea0 tx=0x7f8edc0091d0).stop 2020-11-20T12:19:43.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8ee99af700 1 -- 172.21.15.116:0/3306197680 reap_dead start 2020-11-20T12:19:43.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8eea1b0700 1 -- 172.21.15.116:0/3306197680 shutdown_connections 2020-11-20T12:19:43.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8eea1b0700 1 -- 172.21.15.116:0/3306197680 >> 172.21.15.116:0/3306197680 conn(0x7f8ee40678a0 msgr2=0x7f8ee4067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:43.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8eea1b0700 1 -- 172.21.15.116:0/3306197680 shutdown_connections 2020-11-20T12:19:43.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.056+0000 7f8eea1b0700 1 -- 172.21.15.116:0/3306197680 wait complete. 2020-11-20T12:19:43.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8eea1b0700 1 Processor -- start 2020-11-20T12:19:43.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8eea1b0700 1 -- start start 2020-11-20T12:19:43.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8eea1b0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee4161770 0x7f8ee40619c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8eea1b0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4161b50 0x7f8ee4061eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8eea1b0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40a83a0 0x7f8ee40a8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8eea1b0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee40a4960 con 0x7f8ee4161b50 2020-11-20T12:19:43.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8eea1b0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee40a4c00 con 0x7f8ee4161770 2020-11-20T12:19:43.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8eea1b0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee40a4ea0 con 0x7f8ee40a83a0 2020-11-20T12:19:43.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee89ad700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4161b50 0x7f8ee4061eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee89ad700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4161b50 0x7f8ee4061eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48312/0 (socket says 172.21.15.116:48312) 2020-11-20T12:19:43.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee89ad700 1 -- 172.21.15.116:0/4182512412 learned_addr learned my addr 172.21.15.116:0/4182512412 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:43.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee99af700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40a83a0 0x7f8ee40a8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee91ae700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee4161770 0x7f8ee40619c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee89ad700 1 -- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40a83a0 msgr2=0x7f8ee40a8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee89ad700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40a83a0 0x7f8ee40a8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee89ad700 1 -- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee4161770 msgr2=0x7f8ee40619c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee89ad700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee4161770 0x7f8ee40619c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee89ad700 1 -- 172.21.15.116:0/4182512412 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee40a5140 con 0x7f8ee4161b50 2020-11-20T12:19:43.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee99af700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40a83a0 0x7f8ee40a8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:19:43.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ee89ad700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4161b50 0x7f8ee4061eb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f8ed40010a0 tx=0x7f8ee40acd90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ed17fa700 1 -- 172.21.15.116:0/4182512412 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400e860 con 0x7f8ee4161b50 2020-11-20T12:19:43.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ed17fa700 1 -- 172.21.15.116:0/4182512412 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed4010040 con 0x7f8ee4161b50 2020-11-20T12:19:43.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8ed17fa700 1 -- 172.21.15.116:0/4182512412 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400ebd0 con 0x7f8ee4161b50 2020-11-20T12:19:43.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.060+0000 7f8eea1b0700 1 -- 172.21.15.116:0/4182512412 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee40a53e0 con 0x7f8ee4161b50 2020-11-20T12:19:43.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.064+0000 7f8eea1b0700 1 -- 172.21.15.116:0/4182512412 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee40a4570 con 0x7f8ee4161b50 2020-11-20T12:19:43.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.064+0000 7f8eea1b0700 1 -- 172.21.15.116:0/4182512412 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee40c7e00 con 0x7f8ee4161b50 2020-11-20T12:19:43.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.064+0000 7f8ed17fa700 1 -- 172.21.15.116:0/4182512412 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8ed4015400 con 0x7f8ee4161b50 2020-11-20T12:19:43.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.064+0000 7f8ed17fa700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ebc0557c0 0x7f8ebc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.064+0000 7f8ed17fa700 1 -- 172.21.15.116:0/4182512412 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f8ed406e860 con 0x7f8ee4161b50 2020-11-20T12:19:43.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.068+0000 7f8ee91ae700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ebc0557c0 0x7f8ebc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.068+0000 7f8ee91ae700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ebc0557c0 0x7f8ebc057c80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f8ee0006fd0 tx=0x7f8ee0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.072+0000 7f8ed17fa700 1 -- 172.21.15.116:0/4182512412 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8ee40c7e00 con 0x7f8ee4161b50 2020-11-20T12:19:43.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.228+0000 7f8eea1b0700 1 -- 172.21.15.116:0/4182512412 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f8ee40c7e00 con 0x7f8ee4161b50 2020-11-20T12:19:43.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.228+0000 7f8ed17fa700 1 -- 172.21.15.116:0/4182512412 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v74) v1 ==== 110+0+10 (secure 0 0 0) 0x7f8ee40c7e00 con 0x7f8ee4161b50 2020-11-20T12:19:43.225 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:19:43.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.228+0000 7f8eea1b0700 1 -- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ebc0557c0 msgr2=0x7f8ebc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.228+0000 7f8eea1b0700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ebc0557c0 0x7f8ebc057c80 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f8ee0006fd0 tx=0x7f8ee0008040).stop 2020-11-20T12:19:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.228+0000 7f8eea1b0700 1 -- 172.21.15.116:0/4182512412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4161b50 msgr2=0x7f8ee4061eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.228+0000 7f8eea1b0700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4161b50 0x7f8ee4061eb0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f8ed40010a0 tx=0x7f8ee40acd90).stop 2020-11-20T12:19:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.232+0000 7f8eea1b0700 1 -- 172.21.15.116:0/4182512412 shutdown_connections 2020-11-20T12:19:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.232+0000 7f8eea1b0700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8ebc0557c0 0x7f8ebc057c80 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.232+0000 7f8eea1b0700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ee40a83a0 0x7f8ee40a8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.232+0000 7f8eea1b0700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ee4161770 0x7f8ee40619c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.232+0000 7f8eea1b0700 1 --2- 172.21.15.116:0/4182512412 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ee4161b50 0x7f8ee4061eb0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.232+0000 7f8eea1b0700 1 -- 172.21.15.116:0/4182512412 >> 172.21.15.116:0/4182512412 conn(0x7f8ee40678a0 msgr2=0x7f8ee40c76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.232+0000 7f8eea1b0700 1 -- 172.21.15.116:0/4182512412 shutdown_connections 2020-11-20T12:19:43.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.232+0000 7f8eea1b0700 1 -- 172.21.15.116:0/4182512412 wait complete. 2020-11-20T12:19:43.250 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:19:43.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.428+0000 7f225418d700 1 Processor -- start 2020-11-20T12:19:43.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.428+0000 7f225418d700 1 -- start start 2020-11-20T12:19:43.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f225418d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f224c0606f0 0x7f224c0aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f225418d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224c0a67e0 con 0x7f224c066d90 2020-11-20T12:19:43.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f225418d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f224c0a5fd0 con 0x7f224c060310 2020-11-20T12:19:43.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f225418d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f224c0a6270 con 0x7f224c0606f0 2020-11-20T12:19:43.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f225272a700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f224c066d90 0x7f224c159af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57758/0 (socket says 172.21.15.116:57758) 2020-11-20T12:19:43.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f225272a700 1 -- 172.21.15.116:0/349269362 learned_addr learned my addr 172.21.15.116:0/349269362 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:43.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f2251728700 1 --2- 172.21.15.116:0/349269362 >> v2:172.21.15.116:3300/0 conn(0x7f224c0606f0 0x7f224c0aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f2250f27700 1 -- 172.21.15.116:0/349269362 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 201972789 0 0) 0x7f224c0a67e0 con 0x7f224c066d90 2020-11-20T12:19:43.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f2250f27700 1 -- 172.21.15.116:0/349269362 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2234003240 con 0x7f224c066d90 2020-11-20T12:19:43.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f2250f27700 1 -- 172.21.15.116:0/349269362 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3672227033 0 0) 0x7f224c0a5fd0 con 0x7f224c060310 2020-11-20T12:19:43.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f2251728700 1 -- 172.21.15.116:0/349269362 >> v1:172.21.15.173:6789/0 conn(0x7f224c060310 legacy=0x7f224c066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f2251728700 1 -- 172.21.15.116:0/349269362 >> v1:172.21.15.116:6789/0 conn(0x7f224c066d90 legacy=0x7f224c159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f2251728700 1 -- 172.21.15.116:0/349269362 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c0a6a40 con 0x7f224c0606f0 2020-11-20T12:19:43.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f2251728700 1 --2- 172.21.15.116:0/349269362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f224c0606f0 0x7f224c0aa0d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f223c00b9a0 tx=0x7f223c006df0).ready entity=mon.0 client_cookie=324dae1f4a8f2a81 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f2250f27700 1 -- 172.21.15.116:0/349269362 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c0097b0 con 0x7f224c0606f0 2020-11-20T12:19:43.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f2250f27700 1 -- 172.21.15.116:0/349269362 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f223c009910 con 0x7f224c0606f0 2020-11-20T12:19:43.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f2250f27700 1 -- 172.21.15.116:0/349269362 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f223c009c90 con 0x7f224c0606f0 2020-11-20T12:19:43.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f225418d700 1 -- 172.21.15.116:0/349269362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f224c0606f0 msgr2=0x7f224c0aa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.432+0000 7f225418d700 1 --2- 172.21.15.116:0/349269362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f224c0606f0 0x7f224c0aa0d0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f223c00b9a0 tx=0x7f223c006df0).stop 2020-11-20T12:19:43.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.436+0000 7f225418d700 1 -- 172.21.15.116:0/349269362 shutdown_connections 2020-11-20T12:19:43.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.436+0000 7f225418d700 1 --2- 172.21.15.116:0/349269362 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f224c0606f0 0x7f224c0aa0d0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.436+0000 7f225418d700 1 -- 172.21.15.116:0/349269362 >> 172.21.15.116:0/349269362 conn(0x7f224c05ba80 msgr2=0x7f224c0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:43.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.436+0000 7f225418d700 1 -- 172.21.15.116:0/349269362 shutdown_connections 2020-11-20T12:19:43.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.436+0000 7f225418d700 1 -- 172.21.15.116:0/349269362 wait complete. 2020-11-20T12:19:43.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.440+0000 7f225418d700 1 Processor -- start 2020-11-20T12:19:43.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.440+0000 7f225418d700 1 -- start start 2020-11-20T12:19:43.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.440+0000 7f225418d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f224c060310 0x7f224c15b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.440+0000 7f225418d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f224c0606f0 0x7f224c0bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.440+0000 7f225418d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f224c066d90 0x7f224c0c0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.440+0000 7f225418d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c169960 con 0x7f224c066d90 2020-11-20T12:19:43.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.440+0000 7f225418d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f224c165560 con 0x7f224c060310 2020-11-20T12:19:43.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.440+0000 7f225418d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224c165800 con 0x7f224c0606f0 2020-11-20T12:19:43.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225272a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f224c066d90 0x7f224c0c0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225272a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f224c066d90 0x7f224c0c0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48330/0 (socket says 172.21.15.116:48330) 2020-11-20T12:19:43.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225272a700 1 -- 172.21.15.116:0/2782393745 learned_addr learned my addr 172.21.15.116:0/2782393745 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:43.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f2251728700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f224c0606f0 0x7f224c0bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f2251f29700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f224c060310 0x7f224c15b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225272a700 1 -- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f224c0606f0 msgr2=0x7f224c0bfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225272a700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f224c0606f0 0x7f224c0bfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225272a700 1 -- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f224c060310 msgr2=0x7f224c15b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225272a700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f224c060310 0x7f224c15b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225272a700 1 -- 172.21.15.116:0/2782393745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c165aa0 con 0x7f224c066d90 2020-11-20T12:19:43.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f2251f29700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f224c060310 0x7f224c15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:43.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f2251728700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f224c0606f0 0x7f224c0bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:43.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225272a700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f224c066d90 0x7f224c0c0160 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f2244001420 tx=0x7f2244007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f2239ffb700 1 -- 172.21.15.116:0/2782393745 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244005b80 con 0x7f224c066d90 2020-11-20T12:19:43.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f2239ffb700 1 -- 172.21.15.116:0/2782393745 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2244005ce0 con 0x7f224c066d90 2020-11-20T12:19:43.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225418d700 1 -- 172.21.15.116:0/2782393745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f224c165d40 con 0x7f224c066d90 2020-11-20T12:19:43.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f2239ffb700 1 -- 172.21.15.116:0/2782393745 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244007320 con 0x7f224c066d90 2020-11-20T12:19:43.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225418d700 1 -- 172.21.15.116:0/2782393745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224c1661e0 con 0x7f224c066d90 2020-11-20T12:19:43.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.444+0000 7f225418d700 1 -- 172.21.15.116:0/2782393745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f224c05d2a0 con 0x7f224c066d90 2020-11-20T12:19:43.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.448+0000 7f2239ffb700 1 -- 172.21.15.116:0/2782393745 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2244010030 con 0x7f224c066d90 2020-11-20T12:19:43.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.448+0000 7f2239ffb700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22240557c0 0x7f2224057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.448+0000 7f2239ffb700 1 -- 172.21.15.116:0/2782393745 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f224406e750 con 0x7f224c066d90 2020-11-20T12:19:43.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.448+0000 7f2251f29700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22240557c0 0x7f2224057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.448+0000 7f2251f29700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22240557c0 0x7f2224057c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f22480011a0 tx=0x7f2248009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.452+0000 7f2239ffb700 1 -- 172.21.15.116:0/2782393745 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2244044470 con 0x7f224c066d90 2020-11-20T12:19:43.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.608+0000 7f225418d700 1 -- 172.21.15.116:0/2782393745 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f224c165ed0 con 0x7f224c066d90 2020-11-20T12:19:43.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.612+0000 7f2239ffb700 1 -- 172.21.15.116:0/2782393745 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v74) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2244047a90 con 0x7f224c066d90 2020-11-20T12:19:43.609 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:43.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.612+0000 7f225418d700 1 -- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22240557c0 msgr2=0x7f2224057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.612+0000 7f225418d700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22240557c0 0x7f2224057c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f22480011a0 tx=0x7f2248009040).stop 2020-11-20T12:19:43.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.612+0000 7f225418d700 1 -- 172.21.15.116:0/2782393745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f224c066d90 msgr2=0x7f224c0c0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.612+0000 7f225418d700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f224c066d90 0x7f224c0c0160 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7f2244001420 tx=0x7f2244007e20).stop 2020-11-20T12:19:43.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.612+0000 7f225418d700 1 -- 172.21.15.116:0/2782393745 shutdown_connections 2020-11-20T12:19:43.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.612+0000 7f225418d700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f22240557c0 0x7f2224057c80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.612+0000 7f225418d700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f224c066d90 0x7f224c0c0160 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.612+0000 7f225418d700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f224c060310 0x7f224c15b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.612+0000 7f225418d700 1 --2- 172.21.15.116:0/2782393745 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f224c0606f0 0x7f224c0bfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.612+0000 7f225418d700 1 -- 172.21.15.116:0/2782393745 >> 172.21.15.116:0/2782393745 conn(0x7f224c05ba80 msgr2=0x7f224c0c0e60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:43.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.616+0000 7f225418d700 1 -- 172.21.15.116:0/2782393745 shutdown_connections 2020-11-20T12:19:43.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.616+0000 7f225418d700 1 -- 172.21.15.116:0/2782393745 wait complete. 2020-11-20T12:19:43.636 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:19:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.828+0000 7f7e84223700 1 Processor -- start 2020-11-20T12:19:43.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.828+0000 7f7e84223700 1 -- start start 2020-11-20T12:19:43.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e84223700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e7c0606f0 0x7f7e7c0aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e84223700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7e7c066d90 0x7f7e7c159af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e84223700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e7c0a67e0 con 0x7f7e7c060310 2020-11-20T12:19:43.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e84223700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e7c0a6270 con 0x7f7e7c066d90 2020-11-20T12:19:43.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e84223700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c0a5fd0 con 0x7f7e7c0606f0 2020-11-20T12:19:43.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e827c0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7e7c066d90 0x7f7e7c159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e817be700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e7c0606f0 0x7f7e7c0aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e81fbf700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f7e7c060310 0x7f7e7c066680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:53998/0 (socket says 172.21.15.116:53998) 2020-11-20T12:19:43.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e81fbf700 1 -- 172.21.15.116:0/4122466746 learned_addr learned my addr 172.21.15.116:0/4122466746 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:43.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e80fbd700 1 -- 172.21.15.116:0/4122466746 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3941517960 0 0) 0x7f7e7c0a67e0 con 0x7f7e7c060310 2020-11-20T12:19:43.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e80fbd700 1 -- 172.21.15.116:0/4122466746 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e64003240 con 0x7f7e7c060310 2020-11-20T12:19:43.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e817be700 1 -- 172.21.15.116:0/4122466746 >> v2:172.21.15.173:3300/0 conn(0x7f7e7c066d90 msgr2=0x7f7e7c159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:43.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e817be700 1 --2- 172.21.15.116:0/4122466746 >> v2:172.21.15.173:3300/0 conn(0x7f7e7c066d90 0x7f7e7c159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e817be700 1 -- 172.21.15.116:0/4122466746 >> v1:172.21.15.173:6789/0 conn(0x7f7e7c060310 legacy=0x7f7e7c066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.832+0000 7f7e817be700 1 -- 172.21.15.116:0/4122466746 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c0a6a40 con 0x7f7e7c0606f0 2020-11-20T12:19:43.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e827c0700 1 --2- 172.21.15.116:0/4122466746 >> v2:172.21.15.173:3300/0 conn(0x7f7e7c066d90 0x7f7e7c159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:43.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e817be700 1 --2- 172.21.15.116:0/4122466746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e7c0606f0 0x7f7e7c0aa0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f7e6c00b9a0 tx=0x7f7e6c006df0).ready entity=mon.2 client_cookie=c5fdf15fadeeffaf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e80fbd700 1 -- 172.21.15.116:0/4122466746 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c009680 con 0x7f7e7c0606f0 2020-11-20T12:19:43.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e80fbd700 1 -- 172.21.15.116:0/4122466746 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e6c0097e0 con 0x7f7e7c0606f0 2020-11-20T12:19:43.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e80fbd700 1 -- 172.21.15.116:0/4122466746 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e6c009b60 con 0x7f7e7c0606f0 2020-11-20T12:19:43.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e84223700 1 -- 172.21.15.116:0/4122466746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e7c0606f0 msgr2=0x7f7e7c0aa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e84223700 1 --2- 172.21.15.116:0/4122466746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e7c0606f0 0x7f7e7c0aa0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f7e6c00b9a0 tx=0x7f7e6c006df0).stop 2020-11-20T12:19:43.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e84223700 1 -- 172.21.15.116:0/4122466746 shutdown_connections 2020-11-20T12:19:43.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e84223700 1 --2- 172.21.15.116:0/4122466746 >> v2:172.21.15.173:3300/0 conn(0x7f7e7c066d90 0x7f7e7c159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e84223700 1 --2- 172.21.15.116:0/4122466746 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e7c0606f0 0x7f7e7c0aa0d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e84223700 1 -- 172.21.15.116:0/4122466746 >> 172.21.15.116:0/4122466746 conn(0x7f7e7c05ba80 msgr2=0x7f7e7c0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:43.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e84223700 1 -- 172.21.15.116:0/4122466746 shutdown_connections 2020-11-20T12:19:43.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.836+0000 7f7e84223700 1 -- 172.21.15.116:0/4122466746 wait complete. 2020-11-20T12:19:43.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.840+0000 7f7e84223700 1 Processor -- start 2020-11-20T12:19:43.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.840+0000 7f7e84223700 1 -- start start 2020-11-20T12:19:43.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.840+0000 7f7e84223700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e7c060310 0x7f7e7c153680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.840+0000 7f7e84223700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e7c066d90 0x7f7e7c0ba500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.840+0000 7f7e84223700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e7c0a7680 0x7f7e7c0a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.840+0000 7f7e84223700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c0b9750 con 0x7f7e7c0a7680 2020-11-20T12:19:43.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.840+0000 7f7e84223700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c152d10 con 0x7f7e7c066d90 2020-11-20T12:19:43.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.840+0000 7f7e84223700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e7c166540 con 0x7f7e7c060310 2020-11-20T12:19:43.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e827c0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e7c0a7680 0x7f7e7c0a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e827c0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e7c0a7680 0x7f7e7c0a7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48344/0 (socket says 172.21.15.116:48344) 2020-11-20T12:19:43.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e827c0700 1 -- 172.21.15.116:0/1251605528 learned_addr learned my addr 172.21.15.116:0/1251605528 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:43.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e81fbf700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e7c060310 0x7f7e7c153680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e817be700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e7c066d90 0x7f7e7c0ba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e827c0700 1 -- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e7c060310 msgr2=0x7f7e7c153680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e827c0700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e7c060310 0x7f7e7c153680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e827c0700 1 -- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e7c066d90 msgr2=0x7f7e7c0ba500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:43.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e827c0700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e7c066d90 0x7f7e7c0ba500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:43.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e827c0700 1 -- 172.21.15.116:0/1251605528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e7c1667e0 con 0x7f7e7c0a7680 2020-11-20T12:19:43.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e827c0700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e7c0a7680 0x7f7e7c0a7b00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f7e7400ef50 tx=0x7f7e740091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e69ffb700 1 -- 172.21.15.116:0/1251605528 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7400cc10 con 0x7f7e7c0a7680 2020-11-20T12:19:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e69ffb700 1 -- 172.21.15.116:0/1251605528 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7400cd70 con 0x7f7e7c0a7680 2020-11-20T12:19:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e84223700 1 -- 172.21.15.116:0/1251605528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e7c152f40 con 0x7f7e7c0a7680 2020-11-20T12:19:43.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e69ffb700 1 -- 172.21.15.116:0/1251605528 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e74012620 con 0x7f7e7c0a7680 2020-11-20T12:19:43.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.844+0000 7f7e84223700 1 -- 172.21.15.116:0/1251605528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e7c153b70 con 0x7f7e7c0a7680 2020-11-20T12:19:43.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.848+0000 7f7e84223700 1 -- 172.21.15.116:0/1251605528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e7c05d2a0 con 0x7f7e7c0a7680 2020-11-20T12:19:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.848+0000 7f7e69ffb700 1 -- 172.21.15.116:0/1251605528 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7e74022070 con 0x7f7e7c0a7680 2020-11-20T12:19:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.848+0000 7f7e69ffb700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e540557c0 0x7f7e54057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:43.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.848+0000 7f7e69ffb700 1 -- 172.21.15.116:0/1251605528 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f7e7406f170 con 0x7f7e7c0a7680 2020-11-20T12:19:43.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.848+0000 7f7e81fbf700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e540557c0 0x7f7e54057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:43.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.852+0000 7f7e81fbf700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e540557c0 0x7f7e54057c80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f7e78007fd0 tx=0x7f7e78009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:43.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:43.856+0000 7f7e69ffb700 1 -- 172.21.15.116:0/1251605528 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7e74044d80 con 0x7f7e7c0a7680 2020-11-20T12:19:44.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.012+0000 7f7e84223700 1 -- 172.21.15.116:0/1251605528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7e7c061380 con 0x7f7e7c0a7680 2020-11-20T12:19:44.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.012+0000 7f7e69ffb700 1 -- 172.21.15.116:0/1251605528 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7e740160a0 con 0x7f7e7c0a7680 2020-11-20T12:19:44.012 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:44.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 -- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e540557c0 msgr2=0x7f7e54057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e540557c0 0x7f7e54057c80 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f7e78007fd0 tx=0x7f7e78009040).stop 2020-11-20T12:19:44.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 -- 172.21.15.116:0/1251605528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e7c0a7680 msgr2=0x7f7e7c0a7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e7c0a7680 0x7f7e7c0a7b00 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7f7e7400ef50 tx=0x7f7e740091d0).stop 2020-11-20T12:19:44.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 -- 172.21.15.116:0/1251605528 shutdown_connections 2020-11-20T12:19:44.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7e540557c0 0x7f7e54057c80 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7e7c0a7680 0x7f7e7c0a7b00 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7e7c060310 0x7f7e7c153680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 --2- 172.21.15.116:0/1251605528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7e7c066d90 0x7f7e7c0ba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 -- 172.21.15.116:0/1251605528 >> 172.21.15.116:0/1251605528 conn(0x7f7e7c05ba80 msgr2=0x7f7e7c150fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:44.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 -- 172.21.15.116:0/1251605528 shutdown_connections 2020-11-20T12:19:44.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.016+0000 7f7e84223700 1 -- 172.21.15.116:0/1251605528 wait complete. 2020-11-20T12:19:44.038 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:19:44.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.240+0000 7fc88b59e700 1 Processor -- start 2020-11-20T12:19:44.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88b59e700 1 -- start start 2020-11-20T12:19:44.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88b59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc88c0c1470 0x7fc88c0c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88b59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c066b90 con 0x7fc88c0606f0 2020-11-20T12:19:44.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88b59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88c066e00 con 0x7fc88c060310 2020-11-20T12:19:44.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88b59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88c0670a0 con 0x7fc88c0c1470 2020-11-20T12:19:44.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88ad9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc88c0c1470 0x7fc88c0c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88ad9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc88c0c1470 0x7fc88c0c18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48352/0 (socket says 172.21.15.116:48352) 2020-11-20T12:19:44.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88ad9d700 1 -- 172.21.15.116:0/3618831572 learned_addr learned my addr 172.21.15.116:0/3618831572 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:44.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc889d9b700 1 --1- 172.21.15.116:0/3618831572 >> v1:172.21.15.116:6789/0 conn(0x7fc88c0606f0 0x7fc88c0bf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57782/0 (socket says 172.21.15.116:57782) 2020-11-20T12:19:44.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88ad9d700 1 -- 172.21.15.116:0/3618831572 >> v1:172.21.15.173:6789/0 conn(0x7fc88c060310 legacy=0x7fc88c066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88ad9d700 1 -- 172.21.15.116:0/3618831572 >> v1:172.21.15.116:6789/0 conn(0x7fc88c0606f0 legacy=0x7fc88c0bf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88959a700 1 -- 172.21.15.116:0/3618831572 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557791989 0 0) 0x7fc88c066b90 con 0x7fc88c0606f0 2020-11-20T12:19:44.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.244+0000 7fc88ad9d700 1 -- 172.21.15.116:0/3618831572 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c067340 con 0x7fc88c0c1470 2020-11-20T12:19:44.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.248+0000 7fc88ad9d700 1 --2- 172.21.15.116:0/3618831572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc88c0c1470 0x7fc88c0c18c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fc884009d50 tx=0x7fc884009290).ready entity=mon.0 client_cookie=7ca386a5b52da614 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:44.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.248+0000 7fc88959a700 1 -- 172.21.15.116:0/3618831572 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88400bc80 con 0x7fc88c0c1470 2020-11-20T12:19:44.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.248+0000 7fc88959a700 1 -- 172.21.15.116:0/3618831572 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88400bde0 con 0x7fc88c0c1470 2020-11-20T12:19:44.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.248+0000 7fc88959a700 1 -- 172.21.15.116:0/3618831572 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884011610 con 0x7fc88c0c1470 2020-11-20T12:19:44.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.248+0000 7fc88b59e700 1 -- 172.21.15.116:0/3618831572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc88c0c1470 msgr2=0x7fc88c0c18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.248+0000 7fc88b59e700 1 --2- 172.21.15.116:0/3618831572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc88c0c1470 0x7fc88c0c18c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7fc884009d50 tx=0x7fc884009290).stop 2020-11-20T12:19:44.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.248+0000 7fc88b59e700 1 -- 172.21.15.116:0/3618831572 shutdown_connections 2020-11-20T12:19:44.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.248+0000 7fc88b59e700 1 --2- 172.21.15.116:0/3618831572 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc88c0c1470 0x7fc88c0c18c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.248+0000 7fc88b59e700 1 -- 172.21.15.116:0/3618831572 >> 172.21.15.116:0/3618831572 conn(0x7fc88c068350 msgr2=0x7fc88c0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:44.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.248+0000 7fc88b59e700 1 -- 172.21.15.116:0/3618831572 shutdown_connections 2020-11-20T12:19:44.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.248+0000 7fc88b59e700 1 -- 172.21.15.116:0/3618831572 wait complete. 2020-11-20T12:19:44.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88b59e700 1 Processor -- start 2020-11-20T12:19:44.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88b59e700 1 -- start start 2020-11-20T12:19:44.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c060310 0x7fc88c13d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc88c0606f0 0x7fc88c13d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc88c0c1470 0x7fc88c13ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c13c020 con 0x7fc88c0c1470 2020-11-20T12:19:44.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c13c790 con 0x7fc88c060310 2020-11-20T12:19:44.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c13c250 con 0x7fc88c0606f0 2020-11-20T12:19:44.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc88c0c1470 0x7fc88c13ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc88c0c1470 0x7fc88c13ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48358/0 (socket says 172.21.15.116:48358) 2020-11-20T12:19:44.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc889d9b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc88c0606f0 0x7fc88c13d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88ad9d700 1 -- 172.21.15.116:0/1854040145 learned_addr learned my addr 172.21.15.116:0/1854040145 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:44.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.252+0000 7fc88a59c700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c060310 0x7fc88c13d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc88ad9d700 1 -- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc88c0606f0 msgr2=0x7fc88c13d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc88ad9d700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc88c0606f0 0x7fc88c13d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc88ad9d700 1 -- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c060310 msgr2=0x7fc88c13d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc88ad9d700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c060310 0x7fc88c13d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc88ad9d700 1 -- 172.21.15.116:0/1854040145 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c13c4f0 con 0x7fc88c0c1470 2020-11-20T12:19:44.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc88a59c700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c060310 0x7fc88c13d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:44.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc88ad9d700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc88c0c1470 0x7fc88c13ddb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fc884008000 tx=0x7fc884016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:44.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc87a7fc700 1 -- 172.21.15.116:0/1854040145 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88400b750 con 0x7fc88c0c1470 2020-11-20T12:19:44.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc87a7fc700 1 -- 172.21.15.116:0/1854040145 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc884011e30 con 0x7fc88c0c1470 2020-11-20T12:19:44.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc87a7fc700 1 -- 172.21.15.116:0/1854040145 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc884010dc0 con 0x7fc88c0c1470 2020-11-20T12:19:44.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc88b59e700 1 -- 172.21.15.116:0/1854040145 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc88c0a9560 con 0x7fc88c0c1470 2020-11-20T12:19:44.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc88b59e700 1 -- 172.21.15.116:0/1854040145 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc88c13b7a0 con 0x7fc88c0c1470 2020-11-20T12:19:44.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.256+0000 7fc88b59e700 1 -- 172.21.15.116:0/1854040145 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc88c05cf30 con 0x7fc88c0c1470 2020-11-20T12:19:44.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.260+0000 7fc87a7fc700 1 -- 172.21.15.116:0/1854040145 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc884017070 con 0x7fc88c0c1470 2020-11-20T12:19:44.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.260+0000 7fc87a7fc700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8600557c0 0x7fc860057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.260+0000 7fc87a7fc700 1 -- 172.21.15.116:0/1854040145 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fc884070570 con 0x7fc88c0c1470 2020-11-20T12:19:44.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.260+0000 7fc88a59c700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8600557c0 0x7fc860057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.260+0000 7fc88a59c700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8600557c0 0x7fc860057c80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fc87c001550 tx=0x7fc87c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:44.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.264+0000 7fc87a7fc700 1 -- 172.21.15.116:0/1854040145 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc884046200 con 0x7fc88c0c1470 2020-11-20T12:19:44.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.424+0000 7fc88b59e700 1 -- 172.21.15.116:0/1854040145 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc88c0639c0 con 0x7fc88c0c1470 2020-11-20T12:19:44.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.424+0000 7fc87a7fc700 1 -- 172.21.15.116:0/1854040145 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc884015070 con 0x7fc88c0c1470 2020-11-20T12:19:44.423 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:44.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 -- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8600557c0 msgr2=0x7fc860057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8600557c0 0x7fc860057c80 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fc87c001550 tx=0x7fc87c009040).stop 2020-11-20T12:19:44.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 -- 172.21.15.116:0/1854040145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc88c0c1470 msgr2=0x7fc88c13ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc88c0c1470 0x7fc88c13ddb0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7fc884008000 tx=0x7fc884016000).stop 2020-11-20T12:19:44.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 -- 172.21.15.116:0/1854040145 shutdown_connections 2020-11-20T12:19:44.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc8600557c0 0x7fc860057c80 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc88c0c1470 0x7fc88c13ddb0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc88c060310 0x7fc88c13d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 --2- 172.21.15.116:0/1854040145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc88c0606f0 0x7fc88c13d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 -- 172.21.15.116:0/1854040145 >> 172.21.15.116:0/1854040145 conn(0x7fc88c068350 msgr2=0x7fc88c1516b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:44.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 -- 172.21.15.116:0/1854040145 shutdown_connections 2020-11-20T12:19:44.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.428+0000 7fc88b59e700 1 -- 172.21.15.116:0/1854040145 wait complete. 2020-11-20T12:19:44.451 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get fsvolume_volid pg_num 2020-11-20T12:19:44.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.652+0000 7fc061623700 1 Processor -- start 2020-11-20T12:19:44.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.652+0000 7fc061623700 1 -- start start 2020-11-20T12:19:44.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc061623700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c1516d0 0x7fc05c063920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc061623700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc05c063e10 0x7fc05c0c4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc061623700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc05c0c4840 0x7fc05c0626f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc061623700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc05c064270 con 0x7fc05c063e10 2020-11-20T12:19:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc061623700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc05c0c5060 con 0x7fc05c0c4840 2020-11-20T12:19:44.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc061623700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c155500 con 0x7fc05c1516d0 2020-11-20T12:19:44.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05a7fc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc05c063e10 0x7fc05c0c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05affd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c1516d0 0x7fc05c063920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05affd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c1516d0 0x7fc05c063920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:57312/0 (socket says 172.21.15.116:57312) 2020-11-20T12:19:44.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05a7fc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc05c063e10 0x7fc05c0c4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48364/0 (socket says 172.21.15.116:48364) 2020-11-20T12:19:44.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05affd700 1 -- 172.21.15.116:0/600817509 learned_addr learned my addr 172.21.15.116:0/600817509 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:44.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05bfff700 1 --2- 172.21.15.116:0/600817509 >> v2:172.21.15.173:3300/0 conn(0x7fc05c0c4840 0x7fc05c0626f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05a7fc700 1 -- 172.21.15.116:0/600817509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c1516d0 msgr2=0x7fc05c063920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:44.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05a7fc700 1 --2- 172.21.15.116:0/600817509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c1516d0 0x7fc05c063920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05a7fc700 1 -- 172.21.15.116:0/600817509 >> v2:172.21.15.173:3300/0 conn(0x7fc05c0c4840 msgr2=0x7fc05c0626f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05a7fc700 1 --2- 172.21.15.116:0/600817509 >> v2:172.21.15.173:3300/0 conn(0x7fc05c0c4840 0x7fc05c0626f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05a7fc700 1 -- 172.21.15.116:0/600817509 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c1557a0 con 0x7fc05c063e10 2020-11-20T12:19:44.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05affd700 1 --2- 172.21.15.116:0/600817509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c1516d0 0x7fc05c063920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:44.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc05a7fc700 1 --2- 172.21.15.116:0/600817509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc05c063e10 0x7fc05c0c4350 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fc05c0810f0 tx=0x7fc050006df0).ready entity=mon.0 client_cookie=fa3fb1726e3ab56d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:44.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.656+0000 7fc059ffb700 1 -- 172.21.15.116:0/600817509 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050009640 con 0x7fc05c063e10 2020-11-20T12:19:44.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.660+0000 7fc059ffb700 1 -- 172.21.15.116:0/600817509 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0500097a0 con 0x7fc05c063e10 2020-11-20T12:19:44.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.660+0000 7fc059ffb700 1 -- 172.21.15.116:0/600817509 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc050009b20 con 0x7fc05c063e10 2020-11-20T12:19:44.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.660+0000 7fc061623700 1 -- 172.21.15.116:0/600817509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc05c063e10 msgr2=0x7fc05c0c4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.660+0000 7fc061623700 1 --2- 172.21.15.116:0/600817509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc05c063e10 0x7fc05c0c4350 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7fc05c0810f0 tx=0x7fc050006df0).stop 2020-11-20T12:19:44.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.660+0000 7fc061623700 1 -- 172.21.15.116:0/600817509 shutdown_connections 2020-11-20T12:19:44.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.660+0000 7fc061623700 1 --2- 172.21.15.116:0/600817509 >> v2:172.21.15.173:3300/0 conn(0x7fc05c0c4840 0x7fc05c0626f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.660+0000 7fc061623700 1 --2- 172.21.15.116:0/600817509 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c1516d0 0x7fc05c063920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.660+0000 7fc061623700 1 --2- 172.21.15.116:0/600817509 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc05c063e10 0x7fc05c0c4350 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.660+0000 7fc061623700 1 -- 172.21.15.116:0/600817509 >> 172.21.15.116:0/600817509 conn(0x7fc05c065f70 msgr2=0x7fc05c066380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:44.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.660+0000 7fc061623700 1 -- 172.21.15.116:0/600817509 shutdown_connections 2020-11-20T12:19:44.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.660+0000 7fc061623700 1 -- 172.21.15.116:0/600817509 wait complete. 2020-11-20T12:19:44.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc061623700 1 Processor -- start 2020-11-20T12:19:44.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc061623700 1 -- start start 2020-11-20T12:19:44.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc061623700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c063e10 0x7fc05c13d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc061623700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc05c0c4840 0x7fc05c0c77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc061623700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc05c1516d0 0x7fc05c0a8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc061623700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c1619c0 con 0x7fc05c1516d0 2020-11-20T12:19:44.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc061623700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c161bb0 con 0x7fc05c0c4840 2020-11-20T12:19:44.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc061623700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c161e50 con 0x7fc05c063e10 2020-11-20T12:19:44.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc05c1516d0 0x7fc05c0a8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc05bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc05c1516d0 0x7fc05c0a8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48372/0 (socket says 172.21.15.116:48372) 2020-11-20T12:19:44.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc05bfff700 1 -- 172.21.15.116:0/3581818850 learned_addr learned my addr 172.21.15.116:0/3581818850 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:44.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc05a7fc700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc05c0c4840 0x7fc05c0c77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc05affd700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c063e10 0x7fc05c13d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc05bfff700 1 -- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c063e10 msgr2=0x7fc05c13d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.664+0000 7fc05bfff700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c063e10 0x7fc05c13d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.668+0000 7fc05bfff700 1 -- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc05c0c4840 msgr2=0x7fc05c0c77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.668+0000 7fc05bfff700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc05c0c4840 0x7fc05c0c77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.668+0000 7fc05bfff700 1 -- 172.21.15.116:0/3581818850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c1620f0 con 0x7fc05c1516d0 2020-11-20T12:19:44.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.668+0000 7fc05bfff700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc05c1516d0 0x7fc05c0a8fc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7fc05400ed20 tx=0x7fc0540091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:44.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.668+0000 7fc042ffd700 1 -- 172.21.15.116:0/3581818850 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05400eee0 con 0x7fc05c1516d0 2020-11-20T12:19:44.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.668+0000 7fc042ffd700 1 -- 172.21.15.116:0/3581818850 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc05400cbb0 con 0x7fc05c1516d0 2020-11-20T12:19:44.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.668+0000 7fc061623700 1 -- 172.21.15.116:0/3581818850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc05c162390 con 0x7fc05c1516d0 2020-11-20T12:19:44.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.668+0000 7fc042ffd700 1 -- 172.21.15.116:0/3581818850 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0540123f0 con 0x7fc05c1516d0 2020-11-20T12:19:44.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.668+0000 7fc061623700 1 -- 172.21.15.116:0/3581818850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc05c13db80 con 0x7fc05c1516d0 2020-11-20T12:19:44.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.668+0000 7fc061623700 1 -- 172.21.15.116:0/3581818850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc05c069950 con 0x7fc05c1516d0 2020-11-20T12:19:44.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.672+0000 7fc042ffd700 1 -- 172.21.15.116:0/3581818850 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc054022070 con 0x7fc05c1516d0 2020-11-20T12:19:44.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.672+0000 7fc042ffd700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0380557f0 0x7fc038057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:44.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.672+0000 7fc042ffd700 1 -- 172.21.15.116:0/3581818850 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7fc05406eff0 con 0x7fc05c1516d0 2020-11-20T12:19:44.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.672+0000 7fc05affd700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0380557f0 0x7fc038057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:44.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.676+0000 7fc05affd700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0380557f0 0x7fc038057cb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fc04c006fd0 tx=0x7fc04c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:44.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.680+0000 7fc042ffd700 1 -- 172.21.15.116:0/3581818850 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc054044c00 con 0x7fc05c1516d0 2020-11-20T12:19:44.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.836+0000 7fc061623700 1 -- 172.21.15.116:0/3581818850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "fsvolume_volid", "var": "pg_num"} v 0) v1 -- 0x7fc05c0646b0 con 0x7fc05c1516d0 2020-11-20T12:19:44.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.836+0000 7fc042ffd700 1 -- 172.21.15.116:0/3581818850 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "fsvolume_volid", "var": "pg_num"}]=0 v74) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc054016030 con 0x7fc05c1516d0 2020-11-20T12:19:44.835 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 32 2020-11-20T12:19:44.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 -- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0380557f0 msgr2=0x7fc038057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0380557f0 0x7fc038057cb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7fc04c006fd0 tx=0x7fc04c008040).stop 2020-11-20T12:19:44.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 -- 172.21.15.116:0/3581818850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc05c1516d0 msgr2=0x7fc05c0a8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:44.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc05c1516d0 0x7fc05c0a8fc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7fc05400ed20 tx=0x7fc0540091d0).stop 2020-11-20T12:19:44.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 -- 172.21.15.116:0/3581818850 shutdown_connections 2020-11-20T12:19:44.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0380557f0 0x7fc038057cb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc05c1516d0 0x7fc05c0a8fc0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc05c063e10 0x7fc05c13d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 --2- 172.21.15.116:0/3581818850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc05c0c4840 0x7fc05c0c77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:44.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 -- 172.21.15.116:0/3581818850 >> 172.21.15.116:0/3581818850 conn(0x7fc05c065f70 msgr2=0x7fc05c154d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:44.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 -- 172.21.15.116:0/3581818850 shutdown_connections 2020-11-20T12:19:44.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:44.840+0000 7fc061623700 1 -- 172.21.15.116:0/3581818850 wait complete. 2020-11-20T12:19:44.863 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:19:45.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.040+0000 7f9db19a5700 1 Processor -- start 2020-11-20T12:19:45.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.040+0000 7f9db19a5700 1 -- start start 2020-11-20T12:19:45.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db19a5700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9dac062880 0x7f9dac062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db19a5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dac153450 0x7f9dac0677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db19a5700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dac1543a0 con 0x7f9dac0c4c10 2020-11-20T12:19:45.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db19a5700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dac153920 con 0x7f9dac062880 2020-11-20T12:19:45.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db19a5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac153bc0 con 0x7f9dac153450 2020-11-20T12:19:45.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db11a4700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9dac0c4c10 0x7f9dac0b9710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57814/0 (socket says 172.21.15.116:57814) 2020-11-20T12:19:45.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db11a4700 1 -- 172.21.15.116:0/1279747427 learned_addr learned my addr 172.21.15.116:0/1279747427 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:45.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db09a3700 1 --2- 172.21.15.116:0/1279747427 >> v2:172.21.15.173:3300/0 conn(0x7f9dac062880 0x7f9dac062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9dabfff700 1 --2- 172.21.15.116:0/1279747427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dac153450 0x7f9dac0677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9dab7fe700 1 -- 172.21.15.116:0/1279747427 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972786556 0 0) 0x7f9dac1543a0 con 0x7f9dac0c4c10 2020-11-20T12:19:45.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9dab7fe700 1 -- 172.21.15.116:0/1279747427 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d90003240 con 0x7f9dac0c4c10 2020-11-20T12:19:45.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9dab7fe700 1 -- 172.21.15.116:0/1279747427 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2525751956 0 0) 0x7f9d90003240 con 0x7f9dac0c4c10 2020-11-20T12:19:45.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db09a3700 1 -- 172.21.15.116:0/1279747427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dac153450 msgr2=0x7f9dac0677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:45.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db09a3700 1 --2- 172.21.15.116:0/1279747427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dac153450 0x7f9dac0677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db09a3700 1 -- 172.21.15.116:0/1279747427 >> v1:172.21.15.116:6789/0 conn(0x7f9dac0c4c10 legacy=0x7f9dac0b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db09a3700 1 -- 172.21.15.116:0/1279747427 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac153e60 con 0x7f9dac062880 2020-11-20T12:19:45.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9dabfff700 1 --2- 172.21.15.116:0/1279747427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dac153450 0x7f9dac0677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:45.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9db09a3700 1 --2- 172.21.15.116:0/1279747427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac062880 0x7f9dac062c60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f9da000b9a0 tx=0x7f9da0006df0).ready entity=mon.1 client_cookie=bcbf58f179d6191b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:45.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9dab7fe700 1 -- 172.21.15.116:0/1279747427 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da4001860 con 0x7f9dac062880 2020-11-20T12:19:45.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9dab7fe700 1 -- 172.21.15.116:0/1279747427 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da0011400 con 0x7f9dac062880 2020-11-20T12:19:45.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.044+0000 7f9dab7fe700 1 -- 172.21.15.116:0/1279747427 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0011780 con 0x7f9dac062880 2020-11-20T12:19:45.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.048+0000 7f9db19a5700 1 -- 172.21.15.116:0/1279747427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac062880 msgr2=0x7f9dac062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.048+0000 7f9db19a5700 1 --2- 172.21.15.116:0/1279747427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac062880 0x7f9dac062c60 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f9da000b9a0 tx=0x7f9da0006df0).stop 2020-11-20T12:19:45.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.048+0000 7f9db19a5700 1 -- 172.21.15.116:0/1279747427 shutdown_connections 2020-11-20T12:19:45.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.048+0000 7f9db19a5700 1 --2- 172.21.15.116:0/1279747427 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac062880 0x7f9dac062c60 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.048+0000 7f9db19a5700 1 --2- 172.21.15.116:0/1279747427 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dac153450 0x7f9dac0677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.048+0000 7f9db19a5700 1 -- 172.21.15.116:0/1279747427 >> 172.21.15.116:0/1279747427 conn(0x7f9dac060c40 msgr2=0x7f9dac0c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:45.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.048+0000 7f9db19a5700 1 -- 172.21.15.116:0/1279747427 shutdown_connections 2020-11-20T12:19:45.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.048+0000 7f9db19a5700 1 -- 172.21.15.116:0/1279747427 wait complete. 2020-11-20T12:19:45.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.048+0000 7f9db19a5700 1 Processor -- start 2020-11-20T12:19:45.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.048+0000 7f9db19a5700 1 -- start start 2020-11-20T12:19:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db19a5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac062880 0x7f9dac0bfac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db19a5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dac0c4c10 0x7f9dac0bb460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db19a5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dac153450 0x7f9dac13ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db19a5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac13c840 con 0x7f9dac153450 2020-11-20T12:19:45.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db19a5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac0c2500 con 0x7f9dac062880 2020-11-20T12:19:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db19a5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dac0c27a0 con 0x7f9dac0c4c10 2020-11-20T12:19:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db11a4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dac153450 0x7f9dac13ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db11a4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dac153450 0x7f9dac13ba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48386/0 (socket says 172.21.15.116:48386) 2020-11-20T12:19:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db11a4700 1 -- 172.21.15.116:0/4167083204 learned_addr learned my addr 172.21.15.116:0/4167083204 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:45.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db09a3700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac062880 0x7f9dac0bfac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9dabfff700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dac0c4c10 0x7f9dac0bb460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db11a4700 1 -- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dac0c4c10 msgr2=0x7f9dac0bb460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db11a4700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dac0c4c10 0x7f9dac0bb460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db11a4700 1 -- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac062880 msgr2=0x7f9dac0bfac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db11a4700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac062880 0x7f9dac0bfac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db11a4700 1 -- 172.21.15.116:0/4167083204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dac0c2a40 con 0x7f9dac153450 2020-11-20T12:19:45.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db11a4700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dac153450 0x7f9dac13ba60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f9da400ce60 tx=0x7f9da4001600).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9da8ff9700 1 -- 172.21.15.116:0/4167083204 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da400ab30 con 0x7f9dac153450 2020-11-20T12:19:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9da8ff9700 1 -- 172.21.15.116:0/4167083204 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da400ac90 con 0x7f9dac153450 2020-11-20T12:19:45.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db19a5700 1 -- 172.21.15.116:0/4167083204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dac0c2ce0 con 0x7f9dac153450 2020-11-20T12:19:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9da8ff9700 1 -- 172.21.15.116:0/4167083204 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da40073a0 con 0x7f9dac153450 2020-11-20T12:19:45.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.052+0000 7f9db19a5700 1 -- 172.21.15.116:0/4167083204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dac0c2180 con 0x7f9dac153450 2020-11-20T12:19:45.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.056+0000 7f9db19a5700 1 -- 172.21.15.116:0/4167083204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dac05cf30 con 0x7f9dac153450 2020-11-20T12:19:45.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.056+0000 7f9da8ff9700 1 -- 172.21.15.116:0/4167083204 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9da401b060 con 0x7f9dac153450 2020-11-20T12:19:45.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.056+0000 7f9da8ff9700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d840557c0 0x7f9d84057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.056+0000 7f9da8ff9700 1 -- 172.21.15.116:0/4167083204 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f9da406e990 con 0x7f9dac153450 2020-11-20T12:19:45.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.056+0000 7f9db09a3700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d840557c0 0x7f9d84057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.060+0000 7f9db09a3700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d840557c0 0x7f9d84057c80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f9da000b150 tx=0x7f9da0013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:45.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.064+0000 7f9da8ff9700 1 -- 172.21.15.116:0/4167083204 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9da4044540 con 0x7f9dac153450 2020-11-20T12:19:45.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.264+0000 7f9db19a5700 1 -- 172.21.15.116:0/4167083204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9dac0c1370 con 0x7f9dac153450 2020-11-20T12:19:45.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.268+0000 7f9da8ff9700 1 -- 172.21.15.116:0/4167083204 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+2857 (secure 0 0 0) 0x7f9da4019090 con 0x7f9dac153450 2020-11-20T12:19:45.265 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:45.265 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5681,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3869489418","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3869489418},{"type":"v1","addr":"172.21.15.173:6837","nonce":3869489418}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/2912073472","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2912073472},{"type":"v1","addr":"172.21.15.173:6835","nonce":2912073472}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33},{"gid":6426,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/2221764249","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2221764249},{"type":"v1","addr":"172.21.15.116:6834","nonce":2221764249}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:12.981713+0000","modified":"2020-11-20T12:19:33.421473+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6429},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6429":{"gid":6429,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1946956517","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1946956517},{"type":"v1","addr":"172.21.15.116:6835","nonce":1946956517}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[13,14],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2020-11-20T12:19:45.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.268+0000 7f9db19a5700 1 -- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d840557c0 msgr2=0x7f9d84057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.268+0000 7f9db19a5700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d840557c0 0x7f9d84057c80 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f9da000b150 tx=0x7f9da0013000).stop 2020-11-20T12:19:45.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.268+0000 7f9db19a5700 1 -- 172.21.15.116:0/4167083204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dac153450 msgr2=0x7f9dac13ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.268+0000 7f9db19a5700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dac153450 0x7f9dac13ba60 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f9da400ce60 tx=0x7f9da4001600).stop 2020-11-20T12:19:45.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.268+0000 7f9db19a5700 1 -- 172.21.15.116:0/4167083204 shutdown_connections 2020-11-20T12:19:45.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.268+0000 7f9db19a5700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d840557c0 0x7f9d84057c80 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.268+0000 7f9db19a5700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dac153450 0x7f9dac13ba60 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.268+0000 7f9db19a5700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dac062880 0x7f9dac0bfac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.268+0000 7f9db19a5700 1 --2- 172.21.15.116:0/4167083204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dac0c4c10 0x7f9dac0bb460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.268+0000 7f9db19a5700 1 -- 172.21.15.116:0/4167083204 >> 172.21.15.116:0/4167083204 conn(0x7f9dac060c40 msgr2=0x7f9dac0a5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:45.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.272+0000 7f9db19a5700 1 -- 172.21.15.116:0/4167083204 shutdown_connections 2020-11-20T12:19:45.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.272+0000 7f9db19a5700 1 -- 172.21.15.116:0/4167083204 wait complete. 2020-11-20T12:19:45.271 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 38 2020-11-20T12:19:45.291 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:45.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.468+0000 7f6ada19e700 1 Processor -- start 2020-11-20T12:19:45.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.468+0000 7f6ada19e700 1 -- start start 2020-11-20T12:19:45.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ada19e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6ad4155df0 0x7f6ad4065d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ada19e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad40c0530 con 0x7f6ad4062880 2020-11-20T12:19:45.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ada19e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ad40c07d0 con 0x7f6ad4062c60 2020-11-20T12:19:45.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ada19e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ad40c0a70 con 0x7f6ad4155df0 2020-11-20T12:19:45.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ad919c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6ad4062880 0x7f6ad40677a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57824/0 (socket says 172.21.15.116:57824) 2020-11-20T12:19:45.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ad919c700 1 -- 172.21.15.116:0/3724571794 learned_addr learned my addr 172.21.15.116:0/3724571794 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:45.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ad999d700 1 --2- 172.21.15.116:0/3724571794 >> v2:172.21.15.116:3300/0 conn(0x7f6ad4155df0 0x7f6ad4065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ac3fff700 1 -- 172.21.15.116:0/3724571794 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3335354887 0 0) 0x7f6ad40c0530 con 0x7f6ad4062880 2020-11-20T12:19:45.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ac3fff700 1 -- 172.21.15.116:0/3724571794 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6abc003240 con 0x7f6ad4062880 2020-11-20T12:19:45.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ad999d700 1 -- 172.21.15.116:0/3724571794 >> v1:172.21.15.173:6789/0 conn(0x7f6ad4062c60 legacy=0x7f6ad41556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ad999d700 1 -- 172.21.15.116:0/3724571794 >> v1:172.21.15.116:6789/0 conn(0x7f6ad4062880 legacy=0x7f6ad40677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ad999d700 1 -- 172.21.15.116:0/3724571794 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad41599b0 con 0x7f6ad4155df0 2020-11-20T12:19:45.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ad999d700 1 --2- 172.21.15.116:0/3724571794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ad4155df0 0x7f6ad4065d40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f6acc009c90 tx=0x7f6acc0091d0).ready entity=mon.0 client_cookie=248fe232b257e2e6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:45.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ac3fff700 1 -- 172.21.15.116:0/3724571794 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc00ba70 con 0x7f6ad4155df0 2020-11-20T12:19:45.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ac3fff700 1 -- 172.21.15.116:0/3724571794 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6acc00bbd0 con 0x7f6ad4155df0 2020-11-20T12:19:45.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ac3fff700 1 -- 172.21.15.116:0/3724571794 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6acc011400 con 0x7f6ad4155df0 2020-11-20T12:19:45.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ada19e700 1 -- 172.21.15.116:0/3724571794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ad4155df0 msgr2=0x7f6ad4065d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ada19e700 1 --2- 172.21.15.116:0/3724571794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ad4155df0 0x7f6ad4065d40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f6acc009c90 tx=0x7f6acc0091d0).stop 2020-11-20T12:19:45.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ada19e700 1 -- 172.21.15.116:0/3724571794 shutdown_connections 2020-11-20T12:19:45.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ada19e700 1 --2- 172.21.15.116:0/3724571794 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ad4155df0 0x7f6ad4065d40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.472+0000 7f6ada19e700 1 -- 172.21.15.116:0/3724571794 >> 172.21.15.116:0/3724571794 conn(0x7f6ad4060c40 msgr2=0x7f6ad40c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:45.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.476+0000 7f6ada19e700 1 -- 172.21.15.116:0/3724571794 shutdown_connections 2020-11-20T12:19:45.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.476+0000 7f6ada19e700 1 -- 172.21.15.116:0/3724571794 wait complete. 2020-11-20T12:19:45.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.476+0000 7f6ada19e700 1 Processor -- start 2020-11-20T12:19:45.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.476+0000 7f6ada19e700 1 -- start start 2020-11-20T12:19:45.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ada19e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ad4062880 0x7f6ad41577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ada19e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ad4062c60 0x7f6ad4148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ada19e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ad4155df0 0x7f6ad4066910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ada19e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad41568d0 con 0x7f6ad4062c60 2020-11-20T12:19:45.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ada19e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad4156b70 con 0x7f6ad4155df0 2020-11-20T12:19:45.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ada19e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad4156e10 con 0x7f6ad4062880 2020-11-20T12:19:45.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad899b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ad4062c60 0x7f6ad4148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad899b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ad4062c60 0x7f6ad4148840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48398/0 (socket says 172.21.15.116:48398) 2020-11-20T12:19:45.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad899b700 1 -- 172.21.15.116:0/1954792080 learned_addr learned my addr 172.21.15.116:0/1954792080 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:45.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad919c700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ad4062880 0x7f6ad41577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad999d700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ad4155df0 0x7f6ad4066910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad899b700 1 -- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ad4062880 msgr2=0x7f6ad41577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad899b700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ad4062880 0x7f6ad41577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad899b700 1 -- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ad4155df0 msgr2=0x7f6ad4066910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad899b700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ad4155df0 0x7f6ad4066910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad899b700 1 -- 172.21.15.116:0/1954792080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad41570b0 con 0x7f6ad4062c60 2020-11-20T12:19:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad919c700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ad4062880 0x7f6ad41577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:45.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ad899b700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ad4062c60 0x7f6ad4148840 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f6ac4001e70 tx=0x7f6ac4007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ac17fa700 1 -- 172.21.15.116:0/1954792080 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4007020 con 0x7f6ad4062c60 2020-11-20T12:19:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ac17fa700 1 -- 172.21.15.116:0/1954792080 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ac4007180 con 0x7f6ad4062c60 2020-11-20T12:19:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ada19e700 1 -- 172.21.15.116:0/1954792080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad4157350 con 0x7f6ad4062c60 2020-11-20T12:19:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.480+0000 7f6ac17fa700 1 -- 172.21.15.116:0/1954792080 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac400e560 con 0x7f6ad4062c60 2020-11-20T12:19:45.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.484+0000 7f6ada19e700 1 -- 172.21.15.116:0/1954792080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad4066fa0 con 0x7f6ad4062c60 2020-11-20T12:19:45.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.484+0000 7f6ada19e700 1 -- 172.21.15.116:0/1954792080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad4067280 con 0x7f6ad4062c60 2020-11-20T12:19:45.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.484+0000 7f6ac17fa700 1 -- 172.21.15.116:0/1954792080 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6ac4018020 con 0x7f6ad4062c60 2020-11-20T12:19:45.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.484+0000 7f6ac17fa700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6aac0557c0 0x7f6aac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.488+0000 7f6ac17fa700 1 -- 172.21.15.116:0/1954792080 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f6ac406e180 con 0x7f6ad4062c60 2020-11-20T12:19:45.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.488+0000 7f6ad919c700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6aac0557c0 0x7f6aac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.488+0000 7f6ad919c700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6aac0557c0 0x7f6aac057c80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f6ad00055a0 tx=0x7f6ad0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:45.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.492+0000 7f6ac17fa700 1 -- 172.21.15.116:0/1954792080 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6ac4043e40 con 0x7f6ad4062c60 2020-11-20T12:19:45.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.656+0000 7f6ada19e700 1 -- 172.21.15.116:0/1954792080 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ad405cf30 con 0x7f6ad4062c60 2020-11-20T12:19:45.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.656+0000 7f6ac17fa700 1 -- 172.21.15.116:0/1954792080 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+18428 (secure 0 0 0) 0x7f6ac4016030 con 0x7f6ad4062c60 2020-11-20T12:19:45.655 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:45.655 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":74,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:34.425107+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:11.306087+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"70","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":13,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:11.966077+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}},{"pool":14,"pool_name":"fsvolume_volid","create_time":"2020-11-20T12:19:32.859361+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:45.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 -- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6aac0557c0 msgr2=0x7f6aac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6aac0557c0 0x7f6aac057c80 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f6ad00055a0 tx=0x7f6ad0009040).stop 2020-11-20T12:19:45.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 -- 172.21.15.116:0/1954792080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ad4062c60 msgr2=0x7f6ad4148840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ad4062c60 0x7f6ad4148840 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f6ac4001e70 tx=0x7f6ac4007c50).stop 2020-11-20T12:19:45.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 -- 172.21.15.116:0/1954792080 shutdown_connections 2020-11-20T12:19:45.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6aac0557c0 0x7f6aac057c80 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ad4155df0 0x7f6ad4066910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ad4062880 0x7f6ad41577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 --2- 172.21.15.116:0/1954792080 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ad4062c60 0x7f6ad4148840 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 -- 172.21.15.116:0/1954792080 >> 172.21.15.116:0/1954792080 conn(0x7f6ad4060c40 msgr2=0x7f6ad40c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:45.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 -- 172.21.15.116:0/1954792080 shutdown_connections 2020-11-20T12:19:45.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.660+0000 7f6ada19e700 1 -- 172.21.15.116:0/1954792080 wait complete. 2020-11-20T12:19:45.682 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-20T12:19:45.683 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-20T12:19:45.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.864+0000 7f99c496e700 1 Processor -- start 2020-11-20T12:19:45.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.864+0000 7f99c496e700 1 -- start start 2020-11-20T12:19:45.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.864+0000 7f99c496e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f99c00a5d10 0x7f99c0062750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.864+0000 7f99c496e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c014f4b0 con 0x7f99c00a5930 2020-11-20T12:19:45.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.864+0000 7f99c496e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c014f750 con 0x7f99c0152d40 2020-11-20T12:19:45.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.864+0000 7f99c496e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c014f9f0 con 0x7f99c00a5d10 2020-11-20T12:19:45.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.868+0000 7f99bf7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f99c00a5930 0x7f99c014eb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57838/0 (socket says 172.21.15.116:57838) 2020-11-20T12:19:45.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.868+0000 7f99bf7fe700 1 -- 172.21.15.116:0/3756016179 learned_addr learned my addr 172.21.15.116:0/3756016179 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:45.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.868+0000 7f99bffff700 1 --2- 172.21.15.116:0/3756016179 >> v2:172.21.15.173:3300/0 conn(0x7f99c00a5d10 0x7f99c0062750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.868+0000 7f99be7fc700 1 -- 172.21.15.116:0/3756016179 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3922041380 0 0) 0x7f99c014f4b0 con 0x7f99c00a5930 2020-11-20T12:19:45.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.868+0000 7f99be7fc700 1 -- 172.21.15.116:0/3756016179 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99ac003240 con 0x7f99c00a5930 2020-11-20T12:19:45.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.868+0000 7f99bffff700 1 -- 172.21.15.116:0/3756016179 >> v1:172.21.15.173:6789/0 conn(0x7f99c0152d40 legacy=0x7f99c0067690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.868+0000 7f99bffff700 1 -- 172.21.15.116:0/3756016179 >> v1:172.21.15.116:6789/0 conn(0x7f99c00a5930 legacy=0x7f99c014eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.868+0000 7f99bffff700 1 -- 172.21.15.116:0/3756016179 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00662b0 con 0x7f99c00a5d10 2020-11-20T12:19:45.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.868+0000 7f99bffff700 1 --2- 172.21.15.116:0/3756016179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c00a5d10 0x7f99c0062750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f99b8009c90 tx=0x7f99b80091d0).ready entity=mon.1 client_cookie=15c0d850787010bc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:45.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.872+0000 7f99be7fc700 1 -- 172.21.15.116:0/3756016179 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b800ba70 con 0x7f99c00a5d10 2020-11-20T12:19:45.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.872+0000 7f99be7fc700 1 -- 172.21.15.116:0/3756016179 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b800bbd0 con 0x7f99c00a5d10 2020-11-20T12:19:45.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.872+0000 7f99be7fc700 1 -- 172.21.15.116:0/3756016179 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8011400 con 0x7f99c00a5d10 2020-11-20T12:19:45.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.872+0000 7f99c496e700 1 -- 172.21.15.116:0/3756016179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c00a5d10 msgr2=0x7f99c0062750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.872+0000 7f99c496e700 1 --2- 172.21.15.116:0/3756016179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c00a5d10 0x7f99c0062750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f99b8009c90 tx=0x7f99b80091d0).stop 2020-11-20T12:19:45.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.872+0000 7f99c496e700 1 -- 172.21.15.116:0/3756016179 shutdown_connections 2020-11-20T12:19:45.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.872+0000 7f99c496e700 1 --2- 172.21.15.116:0/3756016179 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c00a5d10 0x7f99c0062750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.872+0000 7f99c496e700 1 -- 172.21.15.116:0/3756016179 >> 172.21.15.116:0/3756016179 conn(0x7f99c0060170 msgr2=0x7f99c0060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:45.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.872+0000 7f99c496e700 1 -- 172.21.15.116:0/3756016179 shutdown_connections 2020-11-20T12:19:45.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.872+0000 7f99c496e700 1 -- 172.21.15.116:0/3756016179 wait complete. 2020-11-20T12:19:45.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99c496e700 1 Processor -- start 2020-11-20T12:19:45.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99c496e700 1 -- start start 2020-11-20T12:19:45.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99c496e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99c00a5930 0x7f99c014b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99c496e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c00a5d10 0x7f99c00c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99c496e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c0152d40 0x7f99c0162270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99c496e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00c4e50 con 0x7f99c0152d40 2020-11-20T12:19:45.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99c496e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00c50f0 con 0x7f99c00a5d10 2020-11-20T12:19:45.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99c496e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0154190 con 0x7f99c00a5930 2020-11-20T12:19:45.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99bffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c0152d40 0x7f99c0162270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99bffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c0152d40 0x7f99c0162270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48414/0 (socket says 172.21.15.116:48414) 2020-11-20T12:19:45.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99bffff700 1 -- 172.21.15.116:0/1747655140 learned_addr learned my addr 172.21.15.116:0/1747655140 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:45.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99bf7fe700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99c00a5930 0x7f99c014b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.876+0000 7f99beffd700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c00a5d10 0x7f99c00c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f99bffff700 1 -- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99c00a5930 msgr2=0x7f99c014b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f99bffff700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99c00a5930 0x7f99c014b180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f99bffff700 1 -- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c00a5d10 msgr2=0x7f99c00c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:45.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f99bffff700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c00a5d10 0x7f99c00c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:45.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f99bffff700 1 -- 172.21.15.116:0/1747655140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0154430 con 0x7f99c0152d40 2020-11-20T12:19:45.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f99bffff700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c0152d40 0x7f99c0162270 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f99b800c010 tx=0x7f99b8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:45.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f999f7fe700 1 -- 172.21.15.116:0/1747655140 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b801c5f0 con 0x7f99c0152d40 2020-11-20T12:19:45.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f999f7fe700 1 -- 172.21.15.116:0/1747655140 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b801c750 con 0x7f99c0152d40 2020-11-20T12:19:45.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f99c496e700 1 -- 172.21.15.116:0/1747655140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c01546d0 con 0x7f99c0152d40 2020-11-20T12:19:45.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f999f7fe700 1 -- 172.21.15.116:0/1747655140 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b801cad0 con 0x7f99c0152d40 2020-11-20T12:19:45.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f99c496e700 1 -- 172.21.15.116:0/1747655140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c01495e0 con 0x7f99c0152d40 2020-11-20T12:19:45.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.880+0000 7f99c496e700 1 -- 172.21.15.116:0/1747655140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c0154890 con 0x7f99c0152d40 2020-11-20T12:19:45.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.884+0000 7f999f7fe700 1 -- 172.21.15.116:0/1747655140 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f99b8017070 con 0x7f99c0152d40 2020-11-20T12:19:45.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.884+0000 7f999f7fe700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99980557c0 0x7f9998057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:45.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.884+0000 7f999f7fe700 1 -- 172.21.15.116:0/1747655140 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6458+0+0 (secure 0 0 0) 0x7f99b8070c40 con 0x7f99c0152d40 2020-11-20T12:19:45.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.884+0000 7f99bf7fe700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99980557c0 0x7f9998057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:45.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.884+0000 7f99bf7fe700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99980557c0 0x7f9998057c80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f99b0007fd0 tx=0x7f99b0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:45.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:45.888+0000 7f999f7fe700 1 -- 172.21.15.116:0/1747655140 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f99b8046850 con 0x7f99c0152d40 2020-11-20T12:19:46.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.100+0000 7f99c496e700 1 -- 172.21.15.116:0/1747655140 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f99c005cd40 con 0x7f99c0152d40 2020-11-20T12:19:46.505 INFO:tasks.ceph.mon.a.smithi116.stderr:2020-11-20T12:19:46.508+0000 7fc111d1b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-20T12:19:46.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.524+0000 7f999f7fe700 1 -- 172.21.15.116:0/1747655140 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f99b8015030 con 0x7f99c0152d40 2020-11-20T12:19:46.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 -- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99980557c0 msgr2=0x7f9998057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:46.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99980557c0 0x7f9998057c80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f99b0007fd0 tx=0x7f99b0009040).stop 2020-11-20T12:19:46.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 -- 172.21.15.116:0/1747655140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c0152d40 msgr2=0x7f99c0162270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:46.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c0152d40 0x7f99c0162270 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f99b800c010 tx=0x7f99b8016000).stop 2020-11-20T12:19:46.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 -- 172.21.15.116:0/1747655140 shutdown_connections 2020-11-20T12:19:46.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99980557c0 0x7f9998057c80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:46.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99c0152d40 0x7f99c0162270 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:46.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99c00a5930 0x7f99c014b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:46.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 --2- 172.21.15.116:0/1747655140 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99c00a5d10 0x7f99c00c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:46.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 -- 172.21.15.116:0/1747655140 >> 172.21.15.116:0/1747655140 conn(0x7f99c0060170 msgr2=0x7f99c0060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:46.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 -- 172.21.15.116:0/1747655140 shutdown_connections 2020-11-20T12:19:46.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.528+0000 7f99c496e700 1 -- 172.21.15.116:0/1747655140 wait complete. 2020-11-20T12:19:46.528 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-20T12:19:46.533 INFO:tasks.ceph.mds.a.smithi116.stderr: -4353> 2020-11-20T12:19:17.368+0000 7ff8cf713700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:19:46.548 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-20T12:19:46.579 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:19:46.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.724+0000 7f3ec0b30700 1 Processor -- start 2020-11-20T12:19:46.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.724+0000 7f3ec0b30700 1 -- start start 2020-11-20T12:19:46.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc1518f0 0x7f3ebc0bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:46.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3ebc0b93c0 0x7f3ebc0c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:46.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ebc0bf510 con 0x7f3ebc151cd0 2020-11-20T12:19:46.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ebc0bf7b0 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0bfa50 con 0x7f3ebc1518f0 2020-11-20T12:19:46.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ebb7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc1518f0 0x7f3ebc0bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:46.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ebbfff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3ebc0b93c0 0x7f3ebc0c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:46.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ebb7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc1518f0 0x7f3ebc0bed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:57380/0 (socket says 172.21.15.116:57380) 2020-11-20T12:19:46.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ebbfff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3ebc0b93c0 0x7f3ebc0c0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:47486/0 (socket says 172.21.15.116:47486) 2020-11-20T12:19:46.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ebb7fe700 1 -- 172.21.15.116:0/771098325 learned_addr learned my addr 172.21.15.116:0/771098325 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:46.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ebb7fe700 1 -- 172.21.15.116:0/771098325 >> v2:172.21.15.173:3300/0 conn(0x7f3ebc0b93c0 msgr2=0x7f3ebc0c0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:46.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ebb7fe700 1 --2- 172.21.15.116:0/771098325 >> v2:172.21.15.173:3300/0 conn(0x7f3ebc0b93c0 0x7f3ebc0c0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:46.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ebb7fe700 1 -- 172.21.15.116:0/771098325 >> v1:172.21.15.173:6789/0 conn(0x7f3ebc151cd0 legacy=0x7f3ebc066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:46.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ebb7fe700 1 -- 172.21.15.116:0/771098325 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc0bfcf0 con 0x7f3ebc1518f0 2020-11-20T12:19:46.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ebb7fe700 1 --2- 172.21.15.116:0/771098325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc1518f0 0x7f3ebc0bed20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f3eac006fd0 tx=0x7f3eac006eb0).ready entity=mon.2 client_cookie=8bc6de0aab1de974 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:46.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3eba7fc700 1 -- 172.21.15.116:0/771098325 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac009810 con 0x7f3ebc1518f0 2020-11-20T12:19:46.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3eba7fc700 1 -- 172.21.15.116:0/771098325 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac009970 con 0x7f3ebc1518f0 2020-11-20T12:19:46.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3eba7fc700 1 -- 172.21.15.116:0/771098325 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac009cf0 con 0x7f3ebc1518f0 2020-11-20T12:19:46.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 -- 172.21.15.116:0/771098325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc1518f0 msgr2=0x7f3ebc0bed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:46.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 --2- 172.21.15.116:0/771098325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc1518f0 0x7f3ebc0bed20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f3eac006fd0 tx=0x7f3eac006eb0).stop 2020-11-20T12:19:46.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 -- 172.21.15.116:0/771098325 shutdown_connections 2020-11-20T12:19:46.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 --2- 172.21.15.116:0/771098325 >> v2:172.21.15.173:3300/0 conn(0x7f3ebc0b93c0 0x7f3ebc0c0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:46.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 --2- 172.21.15.116:0/771098325 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc1518f0 0x7f3ebc0bed20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:46.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 -- 172.21.15.116:0/771098325 >> 172.21.15.116:0/771098325 conn(0x7f3ebc068350 msgr2=0x7f3ebc0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:46.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.728+0000 7f3ec0b30700 1 -- 172.21.15.116:0/771098325 shutdown_connections 2020-11-20T12:19:46.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ec0b30700 1 -- 172.21.15.116:0/771098325 wait complete. 2020-11-20T12:19:46.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ec0b30700 1 Processor -- start 2020-11-20T12:19:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ec0b30700 1 -- start start 2020-11-20T12:19:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ec0b30700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ebc0b93c0 0x7f3ebc14b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:46.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ec0b30700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc151cd0 0x7f3ebc0bdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ec0b30700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ebc0be030 0x7f3ebc1545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ec0b30700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0bcc20 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ec0b30700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0bcec0 con 0x7f3ebc0be030 2020-11-20T12:19:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ec0b30700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ebc0bd160 con 0x7f3ebc151cd0 2020-11-20T12:19:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ebb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ebc0b93c0 0x7f3ebc14b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:46.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ebb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ebc0b93c0 0x7f3ebc14b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48436/0 (socket says 172.21.15.116:48436) 2020-11-20T12:19:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ebb7fe700 1 -- 172.21.15.116:0/1133946566 learned_addr learned my addr 172.21.15.116:0/1133946566 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3ebbfff700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ebc0be030 0x7f3ebc1545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:46.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.732+0000 7f3ebaffd700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc151cd0 0x7f3ebc0bdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3ebb7fe700 1 -- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc151cd0 msgr2=0x7f3ebc0bdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:46.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3ebb7fe700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc151cd0 0x7f3ebc0bdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3ebb7fe700 1 -- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ebc0be030 msgr2=0x7f3ebc1545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3ebb7fe700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ebc0be030 0x7f3ebc1545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3ebb7fe700 1 -- 172.21.15.116:0/1133946566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc0bd400 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3ebaffd700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc151cd0 0x7f3ebc0bdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3ebb7fe700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ebc0b93c0 0x7f3ebc14b0b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f3eac010000 tx=0x7f3eac015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3eb37fe700 1 -- 172.21.15.116:0/1133946566 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac011c80 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3eb37fe700 1 -- 172.21.15.116:0/1133946566 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eac011de0 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3eb37fe700 1 -- 172.21.15.116:0/1133946566 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eac01e6b0 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3ec0b30700 1 -- 172.21.15.116:0/1133946566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ebc0bd6a0 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3ec0b30700 1 -- 172.21.15.116:0/1133946566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ebc0bc830 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.736+0000 7f3ec0b30700 1 -- 172.21.15.116:0/1133946566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ebc0bd830 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.740+0000 7f3eb37fe700 1 -- 172.21.15.116:0/1133946566 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3eac025030 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.740+0000 7f3eb37fe700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e980557c0 0x7f3e98057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:46.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.740+0000 7f3eb37fe700 1 -- 172.21.15.116:0/1133946566 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6544+0+0 (secure 0 0 0) 0x7f3eac06ffe0 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.740+0000 7f3ebaffd700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e980557c0 0x7f3e98057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:46.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.740+0000 7f3ebaffd700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e980557c0 0x7f3e98057c80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f3ebc063220 tx=0x7f3ea4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:46.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.748+0000 7f3eb37fe700 1 -- 172.21.15.116:0/1133946566 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3eac045c20 con 0x7f3ebc0b93c0 2020-11-20T12:19:46.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:46.952+0000 7f3ec0b30700 1 -- 172.21.15.116:0/1133946566 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3ebc05cf30 con 0x7f3ebc0b93c0 2020-11-20T12:19:47.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.532+0000 7f3eb37fe700 1 -- 172.21.15.116:0/1133946566 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3eac023020 con 0x7f3ebc0b93c0 2020-11-20T12:19:47.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 -- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e980557c0 msgr2=0x7f3e98057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:47.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e980557c0 0x7f3e98057c80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f3ebc063220 tx=0x7f3ea4009040).stop 2020-11-20T12:19:47.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 -- 172.21.15.116:0/1133946566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ebc0b93c0 msgr2=0x7f3ebc14b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:47.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ebc0b93c0 0x7f3ebc14b0b0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f3eac010000 tx=0x7f3eac015000).stop 2020-11-20T12:19:47.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 -- 172.21.15.116:0/1133946566 shutdown_connections 2020-11-20T12:19:47.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3e980557c0 0x7f3e98057c80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:47.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3ebc0be030 0x7f3ebc1545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:47.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3ebc0b93c0 0x7f3ebc14b0b0 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:47.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 --2- 172.21.15.116:0/1133946566 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3ebc151cd0 0x7f3ebc0bdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:47.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 -- 172.21.15.116:0/1133946566 >> 172.21.15.116:0/1133946566 conn(0x7f3ebc068350 msgr2=0x7f3ebc0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:47.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 -- 172.21.15.116:0/1133946566 shutdown_connections 2020-11-20T12:19:47.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.536+0000 7f3ec0b30700 1 -- 172.21.15.116:0/1133946566 wait complete. 2020-11-20T12:19:47.561 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-20T12:19:47.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.740+0000 7f8425229700 1 Processor -- start 2020-11-20T12:19:47.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.740+0000 7f8425229700 1 -- start start 2020-11-20T12:19:47.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8425229700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8420066990 0x7f842014d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:47.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8425229700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f842014dd30 0x7f842014e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:47.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8425229700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84200a8070 con 0x7f84200665b0 2020-11-20T12:19:47.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8425229700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84200a8f90 con 0x7f842014dd30 2020-11-20T12:19:47.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8425229700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84200a8310 con 0x7f8420066990 2020-11-20T12:19:47.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f841ffff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f84200665b0 0x7f8420063040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57878/0 (socket says 172.21.15.116:57878) 2020-11-20T12:19:47.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f841ffff700 1 -- 172.21.15.116:0/447882891 learned_addr learned my addr 172.21.15.116:0/447882891 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:47.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8417fff700 1 --2- 172.21.15.116:0/447882891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8420066990 0x7f842014d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:47.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8424a28700 1 --2- 172.21.15.116:0/447882891 >> v2:172.21.15.116:3300/0 conn(0x7f842014dd30 0x7f842014e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:47.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f841f7fe700 1 -- 172.21.15.116:0/447882891 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1341406574 0 0) 0x7f84200a8070 con 0x7f84200665b0 2020-11-20T12:19:47.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f841f7fe700 1 -- 172.21.15.116:0/447882891 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8400003240 con 0x7f84200665b0 2020-11-20T12:19:47.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8424a28700 1 -- 172.21.15.116:0/447882891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8420066990 msgr2=0x7f842014d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:47.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8424a28700 1 --2- 172.21.15.116:0/447882891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8420066990 0x7f842014d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:47.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8424a28700 1 -- 172.21.15.116:0/447882891 >> v1:172.21.15.116:6789/0 conn(0x7f84200665b0 legacy=0x7f8420063040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:47.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8424a28700 1 -- 172.21.15.116:0/447882891 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84200a85b0 con 0x7f842014dd30 2020-11-20T12:19:47.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f8424a28700 1 --2- 172.21.15.116:0/447882891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842014dd30 0x7f842014e1c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f8418009c90 tx=0x7f84180091d0).ready entity=mon.0 client_cookie=3137adc3b803facb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:47.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f841f7fe700 1 -- 172.21.15.116:0/447882891 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800b9c0 con 0x7f842014dd30 2020-11-20T12:19:47.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f841f7fe700 1 -- 172.21.15.116:0/447882891 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f841800bb20 con 0x7f842014dd30 2020-11-20T12:19:47.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.744+0000 7f841f7fe700 1 -- 172.21.15.116:0/447882891 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f841800bea0 con 0x7f842014dd30 2020-11-20T12:19:47.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.748+0000 7f8425229700 1 -- 172.21.15.116:0/447882891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842014dd30 msgr2=0x7f842014e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:47.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.748+0000 7f8425229700 1 --2- 172.21.15.116:0/447882891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842014dd30 0x7f842014e1c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7f8418009c90 tx=0x7f84180091d0).stop 2020-11-20T12:19:47.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.748+0000 7f8425229700 1 -- 172.21.15.116:0/447882891 shutdown_connections 2020-11-20T12:19:47.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.748+0000 7f8425229700 1 --2- 172.21.15.116:0/447882891 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f842014dd30 0x7f842014e1c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:47.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.748+0000 7f8425229700 1 --2- 172.21.15.116:0/447882891 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8420066990 0x7f842014d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:47.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.748+0000 7f8425229700 1 -- 172.21.15.116:0/447882891 >> 172.21.15.116:0/447882891 conn(0x7f84200682e0 msgr2=0x7f8420060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:47.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.748+0000 7f8425229700 1 -- 172.21.15.116:0/447882891 shutdown_connections 2020-11-20T12:19:47.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.748+0000 7f8425229700 1 -- 172.21.15.116:0/447882891 wait complete. 2020-11-20T12:19:47.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.748+0000 7f8425229700 1 Processor -- start 2020-11-20T12:19:47.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.748+0000 7f8425229700 1 -- start start 2020-11-20T12:19:47.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8425229700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84200665b0 0x7f84201467e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:47.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8425229700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8420066990 0x7f8420165e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:47.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8425229700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f842014dd30 0x7f84201681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:47.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8425229700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8420166320 con 0x7f8420066990 2020-11-20T12:19:47.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8425229700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8420146e80 con 0x7f842014dd30 2020-11-20T12:19:47.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8425229700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8420147120 con 0x7f84200665b0 2020-11-20T12:19:47.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8417fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8420066990 0x7f8420165e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:47.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8417fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8420066990 0x7f8420165e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48452/0 (socket says 172.21.15.116:48452) 2020-11-20T12:19:47.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8417fff700 1 -- 172.21.15.116:0/1392615885 learned_addr learned my addr 172.21.15.116:0/1392615885 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:47.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f841ffff700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84200665b0 0x7f84201467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:47.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8424a28700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f842014dd30 0x7f84201681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:47.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8417fff700 1 -- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84200665b0 msgr2=0x7f84201467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:47.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8417fff700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84200665b0 0x7f84201467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:47.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8417fff700 1 -- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f842014dd30 msgr2=0x7f84201681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:47.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8417fff700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f842014dd30 0x7f84201681b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:47.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8417fff700 1 -- 172.21.15.116:0/1392615885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84201473c0 con 0x7f8420066990 2020-11-20T12:19:47.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8417fff700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8420066990 0x7f8420165e30 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7f840800eba0 tx=0x7f8408006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:47.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f841cff9700 1 -- 172.21.15.116:0/1392615885 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800c8e0 con 0x7f8420066990 2020-11-20T12:19:47.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f841cff9700 1 -- 172.21.15.116:0/1392615885 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840800ca40 con 0x7f8420066990 2020-11-20T12:19:47.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8425229700 1 -- 172.21.15.116:0/1392615885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8420147660 con 0x7f8420066990 2020-11-20T12:19:47.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f841cff9700 1 -- 172.21.15.116:0/1392615885 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840800cdc0 con 0x7f8420066990 2020-11-20T12:19:47.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.752+0000 7f8425229700 1 -- 172.21.15.116:0/1392615885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84201686a0 con 0x7f8420066990 2020-11-20T12:19:47.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.756+0000 7f8425229700 1 -- 172.21.15.116:0/1392615885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842005cf30 con 0x7f8420066990 2020-11-20T12:19:47.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.756+0000 7f841cff9700 1 -- 172.21.15.116:0/1392615885 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8408012600 con 0x7f8420066990 2020-11-20T12:19:47.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.756+0000 7f841cff9700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83fc0557f0 0x7f83fc057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:47.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.756+0000 7f841cff9700 1 -- 172.21.15.116:0/1392615885 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6544+0+0 (secure 0 0 0) 0x7f840806ebe0 con 0x7f8420066990 2020-11-20T12:19:47.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.756+0000 7f841ffff700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83fc0557f0 0x7f83fc057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:47.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.760+0000 7f841ffff700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83fc0557f0 0x7f83fc057cb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f8410007fd0 tx=0x7f8410009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:47.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.764+0000 7f841cff9700 1 -- 172.21.15.116:0/1392615885 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8408044850 con 0x7f8420066990 2020-11-20T12:19:47.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:47.920+0000 7f8425229700 1 -- 172.21.15.116:0/1392615885 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8420062300 con 0x7f8420066990 2020-11-20T12:19:48.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.540+0000 7f841cff9700 1 -- 172.21.15.116:0/1392615885 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8408047e70 con 0x7f8420066990 2020-11-20T12:19:48.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 -- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83fc0557f0 msgr2=0x7f83fc057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:48.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83fc0557f0 0x7f83fc057cb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f8410007fd0 tx=0x7f8410009040).stop 2020-11-20T12:19:48.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 -- 172.21.15.116:0/1392615885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8420066990 msgr2=0x7f8420165e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:48.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8420066990 0x7f8420165e30 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7f840800eba0 tx=0x7f8408006df0).stop 2020-11-20T12:19:48.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 -- 172.21.15.116:0/1392615885 shutdown_connections 2020-11-20T12:19:48.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f83fc0557f0 0x7f83fc057cb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:48.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f842014dd30 0x7f84201681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:48.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84200665b0 0x7f84201467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:48.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 --2- 172.21.15.116:0/1392615885 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8420066990 0x7f8420165e30 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:48.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 -- 172.21.15.116:0/1392615885 >> 172.21.15.116:0/1392615885 conn(0x7f84200682e0 msgr2=0x7f84200c33b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:48.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 -- 172.21.15.116:0/1392615885 shutdown_connections 2020-11-20T12:19:48.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.544+0000 7f8425229700 1 -- 172.21.15.116:0/1392615885 wait complete. 2020-11-20T12:19:48.543 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2020-11-20T12:19:48.565 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-20T12:19:48.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.748+0000 7fc19b59e700 1 Processor -- start 2020-11-20T12:19:48.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.752+0000 7fc19b59e700 1 -- start start 2020-11-20T12:19:48.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.752+0000 7fc19b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc19c061670 0x7fc19c061b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.752+0000 7fc19b59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19c0ab240 con 0x7fc19c14bf60 2020-11-20T12:19:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.752+0000 7fc19b59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19c0aafa0 con 0x7fc19c14bb80 2020-11-20T12:19:48.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.752+0000 7fc19b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0acce0 con 0x7fc19c061670 2020-11-20T12:19:48.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc199d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc19c14bf60 0x7fc19c0aa540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57894/0 (socket says 172.21.15.116:57894) 2020-11-20T12:19:48.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc199d9b700 1 -- 172.21.15.116:0/3411785875 learned_addr learned my addr 172.21.15.116:0/3411785875 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:48.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19ad9d700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc19c061670 0x7fc19c061b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:48.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907995195 0 0) 0x7fc19c0ab240 con 0x7fc19c14bf60 2020-11-20T12:19:48.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc180003240 con 0x7fc19c14bf60 2020-11-20T12:19:48.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1917454214 0 0) 0x7fc180003240 con 0x7fc19c14bf60 2020-11-20T12:19:48.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc19c061670 msgr2=0x7fc19c061b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:48.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc19c061670 0x7fc19c061b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:48.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 >> v1:172.21.15.173:6789/0 conn(0x7fc19c14bb80 legacy=0x7fc19c0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:48.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c0aced0 con 0x7fc19c14bf60 2020-11-20T12:19:48.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fc1900014e0 con 0x7fc19c14bf60 2020-11-20T12:19:48.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 >> v1:172.21.15.116:6789/0 conn(0x7fc19c14bf60 legacy=0x7fc19c0aa540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:48.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc180003760 0x7fc180003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:48.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1800044e0 0x7fc180004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc180004e50 0x7fc18000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc180004240 con 0x7fc180003760 2020-11-20T12:19:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18000bc20 con 0x7fc180004e50 2020-11-20T12:19:48.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18000bd60 con 0x7fc1800044e0 2020-11-20T12:19:48.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc199d9b700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1800044e0 0x7fc180004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:48.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19ad9d700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc180004e50 0x7fc18000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:48.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc19a59c700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc180003760 0x7fc180003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:48.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc199d9b700 1 -- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc180004e50 msgr2=0x7fc18000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:48.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc199d9b700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc180004e50 0x7fc18000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:48.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc199d9b700 1 -- 172.21.15.116:0/3411785875 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc180003760 msgr2=0x7fc180003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:48.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc199d9b700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc180003760 0x7fc180003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:48.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.756+0000 7fc199d9b700 1 -- 172.21.15.116:0/3411785875 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc180003240 con 0x7fc1800044e0 2020-11-20T12:19:48.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.760+0000 7fc199d9b700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1800044e0 0x7fc180004960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fc19000c200 tx=0x7fc19c0ab130).ready entity=mon.2 client_cookie=575541485f0e58a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:48.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.760+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19000cdf0 con 0x7fc1800044e0 2020-11-20T12:19:48.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.760+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1900070d0 con 0x7fc1800044e0 2020-11-20T12:19:48.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.760+0000 7fc19959a700 1 -- 172.21.15.116:0/3411785875 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc190007470 con 0x7fc1800044e0 2020-11-20T12:19:48.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.760+0000 7fc19b59e700 1 -- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1800044e0 msgr2=0x7fc180004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:48.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.760+0000 7fc19b59e700 1 --2- 172.21.15.116:0/3411785875 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1800044e0 0x7fc180004960 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fc19000c200 tx=0x7fc19c0ab130).stop 2020-11-20T12:19:48.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.760+0000 7fc19ad9d700 1 -- 172.21.15.116:0/3411785875 reap_dead start 2020-11-20T12:19:48.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.760+0000 7fc19b59e700 1 -- 172.21.15.116:0/3411785875 shutdown_connections 2020-11-20T12:19:48.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.760+0000 7fc19b59e700 1 -- 172.21.15.116:0/3411785875 >> 172.21.15.116:0/3411785875 conn(0x7fc19c0678a0 msgr2=0x7fc19c067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:48.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.760+0000 7fc19b59e700 1 -- 172.21.15.116:0/3411785875 shutdown_connections 2020-11-20T12:19:48.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.760+0000 7fc19b59e700 1 -- 172.21.15.116:0/3411785875 wait complete. 2020-11-20T12:19:48.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19b59e700 1 Processor -- start 2020-11-20T12:19:48.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19b59e700 1 -- start start 2020-11-20T12:19:48.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1800044e0 0x7fc19c0a8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:48.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc19c153400 0x7fc19c155cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:48.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19c1561e0 0x7fc19c148ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:48.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0a90f0 con 0x7fc19c1561e0 2020-11-20T12:19:48.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0a9390 con 0x7fc19c153400 2020-11-20T12:19:48.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0a9630 con 0x7fc1800044e0 2020-11-20T12:19:48.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19c1561e0 0x7fc19c148ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19c1561e0 0x7fc19c148ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48474/0 (socket says 172.21.15.116:48474) 2020-11-20T12:19:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19a59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1800044e0 0x7fc19c0a8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc19ad9d700 1 -- 172.21.15.116:0/60515702 learned_addr learned my addr 172.21.15.116:0/60515702 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.764+0000 7fc199d9b700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc19c153400 0x7fc19c155cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:48.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc19ad9d700 1 -- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1800044e0 msgr2=0x7fc19c0a8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:48.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc19ad9d700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1800044e0 0x7fc19c0a8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:48.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc19ad9d700 1 -- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc19c153400 msgr2=0x7fc19c155cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:48.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc19ad9d700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc19c153400 0x7fc19c155cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:48.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc19ad9d700 1 -- 172.21.15.116:0/60515702 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c161600 con 0x7fc19c1561e0 2020-11-20T12:19:48.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc19ad9d700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19c1561e0 0x7fc19c148ee0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fc19c069dc0 tx=0x7fc194008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:48.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc18a7fc700 1 -- 172.21.15.116:0/60515702 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc19400c070 con 0x7fc19c1561e0 2020-11-20T12:19:48.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc18a7fc700 1 -- 172.21.15.116:0/60515702 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1940136e0 con 0x7fc19c1561e0 2020-11-20T12:19:48.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc19b59e700 1 -- 172.21.15.116:0/60515702 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19c161320 con 0x7fc19c1561e0 2020-11-20T12:19:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc18a7fc700 1 -- 172.21.15.116:0/60515702 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc194013a80 con 0x7fc19c1561e0 2020-11-20T12:19:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc19b59e700 1 -- 172.21.15.116:0/60515702 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c1493d0 con 0x7fc19c1561e0 2020-11-20T12:19:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.768+0000 7fc19b59e700 1 -- 172.21.15.116:0/60515702 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c151b20 con 0x7fc19c1561e0 2020-11-20T12:19:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.772+0000 7fc18a7fc700 1 -- 172.21.15.116:0/60515702 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc194012040 con 0x7fc19c1561e0 2020-11-20T12:19:48.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.772+0000 7fc18a7fc700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1700557c0 0x7fc170057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:48.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.772+0000 7fc18a7fc700 1 -- 172.21.15.116:0/60515702 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 6106+0+0 (secure 0 0 0) 0x7fc19406e660 con 0x7fc19c1561e0 2020-11-20T12:19:48.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.772+0000 7fc19a59c700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1700557c0 0x7fc170057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:48.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.772+0000 7fc19a59c700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1700557c0 0x7fc170057c80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fc18c007fd0 tx=0x7fc18c007b90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:48.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.776+0000 7fc18a7fc700 1 -- 172.21.15.116:0/60515702 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc19c151b20 con 0x7fc19c1561e0 2020-11-20T12:19:48.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:48.936+0000 7fc19b59e700 1 -- 172.21.15.116:0/60515702 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc19c151b20 con 0x7fc19c1561e0 2020-11-20T12:19:49.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.552+0000 7fc18a7fc700 1 -- 172.21.15.116:0/60515702 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc19c151b20 con 0x7fc19c1561e0 2020-11-20T12:19:49.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 -- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1700557c0 msgr2=0x7fc170057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:49.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1700557c0 0x7fc170057c80 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fc18c007fd0 tx=0x7fc18c007b90).stop 2020-11-20T12:19:49.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 -- 172.21.15.116:0/60515702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19c1561e0 msgr2=0x7fc19c148ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:49.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19c1561e0 0x7fc19c148ee0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7fc19c069dc0 tx=0x7fc194008040).stop 2020-11-20T12:19:49.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 -- 172.21.15.116:0/60515702 shutdown_connections 2020-11-20T12:19:49.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1700557c0 0x7fc170057c80 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:49.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc19c1561e0 0x7fc19c148ee0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:49.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc1800044e0 0x7fc19c0a8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:49.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 --2- 172.21.15.116:0/60515702 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc19c153400 0x7fc19c155cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:49.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 -- 172.21.15.116:0/60515702 >> 172.21.15.116:0/60515702 conn(0x7fc19c0678a0 msgr2=0x7fc19c14c580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:49.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 -- 172.21.15.116:0/60515702 shutdown_connections 2020-11-20T12:19:49.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.560+0000 7fc19b59e700 1 -- 172.21.15.116:0/60515702 wait complete. 2020-11-20T12:19:49.560 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2020-11-20T12:19:49.583 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm fsvolume_volid fsvolume_volid --yes-i-really-really-mean-it 2020-11-20T12:19:49.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.772+0000 7fa061188700 1 Processor -- start 2020-11-20T12:19:49.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.772+0000 7fa061188700 1 -- start start 2020-11-20T12:19:49.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.772+0000 7fa061188700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa05c0a7c10 0x7fa05c0c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:49.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.772+0000 7fa061188700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa05c1515f0 0x7fa05c151a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:49.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.772+0000 7fa061188700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa05c0c5140 con 0x7fa05c0a7830 2020-11-20T12:19:49.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.772+0000 7fa061188700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa05c0c3d30 con 0x7fa05c1515f0 2020-11-20T12:19:49.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.772+0000 7fa061188700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c0c3fd0 con 0x7fa05c0a7c10 2020-11-20T12:19:49.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa05b59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa05c1515f0 0x7fa05c151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:49.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa05b59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa05c1515f0 0x7fa05c151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:47532/0 (socket says 172.21.15.116:47532) 2020-11-20T12:19:49.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa05b59e700 1 -- 172.21.15.116:0/4074458654 learned_addr learned my addr 172.21.15.116:0/4074458654 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:49.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa05a59c700 1 --2- 172.21.15.116:0/4074458654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa05c0a7c10 0x7fa05c0c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:49.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa05b59e700 1 -- 172.21.15.116:0/4074458654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa05c0a7c10 msgr2=0x7fa05c0c4c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:49.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa05b59e700 1 --2- 172.21.15.116:0/4074458654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa05c0a7c10 0x7fa05c0c4c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:49.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa05b59e700 1 -- 172.21.15.116:0/4074458654 >> v1:172.21.15.173:6789/0 conn(0x7fa05c0a7830 legacy=0x7fa05c0be110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:49.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa05b59e700 1 -- 172.21.15.116:0/4074458654 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c0c4270 con 0x7fa05c1515f0 2020-11-20T12:19:49.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa05b59e700 1 --2- 172.21.15.116:0/4074458654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa05c1515f0 0x7fa05c151a40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fa054009d50 tx=0x7fa054009290).ready entity=mon.1 client_cookie=3cd1fd8c0972b027 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:49.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa059d9b700 1 -- 172.21.15.116:0/4074458654 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05400bc80 con 0x7fa05c1515f0 2020-11-20T12:19:49.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa059d9b700 1 -- 172.21.15.116:0/4074458654 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa05400bde0 con 0x7fa05c1515f0 2020-11-20T12:19:49.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa059d9b700 1 -- 172.21.15.116:0/4074458654 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054011610 con 0x7fa05c1515f0 2020-11-20T12:19:49.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa061188700 1 -- 172.21.15.116:0/4074458654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa05c1515f0 msgr2=0x7fa05c151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:49.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa061188700 1 --2- 172.21.15.116:0/4074458654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa05c1515f0 0x7fa05c151a40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7fa054009d50 tx=0x7fa054009290).stop 2020-11-20T12:19:49.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa061188700 1 -- 172.21.15.116:0/4074458654 shutdown_connections 2020-11-20T12:19:49.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa061188700 1 --2- 172.21.15.116:0/4074458654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa05c1515f0 0x7fa05c151a40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:49.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa061188700 1 --2- 172.21.15.116:0/4074458654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa05c0a7c10 0x7fa05c0c4c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:49.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa061188700 1 -- 172.21.15.116:0/4074458654 >> 172.21.15.116:0/4074458654 conn(0x7fa05c068320 msgr2=0x7fa05c0c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:49.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa061188700 1 -- 172.21.15.116:0/4074458654 shutdown_connections 2020-11-20T12:19:49.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.776+0000 7fa061188700 1 -- 172.21.15.116:0/4074458654 wait complete. 2020-11-20T12:19:49.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.780+0000 7fa061188700 1 Processor -- start 2020-11-20T12:19:49.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.780+0000 7fa061188700 1 -- start start 2020-11-20T12:19:49.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.780+0000 7fa061188700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa05c0a7830 0x7fa05c0adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:49.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.780+0000 7fa061188700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa05c0a7c10 0x7fa05c1697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:49.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.780+0000 7fa061188700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05c1515f0 0x7fa05c152ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:49.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.780+0000 7fa061188700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c169ce0 con 0x7fa05c1515f0 2020-11-20T12:19:49.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.780+0000 7fa061188700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c0c0df0 con 0x7fa05c0a7830 2020-11-20T12:19:49.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.780+0000 7fa061188700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa05c0c0be0 con 0x7fa05c0a7c10 2020-11-20T12:19:49.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05c1515f0 0x7fa05c152ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:49.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05c1515f0 0x7fa05c152ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48486/0 (socket says 172.21.15.116:48486) 2020-11-20T12:19:49.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05b59e700 1 -- 172.21.15.116:0/2058894674 learned_addr learned my addr 172.21.15.116:0/2058894674 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:49.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05ad9d700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa05c0a7830 0x7fa05c0adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:49.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05a59c700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa05c0a7c10 0x7fa05c1697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:49.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05b59e700 1 -- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa05c0a7c10 msgr2=0x7fa05c1697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:49.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05b59e700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa05c0a7c10 0x7fa05c1697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:49.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05b59e700 1 -- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa05c0a7830 msgr2=0x7fa05c0adee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:49.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05b59e700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa05c0a7830 0x7fa05c0adee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:49.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05b59e700 1 -- 172.21.15.116:0/2058894674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa05c0c1020 con 0x7fa05c1515f0 2020-11-20T12:19:49.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05a59c700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa05c0a7c10 0x7fa05c1697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:49.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa05b59e700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05c1515f0 0x7fa05c152ce0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fa0540092d0 tx=0x7fa054016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:49.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa042ffd700 1 -- 172.21.15.116:0/2058894674 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa05400bc80 con 0x7fa05c1515f0 2020-11-20T12:19:49.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa042ffd700 1 -- 172.21.15.116:0/2058894674 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa054009e90 con 0x7fa05c1515f0 2020-11-20T12:19:49.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa042ffd700 1 -- 172.21.15.116:0/2058894674 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa054010dc0 con 0x7fa05c1515f0 2020-11-20T12:19:49.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa061188700 1 -- 172.21.15.116:0/2058894674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05c0c12c0 con 0x7fa05c1515f0 2020-11-20T12:19:49.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.784+0000 7fa061188700 1 -- 172.21.15.116:0/2058894674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05c1531d0 con 0x7fa05c1515f0 2020-11-20T12:19:49.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.788+0000 7fa061188700 1 -- 172.21.15.116:0/2058894674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa05c05d2a0 con 0x7fa05c1515f0 2020-11-20T12:19:49.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.788+0000 7fa042ffd700 1 -- 172.21.15.116:0/2058894674 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa054017070 con 0x7fa05c1515f0 2020-11-20T12:19:49.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.788+0000 7fa042ffd700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0380557c0 0x7fa038057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:49.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.788+0000 7fa042ffd700 1 -- 172.21.15.116:0/2058894674 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5724+0+0 (secure 0 0 0) 0x7fa054070760 con 0x7fa05c1515f0 2020-11-20T12:19:49.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.788+0000 7fa05ad9d700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0380557c0 0x7fa038057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:49.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.792+0000 7fa05ad9d700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0380557c0 0x7fa038057c80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fa04c003610 tx=0x7fa04c000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:49.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.796+0000 7fa042ffd700 1 -- 172.21.15.116:0/2058894674 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa054046650 con 0x7fa05c1515f0 2020-11-20T12:19:49.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:49.952+0000 7fa061188700 1 -- 172.21.15.116:0/2058894674 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "fsvolume_volid", "pool2": "fsvolume_volid", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa05c0a8130 con 0x7fa05c1515f0 2020-11-20T12:19:50.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.556+0000 7fa042ffd700 1 -- 172.21.15.116:0/2058894674 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "fsvolume_volid", "pool2": "fsvolume_volid", "yes_i_really_really_mean_it": true}]=0 pool 'fsvolume_volid' removed v78) v1 ==== 178+0+0 (secure 0 0 0) 0x7fa0540150d0 con 0x7fa05c1515f0 2020-11-20T12:19:50.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 -- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0380557c0 msgr2=0x7fa038057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:50.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0380557c0 0x7fa038057c80 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fa04c003610 tx=0x7fa04c000b80).stop 2020-11-20T12:19:50.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 -- 172.21.15.116:0/2058894674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05c1515f0 msgr2=0x7fa05c152ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:50.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05c1515f0 0x7fa05c152ce0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7fa0540092d0 tx=0x7fa054016000).stop 2020-11-20T12:19:50.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 -- 172.21.15.116:0/2058894674 shutdown_connections 2020-11-20T12:19:50.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa0380557c0 0x7fa038057c80 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:50.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa05c1515f0 0x7fa05c152ce0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:50.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa05c0a7830 0x7fa05c0adee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:50.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 --2- 172.21.15.116:0/2058894674 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa05c0a7c10 0x7fa05c1697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:50.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 -- 172.21.15.116:0/2058894674 >> 172.21.15.116:0/2058894674 conn(0x7fa05c068320 msgr2=0x7fa05c0a88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:50.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 -- 172.21.15.116:0/2058894674 shutdown_connections 2020-11-20T12:19:50.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.564+0000 7fa061188700 1 -- 172.21.15.116:0/2058894674 wait complete. 2020-11-20T12:19:50.568 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'fsvolume_volid' removed 2020-11-20T12:19:50.590 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T12:19:50.590 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-20T12:19:50.591 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-20T12:19:50.591 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:19:50.591 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-20T12:19:50.592 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-20T12:19:50.592 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-20T12:19:50.592 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:19:50.592 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T12:19:50.593 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-20T12:19:50.593 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-20T12:19:50.593 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:19:50.594 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-20T12:19:50.594 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-20T12:19:50.594 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-20T12:19:50.594 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:19:50.617 INFO:tasks.ceph.mds.a:Stopped 2020-11-20T12:19:50.618 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T12:19:50.620 INFO:tasks.ceph.mds.b:Stopped 2020-11-20T12:19:50.620 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T12:19:50.620 INFO:tasks.ceph.mds.c:Stopped 2020-11-20T12:19:50.621 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-20T12:19:50.622 INFO:tasks.ceph.mds.d:Stopped 2020-11-20T12:19:50.622 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-20T12:19:50.626 INFO:tasks.ceph.mds.a:Started 2020-11-20T12:19:50.627 DEBUG:teuthology.parallel:result is None 2020-11-20T12:19:50.629 INFO:tasks.ceph.mds.c:Started 2020-11-20T12:19:50.630 DEBUG:teuthology.parallel:result is None 2020-11-20T12:19:50.675 INFO:tasks.ceph.mds.b:Started 2020-11-20T12:19:50.677 DEBUG:teuthology.parallel:result is None 2020-11-20T12:19:50.680 INFO:tasks.ceph.mds.d:Started 2020-11-20T12:19:50.681 DEBUG:teuthology.parallel:result is None 2020-11-20T12:19:50.682 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-20T12:19:50.727 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:19:50.734 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:19:50.776 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2020-11-20T12:19:50.784 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2020-11-20T12:19:50.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.856+0000 7f42f33cd700 1 Processor -- start 2020-11-20T12:19:50.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.856+0000 7f42f33cd700 1 -- start start 2020-11-20T12:19:50.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.856+0000 7f42f33cd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f42ec0a6a20 0x7f42ec0c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:50.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.856+0000 7f42f33cd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f42ec152d20 0x7f42ec0c22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:50.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.856+0000 7f42f33cd700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ec152110 con 0x7f42ec063860 2020-11-20T12:19:50.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.856+0000 7f42f33cd700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ec064530 con 0x7f42ec152d20 2020-11-20T12:19:50.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.856+0000 7f42f33cd700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ec152340 con 0x7f42ec0a6a20 2020-11-20T12:19:50.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.856+0000 7f42f2bcc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f42ec063860 0x7f42ec063cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57954/0 (socket says 172.21.15.116:57954) 2020-11-20T12:19:50.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.856+0000 7f42f2bcc700 1 -- 172.21.15.116:0/1189233761 learned_addr learned my addr 172.21.15.116:0/1189233761 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:50.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.856+0000 7f42f1bca700 1 --2- 172.21.15.116:0/1189233761 >> v2:172.21.15.116:3300/0 conn(0x7f42ec152d20 0x7f42ec0c22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:50.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.856+0000 7f42f23cb700 1 --2- 172.21.15.116:0/1189233761 >> v2:172.21.15.173:3300/0 conn(0x7f42ec0a6a20 0x7f42ec0c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:50.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f13c9700 1 -- 172.21.15.116:0/1189233761 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454066666 0 0) 0x7f42ec152110 con 0x7f42ec063860 2020-11-20T12:19:50.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f13c9700 1 -- 172.21.15.116:0/1189233761 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d4003240 con 0x7f42ec063860 2020-11-20T12:19:50.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f1bca700 1 -- 172.21.15.116:0/1189233761 >> v2:172.21.15.173:3300/0 conn(0x7f42ec0a6a20 msgr2=0x7f42ec0c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:50.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f1bca700 1 --2- 172.21.15.116:0/1189233761 >> v2:172.21.15.173:3300/0 conn(0x7f42ec0a6a20 0x7f42ec0c1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:50.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f1bca700 1 -- 172.21.15.116:0/1189233761 >> v1:172.21.15.116:6789/0 conn(0x7f42ec063860 legacy=0x7f42ec063cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:50.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f1bca700 1 -- 172.21.15.116:0/1189233761 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec1525e0 con 0x7f42ec152d20 2020-11-20T12:19:50.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f23cb700 1 --2- 172.21.15.116:0/1189233761 >> v2:172.21.15.173:3300/0 conn(0x7f42ec0a6a20 0x7f42ec0c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:50.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f1bca700 1 --2- 172.21.15.116:0/1189233761 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42ec152d20 0x7f42ec0c22b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f42ec0c5330 tx=0x7f42dc006df0).ready entity=mon.0 client_cookie=407351bd242aa057 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:50.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f13c9700 1 -- 172.21.15.116:0/1189233761 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0097b0 con 0x7f42ec152d20 2020-11-20T12:19:50.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f13c9700 1 -- 172.21.15.116:0/1189233761 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42dc009910 con 0x7f42ec152d20 2020-11-20T12:19:50.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f13c9700 1 -- 172.21.15.116:0/1189233761 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc009c90 con 0x7f42ec152d20 2020-11-20T12:19:50.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f33cd700 1 -- 172.21.15.116:0/1189233761 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42ec152d20 msgr2=0x7f42ec0c22b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:50.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f33cd700 1 --2- 172.21.15.116:0/1189233761 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42ec152d20 0x7f42ec0c22b0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f42ec0c5330 tx=0x7f42dc006df0).stop 2020-11-20T12:19:50.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f33cd700 1 -- 172.21.15.116:0/1189233761 shutdown_connections 2020-11-20T12:19:50.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f33cd700 1 --2- 172.21.15.116:0/1189233761 >> v2:172.21.15.173:3300/0 conn(0x7f42ec0a6a20 0x7f42ec0c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:50.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f33cd700 1 --2- 172.21.15.116:0/1189233761 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42ec152d20 0x7f42ec0c22b0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:50.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f33cd700 1 -- 172.21.15.116:0/1189233761 >> 172.21.15.116:0/1189233761 conn(0x7f42ec061280 msgr2=0x7f42ec061660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:50.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f33cd700 1 -- 172.21.15.116:0/1189233761 shutdown_connections 2020-11-20T12:19:50.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.860+0000 7f42f33cd700 1 -- 172.21.15.116:0/1189233761 wait complete. 2020-11-20T12:19:50.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.864+0000 7f42f33cd700 1 Processor -- start 2020-11-20T12:19:50.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.864+0000 7f42f33cd700 1 -- start start 2020-11-20T12:19:50.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.864+0000 7f42f33cd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42ec063860 0x7f42ec160ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:50.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.864+0000 7f42f33cd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42ec0a6a20 0x7f42ec0bfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:50.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.864+0000 7f42f33cd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42ec152d20 0x7f42ec0a9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:50.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.864+0000 7f42f33cd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0beb50 con 0x7f42ec152d20 2020-11-20T12:19:50.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.864+0000 7f42f33cd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0bedf0 con 0x7f42ec063860 2020-11-20T12:19:50.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.864+0000 7f42f33cd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec0bf090 con 0x7f42ec0a6a20 2020-11-20T12:19:50.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f2bcc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42ec152d20 0x7f42ec0a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:50.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f2bcc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42ec152d20 0x7f42ec0a9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48526/0 (socket says 172.21.15.116:48526) 2020-11-20T12:19:50.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f2bcc700 1 -- 172.21.15.116:0/741243166 learned_addr learned my addr 172.21.15.116:0/741243166 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:50.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f23cb700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42ec063860 0x7f42ec160ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:50.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f1bca700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42ec0a6a20 0x7f42ec0bfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:50.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f2bcc700 1 -- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42ec0a6a20 msgr2=0x7f42ec0bfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:50.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f2bcc700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42ec0a6a20 0x7f42ec0bfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:50.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f2bcc700 1 -- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42ec063860 msgr2=0x7f42ec160ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:50.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f2bcc700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42ec063860 0x7f42ec160ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:50.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f2bcc700 1 -- 172.21.15.116:0/741243166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec0bf330 con 0x7f42ec152d20 2020-11-20T12:19:50.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f2bcc700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42ec152d20 0x7f42ec0a9720 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f42e400ca60 tx=0x7f42e4007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:50.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42da7fc700 1 -- 172.21.15.116:0/741243166 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4005b80 con 0x7f42ec152d20 2020-11-20T12:19:50.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42da7fc700 1 -- 172.21.15.116:0/741243166 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e4005ce0 con 0x7f42ec152d20 2020-11-20T12:19:50.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f33cd700 1 -- 172.21.15.116:0/741243166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ec0bf5d0 con 0x7f42ec152d20 2020-11-20T12:19:50.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42da7fc700 1 -- 172.21.15.116:0/741243166 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4007340 con 0x7f42ec152d20 2020-11-20T12:19:50.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f33cd700 1 -- 172.21.15.116:0/741243166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec0bff40 con 0x7f42ec152d20 2020-11-20T12:19:50.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.868+0000 7f42f33cd700 1 -- 172.21.15.116:0/741243166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ec05cd40 con 0x7f42ec152d20 2020-11-20T12:19:50.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.872+0000 7f42da7fc700 1 -- 172.21.15.116:0/741243166 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f42e4010030 con 0x7f42ec152d20 2020-11-20T12:19:50.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.872+0000 7f42da7fc700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f42c40557c0 0x7f42c4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:50.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.872+0000 7f42da7fc700 1 -- 172.21.15.116:0/741243166 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f42e406e390 con 0x7f42ec152d20 2020-11-20T12:19:50.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.872+0000 7f42f23cb700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f42c40557c0 0x7f42c4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:50.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.872+0000 7f42f23cb700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f42c40557c0 0x7f42c4057c80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f42e800ba80 tx=0x7f42e8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:50.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:50.880+0000 7f42da7fc700 1 -- 172.21.15.116:0/741243166 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f42e4044400 con 0x7f42ec152d20 2020-11-20T12:19:51.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.052+0000 7f42f33cd700 1 -- 172.21.15.116:0/741243166 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f42ec063020 con 0x7f42ec152d20 2020-11-20T12:19:51.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.836+0000 7f42da7fc700 1 -- 172.21.15.116:0/741243166 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v80) v1 ==== 97+0+0 (secure 0 0 0) 0x7f42e4047a20 con 0x7f42ec152d20 2020-11-20T12:19:51.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 -- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f42c40557c0 msgr2=0x7f42c4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:51.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f42c40557c0 0x7f42c4057c80 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f42e800ba80 tx=0x7f42e8008040).stop 2020-11-20T12:19:51.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 -- 172.21.15.116:0/741243166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42ec152d20 msgr2=0x7f42ec0a9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:51.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42ec152d20 0x7f42ec0a9720 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f42e400ca60 tx=0x7f42e4007da0).stop 2020-11-20T12:19:51.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 -- 172.21.15.116:0/741243166 shutdown_connections 2020-11-20T12:19:51.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f42c40557c0 0x7f42c4057c80 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:51.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f42ec152d20 0x7f42ec0a9720 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:51.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f42ec063860 0x7f42ec160ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:51.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 --2- 172.21.15.116:0/741243166 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f42ec0a6a20 0x7f42ec0bfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:51.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 -- 172.21.15.116:0/741243166 >> 172.21.15.116:0/741243166 conn(0x7f42ec061280 msgr2=0x7f42ec0c1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:51.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 -- 172.21.15.116:0/741243166 shutdown_connections 2020-11-20T12:19:51.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:51.840+0000 7f42f33cd700 1 -- 172.21.15.116:0/741243166 wait complete. 2020-11-20T12:19:51.840 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2020-11-20T12:19:51.861 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-20T12:19:52.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.080+0000 7f0bfaf85700 1 Processor -- start 2020-11-20T12:19:52.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.080+0000 7f0bfaf85700 1 -- start start 2020-11-20T12:19:52.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bfaf85700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf4062880 0x7f0bf4062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bfaf85700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf40ba1b0 con 0x7f0bf414b990 2020-11-20T12:19:52.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bfaf85700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf41555a0 con 0x7f0bf40b94e0 2020-11-20T12:19:52.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bfaf85700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4152e60 con 0x7f0bf4062880 2020-11-20T12:19:52.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf9782700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0bf414b990 0x7f0bf414be10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57964/0 (socket says 172.21.15.116:57964) 2020-11-20T12:19:52.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf9782700 1 -- 172.21.15.116:0/2093064308 learned_addr learned my addr 172.21.15.116:0/2093064308 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:52.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf9f83700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf4062880 0x7f0bf4062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201744962 0 0) 0x7f0bf40ba1b0 con 0x7f0bf414b990 2020-11-20T12:19:52.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bdc003240 con 0x7f0bf414b990 2020-11-20T12:19:52.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3305203463 0 0) 0x7f0bf41555a0 con 0x7f0bf40b94e0 2020-11-20T12:19:52.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bf40ba1b0 con 0x7f0bf40b94e0 2020-11-20T12:19:52.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2739064438 0 0) 0x7f0bdc003240 con 0x7f0bf414b990 2020-11-20T12:19:52.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf4062880 msgr2=0x7f0bf4062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:52.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf4062880 0x7f0bf4062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 >> v1:172.21.15.173:6789/0 conn(0x7f0bf40b94e0 legacy=0x7f0bf40b9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf4155840 con 0x7f0bf414b990 2020-11-20T12:19:52.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f0be4001500 con 0x7f0bf414b990 2020-11-20T12:19:52.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 >> v1:172.21.15.116:6789/0 conn(0x7f0bf414b990 legacy=0x7f0bf414be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bdc0038f0 0x7f0bdc003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf4062880 0x7f0bdc0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bdc0049d0 0x7f0bdc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4152e60 con 0x7f0bdc0049d0 2020-11-20T12:19:52.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc005340 con 0x7f0bdc0038f0 2020-11-20T12:19:52.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc00b730 con 0x7f0bf4062880 2020-11-20T12:19:52.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bfa784700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bdc0049d0 0x7f0bdc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.084+0000 7f0bf9f83700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bdc0038f0 0x7f0bdc003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bf9782700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf4062880 0x7f0bdc0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfa784700 1 -- 172.21.15.116:0/2093064308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf4062880 msgr2=0x7f0bdc0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:52.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfa784700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf4062880 0x7f0bdc0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfa784700 1 -- 172.21.15.116:0/2093064308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bdc0038f0 msgr2=0x7f0bdc003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:52.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfa784700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bdc0038f0 0x7f0bdc003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfa784700 1 -- 172.21.15.116:0/2093064308 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bdc003240 con 0x7f0bdc0049d0 2020-11-20T12:19:52.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfa784700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bdc0049d0 0x7f0bdc004e50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f0bec001fd0 tx=0x7f0bec007f10).ready entity=mon.0 client_cookie=c2506a0dd186e835 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:52.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0016a0 con 0x7f0bdc0049d0 2020-11-20T12:19:52.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec00cc80 con 0x7f0bdc0049d0 2020-11-20T12:19:52.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bf8f81700 1 -- 172.21.15.116:0/2093064308 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec006870 con 0x7f0bdc0049d0 2020-11-20T12:19:52.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfaf85700 1 -- 172.21.15.116:0/2093064308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bdc0049d0 msgr2=0x7f0bdc004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfaf85700 1 --2- 172.21.15.116:0/2093064308 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bdc0049d0 0x7f0bdc004e50 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7f0bec001fd0 tx=0x7f0bec007f10).stop 2020-11-20T12:19:52.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfa784700 1 -- 172.21.15.116:0/2093064308 reap_dead start 2020-11-20T12:19:52.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfaf85700 1 -- 172.21.15.116:0/2093064308 shutdown_connections 2020-11-20T12:19:52.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfaf85700 1 -- 172.21.15.116:0/2093064308 >> 172.21.15.116:0/2093064308 conn(0x7f0bf4060c40 msgr2=0x7f0bf40c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:52.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfaf85700 1 -- 172.21.15.116:0/2093064308 shutdown_connections 2020-11-20T12:19:52.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.088+0000 7f0bfaf85700 1 -- 172.21.15.116:0/2093064308 wait complete. 2020-11-20T12:19:52.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bfaf85700 1 Processor -- start 2020-11-20T12:19:52.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bfaf85700 1 -- start start 2020-11-20T12:19:52.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bfaf85700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf40c32f0 0x7f0bf40c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bfaf85700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bf40bb3f0 0x7f0bf40a6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bfaf85700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bf40a7320 0x7f0bf40a7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bfaf85700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4160f30 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bfaf85700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40a6f40 con 0x7f0bf40a7320 2020-11-20T12:19:52.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bfaf85700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40bb9e0 con 0x7f0bf40c32f0 2020-11-20T12:19:52.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bf9782700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bf40bb3f0 0x7f0bf40a6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bf9782700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bf40bb3f0 0x7f0bf40a6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48544/0 (socket says 172.21.15.116:48544) 2020-11-20T12:19:52.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bf9782700 1 -- 172.21.15.116:0/1046220601 learned_addr learned my addr 172.21.15.116:0/1046220601 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:52.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bf9f83700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf40c32f0 0x7f0bf40c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bfa784700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bf40a7320 0x7f0bf40a7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bf9782700 1 -- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf40c32f0 msgr2=0x7f0bf40c36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bf9782700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf40c32f0 0x7f0bf40c36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bf9782700 1 -- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bf40a7320 msgr2=0x7f0bf40a7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bf9782700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bf40a7320 0x7f0bf40a7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bf9782700 1 -- 172.21.15.116:0/1046220601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf40bbd70 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.092+0000 7f0bf9782700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bf40bb3f0 0x7f0bf40a6a50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f0be400ea80 tx=0x7f0bf40677a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:52.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.096+0000 7f0be1ffb700 1 -- 172.21.15.116:0/1046220601 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be400a790 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.096+0000 7f0be1ffb700 1 -- 172.21.15.116:0/1046220601 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be4010040 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.096+0000 7f0be1ffb700 1 -- 172.21.15.116:0/1046220601 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be400ab00 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.096+0000 7f0bfaf85700 1 -- 172.21.15.116:0/1046220601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf40bc010 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.096+0000 7f0bfaf85700 1 -- 172.21.15.116:0/1046220601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf40a77a0 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.096+0000 7f0bfaf85700 1 -- 172.21.15.116:0/1046220601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf40bc4b0 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.096+0000 7f0be1ffb700 1 -- 172.21.15.116:0/1046220601 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0be401b020 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.096+0000 7f0be1ffb700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bcc0557c0 0x7f0bcc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.096+0000 7f0be1ffb700 1 -- 172.21.15.116:0/1046220601 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f0be406da10 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.100+0000 7f0bf9f83700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bcc0557c0 0x7f0bcc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.100+0000 7f0bf9f83700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bcc0557c0 0x7f0bcc057c80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f0bf0006fd0 tx=0x7f0bf0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:52.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.104+0000 7f0be1ffb700 1 -- 172.21.15.116:0/1046220601 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0be4043be0 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.312+0000 7f0bfaf85700 1 -- 172.21.15.116:0/1046220601 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f0bf405cf30 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.312+0000 7f0be1ffb700 1 -- 172.21.15.116:0/1046220601 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5399 (secure 0 0 0) 0x7f0be4047200 con 0x7f0bf40bb3f0 2020-11-20T12:19:52.310 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:52.310 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:19:52.311 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2020-11-20T12:19:52.311 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.311 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2020-11-20T12:19:52.311 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw==", 2020-11-20T12:19:52.312 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.312 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:52.312 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.312 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.312 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.313 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.313 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.313 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.313 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2020-11-20T12:19:52.314 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ==", 2020-11-20T12:19:52.314 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.314 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:52.314 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.314 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.315 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.315 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.315 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.316 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.316 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2020-11-20T12:19:52.316 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag==", 2020-11-20T12:19:52.316 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.317 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:52.317 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.317 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.317 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.318 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.318 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.318 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.318 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2020-11-20T12:19:52.319 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ==", 2020-11-20T12:19:52.319 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.319 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:52.319 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.320 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.320 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.320 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.320 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.320 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.321 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2020-11-20T12:19:52.321 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ==", 2020-11-20T12:19:52.321 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.321 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.322 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.322 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.322 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.322 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.323 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.323 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2020-11-20T12:19:52.323 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q==", 2020-11-20T12:19:52.323 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.324 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.324 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.324 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.324 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2020-11-20T12:19:52.325 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw==", 2020-11-20T12:19:52.326 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.326 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.326 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.326 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.326 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.327 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.327 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.327 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2020-11-20T12:19:52.327 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w==", 2020-11-20T12:19:52.327 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.328 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.328 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.328 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.328 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.329 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.329 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.329 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2020-11-20T12:19:52.330 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA==", 2020-11-20T12:19:52.330 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.330 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.330 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.330 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.331 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.331 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.331 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.332 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2020-11-20T12:19:52.332 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ==", 2020-11-20T12:19:52.332 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.332 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.333 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.333 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.333 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.333 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.333 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.334 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2020-11-20T12:19:52.334 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg==", 2020-11-20T12:19:52.334 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.334 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.335 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.335 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.335 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.335 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.336 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.336 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2020-11-20T12:19:52.336 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg==", 2020-11-20T12:19:52.336 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.337 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.337 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.337 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.337 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.338 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.338 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.338 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2020-11-20T12:19:52.338 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA==", 2020-11-20T12:19:52.339 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.339 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:52.339 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2020-11-20T12:19:52.339 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-20T12:19:52.339 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.340 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.340 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.340 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2020-11-20T12:19:52.340 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow==", 2020-11-20T12:19:52.340 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.341 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:52.341 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:19:52.341 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:19:52.341 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:19:52.342 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.342 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.342 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.342 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2020-11-20T12:19:52.343 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ==", 2020-11-20T12:19:52.343 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.343 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:52.343 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:19:52.344 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:19:52.344 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:19:52.344 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.344 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.344 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.345 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2020-11-20T12:19:52.345 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ==", 2020-11-20T12:19:52.345 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.345 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:19:52.346 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:19:52.346 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:19:52.346 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:19:52.346 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.347 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.347 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.347 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2020-11-20T12:19:52.347 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBDs7dfh+KKOBAAZvcPXWly9/mxRE7NWuR85A==", 2020-11-20T12:19:52.348 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.348 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:19:52.348 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:19:52.349 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.349 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.349 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.349 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.350 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.350 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.manila", 2020-11-20T12:19:52.350 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBUtLdfv7FOEBAAcNA1BqusVps0OxH8B/ZsqA==", 2020-11-20T12:19:52.350 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.350 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:19:52.351 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:19:52.351 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw" 2020-11-20T12:19:52.351 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.351 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.352 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.352 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2020-11-20T12:19:52.352 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A==", 2020-11-20T12:19:52.352 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.353 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:19:52.353 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:19:52.353 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.353 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.353 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:52.354 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:52.354 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2020-11-20T12:19:52.354 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA==", 2020-11-20T12:19:52.354 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:19:52.355 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:19:52.355 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:19:52.355 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:19:52.355 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.356 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:52.356 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:19:52.356 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:19:52.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 -- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bcc0557c0 msgr2=0x7f0bcc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bcc0557c0 0x7f0bcc057c80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f0bf0006fd0 tx=0x7f0bf0008040).stop 2020-11-20T12:19:52.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 -- 172.21.15.116:0/1046220601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bf40bb3f0 msgr2=0x7f0bf40a6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bf40bb3f0 0x7f0bf40a6a50 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7f0be400ea80 tx=0x7f0bf40677a0).stop 2020-11-20T12:19:52.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 -- 172.21.15.116:0/1046220601 shutdown_connections 2020-11-20T12:19:52.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bcc0557c0 0x7f0bcc057c80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0bf40a7320 0x7f0bf40a7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0bf40c32f0 0x7f0bf40c36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 --2- 172.21.15.116:0/1046220601 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0bf40bb3f0 0x7f0bf40a6a50 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 -- 172.21.15.116:0/1046220601 >> 172.21.15.116:0/1046220601 conn(0x7f0bf4060c40 msgr2=0x7f0bf414ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:52.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 -- 172.21.15.116:0/1046220601 shutdown_connections 2020-11-20T12:19:52.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.316+0000 7f0bfaf85700 1 -- 172.21.15.116:0/1046220601 wait complete. 2020-11-20T12:19:52.361 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.manila 2020-11-20T12:19:52.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.536+0000 7f373f59e700 1 Processor -- start 2020-11-20T12:19:52.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.536+0000 7f373f59e700 1 -- start start 2020-11-20T12:19:52.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.536+0000 7f373f59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f37400c8e30 0x7f37400adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.536+0000 7f373f59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37400ad0e0 con 0x7f37400c8a50 2020-11-20T12:19:52.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.536+0000 7f373f59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37400ad310 con 0x7f3740152010 2020-11-20T12:19:52.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.536+0000 7f373f59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37400ad5b0 con 0x7f37400c8e30 2020-11-20T12:19:52.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.536+0000 7f373e59c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f37400c8a50 0x7f37400aa8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:57984/0 (socket says 172.21.15.116:57984) 2020-11-20T12:19:52.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.536+0000 7f373e59c700 1 -- 172.21.15.116:0/979361839 learned_addr learned my addr 172.21.15.116:0/979361839 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:52.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.536+0000 7f3735d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f37400c8e30 0x7f37400adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f3735d9b700 1 -- 172.21.15.116:0/979361839 >> v1:172.21.15.173:6789/0 conn(0x7f3740152010 legacy=0x7f37401524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f373dd9b700 1 -- 172.21.15.116:0/979361839 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950930910 0 0) 0x7f37400ad0e0 con 0x7f37400c8a50 2020-11-20T12:19:52.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f3735d9b700 1 -- 172.21.15.116:0/979361839 >> v1:172.21.15.116:6789/0 conn(0x7f37400c8a50 legacy=0x7f37400aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f3735d9b700 1 -- 172.21.15.116:0/979361839 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37400ad850 con 0x7f37400c8e30 2020-11-20T12:19:52.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f3735d9b700 1 --2- 172.21.15.116:0/979361839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37400c8e30 0x7f37400adcf0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f372800b920 tx=0x7f3728006df0).ready entity=mon.0 client_cookie=a2a60238f7228381 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:52.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f373dd9b700 1 -- 172.21.15.116:0/979361839 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728009730 con 0x7f37400c8e30 2020-11-20T12:19:52.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f373dd9b700 1 -- 172.21.15.116:0/979361839 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3728009890 con 0x7f37400c8e30 2020-11-20T12:19:52.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f373dd9b700 1 -- 172.21.15.116:0/979361839 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3728009c10 con 0x7f37400c8e30 2020-11-20T12:19:52.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f373f59e700 1 -- 172.21.15.116:0/979361839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37400c8e30 msgr2=0x7f37400adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f373f59e700 1 --2- 172.21.15.116:0/979361839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37400c8e30 0x7f37400adcf0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7f372800b920 tx=0x7f3728006df0).stop 2020-11-20T12:19:52.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f373f59e700 1 -- 172.21.15.116:0/979361839 shutdown_connections 2020-11-20T12:19:52.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f373f59e700 1 --2- 172.21.15.116:0/979361839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37400c8e30 0x7f37400adcf0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f373f59e700 1 -- 172.21.15.116:0/979361839 >> 172.21.15.116:0/979361839 conn(0x7f37400615f0 msgr2=0x7f37400619d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:52.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f373f59e700 1 -- 172.21.15.116:0/979361839 shutdown_connections 2020-11-20T12:19:52.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.540+0000 7f373f59e700 1 -- 172.21.15.116:0/979361839 wait complete. 2020-11-20T12:19:52.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373f59e700 1 Processor -- start 2020-11-20T12:19:52.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373f59e700 1 -- start start 2020-11-20T12:19:52.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373f59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37400c8a50 0x7f3740150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373f59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3740152010 0x7f37400a6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37400a7160 0x7f37401499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373f59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374014f530 con 0x7f37400c8a50 2020-11-20T12:19:52.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373f59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f374014f760 con 0x7f3740152010 2020-11-20T12:19:52.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f374014fa00 con 0x7f37400a7160 2020-11-20T12:19:52.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37400c8a50 0x7f3740150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37400c8a50 0x7f3740150680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48556/0 (socket says 172.21.15.116:48556) 2020-11-20T12:19:52.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373e59c700 1 -- 172.21.15.116:0/3671626888 learned_addr learned my addr 172.21.15.116:0/3671626888 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:52.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f3735d9b700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3740152010 0x7f37400a6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.544+0000 7f373ed9d700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37400a7160 0x7f37401499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f373e59c700 1 -- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37400a7160 msgr2=0x7f37401499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f373e59c700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37400a7160 0x7f37401499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f373e59c700 1 -- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3740152010 msgr2=0x7f37400a6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f373e59c700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3740152010 0x7f37400a6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f373e59c700 1 -- 172.21.15.116:0/3671626888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374014fca0 con 0x7f37400c8a50 2020-11-20T12:19:52.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f373e59c700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37400c8a50 0x7f3740150680 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f373000c720 tx=0x7f3730007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:52.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f3736ffd700 1 -- 172.21.15.116:0/3671626888 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730007320 con 0x7f37400c8a50 2020-11-20T12:19:52.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f3736ffd700 1 -- 172.21.15.116:0/3671626888 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3730007480 con 0x7f37400c8a50 2020-11-20T12:19:52.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f373f59e700 1 -- 172.21.15.116:0/3671626888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f374014ff40 con 0x7f37400c8a50 2020-11-20T12:19:52.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f3736ffd700 1 -- 172.21.15.116:0/3671626888 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3730005800 con 0x7f37400c8a50 2020-11-20T12:19:52.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f373f59e700 1 -- 172.21.15.116:0/3671626888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37401623e0 con 0x7f37400c8a50 2020-11-20T12:19:52.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f373f59e700 1 -- 172.21.15.116:0/3671626888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3740153d90 con 0x7f37400c8a50 2020-11-20T12:19:52.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.548+0000 7f3736ffd700 1 -- 172.21.15.116:0/3671626888 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3730010030 con 0x7f37400c8a50 2020-11-20T12:19:52.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.552+0000 7f3736ffd700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 0x7f371c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:52.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.552+0000 7f3736ffd700 1 -- 172.21.15.116:0/3671626888 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f373006e1a0 con 0x7f37400c8a50 2020-11-20T12:19:52.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.552+0000 7f3735d9b700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 0x7f371c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:52.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.552+0000 7f3735d9b700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 0x7f371c057c80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f37400c62f0 tx=0x7f3728015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:52.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.556+0000 7f3736ffd700 1 -- 172.21.15.116:0/3671626888 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3740153d90 con 0x7f37400c8a50 2020-11-20T12:19:52.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.768+0000 7f373f59e700 1 -- 172.21.15.116:0/3671626888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.manila"} v 0) v1 -- 0x7f3740153d90 con 0x7f37400c8a50 2020-11-20T12:19:52.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.776+0000 7f3736ffd700 1 -- 172.21.15.116:0/3671626888 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.manila"}]=0 updated v21) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3740153d90 con 0x7f37400c8a50 2020-11-20T12:19:52.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 -- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 msgr2=0x7f371c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 0x7f371c057c80 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f37400c62f0 tx=0x7f3728015000).stop 2020-11-20T12:19:52.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 -- 172.21.15.116:0/3671626888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37400c8a50 msgr2=0x7f3740150680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:52.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37400c8a50 0x7f3740150680 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7f373000c720 tx=0x7f3730007ac0).stop 2020-11-20T12:19:52.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 -- 172.21.15.116:0/3671626888 shutdown_connections 2020-11-20T12:19:52.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f371c0557c0 0x7f371c057c80 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f37400a7160 0x7f37401499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f37400c8a50 0x7f3740150680 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 --2- 172.21.15.116:0/3671626888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3740152010 0x7f37400a6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:52.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 -- 172.21.15.116:0/3671626888 >> 172.21.15.116:0/3671626888 conn(0x7f37400615f0 msgr2=0x7f37400c64f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:52.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 -- 172.21.15.116:0/3671626888 shutdown_connections 2020-11-20T12:19:52.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.780+0000 7f373f59e700 1 -- 172.21.15.116:0/3671626888 wait complete. 2020-11-20T12:19:52.781 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:19:52.803 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:52.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.980+0000 7f6367494700 1 Processor -- start 2020-11-20T12:19:53.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.980+0000 7f6367494700 1 -- start start 2020-11-20T12:19:53.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f6367494700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f63600c3e10 0x7f63600c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f6367494700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f63600a55f0 0x7f63600c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f6367494700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63600c35e0 con 0x7f63600c93a0 2020-11-20T12:19:53.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f6367494700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63600c3880 con 0x7f63600a55f0 2020-11-20T12:19:53.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f6367494700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63600c7280 con 0x7f63600c3e10 2020-11-20T12:19:53.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f6364a2f700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f63600c93a0 0x7f63600c79c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58000/0 (socket says 172.21.15.116:58000) 2020-11-20T12:19:53.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f6364a2f700 1 -- 172.21.15.116:0/2721999959 learned_addr learned my addr 172.21.15.116:0/2721999959 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:53.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f6365a31700 1 --2- 172.21.15.116:0/2721999959 >> v2:172.21.15.116:3300/0 conn(0x7f63600a55f0 0x7f63600c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f6365230700 1 --2- 172.21.15.116:0/2721999959 >> v2:172.21.15.173:3300/0 conn(0x7f63600c3e10 0x7f63600c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666937712 0 0) 0x7f63600c35e0 con 0x7f63600c93a0 2020-11-20T12:19:53.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6348003240 con 0x7f63600c93a0 2020-11-20T12:19:53.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2220866170 0 0) 0x7f6348003240 con 0x7f63600c93a0 2020-11-20T12:19:53.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 >> v2:172.21.15.173:3300/0 conn(0x7f63600c3e10 msgr2=0x7f63600c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:53.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 --2- 172.21.15.116:0/2721999959 >> v2:172.21.15.173:3300/0 conn(0x7f63600c3e10 0x7f63600c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 >> v2:172.21.15.116:3300/0 conn(0x7f63600a55f0 msgr2=0x7f63600c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:53.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 --2- 172.21.15.116:0/2721999959 >> v2:172.21.15.116:3300/0 conn(0x7f63600a55f0 0x7f63600c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63600c7520 con 0x7f63600c93a0 2020-11-20T12:19:53.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f63500014e0 con 0x7f63600c93a0 2020-11-20T12:19:53.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 >> v1:172.21.15.116:6789/0 conn(0x7f63600c93a0 legacy=0x7f63600c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6348003800 0x7f6348003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63480044e0 0x7f6348004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6348004e50 0x7f634800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600c3880 con 0x7f6348004e50 2020-11-20T12:19:53.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63600c7280 con 0x7f6348003800 2020-11-20T12:19:53.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634800bc20 con 0x7f63480044e0 2020-11-20T12:19:53.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.984+0000 7f6365a31700 1 --2- 172.21.15.116:0/2721999959 >> v2:172.21.15.116:3300/0 conn(0x7f63600a55f0 0x7f63600c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:53.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6365a31700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6348004e50 0x7f634800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6364a2f700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63480044e0 0x7f6348004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6365230700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6348003800 0x7f6348003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6365a31700 1 -- 172.21.15.116:0/2721999959 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63480044e0 msgr2=0x7f6348004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6365a31700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f63480044e0 0x7f6348004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6365a31700 1 -- 172.21.15.116:0/2721999959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6348003800 msgr2=0x7f6348003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6365a31700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6348003800 0x7f6348003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6365a31700 1 -- 172.21.15.116:0/2721999959 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6348003240 con 0x7f6348004e50 2020-11-20T12:19:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6365230700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6348003800 0x7f6348003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6365a31700 1 -- 172.21.15.116:0/2721999959 reap_dead start 2020-11-20T12:19:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6365a31700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6348004e50 0x7f634800b730 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f635800e2d0 tx=0x7f63580091d0).ready entity=mon.0 client_cookie=8198b30919c3ac14 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358009800 con 0x7f6348004e50 2020-11-20T12:19:53.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63580123f0 con 0x7f6348004e50 2020-11-20T12:19:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f634ffff700 1 -- 172.21.15.116:0/2721999959 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6358012770 con 0x7f6348004e50 2020-11-20T12:19:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6367494700 1 -- 172.21.15.116:0/2721999959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6348004e50 msgr2=0x7f634800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6367494700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6348004e50 0x7f634800b730 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f635800e2d0 tx=0x7f63580091d0).stop 2020-11-20T12:19:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6367494700 1 -- 172.21.15.116:0/2721999959 shutdown_connections 2020-11-20T12:19:53.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6367494700 1 --2- 172.21.15.116:0/2721999959 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6348004e50 0x7f634800b730 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.988+0000 7f6367494700 1 -- 172.21.15.116:0/2721999959 >> 172.21.15.116:0/2721999959 conn(0x7f6360068080 msgr2=0x7f63600c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.992+0000 7f6367494700 1 -- 172.21.15.116:0/2721999959 shutdown_connections 2020-11-20T12:19:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.992+0000 7f6367494700 1 -- 172.21.15.116:0/2721999959 wait complete. 2020-11-20T12:19:53.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.992+0000 7f6367494700 1 Processor -- start 2020-11-20T12:19:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.992+0000 7f6367494700 1 -- start start 2020-11-20T12:19:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6367494700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6348004e50 0x7f6360151780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6367494700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63600c1cc0 0x7f63600c2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6367494700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63600a5ce0 0x7f63600a6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6367494700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6360092850 con 0x7f63600c1cc0 2020-11-20T12:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6367494700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6360092ab0 con 0x7f63600a5ce0 2020-11-20T12:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6367494700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6360092d50 con 0x7f6348004e50 2020-11-20T12:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6364a2f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63600c1cc0 0x7f63600c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6364a2f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63600c1cc0 0x7f63600c2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48578/0 (socket says 172.21.15.116:48578) 2020-11-20T12:19:53.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6364a2f700 1 -- 172.21.15.116:0/1553497600 learned_addr learned my addr 172.21.15.116:0/1553497600 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6365230700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6348004e50 0x7f6360151780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6365a31700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63600a5ce0 0x7f63600a6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6364a2f700 1 -- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6348004e50 msgr2=0x7f6360151780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6364a2f700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6348004e50 0x7f6360151780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6364a2f700 1 -- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63600a5ce0 msgr2=0x7f63600a6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6364a2f700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63600a5ce0 0x7f63600a6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6364a2f700 1 -- 172.21.15.116:0/1553497600 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6360092ff0 con 0x7f63600c1cc0 2020-11-20T12:19:53.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6365a31700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63600a5ce0 0x7f63600a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6364a2f700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63600c1cc0 0x7f63600c2140 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f6350004b00 tx=0x7f63600c76b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f634d7fa700 1 -- 172.21.15.116:0/1553497600 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000f8a0 con 0x7f63600c1cc0 2020-11-20T12:19:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f634d7fa700 1 -- 172.21.15.116:0/1553497600 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635000fa00 con 0x7f63600c1cc0 2020-11-20T12:19:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6367494700 1 -- 172.21.15.116:0/1553497600 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6360093290 con 0x7f63600c1cc0 2020-11-20T12:19:53.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f634d7fa700 1 -- 172.21.15.116:0/1553497600 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000fda0 con 0x7f63600c1cc0 2020-11-20T12:19:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:52.996+0000 7f6367494700 1 -- 172.21.15.116:0/1553497600 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6360151c70 con 0x7f63600c1cc0 2020-11-20T12:19:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.000+0000 7f6367494700 1 -- 172.21.15.116:0/1553497600 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f636005d2a0 con 0x7f63600c1cc0 2020-11-20T12:19:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.000+0000 7f634d7fa700 1 -- 172.21.15.116:0/1553497600 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6350001130 con 0x7f63600c1cc0 2020-11-20T12:19:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.000+0000 7f634d7fa700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63380557c0 0x7f6338057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.000+0000 7f634d7fa700 1 -- 172.21.15.116:0/1553497600 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f635006daf0 con 0x7f63600c1cc0 2020-11-20T12:19:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.000+0000 7f6365230700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63380557c0 0x7f6338057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.004+0000 7f6365230700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63380557c0 0x7f6338057c80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f635c006ca0 tx=0x7f635c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.008+0000 7f634d7fa700 1 -- 172.21.15.116:0/1553497600 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6350043bb0 con 0x7f63600c1cc0 2020-11-20T12:19:53.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.168+0000 7f6367494700 1 -- 172.21.15.116:0/1553497600 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6360081c00 con 0x7f63600c1cc0 2020-11-20T12:19:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.168+0000 7f634d7fa700 1 -- 172.21.15.116:0/1553497600 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+13724 (secure 0 0 0) 0x7f6350019070 con 0x7f63600c1cc0 2020-11-20T12:19:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 -- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63380557c0 msgr2=0x7f6338057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63380557c0 0x7f6338057c80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f635c006ca0 tx=0x7f635c008040).stop 2020-11-20T12:19:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 -- 172.21.15.116:0/1553497600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63600c1cc0 msgr2=0x7f63600c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63600c1cc0 0x7f63600c2140 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f6350004b00 tx=0x7f63600c76b0).stop 2020-11-20T12:19:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 -- 172.21.15.116:0/1553497600 shutdown_connections 2020-11-20T12:19:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f63380557c0 0x7f6338057c80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f63600a5ce0 0x7f63600a6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6348004e50 0x7f6360151780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 --2- 172.21.15.116:0/1553497600 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f63600c1cc0 0x7f63600c2140 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 -- 172.21.15.116:0/1553497600 >> 172.21.15.116:0/1553497600 conn(0x7f6360068080 msgr2=0x7f63600be880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 -- 172.21.15.116:0/1553497600 shutdown_connections 2020-11-20T12:19:53.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.172+0000 7f6367494700 1 -- 172.21.15.116:0/1553497600 wait complete. 2020-11-20T12:19:53.347 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:53.347 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":80,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:51.827984+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:53.348 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T12:19:53.349 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:19:53.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.528+0000 7f9d5f59e700 1 Processor -- start 2020-11-20T12:19:53.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.532+0000 7f9d5f59e700 1 -- start start 2020-11-20T12:19:53.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.532+0000 7f9d5f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d600a69d0 0x7f9d6014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.532+0000 7f9d5f59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9d60153270 0x7f9d600a6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.532+0000 7f9d5f59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d600a9010 con 0x7f9d60149820 2020-11-20T12:19:53.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.532+0000 7f9d5f59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d600a72a0 con 0x7f9d60153270 2020-11-20T12:19:53.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.532+0000 7f9d5f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600aa480 con 0x7f9d600a69d0 2020-11-20T12:19:53.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.532+0000 7f9d5ed9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9d60149820 0x7f9d60149c80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58020/0 (socket says 172.21.15.116:58020) 2020-11-20T12:19:53.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.532+0000 7f9d5ed9d700 1 -- 172.21.15.116:0/124675809 learned_addr learned my addr 172.21.15.116:0/124675809 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:53.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492169552 0 0) 0x7f9d600a9010 con 0x7f9d60149820 2020-11-20T12:19:53.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d44003240 con 0x7f9d60149820 2020-11-20T12:19:53.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5e59c700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d600a69d0 0x7f9d6014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5dd9b700 1 --2- 172.21.15.116:0/124675809 >> v2:172.21.15.173:3300/0 conn(0x7f9d60153270 0x7f9d600a6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1284650858 0 0) 0x7f9d44003240 con 0x7f9d60149820 2020-11-20T12:19:53.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d600a69d0 msgr2=0x7f9d6014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:53.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d600a69d0 0x7f9d6014fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 >> v2:172.21.15.173:3300/0 conn(0x7f9d60153270 msgr2=0x7f9d600a6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:53.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 --2- 172.21.15.116:0/124675809 >> v2:172.21.15.173:3300/0 conn(0x7f9d60153270 0x7f9d600a6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d601541a0 con 0x7f9d60149820 2020-11-20T12:19:53.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f9d58001860 con 0x7f9d60149820 2020-11-20T12:19:53.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 >> v1:172.21.15.116:6789/0 conn(0x7f9d60149820 legacy=0x7f9d60149c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d600a69d0 0x7f9d44003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d44003c40 0x7f9d440044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 0x7f9d44004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600a72a0 con 0x7f9d440049d0 2020-11-20T12:19:53.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600aa480 con 0x7f9d44003c40 2020-11-20T12:19:53.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d440042b0 con 0x7f9d600a69d0 2020-11-20T12:19:53.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5dd9b700 1 --2- 172.21.15.116:0/124675809 >> v2:172.21.15.173:3300/0 conn(0x7f9d60153270 0x7f9d600a6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:53.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5ed9d700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 0x7f9d44004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5e59c700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d600a69d0 0x7f9d44003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5dd9b700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d44003c40 0x7f9d440044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5ed9d700 1 -- 172.21.15.116:0/124675809 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d600a69d0 msgr2=0x7f9d44003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:53.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5ed9d700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d600a69d0 0x7f9d44003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5ed9d700 1 -- 172.21.15.116:0/124675809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d44003c40 msgr2=0x7f9d440044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:53.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5ed9d700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d44003c40 0x7f9d440044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5ed9d700 1 -- 172.21.15.116:0/124675809 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d44003240 con 0x7f9d440049d0 2020-11-20T12:19:53.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5ed9d700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 0x7f9d44004e50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f9d5800c840 tx=0x7f9d58007b80).ready entity=mon.0 client_cookie=ed296740bdd2a63 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58005b80 con 0x7f9d440049d0 2020-11-20T12:19:53.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d58005ce0 con 0x7f9d440049d0 2020-11-20T12:19:53.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.536+0000 7f9d5d59a700 1 -- 172.21.15.116:0/124675809 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d58007320 con 0x7f9d440049d0 2020-11-20T12:19:53.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.540+0000 7f9d5f59e700 1 -- 172.21.15.116:0/124675809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 msgr2=0x7f9d44004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.540+0000 7f9d5f59e700 1 --2- 172.21.15.116:0/124675809 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 0x7f9d44004e50 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f9d5800c840 tx=0x7f9d58007b80).stop 2020-11-20T12:19:53.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.540+0000 7f9d5ed9d700 1 -- 172.21.15.116:0/124675809 reap_dead start 2020-11-20T12:19:53.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.540+0000 7f9d5f59e700 1 -- 172.21.15.116:0/124675809 shutdown_connections 2020-11-20T12:19:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.540+0000 7f9d5f59e700 1 -- 172.21.15.116:0/124675809 >> 172.21.15.116:0/124675809 conn(0x7f9d600c16b0 msgr2=0x7f9d600c1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.540+0000 7f9d5f59e700 1 -- 172.21.15.116:0/124675809 shutdown_connections 2020-11-20T12:19:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.540+0000 7f9d5f59e700 1 -- 172.21.15.116:0/124675809 wait complete. 2020-11-20T12:19:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5f59e700 1 Processor -- start 2020-11-20T12:19:53.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5f59e700 1 -- start start 2020-11-20T12:19:53.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5f59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 0x7f9d6015cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6014e760 0x7f9d601457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5f59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d60145cd0 0x7f9d6014c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5f59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6014a2f0 con 0x7f9d440049d0 2020-11-20T12:19:53.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5f59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6014f030 con 0x7f9d60145cd0 2020-11-20T12:19:53.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6014f2d0 con 0x7f9d6014e760 2020-11-20T12:19:53.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 0x7f9d6015cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5e59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 0x7f9d6015cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48596/0 (socket says 172.21.15.116:48596) 2020-11-20T12:19:53.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5e59c700 1 -- 172.21.15.116:0/13963445 learned_addr learned my addr 172.21.15.116:0/13963445 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:53.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5dd9b700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6014e760 0x7f9d601457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5ed9d700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d60145cd0 0x7f9d6014c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5e59c700 1 -- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6014e760 msgr2=0x7f9d601457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5e59c700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6014e760 0x7f9d601457e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5e59c700 1 -- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d60145cd0 msgr2=0x7f9d6014c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5e59c700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d60145cd0 0x7f9d6014c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.544+0000 7f9d5e59c700 1 -- 172.21.15.116:0/13963445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d6014f570 con 0x7f9d440049d0 2020-11-20T12:19:53.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.548+0000 7f9d5e59c700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 0x7f9d6015cc10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f9d50010c20 tx=0x7f9d50008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.548+0000 7f9d4e7fc700 1 -- 172.21.15.116:0/13963445 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5000e9f0 con 0x7f9d440049d0 2020-11-20T12:19:53.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.548+0000 7f9d4e7fc700 1 -- 172.21.15.116:0/13963445 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5000eb50 con 0x7f9d440049d0 2020-11-20T12:19:53.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.548+0000 7f9d5f59e700 1 -- 172.21.15.116:0/13963445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6014f810 con 0x7f9d440049d0 2020-11-20T12:19:53.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.548+0000 7f9d4e7fc700 1 -- 172.21.15.116:0/13963445 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5000eed0 con 0x7f9d440049d0 2020-11-20T12:19:53.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.548+0000 7f9d5f59e700 1 -- 172.21.15.116:0/13963445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6014ede0 con 0x7f9d440049d0 2020-11-20T12:19:53.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.548+0000 7f9d5f59e700 1 -- 172.21.15.116:0/13963445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d600a89b0 con 0x7f9d440049d0 2020-11-20T12:19:53.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.548+0000 7f9d4e7fc700 1 -- 172.21.15.116:0/13963445 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9d50024070 con 0x7f9d440049d0 2020-11-20T12:19:53.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.548+0000 7f9d4e7fc700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d340557c0 0x7f9d34057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.552+0000 7f9d4e7fc700 1 -- 172.21.15.116:0/13963445 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f9d5006ed50 con 0x7f9d440049d0 2020-11-20T12:19:53.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.552+0000 7f9d5dd9b700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d340557c0 0x7f9d34057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.552+0000 7f9d5dd9b700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d340557c0 0x7f9d34057c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f9d54006a90 tx=0x7f9d54008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.556+0000 7f9d4e7fc700 1 -- 172.21.15.116:0/13963445 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9d600a89b0 con 0x7f9d440049d0 2020-11-20T12:19:53.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.716+0000 7f9d5f59e700 1 -- 172.21.15.116:0/13963445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9d600a89b0 con 0x7f9d440049d0 2020-11-20T12:19:53.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.716+0000 7f9d4e7fc700 1 -- 172.21.15.116:0/13963445 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v80) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9d600a89b0 con 0x7f9d440049d0 2020-11-20T12:19:53.714 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:19:53.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.716+0000 7f9d5f59e700 1 -- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d340557c0 msgr2=0x7f9d34057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.716+0000 7f9d5f59e700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d340557c0 0x7f9d34057c80 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f9d54006a90 tx=0x7f9d54008040).stop 2020-11-20T12:19:53.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.716+0000 7f9d5f59e700 1 -- 172.21.15.116:0/13963445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 msgr2=0x7f9d6015cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.716+0000 7f9d5f59e700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 0x7f9d6015cc10 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f9d50010c20 tx=0x7f9d50008040).stop 2020-11-20T12:19:53.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.720+0000 7f9d5f59e700 1 -- 172.21.15.116:0/13963445 shutdown_connections 2020-11-20T12:19:53.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.720+0000 7f9d5f59e700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d340557c0 0x7f9d34057c80 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.720+0000 7f9d5f59e700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d60145cd0 0x7f9d6014c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.720+0000 7f9d5f59e700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d440049d0 0x7f9d6015cc10 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.720+0000 7f9d5f59e700 1 --2- 172.21.15.116:0/13963445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d6014e760 0x7f9d601457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.720+0000 7f9d5f59e700 1 -- 172.21.15.116:0/13963445 >> 172.21.15.116:0/13963445 conn(0x7f9d600c16b0 msgr2=0x7f9d60066c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:53.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.720+0000 7f9d5f59e700 1 -- 172.21.15.116:0/13963445 shutdown_connections 2020-11-20T12:19:53.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.720+0000 7f9d5f59e700 1 -- 172.21.15.116:0/13963445 wait complete. 2020-11-20T12:19:53.743 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:19:53.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.928+0000 7f3d8cf01700 1 Processor -- start 2020-11-20T12:19:53.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.928+0000 7f3d8cf01700 1 -- start start 2020-11-20T12:19:53.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.928+0000 7f3d8cf01700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d88149df0 0x7f3d88153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.928+0000 7f3d8cf01700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3d8814a1d0 0x7f3d880be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.928+0000 7f3d8cf01700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d880a9c20 con 0x7f3d880a9750 2020-11-20T12:19:53.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.928+0000 7f3d8cf01700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d880a9ec0 con 0x7f3d8814a1d0 2020-11-20T12:19:53.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.928+0000 7f3d8cf01700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d88153670 con 0x7f3d88149df0 2020-11-20T12:19:53.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d8659c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d88149df0 0x7f3d88153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d7dd9b700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f3d880a9750 0x7f3d880c0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:54270/0 (socket says 172.21.15.116:54270) 2020-11-20T12:19:53.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d7dd9b700 1 -- 172.21.15.116:0/1628129624 learned_addr learned my addr 172.21.15.116:0/1628129624 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:53.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d86d9d700 1 --2- 172.21.15.116:0/1628129624 >> v2:172.21.15.173:3300/0 conn(0x7f3d8814a1d0 0x7f3d880be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d85d9b700 1 -- 172.21.15.116:0/1628129624 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089651938 0 0) 0x7f3d880a9c20 con 0x7f3d880a9750 2020-11-20T12:19:53.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d85d9b700 1 -- 172.21.15.116:0/1628129624 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d68003240 con 0x7f3d880a9750 2020-11-20T12:19:53.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d86d9d700 1 -- 172.21.15.116:0/1628129624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d88149df0 msgr2=0x7f3d88153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:53.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d86d9d700 1 --2- 172.21.15.116:0/1628129624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d88149df0 0x7f3d88153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d86d9d700 1 -- 172.21.15.116:0/1628129624 >> v1:172.21.15.173:6789/0 conn(0x7f3d880a9750 legacy=0x7f3d880c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d86d9d700 1 -- 172.21.15.116:0/1628129624 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d88153910 con 0x7f3d8814a1d0 2020-11-20T12:19:53.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d8659c700 1 --2- 172.21.15.116:0/1628129624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d88149df0 0x7f3d88153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:53.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d86d9d700 1 --2- 172.21.15.116:0/1628129624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d8814a1d0 0x7f3d880be240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f3d80009c90 tx=0x7f3d800091d0).ready entity=mon.1 client_cookie=38ff242c0943b67b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d85d9b700 1 -- 172.21.15.116:0/1628129624 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8000b9c0 con 0x7f3d8814a1d0 2020-11-20T12:19:53.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d85d9b700 1 -- 172.21.15.116:0/1628129624 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8000bb20 con 0x7f3d8814a1d0 2020-11-20T12:19:53.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d85d9b700 1 -- 172.21.15.116:0/1628129624 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8000bea0 con 0x7f3d8814a1d0 2020-11-20T12:19:53.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1628129624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d8814a1d0 msgr2=0x7f3d880be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d8cf01700 1 --2- 172.21.15.116:0/1628129624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d8814a1d0 0x7f3d880be240 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f3d80009c90 tx=0x7f3d800091d0).stop 2020-11-20T12:19:53.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1628129624 shutdown_connections 2020-11-20T12:19:53.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d8cf01700 1 --2- 172.21.15.116:0/1628129624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d8814a1d0 0x7f3d880be240 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d8cf01700 1 --2- 172.21.15.116:0/1628129624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d88149df0 0x7f3d88153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.932+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1628129624 >> 172.21.15.116:0/1628129624 conn(0x7f3d8805ba80 msgr2=0x7f3d880c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:53.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1628129624 shutdown_connections 2020-11-20T12:19:53.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1628129624 wait complete. 2020-11-20T12:19:53.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d8cf01700 1 Processor -- start 2020-11-20T12:19:53.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d8cf01700 1 -- start start 2020-11-20T12:19:53.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d8cf01700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d880a9750 0x7f3d880a8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d8cf01700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d88149df0 0x7f3d8814fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d8cf01700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d8814a1d0 0x7f3d881502b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d8cf01700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d881648d0 con 0x7f3d8814a1d0 2020-11-20T12:19:53.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d8cf01700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d88164b70 con 0x7f3d880a9750 2020-11-20T12:19:53.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d8cf01700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d88164e10 con 0x7f3d88149df0 2020-11-20T12:19:53.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d86d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d8814a1d0 0x7f3d881502b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d86d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d8814a1d0 0x7f3d881502b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48614/0 (socket says 172.21.15.116:48614) 2020-11-20T12:19:53.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d86d9d700 1 -- 172.21.15.116:0/1158189030 learned_addr learned my addr 172.21.15.116:0/1158189030 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:53.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d8659c700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d880a9750 0x7f3d880a8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.936+0000 7f3d7dd9b700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d88149df0 0x7f3d8814fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d86d9d700 1 -- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d88149df0 msgr2=0x7f3d8814fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d86d9d700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d88149df0 0x7f3d8814fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d86d9d700 1 -- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d880a9750 msgr2=0x7f3d880a8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:53.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d86d9d700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d880a9750 0x7f3d880a8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:53.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d86d9d700 1 -- 172.21.15.116:0/1158189030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d881650b0 con 0x7f3d8814a1d0 2020-11-20T12:19:53.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d86d9d700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d8814a1d0 0x7f3d881502b0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f3d880611d0 tx=0x7f3d80016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d7effd700 1 -- 172.21.15.116:0/1158189030 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8001c5f0 con 0x7f3d8814a1d0 2020-11-20T12:19:53.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d7effd700 1 -- 172.21.15.116:0/1158189030 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d8001c750 con 0x7f3d8814a1d0 2020-11-20T12:19:53.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1158189030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d88165350 con 0x7f3d8814a1d0 2020-11-20T12:19:53.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d7effd700 1 -- 172.21.15.116:0/1158189030 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d8001caf0 con 0x7f3d8814a1d0 2020-11-20T12:19:53.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1158189030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d880a7720 con 0x7f3d8814a1d0 2020-11-20T12:19:53.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.940+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1158189030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d8805d2a0 con 0x7f3d8814a1d0 2020-11-20T12:19:53.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.944+0000 7f3d7effd700 1 -- 172.21.15.116:0/1158189030 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3d80024030 con 0x7f3d8814a1d0 2020-11-20T12:19:53.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.944+0000 7f3d7effd700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d640557c0 0x7f3d64057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:53.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.944+0000 7f3d7effd700 1 -- 172.21.15.116:0/1158189030 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3d80070330 con 0x7f3d8814a1d0 2020-11-20T12:19:53.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.944+0000 7f3d8659c700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d640557c0 0x7f3d64057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:53.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.944+0000 7f3d8659c700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d640557c0 0x7f3d64057c80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f3d880a76b0 tx=0x7f3d78006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:53.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:53.948+0000 7f3d7effd700 1 -- 172.21.15.116:0/1158189030 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3d80046630 con 0x7f3d8814a1d0 2020-11-20T12:19:54.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.108+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1158189030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3d881654e0 con 0x7f3d8814a1d0 2020-11-20T12:19:54.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.108+0000 7f3d7effd700 1 -- 172.21.15.116:0/1158189030 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v80) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3d8002c0a0 con 0x7f3d8814a1d0 2020-11-20T12:19:54.106 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:19:54.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.108+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d640557c0 msgr2=0x7f3d64057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.108+0000 7f3d8cf01700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d640557c0 0x7f3d64057c80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f3d880a76b0 tx=0x7f3d78006df0).stop 2020-11-20T12:19:54.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.108+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1158189030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d8814a1d0 msgr2=0x7f3d881502b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.108+0000 7f3d8cf01700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d8814a1d0 0x7f3d881502b0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f3d880611d0 tx=0x7f3d80016000).stop 2020-11-20T12:19:54.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.112+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1158189030 shutdown_connections 2020-11-20T12:19:54.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.112+0000 7f3d8cf01700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3d640557c0 0x7f3d64057c80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.112+0000 7f3d8cf01700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d8814a1d0 0x7f3d881502b0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.112+0000 7f3d8cf01700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d880a9750 0x7f3d880a8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.112+0000 7f3d8cf01700 1 --2- 172.21.15.116:0/1158189030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d88149df0 0x7f3d8814fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.112+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1158189030 >> 172.21.15.116:0/1158189030 conn(0x7f3d8805ba80 msgr2=0x7f3d88062b20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:54.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.112+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1158189030 shutdown_connections 2020-11-20T12:19:54.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.112+0000 7f3d8cf01700 1 -- 172.21.15.116:0/1158189030 wait complete. 2020-11-20T12:19:54.131 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-20T12:19:54.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.304+0000 7f82d292a700 1 Processor -- start 2020-11-20T12:19:54.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.304+0000 7f82d292a700 1 -- start start 2020-11-20T12:19:54.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.304+0000 7f82d292a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f82cc0c3b60 0x7f82cc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.304+0000 7f82d292a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82cc0c1970 0x7f82cc150560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.304+0000 7f82d292a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82cc0a6950 con 0x7f82cc0c3780 2020-11-20T12:19:54.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.304+0000 7f82d292a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82cc0a6770 con 0x7f82cc0c3b60 2020-11-20T12:19:54.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.304+0000 7f82d292a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc14fed0 con 0x7f82cc0c1970 2020-11-20T12:19:54.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82d0ec7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82cc0c1970 0x7f82cc150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82cbfff700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f82cc0c3780 0x7f82cc0c22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:54282/0 (socket says 172.21.15.116:54282) 2020-11-20T12:19:54.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82cbfff700 1 -- 172.21.15.116:0/4034200292 learned_addr learned my addr 172.21.15.116:0/4034200292 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:54.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82cb7fe700 1 --2- 172.21.15.116:0/4034200292 >> v2:172.21.15.173:3300/0 conn(0x7f82cc0c3b60 0x7f82cc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82cb7fe700 1 -- 172.21.15.116:0/4034200292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82cc0c1970 msgr2=0x7f82cc150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:54.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82cb7fe700 1 --2- 172.21.15.116:0/4034200292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82cc0c1970 0x7f82cc150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82cb7fe700 1 -- 172.21.15.116:0/4034200292 >> v1:172.21.15.173:6789/0 conn(0x7f82cc0c3780 legacy=0x7f82cc0c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82cb7fe700 1 -- 172.21.15.116:0/4034200292 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc0a6bf0 con 0x7f82cc0c3b60 2020-11-20T12:19:54.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82caffd700 1 -- 172.21.15.116:0/4034200292 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1604869226 0 0) 0x7f82cc0a6950 con 0x7f82cc0c3780 2020-11-20T12:19:54.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82cb7fe700 1 --2- 172.21.15.116:0/4034200292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82cc0c3b60 0x7f82cc0a6280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f82c000b9d0 tx=0x7f82c0006df0).ready entity=mon.1 client_cookie=5726401ec3dfc5f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82caffd700 1 -- 172.21.15.116:0/4034200292 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0009770 con 0x7f82cc0c3b60 2020-11-20T12:19:54.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82caffd700 1 -- 172.21.15.116:0/4034200292 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c00098d0 con 0x7f82cc0c3b60 2020-11-20T12:19:54.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.308+0000 7f82caffd700 1 -- 172.21.15.116:0/4034200292 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c0009c50 con 0x7f82cc0c3b60 2020-11-20T12:19:54.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.312+0000 7f82d292a700 1 -- 172.21.15.116:0/4034200292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82cc0c3b60 msgr2=0x7f82cc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.312+0000 7f82d292a700 1 --2- 172.21.15.116:0/4034200292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82cc0c3b60 0x7f82cc0a6280 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f82c000b9d0 tx=0x7f82c0006df0).stop 2020-11-20T12:19:54.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.312+0000 7f82d292a700 1 -- 172.21.15.116:0/4034200292 shutdown_connections 2020-11-20T12:19:54.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.312+0000 7f82d292a700 1 --2- 172.21.15.116:0/4034200292 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82cc0c1970 0x7f82cc150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.312+0000 7f82d292a700 1 --2- 172.21.15.116:0/4034200292 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82cc0c3b60 0x7f82cc0a6280 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.312+0000 7f82d292a700 1 -- 172.21.15.116:0/4034200292 >> 172.21.15.116:0/4034200292 conn(0x7f82cc0c4f80 msgr2=0x7f82cc0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.312+0000 7f82d292a700 1 -- 172.21.15.116:0/4034200292 shutdown_connections 2020-11-20T12:19:54.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.312+0000 7f82d292a700 1 -- 172.21.15.116:0/4034200292 wait complete. 2020-11-20T12:19:54.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82d292a700 1 Processor -- start 2020-11-20T12:19:54.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82d292a700 1 -- start start 2020-11-20T12:19:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82d292a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82cc0c1970 0x7f82cc0a85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82d292a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82cc0c3780 0x7f82cc0bd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82d292a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82cc0c3b60 0x7f82cc0bdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82d292a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc152120 con 0x7f82cc0c3b60 2020-11-20T12:19:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82d292a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc1523c0 con 0x7f82cc0c3780 2020-11-20T12:19:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82d292a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc152660 con 0x7f82cc0c1970 2020-11-20T12:19:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82d0ec7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82cc0c3b60 0x7f82cc0bdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82d0ec7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82cc0c3b60 0x7f82cc0bdcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48628/0 (socket says 172.21.15.116:48628) 2020-11-20T12:19:54.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82d0ec7700 1 -- 172.21.15.116:0/2891097045 learned_addr learned my addr 172.21.15.116:0/2891097045 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82cb7fe700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82cc0c3780 0x7f82cc0bd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.316+0000 7f82cbfff700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82cc0c1970 0x7f82cc0a85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82d0ec7700 1 -- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82cc0c1970 msgr2=0x7f82cc0a85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82d0ec7700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82cc0c1970 0x7f82cc0a85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82d0ec7700 1 -- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82cc0c3780 msgr2=0x7f82cc0bd7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82d0ec7700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82cc0c3780 0x7f82cc0bd7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82d0ec7700 1 -- 172.21.15.116:0/2891097045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82cc0bd080 con 0x7f82cc0c3b60 2020-11-20T12:19:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82cb7fe700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82cc0c3780 0x7f82cc0bd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:54.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82d0ec7700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82cc0c3b60 0x7f82cc0bdcb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f82c400ed20 tx=0x7f82c40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82abfff700 1 -- 172.21.15.116:0/2891097045 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c400cb00 con 0x7f82cc0c3b60 2020-11-20T12:19:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82abfff700 1 -- 172.21.15.116:0/2891097045 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82c400cc60 con 0x7f82cc0c3b60 2020-11-20T12:19:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82d292a700 1 -- 172.21.15.116:0/2891097045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc0bd320 con 0x7f82cc0c3b60 2020-11-20T12:19:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82abfff700 1 -- 172.21.15.116:0/2891097045 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82c40123f0 con 0x7f82cc0c3b60 2020-11-20T12:19:54.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.320+0000 7f82d292a700 1 -- 172.21.15.116:0/2891097045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc0be340 con 0x7f82cc0c3b60 2020-11-20T12:19:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.324+0000 7f82d292a700 1 -- 172.21.15.116:0/2891097045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc05d2a0 con 0x7f82cc0c3b60 2020-11-20T12:19:54.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.324+0000 7f82abfff700 1 -- 172.21.15.116:0/2891097045 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f82c4022070 con 0x7f82cc0c3b60 2020-11-20T12:19:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.324+0000 7f82abfff700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82a40557c0 0x7f82a4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.324+0000 7f82abfff700 1 -- 172.21.15.116:0/2891097045 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f82c406e9d0 con 0x7f82cc0c3b60 2020-11-20T12:19:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.324+0000 7f82cbfff700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82a40557c0 0x7f82a4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.324+0000 7f82cbfff700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82a40557c0 0x7f82a4057c80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f82cc0677d0 tx=0x7f82bc007c20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.332+0000 7f82abfff700 1 -- 172.21.15.116:0/2891097045 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f82c4044ba0 con 0x7f82cc0c3b60 2020-11-20T12:19:54.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.492+0000 7f82d292a700 1 -- 172.21.15.116:0/2891097045 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f82cc0bd4b0 con 0x7f82cc0c3b60 2020-11-20T12:19:54.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.492+0000 7f82abfff700 1 -- 172.21.15.116:0/2891097045 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v80) v1 ==== 84+0+143 (secure 0 0 0) 0x7f82c4016070 con 0x7f82cc0c3b60 2020-11-20T12:19:54.490 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:54.490 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2020-11-20T12:19:54.491 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:54.491 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2020-11-20T12:19:54.491 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "device_health_metrics" 2020-11-20T12:19:54.491 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:19:54.492 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:19:54.492 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 2, 2020-11-20T12:19:54.492 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "rbd" 2020-11-20T12:19:54.492 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:19:54.492 INFO:teuthology.orchestra.run.smithi116.stdout:] 2020-11-20T12:19:54.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 -- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82a40557c0 msgr2=0x7f82a4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82a40557c0 0x7f82a4057c80 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f82cc0677d0 tx=0x7f82bc007c20).stop 2020-11-20T12:19:54.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 -- 172.21.15.116:0/2891097045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82cc0c3b60 msgr2=0x7f82cc0bdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82cc0c3b60 0x7f82cc0bdcb0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f82c400ed20 tx=0x7f82c40091d0).stop 2020-11-20T12:19:54.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 -- 172.21.15.116:0/2891097045 shutdown_connections 2020-11-20T12:19:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f82a40557c0 0x7f82a4057c80 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f82cc0c3b60 0x7f82cc0bdcb0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f82cc0c1970 0x7f82cc0a85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 --2- 172.21.15.116:0/2891097045 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f82cc0c3780 0x7f82cc0bd7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 -- 172.21.15.116:0/2891097045 >> 172.21.15.116:0/2891097045 conn(0x7f82cc0c4f80 msgr2=0x7f82cc0b9780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 -- 172.21.15.116:0/2891097045 shutdown_connections 2020-11-20T12:19:54.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.496+0000 7f82d292a700 1 -- 172.21.15.116:0/2891097045 wait complete. 2020-11-20T12:19:54.520 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T12:19:54.521 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-20T12:19:54.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.708+0000 7fb518248700 1 Processor -- start 2020-11-20T12:19:54.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.708+0000 7fb518248700 1 -- start start 2020-11-20T12:19:54.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb518248700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb510068500 0x7fb5100a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb518248700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5100c49a0 con 0x7fb510151630 2020-11-20T12:19:54.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb518248700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5100c4700 con 0x7fb5101558c0 2020-11-20T12:19:54.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb518248700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5100c9140 con 0x7fb510068500 2020-11-20T12:19:54.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb5167e5700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb510151630 0x7fb510151a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58070/0 (socket says 172.21.15.116:58070) 2020-11-20T12:19:54.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb5167e5700 1 -- 172.21.15.116:0/1726652623 learned_addr learned my addr 172.21.15.116:0/1726652623 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:54.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb5157e3700 1 --2- 172.21.15.116:0/1726652623 >> v2:172.21.15.173:3300/0 conn(0x7fb510068500 0x7fb5100a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb514fe2700 1 -- 172.21.15.116:0/1726652623 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824462546 0 0) 0x7fb5100c49a0 con 0x7fb510151630 2020-11-20T12:19:54.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb514fe2700 1 -- 172.21.15.116:0/1726652623 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4f8003240 con 0x7fb510151630 2020-11-20T12:19:54.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb514fe2700 1 -- 172.21.15.116:0/1726652623 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249391255 0 0) 0x7fb5100c4700 con 0x7fb5101558c0 2020-11-20T12:19:54.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb514fe2700 1 -- 172.21.15.116:0/1726652623 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5100c49a0 con 0x7fb5101558c0 2020-11-20T12:19:54.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb5157e3700 1 -- 172.21.15.116:0/1726652623 >> v1:172.21.15.173:6789/0 conn(0x7fb5101558c0 legacy=0x7fb510155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb5157e3700 1 -- 172.21.15.116:0/1726652623 >> v1:172.21.15.116:6789/0 conn(0x7fb510151630 legacy=0x7fb510151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.712+0000 7fb5157e3700 1 -- 172.21.15.116:0/1726652623 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100c93e0 con 0x7fb510068500 2020-11-20T12:19:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.716+0000 7fb5157e3700 1 --2- 172.21.15.116:0/1726652623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb510068500 0x7fb5100a8ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fb50000b9a0 tx=0x7fb500006df0).ready entity=mon.1 client_cookie=a3a5ef6b377daafb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.716+0000 7fb514fe2700 1 -- 172.21.15.116:0/1726652623 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500009680 con 0x7fb510068500 2020-11-20T12:19:54.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.716+0000 7fb514fe2700 1 -- 172.21.15.116:0/1726652623 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5000097e0 con 0x7fb510068500 2020-11-20T12:19:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.716+0000 7fb514fe2700 1 -- 172.21.15.116:0/1726652623 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500009b60 con 0x7fb510068500 2020-11-20T12:19:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.716+0000 7fb518248700 1 -- 172.21.15.116:0/1726652623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb510068500 msgr2=0x7fb5100a8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.716+0000 7fb518248700 1 --2- 172.21.15.116:0/1726652623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb510068500 0x7fb5100a8ee0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fb50000b9a0 tx=0x7fb500006df0).stop 2020-11-20T12:19:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.716+0000 7fb518248700 1 -- 172.21.15.116:0/1726652623 shutdown_connections 2020-11-20T12:19:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.716+0000 7fb518248700 1 --2- 172.21.15.116:0/1726652623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb510068500 0x7fb5100a8ee0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.716+0000 7fb518248700 1 -- 172.21.15.116:0/1726652623 >> 172.21.15.116:0/1726652623 conn(0x7fb510068080 msgr2=0x7fb5100c8920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:54.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.716+0000 7fb518248700 1 -- 172.21.15.116:0/1726652623 shutdown_connections 2020-11-20T12:19:54.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.716+0000 7fb518248700 1 -- 172.21.15.116:0/1726652623 wait complete. 2020-11-20T12:19:54.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb518248700 1 Processor -- start 2020-11-20T12:19:54.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb518248700 1 -- start start 2020-11-20T12:19:54.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb518248700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb510068500 0x7fb510161e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb518248700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb510151630 0x7fb51014fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb518248700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5101558c0 0x7fb51014eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb518248700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb510162330 con 0x7fb510151630 2020-11-20T12:19:54.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb518248700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb510164e40 con 0x7fb510068500 2020-11-20T12:19:54.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb518248700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5101650e0 con 0x7fb5101558c0 2020-11-20T12:19:54.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb5157e3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb510151630 0x7fb51014fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb515fe4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb510068500 0x7fb510161e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb5157e3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb510151630 0x7fb51014fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48640/0 (socket says 172.21.15.116:48640) 2020-11-20T12:19:54.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb5157e3700 1 -- 172.21.15.116:0/4171142816 learned_addr learned my addr 172.21.15.116:0/4171142816 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:54.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.720+0000 7fb5167e5700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5101558c0 0x7fb51014eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb5157e3700 1 -- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5101558c0 msgr2=0x7fb51014eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb5157e3700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5101558c0 0x7fb51014eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb5157e3700 1 -- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb510068500 msgr2=0x7fb510161e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb5157e3700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb510068500 0x7fb510161e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:54.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb5157e3700 1 -- 172.21.15.116:0/4171142816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb510165380 con 0x7fb510151630 2020-11-20T12:19:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb5157e3700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb510151630 0x7fb51014fb60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7fb500006b80 tx=0x7fb500015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb4fdffb700 1 -- 172.21.15.116:0/4171142816 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500011cc0 con 0x7fb510151630 2020-11-20T12:19:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb4fdffb700 1 -- 172.21.15.116:0/4171142816 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb500011e20 con 0x7fb510151630 2020-11-20T12:19:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb4fdffb700 1 -- 172.21.15.116:0/4171142816 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb50001e670 con 0x7fb510151630 2020-11-20T12:19:54.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb518248700 1 -- 172.21.15.116:0/4171142816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb510165620 con 0x7fb510151630 2020-11-20T12:19:54.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb518248700 1 -- 172.21.15.116:0/4171142816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb510150050 con 0x7fb510151630 2020-11-20T12:19:54.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.724+0000 7fb518248700 1 -- 172.21.15.116:0/4171142816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51005d2a0 con 0x7fb510151630 2020-11-20T12:19:54.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.728+0000 7fb4fdffb700 1 -- 172.21.15.116:0/4171142816 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb500025030 con 0x7fb510151630 2020-11-20T12:19:54.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.728+0000 7fb4fdffb700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb4e80557c0 0x7fb4e8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:54.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.728+0000 7fb4fdffb700 1 -- 172.21.15.116:0/4171142816 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb50006f900 con 0x7fb510151630 2020-11-20T12:19:54.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.728+0000 7fb515fe4700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb4e80557c0 0x7fb4e8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:54.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.728+0000 7fb515fe4700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb4e80557c0 0x7fb4e8057c80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fb50c007fd0 tx=0x7fb50c007a20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:54.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.736+0000 7fb4fdffb700 1 -- 172.21.15.116:0/4171142816 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb500045b80 con 0x7fb510151630 2020-11-20T12:19:54.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:54.900+0000 7fb518248700 1 -- 172.21.15.116:0/4171142816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb510096710 con 0x7fb510151630 2020-11-20T12:19:55.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.836+0000 7fb4fdffb700 1 -- 172.21.15.116:0/4171142816 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v83) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb500023070 con 0x7fb510151630 2020-11-20T12:19:55.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 -- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb4e80557c0 msgr2=0x7fb4e8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:55.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb4e80557c0 0x7fb4e8057c80 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7fb50c007fd0 tx=0x7fb50c007a20).stop 2020-11-20T12:19:55.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 -- 172.21.15.116:0/4171142816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb510151630 msgr2=0x7fb51014fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:55.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb510151630 0x7fb51014fb60 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7fb500006b80 tx=0x7fb500015000).stop 2020-11-20T12:19:55.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 -- 172.21.15.116:0/4171142816 shutdown_connections 2020-11-20T12:19:55.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb4e80557c0 0x7fb4e8057c80 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:55.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5101558c0 0x7fb51014eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:55.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb510068500 0x7fb510161e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:55.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 --2- 172.21.15.116:0/4171142816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb510151630 0x7fb51014fb60 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:55.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 -- 172.21.15.116:0/4171142816 >> 172.21.15.116:0/4171142816 conn(0x7fb510068080 msgr2=0x7fb510061930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:55.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 -- 172.21.15.116:0/4171142816 shutdown_connections 2020-11-20T12:19:55.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:55.844+0000 7fb518248700 1 -- 172.21.15.116:0/4171142816 wait complete. 2020-11-20T12:19:55.843 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2020-11-20T12:19:55.867 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-20T12:19:56.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.056+0000 7fa951286700 1 Processor -- start 2020-11-20T12:19:56.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.056+0000 7fa951286700 1 -- start start 2020-11-20T12:19:56.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa951286700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa94c0bf7d0 0x7fa94c0631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:56.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa951286700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c152cd0 con 0x7fa94c0bf3f0 2020-11-20T12:19:56.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa951286700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c152f70 con 0x7fa94c0a7160 2020-11-20T12:19:56.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa951286700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94c061400 con 0x7fa94c0bf7d0 2020-11-20T12:19:56.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94bfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa94c0bf3f0 0x7fa94c0a6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58084/0 (socket says 172.21.15.116:58084) 2020-11-20T12:19:56.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94bfff700 1 -- 172.21.15.116:0/2811679920 learned_addr learned my addr 172.21.15.116:0/2811679920 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:56.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa950a85700 1 --2- 172.21.15.116:0/2811679920 >> v2:172.21.15.173:3300/0 conn(0x7fa94c0bf7d0 0x7fa94c0631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:56.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2644432127 0 0) 0x7fa94c152cd0 con 0x7fa94c0bf3f0 2020-11-20T12:19:56.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa938003240 con 0x7fa94c0bf3f0 2020-11-20T12:19:56.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3685863322 0 0) 0x7fa94c152f70 con 0x7fa94c0a7160 2020-11-20T12:19:56.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa94c152cd0 con 0x7fa94c0a7160 2020-11-20T12:19:56.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1509799686 0 0) 0x7fa938003240 con 0x7fa94c0bf3f0 2020-11-20T12:19:56.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 >> v2:172.21.15.173:3300/0 conn(0x7fa94c0bf7d0 msgr2=0x7fa94c0631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:56.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 --2- 172.21.15.116:0/2811679920 >> v2:172.21.15.173:3300/0 conn(0x7fa94c0bf7d0 0x7fa94c0631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:56.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 >> v1:172.21.15.173:6789/0 conn(0x7fa94c0a7160 legacy=0x7fa94c14ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:56.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0616a0 con 0x7fa94c0bf3f0 2020-11-20T12:19:56.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fa93c0014e0 con 0x7fa94c0bf3f0 2020-11-20T12:19:56.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 >> v1:172.21.15.116:6789/0 conn(0x7fa94c0bf3f0 legacy=0x7fa94c0a6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:56.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 --2- 172.21.15.116:0/2811679920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9380038c0 0x7fa938003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:56.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 --2- 172.21.15.116:0/2811679920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9380044e0 0x7fa938004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:56.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 --2- 172.21.15.116:0/2811679920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa938004e50 0x7fa93800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:56.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c061400 con 0x7fa938004e50 2020-11-20T12:19:56.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93800bc20 con 0x7fa9380038c0 2020-11-20T12:19:56.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93800bde0 con 0x7fa9380044e0 2020-11-20T12:19:56.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa950a85700 1 --2- 172.21.15.116:0/2811679920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa938004e50 0x7fa93800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:56.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94b7fe700 1 --2- 172.21.15.116:0/2811679920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9380044e0 0x7fa938004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:56.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa94bfff700 1 --2- 172.21.15.116:0/2811679920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9380038c0 0x7fa938003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:56.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa950a85700 1 -- 172.21.15.116:0/2811679920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9380044e0 msgr2=0x7fa938004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:56.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa950a85700 1 --2- 172.21.15.116:0/2811679920 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9380044e0 0x7fa938004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:56.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa950a85700 1 -- 172.21.15.116:0/2811679920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9380038c0 msgr2=0x7fa938003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:56.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.060+0000 7fa950a85700 1 --2- 172.21.15.116:0/2811679920 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9380038c0 0x7fa938003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:56.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa950a85700 1 -- 172.21.15.116:0/2811679920 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa938003240 con 0x7fa938004e50 2020-11-20T12:19:56.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa950a85700 1 -- 172.21.15.116:0/2811679920 reap_dead start 2020-11-20T12:19:56.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa950a85700 1 --2- 172.21.15.116:0/2811679920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa938004e50 0x7fa93800b730 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fa944009ea0 tx=0x7fa9440091d0).ready entity=mon.0 client_cookie=df5a4643c2100e0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:56.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9440095c0 con 0x7fa938004e50 2020-11-20T12:19:56.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94400eda0 con 0x7fa938004e50 2020-11-20T12:19:56.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa94affd700 1 -- 172.21.15.116:0/2811679920 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa944012600 con 0x7fa938004e50 2020-11-20T12:19:56.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa951286700 1 -- 172.21.15.116:0/2811679920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa938004e50 msgr2=0x7fa93800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:56.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa951286700 1 --2- 172.21.15.116:0/2811679920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa938004e50 0x7fa93800b730 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7fa944009ea0 tx=0x7fa9440091d0).stop 2020-11-20T12:19:56.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa951286700 1 -- 172.21.15.116:0/2811679920 shutdown_connections 2020-11-20T12:19:56.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa951286700 1 --2- 172.21.15.116:0/2811679920 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa938004e50 0x7fa93800b730 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:56.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa951286700 1 -- 172.21.15.116:0/2811679920 >> 172.21.15.116:0/2811679920 conn(0x7fa94c0c16f0 msgr2=0x7fa94c0c1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:56.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa951286700 1 -- 172.21.15.116:0/2811679920 shutdown_connections 2020-11-20T12:19:56.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.064+0000 7fa951286700 1 -- 172.21.15.116:0/2811679920 wait complete. 2020-11-20T12:19:56.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.068+0000 7fa951286700 1 Processor -- start 2020-11-20T12:19:56.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.068+0000 7fa951286700 1 -- start start 2020-11-20T12:19:56.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.068+0000 7fa951286700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa938004e50 0x7fa94c146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:56.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.068+0000 7fa951286700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c154140 0x7fa94c0a7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:56.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.068+0000 7fa951286700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa94c0a80f0 0x7fa94c0a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:56.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.068+0000 7fa951286700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0a8510 con 0x7fa94c0a80f0 2020-11-20T12:19:56.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.068+0000 7fa951286700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0aa730 con 0x7fa938004e50 2020-11-20T12:19:56.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.068+0000 7fa951286700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c0aa9d0 con 0x7fa94c154140 2020-11-20T12:19:56.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.068+0000 7fa950a85700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa94c0a80f0 0x7fa94c0a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:56.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa950a85700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa94c0a80f0 0x7fa94c0a9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48666/0 (socket says 172.21.15.116:48666) 2020-11-20T12:19:56.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa950a85700 1 -- 172.21.15.116:0/1227647673 learned_addr learned my addr 172.21.15.116:0/1227647673 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:56.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa94b7fe700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c154140 0x7fa94c0a7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:56.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa94bfff700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa938004e50 0x7fa94c146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:56.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa950a85700 1 -- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c154140 msgr2=0x7fa94c0a7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:56.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa950a85700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c154140 0x7fa94c0a7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:56.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa950a85700 1 -- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa938004e50 msgr2=0x7fa94c146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:56.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa950a85700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa938004e50 0x7fa94c146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:56.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa950a85700 1 -- 172.21.15.116:0/1227647673 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0bc7e0 con 0x7fa94c0a80f0 2020-11-20T12:19:56.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa950a85700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa94c0a80f0 0x7fa94c0a9980 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fa94c0680c0 tx=0x7fa944019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:56.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa92bfff700 1 -- 172.21.15.116:0/1227647673 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94401c650 con 0x7fa94c0a80f0 2020-11-20T12:19:56.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa92bfff700 1 -- 172.21.15.116:0/1227647673 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94401c7b0 con 0x7fa94c0a80f0 2020-11-20T12:19:56.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa951286700 1 -- 172.21.15.116:0/1227647673 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c0bca80 con 0x7fa94c0a80f0 2020-11-20T12:19:56.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa92bfff700 1 -- 172.21.15.116:0/1227647673 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94401cb90 con 0x7fa94c0a80f0 2020-11-20T12:19:56.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa951286700 1 -- 172.21.15.116:0/1227647673 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c146ef0 con 0x7fa94c0a80f0 2020-11-20T12:19:56.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.072+0000 7fa951286700 1 -- 172.21.15.116:0/1227647673 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c0bcc10 con 0x7fa94c0a80f0 2020-11-20T12:19:56.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.076+0000 7fa92bfff700 1 -- 172.21.15.116:0/1227647673 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa94401a400 con 0x7fa94c0a80f0 2020-11-20T12:19:56.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.076+0000 7fa92bfff700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9240557c0 0x7fa924057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:56.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.076+0000 7fa92bfff700 1 -- 172.21.15.116:0/1227647673 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5607+0+0 (secure 0 0 0) 0x7fa9440706e0 con 0x7fa94c0a80f0 2020-11-20T12:19:56.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.076+0000 7fa94bfff700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9240557c0 0x7fa924057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:56.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.076+0000 7fa94bfff700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9240557c0 0x7fa924057c80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fa93c0010e0 tx=0x7fa93c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:56.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.084+0000 7fa92bfff700 1 -- 172.21.15.116:0/1227647673 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa9440465e0 con 0x7fa94c0a80f0 2020-11-20T12:19:56.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.252+0000 7fa951286700 1 -- 172.21.15.116:0/1227647673 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7fa94c05cf30 con 0x7fa94c0a80f0 2020-11-20T12:19:56.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.844+0000 7fa92bfff700 1 -- 172.21.15.116:0/1227647673 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v84) v1 ==== 125+0+0 (secure 0 0 0) 0x7fa9440160a0 con 0x7fa94c0a80f0 2020-11-20T12:19:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 -- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9240557c0 msgr2=0x7fa924057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9240557c0 0x7fa924057c80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fa93c0010e0 tx=0x7fa93c009040).stop 2020-11-20T12:19:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 -- 172.21.15.116:0/1227647673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa94c0a80f0 msgr2=0x7fa94c0a9980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:56.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa94c0a80f0 0x7fa94c0a9980 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fa94c0680c0 tx=0x7fa944019000).stop 2020-11-20T12:19:56.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 -- 172.21.15.116:0/1227647673 shutdown_connections 2020-11-20T12:19:56.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9240557c0 0x7fa924057c80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:56.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa94c0a80f0 0x7fa94c0a9980 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:56.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa938004e50 0x7fa94c146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:56.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 --2- 172.21.15.116:0/1227647673 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa94c154140 0x7fa94c0a7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:56.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 -- 172.21.15.116:0/1227647673 >> 172.21.15.116:0/1227647673 conn(0x7fa94c0c16f0 msgr2=0x7fa94c0c0740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:56.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 -- 172.21.15.116:0/1227647673 shutdown_connections 2020-11-20T12:19:56.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:56.852+0000 7fa951286700 1 -- 172.21.15.116:0/1227647673 wait complete. 2020-11-20T12:19:56.855 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2020-11-20T12:19:56.878 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T12:19:57.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.068+0000 7fcec1673700 1 Processor -- start 2020-11-20T12:19:57.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.068+0000 7fcec1673700 1 -- start start 2020-11-20T12:19:57.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcec1673700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcebc13af90 0x7fcebc13b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcec1673700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcebc0ae1c0 con 0x7fcebc0615f0 2020-11-20T12:19:57.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcec1673700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcebc0aaa90 con 0x7fcebc0aa5f0 2020-11-20T12:19:57.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcec1673700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcebc0aad30 con 0x7fcebc13af90 2020-11-20T12:19:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcebaffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcebc13af90 0x7fcebc13b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcebb7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fcebc0615f0 0x7fcebc0a89e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58108/0 (socket says 172.21.15.116:58108) 2020-11-20T12:19:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcebb7fe700 1 -- 172.21.15.116:0/343633011 learned_addr learned my addr 172.21.15.116:0/343633011 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:57.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcebaffd700 1 -- 172.21.15.116:0/343633011 >> v1:172.21.15.173:6789/0 conn(0x7fcebc0aa5f0 legacy=0x7fcebc0adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcebaffd700 1 -- 172.21.15.116:0/343633011 >> v1:172.21.15.116:6789/0 conn(0x7fcebc0615f0 legacy=0x7fcebc0a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcebaffd700 1 -- 172.21.15.116:0/343633011 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc0aafd0 con 0x7fcebc13af90 2020-11-20T12:19:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fceb9ffb700 1 -- 172.21.15.116:0/343633011 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659732304 0 0) 0x7fceb4001450 con 0x7fcebc0615f0 2020-11-20T12:19:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcebaffd700 1 --2- 172.21.15.116:0/343633011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcebc13af90 0x7fcebc13b370 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fceac00b9a0 tx=0x7fceac006df0).ready entity=mon.0 client_cookie=16cef2ad7e0147b3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:57.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fceb9ffb700 1 -- 172.21.15.116:0/343633011 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac009770 con 0x7fcebc13af90 2020-11-20T12:19:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fceb9ffb700 1 -- 172.21.15.116:0/343633011 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceac0098d0 con 0x7fcebc13af90 2020-11-20T12:19:57.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fceb9ffb700 1 -- 172.21.15.116:0/343633011 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac009c50 con 0x7fcebc13af90 2020-11-20T12:19:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcec1673700 1 -- 172.21.15.116:0/343633011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcebc13af90 msgr2=0x7fcebc13b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcec1673700 1 --2- 172.21.15.116:0/343633011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcebc13af90 0x7fcebc13b370 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7fceac00b9a0 tx=0x7fceac006df0).stop 2020-11-20T12:19:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcec1673700 1 -- 172.21.15.116:0/343633011 shutdown_connections 2020-11-20T12:19:57.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcec1673700 1 --2- 172.21.15.116:0/343633011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcebc13af90 0x7fcebc13b370 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcec1673700 1 -- 172.21.15.116:0/343633011 >> 172.21.15.116:0/343633011 conn(0x7fcebc0678a0 msgr2=0x7fcebc067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcec1673700 1 -- 172.21.15.116:0/343633011 shutdown_connections 2020-11-20T12:19:57.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.072+0000 7fcec1673700 1 -- 172.21.15.116:0/343633011 wait complete. 2020-11-20T12:19:57.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.076+0000 7fcec1673700 1 Processor -- start 2020-11-20T12:19:57.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.076+0000 7fcec1673700 1 -- start start 2020-11-20T12:19:57.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.076+0000 7fcec1673700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcebc0615f0 0x7fcebc138d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.076+0000 7fcec1673700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcebc0aa5f0 0x7fcebc161e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.076+0000 7fcec1673700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcebc13af90 0x7fcebc164aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.076+0000 7fcec1673700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc161470 con 0x7fcebc13af90 2020-11-20T12:19:57.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.076+0000 7fcec1673700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc139230 con 0x7fcebc0615f0 2020-11-20T12:19:57.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.076+0000 7fcec1673700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcebc165500 con 0x7fcebc0aa5f0 2020-11-20T12:19:57.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcebb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcebc13af90 0x7fcebc164aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcebb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcebc13af90 0x7fcebc164aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48680/0 (socket says 172.21.15.116:48680) 2020-11-20T12:19:57.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcebb7fe700 1 -- 172.21.15.116:0/1035717948 learned_addr learned my addr 172.21.15.116:0/1035717948 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:57.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fceba7fc700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcebc0aa5f0 0x7fcebc161e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcebaffd700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcebc0615f0 0x7fcebc138d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcebb7fe700 1 -- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcebc0aa5f0 msgr2=0x7fcebc161e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcebb7fe700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcebc0aa5f0 0x7fcebc161e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcebb7fe700 1 -- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcebc0615f0 msgr2=0x7fcebc138d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcebb7fe700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcebc0615f0 0x7fcebc138d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcebb7fe700 1 -- 172.21.15.116:0/1035717948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc1657a0 con 0x7fcebc13af90 2020-11-20T12:19:57.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcebb7fe700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcebc13af90 0x7fcebc164aa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fceb400ca30 tx=0x7fceb4007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:57.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcea2ffd700 1 -- 172.21.15.116:0/1035717948 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4005b80 con 0x7fcebc13af90 2020-11-20T12:19:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcea2ffd700 1 -- 172.21.15.116:0/1035717948 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceb4005ce0 con 0x7fcebc13af90 2020-11-20T12:19:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcec1673700 1 -- 172.21.15.116:0/1035717948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcebc1616c0 con 0x7fcebc13af90 2020-11-20T12:19:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcea2ffd700 1 -- 172.21.15.116:0/1035717948 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceb4007320 con 0x7fcebc13af90 2020-11-20T12:19:57.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.080+0000 7fcec1673700 1 -- 172.21.15.116:0/1035717948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcebc1622f0 con 0x7fcebc13af90 2020-11-20T12:19:57.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.084+0000 7fcec1673700 1 -- 172.21.15.116:0/1035717948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcebc165930 con 0x7fcebc13af90 2020-11-20T12:19:57.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.084+0000 7fcea2ffd700 1 -- 172.21.15.116:0/1035717948 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fceb4010030 con 0x7fcebc13af90 2020-11-20T12:19:57.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.084+0000 7fcea2ffd700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fce980557c0 0x7fce98057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.084+0000 7fcea2ffd700 1 -- 172.21.15.116:0/1035717948 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5957+0+0 (secure 0 0 0) 0x7fceb406e500 con 0x7fcebc13af90 2020-11-20T12:19:57.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.084+0000 7fcebaffd700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fce980557c0 0x7fce98057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.084+0000 7fcebaffd700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fce980557c0 0x7fce98057c80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fceac006b80 tx=0x7fceac015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:57.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.092+0000 7fcea2ffd700 1 -- 172.21.15.116:0/1035717948 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fceb4044300 con 0x7fcebc13af90 2020-11-20T12:19:57.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.300+0000 7fcec1673700 1 -- 172.21.15.116:0/1035717948 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7fcebc0ae1c0 con 0x7fcebc13af90 2020-11-20T12:19:57.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.312+0000 7fcea2ffd700 1 -- 172.21.15.116:0/1035717948 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 15 and data pool 16 v43) v1 ==== 189+0+0 (secure 0 0 0) 0x7fcebc0ae1c0 con 0x7fcebc13af90 2020-11-20T12:19:57.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.320+0000 7fcec1673700 1 -- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fce980557c0 msgr2=0x7fce98057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.320+0000 7fcec1673700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fce980557c0 0x7fce98057c80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7fceac006b80 tx=0x7fceac015000).stop 2020-11-20T12:19:57.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.320+0000 7fcec1673700 1 -- 172.21.15.116:0/1035717948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcebc13af90 msgr2=0x7fcebc164aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.320+0000 7fcec1673700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcebc13af90 0x7fcebc164aa0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7fceb400ca30 tx=0x7fceb4007be0).stop 2020-11-20T12:19:57.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.324+0000 7fcec1673700 1 -- 172.21.15.116:0/1035717948 shutdown_connections 2020-11-20T12:19:57.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.324+0000 7fcec1673700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fce980557c0 0x7fce98057c80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.324+0000 7fcec1673700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcebc13af90 0x7fcebc164aa0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.324+0000 7fcec1673700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcebc0615f0 0x7fcebc138d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.324+0000 7fcec1673700 1 --2- 172.21.15.116:0/1035717948 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcebc0aa5f0 0x7fcebc161e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.324+0000 7fcec1673700 1 -- 172.21.15.116:0/1035717948 >> 172.21.15.116:0/1035717948 conn(0x7fcebc0678a0 msgr2=0x7fcebc151dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:57.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.324+0000 7fcec1673700 1 -- 172.21.15.116:0/1035717948 shutdown_connections 2020-11-20T12:19:57.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.324+0000 7fcec1673700 1 -- 172.21.15.116:0/1035717948 wait complete. 2020-11-20T12:19:57.328 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 15 and data pool 16 2020-11-20T12:19:57.350 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:57.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.520+0000 7f9e61965700 1 Processor -- start 2020-11-20T12:19:57.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.520+0000 7f9e61965700 1 -- start start 2020-11-20T12:19:57.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.520+0000 7f9e61965700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9e5c066070 0x7f9e5c15c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.520+0000 7f9e61965700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5c060950 con 0x7f9e5c0615d0 2020-11-20T12:19:57.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.520+0000 7f9e61965700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e5c060bf0 con 0x7f9e5c151940 2020-11-20T12:19:57.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.520+0000 7f9e61965700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e5c0c4a90 con 0x7f9e5c066070 2020-11-20T12:19:57.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.520+0000 7f9e5a7fc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9e5c0615d0 0x7f9e5c065310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58128/0 (socket says 172.21.15.116:58128) 2020-11-20T12:19:57.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.520+0000 7f9e5a7fc700 1 -- 172.21.15.116:0/2742406894 learned_addr learned my addr 172.21.15.116:0/2742406894 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:57.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.520+0000 7f9e5bfff700 1 --2- 172.21.15.116:0/2742406894 >> v2:172.21.15.173:3300/0 conn(0x7f9e5c066070 0x7f9e5c15c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.520+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268735407 0 0) 0x7f9e5c060950 con 0x7f9e5c0615d0 2020-11-20T12:19:57.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.520+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e48003240 con 0x7f9e5c0615d0 2020-11-20T12:19:57.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2009426621 0 0) 0x7f9e5c060bf0 con 0x7f9e5c151940 2020-11-20T12:19:57.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e5c060950 con 0x7f9e5c151940 2020-11-20T12:19:57.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3431852087 0 0) 0x7f9e48003240 con 0x7f9e5c0615d0 2020-11-20T12:19:57.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 >> v2:172.21.15.173:3300/0 conn(0x7f9e5c066070 msgr2=0x7f9e5c15c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:57.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 --2- 172.21.15.116:0/2742406894 >> v2:172.21.15.173:3300/0 conn(0x7f9e5c066070 0x7f9e5c15c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 >> v1:172.21.15.173:6789/0 conn(0x7f9e5c151940 legacy=0x7f9e5c0c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c060e90 con 0x7f9e5c0615d0 2020-11-20T12:19:57.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f9e500014e0 con 0x7f9e5c0615d0 2020-11-20T12:19:57.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 >> v1:172.21.15.116:6789/0 conn(0x7f9e5c0615d0 legacy=0x7f9e5c065310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 --2- 172.21.15.116:0/2742406894 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e48003910 0x7f9e48003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 --2- 172.21.15.116:0/2742406894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e480044e0 0x7f9e48004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 --2- 172.21.15.116:0/2742406894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 0x7f9e4800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c0c4a90 con 0x7f9e48004e50 2020-11-20T12:19:57.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4800bc20 con 0x7f9e480044e0 2020-11-20T12:19:57.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4800bd90 con 0x7f9e48003910 2020-11-20T12:19:57.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e5bfff700 1 --2- 172.21.15.116:0/2742406894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 0x7f9e4800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e5affd700 1 --2- 172.21.15.116:0/2742406894 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e48003910 0x7f9e48003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e5a7fc700 1 --2- 172.21.15.116:0/2742406894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e480044e0 0x7f9e48004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e5bfff700 1 -- 172.21.15.116:0/2742406894 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e48003910 msgr2=0x7f9e48003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:57.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e5bfff700 1 --2- 172.21.15.116:0/2742406894 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e48003910 0x7f9e48003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e5bfff700 1 -- 172.21.15.116:0/2742406894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e480044e0 msgr2=0x7f9e48004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:57.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e5bfff700 1 --2- 172.21.15.116:0/2742406894 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e480044e0 0x7f9e48004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e5bfff700 1 -- 172.21.15.116:0/2742406894 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e48003240 con 0x7f9e48004e50 2020-11-20T12:19:57.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e5bfff700 1 -- 172.21.15.116:0/2742406894 reap_dead start 2020-11-20T12:19:57.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.524+0000 7f9e5bfff700 1 --2- 172.21.15.116:0/2742406894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 0x7f9e4800b730 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f9e54009ea0 tx=0x7f9e540091d0).ready entity=mon.0 client_cookie=fe648085f6d58fd0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:57.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.528+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e540095c0 con 0x7f9e48004e50 2020-11-20T12:19:57.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.528+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e5400ec40 con 0x7f9e48004e50 2020-11-20T12:19:57.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.528+0000 7f9e59ffb700 1 -- 172.21.15.116:0/2742406894 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e540123f0 con 0x7f9e48004e50 2020-11-20T12:19:57.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.528+0000 7f9e61965700 1 -- 172.21.15.116:0/2742406894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 msgr2=0x7f9e4800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.528+0000 7f9e61965700 1 --2- 172.21.15.116:0/2742406894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 0x7f9e4800b730 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f9e54009ea0 tx=0x7f9e540091d0).stop 2020-11-20T12:19:57.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.528+0000 7f9e61965700 1 -- 172.21.15.116:0/2742406894 shutdown_connections 2020-11-20T12:19:57.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.528+0000 7f9e61965700 1 --2- 172.21.15.116:0/2742406894 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 0x7f9e4800b730 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.528+0000 7f9e61965700 1 -- 172.21.15.116:0/2742406894 >> 172.21.15.116:0/2742406894 conn(0x7f9e5c0ca850 msgr2=0x7f9e5c0c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:57.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.528+0000 7f9e61965700 1 -- 172.21.15.116:0/2742406894 shutdown_connections 2020-11-20T12:19:57.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.528+0000 7f9e61965700 1 -- 172.21.15.116:0/2742406894 wait complete. 2020-11-20T12:19:57.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e61965700 1 Processor -- start 2020-11-20T12:19:57.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e61965700 1 -- start start 2020-11-20T12:19:57.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e61965700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 0x7f9e5c14f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e61965700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e5c0ab980 0x7f9e5c1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e61965700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e5c0abe00 0x7f9e5c1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e61965700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c161060 con 0x7f9e48004e50 2020-11-20T12:19:57.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e61965700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c161300 con 0x7f9e5c0ab980 2020-11-20T12:19:57.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e61965700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e5c1615a0 con 0x7f9e5c0abe00 2020-11-20T12:19:57.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e5affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 0x7f9e5c14f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e5affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 0x7f9e5c14f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48706/0 (socket says 172.21.15.116:48706) 2020-11-20T12:19:57.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e5affd700 1 -- 172.21.15.116:0/3047476425 learned_addr learned my addr 172.21.15.116:0/3047476425 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:57.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e5a7fc700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e5c0ab980 0x7f9e5c1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e5bfff700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e5c0abe00 0x7f9e5c1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e5affd700 1 -- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e5c0abe00 msgr2=0x7f9e5c1549e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e5affd700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e5c0abe00 0x7f9e5c1549e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e5affd700 1 -- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e5c0ab980 msgr2=0x7f9e5c1544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e5affd700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e5c0ab980 0x7f9e5c1544f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.532+0000 7f9e5affd700 1 -- 172.21.15.116:0/3047476425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e5c161840 con 0x7f9e48004e50 2020-11-20T12:19:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.536+0000 7f9e5affd700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 0x7f9e5c14f0b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f9e4c002cb0 tx=0x7f9e4c007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.536+0000 7f9e42ffd700 1 -- 172.21.15.116:0/3047476425 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c007020 con 0x7f9e48004e50 2020-11-20T12:19:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.536+0000 7f9e42ffd700 1 -- 172.21.15.116:0/3047476425 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e4c007180 con 0x7f9e48004e50 2020-11-20T12:19:57.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.536+0000 7f9e61965700 1 -- 172.21.15.116:0/3047476425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e5c161ae0 con 0x7f9e48004e50 2020-11-20T12:19:57.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.536+0000 7f9e42ffd700 1 -- 172.21.15.116:0/3047476425 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e4c00fa30 con 0x7f9e48004e50 2020-11-20T12:19:57.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.536+0000 7f9e61965700 1 -- 172.21.15.116:0/3047476425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e5c14f5a0 con 0x7f9e48004e50 2020-11-20T12:19:57.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.536+0000 7f9e61965700 1 -- 172.21.15.116:0/3047476425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e5c05d2a0 con 0x7f9e48004e50 2020-11-20T12:19:57.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.536+0000 7f9e42ffd700 1 -- 172.21.15.116:0/3047476425 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9e4c00fda0 con 0x7f9e48004e50 2020-11-20T12:19:57.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.536+0000 7f9e42ffd700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e380557c0 0x7f9e38057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.536+0000 7f9e42ffd700 1 -- 172.21.15.116:0/3047476425 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9e4c06e770 con 0x7f9e48004e50 2020-11-20T12:19:57.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.540+0000 7f9e5a7fc700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e380557c0 0x7f9e38057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.540+0000 7f9e5a7fc700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e380557c0 0x7f9e38057c80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f9e50007720 tx=0x7f9e5c060ae0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:57.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.544+0000 7f9e42ffd700 1 -- 172.21.15.116:0/3047476425 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9e4c044610 con 0x7f9e48004e50 2020-11-20T12:19:57.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.704+0000 7f9e61965700 1 -- 172.21.15.116:0/3047476425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e5c161c70 con 0x7f9e48004e50 2020-11-20T12:19:57.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.708+0000 7f9e42ffd700 1 -- 172.21.15.116:0/3047476425 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16890 (secure 0 0 0) 0x7f9e4c026210 con 0x7f9e48004e50 2020-11-20T12:19:57.706 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:57.706 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":85,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:57.303065+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:54.974437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:56.326129+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:57.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 -- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e380557c0 msgr2=0x7f9e38057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e380557c0 0x7f9e38057c80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f9e50007720 tx=0x7f9e5c060ae0).stop 2020-11-20T12:19:57.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 -- 172.21.15.116:0/3047476425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 msgr2=0x7f9e5c14f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 0x7f9e5c14f0b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f9e4c002cb0 tx=0x7f9e4c007b90).stop 2020-11-20T12:19:57.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 -- 172.21.15.116:0/3047476425 shutdown_connections 2020-11-20T12:19:57.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9e380557c0 0x7f9e38057c80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9e5c0abe00 0x7f9e5c1549e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9e48004e50 0x7f9e5c14f0b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 --2- 172.21.15.116:0/3047476425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9e5c0ab980 0x7f9e5c1544f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 -- 172.21.15.116:0/3047476425 >> 172.21.15.116:0/3047476425 conn(0x7f9e5c0ca850 msgr2=0x7f9e5c0c4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:57.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 -- 172.21.15.116:0/3047476425 shutdown_connections 2020-11-20T12:19:57.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.712+0000 7f9e61965700 1 -- 172.21.15.116:0/3047476425 wait complete. 2020-11-20T12:19:57.735 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:57.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.908+0000 7f6d042f8700 1 Processor -- start 2020-11-20T12:19:57.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.908+0000 7f6d042f8700 1 -- start start 2020-11-20T12:19:57.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d042f8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cfc0ba890 0x7f6cfc0bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d042f8700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cfc066140 con 0x7f6cfc148a20 2020-11-20T12:19:57.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d042f8700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cfc1526c0 con 0x7f6cfc0c2230 2020-11-20T12:19:57.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d042f8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc0c1d80 con 0x7f6cfc0ba890 2020-11-20T12:19:57.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d02895700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6cfc148a20 0x7f6cfc0c0080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58152/0 (socket says 172.21.15.116:58152) 2020-11-20T12:19:57.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d02895700 1 -- 172.21.15.116:0/1124374219 learned_addr learned my addr 172.21.15.116:0/1124374219 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:57.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d02094700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cfc0ba890 0x7f6cfc0bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1449070562 0 0) 0x7f6cfc066140 con 0x7f6cfc148a20 2020-11-20T12:19:57.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ce4003240 con 0x7f6cfc148a20 2020-11-20T12:19:57.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045890437 0 0) 0x7f6cfc1526c0 con 0x7f6cfc0c2230 2020-11-20T12:19:57.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cfc066140 con 0x7f6cfc0c2230 2020-11-20T12:19:57.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1271879249 0 0) 0x7f6ce4003240 con 0x7f6cfc148a20 2020-11-20T12:19:57.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cfc0ba890 msgr2=0x7f6cfc0bac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:57.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cfc0ba890 0x7f6cfc0bac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 >> v1:172.21.15.173:6789/0 conn(0x7f6cfc0c2230 legacy=0x7f6cfc0c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc152960 con 0x7f6cfc148a20 2020-11-20T12:19:57.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f6cf4001860 con 0x7f6cfc148a20 2020-11-20T12:19:57.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 >> v1:172.21.15.116:6789/0 conn(0x7f6cfc148a20 legacy=0x7f6cfc0c0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ce40038f0 0x7f6ce4003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ce40044e0 0x7f6ce4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ce4004e50 0x7f6ce400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc0c1d80 con 0x7f6ce4004e50 2020-11-20T12:19:57.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce400bc20 con 0x7f6ce40038f0 2020-11-20T12:19:57.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce400bd60 con 0x7f6ce40044e0 2020-11-20T12:19:57.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d02094700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cfc0ba890 0x7f6cfc0bac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:19:57.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d02895700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ce4004e50 0x7f6ce400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d01893700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ce40044e0 0x7f6ce4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d02094700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ce40038f0 0x7f6ce4003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.912+0000 7f6d02895700 1 -- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ce40044e0 msgr2=0x7f6ce4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:57.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d02895700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ce40044e0 0x7f6ce4004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d02895700 1 -- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ce40038f0 msgr2=0x7f6ce4003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:57.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d02895700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ce40038f0 0x7f6ce4003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d02895700 1 -- 172.21.15.116:0/1124374219 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4003240 con 0x7f6ce4004e50 2020-11-20T12:19:57.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d01893700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ce40044e0 0x7f6ce4004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:19:57.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d02895700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ce4004e50 0x7f6ce400b730 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f6cf400c560 tx=0x7f6cf4007a20).ready entity=mon.0 client_cookie=ed793a6f2df576cf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:57.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4005b80 con 0x7f6ce4004e50 2020-11-20T12:19:57.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf4005ce0 con 0x7f6ce4004e50 2020-11-20T12:19:57.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d01092700 1 -- 172.21.15.116:0/1124374219 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf4007300 con 0x7f6ce4004e50 2020-11-20T12:19:57.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d042f8700 1 -- 172.21.15.116:0/1124374219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ce4004e50 msgr2=0x7f6ce400b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d042f8700 1 --2- 172.21.15.116:0/1124374219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ce4004e50 0x7f6ce400b730 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f6cf400c560 tx=0x7f6cf4007a20).stop 2020-11-20T12:19:57.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d02895700 1 -- 172.21.15.116:0/1124374219 reap_dead start 2020-11-20T12:19:57.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d042f8700 1 -- 172.21.15.116:0/1124374219 shutdown_connections 2020-11-20T12:19:57.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d042f8700 1 -- 172.21.15.116:0/1124374219 >> 172.21.15.116:0/1124374219 conn(0x7f6cfc0c4f80 msgr2=0x7f6cfc0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:57.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d042f8700 1 -- 172.21.15.116:0/1124374219 shutdown_connections 2020-11-20T12:19:57.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.916+0000 7f6d042f8700 1 -- 172.21.15.116:0/1124374219 wait complete. 2020-11-20T12:19:57.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d042f8700 1 Processor -- start 2020-11-20T12:19:57.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d042f8700 1 -- start start 2020-11-20T12:19:57.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d042f8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ce4004e50 0x7f6cfc0a7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d042f8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cfc150110 0x7f6cfc133e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d042f8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cfc150590 0x7f6cfc1342f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d042f8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc1347e0 con 0x7f6cfc150110 2020-11-20T12:19:57.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d042f8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc161170 con 0x7f6ce4004e50 2020-11-20T12:19:57.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d042f8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cfc161390 con 0x7f6cfc150590 2020-11-20T12:19:57.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d01893700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cfc150110 0x7f6cfc133e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d02895700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cfc150590 0x7f6cfc1342f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d02094700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ce4004e50 0x7f6cfc0a7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d01893700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cfc150110 0x7f6cfc133e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48730/0 (socket says 172.21.15.116:48730) 2020-11-20T12:19:57.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.920+0000 7f6d01893700 1 -- 172.21.15.116:0/1034339625 learned_addr learned my addr 172.21.15.116:0/1034339625 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:57.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6d01893700 1 -- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cfc150590 msgr2=0x7f6cfc1342f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6d01893700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cfc150590 0x7f6cfc1342f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6d01893700 1 -- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ce4004e50 msgr2=0x7f6cfc0a7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:57.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6d01893700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ce4004e50 0x7f6cfc0a7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:57.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6d01893700 1 -- 172.21.15.116:0/1034339625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cfc161630 con 0x7f6cfc150110 2020-11-20T12:19:57.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6d01893700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cfc150110 0x7f6cfc133e00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f6cec000bc0 tx=0x7f6cec009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:57.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6cea7fc700 1 -- 172.21.15.116:0/1034339625 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec001580 con 0x7f6cfc150110 2020-11-20T12:19:57.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6cea7fc700 1 -- 172.21.15.116:0/1034339625 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec010040 con 0x7f6cfc150110 2020-11-20T12:19:57.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6d042f8700 1 -- 172.21.15.116:0/1034339625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cfc1618d0 con 0x7f6cfc150110 2020-11-20T12:19:57.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6cea7fc700 1 -- 172.21.15.116:0/1034339625 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec0018f0 con 0x7f6cfc150110 2020-11-20T12:19:57.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6d042f8700 1 -- 172.21.15.116:0/1034339625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cfc161e10 con 0x7f6cfc150110 2020-11-20T12:19:57.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.924+0000 7f6d042f8700 1 -- 172.21.15.116:0/1034339625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cfc05d2a0 con 0x7f6cfc150110 2020-11-20T12:19:57.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.928+0000 7f6cea7fc700 1 -- 172.21.15.116:0/1034339625 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6cec001c60 con 0x7f6cfc150110 2020-11-20T12:19:57.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.928+0000 7f6cea7fc700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cd40557c0 0x7f6cd4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:57.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.928+0000 7f6cea7fc700 1 -- 172.21.15.116:0/1034339625 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6cec06e920 con 0x7f6cfc150110 2020-11-20T12:19:57.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.928+0000 7f6d02094700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cd40557c0 0x7f6cd4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:57.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.928+0000 7f6d02094700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cd40557c0 0x7f6cd4057c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f6cfc05c830 tx=0x7f6cf8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:57.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:57.932+0000 7f6cea7fc700 1 -- 172.21.15.116:0/1034339625 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6cec0446b0 con 0x7f6cfc150110 2020-11-20T12:19:58.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.092+0000 7f6d042f8700 1 -- 172.21.15.116:0/1034339625 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6cfc0c4150 con 0x7f6cfc150110 2020-11-20T12:19:58.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.096+0000 7f6cea7fc700 1 -- 172.21.15.116:0/1034339625 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16890 (secure 0 0 0) 0x7f6cec047cd0 con 0x7f6cfc150110 2020-11-20T12:19:58.094 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:58.095 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":85,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:57.303065+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:54.974437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:56.326129+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:19:58.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 -- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cd40557c0 msgr2=0x7f6cd4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cd40557c0 0x7f6cd4057c80 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f6cfc05c830 tx=0x7f6cf8008040).stop 2020-11-20T12:19:58.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 -- 172.21.15.116:0/1034339625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cfc150110 msgr2=0x7f6cfc133e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:58.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cfc150110 0x7f6cfc133e00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f6cec000bc0 tx=0x7f6cec009040).stop 2020-11-20T12:19:58.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 -- 172.21.15.116:0/1034339625 shutdown_connections 2020-11-20T12:19:58.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cd40557c0 0x7f6cd4057c80 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cfc150590 0x7f6cfc1342f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ce4004e50 0x7f6cfc0a7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 --2- 172.21.15.116:0/1034339625 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cfc150110 0x7f6cfc133e00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 -- 172.21.15.116:0/1034339625 >> 172.21.15.116:0/1034339625 conn(0x7f6cfc0c4f80 msgr2=0x7f6cfc151600 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:58.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 -- 172.21.15.116:0/1034339625 shutdown_connections 2020-11-20T12:19:58.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.100+0000 7f6d042f8700 1 -- 172.21.15.116:0/1034339625 wait complete. 2020-11-20T12:19:58.121 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-20T12:19:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.312+0000 7f0dec676700 1 Processor -- start 2020-11-20T12:19:58.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.312+0000 7f0dec676700 1 -- start start 2020-11-20T12:19:58.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.312+0000 7f0dec676700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de4149df0 0x7f0de4153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:58.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.312+0000 7f0dec676700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0de40a9750 0x7f0de40c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.312+0000 7f0dec676700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de40a9c20 con 0x7f0de414a1d0 2020-11-20T12:19:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.312+0000 7f0dec676700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de40a9ec0 con 0x7f0de40a9750 2020-11-20T12:19:58.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.312+0000 7f0dec676700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de4153670 con 0x7f0de4149df0 2020-11-20T12:19:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.312+0000 7f0deac13700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0de414a1d0 0x7f0de40be240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58170/0 (socket says 172.21.15.116:58170) 2020-11-20T12:19:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.312+0000 7f0deac13700 1 -- 172.21.15.116:0/2224482730 learned_addr learned my addr 172.21.15.116:0/2224482730 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:19:58.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.312+0000 7f0de9c11700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0de40a9750 0x7f0de40c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:58.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.312+0000 7f0dea412700 1 --2- 172.21.15.116:0/2224482730 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de4149df0 0x7f0de4153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0de9410700 1 -- 172.21.15.116:0/2224482730 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506211920 0 0) 0x7f0de40a9c20 con 0x7f0de414a1d0 2020-11-20T12:19:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0de9410700 1 -- 172.21.15.116:0/2224482730 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dd0003240 con 0x7f0de414a1d0 2020-11-20T12:19:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0de9c11700 1 -- 172.21.15.116:0/2224482730 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de4149df0 msgr2=0x7f0de4153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:58.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0de9c11700 1 --2- 172.21.15.116:0/2224482730 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de4149df0 0x7f0de4153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0de9c11700 1 -- 172.21.15.116:0/2224482730 >> v1:172.21.15.116:6789/0 conn(0x7f0de414a1d0 legacy=0x7f0de40be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0de9c11700 1 -- 172.21.15.116:0/2224482730 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de4153910 con 0x7f0de40a9750 2020-11-20T12:19:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0de9c11700 1 --2- 172.21.15.116:0/2224482730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de40a9750 0x7f0de40c0700 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f0dd800b9a0 tx=0x7f0dd8006df0).ready entity=mon.0 client_cookie=cd300b22e5998414 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0de9410700 1 -- 172.21.15.116:0/2224482730 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd80097b0 con 0x7f0de40a9750 2020-11-20T12:19:58.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0de9410700 1 -- 172.21.15.116:0/2224482730 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd8009910 con 0x7f0de40a9750 2020-11-20T12:19:58.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0de9410700 1 -- 172.21.15.116:0/2224482730 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8009c90 con 0x7f0de40a9750 2020-11-20T12:19:58.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0dec676700 1 -- 172.21.15.116:0/2224482730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de40a9750 msgr2=0x7f0de40c0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0dec676700 1 --2- 172.21.15.116:0/2224482730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de40a9750 0x7f0de40c0700 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f0dd800b9a0 tx=0x7f0dd8006df0).stop 2020-11-20T12:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0dec676700 1 -- 172.21.15.116:0/2224482730 shutdown_connections 2020-11-20T12:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0dec676700 1 --2- 172.21.15.116:0/2224482730 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de4149df0 0x7f0de4153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0dec676700 1 --2- 172.21.15.116:0/2224482730 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de40a9750 0x7f0de40c0700 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0dec676700 1 -- 172.21.15.116:0/2224482730 >> 172.21.15.116:0/2224482730 conn(0x7f0de405ba80 msgr2=0x7f0de40c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0dec676700 1 -- 172.21.15.116:0/2224482730 shutdown_connections 2020-11-20T12:19:58.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.316+0000 7f0dec676700 1 -- 172.21.15.116:0/2224482730 wait complete. 2020-11-20T12:19:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.320+0000 7f0dec676700 1 Processor -- start 2020-11-20T12:19:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.320+0000 7f0dec676700 1 -- start start 2020-11-20T12:19:58.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.320+0000 7f0dec676700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de40a9750 0x7f0de4162ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.320+0000 7f0dec676700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de4149df0 0x7f0de40c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.320+0000 7f0dec676700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de414a1d0 0x7f0de414ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.320+0000 7f0dec676700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de414b7f0 con 0x7f0de40a9750 2020-11-20T12:19:58.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.320+0000 7f0dec676700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de41633c0 con 0x7f0de4149df0 2020-11-20T12:19:58.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.320+0000 7f0dec676700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de414b2b0 con 0x7f0de414a1d0 2020-11-20T12:19:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dea412700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de40a9750 0x7f0de4162ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:58.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dea412700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de40a9750 0x7f0de4162ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48740/0 (socket says 172.21.15.116:48740) 2020-11-20T12:19:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dea412700 1 -- 172.21.15.116:0/2507119549 learned_addr learned my addr 172.21.15.116:0/2507119549 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0deac13700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de414a1d0 0x7f0de414ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dea412700 1 -- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de414a1d0 msgr2=0x7f0de414ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dea412700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de414a1d0 0x7f0de414ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:58.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dea412700 1 -- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de4149df0 msgr2=0x7f0de40c3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:58.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dea412700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de4149df0 0x7f0de40c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:58.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dea412700 1 -- 172.21.15.116:0/2507119549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de414b550 con 0x7f0de40a9750 2020-11-20T12:19:58.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dea412700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de40a9750 0x7f0de4162ed0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f0dd400ec90 tx=0x7f0dd4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:58.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0de27fc700 1 -- 172.21.15.116:0/2507119549 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd400c940 con 0x7f0de40a9750 2020-11-20T12:19:58.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0de27fc700 1 -- 172.21.15.116:0/2507119549 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd400caa0 con 0x7f0de40a9750 2020-11-20T12:19:58.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dec676700 1 -- 172.21.15.116:0/2507119549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0de4146ea0 con 0x7f0de40a9750 2020-11-20T12:19:58.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0de27fc700 1 -- 172.21.15.116:0/2507119549 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd400ce20 con 0x7f0de40a9750 2020-11-20T12:19:58.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dec676700 1 -- 172.21.15.116:0/2507119549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de40c2380 con 0x7f0de40a9750 2020-11-20T12:19:58.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0dec676700 1 -- 172.21.15.116:0/2507119549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de405d2a0 con 0x7f0de40a9750 2020-11-20T12:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.324+0000 7f0de27fc700 1 -- 172.21.15.116:0/2507119549 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0dd4013640 con 0x7f0de40a9750 2020-11-20T12:19:58.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.328+0000 7f0de27fc700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dbc0557c0 0x7f0dbc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:58.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.328+0000 7f0de27fc700 1 -- 172.21.15.116:0/2507119549 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0dd406ea50 con 0x7f0de40a9750 2020-11-20T12:19:58.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.328+0000 7f0de9c11700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dbc0557c0 0x7f0dbc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:58.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.328+0000 7f0de9c11700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dbc0557c0 0x7f0dbc057c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f0de40611a0 tx=0x7f0dd8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:58.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.332+0000 7f0de27fc700 1 -- 172.21.15.116:0/2507119549 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0dd4044890 con 0x7f0de40a9750 2020-11-20T12:19:58.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:58.540+0000 7f0dec676700 1 -- 172.21.15.116:0/2507119549 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0de4061380 con 0x7f0de40a9750 2020-11-20T12:19:59.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.336+0000 7f0de27fc700 1 -- 172.21.15.116:0/2507119549 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v46) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0dd4047eb0 con 0x7f0de40a9750 2020-11-20T12:19:59.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.336+0000 7f0dec676700 1 -- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dbc0557c0 msgr2=0x7f0dbc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:59.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.336+0000 7f0dec676700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dbc0557c0 0x7f0dbc057c80 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f0de40611a0 tx=0x7f0dd8015000).stop 2020-11-20T12:19:59.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.336+0000 7f0dec676700 1 -- 172.21.15.116:0/2507119549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de40a9750 msgr2=0x7f0de4162ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:59.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.336+0000 7f0dec676700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de40a9750 0x7f0de4162ed0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f0dd400ec90 tx=0x7f0dd4008040).stop 2020-11-20T12:19:59.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.340+0000 7f0dec676700 1 -- 172.21.15.116:0/2507119549 shutdown_connections 2020-11-20T12:19:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.340+0000 7f0dec676700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0dbc0557c0 0x7f0dbc057c80 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.340+0000 7f0dec676700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0de414a1d0 0x7f0de414ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.340+0000 7f0dec676700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0de40a9750 0x7f0de4162ed0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.340+0000 7f0dec676700 1 --2- 172.21.15.116:0/2507119549 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0de4149df0 0x7f0de40c3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:19:59.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.340+0000 7f0dec676700 1 -- 172.21.15.116:0/2507119549 >> 172.21.15.116:0/2507119549 conn(0x7f0de405ba80 msgr2=0x7f0de40667e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:59.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.340+0000 7f0dec676700 1 -- 172.21.15.116:0/2507119549 shutdown_connections 2020-11-20T12:19:59.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.340+0000 7f0dec676700 1 -- 172.21.15.116:0/2507119549 wait complete. 2020-11-20T12:19:59.363 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:19:59.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.548+0000 7f19bd8e6700 1 Processor -- start 2020-11-20T12:19:59.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.548+0000 7f19bd8e6700 1 -- start start 2020-11-20T12:19:59.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.548+0000 7f19bd8e6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19b8153450 0x7f19b8062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:59.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.548+0000 7f19bd8e6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f19b80677a0 0x7f19b80b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:59.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.548+0000 7f19bd8e6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b81543a0 con 0x7f19b8062880 2020-11-20T12:19:59.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.548+0000 7f19bd8e6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b8153920 con 0x7f19b80677a0 2020-11-20T12:19:59.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.548+0000 7f19bd8e6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b8153bc0 con 0x7f19b8153450 2020-11-20T12:19:59.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd0e5700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f19b80677a0 0x7f19b80b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:59.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd0e5700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f19b80677a0 0x7f19b80b9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48752/0 (socket says 172.21.15.116:48752) 2020-11-20T12:19:59.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd0e5700 1 -- 172.21.15.116:0/3893051618 learned_addr learned my addr 172.21.15.116:0/3893051618 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:59.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bc8e4700 1 --1- 172.21.15.116:0/3893051618 >> v1:172.21.15.116:6789/0 conn(0x7f19b8062880 0x7f19b80c4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58184/0 (socket says 172.21.15.116:58184) 2020-11-20T12:19:59.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19affff700 1 --2- 172.21.15.116:0/3893051618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19b8153450 0x7f19b8062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:59.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd0e5700 1 -- 172.21.15.116:0/3893051618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19b8153450 msgr2=0x7f19b8062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:19:59.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd0e5700 1 --2- 172.21.15.116:0/3893051618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19b8153450 0x7f19b8062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd0e5700 1 -- 172.21.15.116:0/3893051618 >> v1:172.21.15.116:6789/0 conn(0x7f19b8062880 legacy=0x7f19b80c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:59.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd0e5700 1 -- 172.21.15.116:0/3893051618 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b8067c30 con 0x7f19b80677a0 2020-11-20T12:19:59.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd0e5700 1 --2- 172.21.15.116:0/3893051618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19b80677a0 0x7f19b80b9710 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f19b0009d50 tx=0x7f19b0009290).ready entity=mon.0 client_cookie=a9395f71466f246f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19af7fe700 1 -- 172.21.15.116:0/3893051618 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b000bc80 con 0x7f19b80677a0 2020-11-20T12:19:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19af7fe700 1 -- 172.21.15.116:0/3893051618 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b000bde0 con 0x7f19b80677a0 2020-11-20T12:19:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19af7fe700 1 -- 172.21.15.116:0/3893051618 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b0011610 con 0x7f19b80677a0 2020-11-20T12:19:59.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3893051618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19b80677a0 msgr2=0x7f19b80b9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd8e6700 1 --2- 172.21.15.116:0/3893051618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19b80677a0 0x7f19b80b9710 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f19b0009d50 tx=0x7f19b0009290).stop 2020-11-20T12:19:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3893051618 shutdown_connections 2020-11-20T12:19:59.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd8e6700 1 --2- 172.21.15.116:0/3893051618 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19b80677a0 0x7f19b80b9710 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd8e6700 1 --2- 172.21.15.116:0/3893051618 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19b8153450 0x7f19b8062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3893051618 >> 172.21.15.116:0/3893051618 conn(0x7f19b8060c40 msgr2=0x7f19b80c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:59.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3893051618 shutdown_connections 2020-11-20T12:19:59.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.552+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3893051618 wait complete. 2020-11-20T12:19:59.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.556+0000 7f19bd8e6700 1 Processor -- start 2020-11-20T12:19:59.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.556+0000 7f19bd8e6700 1 -- start start 2020-11-20T12:19:59.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.556+0000 7f19bd8e6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19b8062880 0x7f19b80c00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:59.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.556+0000 7f19bd8e6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b80677a0 0x7f19b8164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:59.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.556+0000 7f19bd8e6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19b8153450 0x7f19b81651c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:59.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.556+0000 7f19bd8e6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80c25b0 con 0x7f19b8062880 2020-11-20T12:19:59.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.556+0000 7f19bd8e6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80c2850 con 0x7f19b80677a0 2020-11-20T12:19:59.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.556+0000 7f19bd8e6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b80c2af0 con 0x7f19b8153450 2020-11-20T12:19:59.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.556+0000 7f19bc8e4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19b8062880 0x7f19b80c00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:59.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bc8e4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19b8062880 0x7f19b80c00b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48754/0 (socket says 172.21.15.116:48754) 2020-11-20T12:19:59.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bc8e4700 1 -- 172.21.15.116:0/3976169405 learned_addr learned my addr 172.21.15.116:0/3976169405 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:19:59.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19affff700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b80677a0 0x7f19b8164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:59.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bd0e5700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19b8153450 0x7f19b81651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:59.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bc8e4700 1 -- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19b8153450 msgr2=0x7f19b81651c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:59.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bc8e4700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19b8153450 0x7f19b81651c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bc8e4700 1 -- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b80677a0 msgr2=0x7f19b8164cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bc8e4700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b80677a0 0x7f19b8164cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bc8e4700 1 -- 172.21.15.116:0/3976169405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b80c2d90 con 0x7f19b8062880 2020-11-20T12:19:59.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bc8e4700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19b8062880 0x7f19b80c00b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f19b400c710 tx=0x7f19b4007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19acff9700 1 -- 172.21.15.116:0/3976169405 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b40072e0 con 0x7f19b8062880 2020-11-20T12:19:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19acff9700 1 -- 172.21.15.116:0/3976169405 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b4005530 con 0x7f19b8062880 2020-11-20T12:19:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3976169405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19b80c3030 con 0x7f19b8062880 2020-11-20T12:19:59.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19acff9700 1 -- 172.21.15.116:0/3976169405 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b40058b0 con 0x7f19b8062880 2020-11-20T12:19:59.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3976169405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b8136d70 con 0x7f19b8062880 2020-11-20T12:19:59.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.560+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3976169405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b80c33f0 con 0x7f19b8062880 2020-11-20T12:19:59.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.564+0000 7f19acff9700 1 -- 172.21.15.116:0/3976169405 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f19b401a060 con 0x7f19b8062880 2020-11-20T12:19:59.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.564+0000 7f19acff9700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19900557c0 0x7f1990057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:19:59.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.564+0000 7f19acff9700 1 -- 172.21.15.116:0/3976169405 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f19b406df50 con 0x7f19b8062880 2020-11-20T12:19:59.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.564+0000 7f19affff700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19900557c0 0x7f1990057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:19:59.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.564+0000 7f19affff700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19900557c0 0x7f1990057c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f19a8006fd0 tx=0x7f19a8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:19:59.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.572+0000 7f19acff9700 1 -- 172.21.15.116:0/3976169405 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f19b4043ce0 con 0x7f19b8062880 2020-11-20T12:19:59.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.780+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3976169405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19b805cf30 con 0x7f19b8062880 2020-11-20T12:19:59.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.780+0000 7f19acff9700 1 -- 172.21.15.116:0/3976169405 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2848 (secure 0 0 0) 0x7f19b4018030 con 0x7f19b8062880 2020-11-20T12:19:59.778 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:19:59.778 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/313210621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":313210621},{"type":"v1","addr":"172.21.15.173:6835","nonce":313210621}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/2140340557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2140340557},{"type":"v1","addr":"172.21.15.173:6837","nonce":2140340557}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3544207168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3544207168},{"type":"v1","addr":"172.21.15.116:6833","nonce":3544207168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:57.303415+0000","modified":"2020-11-20T12:19:59.322603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/657163487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":657163487},{"type":"v1","addr":"172.21.15.116:6835","nonce":657163487}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-20T12:19:59.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19900557c0 msgr2=0x7f1990057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:59.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19900557c0 0x7f1990057c80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f19a8006fd0 tx=0x7f19a8008040).stop 2020-11-20T12:19:59.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3976169405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19b8062880 msgr2=0x7f19b80c00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:19:59.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19b8062880 0x7f19b80c00b0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f19b400c710 tx=0x7f19b4007c40).stop 2020-11-20T12:19:59.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3976169405 shutdown_connections 2020-11-20T12:19:59.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19900557c0 0x7f1990057c80 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19b8153450 0x7f19b81651c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19b8062880 0x7f19b80c00b0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 --2- 172.21.15.116:0/3976169405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19b80677a0 0x7f19b8164cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:19:59.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3976169405 >> 172.21.15.116:0/3976169405 conn(0x7f19b8060c40 msgr2=0x7f19b81528f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:19:59.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3976169405 shutdown_connections 2020-11-20T12:19:59.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.784+0000 7f19bd8e6700 1 -- 172.21.15.116:0/3976169405 wait complete. 2020-11-20T12:19:59.784 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 46 2020-11-20T12:19:59.803 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:19:59.825 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:19:59.831 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:19:59.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.984+0000 7f2d7d40f700 1 Processor -- start 2020-11-20T12:19:59.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.984+0000 7f2d7d40f700 1 -- start start 2020-11-20T12:20:00.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d7d40f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2d780bf020 0x7f2d780c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d7d40f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d780c3600 con 0x7f2d780c3c90 2020-11-20T12:20:00.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d7d40f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d780c38a0 con 0x7f2d78151e90 2020-11-20T12:20:00.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d7d40f700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d780c72a0 con 0x7f2d780bf020 2020-11-20T12:20:00.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d76ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2d780c3c90 0x7f2d780c79e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58196/0 (socket says 172.21.15.116:58196) 2020-11-20T12:20:00.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d76ffd700 1 -- 172.21.15.116:0/3910967966 learned_addr learned my addr 172.21.15.116:0/3910967966 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d777fe700 1 --2- 172.21.15.116:0/3910967966 >> v2:172.21.15.173:3300/0 conn(0x7f2d780bf020 0x7f2d780c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238721459 0 0) 0x7f2d780c3600 con 0x7f2d780c3c90 2020-11-20T12:20:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d64003240 con 0x7f2d780c3c90 2020-11-20T12:20:00.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 198243500 0 0) 0x7f2d780c38a0 con 0x7f2d78151e90 2020-11-20T12:20:00.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d780c3600 con 0x7f2d78151e90 2020-11-20T12:20:00.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2745306532 0 0) 0x7f2d64003240 con 0x7f2d780c3c90 2020-11-20T12:20:00.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 >> v2:172.21.15.173:3300/0 conn(0x7f2d780bf020 msgr2=0x7f2d780c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:00.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 --2- 172.21.15.116:0/3910967966 >> v2:172.21.15.173:3300/0 conn(0x7f2d780bf020 0x7f2d780c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 >> v1:172.21.15.173:6789/0 conn(0x7f2d78151e90 legacy=0x7f2d780c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d780c7540 con 0x7f2d780c3c90 2020-11-20T12:20:00.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f2d68001660 con 0x7f2d780c3c90 2020-11-20T12:20:00.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 >> v1:172.21.15.116:6789/0 conn(0x7f2d780c3c90 legacy=0x7f2d780c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 --2- 172.21.15.116:0/3910967966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d64003910 0x7f2d64003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 --2- 172.21.15.116:0/3910967966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d640044e0 0x7f2d64004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 --2- 172.21.15.116:0/3910967966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d64004e50 0x7f2d6400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780c72a0 con 0x7f2d640044e0 2020-11-20T12:20:00.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6400bc20 con 0x7f2d64004e50 2020-11-20T12:20:00.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.988+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6400bd90 con 0x7f2d64003910 2020-11-20T12:20:00.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d767fc700 1 --2- 172.21.15.116:0/3910967966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d640044e0 0x7f2d64004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d777fe700 1 --2- 172.21.15.116:0/3910967966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d64004e50 0x7f2d6400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d76ffd700 1 --2- 172.21.15.116:0/3910967966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d64003910 0x7f2d64003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d767fc700 1 -- 172.21.15.116:0/3910967966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d64003910 msgr2=0x7f2d64003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:00.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d767fc700 1 --2- 172.21.15.116:0/3910967966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d64003910 0x7f2d64003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d767fc700 1 -- 172.21.15.116:0/3910967966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d64004e50 msgr2=0x7f2d6400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:00.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d767fc700 1 --2- 172.21.15.116:0/3910967966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d64004e50 0x7f2d6400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d767fc700 1 -- 172.21.15.116:0/3910967966 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d64003240 con 0x7f2d640044e0 2020-11-20T12:20:00.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d777fe700 1 -- 172.21.15.116:0/3910967966 reap_dead start 2020-11-20T12:20:00.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d767fc700 1 --2- 172.21.15.116:0/3910967966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d640044e0 0x7f2d64004960 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f2d6c001a80 tx=0x7f2d6c007b90).ready entity=mon.0 client_cookie=3c18e1051a579d09 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:00.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c007020 con 0x7f2d640044e0 2020-11-20T12:20:00.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6c007180 con 0x7f2d640044e0 2020-11-20T12:20:00.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d75ffb700 1 -- 172.21.15.116:0/3910967966 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c00c850 con 0x7f2d640044e0 2020-11-20T12:20:00.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d7d40f700 1 -- 172.21.15.116:0/3910967966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d640044e0 msgr2=0x7f2d64004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d7d40f700 1 --2- 172.21.15.116:0/3910967966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d640044e0 0x7f2d64004960 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f2d6c001a80 tx=0x7f2d6c007b90).stop 2020-11-20T12:20:00.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d7d40f700 1 -- 172.21.15.116:0/3910967966 shutdown_connections 2020-11-20T12:20:00.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d7d40f700 1 --2- 172.21.15.116:0/3910967966 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d640044e0 0x7f2d64004960 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d7d40f700 1 -- 172.21.15.116:0/3910967966 >> 172.21.15.116:0/3910967966 conn(0x7f2d78068080 msgr2=0x7f2d780c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:00.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d7d40f700 1 -- 172.21.15.116:0/3910967966 shutdown_connections 2020-11-20T12:20:00.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.992+0000 7f2d7d40f700 1 -- 172.21.15.116:0/3910967966 wait complete. 2020-11-20T12:20:00.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d7d40f700 1 Processor -- start 2020-11-20T12:20:00.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d7d40f700 1 -- start start 2020-11-20T12:20:00.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d7d40f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d640044e0 0x7f2d780aca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d7d40f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d780acf00 0x7f2d78094dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d7d40f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d780952c0 0x7f2d780a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d7d40f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780ae2b0 con 0x7f2d780952c0 2020-11-20T12:20:00.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d7d40f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780ad390 con 0x7f2d780acf00 2020-11-20T12:20:00.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d7d40f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d780ad630 con 0x7f2d640044e0 2020-11-20T12:20:00.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d777fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d780952c0 0x7f2d780a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d777fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d780952c0 0x7f2d780a64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48778/0 (socket says 172.21.15.116:48778) 2020-11-20T12:20:00.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d777fe700 1 -- 172.21.15.116:0/2097275244 learned_addr learned my addr 172.21.15.116:0/2097275244 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:00.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:19:59.996+0000 7f2d76ffd700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d640044e0 0x7f2d780aca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d767fc700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d780acf00 0x7f2d78094dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d777fe700 1 -- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d640044e0 msgr2=0x7f2d780aca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d777fe700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d640044e0 0x7f2d780aca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d777fe700 1 -- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d780acf00 msgr2=0x7f2d78094dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d777fe700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d780acf00 0x7f2d78094dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d777fe700 1 -- 172.21.15.116:0/2097275244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d780ad8d0 con 0x7f2d780952c0 2020-11-20T12:20:00.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d777fe700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d780952c0 0x7f2d780a64a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f2d78080e50 tx=0x7f2d700091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:00.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d5effd700 1 -- 172.21.15.116:0/2097275244 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7000fb50 con 0x7f2d780952c0 2020-11-20T12:20:00.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d5effd700 1 -- 172.21.15.116:0/2097275244 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7000fcb0 con 0x7f2d780952c0 2020-11-20T12:20:00.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d5effd700 1 -- 172.21.15.116:0/2097275244 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70014440 con 0x7f2d780952c0 2020-11-20T12:20:00.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d7d40f700 1 -- 172.21.15.116:0/2097275244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d780adb70 con 0x7f2d780952c0 2020-11-20T12:20:00.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d7d40f700 1 -- 172.21.15.116:0/2097275244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d780a6990 con 0x7f2d780952c0 2020-11-20T12:20:00.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.000+0000 7f2d7d40f700 1 -- 172.21.15.116:0/2097275244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7805d2a0 con 0x7f2d780952c0 2020-11-20T12:20:00.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.004+0000 7f2d5effd700 1 -- 172.21.15.116:0/2097275244 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2d700147b0 con 0x7f2d780952c0 2020-11-20T12:20:00.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.004+0000 7f2d5effd700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d540557c0 0x7f2d54057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.004+0000 7f2d5effd700 1 -- 172.21.15.116:0/2097275244 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2d7006f4a0 con 0x7f2d780952c0 2020-11-20T12:20:00.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.004+0000 7f2d76ffd700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d540557c0 0x7f2d54057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.004+0000 7f2d76ffd700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d540557c0 0x7f2d54057c80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f2d68000dd0 tx=0x7f2d68009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:00.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.008+0000 7f2d5effd700 1 -- 172.21.15.116:0/2097275244 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2d70045230 con 0x7f2d780952c0 2020-11-20T12:20:00.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.168+0000 7f2d7d40f700 1 -- 172.21.15.116:0/2097275244 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d780c2fe0 con 0x7f2d780952c0 2020-11-20T12:20:00.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.168+0000 7f2d5effd700 1 -- 172.21.15.116:0/2097275244 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f2d70048850 con 0x7f2d780952c0 2020-11-20T12:20:00.165 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:00.166 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":86,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:58.311412+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:54.974437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:56.326129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:00.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.168+0000 7f2d7d40f700 1 -- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d540557c0 msgr2=0x7f2d54057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.168+0000 7f2d7d40f700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d540557c0 0x7f2d54057c80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f2d68000dd0 tx=0x7f2d68009040).stop 2020-11-20T12:20:00.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.168+0000 7f2d7d40f700 1 -- 172.21.15.116:0/2097275244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d780952c0 msgr2=0x7f2d780a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.168+0000 7f2d7d40f700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d780952c0 0x7f2d780a64a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f2d78080e50 tx=0x7f2d700091d0).stop 2020-11-20T12:20:00.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.168+0000 7f2d7d40f700 1 -- 172.21.15.116:0/2097275244 shutdown_connections 2020-11-20T12:20:00.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.168+0000 7f2d7d40f700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2d540557c0 0x7f2d54057c80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.168+0000 7f2d7d40f700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2d780952c0 0x7f2d780a64a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.168+0000 7f2d7d40f700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2d640044e0 0x7f2d780aca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.168+0000 7f2d7d40f700 1 --2- 172.21.15.116:0/2097275244 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2d780acf00 0x7f2d78094dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.172+0000 7f2d7d40f700 1 -- 172.21.15.116:0/2097275244 >> 172.21.15.116:0/2097275244 conn(0x7f2d78068080 msgr2=0x7f2d780c5000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:00.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.172+0000 7f2d7d40f700 1 -- 172.21.15.116:0/2097275244 shutdown_connections 2020-11-20T12:20:00.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.172+0000 7f2d7d40f700 1 -- 172.21.15.116:0/2097275244 wait complete. 2020-11-20T12:20:00.192 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-20T12:20:00.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.384+0000 7f7468229700 1 Processor -- start 2020-11-20T12:20:00.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.388+0000 7f7468229700 1 -- start start 2020-11-20T12:20:00.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.388+0000 7f7468229700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7460154dd0 0x7f7460156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.388+0000 7f7468229700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74600bdfc0 con 0x7f74600bd810 2020-11-20T12:20:00.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.388+0000 7f7468229700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74600bdd20 con 0x7f74601551b0 2020-11-20T12:20:00.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.388+0000 7f7468229700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7460155780 con 0x7f7460154dd0 2020-11-20T12:20:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f74657c4700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f74600bd810 0x7f74600be340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58218/0 (socket says 172.21.15.116:58218) 2020-11-20T12:20:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f74657c4700 1 -- 172.21.15.116:0/2149883582 learned_addr learned my addr 172.21.15.116:0/2149883582 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:00.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7465fc5700 1 --2- 172.21.15.116:0/2149883582 >> v2:172.21.15.116:3300/0 conn(0x7f7460154dd0 0x7f7460156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7464fc3700 1 -- 172.21.15.116:0/2149883582 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2990574512 0 0) 0x7f74600bdfc0 con 0x7f74600bd810 2020-11-20T12:20:00.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7464fc3700 1 -- 172.21.15.116:0/2149883582 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7448003240 con 0x7f74600bd810 2020-11-20T12:20:00.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7464fc3700 1 -- 172.21.15.116:0/2149883582 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3747593659 0 0) 0x7f74600bdd20 con 0x7f74601551b0 2020-11-20T12:20:00.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7465fc5700 1 -- 172.21.15.116:0/2149883582 >> v1:172.21.15.173:6789/0 conn(0x7f74601551b0 legacy=0x7f74600a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7465fc5700 1 -- 172.21.15.116:0/2149883582 >> v1:172.21.15.116:6789/0 conn(0x7f74600bd810 legacy=0x7f74600be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7465fc5700 1 -- 172.21.15.116:0/2149883582 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7460155a20 con 0x7f7460154dd0 2020-11-20T12:20:00.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7465fc5700 1 --2- 172.21.15.116:0/2149883582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7460154dd0 0x7f7460156160 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f745c00b9a0 tx=0x7f745c006df0).ready entity=mon.0 client_cookie=3714fe9daa5d2f79 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7464fc3700 1 -- 172.21.15.116:0/2149883582 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c0097b0 con 0x7f7460154dd0 2020-11-20T12:20:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7464fc3700 1 -- 172.21.15.116:0/2149883582 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f745c009910 con 0x7f7460154dd0 2020-11-20T12:20:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7464fc3700 1 -- 172.21.15.116:0/2149883582 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c009c90 con 0x7f7460154dd0 2020-11-20T12:20:00.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7468229700 1 -- 172.21.15.116:0/2149883582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7460154dd0 msgr2=0x7f7460156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7468229700 1 --2- 172.21.15.116:0/2149883582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7460154dd0 0x7f7460156160 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f745c00b9a0 tx=0x7f745c006df0).stop 2020-11-20T12:20:00.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7468229700 1 -- 172.21.15.116:0/2149883582 shutdown_connections 2020-11-20T12:20:00.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7468229700 1 --2- 172.21.15.116:0/2149883582 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7460154dd0 0x7f7460156160 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.392+0000 7f7468229700 1 -- 172.21.15.116:0/2149883582 >> 172.21.15.116:0/2149883582 conn(0x7f746005ba80 msgr2=0x7f74600c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:00.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f7468229700 1 -- 172.21.15.116:0/2149883582 shutdown_connections 2020-11-20T12:20:00.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f7468229700 1 -- 172.21.15.116:0/2149883582 wait complete. 2020-11-20T12:20:00.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f7468229700 1 Processor -- start 2020-11-20T12:20:00.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f7468229700 1 -- start start 2020-11-20T12:20:00.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f7468229700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74600bd810 0x7f7460156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f7468229700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74601551b0 0x7f74600b9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f7468229700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7460156710 0x7f74600ba140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f7468229700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74600a8340 con 0x7f74601551b0 2020-11-20T12:20:00.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f7468229700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74600a85e0 con 0x7f74600bd810 2020-11-20T12:20:00.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f7468229700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74600b9510 con 0x7f7460156710 2020-11-20T12:20:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f74657c4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74601551b0 0x7f74600b9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f74667c6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7460156710 0x7f74600ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f74657c4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74601551b0 0x7f74600b9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48792/0 (socket says 172.21.15.116:48792) 2020-11-20T12:20:00.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f7465fc5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74600bd810 0x7f7460156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.396+0000 7f74657c4700 1 -- 172.21.15.116:0/3595712908 learned_addr learned my addr 172.21.15.116:0/3595712908 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:00.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f74657c4700 1 -- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7460156710 msgr2=0x7f74600ba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f74657c4700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7460156710 0x7f74600ba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f74657c4700 1 -- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74600bd810 msgr2=0x7f7460156220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f74657c4700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74600bd810 0x7f7460156220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f74657c4700 1 -- 172.21.15.116:0/3595712908 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74600b97b0 con 0x7f74601551b0 2020-11-20T12:20:00.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f74667c6700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7460156710 0x7f74600ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:00.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f74657c4700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74601551b0 0x7f74600b9c50 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f745000c710 tx=0x7f7450007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:00.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f744dffb700 1 -- 172.21.15.116:0/3595712908 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74500072e0 con 0x7f74601551b0 2020-11-20T12:20:00.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f744dffb700 1 -- 172.21.15.116:0/3595712908 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7450007440 con 0x7f74601551b0 2020-11-20T12:20:00.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f7468229700 1 -- 172.21.15.116:0/3595712908 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74600bd480 con 0x7f74601551b0 2020-11-20T12:20:00.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f744dffb700 1 -- 172.21.15.116:0/3595712908 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7450005820 con 0x7f74601551b0 2020-11-20T12:20:00.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f7468229700 1 -- 172.21.15.116:0/3595712908 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74600a7a90 con 0x7f74601551b0 2020-11-20T12:20:00.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.400+0000 7f7468229700 1 -- 172.21.15.116:0/3595712908 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74600b9940 con 0x7f74601551b0 2020-11-20T12:20:00.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.404+0000 7f744dffb700 1 -- 172.21.15.116:0/3595712908 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7450010070 con 0x7f74601551b0 2020-11-20T12:20:00.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.404+0000 7f744dffb700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74380557c0 0x7f7438057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.404+0000 7f744dffb700 1 -- 172.21.15.116:0/3595712908 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f745006dd20 con 0x7f74601551b0 2020-11-20T12:20:00.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.404+0000 7f7465fc5700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74380557c0 0x7f7438057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.404+0000 7f7465fc5700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74380557c0 0x7f7438057c80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f7460064000 tx=0x7f745c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:00.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.408+0000 7f744dffb700 1 -- 172.21.15.116:0/3595712908 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7450043bc0 con 0x7f74601551b0 2020-11-20T12:20:00.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.628+0000 7f7468229700 1 -- 172.21.15.116:0/3595712908 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f746005d2a0 con 0x7f74601551b0 2020-11-20T12:20:00.625 INFO:tasks.ceph.mds.a.smithi116.stderr:2020-11-20T12:20:00.628+0000 7f02a8452700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:20:00.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f744dffb700 1 -- 172.21.15.116:0/3595712908 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 184+0+0 (secure 0 0 0) 0x7f7450018020 con 0x7f74601551b0 2020-11-20T12:20:00.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f7468229700 1 -- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74380557c0 msgr2=0x7f7438057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f7468229700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74380557c0 0x7f7438057c80 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f7460064000 tx=0x7f745c015000).stop 2020-11-20T12:20:00.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f7468229700 1 -- 172.21.15.116:0/3595712908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74601551b0 msgr2=0x7f74600b9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f7468229700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74601551b0 0x7f74600b9c50 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7f745000c710 tx=0x7f7450007a60).stop 2020-11-20T12:20:00.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f7468229700 1 -- 172.21.15.116:0/3595712908 shutdown_connections 2020-11-20T12:20:00.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f7468229700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f74380557c0 0x7f7438057c80 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f7468229700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7460156710 0x7f74600ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f7468229700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f74600bd810 0x7f7460156220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f7468229700 1 --2- 172.21.15.116:0/3595712908 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f74601551b0 0x7f74600b9c50 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f7468229700 1 -- 172.21.15.116:0/3595712908 >> 172.21.15.116:0/3595712908 conn(0x7f746005ba80 msgr2=0x7f74600a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:00.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.636+0000 7f7468229700 1 -- 172.21.15.116:0/3595712908 shutdown_connections 2020-11-20T12:20:00.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.640+0000 7f7468229700 1 -- 172.21.15.116:0/3595712908 wait complete. 2020-11-20T12:20:00.641 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2020-11-20T12:20:00.659 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:00.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.844+0000 7f1106817700 1 Processor -- start 2020-11-20T12:20:00.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.844+0000 7f1106817700 1 -- start start 2020-11-20T12:20:00.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1106817700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f110014bd80 0x7f1100152790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1106817700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11000acd20 0x7f11000ae150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1106817700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11000ad230 con 0x7f11000614b0 2020-11-20T12:20:00.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1106817700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11000a80d0 con 0x7f110014bd80 2020-11-20T12:20:00.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1106817700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1100152c80 con 0x7f11000acd20 2020-11-20T12:20:00.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1105815700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f110014bd80 0x7f1100152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1105815700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f110014bd80 0x7f1100152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48798/0 (socket says 172.21.15.116:48798) 2020-11-20T12:20:00.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1106016700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f11000614b0 0x7f1100152e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:54464/0 (socket says 172.21.15.116:54464) 2020-11-20T12:20:00.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1105815700 1 -- 172.21.15.116:0/714559727 learned_addr learned my addr 172.21.15.116:0/714559727 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:00.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1105014700 1 --2- 172.21.15.116:0/714559727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11000acd20 0x7f11000ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1105815700 1 -- 172.21.15.116:0/714559727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11000acd20 msgr2=0x7f11000ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:00.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1105815700 1 --2- 172.21.15.116:0/714559727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11000acd20 0x7f11000ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1105815700 1 -- 172.21.15.116:0/714559727 >> v1:172.21.15.173:6789/0 conn(0x7f11000614b0 legacy=0x7f1100152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1105815700 1 -- 172.21.15.116:0/714559727 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11000ad4d0 con 0x7f110014bd80 2020-11-20T12:20:00.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1105815700 1 --2- 172.21.15.116:0/714559727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f110014bd80 0x7f1100152790 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f10fc006fd0 tx=0x7f10fc006eb0).ready entity=mon.0 client_cookie=54a6484f0f12b143 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:00.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.848+0000 7f1104813700 1 -- 172.21.15.116:0/714559727 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc009810 con 0x7f110014bd80 2020-11-20T12:20:00.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.852+0000 7f1104813700 1 -- 172.21.15.116:0/714559727 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10fc009970 con 0x7f110014bd80 2020-11-20T12:20:00.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.852+0000 7f1104813700 1 -- 172.21.15.116:0/714559727 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10fc009cf0 con 0x7f110014bd80 2020-11-20T12:20:00.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.852+0000 7f1106817700 1 -- 172.21.15.116:0/714559727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f110014bd80 msgr2=0x7f1100152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.852+0000 7f1106817700 1 --2- 172.21.15.116:0/714559727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f110014bd80 0x7f1100152790 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f10fc006fd0 tx=0x7f10fc006eb0).stop 2020-11-20T12:20:00.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.852+0000 7f1106817700 1 -- 172.21.15.116:0/714559727 shutdown_connections 2020-11-20T12:20:00.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.852+0000 7f1106817700 1 --2- 172.21.15.116:0/714559727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f110014bd80 0x7f1100152790 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.852+0000 7f1106817700 1 --2- 172.21.15.116:0/714559727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11000acd20 0x7f11000ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.852+0000 7f1106817700 1 -- 172.21.15.116:0/714559727 >> 172.21.15.116:0/714559727 conn(0x7f1100068370 msgr2=0x7f11000c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:00.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.852+0000 7f1106817700 1 -- 172.21.15.116:0/714559727 shutdown_connections 2020-11-20T12:20:00.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.852+0000 7f1106817700 1 -- 172.21.15.116:0/714559727 wait complete. 2020-11-20T12:20:00.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106817700 1 Processor -- start 2020-11-20T12:20:00.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106817700 1 -- start start 2020-11-20T12:20:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106817700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11000614b0 0x7f11000bd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106817700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11000acd20 0x7f1100164840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106817700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f11000a9930 0x7f1100164d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106817700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1100149b50 con 0x7f11000a9930 2020-11-20T12:20:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106817700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11000aa0b0 con 0x7f11000acd20 2020-11-20T12:20:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106817700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11000a9e10 con 0x7f11000614b0 2020-11-20T12:20:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106016700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f11000a9930 0x7f1100164d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106016700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f11000a9930 0x7f1100164d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48806/0 (socket says 172.21.15.116:48806) 2020-11-20T12:20:00.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106016700 1 -- 172.21.15.116:0/3552473438 learned_addr learned my addr 172.21.15.116:0/3552473438 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:00.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1105014700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11000acd20 0x7f1100164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1105815700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11000614b0 0x7f11000bd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106016700 1 -- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11000614b0 msgr2=0x7f11000bd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106016700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11000614b0 0x7f11000bd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106016700 1 -- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11000acd20 msgr2=0x7f1100164840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106016700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11000acd20 0x7f1100164840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106016700 1 -- 172.21.15.116:0/3552473438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11000a8760 con 0x7f11000a9930 2020-11-20T12:20:00.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.856+0000 7f1106016700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f11000a9930 0x7f1100164d30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f10f800ca60 tx=0x7f10f8007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:00.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.860+0000 7f10edffb700 1 -- 172.21.15.116:0/3552473438 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8005ac0 con 0x7f11000a9930 2020-11-20T12:20:00.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.860+0000 7f10edffb700 1 -- 172.21.15.116:0/3552473438 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f8005c20 con 0x7f11000a9930 2020-11-20T12:20:00.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.860+0000 7f10edffb700 1 -- 172.21.15.116:0/3552473438 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f8007110 con 0x7f11000a9930 2020-11-20T12:20:00.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.860+0000 7f1106817700 1 -- 172.21.15.116:0/3552473438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11000a8a00 con 0x7f11000a9930 2020-11-20T12:20:00.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.860+0000 7f1106817700 1 -- 172.21.15.116:0/3552473438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11000bdb80 con 0x7f11000a9930 2020-11-20T12:20:00.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.860+0000 7f1106817700 1 -- 172.21.15.116:0/3552473438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110005cf30 con 0x7f11000a9930 2020-11-20T12:20:00.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.860+0000 7f10edffb700 1 -- 172.21.15.116:0/3552473438 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f10f8010030 con 0x7f11000a9930 2020-11-20T12:20:00.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.864+0000 7f10edffb700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d80557c0 0x7f10d8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:00.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.864+0000 7f10edffb700 1 -- 172.21.15.116:0/3552473438 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f10f806e610 con 0x7f11000a9930 2020-11-20T12:20:00.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.864+0000 7f1105815700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d80557c0 0x7f10d8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:00.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.864+0000 7f1105815700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d80557c0 0x7f10d8057c80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f1100045630 tx=0x7f10fc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:00.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:00.868+0000 7f10edffb700 1 -- 172.21.15.116:0/3552473438 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f10f8044530 con 0x7f11000a9930 2020-11-20T12:20:01.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 -- 172.21.15.116:0/3552473438 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11000c5fc0 con 0x7f11000a9930 2020-11-20T12:20:01.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f10edffb700 1 -- 172.21.15.116:0/3552473438 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2848 (secure 0 0 0) 0x7f10f8047b50 con 0x7f11000a9930 2020-11-20T12:20:01.073 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:01.073 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/313210621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":313210621},{"type":"v1","addr":"172.21.15.173:6835","nonce":313210621}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/2140340557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2140340557},{"type":"v1","addr":"172.21.15.173:6837","nonce":2140340557}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3544207168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3544207168},{"type":"v1","addr":"172.21.15.116:6833","nonce":3544207168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:57.303415+0000","modified":"2020-11-20T12:19:59.322603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/657163487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":657163487},{"type":"v1","addr":"172.21.15.116:6835","nonce":657163487}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-20T12:20:01.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 -- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d80557c0 msgr2=0x7f10d8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:01.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d80557c0 0x7f10d8057c80 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f1100045630 tx=0x7f10fc015000).stop 2020-11-20T12:20:01.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 -- 172.21.15.116:0/3552473438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f11000a9930 msgr2=0x7f1100164d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:01.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f11000a9930 0x7f1100164d30 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f10f800ca60 tx=0x7f10f8007f40).stop 2020-11-20T12:20:01.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 -- 172.21.15.116:0/3552473438 shutdown_connections 2020-11-20T12:20:01.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f10d80557c0 0x7f10d8057c80 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:01.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f11000a9930 0x7f1100164d30 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:01.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f11000614b0 0x7f11000bd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:01.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 --2- 172.21.15.116:0/3552473438 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f11000acd20 0x7f1100164840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:01.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 -- 172.21.15.116:0/3552473438 >> 172.21.15.116:0/3552473438 conn(0x7f1100068370 msgr2=0x7f1100149df0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:01.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 -- 172.21.15.116:0/3552473438 shutdown_connections 2020-11-20T12:20:01.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:01.076+0000 7f1106817700 1 -- 172.21.15.116:0/3552473438 wait complete. 2020-11-20T12:20:01.078 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 46 2020-11-20T12:20:01.502 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:19:57.303415+0000', 'modified': '2020-11-20T12:19:59.322603+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6861}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6861': {'gid': 6861, 'name': 'a', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/657163487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 657163487}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 657163487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:20:01.502 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:20:01.503 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:20:01.713 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":6861,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.3899066979999999,"mdsmap_epoch":46,"osdmap_epoch":86,"osdmap_epoch_barrier":86,"uptime":10.95615396} 2020-11-20T12:20:01.714 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 6861, "mdsmap_epoch": 46, "osdmap_epoch": 86, "osdmap_epoch_barrier": 86, "rank_uptime": 4.389906698, "state": "up:active", "uptime": 10.95615396, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:20:01.714 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:20:01.715 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:20:01.715 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:20:01.716 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:20:01.716 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:20:01.716 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:20:01.717 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:20:01.717 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:20:01.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:01.718 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:20:01.737 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:20:01.738 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:20:01.738 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:20:01.738 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:01.739 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:20:01.739 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:01.739 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:01.739 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:01.739 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:01.740 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:01.741 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:01.741 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:01.741 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:20:01.742 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:01.742 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:20:01.742 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:01.742 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:20:01.743 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:01.743 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:20:01.743 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:01.743 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:20:01.744 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:01.744 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:20:01.744 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:01.744 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:20:01.744 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:01.745 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:20:01.745 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:01.745 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:01.745 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:20:01.746 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:20:01.746 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:01.746 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:01.746 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:20:01.746 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:20:01.747 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:01.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:01.748 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:20:01.774 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:20:01.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:01.778 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:20:01.834 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:20:01.835 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:20:01.835 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:20:01.835 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:20:01.836 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:20:01.836 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:20:01.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:01.838 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:20:01.838 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:20:01.839 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:20:01.839 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:20:01.840 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:20:01.840 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:20:01.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:01.938 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:20:01.953 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:20:01.954 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:20:01.955 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:20:01.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:01.956 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2020-11-20T12:20:02.027 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:02.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.200+0000 7f9142bf6700 1 Processor -- start 2020-11-20T12:20:02.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.200+0000 7f9142bf6700 1 -- start start 2020-11-20T12:20:02.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.200+0000 7f9142bf6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f913c0c3b60 0x7f913c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.200+0000 7f9142bf6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f913c0c1970 0x7f913c150560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.200+0000 7f9142bf6700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c0a6950 con 0x7f913c0c3780 2020-11-20T12:20:02.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.200+0000 7f9142bf6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c0a6770 con 0x7f913c0c1970 2020-11-20T12:20:02.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.200+0000 7f9142bf6700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c14fed0 con 0x7f913c0c3b60 2020-11-20T12:20:02.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.200+0000 7f9141193700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f913c0c1970 0x7f913c150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.200+0000 7f9141193700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f913c0c1970 0x7f913c150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48814/0 (socket says 172.21.15.116:48814) 2020-11-20T12:20:02.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.200+0000 7f9141193700 1 -- 172.21.15.116:0/3225374896 learned_addr learned my addr 172.21.15.116:0/3225374896 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:02.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.200+0000 7f913bfff700 1 --2- 172.21.15.116:0/3225374896 >> v2:172.21.15.173:3300/0 conn(0x7f913c0c3b60 0x7f913c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9141193700 1 -- 172.21.15.116:0/3225374896 >> v2:172.21.15.173:3300/0 conn(0x7f913c0c3b60 msgr2=0x7f913c0a6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:02.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9141193700 1 --2- 172.21.15.116:0/3225374896 >> v2:172.21.15.173:3300/0 conn(0x7f913c0c3b60 0x7f913c0a6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9141193700 1 -- 172.21.15.116:0/3225374896 >> v1:172.21.15.173:6789/0 conn(0x7f913c0c3780 legacy=0x7f913c0c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9141193700 1 -- 172.21.15.116:0/3225374896 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c0a6bf0 con 0x7f913c0c1970 2020-11-20T12:20:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f913bfff700 1 --2- 172.21.15.116:0/3225374896 >> v2:172.21.15.173:3300/0 conn(0x7f913c0c3b60 0x7f913c0a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:02.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9141193700 1 --2- 172.21.15.116:0/3225374896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f913c0c1970 0x7f913c150560 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f913c0c4380 tx=0x7f9134009290).ready entity=mon.0 client_cookie=3d769c20762e853d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f913b7fe700 1 -- 172.21.15.116:0/3225374896 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913400bb80 con 0x7f913c0c1970 2020-11-20T12:20:02.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f913b7fe700 1 -- 172.21.15.116:0/3225374896 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913400bce0 con 0x7f913c0c1970 2020-11-20T12:20:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f913b7fe700 1 -- 172.21.15.116:0/3225374896 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134011610 con 0x7f913c0c1970 2020-11-20T12:20:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9142bf6700 1 -- 172.21.15.116:0/3225374896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f913c0c1970 msgr2=0x7f913c150560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9142bf6700 1 --2- 172.21.15.116:0/3225374896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f913c0c1970 0x7f913c150560 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7f913c0c4380 tx=0x7f9134009290).stop 2020-11-20T12:20:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9142bf6700 1 -- 172.21.15.116:0/3225374896 shutdown_connections 2020-11-20T12:20:02.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9142bf6700 1 --2- 172.21.15.116:0/3225374896 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f913c0c1970 0x7f913c150560 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9142bf6700 1 --2- 172.21.15.116:0/3225374896 >> v2:172.21.15.173:3300/0 conn(0x7f913c0c3b60 0x7f913c0a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9142bf6700 1 -- 172.21.15.116:0/3225374896 >> 172.21.15.116:0/3225374896 conn(0x7f913c0c4f80 msgr2=0x7f913c0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9142bf6700 1 -- 172.21.15.116:0/3225374896 shutdown_connections 2020-11-20T12:20:02.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.204+0000 7f9142bf6700 1 -- 172.21.15.116:0/3225374896 wait complete. 2020-11-20T12:20:02.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.208+0000 7f9142bf6700 1 Processor -- start 2020-11-20T12:20:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.208+0000 7f9142bf6700 1 -- start start 2020-11-20T12:20:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.208+0000 7f9142bf6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f913c0c1970 0x7f913c156bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.208+0000 7f9142bf6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913c0c3780 0x7f913c1570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.208+0000 7f9142bf6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f913c0c3b60 0x7f913c162810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.208+0000 7f9142bf6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0a4180 con 0x7f913c0c1970 2020-11-20T12:20:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.208+0000 7f9142bf6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0a4420 con 0x7f913c0c3780 2020-11-20T12:20:02.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.208+0000 7f9142bf6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0a46c0 con 0x7f913c0c3b60 2020-11-20T12:20:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.208+0000 7f9140992700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f913c0c1970 0x7f913c156bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.208+0000 7f913bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913c0c3780 0x7f913c1570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f9141193700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f913c0c3b60 0x7f913c162810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f913bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913c0c3780 0x7f913c1570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:47870/0 (socket says 172.21.15.116:47870) 2020-11-20T12:20:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f913bfff700 1 -- 172.21.15.116:0/4253394479 learned_addr learned my addr 172.21.15.116:0/4253394479 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f913bfff700 1 -- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f913c0c3b60 msgr2=0x7f913c162810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f913bfff700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f913c0c3b60 0x7f913c162810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f913bfff700 1 -- 172.21.15.116:0/4253394479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f913c0c1970 msgr2=0x7f913c156bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f913bfff700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f913c0c1970 0x7f913c156bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f913bfff700 1 -- 172.21.15.116:0/4253394479 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c0a4960 con 0x7f913c0c3780 2020-11-20T12:20:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f9140992700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f913c0c1970 0x7f913c156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f913bfff700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913c0c3780 0x7f913c1570a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f912800ed20 tx=0x7f9128006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:02.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f9138ff9700 1 -- 172.21.15.116:0/4253394479 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f912800caa0 con 0x7f913c0c3780 2020-11-20T12:20:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f9138ff9700 1 -- 172.21.15.116:0/4253394479 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f912800cc00 con 0x7f913c0c3780 2020-11-20T12:20:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f9138ff9700 1 -- 172.21.15.116:0/4253394479 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91280123f0 con 0x7f913c0c3780 2020-11-20T12:20:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f9142bf6700 1 -- 172.21.15.116:0/4253394479 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c0a4c00 con 0x7f913c0c3780 2020-11-20T12:20:02.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.212+0000 7f9142bf6700 1 -- 172.21.15.116:0/4253394479 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c0be7d0 con 0x7f913c0c3780 2020-11-20T12:20:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.216+0000 7f9142bf6700 1 -- 172.21.15.116:0/4253394479 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c0a4d90 con 0x7f913c0c3780 2020-11-20T12:20:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.216+0000 7f9138ff9700 1 -- 172.21.15.116:0/4253394479 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f91280127a0 con 0x7f913c0c3780 2020-11-20T12:20:02.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.216+0000 7f9138ff9700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91140557c0 0x7f9114057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.216+0000 7f9138ff9700 1 -- 172.21.15.116:0/4253394479 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f912806ec60 con 0x7f913c0c3780 2020-11-20T12:20:02.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.216+0000 7f9140992700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91140557c0 0x7f9114057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.220+0000 7f9140992700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91140557c0 0x7f9114057c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f9130001a50 tx=0x7f9130009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:02.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.224+0000 7f9138ff9700 1 -- 172.21.15.116:0/4253394479 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f91280449f0 con 0x7f913c0c3780 2020-11-20T12:20:02.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.384+0000 7f9142bf6700 1 -- 172.21.15.116:0/4253394479 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f913c05d2a0 con 0x7f913c0c3780 2020-11-20T12:20:02.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9138ff9700 1 -- 172.21.15.116:0/4253394479 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f9128048010 con 0x7f913c0c3780 2020-11-20T12:20:02.385 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:02.386 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":86,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:58.311412+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:54.974437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:56.326129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:02.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 -- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91140557c0 msgr2=0x7f9114057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91140557c0 0x7f9114057c80 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f9130001a50 tx=0x7f9130009040).stop 2020-11-20T12:20:02.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 -- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913c0c3780 msgr2=0x7f913c1570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913c0c3780 0x7f913c1570a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f912800ed20 tx=0x7f9128006df0).stop 2020-11-20T12:20:02.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 -- 172.21.15.116:0/4253394479 shutdown_connections 2020-11-20T12:20:02.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f91140557c0 0x7f9114057c80 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f913c0c3b60 0x7f913c162810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f913c0c1970 0x7f913c156bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 --2- 172.21.15.116:0/4253394479 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f913c0c3780 0x7f913c1570a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 -- 172.21.15.116:0/4253394479 >> 172.21.15.116:0/4253394479 conn(0x7f913c0c4f80 msgr2=0x7f913c153930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:02.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 -- 172.21.15.116:0/4253394479 shutdown_connections 2020-11-20T12:20:02.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.388+0000 7f9142bf6700 1 -- 172.21.15.116:0/4253394479 wait complete. 2020-11-20T12:20:02.412 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:19:54.974437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:19:56.326129+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:20:02.413 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:20:02.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f51878a0700 1 Processor -- start 2020-11-20T12:20:02.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f51878a0700 1 -- start start 2020-11-20T12:20:02.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f51878a0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5180065a20 0x7f518014c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f51878a0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5180061990 0x7f5180061e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f51878a0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518014c680 con 0x7f518014bdb0 2020-11-20T12:20:02.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f51878a0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51800622f0 con 0x7f5180065a20 2020-11-20T12:20:02.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f51878a0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5180065ea0 con 0x7f5180061990 2020-11-20T12:20:02.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f518563c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f518014bdb0 0x7f5180152110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58258/0 (socket says 172.21.15.116:58258) 2020-11-20T12:20:02.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f518563c700 1 -- 172.21.15.116:0/655762168 learned_addr learned my addr 172.21.15.116:0/655762168 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:02.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f5184e3b700 1 --2- 172.21.15.116:0/655762168 >> v2:172.21.15.173:3300/0 conn(0x7f5180065a20 0x7f518014c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f5185e3d700 1 --2- 172.21.15.116:0/655762168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5180061990 0x7f5180061e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.600+0000 7f516ffff700 1 -- 172.21.15.116:0/655762168 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2937957044 0 0) 0x7f517c0010d0 con 0x7f518014bdb0 2020-11-20T12:20:02.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f516ffff700 1 -- 172.21.15.116:0/655762168 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5168003240 con 0x7f518014bdb0 2020-11-20T12:20:02.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f5184e3b700 1 -- 172.21.15.116:0/655762168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5180061990 msgr2=0x7f5180061e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:02.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f5184e3b700 1 --2- 172.21.15.116:0/655762168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5180061990 0x7f5180061e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f5184e3b700 1 -- 172.21.15.116:0/655762168 >> v1:172.21.15.116:6789/0 conn(0x7f518014bdb0 legacy=0x7f5180152110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f5184e3b700 1 -- 172.21.15.116:0/655762168 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800605d0 con 0x7f5180065a20 2020-11-20T12:20:02.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f5184e3b700 1 --2- 172.21.15.116:0/655762168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5180065a20 0x7f518014c190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f517000b9a0 tx=0x7f5170006df0).ready entity=mon.1 client_cookie=527ddad3bf5d187f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:02.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f516ffff700 1 -- 172.21.15.116:0/655762168 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170009680 con 0x7f5180065a20 2020-11-20T12:20:02.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f516ffff700 1 -- 172.21.15.116:0/655762168 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51700097e0 con 0x7f5180065a20 2020-11-20T12:20:02.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f516ffff700 1 -- 172.21.15.116:0/655762168 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5170009b60 con 0x7f5180065a20 2020-11-20T12:20:02.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f51878a0700 1 -- 172.21.15.116:0/655762168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5180065a20 msgr2=0x7f518014c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f51878a0700 1 --2- 172.21.15.116:0/655762168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5180065a20 0x7f518014c190 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f517000b9a0 tx=0x7f5170006df0).stop 2020-11-20T12:20:02.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f51878a0700 1 -- 172.21.15.116:0/655762168 shutdown_connections 2020-11-20T12:20:02.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f51878a0700 1 --2- 172.21.15.116:0/655762168 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5180061990 0x7f5180061e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f51878a0700 1 --2- 172.21.15.116:0/655762168 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5180065a20 0x7f518014c190 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f51878a0700 1 -- 172.21.15.116:0/655762168 >> 172.21.15.116:0/655762168 conn(0x7f51800c2bc0 msgr2=0x7f5180061280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:02.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f51878a0700 1 -- 172.21.15.116:0/655762168 shutdown_connections 2020-11-20T12:20:02.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.604+0000 7f51878a0700 1 -- 172.21.15.116:0/655762168 wait complete. 2020-11-20T12:20:02.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.608+0000 7f51878a0700 1 Processor -- start 2020-11-20T12:20:02.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.608+0000 7f51878a0700 1 -- start start 2020-11-20T12:20:02.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f51878a0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5180061990 0x7f5180164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f51878a0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5180065a20 0x7f5180161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f51878a0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518014bdb0 0x7f5180151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f51878a0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5180165570 con 0x7f518014bdb0 2020-11-20T12:20:02.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f51878a0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5180165810 con 0x7f5180061990 2020-11-20T12:20:02.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f51878a0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5180165ab0 con 0x7f5180065a20 2020-11-20T12:20:02.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f5185e3d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518014bdb0 0x7f5180151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f5185e3d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518014bdb0 0x7f5180151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48832/0 (socket says 172.21.15.116:48832) 2020-11-20T12:20:02.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f5185e3d700 1 -- 172.21.15.116:0/3653560271 learned_addr learned my addr 172.21.15.116:0/3653560271 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:02.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f518563c700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5180061990 0x7f5180164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f5184e3b700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5180065a20 0x7f5180161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f5185e3d700 1 -- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5180065a20 msgr2=0x7f5180161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f5185e3d700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5180065a20 0x7f5180161ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f5185e3d700 1 -- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5180061990 msgr2=0x7f5180164cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f5185e3d700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5180061990 0x7f5180164cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f5185e3d700 1 -- 172.21.15.116:0/3653560271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5180161a30 con 0x7f518014bdb0 2020-11-20T12:20:02.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f5185e3d700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518014bdb0 0x7f5180151200 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f517800ef50 tx=0x7f51780091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:02.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f516d7fa700 1 -- 172.21.15.116:0/3653560271 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517800cd20 con 0x7f518014bdb0 2020-11-20T12:20:02.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f516d7fa700 1 -- 172.21.15.116:0/3653560271 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517800ce80 con 0x7f518014bdb0 2020-11-20T12:20:02.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f51878a0700 1 -- 172.21.15.116:0/3653560271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51801651c0 con 0x7f518014bdb0 2020-11-20T12:20:02.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f516d7fa700 1 -- 172.21.15.116:0/3653560271 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178012620 con 0x7f518014bdb0 2020-11-20T12:20:02.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.612+0000 7f51878a0700 1 -- 172.21.15.116:0/3653560271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51801625e0 con 0x7f518014bdb0 2020-11-20T12:20:02.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.616+0000 7f51878a0700 1 -- 172.21.15.116:0/3653560271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5180069950 con 0x7f518014bdb0 2020-11-20T12:20:02.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.616+0000 7f516d7fa700 1 -- 172.21.15.116:0/3653560271 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5178022070 con 0x7f518014bdb0 2020-11-20T12:20:02.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.616+0000 7f516d7fa700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51580557c0 0x7f5158057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.616+0000 7f516d7fa700 1 -- 172.21.15.116:0/3653560271 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f517806f020 con 0x7f518014bdb0 2020-11-20T12:20:02.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.616+0000 7f518563c700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51580557c0 0x7f5158057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.620+0000 7f518563c700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51580557c0 0x7f5158057c80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f517c001230 tx=0x7f517c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:02.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.620+0000 7f516d7fa700 1 -- 172.21.15.116:0/3653560271 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5178044ec0 con 0x7f518014bdb0 2020-11-20T12:20:02.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.788+0000 7f51878a0700 1 -- 172.21.15.116:0/3653560271 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5180161bc0 con 0x7f518014bdb0 2020-11-20T12:20:02.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.788+0000 7f516d7fa700 1 -- 172.21.15.116:0/3653560271 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v86) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5178016030 con 0x7f518014bdb0 2020-11-20T12:20:02.785 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:20:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.788+0000 7f51878a0700 1 -- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51580557c0 msgr2=0x7f5158057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.788+0000 7f51878a0700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51580557c0 0x7f5158057c80 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f517c001230 tx=0x7f517c009040).stop 2020-11-20T12:20:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.788+0000 7f51878a0700 1 -- 172.21.15.116:0/3653560271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518014bdb0 msgr2=0x7f5180151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.788+0000 7f51878a0700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518014bdb0 0x7f5180151200 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f517800ef50 tx=0x7f51780091d0).stop 2020-11-20T12:20:02.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.792+0000 7f51878a0700 1 -- 172.21.15.116:0/3653560271 shutdown_connections 2020-11-20T12:20:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.792+0000 7f51878a0700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51580557c0 0x7f5158057c80 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.792+0000 7f51878a0700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f518014bdb0 0x7f5180151200 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.792+0000 7f51878a0700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5180061990 0x7f5180164cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.792+0000 7f51878a0700 1 --2- 172.21.15.116:0/3653560271 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5180065a20 0x7f5180161ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.792+0000 7f51878a0700 1 -- 172.21.15.116:0/3653560271 >> 172.21.15.116:0/3653560271 conn(0x7f51800c2bc0 msgr2=0x7f5180067550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.792+0000 7f51878a0700 1 -- 172.21.15.116:0/3653560271 shutdown_connections 2020-11-20T12:20:02.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.792+0000 7f51878a0700 1 -- 172.21.15.116:0/3653560271 wait complete. 2020-11-20T12:20:02.810 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:20:02.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.992+0000 7f1331207700 1 Processor -- start 2020-11-20T12:20:02.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.996+0000 7f1331207700 1 -- start start 2020-11-20T12:20:02.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.996+0000 7f1331207700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c151940 0x7f132c0a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:02.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.996+0000 7f1331207700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132c0a7220 con 0x7f132c062f10 2020-11-20T12:20:02.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.996+0000 7f1331207700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f132c062530 con 0x7f132c151d20 2020-11-20T12:20:02.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.996+0000 7f1331207700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132c0633c0 con 0x7f132c151940 2020-11-20T12:20:02.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.996+0000 7f132b59e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f132c062f10 0x7f132c0bfa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58276/0 (socket says 172.21.15.116:58276) 2020-11-20T12:20:02.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.996+0000 7f132b59e700 1 -- 172.21.15.116:0/863189561 learned_addr learned my addr 172.21.15.116:0/863189561 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:02.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.996+0000 7f132ad9d700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c151940 0x7f132c0a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:02.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.996+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452405941 0 0) 0x7f132c0a7220 con 0x7f132c062f10 2020-11-20T12:20:02.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:02.996+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1318003240 con 0x7f132c062f10 2020-11-20T12:20:02.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3854978664 0 0) 0x7f1318003240 con 0x7f132c062f10 2020-11-20T12:20:02.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c151940 msgr2=0x7f132c0a6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:02.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c151940 0x7f132c0a6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:02.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 >> v1:172.21.15.173:6789/0 conn(0x7f132c151d20 legacy=0x7f132c064a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:02.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c0627d0 con 0x7f132c062f10 2020-11-20T12:20:02.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f1324001860 con 0x7f132c062f10 2020-11-20T12:20:02.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 >> v1:172.21.15.116:6789/0 conn(0x7f132c062f10 legacy=0x7f132c0bfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c151940 0x7f1318003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1318003cc0 0x7f13180044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13180049d0 0x7f1318004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c0633c0 con 0x7f13180049d0 2020-11-20T12:20:03.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13180042b0 con 0x7f1318003cc0 2020-11-20T12:20:03.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f131800b730 con 0x7f132c151940 2020-11-20T12:20:03.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f132b59e700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13180049d0 0x7f1318004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f132ad9d700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c151940 0x7f1318003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f132a59c700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1318003cc0 0x7f13180044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f132b59e700 1 -- 172.21.15.116:0/863189561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c151940 msgr2=0x7f1318003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:03.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f132b59e700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c151940 0x7f1318003760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f132b59e700 1 -- 172.21.15.116:0/863189561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1318003cc0 msgr2=0x7f13180044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:03.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f132b59e700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1318003cc0 0x7f13180044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f132b59e700 1 -- 172.21.15.116:0/863189561 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1318003240 con 0x7f13180049d0 2020-11-20T12:20:03.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f132b59e700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13180049d0 0x7f1318004e50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f132400c600 tx=0x7f132c0a73b0).ready entity=mon.0 client_cookie=eed1630d2e3a0fdc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:03.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324005b80 con 0x7f13180049d0 2020-11-20T12:20:03.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1324005ce0 con 0x7f13180049d0 2020-11-20T12:20:03.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.000+0000 7f1329d9b700 1 -- 172.21.15.116:0/863189561 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13240072e0 con 0x7f13180049d0 2020-11-20T12:20:03.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.004+0000 7f1331207700 1 -- 172.21.15.116:0/863189561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13180049d0 msgr2=0x7f1318004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.004+0000 7f1331207700 1 --2- 172.21.15.116:0/863189561 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f13180049d0 0x7f1318004e50 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7f132400c600 tx=0x7f132c0a73b0).stop 2020-11-20T12:20:03.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.004+0000 7f132b59e700 1 -- 172.21.15.116:0/863189561 reap_dead start 2020-11-20T12:20:03.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.004+0000 7f1331207700 1 -- 172.21.15.116:0/863189561 shutdown_connections 2020-11-20T12:20:03.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.004+0000 7f1331207700 1 -- 172.21.15.116:0/863189561 >> 172.21.15.116:0/863189561 conn(0x7f132c060c00 msgr2=0x7f132c0c66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:03.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.004+0000 7f1331207700 1 -- 172.21.15.116:0/863189561 shutdown_connections 2020-11-20T12:20:03.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.004+0000 7f1331207700 1 -- 172.21.15.116:0/863189561 wait complete. 2020-11-20T12:20:03.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.004+0000 7f1331207700 1 Processor -- start 2020-11-20T12:20:03.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.004+0000 7f1331207700 1 -- start start 2020-11-20T12:20:03.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f1331207700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c14c4f0 0x7f132c09d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f1331207700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f132c14c8d0 0x7f132c165fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f1331207700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c09e1a0 0x7f132c0a56d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f1331207700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c09e990 con 0x7f132c09e1a0 2020-11-20T12:20:03.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f1331207700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f132c09ec30 con 0x7f132c14c8d0 2020-11-20T12:20:03.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f1331207700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f132c1655f0 con 0x7f132c14c4f0 2020-11-20T12:20:03.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c09e1a0 0x7f132c0a56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c09e1a0 0x7f132c0a56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48854/0 (socket says 172.21.15.116:48854) 2020-11-20T12:20:03.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132b59e700 1 -- 172.21.15.116:0/3959065737 learned_addr learned my addr 172.21.15.116:0/3959065737 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:03.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132a59c700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f132c14c8d0 0x7f132c165fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132ad9d700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c14c4f0 0x7f132c09d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132b59e700 1 -- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c14c4f0 msgr2=0x7f132c09d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132b59e700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c14c4f0 0x7f132c09d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132b59e700 1 -- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f132c14c8d0 msgr2=0x7f132c165fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132b59e700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f132c14c8d0 0x7f132c165fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132b59e700 1 -- 172.21.15.116:0/3959065737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132c165890 con 0x7f132c09e1a0 2020-11-20T12:20:03.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132ad9d700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c14c4f0 0x7f132c09d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:03.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f132b59e700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c09e1a0 0x7f132c0a56d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f132400c5d0 tx=0x7f1324013000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:03.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f1312ffd700 1 -- 172.21.15.116:0/3959065737 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324001860 con 0x7f132c09e1a0 2020-11-20T12:20:03.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f1312ffd700 1 -- 172.21.15.116:0/3959065737 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1324001290 con 0x7f132c09e1a0 2020-11-20T12:20:03.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f1331207700 1 -- 172.21.15.116:0/3959065737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f132c165b30 con 0x7f132c09e1a0 2020-11-20T12:20:03.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f1312ffd700 1 -- 172.21.15.116:0/3959065737 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1324013cd0 con 0x7f132c09e1a0 2020-11-20T12:20:03.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.008+0000 7f1331207700 1 -- 172.21.15.116:0/3959065737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f132c0a5bc0 con 0x7f132c09e1a0 2020-11-20T12:20:03.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.012+0000 7f1331207700 1 -- 172.21.15.116:0/3959065737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f132c165cc0 con 0x7f132c09e1a0 2020-11-20T12:20:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.012+0000 7f1312ffd700 1 -- 172.21.15.116:0/3959065737 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f132401c020 con 0x7f132c09e1a0 2020-11-20T12:20:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.012+0000 7f1312ffd700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f13080557c0 0x7f1308057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.012+0000 7f1312ffd700 1 -- 172.21.15.116:0/3959065737 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f132406fc80 con 0x7f132c09e1a0 2020-11-20T12:20:03.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.012+0000 7f132ad9d700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f13080557c0 0x7f1308057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.016+0000 7f132ad9d700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f13080557c0 0x7f1308057c80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f131c006bd0 tx=0x7f131c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:03.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.016+0000 7f1312ffd700 1 -- 172.21.15.116:0/3959065737 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1324045a10 con 0x7f132c09e1a0 2020-11-20T12:20:03.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.172+0000 7f1331207700 1 -- 172.21.15.116:0/3959065737 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f132c05d2a0 con 0x7f132c09e1a0 2020-11-20T12:20:03.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.172+0000 7f1312ffd700 1 -- 172.21.15.116:0/3959065737 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1324049030 con 0x7f132c09e1a0 2020-11-20T12:20:03.169 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 -- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f13080557c0 msgr2=0x7f1308057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f13080557c0 0x7f1308057c80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f131c006bd0 tx=0x7f131c008040).stop 2020-11-20T12:20:03.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 -- 172.21.15.116:0/3959065737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c09e1a0 msgr2=0x7f132c0a56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c09e1a0 0x7f132c0a56d0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f132400c5d0 tx=0x7f1324013000).stop 2020-11-20T12:20:03.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 -- 172.21.15.116:0/3959065737 shutdown_connections 2020-11-20T12:20:03.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f13080557c0 0x7f1308057c80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f132c09e1a0 0x7f132c0a56d0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f132c14c4f0 0x7f132c09d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 --2- 172.21.15.116:0/3959065737 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f132c14c8d0 0x7f132c165fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 -- 172.21.15.116:0/3959065737 >> 172.21.15.116:0/3959065737 conn(0x7f132c060c00 msgr2=0x7f132c154200 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:03.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 -- 172.21.15.116:0/3959065737 shutdown_connections 2020-11-20T12:20:03.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.176+0000 7f1331207700 1 -- 172.21.15.116:0/3959065737 wait complete. 2020-11-20T12:20:03.197 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:20:03.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f2f2e700 1 Processor -- start 2020-11-20T12:20:03.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f2f2e700 1 -- start start 2020-11-20T12:20:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f2f2e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff4ec156440 0x7ff4ec0aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f2f2e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec0ace40 con 0x7ff4ec0a8400 2020-11-20T12:20:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f2f2e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4ec1526c0 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f2f2e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4ec152960 con 0x7ff4ec156440 2020-11-20T12:20:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f172b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff4ec0a8400 0x7ff4ec0ad540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58294/0 (socket says 172.21.15.116:58294) 2020-11-20T12:20:03.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f172b700 1 -- 172.21.15.116:0/27653558 learned_addr learned my addr 172.21.15.116:0/27653558 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f272d700 1 --2- 172.21.15.116:0/27653558 >> v2:172.21.15.173:3300/0 conn(0x7ff4ec156440 0x7ff4ec0aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f0f2a700 1 -- 172.21.15.116:0/27653558 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655233559 0 0) 0x7ff4ec0ace40 con 0x7ff4ec0a8400 2020-11-20T12:20:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f0f2a700 1 -- 172.21.15.116:0/27653558 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4d4003240 con 0x7ff4ec0a8400 2020-11-20T12:20:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f0f2a700 1 -- 172.21.15.116:0/27653558 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2209244663 0 0) 0x7ff4ec1526c0 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.380+0000 7ff4f0f2a700 1 -- 172.21.15.116:0/27653558 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4ec0ace40 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f272d700 1 -- 172.21.15.116:0/27653558 >> v1:172.21.15.173:6789/0 conn(0x7ff4ec0aa7d0 legacy=0x7ff4ec151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f272d700 1 -- 172.21.15.116:0/27653558 >> v1:172.21.15.116:6789/0 conn(0x7ff4ec0a8400 legacy=0x7ff4ec0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f0f2a700 1 -- 172.21.15.116:0/27653558 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3969080117 0 0) 0x7ff4d4003240 con 0x7ff4ec0a8400 2020-11-20T12:20:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f272d700 1 -- 172.21.15.116:0/27653558 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec152c00 con 0x7ff4ec156440 2020-11-20T12:20:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f272d700 1 --2- 172.21.15.116:0/27653558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ec156440 0x7ff4ec0aabb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7ff4e4009c90 tx=0x7ff4e40091d0).ready entity=mon.1 client_cookie=269c9d854de14259 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:03.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f0f2a700 1 -- 172.21.15.116:0/27653558 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400b9c0 con 0x7ff4ec156440 2020-11-20T12:20:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f0f2a700 1 -- 172.21.15.116:0/27653558 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e400bb20 con 0x7ff4ec156440 2020-11-20T12:20:03.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f0f2a700 1 -- 172.21.15.116:0/27653558 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e400bea0 con 0x7ff4ec156440 2020-11-20T12:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/27653558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ec156440 msgr2=0x7ff4ec0aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f2f2e700 1 --2- 172.21.15.116:0/27653558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ec156440 0x7ff4ec0aabb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7ff4e4009c90 tx=0x7ff4e40091d0).stop 2020-11-20T12:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/27653558 shutdown_connections 2020-11-20T12:20:03.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f2f2e700 1 --2- 172.21.15.116:0/27653558 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ec156440 0x7ff4ec0aabb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/27653558 >> 172.21.15.116:0/27653558 conn(0x7ff4ec0678a0 msgr2=0x7ff4ec067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/27653558 shutdown_connections 2020-11-20T12:20:03.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.384+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/27653558 wait complete. 2020-11-20T12:20:03.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f2f2e700 1 Processor -- start 2020-11-20T12:20:03.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f2f2e700 1 -- start start 2020-11-20T12:20:03.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f2f2e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4ec0a8400 0x7ff4ec150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f2f2e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ec0aa7d0 0x7ff4ec14a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f2f2e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ec156440 0x7ff4ec1642b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f2f2e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec14cf80 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f2f2e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec14d1b0 con 0x7ff4ec156440 2020-11-20T12:20:03.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f2f2e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec14d450 con 0x7ff4ec0a8400 2020-11-20T12:20:03.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f172b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ec0aa7d0 0x7ff4ec14a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f172b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ec0aa7d0 0x7ff4ec14a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48866/0 (socket says 172.21.15.116:48866) 2020-11-20T12:20:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f172b700 1 -- 172.21.15.116:0/3821575680 learned_addr learned my addr 172.21.15.116:0/3821575680 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f272d700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ec156440 0x7ff4ec1642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f1f2c700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4ec0a8400 0x7ff4ec150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f172b700 1 -- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4ec0a8400 msgr2=0x7ff4ec150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f172b700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4ec0a8400 0x7ff4ec150e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f172b700 1 -- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ec156440 msgr2=0x7ff4ec1642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f172b700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ec156440 0x7ff4ec1642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f172b700 1 -- 172.21.15.116:0/3821575680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec14d6f0 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.388+0000 7ff4f272d700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ec156440 0x7ff4ec1642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.392+0000 7ff4f172b700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ec0aa7d0 0x7ff4ec14a1f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7ff4dc00c760 tx=0x7ff4dc007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.392+0000 7ff4d9ffb700 1 -- 172.21.15.116:0/3821575680 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc007170 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.392+0000 7ff4d9ffb700 1 -- 172.21.15.116:0/3821575680 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4dc0072d0 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.392+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/3821575680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4ec14d990 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.392+0000 7ff4d9ffb700 1 -- 172.21.15.116:0/3821575680 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4dc004920 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.392+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/3821575680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4ec151310 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.392+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/3821575680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4ec05cf30 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.392+0000 7ff4d9ffb700 1 -- 172.21.15.116:0/3821575680 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff4dc01b070 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.396+0000 7ff4d9ffb700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4c40557c0 0x7ff4c4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.396+0000 7ff4d9ffb700 1 -- 172.21.15.116:0/3821575680 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff4dc06de10 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.396+0000 7ff4f1f2c700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4c40557c0 0x7ff4c4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.396+0000 7ff4f1f2c700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4c40557c0 0x7ff4c4057c80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7ff4ec154f10 tx=0x7ff4e8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:03.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.400+0000 7ff4d9ffb700 1 -- 172.21.15.116:0/3821575680 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff4dc043cb0 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.560+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/3821575680 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff4ec0601b0 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.560+0000 7ff4d9ffb700 1 -- 172.21.15.116:0/3821575680 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff4dc019030 con 0x7ff4ec0aa7d0 2020-11-20T12:20:03.559 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:03.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4c40557c0 msgr2=0x7ff4c4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4c40557c0 0x7ff4c4057c80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7ff4ec154f10 tx=0x7ff4e8009040).stop 2020-11-20T12:20:03.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/3821575680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ec0aa7d0 msgr2=0x7ff4ec14a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ec0aa7d0 0x7ff4ec14a1f0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7ff4dc00c760 tx=0x7ff4dc007a90).stop 2020-11-20T12:20:03.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/3821575680 shutdown_connections 2020-11-20T12:20:03.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff4c40557c0 0x7ff4c4057c80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4ec156440 0x7ff4ec1642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4ec0a8400 0x7ff4ec150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 --2- 172.21.15.116:0/3821575680 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4ec0aa7d0 0x7ff4ec14a1f0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/3821575680 >> 172.21.15.116:0/3821575680 conn(0x7ff4ec0678a0 msgr2=0x7ff4ec061db0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:03.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/3821575680 shutdown_connections 2020-11-20T12:20:03.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.564+0000 7ff4f2f2e700 1 -- 172.21.15.116:0/3821575680 wait complete. 2020-11-20T12:20:03.588 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:20:03.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.764+0000 7f88df59e700 1 Processor -- start 2020-11-20T12:20:03.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.764+0000 7f88df59e700 1 -- start start 2020-11-20T12:20:03.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88df59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88e014f880 0x7f88e00b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88df59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f88e00c0e90 0x7f88e00c1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88df59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88e00a8930 con 0x7f88e00a81f0 2020-11-20T12:20:03.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88df59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e00ba2a0 con 0x7f88e00c0e90 2020-11-20T12:20:03.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88df59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e00a8690 con 0x7f88e014f880 2020-11-20T12:20:03.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88ded9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f88e00c0e90 0x7f88e00c1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88ddd9b700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f88e00a81f0 0x7f88e0060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:54536/0 (socket says 172.21.15.116:54536) 2020-11-20T12:20:03.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88ddd9b700 1 -- 172.21.15.116:0/3711205364 learned_addr learned my addr 172.21.15.116:0/3711205364 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:03.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88de59c700 1 --2- 172.21.15.116:0/3711205364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88e014f880 0x7f88e00b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88ded9d700 1 -- 172.21.15.116:0/3711205364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88e014f880 msgr2=0x7f88e00b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:03.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88ded9d700 1 --2- 172.21.15.116:0/3711205364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88e014f880 0x7f88e00b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88ded9d700 1 -- 172.21.15.116:0/3711205364 >> v1:172.21.15.173:6789/0 conn(0x7f88e00a81f0 legacy=0x7f88e0060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88dd59a700 1 -- 172.21.15.116:0/3711205364 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760799407 0 0) 0x7f88e00a8930 con 0x7f88e00a81f0 2020-11-20T12:20:03.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88ded9d700 1 -- 172.21.15.116:0/3711205364 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e00b94d0 con 0x7f88e00c0e90 2020-11-20T12:20:03.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88ded9d700 1 --2- 172.21.15.116:0/3711205364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88e00c0e90 0x7f88e00c1300 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f88d8009c90 tx=0x7f88d80091d0).ready entity=mon.0 client_cookie=f5b18e25e1145724 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:03.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88dd59a700 1 -- 172.21.15.116:0/3711205364 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d800ba70 con 0x7f88e00c0e90 2020-11-20T12:20:03.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88dd59a700 1 -- 172.21.15.116:0/3711205364 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d800bbd0 con 0x7f88e00c0e90 2020-11-20T12:20:03.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.768+0000 7f88dd59a700 1 -- 172.21.15.116:0/3711205364 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d8011400 con 0x7f88e00c0e90 2020-11-20T12:20:03.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.772+0000 7f88df59e700 1 -- 172.21.15.116:0/3711205364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88e00c0e90 msgr2=0x7f88e00c1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.772+0000 7f88df59e700 1 --2- 172.21.15.116:0/3711205364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88e00c0e90 0x7f88e00c1300 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f88d8009c90 tx=0x7f88d80091d0).stop 2020-11-20T12:20:03.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.772+0000 7f88df59e700 1 -- 172.21.15.116:0/3711205364 shutdown_connections 2020-11-20T12:20:03.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.772+0000 7f88df59e700 1 --2- 172.21.15.116:0/3711205364 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88e00c0e90 0x7f88e00c1300 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.772+0000 7f88df59e700 1 --2- 172.21.15.116:0/3711205364 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88e014f880 0x7f88e00b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.772+0000 7f88df59e700 1 -- 172.21.15.116:0/3711205364 >> 172.21.15.116:0/3711205364 conn(0x7f88e0068350 msgr2=0x7f88e00627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.772+0000 7f88df59e700 1 -- 172.21.15.116:0/3711205364 shutdown_connections 2020-11-20T12:20:03.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.772+0000 7f88df59e700 1 -- 172.21.15.116:0/3711205364 wait complete. 2020-11-20T12:20:03.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.772+0000 7f88df59e700 1 Processor -- start 2020-11-20T12:20:03.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.772+0000 7f88df59e700 1 -- start start 2020-11-20T12:20:03.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88df59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88e00a81f0 0x7f88e01619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88df59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88e00c0e90 0x7f88e00bc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88df59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88e014f880 0x7f88e01646c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88df59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e0165120 con 0x7f88e014f880 2020-11-20T12:20:03.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88df59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e01653c0 con 0x7f88e00c0e90 2020-11-20T12:20:03.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88df59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e0161270 con 0x7f88e00a81f0 2020-11-20T12:20:03.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ded9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88e014f880 0x7f88e01646c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ded9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88e014f880 0x7f88e01646c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48882/0 (socket says 172.21.15.116:48882) 2020-11-20T12:20:03.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ded9d700 1 -- 172.21.15.116:0/3780117958 learned_addr learned my addr 172.21.15.116:0/3780117958 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:03.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ddd9b700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88e00c0e90 0x7f88e00bc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88de59c700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88e00a81f0 0x7f88e01619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ded9d700 1 -- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88e00a81f0 msgr2=0x7f88e01619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ded9d700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88e00a81f0 0x7f88e01619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ded9d700 1 -- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88e00c0e90 msgr2=0x7f88e00bc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ded9d700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88e00c0e90 0x7f88e00bc330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ded9d700 1 -- 172.21.15.116:0/3780117958 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e0161510 con 0x7f88e014f880 2020-11-20T12:20:03.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ded9d700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88e014f880 0x7f88e01646c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f88d8011f40 tx=0x7f88d8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:03.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ce7fc700 1 -- 172.21.15.116:0/3780117958 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d801c5f0 con 0x7f88e014f880 2020-11-20T12:20:03.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ce7fc700 1 -- 172.21.15.116:0/3780117958 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88d801c750 con 0x7f88e014f880 2020-11-20T12:20:03.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88df59e700 1 -- 172.21.15.116:0/3780117958 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e0151760 con 0x7f88e014f880 2020-11-20T12:20:03.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88ce7fc700 1 -- 172.21.15.116:0/3780117958 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88d801cad0 con 0x7f88e014f880 2020-11-20T12:20:03.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.776+0000 7f88df59e700 1 -- 172.21.15.116:0/3780117958 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e0162040 con 0x7f88e014f880 2020-11-20T12:20:03.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.780+0000 7f88df59e700 1 -- 172.21.15.116:0/3780117958 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e0165590 con 0x7f88e014f880 2020-11-20T12:20:03.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.780+0000 7f88ce7fc700 1 -- 172.21.15.116:0/3780117958 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f88d8024030 con 0x7f88e014f880 2020-11-20T12:20:03.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.780+0000 7f88ce7fc700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88b4055810 0x7f88b4057cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:03.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.780+0000 7f88ce7fc700 1 -- 172.21.15.116:0/3780117958 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f88d80700d0 con 0x7f88e014f880 2020-11-20T12:20:03.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.780+0000 7f88de59c700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88b4055810 0x7f88b4057cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:03.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.780+0000 7f88de59c700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88b4055810 0x7f88b4057cd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f88d0006fd0 tx=0x7f88d0006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:03.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.788+0000 7f88ce7fc700 1 -- 172.21.15.116:0/3780117958 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f88d80460a0 con 0x7f88e014f880 2020-11-20T12:20:03.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.944+0000 7f88df59e700 1 -- 172.21.15.116:0/3780117958 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f88e01616a0 con 0x7f88e014f880 2020-11-20T12:20:03.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.944+0000 7f88ce7fc700 1 -- 172.21.15.116:0/3780117958 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+10 (secure 0 0 0) 0x7f88d8022020 con 0x7f88e014f880 2020-11-20T12:20:03.942 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 -- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88b4055810 msgr2=0x7f88b4057cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88b4055810 0x7f88b4057cd0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f88d0006fd0 tx=0x7f88d0006df0).stop 2020-11-20T12:20:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 -- 172.21.15.116:0/3780117958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88e014f880 msgr2=0x7f88e01646c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:03.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88e014f880 0x7f88e01646c0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f88d8011f40 tx=0x7f88d8016000).stop 2020-11-20T12:20:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 -- 172.21.15.116:0/3780117958 shutdown_connections 2020-11-20T12:20:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88b4055810 0x7f88b4057cd0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88e014f880 0x7f88e01646c0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88e00a81f0 0x7f88e01619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 --2- 172.21.15.116:0/3780117958 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88e00c0e90 0x7f88e00bc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 -- 172.21.15.116:0/3780117958 >> 172.21.15.116:0/3780117958 conn(0x7f88e0068350 msgr2=0x7f88e00bb520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 -- 172.21.15.116:0/3780117958 shutdown_connections 2020-11-20T12:20:03.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:03.948+0000 7f88df59e700 1 -- 172.21.15.116:0/3780117958 wait complete. 2020-11-20T12:20:03.967 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:20:03.968 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:04.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.144+0000 7fd906a1c700 1 Processor -- start 2020-11-20T12:20:04.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.144+0000 7fd906a1c700 1 -- start start 2020-11-20T12:20:04.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.148+0000 7fd906a1c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd9000c3780 0x7fd9000c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:04.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.148+0000 7fd906a1c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9000a6280 0x7fd9000c1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:04.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.148+0000 7fd906a1c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9000c1e60 con 0x7fd9000c22f0 2020-11-20T12:20:04.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.148+0000 7fd906a1c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90014fed0 con 0x7fd9000c3780 2020-11-20T12:20:04.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.148+0000 7fd906a1c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000a6700 con 0x7fd9000a6280 2020-11-20T12:20:04.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.148+0000 7fd904fb9700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd9000c22f0 0x7fd900150560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58324/0 (socket says 172.21.15.116:58324) 2020-11-20T12:20:04.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.148+0000 7fd904fb9700 1 -- 172.21.15.116:0/3471242966 learned_addr learned my addr 172.21.15.116:0/3471242966 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:04.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.148+0000 7fd8fffff700 1 --2- 172.21.15.116:0/3471242966 >> v2:172.21.15.173:3300/0 conn(0x7fd9000c3780 0x7fd9000c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:04.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.148+0000 7fd8f77fe700 1 --2- 172.21.15.116:0/3471242966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9000a6280 0x7fd9000c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:04.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.148+0000 7fd8ff7fe700 1 -- 172.21.15.116:0/3471242966 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 947062154 0 0) 0x7fd8f8001450 con 0x7fd9000c22f0 2020-11-20T12:20:04.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.148+0000 7fd8ff7fe700 1 -- 172.21.15.116:0/3471242966 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e0003240 con 0x7fd9000c22f0 2020-11-20T12:20:04.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd8fffff700 1 -- 172.21.15.116:0/3471242966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9000a6280 msgr2=0x7fd9000c1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:04.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd8fffff700 1 --2- 172.21.15.116:0/3471242966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9000a6280 0x7fd9000c1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:04.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd8fffff700 1 -- 172.21.15.116:0/3471242966 >> v1:172.21.15.116:6789/0 conn(0x7fd9000c22f0 legacy=0x7fd900150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:04.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd8fffff700 1 -- 172.21.15.116:0/3471242966 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9000a6950 con 0x7fd9000c3780 2020-11-20T12:20:04.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd8f77fe700 1 --2- 172.21.15.116:0/3471242966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9000a6280 0x7fd9000c1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:04.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd8fffff700 1 --2- 172.21.15.116:0/3471242966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9000c3780 0x7fd9000c3b60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fd8f000b9d0 tx=0x7fd8f0006df0).ready entity=mon.1 client_cookie=206a02b89e09d761 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:04.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd8ff7fe700 1 -- 172.21.15.116:0/3471242966 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0009640 con 0x7fd9000c3780 2020-11-20T12:20:04.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd8ff7fe700 1 -- 172.21.15.116:0/3471242966 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f00097a0 con 0x7fd9000c3780 2020-11-20T12:20:04.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd8ff7fe700 1 -- 172.21.15.116:0/3471242966 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0009b20 con 0x7fd9000c3780 2020-11-20T12:20:04.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd906a1c700 1 -- 172.21.15.116:0/3471242966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9000c3780 msgr2=0x7fd9000c3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:04.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd906a1c700 1 --2- 172.21.15.116:0/3471242966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9000c3780 0x7fd9000c3b60 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7fd8f000b9d0 tx=0x7fd8f0006df0).stop 2020-11-20T12:20:04.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd906a1c700 1 -- 172.21.15.116:0/3471242966 shutdown_connections 2020-11-20T12:20:04.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd906a1c700 1 --2- 172.21.15.116:0/3471242966 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9000c3780 0x7fd9000c3b60 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:04.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd906a1c700 1 --2- 172.21.15.116:0/3471242966 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9000a6280 0x7fd9000c1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:04.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd906a1c700 1 -- 172.21.15.116:0/3471242966 >> 172.21.15.116:0/3471242966 conn(0x7fd9000c4f80 msgr2=0x7fd9000c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:04.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd906a1c700 1 -- 172.21.15.116:0/3471242966 shutdown_connections 2020-11-20T12:20:04.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.152+0000 7fd906a1c700 1 -- 172.21.15.116:0/3471242966 wait complete. 2020-11-20T12:20:04.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.156+0000 7fd906a1c700 1 Processor -- start 2020-11-20T12:20:04.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.156+0000 7fd906a1c700 1 -- start start 2020-11-20T12:20:04.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd906a1c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9000a6280 0x7fd90014d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:04.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd906a1c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9000c22f0 0x7fd9000a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:04.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd906a1c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9000c3780 0x7fd9000a5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:04.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd906a1c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000a9520 con 0x7fd9000c22f0 2020-11-20T12:20:04.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd906a1c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd900162260 con 0x7fd9000c3780 2020-11-20T12:20:04.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd906a1c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd900162500 con 0x7fd9000a6280 2020-11-20T12:20:04.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8f77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9000c22f0 0x7fd9000a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:04.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8f77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9000c22f0 0x7fd9000a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48894/0 (socket says 172.21.15.116:48894) 2020-11-20T12:20:04.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8f77fe700 1 -- 172.21.15.116:0/3692064457 learned_addr learned my addr 172.21.15.116:0/3692064457 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:04.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd904fb9700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9000c3780 0x7fd9000a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:04.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8fffff700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9000a6280 0x7fd90014d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:04.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8f77fe700 1 -- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9000a6280 msgr2=0x7fd90014d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:04.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8f77fe700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9000a6280 0x7fd90014d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:04.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8f77fe700 1 -- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9000c3780 msgr2=0x7fd9000a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:04.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8f77fe700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9000c3780 0x7fd9000a5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:04.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8f77fe700 1 -- 172.21.15.116:0/3692064457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9000a9710 con 0x7fd9000c22f0 2020-11-20T12:20:04.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8f77fe700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9000c22f0 0x7fd9000a9e50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fd8e800ed20 tx=0x7fd8e8006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:04.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8fcff9700 1 -- 172.21.15.116:0/3692064457 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800caa0 con 0x7fd9000c22f0 2020-11-20T12:20:04.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8fcff9700 1 -- 172.21.15.116:0/3692064457 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e800cc00 con 0x7fd9000c22f0 2020-11-20T12:20:04.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd906a1c700 1 -- 172.21.15.116:0/3692064457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9000a99b0 con 0x7fd9000c22f0 2020-11-20T12:20:04.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd8fcff9700 1 -- 172.21.15.116:0/3692064457 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e80123f0 con 0x7fd9000c22f0 2020-11-20T12:20:04.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.160+0000 7fd906a1c700 1 -- 172.21.15.116:0/3692064457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9000a8f00 con 0x7fd9000c22f0 2020-11-20T12:20:04.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.164+0000 7fd906a1c700 1 -- 172.21.15.116:0/3692064457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9000a9b40 con 0x7fd9000c22f0 2020-11-20T12:20:04.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.164+0000 7fd8fcff9700 1 -- 172.21.15.116:0/3692064457 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd8e80127a0 con 0x7fd9000c22f0 2020-11-20T12:20:04.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.164+0000 7fd8fcff9700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8dc0557c0 0x7fd8dc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:04.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.164+0000 7fd8fcff9700 1 -- 172.21.15.116:0/3692064457 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd8e806ebc0 con 0x7fd9000c22f0 2020-11-20T12:20:04.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.164+0000 7fd8fffff700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8dc0557c0 0x7fd8dc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:04.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.168+0000 7fd8fffff700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8dc0557c0 0x7fd8dc057c80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fd8f0006e30 tx=0x7fd8f0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:04.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.172+0000 7fd8fcff9700 1 -- 172.21.15.116:0/3692064457 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd8e8044a60 con 0x7fd9000c22f0 2020-11-20T12:20:04.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.380+0000 7fd906a1c700 1 -- 172.21.15.116:0/3692064457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd90005d2a0 con 0x7fd9000c22f0 2020-11-20T12:20:04.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.380+0000 7fd8fcff9700 1 -- 172.21.15.116:0/3692064457 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2848 (secure 0 0 0) 0x7fd8e8048080 con 0x7fd9000c22f0 2020-11-20T12:20:04.379 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:04.379 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/313210621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":313210621},{"type":"v1","addr":"172.21.15.173:6835","nonce":313210621}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/2140340557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2140340557},{"type":"v1","addr":"172.21.15.173:6837","nonce":2140340557}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3544207168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3544207168},{"type":"v1","addr":"172.21.15.116:6833","nonce":3544207168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:57.303415+0000","modified":"2020-11-20T12:19:59.322603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/657163487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":657163487},{"type":"v1","addr":"172.21.15.116:6835","nonce":657163487}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-20T12:20:04.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 -- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8dc0557c0 msgr2=0x7fd8dc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:04.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8dc0557c0 0x7fd8dc057c80 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7fd8f0006e30 tx=0x7fd8f0015000).stop 2020-11-20T12:20:04.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 -- 172.21.15.116:0/3692064457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9000c22f0 msgr2=0x7fd9000a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:04.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9000c22f0 0x7fd9000a9e50 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7fd8e800ed20 tx=0x7fd8e8006df0).stop 2020-11-20T12:20:04.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 -- 172.21.15.116:0/3692064457 shutdown_connections 2020-11-20T12:20:04.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8dc0557c0 0x7fd8dc057c80 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:04.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9000c3780 0x7fd9000a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:04.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9000a6280 0x7fd90014d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:04.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 --2- 172.21.15.116:0/3692064457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9000c22f0 0x7fd9000a9e50 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:04.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 -- 172.21.15.116:0/3692064457 >> 172.21.15.116:0/3692064457 conn(0x7fd9000c4f80 msgr2=0x7fd9000bff30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:04.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 -- 172.21.15.116:0/3692064457 shutdown_connections 2020-11-20T12:20:04.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:04.384+0000 7fd906a1c700 1 -- 172.21.15.116:0/3692064457 wait complete. 2020-11-20T12:20:04.386 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 46 2020-11-20T12:20:04.806 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:19:57.303415+0000', 'modified': '2020-11-20T12:19:59.322603+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6861}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6861': {'gid': 6861, 'name': 'a', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/657163487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 657163487}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 657163487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:20:04.807 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:20:04.808 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:20:05.002 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":6861,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.679263443,"mdsmap_epoch":46,"osdmap_epoch":86,"osdmap_epoch_barrier":86,"uptime":14.245492125} 2020-11-20T12:20:05.002 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 6861, "mdsmap_epoch": 46, "osdmap_epoch": 86, "osdmap_epoch_barrier": 86, "rank_uptime": 7.679263443, "state": "up:active", "uptime": 14.245492125, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:20:05.003 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:20:05.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:05.004 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-20T12:20:05.024 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:20:05.024 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:05.142 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:05.143 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:20:05.170 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:05.170 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:20:05.181 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:20:05.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:05.183 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-20T12:20:05.219 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:20:05.220 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:20:05.220 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-20T12:20:05.221 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-20T12:20:05.221 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:20:05.221 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:20:05.222 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:20:05.222 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:20:05.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:05.223 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:20:05.244 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:20:05.245 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:20:05.245 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:20:05.245 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:05.246 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:20:05.246 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:05.246 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:05.246 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:05.246 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:05.247 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:05.247 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:05.247 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:05.247 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:20:05.247 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:05.248 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:20:05.248 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:05.249 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:20:05.250 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:05.250 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:20:05.251 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:05.251 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:20:05.251 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:05.252 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:20:05.252 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:05.253 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:20:05.254 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:05.254 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:20:05.255 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:05.255 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:05.255 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:20:05.256 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:20:05.256 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:05.256 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:05.257 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:20:05.257 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:20:05.258 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:05.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:05.259 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:20:05.323 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:20:05.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:05.327 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:20:05.345 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:20:05.345 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:20:05.346 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:20:05.346 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:20:05.346 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:20:05.346 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:20:05.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:05.348 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:20:05.348 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:20:05.349 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:20:05.349 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:20:05.350 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:20:05.350 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:20:05.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:05.439 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:20:05.453 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:20:05.453 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:20:05.454 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:20:05.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:05.455 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2020-11-20T12:20:05.489 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:05.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.668+0000 7f389ad4e700 1 Processor -- start 2020-11-20T12:20:05.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.672+0000 7f389ad4e700 1 -- start start 2020-11-20T12:20:05.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.672+0000 7f389ad4e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894062880 0x7f3894062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.672+0000 7f389ad4e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3894065d40 0x7f38940bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.672+0000 7f389ad4e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38940c0530 con 0x7f38940677a0 2020-11-20T12:20:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.672+0000 7f389ad4e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38940661a0 con 0x7f3894065d40 2020-11-20T12:20:05.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.672+0000 7f389ad4e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38940c0310 con 0x7f3894062880 2020-11-20T12:20:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.672+0000 7f3899d4c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894062880 0x7f3894062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.672+0000 7f3899d4c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894062880 0x7f3894062c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:57854/0 (socket says 172.21.15.116:57854) 2020-11-20T12:20:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.672+0000 7f3899d4c700 1 -- 172.21.15.116:0/963931656 learned_addr learned my addr 172.21.15.116:0/963931656 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:05.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.672+0000 7f389954b700 1 --2- 172.21.15.116:0/963931656 >> v2:172.21.15.173:3300/0 conn(0x7f3894065d40 0x7f38940bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:05.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f3899d4c700 1 -- 172.21.15.116:0/963931656 >> v2:172.21.15.173:3300/0 conn(0x7f3894065d40 msgr2=0x7f38940bfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:05.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f3899d4c700 1 --2- 172.21.15.116:0/963931656 >> v2:172.21.15.173:3300/0 conn(0x7f3894065d40 0x7f38940bfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:05.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f3899d4c700 1 -- 172.21.15.116:0/963931656 >> v1:172.21.15.173:6789/0 conn(0x7f38940677a0 legacy=0x7f38941556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:05.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f3899d4c700 1 -- 172.21.15.116:0/963931656 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38940c07d0 con 0x7f3894062880 2020-11-20T12:20:05.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f3899d4c700 1 --2- 172.21.15.116:0/963931656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894062880 0x7f3894062c60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f3890006fd0 tx=0x7f3890006eb0).ready entity=mon.2 client_cookie=857e11ac197e7431 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:05.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f3898d4a700 1 -- 172.21.15.116:0/963931656 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38900098d0 con 0x7f3894062880 2020-11-20T12:20:05.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f3898d4a700 1 -- 172.21.15.116:0/963931656 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3890009a30 con 0x7f3894062880 2020-11-20T12:20:05.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f3898d4a700 1 -- 172.21.15.116:0/963931656 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3890009db0 con 0x7f3894062880 2020-11-20T12:20:05.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f389ad4e700 1 -- 172.21.15.116:0/963931656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894062880 msgr2=0x7f3894062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:05.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f389ad4e700 1 --2- 172.21.15.116:0/963931656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894062880 0x7f3894062c60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f3890006fd0 tx=0x7f3890006eb0).stop 2020-11-20T12:20:05.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f389ad4e700 1 -- 172.21.15.116:0/963931656 shutdown_connections 2020-11-20T12:20:05.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f389ad4e700 1 --2- 172.21.15.116:0/963931656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894062880 0x7f3894062c60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:05.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f389ad4e700 1 --2- 172.21.15.116:0/963931656 >> v2:172.21.15.173:3300/0 conn(0x7f3894065d40 0x7f38940bfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:05.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f389ad4e700 1 -- 172.21.15.116:0/963931656 >> 172.21.15.116:0/963931656 conn(0x7f3894060c40 msgr2=0x7f38940c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:05.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f389ad4e700 1 -- 172.21.15.116:0/963931656 shutdown_connections 2020-11-20T12:20:05.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.676+0000 7f389ad4e700 1 -- 172.21.15.116:0/963931656 wait complete. 2020-11-20T12:20:05.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.680+0000 7f389ad4e700 1 Processor -- start 2020-11-20T12:20:05.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.680+0000 7f389ad4e700 1 -- start start 2020-11-20T12:20:05.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.680+0000 7f389ad4e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3894062880 0x7f3894156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:05.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.680+0000 7f389ad4e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894065d40 0x7f38941533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:05.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.680+0000 7f389ad4e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38940677a0 0x7f3894151f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:05.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.680+0000 7f389ad4e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38941558d0 con 0x7f3894062880 2020-11-20T12:20:05.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.680+0000 7f389ad4e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3894155ac0 con 0x7f38940677a0 2020-11-20T12:20:05.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.680+0000 7f389ad4e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3894166210 con 0x7f3894065d40 2020-11-20T12:20:05.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.680+0000 7f3899d4c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3894062880 0x7f3894156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:05.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.680+0000 7f3899d4c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3894062880 0x7f3894156460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48910/0 (socket says 172.21.15.116:48910) 2020-11-20T12:20:05.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.680+0000 7f3899d4c700 1 -- 172.21.15.116:0/2066987615 learned_addr learned my addr 172.21.15.116:0/2066987615 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:05.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f389954b700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894065d40 0x7f38941533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:05.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f389a54d700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38940677a0 0x7f3894151f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:05.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f3899d4c700 1 -- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894065d40 msgr2=0x7f38941533d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:05.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f3899d4c700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894065d40 0x7f38941533d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:05.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f3899d4c700 1 -- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38940677a0 msgr2=0x7f3894151f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:05.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f3899d4c700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38940677a0 0x7f3894151f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:05.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f3899d4c700 1 -- 172.21.15.116:0/2066987615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3894165f70 con 0x7f3894062880 2020-11-20T12:20:05.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f389a54d700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38940677a0 0x7f3894151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:05.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f3899d4c700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3894062880 0x7f3894156460 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f3890006ef0 tx=0x7f3890015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:05.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f3881ffb700 1 -- 172.21.15.116:0/2066987615 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389001e460 con 0x7f3894062880 2020-11-20T12:20:05.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f3881ffb700 1 -- 172.21.15.116:0/2066987615 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389001e5c0 con 0x7f3894062880 2020-11-20T12:20:05.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f3881ffb700 1 -- 172.21.15.116:0/2066987615 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389001e980 con 0x7f3894062880 2020-11-20T12:20:05.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f389ad4e700 1 -- 172.21.15.116:0/2066987615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3894155d20 con 0x7f3894062880 2020-11-20T12:20:05.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.684+0000 7f389ad4e700 1 -- 172.21.15.116:0/2066987615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38941569d0 con 0x7f3894062880 2020-11-20T12:20:05.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.688+0000 7f389ad4e700 1 -- 172.21.15.116:0/2066987615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f389405cf30 con 0x7f3894062880 2020-11-20T12:20:05.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.688+0000 7f3881ffb700 1 -- 172.21.15.116:0/2066987615 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3890025030 con 0x7f3894062880 2020-11-20T12:20:05.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.688+0000 7f3881ffb700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f386c0557c0 0x7f386c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:05.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.688+0000 7f3881ffb700 1 -- 172.21.15.116:0/2066987615 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f389006fec0 con 0x7f3894062880 2020-11-20T12:20:05.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.688+0000 7f389954b700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f386c0557c0 0x7f386c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:05.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.692+0000 7f389954b700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f386c0557c0 0x7f386c057c80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f3894066ce0 tx=0x7f3884008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:05.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.696+0000 7f3881ffb700 1 -- 172.21.15.116:0/2066987615 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3890045d00 con 0x7f3894062880 2020-11-20T12:20:05.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.852+0000 7f389ad4e700 1 -- 172.21.15.116:0/2066987615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3894061230 con 0x7f3894062880 2020-11-20T12:20:05.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.856+0000 7f3881ffb700 1 -- 172.21.15.116:0/2066987615 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f3890031200 con 0x7f3894062880 2020-11-20T12:20:05.854 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:05.854 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":86,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:58.311412+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:54.974437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:56.326129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:05.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 -- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f386c0557c0 msgr2=0x7f386c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:05.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f386c0557c0 0x7f386c057c80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f3894066ce0 tx=0x7f3884008040).stop 2020-11-20T12:20:05.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 -- 172.21.15.116:0/2066987615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3894062880 msgr2=0x7f3894156460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:05.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3894062880 0x7f3894156460 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f3890006ef0 tx=0x7f3890015000).stop 2020-11-20T12:20:05.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 -- 172.21.15.116:0/2066987615 shutdown_connections 2020-11-20T12:20:05.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f386c0557c0 0x7f386c057c80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:05.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f38940677a0 0x7f3894151f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:05.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3894062880 0x7f3894156460 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:05.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 --2- 172.21.15.116:0/2066987615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3894065d40 0x7f38941533d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:05.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 -- 172.21.15.116:0/2066987615 >> 172.21.15.116:0/2066987615 conn(0x7f3894060c40 msgr2=0x7f3894154790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:05.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 -- 172.21.15.116:0/2066987615 shutdown_connections 2020-11-20T12:20:05.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:05.860+0000 7f389ad4e700 1 -- 172.21.15.116:0/2066987615 wait complete. 2020-11-20T12:20:05.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:19:54.974437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:19:56.326129+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:20:05.891 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:20:06.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.076+0000 7fd55aec4700 1 Processor -- start 2020-11-20T12:20:06.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.076+0000 7fd55aec4700 1 -- start start 2020-11-20T12:20:06.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.076+0000 7fd55aec4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd554155c80 0x7fd554062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.076+0000 7fd55aec4700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd554156060 0x7fd5540c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.076+0000 7fd55aec4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5540c12c0 0x7fd554060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd55aec4700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5540ba580 con 0x7fd554155c80 2020-11-20T12:20:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd55aec4700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5540b93c0 con 0x7fd554156060 2020-11-20T12:20:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd55aec4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540b9660 con 0x7fd5540c12c0 2020-11-20T12:20:06.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd559ec2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd554155c80 0x7fd554062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd559ec2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd554155c80 0x7fd554062ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48918/0 (socket says 172.21.15.116:48918) 2020-11-20T12:20:06.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd559ec2700 1 -- 172.21.15.116:0/2004282753 learned_addr learned my addr 172.21.15.116:0/2004282753 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd55a6c3700 1 --2- 172.21.15.116:0/2004282753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5540c12c0 0x7fd554060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd5596c1700 1 --2- 172.21.15.116:0/2004282753 >> v2:172.21.15.173:3300/0 conn(0x7fd554156060 0x7fd5540c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd559ec2700 1 -- 172.21.15.116:0/2004282753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5540c12c0 msgr2=0x7fd554060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:06.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd559ec2700 1 --2- 172.21.15.116:0/2004282753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5540c12c0 0x7fd554060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd559ec2700 1 -- 172.21.15.116:0/2004282753 >> v2:172.21.15.173:3300/0 conn(0x7fd554156060 msgr2=0x7fd5540c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:06.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd559ec2700 1 --2- 172.21.15.116:0/2004282753 >> v2:172.21.15.173:3300/0 conn(0x7fd554156060 0x7fd5540c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd559ec2700 1 -- 172.21.15.116:0/2004282753 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5540b9900 con 0x7fd554155c80 2020-11-20T12:20:06.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd559ec2700 1 --2- 172.21.15.116:0/2004282753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd554155c80 0x7fd554062ed0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7fd554065d50 tx=0x7fd550006eb0).ready entity=mon.0 client_cookie=a3f6c9be2e95404c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:06.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd558ec0700 1 -- 172.21.15.116:0/2004282753 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550009720 con 0x7fd554155c80 2020-11-20T12:20:06.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd558ec0700 1 -- 172.21.15.116:0/2004282753 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd550009880 con 0x7fd554155c80 2020-11-20T12:20:06.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.080+0000 7fd558ec0700 1 -- 172.21.15.116:0/2004282753 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd550009c00 con 0x7fd554155c80 2020-11-20T12:20:06.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.084+0000 7fd55aec4700 1 -- 172.21.15.116:0/2004282753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd554155c80 msgr2=0x7fd554062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.084+0000 7fd55aec4700 1 --2- 172.21.15.116:0/2004282753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd554155c80 0x7fd554062ed0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7fd554065d50 tx=0x7fd550006eb0).stop 2020-11-20T12:20:06.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.084+0000 7fd55aec4700 1 -- 172.21.15.116:0/2004282753 shutdown_connections 2020-11-20T12:20:06.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.084+0000 7fd55aec4700 1 --2- 172.21.15.116:0/2004282753 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5540c12c0 0x7fd554060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.084+0000 7fd55aec4700 1 --2- 172.21.15.116:0/2004282753 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd554155c80 0x7fd554062ed0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.084+0000 7fd55aec4700 1 --2- 172.21.15.116:0/2004282753 >> v2:172.21.15.173:3300/0 conn(0x7fd554156060 0x7fd5540c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.084+0000 7fd55aec4700 1 -- 172.21.15.116:0/2004282753 >> 172.21.15.116:0/2004282753 conn(0x7fd554068350 msgr2=0x7fd5540627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:06.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.084+0000 7fd55aec4700 1 -- 172.21.15.116:0/2004282753 shutdown_connections 2020-11-20T12:20:06.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.084+0000 7fd55aec4700 1 -- 172.21.15.116:0/2004282753 wait complete. 2020-11-20T12:20:06.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd55aec4700 1 Processor -- start 2020-11-20T12:20:06.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd55aec4700 1 -- start start 2020-11-20T12:20:06.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd55aec4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5540c12c0 0x7fd5541662a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd55aec4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd554155c80 0x7fd554150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd55aec4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd554156060 0x7fd55415bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd55aec4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5540be4d0 con 0x7fd554155c80 2020-11-20T12:20:06.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd55aec4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd554169970 con 0x7fd5540c12c0 2020-11-20T12:20:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd55aec4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd554165620 con 0x7fd554156060 2020-11-20T12:20:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd5596c1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd554155c80 0x7fd554150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd5596c1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd554155c80 0x7fd554150e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48926/0 (socket says 172.21.15.116:48926) 2020-11-20T12:20:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd5596c1700 1 -- 172.21.15.116:0/3901722812 learned_addr learned my addr 172.21.15.116:0/3901722812 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:06.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd559ec2700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5540c12c0 0x7fd5541662a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd55a6c3700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd554156060 0x7fd55415bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd5596c1700 1 -- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd554156060 msgr2=0x7fd55415bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd5596c1700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd554156060 0x7fd55415bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd5596c1700 1 -- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5540c12c0 msgr2=0x7fd5541662a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd5596c1700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5540c12c0 0x7fd5541662a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd5596c1700 1 -- 172.21.15.116:0/3901722812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5541658c0 con 0x7fd554155c80 2020-11-20T12:20:06.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd559ec2700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5540c12c0 0x7fd5541662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:06.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.088+0000 7fd5596c1700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd554155c80 0x7fd554150e80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fd54400eba0 tx=0x7fd544006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:06.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.092+0000 7fd541ffb700 1 -- 172.21.15.116:0/3901722812 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd54400ca60 con 0x7fd554155c80 2020-11-20T12:20:06.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.092+0000 7fd541ffb700 1 -- 172.21.15.116:0/3901722812 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd54400cbc0 con 0x7fd554155c80 2020-11-20T12:20:06.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.092+0000 7fd55aec4700 1 -- 172.21.15.116:0/3901722812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd554165b60 con 0x7fd554155c80 2020-11-20T12:20:06.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.092+0000 7fd541ffb700 1 -- 172.21.15.116:0/3901722812 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5440123f0 con 0x7fd554155c80 2020-11-20T12:20:06.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.092+0000 7fd55aec4700 1 -- 172.21.15.116:0/3901722812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd554151370 con 0x7fd554155c80 2020-11-20T12:20:06.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.092+0000 7fd55aec4700 1 -- 172.21.15.116:0/3901722812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55405cf30 con 0x7fd554155c80 2020-11-20T12:20:06.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.092+0000 7fd541ffb700 1 -- 172.21.15.116:0/3901722812 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd5440127a0 con 0x7fd554155c80 2020-11-20T12:20:06.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.092+0000 7fd541ffb700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd52c0557c0 0x7fd52c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.092+0000 7fd541ffb700 1 -- 172.21.15.116:0/3901722812 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd54406eba0 con 0x7fd554155c80 2020-11-20T12:20:06.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.096+0000 7fd559ec2700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd52c0557c0 0x7fd52c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.096+0000 7fd559ec2700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd52c0557c0 0x7fd52c057c80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fd55000b1b0 tx=0x7fd550015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:06.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.100+0000 7fd541ffb700 1 -- 172.21.15.116:0/3901722812 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd544044a40 con 0x7fd554155c80 2020-11-20T12:20:06.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.260+0000 7fd55aec4700 1 -- 172.21.15.116:0/3901722812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd5540639c0 con 0x7fd554155c80 2020-11-20T12:20:06.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.260+0000 7fd541ffb700 1 -- 172.21.15.116:0/3901722812 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v86) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd544048060 con 0x7fd554155c80 2020-11-20T12:20:06.261 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:20:06.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 -- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd52c0557c0 msgr2=0x7fd52c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd52c0557c0 0x7fd52c057c80 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fd55000b1b0 tx=0x7fd550015000).stop 2020-11-20T12:20:06.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 -- 172.21.15.116:0/3901722812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd554155c80 msgr2=0x7fd554150e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd554155c80 0x7fd554150e80 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7fd54400eba0 tx=0x7fd544006df0).stop 2020-11-20T12:20:06.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 -- 172.21.15.116:0/3901722812 shutdown_connections 2020-11-20T12:20:06.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd52c0557c0 0x7fd52c057c80 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd554156060 0x7fd55415bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5540c12c0 0x7fd5541662a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 --2- 172.21.15.116:0/3901722812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd554155c80 0x7fd554150e80 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 -- 172.21.15.116:0/3901722812 >> 172.21.15.116:0/3901722812 conn(0x7fd554068350 msgr2=0x7fd5540bf420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:06.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 -- 172.21.15.116:0/3901722812 shutdown_connections 2020-11-20T12:20:06.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.264+0000 7fd55aec4700 1 -- 172.21.15.116:0/3901722812 wait complete. 2020-11-20T12:20:06.287 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:20:06.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.472+0000 7f648fa2e700 1 Processor -- start 2020-11-20T12:20:06.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.472+0000 7f648fa2e700 1 -- start start 2020-11-20T12:20:06.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.472+0000 7f648fa2e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f648814a1d0 0x7f64880be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.472+0000 7f648fa2e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64880a9c20 con 0x7f64880a9750 2020-11-20T12:20:06.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.472+0000 7f648fa2e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64880a9ec0 con 0x7f6488149df0 2020-11-20T12:20:06.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.472+0000 7f648fa2e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6488153670 con 0x7f648814a1d0 2020-11-20T12:20:06.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648dfcb700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f648814a1d0 0x7f64880be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648cfc9700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f64880a9750 0x7f64880c0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58368/0 (socket says 172.21.15.116:58368) 2020-11-20T12:20:06.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648cfc9700 1 -- 172.21.15.116:0/1331047492 learned_addr learned my addr 172.21.15.116:0/1331047492 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:06.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f6477fff700 1 -- 172.21.15.116:0/1331047492 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2734050492 0 0) 0x7f64880a9c20 con 0x7f64880a9750 2020-11-20T12:20:06.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f6477fff700 1 -- 172.21.15.116:0/1331047492 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6470003240 con 0x7f64880a9750 2020-11-20T12:20:06.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648dfcb700 1 -- 172.21.15.116:0/1331047492 >> v1:172.21.15.173:6789/0 conn(0x7f6488149df0 legacy=0x7f6488153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648dfcb700 1 -- 172.21.15.116:0/1331047492 >> v1:172.21.15.116:6789/0 conn(0x7f64880a9750 legacy=0x7f64880c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648dfcb700 1 -- 172.21.15.116:0/1331047492 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6488153910 con 0x7f648814a1d0 2020-11-20T12:20:06.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648dfcb700 1 --2- 172.21.15.116:0/1331047492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f648814a1d0 0x7f64880be240 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f6480009c90 tx=0x7f64800091d0).ready entity=mon.0 client_cookie=961dd8d9aa9aa28b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:06.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f6477fff700 1 -- 172.21.15.116:0/1331047492 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648000ba70 con 0x7f648814a1d0 2020-11-20T12:20:06.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f6477fff700 1 -- 172.21.15.116:0/1331047492 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648000bbd0 con 0x7f648814a1d0 2020-11-20T12:20:06.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f6477fff700 1 -- 172.21.15.116:0/1331047492 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480011400 con 0x7f648814a1d0 2020-11-20T12:20:06.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648fa2e700 1 -- 172.21.15.116:0/1331047492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f648814a1d0 msgr2=0x7f64880be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648fa2e700 1 --2- 172.21.15.116:0/1331047492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f648814a1d0 0x7f64880be240 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f6480009c90 tx=0x7f64800091d0).stop 2020-11-20T12:20:06.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648fa2e700 1 -- 172.21.15.116:0/1331047492 shutdown_connections 2020-11-20T12:20:06.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648fa2e700 1 --2- 172.21.15.116:0/1331047492 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f648814a1d0 0x7f64880be240 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648fa2e700 1 -- 172.21.15.116:0/1331047492 >> 172.21.15.116:0/1331047492 conn(0x7f648805ba80 msgr2=0x7f64880c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:06.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648fa2e700 1 -- 172.21.15.116:0/1331047492 shutdown_connections 2020-11-20T12:20:06.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.476+0000 7f648fa2e700 1 -- 172.21.15.116:0/1331047492 wait complete. 2020-11-20T12:20:06.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.480+0000 7f648fa2e700 1 Processor -- start 2020-11-20T12:20:06.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.480+0000 7f648fa2e700 1 -- start start 2020-11-20T12:20:06.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.480+0000 7f648fa2e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64880a9750 0x7f648814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.480+0000 7f648fa2e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6488149df0 0x7f648814c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.480+0000 7f648fa2e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f648814a1d0 0x7f648814cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.480+0000 7f648fa2e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648814d0a0 con 0x7f648814a1d0 2020-11-20T12:20:06.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.480+0000 7f648fa2e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648814d340 con 0x7f6488149df0 2020-11-20T12:20:06.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.480+0000 7f648fa2e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6488164920 con 0x7f64880a9750 2020-11-20T12:20:06.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648dfcb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f648814a1d0 0x7f648814cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648dfcb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f648814a1d0 0x7f648814cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48942/0 (socket says 172.21.15.116:48942) 2020-11-20T12:20:06.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648dfcb700 1 -- 172.21.15.116:0/2387361743 learned_addr learned my addr 172.21.15.116:0/2387361743 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:06.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648cfc9700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6488149df0 0x7f648814c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648d7ca700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64880a9750 0x7f648814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648dfcb700 1 -- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64880a9750 msgr2=0x7f648814acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648dfcb700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64880a9750 0x7f648814acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648dfcb700 1 -- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6488149df0 msgr2=0x7f648814c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648dfcb700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6488149df0 0x7f648814c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648dfcb700 1 -- 172.21.15.116:0/2387361743 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6488164bc0 con 0x7f648814a1d0 2020-11-20T12:20:06.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648dfcb700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f648814a1d0 0x7f648814cac0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f6480008000 tx=0x7f6480016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:06.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f64757fa700 1 -- 172.21.15.116:0/2387361743 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648001c5f0 con 0x7f648814a1d0 2020-11-20T12:20:06.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f64757fa700 1 -- 172.21.15.116:0/2387361743 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648001c750 con 0x7f648814a1d0 2020-11-20T12:20:06.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f64757fa700 1 -- 172.21.15.116:0/2387361743 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648001cad0 con 0x7f648814a1d0 2020-11-20T12:20:06.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648fa2e700 1 -- 172.21.15.116:0/2387361743 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64880a92b0 con 0x7f648814a1d0 2020-11-20T12:20:06.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648fa2e700 1 -- 172.21.15.116:0/2387361743 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648814b340 con 0x7f648814a1d0 2020-11-20T12:20:06.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.484+0000 7f648fa2e700 1 -- 172.21.15.116:0/2387361743 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6488067670 con 0x7f648814a1d0 2020-11-20T12:20:06.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.488+0000 7f64757fa700 1 -- 172.21.15.116:0/2387361743 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6480024030 con 0x7f648814a1d0 2020-11-20T12:20:06.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.488+0000 7f64757fa700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64600557c0 0x7f6460057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.488+0000 7f64757fa700 1 -- 172.21.15.116:0/2387361743 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6480070360 con 0x7f648814a1d0 2020-11-20T12:20:06.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.488+0000 7f648d7ca700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64600557c0 0x7f6460057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.488+0000 7f648d7ca700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64600557c0 0x7f6460057c80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f6484003510 tx=0x7f6484009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:06.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.492+0000 7f64757fa700 1 -- 172.21.15.116:0/2387361743 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6488067670 con 0x7f648814a1d0 2020-11-20T12:20:06.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.652+0000 7f648fa2e700 1 -- 172.21.15.116:0/2387361743 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6488067670 con 0x7f648814a1d0 2020-11-20T12:20:06.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.652+0000 7f64757fa700 1 -- 172.21.15.116:0/2387361743 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6488067670 con 0x7f648814a1d0 2020-11-20T12:20:06.651 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:06.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 -- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64600557c0 msgr2=0x7f6460057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64600557c0 0x7f6460057c80 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f6484003510 tx=0x7f6484009040).stop 2020-11-20T12:20:06.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 -- 172.21.15.116:0/2387361743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f648814a1d0 msgr2=0x7f648814cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f648814a1d0 0x7f648814cac0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f6480008000 tx=0x7f6480016000).stop 2020-11-20T12:20:06.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 -- 172.21.15.116:0/2387361743 shutdown_connections 2020-11-20T12:20:06.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f64600557c0 0x7f6460057c80 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f648814a1d0 0x7f648814cac0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f64880a9750 0x7f648814acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 --2- 172.21.15.116:0/2387361743 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6488149df0 0x7f648814c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 -- 172.21.15.116:0/2387361743 >> 172.21.15.116:0/2387361743 conn(0x7f648805ba80 msgr2=0x7f648814fdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:06.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 -- 172.21.15.116:0/2387361743 shutdown_connections 2020-11-20T12:20:06.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.656+0000 7f648fa2e700 1 -- 172.21.15.116:0/2387361743 wait complete. 2020-11-20T12:20:06.679 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:20:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.868+0000 7f05f682c700 1 Processor -- start 2020-11-20T12:20:06.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.868+0000 7f05f682c700 1 -- start start 2020-11-20T12:20:06.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05f682c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05f0063ad0 0x7f05f00ba640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05f682c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f05f01533c0 0x7f05f0153840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05f682c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f00c2140 con 0x7f05f014fdf0 2020-11-20T12:20:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05f682c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f0061180 con 0x7f05f01533c0 2020-11-20T12:20:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05f682c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f00c2370 con 0x7f05f0063ad0 2020-11-20T12:20:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05ef7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f05f01533c0 0x7f05f0153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05ef7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f05f01533c0 0x7f05f0153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48948/0 (socket says 172.21.15.116:48948) 2020-11-20T12:20:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05ef7fe700 1 -- 172.21.15.116:0/3821180365 learned_addr learned my addr 172.21.15.116:0/3821180365 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05effff700 1 --2- 172.21.15.116:0/3821180365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05f0063ad0 0x7f05f00ba640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05ef7fe700 1 -- 172.21.15.116:0/3821180365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05f0063ad0 msgr2=0x7f05f00ba640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05ef7fe700 1 --2- 172.21.15.116:0/3821180365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05f0063ad0 0x7f05f00ba640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05ef7fe700 1 -- 172.21.15.116:0/3821180365 >> v1:172.21.15.173:6789/0 conn(0x7f05f014fdf0 legacy=0x7f05f0150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05ef7fe700 1 -- 172.21.15.116:0/3821180365 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f00c2610 con 0x7f05f01533c0 2020-11-20T12:20:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05effff700 1 --2- 172.21.15.116:0/3821180365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05f0063ad0 0x7f05f00ba640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05ef7fe700 1 --2- 172.21.15.116:0/3821180365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f01533c0 0x7f05f0153840 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f05e4006fd0 tx=0x7f05e4006eb0).ready entity=mon.0 client_cookie=edf32489b3feb275 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05eeffd700 1 -- 172.21.15.116:0/3821180365 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4009890 con 0x7f05f01533c0 2020-11-20T12:20:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05eeffd700 1 -- 172.21.15.116:0/3821180365 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e40099f0 con 0x7f05f01533c0 2020-11-20T12:20:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05eeffd700 1 -- 172.21.15.116:0/3821180365 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4009d70 con 0x7f05f01533c0 2020-11-20T12:20:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05f682c700 1 -- 172.21.15.116:0/3821180365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f01533c0 msgr2=0x7f05f0153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05f682c700 1 --2- 172.21.15.116:0/3821180365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f01533c0 0x7f05f0153840 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f05e4006fd0 tx=0x7f05e4006eb0).stop 2020-11-20T12:20:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05f682c700 1 -- 172.21.15.116:0/3821180365 shutdown_connections 2020-11-20T12:20:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05f682c700 1 --2- 172.21.15.116:0/3821180365 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05f0063ad0 0x7f05f00ba640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05f682c700 1 --2- 172.21.15.116:0/3821180365 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f01533c0 0x7f05f0153840 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.876+0000 7f05f682c700 1 -- 172.21.15.116:0/3821180365 >> 172.21.15.116:0/3821180365 conn(0x7f05f00c5820 msgr2=0x7f05f00c17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.880+0000 7f05f682c700 1 -- 172.21.15.116:0/3821180365 shutdown_connections 2020-11-20T12:20:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.880+0000 7f05f682c700 1 -- 172.21.15.116:0/3821180365 wait complete. 2020-11-20T12:20:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.880+0000 7f05f682c700 1 Processor -- start 2020-11-20T12:20:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.880+0000 7f05f682c700 1 -- start start 2020-11-20T12:20:06.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05f682c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05f0063ad0 0x7f05f00aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05f682c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f014fdf0 0x7f05f00a7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05f682c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f0152270 0x7f05f00a5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05f682c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f0164f90 con 0x7f05f014fdf0 2020-11-20T12:20:06.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05f682c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f00c4990 con 0x7f05f0152270 2020-11-20T12:20:06.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05f682c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f00aaac0 con 0x7f05f0063ad0 2020-11-20T12:20:06.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05ef7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f014fdf0 0x7f05f00a7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05ef7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f014fdf0 0x7f05f00a7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48954/0 (socket says 172.21.15.116:48954) 2020-11-20T12:20:06.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05ef7fe700 1 -- 172.21.15.116:0/1704878263 learned_addr learned my addr 172.21.15.116:0/1704878263 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:06.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05effff700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05f0063ad0 0x7f05f00aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05f4dc9700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f0152270 0x7f05f00a5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05ef7fe700 1 -- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05f0063ad0 msgr2=0x7f05f00aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05ef7fe700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05f0063ad0 0x7f05f00aa5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05ef7fe700 1 -- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f0152270 msgr2=0x7f05f00a5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05ef7fe700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f0152270 0x7f05f00a5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05ef7fe700 1 -- 172.21.15.116:0/1704878263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f0152780 con 0x7f05f014fdf0 2020-11-20T12:20:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05ef7fe700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f014fdf0 0x7f05f00a7d00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7f05e400b1b0 tx=0x7f05e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05cffff700 1 -- 172.21.15.116:0/1704878263 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e4011c80 con 0x7f05f014fdf0 2020-11-20T12:20:06.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05cffff700 1 -- 172.21.15.116:0/1704878263 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e4011de0 con 0x7f05f014fdf0 2020-11-20T12:20:06.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05f682c700 1 -- 172.21.15.116:0/1704878263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f0152a20 con 0x7f05f014fdf0 2020-11-20T12:20:06.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05cffff700 1 -- 172.21.15.116:0/1704878263 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e401e6b0 con 0x7f05f014fdf0 2020-11-20T12:20:06.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.884+0000 7f05f682c700 1 -- 172.21.15.116:0/1704878263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f0152f60 con 0x7f05f014fdf0 2020-11-20T12:20:06.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.888+0000 7f05f682c700 1 -- 172.21.15.116:0/1704878263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f0069950 con 0x7f05f014fdf0 2020-11-20T12:20:06.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.888+0000 7f05cffff700 1 -- 172.21.15.116:0/1704878263 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f05e4025030 con 0x7f05f014fdf0 2020-11-20T12:20:06.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.888+0000 7f05cffff700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05c80557c0 0x7f05c8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:06.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.888+0000 7f05cffff700 1 -- 172.21.15.116:0/1704878263 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f05e406fd00 con 0x7f05f014fdf0 2020-11-20T12:20:06.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.888+0000 7f05effff700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05c80557c0 0x7f05c8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:06.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.892+0000 7f05effff700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05c80557c0 0x7f05c8057c80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f05f014d790 tx=0x7f05e0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:06.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:06.896+0000 7f05cffff700 1 -- 172.21.15.116:0/1704878263 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f05e4045b40 con 0x7f05f014fdf0 2020-11-20T12:20:07.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.056+0000 7f05f682c700 1 -- 172.21.15.116:0/1704878263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05f01650d0 con 0x7f05f014fdf0 2020-11-20T12:20:07.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.056+0000 7f05cffff700 1 -- 172.21.15.116:0/1704878263 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+10 (secure 0 0 0) 0x7f05e4023070 con 0x7f05f014fdf0 2020-11-20T12:20:07.054 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:07.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 -- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05c80557c0 msgr2=0x7f05c8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05c80557c0 0x7f05c8057c80 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f05f014d790 tx=0x7f05e0008040).stop 2020-11-20T12:20:07.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 -- 172.21.15.116:0/1704878263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f014fdf0 msgr2=0x7f05f00a7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f014fdf0 0x7f05f00a7d00 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7f05e400b1b0 tx=0x7f05e4015000).stop 2020-11-20T12:20:07.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 -- 172.21.15.116:0/1704878263 shutdown_connections 2020-11-20T12:20:07.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f05c80557c0 0x7f05c8057c80 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f05f0152270 0x7f05f00a5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f05f0063ad0 0x7f05f00aa5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 --2- 172.21.15.116:0/1704878263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f05f014fdf0 0x7f05f00a7d00 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 -- 172.21.15.116:0/1704878263 >> 172.21.15.116:0/1704878263 conn(0x7f05f00c5820 msgr2=0x7f05f0159fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:07.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 -- 172.21.15.116:0/1704878263 shutdown_connections 2020-11-20T12:20:07.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.060+0000 7f05f682c700 1 -- 172.21.15.116:0/1704878263 wait complete. 2020-11-20T12:20:07.081 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:20:07.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.272+0000 7fa9150eb700 1 Processor -- start 2020-11-20T12:20:07.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.272+0000 7fa9150eb700 1 -- start start 2020-11-20T12:20:07.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa9150eb700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa910156440 0x7fa9100aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa9150eb700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9100ace40 con 0x7fa9100a8400 2020-11-20T12:20:07.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa9150eb700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9101526c0 con 0x7fa9100aa7d0 2020-11-20T12:20:07.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa9150eb700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa910152960 con 0x7fa910156440 2020-11-20T12:20:07.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa90f7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa9100a8400 0x7fa9100ad540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58396/0 (socket says 172.21.15.116:58396) 2020-11-20T12:20:07.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa90f7fe700 1 -- 172.21.15.116:0/2496240747 learned_addr learned my addr 172.21.15.116:0/2496240747 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:07.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa9148ea700 1 --2- 172.21.15.116:0/2496240747 >> v2:172.21.15.173:3300/0 conn(0x7fa910156440 0x7fa9100aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa90effd700 1 -- 172.21.15.116:0/2496240747 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949407020 0 0) 0x7fa9100ace40 con 0x7fa9100a8400 2020-11-20T12:20:07.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa90effd700 1 -- 172.21.15.116:0/2496240747 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8fc003240 con 0x7fa9100a8400 2020-11-20T12:20:07.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa90effd700 1 -- 172.21.15.116:0/2496240747 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282092028 0 0) 0x7fa9101526c0 con 0x7fa9100aa7d0 2020-11-20T12:20:07.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa90effd700 1 -- 172.21.15.116:0/2496240747 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9100ace40 con 0x7fa9100aa7d0 2020-11-20T12:20:07.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa90effd700 1 -- 172.21.15.116:0/2496240747 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 576193084 0 0) 0x7fa8fc003240 con 0x7fa9100a8400 2020-11-20T12:20:07.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa9148ea700 1 -- 172.21.15.116:0/2496240747 >> v1:172.21.15.173:6789/0 conn(0x7fa9100aa7d0 legacy=0x7fa910151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa9148ea700 1 -- 172.21.15.116:0/2496240747 >> v1:172.21.15.116:6789/0 conn(0x7fa9100a8400 legacy=0x7fa9100ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.276+0000 7fa9148ea700 1 -- 172.21.15.116:0/2496240747 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa910152c00 con 0x7fa910156440 2020-11-20T12:20:07.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.280+0000 7fa9148ea700 1 --2- 172.21.15.116:0/2496240747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa910156440 0x7fa9100aabb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fa908009c90 tx=0x7fa9080091d0).ready entity=mon.1 client_cookie=1bb7788865b70f0a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:07.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.280+0000 7fa90effd700 1 -- 172.21.15.116:0/2496240747 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9040014e0 con 0x7fa910156440 2020-11-20T12:20:07.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.280+0000 7fa90effd700 1 -- 172.21.15.116:0/2496240747 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90800b6b0 con 0x7fa910156440 2020-11-20T12:20:07.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.280+0000 7fa90effd700 1 -- 172.21.15.116:0/2496240747 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90800ba30 con 0x7fa910156440 2020-11-20T12:20:07.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.280+0000 7fa9150eb700 1 -- 172.21.15.116:0/2496240747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa910156440 msgr2=0x7fa9100aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.280+0000 7fa9150eb700 1 --2- 172.21.15.116:0/2496240747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa910156440 0x7fa9100aabb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7fa908009c90 tx=0x7fa9080091d0).stop 2020-11-20T12:20:07.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.280+0000 7fa9150eb700 1 -- 172.21.15.116:0/2496240747 shutdown_connections 2020-11-20T12:20:07.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.280+0000 7fa9150eb700 1 --2- 172.21.15.116:0/2496240747 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa910156440 0x7fa9100aabb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.280+0000 7fa9150eb700 1 -- 172.21.15.116:0/2496240747 >> 172.21.15.116:0/2496240747 conn(0x7fa9100678a0 msgr2=0x7fa910067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:07.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.280+0000 7fa9150eb700 1 -- 172.21.15.116:0/2496240747 shutdown_connections 2020-11-20T12:20:07.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.280+0000 7fa9150eb700 1 -- 172.21.15.116:0/2496240747 wait complete. 2020-11-20T12:20:07.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.284+0000 7fa9150eb700 1 Processor -- start 2020-11-20T12:20:07.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.284+0000 7fa9150eb700 1 -- start start 2020-11-20T12:20:07.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.284+0000 7fa9150eb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9100a8400 0x7fa910149880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.284+0000 7fa9150eb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9100aa7d0 0x7fa91014cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.284+0000 7fa9150eb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa910156440 0x7fa910154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.284+0000 7fa9150eb700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9101613a0 con 0x7fa9100a8400 2020-11-20T12:20:07.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa9150eb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100a0900 con 0x7fa910156440 2020-11-20T12:20:07.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa9150eb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9100a0ba0 con 0x7fa9100aa7d0 2020-11-20T12:20:07.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa90ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9100a8400 0x7fa910149880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa90ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9100a8400 0x7fa910149880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48966/0 (socket says 172.21.15.116:48966) 2020-11-20T12:20:07.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa90ffff700 1 -- 172.21.15.116:0/1865009769 learned_addr learned my addr 172.21.15.116:0/1865009769 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:07.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa9148ea700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa910156440 0x7fa910154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa90f7fe700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9100aa7d0 0x7fa91014cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa90ffff700 1 -- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9100aa7d0 msgr2=0x7fa91014cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa90ffff700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9100aa7d0 0x7fa91014cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa90ffff700 1 -- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa910156440 msgr2=0x7fa910154070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa90ffff700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa910156440 0x7fa910154070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa90ffff700 1 -- 172.21.15.116:0/1865009769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9100a0e40 con 0x7fa9100a8400 2020-11-20T12:20:07.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa90ffff700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9100a8400 0x7fa910149880 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fa900007fd0 tx=0x7fa900007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:07.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa8effff700 1 -- 172.21.15.116:0/1865009769 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000e560 con 0x7fa9100a8400 2020-11-20T12:20:07.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa8effff700 1 -- 172.21.15.116:0/1865009769 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa90000e6e0 con 0x7fa9100a8400 2020-11-20T12:20:07.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa9150eb700 1 -- 172.21.15.116:0/1865009769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9100a10e0 con 0x7fa9100a8400 2020-11-20T12:20:07.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa8effff700 1 -- 172.21.15.116:0/1865009769 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa90000eaa0 con 0x7fa9100a8400 2020-11-20T12:20:07.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.288+0000 7fa9150eb700 1 -- 172.21.15.116:0/1865009769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa910149d70 con 0x7fa9100a8400 2020-11-20T12:20:07.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.292+0000 7fa9150eb700 1 -- 172.21.15.116:0/1865009769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa91005cf30 con 0x7fa9100a8400 2020-11-20T12:20:07.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.292+0000 7fa8effff700 1 -- 172.21.15.116:0/1865009769 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa900019030 con 0x7fa9100a8400 2020-11-20T12:20:07.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.292+0000 7fa8effff700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8e80557c0 0x7fa8e8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.292+0000 7fa8effff700 1 -- 172.21.15.116:0/1865009769 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa90006dc50 con 0x7fa9100a8400 2020-11-20T12:20:07.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.296+0000 7fa90f7fe700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8e80557c0 0x7fa8e8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.296+0000 7fa90f7fe700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8e80557c0 0x7fa8e8057c80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fa910164550 tx=0x7fa9040019a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:07.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.300+0000 7fa8effff700 1 -- 172.21.15.116:0/1865009769 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa900043af0 con 0x7fa9100a8400 2020-11-20T12:20:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.456+0000 7fa9150eb700 1 -- 172.21.15.116:0/1865009769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa9100a8d90 con 0x7fa9100a8400 2020-11-20T12:20:07.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.456+0000 7fa8effff700 1 -- 172.21.15.116:0/1865009769 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+10 (secure 0 0 0) 0x7fa900047110 con 0x7fa9100a8400 2020-11-20T12:20:07.456 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 -- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8e80557c0 msgr2=0x7fa8e8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8e80557c0 0x7fa8e8057c80 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7fa910164550 tx=0x7fa9040019a0).stop 2020-11-20T12:20:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 -- 172.21.15.116:0/1865009769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9100a8400 msgr2=0x7fa910149880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9100a8400 0x7fa910149880 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fa900007fd0 tx=0x7fa900007ac0).stop 2020-11-20T12:20:07.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 -- 172.21.15.116:0/1865009769 shutdown_connections 2020-11-20T12:20:07.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa8e80557c0 0x7fa8e8057c80 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa910156440 0x7fa910154070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9100a8400 0x7fa910149880 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 --2- 172.21.15.116:0/1865009769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9100aa7d0 0x7fa91014cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 -- 172.21.15.116:0/1865009769 >> 172.21.15.116:0/1865009769 conn(0x7fa9100678a0 msgr2=0x7fa91014bab0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 -- 172.21.15.116:0/1865009769 shutdown_connections 2020-11-20T12:20:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.460+0000 7fa9150eb700 1 -- 172.21.15.116:0/1865009769 wait complete. 2020-11-20T12:20:07.484 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:20:07.485 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:07.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.668+0000 7fd9d1b14700 1 Processor -- start 2020-11-20T12:20:07.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.668+0000 7fd9d1b14700 1 -- start start 2020-11-20T12:20:07.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.668+0000 7fd9d1b14700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd9cc151e90 0x7fd9cc0c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.668+0000 7fd9d1b14700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc0bf020 0x7fd9cc0a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.668+0000 7fd9d1b14700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9cc0c3600 con 0x7fd9cc0c3c90 2020-11-20T12:20:07.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.668+0000 7fd9d1b14700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9cc0c38a0 con 0x7fd9cc151e90 2020-11-20T12:20:07.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.668+0000 7fd9d1b14700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc0c72a0 con 0x7fd9cc0bf020 2020-11-20T12:20:07.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.668+0000 7fd9cb7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd9cc0c3c90 0x7fd9cc0c79e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58408/0 (socket says 172.21.15.116:58408) 2020-11-20T12:20:07.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.668+0000 7fd9cb7fe700 1 -- 172.21.15.116:0/1924105771 learned_addr learned my addr 172.21.15.116:0/1924105771 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:07.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9cbfff700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc0bf020 0x7fd9cc0a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9c2ffd700 1 --2- 172.21.15.116:0/1924105771 >> v2:172.21.15.173:3300/0 conn(0x7fd9cc151e90 0x7fd9cc0c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 909526359 0 0) 0x7fd9cc0c3600 con 0x7fd9cc0c3c90 2020-11-20T12:20:07.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9ac003240 con 0x7fd9cc0c3c90 2020-11-20T12:20:07.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 173542929 0 0) 0x7fd9ac003240 con 0x7fd9cc0c3c90 2020-11-20T12:20:07.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc0bf020 msgr2=0x7fd9cc0a5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:07.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc0bf020 0x7fd9cc0a5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 >> v2:172.21.15.173:3300/0 conn(0x7fd9cc151e90 msgr2=0x7fd9cc0c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:07.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 --2- 172.21.15.116:0/1924105771 >> v2:172.21.15.173:3300/0 conn(0x7fd9cc151e90 0x7fd9cc0c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0c7540 con 0x7fd9cc0c3c90 2020-11-20T12:20:07.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fd9bc001660 con 0x7fd9cc0c3c90 2020-11-20T12:20:07.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9cbfff700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc0bf020 0x7fd9cc0a5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:07.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 >> v1:172.21.15.116:6789/0 conn(0x7fd9cc0c3c90 legacy=0x7fd9cc0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc0bf020 0x7fd9ac003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9ac003c20 0x7fd9ac0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9ac0049d0 0x7fd9ac004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc0c38a0 con 0x7fd9ac0049d0 2020-11-20T12:20:07.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc0c72a0 con 0x7fd9ac003c20 2020-11-20T12:20:07.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ac0041c0 con 0x7fd9cc0bf020 2020-11-20T12:20:07.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9cbfff700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9ac0049d0 0x7fd9ac004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9cb7fe700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc0bf020 0x7fd9ac003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9c2ffd700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9ac003c20 0x7fd9ac0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9cbfff700 1 -- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc0bf020 msgr2=0x7fd9ac003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:07.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9cbfff700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc0bf020 0x7fd9ac003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9cbfff700 1 -- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9ac003c20 msgr2=0x7fd9ac0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:07.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9cbfff700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9ac003c20 0x7fd9ac0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9cbfff700 1 -- 172.21.15.116:0/1924105771 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac003240 con 0x7fd9ac0049d0 2020-11-20T12:20:07.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9cbfff700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9ac0049d0 0x7fd9ac004e50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7fd9c400e2d0 tx=0x7fd9c40091d0).ready entity=mon.0 client_cookie=7bf9c5692639d469 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:07.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4009800 con 0x7fd9ac0049d0 2020-11-20T12:20:07.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c400ee40 con 0x7fd9ac0049d0 2020-11-20T12:20:07.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.672+0000 7fd9caffd700 1 -- 172.21.15.116:0/1924105771 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c4012600 con 0x7fd9ac0049d0 2020-11-20T12:20:07.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.676+0000 7fd9d1b14700 1 -- 172.21.15.116:0/1924105771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9ac0049d0 msgr2=0x7fd9ac004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.676+0000 7fd9d1b14700 1 --2- 172.21.15.116:0/1924105771 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9ac0049d0 0x7fd9ac004e50 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7fd9c400e2d0 tx=0x7fd9c40091d0).stop 2020-11-20T12:20:07.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.676+0000 7fd9cbfff700 1 -- 172.21.15.116:0/1924105771 reap_dead start 2020-11-20T12:20:07.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.676+0000 7fd9d1b14700 1 -- 172.21.15.116:0/1924105771 shutdown_connections 2020-11-20T12:20:07.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.676+0000 7fd9d1b14700 1 -- 172.21.15.116:0/1924105771 >> 172.21.15.116:0/1924105771 conn(0x7fd9cc068080 msgr2=0x7fd9cc0c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:07.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.676+0000 7fd9d1b14700 1 -- 172.21.15.116:0/1924105771 shutdown_connections 2020-11-20T12:20:07.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.676+0000 7fd9d1b14700 1 -- 172.21.15.116:0/1924105771 wait complete. 2020-11-20T12:20:07.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.676+0000 7fd9d1b14700 1 Processor -- start 2020-11-20T12:20:07.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.676+0000 7fd9d1b14700 1 -- start start 2020-11-20T12:20:07.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9d1b14700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9cc150260 0x7fd9cc0a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9d1b14700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9cc150640 0x7fd9cc092710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9d1b14700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc161900 0x7fd9cc0c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9d1b14700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc0a90c0 con 0x7fd9cc150260 2020-11-20T12:20:07.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9d1b14700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc092c00 con 0x7fd9cc150640 2020-11-20T12:20:07.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9d1b14700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9cc092e20 con 0x7fd9cc161900 2020-11-20T12:20:07.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9cb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9cc150260 0x7fd9cc0a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9cb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9cc150260 0x7fd9cc0a8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:48986/0 (socket says 172.21.15.116:48986) 2020-11-20T12:20:07.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9cb7fe700 1 -- 172.21.15.116:0/2231122626 learned_addr learned my addr 172.21.15.116:0/2231122626 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:07.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9c2ffd700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9cc150640 0x7fd9cc092710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9cbfff700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc161900 0x7fd9cc0c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9cb7fe700 1 -- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc161900 msgr2=0x7fd9cc0c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9cb7fe700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc161900 0x7fd9cc0c1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9cb7fe700 1 -- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9cc150640 msgr2=0x7fd9cc092710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9cb7fe700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9cc150640 0x7fd9cc092710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9cb7fe700 1 -- 172.21.15.116:0/2231122626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9cc0930c0 con 0x7fd9cc150260 2020-11-20T12:20:07.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9cbfff700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc161900 0x7fd9cc0c1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:07.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9cb7fe700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9cc150260 0x7fd9cc0a8bd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7fd9bc00fc00 tx=0x7fd9bc009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:07.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9c3fff700 1 -- 172.21.15.116:0/2231122626 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc0070d0 con 0x7fd9cc150260 2020-11-20T12:20:07.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9c3fff700 1 -- 172.21.15.116:0/2231122626 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9bc007230 con 0x7fd9cc150260 2020-11-20T12:20:07.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9d1b14700 1 -- 172.21.15.116:0/2231122626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9cc093360 con 0x7fd9cc150260 2020-11-20T12:20:07.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9c3fff700 1 -- 172.21.15.116:0/2231122626 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc00db60 con 0x7fd9cc150260 2020-11-20T12:20:07.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.680+0000 7fd9d1b14700 1 -- 172.21.15.116:0/2231122626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9cc161460 con 0x7fd9cc150260 2020-11-20T12:20:07.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.684+0000 7fd9d1b14700 1 -- 172.21.15.116:0/2231122626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9cc0c4dc0 con 0x7fd9cc150260 2020-11-20T12:20:07.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.684+0000 7fd9c3fff700 1 -- 172.21.15.116:0/2231122626 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd9bc01a030 con 0x7fd9cc150260 2020-11-20T12:20:07.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.684+0000 7fd9c3fff700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd9a80557f0 0x7fd9a8057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:07.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.684+0000 7fd9c3fff700 1 -- 172.21.15.116:0/2231122626 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd9bc06dd60 con 0x7fd9cc150260 2020-11-20T12:20:07.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.684+0000 7fd9c2ffd700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd9a80557f0 0x7fd9a8057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:07.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.684+0000 7fd9c2ffd700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd9a80557f0 0x7fd9a8057cb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fd9cc0aaec0 tx=0x7fd9b4006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:07.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.692+0000 7fd9c3fff700 1 -- 172.21.15.116:0/2231122626 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd9cc0c4dc0 con 0x7fd9cc150260 2020-11-20T12:20:07.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.892+0000 7fd9d1b14700 1 -- 172.21.15.116:0/2231122626 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd9cc0c4dc0 con 0x7fd9cc150260 2020-11-20T12:20:07.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.896+0000 7fd9c3fff700 1 -- 172.21.15.116:0/2231122626 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2848 (secure 0 0 0) 0x7fd9cc0c4dc0 con 0x7fd9cc150260 2020-11-20T12:20:07.892 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:07.893 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/313210621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":313210621},{"type":"v1","addr":"172.21.15.173:6835","nonce":313210621}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/2140340557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2140340557},{"type":"v1","addr":"172.21.15.173:6837","nonce":2140340557}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3544207168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3544207168},{"type":"v1","addr":"172.21.15.116:6833","nonce":3544207168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:57.303415+0000","modified":"2020-11-20T12:19:59.322603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/657163487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":657163487},{"type":"v1","addr":"172.21.15.116:6835","nonce":657163487}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-20T12:20:07.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 -- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd9a80557f0 msgr2=0x7fd9a8057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd9a80557f0 0x7fd9a8057cb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fd9cc0aaec0 tx=0x7fd9b4006df0).stop 2020-11-20T12:20:07.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 -- 172.21.15.116:0/2231122626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9cc150260 msgr2=0x7fd9cc0a8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:07.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9cc150260 0x7fd9cc0a8bd0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7fd9bc00fc00 tx=0x7fd9bc009040).stop 2020-11-20T12:20:07.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 -- 172.21.15.116:0/2231122626 shutdown_connections 2020-11-20T12:20:07.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd9a80557f0 0x7fd9a8057cb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9cc161900 0x7fd9cc0c1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9cc150260 0x7fd9cc0a8bd0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 --2- 172.21.15.116:0/2231122626 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd9cc150640 0x7fd9cc092710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:07.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 -- 172.21.15.116:0/2231122626 >> 172.21.15.116:0/2231122626 conn(0x7fd9cc068080 msgr2=0x7fd9cc061500 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:07.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 -- 172.21.15.116:0/2231122626 shutdown_connections 2020-11-20T12:20:07.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:07.900+0000 7fd9d1b14700 1 -- 172.21.15.116:0/2231122626 wait complete. 2020-11-20T12:20:07.902 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 46 2020-11-20T12:20:08.324 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:19:57.303415+0000', 'modified': '2020-11-20T12:19:59.322603+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6861}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6861': {'gid': 6861, 'name': 'a', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/657163487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 657163487}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 657163487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:20:08.324 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:20:08.325 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:20:08.517 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":6861,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.194108394000001,"mdsmap_epoch":46,"osdmap_epoch":86,"osdmap_epoch_barrier":86,"uptime":17.760329585000001} 2020-11-20T12:20:08.518 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 6861, "mdsmap_epoch": 46, "osdmap_epoch": 86, "osdmap_epoch_barrier": 86, "rank_uptime": 11.194108394, "state": "up:active", "uptime": 17.760329585, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:20:08.519 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:20:08.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:08.521 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-20T12:20:08.540 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:20:08.541 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:08.655 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:08.656 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:20:08.680 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:08.680 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:20:08.688 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:20:08.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:08.689 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-20T12:20:08.716 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:20:08.717 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:20:08.717 INFO:tasks.cephfs.mount:self.client.name = client.2 2020-11-20T12:20:08.718 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2020-11-20T12:20:08.718 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:20:08.718 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:20:08.719 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:20:08.719 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:20:08.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:08.720 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:20:08.741 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:20:08.741 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:20:08.741 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:20:08.741 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:08.742 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:20:08.742 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:08.742 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:08.742 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:08.742 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:08.743 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:08.743 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:08.743 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:08.743 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:20:08.743 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:08.744 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:20:08.744 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:08.744 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:20:08.744 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:08.744 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:20:08.745 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:08.745 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:20:08.745 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:08.745 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:20:08.745 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:08.746 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:20:08.746 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:08.746 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:20:08.746 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:08.746 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:08.747 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:20:08.747 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:20:08.747 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:08.747 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:08.748 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:20:08.748 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:20:08.748 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:08.749 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:08.749 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:20:08.825 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:20:08.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:08.829 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:20:08.845 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:20:08.846 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:20:08.846 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:20:08.846 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:20:08.847 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:20:08.847 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:20:08.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:08.850 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:20:08.850 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:20:08.851 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:20:08.852 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:20:08.853 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:20:08.853 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:20:08.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:08.962 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:20:08.980 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:20:08.980 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:20:08.981 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:20:08.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:08.982 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2020-11-20T12:20:09.059 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:09.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.240+0000 7f2b265de700 1 Processor -- start 2020-11-20T12:20:09.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.240+0000 7f2b265de700 1 -- start start 2020-11-20T12:20:09.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b265de700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2b20150680 0x7f2b20066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b265de700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b201573d0 con 0x7f2b200b9700 2020-11-20T12:20:09.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b265de700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b20066af0 con 0x7f2b20156f20 2020-11-20T12:20:09.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b265de700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b20150a60 con 0x7f2b20150680 2020-11-20T12:20:09.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b24b7b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2b200b9700 0x7f2b200b9b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58432/0 (socket says 172.21.15.116:58432) 2020-11-20T12:20:09.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b24b7b700 1 -- 172.21.15.116:0/184478096 learned_addr learned my addr 172.21.15.116:0/184478096 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:09.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1ffff700 1 --2- 172.21.15.116:0/184478096 >> v2:172.21.15.173:3300/0 conn(0x7f2b20150680 0x7f2b20066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682397456 0 0) 0x7f2b201573d0 con 0x7f2b200b9700 2020-11-20T12:20:09.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b00003240 con 0x7f2b200b9700 2020-11-20T12:20:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 811549091 0 0) 0x7f2b20066af0 con 0x7f2b20156f20 2020-11-20T12:20:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b201573d0 con 0x7f2b20156f20 2020-11-20T12:20:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 958807426 0 0) 0x7f2b00003240 con 0x7f2b200b9700 2020-11-20T12:20:09.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 >> v2:172.21.15.173:3300/0 conn(0x7f2b20150680 msgr2=0x7f2b20066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 --2- 172.21.15.116:0/184478096 >> v2:172.21.15.173:3300/0 conn(0x7f2b20150680 0x7f2b20066450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 >> v1:172.21.15.173:6789/0 conn(0x7f2b20156f20 legacy=0x7f2b200bf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b20066d90 con 0x7f2b200b9700 2020-11-20T12:20:09.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f2b18001860 con 0x7f2b200b9700 2020-11-20T12:20:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 >> v1:172.21.15.116:6789/0 conn(0x7f2b200b9700 legacy=0x7f2b200b9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b00003910 0x7f2b00003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b000044e0 0x7f2b00004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b00004e50 0x7f2b0000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b20150a60 con 0x7f2b000044e0 2020-11-20T12:20:09.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0000bc20 con 0x7f2b00003910 2020-11-20T12:20:09.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0000bd90 con 0x7f2b00004e50 2020-11-20T12:20:09.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.244+0000 7f2b1ffff700 1 --2- 172.21.15.116:0/184478096 >> v2:172.21.15.173:3300/0 conn(0x7f2b20150680 0x7f2b20066450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:09.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b17fff700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b000044e0 0x7f2b00004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b24b7b700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b00004e50 0x7f2b0000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b1ffff700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b00003910 0x7f2b00003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b17fff700 1 -- 172.21.15.116:0/184478096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b00004e50 msgr2=0x7f2b0000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:09.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b17fff700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b00004e50 0x7f2b0000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b17fff700 1 -- 172.21.15.116:0/184478096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b00003910 msgr2=0x7f2b00003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:09.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b17fff700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b00003910 0x7f2b00003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b17fff700 1 -- 172.21.15.116:0/184478096 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b00003240 con 0x7f2b000044e0 2020-11-20T12:20:09.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b1ffff700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b00003910 0x7f2b00003d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:09.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b24b7b700 1 -- 172.21.15.116:0/184478096 reap_dead start 2020-11-20T12:20:09.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b17fff700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b000044e0 0x7f2b00004960 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f2b08001a80 tx=0x7f2b08007b90).ready entity=mon.0 client_cookie=4b8644c50a020403 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:09.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08007020 con 0x7f2b000044e0 2020-11-20T12:20:09.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b08007180 con 0x7f2b000044e0 2020-11-20T12:20:09.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b1f7fe700 1 -- 172.21.15.116:0/184478096 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b0800c890 con 0x7f2b000044e0 2020-11-20T12:20:09.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b265de700 1 -- 172.21.15.116:0/184478096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b000044e0 msgr2=0x7f2b00004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b265de700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b000044e0 0x7f2b00004960 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f2b08001a80 tx=0x7f2b08007b90).stop 2020-11-20T12:20:09.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b265de700 1 -- 172.21.15.116:0/184478096 shutdown_connections 2020-11-20T12:20:09.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b265de700 1 --2- 172.21.15.116:0/184478096 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b000044e0 0x7f2b00004960 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b265de700 1 -- 172.21.15.116:0/184478096 >> 172.21.15.116:0/184478096 conn(0x7f2b200c4f80 msgr2=0x7f2b200c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:09.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b265de700 1 -- 172.21.15.116:0/184478096 shutdown_connections 2020-11-20T12:20:09.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.248+0000 7f2b265de700 1 -- 172.21.15.116:0/184478096 wait complete. 2020-11-20T12:20:09.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.252+0000 7f2b265de700 1 Processor -- start 2020-11-20T12:20:09.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.252+0000 7f2b265de700 1 -- start start 2020-11-20T12:20:09.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b265de700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b20143640 0x7f2b200a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b265de700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b200a4490 0x7f2b20063190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b265de700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b20143a20 0x7f2b20063680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b265de700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200a4920 con 0x7f2b20143a20 2020-11-20T12:20:09.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b265de700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200a4bc0 con 0x7f2b20143640 2020-11-20T12:20:09.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b265de700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b200a4e60 con 0x7f2b200a4490 2020-11-20T12:20:09.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b24b7b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b20143a20 0x7f2b20063680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b1ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b20143640 0x7f2b200a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b24b7b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b20143a20 0x7f2b20063680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49010/0 (socket says 172.21.15.116:49010) 2020-11-20T12:20:09.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b24b7b700 1 -- 172.21.15.116:0/3409198500 learned_addr learned my addr 172.21.15.116:0/3409198500 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:09.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b17fff700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b200a4490 0x7f2b20063190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b24b7b700 1 -- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b200a4490 msgr2=0x7f2b20063190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b24b7b700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b200a4490 0x7f2b20063190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b24b7b700 1 -- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b20143640 msgr2=0x7f2b200a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b24b7b700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b20143640 0x7f2b200a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b24b7b700 1 -- 172.21.15.116:0/3409198500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b200a5100 con 0x7f2b20143a20 2020-11-20T12:20:09.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b1ffff700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b20143640 0x7f2b200a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:09.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b24b7b700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b20143a20 0x7f2b20063680 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f2b200bea40 tx=0x7f2b18007a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:09.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b1cff9700 1 -- 172.21.15.116:0/3409198500 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800f960 con 0x7f2b20143a20 2020-11-20T12:20:09.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b1cff9700 1 -- 172.21.15.116:0/3409198500 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1800fac0 con 0x7f2b20143a20 2020-11-20T12:20:09.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b265de700 1 -- 172.21.15.116:0/3409198500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b201659a0 con 0x7f2b20143a20 2020-11-20T12:20:09.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b1cff9700 1 -- 172.21.15.116:0/3409198500 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800fe40 con 0x7f2b20143a20 2020-11-20T12:20:09.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.256+0000 7f2b265de700 1 -- 172.21.15.116:0/3409198500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b20151b70 con 0x7f2b20143a20 2020-11-20T12:20:09.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.260+0000 7f2b265de700 1 -- 172.21.15.116:0/3409198500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b20165b30 con 0x7f2b20143a20 2020-11-20T12:20:09.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.260+0000 7f2b1cff9700 1 -- 172.21.15.116:0/3409198500 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2b1801b030 con 0x7f2b20143a20 2020-11-20T12:20:09.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.260+0000 7f2b1cff9700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2afc0557f0 0x7f2afc057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.260+0000 7f2b1cff9700 1 -- 172.21.15.116:0/3409198500 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2b1806de40 con 0x7f2b20143a20 2020-11-20T12:20:09.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.264+0000 7f2b1ffff700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2afc0557f0 0x7f2afc057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.264+0000 7f2b1ffff700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2afc0557f0 0x7f2afc057cb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f2b10006d00 tx=0x7f2b10008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:09.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.268+0000 7f2b1cff9700 1 -- 172.21.15.116:0/3409198500 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2b18043bd0 con 0x7f2b20143a20 2020-11-20T12:20:09.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.424+0000 7f2b265de700 1 -- 172.21.15.116:0/3409198500 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b2005d2a0 con 0x7f2b20143a20 2020-11-20T12:20:09.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.428+0000 7f2b1cff9700 1 -- 172.21.15.116:0/3409198500 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f2b18019030 con 0x7f2b20143a20 2020-11-20T12:20:09.425 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:09.425 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":86,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:58.311412+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:54.974437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:56.326129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:09.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 -- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2afc0557f0 msgr2=0x7f2afc057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2afc0557f0 0x7f2afc057cb0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f2b10006d00 tx=0x7f2b10008040).stop 2020-11-20T12:20:09.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 -- 172.21.15.116:0/3409198500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b20143a20 msgr2=0x7f2b20063680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b20143a20 0x7f2b20063680 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f2b200bea40 tx=0x7f2b18007a20).stop 2020-11-20T12:20:09.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 -- 172.21.15.116:0/3409198500 shutdown_connections 2020-11-20T12:20:09.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2afc0557f0 0x7f2afc057cb0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2b20143a20 0x7f2b20063680 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2b20143640 0x7f2b200a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 --2- 172.21.15.116:0/3409198500 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2b200a4490 0x7f2b20063190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 -- 172.21.15.116:0/3409198500 >> 172.21.15.116:0/3409198500 conn(0x7f2b200c4f80 msgr2=0x7f2b200bed70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:09.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 -- 172.21.15.116:0/3409198500 shutdown_connections 2020-11-20T12:20:09.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.432+0000 7f2b265de700 1 -- 172.21.15.116:0/3409198500 wait complete. 2020-11-20T12:20:09.453 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:19:54.974437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:19:56.326129+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:20:09.454 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:20:09.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.628+0000 7f7a4575e700 1 Processor -- start 2020-11-20T12:20:09.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.632+0000 7f7a4575e700 1 -- start start 2020-11-20T12:20:09.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.632+0000 7f7a4575e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a40065770 0x7f7a40065b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.632+0000 7f7a4575e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7a400ca810 0x7f7a401557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.632+0000 7f7a4575e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a40155eb0 con 0x7f7a400a8d20 2020-11-20T12:20:09.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.632+0000 7f7a4575e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a400cac90 con 0x7f7a400ca810 2020-11-20T12:20:09.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.632+0000 7f7a4575e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a40155c90 con 0x7f7a40065770 2020-11-20T12:20:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3ffff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7a400a8d20 0x7f7a401521e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58452/0 (socket says 172.21.15.116:58452) 2020-11-20T12:20:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3ffff700 1 -- 172.21.15.116:0/2043159640 learned_addr learned my addr 172.21.15.116:0/2043159640 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3effd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a40065770 0x7f7a40065b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3e7fc700 1 --2- 172.21.15.116:0/2043159640 >> v2:172.21.15.173:3300/0 conn(0x7f7a400ca810 0x7f7a401557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3dffb700 1 -- 172.21.15.116:0/2043159640 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2564892552 0 0) 0x7f7a40155eb0 con 0x7f7a400a8d20 2020-11-20T12:20:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3e7fc700 1 -- 172.21.15.116:0/2043159640 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a40065770 msgr2=0x7f7a40065b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3e7fc700 1 --2- 172.21.15.116:0/2043159640 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a40065770 0x7f7a40065b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3e7fc700 1 -- 172.21.15.116:0/2043159640 >> v1:172.21.15.116:6789/0 conn(0x7f7a400a8d20 legacy=0x7f7a401521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3e7fc700 1 -- 172.21.15.116:0/2043159640 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a40156150 con 0x7f7a400ca810 2020-11-20T12:20:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3e7fc700 1 --2- 172.21.15.116:0/2043159640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a400ca810 0x7f7a401557a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f7a3400b9d0 tx=0x7f7a34006df0).ready entity=mon.1 client_cookie=621c3db84fc0ac11 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3dffb700 1 -- 172.21.15.116:0/2043159640 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34009770 con 0x7f7a400ca810 2020-11-20T12:20:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3dffb700 1 -- 172.21.15.116:0/2043159640 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a340098d0 con 0x7f7a400ca810 2020-11-20T12:20:09.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a3dffb700 1 -- 172.21.15.116:0/2043159640 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34009c50 con 0x7f7a400ca810 2020-11-20T12:20:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a4575e700 1 -- 172.21.15.116:0/2043159640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a400ca810 msgr2=0x7f7a401557a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a4575e700 1 --2- 172.21.15.116:0/2043159640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a400ca810 0x7f7a401557a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f7a3400b9d0 tx=0x7f7a34006df0).stop 2020-11-20T12:20:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a4575e700 1 -- 172.21.15.116:0/2043159640 shutdown_connections 2020-11-20T12:20:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a4575e700 1 --2- 172.21.15.116:0/2043159640 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a40065770 0x7f7a40065b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a4575e700 1 --2- 172.21.15.116:0/2043159640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a400ca810 0x7f7a401557a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a4575e700 1 -- 172.21.15.116:0/2043159640 >> 172.21.15.116:0/2043159640 conn(0x7f7a400647f0 msgr2=0x7f7a400c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:09.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a4575e700 1 -- 172.21.15.116:0/2043159640 shutdown_connections 2020-11-20T12:20:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.636+0000 7f7a4575e700 1 -- 172.21.15.116:0/2043159640 wait complete. 2020-11-20T12:20:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.640+0000 7f7a4575e700 1 Processor -- start 2020-11-20T12:20:09.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.640+0000 7f7a4575e700 1 -- start start 2020-11-20T12:20:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a4575e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a40065770 0x7f7a40081470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a4575e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a400a8d20 0x7f7a40169810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a4575e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a400ca810 0x7f7a40141500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a4575e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a40169d00 con 0x7f7a400a8d20 2020-11-20T12:20:09.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a4575e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4014f000 con 0x7f7a40065770 2020-11-20T12:20:09.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a4575e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a40081a60 con 0x7f7a400ca810 2020-11-20T12:20:09.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3e7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a400a8d20 0x7f7a40169810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3e7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a400a8d20 0x7f7a40169810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49020/0 (socket says 172.21.15.116:49020) 2020-11-20T12:20:09.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3e7fc700 1 -- 172.21.15.116:0/1645894937 learned_addr learned my addr 172.21.15.116:0/1645894937 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:09.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3ffff700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a400ca810 0x7f7a40141500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3effd700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a40065770 0x7f7a40081470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3e7fc700 1 -- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a400ca810 msgr2=0x7f7a40141500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3e7fc700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a400ca810 0x7f7a40141500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3e7fc700 1 -- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a40065770 msgr2=0x7f7a40081470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3e7fc700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a40065770 0x7f7a40081470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3e7fc700 1 -- 172.21.15.116:0/1645894937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a40081d00 con 0x7f7a400a8d20 2020-11-20T12:20:09.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3ffff700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a400ca810 0x7f7a40141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:09.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a3e7fc700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a400a8d20 0x7f7a40169810 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f7a34010000 tx=0x7f7a34015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:09.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a26ffd700 1 -- 172.21.15.116:0/1645894937 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a34011d50 con 0x7f7a400a8d20 2020-11-20T12:20:09.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a26ffd700 1 -- 172.21.15.116:0/1645894937 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3401e460 con 0x7f7a400a8d20 2020-11-20T12:20:09.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a4575e700 1 -- 172.21.15.116:0/1645894937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a40081fa0 con 0x7f7a400a8d20 2020-11-20T12:20:09.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a26ffd700 1 -- 172.21.15.116:0/1645894937 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3401e7e0 con 0x7f7a400a8d20 2020-11-20T12:20:09.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.644+0000 7f7a4575e700 1 -- 172.21.15.116:0/1645894937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a400824e0 con 0x7f7a400a8d20 2020-11-20T12:20:09.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.648+0000 7f7a4575e700 1 -- 172.21.15.116:0/1645894937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4005d2a0 con 0x7f7a400a8d20 2020-11-20T12:20:09.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.648+0000 7f7a26ffd700 1 -- 172.21.15.116:0/1645894937 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7a34025030 con 0x7f7a400a8d20 2020-11-20T12:20:09.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.648+0000 7f7a26ffd700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a1c0557c0 0x7f7a1c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:09.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.648+0000 7f7a26ffd700 1 -- 172.21.15.116:0/1645894937 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7a3406fba0 con 0x7f7a400a8d20 2020-11-20T12:20:09.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.652+0000 7f7a3effd700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a1c0557c0 0x7f7a1c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:09.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.652+0000 7f7a3effd700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a1c0557c0 0x7f7a1c057c80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f7a30006fd0 tx=0x7f7a30006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:09.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.652+0000 7f7a26ffd700 1 -- 172.21.15.116:0/1645894937 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7a34045af0 con 0x7f7a400a8d20 2020-11-20T12:20:09.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.812+0000 7f7a4575e700 1 -- 172.21.15.116:0/1645894937 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f7a40063850 con 0x7f7a400a8d20 2020-11-20T12:20:09.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a26ffd700 1 -- 172.21.15.116:0/1645894937 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v86) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7a34023030 con 0x7f7a400a8d20 2020-11-20T12:20:09.812 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:20:09.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 -- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a1c0557c0 msgr2=0x7f7a1c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a1c0557c0 0x7f7a1c057c80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f7a30006fd0 tx=0x7f7a30006df0).stop 2020-11-20T12:20:09.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 -- 172.21.15.116:0/1645894937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a400a8d20 msgr2=0x7f7a40169810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:09.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a400a8d20 0x7f7a40169810 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7f7a34010000 tx=0x7f7a34015000).stop 2020-11-20T12:20:09.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 -- 172.21.15.116:0/1645894937 shutdown_connections 2020-11-20T12:20:09.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7a1c0557c0 0x7f7a1c057c80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7a400ca810 0x7f7a40141500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7a40065770 0x7f7a40081470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 --2- 172.21.15.116:0/1645894937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7a400a8d20 0x7f7a40169810 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:09.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 -- 172.21.15.116:0/1645894937 >> 172.21.15.116:0/1645894937 conn(0x7f7a400647f0 msgr2=0x7f7a40153bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:09.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 -- 172.21.15.116:0/1645894937 shutdown_connections 2020-11-20T12:20:09.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:09.816+0000 7f7a4575e700 1 -- 172.21.15.116:0/1645894937 wait complete. 2020-11-20T12:20:09.837 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:20:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.012+0000 7f19e89e2700 1 Processor -- start 2020-11-20T12:20:10.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.012+0000 7f19e89e2700 1 -- start start 2020-11-20T12:20:10.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e89e2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19e40aa7d0 0x7f19e40a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e89e2700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f19e40ad540 0x7f19e40ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e89e2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19e40ace40 con 0x7f19e40aabb0 2020-11-20T12:20:10.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e89e2700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19e41526c0 con 0x7f19e40ad540 2020-11-20T12:20:10.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e89e2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e4152960 con 0x7f19e40aa7d0 2020-11-20T12:20:10.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e359e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f19e40ad540 0x7f19e40ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e2d9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19e40aa7d0 0x7f19e40a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e259c700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f19e40aabb0 0x7f19e4151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:54694/0 (socket says 172.21.15.116:54694) 2020-11-20T12:20:10.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e259c700 1 -- 172.21.15.116:0/1378211147 learned_addr learned my addr 172.21.15.116:0/1378211147 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:10.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e2d9d700 1 -- 172.21.15.116:0/1378211147 >> v2:172.21.15.173:3300/0 conn(0x7f19e40ad540 msgr2=0x7f19e40ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:10.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e2d9d700 1 --2- 172.21.15.116:0/1378211147 >> v2:172.21.15.173:3300/0 conn(0x7f19e40ad540 0x7f19e40ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e2d9d700 1 -- 172.21.15.116:0/1378211147 >> v1:172.21.15.173:6789/0 conn(0x7f19e40aabb0 legacy=0x7f19e4151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e2d9d700 1 -- 172.21.15.116:0/1378211147 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4152c00 con 0x7f19e40aa7d0 2020-11-20T12:20:10.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.016+0000 7f19e2d9d700 1 --2- 172.21.15.116:0/1378211147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19e40aa7d0 0x7f19e40a8400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f19d400b9a0 tx=0x7f19d4006df0).ready entity=mon.2 client_cookie=a7619620520a7e21 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e1d9b700 1 -- 172.21.15.116:0/1378211147 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d40097b0 con 0x7f19e40aa7d0 2020-11-20T12:20:10.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e1d9b700 1 -- 172.21.15.116:0/1378211147 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d4009910 con 0x7f19e40aa7d0 2020-11-20T12:20:10.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e1d9b700 1 -- 172.21.15.116:0/1378211147 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4009c90 con 0x7f19e40aa7d0 2020-11-20T12:20:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e89e2700 1 -- 172.21.15.116:0/1378211147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19e40aa7d0 msgr2=0x7f19e40a8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e89e2700 1 --2- 172.21.15.116:0/1378211147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19e40aa7d0 0x7f19e40a8400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f19d400b9a0 tx=0x7f19d4006df0).stop 2020-11-20T12:20:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e89e2700 1 -- 172.21.15.116:0/1378211147 shutdown_connections 2020-11-20T12:20:10.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e89e2700 1 --2- 172.21.15.116:0/1378211147 >> v2:172.21.15.173:3300/0 conn(0x7f19e40ad540 0x7f19e40ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e89e2700 1 --2- 172.21.15.116:0/1378211147 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19e40aa7d0 0x7f19e40a8400 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e89e2700 1 -- 172.21.15.116:0/1378211147 >> 172.21.15.116:0/1378211147 conn(0x7f19e40678a0 msgr2=0x7f19e4067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e89e2700 1 -- 172.21.15.116:0/1378211147 shutdown_connections 2020-11-20T12:20:10.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e89e2700 1 -- 172.21.15.116:0/1378211147 wait complete. 2020-11-20T12:20:10.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e89e2700 1 Processor -- start 2020-11-20T12:20:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.020+0000 7f19e89e2700 1 -- start start 2020-11-20T12:20:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e89e2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19e40aa7d0 0x7f19e414ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e89e2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19e40aabb0 0x7f19e40c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e89e2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19e40ad540 0x7f19e414f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e89e2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e414abd0 con 0x7f19e40aabb0 2020-11-20T12:20:10.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e89e2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19e41654e0 con 0x7f19e40aa7d0 2020-11-20T12:20:10.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e89e2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e41645c0 con 0x7f19e40ad540 2020-11-20T12:20:10.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19e40aabb0 0x7f19e40c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19e40ad540 0x7f19e414f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19e40aabb0 0x7f19e40c4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49038/0 (socket says 172.21.15.116:49038) 2020-11-20T12:20:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e259c700 1 -- 172.21.15.116:0/4254752238 learned_addr learned my addr 172.21.15.116:0/4254752238 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:10.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e2d9d700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19e40aa7d0 0x7f19e414ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e259c700 1 -- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19e40ad540 msgr2=0x7f19e414f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e259c700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19e40ad540 0x7f19e414f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e259c700 1 -- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19e40aa7d0 msgr2=0x7f19e414ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e259c700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19e40aa7d0 0x7f19e414ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e259c700 1 -- 172.21.15.116:0/4254752238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e4164860 con 0x7f19e40aabb0 2020-11-20T12:20:10.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e259c700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19e40aabb0 0x7f19e40c4fa0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f19d800c710 tx=0x7f19d8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19caffd700 1 -- 172.21.15.116:0/4254752238 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d80072e0 con 0x7f19e40aabb0 2020-11-20T12:20:10.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19caffd700 1 -- 172.21.15.116:0/4254752238 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19d8007440 con 0x7f19e40aabb0 2020-11-20T12:20:10.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e89e2700 1 -- 172.21.15.116:0/4254752238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19e4164b00 con 0x7f19e40aabb0 2020-11-20T12:20:10.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19caffd700 1 -- 172.21.15.116:0/4254752238 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d8005800 con 0x7f19e40aabb0 2020-11-20T12:20:10.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.024+0000 7f19e89e2700 1 -- 172.21.15.116:0/4254752238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19e414f200 con 0x7f19e40aabb0 2020-11-20T12:20:10.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.028+0000 7f19e89e2700 1 -- 172.21.15.116:0/4254752238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19e4164c90 con 0x7f19e40aabb0 2020-11-20T12:20:10.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.028+0000 7f19caffd700 1 -- 172.21.15.116:0/4254752238 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f19d8010070 con 0x7f19e40aabb0 2020-11-20T12:20:10.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.028+0000 7f19caffd700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19c00557c0 0x7f19c0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.028+0000 7f19caffd700 1 -- 172.21.15.116:0/4254752238 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f19d806dd60 con 0x7f19e40aabb0 2020-11-20T12:20:10.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.032+0000 7f19e2d9d700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19c00557c0 0x7f19c0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.032+0000 7f19e2d9d700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19c00557c0 0x7f19c0057c80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f19d400b150 tx=0x7f19d4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.032+0000 7f19caffd700 1 -- 172.21.15.116:0/4254752238 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f19d8043af0 con 0x7f19e40aabb0 2020-11-20T12:20:10.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.192+0000 7f19e89e2700 1 -- 172.21.15.116:0/4254752238 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f19e405cf30 con 0x7f19e40aabb0 2020-11-20T12:20:10.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19caffd700 1 -- 172.21.15.116:0/4254752238 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7f19d8047110 con 0x7f19e40aabb0 2020-11-20T12:20:10.192 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:10.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 -- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19c00557c0 msgr2=0x7f19c0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19c00557c0 0x7f19c0057c80 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f19d400b150 tx=0x7f19d4015000).stop 2020-11-20T12:20:10.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 -- 172.21.15.116:0/4254752238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19e40aabb0 msgr2=0x7f19e40c4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19e40aabb0 0x7f19e40c4fa0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7f19d800c710 tx=0x7f19d8007c10).stop 2020-11-20T12:20:10.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 -- 172.21.15.116:0/4254752238 shutdown_connections 2020-11-20T12:20:10.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f19c00557c0 0x7f19c0057c80 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f19e40ad540 0x7f19e414f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f19e40aa7d0 0x7f19e414ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 --2- 172.21.15.116:0/4254752238 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f19e40aabb0 0x7f19e40c4fa0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 -- 172.21.15.116:0/4254752238 >> 172.21.15.116:0/4254752238 conn(0x7f19e40678a0 msgr2=0x7f19e41549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 -- 172.21.15.116:0/4254752238 shutdown_connections 2020-11-20T12:20:10.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.196+0000 7f19e89e2700 1 -- 172.21.15.116:0/4254752238 wait complete. 2020-11-20T12:20:10.222 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:20:10.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d36718700 1 Processor -- start 2020-11-20T12:20:10.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d36718700 1 -- start start 2020-11-20T12:20:10.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d36718700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9d300aa7d0 0x7f9d300a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d36718700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d300ad540 0x7f9d300ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d36718700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d300ace40 con 0x7f9d300aabb0 2020-11-20T12:20:10.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d36718700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d301526c0 con 0x7f9d300ad540 2020-11-20T12:20:10.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d36718700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d30152960 con 0x7f9d300aa7d0 2020-11-20T12:20:10.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d34f15700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9d300aabb0 0x7f9d30151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58480/0 (socket says 172.21.15.116:58480) 2020-11-20T12:20:10.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d34f15700 1 -- 172.21.15.116:0/1651745385 learned_addr learned my addr 172.21.15.116:0/1651745385 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:10.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d35f17700 1 --2- 172.21.15.116:0/1651745385 >> v2:172.21.15.116:3300/0 conn(0x7f9d300ad540 0x7f9d300ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d35716700 1 --2- 172.21.15.116:0/1651745385 >> v2:172.21.15.173:3300/0 conn(0x7f9d300aa7d0 0x7f9d300a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d1ffff700 1 -- 172.21.15.116:0/1651745385 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199460857 0 0) 0x7f9d300ace40 con 0x7f9d300aabb0 2020-11-20T12:20:10.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d1ffff700 1 -- 172.21.15.116:0/1651745385 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d18003240 con 0x7f9d300aabb0 2020-11-20T12:20:10.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d35f17700 1 -- 172.21.15.116:0/1651745385 >> v2:172.21.15.173:3300/0 conn(0x7f9d300aa7d0 msgr2=0x7f9d300a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:10.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d35f17700 1 --2- 172.21.15.116:0/1651745385 >> v2:172.21.15.173:3300/0 conn(0x7f9d300aa7d0 0x7f9d300a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d35f17700 1 -- 172.21.15.116:0/1651745385 >> v1:172.21.15.116:6789/0 conn(0x7f9d300aabb0 legacy=0x7f9d30151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.428+0000 7f9d35f17700 1 -- 172.21.15.116:0/1651745385 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d30152c00 con 0x7f9d300ad540 2020-11-20T12:20:10.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d35f17700 1 --2- 172.21.15.116:0/1651745385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300ad540 0x7f9d300ad980 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f9d30081950 tx=0x7f9d280091d0).ready entity=mon.0 client_cookie=e4de4e45ea53ed7c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d1ffff700 1 -- 172.21.15.116:0/1651745385 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800b9c0 con 0x7f9d300ad540 2020-11-20T12:20:10.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d1ffff700 1 -- 172.21.15.116:0/1651745385 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2800bb20 con 0x7f9d300ad540 2020-11-20T12:20:10.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d1ffff700 1 -- 172.21.15.116:0/1651745385 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2800bea0 con 0x7f9d300ad540 2020-11-20T12:20:10.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d36718700 1 -- 172.21.15.116:0/1651745385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300ad540 msgr2=0x7f9d300ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d36718700 1 --2- 172.21.15.116:0/1651745385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300ad540 0x7f9d300ad980 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f9d30081950 tx=0x7f9d280091d0).stop 2020-11-20T12:20:10.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d36718700 1 -- 172.21.15.116:0/1651745385 shutdown_connections 2020-11-20T12:20:10.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d36718700 1 --2- 172.21.15.116:0/1651745385 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300ad540 0x7f9d300ad980 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d36718700 1 --2- 172.21.15.116:0/1651745385 >> v2:172.21.15.173:3300/0 conn(0x7f9d300aa7d0 0x7f9d300a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d36718700 1 -- 172.21.15.116:0/1651745385 >> 172.21.15.116:0/1651745385 conn(0x7f9d300678a0 msgr2=0x7f9d30067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:10.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d36718700 1 -- 172.21.15.116:0/1651745385 shutdown_connections 2020-11-20T12:20:10.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.432+0000 7f9d36718700 1 -- 172.21.15.116:0/1651745385 wait complete. 2020-11-20T12:20:10.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.436+0000 7f9d36718700 1 Processor -- start 2020-11-20T12:20:10.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.436+0000 7f9d36718700 1 -- start start 2020-11-20T12:20:10.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.436+0000 7f9d36718700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300aa7d0 0x7f9d30162060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.436+0000 7f9d36718700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d300aabb0 0x7f9d30163be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.436+0000 7f9d36718700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d300ad540 0x7f9d300bda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.436+0000 7f9d36718700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d301653e0 con 0x7f9d300aa7d0 2020-11-20T12:20:10.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.436+0000 7f9d36718700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30164960 con 0x7f9d300ad540 2020-11-20T12:20:10.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.436+0000 7f9d36718700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30164c00 con 0x7f9d300aabb0 2020-11-20T12:20:10.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.436+0000 7f9d35716700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300aa7d0 0x7f9d30162060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.436+0000 7f9d35716700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300aa7d0 0x7f9d30162060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49048/0 (socket says 172.21.15.116:49048) 2020-11-20T12:20:10.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.436+0000 7f9d35716700 1 -- 172.21.15.116:0/3184727647 learned_addr learned my addr 172.21.15.116:0/3184727647 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:10.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d35f17700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d300ad540 0x7f9d300bda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d34f15700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d300aabb0 0x7f9d30163be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d35716700 1 -- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d300aabb0 msgr2=0x7f9d30163be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d35716700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d300aabb0 0x7f9d30163be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d35716700 1 -- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d300ad540 msgr2=0x7f9d300bda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d35716700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d300ad540 0x7f9d300bda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d35716700 1 -- 172.21.15.116:0/3184727647 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d30164ea0 con 0x7f9d300aa7d0 2020-11-20T12:20:10.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d35716700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300aa7d0 0x7f9d30162060 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f9d2c00eba0 tx=0x7f9d2c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d1d7fa700 1 -- 172.21.15.116:0/3184727647 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c00ca60 con 0x7f9d300aa7d0 2020-11-20T12:20:10.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d1d7fa700 1 -- 172.21.15.116:0/3184727647 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d2c00cbc0 con 0x7f9d300aa7d0 2020-11-20T12:20:10.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d36718700 1 -- 172.21.15.116:0/3184727647 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d30165140 con 0x7f9d300aa7d0 2020-11-20T12:20:10.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d1d7fa700 1 -- 172.21.15.116:0/3184727647 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0123f0 con 0x7f9d300aa7d0 2020-11-20T12:20:10.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d36718700 1 -- 172.21.15.116:0/3184727647 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d30164500 con 0x7f9d300aa7d0 2020-11-20T12:20:10.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.440+0000 7f9d36718700 1 -- 172.21.15.116:0/3184727647 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d30161010 con 0x7f9d300aa7d0 2020-11-20T12:20:10.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.444+0000 7f9d1d7fa700 1 -- 172.21.15.116:0/3184727647 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9d2c0127a0 con 0x7f9d300aa7d0 2020-11-20T12:20:10.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.444+0000 7f9d1d7fa700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d080557c0 0x7f9d08057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.444+0000 7f9d1d7fa700 1 -- 172.21.15.116:0/3184727647 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9d2c06eba0 con 0x7f9d300aa7d0 2020-11-20T12:20:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.444+0000 7f9d34f15700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d080557c0 0x7f9d08057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.444+0000 7f9d34f15700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d080557c0 0x7f9d08057c80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f9d200055a0 tx=0x7f9d20009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.448+0000 7f9d1d7fa700 1 -- 172.21.15.116:0/3184727647 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9d30161010 con 0x7f9d300aa7d0 2020-11-20T12:20:10.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.612+0000 7f9d36718700 1 -- 172.21.15.116:0/3184727647 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9d3005cf30 con 0x7f9d300aa7d0 2020-11-20T12:20:10.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.612+0000 7f9d1d7fa700 1 -- 172.21.15.116:0/3184727647 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9d2c044a40 con 0x7f9d300aa7d0 2020-11-20T12:20:10.610 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:10.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 -- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d080557c0 msgr2=0x7f9d08057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d080557c0 0x7f9d08057c80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f9d200055a0 tx=0x7f9d20009040).stop 2020-11-20T12:20:10.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 -- 172.21.15.116:0/3184727647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300aa7d0 msgr2=0x7f9d30162060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300aa7d0 0x7f9d30162060 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f9d2c00eba0 tx=0x7f9d2c006df0).stop 2020-11-20T12:20:10.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 -- 172.21.15.116:0/3184727647 shutdown_connections 2020-11-20T12:20:10.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d080557c0 0x7f9d08057c80 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d300ad540 0x7f9d300bda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d300aa7d0 0x7f9d30162060 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 --2- 172.21.15.116:0/3184727647 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d300aabb0 0x7f9d30163be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 -- 172.21.15.116:0/3184727647 >> 172.21.15.116:0/3184727647 conn(0x7f9d300678a0 msgr2=0x7f9d30067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:10.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 -- 172.21.15.116:0/3184727647 shutdown_connections 2020-11-20T12:20:10.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.616+0000 7f9d36718700 1 -- 172.21.15.116:0/3184727647 wait complete. 2020-11-20T12:20:10.637 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:20:10.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.816+0000 7fedd359e700 1 Processor -- start 2020-11-20T12:20:10.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.816+0000 7fedd359e700 1 -- start start 2020-11-20T12:20:10.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedd414f880 0x7fedd40b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd359e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fedd40c0e90 0x7fedd40c1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd359e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedd40a8930 con 0x7fedd40a81f0 2020-11-20T12:20:10.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd359e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedd40ba2a0 con 0x7fedd40c0e90 2020-11-20T12:20:10.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd40a8690 con 0x7fedd414f880 2020-11-20T12:20:10.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd2d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fedd40c0e90 0x7fedd40c1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd2d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fedd40c0e90 0x7fedd40c1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49060/0 (socket says 172.21.15.116:49060) 2020-11-20T12:20:10.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd2d9d700 1 -- 172.21.15.116:0/4071724032 learned_addr learned my addr 172.21.15.116:0/4071724032 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:10.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd259c700 1 --2- 172.21.15.116:0/4071724032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedd414f880 0x7fedd40b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd2d9d700 1 -- 172.21.15.116:0/4071724032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedd414f880 msgr2=0x7fedd40b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:10.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd2d9d700 1 --2- 172.21.15.116:0/4071724032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedd414f880 0x7fedd40b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd2d9d700 1 -- 172.21.15.116:0/4071724032 >> v1:172.21.15.173:6789/0 conn(0x7fedd40a81f0 legacy=0x7fedd4060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd2d9d700 1 -- 172.21.15.116:0/4071724032 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd40b94d0 con 0x7fedd40c0e90 2020-11-20T12:20:10.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd2d9d700 1 --2- 172.21.15.116:0/4071724032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedd40c0e90 0x7fedd40c1300 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fedcc006fd0 tx=0x7fedcc006eb0).ready entity=mon.0 client_cookie=e1d6f0a7454abf10 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.820+0000 7fedd159a700 1 -- 172.21.15.116:0/4071724032 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc0098d0 con 0x7fedd40c0e90 2020-11-20T12:20:10.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.824+0000 7fedd159a700 1 -- 172.21.15.116:0/4071724032 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedcc009a30 con 0x7fedd40c0e90 2020-11-20T12:20:10.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.824+0000 7fedd159a700 1 -- 172.21.15.116:0/4071724032 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc009db0 con 0x7fedd40c0e90 2020-11-20T12:20:10.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.824+0000 7fedd359e700 1 -- 172.21.15.116:0/4071724032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedd40c0e90 msgr2=0x7fedd40c1300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.824+0000 7fedd359e700 1 --2- 172.21.15.116:0/4071724032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedd40c0e90 0x7fedd40c1300 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7fedcc006fd0 tx=0x7fedcc006eb0).stop 2020-11-20T12:20:10.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.824+0000 7fedd359e700 1 -- 172.21.15.116:0/4071724032 shutdown_connections 2020-11-20T12:20:10.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.824+0000 7fedd359e700 1 --2- 172.21.15.116:0/4071724032 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedd40c0e90 0x7fedd40c1300 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.824+0000 7fedd359e700 1 --2- 172.21.15.116:0/4071724032 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedd414f880 0x7fedd40b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.824+0000 7fedd359e700 1 -- 172.21.15.116:0/4071724032 >> 172.21.15.116:0/4071724032 conn(0x7fedd4068350 msgr2=0x7fedd40627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:10.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.824+0000 7fedd359e700 1 -- 172.21.15.116:0/4071724032 shutdown_connections 2020-11-20T12:20:10.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.824+0000 7fedd359e700 1 -- 172.21.15.116:0/4071724032 wait complete. 2020-11-20T12:20:10.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd359e700 1 Processor -- start 2020-11-20T12:20:10.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd359e700 1 -- start start 2020-11-20T12:20:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedd40a81f0 0x7fedd409dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedd40c0e90 0x7fedd40a5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedd414f880 0x7fedd40a6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd415fc30 con 0x7fedd40a81f0 2020-11-20T12:20:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedd409d5c0 con 0x7fedd40c0e90 2020-11-20T12:20:10.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedd409d860 con 0x7fedd414f880 2020-11-20T12:20:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedd40a81f0 0x7fedd409dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedd40a81f0 0x7fedd409dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49064/0 (socket says 172.21.15.116:49064) 2020-11-20T12:20:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd259c700 1 -- 172.21.15.116:0/3864144554 learned_addr learned my addr 172.21.15.116:0/3864144554 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd1d9b700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedd40c0e90 0x7fedd40a5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd2d9d700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedd414f880 0x7fedd40a6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd259c700 1 -- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedd414f880 msgr2=0x7fedd40a6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd259c700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedd414f880 0x7fedd40a6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd259c700 1 -- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedd40c0e90 msgr2=0x7fedd40a5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd259c700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedd40c0e90 0x7fedd40a5dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.828+0000 7fedd259c700 1 -- 172.21.15.116:0/3864144554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedd409db00 con 0x7fedd40a81f0 2020-11-20T12:20:10.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.832+0000 7fedd259c700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedd40a81f0 0x7fedd409dfa0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fedc400ec90 tx=0x7fedc4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.832+0000 7fedc27fc700 1 -- 172.21.15.116:0/3864144554 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400c790 con 0x7fedd40a81f0 2020-11-20T12:20:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.832+0000 7fedc27fc700 1 -- 172.21.15.116:0/3864144554 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc400c8f0 con 0x7fedd40a81f0 2020-11-20T12:20:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.832+0000 7fedd359e700 1 -- 172.21.15.116:0/3864144554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedd409e490 con 0x7fedd40a81f0 2020-11-20T12:20:10.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.832+0000 7fedc27fc700 1 -- 172.21.15.116:0/3864144554 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc400cc70 con 0x7fedd40a81f0 2020-11-20T12:20:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.832+0000 7fedd359e700 1 -- 172.21.15.116:0/3864144554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedd40a6af0 con 0x7fedd40a81f0 2020-11-20T12:20:10.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.832+0000 7fedd359e700 1 -- 172.21.15.116:0/3864144554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedd405cf30 con 0x7fedd40a81f0 2020-11-20T12:20:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.832+0000 7fedc27fc700 1 -- 172.21.15.116:0/3864144554 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fedc401d460 con 0x7fedd40a81f0 2020-11-20T12:20:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.836+0000 7fedc27fc700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feda80557c0 0x7feda8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.836+0000 7fedc27fc700 1 -- 172.21.15.116:0/3864144554 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fedc406e6b0 con 0x7fedd40a81f0 2020-11-20T12:20:10.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.836+0000 7fedd1d9b700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feda80557c0 0x7feda8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:10.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.836+0000 7fedd1d9b700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feda80557c0 0x7feda8057c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fedd414dea0 tx=0x7fedc8007b90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:10.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:10.840+0000 7fedc27fc700 1 -- 172.21.15.116:0/3864144554 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fedc40444f0 con 0x7fedd40a81f0 2020-11-20T12:20:10.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.000+0000 7fedd359e700 1 -- 172.21.15.116:0/3864144554 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fedd4061310 con 0x7fedd40a81f0 2020-11-20T12:20:11.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.000+0000 7fedc27fc700 1 -- 172.21.15.116:0/3864144554 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+10 (secure 0 0 0) 0x7fedc4047b10 con 0x7fedd40a81f0 2020-11-20T12:20:11.409 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:11.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 -- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feda80557c0 msgr2=0x7feda8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:11.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feda80557c0 0x7feda8057c80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fedd414dea0 tx=0x7fedc8007b90).stop 2020-11-20T12:20:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 -- 172.21.15.116:0/3864144554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedd40a81f0 msgr2=0x7fedd409dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedd40a81f0 0x7fedd409dfa0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7fedc400ec90 tx=0x7fedc4008040).stop 2020-11-20T12:20:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 -- 172.21.15.116:0/3864144554 shutdown_connections 2020-11-20T12:20:11.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feda80557c0 0x7feda8057c80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fedd414f880 0x7fedd40a6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fedd40a81f0 0x7fedd409dfa0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 --2- 172.21.15.116:0/3864144554 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fedd40c0e90 0x7fedd40a5dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 -- 172.21.15.116:0/3864144554 >> 172.21.15.116:0/3864144554 conn(0x7fedd4068350 msgr2=0x7fedd40bd760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:11.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 -- 172.21.15.116:0/3864144554 shutdown_connections 2020-11-20T12:20:11.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.004+0000 7fedd359e700 1 -- 172.21.15.116:0/3864144554 wait complete. 2020-11-20T12:20:11.415 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:20:11.415 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.596+0000 7f5449214700 1 Processor -- start 2020-11-20T12:20:11.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.596+0000 7f5449214700 1 -- start start 2020-11-20T12:20:11.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.596+0000 7f5449214700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5444062880 0x7f5444062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:11.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.596+0000 7f5449214700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5444065d40 0x7f54440bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:11.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.596+0000 7f5449214700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54440c0530 con 0x7f54440677a0 2020-11-20T12:20:11.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.596+0000 7f5449214700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54440661a0 con 0x7f5444065d40 2020-11-20T12:20:11.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.596+0000 7f5449214700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54440c0310 con 0x7f5444062880 2020-11-20T12:20:11.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f54437fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5444065d40 0x7f54440bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:11.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f54437fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5444065d40 0x7f54440bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49070/0 (socket says 172.21.15.116:49070) 2020-11-20T12:20:11.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f54437fe700 1 -- 172.21.15.116:0/3945804853 learned_addr learned my addr 172.21.15.116:0/3945804853 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:11.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5443fff700 1 --2- 172.21.15.116:0/3945804853 >> v2:172.21.15.173:3300/0 conn(0x7f5444062880 0x7f5444062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:11.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f54437fe700 1 -- 172.21.15.116:0/3945804853 >> v2:172.21.15.173:3300/0 conn(0x7f5444062880 msgr2=0x7f5444062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:11.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f54437fe700 1 --2- 172.21.15.116:0/3945804853 >> v2:172.21.15.173:3300/0 conn(0x7f5444062880 0x7f5444062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f54437fe700 1 -- 172.21.15.116:0/3945804853 >> v1:172.21.15.173:6789/0 conn(0x7f54440677a0 legacy=0x7f54441556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:11.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f54437fe700 1 -- 172.21.15.116:0/3945804853 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54440c07d0 con 0x7f5444065d40 2020-11-20T12:20:11.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5443fff700 1 --2- 172.21.15.116:0/3945804853 >> v2:172.21.15.173:3300/0 conn(0x7f5444062880 0x7f5444062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:11.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f54437fe700 1 --2- 172.21.15.116:0/3945804853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5444065d40 0x7f54440bfe20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f54440c1e20 tx=0x7f5438006eb0).ready entity=mon.0 client_cookie=98d950e8e33b06ae server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:11.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5442ffd700 1 -- 172.21.15.116:0/3945804853 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438009890 con 0x7f5444065d40 2020-11-20T12:20:11.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5442ffd700 1 -- 172.21.15.116:0/3945804853 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54380099f0 con 0x7f5444065d40 2020-11-20T12:20:11.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5442ffd700 1 -- 172.21.15.116:0/3945804853 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5438009d70 con 0x7f5444065d40 2020-11-20T12:20:11.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5449214700 1 -- 172.21.15.116:0/3945804853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5444065d40 msgr2=0x7f54440bfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:11.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5449214700 1 --2- 172.21.15.116:0/3945804853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5444065d40 0x7f54440bfe20 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f54440c1e20 tx=0x7f5438006eb0).stop 2020-11-20T12:20:11.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5449214700 1 -- 172.21.15.116:0/3945804853 shutdown_connections 2020-11-20T12:20:11.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5449214700 1 --2- 172.21.15.116:0/3945804853 >> v2:172.21.15.173:3300/0 conn(0x7f5444062880 0x7f5444062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5449214700 1 --2- 172.21.15.116:0/3945804853 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5444065d40 0x7f54440bfe20 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5449214700 1 -- 172.21.15.116:0/3945804853 >> 172.21.15.116:0/3945804853 conn(0x7f5444060c40 msgr2=0x7f54440c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:11.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.600+0000 7f5449214700 1 -- 172.21.15.116:0/3945804853 shutdown_connections 2020-11-20T12:20:11.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.604+0000 7f5449214700 1 -- 172.21.15.116:0/3945804853 wait complete. 2020-11-20T12:20:11.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5449214700 1 Processor -- start 2020-11-20T12:20:11.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5449214700 1 -- start start 2020-11-20T12:20:11.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5449214700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5444062880 0x7f5444156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:11.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5449214700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5444065d40 0x7f54440a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:11.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5449214700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54440677a0 0x7f5444145c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:11.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5449214700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5444156af0 con 0x7f5444062880 2020-11-20T12:20:11.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5449214700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5444156d90 con 0x7f5444065d40 2020-11-20T12:20:11.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5449214700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5444157030 con 0x7f54440677a0 2020-11-20T12:20:11.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5443fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5444062880 0x7f5444156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:11.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5443fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5444062880 0x7f5444156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49076/0 (socket says 172.21.15.116:49076) 2020-11-20T12:20:11.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5443fff700 1 -- 172.21.15.116:0/2153871897 learned_addr learned my addr 172.21.15.116:0/2153871897 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:11.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f54437fe700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5444065d40 0x7f54440a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:11.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5448a13700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54440677a0 0x7f5444145c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:11.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5443fff700 1 -- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54440677a0 msgr2=0x7f5444145c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:11.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5443fff700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54440677a0 0x7f5444145c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5443fff700 1 -- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5444065d40 msgr2=0x7f54440a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:11.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5443fff700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5444065d40 0x7f54440a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f5443fff700 1 -- 172.21.15.116:0/2153871897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54441572d0 con 0x7f5444062880 2020-11-20T12:20:11.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.608+0000 7f54437fe700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5444065d40 0x7f54440a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:11.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.612+0000 7f5443fff700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5444062880 0x7f5444156250 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f543400cb50 tx=0x7f5434008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:11.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.612+0000 7f5423fff700 1 -- 172.21.15.116:0/2153871897 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54340077d0 con 0x7f5444062880 2020-11-20T12:20:11.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.612+0000 7f5423fff700 1 -- 172.21.15.116:0/2153871897 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5434007930 con 0x7f5444062880 2020-11-20T12:20:11.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.612+0000 7f5449214700 1 -- 172.21.15.116:0/2153871897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5444157570 con 0x7f5444062880 2020-11-20T12:20:11.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.612+0000 7f5423fff700 1 -- 172.21.15.116:0/2153871897 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5434007cb0 con 0x7f5444062880 2020-11-20T12:20:11.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.612+0000 7f5449214700 1 -- 172.21.15.116:0/2153871897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54440a4030 con 0x7f5444062880 2020-11-20T12:20:11.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.612+0000 7f5449214700 1 -- 172.21.15.116:0/2153871897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f544405cf30 con 0x7f5444062880 2020-11-20T12:20:11.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.612+0000 7f5423fff700 1 -- 172.21.15.116:0/2153871897 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5434011620 con 0x7f5444062880 2020-11-20T12:20:11.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.612+0000 7f5423fff700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f541c0557c0 0x7f541c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:11.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.616+0000 7f5423fff700 1 -- 172.21.15.116:0/2153871897 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f543406e8a0 con 0x7f5444062880 2020-11-20T12:20:11.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.616+0000 7f54437fe700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f541c0557c0 0x7f541c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:11.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.616+0000 7f54437fe700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f541c0557c0 0x7f541c057c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f5438006ef0 tx=0x7f5438015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:11.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.620+0000 7f5423fff700 1 -- 172.21.15.116:0/2153871897 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5434044740 con 0x7f5444062880 2020-11-20T12:20:11.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.828+0000 7f5449214700 1 -- 172.21.15.116:0/2153871897 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5444061230 con 0x7f5444062880 2020-11-20T12:20:11.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.828+0000 7f5423fff700 1 -- 172.21.15.116:0/2153871897 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2848 (secure 0 0 0) 0x7f5434047d60 con 0x7f5444062880 2020-11-20T12:20:11.826 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:11.826 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/313210621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":313210621},{"type":"v1","addr":"172.21.15.173:6835","nonce":313210621}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/2140340557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2140340557},{"type":"v1","addr":"172.21.15.173:6837","nonce":2140340557}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3544207168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3544207168},{"type":"v1","addr":"172.21.15.116:6833","nonce":3544207168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:57.303415+0000","modified":"2020-11-20T12:19:59.322603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/657163487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":657163487},{"type":"v1","addr":"172.21.15.116:6835","nonce":657163487}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-20T12:20:11.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 -- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f541c0557c0 msgr2=0x7f541c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:11.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f541c0557c0 0x7f541c057c80 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f5438006ef0 tx=0x7f5438015000).stop 2020-11-20T12:20:11.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 -- 172.21.15.116:0/2153871897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5444062880 msgr2=0x7f5444156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:11.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5444062880 0x7f5444156250 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f543400cb50 tx=0x7f5434008040).stop 2020-11-20T12:20:11.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 -- 172.21.15.116:0/2153871897 shutdown_connections 2020-11-20T12:20:11.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f541c0557c0 0x7f541c057c80 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f54440677a0 0x7f5444145c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5444062880 0x7f5444156250 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 --2- 172.21.15.116:0/2153871897 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5444065d40 0x7f54440a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:11.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 -- 172.21.15.116:0/2153871897 >> 172.21.15.116:0/2153871897 conn(0x7f5444060c40 msgr2=0x7f5444154840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:11.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 -- 172.21.15.116:0/2153871897 shutdown_connections 2020-11-20T12:20:11.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:11.832+0000 7f5449214700 1 -- 172.21.15.116:0/2153871897 wait complete. 2020-11-20T12:20:11.832 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 46 2020-11-20T12:20:12.262 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:19:57.303415+0000', 'modified': '2020-11-20T12:19:59.322603+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6861}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6861': {'gid': 6861, 'name': 'a', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/657163487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 657163487}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 657163487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:20:12.262 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:20:12.263 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:20:12.458 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":6861,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.135372290999999,"mdsmap_epoch":46,"osdmap_epoch":86,"osdmap_epoch_barrier":86,"uptime":21.701600478} 2020-11-20T12:20:12.459 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 6861, "mdsmap_epoch": 46, "osdmap_epoch": 86, "osdmap_epoch_barrier": 86, "rank_uptime": 15.135372291, "state": "up:active", "uptime": 21.701600478, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:20:12.460 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:20:12.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:12.462 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.2 2020-11-20T12:20:12.484 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:20:12.484 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:12.560 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:12.560 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:20:12.584 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:12.585 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:20:12.591 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:20:12.592 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:12.592 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2020-11-20T12:20:12.623 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:20:12.623 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:20:12.623 INFO:tasks.cephfs.mount:self.client.name = client.3 2020-11-20T12:20:12.624 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2020-11-20T12:20:12.624 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:20:12.624 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:20:12.625 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:20:12.625 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:20:12.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:12.626 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:20:12.644 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:20:12.645 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:20:12.645 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:20:12.645 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:12.646 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:20:12.646 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:12.646 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:12.646 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:12.646 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:12.647 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:12.647 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:12.647 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:12.647 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:20:12.647 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:12.648 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:20:12.648 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:12.648 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:20:12.648 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:12.648 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:20:12.649 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:12.649 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:20:12.649 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:12.649 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:20:12.650 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:12.650 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:20:12.650 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:12.650 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:20:12.650 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:12.651 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:12.651 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:20:12.651 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:20:12.651 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:12.651 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:12.652 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:20:12.652 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:20:12.652 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:12.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:12.654 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:20:12.735 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:20:12.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:12.739 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:20:12.758 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:20:12.758 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:20:12.759 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:20:12.759 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:20:12.759 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:20:12.759 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:20:12.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:12.761 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:20:12.761 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:20:12.762 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:20:12.762 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:20:12.763 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:20:12.763 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:20:12.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:12.899 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:20:12.918 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:20:12.919 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:20:12.920 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:20:12.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:12.922 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2020-11-20T12:20:13.002 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:13.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.196+0000 7f88957ff700 1 Processor -- start 2020-11-20T12:20:13.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.196+0000 7f88957ff700 1 -- start start 2020-11-20T12:20:13.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.196+0000 7f88957ff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f88900aa570 0x7f88900a7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.196+0000 7f88957ff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88900a7810 0x7f88900a54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.196+0000 7f88957ff700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f889014fbe0 con 0x7f889014f800 2020-11-20T12:20:13.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.196+0000 7f88957ff700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88900a98f0 con 0x7f88900aa570 2020-11-20T12:20:13.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.196+0000 7f88957ff700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88900a9b90 con 0x7f88900a7810 2020-11-20T12:20:13.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.196+0000 7f888effd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f889014f800 0x7f8890065330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58522/0 (socket says 172.21.15.116:58522) 2020-11-20T12:20:13.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.196+0000 7f888effd700 1 -- 172.21.15.116:0/1054526561 learned_addr learned my addr 172.21.15.116:0/1054526561 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:13.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.196+0000 7f888ffff700 1 --2- 172.21.15.116:0/1054526561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88900a7810 0x7f88900a54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.196+0000 7f88867fc700 1 --2- 172.21.15.116:0/1054526561 >> v2:172.21.15.173:3300/0 conn(0x7f88900aa570 0x7f88900a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f888e7fc700 1 -- 172.21.15.116:0/1054526561 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038813471 0 0) 0x7f889014fbe0 con 0x7f889014f800 2020-11-20T12:20:13.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f888e7fc700 1 -- 172.21.15.116:0/1054526561 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8870003240 con 0x7f889014f800 2020-11-20T12:20:13.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f888ffff700 1 -- 172.21.15.116:0/1054526561 >> v2:172.21.15.173:3300/0 conn(0x7f88900aa570 msgr2=0x7f88900a7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:13.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f888ffff700 1 --2- 172.21.15.116:0/1054526561 >> v2:172.21.15.173:3300/0 conn(0x7f88900aa570 0x7f88900a7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f888ffff700 1 -- 172.21.15.116:0/1054526561 >> v1:172.21.15.116:6789/0 conn(0x7f889014f800 legacy=0x7f8890065330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f888ffff700 1 -- 172.21.15.116:0/1054526561 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88900a9e30 con 0x7f88900a7810 2020-11-20T12:20:13.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f88867fc700 1 --2- 172.21.15.116:0/1054526561 >> v2:172.21.15.173:3300/0 conn(0x7f88900aa570 0x7f88900a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:13.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f888ffff700 1 --2- 172.21.15.116:0/1054526561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88900a7810 0x7f88900a54d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f8888009c90 tx=0x7f88880091d0).ready entity=mon.2 client_cookie=239dea509a8d949a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:13.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f888e7fc700 1 -- 172.21.15.116:0/1054526561 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888800b9c0 con 0x7f88900a7810 2020-11-20T12:20:13.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f888e7fc700 1 -- 172.21.15.116:0/1054526561 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888800bb20 con 0x7f88900a7810 2020-11-20T12:20:13.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f888e7fc700 1 -- 172.21.15.116:0/1054526561 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888800bea0 con 0x7f88900a7810 2020-11-20T12:20:13.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f88957ff700 1 -- 172.21.15.116:0/1054526561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88900a7810 msgr2=0x7f88900a54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f88957ff700 1 --2- 172.21.15.116:0/1054526561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88900a7810 0x7f88900a54d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f8888009c90 tx=0x7f88880091d0).stop 2020-11-20T12:20:13.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f88957ff700 1 -- 172.21.15.116:0/1054526561 shutdown_connections 2020-11-20T12:20:13.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f88957ff700 1 --2- 172.21.15.116:0/1054526561 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88900a7810 0x7f88900a54d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f88957ff700 1 --2- 172.21.15.116:0/1054526561 >> v2:172.21.15.173:3300/0 conn(0x7f88900aa570 0x7f88900a7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f88957ff700 1 -- 172.21.15.116:0/1054526561 >> 172.21.15.116:0/1054526561 conn(0x7f88900c63f0 msgr2=0x7f88900c23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:13.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f88957ff700 1 -- 172.21.15.116:0/1054526561 shutdown_connections 2020-11-20T12:20:13.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.200+0000 7f88957ff700 1 -- 172.21.15.116:0/1054526561 wait complete. 2020-11-20T12:20:13.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.204+0000 7f88957ff700 1 Processor -- start 2020-11-20T12:20:13.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.204+0000 7f88957ff700 1 -- start start 2020-11-20T12:20:13.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88957ff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88900a7810 0x7f8890139f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88957ff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88900aa570 0x7f88901393c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88957ff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f889014f800 0x7f8890154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88957ff700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8890150ea0 con 0x7f88900a7810 2020-11-20T12:20:13.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88957ff700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8890151140 con 0x7f88900aa570 2020-11-20T12:20:13.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88957ff700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88901513e0 con 0x7f889014f800 2020-11-20T12:20:13.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f888effd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88900a7810 0x7f8890139f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f888effd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88900a7810 0x7f8890139f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49094/0 (socket says 172.21.15.116:49094) 2020-11-20T12:20:13.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f888effd700 1 -- 172.21.15.116:0/663896409 learned_addr learned my addr 172.21.15.116:0/663896409 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:13.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88867fc700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88900aa570 0x7f88901393c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f888ffff700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f889014f800 0x7f8890154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f888effd700 1 -- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f889014f800 msgr2=0x7f8890154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f888effd700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f889014f800 0x7f8890154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f888effd700 1 -- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88900aa570 msgr2=0x7f88901393c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f888effd700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88900aa570 0x7f88901393c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f888effd700 1 -- 172.21.15.116:0/663896409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8890151680 con 0x7f88900a7810 2020-11-20T12:20:13.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f888ffff700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f889014f800 0x7f8890154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:13.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f888effd700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88900a7810 0x7f8890139f50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f888000c6e0 tx=0x7f8880007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88877fe700 1 -- 172.21.15.116:0/663896409 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880007320 con 0x7f88900a7810 2020-11-20T12:20:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88877fe700 1 -- 172.21.15.116:0/663896409 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8880007480 con 0x7f88900a7810 2020-11-20T12:20:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88957ff700 1 -- 172.21.15.116:0/663896409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88901398b0 con 0x7f88900a7810 2020-11-20T12:20:13.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88877fe700 1 -- 172.21.15.116:0/663896409 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8880005820 con 0x7f88900a7810 2020-11-20T12:20:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.208+0000 7f88957ff700 1 -- 172.21.15.116:0/663896409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88901554f0 con 0x7f88900a7810 2020-11-20T12:20:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.212+0000 7f88957ff700 1 -- 172.21.15.116:0/663896409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8890155770 con 0x7f88900a7810 2020-11-20T12:20:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.212+0000 7f88877fe700 1 -- 172.21.15.116:0/663896409 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8880010030 con 0x7f88900a7810 2020-11-20T12:20:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.212+0000 7f88877fe700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f886c0557c0 0x7f886c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.212+0000 7f88877fe700 1 -- 172.21.15.116:0/663896409 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f888006e3a0 con 0x7f88900a7810 2020-11-20T12:20:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.216+0000 7f88867fc700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f886c0557c0 0x7f886c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.216+0000 7f88867fc700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f886c0557c0 0x7f886c057c80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f8890064640 tx=0x7f8878008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:13.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.220+0000 7f88877fe700 1 -- 172.21.15.116:0/663896409 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8880044130 con 0x7f88900a7810 2020-11-20T12:20:13.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.380+0000 7f88957ff700 1 -- 172.21.15.116:0/663896409 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8890154bf0 con 0x7f88900a7810 2020-11-20T12:20:13.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88877fe700 1 -- 172.21.15.116:0/663896409 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f8890154bf0 con 0x7f88900a7810 2020-11-20T12:20:13.380 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:13.381 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":86,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:58.311412+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:54.974437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:56.326129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:13.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 -- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f886c0557c0 msgr2=0x7f886c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f886c0557c0 0x7f886c057c80 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f8890064640 tx=0x7f8878008040).stop 2020-11-20T12:20:13.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 -- 172.21.15.116:0/663896409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88900a7810 msgr2=0x7f8890139f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88900a7810 0x7f8890139f50 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f888000c6e0 tx=0x7f8880007ac0).stop 2020-11-20T12:20:13.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 -- 172.21.15.116:0/663896409 shutdown_connections 2020-11-20T12:20:13.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f886c0557c0 0x7f886c057c80 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f889014f800 0x7f8890154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88900a7810 0x7f8890139f50 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 --2- 172.21.15.116:0/663896409 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88900aa570 0x7f88901393c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 -- 172.21.15.116:0/663896409 >> 172.21.15.116:0/663896409 conn(0x7f88900c63f0 msgr2=0x7f889014e860 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:13.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 -- 172.21.15.116:0/663896409 shutdown_connections 2020-11-20T12:20:13.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.384+0000 7f88957ff700 1 -- 172.21.15.116:0/663896409 wait complete. 2020-11-20T12:20:13.405 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:19:54.974437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:19:56.326129+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:20:13.406 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:20:13.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.588+0000 7f16af59e700 1 Processor -- start 2020-11-20T12:20:13.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.588+0000 7f16af59e700 1 -- start start 2020-11-20T12:20:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.592+0000 7f16af59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f16b014fcf0 0x7f16b00c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.592+0000 7f16af59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b01500d0 0x7f16b0150530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.592+0000 7f16af59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f16b00a8eb0 0x7f16b00a9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.592+0000 7f16af59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b00a76e0 con 0x7f16b014fcf0 2020-11-20T12:20:13.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.592+0000 7f16af59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b00a4190 con 0x7f16b00a8eb0 2020-11-20T12:20:13.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.592+0000 7f16af59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b0060b80 con 0x7f16b01500d0 2020-11-20T12:20:13.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.592+0000 7f16ae59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f16b014fcf0 0x7f16b00c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.592+0000 7f16ae59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f16b014fcf0 0x7f16b00c0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49104/0 (socket says 172.21.15.116:49104) 2020-11-20T12:20:13.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.592+0000 7f16ae59c700 1 -- 172.21.15.116:0/2853842030 learned_addr learned my addr 172.21.15.116:0/2853842030 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:13.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.592+0000 7f16add9b700 1 --2- 172.21.15.116:0/2853842030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b01500d0 0x7f16b0150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.592+0000 7f16aed9d700 1 --2- 172.21.15.116:0/2853842030 >> v2:172.21.15.173:3300/0 conn(0x7f16b00a8eb0 0x7f16b00a9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16ae59c700 1 -- 172.21.15.116:0/2853842030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b01500d0 msgr2=0x7f16b0150530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:13.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16ae59c700 1 --2- 172.21.15.116:0/2853842030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b01500d0 0x7f16b0150530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16ae59c700 1 -- 172.21.15.116:0/2853842030 >> v2:172.21.15.173:3300/0 conn(0x7f16b00a8eb0 msgr2=0x7f16b00a9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:13.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16ae59c700 1 --2- 172.21.15.116:0/2853842030 >> v2:172.21.15.173:3300/0 conn(0x7f16b00a8eb0 0x7f16b00a9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16ae59c700 1 -- 172.21.15.116:0/2853842030 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b0060e20 con 0x7f16b014fcf0 2020-11-20T12:20:13.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16ae59c700 1 --2- 172.21.15.116:0/2853842030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b014fcf0 0x7f16b00c0e50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f16b00a8b00 tx=0x7f16a0006eb0).ready entity=mon.0 client_cookie=d8373da2c485a87 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:13.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16ad59a700 1 -- 172.21.15.116:0/2853842030 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0009720 con 0x7f16b014fcf0 2020-11-20T12:20:13.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16ad59a700 1 -- 172.21.15.116:0/2853842030 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a0009880 con 0x7f16b014fcf0 2020-11-20T12:20:13.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16ad59a700 1 -- 172.21.15.116:0/2853842030 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0009c00 con 0x7f16b014fcf0 2020-11-20T12:20:13.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16af59e700 1 -- 172.21.15.116:0/2853842030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b014fcf0 msgr2=0x7f16b00c0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16af59e700 1 --2- 172.21.15.116:0/2853842030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b014fcf0 0x7f16b00c0e50 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f16b00a8b00 tx=0x7f16a0006eb0).stop 2020-11-20T12:20:13.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16af59e700 1 -- 172.21.15.116:0/2853842030 shutdown_connections 2020-11-20T12:20:13.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16af59e700 1 --2- 172.21.15.116:0/2853842030 >> v2:172.21.15.173:3300/0 conn(0x7f16b00a8eb0 0x7f16b00a9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16af59e700 1 --2- 172.21.15.116:0/2853842030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b014fcf0 0x7f16b00c0e50 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16af59e700 1 --2- 172.21.15.116:0/2853842030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b01500d0 0x7f16b0150530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16af59e700 1 -- 172.21.15.116:0/2853842030 >> 172.21.15.116:0/2853842030 conn(0x7f16b00c1710 msgr2=0x7f16b00c1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:13.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16af59e700 1 -- 172.21.15.116:0/2853842030 shutdown_connections 2020-11-20T12:20:13.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.596+0000 7f16af59e700 1 -- 172.21.15.116:0/2853842030 wait complete. 2020-11-20T12:20:13.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.600+0000 7f16af59e700 1 Processor -- start 2020-11-20T12:20:13.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.600+0000 7f16af59e700 1 -- start start 2020-11-20T12:20:13.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.600+0000 7f16af59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b00a8eb0 0x7f16b0162280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.600+0000 7f16af59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b014fcf0 0x7f16b00c2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.600+0000 7f16af59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b01500d0 0x7f16b015f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.600+0000 7f16af59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b015ecb0 con 0x7f16b00a8eb0 2020-11-20T12:20:13.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.600+0000 7f16af59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b0162a30 con 0x7f16b01500d0 2020-11-20T12:20:13.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.600+0000 7f16af59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b0162cd0 con 0x7f16b014fcf0 2020-11-20T12:20:13.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16ae59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b00a8eb0 0x7f16b0162280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16ae59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b00a8eb0 0x7f16b0162280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49108/0 (socket says 172.21.15.116:49108) 2020-11-20T12:20:13.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16ae59c700 1 -- 172.21.15.116:0/1179948623 learned_addr learned my addr 172.21.15.116:0/1179948623 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:13.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16add9b700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b014fcf0 0x7f16b00c2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16aed9d700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b01500d0 0x7f16b015f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16ae59c700 1 -- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b014fcf0 msgr2=0x7f16b00c2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16ae59c700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b014fcf0 0x7f16b00c2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16ae59c700 1 -- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b01500d0 msgr2=0x7f16b015f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16ae59c700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b01500d0 0x7f16b015f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16ae59c700 1 -- 172.21.15.116:0/1179948623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b0162f70 con 0x7f16b00a8eb0 2020-11-20T12:20:13.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16ae59c700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b00a8eb0 0x7f16b0162280 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f16b00669d0 tx=0x7f16a0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:13.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f169e7fc700 1 -- 172.21.15.116:0/1179948623 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a0011bc0 con 0x7f16b00a8eb0 2020-11-20T12:20:13.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f169e7fc700 1 -- 172.21.15.116:0/1179948623 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a0011d20 con 0x7f16b00a8eb0 2020-11-20T12:20:13.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16af59e700 1 -- 172.21.15.116:0/1179948623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b015eea0 con 0x7f16b00a8eb0 2020-11-20T12:20:13.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f169e7fc700 1 -- 172.21.15.116:0/1179948623 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a001e670 con 0x7f16b00a8eb0 2020-11-20T12:20:13.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16af59e700 1 -- 172.21.15.116:0/1179948623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b00c3110 con 0x7f16b00a8eb0 2020-11-20T12:20:13.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.604+0000 7f16af59e700 1 -- 172.21.15.116:0/1179948623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b00c3360 con 0x7f16b00a8eb0 2020-11-20T12:20:13.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.608+0000 7f169e7fc700 1 -- 172.21.15.116:0/1179948623 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f16a0025030 con 0x7f16b00a8eb0 2020-11-20T12:20:13.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.608+0000 7f169e7fc700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16840557c0 0x7f1684057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.608+0000 7f169e7fc700 1 -- 172.21.15.116:0/1179948623 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f16a006fbc0 con 0x7f16b00a8eb0 2020-11-20T12:20:13.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.608+0000 7f16add9b700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16840557c0 0x7f1684057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.608+0000 7f16add9b700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16840557c0 0x7f1684057c80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f16b0149e40 tx=0x7f16a4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:13.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.612+0000 7f169e7fc700 1 -- 172.21.15.116:0/1179948623 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f16a0045b10 con 0x7f16b00a8eb0 2020-11-20T12:20:13.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.772+0000 7f16af59e700 1 -- 172.21.15.116:0/1179948623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f16b005cf30 con 0x7f16b00a8eb0 2020-11-20T12:20:13.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.772+0000 7f169e7fc700 1 -- 172.21.15.116:0/1179948623 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v86) v1 ==== 110+0+10 (secure 0 0 0) 0x7f16a0023070 con 0x7f16b00a8eb0 2020-11-20T12:20:13.769 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:20:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.772+0000 7f16af59e700 1 -- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16840557c0 msgr2=0x7f1684057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.776+0000 7f16af59e700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16840557c0 0x7f1684057c80 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f16b0149e40 tx=0x7f16a4008040).stop 2020-11-20T12:20:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.776+0000 7f16af59e700 1 -- 172.21.15.116:0/1179948623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b00a8eb0 msgr2=0x7f16b0162280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.776+0000 7f16af59e700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b00a8eb0 0x7f16b0162280 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f16b00669d0 tx=0x7f16a0015000).stop 2020-11-20T12:20:13.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.776+0000 7f16af59e700 1 -- 172.21.15.116:0/1179948623 shutdown_connections 2020-11-20T12:20:13.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.776+0000 7f16af59e700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16840557c0 0x7f1684057c80 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.776+0000 7f16af59e700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b01500d0 0x7f16b015f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.776+0000 7f16af59e700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b00a8eb0 0x7f16b0162280 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.776+0000 7f16af59e700 1 --2- 172.21.15.116:0/1179948623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b014fcf0 0x7f16b00c2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.776+0000 7f16af59e700 1 -- 172.21.15.116:0/1179948623 >> 172.21.15.116:0/1179948623 conn(0x7f16b00c1710 msgr2=0x7f16b0152540 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:13.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.776+0000 7f16af59e700 1 -- 172.21.15.116:0/1179948623 shutdown_connections 2020-11-20T12:20:13.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.776+0000 7f16af59e700 1 -- 172.21.15.116:0/1179948623 wait complete. 2020-11-20T12:20:13.793 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:20:13.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.988+0000 7f8e2759e700 1 Processor -- start 2020-11-20T12:20:13.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.988+0000 7f8e2759e700 1 -- start start 2020-11-20T12:20:13.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2759e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e280606f0 0x7f8e280bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e280bfaa0 0x7f8e280c1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:13.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2759e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e28066b90 con 0x7f8e28060310 2020-11-20T12:20:13.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2759e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e28066e00 con 0x7f8e280606f0 2020-11-20T12:20:13.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e280670a0 con 0x7f8e280bfaa0 2020-11-20T12:20:13.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e25d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e280606f0 0x7f8e280bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e26d9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e280bfaa0 0x7f8e280c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:13.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2659c700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f8e28060310 0x7f8e28066480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:54780/0 (socket says 172.21.15.116:54780) 2020-11-20T12:20:13.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2659c700 1 -- 172.21.15.116:0/448674839 learned_addr learned my addr 172.21.15.116:0/448674839 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:13.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e25d9b700 1 -- 172.21.15.116:0/448674839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e280bfaa0 msgr2=0x7f8e280c1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:13.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e25d9b700 1 --2- 172.21.15.116:0/448674839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e280bfaa0 0x7f8e280c1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e25d9b700 1 -- 172.21.15.116:0/448674839 >> v1:172.21.15.173:6789/0 conn(0x7f8e28060310 legacy=0x7f8e28066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e25d9b700 1 -- 172.21.15.116:0/448674839 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e28067340 con 0x7f8e280606f0 2020-11-20T12:20:13.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2559a700 1 -- 172.21.15.116:0/448674839 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228446429 0 0) 0x7f8e28066b90 con 0x7f8e28060310 2020-11-20T12:20:13.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e26d9d700 1 --2- 172.21.15.116:0/448674839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e280bfaa0 0x7f8e280c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:13.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e25d9b700 1 --2- 172.21.15.116:0/448674839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e280606f0 0x7f8e280bf5b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f8e1c00b9a0 tx=0x7f8e1c006df0).ready entity=mon.0 client_cookie=56a0ed4ae3f35823 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:13.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2559a700 1 -- 172.21.15.116:0/448674839 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0097b0 con 0x7f8e280606f0 2020-11-20T12:20:13.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2559a700 1 -- 172.21.15.116:0/448674839 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c009910 con 0x7f8e280606f0 2020-11-20T12:20:13.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2559a700 1 -- 172.21.15.116:0/448674839 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c009c90 con 0x7f8e280606f0 2020-11-20T12:20:13.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2759e700 1 -- 172.21.15.116:0/448674839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e280606f0 msgr2=0x7f8e280bf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:13.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.992+0000 7f8e2759e700 1 --2- 172.21.15.116:0/448674839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e280606f0 0x7f8e280bf5b0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f8e1c00b9a0 tx=0x7f8e1c006df0).stop 2020-11-20T12:20:13.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 -- 172.21.15.116:0/448674839 shutdown_connections 2020-11-20T12:20:13.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 --2- 172.21.15.116:0/448674839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e280bfaa0 0x7f8e280c1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 --2- 172.21.15.116:0/448674839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e280606f0 0x7f8e280bf5b0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:13.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 -- 172.21.15.116:0/448674839 >> 172.21.15.116:0/448674839 conn(0x7f8e28068350 msgr2=0x7f8e280627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:13.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 -- 172.21.15.116:0/448674839 shutdown_connections 2020-11-20T12:20:14.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 -- 172.21.15.116:0/448674839 wait complete. 2020-11-20T12:20:14.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 Processor -- start 2020-11-20T12:20:14.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 -- start start 2020-11-20T12:20:14.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e28060310 0x7f8e2814d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e280606f0 0x7f8e2813ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e280bfaa0 0x7f8e281507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2814dc90 con 0x7f8e280606f0 2020-11-20T12:20:14.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2814df30 con 0x7f8e280bfaa0 2020-11-20T12:20:14.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:13.996+0000 7f8e2759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e2814e1d0 con 0x7f8e28060310 2020-11-20T12:20:14.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e25d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e280606f0 0x7f8e2813ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e25d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e280606f0 0x7f8e2813ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49126/0 (socket says 172.21.15.116:49126) 2020-11-20T12:20:14.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e25d9b700 1 -- 172.21.15.116:0/869398726 learned_addr learned my addr 172.21.15.116:0/869398726 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:14.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e26d9d700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e280bfaa0 0x7f8e281507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e2659c700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e28060310 0x7f8e2814d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e25d9b700 1 -- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e28060310 msgr2=0x7f8e2814d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e25d9b700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e28060310 0x7f8e2814d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e25d9b700 1 -- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e280bfaa0 msgr2=0x7f8e281507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e25d9b700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e280bfaa0 0x7f8e281507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e25d9b700 1 -- 172.21.15.116:0/869398726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2814e470 con 0x7f8e280606f0 2020-11-20T12:20:14.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e25d9b700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e280606f0 0x7f8e2813ad20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f8e280c54a0 tx=0x7f8e1c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:14.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e167fc700 1 -- 172.21.15.116:0/869398726 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c011c80 con 0x7f8e280606f0 2020-11-20T12:20:14.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e167fc700 1 -- 172.21.15.116:0/869398726 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c011de0 con 0x7f8e280606f0 2020-11-20T12:20:14.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e2759e700 1 -- 172.21.15.116:0/869398726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2814e710 con 0x7f8e280606f0 2020-11-20T12:20:14.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e167fc700 1 -- 172.21.15.116:0/869398726 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c01e670 con 0x7f8e280606f0 2020-11-20T12:20:14.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e2759e700 1 -- 172.21.15.116:0/869398726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e2813b210 con 0x7f8e280606f0 2020-11-20T12:20:14.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.000+0000 7f8e2759e700 1 -- 172.21.15.116:0/869398726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2814e8a0 con 0x7f8e280606f0 2020-11-20T12:20:14.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.004+0000 7f8e167fc700 1 -- 172.21.15.116:0/869398726 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8e1c025030 con 0x7f8e280606f0 2020-11-20T12:20:14.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.004+0000 7f8e167fc700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dfc0557f0 0x7f8dfc057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.004+0000 7f8e167fc700 1 -- 172.21.15.116:0/869398726 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8e1c06f9c0 con 0x7f8e280606f0 2020-11-20T12:20:14.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.004+0000 7f8e2659c700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dfc0557f0 0x7f8dfc057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.004+0000 7f8e2659c700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dfc0557f0 0x7f8dfc057cb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f8e18006780 tx=0x7f8e18007c20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:14.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.008+0000 7f8e167fc700 1 -- 172.21.15.116:0/869398726 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8e1c045800 con 0x7f8e280606f0 2020-11-20T12:20:14.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 -- 172.21.15.116:0/869398726 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8e2805cf30 con 0x7f8e280606f0 2020-11-20T12:20:14.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e167fc700 1 -- 172.21.15.116:0/869398726 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8e1c023070 con 0x7f8e280606f0 2020-11-20T12:20:14.169 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:14.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 -- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dfc0557f0 msgr2=0x7f8dfc057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dfc0557f0 0x7f8dfc057cb0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f8e18006780 tx=0x7f8e18007c20).stop 2020-11-20T12:20:14.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 -- 172.21.15.116:0/869398726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e280606f0 msgr2=0x7f8e2813ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e280606f0 0x7f8e2813ad20 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f8e280c54a0 tx=0x7f8e1c015000).stop 2020-11-20T12:20:14.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 -- 172.21.15.116:0/869398726 shutdown_connections 2020-11-20T12:20:14.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dfc0557f0 0x7f8dfc057cb0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e280bfaa0 0x7f8e281507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e28060310 0x7f8e2814d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 --2- 172.21.15.116:0/869398726 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e280606f0 0x7f8e2813ad20 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 -- 172.21.15.116:0/869398726 >> 172.21.15.116:0/869398726 conn(0x7f8e28068350 msgr2=0x7f8e280bb780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:14.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 -- 172.21.15.116:0/869398726 shutdown_connections 2020-11-20T12:20:14.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.172+0000 7f8e2759e700 1 -- 172.21.15.116:0/869398726 wait complete. 2020-11-20T12:20:14.195 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:20:14.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.372+0000 7f782ebc9700 1 Processor -- start 2020-11-20T12:20:14.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.372+0000 7f782ebc9700 1 -- start start 2020-11-20T12:20:14.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.376+0000 7f782ebc9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7828155c80 0x7f7828062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.376+0000 7f782ebc9700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7828156060 0x7f78280c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.376+0000 7f782ebc9700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78280b93c0 con 0x7f78280c12c0 2020-11-20T12:20:14.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.376+0000 7f782ebc9700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78280ba580 con 0x7f7828155c80 2020-11-20T12:20:14.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.376+0000 7f782ebc9700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78280b9660 con 0x7f7828156060 2020-11-20T12:20:14.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.376+0000 7f782dbc7700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7828155c80 0x7f7828062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.376+0000 7f782d3c6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7828156060 0x7f78280c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.376+0000 7f782d3c6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7828156060 0x7f78280c0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:48184/0 (socket says 172.21.15.116:48184) 2020-11-20T12:20:14.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.376+0000 7f782d3c6700 1 -- 172.21.15.116:0/3908882677 learned_addr learned my addr 172.21.15.116:0/3908882677 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:14.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782d3c6700 1 -- 172.21.15.116:0/3908882677 >> v2:172.21.15.116:3300/0 conn(0x7f7828155c80 msgr2=0x7f7828062ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:14.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782d3c6700 1 --2- 172.21.15.116:0/3908882677 >> v2:172.21.15.116:3300/0 conn(0x7f7828155c80 0x7f7828062ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782d3c6700 1 -- 172.21.15.116:0/3908882677 >> v1:172.21.15.173:6789/0 conn(0x7f78280c12c0 legacy=0x7f7828060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782d3c6700 1 -- 172.21.15.116:0/3908882677 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280b9900 con 0x7f7828156060 2020-11-20T12:20:14.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782d3c6700 1 --2- 172.21.15.116:0/3908882677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7828156060 0x7f78280c0dd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f7828065d50 tx=0x7f7818006eb0).ready entity=mon.1 client_cookie=2edadeda8b4eee05 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:14.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782cbc5700 1 -- 172.21.15.116:0/3908882677 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818009810 con 0x7f7828156060 2020-11-20T12:20:14.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782cbc5700 1 -- 172.21.15.116:0/3908882677 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7818009970 con 0x7f7828156060 2020-11-20T12:20:14.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782cbc5700 1 -- 172.21.15.116:0/3908882677 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818009cf0 con 0x7f7828156060 2020-11-20T12:20:14.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782ebc9700 1 -- 172.21.15.116:0/3908882677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7828156060 msgr2=0x7f78280c0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782ebc9700 1 --2- 172.21.15.116:0/3908882677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7828156060 0x7f78280c0dd0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f7828065d50 tx=0x7f7818006eb0).stop 2020-11-20T12:20:14.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782ebc9700 1 -- 172.21.15.116:0/3908882677 shutdown_connections 2020-11-20T12:20:14.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782ebc9700 1 --2- 172.21.15.116:0/3908882677 >> v2:172.21.15.116:3300/0 conn(0x7f7828155c80 0x7f7828062ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782ebc9700 1 --2- 172.21.15.116:0/3908882677 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7828156060 0x7f78280c0dd0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782ebc9700 1 -- 172.21.15.116:0/3908882677 >> 172.21.15.116:0/3908882677 conn(0x7f7828068350 msgr2=0x7f78280627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:14.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782ebc9700 1 -- 172.21.15.116:0/3908882677 shutdown_connections 2020-11-20T12:20:14.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.380+0000 7f782ebc9700 1 -- 172.21.15.116:0/3908882677 wait complete. 2020-11-20T12:20:14.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.384+0000 7f782ebc9700 1 Processor -- start 2020-11-20T12:20:14.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.384+0000 7f782ebc9700 1 -- start start 2020-11-20T12:20:14.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.384+0000 7f782ebc9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78280c12c0 0x7f7828165550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.384+0000 7f782ebc9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7828155c80 0x7f78280a94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.384+0000 7f782ebc9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7828156060 0x7f7828150e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.384+0000 7f782ebc9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78280a8270 con 0x7f7828155c80 2020-11-20T12:20:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.384+0000 7f782ebc9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78280aa550 con 0x7f78280c12c0 2020-11-20T12:20:14.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.384+0000 7f782ebc9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78280a9d70 con 0x7f7828156060 2020-11-20T12:20:14.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782d3c6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7828155c80 0x7f78280a94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782d3c6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7828155c80 0x7f78280a94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49138/0 (socket says 172.21.15.116:49138) 2020-11-20T12:20:14.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782d3c6700 1 -- 172.21.15.116:0/1095012941 learned_addr learned my addr 172.21.15.116:0/1095012941 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:14.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782dbc7700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78280c12c0 0x7f7828165550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782e3c8700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7828156060 0x7f7828150e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782d3c6700 1 -- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7828156060 msgr2=0x7f7828150e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782d3c6700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7828156060 0x7f7828150e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782d3c6700 1 -- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78280c12c0 msgr2=0x7f7828165550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782d3c6700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78280c12c0 0x7f7828165550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782d3c6700 1 -- 172.21.15.116:0/1095012941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78280aa010 con 0x7f7828155c80 2020-11-20T12:20:14.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782d3c6700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7828155c80 0x7f78280a94a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f78280b97a0 tx=0x7f7818015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:14.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f7815ffb700 1 -- 172.21.15.116:0/1095012941 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7818009810 con 0x7f7828155c80 2020-11-20T12:20:14.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f7815ffb700 1 -- 172.21.15.116:0/1095012941 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7818006760 con 0x7f7828155c80 2020-11-20T12:20:14.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f7815ffb700 1 -- 172.21.15.116:0/1095012941 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f781801e460 con 0x7f7828155c80 2020-11-20T12:20:14.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782ebc9700 1 -- 172.21.15.116:0/1095012941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78280aa2b0 con 0x7f7828155c80 2020-11-20T12:20:14.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.388+0000 7f782ebc9700 1 -- 172.21.15.116:0/1095012941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78280a9990 con 0x7f7828155c80 2020-11-20T12:20:14.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.392+0000 7f782ebc9700 1 -- 172.21.15.116:0/1095012941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f782805cf30 con 0x7f7828155c80 2020-11-20T12:20:14.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.392+0000 7f7815ffb700 1 -- 172.21.15.116:0/1095012941 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7818025030 con 0x7f7828155c80 2020-11-20T12:20:14.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.392+0000 7f7815ffb700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78000557c0 0x7f7800057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.392+0000 7f7815ffb700 1 -- 172.21.15.116:0/1095012941 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f781806fcc0 con 0x7f7828155c80 2020-11-20T12:20:14.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.392+0000 7f782dbc7700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78000557c0 0x7f7800057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.396+0000 7f782dbc7700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78000557c0 0x7f7800057c80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f7824006fd0 tx=0x7f7824006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:14.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.400+0000 7f7815ffb700 1 -- 172.21.15.116:0/1095012941 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7818076080 con 0x7f7828155c80 2020-11-20T12:20:14.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.560+0000 7f782ebc9700 1 -- 172.21.15.116:0/1095012941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78280639c0 con 0x7f7828155c80 2020-11-20T12:20:14.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f7815ffb700 1 -- 172.21.15.116:0/1095012941 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7818023020 con 0x7f7828155c80 2020-11-20T12:20:14.560 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:14.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 -- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78000557c0 msgr2=0x7f7800057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78000557c0 0x7f7800057c80 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f7824006fd0 tx=0x7f7824006df0).stop 2020-11-20T12:20:14.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 -- 172.21.15.116:0/1095012941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7828155c80 msgr2=0x7f78280a94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7828155c80 0x7f78280a94a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f78280b97a0 tx=0x7f7818015000).stop 2020-11-20T12:20:14.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 -- 172.21.15.116:0/1095012941 shutdown_connections 2020-11-20T12:20:14.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f78000557c0 0x7f7800057c80 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7828156060 0x7f7828150e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f78280c12c0 0x7f7828165550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 --2- 172.21.15.116:0/1095012941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7828155c80 0x7f78280a94a0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 -- 172.21.15.116:0/1095012941 >> 172.21.15.116:0/1095012941 conn(0x7f7828068350 msgr2=0x7f78280c2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:14.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 -- 172.21.15.116:0/1095012941 shutdown_connections 2020-11-20T12:20:14.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.564+0000 7f782ebc9700 1 -- 172.21.15.116:0/1095012941 wait complete. 2020-11-20T12:20:14.587 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:20:14.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.768+0000 7f518f5c5700 1 Processor -- start 2020-11-20T12:20:14.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.768+0000 7f518f5c5700 1 -- start start 2020-11-20T12:20:14.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f518f5c5700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f51880606f0 0x7f51880aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f518f5c5700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51880a67e0 con 0x7f5188066d90 2020-11-20T12:20:14.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f518f5c5700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51880a5fd0 con 0x7f5188060310 2020-11-20T12:20:14.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f518f5c5700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51880a6270 con 0x7f51880606f0 2020-11-20T12:20:14.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f518db62700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5188066d90 0x7f5188159af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58580/0 (socket says 172.21.15.116:58580) 2020-11-20T12:20:14.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f518db62700 1 -- 172.21.15.116:0/3001732638 learned_addr learned my addr 172.21.15.116:0/3001732638 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:14.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f518cb60700 1 --2- 172.21.15.116:0/3001732638 >> v2:172.21.15.116:3300/0 conn(0x7f51880606f0 0x7f51880aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f5177fff700 1 -- 172.21.15.116:0/3001732638 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747438054 0 0) 0x7f51880a67e0 con 0x7f5188066d90 2020-11-20T12:20:14.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f5177fff700 1 -- 172.21.15.116:0/3001732638 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5170003240 con 0x7f5188066d90 2020-11-20T12:20:14.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f5177fff700 1 -- 172.21.15.116:0/3001732638 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1641609117 0 0) 0x7f51880a5fd0 con 0x7f5188060310 2020-11-20T12:20:14.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f5177fff700 1 -- 172.21.15.116:0/3001732638 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51880a67e0 con 0x7f5188060310 2020-11-20T12:20:14.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f518cb60700 1 -- 172.21.15.116:0/3001732638 >> v1:172.21.15.173:6789/0 conn(0x7f5188060310 legacy=0x7f5188066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f518cb60700 1 -- 172.21.15.116:0/3001732638 >> v1:172.21.15.116:6789/0 conn(0x7f5188066d90 legacy=0x7f5188159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.772+0000 7f518cb60700 1 -- 172.21.15.116:0/3001732638 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51880a6a40 con 0x7f51880606f0 2020-11-20T12:20:14.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.776+0000 7f518cb60700 1 --2- 172.21.15.116:0/3001732638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51880606f0 0x7f51880aa0d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f517800b9a0 tx=0x7f5178006df0).ready entity=mon.0 client_cookie=a6677bb10e380c0b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:14.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.776+0000 7f5177fff700 1 -- 172.21.15.116:0/3001732638 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178009680 con 0x7f51880606f0 2020-11-20T12:20:14.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.776+0000 7f5177fff700 1 -- 172.21.15.116:0/3001732638 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51780097e0 con 0x7f51880606f0 2020-11-20T12:20:14.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.776+0000 7f5177fff700 1 -- 172.21.15.116:0/3001732638 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178009b60 con 0x7f51880606f0 2020-11-20T12:20:14.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.776+0000 7f518f5c5700 1 -- 172.21.15.116:0/3001732638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51880606f0 msgr2=0x7f51880aa0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.776+0000 7f518f5c5700 1 --2- 172.21.15.116:0/3001732638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51880606f0 0x7f51880aa0d0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7f517800b9a0 tx=0x7f5178006df0).stop 2020-11-20T12:20:14.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.776+0000 7f518f5c5700 1 -- 172.21.15.116:0/3001732638 shutdown_connections 2020-11-20T12:20:14.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.776+0000 7f518f5c5700 1 --2- 172.21.15.116:0/3001732638 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f51880606f0 0x7f51880aa0d0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.776+0000 7f518f5c5700 1 -- 172.21.15.116:0/3001732638 >> 172.21.15.116:0/3001732638 conn(0x7f518805ba80 msgr2=0x7f51880c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:14.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.776+0000 7f518f5c5700 1 -- 172.21.15.116:0/3001732638 shutdown_connections 2020-11-20T12:20:14.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.776+0000 7f518f5c5700 1 -- 172.21.15.116:0/3001732638 wait complete. 2020-11-20T12:20:14.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.780+0000 7f518f5c5700 1 Processor -- start 2020-11-20T12:20:14.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.780+0000 7f518f5c5700 1 -- start start 2020-11-20T12:20:14.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.780+0000 7f518f5c5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5188060310 0x7f51880b9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.780+0000 7f518f5c5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51880606f0 0x7f51880c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.780+0000 7f518f5c5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5188066d90 0x7f51881655e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.780+0000 7f518f5c5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51880ba840 con 0x7f5188066d90 2020-11-20T12:20:14.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.780+0000 7f518f5c5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51880baae0 con 0x7f51880606f0 2020-11-20T12:20:14.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.780+0000 7f518f5c5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51880bad80 con 0x7f5188060310 2020-11-20T12:20:14.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.780+0000 7f518db62700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5188066d90 0x7f51881655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518db62700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5188066d90 0x7f51881655e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49154/0 (socket says 172.21.15.116:49154) 2020-11-20T12:20:14.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518db62700 1 -- 172.21.15.116:0/3796060678 learned_addr learned my addr 172.21.15.116:0/3796060678 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:14.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518d361700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5188060310 0x7f51880b9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518cb60700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51880606f0 0x7f51880c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518db62700 1 -- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5188060310 msgr2=0x7f51880b9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518db62700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5188060310 0x7f51880b9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518db62700 1 -- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51880606f0 msgr2=0x7f51880c3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518db62700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51880606f0 0x7f51880c3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518db62700 1 -- 172.21.15.116:0/3796060678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5188168be0 con 0x7f5188066d90 2020-11-20T12:20:14.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518cb60700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51880606f0 0x7f51880c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:14.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518db62700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5188066d90 0x7f51881655e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f518000ca60 tx=0x7f5180007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:14.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f51757fa700 1 -- 172.21.15.116:0/3796060678 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180005b80 con 0x7f5188066d90 2020-11-20T12:20:14.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f51757fa700 1 -- 172.21.15.116:0/3796060678 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5180005ce0 con 0x7f5188066d90 2020-11-20T12:20:14.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518f5c5700 1 -- 172.21.15.116:0/3796060678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5188168e80 con 0x7f5188066d90 2020-11-20T12:20:14.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f51757fa700 1 -- 172.21.15.116:0/3796060678 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5180007320 con 0x7f5188066d90 2020-11-20T12:20:14.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518f5c5700 1 -- 172.21.15.116:0/3796060678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51880c41c0 con 0x7f5188066d90 2020-11-20T12:20:14.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.784+0000 7f518f5c5700 1 -- 172.21.15.116:0/3796060678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5188169010 con 0x7f5188066d90 2020-11-20T12:20:14.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.788+0000 7f51757fa700 1 -- 172.21.15.116:0/3796060678 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5180010030 con 0x7f5188066d90 2020-11-20T12:20:14.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.788+0000 7f51757fa700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51600557f0 0x7f5160057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:14.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.788+0000 7f51757fa700 1 -- 172.21.15.116:0/3796060678 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f518006e6a0 con 0x7f5188066d90 2020-11-20T12:20:14.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.788+0000 7f518d361700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51600557f0 0x7f5160057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:14.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.788+0000 7f518d361700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51600557f0 0x7f5160057cb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f5184003740 tx=0x7f5184007a20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:14.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.792+0000 7f51757fa700 1 -- 172.21.15.116:0/3796060678 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5180044430 con 0x7f5188066d90 2020-11-20T12:20:14.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.952+0000 7f518f5c5700 1 -- 172.21.15.116:0/3796060678 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f518805d2a0 con 0x7f5188066d90 2020-11-20T12:20:14.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.952+0000 7f51757fa700 1 -- 172.21.15.116:0/3796060678 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+10 (secure 0 0 0) 0x7f5180047a50 con 0x7f5188066d90 2020-11-20T12:20:14.950 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:14.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.952+0000 7f518f5c5700 1 -- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51600557f0 msgr2=0x7f5160057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.952+0000 7f518f5c5700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51600557f0 0x7f5160057cb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f5184003740 tx=0x7f5184007a20).stop 2020-11-20T12:20:14.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.952+0000 7f518f5c5700 1 -- 172.21.15.116:0/3796060678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5188066d90 msgr2=0x7f51881655e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:14.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.952+0000 7f518f5c5700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5188066d90 0x7f51881655e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f518000ca60 tx=0x7f5180007e20).stop 2020-11-20T12:20:14.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.956+0000 7f518f5c5700 1 -- 172.21.15.116:0/3796060678 shutdown_connections 2020-11-20T12:20:14.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.956+0000 7f518f5c5700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51600557f0 0x7f5160057cb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.956+0000 7f518f5c5700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5188066d90 0x7f51881655e0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.956+0000 7f518f5c5700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5188060310 0x7f51880b9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.956+0000 7f518f5c5700 1 --2- 172.21.15.116:0/3796060678 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f51880606f0 0x7f51880c3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:14.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.956+0000 7f518f5c5700 1 -- 172.21.15.116:0/3796060678 >> 172.21.15.116:0/3796060678 conn(0x7f518805ba80 msgr2=0x7f51880be700 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:14.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.956+0000 7f518f5c5700 1 -- 172.21.15.116:0/3796060678 shutdown_connections 2020-11-20T12:20:14.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:14.956+0000 7f518f5c5700 1 -- 172.21.15.116:0/3796060678 wait complete. 2020-11-20T12:20:14.974 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:20:14.975 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:15.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.148+0000 7fc0e2538700 1 Processor -- start 2020-11-20T12:20:15.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.152+0000 7fc0e2538700 1 -- start start 2020-11-20T12:20:15.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.152+0000 7fc0e2538700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc0dc13afa0 0x7fc0dc0aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:15.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.152+0000 7fc0e2538700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0dc0ada00 con 0x7fc0dc0ad440 2020-11-20T12:20:15.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.152+0000 7fc0e2538700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0dc0adca0 con 0x7fc0dc13b380 2020-11-20T12:20:15.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.152+0000 7fc0e2538700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0dc0adf40 con 0x7fc0dc13afa0 2020-11-20T12:20:15.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e1d37700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc0dc0ad440 0x7fc0dc0a8790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58596/0 (socket says 172.21.15.116:58596) 2020-11-20T12:20:15.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e1d37700 1 -- 172.21.15.116:0/358041915 learned_addr learned my addr 172.21.15.116:0/358041915 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:15.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e1536700 1 --2- 172.21.15.116:0/358041915 >> v2:172.21.15.116:3300/0 conn(0x7fc0dc13afa0 0x7fc0dc0aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:15.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0cbfff700 1 -- 172.21.15.116:0/358041915 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609055480 0 0) 0x7fc0dc0ada00 con 0x7fc0dc0ad440 2020-11-20T12:20:15.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0cbfff700 1 -- 172.21.15.116:0/358041915 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0c4003240 con 0x7fc0dc0ad440 2020-11-20T12:20:15.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0cbfff700 1 -- 172.21.15.116:0/358041915 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1273426143 0 0) 0x7fc0dc0adca0 con 0x7fc0dc13b380 2020-11-20T12:20:15.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0cbfff700 1 -- 172.21.15.116:0/358041915 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0dc0ada00 con 0x7fc0dc13b380 2020-11-20T12:20:15.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e1536700 1 -- 172.21.15.116:0/358041915 >> v1:172.21.15.173:6789/0 conn(0x7fc0dc13b380 legacy=0x7fc0dc0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:15.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e1536700 1 -- 172.21.15.116:0/358041915 >> v1:172.21.15.116:6789/0 conn(0x7fc0dc0ad440 legacy=0x7fc0dc0a8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:15.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e1536700 1 -- 172.21.15.116:0/358041915 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc0ae1e0 con 0x7fc0dc13afa0 2020-11-20T12:20:15.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e1536700 1 --2- 172.21.15.116:0/358041915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0dc13afa0 0x7fc0dc0aa5b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7fc0d800b9a0 tx=0x7fc0d8006df0).ready entity=mon.0 client_cookie=3c80cdbb66832c0f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:15.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0cbfff700 1 -- 172.21.15.116:0/358041915 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8009680 con 0x7fc0dc13afa0 2020-11-20T12:20:15.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0cbfff700 1 -- 172.21.15.116:0/358041915 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d80097e0 con 0x7fc0dc13afa0 2020-11-20T12:20:15.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0cbfff700 1 -- 172.21.15.116:0/358041915 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8009b60 con 0x7fc0dc13afa0 2020-11-20T12:20:15.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e2538700 1 -- 172.21.15.116:0/358041915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0dc13afa0 msgr2=0x7fc0dc0aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:15.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e2538700 1 --2- 172.21.15.116:0/358041915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0dc13afa0 0x7fc0dc0aa5b0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7fc0d800b9a0 tx=0x7fc0d8006df0).stop 2020-11-20T12:20:15.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e2538700 1 -- 172.21.15.116:0/358041915 shutdown_connections 2020-11-20T12:20:15.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e2538700 1 --2- 172.21.15.116:0/358041915 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0dc13afa0 0x7fc0dc0aa5b0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:15.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.156+0000 7fc0e2538700 1 -- 172.21.15.116:0/358041915 >> 172.21.15.116:0/358041915 conn(0x7fc0dc0678a0 msgr2=0x7fc0dc067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:15.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.160+0000 7fc0e2538700 1 -- 172.21.15.116:0/358041915 shutdown_connections 2020-11-20T12:20:15.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.160+0000 7fc0e2538700 1 -- 172.21.15.116:0/358041915 wait complete. 2020-11-20T12:20:15.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e2538700 1 Processor -- start 2020-11-20T12:20:15.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e2538700 1 -- start start 2020-11-20T12:20:15.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e2538700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0dc0ad440 0x7fc0dc151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:15.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e2538700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0dc13b380 0x7fc0dc1511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:15.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e2538700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0dc0a2d50 0x7fc0dc0a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:15.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e2538700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc0a4000 con 0x7fc0dc0ad440 2020-11-20T12:20:15.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e2538700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc0a3820 con 0x7fc0dc13b380 2020-11-20T12:20:15.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e2538700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0dc0a3ac0 con 0x7fc0dc0a2d50 2020-11-20T12:20:15.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e1536700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0dc0ad440 0x7fc0dc151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:15.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e1536700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0dc0ad440 0x7fc0dc151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49164/0 (socket says 172.21.15.116:49164) 2020-11-20T12:20:15.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e1536700 1 -- 172.21.15.116:0/2119240979 learned_addr learned my addr 172.21.15.116:0/2119240979 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:15.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e1d37700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0dc0a2d50 0x7fc0dc0a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:15.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e0d35700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0dc13b380 0x7fc0dc1511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:15.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e1536700 1 -- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0dc0a2d50 msgr2=0x7fc0dc0a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:15.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e1536700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0dc0a2d50 0x7fc0dc0a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:15.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e1536700 1 -- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0dc13b380 msgr2=0x7fc0dc1511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:15.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e1536700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0dc13b380 0x7fc0dc1511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:15.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e1536700 1 -- 172.21.15.116:0/2119240979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0dc0a3d60 con 0x7fc0dc0ad440 2020-11-20T12:20:15.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.164+0000 7fc0e1536700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0dc0ad440 0x7fc0dc151ee0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7fc0d80097b0 tx=0x7fc0d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:15.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.168+0000 7fc0c97fa700 1 -- 172.21.15.116:0/2119240979 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d8021070 con 0x7fc0dc0ad440 2020-11-20T12:20:15.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.168+0000 7fc0e2538700 1 -- 172.21.15.116:0/2119240979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0dc09f4a0 con 0x7fc0dc0ad440 2020-11-20T12:20:15.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.168+0000 7fc0c97fa700 1 -- 172.21.15.116:0/2119240979 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d8011c70 con 0x7fc0dc0ad440 2020-11-20T12:20:15.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.168+0000 7fc0c97fa700 1 -- 172.21.15.116:0/2119240979 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d801d450 con 0x7fc0dc0ad440 2020-11-20T12:20:15.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.168+0000 7fc0e2538700 1 -- 172.21.15.116:0/2119240979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0dc1516b0 con 0x7fc0dc0ad440 2020-11-20T12:20:15.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.168+0000 7fc0e2538700 1 -- 172.21.15.116:0/2119240979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0dc09f630 con 0x7fc0dc0ad440 2020-11-20T12:20:15.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.172+0000 7fc0c97fa700 1 -- 172.21.15.116:0/2119240979 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc0d800f070 con 0x7fc0dc0ad440 2020-11-20T12:20:15.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.172+0000 7fc0c97fa700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0b40557c0 0x7fc0b4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:15.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.172+0000 7fc0c97fa700 1 -- 172.21.15.116:0/2119240979 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc0d806fe40 con 0x7fc0dc0ad440 2020-11-20T12:20:15.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.172+0000 7fc0e0d35700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0b40557c0 0x7fc0b4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:15.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.176+0000 7fc0e0d35700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0b40557c0 0x7fc0b4057c80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc0dc154e80 tx=0x7fc0cc007a20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:15.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.176+0000 7fc0c97fa700 1 -- 172.21.15.116:0/2119240979 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc0d8045ce0 con 0x7fc0dc0ad440 2020-11-20T12:20:15.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.380+0000 7fc0e2538700 1 -- 172.21.15.116:0/2119240979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc0dc05cf30 con 0x7fc0dc0ad440 2020-11-20T12:20:15.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.380+0000 7fc0c97fa700 1 -- 172.21.15.116:0/2119240979 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2848 (secure 0 0 0) 0x7fc0d802c020 con 0x7fc0dc0ad440 2020-11-20T12:20:15.379 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:15.379 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/313210621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":313210621},{"type":"v1","addr":"172.21.15.173:6835","nonce":313210621}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/2140340557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2140340557},{"type":"v1","addr":"172.21.15.173:6837","nonce":2140340557}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3544207168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3544207168},{"type":"v1","addr":"172.21.15.116:6833","nonce":3544207168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:57.303415+0000","modified":"2020-11-20T12:19:59.322603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/657163487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":657163487},{"type":"v1","addr":"172.21.15.116:6835","nonce":657163487}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-20T12:20:15.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 -- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0b40557c0 msgr2=0x7fc0b4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:15.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0b40557c0 0x7fc0b4057c80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc0dc154e80 tx=0x7fc0cc007a20).stop 2020-11-20T12:20:15.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 -- 172.21.15.116:0/2119240979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0dc0ad440 msgr2=0x7fc0dc151ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:15.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0dc0ad440 0x7fc0dc151ee0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7fc0d80097b0 tx=0x7fc0d8015000).stop 2020-11-20T12:20:15.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 -- 172.21.15.116:0/2119240979 shutdown_connections 2020-11-20T12:20:15.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc0b40557c0 0x7fc0b4057c80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:15.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc0dc0a2d50 0x7fc0dc0a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:15.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc0dc0ad440 0x7fc0dc151ee0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:15.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 --2- 172.21.15.116:0/2119240979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc0dc13b380 0x7fc0dc1511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:15.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 -- 172.21.15.116:0/2119240979 >> 172.21.15.116:0/2119240979 conn(0x7fc0dc0678a0 msgr2=0x7fc0dc0a9ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:15.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 -- 172.21.15.116:0/2119240979 shutdown_connections 2020-11-20T12:20:15.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:15.384+0000 7fc0e2538700 1 -- 172.21.15.116:0/2119240979 wait complete. 2020-11-20T12:20:15.384 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 46 2020-11-20T12:20:15.807 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:19:57.303415+0000', 'modified': '2020-11-20T12:19:59.322603+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6861}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6861': {'gid': 6861, 'name': 'a', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/657163487', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6834', 'nonce': 657163487}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 657163487}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:20:15.808 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:20:15.809 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:20:16.007 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":6861,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.683736921000001,"mdsmap_epoch":46,"osdmap_epoch":86,"osdmap_epoch_barrier":86,"uptime":25.249967040000001} 2020-11-20T12:20:16.008 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 6861, "mdsmap_epoch": 46, "osdmap_epoch": 86, "osdmap_epoch_barrier": 86, "rank_uptime": 18.683736921, "state": "up:active", "uptime": 25.24996704, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:20:16.008 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:20:16.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:16.009 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.3 2020-11-20T12:20:16.028 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:20:16.029 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:16.145 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:16.145 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:20:16.171 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:16.171 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:20:16.179 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:20:16.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:16.180 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2020-11-20T12:20:16.209 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-20T12:20:16.210 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:20:16.210 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 2020-11-20T12:20:16.283 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:20:16.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:16.284 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T12:20:16.303 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.manila mds 'allow *' osd 'allow rw' mon 'allow *' 2020-11-20T12:20:16.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.492+0000 7f39918ba700 1 Processor -- start 2020-11-20T12:20:16.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.492+0000 7f39918ba700 1 -- start start 2020-11-20T12:20:16.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f39918ba700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c13b0d0 0x7f398c0a8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:16.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f39918ba700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f398c151ee0 0x7f398c152390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:16.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f39918ba700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c1557f0 con 0x7f398c13b4b0 2020-11-20T12:20:16.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f39918ba700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c152880 con 0x7f398c151ee0 2020-11-20T12:20:16.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f39918ba700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c152b10 con 0x7f398c13b0d0 2020-11-20T12:20:16.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f398b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f398c151ee0 0x7f398c152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:16.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f398b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f398c151ee0 0x7f398c152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49176/0 (socket says 172.21.15.116:49176) 2020-11-20T12:20:16.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f398b7fe700 1 -- 172.21.15.116:0/3749643121 learned_addr learned my addr 172.21.15.116:0/3749643121 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:16.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f398affd700 1 --2- 172.21.15.116:0/3749643121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c13b0d0 0x7f398c0a8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:16.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f398b7fe700 1 -- 172.21.15.116:0/3749643121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c13b0d0 msgr2=0x7f398c0a8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:16.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f398b7fe700 1 --2- 172.21.15.116:0/3749643121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c13b0d0 0x7f398c0a8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:16.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f398b7fe700 1 -- 172.21.15.116:0/3749643121 >> v1:172.21.15.173:6789/0 conn(0x7f398c13b4b0 legacy=0x7f398c0aa7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:16.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.496+0000 7f398b7fe700 1 -- 172.21.15.116:0/3749643121 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c152db0 con 0x7f398c151ee0 2020-11-20T12:20:16.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f398b7fe700 1 --2- 172.21.15.116:0/3749643121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f398c151ee0 0x7f398c152390 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f3984009d50 tx=0x7f3984009290).ready entity=mon.0 client_cookie=18404ab368cfccea server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:16.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f3989ffb700 1 -- 172.21.15.116:0/3749643121 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398400bc80 con 0x7f398c151ee0 2020-11-20T12:20:16.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f3989ffb700 1 -- 172.21.15.116:0/3749643121 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f398400bde0 con 0x7f398c151ee0 2020-11-20T12:20:16.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f3989ffb700 1 -- 172.21.15.116:0/3749643121 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984011610 con 0x7f398c151ee0 2020-11-20T12:20:16.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f39918ba700 1 -- 172.21.15.116:0/3749643121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f398c151ee0 msgr2=0x7f398c152390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:16.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f39918ba700 1 --2- 172.21.15.116:0/3749643121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f398c151ee0 0x7f398c152390 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f3984009d50 tx=0x7f3984009290).stop 2020-11-20T12:20:16.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f39918ba700 1 -- 172.21.15.116:0/3749643121 shutdown_connections 2020-11-20T12:20:16.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f39918ba700 1 --2- 172.21.15.116:0/3749643121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f398c151ee0 0x7f398c152390 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:16.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f39918ba700 1 --2- 172.21.15.116:0/3749643121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c13b0d0 0x7f398c0a8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:16.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f39918ba700 1 -- 172.21.15.116:0/3749643121 >> 172.21.15.116:0/3749643121 conn(0x7f398c0678a0 msgr2=0x7f398c067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:16.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f39918ba700 1 -- 172.21.15.116:0/3749643121 shutdown_connections 2020-11-20T12:20:16.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f39918ba700 1 -- 172.21.15.116:0/3749643121 wait complete. 2020-11-20T12:20:16.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.500+0000 7f39918ba700 1 Processor -- start 2020-11-20T12:20:16.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f39918ba700 1 -- start start 2020-11-20T12:20:16.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f39918ba700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c13b0d0 0x7f398c160e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:16.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f39918ba700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f398c13b4b0 0x7f398c1512f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:16.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f39918ba700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f398c151ee0 0x7f398c0c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:16.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f39918ba700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c1616f0 con 0x7f398c151ee0 2020-11-20T12:20:16.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f39918ba700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c161990 con 0x7f398c13b4b0 2020-11-20T12:20:16.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f39918ba700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c161c30 con 0x7f398c13b0d0 2020-11-20T12:20:16.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f398b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f398c151ee0 0x7f398c0c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:16.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f398b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f398c151ee0 0x7f398c0c11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49182/0 (socket says 172.21.15.116:49182) 2020-11-20T12:20:16.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f398b7fe700 1 -- 172.21.15.116:0/1105422340 learned_addr learned my addr 172.21.15.116:0/1105422340 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:16.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f398affd700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c13b0d0 0x7f398c160e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:16.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f398a7fc700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f398c13b4b0 0x7f398c1512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:16.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f398b7fe700 1 -- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c13b0d0 msgr2=0x7f398c160e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:16.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f398b7fe700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c13b0d0 0x7f398c160e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:16.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f398b7fe700 1 -- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f398c13b4b0 msgr2=0x7f398c1512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:16.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f398b7fe700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f398c13b4b0 0x7f398c1512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:16.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f398b7fe700 1 -- 172.21.15.116:0/1105422340 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0bf8b0 con 0x7f398c151ee0 2020-11-20T12:20:16.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f398b7fe700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f398c151ee0 0x7f398c0c11c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f398c081a20 tx=0x7f3984016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:16.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f3972ffd700 1 -- 172.21.15.116:0/1105422340 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f398400bc80 con 0x7f398c151ee0 2020-11-20T12:20:16.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f3972ffd700 1 -- 172.21.15.116:0/1105422340 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3984009e90 con 0x7f398c151ee0 2020-11-20T12:20:16.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f39918ba700 1 -- 172.21.15.116:0/1105422340 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c161380 con 0x7f398c151ee0 2020-11-20T12:20:16.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f3972ffd700 1 -- 172.21.15.116:0/1105422340 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984010d80 con 0x7f398c151ee0 2020-11-20T12:20:16.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.504+0000 7f39918ba700 1 -- 172.21.15.116:0/1105422340 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c0c16b0 con 0x7f398c151ee0 2020-11-20T12:20:16.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.508+0000 7f39918ba700 1 -- 172.21.15.116:0/1105422340 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c0c1930 con 0x7f398c151ee0 2020-11-20T12:20:16.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.508+0000 7f3972ffd700 1 -- 172.21.15.116:0/1105422340 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3984017070 con 0x7f398c151ee0 2020-11-20T12:20:16.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.508+0000 7f3972ffd700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f39680557c0 0x7f3968057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:16.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.508+0000 7f3972ffd700 1 -- 172.21.15.116:0/1105422340 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3984070620 con 0x7f398c151ee0 2020-11-20T12:20:16.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.512+0000 7f398affd700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f39680557c0 0x7f3968057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:16.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.512+0000 7f398affd700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f39680557c0 0x7f3968057c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f397c006fd0 tx=0x7f397c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:16.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.512+0000 7f3972ffd700 1 -- 172.21.15.116:0/1105422340 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f39840464c0 con 0x7f398c151ee0 2020-11-20T12:20:16.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.728+0000 7f39918ba700 1 -- 172.21.15.116:0/1105422340 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]} v 0) v1 -- 0x7f398c0bfa40 con 0x7f398c151ee0 2020-11-20T12:20:16.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f3972ffd700 1 -- 172.21.15.116:0/1105422340 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]}]=0 v23) v1 ==== 158+0+64 (secure 0 0 0) 0x7f3984015020 con 0x7f398c151ee0 2020-11-20T12:20:16.732 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:20:16.733 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQCAtLdf7UOCKxAAAQ2Jig7S/lAPiwQX/PHdEw== 2020-11-20T12:20:16.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 -- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f39680557c0 msgr2=0x7f3968057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:16.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f39680557c0 0x7f3968057c80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f397c006fd0 tx=0x7f397c008040).stop 2020-11-20T12:20:16.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 -- 172.21.15.116:0/1105422340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f398c151ee0 msgr2=0x7f398c0c11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:16.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f398c151ee0 0x7f398c0c11c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f398c081a20 tx=0x7f3984016000).stop 2020-11-20T12:20:16.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 -- 172.21.15.116:0/1105422340 shutdown_connections 2020-11-20T12:20:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f39680557c0 0x7f3968057c80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f398c151ee0 0x7f398c0c11c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f398c13b0d0 0x7f398c160e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:16.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 --2- 172.21.15.116:0/1105422340 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f398c13b4b0 0x7f398c1512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:16.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 -- 172.21.15.116:0/1105422340 >> 172.21.15.116:0/1105422340 conn(0x7f398c0678a0 msgr2=0x7f398c0c59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:16.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 -- 172.21.15.116:0/1105422340 shutdown_connections 2020-11-20T12:20:16.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:16.736+0000 7f39918ba700 1 -- 172.21.15.116:0/1105422340 wait complete. 2020-11-20T12:20:16.758 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:20:16.759 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:20:16.772 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:20:16.786 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:20:16.787 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:20:16.787 INFO:teuthology.orchestra.run.smithi173.stderr:64 bytes copied, 0.000327195 s, 196 kB/s 2020-11-20T12:20:16.790 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:20:16.797 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:20:16.855 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:20:16.886 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:20:16.887 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:20:16.887 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:20:16.887 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.887 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:20:16.888 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.888 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:20:16.888 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.888 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:20:16.888 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:20:16.889 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.889 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:20:16.889 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:20:16.889 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.890 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:20:16.890 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:20:16.890 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.890 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:20:16.890 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:20:16.891 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.891 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:20:16.891 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.891 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:20:16.891 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:20:16.892 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:20:16.892 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:20:16.892 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:20:16.892 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:20:16.892 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:20:16.893 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:20:16.893 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:20:16.893 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:20:16.893 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.893 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:20:16.894 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.894 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:20:16.894 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.894 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:20:16.894 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.895 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:20:16.895 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:20:16.895 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:20:16.895 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:20:16.895 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:20:16.896 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:20:16.896 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.896 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:20:16.896 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:20:16.896 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.897 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:20:16.897 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:20:16.897 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.897 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:20:16.897 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:20:16.898 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.898 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:20:16.898 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:20:16.898 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:20:16.898 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.899 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:20:16.899 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:20:16.899 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.899 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:20:16.899 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.900 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:20:16.900 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.900 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:20:16.900 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.900 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:20:16.901 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:20:16.901 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.901 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:20:16.901 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:20:16.901 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:20:16.902 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:20:16.902 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:20:16.902 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:20:16.902 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:20:16.902 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:20:16.903 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:20:16.903 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:20:16.903 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:20:16.903 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:20:16.903 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:20:16.904 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:20:16.904 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:20:16.904 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:20:16.904 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:20:16.904 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:20:16.905 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:20:16.905 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:20:16.905 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.905 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:20:16.905 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:20:16.906 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:20:16.906 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:20:16.906 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:20:16.906 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:20:16.906 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:20:16.907 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:20:16.907 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.907 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:20:16.907 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:20:16.907 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:20:16.908 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:20:16.908 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:20:16.908 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:20:16.908 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:20:16.908 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:20:16.909 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:20:16.909 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:20:16.909 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:20:16.909 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:20:16.909 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:16.910 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:20:16.910 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:20:16.910 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:20:16.910 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:20:16.910 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:20:16.911 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:20:16.911 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:20:16.911 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:20:16.911 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:20:16.912 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:20:16.912 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:20:16.912 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:20:16.912 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:20:16.912 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:20:16.913 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:20:16.913 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:20:16.913 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:20:16.913 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:20:16.913 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:20:16.914 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:20:16.914 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:20:16.914 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:20:16.914 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:20:16.914 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:20:16.915 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:20:16.915 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:20:16.915 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:20:16.915 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:20:16.915 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:20:16.916 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:20:16.916 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:20:16.916 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:20:16.916 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:20:16.916 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:20:16.917 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:20:16.917 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:20:16.917 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:20:16.918 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:20:16.919 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:20:16.919 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:20:16.919 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:20:16.919 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.920 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:20:16.920 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.920 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:20:16.920 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.920 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:20:16.921 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:20:16.921 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.921 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:20:16.921 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:20:16.921 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.922 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:20:16.922 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:20:16.922 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.922 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:20:16.922 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:20:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:20:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:20:16.923 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:20:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:20:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:20:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:20:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:20:16.924 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:20:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:20:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:20:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:20:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.925 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:20:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:20:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:20:16.926 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.927 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:20:16.927 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:20:16.927 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:20:16.927 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:20:16.927 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:20:16.928 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:20:16.928 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.928 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:20:16.928 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:20:16.928 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.929 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:20:16.929 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:20:16.929 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.929 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:20:16.929 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:20:16.930 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.930 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:20:16.930 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:20:16.930 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:20:16.930 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.931 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:20:16.931 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:20:16.931 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.931 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:20:16.931 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.932 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:20:16.932 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.932 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:20:16.932 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.932 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:20:16.933 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:20:16.933 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.933 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:20:16.933 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:20:16.933 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:20:16.934 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:20:16.934 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:20:16.934 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:20:16.934 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:20:16.934 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:20:16.935 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:20:16.935 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:20:16.935 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:20:16.935 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:20:16.935 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:20:16.936 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:20:16.936 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:20:16.936 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:20:16.936 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:20:16.936 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:20:16.937 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:20:16.937 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:20:16.937 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.937 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:20:16.937 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:20:16.938 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:20:16.938 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:20:16.938 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:20:16.938 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:20:16.938 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:20:16.939 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:20:16.939 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.939 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:20:16.939 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:20:16.939 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:20:16.940 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:20:16.940 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:20:16.940 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:20:16.940 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:20:16.940 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:20:16.940 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:20:16.941 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:20:16.941 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:20:16.941 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:20:16.941 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:16.941 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:20:16.942 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:20:16.942 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:20:16.942 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:20:16.942 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:20:16.942 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:20:16.943 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:20:16.943 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:20:16.943 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:20:16.943 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:20:16.943 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:20:16.944 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:20:16.944 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:20:16.944 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:20:16.944 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:20:16.944 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:20:16.945 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:20:16.945 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:20:16.945 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:20:16.945 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:20:16.945 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:20:16.946 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:20:16.946 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:20:16.946 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:20:16.946 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:20:16.946 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:20:16.947 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:20:16.947 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:20:16.947 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:20:16.947 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:20:16.947 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:20:16.948 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:20:16.948 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:20:16.948 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:20:16.948 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:20:16.948 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:20:16.949 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:20:16.949 INFO:teuthology.orchestra.run.smithi116:> rados -p cephfs_data put test_vc_obj_3 - 2020-11-20T12:20:17.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e2eec3c40 1 Processor -- start 2020-11-20T12:20:17.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e2eec3c40 1 -- start start 2020-11-20T12:20:17.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e2eec3c40 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7e2219bd0 0x55b7e2172c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e2eec3c40 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e2219fb0 0x55b7e21730f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e2eec3c40 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55b7e216f040 0x55b7e2171420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e2eec3c40 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7e20ec190 con 0x55b7e2219fb0 2020-11-20T12:20:17.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e2eec3c40 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7e20ed1f0 con 0x55b7e2219bd0 2020-11-20T12:20:17.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e2eec3c40 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7e20bbbe0 con 0x55b7e216f040 2020-11-20T12:20:17.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e1fcc8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e2219fb0 0x55b7e21730f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e1fcc8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e2219fb0 0x55b7e21730f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49200/0 (socket says 172.21.15.116:49200) 2020-11-20T12:20:17.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e1fcc8700 1 -- 172.21.15.116:0/2643644578 learned_addr learned my addr 172.21.15.116:0/2643644578 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:17.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e204c9700 1 --2- 172.21.15.116:0/2643644578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7e2219bd0 0x55b7e2172c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e20cca700 1 --2- 172.21.15.116:0/2643644578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55b7e216f040 0x55b7e2171420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e1fcc8700 1 -- 172.21.15.116:0/2643644578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55b7e216f040 msgr2=0x55b7e2171420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:17.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e1fcc8700 1 --2- 172.21.15.116:0/2643644578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55b7e216f040 0x55b7e2171420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e1fcc8700 1 -- 172.21.15.116:0/2643644578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7e2219bd0 msgr2=0x55b7e2172c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:17.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e1fcc8700 1 --2- 172.21.15.116:0/2643644578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7e2219bd0 0x55b7e2172c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e1fcc8700 1 -- 172.21.15.116:0/2643644578 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7e20e1280 con 0x55b7e2219fb0 2020-11-20T12:20:17.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.008+0000 7f6e204c9700 1 --2- 172.21.15.116:0/2643644578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7e2219bd0 0x55b7e2172c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:17.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e1fcc8700 1 --2- 172.21.15.116:0/2643644578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e2219fb0 0x55b7e21730f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f6e1400e670 tx=0x7f6e14001980).ready entity=mon.0 client_cookie=3afb3dd79d46283d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:17.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e1ecc6700 1 -- 172.21.15.116:0/2643644578 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e14001b20 con 0x55b7e2219fb0 2020-11-20T12:20:17.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e1ecc6700 1 -- 172.21.15.116:0/2643644578 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e14001c80 con 0x55b7e2219fb0 2020-11-20T12:20:17.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e1ecc6700 1 -- 172.21.15.116:0/2643644578 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e14010710 con 0x55b7e2219fb0 2020-11-20T12:20:17.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/2643644578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e2219fb0 msgr2=0x55b7e21730f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e2eec3c40 1 --2- 172.21.15.116:0/2643644578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e2219fb0 0x55b7e21730f0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7f6e1400e670 tx=0x7f6e14001980).stop 2020-11-20T12:20:17.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/2643644578 shutdown_connections 2020-11-20T12:20:17.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e2eec3c40 1 --2- 172.21.15.116:0/2643644578 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55b7e216f040 0x55b7e2171420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e2eec3c40 1 --2- 172.21.15.116:0/2643644578 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7e2219bd0 0x55b7e2172c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e2eec3c40 1 --2- 172.21.15.116:0/2643644578 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e2219fb0 0x55b7e21730f0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/2643644578 >> 172.21.15.116:0/2643644578 conn(0x55b7e218a4a0 msgr2=0x55b7e213f0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:17.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/2643644578 shutdown_connections 2020-11-20T12:20:17.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/2643644578 wait complete. 2020-11-20T12:20:17.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e2eec3c40 1 Processor -- start 2020-11-20T12:20:17.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.012+0000 7f6e2eec3c40 1 -- start start 2020-11-20T12:20:17.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e2eec3c40 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e216f040 0x55b7e2216020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e2eec3c40 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55b7e2219bd0 0x55b7e2185340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e2eec3c40 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7e2219fb0 0x55b7e2185830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e2eec3c40 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7e222c130 con 0x55b7e216f040 2020-11-20T12:20:17.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e2eec3c40 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b7e213dee0 con 0x55b7e2219fb0 2020-11-20T12:20:17.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e2eec3c40 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b7e2216510 con 0x55b7e2219bd0 2020-11-20T12:20:17.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e204c9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e216f040 0x55b7e2216020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e204c9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e216f040 0x55b7e2216020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49204/0 (socket says 172.21.15.116:49204) 2020-11-20T12:20:17.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e204c9700 1 -- 172.21.15.116:0/1706740162 learned_addr learned my addr 172.21.15.116:0/1706740162 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:17.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e1fcc8700 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55b7e2219bd0 0x55b7e2185340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e20cca700 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7e2219fb0 0x55b7e2185830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e204c9700 1 -- 172.21.15.116:0/1706740162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55b7e2219bd0 msgr2=0x55b7e2185340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e204c9700 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55b7e2219bd0 0x55b7e2185340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e204c9700 1 -- 172.21.15.116:0/1706740162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7e2219fb0 msgr2=0x55b7e2185830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e204c9700 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55b7e2219fb0 0x55b7e2185830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e204c9700 1 -- 172.21.15.116:0/1706740162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b7e20bf620 con 0x55b7e216f040 2020-11-20T12:20:17.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e204c9700 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e216f040 0x55b7e2216020 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x55b7e21706d0 tx=0x7f6e10018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:17.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e1d4c3700 1 -- 172.21.15.116:0/1706740162 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e10011af0 con 0x55b7e216f040 2020-11-20T12:20:17.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e1d4c3700 1 -- 172.21.15.116:0/1706740162 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e10011c50 con 0x55b7e216f040 2020-11-20T12:20:17.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/1706740162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b7e21864c0 con 0x55b7e216f040 2020-11-20T12:20:17.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e1d4c3700 1 -- 172.21.15.116:0/1706740162 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1001c440 con 0x55b7e216f040 2020-11-20T12:20:17.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.016+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/1706740162 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b7e2186a00 con 0x55b7e216f040 2020-11-20T12:20:17.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.020+0000 7f6e1d4c3700 1 -- 172.21.15.116:0/1706740162 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6e1001c830 con 0x55b7e216f040 2020-11-20T12:20:17.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.020+0000 7f6e1d4c3700 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e0c054440 0x7f6e0c056900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.020+0000 7f6e1d4c3700 1 -- 172.21.15.116:0/1706740162 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6e1006f850 con 0x55b7e216f040 2020-11-20T12:20:17.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.020+0000 7f6e2eec3c40 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x55b7e221c310 0x55b7e221c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.020+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/1706740162 --> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] -- osd_op(unknown.0.0:1 16.4 16:26661a82:::test_vc_obj_3:head [writefull 0~9 in=9b] snapc 0=[] ondisk+write+known_if_redirected e86) v8 -- 0x55b7e221cbe0 con 0x55b7e221c310 2020-11-20T12:20:17.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.020+0000 7f6e1fcc8700 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e0c054440 0x7f6e0c056900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.020+0000 7f6e20cca700 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x55b7e221c310 0x55b7e221c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.020+0000 7f6e1fcc8700 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e0c054440 0x7f6e0c056900 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x55b7e21737a0 tx=0x7f6e14017000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:17.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.020+0000 7f6e20cca700 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x55b7e221c310 0x55b7e221c6f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:17.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.024+0000 7f6e20cca700 1 -- 172.21.15.116:0/1706740162 <== osd.3 v2:172.21.15.116:6802/14055 1 ==== osd_op_reply(1 test_vc_obj_3 [writefull 0~9] v86'1 uv1 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f6e180109d0 con 0x55b7e221c310 2020-11-20T12:20:17.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.024+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/1706740162 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x55b7e221c310 msgr2=0x55b7e221c6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.024+0000 7f6e2eec3c40 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x55b7e221c310 0x55b7e221c6f0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.024+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/1706740162 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e0c054440 msgr2=0x7f6e0c056900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.024+0000 7f6e2eec3c40 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e0c054440 0x7f6e0c056900 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x55b7e21737a0 tx=0x7f6e14017000).stop 2020-11-20T12:20:17.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.024+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/1706740162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e216f040 msgr2=0x55b7e2216020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.024+0000 7f6e2eec3c40 1 --2- 172.21.15.116:0/1706740162 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55b7e216f040 0x55b7e2216020 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x55b7e21706d0 tx=0x7f6e10018000).stop 2020-11-20T12:20:17.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.024+0000 7f6e20cca700 1 -- 172.21.15.116:0/1706740162 reap_dead start 2020-11-20T12:20:17.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.024+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/1706740162 shutdown_connections 2020-11-20T12:20:17.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.024+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/1706740162 >> 172.21.15.116:0/1706740162 conn(0x55b7e218a4a0 msgr2=0x55b7e2216850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:17.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.028+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/1706740162 shutdown_connections 2020-11-20T12:20:17.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.028+0000 7f6e2eec3c40 1 -- 172.21.15.116:0/1706740162 wait complete. 2020-11-20T12:20:17.029 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:20:17.030 INFO:teuthology.orchestra.run.smithi173:> from __future__ import print_function 2020-11-20T12:20:17.030 INFO:teuthology.orchestra.run.smithi173:> from ceph_volume_client import CephFSVolumeClient, VolumePath 2020-11-20T12:20:17.031 INFO:teuthology.orchestra.run.smithi173:> from sys import version_info as sys_version_info 2020-11-20T12:20:17.031 INFO:teuthology.orchestra.run.smithi173:> from rados import OSError as rados_OSError 2020-11-20T12:20:17.031 INFO:teuthology.orchestra.run.smithi173:> import logging 2020-11-20T12:20:17.032 INFO:teuthology.orchestra.run.smithi173:> log = logging.getLogger("ceph_volume_client") 2020-11-20T12:20:17.032 INFO:teuthology.orchestra.run.smithi173:> log.addHandler(logging.StreamHandler()) 2020-11-20T12:20:17.032 INFO:teuthology.orchestra.run.smithi173:> log.setLevel(logging.DEBUG) 2020-11-20T12:20:17.033 INFO:teuthology.orchestra.run.smithi173:> vc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None) 2020-11-20T12:20:17.033 INFO:teuthology.orchestra.run.smithi173:> vc.connect() 2020-11-20T12:20:17.034 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:20:17.034 INFO:teuthology.orchestra.run.smithi173:> data_read = vc.delete_object("cephfs_data", "test_vc_obj_3") 2020-11-20T12:20:17.034 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:20:17.035 INFO:teuthology.orchestra.run.smithi173:> vc.disconnect() 2020-11-20T12:20:17.035 INFO:teuthology.orchestra.run.smithi173:> ' 2020-11-20T12:20:17.221 INFO:teuthology.orchestra.run.smithi173.stderr:Configuring to RADOS with config /etc/ceph/ceph.conf... 2020-11-20T12:20:17.226 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to RADOS... 2020-11-20T12:20:17.238 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to RADOS complete 2020-11-20T12:20:17.238 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to cephfs... 2020-11-20T12:20:17.238 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS initializing... 2020-11-20T12:20:17.245 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS mounting... 2020-11-20T12:20:17.255 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to cephfs complete 2020-11-20T12:20:17.255 INFO:teuthology.orchestra.run.smithi173.stderr:Recovering from partial auth updates (if any)... 2020-11-20T12:20:17.255 INFO:teuthology.orchestra.run.smithi173.stderr:Nothing to recover. No auth meta files. 2020-11-20T12:20:17.260 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:20:17.261 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs... 2020-11-20T12:20:17.268 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs complete 2020-11-20T12:20:17.268 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados... 2020-11-20T12:20:17.271 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados complete 2020-11-20T12:20:17.275 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:20:17.339 INFO:teuthology.orchestra.run.smithi116:> rados -p cephfs_data stat test_vc_obj_3 2020-11-20T12:20:17.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb79f44cc40 1 Processor -- start 2020-11-20T12:20:17.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb79f44cc40 1 -- start start 2020-11-20T12:20:17.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb79f44cc40 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ef755c9c60 0x55ef755fcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb79f44cc40 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ef756124f0 0x55ef7569f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb79f44cc40 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756896b0 0x55ef7569fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb79f44cc40 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef755771f0 con 0x55ef756896b0 2020-11-20T12:20:17.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb79f44cc40 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef75545be0 con 0x55ef755c9c60 2020-11-20T12:20:17.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb79f44cc40 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef756a5680 con 0x55ef756124f0 2020-11-20T12:20:17.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb791253700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756896b0 0x55ef7569fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb791253700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756896b0 0x55ef7569fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49224/0 (socket says 172.21.15.116:49224) 2020-11-20T12:20:17.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb791253700 1 -- 172.21.15.116:0/1210487011 learned_addr learned my addr 172.21.15.116:0/1210487011 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:17.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb790251700 1 --2- 172.21.15.116:0/1210487011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ef756124f0 0x55ef7569f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb790a52700 1 --2- 172.21.15.116:0/1210487011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ef755c9c60 0x55ef755fcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb791253700 1 -- 172.21.15.116:0/1210487011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ef756124f0 msgr2=0x55ef7569f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:17.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb791253700 1 --2- 172.21.15.116:0/1210487011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ef756124f0 0x55ef7569f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb791253700 1 -- 172.21.15.116:0/1210487011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ef755c9c60 msgr2=0x55ef755fcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:17.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb791253700 1 --2- 172.21.15.116:0/1210487011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ef755c9c60 0x55ef755fcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb791253700 1 -- 172.21.15.116:0/1210487011 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef7556f8a0 con 0x55ef756896b0 2020-11-20T12:20:17.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.396+0000 7fb790a52700 1 --2- 172.21.15.116:0/1210487011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ef755c9c60 0x55ef755fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:17.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb791253700 1 --2- 172.21.15.116:0/1210487011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756896b0 0x55ef7569fa50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x55ef755b7290 tx=0x7fb788014000).ready entity=mon.0 client_cookie=d66551ff5242dec3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:17.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb78f24f700 1 -- 172.21.15.116:0/1210487011 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb788010d00 con 0x55ef756896b0 2020-11-20T12:20:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb78f24f700 1 -- 172.21.15.116:0/1210487011 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb788010e60 con 0x55ef756896b0 2020-11-20T12:20:17.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb78f24f700 1 -- 172.21.15.116:0/1210487011 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78801c6a0 con 0x55ef756896b0 2020-11-20T12:20:17.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb79f44cc40 1 -- 172.21.15.116:0/1210487011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756896b0 msgr2=0x55ef7569fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb79f44cc40 1 --2- 172.21.15.116:0/1210487011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756896b0 0x55ef7569fa50 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x55ef755b7290 tx=0x7fb788014000).stop 2020-11-20T12:20:17.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb79f44cc40 1 -- 172.21.15.116:0/1210487011 shutdown_connections 2020-11-20T12:20:17.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb79f44cc40 1 --2- 172.21.15.116:0/1210487011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756896b0 0x55ef7569fa50 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb79f44cc40 1 --2- 172.21.15.116:0/1210487011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ef755c9c60 0x55ef755fcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb79f44cc40 1 --2- 172.21.15.116:0/1210487011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ef756124f0 0x55ef7569f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb79f44cc40 1 -- 172.21.15.116:0/1210487011 >> 172.21.15.116:0/1210487011 conn(0x55ef756144a0 msgr2=0x55ef755c7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:17.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb79f44cc40 1 -- 172.21.15.116:0/1210487011 shutdown_connections 2020-11-20T12:20:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb79f44cc40 1 -- 172.21.15.116:0/1210487011 wait complete. 2020-11-20T12:20:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb79f44cc40 1 Processor -- start 2020-11-20T12:20:17.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.400+0000 7fb79f44cc40 1 -- start start 2020-11-20T12:20:17.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb79f44cc40 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ef755c9c60 0x55ef7569c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb79f44cc40 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756124f0 0x55ef75684e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb79f44cc40 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ef756896b0 0x55ef756a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb79f44cc40 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7569d6a0 con 0x55ef756124f0 2020-11-20T12:20:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb79f44cc40 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef7569d940 con 0x55ef756896b0 2020-11-20T12:20:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb79f44cc40 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ef7569dbe0 con 0x55ef755c9c60 2020-11-20T12:20:17.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb790251700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756124f0 0x55ef75684e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb790251700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756124f0 0x55ef75684e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49226/0 (socket says 172.21.15.116:49226) 2020-11-20T12:20:17.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb790251700 1 -- 172.21.15.116:0/2489169417 learned_addr learned my addr 172.21.15.116:0/2489169417 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:17.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb791253700 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ef756896b0 0x55ef756a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb790a52700 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ef755c9c60 0x55ef7569c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb790251700 1 -- 172.21.15.116:0/2489169417 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ef755c9c60 msgr2=0x55ef7569c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb790251700 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x55ef755c9c60 0x55ef7569c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb790251700 1 -- 172.21.15.116:0/2489169417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ef756896b0 msgr2=0x55ef756a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb790251700 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x55ef756896b0 0x55ef756a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb790251700 1 -- 172.21.15.116:0/2489169417 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef7556b280 con 0x55ef756124f0 2020-11-20T12:20:17.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb790251700 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756124f0 0x55ef75684e00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x55ef755f7520 tx=0x7fb784001670).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:17.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb78da4c700 1 -- 172.21.15.116:0/2489169417 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb784001cf0 con 0x55ef756124f0 2020-11-20T12:20:17.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb78da4c700 1 -- 172.21.15.116:0/2489169417 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78400f7b0 con 0x55ef756124f0 2020-11-20T12:20:17.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb79f44cc40 1 -- 172.21.15.116:0/2489169417 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55ef75549620 con 0x55ef756124f0 2020-11-20T12:20:17.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb78da4c700 1 -- 172.21.15.116:0/2489169417 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78400fb30 con 0x55ef756124f0 2020-11-20T12:20:17.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.404+0000 7fb79f44cc40 1 -- 172.21.15.116:0/2489169417 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef7569e120 con 0x55ef756124f0 2020-11-20T12:20:17.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.408+0000 7fb78da4c700 1 -- 172.21.15.116:0/2489169417 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb7840173c0 con 0x55ef756124f0 2020-11-20T12:20:17.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.408+0000 7fb78da4c700 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb77c054440 0x7fb77c056900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.408+0000 7fb78da4c700 1 -- 172.21.15.116:0/2489169417 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb78406daf0 con 0x55ef756124f0 2020-11-20T12:20:17.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.408+0000 7fb79f44cc40 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x55ef756852f0 0x55ef7569ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:17.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.408+0000 7fb790a52700 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb77c054440 0x7fb77c056900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.408+0000 7fb79f44cc40 1 -- 172.21.15.116:0/2489169417 --> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] -- osd_op(unknown.0.0:1 16.4 16:26661a82:::test_vc_obj_3:head [stat] snapc 0=[] ondisk+read+known_if_redirected e86) v8 -- 0x55ef755f7f40 con 0x55ef756852f0 2020-11-20T12:20:17.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb791253700 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x55ef756852f0 0x55ef7569ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:17.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb790a52700 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb77c054440 0x7fb77c056900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fb780008b50 tx=0x7fb78000c040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:17.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb791253700 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x55ef756852f0 0x55ef7569ead0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:17.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb791253700 1 -- 172.21.15.116:0/2489169417 <== osd.3 v2:172.21.15.116:6802/14055 1 ==== osd_op_reply(1 test_vc_obj_3 [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 157+0+0 (crc 0 0 0) 0x7fb788022020 con 0x55ef756852f0 2020-11-20T12:20:17.433 INFO:teuthology.orchestra.run.smithi116.stderr: error stat-ing cephfs_data/test_vc_obj_3: (2) No such file or directory 2020-11-20T12:20:17.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb79f44cc40 1 -- 172.21.15.116:0/2489169417 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x55ef756852f0 msgr2=0x55ef7569ead0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb79f44cc40 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.116:6802/14055,v1:172.21.15.116:6804/14055] conn(0x55ef756852f0 0x55ef7569ead0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:17.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb79f44cc40 1 -- 172.21.15.116:0/2489169417 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb77c054440 msgr2=0x7fb77c056900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb79f44cc40 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb77c054440 0x7fb77c056900 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fb780008b50 tx=0x7fb78000c040).stop 2020-11-20T12:20:17.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb79f44cc40 1 -- 172.21.15.116:0/2489169417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756124f0 msgr2=0x55ef75684e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:17.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb79f44cc40 1 --2- 172.21.15.116:0/2489169417 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x55ef756124f0 0x55ef75684e00 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x55ef755f7520 tx=0x7fb784001670).stop 2020-11-20T12:20:17.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb791253700 1 -- 172.21.15.116:0/2489169417 reap_dead start 2020-11-20T12:20:17.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb79f44cc40 1 -- 172.21.15.116:0/2489169417 shutdown_connections 2020-11-20T12:20:17.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb79f44cc40 1 -- 172.21.15.116:0/2489169417 >> 172.21.15.116:0/2489169417 conn(0x55ef756144a0 msgr2=0x55ef75616170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:17.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb79f44cc40 1 -- 172.21.15.116:0/2489169417 shutdown_connections 2020-11-20T12:20:17.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:17.412+0000 7fb79f44cc40 1 -- 172.21.15.116:0/2489169417 wait complete. 2020-11-20T12:20:17.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2020-11-20T12:20:17.439 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:20:17.439 INFO:teuthology.orchestra.run.smithi173:> from __future__ import print_function 2020-11-20T12:20:17.440 INFO:teuthology.orchestra.run.smithi173:> from ceph_volume_client import CephFSVolumeClient, VolumePath 2020-11-20T12:20:17.441 INFO:teuthology.orchestra.run.smithi173:> from sys import version_info as sys_version_info 2020-11-20T12:20:17.442 INFO:teuthology.orchestra.run.smithi173:> from rados import OSError as rados_OSError 2020-11-20T12:20:17.442 INFO:teuthology.orchestra.run.smithi173:> import logging 2020-11-20T12:20:17.443 INFO:teuthology.orchestra.run.smithi173:> log = logging.getLogger("ceph_volume_client") 2020-11-20T12:20:17.444 INFO:teuthology.orchestra.run.smithi173:> log.addHandler(logging.StreamHandler()) 2020-11-20T12:20:17.445 INFO:teuthology.orchestra.run.smithi173:> log.setLevel(logging.DEBUG) 2020-11-20T12:20:17.445 INFO:teuthology.orchestra.run.smithi173:> vc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None) 2020-11-20T12:20:17.446 INFO:teuthology.orchestra.run.smithi173:> vc.connect() 2020-11-20T12:20:17.447 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:20:17.447 INFO:teuthology.orchestra.run.smithi173:> data_read = vc.delete_object("cephfs_data", "test_vc_obj_3") 2020-11-20T12:20:17.448 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:20:17.448 INFO:teuthology.orchestra.run.smithi173:> vc.disconnect() 2020-11-20T12:20:17.448 INFO:teuthology.orchestra.run.smithi173:> ' 2020-11-20T12:20:17.627 INFO:teuthology.orchestra.run.smithi173.stderr:Configuring to RADOS with config /etc/ceph/ceph.conf... 2020-11-20T12:20:17.631 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to RADOS... 2020-11-20T12:20:17.642 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to RADOS complete 2020-11-20T12:20:17.643 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to cephfs... 2020-11-20T12:20:17.643 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS initializing... 2020-11-20T12:20:17.649 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS mounting... 2020-11-20T12:20:17.659 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to cephfs complete 2020-11-20T12:20:17.660 INFO:teuthology.orchestra.run.smithi173.stderr:Recovering from partial auth updates (if any)... 2020-11-20T12:20:17.660 INFO:teuthology.orchestra.run.smithi173.stderr:Nothing to recover. No auth meta files. 2020-11-20T12:20:17.665 INFO:teuthology.orchestra.run.smithi173.stderr:Object 'test_vc_obj_3' was already removed 2020-11-20T12:20:17.665 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:20:17.665 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs... 2020-11-20T12:20:17.673 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs complete 2020-11-20T12:20:17.673 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados... 2020-11-20T12:20:17.676 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados complete 2020-11-20T12:20:17.680 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:20:17.750 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:20:17.751 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:20:17.790 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:20:17.838 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:20:17.838 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-20T12:20:17.839 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:20:17.839 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-20T12:20:17.915 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:20:17.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:17.916 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:20:17.933 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2020-11-20T12:20:17.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:20:17.935 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.2 2020-11-20T12:20:18.060 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:20:18.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:18.062 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T12:20:18.090 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2020-11-20T12:20:18.091 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:20:18.091 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.3 2020-11-20T12:20:18.169 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:20:18.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:18.170 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T12:20:18.201 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:20:18.213 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:20:18.279 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:20:18.279 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:20:18.279 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:20:18.279 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:20:18.280 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.280 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:20:18.280 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.280 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:20:18.281 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.281 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:20:18.281 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:20:18.281 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.281 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:20:18.282 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:20:18.282 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.282 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:20:18.282 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:20:18.282 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.283 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:20:18.283 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:20:18.283 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.283 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:20:18.283 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.284 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:20:18.284 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:20:18.284 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:20:18.284 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:20:18.284 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:20:18.285 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:20:18.285 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:20:18.285 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:20:18.285 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:20:18.286 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:20:18.286 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.286 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:20:18.286 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.286 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:20:18.287 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.287 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:20:18.287 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.287 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:20:18.287 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:20:18.288 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:20:18.288 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:20:18.288 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:20:18.288 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:20:18.288 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.289 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:20:18.289 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:20:18.289 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.289 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:20:18.289 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:20:18.290 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.290 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:20:18.290 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:20:18.290 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.290 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:20:18.291 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:20:18.291 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:20:18.291 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.291 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:20:18.291 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:20:18.292 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.292 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:20:18.292 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.292 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:20:18.292 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.293 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:20:18.293 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.293 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:20:18.293 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:20:18.293 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.294 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:20:18.294 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:20:18.294 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:20:18.294 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:20:18.294 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:20:18.295 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:20:18.295 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:20:18.295 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:20:18.295 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:20:18.295 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:20:18.296 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:20:18.296 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:20:18.296 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:20:18.296 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:20:18.296 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:20:18.297 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:20:18.297 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:20:18.297 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:20:18.297 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:20:18.298 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:20:18.298 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.298 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:20:18.298 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:20:18.298 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:20:18.299 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:20:18.299 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:20:18.299 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:20:18.299 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:20:18.299 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:20:18.300 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.300 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:20:18.300 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:20:18.300 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:20:18.300 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:20:18.301 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:20:18.301 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:20:18.301 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:20:18.301 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:20:18.301 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:20:18.302 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:20:18.302 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:20:18.302 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:20:18.302 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:18.302 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:20:18.303 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:20:18.303 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:20:18.303 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:20:18.303 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:20:18.303 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:20:18.304 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:20:18.304 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:20:18.304 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:20:18.304 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:20:18.304 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:20:18.305 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:20:18.305 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:20:18.305 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:20:18.305 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:20:18.305 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:20:18.306 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:20:18.306 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:20:18.306 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:20:18.306 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:20:18.306 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:20:18.307 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:20:18.307 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:20:18.307 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:20:18.307 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:20:18.307 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:20:18.308 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:20:18.308 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:20:18.308 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:20:18.308 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:20:18.308 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:20:18.309 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:20:18.309 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:20:18.309 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:20:18.309 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:20:18.309 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:20:18.312 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:20:18.312 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:20:18.312 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:20:18.313 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:20:18.313 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.313 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:20:18.313 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.313 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:20:18.314 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.314 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:20:18.314 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:20:18.314 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.314 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:20:18.315 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:20:18.315 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.315 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:20:18.315 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:20:18.315 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.316 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:20:18.316 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:20:18.316 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.316 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:20:18.316 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.317 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:20:18.317 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:20:18.317 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:20:18.317 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:20:18.318 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:20:18.318 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:20:18.318 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:20:18.318 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:20:18.318 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:20:18.319 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:20:18.319 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.319 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:20:18.319 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.319 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:20:18.320 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.320 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:20:18.320 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.320 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:20:18.320 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:20:18.321 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:20:18.321 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:20:18.321 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:20:18.321 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:20:18.321 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.322 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:20:18.322 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:20:18.322 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.322 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:20:18.323 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:20:18.323 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.323 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:20:18.323 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:20:18.323 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.324 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:20:18.324 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:20:18.324 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:20:18.324 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.324 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:20:18.325 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:20:18.325 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.325 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:20:18.325 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.325 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:20:18.326 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.326 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:20:18.326 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.326 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:20:18.326 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:20:18.327 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.327 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:20:18.327 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:20:18.327 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:20:18.327 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:20:18.328 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:20:18.328 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:20:18.328 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:20:18.328 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:20:18.328 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:20:18.329 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:20:18.329 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:20:18.329 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:20:18.329 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:20:18.329 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:20:18.330 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:20:18.330 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:20:18.330 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:20:18.330 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:20:18.330 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:20:18.331 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:20:18.331 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.331 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:20:18.331 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:20:18.331 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:20:18.332 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:20:18.332 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:20:18.332 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:20:18.332 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:20:18.332 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:20:18.333 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.333 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:20:18.333 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:20:18.333 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:20:18.333 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:20:18.334 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:20:18.334 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:20:18.334 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:20:18.334 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:20:18.334 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:20:18.335 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:20:18.335 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:20:18.335 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:20:18.335 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:20:18.335 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:20:18.336 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:20:18.336 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:20:18.336 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:20:18.336 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:20:18.336 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:20:18.337 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:20:18.337 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:20:18.337 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:20:18.337 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:20:18.337 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:20:18.338 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:20:18.338 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:20:18.338 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:20:18.338 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:20:18.338 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:20:18.339 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:20:18.339 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:20:18.339 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:20:18.339 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:20:18.339 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:20:18.340 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:20:18.340 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:20:18.340 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:20:18.340 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:20:18.340 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:20:18.341 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:20:18.341 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:20:18.341 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:20:18.341 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:20:18.341 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:20:18.342 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:20:18.342 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:20:18.342 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:20:18.342 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:20:18.342 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:20:18.343 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-20T12:20:18.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.544+0000 7f3437cd1700 1 Processor -- start 2020-11-20T12:20:18.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.548+0000 7f3437cd1700 1 -- start start 2020-11-20T12:20:18.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.548+0000 7f3437cd1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f343014bee0 0x7f3430152990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.548+0000 7f3437cd1700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f34300c8fa0 0x7f34300adba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.548+0000 7f3437cd1700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34300abc10 con 0x7f34300ae090 2020-11-20T12:20:18.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.548+0000 7f3437cd1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34300acf60 con 0x7f343014bee0 2020-11-20T12:20:18.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.548+0000 7f3437cd1700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3430152e80 con 0x7f34300c8fa0 2020-11-20T12:20:18.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3435a6d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f343014bee0 0x7f3430152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3435a6d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f343014bee0 0x7f3430152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49236/0 (socket says 172.21.15.116:49236) 2020-11-20T12:20:18.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3435a6d700 1 -- 172.21.15.116:0/120516960 learned_addr learned my addr 172.21.15.116:0/120516960 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:18.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f343526c700 1 --2- 172.21.15.116:0/120516960 >> v2:172.21.15.173:3300/0 conn(0x7f34300c8fa0 0x7f34300adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3435a6d700 1 -- 172.21.15.116:0/120516960 >> v2:172.21.15.173:3300/0 conn(0x7f34300c8fa0 msgr2=0x7f34300adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:18.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3435a6d700 1 --2- 172.21.15.116:0/120516960 >> v2:172.21.15.173:3300/0 conn(0x7f34300c8fa0 0x7f34300adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3435a6d700 1 -- 172.21.15.116:0/120516960 >> v1:172.21.15.173:6789/0 conn(0x7f34300ae090 legacy=0x7f34300616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:18.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3435a6d700 1 -- 172.21.15.116:0/120516960 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34300c9520 con 0x7f343014bee0 2020-11-20T12:20:18.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3435a6d700 1 --2- 172.21.15.116:0/120516960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f343014bee0 0x7f3430152990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f34300c9430 tx=0x7f342c006eb0).ready entity=mon.0 client_cookie=934197c721a3171d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:18.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3434a6b700 1 -- 172.21.15.116:0/120516960 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c009810 con 0x7f343014bee0 2020-11-20T12:20:18.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3434a6b700 1 -- 172.21.15.116:0/120516960 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342c009970 con 0x7f343014bee0 2020-11-20T12:20:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3434a6b700 1 -- 172.21.15.116:0/120516960 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c009cf0 con 0x7f343014bee0 2020-11-20T12:20:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3437cd1700 1 -- 172.21.15.116:0/120516960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f343014bee0 msgr2=0x7f3430152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3437cd1700 1 --2- 172.21.15.116:0/120516960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f343014bee0 0x7f3430152990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7f34300c9430 tx=0x7f342c006eb0).stop 2020-11-20T12:20:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3437cd1700 1 -- 172.21.15.116:0/120516960 shutdown_connections 2020-11-20T12:20:18.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3437cd1700 1 --2- 172.21.15.116:0/120516960 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f343014bee0 0x7f3430152990 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3437cd1700 1 --2- 172.21.15.116:0/120516960 >> v2:172.21.15.173:3300/0 conn(0x7f34300c8fa0 0x7f34300adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.552+0000 7f3437cd1700 1 -- 172.21.15.116:0/120516960 >> 172.21.15.116:0/120516960 conn(0x7f3430068370 msgr2=0x7f34300c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:18.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.556+0000 7f3437cd1700 1 -- 172.21.15.116:0/120516960 shutdown_connections 2020-11-20T12:20:18.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.556+0000 7f3437cd1700 1 -- 172.21.15.116:0/120516960 wait complete. 2020-11-20T12:20:18.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.556+0000 7f3437cd1700 1 Processor -- start 2020-11-20T12:20:18.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.556+0000 7f3437cd1700 1 -- start start 2020-11-20T12:20:18.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.556+0000 7f3437cd1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34300ae090 0x7f343014e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.556+0000 7f3437cd1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34300c8fa0 0x7f3430150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.556+0000 7f3437cd1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3430161eb0 0x7f34301512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.556+0000 7f3437cd1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343014f0f0 con 0x7f34300ae090 2020-11-20T12:20:18.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.556+0000 7f3437cd1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3430161230 con 0x7f3430161eb0 2020-11-20T12:20:18.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.556+0000 7f3437cd1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34301614d0 con 0x7f34300c8fa0 2020-11-20T12:20:18.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3435a6d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34300ae090 0x7f343014e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3435a6d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34300ae090 0x7f343014e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49242/0 (socket says 172.21.15.116:49242) 2020-11-20T12:20:18.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3435a6d700 1 -- 172.21.15.116:0/1888830598 learned_addr learned my addr 172.21.15.116:0/1888830598 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:18.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f343526c700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34300c8fa0 0x7f3430150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f343626e700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3430161eb0 0x7f34301512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3435a6d700 1 -- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34300c8fa0 msgr2=0x7f3430150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:18.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3435a6d700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34300c8fa0 0x7f3430150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3435a6d700 1 -- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3430161eb0 msgr2=0x7f34301512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:18.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3435a6d700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3430161eb0 0x7f34301512d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3435a6d700 1 -- 172.21.15.116:0/1888830598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3430161770 con 0x7f34300ae090 2020-11-20T12:20:18.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f343526c700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34300c8fa0 0x7f3430150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:18.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f343626e700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3430161eb0 0x7f34301512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:18.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3435a6d700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34300ae090 0x7f343014e700 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f3430060bb0 tx=0x7f342c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f341dffb700 1 -- 172.21.15.116:0/1888830598 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c009810 con 0x7f34300ae090 2020-11-20T12:20:18.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f341dffb700 1 -- 172.21.15.116:0/1888830598 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f342c011e90 con 0x7f34300ae090 2020-11-20T12:20:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3437cd1700 1 -- 172.21.15.116:0/1888830598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3430161a10 con 0x7f34300ae090 2020-11-20T12:20:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f341dffb700 1 -- 172.21.15.116:0/1888830598 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f342c01e6b0 con 0x7f34300ae090 2020-11-20T12:20:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3437cd1700 1 -- 172.21.15.116:0/1888830598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343014ed90 con 0x7f34300ae090 2020-11-20T12:20:18.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.560+0000 7f3437cd1700 1 -- 172.21.15.116:0/1888830598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343005d2a0 con 0x7f34300ae090 2020-11-20T12:20:18.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.564+0000 7f341dffb700 1 -- 172.21.15.116:0/1888830598 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f342c025030 con 0x7f34300ae090 2020-11-20T12:20:18.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.564+0000 7f341dffb700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f34080557c0 0x7f3408057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.564+0000 7f341dffb700 1 -- 172.21.15.116:0/1888830598 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f342c06fe00 con 0x7f34300ae090 2020-11-20T12:20:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.564+0000 7f343526c700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f34080557c0 0x7f3408057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.568+0000 7f343526c700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f34080557c0 0x7f3408057c80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f34300aa660 tx=0x7f3420008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:18.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.572+0000 7f341dffb700 1 -- 172.21.15.116:0/1888830598 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f342c045c40 con 0x7f34300ae090 2020-11-20T12:20:18.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.732+0000 7f3437cd1700 1 -- 172.21.15.116:0/1888830598 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f34300608c0 con 0x7f34300ae090 2020-11-20T12:20:18.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.740+0000 7f341dffb700 1 -- 172.21.15.116:0/1888830598 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f342c023070 con 0x7f34300ae090 2020-11-20T12:20:18.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 -- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f34080557c0 msgr2=0x7f3408057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:18.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f34080557c0 0x7f3408057c80 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f34300aa660 tx=0x7f3420008040).stop 2020-11-20T12:20:18.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 -- 172.21.15.116:0/1888830598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34300ae090 msgr2=0x7f343014e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:18.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34300ae090 0x7f343014e700 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7f3430060bb0 tx=0x7f342c015000).stop 2020-11-20T12:20:18.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 -- 172.21.15.116:0/1888830598 shutdown_connections 2020-11-20T12:20:18.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f34080557c0 0x7f3408057c80 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3430161eb0 0x7f34301512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f34300ae090 0x7f343014e700 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 --2- 172.21.15.116:0/1888830598 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f34300c8fa0 0x7f3430150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 -- 172.21.15.116:0/1888830598 >> 172.21.15.116:0/1888830598 conn(0x7f3430068370 msgr2=0x7f343014fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:18.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 -- 172.21.15.116:0/1888830598 shutdown_connections 2020-11-20T12:20:18.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.744+0000 7f3437cd1700 1 -- 172.21.15.116:0/1888830598 wait complete. 2020-11-20T12:20:18.766 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_delete_object' 2020-11-20T12:20:18.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.948+0000 7f381f17d700 1 Processor -- start 2020-11-20T12:20:18.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.948+0000 7f381f17d700 1 -- start start 2020-11-20T12:20:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381f17d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3818062880 0x7f3818062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381f17d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3818153450 0x7f38180677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381f17d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38181543a0 con 0x7f38180c4c10 2020-11-20T12:20:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381f17d700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3818153920 con 0x7f3818153450 2020-11-20T12:20:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381f17d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3818153bc0 con 0x7f3818062880 2020-11-20T12:20:18.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381e17b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3818062880 0x7f3818062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381e17b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3818062880 0x7f3818062c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:58200/0 (socket says 172.21.15.116:58200) 2020-11-20T12:20:18.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381e17b700 1 -- 172.21.15.116:0/3810720402 learned_addr learned my addr 172.21.15.116:0/3810720402 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:18.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381d97a700 1 --2- 172.21.15.116:0/3810720402 >> v2:172.21.15.173:3300/0 conn(0x7f3818153450 0x7f38180677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381e17b700 1 -- 172.21.15.116:0/3810720402 >> v2:172.21.15.173:3300/0 conn(0x7f3818153450 msgr2=0x7f38180677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:18.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381e17b700 1 --2- 172.21.15.116:0/3810720402 >> v2:172.21.15.173:3300/0 conn(0x7f3818153450 0x7f38180677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381e17b700 1 -- 172.21.15.116:0/3810720402 >> v1:172.21.15.173:6789/0 conn(0x7f38180c4c10 legacy=0x7f38180b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.952+0000 7f381e17b700 1 -- 172.21.15.116:0/3810720402 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3818153e60 con 0x7f3818062880 2020-11-20T12:20:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381e17b700 1 --2- 172.21.15.116:0/3810720402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3818062880 0x7f3818062c60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3814009d50 tx=0x7f3814009290).ready entity=mon.2 client_cookie=826b669da8909359 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:18.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381d179700 1 -- 172.21.15.116:0/3810720402 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381400bbc0 con 0x7f3818062880 2020-11-20T12:20:18.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381d179700 1 -- 172.21.15.116:0/3810720402 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381400bd20 con 0x7f3818062880 2020-11-20T12:20:18.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381d179700 1 -- 172.21.15.116:0/3810720402 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3814011610 con 0x7f3818062880 2020-11-20T12:20:18.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381f17d700 1 -- 172.21.15.116:0/3810720402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3818062880 msgr2=0x7f3818062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:18.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381f17d700 1 --2- 172.21.15.116:0/3810720402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3818062880 0x7f3818062c60 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3814009d50 tx=0x7f3814009290).stop 2020-11-20T12:20:18.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381f17d700 1 -- 172.21.15.116:0/3810720402 shutdown_connections 2020-11-20T12:20:18.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381f17d700 1 --2- 172.21.15.116:0/3810720402 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3818062880 0x7f3818062c60 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381f17d700 1 --2- 172.21.15.116:0/3810720402 >> v2:172.21.15.173:3300/0 conn(0x7f3818153450 0x7f38180677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381f17d700 1 -- 172.21.15.116:0/3810720402 >> 172.21.15.116:0/3810720402 conn(0x7f3818060c40 msgr2=0x7f38180c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:18.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381f17d700 1 -- 172.21.15.116:0/3810720402 shutdown_connections 2020-11-20T12:20:18.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.956+0000 7f381f17d700 1 -- 172.21.15.116:0/3810720402 wait complete. 2020-11-20T12:20:18.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381f17d700 1 Processor -- start 2020-11-20T12:20:18.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381f17d700 1 -- start start 2020-11-20T12:20:18.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381f17d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3818062880 0x7f3818165740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381f17d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38180c4c10 0x7f38181616f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381f17d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3818153450 0x7f3818067f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381f17d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38181510e0 con 0x7f3818153450 2020-11-20T12:20:18.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381f17d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3818162650 con 0x7f3818062880 2020-11-20T12:20:18.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381f17d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3818161be0 con 0x7f38180c4c10 2020-11-20T12:20:18.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381e97c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3818153450 0x7f3818067f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381e97c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3818153450 0x7f3818067f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49260/0 (socket says 172.21.15.116:49260) 2020-11-20T12:20:18.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381e97c700 1 -- 172.21.15.116:0/1893179872 learned_addr learned my addr 172.21.15.116:0/1893179872 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:18.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381d97a700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38180c4c10 0x7f38181616f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381e17b700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3818062880 0x7f3818165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381e97c700 1 -- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38180c4c10 msgr2=0x7f38181616f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:18.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381e97c700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38180c4c10 0x7f38181616f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381e97c700 1 -- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3818062880 msgr2=0x7f3818165740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:18.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381e97c700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3818062880 0x7f3818165740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:18.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381e97c700 1 -- 172.21.15.116:0/1893179872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3818161e70 con 0x7f3818153450 2020-11-20T12:20:18.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381d97a700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38180c4c10 0x7f38181616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:18.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.968+0000 7f381e97c700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3818153450 0x7f3818067f40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7f381000c710 tx=0x7f3810007b80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:18.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.972+0000 7f38067fc700 1 -- 172.21.15.116:0/1893179872 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381000e490 con 0x7f3818153450 2020-11-20T12:20:18.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.972+0000 7f38067fc700 1 -- 172.21.15.116:0/1893179872 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381000e5f0 con 0x7f3818153450 2020-11-20T12:20:18.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.972+0000 7f381f17d700 1 -- 172.21.15.116:0/1893179872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3818162110 con 0x7f3818153450 2020-11-20T12:20:18.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.972+0000 7f38067fc700 1 -- 172.21.15.116:0/1893179872 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381000e990 con 0x7f3818153450 2020-11-20T12:20:18.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.972+0000 7f381f17d700 1 -- 172.21.15.116:0/1893179872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3818068430 con 0x7f3818153450 2020-11-20T12:20:18.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.972+0000 7f381f17d700 1 -- 172.21.15.116:0/1893179872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3818068710 con 0x7f3818153450 2020-11-20T12:20:18.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.972+0000 7f38067fc700 1 -- 172.21.15.116:0/1893179872 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3810010030 con 0x7f3818153450 2020-11-20T12:20:18.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.976+0000 7f38067fc700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37f00557c0 0x7f37f0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:18.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.976+0000 7f38067fc700 1 -- 172.21.15.116:0/1893179872 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3810001e40 con 0x7f3818153450 2020-11-20T12:20:18.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.976+0000 7f381e17b700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37f00557c0 0x7f37f0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:18.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.976+0000 7f381e17b700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37f00557c0 0x7f37f0057c80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f3814009fd0 tx=0x7f3814016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:18.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:18.980+0000 7f38067fc700 1 -- 172.21.15.116:0/1893179872 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3810044390 con 0x7f3818153450 2020-11-20T12:20:19.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.196+0000 7f381f17d700 1 -- 172.21.15.116:0/1893179872 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_delete_object"]} v 0) v1 -- 0x7f381805cf30 con 0x7f3818153450 2020-11-20T12:20:19.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.680+0000 7f38067fc700 1 -- 172.21.15.116:0/1893179872 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_delete_object"]}]=0 v252) v1 ==== 146+0+0 (secure 0 0 0) 0x7f38100479b0 con 0x7f3818153450 2020-11-20T12:20:19.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 -- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37f00557c0 msgr2=0x7f37f0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:19.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37f00557c0 0x7f37f0057c80 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f3814009fd0 tx=0x7f3814016000).stop 2020-11-20T12:20:19.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 -- 172.21.15.116:0/1893179872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3818153450 msgr2=0x7f3818067f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:19.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3818153450 0x7f3818067f40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7f381000c710 tx=0x7f3810007b80).stop 2020-11-20T12:20:19.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 -- 172.21.15.116:0/1893179872 shutdown_connections 2020-11-20T12:20:19.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37f00557c0 0x7f37f0057c80 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:19.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3818153450 0x7f3818067f40 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:19.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3818062880 0x7f3818165740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:19.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 --2- 172.21.15.116:0/1893179872 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f38180c4c10 0x7f38181616f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:19.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 -- 172.21.15.116:0/1893179872 >> 172.21.15.116:0/1893179872 conn(0x7f3818060c40 msgr2=0x7f381814f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:19.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 -- 172.21.15.116:0/1893179872 shutdown_connections 2020-11-20T12:20:19.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.684+0000 7f381f17d700 1 -- 172.21.15.116:0/1893179872 wait complete. 2020-11-20T12:20:19.706 INFO:tasks.cephfs_test_runner:test_delete_object (tasks.cephfs.test_volume_client.TestVolumeClient) ... ok 2020-11-20T12:20:19.707 INFO:tasks.cephfs_test_runner:Starting test: test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient) 2020-11-20T12:20:19.707 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_evict_client' 2020-11-20T12:20:19.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.888+0000 7f787a46f700 1 Processor -- start 2020-11-20T12:20:19.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.888+0000 7f787a46f700 1 -- start start 2020-11-20T12:20:19.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.888+0000 7f787a46f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7874153450 0x7f7874062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:19.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.888+0000 7f787a46f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78740677a0 0x7f78740b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:19.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.888+0000 7f787a46f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78741543a0 con 0x7f7874062880 2020-11-20T12:20:19.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.888+0000 7f787a46f700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7874067c00 con 0x7f7874153450 2020-11-20T12:20:19.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.888+0000 7f787a46f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7874153920 con 0x7f78740677a0 2020-11-20T12:20:19.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.888+0000 7f787946d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7874062880 0x7f78740c4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58698/0 (socket says 172.21.15.116:58698) 2020-11-20T12:20:19.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.888+0000 7f787946d700 1 -- 172.21.15.116:0/3014519590 learned_addr learned my addr 172.21.15.116:0/3014519590 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:19.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.888+0000 7f7878c6c700 1 --2- 172.21.15.116:0/3014519590 >> v2:172.21.15.173:3300/0 conn(0x7f7874153450 0x7f7874062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:19.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.888+0000 7f7879c6e700 1 --2- 172.21.15.116:0/3014519590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78740677a0 0x7f78740b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:19.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f7863fff700 1 -- 172.21.15.116:0/3014519590 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285252424 0 0) 0x7f78741543a0 con 0x7f7874062880 2020-11-20T12:20:19.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f7863fff700 1 -- 172.21.15.116:0/3014519590 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f785c003240 con 0x7f7874062880 2020-11-20T12:20:19.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f7878c6c700 1 -- 172.21.15.116:0/3014519590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78740677a0 msgr2=0x7f78740b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:19.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f7878c6c700 1 --2- 172.21.15.116:0/3014519590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78740677a0 0x7f78740b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:19.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f7878c6c700 1 -- 172.21.15.116:0/3014519590 >> v1:172.21.15.116:6789/0 conn(0x7f7874062880 legacy=0x7f78740c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:19.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f7878c6c700 1 -- 172.21.15.116:0/3014519590 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7874153bc0 con 0x7f7874153450 2020-11-20T12:20:19.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f7878c6c700 1 --2- 172.21.15.116:0/3014519590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7874153450 0x7f7874062c60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f786400b9a0 tx=0x7f7864006df0).ready entity=mon.1 client_cookie=60967f5a6ead8955 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f7863fff700 1 -- 172.21.15.116:0/3014519590 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864009680 con 0x7f7874153450 2020-11-20T12:20:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f7863fff700 1 -- 172.21.15.116:0/3014519590 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78640097e0 con 0x7f7874153450 2020-11-20T12:20:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f7863fff700 1 -- 172.21.15.116:0/3014519590 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7864009b60 con 0x7f7874153450 2020-11-20T12:20:19.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f787a46f700 1 -- 172.21.15.116:0/3014519590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7874153450 msgr2=0x7f7874062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f787a46f700 1 --2- 172.21.15.116:0/3014519590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7874153450 0x7f7874062c60 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f786400b9a0 tx=0x7f7864006df0).stop 2020-11-20T12:20:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f787a46f700 1 -- 172.21.15.116:0/3014519590 shutdown_connections 2020-11-20T12:20:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f787a46f700 1 --2- 172.21.15.116:0/3014519590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78740677a0 0x7f78740b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f787a46f700 1 --2- 172.21.15.116:0/3014519590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7874153450 0x7f7874062c60 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:19.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f787a46f700 1 -- 172.21.15.116:0/3014519590 >> 172.21.15.116:0/3014519590 conn(0x7f7874060c40 msgr2=0x7f78740c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f787a46f700 1 -- 172.21.15.116:0/3014519590 shutdown_connections 2020-11-20T12:20:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.892+0000 7f787a46f700 1 -- 172.21.15.116:0/3014519590 wait complete. 2020-11-20T12:20:19.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.896+0000 7f787a46f700 1 Processor -- start 2020-11-20T12:20:19.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.896+0000 7f787a46f700 1 -- start start 2020-11-20T12:20:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.896+0000 7f787a46f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7874062880 0x7f787413cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.896+0000 7f787a46f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78740677a0 0x7f78740a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.896+0000 7f787a46f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7874153450 0x7f78741510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.896+0000 7f787a46f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740a8eb0 con 0x7f7874153450 2020-11-20T12:20:19.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.896+0000 7f787a46f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78740a9150 con 0x7f7874062880 2020-11-20T12:20:19.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.896+0000 7f787a46f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78740a93f0 con 0x7f78740677a0 2020-11-20T12:20:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f7879c6e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7874153450 0x7f78741510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f7879c6e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7874153450 0x7f78741510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49272/0 (socket says 172.21.15.116:49272) 2020-11-20T12:20:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f7879c6e700 1 -- 172.21.15.116:0/3765678989 learned_addr learned my addr 172.21.15.116:0/3765678989 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f787946d700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7874062880 0x7f787413cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f7878c6c700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78740677a0 0x7f78740a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f7879c6e700 1 -- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78740677a0 msgr2=0x7f78740a83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f7879c6e700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78740677a0 0x7f78740a83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f7879c6e700 1 -- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7874062880 msgr2=0x7f787413cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f7879c6e700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7874062880 0x7f787413cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f7879c6e700 1 -- 172.21.15.116:0/3765678989 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78740a9690 con 0x7f7874153450 2020-11-20T12:20:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f7878c6c700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78740677a0 0x7f78740a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f7879c6e700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7874153450 0x7f78741510d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f786c00ef50 tx=0x7f786c0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f78617fa700 1 -- 172.21.15.116:0/3765678989 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c00cc80 con 0x7f7874153450 2020-11-20T12:20:19.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f78617fa700 1 -- 172.21.15.116:0/3765678989 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c00cde0 con 0x7f7874153450 2020-11-20T12:20:19.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f787a46f700 1 -- 172.21.15.116:0/3765678989 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78740a88d0 con 0x7f7874153450 2020-11-20T12:20:19.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f78617fa700 1 -- 172.21.15.116:0/3765678989 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c012620 con 0x7f7874153450 2020-11-20T12:20:19.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f787a46f700 1 -- 172.21.15.116:0/3765678989 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7874151760 con 0x7f7874153450 2020-11-20T12:20:19.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.900+0000 7f787a46f700 1 -- 172.21.15.116:0/3765678989 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787405cf30 con 0x7f7874153450 2020-11-20T12:20:19.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.904+0000 7f78617fa700 1 -- 172.21.15.116:0/3765678989 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f786c022070 con 0x7f7874153450 2020-11-20T12:20:19.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.904+0000 7f78617fa700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f784c0557c0 0x7f784c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:19.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.904+0000 7f78617fa700 1 -- 172.21.15.116:0/3765678989 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f786c06ef70 con 0x7f7874153450 2020-11-20T12:20:19.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.904+0000 7f787946d700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f784c0557c0 0x7f784c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:19.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.904+0000 7f787946d700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f784c0557c0 0x7f784c057c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f7870007fd0 tx=0x7f7870009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:19.908+0000 7f78617fa700 1 -- 172.21.15.116:0/3765678989 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f786c044e10 con 0x7f7874153450 2020-11-20T12:20:20.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.124+0000 7f787a46f700 1 -- 172.21.15.116:0/3765678989 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_evict_client"]} v 0) v1 -- 0x7f78740a9820 con 0x7f7874153450 2020-11-20T12:20:20.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.688+0000 7f78617fa700 1 -- 172.21.15.116:0/3765678989 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_evict_client"]}]=0 v253) v1 ==== 148+0+0 (secure 0 0 0) 0x7f786c016070 con 0x7f7874153450 2020-11-20T12:20:20.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 -- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f784c0557c0 msgr2=0x7f784c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:20.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f784c0557c0 0x7f784c057c80 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f7870007fd0 tx=0x7f7870009040).stop 2020-11-20T12:20:20.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 -- 172.21.15.116:0/3765678989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7874153450 msgr2=0x7f78741510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:20.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7874153450 0x7f78741510d0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f786c00ef50 tx=0x7f786c0091d0).stop 2020-11-20T12:20:20.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 -- 172.21.15.116:0/3765678989 shutdown_connections 2020-11-20T12:20:20.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f784c0557c0 0x7f784c057c80 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:20.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7874153450 0x7f78741510d0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:20.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7874062880 0x7f787413cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:20.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 --2- 172.21.15.116:0/3765678989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f78740677a0 0x7f78740a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:20.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 -- 172.21.15.116:0/3765678989 >> 172.21.15.116:0/3765678989 conn(0x7f7874060c40 msgr2=0x7f78740c0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:20.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 -- 172.21.15.116:0/3765678989 shutdown_connections 2020-11-20T12:20:20.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.692+0000 7f787a46f700 1 -- 172.21.15.116:0/3765678989 wait complete. 2020-11-20T12:20:20.714 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-20T12:20:20.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.904+0000 7f608d3ef700 1 Processor -- start 2020-11-20T12:20:20.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.904+0000 7f608d3ef700 1 -- start start 2020-11-20T12:20:20.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.904+0000 7f608d3ef700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6088156440 0x7f60880aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:20.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f608d3ef700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60880ace40 con 0x7f60880a8400 2020-11-20T12:20:20.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f608d3ef700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60881526c0 con 0x7f60880aa7d0 2020-11-20T12:20:20.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f608d3ef700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6088152960 con 0x7f6088156440 2020-11-20T12:20:20.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f60877fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f60880a8400 0x7f60880ad540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58714/0 (socket says 172.21.15.116:58714) 2020-11-20T12:20:20.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f60877fe700 1 -- 172.21.15.116:0/1106233772 learned_addr learned my addr 172.21.15.116:0/1106233772 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:20.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f6087fff700 1 --1- 172.21.15.116:0/1106233772 >> v1:172.21.15.173:6789/0 conn(0x7f60880aa7d0 0x7f6088151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:54942/0 (socket says 172.21.15.116:54942) 2020-11-20T12:20:20.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f608cbee700 1 --2- 172.21.15.116:0/1106233772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6088156440 0x7f60880aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:20.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f6086ffd700 1 -- 172.21.15.116:0/1106233772 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245314871 0 0) 0x7f60881526c0 con 0x7f60880aa7d0 2020-11-20T12:20:20.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f6086ffd700 1 -- 172.21.15.116:0/1106233772 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6074003240 con 0x7f60880aa7d0 2020-11-20T12:20:20.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f608cbee700 1 -- 172.21.15.116:0/1106233772 >> v1:172.21.15.173:6789/0 conn(0x7f60880aa7d0 legacy=0x7f6088151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:20.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f6086ffd700 1 -- 172.21.15.116:0/1106233772 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292214688 0 0) 0x7f60880ace40 con 0x7f60880a8400 2020-11-20T12:20:20.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f608cbee700 1 -- 172.21.15.116:0/1106233772 >> v1:172.21.15.116:6789/0 conn(0x7f60880a8400 legacy=0x7f60880ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:20.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f608cbee700 1 -- 172.21.15.116:0/1106233772 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6088152c00 con 0x7f6088156440 2020-11-20T12:20:20.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f608cbee700 1 --2- 172.21.15.116:0/1106233772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6088156440 0x7f60880aabb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f6080009cc0 tx=0x7f60800091d0).ready entity=mon.2 client_cookie=f112fe08677f9160 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:20.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f6086ffd700 1 -- 172.21.15.116:0/1106233772 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f608000bad0 con 0x7f6088156440 2020-11-20T12:20:20.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f6086ffd700 1 -- 172.21.15.116:0/1106233772 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608000bc30 con 0x7f6088156440 2020-11-20T12:20:20.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.908+0000 7f6086ffd700 1 -- 172.21.15.116:0/1106233772 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080011400 con 0x7f6088156440 2020-11-20T12:20:20.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.912+0000 7f608d3ef700 1 -- 172.21.15.116:0/1106233772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6088156440 msgr2=0x7f60880aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:20.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.912+0000 7f608d3ef700 1 --2- 172.21.15.116:0/1106233772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6088156440 0x7f60880aabb0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f6080009cc0 tx=0x7f60800091d0).stop 2020-11-20T12:20:20.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.912+0000 7f608d3ef700 1 -- 172.21.15.116:0/1106233772 shutdown_connections 2020-11-20T12:20:20.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.912+0000 7f608d3ef700 1 --2- 172.21.15.116:0/1106233772 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6088156440 0x7f60880aabb0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:20.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.912+0000 7f608d3ef700 1 -- 172.21.15.116:0/1106233772 >> 172.21.15.116:0/1106233772 conn(0x7f60880678a0 msgr2=0x7f6088067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:20.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.912+0000 7f608d3ef700 1 -- 172.21.15.116:0/1106233772 shutdown_connections 2020-11-20T12:20:20.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.912+0000 7f608d3ef700 1 -- 172.21.15.116:0/1106233772 wait complete. 2020-11-20T12:20:20.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f608d3ef700 1 Processor -- start 2020-11-20T12:20:20.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f608d3ef700 1 -- start start 2020-11-20T12:20:20.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f608d3ef700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60880a8400 0x7f6088164260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:20.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f608d3ef700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60880aa7d0 0x7f6088164750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:20.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f608d3ef700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6088156440 0x7f60880a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:20.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f608d3ef700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6088138030 con 0x7f60880a8400 2020-11-20T12:20:20.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f608d3ef700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60881382d0 con 0x7f6088156440 2020-11-20T12:20:20.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f608d3ef700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6088138570 con 0x7f60880aa7d0 2020-11-20T12:20:20.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6087fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60880a8400 0x7f6088164260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:20.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6087fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60880a8400 0x7f6088164260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49284/0 (socket says 172.21.15.116:49284) 2020-11-20T12:20:20.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6087fff700 1 -- 172.21.15.116:0/3918145386 learned_addr learned my addr 172.21.15.116:0/3918145386 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:20.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f60877fe700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60880aa7d0 0x7f6088164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:20.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f608cbee700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6088156440 0x7f60880a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:20.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6087fff700 1 -- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60880aa7d0 msgr2=0x7f6088164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:20.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6087fff700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60880aa7d0 0x7f6088164750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:20.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6087fff700 1 -- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6088156440 msgr2=0x7f60880a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:20.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6087fff700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6088156440 0x7f60880a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:20.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6087fff700 1 -- 172.21.15.116:0/3918145386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6088138810 con 0x7f60880a8400 2020-11-20T12:20:20.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6087fff700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60880a8400 0x7f6088164260 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f607800c710 tx=0x7f6078007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:20.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6067fff700 1 -- 172.21.15.116:0/3918145386 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6078007320 con 0x7f60880a8400 2020-11-20T12:20:20.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6067fff700 1 -- 172.21.15.116:0/3918145386 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6078005530 con 0x7f60880a8400 2020-11-20T12:20:20.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f608d3ef700 1 -- 172.21.15.116:0/3918145386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6088138ab0 con 0x7f60880a8400 2020-11-20T12:20:20.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f6067fff700 1 -- 172.21.15.116:0/3918145386 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60780058b0 con 0x7f60880a8400 2020-11-20T12:20:20.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.916+0000 7f608d3ef700 1 -- 172.21.15.116:0/3918145386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60880a0c80 con 0x7f60880a8400 2020-11-20T12:20:20.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.920+0000 7f608d3ef700 1 -- 172.21.15.116:0/3918145386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f608805cf30 con 0x7f60880a8400 2020-11-20T12:20:20.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.920+0000 7f6067fff700 1 -- 172.21.15.116:0/3918145386 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6078010030 con 0x7f60880a8400 2020-11-20T12:20:20.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.920+0000 7f6067fff700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60600557c0 0x7f6060057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:20.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.920+0000 7f6067fff700 1 -- 172.21.15.116:0/3918145386 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f607806e4d0 con 0x7f60880a8400 2020-11-20T12:20:20.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.920+0000 7f60877fe700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60600557c0 0x7f6060057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:20.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.924+0000 7f60877fe700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60600557c0 0x7f6060057c80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f608814aa20 tx=0x7f607c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:20.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:20.928+0000 7f6067fff700 1 -- 172.21.15.116:0/3918145386 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6078044310 con 0x7f60880a8400 2020-11-20T12:20:21.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.088+0000 7f608d3ef700 1 -- 172.21.15.116:0/3918145386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f60880601b0 con 0x7f60880a8400 2020-11-20T12:20:21.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.096+0000 7f6067fff700 1 -- 172.21.15.116:0/3918145386 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6078047930 con 0x7f60880a8400 2020-11-20T12:20:21.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 -- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60600557c0 msgr2=0x7f6060057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60600557c0 0x7f6060057c80 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f608814aa20 tx=0x7f607c009040).stop 2020-11-20T12:20:21.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 -- 172.21.15.116:0/3918145386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60880a8400 msgr2=0x7f6088164260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60880a8400 0x7f6088164260 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f607800c710 tx=0x7f6078007ad0).stop 2020-11-20T12:20:21.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 -- 172.21.15.116:0/3918145386 shutdown_connections 2020-11-20T12:20:21.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60600557c0 0x7f6060057c80 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6088156440 0x7f60880a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60880a8400 0x7f6088164260 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 --2- 172.21.15.116:0/3918145386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60880aa7d0 0x7f6088164750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 -- 172.21.15.116:0/3918145386 >> 172.21.15.116:0/3918145386 conn(0x7f60880678a0 msgr2=0x7f6088061a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:21.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 -- 172.21.15.116:0/3918145386 shutdown_connections 2020-11-20T12:20:21.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.100+0000 7f608d3ef700 1 -- 172.21.15.116:0/3918145386 wait complete. 2020-11-20T12:20:21.120 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:20:21.121 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:20:21.158 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:20:21.198 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:20:21.199 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2020-11-20T12:20:21.199 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:21.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.380+0000 7f106c4f6700 1 Processor -- start 2020-11-20T12:20:21.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.380+0000 7f106c4f6700 1 -- start start 2020-11-20T12:20:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106c4f6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10640bcb10 0x7f10640bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106c4f6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1064066140 0x7f10640c3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106c4f6700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10640c40e0 con 0x7f106414fe10 2020-11-20T12:20:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106c4f6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1064153570 con 0x7f10640bcb10 2020-11-20T12:20:21.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106c4f6700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10640665a0 con 0x7f1064066140 2020-11-20T12:20:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106a292700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10640bcb10 0x7f10640bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106a292700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10640bcb10 0x7f10640bcef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49292/0 (socket says 172.21.15.116:49292) 2020-11-20T12:20:21.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106a292700 1 -- 172.21.15.116:0/3439079873 learned_addr learned my addr 172.21.15.116:0/3439079873 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f1069a91700 1 --2- 172.21.15.116:0/3439079873 >> v2:172.21.15.173:3300/0 conn(0x7f1064066140 0x7f10640c3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106a292700 1 -- 172.21.15.116:0/3439079873 >> v2:172.21.15.173:3300/0 conn(0x7f1064066140 msgr2=0x7f10640c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:21.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106a292700 1 --2- 172.21.15.116:0/3439079873 >> v2:172.21.15.173:3300/0 conn(0x7f1064066140 0x7f10640c3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106a292700 1 -- 172.21.15.116:0/3439079873 >> v1:172.21.15.173:6789/0 conn(0x7f106414fe10 legacy=0x7f10640c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106a292700 1 -- 172.21.15.116:0/3439079873 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1064066810 con 0x7f10640bcb10 2020-11-20T12:20:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f106a292700 1 --2- 172.21.15.116:0/3439079873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10640bcb10 0x7f10640bcef0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7f1064067ce0 tx=0x7f1060006eb0).ready entity=mon.0 client_cookie=15ccc3e615706f8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:21.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f1069290700 1 -- 172.21.15.116:0/3439079873 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10600098d0 con 0x7f10640bcb10 2020-11-20T12:20:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f1069290700 1 -- 172.21.15.116:0/3439079873 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1060009a30 con 0x7f10640bcb10 2020-11-20T12:20:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.384+0000 7f1069290700 1 -- 172.21.15.116:0/3439079873 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1060009db0 con 0x7f10640bcb10 2020-11-20T12:20:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.388+0000 7f106c4f6700 1 -- 172.21.15.116:0/3439079873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10640bcb10 msgr2=0x7f10640bcef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.388+0000 7f106c4f6700 1 --2- 172.21.15.116:0/3439079873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10640bcb10 0x7f10640bcef0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7f1064067ce0 tx=0x7f1060006eb0).stop 2020-11-20T12:20:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.388+0000 7f106c4f6700 1 -- 172.21.15.116:0/3439079873 shutdown_connections 2020-11-20T12:20:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.388+0000 7f106c4f6700 1 --2- 172.21.15.116:0/3439079873 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10640bcb10 0x7f10640bcef0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.388+0000 7f106c4f6700 1 --2- 172.21.15.116:0/3439079873 >> v2:172.21.15.173:3300/0 conn(0x7f1064066140 0x7f10640c3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.388+0000 7f106c4f6700 1 -- 172.21.15.116:0/3439079873 >> 172.21.15.116:0/3439079873 conn(0x7f10640c7210 msgr2=0x7f10640c31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.388+0000 7f106c4f6700 1 -- 172.21.15.116:0/3439079873 shutdown_connections 2020-11-20T12:20:21.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.388+0000 7f106c4f6700 1 -- 172.21.15.116:0/3439079873 wait complete. 2020-11-20T12:20:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.388+0000 7f106c4f6700 1 Processor -- start 2020-11-20T12:20:21.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.388+0000 7f106c4f6700 1 -- start start 2020-11-20T12:20:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f106c4f6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1064066140 0x7f10640a9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f106c4f6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10640bcb10 0x7f1064161da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f106c4f6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f106414fe10 0x7f106414f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:21.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f106c4f6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1064162290 con 0x7f10640bcb10 2020-11-20T12:20:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f106c4f6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106414f8a0 con 0x7f1064066140 2020-11-20T12:20:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f106c4f6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10640ab300 con 0x7f106414fe10 2020-11-20T12:20:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f1069a91700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10640bcb10 0x7f1064161da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:21.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f1069a91700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10640bcb10 0x7f1064161da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49300/0 (socket says 172.21.15.116:49300) 2020-11-20T12:20:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f1069a91700 1 -- 172.21.15.116:0/388609519 learned_addr learned my addr 172.21.15.116:0/388609519 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f106a292700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1064066140 0x7f10640a9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f106aa93700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f106414fe10 0x7f106414f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:21.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f1069a91700 1 -- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f106414fe10 msgr2=0x7f106414f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f1069a91700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f106414fe10 0x7f106414f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f1069a91700 1 -- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1064066140 msgr2=0x7f10640a9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f1069a91700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1064066140 0x7f10640a9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f1069a91700 1 -- 172.21.15.116:0/388609519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10640ab5a0 con 0x7f10640bcb10 2020-11-20T12:20:21.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f106a292700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1064066140 0x7f10640a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:21.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f1069a91700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10640bcb10 0x7f1064161da0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f105400db60 tx=0x7f1054008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:21.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f10527fc700 1 -- 172.21.15.116:0/388609519 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054009790 con 0x7f10640bcb10 2020-11-20T12:20:21.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f10527fc700 1 -- 172.21.15.116:0/388609519 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10540098f0 con 0x7f10640bcb10 2020-11-20T12:20:21.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f106c4f6700 1 -- 172.21.15.116:0/388609519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10640ab840 con 0x7f10640bcb10 2020-11-20T12:20:21.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.392+0000 7f10527fc700 1 -- 172.21.15.116:0/388609519 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1054009cb0 con 0x7f10640bcb10 2020-11-20T12:20:21.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.396+0000 7f106c4f6700 1 -- 172.21.15.116:0/388609519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10640abd80 con 0x7f10640bcb10 2020-11-20T12:20:21.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.396+0000 7f106c4f6700 1 -- 172.21.15.116:0/388609519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f106405d2a0 con 0x7f10640bcb10 2020-11-20T12:20:21.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.396+0000 7f10527fc700 1 -- 172.21.15.116:0/388609519 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1054012630 con 0x7f10640bcb10 2020-11-20T12:20:21.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.396+0000 7f10527fc700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f103c0557f0 0x7f103c057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:21.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.396+0000 7f10527fc700 1 -- 172.21.15.116:0/388609519 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f105406e9c0 con 0x7f10640bcb10 2020-11-20T12:20:21.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.400+0000 7f106a292700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f103c0557f0 0x7f103c057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:21.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.400+0000 7f106a292700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f103c0557f0 0x7f103c057cb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f1060006ef0 tx=0x7f1060015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:21.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.404+0000 7f10527fc700 1 -- 172.21.15.116:0/388609519 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1054044860 con 0x7f10640bcb10 2020-11-20T12:20:21.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.612+0000 7f106c4f6700 1 -- 172.21.15.116:0/388609519 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f10640645a0 con 0x7f10640bcb10 2020-11-20T12:20:21.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.612+0000 7f10527fc700 1 -- 172.21.15.116:0/388609519 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2848 (secure 0 0 0) 0x7f1054047e80 con 0x7f10640bcb10 2020-11-20T12:20:21.610 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:21.610 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/313210621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":313210621},{"type":"v1","addr":"172.21.15.173:6835","nonce":313210621}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/2140340557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2140340557},{"type":"v1","addr":"172.21.15.173:6837","nonce":2140340557}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3544207168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3544207168},{"type":"v1","addr":"172.21.15.116:6833","nonce":3544207168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:57.303415+0000","modified":"2020-11-20T12:19:59.322603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/657163487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":657163487},{"type":"v1","addr":"172.21.15.116:6835","nonce":657163487}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-20T12:20:21.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 -- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f103c0557f0 msgr2=0x7f103c057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f103c0557f0 0x7f103c057cb0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f1060006ef0 tx=0x7f1060015000).stop 2020-11-20T12:20:21.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 -- 172.21.15.116:0/388609519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10640bcb10 msgr2=0x7f1064161da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10640bcb10 0x7f1064161da0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f105400db60 tx=0x7f1054008040).stop 2020-11-20T12:20:21.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 -- 172.21.15.116:0/388609519 shutdown_connections 2020-11-20T12:20:21.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f103c0557f0 0x7f103c057cb0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f106414fe10 0x7f106414f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1064066140 0x7f10640a9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 --2- 172.21.15.116:0/388609519 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10640bcb10 0x7f1064161da0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 -- 172.21.15.116:0/388609519 >> 172.21.15.116:0/388609519 conn(0x7f10640c7210 msgr2=0x7f1064064840 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:21.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 -- 172.21.15.116:0/388609519 shutdown_connections 2020-11-20T12:20:21.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.616+0000 7f106c4f6700 1 -- 172.21.15.116:0/388609519 wait complete. 2020-11-20T12:20:21.617 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 46 2020-11-20T12:20:21.636 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:21.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.812+0000 7fc327fb1700 1 Processor -- start 2020-11-20T12:20:21.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.812+0000 7fc327fb1700 1 -- start start 2020-11-20T12:20:21.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc327fb1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc320066450 0x7fc320156f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:21.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc327fb1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc320066af0 con 0x7fc3200b9700 2020-11-20T12:20:21.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc327fb1700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc320066d90 con 0x7fc320150680 2020-11-20T12:20:21.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc327fb1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc320066900 con 0x7fc320066450 2020-11-20T12:20:21.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc32654e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc3200b9700 0x7fc3200b9b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58744/0 (socket says 172.21.15.116:58744) 2020-11-20T12:20:21.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc32654e700 1 -- 172.21.15.116:0/2045721818 learned_addr learned my addr 172.21.15.116:0/2045721818 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:21.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc32554c700 1 --2- 172.21.15.116:0/2045721818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc320066450 0x7fc320156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:21.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc324d4b700 1 -- 172.21.15.116:0/2045721818 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 214179379 0 0) 0x7fc320066af0 con 0x7fc3200b9700 2020-11-20T12:20:21.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc324d4b700 1 -- 172.21.15.116:0/2045721818 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc308003240 con 0x7fc3200b9700 2020-11-20T12:20:21.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc324d4b700 1 -- 172.21.15.116:0/2045721818 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651384298 0 0) 0x7fc320066d90 con 0x7fc320150680 2020-11-20T12:20:21.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc324d4b700 1 -- 172.21.15.116:0/2045721818 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc320066af0 con 0x7fc320150680 2020-11-20T12:20:21.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc32554c700 1 -- 172.21.15.116:0/2045721818 >> v1:172.21.15.173:6789/0 conn(0x7fc320150680 legacy=0x7fc3200bf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc32554c700 1 -- 172.21.15.116:0/2045721818 >> v1:172.21.15.116:6789/0 conn(0x7fc3200b9700 legacy=0x7fc3200b9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.816+0000 7fc32554c700 1 -- 172.21.15.116:0/2045721818 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc320067030 con 0x7fc320066450 2020-11-20T12:20:21.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.820+0000 7fc32554c700 1 --2- 172.21.15.116:0/2045721818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc320066450 0x7fc320156f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fc31000b9d0 tx=0x7fc310006df0).ready entity=mon.2 client_cookie=5ef6112f5dedae3c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:21.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.820+0000 7fc324d4b700 1 -- 172.21.15.116:0/2045721818 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310009680 con 0x7fc320066450 2020-11-20T12:20:21.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.820+0000 7fc324d4b700 1 -- 172.21.15.116:0/2045721818 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3100097e0 con 0x7fc320066450 2020-11-20T12:20:21.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.820+0000 7fc324d4b700 1 -- 172.21.15.116:0/2045721818 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc310009b60 con 0x7fc320066450 2020-11-20T12:20:21.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.820+0000 7fc327fb1700 1 -- 172.21.15.116:0/2045721818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc320066450 msgr2=0x7fc320156f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.820+0000 7fc327fb1700 1 --2- 172.21.15.116:0/2045721818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc320066450 0x7fc320156f20 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fc31000b9d0 tx=0x7fc310006df0).stop 2020-11-20T12:20:21.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.820+0000 7fc327fb1700 1 -- 172.21.15.116:0/2045721818 shutdown_connections 2020-11-20T12:20:21.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.820+0000 7fc327fb1700 1 --2- 172.21.15.116:0/2045721818 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc320066450 0x7fc320156f20 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.820+0000 7fc327fb1700 1 -- 172.21.15.116:0/2045721818 >> 172.21.15.116:0/2045721818 conn(0x7fc3200c4f80 msgr2=0x7fc3200c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:21.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.820+0000 7fc327fb1700 1 -- 172.21.15.116:0/2045721818 shutdown_connections 2020-11-20T12:20:21.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.820+0000 7fc327fb1700 1 -- 172.21.15.116:0/2045721818 wait complete. 2020-11-20T12:20:21.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc327fb1700 1 Processor -- start 2020-11-20T12:20:21.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc327fb1700 1 -- start start 2020-11-20T12:20:21.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc327fb1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc320066450 0x7fc3200a4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:21.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc327fb1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3200b9700 0x7fc320169bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:21.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc327fb1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc320150680 0x7fc3200a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:21.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc327fb1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32014f100 con 0x7fc320066450 2020-11-20T12:20:21.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc327fb1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200a7040 con 0x7fc3200b9700 2020-11-20T12:20:21.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc327fb1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3200aa2a0 con 0x7fc320150680 2020-11-20T12:20:21.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc325d4d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc320066450 0x7fc3200a4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:21.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc325d4d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc320066450 0x7fc3200a4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49312/0 (socket says 172.21.15.116:49312) 2020-11-20T12:20:21.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc325d4d700 1 -- 172.21.15.116:0/310275231 learned_addr learned my addr 172.21.15.116:0/310275231 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:21.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc32554c700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3200b9700 0x7fc320169bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:21.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc32654e700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc320150680 0x7fc3200a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:21.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc325d4d700 1 -- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc320150680 msgr2=0x7fc3200a9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc325d4d700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc320150680 0x7fc3200a9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc325d4d700 1 -- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3200b9700 msgr2=0x7fc320169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.824+0000 7fc325d4d700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3200b9700 0x7fc320169bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:21.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.828+0000 7fc325d4d700 1 -- 172.21.15.116:0/310275231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200a6da0 con 0x7fc320066450 2020-11-20T12:20:21.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.828+0000 7fc32554c700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3200b9700 0x7fc320169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:21.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.828+0000 7fc325d4d700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc320066450 0x7fc3200a4220 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7fc31c00c710 tx=0x7fc31c007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:21.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.828+0000 7fc30dffb700 1 -- 172.21.15.116:0/310275231 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c00e560 con 0x7fc320066450 2020-11-20T12:20:21.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.828+0000 7fc30dffb700 1 -- 172.21.15.116:0/310275231 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c00e6c0 con 0x7fc320066450 2020-11-20T12:20:21.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.828+0000 7fc327fb1700 1 -- 172.21.15.116:0/310275231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc320169210 con 0x7fc320066450 2020-11-20T12:20:21.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.828+0000 7fc30dffb700 1 -- 172.21.15.116:0/310275231 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c00ea60 con 0x7fc320066450 2020-11-20T12:20:21.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.828+0000 7fc327fb1700 1 -- 172.21.15.116:0/310275231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc320169750 con 0x7fc320066450 2020-11-20T12:20:21.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.828+0000 7fc327fb1700 1 -- 172.21.15.116:0/310275231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc320169910 con 0x7fc320066450 2020-11-20T12:20:21.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.828+0000 7fc30dffb700 1 -- 172.21.15.116:0/310275231 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc31c019030 con 0x7fc320066450 2020-11-20T12:20:21.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.832+0000 7fc30dffb700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2f80557c0 0x7fc2f8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:21.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.832+0000 7fc30dffb700 1 -- 172.21.15.116:0/310275231 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc31c06db90 con 0x7fc320066450 2020-11-20T12:20:21.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.832+0000 7fc32554c700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2f80557c0 0x7fc2f8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:21.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.832+0000 7fc32554c700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2f80557c0 0x7fc2f8057c80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fc31000b150 tx=0x7fc310015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:21.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.836+0000 7fc30dffb700 1 -- 172.21.15.116:0/310275231 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc31c0439d0 con 0x7fc320066450 2020-11-20T12:20:21.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:21.996+0000 7fc327fb1700 1 -- 172.21.15.116:0/310275231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc32005d2a0 con 0x7fc320066450 2020-11-20T12:20:21.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.000+0000 7fc30dffb700 1 -- 172.21.15.116:0/310275231 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16877 (secure 0 0 0) 0x7fc31c025200 con 0x7fc320066450 2020-11-20T12:20:21.997 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:21.997 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":86,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:58.311412+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:54.974437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:56.326129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:21.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.000+0000 7fc327fb1700 1 -- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2f80557c0 msgr2=0x7fc2f8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:21.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.000+0000 7fc327fb1700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2f80557c0 0x7fc2f8057c80 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fc31000b150 tx=0x7fc310015000).stop 2020-11-20T12:20:22.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.000+0000 7fc327fb1700 1 -- 172.21.15.116:0/310275231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc320066450 msgr2=0x7fc3200a4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.000+0000 7fc327fb1700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc320066450 0x7fc3200a4220 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7fc31c00c710 tx=0x7fc31c007a90).stop 2020-11-20T12:20:22.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.004+0000 7fc327fb1700 1 -- 172.21.15.116:0/310275231 shutdown_connections 2020-11-20T12:20:22.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.004+0000 7fc327fb1700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc2f80557c0 0x7fc2f8057c80 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.004+0000 7fc327fb1700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc320150680 0x7fc3200a9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.004+0000 7fc327fb1700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc320066450 0x7fc3200a4220 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.004+0000 7fc327fb1700 1 --2- 172.21.15.116:0/310275231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc3200b9700 0x7fc320169bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.004+0000 7fc327fb1700 1 -- 172.21.15.116:0/310275231 >> 172.21.15.116:0/310275231 conn(0x7fc3200c4f80 msgr2=0x7fc3200c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:22.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.004+0000 7fc327fb1700 1 -- 172.21.15.116:0/310275231 shutdown_connections 2020-11-20T12:20:22.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.004+0000 7fc327fb1700 1 -- 172.21.15.116:0/310275231 wait complete. 2020-11-20T12:20:22.021 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:19:54.974437+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:19:56.326129+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:20:22.022 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:20:22.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48e5a5b700 1 Processor -- start 2020-11-20T12:20:22.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48e5a5b700 1 -- start start 2020-11-20T12:20:22.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48e5a5b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f48e0065800 0x7f48e0065be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48e5a5b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f48e0150cb0 0x7f48e00c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48e5a5b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e00c1f60 0x7f48e014f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48e5a5b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e0150070 con 0x7f48e0065800 2020-11-20T12:20:22.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48e5a5b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48e01502d0 con 0x7f48e0150cb0 2020-11-20T12:20:22.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48e5a5b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e0150570 con 0x7f48e00c1f60 2020-11-20T12:20:22.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48deffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f48e0065800 0x7f48e0065be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48deffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f48e0065800 0x7f48e0065be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49320/0 (socket says 172.21.15.116:49320) 2020-11-20T12:20:22.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48deffd700 1 -- 172.21.15.116:0/2972241058 learned_addr learned my addr 172.21.15.116:0/2972241058 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:22.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48d67fc700 1 --2- 172.21.15.116:0/2972241058 >> v2:172.21.15.173:3300/0 conn(0x7f48e0150cb0 0x7f48e00c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48dffff700 1 --2- 172.21.15.116:0/2972241058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e00c1f60 0x7f48e014f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48deffd700 1 -- 172.21.15.116:0/2972241058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e00c1f60 msgr2=0x7f48e014f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:22.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48deffd700 1 --2- 172.21.15.116:0/2972241058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e00c1f60 0x7f48e014f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48deffd700 1 -- 172.21.15.116:0/2972241058 >> v2:172.21.15.173:3300/0 conn(0x7f48e0150cb0 msgr2=0x7f48e00c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:22.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48deffd700 1 --2- 172.21.15.116:0/2972241058 >> v2:172.21.15.173:3300/0 conn(0x7f48e0150cb0 0x7f48e00c2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48deffd700 1 -- 172.21.15.116:0/2972241058 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e0150810 con 0x7f48e0065800 2020-11-20T12:20:22.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.196+0000 7f48d67fc700 1 --2- 172.21.15.116:0/2972241058 >> v2:172.21.15.173:3300/0 conn(0x7f48e0150cb0 0x7f48e00c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:22.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48deffd700 1 --2- 172.21.15.116:0/2972241058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e0065800 0x7f48e0065be0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f48e00a9a30 tx=0x7f48d0006eb0).ready entity=mon.0 client_cookie=a628fb890092883b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:22.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48de7fc700 1 -- 172.21.15.116:0/2972241058 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d00098d0 con 0x7f48e0065800 2020-11-20T12:20:22.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48de7fc700 1 -- 172.21.15.116:0/2972241058 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d0009a30 con 0x7f48e0065800 2020-11-20T12:20:22.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48de7fc700 1 -- 172.21.15.116:0/2972241058 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d0009db0 con 0x7f48e0065800 2020-11-20T12:20:22.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48e5a5b700 1 -- 172.21.15.116:0/2972241058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e0065800 msgr2=0x7f48e0065be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48e5a5b700 1 --2- 172.21.15.116:0/2972241058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e0065800 0x7f48e0065be0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f48e00a9a30 tx=0x7f48d0006eb0).stop 2020-11-20T12:20:22.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48e5a5b700 1 -- 172.21.15.116:0/2972241058 shutdown_connections 2020-11-20T12:20:22.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48e5a5b700 1 --2- 172.21.15.116:0/2972241058 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e00c1f60 0x7f48e014f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48e5a5b700 1 --2- 172.21.15.116:0/2972241058 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e0065800 0x7f48e0065be0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48e5a5b700 1 --2- 172.21.15.116:0/2972241058 >> v2:172.21.15.173:3300/0 conn(0x7f48e0150cb0 0x7f48e00c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48e5a5b700 1 -- 172.21.15.116:0/2972241058 >> 172.21.15.116:0/2972241058 conn(0x7f48e00c6480 msgr2=0x7f48e00c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:22.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48e5a5b700 1 -- 172.21.15.116:0/2972241058 shutdown_connections 2020-11-20T12:20:22.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.200+0000 7f48e5a5b700 1 -- 172.21.15.116:0/2972241058 wait complete. 2020-11-20T12:20:22.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48e5a5b700 1 Processor -- start 2020-11-20T12:20:22.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48e5a5b700 1 -- start start 2020-11-20T12:20:22.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48e5a5b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0065800 0x7f48e00be670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48e5a5b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48e00c1f60 0x7f48e0155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48e5a5b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e0150cb0 0x7f48e00c0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48e5a5b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e00bd750 con 0x7f48e0150cb0 2020-11-20T12:20:22.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48e5a5b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48e00bd9f0 con 0x7f48e00c1f60 2020-11-20T12:20:22.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48e5a5b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48e00bdc90 con 0x7f48e0065800 2020-11-20T12:20:22.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48dffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e0150cb0 0x7f48e00c0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48dffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e0150cb0 0x7f48e00c0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49330/0 (socket says 172.21.15.116:49330) 2020-11-20T12:20:22.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48dffff700 1 -- 172.21.15.116:0/4256654445 learned_addr learned my addr 172.21.15.116:0/4256654445 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:22.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48deffd700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0065800 0x7f48e00be670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.204+0000 7f48d67fc700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48e00c1f60 0x7f48e0155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48dffff700 1 -- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0065800 msgr2=0x7f48e00be670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48dffff700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0065800 0x7f48e00be670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48dffff700 1 -- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48e00c1f60 msgr2=0x7f48e0155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48dffff700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48e00c1f60 0x7f48e0155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48dffff700 1 -- 172.21.15.116:0/4256654445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48e00bdf30 con 0x7f48e0150cb0 2020-11-20T12:20:22.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48dffff700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e0150cb0 0x7f48e00c0800 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f48d800eb40 tx=0x7f48d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:22.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48d77fe700 1 -- 172.21.15.116:0/4256654445 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d800c700 con 0x7f48e0150cb0 2020-11-20T12:20:22.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48d77fe700 1 -- 172.21.15.116:0/4256654445 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48d800c860 con 0x7f48e0150cb0 2020-11-20T12:20:22.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48e5a5b700 1 -- 172.21.15.116:0/4256654445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48e00be1d0 con 0x7f48e0150cb0 2020-11-20T12:20:22.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48d77fe700 1 -- 172.21.15.116:0/4256654445 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48d800cc00 con 0x7f48e0150cb0 2020-11-20T12:20:22.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48e5a5b700 1 -- 172.21.15.116:0/4256654445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48e01635d0 con 0x7f48e0150cb0 2020-11-20T12:20:22.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48e5a5b700 1 -- 172.21.15.116:0/4256654445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48e00be360 con 0x7f48e0150cb0 2020-11-20T12:20:22.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.208+0000 7f48d77fe700 1 -- 172.21.15.116:0/4256654445 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f48d8013430 con 0x7f48e0150cb0 2020-11-20T12:20:22.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.212+0000 7f48d77fe700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f48bc0557c0 0x7f48bc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.212+0000 7f48d77fe700 1 -- 172.21.15.116:0/4256654445 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f48d806e2f0 con 0x7f48e0150cb0 2020-11-20T12:20:22.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.212+0000 7f48deffd700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f48bc0557c0 0x7f48bc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.212+0000 7f48deffd700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f48bc0557c0 0x7f48bc057c80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f48d000b1b0 tx=0x7f48d0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:22.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.216+0000 7f48d77fe700 1 -- 172.21.15.116:0/4256654445 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f48d8044130 con 0x7f48e0150cb0 2020-11-20T12:20:22.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.376+0000 7f48e5a5b700 1 -- 172.21.15.116:0/4256654445 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f48e005d2a0 con 0x7f48e0150cb0 2020-11-20T12:20:22.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.376+0000 7f48d77fe700 1 -- 172.21.15.116:0/4256654445 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v86) v1 ==== 110+0+10 (secure 0 0 0) 0x7f48d8047750 con 0x7f48e0150cb0 2020-11-20T12:20:22.375 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:20:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 -- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f48bc0557c0 msgr2=0x7f48bc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f48bc0557c0 0x7f48bc057c80 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f48d000b1b0 tx=0x7f48d0015000).stop 2020-11-20T12:20:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 -- 172.21.15.116:0/4256654445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e0150cb0 msgr2=0x7f48e00c0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e0150cb0 0x7f48e00c0800 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f48d800eb40 tx=0x7f48d8008040).stop 2020-11-20T12:20:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 -- 172.21.15.116:0/4256654445 shutdown_connections 2020-11-20T12:20:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f48bc0557c0 0x7f48bc057c80 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f48e0150cb0 0x7f48e00c0800 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f48e0065800 0x7f48e00be670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 --2- 172.21.15.116:0/4256654445 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f48e00c1f60 0x7f48e0155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 -- 172.21.15.116:0/4256654445 >> 172.21.15.116:0/4256654445 conn(0x7f48e00c6480 msgr2=0x7f48e014f510 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 -- 172.21.15.116:0/4256654445 shutdown_connections 2020-11-20T12:20:22.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.380+0000 7f48e5a5b700 1 -- 172.21.15.116:0/4256654445 wait complete. 2020-11-20T12:20:22.402 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:20:22.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.584+0000 7eff5716c700 1 Processor -- start 2020-11-20T12:20:22.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.584+0000 7eff5716c700 1 -- start start 2020-11-20T12:20:22.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff5716c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7eff500a8400 0x7eff500aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff5716c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff500ace40 con 0x7eff500ad540 2020-11-20T12:20:22.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff5716c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff501526c0 con 0x7eff500aa7d0 2020-11-20T12:20:22.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff5716c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff50152960 con 0x7eff500a8400 2020-11-20T12:20:22.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff5696b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7eff500ad540 0x7eff500ad9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58770/0 (socket says 172.21.15.116:58770) 2020-11-20T12:20:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff5696b700 1 -- 172.21.15.116:0/1286498818 learned_addr learned my addr 172.21.15.116:0/1286498818 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff55969700 1 --2- 172.21.15.116:0/1286498818 >> v2:172.21.15.116:3300/0 conn(0x7eff500a8400 0x7eff500aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff55168700 1 -- 172.21.15.116:0/1286498818 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993794499 0 0) 0x7eff500ace40 con 0x7eff500ad540 2020-11-20T12:20:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff55168700 1 -- 172.21.15.116:0/1286498818 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff38003240 con 0x7eff500ad540 2020-11-20T12:20:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff55168700 1 -- 172.21.15.116:0/1286498818 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845102805 0 0) 0x7eff501526c0 con 0x7eff500aa7d0 2020-11-20T12:20:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff55969700 1 -- 172.21.15.116:0/1286498818 >> v1:172.21.15.173:6789/0 conn(0x7eff500aa7d0 legacy=0x7eff50151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff55969700 1 -- 172.21.15.116:0/1286498818 >> v1:172.21.15.116:6789/0 conn(0x7eff500ad540 legacy=0x7eff500ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.588+0000 7eff55969700 1 -- 172.21.15.116:0/1286498818 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff50152c00 con 0x7eff500a8400 2020-11-20T12:20:22.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff55969700 1 --2- 172.21.15.116:0/1286498818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff500a8400 0x7eff500aabb0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7eff4000b9a0 tx=0x7eff40006df0).ready entity=mon.0 client_cookie=a79138c6c00ab5f3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:22.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff55168700 1 -- 172.21.15.116:0/1286498818 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff400097b0 con 0x7eff500a8400 2020-11-20T12:20:22.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff55168700 1 -- 172.21.15.116:0/1286498818 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff40009910 con 0x7eff500a8400 2020-11-20T12:20:22.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff55168700 1 -- 172.21.15.116:0/1286498818 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40009c90 con 0x7eff500a8400 2020-11-20T12:20:22.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff5716c700 1 -- 172.21.15.116:0/1286498818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff500a8400 msgr2=0x7eff500aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff5716c700 1 --2- 172.21.15.116:0/1286498818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff500a8400 0x7eff500aabb0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7eff4000b9a0 tx=0x7eff40006df0).stop 2020-11-20T12:20:22.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff5716c700 1 -- 172.21.15.116:0/1286498818 shutdown_connections 2020-11-20T12:20:22.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff5716c700 1 --2- 172.21.15.116:0/1286498818 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff500a8400 0x7eff500aabb0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff5716c700 1 -- 172.21.15.116:0/1286498818 >> 172.21.15.116:0/1286498818 conn(0x7eff500678a0 msgr2=0x7eff50067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:22.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff5716c700 1 -- 172.21.15.116:0/1286498818 shutdown_connections 2020-11-20T12:20:22.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff5716c700 1 -- 172.21.15.116:0/1286498818 wait complete. 2020-11-20T12:20:22.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff5716c700 1 Processor -- start 2020-11-20T12:20:22.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.592+0000 7eff5716c700 1 -- start start 2020-11-20T12:20:22.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5716c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500a8400 0x7eff500c0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5716c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff500aa7d0 0x7eff500a88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5716c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff500ad540 0x7eff500a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5716c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff500bfa00 con 0x7eff500aa7d0 2020-11-20T12:20:22.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5716c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff50151320 con 0x7eff500a8400 2020-11-20T12:20:22.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5716c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff500be4f0 con 0x7eff500ad540 2020-11-20T12:20:22.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff55969700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff500aa7d0 0x7eff500a88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff55969700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff500aa7d0 0x7eff500a88e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49342/0 (socket says 172.21.15.116:49342) 2020-11-20T12:20:22.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff55969700 1 -- 172.21.15.116:0/2435990803 learned_addr learned my addr 172.21.15.116:0/2435990803 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:22.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5616a700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500a8400 0x7eff500c0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5696b700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff500ad540 0x7eff500a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff55969700 1 -- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff500ad540 msgr2=0x7eff500a8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff55969700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff500ad540 0x7eff500a8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff55969700 1 -- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500a8400 msgr2=0x7eff500c0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff55969700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500a8400 0x7eff500c0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff55969700 1 -- 172.21.15.116:0/2435990803 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff500bfc60 con 0x7eff500aa7d0 2020-11-20T12:20:22.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5616a700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500a8400 0x7eff500c0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:22.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5696b700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff500ad540 0x7eff500a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:22.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff55969700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff500aa7d0 0x7eff500a88e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7eff40006fd0 tx=0x7eff40015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:22.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff3e7fc700 1 -- 172.21.15.116:0/2435990803 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff40011dd0 con 0x7eff500aa7d0 2020-11-20T12:20:22.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff3e7fc700 1 -- 172.21.15.116:0/2435990803 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4001e460 con 0x7eff500aa7d0 2020-11-20T12:20:22.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5716c700 1 -- 172.21.15.116:0/2435990803 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff500bff00 con 0x7eff500aa7d0 2020-11-20T12:20:22.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff3e7fc700 1 -- 172.21.15.116:0/2435990803 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4001e800 con 0x7eff500aa7d0 2020-11-20T12:20:22.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.596+0000 7eff5716c700 1 -- 172.21.15.116:0/2435990803 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff500c0b30 con 0x7eff500aa7d0 2020-11-20T12:20:22.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.600+0000 7eff5716c700 1 -- 172.21.15.116:0/2435990803 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5005cf30 con 0x7eff500aa7d0 2020-11-20T12:20:22.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.600+0000 7eff3e7fc700 1 -- 172.21.15.116:0/2435990803 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7eff40025030 con 0x7eff500aa7d0 2020-11-20T12:20:22.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.600+0000 7eff3e7fc700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff280557c0 0x7eff28057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:22.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.600+0000 7eff3e7fc700 1 -- 172.21.15.116:0/2435990803 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7eff4006fda0 con 0x7eff500aa7d0 2020-11-20T12:20:22.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.604+0000 7eff5616a700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff280557c0 0x7eff28057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:22.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.604+0000 7eff5616a700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff280557c0 0x7eff28057c80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7eff50061a30 tx=0x7eff4c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:22.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.608+0000 7eff3e7fc700 1 -- 172.21.15.116:0/2435990803 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7eff40045cf0 con 0x7eff500aa7d0 2020-11-20T12:20:22.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.776+0000 7eff5716c700 1 -- 172.21.15.116:0/2435990803 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7eff500aae80 con 0x7eff500aa7d0 2020-11-20T12:20:22.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.776+0000 7eff3e7fc700 1 -- 172.21.15.116:0/2435990803 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v86) v1 ==== 92+0+10 (secure 0 0 0) 0x7eff40023030 con 0x7eff500aa7d0 2020-11-20T12:20:22.773 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:22.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 -- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff280557c0 msgr2=0x7eff28057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff280557c0 0x7eff28057c80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7eff50061a30 tx=0x7eff4c009040).stop 2020-11-20T12:20:22.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 -- 172.21.15.116:0/2435990803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff500aa7d0 msgr2=0x7eff500a88e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:22.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff500aa7d0 0x7eff500a88e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7eff40006fd0 tx=0x7eff40015000).stop 2020-11-20T12:20:22.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 -- 172.21.15.116:0/2435990803 shutdown_connections 2020-11-20T12:20:22.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7eff280557c0 0x7eff28057c80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7eff500ad540 0x7eff500a8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7eff500a8400 0x7eff500c0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 --2- 172.21.15.116:0/2435990803 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7eff500aa7d0 0x7eff500a88e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:22.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 -- 172.21.15.116:0/2435990803 >> 172.21.15.116:0/2435990803 conn(0x7eff500678a0 msgr2=0x7eff5014aa80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:22.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 -- 172.21.15.116:0/2435990803 shutdown_connections 2020-11-20T12:20:22.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.780+0000 7eff5716c700 1 -- 172.21.15.116:0/2435990803 wait complete. 2020-11-20T12:20:22.801 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:20:22.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.984+0000 7fa0cbead700 1 Processor -- start 2020-11-20T12:20:23.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.984+0000 7fa0cbead700 1 -- start start 2020-11-20T12:20:23.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0cbead700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa0c4060310 0x7fa0c40606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0cbead700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c40aa0d0 0x7fa0c4066680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0cbead700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c40a67e0 con 0x7fa0c4066b70 2020-11-20T12:20:23.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0cbead700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c40a6270 con 0x7fa0c4060310 2020-11-20T12:20:23.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0cbead700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40aa530 con 0x7fa0c40aa0d0 2020-11-20T12:20:23.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0ca44a700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa0c4066b70 0x7fa0c40bfd90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58784/0 (socket says 172.21.15.116:58784) 2020-11-20T12:20:23.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0ca44a700 1 -- 172.21.15.116:0/384024274 learned_addr learned my addr 172.21.15.116:0/384024274 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:23.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c9c49700 1 --2- 172.21.15.116:0/384024274 >> v2:172.21.15.116:3300/0 conn(0x7fa0c4060310 0x7fa0c40606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c9448700 1 --2- 172.21.15.116:0/384024274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c40aa0d0 0x7fa0c4066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c8c47700 1 -- 172.21.15.116:0/384024274 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2825182259 0 0) 0x7fa0c40a67e0 con 0x7fa0c4066b70 2020-11-20T12:20:23.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c8c47700 1 -- 172.21.15.116:0/384024274 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0ac003240 con 0x7fa0c4066b70 2020-11-20T12:20:23.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c9c49700 1 -- 172.21.15.116:0/384024274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c40aa0d0 msgr2=0x7fa0c4066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:23.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c9c49700 1 --2- 172.21.15.116:0/384024274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c40aa0d0 0x7fa0c4066680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c9c49700 1 -- 172.21.15.116:0/384024274 >> v1:172.21.15.116:6789/0 conn(0x7fa0c4066b70 legacy=0x7fa0c40bfd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c9c49700 1 -- 172.21.15.116:0/384024274 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40a5fd0 con 0x7fa0c4060310 2020-11-20T12:20:23.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c9c49700 1 --2- 172.21.15.116:0/384024274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c4060310 0x7fa0c40606f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fa0c000b9a0 tx=0x7fa0c0006df0).ready entity=mon.0 client_cookie=9f03f97a7fd23b90 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:23.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c8c47700 1 -- 172.21.15.116:0/384024274 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0009680 con 0x7fa0c4060310 2020-11-20T12:20:23.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c8c47700 1 -- 172.21.15.116:0/384024274 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c00097e0 con 0x7fa0c4060310 2020-11-20T12:20:23.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.988+0000 7fa0c8c47700 1 -- 172.21.15.116:0/384024274 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c0009b60 con 0x7fa0c4060310 2020-11-20T12:20:23.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.992+0000 7fa0cbead700 1 -- 172.21.15.116:0/384024274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c4060310 msgr2=0x7fa0c40606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.992+0000 7fa0cbead700 1 --2- 172.21.15.116:0/384024274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c4060310 0x7fa0c40606f0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fa0c000b9a0 tx=0x7fa0c0006df0).stop 2020-11-20T12:20:23.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.992+0000 7fa0cbead700 1 -- 172.21.15.116:0/384024274 shutdown_connections 2020-11-20T12:20:23.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.992+0000 7fa0cbead700 1 --2- 172.21.15.116:0/384024274 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c4060310 0x7fa0c40606f0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.992+0000 7fa0cbead700 1 --2- 172.21.15.116:0/384024274 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c40aa0d0 0x7fa0c4066680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.992+0000 7fa0cbead700 1 -- 172.21.15.116:0/384024274 >> 172.21.15.116:0/384024274 conn(0x7fa0c405ba80 msgr2=0x7fa0c40c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:23.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.992+0000 7fa0cbead700 1 -- 172.21.15.116:0/384024274 shutdown_connections 2020-11-20T12:20:23.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.992+0000 7fa0cbead700 1 -- 172.21.15.116:0/384024274 wait complete. 2020-11-20T12:20:23.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0cbead700 1 Processor -- start 2020-11-20T12:20:23.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0cbead700 1 -- start start 2020-11-20T12:20:23.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0cbead700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c4066b70 0x7fa0c41662e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0cbead700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c40aa0d0 0x7fa0c40bb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0cbead700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c4154ec0 0x7fa0c4155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0cbead700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4151d50 con 0x7fa0c4154ec0 2020-11-20T12:20:23.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0cbead700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4155c70 con 0x7fa0c4066b70 2020-11-20T12:20:23.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0cbead700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c41559d0 con 0x7fa0c40aa0d0 2020-11-20T12:20:23.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0ca44a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c4154ec0 0x7fa0c4155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0ca44a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c4154ec0 0x7fa0c4155340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49358/0 (socket says 172.21.15.116:49358) 2020-11-20T12:20:23.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0ca44a700 1 -- 172.21.15.116:0/2045271790 learned_addr learned my addr 172.21.15.116:0/2045271790 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:23.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0c9448700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c40aa0d0 0x7fa0c40bb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0c9c49700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c4066b70 0x7fa0c41662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0ca44a700 1 -- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c40aa0d0 msgr2=0x7fa0c40bb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0ca44a700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c40aa0d0 0x7fa0c40bb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0ca44a700 1 -- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c4066b70 msgr2=0x7fa0c41662e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0ca44a700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c4066b70 0x7fa0c41662e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:22.996+0000 7fa0ca44a700 1 -- 172.21.15.116:0/2045271790 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c4165900 con 0x7fa0c4154ec0 2020-11-20T12:20:23.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.000+0000 7fa0ca44a700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c4154ec0 0x7fa0c4155340 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7fa0bc00c9e0 tx=0x7fa0bc007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:23.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.000+0000 7fa0b1ffb700 1 -- 172.21.15.116:0/2045271790 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc005b80 con 0x7fa0c4154ec0 2020-11-20T12:20:23.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.000+0000 7fa0b1ffb700 1 -- 172.21.15.116:0/2045271790 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc005ce0 con 0x7fa0c4154ec0 2020-11-20T12:20:23.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.000+0000 7fa0cbead700 1 -- 172.21.15.116:0/2045271790 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c4165ba0 con 0x7fa0c4154ec0 2020-11-20T12:20:23.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.000+0000 7fa0b1ffb700 1 -- 172.21.15.116:0/2045271790 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc007320 con 0x7fa0c4154ec0 2020-11-20T12:20:23.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.000+0000 7fa0cbead700 1 -- 172.21.15.116:0/2045271790 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40c00a0 con 0x7fa0c4154ec0 2020-11-20T12:20:23.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.000+0000 7fa0cbead700 1 -- 172.21.15.116:0/2045271790 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c4165740 con 0x7fa0c4154ec0 2020-11-20T12:20:23.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.000+0000 7fa0b1ffb700 1 -- 172.21.15.116:0/2045271790 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa0bc010030 con 0x7fa0c4154ec0 2020-11-20T12:20:23.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.000+0000 7fa0b1ffb700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa09c0557f0 0x7fa09c057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.000+0000 7fa0b1ffb700 1 -- 172.21.15.116:0/2045271790 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa0bc06e5d0 con 0x7fa0c4154ec0 2020-11-20T12:20:23.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.004+0000 7fa0c9c49700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa09c0557f0 0x7fa09c057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.004+0000 7fa0c9c49700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa09c0557f0 0x7fa09c057cb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fa0c4064000 tx=0x7fa0c0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:23.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.008+0000 7fa0b1ffb700 1 -- 172.21.15.116:0/2045271790 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa0bc044470 con 0x7fa0c4154ec0 2020-11-20T12:20:23.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.164+0000 7fa0cbead700 1 -- 172.21.15.116:0/2045271790 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa0c405d2a0 con 0x7fa0c4154ec0 2020-11-20T12:20:23.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.164+0000 7fa0b1ffb700 1 -- 172.21.15.116:0/2045271790 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa0bc047a90 con 0x7fa0c4154ec0 2020-11-20T12:20:23.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 -- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa09c0557f0 msgr2=0x7fa09c057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa09c0557f0 0x7fa09c057cb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7fa0c4064000 tx=0x7fa0c0015000).stop 2020-11-20T12:20:23.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 -- 172.21.15.116:0/2045271790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c4154ec0 msgr2=0x7fa0c4155340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c4154ec0 0x7fa0c4155340 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7fa0bc00c9e0 tx=0x7fa0bc007e20).stop 2020-11-20T12:20:23.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 -- 172.21.15.116:0/2045271790 shutdown_connections 2020-11-20T12:20:23.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa09c0557f0 0x7fa09c057cb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa0c4154ec0 0x7fa0c4155340 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0c4066b70 0x7fa0c41662e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 --2- 172.21.15.116:0/2045271790 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0c40aa0d0 0x7fa0c40bb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 -- 172.21.15.116:0/2045271790 >> 172.21.15.116:0/2045271790 conn(0x7fa0c405ba80 msgr2=0x7fa0c4152520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:23.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 -- 172.21.15.116:0/2045271790 shutdown_connections 2020-11-20T12:20:23.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.168+0000 7fa0cbead700 1 -- 172.21.15.116:0/2045271790 wait complete. 2020-11-20T12:20:23.370 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:23.370 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:20:23.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.560+0000 7f26b759e700 1 Processor -- start 2020-11-20T12:20:23.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.560+0000 7f26b759e700 1 -- start start 2020-11-20T12:20:23.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.560+0000 7f26b759e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f26b80a81f0 0x7f26b80b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.560+0000 7f26b759e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b80a8930 con 0x7f26b80c0e90 2020-11-20T12:20:23.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.560+0000 7f26b759e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26b80ba2a0 con 0x7f26b814f880 2020-11-20T12:20:23.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.560+0000 7f26b759e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26b80a8690 con 0x7f26b80a81f0 2020-11-20T12:20:23.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b6d9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f26b80c0e90 0x7f26b80c1320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58798/0 (socket says 172.21.15.116:58798) 2020-11-20T12:20:23.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b6d9d700 1 -- 172.21.15.116:0/2880154989 learned_addr learned my addr 172.21.15.116:0/2880154989 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:23.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b5d9b700 1 --2- 172.21.15.116:0/2880154989 >> v2:172.21.15.116:3300/0 conn(0x7f26b80a81f0 0x7f26b80b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877005892 0 0) 0x7f26b80a8930 con 0x7f26b80c0e90 2020-11-20T12:20:23.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f269c003240 con 0x7f26b80c0e90 2020-11-20T12:20:23.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3300417245 0 0) 0x7f26b80ba2a0 con 0x7f26b814f880 2020-11-20T12:20:23.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b80a8930 con 0x7f26b814f880 2020-11-20T12:20:23.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 4053286922 0 0) 0x7f269c003240 con 0x7f26b80c0e90 2020-11-20T12:20:23.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 >> v2:172.21.15.116:3300/0 conn(0x7f26b80a81f0 msgr2=0x7f26b80b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:23.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 --2- 172.21.15.116:0/2880154989 >> v2:172.21.15.116:3300/0 conn(0x7f26b80a81f0 0x7f26b80b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 >> v1:172.21.15.173:6789/0 conn(0x7f26b814f880 legacy=0x7f26b8060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b80b94d0 con 0x7f26b80c0e90 2020-11-20T12:20:23.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f26b0001860 con 0x7f26b80c0e90 2020-11-20T12:20:23.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 >> v1:172.21.15.116:6789/0 conn(0x7f26b80c0e90 legacy=0x7f26b80c1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c004920 0x7f269c004d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c003480 0x7f269c003900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f269c003df0 0x7f269c00fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80a8690 con 0x7f269c004920 2020-11-20T12:20:23.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f269c010030 con 0x7f269c003480 2020-11-20T12:20:23.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f269c0101f0 con 0x7f269c003df0 2020-11-20T12:20:23.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b659c700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c004920 0x7f269c004d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b5d9b700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c003480 0x7f269c003900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b659c700 1 -- 172.21.15.116:0/2880154989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f269c003df0 msgr2=0x7f269c00fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:23.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b6d9d700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f269c003df0 0x7f269c00fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b659c700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f269c003df0 0x7f269c00fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b659c700 1 -- 172.21.15.116:0/2880154989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c003480 msgr2=0x7f269c003900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:23.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b659c700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c003480 0x7f269c003900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b659c700 1 -- 172.21.15.116:0/2880154989 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b80b94d0 con 0x7f269c004920 2020-11-20T12:20:23.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b6d9d700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f269c003df0 0x7f269c00fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:23.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b5d9b700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c003480 0x7f269c003900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:23.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.564+0000 7f26b6d9d700 1 -- 172.21.15.116:0/2880154989 reap_dead start 2020-11-20T12:20:23.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b659c700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c004920 0x7f269c004d20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f26a8001a80 tx=0x7f26a8007b90).ready entity=mon.0 client_cookie=ead2b54cb3b2509e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:23.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a8007020 con 0x7f269c004920 2020-11-20T12:20:23.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26a8007180 con 0x7f269c004920 2020-11-20T12:20:23.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b559a700 1 -- 172.21.15.116:0/2880154989 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26a800c890 con 0x7f269c004920 2020-11-20T12:20:23.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b759e700 1 -- 172.21.15.116:0/2880154989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c004920 msgr2=0x7f269c004d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b759e700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c004920 0x7f269c004d20 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f26a8001a80 tx=0x7f26a8007b90).stop 2020-11-20T12:20:23.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b759e700 1 -- 172.21.15.116:0/2880154989 shutdown_connections 2020-11-20T12:20:23.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b759e700 1 --2- 172.21.15.116:0/2880154989 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f269c004920 0x7f269c004d20 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b759e700 1 -- 172.21.15.116:0/2880154989 >> 172.21.15.116:0/2880154989 conn(0x7f26b8068350 msgr2=0x7f26b80627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:23.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b759e700 1 -- 172.21.15.116:0/2880154989 shutdown_connections 2020-11-20T12:20:23.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b759e700 1 -- 172.21.15.116:0/2880154989 wait complete. 2020-11-20T12:20:23.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.568+0000 7f26b759e700 1 Processor -- start 2020-11-20T12:20:23.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b759e700 1 -- start start 2020-11-20T12:20:23.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c004920 0x7f26b815d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26b814dc20 0x7f26b814abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b814b0d0 0x7f26b81600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b815cbe0 con 0x7f26b814dc20 2020-11-20T12:20:23.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b8160d00 con 0x7f269c004920 2020-11-20T12:20:23.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b815ce40 con 0x7f26b814b0d0 2020-11-20T12:20:23.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b5d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26b814dc20 0x7f26b814abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b5d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26b814dc20 0x7f26b814abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49378/0 (socket says 172.21.15.116:49378) 2020-11-20T12:20:23.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b5d9b700 1 -- 172.21.15.116:0/2412308841 learned_addr learned my addr 172.21.15.116:0/2412308841 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:23.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b659c700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c004920 0x7f26b815d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b6d9d700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b814b0d0 0x7f26b81600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b5d9b700 1 -- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b814b0d0 msgr2=0x7f26b81600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b5d9b700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b814b0d0 0x7f26b81600a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b5d9b700 1 -- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c004920 msgr2=0x7f26b815d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b5d9b700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c004920 0x7f26b815d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b5d9b700 1 -- 172.21.15.116:0/2412308841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b815d0e0 con 0x7f26b814dc20 2020-11-20T12:20:23.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b6d9d700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b814b0d0 0x7f26b81600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:23.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b5d9b700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26b814dc20 0x7f26b814abe0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f26ac011cf0 tx=0x7f26ac006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:23.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26a67fc700 1 -- 172.21.15.116:0/2412308841 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00faf0 con 0x7f26b814dc20 2020-11-20T12:20:23.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26a67fc700 1 -- 172.21.15.116:0/2412308841 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26ac00fc50 con 0x7f26b814dc20 2020-11-20T12:20:23.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26a67fc700 1 -- 172.21.15.116:0/2412308841 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac014440 con 0x7f26b814dc20 2020-11-20T12:20:23.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b759e700 1 -- 172.21.15.116:0/2412308841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b815da70 con 0x7f26b814dc20 2020-11-20T12:20:23.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.572+0000 7f26b759e700 1 -- 172.21.15.116:0/2412308841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b8160590 con 0x7f26b814dc20 2020-11-20T12:20:23.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.576+0000 7f26b759e700 1 -- 172.21.15.116:0/2412308841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b814b4b0 con 0x7f26b814dc20 2020-11-20T12:20:23.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.576+0000 7f26a67fc700 1 -- 172.21.15.116:0/2412308841 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f26ac014830 con 0x7f26b814dc20 2020-11-20T12:20:23.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.576+0000 7f26a67fc700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f268c0557c0 0x7f268c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.576+0000 7f26a67fc700 1 -- 172.21.15.116:0/2412308841 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f26ac06f280 con 0x7f26b814dc20 2020-11-20T12:20:23.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.576+0000 7f26b659c700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f268c0557c0 0x7f268c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.576+0000 7f26b659c700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f268c0557c0 0x7f268c057c80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f26a8000bc0 tx=0x7f26a8016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:23.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.580+0000 7f26a67fc700 1 -- 172.21.15.116:0/2412308841 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f26ac045120 con 0x7f26b814dc20 2020-11-20T12:20:23.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26b759e700 1 -- 172.21.15.116:0/2412308841 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f26b805cf30 con 0x7f26b814dc20 2020-11-20T12:20:23.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26a67fc700 1 -- 172.21.15.116:0/2412308841 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+10 (secure 0 0 0) 0x7f26ac048740 con 0x7f26b814dc20 2020-11-20T12:20:23.740 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:23.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26b759e700 1 -- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f268c0557c0 msgr2=0x7f268c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26b759e700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f268c0557c0 0x7f268c057c80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f26a8000bc0 tx=0x7f26a8016000).stop 2020-11-20T12:20:23.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26b759e700 1 -- 172.21.15.116:0/2412308841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26b814dc20 msgr2=0x7f26b814abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26b759e700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26b814dc20 0x7f26b814abe0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f26ac011cf0 tx=0x7f26ac006df0).stop 2020-11-20T12:20:23.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26b759e700 1 -- 172.21.15.116:0/2412308841 shutdown_connections 2020-11-20T12:20:23.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26b759e700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f268c0557c0 0x7f268c057c80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26b759e700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26b814b0d0 0x7f26b81600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26b759e700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f269c004920 0x7f26b815d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26b759e700 1 --2- 172.21.15.116:0/2412308841 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26b814dc20 0x7f26b814abe0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.744+0000 7f26b759e700 1 -- 172.21.15.116:0/2412308841 >> 172.21.15.116:0/2412308841 conn(0x7f26b8068350 msgr2=0x7f26b814ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:23.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.748+0000 7f26b759e700 1 -- 172.21.15.116:0/2412308841 shutdown_connections 2020-11-20T12:20:23.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.748+0000 7f26b759e700 1 -- 172.21.15.116:0/2412308841 wait complete. 2020-11-20T12:20:23.764 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:23.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.936+0000 7f47da3d9700 1 Processor -- start 2020-11-20T12:20:23.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.936+0000 7f47da3d9700 1 -- start start 2020-11-20T12:20:23.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.940+0000 7f47da3d9700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f47d40acd20 0x7f47d4152790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.940+0000 7f47da3d9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d40614b0 0x7f47d40618f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.940+0000 7f47da3d9700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47d40ad230 con 0x7f47d414bd80 2020-11-20T12:20:23.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.940+0000 7f47da3d9700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47d40a80d0 con 0x7f47d40acd20 2020-11-20T12:20:23.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.940+0000 7f47da3d9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4061de0 con 0x7f47d40614b0 2020-11-20T12:20:23.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.940+0000 7f47d93d7700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f47d414bd80 0x7f47d40ae150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58816/0 (socket says 172.21.15.116:58816) 2020-11-20T12:20:23.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.940+0000 7f47d93d7700 1 -- 172.21.15.116:0/630944038 learned_addr learned my addr 172.21.15.116:0/630944038 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:23.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.940+0000 7f47d9bd8700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d40614b0 0x7f47d40618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.940+0000 7f47d8bd6700 1 --2- 172.21.15.116:0/630944038 >> v2:172.21.15.173:3300/0 conn(0x7f47d40acd20 0x7f47d4152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.940+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3935302221 0 0) 0x7f47d40ad230 con 0x7f47d414bd80 2020-11-20T12:20:23.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.940+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47bc003240 con 0x7f47d414bd80 2020-11-20T12:20:23.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 699349495 0 0) 0x7f47bc003240 con 0x7f47d414bd80 2020-11-20T12:20:23.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d40614b0 msgr2=0x7f47d40618f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:23.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d40614b0 0x7f47d40618f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 >> v2:172.21.15.173:3300/0 conn(0x7f47d40acd20 msgr2=0x7f47d4152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:23.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 --2- 172.21.15.116:0/630944038 >> v2:172.21.15.173:3300/0 conn(0x7f47d40acd20 0x7f47d4152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40ad4d0 con 0x7f47d414bd80 2020-11-20T12:20:23.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f47d0001860 con 0x7f47d414bd80 2020-11-20T12:20:23.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 >> v1:172.21.15.116:6789/0 conn(0x7f47d414bd80 legacy=0x7f47d40ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47bc0037b0 0x7f47bc003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47bc0044e0 0x7f47bc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d40614b0 0x7f47bc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40a80d0 con 0x7f47bc0037b0 2020-11-20T12:20:23.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d4061de0 con 0x7f47bc0044e0 2020-11-20T12:20:23.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47bc005370 con 0x7f47d40614b0 2020-11-20T12:20:23.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47d93d7700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47bc0037b0 0x7f47bc003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47d8bd6700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47bc0044e0 0x7f47bc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47d9bd8700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d40614b0 0x7f47bc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47d93d7700 1 -- 172.21.15.116:0/630944038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d40614b0 msgr2=0x7f47bc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:23.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47d93d7700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d40614b0 0x7f47bc004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47d93d7700 1 -- 172.21.15.116:0/630944038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47bc0044e0 msgr2=0x7f47bc004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:23.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47d93d7700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47bc0044e0 0x7f47bc004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47d93d7700 1 -- 172.21.15.116:0/630944038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47bc003240 con 0x7f47bc0037b0 2020-11-20T12:20:23.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47d93d7700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47bc0037b0 0x7f47bc003bd0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f47d000c610 tx=0x7f47d4151d50).ready entity=mon.0 client_cookie=fa0d1f967ef8f7a1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:23.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0005b80 con 0x7f47bc0037b0 2020-11-20T12:20:23.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47d0005ce0 con 0x7f47bc0037b0 2020-11-20T12:20:23.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.944+0000 7f47c3fff700 1 -- 172.21.15.116:0/630944038 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47d0007320 con 0x7f47bc0037b0 2020-11-20T12:20:23.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.948+0000 7f47da3d9700 1 -- 172.21.15.116:0/630944038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47bc0037b0 msgr2=0x7f47bc003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.948+0000 7f47da3d9700 1 --2- 172.21.15.116:0/630944038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47bc0037b0 0x7f47bc003bd0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f47d000c610 tx=0x7f47d4151d50).stop 2020-11-20T12:20:23.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.948+0000 7f47d9bd8700 1 -- 172.21.15.116:0/630944038 reap_dead start 2020-11-20T12:20:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.948+0000 7f47da3d9700 1 -- 172.21.15.116:0/630944038 shutdown_connections 2020-11-20T12:20:23.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.948+0000 7f47da3d9700 1 -- 172.21.15.116:0/630944038 >> 172.21.15.116:0/630944038 conn(0x7f47d4068370 msgr2=0x7f47d40c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.948+0000 7f47da3d9700 1 -- 172.21.15.116:0/630944038 shutdown_connections 2020-11-20T12:20:23.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.948+0000 7f47da3d9700 1 -- 172.21.15.116:0/630944038 wait complete. 2020-11-20T12:20:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47da3d9700 1 Processor -- start 2020-11-20T12:20:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47da3d9700 1 -- start start 2020-11-20T12:20:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47da3d9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d414a2c0 0x7f47d414a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47da3d9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47d40aaf80 0x7f47d41471f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47da3d9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47d414ae00 0x7f47d41476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47da3d9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40c8520 con 0x7f47d414ae00 2020-11-20T12:20:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47da3d9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40c87c0 con 0x7f47d40aaf80 2020-11-20T12:20:23.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47da3d9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d40c8a60 con 0x7f47d414a2c0 2020-11-20T12:20:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47d9bd8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47d414ae00 0x7f47d41476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47d9bd8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47d414ae00 0x7f47d41476e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49398/0 (socket says 172.21.15.116:49398) 2020-11-20T12:20:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47d9bd8700 1 -- 172.21.15.116:0/1162971448 learned_addr learned my addr 172.21.15.116:0/1162971448 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:23.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47d8bd6700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47d40aaf80 0x7f47d41471f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47d93d7700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d414a2c0 0x7f47d414a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47d9bd8700 1 -- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d414a2c0 msgr2=0x7f47d414a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47d9bd8700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d414a2c0 0x7f47d414a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47d9bd8700 1 -- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47d40aaf80 msgr2=0x7f47d41471f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:23.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47d9bd8700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47d40aaf80 0x7f47d41471f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47d9bd8700 1 -- 172.21.15.116:0/1162971448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d40c8d00 con 0x7f47d414ae00 2020-11-20T12:20:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.952+0000 7f47d8bd6700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47d40aaf80 0x7f47d41471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.956+0000 7f47d9bd8700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47d414ae00 0x7f47d41476e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f47d4060ae0 tx=0x7f47cc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.956+0000 7f47c17fa700 1 -- 172.21.15.116:0/1162971448 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc018070 con 0x7f47d414ae00 2020-11-20T12:20:23.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.956+0000 7f47c17fa700 1 -- 172.21.15.116:0/1162971448 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47cc00f910 con 0x7f47d414ae00 2020-11-20T12:20:23.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.956+0000 7f47da3d9700 1 -- 172.21.15.116:0/1162971448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d40c8fa0 con 0x7f47d414ae00 2020-11-20T12:20:23.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.956+0000 7f47c17fa700 1 -- 172.21.15.116:0/1162971448 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00fcb0 con 0x7f47d414ae00 2020-11-20T12:20:23.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.956+0000 7f47da3d9700 1 -- 172.21.15.116:0/1162971448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d40c82e0 con 0x7f47d414ae00 2020-11-20T12:20:23.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.956+0000 7f47da3d9700 1 -- 172.21.15.116:0/1162971448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d405cf30 con 0x7f47d414ae00 2020-11-20T12:20:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.956+0000 7f47c17fa700 1 -- 172.21.15.116:0/1162971448 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f47cc014650 con 0x7f47d414ae00 2020-11-20T12:20:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.960+0000 7f47c17fa700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47ac0557c0 0x7f47ac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.960+0000 7f47c17fa700 1 -- 172.21.15.116:0/1162971448 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f47cc06eab0 con 0x7f47d414ae00 2020-11-20T12:20:23.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.960+0000 7f47d93d7700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47ac0557c0 0x7f47ac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:23.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.960+0000 7f47d93d7700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47ac0557c0 0x7f47ac057c80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f47d405f2f0 tx=0x7f47d0013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:23.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:23.964+0000 7f47c17fa700 1 -- 172.21.15.116:0/1162971448 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f47cc044950 con 0x7f47d414ae00 2020-11-20T12:20:24.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.172+0000 7f47da3d9700 1 -- 172.21.15.116:0/1162971448 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47d40c82e0 con 0x7f47d414ae00 2020-11-20T12:20:24.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.176+0000 7f47c17fa700 1 -- 172.21.15.116:0/1162971448 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+2848 (secure 0 0 0) 0x7f47d40c82e0 con 0x7f47d414ae00 2020-11-20T12:20:24.172 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:24.172 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":5987,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/313210621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":313210621},{"type":"v1","addr":"172.21.15.173:6835","nonce":313210621}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":41},{"gid":5990,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/2140340557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":2140340557},{"type":"v1","addr":"172.21.15.173:6837","nonce":2140340557}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42},{"gid":6858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6833/3544207168","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3544207168},{"type":"v1","addr":"172.21.15.116:6833","nonce":3544207168}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:19:57.303415+0000","modified":"2020-11-20T12:19:59.322603+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6861},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6861":{"gid":6861,"name":"a","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/657163487","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":657163487},{"type":"v1","addr":"172.21.15.116:6835","nonce":657163487}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2020-11-20T12:20:24.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.176+0000 7f47da3d9700 1 -- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47ac0557c0 msgr2=0x7f47ac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.176+0000 7f47da3d9700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47ac0557c0 0x7f47ac057c80 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f47d405f2f0 tx=0x7f47d0013000).stop 2020-11-20T12:20:24.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.176+0000 7f47da3d9700 1 -- 172.21.15.116:0/1162971448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47d414ae00 msgr2=0x7f47d41476e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.176+0000 7f47da3d9700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47d414ae00 0x7f47d41476e0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f47d4060ae0 tx=0x7f47cc006df0).stop 2020-11-20T12:20:24.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.176+0000 7f47da3d9700 1 -- 172.21.15.116:0/1162971448 shutdown_connections 2020-11-20T12:20:24.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.176+0000 7f47da3d9700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f47ac0557c0 0x7f47ac057c80 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.176+0000 7f47da3d9700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47d414ae00 0x7f47d41476e0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.176+0000 7f47da3d9700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47d414a2c0 0x7f47d414a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.176+0000 7f47da3d9700 1 --2- 172.21.15.116:0/1162971448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47d40aaf80 0x7f47d41471f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.176+0000 7f47da3d9700 1 -- 172.21.15.116:0/1162971448 >> 172.21.15.116:0/1162971448 conn(0x7f47d4068370 msgr2=0x7f47d4153430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:24.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.180+0000 7f47da3d9700 1 -- 172.21.15.116:0/1162971448 shutdown_connections 2020-11-20T12:20:24.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.180+0000 7f47da3d9700 1 -- 172.21.15.116:0/1162971448 wait complete. 2020-11-20T12:20:24.177 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 46 2020-11-20T12:20:24.198 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:24.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.376+0000 7f0bebade700 1 Processor -- start 2020-11-20T12:20:24.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.376+0000 7f0bebade700 1 -- start start 2020-11-20T12:20:24.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0bebade700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0be4066450 0x7f0be4156f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0bebade700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0be40b9700 0x7f0be40b9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0bebade700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0be4066af0 con 0x7f0be4150680 2020-11-20T12:20:24.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0bebade700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0be4066900 con 0x7f0be40b9700 2020-11-20T12:20:24.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0bebade700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be4150a60 con 0x7f0be4066450 2020-11-20T12:20:24.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0be987a700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0be4150680 0x7f0be40bf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58836/0 (socket says 172.21.15.116:58836) 2020-11-20T12:20:24.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0be987a700 1 -- 172.21.15.116:0/505262285 learned_addr learned my addr 172.21.15.116:0/505262285 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:24.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0bea07b700 1 --2- 172.21.15.116:0/505262285 >> v2:172.21.15.173:3300/0 conn(0x7f0be40b9700 0x7f0be40b9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0be9079700 1 --2- 172.21.15.116:0/505262285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0be4066450 0x7f0be4156f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0be8878700 1 -- 172.21.15.116:0/505262285 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862827988 0 0) 0x7f0be4066af0 con 0x7f0be4150680 2020-11-20T12:20:24.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0be8878700 1 -- 172.21.15.116:0/505262285 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bcc003240 con 0x7f0be4150680 2020-11-20T12:20:24.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0bea07b700 1 -- 172.21.15.116:0/505262285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0be4066450 msgr2=0x7f0be4156f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:24.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0bea07b700 1 --2- 172.21.15.116:0/505262285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0be4066450 0x7f0be4156f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0bea07b700 1 -- 172.21.15.116:0/505262285 >> v1:172.21.15.116:6789/0 conn(0x7f0be4150680 legacy=0x7f0be40bf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0bea07b700 1 -- 172.21.15.116:0/505262285 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be4066d90 con 0x7f0be40b9700 2020-11-20T12:20:24.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0bea07b700 1 --2- 172.21.15.116:0/505262285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be40b9700 0x7f0be40b9b70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f0bdc009c90 tx=0x7f0bdc0091d0).ready entity=mon.1 client_cookie=4f1abae40b5e09a5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:24.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0be8878700 1 -- 172.21.15.116:0/505262285 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00b9c0 con 0x7f0be40b9700 2020-11-20T12:20:24.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0be8878700 1 -- 172.21.15.116:0/505262285 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc00bb20 con 0x7f0be40b9700 2020-11-20T12:20:24.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.380+0000 7f0be8878700 1 -- 172.21.15.116:0/505262285 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc00bea0 con 0x7f0be40b9700 2020-11-20T12:20:24.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.384+0000 7f0bebade700 1 -- 172.21.15.116:0/505262285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be40b9700 msgr2=0x7f0be40b9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.384+0000 7f0bebade700 1 --2- 172.21.15.116:0/505262285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be40b9700 0x7f0be40b9b70 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f0bdc009c90 tx=0x7f0bdc0091d0).stop 2020-11-20T12:20:24.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.384+0000 7f0bebade700 1 -- 172.21.15.116:0/505262285 shutdown_connections 2020-11-20T12:20:24.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.384+0000 7f0bebade700 1 --2- 172.21.15.116:0/505262285 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be40b9700 0x7f0be40b9b70 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.384+0000 7f0bebade700 1 --2- 172.21.15.116:0/505262285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0be4066450 0x7f0be4156f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.384+0000 7f0bebade700 1 -- 172.21.15.116:0/505262285 >> 172.21.15.116:0/505262285 conn(0x7f0be40c4f80 msgr2=0x7f0be40c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:24.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.384+0000 7f0bebade700 1 -- 172.21.15.116:0/505262285 shutdown_connections 2020-11-20T12:20:24.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.384+0000 7f0bebade700 1 -- 172.21.15.116:0/505262285 wait complete. 2020-11-20T12:20:24.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0bebade700 1 Processor -- start 2020-11-20T12:20:24.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0bebade700 1 -- start start 2020-11-20T12:20:24.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0bebade700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be4066450 0x7f0be40a4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0bebade700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0be40b9700 0x7f0be40a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0bebade700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0be4150680 0x7f0be40a7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0bebade700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be4150130 con 0x7f0be4150680 2020-11-20T12:20:24.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0bebade700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0be40a4fd0 con 0x7f0be4066450 2020-11-20T12:20:24.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0bebade700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0be40a5270 con 0x7f0be40b9700 2020-11-20T12:20:24.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0bea07b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0be4150680 0x7f0be40a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0bea07b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0be4150680 0x7f0be40a7960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49412/0 (socket says 172.21.15.116:49412) 2020-11-20T12:20:24.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0bea07b700 1 -- 172.21.15.116:0/180762124 learned_addr learned my addr 172.21.15.116:0/180762124 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:24.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0be987a700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be4066450 0x7f0be40a4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.388+0000 7f0be9079700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0be40b9700 0x7f0be40a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bea07b700 1 -- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0be40b9700 msgr2=0x7f0be40a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bea07b700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0be40b9700 0x7f0be40a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bea07b700 1 -- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be4066450 msgr2=0x7f0be40a4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bea07b700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be4066450 0x7f0be40a4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bea07b700 1 -- 172.21.15.116:0/180762124 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be40a5510 con 0x7f0be4150680 2020-11-20T12:20:24.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bea07b700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0be4150680 0x7f0be40a7960 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f0be405c8b0 tx=0x7f0bdc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:24.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bd1ffb700 1 -- 172.21.15.116:0/180762124 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc01c6c0 con 0x7f0be4150680 2020-11-20T12:20:24.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bd1ffb700 1 -- 172.21.15.116:0/180762124 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bdc01c820 con 0x7f0be4150680 2020-11-20T12:20:24.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bebade700 1 -- 172.21.15.116:0/180762124 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0be40a57b0 con 0x7f0be4150680 2020-11-20T12:20:24.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bd1ffb700 1 -- 172.21.15.116:0/180762124 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bdc01cba0 con 0x7f0be4150680 2020-11-20T12:20:24.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bebade700 1 -- 172.21.15.116:0/180762124 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0be40a5ef0 con 0x7f0be4150680 2020-11-20T12:20:24.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.392+0000 7f0bebade700 1 -- 172.21.15.116:0/180762124 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0be405d2a0 con 0x7f0be4150680 2020-11-20T12:20:24.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.396+0000 7f0bd1ffb700 1 -- 172.21.15.116:0/180762124 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0bdc024030 con 0x7f0be4150680 2020-11-20T12:20:24.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.396+0000 7f0bd1ffb700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bbc0557c0 0x7f0bbc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.396+0000 7f0bd1ffb700 1 -- 172.21.15.116:0/180762124 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0bdc070120 con 0x7f0be4150680 2020-11-20T12:20:24.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.396+0000 7f0be987a700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bbc0557c0 0x7f0bbc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.400+0000 7f0be987a700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bbc0557c0 0x7f0bbc057c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f0be0007fd0 tx=0x7f0be0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:24.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.400+0000 7f0bd1ffb700 1 -- 172.21.15.116:0/180762124 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0bdc0460f0 con 0x7f0be4150680 2020-11-20T12:20:24.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.564+0000 7f0bebade700 1 -- 172.21.15.116:0/180762124 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0be40a3d80 con 0x7f0be4150680 2020-11-20T12:20:24.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.564+0000 7f0bd1ffb700 1 -- 172.21.15.116:0/180762124 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f0bdc030200 con 0x7f0be4150680 2020-11-20T12:20:24.561 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:24.562 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":86,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:19:58.311412+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:19:54.974437+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2020-11-20T12:19:56.326129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:24.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.564+0000 7f0bebade700 1 -- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bbc0557c0 msgr2=0x7f0bbc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.564+0000 7f0bebade700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bbc0557c0 0x7f0bbc057c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f0be0007fd0 tx=0x7f0be0009040).stop 2020-11-20T12:20:24.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.564+0000 7f0bebade700 1 -- 172.21.15.116:0/180762124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0be4150680 msgr2=0x7f0be40a7960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.564+0000 7f0bebade700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0be4150680 0x7f0be40a7960 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f0be405c8b0 tx=0x7f0bdc016000).stop 2020-11-20T12:20:24.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.568+0000 7f0bebade700 1 -- 172.21.15.116:0/180762124 shutdown_connections 2020-11-20T12:20:24.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.568+0000 7f0bebade700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0bbc0557c0 0x7f0bbc057c80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.568+0000 7f0bebade700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0be4150680 0x7f0be40a7960 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.568+0000 7f0bebade700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0be4066450 0x7f0be40a4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.568+0000 7f0bebade700 1 --2- 172.21.15.116:0/180762124 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0be40b9700 0x7f0be40a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.568+0000 7f0bebade700 1 -- 172.21.15.116:0/180762124 >> 172.21.15.116:0/180762124 conn(0x7f0be40c4f80 msgr2=0x7f0be4154770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:24.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.568+0000 7f0bebade700 1 -- 172.21.15.116:0/180762124 shutdown_connections 2020-11-20T12:20:24.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.568+0000 7f0bebade700 1 -- 172.21.15.116:0/180762124 wait complete. 2020-11-20T12:20:24.586 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-20T12:20:24.587 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-20T12:20:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.780+0000 7f9c2b23b700 1 Processor -- start 2020-11-20T12:20:24.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.780+0000 7f9c2b23b700 1 -- start start 2020-11-20T12:20:24.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2b23b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c2414b9a0 0x7f9c240a46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2b23b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9c24066830 0x7f9c24066cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2b23b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c240a5790 con 0x7f9c2414e3e0 2020-11-20T12:20:24.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2b23b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c2414f290 con 0x7f9c24066830 2020-11-20T12:20:24.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2b23b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240a4fb0 con 0x7f9c2414b9a0 2020-11-20T12:20:24.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2aa3a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9c24066830 0x7f9c24066cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c29a38700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f9c2414e3e0 0x7f9c240c0c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:55080/0 (socket says 172.21.15.116:55080) 2020-11-20T12:20:24.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c29a38700 1 -- 172.21.15.116:0/998350573 learned_addr learned my addr 172.21.15.116:0/998350573 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:24.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2a239700 1 --2- 172.21.15.116:0/998350573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c2414b9a0 0x7f9c240a46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2aa3a700 1 -- 172.21.15.116:0/998350573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c2414b9a0 msgr2=0x7f9c240a46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:24.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2aa3a700 1 --2- 172.21.15.116:0/998350573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c2414b9a0 0x7f9c240a46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2aa3a700 1 -- 172.21.15.116:0/998350573 >> v1:172.21.15.173:6789/0 conn(0x7f9c2414e3e0 legacy=0x7f9c240c0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2aa3a700 1 -- 172.21.15.116:0/998350573 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c240a5250 con 0x7f9c24066830 2020-11-20T12:20:24.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c2aa3a700 1 --2- 172.21.15.116:0/998350573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c24066830 0x7f9c24066cc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f9c1c009c90 tx=0x7f9c1c0091d0).ready entity=mon.1 client_cookie=ec833d2e86423fae server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:24.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c29237700 1 -- 172.21.15.116:0/998350573 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00ba70 con 0x7f9c24066830 2020-11-20T12:20:24.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c29237700 1 -- 172.21.15.116:0/998350573 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c00bbd0 con 0x7f9c24066830 2020-11-20T12:20:24.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.784+0000 7f9c29237700 1 -- 172.21.15.116:0/998350573 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c011400 con 0x7f9c24066830 2020-11-20T12:20:24.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.788+0000 7f9c2b23b700 1 -- 172.21.15.116:0/998350573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c24066830 msgr2=0x7f9c24066cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.788+0000 7f9c2b23b700 1 --2- 172.21.15.116:0/998350573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c24066830 0x7f9c24066cc0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f9c1c009c90 tx=0x7f9c1c0091d0).stop 2020-11-20T12:20:24.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.788+0000 7f9c2b23b700 1 -- 172.21.15.116:0/998350573 shutdown_connections 2020-11-20T12:20:24.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.788+0000 7f9c2b23b700 1 --2- 172.21.15.116:0/998350573 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c24066830 0x7f9c24066cc0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.788+0000 7f9c2b23b700 1 --2- 172.21.15.116:0/998350573 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c2414b9a0 0x7f9c240a46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.788+0000 7f9c2b23b700 1 -- 172.21.15.116:0/998350573 >> 172.21.15.116:0/998350573 conn(0x7f9c24060170 msgr2=0x7f9c24060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:24.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.788+0000 7f9c2b23b700 1 -- 172.21.15.116:0/998350573 shutdown_connections 2020-11-20T12:20:24.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.788+0000 7f9c2b23b700 1 -- 172.21.15.116:0/998350573 wait complete. 2020-11-20T12:20:24.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2b23b700 1 Processor -- start 2020-11-20T12:20:24.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2b23b700 1 -- start start 2020-11-20T12:20:24.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2b23b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c24066830 0x7f9c24144240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2b23b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c2414b9a0 0x7f9c24158d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2b23b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c2414e3e0 0x7f9c24159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2b23b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240a2090 con 0x7f9c2414e3e0 2020-11-20T12:20:24.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2b23b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240a2330 con 0x7f9c2414b9a0 2020-11-20T12:20:24.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2b23b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c240a25d0 con 0x7f9c24066830 2020-11-20T12:20:24.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2aa3a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c2414e3e0 0x7f9c24159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2aa3a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c2414e3e0 0x7f9c24159200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49426/0 (socket says 172.21.15.116:49426) 2020-11-20T12:20:24.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2aa3a700 1 -- 172.21.15.116:0/1855534106 learned_addr learned my addr 172.21.15.116:0/1855534106 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:24.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c2a239700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c24066830 0x7f9c24144240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.792+0000 7f9c29a38700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c2414b9a0 0x7f9c24158d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c2aa3a700 1 -- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c24066830 msgr2=0x7f9c24144240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c2aa3a700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c24066830 0x7f9c24144240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c2aa3a700 1 -- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c2414b9a0 msgr2=0x7f9c24158d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:24.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c2aa3a700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c2414b9a0 0x7f9c24158d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:24.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c2aa3a700 1 -- 172.21.15.116:0/1855534106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c240a2870 con 0x7f9c2414e3e0 2020-11-20T12:20:24.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c29a38700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c2414b9a0 0x7f9c24158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:24.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c2aa3a700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c2414e3e0 0x7f9c24159200 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f9c1c00b770 tx=0x7f9c1c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:24.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c127fc700 1 -- 172.21.15.116:0/1855534106 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c010780 con 0x7f9c2414e3e0 2020-11-20T12:20:24.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c127fc700 1 -- 172.21.15.116:0/1855534106 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1c0108e0 con 0x7f9c2414e3e0 2020-11-20T12:20:24.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c127fc700 1 -- 172.21.15.116:0/1855534106 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c010c60 con 0x7f9c2414e3e0 2020-11-20T12:20:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c2b23b700 1 -- 172.21.15.116:0/1855534106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c24144730 con 0x7f9c2414e3e0 2020-11-20T12:20:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c2b23b700 1 -- 172.21.15.116:0/1855534106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c240a2d10 con 0x7f9c2414e3e0 2020-11-20T12:20:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.796+0000 7f9c2b23b700 1 -- 172.21.15.116:0/1855534106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c240a2a00 con 0x7f9c2414e3e0 2020-11-20T12:20:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.800+0000 7f9c127fc700 1 -- 172.21.15.116:0/1855534106 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9c1c024030 con 0x7f9c2414e3e0 2020-11-20T12:20:24.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.800+0000 7f9c127fc700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9bfc0557c0 0x7f9bfc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:24.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.800+0000 7f9c127fc700 1 -- 172.21.15.116:0/1855534106 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9c1c0700c0 con 0x7f9c2414e3e0 2020-11-20T12:20:24.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.800+0000 7f9c2a239700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9bfc0557c0 0x7f9bfc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:24.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.800+0000 7f9c2a239700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9bfc0557c0 0x7f9bfc057c80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f9c20006fd0 tx=0x7f9c20006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:24.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:24.804+0000 7f9c127fc700 1 -- 172.21.15.116:0/1855534106 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9c1c046010 con 0x7f9c2414e3e0 2020-11-20T12:20:25.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.012+0000 7f9c2b23b700 1 -- 172.21.15.116:0/1855534106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f9c2405cd40 con 0x7f9c2414e3e0 2020-11-20T12:20:25.716 INFO:tasks.ceph.mon.a.smithi116.stderr:2020-11-20T12:20:25.720+0000 7fc111d1b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-20T12:20:25.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.732+0000 7f9c127fc700 1 -- 172.21.15.116:0/1855534106 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7f9c1c022020 con 0x7f9c2414e3e0 2020-11-20T12:20:25.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 -- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9bfc0557c0 msgr2=0x7f9bfc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:25.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9bfc0557c0 0x7f9bfc057c80 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f9c20006fd0 tx=0x7f9c20006df0).stop 2020-11-20T12:20:25.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 -- 172.21.15.116:0/1855534106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c2414e3e0 msgr2=0x7f9c24159200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:25.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c2414e3e0 0x7f9c24159200 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f9c1c00b770 tx=0x7f9c1c016000).stop 2020-11-20T12:20:25.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 -- 172.21.15.116:0/1855534106 shutdown_connections 2020-11-20T12:20:25.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9bfc0557c0 0x7f9bfc057c80 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:25.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c2414e3e0 0x7f9c24159200 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:25.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c24066830 0x7f9c24144240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:25.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 --2- 172.21.15.116:0/1855534106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c2414b9a0 0x7f9c24158d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:25.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 -- 172.21.15.116:0/1855534106 >> 172.21.15.116:0/1855534106 conn(0x7f9c24060170 msgr2=0x7f9c240a36b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:25.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 -- 172.21.15.116:0/1855534106 shutdown_connections 2020-11-20T12:20:25.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.736+0000 7f9c2b23b700 1 -- 172.21.15.116:0/1855534106 wait complete. 2020-11-20T12:20:25.736 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-20T12:20:25.739 INFO:tasks.ceph.mds.a.smithi116.stderr: -1582> 2020-11-20T12:20:00.628+0000 7f02a8452700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:20:25.760 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-20T12:20:25.776 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:20:25.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.940+0000 7f7d14a91700 1 Processor -- start 2020-11-20T12:20:25.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.940+0000 7f7d14a91700 1 -- start start 2020-11-20T12:20:25.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.940+0000 7f7d14a91700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d100a8400 0x7f7d100aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:25.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.940+0000 7f7d14a91700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d100ad540 0x7f7d100ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:25.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.940+0000 7f7d14a91700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d100ace40 con 0x7f7d100aa7d0 2020-11-20T12:20:25.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.940+0000 7f7d14a91700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d101526c0 con 0x7f7d100a8400 2020-11-20T12:20:25.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.940+0000 7f7d14a91700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10152960 con 0x7f7d100ad540 2020-11-20T12:20:25.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.940+0000 7f7d06ffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d100a8400 0x7f7d100aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:25.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.940+0000 7f7d06ffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7d100a8400 0x7f7d100aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49444/0 (socket says 172.21.15.116:49444) 2020-11-20T12:20:25.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.940+0000 7f7d06ffd700 1 -- 172.21.15.116:0/3044329521 learned_addr learned my addr 172.21.15.116:0/3044329521 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:25.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.940+0000 7f7d0ffff700 1 --2- 172.21.15.116:0/3044329521 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d100ad540 0x7f7d100ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:25.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d06ffd700 1 -- 172.21.15.116:0/3044329521 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d100ad540 msgr2=0x7f7d100ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:25.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d06ffd700 1 --2- 172.21.15.116:0/3044329521 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d100ad540 0x7f7d100ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:25.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d06ffd700 1 -- 172.21.15.116:0/3044329521 >> v1:172.21.15.173:6789/0 conn(0x7f7d100aa7d0 legacy=0x7f7d10151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:25.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d06ffd700 1 -- 172.21.15.116:0/3044329521 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10152c00 con 0x7f7d100a8400 2020-11-20T12:20:25.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d06ffd700 1 --2- 172.21.15.116:0/3044329521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d100a8400 0x7f7d100aabb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f7cf8006fd0 tx=0x7f7cf8006eb0).ready entity=mon.0 client_cookie=a9f5d3e9ec82bb2b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:25.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d0effd700 1 -- 172.21.15.116:0/3044329521 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8009810 con 0x7f7d100a8400 2020-11-20T12:20:25.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d0effd700 1 -- 172.21.15.116:0/3044329521 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf8009970 con 0x7f7d100a8400 2020-11-20T12:20:25.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d0effd700 1 -- 172.21.15.116:0/3044329521 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf8009cf0 con 0x7f7d100a8400 2020-11-20T12:20:25.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d14a91700 1 -- 172.21.15.116:0/3044329521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d100a8400 msgr2=0x7f7d100aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:25.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d14a91700 1 --2- 172.21.15.116:0/3044329521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d100a8400 0x7f7d100aabb0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f7cf8006fd0 tx=0x7f7cf8006eb0).stop 2020-11-20T12:20:25.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d14a91700 1 -- 172.21.15.116:0/3044329521 shutdown_connections 2020-11-20T12:20:25.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d14a91700 1 --2- 172.21.15.116:0/3044329521 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d100ad540 0x7f7d100ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:25.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d14a91700 1 --2- 172.21.15.116:0/3044329521 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d100a8400 0x7f7d100aabb0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:25.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d14a91700 1 -- 172.21.15.116:0/3044329521 >> 172.21.15.116:0/3044329521 conn(0x7f7d100678a0 msgr2=0x7f7d10067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:25.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d14a91700 1 -- 172.21.15.116:0/3044329521 shutdown_connections 2020-11-20T12:20:25.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.944+0000 7f7d14a91700 1 -- 172.21.15.116:0/3044329521 wait complete. 2020-11-20T12:20:25.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d14a91700 1 Processor -- start 2020-11-20T12:20:25.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d14a91700 1 -- start start 2020-11-20T12:20:25.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d14a91700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d100aa7d0 0x7f7d1014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:25.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d14a91700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d100ad540 0x7f7d101619b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:25.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d14a91700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d1014d540 0x7f7d10149790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:25.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d14a91700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10150310 con 0x7f7d100aa7d0 2020-11-20T12:20:25.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d14a91700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d101629a0 con 0x7f7d1014d540 2020-11-20T12:20:25.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d14a91700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d101621c0 con 0x7f7d100ad540 2020-11-20T12:20:25.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d0f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d100aa7d0 0x7f7d1014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:25.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d0f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d100aa7d0 0x7f7d1014fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49448/0 (socket says 172.21.15.116:49448) 2020-11-20T12:20:25.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d0ffff700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d1014d540 0x7f7d10149790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:25.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d0f7fe700 1 -- 172.21.15.116:0/3206989844 learned_addr learned my addr 172.21.15.116:0/3206989844 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:25.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.948+0000 7f7d06ffd700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d100ad540 0x7f7d101619b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:25.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d0f7fe700 1 -- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d100ad540 msgr2=0x7f7d101619b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:25.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d0f7fe700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d100ad540 0x7f7d101619b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:25.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d0f7fe700 1 -- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d1014d540 msgr2=0x7f7d10149790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:25.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d0f7fe700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d1014d540 0x7f7d10149790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:25.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d0f7fe700 1 -- 172.21.15.116:0/3206989844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d10162460 con 0x7f7d100aa7d0 2020-11-20T12:20:25.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d0f7fe700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d100aa7d0 0x7f7d1014fe20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f7d0000c710 tx=0x7f7d00007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:25.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d07fff700 1 -- 172.21.15.116:0/3206989844 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000e4a0 con 0x7f7d100aa7d0 2020-11-20T12:20:25.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d07fff700 1 -- 172.21.15.116:0/3206989844 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0000e600 con 0x7f7d100aa7d0 2020-11-20T12:20:25.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d07fff700 1 -- 172.21.15.116:0/3206989844 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0000e9a0 con 0x7f7d100aa7d0 2020-11-20T12:20:25.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d14a91700 1 -- 172.21.15.116:0/3206989844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d10162700 con 0x7f7d100aa7d0 2020-11-20T12:20:25.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d14a91700 1 -- 172.21.15.116:0/3206989844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d10161ea0 con 0x7f7d100aa7d0 2020-11-20T12:20:25.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.952+0000 7f7d14a91700 1 -- 172.21.15.116:0/3206989844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d1005cf30 con 0x7f7d100aa7d0 2020-11-20T12:20:25.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.956+0000 7f7d07fff700 1 -- 172.21.15.116:0/3206989844 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7d00010030 con 0x7f7d100aa7d0 2020-11-20T12:20:25.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.956+0000 7f7d07fff700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7cec0557c0 0x7f7cec057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:25.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.956+0000 7f7d07fff700 1 -- 172.21.15.116:0/3206989844 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7d0006e560 con 0x7f7d100aa7d0 2020-11-20T12:20:25.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.956+0000 7f7d06ffd700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7cec0557c0 0x7f7cec057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:25.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.956+0000 7f7d06ffd700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7cec0557c0 0x7f7cec057c80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f7d10068770 tx=0x7f7cf8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:25.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:25.960+0000 7f7d07fff700 1 -- 172.21.15.116:0/3206989844 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7d000442a0 con 0x7f7d100aa7d0 2020-11-20T12:20:26.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.168+0000 7f7d14a91700 1 -- 172.21.15.116:0/3206989844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7d1014c250 con 0x7f7d100aa7d0 2020-11-20T12:20:26.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.740+0000 7f7d07fff700 1 -- 172.21.15.116:0/3206989844 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7d000478c0 con 0x7f7d100aa7d0 2020-11-20T12:20:26.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 -- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7cec0557c0 msgr2=0x7f7cec057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:26.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7cec0557c0 0x7f7cec057c80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f7d10068770 tx=0x7f7cf8015000).stop 2020-11-20T12:20:26.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 -- 172.21.15.116:0/3206989844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d100aa7d0 msgr2=0x7f7d1014fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:26.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d100aa7d0 0x7f7d1014fe20 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f7d0000c710 tx=0x7f7d00007c10).stop 2020-11-20T12:20:26.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 -- 172.21.15.116:0/3206989844 shutdown_connections 2020-11-20T12:20:26.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7cec0557c0 0x7f7cec057c80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:26.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7d1014d540 0x7f7d10149790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:26.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7d100aa7d0 0x7f7d1014fe20 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:26.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 --2- 172.21.15.116:0/3206989844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7d100ad540 0x7f7d101619b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:26.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 -- 172.21.15.116:0/3206989844 >> 172.21.15.116:0/3206989844 conn(0x7f7d100678a0 msgr2=0x7f7d1014afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:26.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 -- 172.21.15.116:0/3206989844 shutdown_connections 2020-11-20T12:20:26.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:26.744+0000 7f7d14a91700 1 -- 172.21.15.116:0/3206989844 wait complete. 2020-11-20T12:20:26.769 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-20T12:20:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.004+0000 7f2a99346700 1 Processor -- start 2020-11-20T12:20:27.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.004+0000 7f2a99346700 1 -- start start 2020-11-20T12:20:27.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.004+0000 7f2a99346700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2a941390e0 0x7f2a940a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.004+0000 7f2a99346700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a94062690 con 0x7f2a94138d00 2020-11-20T12:20:27.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.004+0000 7f2a99346700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a94150520 con 0x7f2a94066f00 2020-11-20T12:20:27.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.004+0000 7f2a99346700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a941507c0 con 0x7f2a941390e0 2020-11-20T12:20:27.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.004+0000 7f2a93fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2a94138d00 0x7f2a9414fcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58890/0 (socket says 172.21.15.116:58890) 2020-11-20T12:20:27.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.004+0000 7f2a93fff700 1 -- 172.21.15.116:0/21147197 learned_addr learned my addr 172.21.15.116:0/21147197 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:27.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.004+0000 7f2a98b45700 1 --2- 172.21.15.116:0/21147197 >> v2:172.21.15.116:3300/0 conn(0x7f2a941390e0 0x7f2a940a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a937fe700 1 -- 172.21.15.116:0/21147197 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113827035 0 0) 0x7f2a94062690 con 0x7f2a94138d00 2020-11-20T12:20:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a937fe700 1 -- 172.21.15.116:0/21147197 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a74003240 con 0x7f2a94138d00 2020-11-20T12:20:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a98b45700 1 -- 172.21.15.116:0/21147197 >> v1:172.21.15.173:6789/0 conn(0x7f2a94066f00 legacy=0x7f2a94150df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a98b45700 1 -- 172.21.15.116:0/21147197 >> v1:172.21.15.116:6789/0 conn(0x7f2a94138d00 legacy=0x7f2a9414fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a98b45700 1 -- 172.21.15.116:0/21147197 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94150a60 con 0x7f2a941390e0 2020-11-20T12:20:27.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a98b45700 1 --2- 172.21.15.116:0/21147197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a941390e0 0x7f2a940a97c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f2a8c009c90 tx=0x7f2a8c0091d0).ready entity=mon.0 client_cookie=525bc143c26d04bc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:27.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a937fe700 1 -- 172.21.15.116:0/21147197 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c00ba70 con 0x7f2a941390e0 2020-11-20T12:20:27.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a937fe700 1 -- 172.21.15.116:0/21147197 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a8c00bbd0 con 0x7f2a941390e0 2020-11-20T12:20:27.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a937fe700 1 -- 172.21.15.116:0/21147197 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a8c011400 con 0x7f2a941390e0 2020-11-20T12:20:27.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a99346700 1 -- 172.21.15.116:0/21147197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a941390e0 msgr2=0x7f2a940a97c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a99346700 1 --2- 172.21.15.116:0/21147197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a941390e0 0x7f2a940a97c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f2a8c009c90 tx=0x7f2a8c0091d0).stop 2020-11-20T12:20:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a99346700 1 -- 172.21.15.116:0/21147197 shutdown_connections 2020-11-20T12:20:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a99346700 1 --2- 172.21.15.116:0/21147197 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a941390e0 0x7f2a940a97c0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a99346700 1 -- 172.21.15.116:0/21147197 >> 172.21.15.116:0/21147197 conn(0x7f2a940682e0 msgr2=0x7f2a94060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:27.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a99346700 1 -- 172.21.15.116:0/21147197 shutdown_connections 2020-11-20T12:20:27.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.008+0000 7f2a99346700 1 -- 172.21.15.116:0/21147197 wait complete. 2020-11-20T12:20:27.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.012+0000 7f2a99346700 1 Processor -- start 2020-11-20T12:20:27.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.012+0000 7f2a99346700 1 -- start start 2020-11-20T12:20:27.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a99346700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a94066f00 0x7f2a940c38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a99346700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a94138d00 0x7f2a941342d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a99346700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a941390e0 0x7f2a941347c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a99346700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94134cb0 con 0x7f2a94138d00 2020-11-20T12:20:27.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a99346700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a940bd630 con 0x7f2a941390e0 2020-11-20T12:20:27.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a99346700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a94162660 con 0x7f2a94066f00 2020-11-20T12:20:27.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a8b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a94138d00 0x7f2a941342d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a93fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a94066f00 0x7f2a940c38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a93fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a94066f00 0x7f2a940c38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:58412/0 (socket says 172.21.15.116:58412) 2020-11-20T12:20:27.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a93fff700 1 -- 172.21.15.116:0/636580662 learned_addr learned my addr 172.21.15.116:0/636580662 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:27.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a98b45700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a941390e0 0x7f2a941347c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a8b7fe700 1 -- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a94066f00 msgr2=0x7f2a940c38f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a8b7fe700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a94066f00 0x7f2a940c38f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a8b7fe700 1 -- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a941390e0 msgr2=0x7f2a941347c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a8b7fe700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a941390e0 0x7f2a941347c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a8b7fe700 1 -- 172.21.15.116:0/636580662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a94162900 con 0x7f2a94138d00 2020-11-20T12:20:27.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.016+0000 7f2a8b7fe700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a94138d00 0x7f2a941342d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f2a7c00c6e0 tx=0x7f2a7c007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:27.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.020+0000 7f2a90ff9700 1 -- 172.21.15.116:0/636580662 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c007020 con 0x7f2a94138d00 2020-11-20T12:20:27.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.020+0000 7f2a90ff9700 1 -- 172.21.15.116:0/636580662 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c007180 con 0x7f2a94138d00 2020-11-20T12:20:27.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.020+0000 7f2a90ff9700 1 -- 172.21.15.116:0/636580662 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c0064f0 con 0x7f2a94138d00 2020-11-20T12:20:27.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.020+0000 7f2a99346700 1 -- 172.21.15.116:0/636580662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a94162ba0 con 0x7f2a94138d00 2020-11-20T12:20:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.020+0000 7f2a99346700 1 -- 172.21.15.116:0/636580662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a940c3de0 con 0x7f2a94138d00 2020-11-20T12:20:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.020+0000 7f2a99346700 1 -- 172.21.15.116:0/636580662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a940c4060 con 0x7f2a94138d00 2020-11-20T12:20:27.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.020+0000 7f2a90ff9700 1 -- 172.21.15.116:0/636580662 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2a7c010030 con 0x7f2a94138d00 2020-11-20T12:20:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.024+0000 7f2a90ff9700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a700557c0 0x7f2a70057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.024+0000 7f2a90ff9700 1 -- 172.21.15.116:0/636580662 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2a7c06dd90 con 0x7f2a94138d00 2020-11-20T12:20:27.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.024+0000 7f2a93fff700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a700557c0 0x7f2a70057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.024+0000 7f2a93fff700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a700557c0 0x7f2a70057c80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f2a84001770 tx=0x7f2a84009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:27.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.028+0000 7f2a90ff9700 1 -- 172.21.15.116:0/636580662 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2a7c043be0 con 0x7f2a94138d00 2020-11-20T12:20:27.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.188+0000 7f2a99346700 1 -- 172.21.15.116:0/636580662 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2a9405cf30 con 0x7f2a94138d00 2020-11-20T12:20:27.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.748+0000 7f2a90ff9700 1 -- 172.21.15.116:0/636580662 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v88) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2a7c047200 con 0x7f2a94138d00 2020-11-20T12:20:27.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 -- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a700557c0 msgr2=0x7f2a70057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a700557c0 0x7f2a70057c80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f2a84001770 tx=0x7f2a84009040).stop 2020-11-20T12:20:27.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 -- 172.21.15.116:0/636580662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a94138d00 msgr2=0x7f2a941342d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a94138d00 0x7f2a941342d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7f2a7c00c6e0 tx=0x7f2a7c007b90).stop 2020-11-20T12:20:27.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 -- 172.21.15.116:0/636580662 shutdown_connections 2020-11-20T12:20:27.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a700557c0 0x7f2a70057c80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2a941390e0 0x7f2a941347c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2a94066f00 0x7f2a940c38f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 --2- 172.21.15.116:0/636580662 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2a94138d00 0x7f2a941342d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 -- 172.21.15.116:0/636580662 >> 172.21.15.116:0/636580662 conn(0x7f2a940682e0 msgr2=0x7f2a94066410 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:27.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 -- 172.21.15.116:0/636580662 shutdown_connections 2020-11-20T12:20:27.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.752+0000 7f2a99346700 1 -- 172.21.15.116:0/636580662 wait complete. 2020-11-20T12:20:27.752 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2020-11-20T12:20:27.776 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-20T12:20:27.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.968+0000 7f0ef9ee2700 1 Processor -- start 2020-11-20T12:20:27.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.968+0000 7f0ef9ee2700 1 -- start start 2020-11-20T12:20:27.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.968+0000 7f0ef9ee2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ef4062860 0x7f0ef4062c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef9ee2700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef40bb2b0 con 0x7f0ef414f820 2020-11-20T12:20:27.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef9ee2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef40bada0 con 0x7f0ef40c4f10 2020-11-20T12:20:27.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef9ee2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef414fc70 con 0x7f0ef4062860 2020-11-20T12:20:27.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef3fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0ef414f820 0x7f0ef414bda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58906/0 (socket says 172.21.15.116:58906) 2020-11-20T12:20:27.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef3fff700 1 -- 172.21.15.116:0/1744092226 learned_addr learned my addr 172.21.15.116:0/1744092226 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:27.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef37fe700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ef4062860 0x7f0ef4062c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 17609944 0 0) 0x7f0ef40bb2b0 con 0x7f0ef414f820 2020-11-20T12:20:27.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ee0003240 con 0x7f0ef414f820 2020-11-20T12:20:27.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 784045101 0 0) 0x7f0ef40bada0 con 0x7f0ef40c4f10 2020-11-20T12:20:27.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ef40bb2b0 con 0x7f0ef40c4f10 2020-11-20T12:20:27.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2531169748 0 0) 0x7f0ee0003240 con 0x7f0ef414f820 2020-11-20T12:20:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ef4062860 msgr2=0x7f0ef4062c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ef4062860 0x7f0ef4062c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 >> v1:172.21.15.173:6789/0 conn(0x7f0ef40c4f10 legacy=0x7f0ef40ba690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef40bb010 con 0x7f0ef414f820 2020-11-20T12:20:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f0eec001860 con 0x7f0ef414f820 2020-11-20T12:20:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 >> v1:172.21.15.116:6789/0 conn(0x7f0ef414f820 legacy=0x7f0ef414bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ee00038f0 0x7f0ee0003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee00044e0 0x7f0ee0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0004e50 0x7f0ee000d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef414fc70 con 0x7f0ee0004e50 2020-11-20T12:20:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee000de30 con 0x7f0ee00044e0 2020-11-20T12:20:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee000df70 con 0x7f0ee00038f0 2020-11-20T12:20:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef3fff700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0004e50 0x7f0ee000d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef37fe700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ee00038f0 0x7f0ee0003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.972+0000 7f0ef2ffd700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee00044e0 0x7f0ee0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef3fff700 1 -- 172.21.15.116:0/1744092226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ee00038f0 msgr2=0x7f0ee0003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:27.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef3fff700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ee00038f0 0x7f0ee0003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef3fff700 1 -- 172.21.15.116:0/1744092226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee00044e0 msgr2=0x7f0ee0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef3fff700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ee00044e0 0x7f0ee0004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef3fff700 1 -- 172.21.15.116:0/1744092226 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0003240 con 0x7f0ee0004e50 2020-11-20T12:20:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef3fff700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0004e50 0x7f0ee000d940 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f0eec00c630 tx=0x7f0ef40bb1a0).ready entity=mon.0 client_cookie=c19dcbd936f04ed1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec005b80 con 0x7f0ee0004e50 2020-11-20T12:20:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec005ce0 con 0x7f0ee0004e50 2020-11-20T12:20:27.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef27fc700 1 -- 172.21.15.116:0/1744092226 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec007320 con 0x7f0ee0004e50 2020-11-20T12:20:27.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/1744092226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0004e50 msgr2=0x7f0ee000d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef9ee2700 1 --2- 172.21.15.116:0/1744092226 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ee0004e50 0x7f0ee000d940 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f0eec00c630 tx=0x7f0ef40bb1a0).stop 2020-11-20T12:20:27.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef3fff700 1 -- 172.21.15.116:0/1744092226 reap_dead start 2020-11-20T12:20:27.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/1744092226 shutdown_connections 2020-11-20T12:20:27.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/1744092226 >> 172.21.15.116:0/1744092226 conn(0x7f0ef40682c0 msgr2=0x7f0ef40c4470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:27.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/1744092226 shutdown_connections 2020-11-20T12:20:27.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.976+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/1744092226 wait complete. 2020-11-20T12:20:27.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef9ee2700 1 Processor -- start 2020-11-20T12:20:27.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef9ee2700 1 -- start start 2020-11-20T12:20:27.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef9ee2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ef40c2f30 0x7f0ef40c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef9ee2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ef40c0130 0x7f0ef4144e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef9ee2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ef40c05b0 0x7f0ef4145330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef9ee2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4145820 con 0x7f0ef40c05b0 2020-11-20T12:20:27.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef9ee2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef4149210 con 0x7f0ef40c2f30 2020-11-20T12:20:27.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef9ee2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef40beea0 con 0x7f0ef40c0130 2020-11-20T12:20:27.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ef40c05b0 0x7f0ef4145330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ef40c05b0 0x7f0ef4145330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49486/0 (socket says 172.21.15.116:49486) 2020-11-20T12:20:27.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef3fff700 1 -- 172.21.15.116:0/209952482 learned_addr learned my addr 172.21.15.116:0/209952482 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:27.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef2ffd700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ef40c0130 0x7f0ef4144e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.980+0000 7f0ef37fe700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ef40c2f30 0x7f0ef40c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ef3fff700 1 -- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ef40c0130 msgr2=0x7f0ef4144e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ef3fff700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ef40c0130 0x7f0ef4144e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ef3fff700 1 -- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ef40c2f30 msgr2=0x7f0ef40c3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:27.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ef3fff700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ef40c2f30 0x7f0ef40c3310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:27.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ef3fff700 1 -- 172.21.15.116:0/209952482 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef4148cd0 con 0x7f0ef40c05b0 2020-11-20T12:20:27.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ef3fff700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ef40c05b0 0x7f0ef4145330 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f0eec008fd0 tx=0x7f0eec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:27.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ed37fe700 1 -- 172.21.15.116:0/209952482 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec007750 con 0x7f0ef40c05b0 2020-11-20T12:20:27.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ed37fe700 1 -- 172.21.15.116:0/209952482 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec0148c0 con 0x7f0ef40c05b0 2020-11-20T12:20:27.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ed37fe700 1 -- 172.21.15.116:0/209952482 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec0078b0 con 0x7f0ef40c05b0 2020-11-20T12:20:27.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/209952482 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef4148f70 con 0x7f0ef40c05b0 2020-11-20T12:20:27.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/209952482 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef40bf5a0 con 0x7f0ef40c05b0 2020-11-20T12:20:27.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/209952482 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ef405d2a0 con 0x7f0ef40c05b0 2020-11-20T12:20:27.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.984+0000 7f0ed37fe700 1 -- 172.21.15.116:0/209952482 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0eec010070 con 0x7f0ef40c05b0 2020-11-20T12:20:27.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.988+0000 7f0ed37fe700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ecc0557c0 0x7f0ecc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:27.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.988+0000 7f0ed37fe700 1 -- 172.21.15.116:0/209952482 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f0eec06f610 con 0x7f0ef40c05b0 2020-11-20T12:20:27.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.988+0000 7f0ef37fe700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ecc0557c0 0x7f0ecc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:27.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.988+0000 7f0ef37fe700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ecc0557c0 0x7f0ecc057c80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f0ee4006d00 tx=0x7f0ee4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:27.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:27.992+0000 7f0ed37fe700 1 -- 172.21.15.116:0/209952482 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0eec045610 con 0x7f0ef40c05b0 2020-11-20T12:20:28.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.160+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/209952482 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0ef4062330 con 0x7f0ef40c05b0 2020-11-20T12:20:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.760+0000 7f0ed37fe700 1 -- 172.21.15.116:0/209952482 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v89) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0eec048c30 con 0x7f0ef40c05b0 2020-11-20T12:20:28.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.764+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ecc0557c0 msgr2=0x7f0ecc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:28.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.764+0000 7f0ef9ee2700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ecc0557c0 0x7f0ecc057c80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f0ee4006d00 tx=0x7f0ee4008040).stop 2020-11-20T12:20:28.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.764+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/209952482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ef40c05b0 msgr2=0x7f0ef4145330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:28.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.764+0000 7f0ef9ee2700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ef40c05b0 0x7f0ef4145330 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7f0eec008fd0 tx=0x7f0eec015000).stop 2020-11-20T12:20:28.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.764+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/209952482 shutdown_connections 2020-11-20T12:20:28.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.764+0000 7f0ef9ee2700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ecc0557c0 0x7f0ecc057c80 secure :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0x7f0ee4006d00 tx=0x7f0ee4008040).stop 2020-11-20T12:20:28.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.764+0000 7f0ef9ee2700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ef40c05b0 0x7f0ef4145330 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:28.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.764+0000 7f0ef9ee2700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ef40c2f30 0x7f0ef40c3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:28.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.764+0000 7f0ef9ee2700 1 --2- 172.21.15.116:0/209952482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ef40c0130 0x7f0ef4144e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:28.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.764+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/209952482 >> 172.21.15.116:0/209952482 conn(0x7f0ef40682c0 msgr2=0x7f0ef40a9780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:28.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.768+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/209952482 shutdown_connections 2020-11-20T12:20:28.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:28.768+0000 7f0ef9ee2700 1 -- 172.21.15.116:0/209952482 wait complete. 2020-11-20T12:20:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2020-11-20T12:20:28.787 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T12:20:28.788 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-20T12:20:28.788 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-20T12:20:28.788 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:20:28.789 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-20T12:20:28.789 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-20T12:20:28.789 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-20T12:20:28.790 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:20:28.790 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T12:20:28.790 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-20T12:20:28.791 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-20T12:20:28.791 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:20:28.791 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-20T12:20:28.791 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-20T12:20:28.792 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-20T12:20:28.793 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:20:28.811 INFO:tasks.ceph.mds.b:Stopped 2020-11-20T12:20:28.812 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T12:20:28.814 INFO:tasks.ceph.mds.a:Stopped 2020-11-20T12:20:28.814 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T12:20:28.818 INFO:tasks.ceph.mds.b:Started 2020-11-20T12:20:28.819 INFO:tasks.ceph.mds.d:Stopped 2020-11-20T12:20:28.819 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-20T12:20:28.820 DEBUG:teuthology.parallel:result is None 2020-11-20T12:20:28.821 INFO:tasks.ceph.mds.a:Started 2020-11-20T12:20:28.822 INFO:tasks.ceph.mds.c:Stopped 2020-11-20T12:20:28.822 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-20T12:20:28.823 DEBUG:teuthology.parallel:result is None 2020-11-20T12:20:28.825 INFO:tasks.ceph.mds.d:Started 2020-11-20T12:20:28.825 DEBUG:teuthology.parallel:result is None 2020-11-20T12:20:28.827 INFO:tasks.ceph.mds.c:Started 2020-11-20T12:20:28.827 DEBUG:teuthology.parallel:result is None 2020-11-20T12:20:28.828 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-20T12:20:28.920 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:20:28.923 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2020-11-20T12:20:28.930 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:20:28.932 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2020-11-20T12:20:29.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.008+0000 7f66e73f3700 1 Processor -- start 2020-11-20T12:20:29.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.008+0000 7f66e73f3700 1 -- start start 2020-11-20T12:20:29.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.008+0000 7f66e73f3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f66e00c5420 0x7f66e00c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:29.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.008+0000 7f66e73f3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c3880 0x7f66e00c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:29.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e73f3700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66e0155580 con 0x7f66e0151440 2020-11-20T12:20:29.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e73f3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66e01557b0 con 0x7f66e00c5420 2020-11-20T12:20:29.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e73f3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0155a50 con 0x7f66e00c3880 2020-11-20T12:20:29.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e518f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f66e00c5420 0x7f66e00c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:29.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e5990700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f66e0151440 0x7f66e01518d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58958/0 (socket says 172.21.15.116:58958) 2020-11-20T12:20:29.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e5990700 1 -- 172.21.15.116:0/1906754220 learned_addr learned my addr 172.21.15.116:0/1906754220 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:29.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e498e700 1 --2- 172.21.15.116:0/1906754220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c3880 0x7f66e00c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:29.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66d7fff700 1 -- 172.21.15.116:0/1906754220 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 93256320 0 0) 0x7f66e0155580 con 0x7f66e0151440 2020-11-20T12:20:29.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66d7fff700 1 -- 172.21.15.116:0/1906754220 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66c4003240 con 0x7f66e0151440 2020-11-20T12:20:29.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e498e700 1 -- 172.21.15.116:0/1906754220 >> v2:172.21.15.116:3300/0 conn(0x7f66e00c5420 msgr2=0x7f66e00c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:29.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e498e700 1 --2- 172.21.15.116:0/1906754220 >> v2:172.21.15.116:3300/0 conn(0x7f66e00c5420 0x7f66e00c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:29.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e498e700 1 -- 172.21.15.116:0/1906754220 >> v1:172.21.15.116:6789/0 conn(0x7f66e0151440 legacy=0x7f66e01518d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:29.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e498e700 1 -- 172.21.15.116:0/1906754220 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e0155cf0 con 0x7f66e00c3880 2020-11-20T12:20:29.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e498e700 1 --2- 172.21.15.116:0/1906754220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c3880 0x7f66e00c3ce0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f66d000b9a0 tx=0x7f66d0006df0).ready entity=mon.2 client_cookie=bee1c6e9b6100ea1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:29.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66d7fff700 1 -- 172.21.15.116:0/1906754220 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0009680 con 0x7f66e00c3880 2020-11-20T12:20:29.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66d7fff700 1 -- 172.21.15.116:0/1906754220 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66d00097e0 con 0x7f66e00c3880 2020-11-20T12:20:29.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66d7fff700 1 -- 172.21.15.116:0/1906754220 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66d0009b60 con 0x7f66e00c3880 2020-11-20T12:20:29.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e73f3700 1 -- 172.21.15.116:0/1906754220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c3880 msgr2=0x7f66e00c3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:29.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e73f3700 1 --2- 172.21.15.116:0/1906754220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c3880 0x7f66e00c3ce0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f66d000b9a0 tx=0x7f66d0006df0).stop 2020-11-20T12:20:29.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e73f3700 1 -- 172.21.15.116:0/1906754220 shutdown_connections 2020-11-20T12:20:29.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e73f3700 1 --2- 172.21.15.116:0/1906754220 >> v2:172.21.15.116:3300/0 conn(0x7f66e00c5420 0x7f66e00c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:29.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e73f3700 1 --2- 172.21.15.116:0/1906754220 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c3880 0x7f66e00c3ce0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:29.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.012+0000 7f66e73f3700 1 -- 172.21.15.116:0/1906754220 >> 172.21.15.116:0/1906754220 conn(0x7f66e00678a0 msgr2=0x7f66e0067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:29.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.016+0000 7f66e73f3700 1 -- 172.21.15.116:0/1906754220 shutdown_connections 2020-11-20T12:20:29.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.016+0000 7f66e73f3700 1 -- 172.21.15.116:0/1906754220 wait complete. 2020-11-20T12:20:29.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.016+0000 7f66e73f3700 1 Processor -- start 2020-11-20T12:20:29.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.016+0000 7f66e73f3700 1 -- start start 2020-11-20T12:20:29.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e73f3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e00c3880 0x7f66e01532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:29.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e73f3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c5420 0x7f66e0168850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:29.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e73f3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e0151440 0x7f66e0154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:29.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e73f3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0165280 con 0x7f66e00c3880 2020-11-20T12:20:29.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e73f3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66e014abf0 con 0x7f66e0151440 2020-11-20T12:20:29.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e73f3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66e0168d40 con 0x7f66e00c5420 2020-11-20T12:20:29.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e518f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e00c3880 0x7f66e01532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:29.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e518f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e00c3880 0x7f66e01532f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49526/0 (socket says 172.21.15.116:49526) 2020-11-20T12:20:29.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e518f700 1 -- 172.21.15.116:0/940985943 learned_addr learned my addr 172.21.15.116:0/940985943 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:29.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e5990700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e0151440 0x7f66e0154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:29.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e498e700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c5420 0x7f66e0168850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:29.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e518f700 1 -- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c5420 msgr2=0x7f66e0168850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:29.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e518f700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c5420 0x7f66e0168850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:29.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e518f700 1 -- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e0151440 msgr2=0x7f66e0154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:29.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e518f700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e0151440 0x7f66e0154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:29.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e518f700 1 -- 172.21.15.116:0/940985943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e0168fd0 con 0x7f66e00c3880 2020-11-20T12:20:29.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e498e700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c5420 0x7f66e0168850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:29.022 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2020-11-20T12:20:29.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e518f700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e00c3880 0x7f66e01532f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f66dc00eba0 tx=0x7f66dc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:29.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66d57fa700 1 -- 172.21.15.116:0/940985943 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc00ca60 con 0x7f66e00c3880 2020-11-20T12:20:29.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66d57fa700 1 -- 172.21.15.116:0/940985943 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66dc00cbc0 con 0x7f66e00c3880 2020-11-20T12:20:29.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e73f3700 1 -- 172.21.15.116:0/940985943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66e0169270 con 0x7f66e00c3880 2020-11-20T12:20:29.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66d57fa700 1 -- 172.21.15.116:0/940985943 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66dc0123f0 con 0x7f66e00c3880 2020-11-20T12:20:29.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.020+0000 7f66e73f3700 1 -- 172.21.15.116:0/940985943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e0165470 con 0x7f66e00c3880 2020-11-20T12:20:29.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.024+0000 7f66e73f3700 1 -- 172.21.15.116:0/940985943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66e005d2a0 con 0x7f66e00c3880 2020-11-20T12:20:29.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.024+0000 7f66d57fa700 1 -- 172.21.15.116:0/940985943 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f66dc0127a0 con 0x7f66e00c3880 2020-11-20T12:20:29.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.024+0000 7f66d57fa700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66b80557c0 0x7f66b8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:29.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.024+0000 7f66d57fa700 1 -- 172.21.15.116:0/940985943 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f66dc06ea10 con 0x7f66e00c3880 2020-11-20T12:20:29.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.024+0000 7f66e498e700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66b80557c0 0x7f66b8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:29.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.024+0000 7f66e498e700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66b80557c0 0x7f66b8057c80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f66e00a9630 tx=0x7f66d0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:29.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.028+0000 7f66d57fa700 1 -- 172.21.15.116:0/940985943 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f66dc044a80 con 0x7f66e00c3880 2020-11-20T12:20:29.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.200+0000 7f66e73f3700 1 -- 172.21.15.116:0/940985943 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f66e0060170 con 0x7f66e00c3880 2020-11-20T12:20:29.911 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:20:29.917 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:20:29.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.960+0000 7f66d57fa700 1 -- 172.21.15.116:0/940985943 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v90) v1 ==== 97+0+0 (secure 0 0 0) 0x7f66dc0480a0 con 0x7f66e00c3880 2020-11-20T12:20:29.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 -- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66b80557c0 msgr2=0x7f66b8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:29.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66b80557c0 0x7f66b8057c80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f66e00a9630 tx=0x7f66d0015000).stop 2020-11-20T12:20:29.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 -- 172.21.15.116:0/940985943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e00c3880 msgr2=0x7f66e01532f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:29.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e00c3880 0x7f66e01532f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f66dc00eba0 tx=0x7f66dc006df0).stop 2020-11-20T12:20:29.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 -- 172.21.15.116:0/940985943 shutdown_connections 2020-11-20T12:20:29.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66b80557c0 0x7f66b8057c80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:29.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66e0151440 0x7f66e0154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:29.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66e00c3880 0x7f66e01532f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:29.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 --2- 172.21.15.116:0/940985943 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66e00c5420 0x7f66e0168850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:29.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 -- 172.21.15.116:0/940985943 >> 172.21.15.116:0/940985943 conn(0x7f66e00678a0 msgr2=0x7f66e00a7b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:29.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 -- 172.21.15.116:0/940985943 shutdown_connections 2020-11-20T12:20:29.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:29.964+0000 7f66e73f3700 1 -- 172.21.15.116:0/940985943 wait complete. 2020-11-20T12:20:29.965 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2020-11-20T12:20:29.986 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-20T12:20:30.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.160+0000 7fb83e2bb700 1 Processor -- start 2020-11-20T12:20:30.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.160+0000 7fb83e2bb700 1 -- start start 2020-11-20T12:20:30.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83e2bb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb838068420 0x7fb838148420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83e2bb700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb838067190 con 0x7fb838148040 2020-11-20T12:20:30.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83e2bb700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8380677a0 con 0x7fb838066480 2020-11-20T12:20:30.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83e2bb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb838067a40 con 0x7fb838068420 2020-11-20T12:20:30.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83d2b9700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb838148040 0x7fb8380626e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58970/0 (socket says 172.21.15.116:58970) 2020-11-20T12:20:30.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83d2b9700 1 -- 172.21.15.116:0/2013618471 learned_addr learned my addr 172.21.15.116:0/2013618471 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:30.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4189482699 0 0) 0x7fb838067190 con 0x7fb838148040 2020-11-20T12:20:30.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb820003240 con 0x7fb838148040 2020-11-20T12:20:30.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83cab8700 1 --2- 172.21.15.116:0/2013618471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb838068420 0x7fb838148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2565138765 0 0) 0x7fb820003240 con 0x7fb838148040 2020-11-20T12:20:30.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb838068420 msgr2=0x7fb838148420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:30.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 --2- 172.21.15.116:0/2013618471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb838068420 0x7fb838148420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 >> v1:172.21.15.173:6789/0 conn(0x7fb838066480 legacy=0x7fb838066930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb838067ce0 con 0x7fb838148040 2020-11-20T12:20:30.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fb8340014e0 con 0x7fb838148040 2020-11-20T12:20:30.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 >> v1:172.21.15.116:6789/0 conn(0x7fb838148040 legacy=0x7fb8380626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 --2- 172.21.15.116:0/2013618471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb820003760 0x7fb820003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 --2- 172.21.15.116:0/2013618471 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8200044e0 0x7fb820004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 --2- 172.21.15.116:0/2013618471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb820004e50 0x7fb82000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb838067a40 con 0x7fb820003760 2020-11-20T12:20:30.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8200052d0 con 0x7fb8200044e0 2020-11-20T12:20:30.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb82000de30 con 0x7fb820004e50 2020-11-20T12:20:30.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83d2b9700 1 --2- 172.21.15.116:0/2013618471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb820003760 0x7fb820003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83cab8700 1 --2- 172.21.15.116:0/2013618471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb838068420 0x7fb838148420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:30.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83d2b9700 1 -- 172.21.15.116:0/2013618471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb820004e50 msgr2=0x7fb82000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:30.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83d2b9700 1 --2- 172.21.15.116:0/2013618471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb820004e50 0x7fb82000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:30.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.164+0000 7fb83d2b9700 1 -- 172.21.15.116:0/2013618471 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8200044e0 msgr2=0x7fb820004960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2020-11-20T12:20:30.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb83d2b9700 1 --2- 172.21.15.116:0/2013618471 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8200044e0 0x7fb820004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:30.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb83d2b9700 1 -- 172.21.15.116:0/2013618471 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820003240 con 0x7fb820003760 2020-11-20T12:20:30.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb83d2b9700 1 --2- 172.21.15.116:0/2013618471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb820003760 0x7fb820003b80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7fb83400c210 tx=0x7fb8380bf1c0).ready entity=mon.0 client_cookie=4ea1e67a13be6485 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:30.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8340070d0 con 0x7fb820003760 2020-11-20T12:20:30.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb834007230 con 0x7fb820003760 2020-11-20T12:20:30.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb827fff700 1 -- 172.21.15.116:0/2013618471 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8340055f0 con 0x7fb820003760 2020-11-20T12:20:30.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2013618471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb820003760 msgr2=0x7fb820003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb83e2bb700 1 --2- 172.21.15.116:0/2013618471 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb820003760 0x7fb820003b80 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7fb83400c210 tx=0x7fb8380bf1c0).stop 2020-11-20T12:20:30.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb83daba700 1 -- 172.21.15.116:0/2013618471 reap_dead start 2020-11-20T12:20:30.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2013618471 shutdown_connections 2020-11-20T12:20:30.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2013618471 >> 172.21.15.116:0/2013618471 conn(0x7fb838060c40 msgr2=0x7fb8380c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:30.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2013618471 shutdown_connections 2020-11-20T12:20:30.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.168+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2013618471 wait complete. 2020-11-20T12:20:30.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83e2bb700 1 Processor -- start 2020-11-20T12:20:30.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83e2bb700 1 -- start start 2020-11-20T12:20:30.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83e2bb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8380c03e0 0x7fb8380c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83e2bb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8380a7320 0x7fb8380a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83e2bb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8380a6c10 0x7fb8380bd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83e2bb700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8380a7ca0 con 0x7fb8380a6c10 2020-11-20T12:20:30.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83e2bb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8380a4be0 con 0x7fb8380c03e0 2020-11-20T12:20:30.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83e2bb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8380bc730 con 0x7fb8380a7320 2020-11-20T12:20:30.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83daba700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8380a6c10 0x7fb8380bd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83daba700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8380a6c10 0x7fb8380bd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49552/0 (socket says 172.21.15.116:49552) 2020-11-20T12:20:30.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83daba700 1 -- 172.21.15.116:0/2199438231 learned_addr learned my addr 172.21.15.116:0/2199438231 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:30.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83cab8700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8380a7320 0x7fb8380a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83d2b9700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8380c03e0 0x7fb8380c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83daba700 1 -- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8380a7320 msgr2=0x7fb8380a77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83daba700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8380a7320 0x7fb8380a77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83daba700 1 -- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8380c03e0 msgr2=0x7fb8380c07c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83daba700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8380c03e0 0x7fb8380c07c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83daba700 1 -- 172.21.15.116:0/2199438231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8380bc9d0 con 0x7fb8380a6c10 2020-11-20T12:20:30.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.172+0000 7fb83daba700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8380a6c10 0x7fb8380bd3b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7fb838061cf0 tx=0x7fb830007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:30.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.176+0000 7fb8257fa700 1 -- 172.21.15.116:0/2199438231 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb830005a70 con 0x7fb8380a6c10 2020-11-20T12:20:30.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.176+0000 7fb8257fa700 1 -- 172.21.15.116:0/2199438231 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb830005bf0 con 0x7fb8380a6c10 2020-11-20T12:20:30.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.176+0000 7fb8257fa700 1 -- 172.21.15.116:0/2199438231 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8300070d0 con 0x7fb8380a6c10 2020-11-20T12:20:30.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.176+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2199438231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8380bcc70 con 0x7fb8380a6c10 2020-11-20T12:20:30.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.176+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2199438231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8380a4860 con 0x7fb8380a6c10 2020-11-20T12:20:30.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.176+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2199438231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83805cf30 con 0x7fb8380a6c10 2020-11-20T12:20:30.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.176+0000 7fb8257fa700 1 -- 172.21.15.116:0/2199438231 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb83001c070 con 0x7fb8380a6c10 2020-11-20T12:20:30.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.176+0000 7fb8257fa700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb8100557f0 0x7fb810057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.180+0000 7fb8257fa700 1 -- 172.21.15.116:0/2199438231 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb83006da00 con 0x7fb8380a6c10 2020-11-20T12:20:30.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.180+0000 7fb83d2b9700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb8100557f0 0x7fb810057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.180+0000 7fb83d2b9700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb8100557f0 0x7fb810057cb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fb83805f2f0 tx=0x7fb834015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:30.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.180+0000 7fb8257fa700 1 -- 172.21.15.116:0/2199438231 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb830043bd0 con 0x7fb8380a6c10 2020-11-20T12:20:30.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.392+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2199438231 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb838061230 con 0x7fb8380a6c10 2020-11-20T12:20:30.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.392+0000 7fb8257fa700 1 -- 172.21.15.116:0/2199438231 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5399 (secure 0 0 0) 0x7fb8300471f0 con 0x7fb8380a6c10 2020-11-20T12:20:30.389 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:30.390 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:20:30.390 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2020-11-20T12:20:30.390 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.390 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2020-11-20T12:20:30.391 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw==", 2020-11-20T12:20:30.391 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.391 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:20:30.391 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.391 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.392 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.392 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.392 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.392 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.393 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2020-11-20T12:20:30.393 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ==", 2020-11-20T12:20:30.393 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.393 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:20:30.393 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.394 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.394 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.394 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.394 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.395 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.395 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2020-11-20T12:20:30.395 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag==", 2020-11-20T12:20:30.395 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.395 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:20:30.396 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.396 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.396 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.396 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.396 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.397 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.397 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2020-11-20T12:20:30.397 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ==", 2020-11-20T12:20:30.397 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.397 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:20:30.398 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.398 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.398 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.398 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.398 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.399 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.399 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2020-11-20T12:20:30.399 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ==", 2020-11-20T12:20:30.399 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.400 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.400 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.400 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.400 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.400 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.401 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.401 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2020-11-20T12:20:30.401 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q==", 2020-11-20T12:20:30.401 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.401 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.402 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.402 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.402 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.402 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.402 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.403 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2020-11-20T12:20:30.403 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw==", 2020-11-20T12:20:30.403 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.403 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.403 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.404 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.404 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.404 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.404 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.404 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2020-11-20T12:20:30.405 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w==", 2020-11-20T12:20:30.405 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.405 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.405 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.405 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.406 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.406 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.406 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.406 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2020-11-20T12:20:30.406 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA==", 2020-11-20T12:20:30.407 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.407 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.407 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.407 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.408 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.408 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.408 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.408 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2020-11-20T12:20:30.408 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ==", 2020-11-20T12:20:30.409 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.409 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.409 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.409 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.409 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.410 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.410 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.410 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2020-11-20T12:20:30.410 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg==", 2020-11-20T12:20:30.410 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.411 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.411 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.411 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.411 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.411 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.412 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.412 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2020-11-20T12:20:30.412 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg==", 2020-11-20T12:20:30.412 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.412 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.413 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.413 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.413 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.413 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.413 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.414 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2020-11-20T12:20:30.414 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA==", 2020-11-20T12:20:30.414 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.414 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:20:30.414 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2020-11-20T12:20:30.415 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-20T12:20:30.415 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.415 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.415 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.415 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2020-11-20T12:20:30.416 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow==", 2020-11-20T12:20:30.416 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.416 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:20:30.416 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:20:30.416 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:20:30.417 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:20:30.417 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.417 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.417 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.418 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2020-11-20T12:20:30.418 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ==", 2020-11-20T12:20:30.418 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.418 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:20:30.418 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:20:30.419 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:20:30.419 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:20:30.419 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.419 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.419 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.420 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2020-11-20T12:20:30.420 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ==", 2020-11-20T12:20:30.420 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.420 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:20:30.420 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:20:30.421 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:20:30.421 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:20:30.421 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.421 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.421 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.422 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2020-11-20T12:20:30.422 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBDs7dfh+KKOBAAZvcPXWly9/mxRE7NWuR85A==", 2020-11-20T12:20:30.422 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.422 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:20:30.422 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:20:30.423 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.423 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.423 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.423 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.423 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.424 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.manila", 2020-11-20T12:20:30.424 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQCAtLdf7UOCKxAAAQ2Jig7S/lAPiwQX/PHdEw==", 2020-11-20T12:20:30.424 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.424 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:20:30.424 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:20:30.425 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw" 2020-11-20T12:20:30.425 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.425 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.425 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.425 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2020-11-20T12:20:30.426 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A==", 2020-11-20T12:20:30.426 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.426 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:20:30.426 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:20:30.426 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.427 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.427 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:30.427 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:30.427 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2020-11-20T12:20:30.427 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA==", 2020-11-20T12:20:30.428 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:20:30.428 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:20:30.428 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:20:30.428 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:20:30.428 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.429 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:30.429 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:20:30.429 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:20:30.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb8100557f0 msgr2=0x7fb810057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb8100557f0 0x7fb810057cb0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fb83805f2f0 tx=0x7fb834015000).stop 2020-11-20T12:20:30.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2199438231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8380a6c10 msgr2=0x7fb8380bd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8380a6c10 0x7fb8380bd3b0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7fb838061cf0 tx=0x7fb830007d90).stop 2020-11-20T12:20:30.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2199438231 shutdown_connections 2020-11-20T12:20:30.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb8100557f0 0x7fb810057cb0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb8380a6c10 0x7fb8380bd3b0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8380c03e0 0x7fb8380c07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 --2- 172.21.15.116:0/2199438231 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8380a7320 0x7fb8380a77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2199438231 >> 172.21.15.116:0/2199438231 conn(0x7fb838060c40 msgr2=0x7fb83814d960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:30.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2199438231 shutdown_connections 2020-11-20T12:20:30.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.396+0000 7fb83e2bb700 1 -- 172.21.15.116:0/2199438231 wait complete. 2020-11-20T12:20:30.434 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.manila 2020-11-20T12:20:30.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.608+0000 7f6effb7b700 1 Processor -- start 2020-11-20T12:20:30.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.612+0000 7f6effb7b700 1 -- start start 2020-11-20T12:20:30.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.612+0000 7f6effb7b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6ef8062560 0x7f6ef8066ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.612+0000 7f6effb7b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef8156f90 con 0x7f6ef8062940 2020-11-20T12:20:30.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.612+0000 7f6effb7b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef8157230 con 0x7f6ef80677e0 2020-11-20T12:20:30.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.612+0000 7f6effb7b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef8066560 con 0x7f6ef8062560 2020-11-20T12:20:30.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.612+0000 7f6efd116700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6ef8062940 0x7f6ef81564c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:58990/0 (socket says 172.21.15.116:58990) 2020-11-20T12:20:30.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.612+0000 7f6efd116700 1 -- 172.21.15.116:0/691572565 learned_addr learned my addr 172.21.15.116:0/691572565 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:30.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.612+0000 7f6efd917700 1 --2- 172.21.15.116:0/691572565 >> v2:172.21.15.173:3300/0 conn(0x7f6ef8062560 0x7f6ef8066ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.612+0000 7f6efc915700 1 -- 172.21.15.116:0/691572565 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256234742 0 0) 0x7f6ef8156f90 con 0x7f6ef8062940 2020-11-20T12:20:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.612+0000 7f6efc915700 1 -- 172.21.15.116:0/691572565 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ee0003240 con 0x7f6ef8062940 2020-11-20T12:20:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.612+0000 7f6efc915700 1 -- 172.21.15.116:0/691572565 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 16851734 0 0) 0x7f6ef8157230 con 0x7f6ef80677e0 2020-11-20T12:20:30.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6efc915700 1 -- 172.21.15.116:0/691572565 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ef8156f90 con 0x7f6ef80677e0 2020-11-20T12:20:30.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6efd917700 1 -- 172.21.15.116:0/691572565 >> v1:172.21.15.173:6789/0 conn(0x7f6ef80677e0 legacy=0x7f6ef80a52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6efd917700 1 -- 172.21.15.116:0/691572565 >> v1:172.21.15.116:6789/0 conn(0x7f6ef8062940 legacy=0x7f6ef81564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6efd917700 1 -- 172.21.15.116:0/691572565 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef8066800 con 0x7f6ef8062560 2020-11-20T12:20:30.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6efd917700 1 --2- 172.21.15.116:0/691572565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef8062560 0x7f6ef8066ca0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f6ef400b9a0 tx=0x7f6ef4006df0).ready entity=mon.1 client_cookie=f5b8684764ae895a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:30.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6efc915700 1 -- 172.21.15.116:0/691572565 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4009680 con 0x7f6ef8062560 2020-11-20T12:20:30.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6efc915700 1 -- 172.21.15.116:0/691572565 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef40097e0 con 0x7f6ef8062560 2020-11-20T12:20:30.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6efc915700 1 -- 172.21.15.116:0/691572565 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef4009b60 con 0x7f6ef8062560 2020-11-20T12:20:30.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6effb7b700 1 -- 172.21.15.116:0/691572565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef8062560 msgr2=0x7f6ef8066ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6effb7b700 1 --2- 172.21.15.116:0/691572565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef8062560 0x7f6ef8066ca0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f6ef400b9a0 tx=0x7f6ef4006df0).stop 2020-11-20T12:20:30.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6effb7b700 1 -- 172.21.15.116:0/691572565 shutdown_connections 2020-11-20T12:20:30.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6effb7b700 1 --2- 172.21.15.116:0/691572565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef8062560 0x7f6ef8066ca0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6effb7b700 1 -- 172.21.15.116:0/691572565 >> 172.21.15.116:0/691572565 conn(0x7f6ef8060170 msgr2=0x7f6ef8060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:30.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6effb7b700 1 -- 172.21.15.116:0/691572565 shutdown_connections 2020-11-20T12:20:30.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.616+0000 7f6effb7b700 1 -- 172.21.15.116:0/691572565 wait complete. 2020-11-20T12:20:30.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6effb7b700 1 Processor -- start 2020-11-20T12:20:30.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6effb7b700 1 -- start start 2020-11-20T12:20:30.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6effb7b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ef8062560 0x7f6ef808d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6effb7b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef8062940 0x7f6ef80a9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6effb7b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ef80677e0 0x7f6ef8091b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6effb7b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef80911f0 con 0x7f6ef8062560 2020-11-20T12:20:30.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6effb7b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef80aa030 con 0x7f6ef8062940 2020-11-20T12:20:30.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6effb7b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef80aa2d0 con 0x7f6ef80677e0 2020-11-20T12:20:30.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6efd917700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ef8062560 0x7f6ef808d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6efd917700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ef8062560 0x7f6ef808d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49562/0 (socket says 172.21.15.116:49562) 2020-11-20T12:20:30.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6efd917700 1 -- 172.21.15.116:0/3288344207 learned_addr learned my addr 172.21.15.116:0/3288344207 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:30.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6efd116700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef8062940 0x7f6ef80a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6efe118700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ef80677e0 0x7f6ef8091b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6efd917700 1 -- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ef80677e0 msgr2=0x7f6ef8091b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6efd917700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ef80677e0 0x7f6ef8091b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6efd917700 1 -- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef8062940 msgr2=0x7f6ef80a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.620+0000 7f6efd917700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef8062940 0x7f6ef80a9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.624+0000 7f6efd917700 1 -- 172.21.15.116:0/3288344207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef8091450 con 0x7f6ef8062560 2020-11-20T12:20:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.624+0000 7f6efd917700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ef8062560 0x7f6ef808d1f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f6ef400c010 tx=0x7f6ef4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.624+0000 7f6ee5ffb700 1 -- 172.21.15.116:0/3288344207 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef40067e0 con 0x7f6ef8062560 2020-11-20T12:20:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.624+0000 7f6ee5ffb700 1 -- 172.21.15.116:0/3288344207 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef4011df0 con 0x7f6ef8062560 2020-11-20T12:20:30.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.624+0000 7f6effb7b700 1 -- 172.21.15.116:0/3288344207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef80916f0 con 0x7f6ef8062560 2020-11-20T12:20:30.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.624+0000 7f6ee5ffb700 1 -- 172.21.15.116:0/3288344207 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef401e670 con 0x7f6ef8062560 2020-11-20T12:20:30.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.624+0000 7f6effb7b700 1 -- 172.21.15.116:0/3288344207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef8092080 con 0x7f6ef8062560 2020-11-20T12:20:30.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.624+0000 7f6effb7b700 1 -- 172.21.15.116:0/3288344207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ef8091880 con 0x7f6ef8062560 2020-11-20T12:20:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.628+0000 7f6ee5ffb700 1 -- 172.21.15.116:0/3288344207 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6ef4025030 con 0x7f6ef8062560 2020-11-20T12:20:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.628+0000 7f6ee5ffb700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ed00557c0 0x7f6ed0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:30.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.628+0000 7f6ee5ffb700 1 -- 172.21.15.116:0/3288344207 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f6ef406f8a0 con 0x7f6ef8062560 2020-11-20T12:20:30.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.628+0000 7f6efd116700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ed00557c0 0x7f6ed0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:30.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.628+0000 7f6efd116700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ed00557c0 0x7f6ed0057c80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f6ef8169ab0 tx=0x7f6ee8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:30.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.632+0000 7f6ee5ffb700 1 -- 172.21.15.116:0/3288344207 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6ef4045b20 con 0x7f6ef8062560 2020-11-20T12:20:30.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.852+0000 7f6effb7b700 1 -- 172.21.15.116:0/3288344207 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.manila"} v 0) v1 -- 0x7f6ef805d2a0 con 0x7f6ef8062560 2020-11-20T12:20:30.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.864+0000 7f6ee5ffb700 1 -- 172.21.15.116:0/3288344207 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.manila"}]=0 updated v24) v1 ==== 90+0+0 (secure 0 0 0) 0x7f6ef4023070 con 0x7f6ef8062560 2020-11-20T12:20:30.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 -- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ed00557c0 msgr2=0x7f6ed0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ed00557c0 0x7f6ed0057c80 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f6ef8169ab0 tx=0x7f6ee8009040).stop 2020-11-20T12:20:30.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 -- 172.21.15.116:0/3288344207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ef8062560 msgr2=0x7f6ef808d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:30.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ef8062560 0x7f6ef808d1f0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f6ef400c010 tx=0x7f6ef4015000).stop 2020-11-20T12:20:30.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 -- 172.21.15.116:0/3288344207 shutdown_connections 2020-11-20T12:20:30.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ed00557c0 0x7f6ed0057c80 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ef80677e0 0x7f6ef8091b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ef8062560 0x7f6ef808d1f0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 --2- 172.21.15.116:0/3288344207 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ef8062940 0x7f6ef80a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:30.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 -- 172.21.15.116:0/3288344207 >> 172.21.15.116:0/3288344207 conn(0x7f6ef8060170 msgr2=0x7f6ef80a5040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:30.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 -- 172.21.15.116:0/3288344207 shutdown_connections 2020-11-20T12:20:30.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:30.868+0000 7f6effb7b700 1 -- 172.21.15.116:0/3288344207 wait complete. 2020-11-20T12:20:30.870 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:20:30.889 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:31.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.068+0000 7fe15c464700 1 Processor -- start 2020-11-20T12:20:31.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.068+0000 7fe15c464700 1 -- start start 2020-11-20T12:20:31.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.068+0000 7fe15c464700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe1540bef70 0x7fe1540c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.068+0000 7fe15c464700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1540c35e0 con 0x7fe1540c93a0 2020-11-20T12:20:31.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.068+0000 7fe15c464700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1540c3880 con 0x7fe1540c3e10 2020-11-20T12:20:31.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.068+0000 7fe15c464700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1540c7280 con 0x7fe1540bef70 2020-11-20T12:20:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.068+0000 7fe1599ff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe1540c93a0 0x7fe1540c80d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59006/0 (socket says 172.21.15.116:59006) 2020-11-20T12:20:31.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.068+0000 7fe1599ff700 1 -- 172.21.15.116:0/3047943927 learned_addr learned my addr 172.21.15.116:0/3047943927 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:31.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.068+0000 7fe15aa01700 1 --2- 172.21.15.116:0/3047943927 >> v2:172.21.15.173:3300/0 conn(0x7fe1540bef70 0x7fe1540c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.068+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105997581 0 0) 0x7fe1540c35e0 con 0x7fe1540c93a0 2020-11-20T12:20:31.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.068+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe140003240 con 0x7fe1540c93a0 2020-11-20T12:20:31.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2798975688 0 0) 0x7fe140003240 con 0x7fe1540c93a0 2020-11-20T12:20:31.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 >> v2:172.21.15.173:3300/0 conn(0x7fe1540bef70 msgr2=0x7fe1540c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:31.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 --2- 172.21.15.116:0/3047943927 >> v2:172.21.15.173:3300/0 conn(0x7fe1540bef70 0x7fe1540c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 >> v1:172.21.15.173:6789/0 conn(0x7fe1540c3e10 legacy=0x7fe1540c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1540c7520 con 0x7fe1540c93a0 2020-11-20T12:20:31.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fe1480014e0 con 0x7fe1540c93a0 2020-11-20T12:20:31.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 >> v1:172.21.15.116:6789/0 conn(0x7fe1540c93a0 legacy=0x7fe1540c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 --2- 172.21.15.116:0/3047943927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe140003780 0x7fe140003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 --2- 172.21.15.116:0/3047943927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 0x7fe140004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 --2- 172.21.15.116:0/3047943927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe140004e50 0x7fe14000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540c7280 con 0x7fe1400044e0 2020-11-20T12:20:31.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe140005330 con 0x7fe140004e50 2020-11-20T12:20:31.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14000bc20 con 0x7fe140003780 2020-11-20T12:20:31.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1599ff700 1 --2- 172.21.15.116:0/3047943927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 0x7fe140004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe15aa01700 1 --2- 172.21.15.116:0/3047943927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe140004e50 0x7fe14000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe15a200700 1 --2- 172.21.15.116:0/3047943927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe140003780 0x7fe140003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1599ff700 1 -- 172.21.15.116:0/3047943927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe140003780 msgr2=0x7fe140003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:31.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1599ff700 1 --2- 172.21.15.116:0/3047943927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe140003780 0x7fe140003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1599ff700 1 -- 172.21.15.116:0/3047943927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe140004e50 msgr2=0x7fe14000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1599ff700 1 --2- 172.21.15.116:0/3047943927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe140004e50 0x7fe14000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1599ff700 1 -- 172.21.15.116:0/3047943927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe140003240 con 0x7fe1400044e0 2020-11-20T12:20:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe15aa01700 1 -- 172.21.15.116:0/3047943927 reap_dead start 2020-11-20T12:20:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1599ff700 1 --2- 172.21.15.116:0/3047943927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 0x7fe140004960 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7fe14800c200 tx=0x7fe1540c76b0).ready entity=mon.0 client_cookie=4823b1d39e51f23b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480070d0 con 0x7fe1400044e0 2020-11-20T12:20:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe148007230 con 0x7fe1400044e0 2020-11-20T12:20:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe1591fe700 1 -- 172.21.15.116:0/3047943927 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480055f0 con 0x7fe1400044e0 2020-11-20T12:20:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe15c464700 1 -- 172.21.15.116:0/3047943927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 msgr2=0x7fe140004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe15c464700 1 --2- 172.21.15.116:0/3047943927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 0x7fe140004960 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7fe14800c200 tx=0x7fe1540c76b0).stop 2020-11-20T12:20:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe15c464700 1 -- 172.21.15.116:0/3047943927 shutdown_connections 2020-11-20T12:20:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe15c464700 1 --2- 172.21.15.116:0/3047943927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 0x7fe140004960 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe15c464700 1 -- 172.21.15.116:0/3047943927 >> 172.21.15.116:0/3047943927 conn(0x7fe154068080 msgr2=0x7fe1540c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe15c464700 1 -- 172.21.15.116:0/3047943927 shutdown_connections 2020-11-20T12:20:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.072+0000 7fe15c464700 1 -- 172.21.15.116:0/3047943927 wait complete. 2020-11-20T12:20:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15c464700 1 Processor -- start 2020-11-20T12:20:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15c464700 1 -- start start 2020-11-20T12:20:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15c464700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 0x7fe1540930d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15c464700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1540945d0 0x7fe15414f990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15c464700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe154095bc0 0x7fe154096020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15c464700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe154094ca0 con 0x7fe1400044e0 2020-11-20T12:20:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15c464700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe154094f40 con 0x7fe154095bc0 2020-11-20T12:20:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15c464700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540951e0 con 0x7fe1540945d0 2020-11-20T12:20:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15a200700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 0x7fe1540930d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15a200700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 0x7fe1540930d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49582/0 (socket says 172.21.15.116:49582) 2020-11-20T12:20:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15a200700 1 -- 172.21.15.116:0/3659296204 learned_addr learned my addr 172.21.15.116:0/3659296204 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15aa01700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe154095bc0 0x7fe154096020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe1599ff700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1540945d0 0x7fe15414f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15a200700 1 -- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1540945d0 msgr2=0x7fe15414f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15a200700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1540945d0 0x7fe15414f990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15a200700 1 -- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe154095bc0 msgr2=0x7fe154096020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15a200700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe154095bc0 0x7fe154096020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.076+0000 7fe15a200700 1 -- 172.21.15.116:0/3659296204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe154095480 con 0x7fe1400044e0 2020-11-20T12:20:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.080+0000 7fe15a200700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 0x7fe1540930d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7fe14400eae0 tx=0x7fe144009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.080+0000 7fe1527fc700 1 -- 172.21.15.116:0/3659296204 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe144003320 con 0x7fe1400044e0 2020-11-20T12:20:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.080+0000 7fe1527fc700 1 -- 172.21.15.116:0/3659296204 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14400c540 con 0x7fe1400044e0 2020-11-20T12:20:31.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.080+0000 7fe15c464700 1 -- 172.21.15.116:0/3659296204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe154095720 con 0x7fe1400044e0 2020-11-20T12:20:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.080+0000 7fe1527fc700 1 -- 172.21.15.116:0/3659296204 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14400c8c0 con 0x7fe1400044e0 2020-11-20T12:20:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.080+0000 7fe15c464700 1 -- 172.21.15.116:0/3659296204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe154093760 con 0x7fe1400044e0 2020-11-20T12:20:31.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.080+0000 7fe15c464700 1 -- 172.21.15.116:0/3659296204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1540958b0 con 0x7fe1400044e0 2020-11-20T12:20:31.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.080+0000 7fe1527fc700 1 -- 172.21.15.116:0/3659296204 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe14400cc70 con 0x7fe1400044e0 2020-11-20T12:20:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.084+0000 7fe1527fc700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe12c0557c0 0x7fe12c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.084+0000 7fe1527fc700 1 -- 172.21.15.116:0/3659296204 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe14406e070 con 0x7fe1400044e0 2020-11-20T12:20:31.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.084+0000 7fe1599ff700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe12c0557c0 0x7fe12c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.084+0000 7fe1599ff700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe12c0557c0 0x7fe12c057c80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fe1540ab0a0 tx=0x7fe148013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:31.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.088+0000 7fe1527fc700 1 -- 172.21.15.116:0/3659296204 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe14406bfd0 con 0x7fe1400044e0 2020-11-20T12:20:31.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.248+0000 7fe15c464700 1 -- 172.21.15.116:0/3659296204 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe15405d2a0 con 0x7fe1400044e0 2020-11-20T12:20:31.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.252+0000 7fe1527fc700 1 -- 172.21.15.116:0/3659296204 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+13724 (secure 0 0 0) 0x7fe1440255b0 con 0x7fe1400044e0 2020-11-20T12:20:31.249 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:31.249 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":90,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:20:29.956407+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:31.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.252+0000 7fe15c464700 1 -- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe12c0557c0 msgr2=0x7fe12c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.252+0000 7fe15c464700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe12c0557c0 0x7fe12c057c80 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fe1540ab0a0 tx=0x7fe148013000).stop 2020-11-20T12:20:31.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.252+0000 7fe15c464700 1 -- 172.21.15.116:0/3659296204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 msgr2=0x7fe1540930d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.252+0000 7fe15c464700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 0x7fe1540930d0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7fe14400eae0 tx=0x7fe144009040).stop 2020-11-20T12:20:31.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.252+0000 7fe15c464700 1 -- 172.21.15.116:0/3659296204 shutdown_connections 2020-11-20T12:20:31.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.252+0000 7fe15c464700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe12c0557c0 0x7fe12c057c80 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.252+0000 7fe15c464700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe154095bc0 0x7fe154096020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.252+0000 7fe15c464700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1400044e0 0x7fe1540930d0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.252+0000 7fe15c464700 1 --2- 172.21.15.116:0/3659296204 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1540945d0 0x7fe15414f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.252+0000 7fe15c464700 1 -- 172.21.15.116:0/3659296204 >> 172.21.15.116:0/3659296204 conn(0x7fe154068080 msgr2=0x7fe1540abfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:31.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.256+0000 7fe15c464700 1 -- 172.21.15.116:0/3659296204 shutdown_connections 2020-11-20T12:20:31.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.256+0000 7fe15c464700 1 -- 172.21.15.116:0/3659296204 wait complete. 2020-11-20T12:20:31.279 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T12:20:31.280 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:20:31.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.460+0000 7f6edf48f700 1 Processor -- start 2020-11-20T12:20:31.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.460+0000 7f6edf48f700 1 -- start start 2020-11-20T12:20:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.464+0000 7f6edf48f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6ed8063100 0x7f6ed80bff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.464+0000 7f6edf48f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ed80a66e0 0x7f6ed80a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.464+0000 7f6edf48f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ed80c0400 con 0x7f6ed814e590 2020-11-20T12:20:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.464+0000 7f6edf48f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ed80a7040 con 0x7f6ed8063100 2020-11-20T12:20:31.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.464+0000 7f6edf48f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80c0690 con 0x7f6ed80a66e0 2020-11-20T12:20:31.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.464+0000 7f6ede48d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6ed814e590 0x7f6ed814f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59024/0 (socket says 172.21.15.116:59024) 2020-11-20T12:20:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.464+0000 7f6ede48d700 1 -- 172.21.15.116:0/2834611927 learned_addr learned my addr 172.21.15.116:0/2834611927 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.464+0000 7f6eddc8c700 1 --2- 172.21.15.116:0/2834611927 >> v2:172.21.15.116:3300/0 conn(0x7f6ed8063100 0x7f6ed80bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2406108442 0 0) 0x7f6ed80c0400 con 0x7f6ed814e590 2020-11-20T12:20:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec0003240 con 0x7f6ed814e590 2020-11-20T12:20:31.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edec8e700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ed80a66e0 0x7f6ed80a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 684924744 0 0) 0x7f6ec0003240 con 0x7f6ed814e590 2020-11-20T12:20:31.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ed80a66e0 msgr2=0x7f6ed80a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:31.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ed80a66e0 0x7f6ed80a6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 >> v2:172.21.15.116:3300/0 conn(0x7f6ed8063100 msgr2=0x7f6ed80bff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 --2- 172.21.15.116:0/2834611927 >> v2:172.21.15.116:3300/0 conn(0x7f6ed8063100 0x7f6ed80bff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed80c0930 con 0x7f6ed814e590 2020-11-20T12:20:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f6ed40014e0 con 0x7f6ed814e590 2020-11-20T12:20:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 >> v1:172.21.15.116:6789/0 conn(0x7f6ed814e590 legacy=0x7f6ed814f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ec00037b0 0x7f6ec0003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ed80a66e0 0x7f6ec00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec00049d0 0x7f6ec0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80a7040 con 0x7f6ec00049d0 2020-11-20T12:20:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80c0690 con 0x7f6ec00037b0 2020-11-20T12:20:31.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec0004320 con 0x7f6ed80a66e0 2020-11-20T12:20:31.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edec8e700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec00049d0 0x7f6ec0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6ede48d700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ec00037b0 0x7f6ec0003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6eddc8c700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ed80a66e0 0x7f6ec00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edec8e700 1 -- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ed80a66e0 msgr2=0x7f6ec00044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:31.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edec8e700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ed80a66e0 0x7f6ec00044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edec8e700 1 -- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ec00037b0 msgr2=0x7f6ec0003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:31.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edec8e700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ec00037b0 0x7f6ec0003bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6edec8e700 1 -- 172.21.15.116:0/2834611927 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec0003240 con 0x7f6ec00049d0 2020-11-20T12:20:31.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.468+0000 7f6ede48d700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ec00037b0 0x7f6ec0003bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:31.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.472+0000 7f6edec8e700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec00049d0 0x7f6ec0004e50 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f6ed0009d80 tx=0x7f6ed0008040).ready entity=mon.0 client_cookie=b165c713dc4d8af0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:31.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.472+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000c070 con 0x7f6ec00049d0 2020-11-20T12:20:31.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.472+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed000e910 con 0x7f6ec00049d0 2020-11-20T12:20:31.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.472+0000 7f6edd48b700 1 -- 172.21.15.116:0/2834611927 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed000ecd0 con 0x7f6ec00049d0 2020-11-20T12:20:31.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.472+0000 7f6edf48f700 1 -- 172.21.15.116:0/2834611927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec00049d0 msgr2=0x7f6ec0004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.472+0000 7f6edf48f700 1 --2- 172.21.15.116:0/2834611927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ec00049d0 0x7f6ec0004e50 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f6ed0009d80 tx=0x7f6ed0008040).stop 2020-11-20T12:20:31.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.472+0000 7f6edec8e700 1 -- 172.21.15.116:0/2834611927 reap_dead start 2020-11-20T12:20:31.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.472+0000 7f6edf48f700 1 -- 172.21.15.116:0/2834611927 shutdown_connections 2020-11-20T12:20:31.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.472+0000 7f6edf48f700 1 -- 172.21.15.116:0/2834611927 >> 172.21.15.116:0/2834611927 conn(0x7f6ed8068350 msgr2=0x7f6ed80627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:31.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.472+0000 7f6edf48f700 1 -- 172.21.15.116:0/2834611927 shutdown_connections 2020-11-20T12:20:31.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.472+0000 7f6edf48f700 1 -- 172.21.15.116:0/2834611927 wait complete. 2020-11-20T12:20:31.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edf48f700 1 Processor -- start 2020-11-20T12:20:31.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edf48f700 1 -- start start 2020-11-20T12:20:31.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edf48f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ec00049d0 0x7f6ed80bdc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edf48f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8150360 0x7f6ed81507e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edf48f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ed80c15e0 0x7f6ed80c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edf48f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed80be160 con 0x7f6ed80c15e0 2020-11-20T12:20:31.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edf48f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed8161d60 con 0x7f6ed8150360 2020-11-20T12:20:31.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edf48f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed81610e0 con 0x7f6ec00049d0 2020-11-20T12:20:31.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edec8e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ed80c15e0 0x7f6ed80c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edec8e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ed80c15e0 0x7f6ed80c1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49606/0 (socket says 172.21.15.116:49606) 2020-11-20T12:20:31.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edec8e700 1 -- 172.21.15.116:0/1857760352 learned_addr learned my addr 172.21.15.116:0/1857760352 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:31.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6ede48d700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ec00049d0 0x7f6ed80bdc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6eddc8c700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8150360 0x7f6ed81507e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edec8e700 1 -- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ec00049d0 msgr2=0x7f6ed80bdc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edec8e700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ec00049d0 0x7f6ed80bdc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edec8e700 1 -- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8150360 msgr2=0x7f6ed81507e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edec8e700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8150360 0x7f6ed81507e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edec8e700 1 -- 172.21.15.116:0/1857760352 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed8161380 con 0x7f6ed80c15e0 2020-11-20T12:20:31.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edec8e700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ed80c15e0 0x7f6ed80c1a60 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f6ed0013000 tx=0x7f6ed0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6ec67fc700 1 -- 172.21.15.116:0/1857760352 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed001e630 con 0x7f6ed80c15e0 2020-11-20T12:20:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6ec67fc700 1 -- 172.21.15.116:0/1857760352 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed001e790 con 0x7f6ed80c15e0 2020-11-20T12:20:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edf48f700 1 -- 172.21.15.116:0/1857760352 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed8161620 con 0x7f6ed80c15e0 2020-11-20T12:20:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6ec67fc700 1 -- 172.21.15.116:0/1857760352 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed001eb30 con 0x7f6ed80c15e0 2020-11-20T12:20:31.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.476+0000 7f6edf48f700 1 -- 172.21.15.116:0/1857760352 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed8160e00 con 0x7f6ed80c15e0 2020-11-20T12:20:31.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.480+0000 7f6edf48f700 1 -- 172.21.15.116:0/1857760352 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed805cf30 con 0x7f6ed80c15e0 2020-11-20T12:20:31.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.480+0000 7f6ec67fc700 1 -- 172.21.15.116:0/1857760352 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6ed0028410 con 0x7f6ed80c15e0 2020-11-20T12:20:31.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.480+0000 7f6ec67fc700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6eb00557c0 0x7f6eb0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.480+0000 7f6ec67fc700 1 -- 172.21.15.116:0/1857760352 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f6ed0002030 con 0x7f6ed80c15e0 2020-11-20T12:20:31.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.480+0000 7f6ede48d700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6eb00557c0 0x7f6eb0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.484+0000 7f6ede48d700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6eb00557c0 0x7f6eb0057c80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f6ed4001da0 tx=0x7f6ed4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:31.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.488+0000 7f6ec67fc700 1 -- 172.21.15.116:0/1857760352 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6ed0046100 con 0x7f6ed80c15e0 2020-11-20T12:20:31.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.644+0000 7f6edf48f700 1 -- 172.21.15.116:0/1857760352 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f6ed80639c0 con 0x7f6ed80c15e0 2020-11-20T12:20:31.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6ec67fc700 1 -- 172.21.15.116:0/1857760352 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v90) v1 ==== 110+0+10 (secure 0 0 0) 0x7f6ed0049720 con 0x7f6ed80c15e0 2020-11-20T12:20:31.645 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:20:31.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 -- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6eb00557c0 msgr2=0x7f6eb0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6eb00557c0 0x7f6eb0057c80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f6ed4001da0 tx=0x7f6ed4009040).stop 2020-11-20T12:20:31.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 -- 172.21.15.116:0/1857760352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ed80c15e0 msgr2=0x7f6ed80c1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ed80c15e0 0x7f6ed80c1a60 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f6ed0013000 tx=0x7f6ed0008040).stop 2020-11-20T12:20:31.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 -- 172.21.15.116:0/1857760352 shutdown_connections 2020-11-20T12:20:31.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6eb00557c0 0x7f6eb0057c80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ed80c15e0 0x7f6ed80c1a60 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ec00049d0 0x7f6ed80bdc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 --2- 172.21.15.116:0/1857760352 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ed8150360 0x7f6ed81507e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 -- 172.21.15.116:0/1857760352 >> 172.21.15.116:0/1857760352 conn(0x7f6ed8068350 msgr2=0x7f6ed80a9570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:31.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 -- 172.21.15.116:0/1857760352 shutdown_connections 2020-11-20T12:20:31.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.648+0000 7f6edf48f700 1 -- 172.21.15.116:0/1857760352 wait complete. 2020-11-20T12:20:31.669 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:20:31.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.852+0000 7f304fc37700 1 Processor -- start 2020-11-20T12:20:31.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.852+0000 7f304fc37700 1 -- start start 2020-11-20T12:20:31.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304fc37700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f30481518c0 0x7f3048151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304fc37700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f30480aa210 0x7f30480aa6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304fc37700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30480a92f0 con 0x7f30480c9010 2020-11-20T12:20:31.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304fc37700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30480c9450 con 0x7f30481518c0 2020-11-20T12:20:31.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304fc37700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30480a9590 con 0x7f30480aa210 2020-11-20T12:20:31.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304d1d2700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f30480c9010 0x7f3048061550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59044/0 (socket says 172.21.15.116:59044) 2020-11-20T12:20:31.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304d1d2700 1 -- 172.21.15.116:0/3853905310 learned_addr learned my addr 172.21.15.116:0/3853905310 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:31.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304d9d3700 1 --2- 172.21.15.116:0/3853905310 >> v2:172.21.15.116:3300/0 conn(0x7f30481518c0 0x7f3048151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304e1d4700 1 --2- 172.21.15.116:0/3853905310 >> v2:172.21.15.173:3300/0 conn(0x7f30480aa210 0x7f30480aa6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304c9d1700 1 -- 172.21.15.116:0/3853905310 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625831599 0 0) 0x7f30480a92f0 con 0x7f30480c9010 2020-11-20T12:20:31.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304c9d1700 1 -- 172.21.15.116:0/3853905310 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3030003240 con 0x7f30480c9010 2020-11-20T12:20:31.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304d9d3700 1 -- 172.21.15.116:0/3853905310 >> v2:172.21.15.173:3300/0 conn(0x7f30480aa210 msgr2=0x7f30480aa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:31.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304d9d3700 1 --2- 172.21.15.116:0/3853905310 >> v2:172.21.15.173:3300/0 conn(0x7f30480aa210 0x7f30480aa6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304d9d3700 1 -- 172.21.15.116:0/3853905310 >> v1:172.21.15.116:6789/0 conn(0x7f30480c9010 legacy=0x7f3048061550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304d9d3700 1 -- 172.21.15.116:0/3853905310 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30480a9830 con 0x7f30481518c0 2020-11-20T12:20:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304d9d3700 1 --2- 172.21.15.116:0/3853905310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30481518c0 0x7f3048151ca0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f304400b990 tx=0x7f3044006df0).ready entity=mon.0 client_cookie=d96d7411dc0c32b8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:31.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304c9d1700 1 -- 172.21.15.116:0/3853905310 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3044006e60 con 0x7f30481518c0 2020-11-20T12:20:31.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304c9d1700 1 -- 172.21.15.116:0/3853905310 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3044009640 con 0x7f30481518c0 2020-11-20T12:20:31.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.856+0000 7f304c9d1700 1 -- 172.21.15.116:0/3853905310 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30440099c0 con 0x7f30481518c0 2020-11-20T12:20:31.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.860+0000 7f304fc37700 1 -- 172.21.15.116:0/3853905310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30481518c0 msgr2=0x7f3048151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.860+0000 7f304fc37700 1 --2- 172.21.15.116:0/3853905310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30481518c0 0x7f3048151ca0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7f304400b990 tx=0x7f3044006df0).stop 2020-11-20T12:20:31.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.860+0000 7f304fc37700 1 -- 172.21.15.116:0/3853905310 shutdown_connections 2020-11-20T12:20:31.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.860+0000 7f304fc37700 1 --2- 172.21.15.116:0/3853905310 >> v2:172.21.15.173:3300/0 conn(0x7f30480aa210 0x7f30480aa6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.860+0000 7f304fc37700 1 --2- 172.21.15.116:0/3853905310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30481518c0 0x7f3048151ca0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.860+0000 7f304fc37700 1 -- 172.21.15.116:0/3853905310 >> 172.21.15.116:0/3853905310 conn(0x7f3048068370 msgr2=0x7f30480c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:31.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.860+0000 7f304fc37700 1 -- 172.21.15.116:0/3853905310 shutdown_connections 2020-11-20T12:20:31.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.860+0000 7f304fc37700 1 -- 172.21.15.116:0/3853905310 wait complete. 2020-11-20T12:20:31.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.860+0000 7f304fc37700 1 Processor -- start 2020-11-20T12:20:31.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304fc37700 1 -- start start 2020-11-20T12:20:31.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304fc37700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30480aa210 0x7f304813c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304fc37700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30480c9010 0x7f30480c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304fc37700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30481518c0 0x7f30480c7c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304fc37700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3048065080 con 0x7f30480c9010 2020-11-20T12:20:31.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304fc37700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3048166410 con 0x7f30480aa210 2020-11-20T12:20:31.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304fc37700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30480652e0 con 0x7f30481518c0 2020-11-20T12:20:31.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304d1d2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30480c9010 0x7f30480c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304d1d2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30480c9010 0x7f30480c7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49618/0 (socket says 172.21.15.116:49618) 2020-11-20T12:20:31.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304d1d2700 1 -- 172.21.15.116:0/2967853027 learned_addr learned my addr 172.21.15.116:0/2967853027 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:31.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304d9d3700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30480aa210 0x7f304813c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304e1d4700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30481518c0 0x7f30480c7c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304d1d2700 1 -- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30481518c0 msgr2=0x7f30480c7c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304d1d2700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30481518c0 0x7f30480c7c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304d1d2700 1 -- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30480aa210 msgr2=0x7f304813c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:31.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304d1d2700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30480aa210 0x7f304813c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:31.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.864+0000 7f304d1d2700 1 -- 172.21.15.116:0/2967853027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3048065580 con 0x7f30480c9010 2020-11-20T12:20:31.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.868+0000 7f304d1d2700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30480c9010 0x7f30480c7710 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f303800c710 tx=0x7f3038007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:31.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.868+0000 7f3035ffb700 1 -- 172.21.15.116:0/2967853027 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30380072e0 con 0x7f30480c9010 2020-11-20T12:20:31.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.868+0000 7f3035ffb700 1 -- 172.21.15.116:0/2967853027 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3038007440 con 0x7f30480c9010 2020-11-20T12:20:31.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.868+0000 7f304fc37700 1 -- 172.21.15.116:0/2967853027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30481666b0 con 0x7f30480c9010 2020-11-20T12:20:31.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.868+0000 7f3035ffb700 1 -- 172.21.15.116:0/2967853027 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3038005800 con 0x7f30480c9010 2020-11-20T12:20:31.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.868+0000 7f304fc37700 1 -- 172.21.15.116:0/2967853027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3048165700 con 0x7f30480c9010 2020-11-20T12:20:31.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.868+0000 7f304fc37700 1 -- 172.21.15.116:0/2967853027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3048069950 con 0x7f30480c9010 2020-11-20T12:20:31.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.868+0000 7f3035ffb700 1 -- 172.21.15.116:0/2967853027 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f303801a060 con 0x7f30480c9010 2020-11-20T12:20:31.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.872+0000 7f3035ffb700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30200557c0 0x7f3020057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:31.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.872+0000 7f3035ffb700 1 -- 172.21.15.116:0/2967853027 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f303806dbb0 con 0x7f30480c9010 2020-11-20T12:20:31.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.872+0000 7f304d9d3700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30200557c0 0x7f3020057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:31.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.872+0000 7f304d9d3700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30200557c0 0x7f3020057c80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3044009340 tx=0x7f3044015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:31.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:31.876+0000 7f3035ffb700 1 -- 172.21.15.116:0/2967853027 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3038043c70 con 0x7f30480c9010 2020-11-20T12:20:32.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.032+0000 7f304fc37700 1 -- 172.21.15.116:0/2967853027 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f30480c8df0 con 0x7f30480c9010 2020-11-20T12:20:32.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.032+0000 7f3035ffb700 1 -- 172.21.15.116:0/2967853027 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v90) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3038018030 con 0x7f30480c9010 2020-11-20T12:20:32.031 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:32.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 -- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30200557c0 msgr2=0x7f3020057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30200557c0 0x7f3020057c80 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f3044009340 tx=0x7f3044015000).stop 2020-11-20T12:20:32.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 -- 172.21.15.116:0/2967853027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30480c9010 msgr2=0x7f30480c7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30480c9010 0x7f30480c7710 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f303800c710 tx=0x7f3038007ad0).stop 2020-11-20T12:20:32.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 -- 172.21.15.116:0/2967853027 shutdown_connections 2020-11-20T12:20:32.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f30200557c0 0x7f3020057c80 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f30481518c0 0x7f30480c7c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f30480aa210 0x7f304813c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 --2- 172.21.15.116:0/2967853027 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f30480c9010 0x7f30480c7710 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 -- 172.21.15.116:0/2967853027 >> 172.21.15.116:0/2967853027 conn(0x7f3048068370 msgr2=0x7f3048150210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:32.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 -- 172.21.15.116:0/2967853027 shutdown_connections 2020-11-20T12:20:32.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.036+0000 7f304fc37700 1 -- 172.21.15.116:0/2967853027 wait complete. 2020-11-20T12:20:32.057 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-20T12:20:32.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.232+0000 7f3c25d77700 1 Processor -- start 2020-11-20T12:20:32.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.232+0000 7f3c25d77700 1 -- start start 2020-11-20T12:20:32.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25d77700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c2014bd80 0x7f3c20152790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25d77700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3c200614b0 0x7f3c200618f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25d77700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c200ad230 con 0x7f3c200acd20 2020-11-20T12:20:32.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25d77700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c200a80d0 con 0x7f3c200614b0 2020-11-20T12:20:32.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25d77700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c20061de0 con 0x7f3c2014bd80 2020-11-20T12:20:32.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25576700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3c200614b0 0x7f3c200618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c24d75700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c2014bd80 0x7f3c20152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25576700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3c200614b0 0x7f3c200618f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:48678/0 (socket says 172.21.15.116:48678) 2020-11-20T12:20:32.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25576700 1 -- 172.21.15.116:0/2277901719 learned_addr learned my addr 172.21.15.116:0/2277901719 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:32.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25576700 1 -- 172.21.15.116:0/2277901719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c2014bd80 msgr2=0x7f3c20152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:32.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25576700 1 --2- 172.21.15.116:0/2277901719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c2014bd80 0x7f3c20152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25576700 1 -- 172.21.15.116:0/2277901719 >> v1:172.21.15.173:6789/0 conn(0x7f3c200acd20 legacy=0x7f3c200ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25576700 1 -- 172.21.15.116:0/2277901719 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c200ad4d0 con 0x7f3c200614b0 2020-11-20T12:20:32.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c25576700 1 --2- 172.21.15.116:0/2277901719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c200614b0 0x7f3c200618f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f3c18009d50 tx=0x7f3c18009290).ready entity=mon.1 client_cookie=f119cb16e4654318 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:32.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.236+0000 7f3c1f7fe700 1 -- 172.21.15.116:0/2277901719 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1800bbc0 con 0x7f3c200614b0 2020-11-20T12:20:32.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.240+0000 7f3c1f7fe700 1 -- 172.21.15.116:0/2277901719 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1800bd20 con 0x7f3c200614b0 2020-11-20T12:20:32.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.240+0000 7f3c1f7fe700 1 -- 172.21.15.116:0/2277901719 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c18011610 con 0x7f3c200614b0 2020-11-20T12:20:32.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.240+0000 7f3c25d77700 1 -- 172.21.15.116:0/2277901719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c200614b0 msgr2=0x7f3c200618f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.240+0000 7f3c25d77700 1 --2- 172.21.15.116:0/2277901719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c200614b0 0x7f3c200618f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7f3c18009d50 tx=0x7f3c18009290).stop 2020-11-20T12:20:32.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.240+0000 7f3c25d77700 1 -- 172.21.15.116:0/2277901719 shutdown_connections 2020-11-20T12:20:32.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.240+0000 7f3c25d77700 1 --2- 172.21.15.116:0/2277901719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c200614b0 0x7f3c200618f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.240+0000 7f3c25d77700 1 --2- 172.21.15.116:0/2277901719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c2014bd80 0x7f3c20152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.240+0000 7f3c25d77700 1 -- 172.21.15.116:0/2277901719 >> 172.21.15.116:0/2277901719 conn(0x7f3c20068370 msgr2=0x7f3c200c8730 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:32.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.240+0000 7f3c25d77700 1 -- 172.21.15.116:0/2277901719 shutdown_connections 2020-11-20T12:20:32.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.240+0000 7f3c25d77700 1 -- 172.21.15.116:0/2277901719 wait complete. 2020-11-20T12:20:32.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.240+0000 7f3c25d77700 1 Processor -- start 2020-11-20T12:20:32.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c25d77700 1 -- start start 2020-11-20T12:20:32.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c25d77700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c200614b0 0x7f3c200a8f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c25d77700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c200acd20 0x7f3c200c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c25d77700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c2014bd80 0x7f3c200c84f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.247 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c25d77700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c20168850 con 0x7f3c200614b0 2020-11-20T12:20:32.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c25d77700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c20168af0 con 0x7f3c200acd20 2020-11-20T12:20:32.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c25d77700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c20168d90 con 0x7f3c2014bd80 2020-11-20T12:20:32.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c24d75700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c200614b0 0x7f3c200a8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c24d75700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c200614b0 0x7f3c200a8f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49630/0 (socket says 172.21.15.116:49630) 2020-11-20T12:20:32.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c24d75700 1 -- 172.21.15.116:0/708153494 learned_addr learned my addr 172.21.15.116:0/708153494 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:32.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c1ffff700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c200acd20 0x7f3c200c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c25576700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c2014bd80 0x7f3c200c84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c24d75700 1 -- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c2014bd80 msgr2=0x7f3c200c84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c24d75700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c2014bd80 0x7f3c200c84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c24d75700 1 -- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c200acd20 msgr2=0x7f3c200c52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c24d75700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c200acd20 0x7f3c200c52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c24d75700 1 -- 172.21.15.116:0/708153494 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c20169030 con 0x7f3c200614b0 2020-11-20T12:20:32.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c24d75700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c200614b0 0x7f3c200a8f80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f3c1400ebd0 tx=0x7f3c14006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:32.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.244+0000 7f3c1cff9700 1 -- 172.21.15.116:0/708153494 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1400ba30 con 0x7f3c200614b0 2020-11-20T12:20:32.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.248+0000 7f3c1cff9700 1 -- 172.21.15.116:0/708153494 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1400bb90 con 0x7f3c200614b0 2020-11-20T12:20:32.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.248+0000 7f3c1cff9700 1 -- 172.21.15.116:0/708153494 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c140123f0 con 0x7f3c200614b0 2020-11-20T12:20:32.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.248+0000 7f3c25d77700 1 -- 172.21.15.116:0/708153494 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c201692d0 con 0x7f3c200614b0 2020-11-20T12:20:32.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.248+0000 7f3c25d77700 1 -- 172.21.15.116:0/708153494 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c200a9470 con 0x7f3c200614b0 2020-11-20T12:20:32.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.248+0000 7f3c25d77700 1 -- 172.21.15.116:0/708153494 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c20169690 con 0x7f3c200614b0 2020-11-20T12:20:32.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.248+0000 7f3c1cff9700 1 -- 172.21.15.116:0/708153494 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3c14012760 con 0x7f3c200614b0 2020-11-20T12:20:32.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.248+0000 7f3c1cff9700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bf80557c0 0x7f3bf8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.252+0000 7f3c1cff9700 1 -- 172.21.15.116:0/708153494 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f3c1406e7c0 con 0x7f3c200614b0 2020-11-20T12:20:32.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.252+0000 7f3c1ffff700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bf80557c0 0x7f3bf8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.252+0000 7f3c1ffff700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bf80557c0 0x7f3bf8057c80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f3c200c66c0 tx=0x7f3c0c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:32.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.256+0000 7f3c1cff9700 1 -- 172.21.15.116:0/708153494 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3c14044990 con 0x7f3c200614b0 2020-11-20T12:20:32.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.416+0000 7f3c25d77700 1 -- 172.21.15.116:0/708153494 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3c2005cf30 con 0x7f3c200614b0 2020-11-20T12:20:32.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.416+0000 7f3c1cff9700 1 -- 172.21.15.116:0/708153494 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v90) v1 ==== 84+0+143 (secure 0 0 0) 0x7f3c14047fb0 con 0x7f3c200614b0 2020-11-20T12:20:32.415 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:32.415 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2020-11-20T12:20:32.416 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:32.416 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2020-11-20T12:20:32.416 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "device_health_metrics" 2020-11-20T12:20:32.416 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:20:32.417 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:20:32.417 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 2, 2020-11-20T12:20:32.417 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "rbd" 2020-11-20T12:20:32.418 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:20:32.418 INFO:teuthology.orchestra.run.smithi116.stdout:] 2020-11-20T12:20:32.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 -- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bf80557c0 msgr2=0x7f3bf8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bf80557c0 0x7f3bf8057c80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f3c200c66c0 tx=0x7f3c0c009040).stop 2020-11-20T12:20:32.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 -- 172.21.15.116:0/708153494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c200614b0 msgr2=0x7f3c200a8f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c200614b0 0x7f3c200a8f80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f3c1400ebd0 tx=0x7f3c14006df0).stop 2020-11-20T12:20:32.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 -- 172.21.15.116:0/708153494 shutdown_connections 2020-11-20T12:20:32.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bf80557c0 0x7f3bf8057c80 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c2014bd80 0x7f3c200c84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c200614b0 0x7f3c200a8f80 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 --2- 172.21.15.116:0/708153494 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c200acd20 0x7f3c200c52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 -- 172.21.15.116:0/708153494 >> 172.21.15.116:0/708153494 conn(0x7f3c20068370 msgr2=0x7f3c20151130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:32.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 -- 172.21.15.116:0/708153494 shutdown_connections 2020-11-20T12:20:32.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.420+0000 7f3c25d77700 1 -- 172.21.15.116:0/708153494 wait complete. 2020-11-20T12:20:32.441 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T12:20:32.442 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-20T12:20:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.624+0000 7fb6adb86700 1 Processor -- start 2020-11-20T12:20:32.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.624+0000 7fb6adb86700 1 -- start start 2020-11-20T12:20:32.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.624+0000 7fb6adb86700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb6a80a9750 0x7fb6a80c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.624+0000 7fb6adb86700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb6a814a1d0 0x7fb6a80be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.624+0000 7fb6adb86700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6a80a9c20 con 0x7fb6a8149df0 2020-11-20T12:20:32.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.624+0000 7fb6adb86700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a80a9ec0 con 0x7fb6a80a9750 2020-11-20T12:20:32.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.624+0000 7fb6adb86700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6a8153670 con 0x7fb6a814a1d0 2020-11-20T12:20:32.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb69effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb6a80a9750 0x7fb6a80c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb69effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb6a80a9750 0x7fb6a80c0700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49640/0 (socket says 172.21.15.116:49640) 2020-11-20T12:20:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb69effd700 1 -- 172.21.15.116:0/224632 learned_addr learned my addr 172.21.15.116:0/224632 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb6a7fff700 1 --2- 172.21.15.116:0/224632 >> v2:172.21.15.173:3300/0 conn(0x7fb6a814a1d0 0x7fb6a80be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb69effd700 1 -- 172.21.15.116:0/224632 >> v2:172.21.15.173:3300/0 conn(0x7fb6a814a1d0 msgr2=0x7fb6a80be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:32.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb69effd700 1 --2- 172.21.15.116:0/224632 >> v2:172.21.15.173:3300/0 conn(0x7fb6a814a1d0 0x7fb6a80be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb69effd700 1 -- 172.21.15.116:0/224632 >> v1:172.21.15.173:6789/0 conn(0x7fb6a8149df0 legacy=0x7fb6a8153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb69effd700 1 -- 172.21.15.116:0/224632 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a8153910 con 0x7fb6a80a9750 2020-11-20T12:20:32.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb69effd700 1 --2- 172.21.15.116:0/224632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6a80a9750 0x7fb6a80c0700 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fb6a80c2370 tx=0x7fb690006eb0).ready entity=mon.0 client_cookie=f7b882880625238c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:32.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb6a6ffd700 1 -- 172.21.15.116:0/224632 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6900098d0 con 0x7fb6a80a9750 2020-11-20T12:20:32.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb6a6ffd700 1 -- 172.21.15.116:0/224632 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb690009a30 con 0x7fb6a80a9750 2020-11-20T12:20:32.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb6a6ffd700 1 -- 172.21.15.116:0/224632 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690009db0 con 0x7fb6a80a9750 2020-11-20T12:20:32.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb6adb86700 1 -- 172.21.15.116:0/224632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6a80a9750 msgr2=0x7fb6a80c0700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb6adb86700 1 --2- 172.21.15.116:0/224632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6a80a9750 0x7fb6a80c0700 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fb6a80c2370 tx=0x7fb690006eb0).stop 2020-11-20T12:20:32.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb6adb86700 1 -- 172.21.15.116:0/224632 shutdown_connections 2020-11-20T12:20:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb6adb86700 1 --2- 172.21.15.116:0/224632 >> v2:172.21.15.173:3300/0 conn(0x7fb6a814a1d0 0x7fb6a80be240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb6adb86700 1 --2- 172.21.15.116:0/224632 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6a80a9750 0x7fb6a80c0700 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.628+0000 7fb6adb86700 1 -- 172.21.15.116:0/224632 >> 172.21.15.116:0/224632 conn(0x7fb6a805ba80 msgr2=0x7fb6a80c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:32.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.632+0000 7fb6adb86700 1 -- 172.21.15.116:0/224632 shutdown_connections 2020-11-20T12:20:32.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.632+0000 7fb6adb86700 1 -- 172.21.15.116:0/224632 wait complete. 2020-11-20T12:20:32.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.632+0000 7fb6adb86700 1 Processor -- start 2020-11-20T12:20:32.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.632+0000 7fb6adb86700 1 -- start start 2020-11-20T12:20:32.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb6adb86700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6a8149df0 0x7fb6a8140180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb6adb86700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6a814a1d0 0x7fb6a814cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb6adb86700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6a814d4a0 0x7fb6a813e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb6adb86700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a814f930 con 0x7fb6a814a1d0 2020-11-20T12:20:32.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb6adb86700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a813f3f0 con 0x7fb6a8149df0 2020-11-20T12:20:32.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb6adb86700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6a813fb60 con 0x7fb6a814d4a0 2020-11-20T12:20:32.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69effd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6a814a1d0 0x7fb6a814cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69effd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6a814a1d0 0x7fb6a814cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49648/0 (socket says 172.21.15.116:49648) 2020-11-20T12:20:32.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69effd700 1 -- 172.21.15.116:0/2865360414 learned_addr learned my addr 172.21.15.116:0/2865360414 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:32.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb6a7fff700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6a814d4a0 0x7fb6a813e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb6a77fe700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6a8149df0 0x7fb6a8140180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69effd700 1 -- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6a814d4a0 msgr2=0x7fb6a813e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69effd700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6a814d4a0 0x7fb6a813e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69effd700 1 -- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6a8149df0 msgr2=0x7fb6a8140180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69effd700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6a8149df0 0x7fb6a8140180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69effd700 1 -- 172.21.15.116:0/2865360414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6a8142440 con 0x7fb6a814a1d0 2020-11-20T12:20:32.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69effd700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6a814a1d0 0x7fb6a814cfb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fb6a80611d0 tx=0x7fb690015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:32.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69ffff700 1 -- 172.21.15.116:0/2865360414 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb690011c80 con 0x7fb6a814a1d0 2020-11-20T12:20:32.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69ffff700 1 -- 172.21.15.116:0/2865360414 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb690011de0 con 0x7fb6a814a1d0 2020-11-20T12:20:32.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb6adb86700 1 -- 172.21.15.116:0/2865360414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6a81426b0 con 0x7fb6a814a1d0 2020-11-20T12:20:32.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb69ffff700 1 -- 172.21.15.116:0/2865360414 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb69001e670 con 0x7fb6a814a1d0 2020-11-20T12:20:32.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.636+0000 7fb6adb86700 1 -- 172.21.15.116:0/2865360414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6a8140670 con 0x7fb6a814a1d0 2020-11-20T12:20:32.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.640+0000 7fb6adb86700 1 -- 172.21.15.116:0/2865360414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6a813fec0 con 0x7fb6a814a1d0 2020-11-20T12:20:32.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.640+0000 7fb69ffff700 1 -- 172.21.15.116:0/2865360414 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb690025030 con 0x7fb6a814a1d0 2020-11-20T12:20:32.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.640+0000 7fb69ffff700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6840557c0 0x7fb684057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:32.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.640+0000 7fb69ffff700 1 -- 172.21.15.116:0/2865360414 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb69006f980 con 0x7fb6a814a1d0 2020-11-20T12:20:32.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.644+0000 7fb6a77fe700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6840557c0 0x7fb684057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:32.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.644+0000 7fb6a77fe700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6840557c0 0x7fb684057c80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fb6a81498c0 tx=0x7fb698009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:32.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.648+0000 7fb69ffff700 1 -- 172.21.15.116:0/2865360414 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb6a813fec0 con 0x7fb6a814a1d0 2020-11-20T12:20:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.804+0000 7fb6adb86700 1 -- 172.21.15.116:0/2865360414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fb6a813fec0 con 0x7fb6a814a1d0 2020-11-20T12:20:32.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.968+0000 7fb69ffff700 1 -- 172.21.15.116:0/2865360414 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v91) v1 ==== 133+0+0 (secure 0 0 0) 0x7fb6a813fec0 con 0x7fb6a814a1d0 2020-11-20T12:20:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 -- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6840557c0 msgr2=0x7fb684057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6840557c0 0x7fb684057c80 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fb6a81498c0 tx=0x7fb698009040).stop 2020-11-20T12:20:32.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 -- 172.21.15.116:0/2865360414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6a814a1d0 msgr2=0x7fb6a814cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6a814a1d0 0x7fb6a814cfb0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fb6a80611d0 tx=0x7fb690015000).stop 2020-11-20T12:20:32.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 -- 172.21.15.116:0/2865360414 shutdown_connections 2020-11-20T12:20:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6840557c0 0x7fb684057c80 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6a814d4a0 0x7fb6a813e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6a8149df0 0x7fb6a8140180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 --2- 172.21.15.116:0/2865360414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6a814a1d0 0x7fb6a814cfb0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:32.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 -- 172.21.15.116:0/2865360414 >> 172.21.15.116:0/2865360414 conn(0x7fb6a805ba80 msgr2=0x7fb6a8153cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:32.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 -- 172.21.15.116:0/2865360414 shutdown_connections 2020-11-20T12:20:32.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:32.976+0000 7fb6adb86700 1 -- 172.21.15.116:0/2865360414 wait complete. 2020-11-20T12:20:32.975 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2020-11-20T12:20:33.000 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-20T12:20:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.180+0000 7f9416575700 1 Processor -- start 2020-11-20T12:20:33.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.180+0000 7f9416575700 1 -- start start 2020-11-20T12:20:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.184+0000 7f9416575700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f94100aa7d0 0x7f94100a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.184+0000 7f9416575700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f94100ad540 0x7f94100ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:33.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.184+0000 7f9416575700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94100ace40 con 0x7f94100aabb0 2020-11-20T12:20:33.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.184+0000 7f9416575700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94101526c0 con 0x7f94100ad540 2020-11-20T12:20:33.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.184+0000 7f9416575700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9410152960 con 0x7f94100aa7d0 2020-11-20T12:20:33.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.184+0000 7f9414d72700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f94100aabb0 0x7f9410151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59088/0 (socket says 172.21.15.116:59088) 2020-11-20T12:20:33.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.184+0000 7f9414d72700 1 -- 172.21.15.116:0/1845435928 learned_addr learned my addr 172.21.15.116:0/1845435928 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:33.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.184+0000 7f9415d74700 1 --2- 172.21.15.116:0/1845435928 >> v2:172.21.15.116:3300/0 conn(0x7f94100ad540 0x7f94100ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:33.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.184+0000 7f9415573700 1 --2- 172.21.15.116:0/1845435928 >> v2:172.21.15.173:3300/0 conn(0x7f94100aa7d0 0x7f94100a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:33.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f93fffff700 1 -- 172.21.15.116:0/1845435928 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3625421689 0 0) 0x7f94100ace40 con 0x7f94100aabb0 2020-11-20T12:20:33.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f93fffff700 1 -- 172.21.15.116:0/1845435928 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93f8003240 con 0x7f94100aabb0 2020-11-20T12:20:33.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9415d74700 1 -- 172.21.15.116:0/1845435928 >> v2:172.21.15.173:3300/0 conn(0x7f94100aa7d0 msgr2=0x7f94100a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:33.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9415d74700 1 --2- 172.21.15.116:0/1845435928 >> v2:172.21.15.173:3300/0 conn(0x7f94100aa7d0 0x7f94100a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:33.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9415d74700 1 -- 172.21.15.116:0/1845435928 >> v1:172.21.15.116:6789/0 conn(0x7f94100aabb0 legacy=0x7f9410151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:33.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9415d74700 1 -- 172.21.15.116:0/1845435928 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9410152c00 con 0x7f94100ad540 2020-11-20T12:20:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9415d74700 1 --2- 172.21.15.116:0/1845435928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94100ad540 0x7f94100ad980 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f9410081950 tx=0x7f94080091d0).ready entity=mon.0 client_cookie=88e0ecbb2a2a7700 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f93fffff700 1 -- 172.21.15.116:0/1845435928 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940800b9c0 con 0x7f94100ad540 2020-11-20T12:20:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f93fffff700 1 -- 172.21.15.116:0/1845435928 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940800bb20 con 0x7f94100ad540 2020-11-20T12:20:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f93fffff700 1 -- 172.21.15.116:0/1845435928 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940800bea0 con 0x7f94100ad540 2020-11-20T12:20:33.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9416575700 1 -- 172.21.15.116:0/1845435928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94100ad540 msgr2=0x7f94100ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:33.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9416575700 1 --2- 172.21.15.116:0/1845435928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94100ad540 0x7f94100ad980 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f9410081950 tx=0x7f94080091d0).stop 2020-11-20T12:20:33.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9416575700 1 -- 172.21.15.116:0/1845435928 shutdown_connections 2020-11-20T12:20:33.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9416575700 1 --2- 172.21.15.116:0/1845435928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94100ad540 0x7f94100ad980 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:33.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9416575700 1 --2- 172.21.15.116:0/1845435928 >> v2:172.21.15.173:3300/0 conn(0x7f94100aa7d0 0x7f94100a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:33.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9416575700 1 -- 172.21.15.116:0/1845435928 >> 172.21.15.116:0/1845435928 conn(0x7f94100678a0 msgr2=0x7f9410067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:33.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9416575700 1 -- 172.21.15.116:0/1845435928 shutdown_connections 2020-11-20T12:20:33.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.188+0000 7f9416575700 1 -- 172.21.15.116:0/1845435928 wait complete. 2020-11-20T12:20:33.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.192+0000 7f9416575700 1 Processor -- start 2020-11-20T12:20:33.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.192+0000 7f9416575700 1 -- start start 2020-11-20T12:20:33.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.192+0000 7f9416575700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94100aa7d0 0x7f94100a20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:33.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.192+0000 7f9416575700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94100aabb0 0x7f94100a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:33.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.192+0000 7f9416575700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94100ad540 0x7f94101543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:33.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.192+0000 7f9416575700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9410161dd0 con 0x7f94100ad540 2020-11-20T12:20:33.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.192+0000 7f9416575700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94100a1440 con 0x7f94100aabb0 2020-11-20T12:20:33.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.192+0000 7f9416575700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94100a16e0 con 0x7f94100aa7d0 2020-11-20T12:20:33.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.192+0000 7f9415d74700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94100ad540 0x7f94101543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:33.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9415d74700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94100ad540 0x7f94101543b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49662/0 (socket says 172.21.15.116:49662) 2020-11-20T12:20:33.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9415d74700 1 -- 172.21.15.116:0/881921972 learned_addr learned my addr 172.21.15.116:0/881921972 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:33.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9414d72700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94100aabb0 0x7f94100a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:33.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9415573700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94100aa7d0 0x7f94100a20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:33.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9415d74700 1 -- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94100aa7d0 msgr2=0x7f94100a20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:33.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9415d74700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94100aa7d0 0x7f94100a20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:33.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9415d74700 1 -- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94100aabb0 msgr2=0x7f94100a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:33.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9415d74700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94100aabb0 0x7f94100a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:33.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9415d74700 1 -- 172.21.15.116:0/881921972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94100a1980 con 0x7f94100ad540 2020-11-20T12:20:33.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9414d72700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94100aabb0 0x7f94100a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:33.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9415573700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94100aa7d0 0x7f94100a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:33.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9415d74700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94100ad540 0x7f94101543b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f9408011da0 tx=0x7f9408016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:33.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f93fd7fa700 1 -- 172.21.15.116:0/881921972 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9408006b80 con 0x7f94100ad540 2020-11-20T12:20:33.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f93fd7fa700 1 -- 172.21.15.116:0/881921972 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f940801c5f0 con 0x7f94100ad540 2020-11-20T12:20:33.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f93fd7fa700 1 -- 172.21.15.116:0/881921972 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f940801c970 con 0x7f94100ad540 2020-11-20T12:20:33.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9416575700 1 -- 172.21.15.116:0/881921972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94100a1c20 con 0x7f94100ad540 2020-11-20T12:20:33.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.196+0000 7f9416575700 1 -- 172.21.15.116:0/881921972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94100a4ad0 con 0x7f94100ad540 2020-11-20T12:20:33.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.200+0000 7f9416575700 1 -- 172.21.15.116:0/881921972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94100a1db0 con 0x7f94100ad540 2020-11-20T12:20:33.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.200+0000 7f93fd7fa700 1 -- 172.21.15.116:0/881921972 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9408024030 con 0x7f94100ad540 2020-11-20T12:20:33.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.200+0000 7f93fd7fa700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f93e80557c0 0x7f93e8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:33.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.200+0000 7f93fd7fa700 1 -- 172.21.15.116:0/881921972 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f94080705d0 con 0x7f94100ad540 2020-11-20T12:20:33.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.200+0000 7f9415573700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f93e80557c0 0x7f93e8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:33.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.204+0000 7f9415573700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f93e80557c0 0x7f93e8057c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f940c006fd0 tx=0x7f940c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:33.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.208+0000 7f93fd7fa700 1 -- 172.21.15.116:0/881921972 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f94080465e0 con 0x7f94100ad540 2020-11-20T12:20:33.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:33.364+0000 7f9416575700 1 -- 172.21.15.116:0/881921972 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f941005cf30 con 0x7f94100ad540 2020-11-20T12:20:33.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.000+0000 7f93fd7fa700 1 -- 172.21.15.116:0/881921972 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v94) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9408022020 con 0x7f94100ad540 2020-11-20T12:20:34.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 -- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f93e80557c0 msgr2=0x7f93e8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f93e80557c0 0x7f93e8057c80 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f940c006fd0 tx=0x7f940c008040).stop 2020-11-20T12:20:34.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 -- 172.21.15.116:0/881921972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94100ad540 msgr2=0x7f94101543b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94100ad540 0x7f94101543b0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f9408011da0 tx=0x7f9408016000).stop 2020-11-20T12:20:34.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 -- 172.21.15.116:0/881921972 shutdown_connections 2020-11-20T12:20:34.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f93e80557c0 0x7f93e8057c80 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94100ad540 0x7f94101543b0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94100aa7d0 0x7f94100a20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 --2- 172.21.15.116:0/881921972 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94100aabb0 0x7f94100a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 -- 172.21.15.116:0/881921972 >> 172.21.15.116:0/881921972 conn(0x7f94100678a0 msgr2=0x7f94101538a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:34.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 -- 172.21.15.116:0/881921972 shutdown_connections 2020-11-20T12:20:34.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.004+0000 7f9416575700 1 -- 172.21.15.116:0/881921972 wait complete. 2020-11-20T12:20:34.006 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2020-11-20T12:20:34.027 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T12:20:34.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.212+0000 7ffa24ce2700 1 Processor -- start 2020-11-20T12:20:34.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.212+0000 7ffa24ce2700 1 -- start start 2020-11-20T12:20:34.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.212+0000 7ffa24ce2700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ffa2014f270 0x7ffa200aab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.212+0000 7ffa24ce2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20152ac0 0x7ffa20155db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.212+0000 7ffa24ce2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffa200a9e10 0x7ffa200aa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.212+0000 7ffa24ce2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa201562a0 con 0x7ffa200a9e10 2020-11-20T12:20:34.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.212+0000 7ffa24ce2700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa200ab150 con 0x7ffa2014f270 2020-11-20T12:20:34.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.212+0000 7ffa24ce2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa201520e0 con 0x7ffa20152ac0 2020-11-20T12:20:34.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.212+0000 7ffa1ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffa200a9e10 0x7ffa200aa2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.212+0000 7ffa1ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ffa200a9e10 0x7ffa200aa2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49672/0 (socket says 172.21.15.116:49672) 2020-11-20T12:20:34.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.212+0000 7ffa1ffff700 1 -- 172.21.15.116:0/3319192909 learned_addr learned my addr 172.21.15.116:0/3319192909 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:34.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa1effd700 1 --2- 172.21.15.116:0/3319192909 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20152ac0 0x7ffa20155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa1f7fe700 1 --2- 172.21.15.116:0/3319192909 >> v2:172.21.15.173:3300/0 conn(0x7ffa2014f270 0x7ffa200aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa1ffff700 1 -- 172.21.15.116:0/3319192909 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20152ac0 msgr2=0x7ffa20155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:34.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa1ffff700 1 --2- 172.21.15.116:0/3319192909 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20152ac0 0x7ffa20155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa1ffff700 1 -- 172.21.15.116:0/3319192909 >> v2:172.21.15.173:3300/0 conn(0x7ffa2014f270 msgr2=0x7ffa200aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:34.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa1ffff700 1 --2- 172.21.15.116:0/3319192909 >> v2:172.21.15.173:3300/0 conn(0x7ffa2014f270 0x7ffa200aab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa1ffff700 1 -- 172.21.15.116:0/3319192909 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa20152380 con 0x7ffa200a9e10 2020-11-20T12:20:34.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa1ffff700 1 --2- 172.21.15.116:0/3319192909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa200a9e10 0x7ffa200aa2c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7ffa200c9780 tx=0x7ffa18009290).ready entity=mon.0 client_cookie=571f49d8b1d6c3b1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:34.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa1e7fc700 1 -- 172.21.15.116:0/3319192909 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1800bb10 con 0x7ffa200a9e10 2020-11-20T12:20:34.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa1e7fc700 1 -- 172.21.15.116:0/3319192909 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1800bc70 con 0x7ffa200a9e10 2020-11-20T12:20:34.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa1e7fc700 1 -- 172.21.15.116:0/3319192909 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa18011400 con 0x7ffa200a9e10 2020-11-20T12:20:34.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3319192909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa200a9e10 msgr2=0x7ffa200aa2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa24ce2700 1 --2- 172.21.15.116:0/3319192909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa200a9e10 0x7ffa200aa2c0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7ffa200c9780 tx=0x7ffa18009290).stop 2020-11-20T12:20:34.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3319192909 shutdown_connections 2020-11-20T12:20:34.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa24ce2700 1 --2- 172.21.15.116:0/3319192909 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa200a9e10 0x7ffa200aa2c0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa24ce2700 1 --2- 172.21.15.116:0/3319192909 >> v2:172.21.15.173:3300/0 conn(0x7ffa2014f270 0x7ffa200aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa24ce2700 1 --2- 172.21.15.116:0/3319192909 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa20152ac0 0x7ffa20155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3319192909 >> 172.21.15.116:0/3319192909 conn(0x7ffa200678a0 msgr2=0x7ffa20067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:34.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3319192909 shutdown_connections 2020-11-20T12:20:34.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.216+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3319192909 wait complete. 2020-11-20T12:20:34.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa24ce2700 1 Processor -- start 2020-11-20T12:20:34.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa24ce2700 1 -- start start 2020-11-20T12:20:34.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa24ce2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200a9e10 0x7ffa20161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa24ce2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa2014f270 0x7ffa2014d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa24ce2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa20152ac0 0x7ffa2014dc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa24ce2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2009f320 con 0x7ffa20152ac0 2020-11-20T12:20:34.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa24ce2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2014ad90 con 0x7ffa200a9e10 2020-11-20T12:20:34.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa24ce2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2009f550 con 0x7ffa2014f270 2020-11-20T12:20:34.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa20152ac0 0x7ffa2014dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa20152ac0 0x7ffa2014dc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49676/0 (socket says 172.21.15.116:49676) 2020-11-20T12:20:34.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1ffff700 1 -- 172.21.15.116:0/3016387581 learned_addr learned my addr 172.21.15.116:0/3016387581 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:34.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1f7fe700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200a9e10 0x7ffa20161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1effd700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa2014f270 0x7ffa2014d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1ffff700 1 -- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa2014f270 msgr2=0x7ffa2014d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1ffff700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa2014f270 0x7ffa2014d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1ffff700 1 -- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200a9e10 msgr2=0x7ffa20161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1ffff700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200a9e10 0x7ffa20161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1ffff700 1 -- 172.21.15.116:0/3016387581 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2009f7f0 con 0x7ffa20152ac0 2020-11-20T12:20:34.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1f7fe700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200a9e10 0x7ffa20161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:34.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.220+0000 7ffa1ffff700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa20152ac0 0x7ffa2014dc30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7ffa20081b20 tx=0x7ffa18016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:34.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.224+0000 7ff9ff7fe700 1 -- 172.21.15.116:0/3016387581 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1801b6b0 con 0x7ffa20152ac0 2020-11-20T12:20:34.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.224+0000 7ff9ff7fe700 1 -- 172.21.15.116:0/3016387581 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa1801b810 con 0x7ffa20152ac0 2020-11-20T12:20:34.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.224+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3016387581 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2009fa90 con 0x7ffa20152ac0 2020-11-20T12:20:34.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.224+0000 7ff9ff7fe700 1 -- 172.21.15.116:0/3016387581 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa1801bb90 con 0x7ffa20152ac0 2020-11-20T12:20:34.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.224+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3016387581 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa20160e50 con 0x7ffa20152ac0 2020-11-20T12:20:34.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.224+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3016387581 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa200a00f0 con 0x7ffa20152ac0 2020-11-20T12:20:34.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.224+0000 7ff9ff7fe700 1 -- 172.21.15.116:0/3016387581 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ffa18017070 con 0x7ffa20152ac0 2020-11-20T12:20:34.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.228+0000 7ff9ff7fe700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9f80557c0 0x7ff9f8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.228+0000 7ff9ff7fe700 1 -- 172.21.15.116:0/3016387581 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5957+0+0 (secure 0 0 0) 0x7ffa18070780 con 0x7ffa20152ac0 2020-11-20T12:20:34.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.228+0000 7ffa1f7fe700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9f80557c0 0x7ff9f8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.228+0000 7ffa1f7fe700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9f80557c0 0x7ff9f8057c80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7ffa200a9a90 tx=0x7ffa10008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:34.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.232+0000 7ff9ff7fe700 1 -- 172.21.15.116:0/3016387581 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ffa200a00f0 con 0x7ffa20152ac0 2020-11-20T12:20:34.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.444+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3016387581 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7ffa200a00f0 con 0x7ffa20152ac0 2020-11-20T12:20:34.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.456+0000 7ff9ff7fe700 1 -- 172.21.15.116:0/3016387581 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 17 and data pool 18 v51) v1 ==== 189+0+0 (secure 0 0 0) 0x7ffa200a00f0 con 0x7ffa20152ac0 2020-11-20T12:20:34.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9f80557c0 msgr2=0x7ff9f8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9f80557c0 0x7ff9f8057c80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7ffa200a9a90 tx=0x7ffa10008040).stop 2020-11-20T12:20:34.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3016387581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa20152ac0 msgr2=0x7ffa2014dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa20152ac0 0x7ffa2014dc30 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7ffa20081b20 tx=0x7ffa18016000).stop 2020-11-20T12:20:34.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3016387581 shutdown_connections 2020-11-20T12:20:34.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9f80557c0 0x7ff9f8057c80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ffa20152ac0 0x7ffa2014dc30 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ffa200a9e10 0x7ffa20161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 --2- 172.21.15.116:0/3016387581 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ffa2014f270 0x7ffa2014d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3016387581 >> 172.21.15.116:0/3016387581 conn(0x7ffa200678a0 msgr2=0x7ffa200aa1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:34.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3016387581 shutdown_connections 2020-11-20T12:20:34.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.464+0000 7ffa24ce2700 1 -- 172.21.15.116:0/3016387581 wait complete. 2020-11-20T12:20:34.467 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 17 and data pool 18 2020-11-20T12:20:34.489 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:34.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.656+0000 7f8a127e2700 1 Processor -- start 2020-11-20T12:20:34.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.656+0000 7f8a127e2700 1 -- start start 2020-11-20T12:20:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a127e2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8a0c14e220 0x7f8a0c0bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a127e2700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0b9d20 con 0x7f8a0c0bfa80 2020-11-20T12:20:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a127e2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0b9f80 con 0x7f8a0c0bf100 2020-11-20T12:20:34.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a127e2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c0ba220 con 0x7f8a0c14e220 2020-11-20T12:20:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a0bfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8a0c0bfa80 0x7f8a0c066e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59116/0 (socket says 172.21.15.116:59116) 2020-11-20T12:20:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a0bfff700 1 -- 172.21.15.116:0/3942534035 learned_addr learned my addr 172.21.15.116:0/3942534035 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:34.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a10d7f700 1 --2- 172.21.15.116:0/3942534035 >> v2:172.21.15.116:3300/0 conn(0x7f8a0c14e220 0x7f8a0c0bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a0affd700 1 -- 172.21.15.116:0/3942534035 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 215954748 0 0) 0x7f8a0c0b9d20 con 0x7f8a0c0bfa80 2020-11-20T12:20:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a0affd700 1 -- 172.21.15.116:0/3942534035 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89f8003240 con 0x7f8a0c0bfa80 2020-11-20T12:20:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a10d7f700 1 -- 172.21.15.116:0/3942534035 >> v1:172.21.15.173:6789/0 conn(0x7f8a0c0bf100 legacy=0x7f8a0c14db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a10d7f700 1 -- 172.21.15.116:0/3942534035 >> v1:172.21.15.116:6789/0 conn(0x7f8a0c0bfa80 legacy=0x7f8a0c066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a10d7f700 1 -- 172.21.15.116:0/3942534035 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0ba4c0 con 0x7f8a0c14e220 2020-11-20T12:20:34.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.660+0000 7f8a10d7f700 1 --2- 172.21.15.116:0/3942534035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0c14e220 0x7f8a0c0bfe60 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f8a04009c90 tx=0x7f8a040091d0).ready entity=mon.0 client_cookie=9b6decbcaded909c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:34.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.664+0000 7f8a0affd700 1 -- 172.21.15.116:0/3942534035 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400ba70 con 0x7f8a0c14e220 2020-11-20T12:20:34.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.664+0000 7f8a0affd700 1 -- 172.21.15.116:0/3942534035 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0400bbd0 con 0x7f8a0c14e220 2020-11-20T12:20:34.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.664+0000 7f8a0affd700 1 -- 172.21.15.116:0/3942534035 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04011400 con 0x7f8a0c14e220 2020-11-20T12:20:34.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.664+0000 7f8a127e2700 1 -- 172.21.15.116:0/3942534035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0c14e220 msgr2=0x7f8a0c0bfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.664+0000 7f8a127e2700 1 --2- 172.21.15.116:0/3942534035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0c14e220 0x7f8a0c0bfe60 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7f8a04009c90 tx=0x7f8a040091d0).stop 2020-11-20T12:20:34.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.664+0000 7f8a127e2700 1 -- 172.21.15.116:0/3942534035 shutdown_connections 2020-11-20T12:20:34.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.664+0000 7f8a127e2700 1 --2- 172.21.15.116:0/3942534035 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0c14e220 0x7f8a0c0bfe60 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.664+0000 7f8a127e2700 1 -- 172.21.15.116:0/3942534035 >> 172.21.15.116:0/3942534035 conn(0x7f8a0c0c4f80 msgr2=0x7f8a0c0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:34.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.664+0000 7f8a127e2700 1 -- 172.21.15.116:0/3942534035 shutdown_connections 2020-11-20T12:20:34.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.664+0000 7f8a127e2700 1 -- 172.21.15.116:0/3942534035 wait complete. 2020-11-20T12:20:34.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a127e2700 1 Processor -- start 2020-11-20T12:20:34.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a127e2700 1 -- start start 2020-11-20T12:20:34.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a127e2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0c0bf100 0x7f8a0c0bb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a127e2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a0c0bfa80 0x7f8a0c148f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a127e2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a0c14e220 0x7f8a0c0a9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a127e2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0bd160 con 0x7f8a0c0bf100 2020-11-20T12:20:34.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a127e2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c1524f0 con 0x7f8a0c14e220 2020-11-20T12:20:34.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a127e2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c0bbb70 con 0x7f8a0c0bfa80 2020-11-20T12:20:34.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a0bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0c0bf100 0x7f8a0c0bb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a0bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0c0bf100 0x7f8a0c0bb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49688/0 (socket says 172.21.15.116:49688) 2020-11-20T12:20:34.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a0bfff700 1 -- 172.21.15.116:0/1008030685 learned_addr learned my addr 172.21.15.116:0/1008030685 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:34.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.668+0000 7f8a10d7f700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a0c14e220 0x7f8a0c0a9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f8a0bfff700 1 -- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a0c0bfa80 msgr2=0x7f8a0c148f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f8a0bfff700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a0c0bfa80 0x7f8a0c148f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:34.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f8a0bfff700 1 -- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a0c14e220 msgr2=0x7f8a0c0a9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f8a0bfff700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a0c14e220 0x7f8a0c0a9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f8a0bfff700 1 -- 172.21.15.116:0/1008030685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c0bbe10 con 0x7f8a0c0bf100 2020-11-20T12:20:34.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f8a0bfff700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0c0bf100 0x7f8a0c0bb540 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f89fc00c6f0 tx=0x7f89fc007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:34.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f89ebfff700 1 -- 172.21.15.116:0/1008030685 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc007320 con 0x7f8a0c0bf100 2020-11-20T12:20:34.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f89ebfff700 1 -- 172.21.15.116:0/1008030685 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc007480 con 0x7f8a0c0bf100 2020-11-20T12:20:34.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f8a127e2700 1 -- 172.21.15.116:0/1008030685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c0bc0b0 con 0x7f8a0c0bf100 2020-11-20T12:20:34.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f89ebfff700 1 -- 172.21.15.116:0/1008030685 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc005800 con 0x7f8a0c0bf100 2020-11-20T12:20:34.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f8a127e2700 1 -- 172.21.15.116:0/1008030685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c149470 con 0x7f8a0c0bf100 2020-11-20T12:20:34.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f8a127e2700 1 -- 172.21.15.116:0/1008030685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c0bc550 con 0x7f8a0c0bf100 2020-11-20T12:20:34.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.672+0000 7f89ebfff700 1 -- 172.21.15.116:0/1008030685 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f89fc010030 con 0x7f8a0c0bf100 2020-11-20T12:20:34.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.676+0000 7f89ebfff700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f89e40557c0 0x7f89e4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:34.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.676+0000 7f89ebfff700 1 -- 172.21.15.116:0/1008030685 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f89fc06eba0 con 0x7f8a0c0bf100 2020-11-20T12:20:34.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.676+0000 7f8a0b7fe700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f89e40557c0 0x7f89e4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:34.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.676+0000 7f8a0b7fe700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f89e40557c0 0x7f89e4057c80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f8a0c0c1b30 tx=0x7f8a00007810).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:34.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.680+0000 7f89ebfff700 1 -- 172.21.15.116:0/1008030685 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f89fc0449e0 con 0x7f8a0c0bf100 2020-11-20T12:20:34.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.840+0000 7f8a127e2700 1 -- 172.21.15.116:0/1008030685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a0c05d2a0 con 0x7f8a0c0bf100 2020-11-20T12:20:34.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.844+0000 7f89ebfff700 1 -- 172.21.15.116:0/1008030685 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16890 (secure 0 0 0) 0x7f89fc048000 con 0x7f8a0c0bf100 2020-11-20T12:20:34.841 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:34.841 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":95,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:20:34.448953+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:20:32.886768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-20T12:20:33.446960+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:34.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.844+0000 7f8a127e2700 1 -- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f89e40557c0 msgr2=0x7f89e4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.844+0000 7f8a127e2700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f89e40557c0 0x7f89e4057c80 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f8a0c0c1b30 tx=0x7f8a00007810).stop 2020-11-20T12:20:34.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.844+0000 7f8a127e2700 1 -- 172.21.15.116:0/1008030685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0c0bf100 msgr2=0x7f8a0c0bb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:34.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.844+0000 7f8a127e2700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0c0bf100 0x7f8a0c0bb540 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f89fc00c6f0 tx=0x7f89fc007ad0).stop 2020-11-20T12:20:34.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.844+0000 7f8a127e2700 1 -- 172.21.15.116:0/1008030685 shutdown_connections 2020-11-20T12:20:34.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.844+0000 7f8a127e2700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f89e40557c0 0x7f89e4057c80 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.844+0000 7f8a127e2700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8a0c14e220 0x7f8a0c0a9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.844+0000 7f8a127e2700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8a0c0bf100 0x7f8a0c0bb540 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:34.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.844+0000 7f8a127e2700 1 --2- 172.21.15.116:0/1008030685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8a0c0bfa80 0x7f8a0c148f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:34.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.848+0000 7f8a127e2700 1 -- 172.21.15.116:0/1008030685 >> 172.21.15.116:0/1008030685 conn(0x7f8a0c0c4f80 msgr2=0x7f8a0c063990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:34.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.848+0000 7f8a127e2700 1 -- 172.21.15.116:0/1008030685 shutdown_connections 2020-11-20T12:20:34.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:34.848+0000 7f8a127e2700 1 -- 172.21.15.116:0/1008030685 wait complete. 2020-11-20T12:20:34.870 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:35.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.040+0000 7f00fef21700 1 Processor -- start 2020-11-20T12:20:35.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.040+0000 7f00fef21700 1 -- start start 2020-11-20T12:20:35.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fef21700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f00f8062880 0x7f00f8062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fef21700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f00f8065d40 0x7f00f80bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fef21700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f80c0530 con 0x7f00f80677a0 2020-11-20T12:20:35.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fef21700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f80661a0 con 0x7f00f8065d40 2020-11-20T12:20:35.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fef21700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00f80c0310 con 0x7f00f8062880 2020-11-20T12:20:35.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fd71e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f00f8065d40 0x7f00f80bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fd71e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f00f8065d40 0x7f00f80bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49698/0 (socket says 172.21.15.116:49698) 2020-11-20T12:20:35.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fd71e700 1 -- 172.21.15.116:0/2347122084 learned_addr learned my addr 172.21.15.116:0/2347122084 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:35.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fe720700 1 --1- 172.21.15.116:0/2347122084 >> v1:172.21.15.116:6789/0 conn(0x7f00f80677a0 0x7f00f81556e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59134/0 (socket says 172.21.15.116:59134) 2020-11-20T12:20:35.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fdf1f700 1 --2- 172.21.15.116:0/2347122084 >> v2:172.21.15.173:3300/0 conn(0x7f00f8062880 0x7f00f8062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fd71e700 1 -- 172.21.15.116:0/2347122084 >> v2:172.21.15.173:3300/0 conn(0x7f00f8062880 msgr2=0x7f00f8062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:35.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fd71e700 1 --2- 172.21.15.116:0/2347122084 >> v2:172.21.15.173:3300/0 conn(0x7f00f8062880 0x7f00f8062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fd71e700 1 -- 172.21.15.116:0/2347122084 >> v1:172.21.15.116:6789/0 conn(0x7f00f80677a0 legacy=0x7f00f81556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:35.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fd71e700 1 -- 172.21.15.116:0/2347122084 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f80c07d0 con 0x7f00f8065d40 2020-11-20T12:20:35.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fd71e700 1 --2- 172.21.15.116:0/2347122084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f8065d40 0x7f00f80bfe20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f00f80c1e20 tx=0x7f00e8006eb0).ready entity=mon.0 client_cookie=123dee9376fcc580 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fcf1d700 1 -- 172.21.15.116:0/2347122084 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e80097d0 con 0x7f00f8065d40 2020-11-20T12:20:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fcf1d700 1 -- 172.21.15.116:0/2347122084 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e8009930 con 0x7f00f8065d40 2020-11-20T12:20:35.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fcf1d700 1 -- 172.21.15.116:0/2347122084 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e8009cb0 con 0x7f00f8065d40 2020-11-20T12:20:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fef21700 1 -- 172.21.15.116:0/2347122084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f8065d40 msgr2=0x7f00f80bfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:35.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fef21700 1 --2- 172.21.15.116:0/2347122084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f8065d40 0x7f00f80bfe20 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f00f80c1e20 tx=0x7f00e8006eb0).stop 2020-11-20T12:20:35.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fef21700 1 -- 172.21.15.116:0/2347122084 shutdown_connections 2020-11-20T12:20:35.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fef21700 1 --2- 172.21.15.116:0/2347122084 >> v2:172.21.15.173:3300/0 conn(0x7f00f8062880 0x7f00f8062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fef21700 1 --2- 172.21.15.116:0/2347122084 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f8065d40 0x7f00f80bfe20 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.044+0000 7f00fef21700 1 -- 172.21.15.116:0/2347122084 >> 172.21.15.116:0/2347122084 conn(0x7f00f8060c40 msgr2=0x7f00f80c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:35.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.048+0000 7f00fef21700 1 -- 172.21.15.116:0/2347122084 shutdown_connections 2020-11-20T12:20:35.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.048+0000 7f00fef21700 1 -- 172.21.15.116:0/2347122084 wait complete. 2020-11-20T12:20:35.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.048+0000 7f00fef21700 1 Processor -- start 2020-11-20T12:20:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.048+0000 7f00fef21700 1 -- start start 2020-11-20T12:20:35.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fef21700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00f8062880 0x7f00f8141610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fef21700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f80677a0 0x7f00f80bb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fef21700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00f80bd180 0x7f00f80bd600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fef21700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f80a5500 con 0x7f00f80677a0 2020-11-20T12:20:35.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fef21700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f80a57a0 con 0x7f00f8062880 2020-11-20T12:20:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fef21700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f80a5a40 con 0x7f00f80bd180 2020-11-20T12:20:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fd71e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f80677a0 0x7f00f80bb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fd71e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f80677a0 0x7f00f80bb3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49704/0 (socket says 172.21.15.116:49704) 2020-11-20T12:20:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fd71e700 1 -- 172.21.15.116:0/2577494280 learned_addr learned my addr 172.21.15.116:0/2577494280 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:35.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fe720700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00f80bd180 0x7f00f80bd600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fdf1f700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00f8062880 0x7f00f8141610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fd71e700 1 -- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00f80bd180 msgr2=0x7f00f80bd600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:35.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fd71e700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00f80bd180 0x7f00f80bd600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fd71e700 1 -- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00f8062880 msgr2=0x7f00f8141610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:35.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fd71e700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00f8062880 0x7f00f8141610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fd71e700 1 -- 172.21.15.116:0/2577494280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f80a5fb0 con 0x7f00f80677a0 2020-11-20T12:20:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fd71e700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f80677a0 0x7f00f80bb3f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f00e8006c40 tx=0x7f00e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00e5ffb700 1 -- 172.21.15.116:0/2577494280 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e8021070 con 0x7f00f80677a0 2020-11-20T12:20:35.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00e5ffb700 1 -- 172.21.15.116:0/2577494280 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e8011e30 con 0x7f00f80677a0 2020-11-20T12:20:35.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fef21700 1 -- 172.21.15.116:0/2577494280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f80a5cf0 con 0x7f00f80677a0 2020-11-20T12:20:35.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00e5ffb700 1 -- 172.21.15.116:0/2577494280 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e801d680 con 0x7f00f80677a0 2020-11-20T12:20:35.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.052+0000 7f00fef21700 1 -- 172.21.15.116:0/2577494280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f815c6e0 con 0x7f00f80677a0 2020-11-20T12:20:35.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.056+0000 7f00fef21700 1 -- 172.21.15.116:0/2577494280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f80a6140 con 0x7f00f80677a0 2020-11-20T12:20:35.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.056+0000 7f00e5ffb700 1 -- 172.21.15.116:0/2577494280 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f00e800f070 con 0x7f00f80677a0 2020-11-20T12:20:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.056+0000 7f00e5ffb700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00d00557c0 0x7f00d0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.056+0000 7f00e5ffb700 1 -- 172.21.15.116:0/2577494280 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f00e80701d0 con 0x7f00f80677a0 2020-11-20T12:20:35.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.056+0000 7f00fdf1f700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00d00557c0 0x7f00d0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.060+0000 7f00fdf1f700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00d00557c0 0x7f00d0057c80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f00f4006fd0 tx=0x7f00f4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:35.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.064+0000 7f00e5ffb700 1 -- 172.21.15.116:0/2577494280 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f00e8045fa0 con 0x7f00f80677a0 2020-11-20T12:20:35.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.224+0000 7f00fef21700 1 -- 172.21.15.116:0/2577494280 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00f805cf30 con 0x7f00f80677a0 2020-11-20T12:20:35.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.228+0000 7f00e5ffb700 1 -- 172.21.15.116:0/2577494280 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16890 (secure 0 0 0) 0x7f00e80495c0 con 0x7f00f80677a0 2020-11-20T12:20:35.225 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:35.225 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":95,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:20:34.448953+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:20:32.886768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-20T12:20:33.446960+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:35.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 -- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00d00557c0 msgr2=0x7f00d0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:35.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00d00557c0 0x7f00d0057c80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7f00f4006fd0 tx=0x7f00f4008040).stop 2020-11-20T12:20:35.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 -- 172.21.15.116:0/2577494280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f80677a0 msgr2=0x7f00f80bb3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f80677a0 0x7f00f80bb3f0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f00e8006c40 tx=0x7f00e8015000).stop 2020-11-20T12:20:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 -- 172.21.15.116:0/2577494280 shutdown_connections 2020-11-20T12:20:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f00d00557c0 0x7f00d0057c80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f00f80bd180 0x7f00f80bd600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f00f8062880 0x7f00f8141610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 --2- 172.21.15.116:0/2577494280 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f00f80677a0 0x7f00f80bb3f0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 -- 172.21.15.116:0/2577494280 >> 172.21.15.116:0/2577494280 conn(0x7f00f8060c40 msgr2=0x7f00f81549f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 -- 172.21.15.116:0/2577494280 shutdown_connections 2020-11-20T12:20:35.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.232+0000 7f00fef21700 1 -- 172.21.15.116:0/2577494280 wait complete. 2020-11-20T12:20:35.250 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-20T12:20:35.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.432+0000 7f8595a67700 1 Processor -- start 2020-11-20T12:20:35.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.432+0000 7f8595a67700 1 -- start start 2020-11-20T12:20:35.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f8595a67700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8590068500 0x7f85900a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f8595a67700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8590151630 0x7f8590151a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f8595a67700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85900c49a0 con 0x7f85901558c0 2020-11-20T12:20:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f8595a67700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85900c4700 con 0x7f8590068500 2020-11-20T12:20:35.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f8595a67700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85900c9140 con 0x7f8590151630 2020-11-20T12:20:35.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f858effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8590068500 0x7f85900a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f858effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8590068500 0x7f85900a8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49710/0 (socket says 172.21.15.116:49710) 2020-11-20T12:20:35.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f858effd700 1 -- 172.21.15.116:0/3997778899 learned_addr learned my addr 172.21.15.116:0/3997778899 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:35.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f858ffff700 1 --2- 172.21.15.116:0/3997778899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8590151630 0x7f8590151a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f858effd700 1 -- 172.21.15.116:0/3997778899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8590151630 msgr2=0x7f8590151a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f858effd700 1 --2- 172.21.15.116:0/3997778899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8590151630 0x7f8590151a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f858effd700 1 -- 172.21.15.116:0/3997778899 >> v1:172.21.15.173:6789/0 conn(0x7f85901558c0 legacy=0x7f8590155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f858effd700 1 -- 172.21.15.116:0/3997778899 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85900c93e0 con 0x7f8590068500 2020-11-20T12:20:35.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.436+0000 7f858effd700 1 --2- 172.21.15.116:0/3997778899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590068500 0x7f85900a8ee0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f8584006fd0 tx=0x7f8584006eb0).ready entity=mon.0 client_cookie=7fd2ee16e2d06302 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:35.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.440+0000 7f858e7fc700 1 -- 172.21.15.116:0/3997778899 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584009810 con 0x7f8590068500 2020-11-20T12:20:35.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.440+0000 7f858e7fc700 1 -- 172.21.15.116:0/3997778899 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8584009970 con 0x7f8590068500 2020-11-20T12:20:35.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.440+0000 7f858e7fc700 1 -- 172.21.15.116:0/3997778899 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584009cf0 con 0x7f8590068500 2020-11-20T12:20:35.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.440+0000 7f8595a67700 1 -- 172.21.15.116:0/3997778899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590068500 msgr2=0x7f85900a8ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:35.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.440+0000 7f8595a67700 1 --2- 172.21.15.116:0/3997778899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590068500 0x7f85900a8ee0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f8584006fd0 tx=0x7f8584006eb0).stop 2020-11-20T12:20:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.440+0000 7f8595a67700 1 -- 172.21.15.116:0/3997778899 shutdown_connections 2020-11-20T12:20:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.440+0000 7f8595a67700 1 --2- 172.21.15.116:0/3997778899 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8590151630 0x7f8590151a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.440+0000 7f8595a67700 1 --2- 172.21.15.116:0/3997778899 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8590068500 0x7f85900a8ee0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.440+0000 7f8595a67700 1 -- 172.21.15.116:0/3997778899 >> 172.21.15.116:0/3997778899 conn(0x7f8590068080 msgr2=0x7f85900c8920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.440+0000 7f8595a67700 1 -- 172.21.15.116:0/3997778899 shutdown_connections 2020-11-20T12:20:35.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.440+0000 7f8595a67700 1 -- 172.21.15.116:0/3997778899 wait complete. 2020-11-20T12:20:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f8595a67700 1 Processor -- start 2020-11-20T12:20:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f8595a67700 1 -- start start 2020-11-20T12:20:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f8595a67700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8590151630 0x7f859014dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f8595a67700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85901558c0 0x7f8590154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f8595a67700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85900abae0 0x7f8590164dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f8595a67700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85900ac060 con 0x7f85901558c0 2020-11-20T12:20:35.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f8595a67700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85900ac300 con 0x7f8590151630 2020-11-20T12:20:35.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f8595a67700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85900ac5a0 con 0x7f85900abae0 2020-11-20T12:20:35.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f858effd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85901558c0 0x7f8590154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f858effd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85901558c0 0x7f8590154a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49718/0 (socket says 172.21.15.116:49718) 2020-11-20T12:20:35.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f858effd700 1 -- 172.21.15.116:0/4243720460 learned_addr learned my addr 172.21.15.116:0/4243720460 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:35.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f858f7fe700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8590151630 0x7f859014dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f858ffff700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85900abae0 0x7f8590164dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f858effd700 1 -- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85900abae0 msgr2=0x7f8590164dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:35.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f858effd700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85900abae0 0x7f8590164dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f858effd700 1 -- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8590151630 msgr2=0x7f859014dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:35.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f858effd700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8590151630 0x7f859014dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:35.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f858effd700 1 -- 172.21.15.116:0/4243720460 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85900ac840 con 0x7f85901558c0 2020-11-20T12:20:35.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.444+0000 7f858effd700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85901558c0 0x7f8590154a90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f8584010000 tx=0x7f8584015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.448+0000 7f856f7fe700 1 -- 172.21.15.116:0/4243720460 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584011c80 con 0x7f85901558c0 2020-11-20T12:20:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.448+0000 7f8595a67700 1 -- 172.21.15.116:0/4243720460 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85900acae0 con 0x7f85901558c0 2020-11-20T12:20:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.448+0000 7f856f7fe700 1 -- 172.21.15.116:0/4243720460 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8584011de0 con 0x7f85901558c0 2020-11-20T12:20:35.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.448+0000 7f856f7fe700 1 -- 172.21.15.116:0/4243720460 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858401e6b0 con 0x7f85901558c0 2020-11-20T12:20:35.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.448+0000 7f8595a67700 1 -- 172.21.15.116:0/4243720460 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8590154f80 con 0x7f85901558c0 2020-11-20T12:20:35.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.448+0000 7f8595a67700 1 -- 172.21.15.116:0/4243720460 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f859005d2a0 con 0x7f85901558c0 2020-11-20T12:20:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.448+0000 7f856f7fe700 1 -- 172.21.15.116:0/4243720460 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8584025030 con 0x7f85901558c0 2020-11-20T12:20:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.448+0000 7f856f7fe700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f85680557c0 0x7f8568057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.448+0000 7f856f7fe700 1 -- 172.21.15.116:0/4243720460 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f858406fd40 con 0x7f85901558c0 2020-11-20T12:20:35.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.448+0000 7f858f7fe700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f85680557c0 0x7f8568057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:35.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.452+0000 7f858f7fe700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f85680557c0 0x7f8568057c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f8580003610 tx=0x7f8580000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:35.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.456+0000 7f856f7fe700 1 -- 172.21.15.116:0/4243720460 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8584045be0 con 0x7f85901558c0 2020-11-20T12:20:35.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:35.664+0000 7f8595a67700 1 -- 172.21.15.116:0/4243720460 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8590096710 con 0x7f85901558c0 2020-11-20T12:20:35.743 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2020-11-20T12:20:36.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.480+0000 7f856f7fe700 1 -- 172.21.15.116:0/4243720460 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8584023070 con 0x7f85901558c0 2020-11-20T12:20:36.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 -- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f85680557c0 msgr2=0x7f8568057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:36.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f85680557c0 0x7f8568057c80 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f8580003610 tx=0x7f8580000b80).stop 2020-11-20T12:20:36.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 -- 172.21.15.116:0/4243720460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85901558c0 msgr2=0x7f8590154a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:36.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85901558c0 0x7f8590154a90 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f8584010000 tx=0x7f8584015000).stop 2020-11-20T12:20:36.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 -- 172.21.15.116:0/4243720460 shutdown_connections 2020-11-20T12:20:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f85680557c0 0x7f8568057c80 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85900abae0 0x7f8590164dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8590151630 0x7f859014dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 --2- 172.21.15.116:0/4243720460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85901558c0 0x7f8590154a90 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 -- 172.21.15.116:0/4243720460 >> 172.21.15.116:0/4243720460 conn(0x7f8590068080 msgr2=0x7f8590062390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:36.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 -- 172.21.15.116:0/4243720460 shutdown_connections 2020-11-20T12:20:36.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.484+0000 7f8595a67700 1 -- 172.21.15.116:0/4243720460 wait complete. 2020-11-20T12:20:36.509 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:36.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.684+0000 7f9dff008700 1 Processor -- start 2020-11-20T12:20:36.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.684+0000 7f9dff008700 1 -- start start 2020-11-20T12:20:36.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.688+0000 7f9dff008700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9df8148040 0x7f9df8148420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:36.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.688+0000 7f9dff008700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9df8068420 0x7f9df80626e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:36.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.688+0000 7f9dff008700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9df8062bd0 0x7f9df80c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:36.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.688+0000 7f9dff008700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df8067190 con 0x7f9df8148040 2020-11-20T12:20:36.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.688+0000 7f9dff008700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df80677a0 con 0x7f9df8068420 2020-11-20T12:20:36.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.688+0000 7f9dff008700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df8067a40 con 0x7f9df8062bd0 2020-11-20T12:20:36.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.688+0000 7f9dfe006700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9df8148040 0x7f9df8148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:36.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.688+0000 7f9dfe006700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9df8148040 0x7f9df8148420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49724/0 (socket says 172.21.15.116:49724) 2020-11-20T12:20:36.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.688+0000 7f9dfe006700 1 -- 172.21.15.116:0/2874913465 learned_addr learned my addr 172.21.15.116:0/2874913465 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:36.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dfd805700 1 --2- 172.21.15.116:0/2874913465 >> v2:172.21.15.173:3300/0 conn(0x7f9df8068420 0x7f9df80626e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:36.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dfe807700 1 --2- 172.21.15.116:0/2874913465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9df8062bd0 0x7f9df80c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:36.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dfe006700 1 -- 172.21.15.116:0/2874913465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9df8062bd0 msgr2=0x7f9df80c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:36.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dfe006700 1 --2- 172.21.15.116:0/2874913465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9df8062bd0 0x7f9df80c4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dfe006700 1 -- 172.21.15.116:0/2874913465 >> v2:172.21.15.173:3300/0 conn(0x7f9df8068420 msgr2=0x7f9df80626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:36.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dfe006700 1 --2- 172.21.15.116:0/2874913465 >> v2:172.21.15.173:3300/0 conn(0x7f9df8068420 0x7f9df80626e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dfe006700 1 -- 172.21.15.116:0/2874913465 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df8067ce0 con 0x7f9df8148040 2020-11-20T12:20:36.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dfe006700 1 --2- 172.21.15.116:0/2874913465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9df8148040 0x7f9df8148420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7f9df8061c70 tx=0x7f9df4006eb0).ready entity=mon.0 client_cookie=e3f5af9f1de6dced server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:36.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dfd004700 1 -- 172.21.15.116:0/2874913465 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df40098d0 con 0x7f9df8148040 2020-11-20T12:20:36.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dfd004700 1 -- 172.21.15.116:0/2874913465 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9df4009a30 con 0x7f9df8148040 2020-11-20T12:20:36.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dfd004700 1 -- 172.21.15.116:0/2874913465 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9df4009db0 con 0x7f9df8148040 2020-11-20T12:20:36.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dff008700 1 -- 172.21.15.116:0/2874913465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9df8148040 msgr2=0x7f9df8148420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:36.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dff008700 1 --2- 172.21.15.116:0/2874913465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9df8148040 0x7f9df8148420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7f9df8061c70 tx=0x7f9df4006eb0).stop 2020-11-20T12:20:36.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dff008700 1 -- 172.21.15.116:0/2874913465 shutdown_connections 2020-11-20T12:20:36.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dff008700 1 --2- 172.21.15.116:0/2874913465 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9df8062bd0 0x7f9df80c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dff008700 1 --2- 172.21.15.116:0/2874913465 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9df8148040 0x7f9df8148420 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dff008700 1 --2- 172.21.15.116:0/2874913465 >> v2:172.21.15.173:3300/0 conn(0x7f9df8068420 0x7f9df80626e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dff008700 1 -- 172.21.15.116:0/2874913465 >> 172.21.15.116:0/2874913465 conn(0x7f9df8060c40 msgr2=0x7f9df80c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:36.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dff008700 1 -- 172.21.15.116:0/2874913465 shutdown_connections 2020-11-20T12:20:36.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.692+0000 7f9dff008700 1 -- 172.21.15.116:0/2874913465 wait complete. 2020-11-20T12:20:36.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dff008700 1 Processor -- start 2020-11-20T12:20:36.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dff008700 1 -- start start 2020-11-20T12:20:36.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dff008700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9df8062bd0 0x7f9df8161810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:36.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dff008700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9df8068420 0x7f9df814ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:36.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dff008700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9df8148040 0x7f9df8137da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:36.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dff008700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df809e270 con 0x7f9df8068420 2020-11-20T12:20:36.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dff008700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df809e510 con 0x7f9df8062bd0 2020-11-20T12:20:36.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dff008700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df809e7b0 con 0x7f9df8148040 2020-11-20T12:20:36.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dfd805700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9df8068420 0x7f9df814ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:36.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dfd805700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9df8068420 0x7f9df814ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49732/0 (socket says 172.21.15.116:49732) 2020-11-20T12:20:36.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dfd805700 1 -- 172.21.15.116:0/3157944395 learned_addr learned my addr 172.21.15.116:0/3157944395 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:36.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dfe006700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9df8062bd0 0x7f9df8161810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:36.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.696+0000 7f9dfe807700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9df8148040 0x7f9df8137da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9dfd805700 1 -- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9df8148040 msgr2=0x7f9df8137da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9dfd805700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9df8148040 0x7f9df8137da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9dfd805700 1 -- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9df8062bd0 msgr2=0x7f9df8161810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9dfd805700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9df8062bd0 0x7f9df8161810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9dfd805700 1 -- 172.21.15.116:0/3157944395 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df809ea50 con 0x7f9df8068420 2020-11-20T12:20:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9dfd805700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9df8068420 0x7f9df814ca30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f9de800db30 tx=0x7f9de8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9de67fc700 1 -- 172.21.15.116:0/3157944395 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8009730 con 0x7f9df8068420 2020-11-20T12:20:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9de67fc700 1 -- 172.21.15.116:0/3157944395 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de8009890 con 0x7f9df8068420 2020-11-20T12:20:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9de67fc700 1 -- 172.21.15.116:0/3157944395 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8009c10 con 0x7f9df8068420 2020-11-20T12:20:36.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9dff008700 1 -- 172.21.15.116:0/3157944395 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df809ecf0 con 0x7f9df8068420 2020-11-20T12:20:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9dff008700 1 -- 172.21.15.116:0/3157944395 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df8161d00 con 0x7f9df8068420 2020-11-20T12:20:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9dff008700 1 -- 172.21.15.116:0/3157944395 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df809ee80 con 0x7f9df8068420 2020-11-20T12:20:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.700+0000 7f9de67fc700 1 -- 172.21.15.116:0/3157944395 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9de8013480 con 0x7f9df8068420 2020-11-20T12:20:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.704+0000 7f9de67fc700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9dd00557c0 0x7f9dd0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:36.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.704+0000 7f9de67fc700 1 -- 172.21.15.116:0/3157944395 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9de806e980 con 0x7f9df8068420 2020-11-20T12:20:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.704+0000 7f9dfe006700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9dd00557c0 0x7f9dd0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.704+0000 7f9dfe006700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9dd00557c0 0x7f9dd0057c80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f9df4006ef0 tx=0x7f9df4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:36.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.708+0000 7f9de67fc700 1 -- 172.21.15.116:0/3157944395 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9de80447c0 con 0x7f9df8068420 2020-11-20T12:20:36.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.912+0000 7f9dff008700 1 -- 172.21.15.116:0/3157944395 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9df805cf30 con 0x7f9df8068420 2020-11-20T12:20:36.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.912+0000 7f9de67fc700 1 -- 172.21.15.116:0/3157944395 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2848 (secure 0 0 0) 0x7f9de8047de0 con 0x7f9df8068420 2020-11-20T12:20:36.911 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:36.911 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3105911814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3105911814},{"type":"v1","addr":"172.21.15.173:6837","nonce":3105911814}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/3457547857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3457547857},{"type":"v1","addr":"172.21.15.116:6835","nonce":3457547857}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/81195738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":81195738},{"type":"v1","addr":"172.21.15.116:6833","nonce":81195738}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:20:34.449524+0000","modified":"2020-11-20T12:20:36.475264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/2361610151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2361610151},{"type":"v1","addr":"172.21.15.173:6835","nonce":2361610151}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-20T12:20:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 -- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9dd00557c0 msgr2=0x7f9dd0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:36.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9dd00557c0 0x7f9dd0057c80 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f9df4006ef0 tx=0x7f9df4015000).stop 2020-11-20T12:20:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 -- 172.21.15.116:0/3157944395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9df8068420 msgr2=0x7f9df814ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9df8068420 0x7f9df814ca30 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7f9de800db30 tx=0x7f9de8008040).stop 2020-11-20T12:20:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 -- 172.21.15.116:0/3157944395 shutdown_connections 2020-11-20T12:20:36.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9dd00557c0 0x7f9dd0057c80 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9df8148040 0x7f9df8137da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9df8062bd0 0x7f9df8161810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 --2- 172.21.15.116:0/3157944395 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9df8068420 0x7f9df814ca30 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 -- 172.21.15.116:0/3157944395 >> 172.21.15.116:0/3157944395 conn(0x7f9df8060c40 msgr2=0x7f9df80c2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:36.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 -- 172.21.15.116:0/3157944395 shutdown_connections 2020-11-20T12:20:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:36.916+0000 7f9dff008700 1 -- 172.21.15.116:0/3157944395 wait complete. 2020-11-20T12:20:36.917 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 54 2020-11-20T12:20:36.937 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:37.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.116+0000 7f9d2ac72700 1 Processor -- start 2020-11-20T12:20:37.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.116+0000 7f9d2ac72700 1 -- start start 2020-11-20T12:20:37.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2ac72700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d240bf100 0x7f9d240bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2ac72700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9d240bac00 0x7f9d2414db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2ac72700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d240b9d20 con 0x7f9d240bfa80 2020-11-20T12:20:37.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2ac72700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d240b9f80 con 0x7f9d240bf100 2020-11-20T12:20:37.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2ac72700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d240ba220 con 0x7f9d240bac00 2020-11-20T12:20:37.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d23fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d240bf100 0x7f9d240bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d23fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d240bf100 0x7f9d240bfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49740/0 (socket says 172.21.15.116:49740) 2020-11-20T12:20:37.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d23fff700 1 -- 172.21.15.116:0/3211436287 learned_addr learned my addr 172.21.15.116:0/3211436287 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:37.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2920f700 1 --2- 172.21.15.116:0/3211436287 >> v2:172.21.15.173:3300/0 conn(0x7f9d240bac00 0x7f9d2414db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d23fff700 1 -- 172.21.15.116:0/3211436287 >> v2:172.21.15.173:3300/0 conn(0x7f9d240bac00 msgr2=0x7f9d2414db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:37.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d23fff700 1 --2- 172.21.15.116:0/3211436287 >> v2:172.21.15.173:3300/0 conn(0x7f9d240bac00 0x7f9d2414db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d23fff700 1 -- 172.21.15.116:0/3211436287 >> v1:172.21.15.173:6789/0 conn(0x7f9d240bfa80 legacy=0x7f9d24066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:37.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d23fff700 1 -- 172.21.15.116:0/3211436287 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d240ba4c0 con 0x7f9d240bf100 2020-11-20T12:20:37.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d23fff700 1 --2- 172.21.15.116:0/3211436287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d240bf100 0x7f9d240bfe60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f9d24065280 tx=0x7f9d14006f40).ready entity=mon.0 client_cookie=e8e17bf7b3a13d0e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:37.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d237fe700 1 -- 172.21.15.116:0/3211436287 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1400dc20 con 0x7f9d240bf100 2020-11-20T12:20:37.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d237fe700 1 -- 172.21.15.116:0/3211436287 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1400dd80 con 0x7f9d240bf100 2020-11-20T12:20:37.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d237fe700 1 -- 172.21.15.116:0/3211436287 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d14010680 con 0x7f9d240bf100 2020-11-20T12:20:37.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2ac72700 1 -- 172.21.15.116:0/3211436287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d240bf100 msgr2=0x7f9d240bfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2ac72700 1 --2- 172.21.15.116:0/3211436287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d240bf100 0x7f9d240bfe60 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f9d24065280 tx=0x7f9d14006f40).stop 2020-11-20T12:20:37.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2ac72700 1 -- 172.21.15.116:0/3211436287 shutdown_connections 2020-11-20T12:20:37.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2ac72700 1 --2- 172.21.15.116:0/3211436287 >> v2:172.21.15.173:3300/0 conn(0x7f9d240bac00 0x7f9d2414db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2ac72700 1 --2- 172.21.15.116:0/3211436287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d240bf100 0x7f9d240bfe60 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.120+0000 7f9d2ac72700 1 -- 172.21.15.116:0/3211436287 >> 172.21.15.116:0/3211436287 conn(0x7f9d240c4f80 msgr2=0x7f9d240c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:37.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.124+0000 7f9d2ac72700 1 -- 172.21.15.116:0/3211436287 shutdown_connections 2020-11-20T12:20:37.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.124+0000 7f9d2ac72700 1 -- 172.21.15.116:0/3211436287 wait complete. 2020-11-20T12:20:37.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.124+0000 7f9d2ac72700 1 Processor -- start 2020-11-20T12:20:37.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.124+0000 7f9d2ac72700 1 -- start start 2020-11-20T12:20:37.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d2ac72700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d240bac00 0x7f9d24160d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d2ac72700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d240bf100 0x7f9d240bbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d2ac72700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d240bfa80 0x7f9d240bcf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d2ac72700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2414c950 con 0x7f9d240bac00 2020-11-20T12:20:37.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d2ac72700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d240a5940 con 0x7f9d240bf100 2020-11-20T12:20:37.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d2ac72700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d24161270 con 0x7f9d240bfa80 2020-11-20T12:20:37.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d28a0e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d240bac00 0x7f9d24160d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d28a0e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d240bac00 0x7f9d24160d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49744/0 (socket says 172.21.15.116:49744) 2020-11-20T12:20:37.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d28a0e700 1 -- 172.21.15.116:0/76874876 learned_addr learned my addr 172.21.15.116:0/76874876 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:37.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d23fff700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d240bf100 0x7f9d240bbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d2920f700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d240bfa80 0x7f9d240bcf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d28a0e700 1 -- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d240bfa80 msgr2=0x7f9d240bcf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d28a0e700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d240bfa80 0x7f9d240bcf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d28a0e700 1 -- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d240bf100 msgr2=0x7f9d240bbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d28a0e700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d240bf100 0x7f9d240bbf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d28a0e700 1 -- 172.21.15.116:0/76874876 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d240a4a20 con 0x7f9d240bac00 2020-11-20T12:20:37.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d28a0e700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d240bac00 0x7f9d24160d80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7f9d0c009710 tx=0x7f9d0c006d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:37.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.128+0000 7f9d20ff9700 1 -- 172.21.15.116:0/76874876 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00d490 con 0x7f9d240bac00 2020-11-20T12:20:37.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.132+0000 7f9d20ff9700 1 -- 172.21.15.116:0/76874876 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0c000e90 con 0x7f9d240bac00 2020-11-20T12:20:37.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.132+0000 7f9d2ac72700 1 -- 172.21.15.116:0/76874876 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d240a4cc0 con 0x7f9d240bac00 2020-11-20T12:20:37.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.132+0000 7f9d20ff9700 1 -- 172.21.15.116:0/76874876 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0c00d860 con 0x7f9d240bac00 2020-11-20T12:20:37.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.132+0000 7f9d2ac72700 1 -- 172.21.15.116:0/76874876 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d240a54a0 con 0x7f9d240bac00 2020-11-20T12:20:37.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.132+0000 7f9d2ac72700 1 -- 172.21.15.116:0/76874876 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2405d2a0 con 0x7f9d240bac00 2020-11-20T12:20:37.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.132+0000 7f9d20ff9700 1 -- 172.21.15.116:0/76874876 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9d0c00f030 con 0x7f9d240bac00 2020-11-20T12:20:37.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.132+0000 7f9d20ff9700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d000557f0 0x7f9d00057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.136+0000 7f9d20ff9700 1 -- 172.21.15.116:0/76874876 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9d0c06e2c0 con 0x7f9d240bac00 2020-11-20T12:20:37.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.136+0000 7f9d23fff700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d000557f0 0x7f9d00057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.136+0000 7f9d23fff700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d000557f0 0x7f9d00057cb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f9d24065280 tx=0x7f9d240b9c70).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:37.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.140+0000 7f9d20ff9700 1 -- 172.21.15.116:0/76874876 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9d0c044050 con 0x7f9d240bac00 2020-11-20T12:20:37.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.300+0000 7f9d2ac72700 1 -- 172.21.15.116:0/76874876 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d24061380 con 0x7f9d240bac00 2020-11-20T12:20:37.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d20ff9700 1 -- 172.21.15.116:0/76874876 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f9d0c047670 con 0x7f9d240bac00 2020-11-20T12:20:37.300 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:37.300 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":96,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:20:35.459061+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:20:32.886768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-20T12:20:33.446960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:37.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 -- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d000557f0 msgr2=0x7f9d00057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d000557f0 0x7f9d00057cb0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f9d24065280 tx=0x7f9d240b9c70).stop 2020-11-20T12:20:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 -- 172.21.15.116:0/76874876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d240bac00 msgr2=0x7f9d24160d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d240bac00 0x7f9d24160d80 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7f9d0c009710 tx=0x7f9d0c006d70).stop 2020-11-20T12:20:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 -- 172.21.15.116:0/76874876 shutdown_connections 2020-11-20T12:20:37.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9d000557f0 0x7f9d00057cb0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d240bfa80 0x7f9d240bcf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d240bac00 0x7f9d24160d80 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 --2- 172.21.15.116:0/76874876 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d240bf100 0x7f9d240bbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 -- 172.21.15.116:0/76874876 >> 172.21.15.116:0/76874876 conn(0x7f9d240c4f80 msgr2=0x7f9d240a1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 -- 172.21.15.116:0/76874876 shutdown_connections 2020-11-20T12:20:37.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.304+0000 7f9d2ac72700 1 -- 172.21.15.116:0/76874876 wait complete. 2020-11-20T12:20:37.327 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-20T12:20:37.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.516+0000 7f31309fa700 1 Processor -- start 2020-11-20T12:20:37.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.516+0000 7f31309fa700 1 -- start start 2020-11-20T12:20:37.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.516+0000 7f31309fa700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f312c0c0220 0x7f312c0c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.516+0000 7f31309fa700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f312c0664f0 0x7f312c066930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.516+0000 7f31309fa700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f312c062880 con 0x7f312c063490 2020-11-20T12:20:37.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.516+0000 7f31309fa700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312c067200 con 0x7f312c0c0220 2020-11-20T12:20:37.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.516+0000 7f31309fa700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f312c062ab0 con 0x7f312c0664f0 2020-11-20T12:20:37.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.516+0000 7f3129d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f312c063490 0x7f312c060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59186/0 (socket says 172.21.15.116:59186) 2020-11-20T12:20:37.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.516+0000 7f3129d9b700 1 -- 172.21.15.116:0/3620074244 learned_addr learned my addr 172.21.15.116:0/3620074244 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:37.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.516+0000 7f312a59c700 1 --2- 172.21.15.116:0/3620074244 >> v2:172.21.15.116:3300/0 conn(0x7f312c0c0220 0x7f312c0c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f312ad9d700 1 --2- 172.21.15.116:0/3620074244 >> v2:172.21.15.173:3300/0 conn(0x7f312c0664f0 0x7f312c066930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f312959a700 1 -- 172.21.15.116:0/3620074244 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 256319600 0 0) 0x7f312c062880 con 0x7f312c063490 2020-11-20T12:20:37.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f312959a700 1 -- 172.21.15.116:0/3620074244 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3110003240 con 0x7f312c063490 2020-11-20T12:20:37.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f312a59c700 1 -- 172.21.15.116:0/3620074244 >> v2:172.21.15.173:3300/0 conn(0x7f312c0664f0 msgr2=0x7f312c066930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:37.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f312a59c700 1 --2- 172.21.15.116:0/3620074244 >> v2:172.21.15.173:3300/0 conn(0x7f312c0664f0 0x7f312c066930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f312a59c700 1 -- 172.21.15.116:0/3620074244 >> v1:172.21.15.116:6789/0 conn(0x7f312c063490 legacy=0x7f312c060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f312a59c700 1 -- 172.21.15.116:0/3620074244 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c062d50 con 0x7f312c0c0220 2020-11-20T12:20:37.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f312a59c700 1 --2- 172.21.15.116:0/3620074244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f312c0c0220 0x7f312c0c0600 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f312c05d100 tx=0x7f311c006e80).ready entity=mon.0 client_cookie=60e5305c4a00ba3a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:37.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f312959a700 1 -- 172.21.15.116:0/3620074244 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c00bb50 con 0x7f312c0c0220 2020-11-20T12:20:37.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f312959a700 1 -- 172.21.15.116:0/3620074244 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f311c00bcb0 con 0x7f312c0c0220 2020-11-20T12:20:37.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f312959a700 1 -- 172.21.15.116:0/3620074244 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f311c010680 con 0x7f312c0c0220 2020-11-20T12:20:37.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f31309fa700 1 -- 172.21.15.116:0/3620074244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f312c0c0220 msgr2=0x7f312c0c0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f31309fa700 1 --2- 172.21.15.116:0/3620074244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f312c0c0220 0x7f312c0c0600 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f312c05d100 tx=0x7f311c006e80).stop 2020-11-20T12:20:37.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f31309fa700 1 -- 172.21.15.116:0/3620074244 shutdown_connections 2020-11-20T12:20:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f31309fa700 1 --2- 172.21.15.116:0/3620074244 >> v2:172.21.15.173:3300/0 conn(0x7f312c0664f0 0x7f312c066930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f31309fa700 1 --2- 172.21.15.116:0/3620074244 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f312c0c0220 0x7f312c0c0600 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f31309fa700 1 -- 172.21.15.116:0/3620074244 >> 172.21.15.116:0/3620074244 conn(0x7f312c05ba80 msgr2=0x7f312c0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f31309fa700 1 -- 172.21.15.116:0/3620074244 shutdown_connections 2020-11-20T12:20:37.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.520+0000 7f31309fa700 1 -- 172.21.15.116:0/3620074244 wait complete. 2020-11-20T12:20:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f31309fa700 1 Processor -- start 2020-11-20T12:20:37.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f31309fa700 1 -- start start 2020-11-20T12:20:37.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f31309fa700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f312c063490 0x7f312c138e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f31309fa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f312c0664f0 0x7f312c14fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f31309fa700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f312c0c0220 0x7f312c1503c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f31309fa700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c151400 con 0x7f312c0c0220 2020-11-20T12:20:37.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f31309fa700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f312c150980 con 0x7f312c0664f0 2020-11-20T12:20:37.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f31309fa700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f312c150c20 con 0x7f312c063490 2020-11-20T12:20:37.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f312a59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f312c063490 0x7f312c138e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f3129d9b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f312c0664f0 0x7f312c14fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f312a59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f312c063490 0x7f312c138e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:58708/0 (socket says 172.21.15.116:58708) 2020-11-20T12:20:37.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.524+0000 7f312a59c700 1 -- 172.21.15.116:0/372548685 learned_addr learned my addr 172.21.15.116:0/372548685 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:37.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f312ad9d700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f312c0c0220 0x7f312c1503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f312ad9d700 1 -- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f312c063490 msgr2=0x7f312c138e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f312ad9d700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f312c063490 0x7f312c138e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f312ad9d700 1 -- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f312c0664f0 msgr2=0x7f312c14fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f312ad9d700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f312c0664f0 0x7f312c14fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f312ad9d700 1 -- 172.21.15.116:0/372548685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c150ec0 con 0x7f312c0c0220 2020-11-20T12:20:37.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f312ad9d700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f312c0c0220 0x7f312c1503c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f312400d570 tx=0x7f31240060d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:37.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f311a7fc700 1 -- 172.21.15.116:0/372548685 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312400de00 con 0x7f312c0c0220 2020-11-20T12:20:37.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f311a7fc700 1 -- 172.21.15.116:0/372548685 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3124012600 con 0x7f312c0c0220 2020-11-20T12:20:37.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f31309fa700 1 -- 172.21.15.116:0/372548685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f312c151160 con 0x7f312c0c0220 2020-11-20T12:20:37.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f311a7fc700 1 -- 172.21.15.116:0/372548685 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31240129a0 con 0x7f312c0c0220 2020-11-20T12:20:37.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f31309fa700 1 -- 172.21.15.116:0/372548685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f312c139330 con 0x7f312c0c0220 2020-11-20T12:20:37.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.528+0000 7f31309fa700 1 -- 172.21.15.116:0/372548685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f312c148b80 con 0x7f312c0c0220 2020-11-20T12:20:37.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.532+0000 7f311a7fc700 1 -- 172.21.15.116:0/372548685 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3124022070 con 0x7f312c0c0220 2020-11-20T12:20:37.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.532+0000 7f311a7fc700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f31000557c0 0x7f3100057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.532+0000 7f311a7fc700 1 -- 172.21.15.116:0/372548685 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f312406f2e0 con 0x7f312c0c0220 2020-11-20T12:20:37.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.532+0000 7f312a59c700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f31000557c0 0x7f3100057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.532+0000 7f312a59c700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f31000557c0 0x7f3100057c80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f311c00d3e0 tx=0x7f312c0674a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:37.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.536+0000 7f311a7fc700 1 -- 172.21.15.116:0/372548685 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f312c148b80 con 0x7f312c0c0220 2020-11-20T12:20:37.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.748+0000 7f31309fa700 1 -- 172.21.15.116:0/372548685 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f312c148b80 con 0x7f312c0c0220 2020-11-20T12:20:37.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.756+0000 7f311a7fc700 1 -- 172.21.15.116:0/372548685 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 184+0+0 (secure 0 0 0) 0x7f312c148b80 con 0x7f312c0c0220 2020-11-20T12:20:37.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 -- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f31000557c0 msgr2=0x7f3100057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f31000557c0 0x7f3100057c80 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f311c00d3e0 tx=0x7f312c0674a0).stop 2020-11-20T12:20:37.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 -- 172.21.15.116:0/372548685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f312c0c0220 msgr2=0x7f312c1503c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f312c0c0220 0x7f312c1503c0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f312400d570 tx=0x7f31240060d0).stop 2020-11-20T12:20:37.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 -- 172.21.15.116:0/372548685 shutdown_connections 2020-11-20T12:20:37.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f31000557c0 0x7f3100057c80 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f312c0c0220 0x7f312c1503c0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f312c063490 0x7f312c138e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 --2- 172.21.15.116:0/372548685 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f312c0664f0 0x7f312c14fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 -- 172.21.15.116:0/372548685 >> 172.21.15.116:0/372548685 conn(0x7f312c05ba80 msgr2=0x7f312c147fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:37.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 -- 172.21.15.116:0/372548685 shutdown_connections 2020-11-20T12:20:37.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.760+0000 7f31309fa700 1 -- 172.21.15.116:0/372548685 wait complete. 2020-11-20T12:20:37.761 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2020-11-20T12:20:37.778 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:37.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.952+0000 7fc96739d700 1 Processor -- start 2020-11-20T12:20:37.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.952+0000 7fc96739d700 1 -- start start 2020-11-20T12:20:37.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.952+0000 7fc96739d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc960062880 0x7fc960062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.952+0000 7fc96739d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc96014b990 0x7fc96014be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.952+0000 7fc96739d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc9600b94e0 0x7fc9600b9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.952+0000 7fc96739d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9600ba1b0 con 0x7fc9600b94e0 2020-11-20T12:20:37.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.952+0000 7fc96739d700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9601555a0 con 0x7fc960062880 2020-11-20T12:20:37.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.952+0000 7fc96739d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc960155840 con 0x7fc96014b990 2020-11-20T12:20:37.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.952+0000 7fc966b9c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc9600b94e0 0x7fc9600b9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc965b9a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc96014b990 0x7fc96014be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc96639b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc960062880 0x7fc960062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc966b9c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc9600b94e0 0x7fc9600b9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49770/0 (socket says 172.21.15.116:49770) 2020-11-20T12:20:37.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc966b9c700 1 -- 172.21.15.116:0/3568159542 learned_addr learned my addr 172.21.15.116:0/3568159542 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:37.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc966b9c700 1 -- 172.21.15.116:0/3568159542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc96014b990 msgr2=0x7fc96014be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:37.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc966b9c700 1 --2- 172.21.15.116:0/3568159542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc96014b990 0x7fc96014be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc966b9c700 1 -- 172.21.15.116:0/3568159542 >> v2:172.21.15.173:3300/0 conn(0x7fc960062880 msgr2=0x7fc960062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:37.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc966b9c700 1 --2- 172.21.15.116:0/3568159542 >> v2:172.21.15.173:3300/0 conn(0x7fc960062880 0x7fc960062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc966b9c700 1 -- 172.21.15.116:0/3568159542 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc960155ae0 con 0x7fc9600b94e0 2020-11-20T12:20:37.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc96639b700 1 --2- 172.21.15.116:0/3568159542 >> v2:172.21.15.173:3300/0 conn(0x7fc960062880 0x7fc960062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:37.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc966b9c700 1 --2- 172.21.15.116:0/3568159542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9600b94e0 0x7fc9600b9940 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fc9600c1e20 tx=0x7fc958009290).ready entity=mon.0 client_cookie=e0d655d4abe445b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:37.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc965399700 1 -- 172.21.15.116:0/3568159542 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95800bb10 con 0x7fc9600b94e0 2020-11-20T12:20:37.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc965399700 1 -- 172.21.15.116:0/3568159542 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95800bc70 con 0x7fc9600b94e0 2020-11-20T12:20:37.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc965399700 1 -- 172.21.15.116:0/3568159542 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc958011400 con 0x7fc9600b94e0 2020-11-20T12:20:37.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc96739d700 1 -- 172.21.15.116:0/3568159542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9600b94e0 msgr2=0x7fc9600b9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc96739d700 1 --2- 172.21.15.116:0/3568159542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9600b94e0 0x7fc9600b9940 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fc9600c1e20 tx=0x7fc958009290).stop 2020-11-20T12:20:37.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc96739d700 1 -- 172.21.15.116:0/3568159542 shutdown_connections 2020-11-20T12:20:37.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc96739d700 1 --2- 172.21.15.116:0/3568159542 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9600b94e0 0x7fc9600b9940 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc96739d700 1 --2- 172.21.15.116:0/3568159542 >> v2:172.21.15.173:3300/0 conn(0x7fc960062880 0x7fc960062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc96739d700 1 --2- 172.21.15.116:0/3568159542 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc96014b990 0x7fc96014be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc96739d700 1 -- 172.21.15.116:0/3568159542 >> 172.21.15.116:0/3568159542 conn(0x7fc960060c40 msgr2=0x7fc9600c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:37.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.956+0000 7fc96739d700 1 -- 172.21.15.116:0/3568159542 shutdown_connections 2020-11-20T12:20:37.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.960+0000 7fc96739d700 1 -- 172.21.15.116:0/3568159542 wait complete. 2020-11-20T12:20:37.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.960+0000 7fc96739d700 1 Processor -- start 2020-11-20T12:20:37.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.960+0000 7fc96739d700 1 -- start start 2020-11-20T12:20:37.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc96739d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc960062880 0x7fc960161b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc96739d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9600b94e0 0x7fc9600bd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc96739d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96014b990 0x7fc9600a53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc96739d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600a79f0 con 0x7fc96014b990 2020-11-20T12:20:37.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc96739d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc960162070 con 0x7fc9600b94e0 2020-11-20T12:20:37.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc96739d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9600a7750 con 0x7fc960062880 2020-11-20T12:20:37.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc966b9c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96014b990 0x7fc9600a53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc96639b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc960062880 0x7fc960161b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc966b9c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96014b990 0x7fc9600a53b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49776/0 (socket says 172.21.15.116:49776) 2020-11-20T12:20:37.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc966b9c700 1 -- 172.21.15.116:0/2907200888 learned_addr learned my addr 172.21.15.116:0/2907200888 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:37.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc965b9a700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9600b94e0 0x7fc9600bd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc966b9c700 1 -- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc960062880 msgr2=0x7fc960161b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc966b9c700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc960062880 0x7fc960161b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc966b9c700 1 -- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9600b94e0 msgr2=0x7fc9600bd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:37.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc966b9c700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9600b94e0 0x7fc9600bd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:37.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc966b9c700 1 -- 172.21.15.116:0/2907200888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc960067b60 con 0x7fc96014b990 2020-11-20T12:20:37.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc966b9c700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96014b990 0x7fc9600a53b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7fc96005d820 tx=0x7fc958016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:37.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc94e7fc700 1 -- 172.21.15.116:0/2907200888 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95801b6b0 con 0x7fc96014b990 2020-11-20T12:20:37.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc94e7fc700 1 -- 172.21.15.116:0/2907200888 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc95801b810 con 0x7fc96014b990 2020-11-20T12:20:37.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc94e7fc700 1 -- 172.21.15.116:0/2907200888 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc95801bb90 con 0x7fc96014b990 2020-11-20T12:20:37.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc96739d700 1 -- 172.21.15.116:0/2907200888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc960067e00 con 0x7fc96014b990 2020-11-20T12:20:37.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.964+0000 7fc96739d700 1 -- 172.21.15.116:0/2907200888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9600a6a70 con 0x7fc96014b990 2020-11-20T12:20:37.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.968+0000 7fc96739d700 1 -- 172.21.15.116:0/2907200888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc96014b500 con 0x7fc96014b990 2020-11-20T12:20:37.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.968+0000 7fc94e7fc700 1 -- 172.21.15.116:0/2907200888 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc958017070 con 0x7fc96014b990 2020-11-20T12:20:37.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.968+0000 7fc94e7fc700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9380557c0 0x7fc938057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:37.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.968+0000 7fc94e7fc700 1 -- 172.21.15.116:0/2907200888 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc958070580 con 0x7fc96014b990 2020-11-20T12:20:37.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.972+0000 7fc96639b700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9380557c0 0x7fc938057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:37.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.972+0000 7fc96639b700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9380557c0 0x7fc938057c80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fc95c006fd0 tx=0x7fc95c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:37.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:37.972+0000 7fc94e7fc700 1 -- 172.21.15.116:0/2907200888 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc96014b500 con 0x7fc96014b990 2020-11-20T12:20:38.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.180+0000 7fc96739d700 1 -- 172.21.15.116:0/2907200888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc96014b500 con 0x7fc96014b990 2020-11-20T12:20:38.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.180+0000 7fc94e7fc700 1 -- 172.21.15.116:0/2907200888 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2848 (secure 0 0 0) 0x7fc96014b500 con 0x7fc96014b990 2020-11-20T12:20:38.178 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:38.178 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3105911814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3105911814},{"type":"v1","addr":"172.21.15.173:6837","nonce":3105911814}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/3457547857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3457547857},{"type":"v1","addr":"172.21.15.116:6835","nonce":3457547857}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/81195738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":81195738},{"type":"v1","addr":"172.21.15.116:6833","nonce":81195738}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:20:34.449524+0000","modified":"2020-11-20T12:20:36.475264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/2361610151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2361610151},{"type":"v1","addr":"172.21.15.173:6835","nonce":2361610151}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-20T12:20:38.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 -- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9380557c0 msgr2=0x7fc938057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:38.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9380557c0 0x7fc938057c80 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fc95c006fd0 tx=0x7fc95c006df0).stop 2020-11-20T12:20:38.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 -- 172.21.15.116:0/2907200888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96014b990 msgr2=0x7fc9600a53b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:38.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96014b990 0x7fc9600a53b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7fc96005d820 tx=0x7fc958016000).stop 2020-11-20T12:20:38.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 -- 172.21.15.116:0/2907200888 shutdown_connections 2020-11-20T12:20:38.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9380557c0 0x7fc938057c80 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:38.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc96014b990 0x7fc9600a53b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:38.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc960062880 0x7fc960161b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:38.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 --2- 172.21.15.116:0/2907200888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9600b94e0 0x7fc9600bd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:38.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 -- 172.21.15.116:0/2907200888 >> 172.21.15.116:0/2907200888 conn(0x7fc960060c40 msgr2=0x7fc9600c3570 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:38.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 -- 172.21.15.116:0/2907200888 shutdown_connections 2020-11-20T12:20:38.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:38.184+0000 7fc96739d700 1 -- 172.21.15.116:0/2907200888 wait complete. 2020-11-20T12:20:38.183 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 54 2020-11-20T12:20:38.605 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:20:34.449524+0000', 'modified': '2020-11-20T12:20:36.475264+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7311}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6835/2361610151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2361610151}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2361610151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:20:38.606 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:20:38.607 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-20T12:20:38.802 INFO:teuthology.orchestra.run.smithi173.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":7311,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.3346275729999997,"mdsmap_epoch":54,"osdmap_epoch":96,"osdmap_epoch_barrier":96,"uptime":9.8542675269999993} 2020-11-20T12:20:38.803 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 7311, "mdsmap_epoch": 54, "osdmap_epoch": 96, "osdmap_epoch_barrier": 96, "rank_uptime": 4.334627573, "state": "up:active", "uptime": 9.854267527, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:20:38.803 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:20:38.803 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:20:38.804 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:20:38.804 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:20:38.805 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:20:38.805 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:20:38.805 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:20:38.806 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:20:38.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:38.806 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:20:38.812 INFO:tasks.ceph.mds.b.smithi173.stderr:2020-11-20T12:20:38.818+0000 7f14ba2ca700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:20:38.822 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:20:38.822 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:20:38.822 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:20:38.823 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:38.823 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:20:38.823 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:38.823 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:38.824 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:38.824 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:38.824 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:38.824 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:38.824 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:38.825 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:20:38.825 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:38.825 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:20:38.825 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:38.825 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:20:38.826 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:38.826 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:20:38.826 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:38.826 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:20:38.826 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:38.827 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:20:38.827 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:38.827 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:20:38.827 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:38.827 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:20:38.828 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:38.828 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:38.828 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:20:38.828 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:20:38.828 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:38.829 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:38.829 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:20:38.829 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:20:38.829 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:38.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:38.831 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:20:38.863 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:20:38.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:38.867 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:20:38.883 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:20:38.883 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:20:38.883 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:20:38.884 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:20:38.884 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:20:38.884 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:20:38.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:38.886 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:20:38.886 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:20:38.887 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:20:38.887 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:20:38.887 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:20:38.888 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:20:38.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:38.990 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:20:39.005 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:20:39.005 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:20:39.006 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:20:39.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:39.006 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2020-11-20T12:20:39.037 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:39.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.208+0000 7f9828c0f700 1 Processor -- start 2020-11-20T12:20:39.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.208+0000 7f9828c0f700 1 -- start start 2020-11-20T12:20:39.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f9828c0f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9824062880 0x7f9824062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f9828c0f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f98240bfe20 0x7f98241556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f9828c0f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98240c02d0 con 0x7f9824065d40 2020-11-20T12:20:39.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f9828c0f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98240661a0 con 0x7f9824062880 2020-11-20T12:20:39.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f9828c0f700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98240c0530 con 0x7f98240bfe20 2020-11-20T12:20:39.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f98237fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9824062880 0x7f9824062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f98237fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9824062880 0x7f9824062c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49780/0 (socket says 172.21.15.116:49780) 2020-11-20T12:20:39.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f98237fe700 1 -- 172.21.15.116:0/2328522658 learned_addr learned my addr 172.21.15.116:0/2328522658 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:39.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f98237fe700 1 -- 172.21.15.116:0/2328522658 >> v2:172.21.15.173:3300/0 conn(0x7f98240bfe20 msgr2=0x7f98241556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:39.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f98237fe700 1 --2- 172.21.15.116:0/2328522658 >> v2:172.21.15.173:3300/0 conn(0x7f98240bfe20 0x7f98241556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:39.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f98237fe700 1 -- 172.21.15.116:0/2328522658 >> v1:172.21.15.173:6789/0 conn(0x7f9824065d40 legacy=0x7f98240677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:39.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f98237fe700 1 -- 172.21.15.116:0/2328522658 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98240c07d0 con 0x7f9824062880 2020-11-20T12:20:39.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f98237fe700 1 --2- 172.21.15.116:0/2328522658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9824062880 0x7f9824062c60 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f98240a70a0 tx=0x7f9814006eb0).ready entity=mon.0 client_cookie=4731b7e7057bad66 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:39.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f9822ffd700 1 -- 172.21.15.116:0/2328522658 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814009890 con 0x7f9824062880 2020-11-20T12:20:39.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f9822ffd700 1 -- 172.21.15.116:0/2328522658 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98140099f0 con 0x7f9824062880 2020-11-20T12:20:39.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.212+0000 7f9822ffd700 1 -- 172.21.15.116:0/2328522658 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9814009d70 con 0x7f9824062880 2020-11-20T12:20:39.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.216+0000 7f9828c0f700 1 -- 172.21.15.116:0/2328522658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9824062880 msgr2=0x7f9824062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.216+0000 7f9828c0f700 1 --2- 172.21.15.116:0/2328522658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9824062880 0x7f9824062c60 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f98240a70a0 tx=0x7f9814006eb0).stop 2020-11-20T12:20:39.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.216+0000 7f9828c0f700 1 -- 172.21.15.116:0/2328522658 shutdown_connections 2020-11-20T12:20:39.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.216+0000 7f9828c0f700 1 --2- 172.21.15.116:0/2328522658 >> v2:172.21.15.173:3300/0 conn(0x7f98240bfe20 0x7f98241556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:20:39.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.216+0000 7f9828c0f700 1 --2- 172.21.15.116:0/2328522658 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9824062880 0x7f9824062c60 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.216+0000 7f9828c0f700 1 -- 172.21.15.116:0/2328522658 >> 172.21.15.116:0/2328522658 conn(0x7f9824060c40 msgr2=0x7f98240c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:39.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.216+0000 7f9828c0f700 1 -- 172.21.15.116:0/2328522658 shutdown_connections 2020-11-20T12:20:39.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.216+0000 7f9828c0f700 1 -- 172.21.15.116:0/2328522658 wait complete. 2020-11-20T12:20:39.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.216+0000 7f9828c0f700 1 Processor -- start 2020-11-20T12:20:39.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.216+0000 7f9828c0f700 1 -- start start 2020-11-20T12:20:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f9828c0f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9824062880 0x7f98241694d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f9828c0f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9824065d40 0x7f9824152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f9828c0f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98240bfe20 0x7f98240a9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f9828c0f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98241699c0 con 0x7f9824065d40 2020-11-20T12:20:39.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f9828c0f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98240a6460 con 0x7f9824062880 2020-11-20T12:20:39.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f9828c0f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98240a7cd0 con 0x7f98240bfe20 2020-11-20T12:20:39.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9824065d40 0x7f9824152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9824065d40 0x7f9824152920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49786/0 (socket says 172.21.15.116:49786) 2020-11-20T12:20:39.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981affd700 1 -- 172.21.15.116:0/4181768952 learned_addr learned my addr 172.21.15.116:0/4181768952 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:39.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f98237fe700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9824062880 0x7f98241694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f9823fff700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98240bfe20 0x7f98240a9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981affd700 1 -- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98240bfe20 msgr2=0x7f98240a9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981affd700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98240bfe20 0x7f98240a9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981affd700 1 -- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9824062880 msgr2=0x7f98241694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981affd700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9824062880 0x7f98241694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981affd700 1 -- 172.21.15.116:0/4181768952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98240a7f70 con 0x7f9824065d40 2020-11-20T12:20:39.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f98237fe700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9824062880 0x7f98241694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:39.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f9823fff700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98240bfe20 0x7f98240a9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:39.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981affd700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9824065d40 0x7f9824152920 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f980c0096b0 tx=0x7f980c006de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:39.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981bfff700 1 -- 172.21.15.116:0/4181768952 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c0113f0 con 0x7f9824065d40 2020-11-20T12:20:39.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981bfff700 1 -- 172.21.15.116:0/4181768952 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f980c011550 con 0x7f9824065d40 2020-11-20T12:20:39.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f9828c0f700 1 -- 172.21.15.116:0/4181768952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98240a8210 con 0x7f9824065d40 2020-11-20T12:20:39.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f981bfff700 1 -- 172.21.15.116:0/4181768952 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980c011910 con 0x7f9824065d40 2020-11-20T12:20:39.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f9828c0f700 1 -- 172.21.15.116:0/4181768952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98240a8750 con 0x7f9824065d40 2020-11-20T12:20:39.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.220+0000 7f9828c0f700 1 -- 172.21.15.116:0/4181768952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f982405cf30 con 0x7f9824065d40 2020-11-20T12:20:39.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.224+0000 7f981bfff700 1 -- 172.21.15.116:0/4181768952 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f980c011c80 con 0x7f9824065d40 2020-11-20T12:20:39.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.224+0000 7f981bfff700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98000557c0 0x7f9800057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.224+0000 7f981bfff700 1 -- 172.21.15.116:0/4181768952 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f980c06dc20 con 0x7f9824065d40 2020-11-20T12:20:39.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.224+0000 7f98237fe700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98000557c0 0x7f9800057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.228+0000 7f98237fe700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98000557c0 0x7f9800057c80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f9824155290 tx=0x7f9814015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:39.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.232+0000 7f981bfff700 1 -- 172.21.15.116:0/4181768952 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f980c043ac0 con 0x7f9824065d40 2020-11-20T12:20:39.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.388+0000 7f9828c0f700 1 -- 172.21.15.116:0/4181768952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98240be800 con 0x7f9824065d40 2020-11-20T12:20:39.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.392+0000 7f981bfff700 1 -- 172.21.15.116:0/4181768952 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f980c0470e0 con 0x7f9824065d40 2020-11-20T12:20:39.388 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:39.389 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":96,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:20:35.459061+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:20:32.886768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-20T12:20:33.446960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:39.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.392+0000 7f9828c0f700 1 -- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98000557c0 msgr2=0x7f9800057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.392+0000 7f9828c0f700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98000557c0 0x7f9800057c80 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f9824155290 tx=0x7f9814015000).stop 2020-11-20T12:20:39.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.392+0000 7f9828c0f700 1 -- 172.21.15.116:0/4181768952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9824065d40 msgr2=0x7f9824152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.392+0000 7f9828c0f700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9824065d40 0x7f9824152920 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f980c0096b0 tx=0x7f980c006de0).stop 2020-11-20T12:20:39.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.392+0000 7f9828c0f700 1 -- 172.21.15.116:0/4181768952 shutdown_connections 2020-11-20T12:20:39.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.392+0000 7f9828c0f700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98000557c0 0x7f9800057c80 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.392+0000 7f9828c0f700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98240bfe20 0x7f98240a9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.392+0000 7f9828c0f700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9824062880 0x7f98241694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.392+0000 7f9828c0f700 1 --2- 172.21.15.116:0/4181768952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9824065d40 0x7f9824152920 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.392+0000 7f9828c0f700 1 -- 172.21.15.116:0/4181768952 >> 172.21.15.116:0/4181768952 conn(0x7f9824060c40 msgr2=0x7f9824154820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.396+0000 7f9828c0f700 1 -- 172.21.15.116:0/4181768952 shutdown_connections 2020-11-20T12:20:39.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.396+0000 7f9828c0f700 1 -- 172.21.15.116:0/4181768952 wait complete. 2020-11-20T12:20:39.416 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:20:32.886768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:20:33.446960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:20:39.417 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:20:39.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.596+0000 7f2b00b83700 1 Processor -- start 2020-11-20T12:20:39.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.596+0000 7f2b00b83700 1 -- start start 2020-11-20T12:20:39.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.596+0000 7f2b00b83700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2afc060310 0x7f2afc0606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.596+0000 7f2b00b83700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2afc066480 0x7f2afc0bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.596+0000 7f2b00b83700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2afc0668e0 0x7f2afc0bfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.596+0000 7f2b00b83700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc066e00 con 0x7f2afc060310 2020-11-20T12:20:39.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.596+0000 7f2b00b83700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2afc0670a0 con 0x7f2afc066480 2020-11-20T12:20:39.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.596+0000 7f2b00b83700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc067340 con 0x7f2afc0668e0 2020-11-20T12:20:39.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afb7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2afc060310 0x7f2afc0606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afb7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2afc060310 0x7f2afc0606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49796/0 (socket says 172.21.15.116:49796) 2020-11-20T12:20:39.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afb7fe700 1 -- 172.21.15.116:0/3309025113 learned_addr learned my addr 172.21.15.116:0/3309025113 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:39.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afbfff700 1 --2- 172.21.15.116:0/3309025113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2afc0668e0 0x7f2afc0bfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afaffd700 1 --2- 172.21.15.116:0/3309025113 >> v2:172.21.15.173:3300/0 conn(0x7f2afc066480 0x7f2afc0bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afb7fe700 1 -- 172.21.15.116:0/3309025113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2afc0668e0 msgr2=0x7f2afc0bfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:39.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afb7fe700 1 --2- 172.21.15.116:0/3309025113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2afc0668e0 0x7f2afc0bfaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afb7fe700 1 -- 172.21.15.116:0/3309025113 >> v2:172.21.15.173:3300/0 conn(0x7f2afc066480 msgr2=0x7f2afc0bf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:39.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afb7fe700 1 --2- 172.21.15.116:0/3309025113 >> v2:172.21.15.173:3300/0 conn(0x7f2afc066480 0x7f2afc0bf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afb7fe700 1 -- 172.21.15.116:0/3309025113 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc0c0d30 con 0x7f2afc060310 2020-11-20T12:20:39.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afbfff700 1 --2- 172.21.15.116:0/3309025113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2afc0668e0 0x7f2afc0bfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:39.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afb7fe700 1 --2- 172.21.15.116:0/3309025113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2afc060310 0x7f2afc0606f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f2afc0bf3c0 tx=0x7f2aec006eb0).ready entity=mon.0 client_cookie=dc7dd9d022da1a6e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:39.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afa7fc700 1 -- 172.21.15.116:0/3309025113 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec009720 con 0x7f2afc060310 2020-11-20T12:20:39.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afa7fc700 1 -- 172.21.15.116:0/3309025113 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec009880 con 0x7f2afc060310 2020-11-20T12:20:39.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2afa7fc700 1 -- 172.21.15.116:0/3309025113 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec009c00 con 0x7f2afc060310 2020-11-20T12:20:39.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2b00b83700 1 -- 172.21.15.116:0/3309025113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2afc060310 msgr2=0x7f2afc0606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.600+0000 7f2b00b83700 1 --2- 172.21.15.116:0/3309025113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2afc060310 0x7f2afc0606f0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f2afc0bf3c0 tx=0x7f2aec006eb0).stop 2020-11-20T12:20:39.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.604+0000 7f2b00b83700 1 -- 172.21.15.116:0/3309025113 shutdown_connections 2020-11-20T12:20:39.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.604+0000 7f2b00b83700 1 --2- 172.21.15.116:0/3309025113 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2afc0668e0 0x7f2afc0bfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.604+0000 7f2b00b83700 1 --2- 172.21.15.116:0/3309025113 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2afc060310 0x7f2afc0606f0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.604+0000 7f2b00b83700 1 --2- 172.21.15.116:0/3309025113 >> v2:172.21.15.173:3300/0 conn(0x7f2afc066480 0x7f2afc0bf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.604+0000 7f2b00b83700 1 -- 172.21.15.116:0/3309025113 >> 172.21.15.116:0/3309025113 conn(0x7f2afc068350 msgr2=0x7f2afc0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:39.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.604+0000 7f2b00b83700 1 -- 172.21.15.116:0/3309025113 shutdown_connections 2020-11-20T12:20:39.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.604+0000 7f2b00b83700 1 -- 172.21.15.116:0/3309025113 wait complete. 2020-11-20T12:20:39.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.604+0000 7f2b00b83700 1 Processor -- start 2020-11-20T12:20:39.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.604+0000 7f2b00b83700 1 -- start start 2020-11-20T12:20:39.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2b00b83700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2afc060310 0x7f2afc0a9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2b00b83700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2afc066480 0x7f2afc146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2b00b83700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2afc0668e0 0x7f2afc147cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2b00b83700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc0a9b00 con 0x7f2afc066480 2020-11-20T12:20:39.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2b00b83700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc15f320 con 0x7f2afc060310 2020-11-20T12:20:39.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2b00b83700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2afc15f5c0 con 0x7f2afc0668e0 2020-11-20T12:20:39.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2afaffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2afc066480 0x7f2afc146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2afaffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2afc066480 0x7f2afc146e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49802/0 (socket says 172.21.15.116:49802) 2020-11-20T12:20:39.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2afaffd700 1 -- 172.21.15.116:0/795823038 learned_addr learned my addr 172.21.15.116:0/795823038 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:39.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2afb7fe700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2afc060310 0x7f2afc0a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2afbfff700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2afc0668e0 0x7f2afc147cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2afaffd700 1 -- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2afc0668e0 msgr2=0x7f2afc147cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2afaffd700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2afc0668e0 0x7f2afc147cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2afaffd700 1 -- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2afc060310 msgr2=0x7f2afc0a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2afaffd700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2afc060310 0x7f2afc0a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2afaffd700 1 -- 172.21.15.116:0/795823038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2afc15f860 con 0x7f2afc066480 2020-11-20T12:20:39.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2afaffd700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2afc066480 0x7f2afc146e60 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f2af000ebd0 tx=0x7f2af0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:39.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2adb7fe700 1 -- 172.21.15.116:0/795823038 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000b9a0 con 0x7f2afc066480 2020-11-20T12:20:39.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2adb7fe700 1 -- 172.21.15.116:0/795823038 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000bb00 con 0x7f2afc066480 2020-11-20T12:20:39.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2b00b83700 1 -- 172.21.15.116:0/795823038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2afc15fb00 con 0x7f2afc066480 2020-11-20T12:20:39.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2adb7fe700 1 -- 172.21.15.116:0/795823038 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000be80 con 0x7f2afc066480 2020-11-20T12:20:39.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.608+0000 7f2b00b83700 1 -- 172.21.15.116:0/795823038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2afc15efb0 con 0x7f2afc066480 2020-11-20T12:20:39.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.612+0000 7f2b00b83700 1 -- 172.21.15.116:0/795823038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2afc05cf30 con 0x7f2afc066480 2020-11-20T12:20:39.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.612+0000 7f2adb7fe700 1 -- 172.21.15.116:0/795823038 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2af0012600 con 0x7f2afc066480 2020-11-20T12:20:39.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.612+0000 7f2adb7fe700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2ad40557f0 0x7f2ad4057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.612+0000 7f2adb7fe700 1 -- 172.21.15.116:0/795823038 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2af006ea40 con 0x7f2afc066480 2020-11-20T12:20:39.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.612+0000 7f2afb7fe700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2ad40557f0 0x7f2ad4057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.616+0000 7f2afb7fe700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2ad40557f0 0x7f2ad4057cb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f2aec009420 tx=0x7f2aec015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:39.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.620+0000 7f2adb7fe700 1 -- 172.21.15.116:0/795823038 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2af0044880 con 0x7f2afc066480 2020-11-20T12:20:39.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.776+0000 7f2b00b83700 1 -- 172.21.15.116:0/795823038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2afc0639c0 con 0x7f2afc066480 2020-11-20T12:20:39.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.776+0000 7f2adb7fe700 1 -- 172.21.15.116:0/795823038 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v96) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2af0047ea0 con 0x7f2afc066480 2020-11-20T12:20:39.774 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:20:39.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 -- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2ad40557f0 msgr2=0x7f2ad4057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2ad40557f0 0x7f2ad4057cb0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f2aec009420 tx=0x7f2aec015000).stop 2020-11-20T12:20:39.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 -- 172.21.15.116:0/795823038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2afc066480 msgr2=0x7f2afc146e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2afc066480 0x7f2afc146e60 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f2af000ebd0 tx=0x7f2af0006df0).stop 2020-11-20T12:20:39.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 -- 172.21.15.116:0/795823038 shutdown_connections 2020-11-20T12:20:39.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2ad40557f0 0x7f2ad4057cb0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2afc0668e0 0x7f2afc147cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2afc060310 0x7f2afc0a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 --2- 172.21.15.116:0/795823038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2afc066480 0x7f2afc146e60 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 -- 172.21.15.116:0/795823038 >> 172.21.15.116:0/795823038 conn(0x7f2afc068350 msgr2=0x7f2afc0c3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 -- 172.21.15.116:0/795823038 shutdown_connections 2020-11-20T12:20:39.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.780+0000 7f2b00b83700 1 -- 172.21.15.116:0/795823038 wait complete. 2020-11-20T12:20:39.804 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:20:39.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.984+0000 7fd789355700 1 Processor -- start 2020-11-20T12:20:39.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.984+0000 7fd789355700 1 -- start start 2020-11-20T12:20:39.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.984+0000 7fd789355700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd7840a7e50 0x7fd7840a8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.984+0000 7fd789355700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7840a8f20 0x7fd784157b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.984+0000 7fd789355700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7840a5300 con 0x7fd7841485c0 2020-11-20T12:20:39.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.984+0000 7fd789355700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7840a9390 con 0x7fd7840a7e50 2020-11-20T12:20:39.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.984+0000 7fd789355700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7840a82b0 con 0x7fd7840a8f20 2020-11-20T12:20:39.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd77a7fc700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd7840a7e50 0x7fd7840a8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd77a7fc700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd7840a7e50 0x7fd7840a8a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:48862/0 (socket says 172.21.15.116:48862) 2020-11-20T12:20:39.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd77a7fc700 1 -- 172.21.15.116:0/3647009779 learned_addr learned my addr 172.21.15.116:0/3647009779 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:39.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd783fff700 1 --2- 172.21.15.116:0/3647009779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7840a8f20 0x7fd784157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd783fff700 1 -- 172.21.15.116:0/3647009779 >> v2:172.21.15.173:3300/0 conn(0x7fd7840a7e50 msgr2=0x7fd7840a8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:39.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd783fff700 1 --2- 172.21.15.116:0/3647009779 >> v2:172.21.15.173:3300/0 conn(0x7fd7840a7e50 0x7fd7840a8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd783fff700 1 -- 172.21.15.116:0/3647009779 >> v1:172.21.15.173:6789/0 conn(0x7fd7841485c0 legacy=0x7fd7840a4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd7827fc700 1 -- 172.21.15.116:0/3647009779 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2322815879 0 0) 0x7fd7840a5300 con 0x7fd7841485c0 2020-11-20T12:20:39.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd783fff700 1 -- 172.21.15.116:0/3647009779 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78414e6f0 con 0x7fd7840a8f20 2020-11-20T12:20:39.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd77a7fc700 1 --2- 172.21.15.116:0/3647009779 >> v2:172.21.15.173:3300/0 conn(0x7fd7840a7e50 0x7fd7840a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:39.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd783fff700 1 --2- 172.21.15.116:0/3647009779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7840a8f20 0x7fd784157b70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fd77c00b9a0 tx=0x7fd77c006df0).ready entity=mon.2 client_cookie=1ec10c2c2d08ab90 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:39.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd7827fc700 1 -- 172.21.15.116:0/3647009779 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c0097b0 con 0x7fd7840a8f20 2020-11-20T12:20:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd7827fc700 1 -- 172.21.15.116:0/3647009779 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77c009910 con 0x7fd7840a8f20 2020-11-20T12:20:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd7827fc700 1 -- 172.21.15.116:0/3647009779 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c009c90 con 0x7fd7840a8f20 2020-11-20T12:20:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd789355700 1 -- 172.21.15.116:0/3647009779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7840a8f20 msgr2=0x7fd784157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.988+0000 7fd789355700 1 --2- 172.21.15.116:0/3647009779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7840a8f20 0x7fd784157b70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fd77c00b9a0 tx=0x7fd77c006df0).stop 2020-11-20T12:20:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.992+0000 7fd789355700 1 -- 172.21.15.116:0/3647009779 shutdown_connections 2020-11-20T12:20:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.992+0000 7fd789355700 1 --2- 172.21.15.116:0/3647009779 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7840a8f20 0x7fd784157b70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.992+0000 7fd789355700 1 --2- 172.21.15.116:0/3647009779 >> v2:172.21.15.173:3300/0 conn(0x7fd7840a7e50 0x7fd7840a8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.992+0000 7fd789355700 1 -- 172.21.15.116:0/3647009779 >> 172.21.15.116:0/3647009779 conn(0x7fd7840c6480 msgr2=0x7fd7840c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:39.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.992+0000 7fd789355700 1 -- 172.21.15.116:0/3647009779 shutdown_connections 2020-11-20T12:20:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.992+0000 7fd789355700 1 -- 172.21.15.116:0/3647009779 wait complete. 2020-11-20T12:20:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd789355700 1 Processor -- start 2020-11-20T12:20:39.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd789355700 1 -- start start 2020-11-20T12:20:39.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd789355700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7840a7e50 0x7fd784160df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd789355700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7840a8f20 0x7fd784146f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd789355700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7841485c0 0x7fd78414be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd789355700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7841612e0 con 0x7fd7840a7e50 2020-11-20T12:20:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd789355700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78414ac30 con 0x7fd7840a8f20 2020-11-20T12:20:39.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd789355700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78414aed0 con 0x7fd7841485c0 2020-11-20T12:20:39.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd782ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7840a7e50 0x7fd784160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd77a7fc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7840a8f20 0x7fd784146f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd77a7fc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7840a8f20 0x7fd784146f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:48866/0 (socket says 172.21.15.116:48866) 2020-11-20T12:20:39.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd77a7fc700 1 -- 172.21.15.116:0/2241167934 learned_addr learned my addr 172.21.15.116:0/2241167934 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:39.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd783fff700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7841485c0 0x7fd78414be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:39.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd782ffd700 1 -- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7841485c0 msgr2=0x7fd78414be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd782ffd700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7841485c0 0x7fd78414be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd782ffd700 1 -- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7840a8f20 msgr2=0x7fd784146f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:39.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd782ffd700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7840a8f20 0x7fd784146f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:39.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd782ffd700 1 -- 172.21.15.116:0/2241167934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78414b170 con 0x7fd7840a7e50 2020-11-20T12:20:39.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:39.996+0000 7fd77a7fc700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7840a8f20 0x7fd784146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:39.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.000+0000 7fd782ffd700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7840a7e50 0x7fd784160df0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7fd77400cae0 tx=0x7fd774007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:39.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.000+0000 7fd77b7fe700 1 -- 172.21.15.116:0/2241167934 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774006870 con 0x7fd7840a7e50 2020-11-20T12:20:39.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.000+0000 7fd77b7fe700 1 -- 172.21.15.116:0/2241167934 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7740069d0 con 0x7fd7840a7e50 2020-11-20T12:20:39.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.000+0000 7fd789355700 1 -- 172.21.15.116:0/2241167934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78414b6c0 con 0x7fd7840a7e50 2020-11-20T12:20:39.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.000+0000 7fd77b7fe700 1 -- 172.21.15.116:0/2241167934 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774006d90 con 0x7fd7840a7e50 2020-11-20T12:20:39.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.000+0000 7fd789355700 1 -- 172.21.15.116:0/2241167934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd784147470 con 0x7fd7840a7e50 2020-11-20T12:20:40.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.000+0000 7fd789355700 1 -- 172.21.15.116:0/2241167934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78414b520 con 0x7fd7840a7e50 2020-11-20T12:20:40.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.000+0000 7fd77b7fe700 1 -- 172.21.15.116:0/2241167934 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd774010070 con 0x7fd7840a7e50 2020-11-20T12:20:40.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.000+0000 7fd77b7fe700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd7600557c0 0x7fd760057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.004+0000 7fd77b7fe700 1 -- 172.21.15.116:0/2241167934 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd77406dd50 con 0x7fd7840a7e50 2020-11-20T12:20:40.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.004+0000 7fd77a7fc700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd7600557c0 0x7fd760057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.004+0000 7fd77a7fc700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd7600557c0 0x7fd760057c80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fd76c00bfd0 tx=0x7fd76c006eb0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:40.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.008+0000 7fd77b7fe700 1 -- 172.21.15.116:0/2241167934 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd78414b520 con 0x7fd7840a7e50 2020-11-20T12:20:40.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.164+0000 7fd789355700 1 -- 172.21.15.116:0/2241167934 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd78414b520 con 0x7fd7840a7e50 2020-11-20T12:20:40.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.164+0000 7fd77b7fe700 1 -- 172.21.15.116:0/2241167934 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v96) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd78414b520 con 0x7fd7840a7e50 2020-11-20T12:20:40.161 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:40.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 -- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd7600557c0 msgr2=0x7fd760057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd7600557c0 0x7fd760057c80 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7fd76c00bfd0 tx=0x7fd76c006eb0).stop 2020-11-20T12:20:40.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 -- 172.21.15.116:0/2241167934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7840a7e50 msgr2=0x7fd784160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7840a7e50 0x7fd784160df0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7fd77400cae0 tx=0x7fd774007f10).stop 2020-11-20T12:20:40.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 -- 172.21.15.116:0/2241167934 shutdown_connections 2020-11-20T12:20:40.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd7600557c0 0x7fd760057c80 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd7841485c0 0x7fd78414be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd7840a7e50 0x7fd784160df0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 --2- 172.21.15.116:0/2241167934 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd7840a8f20 0x7fd784146f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 -- 172.21.15.116:0/2241167934 >> 172.21.15.116:0/2241167934 conn(0x7fd7840c6480 msgr2=0x7fd7840618c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:40.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 -- 172.21.15.116:0/2241167934 shutdown_connections 2020-11-20T12:20:40.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.168+0000 7fd789355700 1 -- 172.21.15.116:0/2241167934 wait complete. 2020-11-20T12:20:40.186 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:20:40.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.368+0000 7fe1a6b53700 1 Processor -- start 2020-11-20T12:20:40.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.372+0000 7fe1a6b53700 1 -- start start 2020-11-20T12:20:40.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.372+0000 7fe1a6b53700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe1a01518c0 0x7fe1a0151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.372+0000 7fe1a6b53700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1a00c9010 0x7fe1a0061550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.372+0000 7fe1a6b53700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe1a0061a40 0x7fe1a00ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.372+0000 7fe1a6b53700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a0061ea0 con 0x7fe1a0061a40 2020-11-20T12:20:40.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.372+0000 7fe1a6b53700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a00c9450 con 0x7fe1a01518c0 2020-11-20T12:20:40.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.372+0000 7fe1a6b53700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00a92f0 con 0x7fe1a00c9010 2020-11-20T12:20:40.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.372+0000 7fe1a48ef700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe1a01518c0 0x7fe1a0151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.372+0000 7fe1a50f0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe1a0061a40 0x7fe1a00ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.372+0000 7fe1a48ef700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe1a01518c0 0x7fe1a0151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:48874/0 (socket says 172.21.15.116:48874) 2020-11-20T12:20:40.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.372+0000 7fe1a48ef700 1 -- 172.21.15.116:0/2372792433 learned_addr learned my addr 172.21.15.116:0/2372792433 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:40.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe19ffff700 1 --2- 172.21.15.116:0/2372792433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1a00c9010 0x7fe1a0061550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a50f0700 1 -- 172.21.15.116:0/2372792433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1a00c9010 msgr2=0x7fe1a0061550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:40.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a50f0700 1 --2- 172.21.15.116:0/2372792433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1a00c9010 0x7fe1a0061550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a50f0700 1 -- 172.21.15.116:0/2372792433 >> v2:172.21.15.173:3300/0 conn(0x7fe1a01518c0 msgr2=0x7fe1a0151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:40.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a50f0700 1 --2- 172.21.15.116:0/2372792433 >> v2:172.21.15.173:3300/0 conn(0x7fe1a01518c0 0x7fe1a0151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a50f0700 1 -- 172.21.15.116:0/2372792433 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00a9590 con 0x7fe1a0061a40 2020-11-20T12:20:40.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a50f0700 1 --2- 172.21.15.116:0/2372792433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1a0061a40 0x7fe1a00ad010 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7fe1a0060170 tx=0x7fe1980091d0).ready entity=mon.0 client_cookie=daabb32d125542cc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:40.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe19f7fe700 1 -- 172.21.15.116:0/2372792433 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19800ba30 con 0x7fe1a0061a40 2020-11-20T12:20:40.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe19f7fe700 1 -- 172.21.15.116:0/2372792433 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19800bb90 con 0x7fe1a0061a40 2020-11-20T12:20:40.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe19f7fe700 1 -- 172.21.15.116:0/2372792433 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198011400 con 0x7fe1a0061a40 2020-11-20T12:20:40.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a6b53700 1 -- 172.21.15.116:0/2372792433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1a0061a40 msgr2=0x7fe1a00ad010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a6b53700 1 --2- 172.21.15.116:0/2372792433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1a0061a40 0x7fe1a00ad010 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7fe1a0060170 tx=0x7fe1980091d0).stop 2020-11-20T12:20:40.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a6b53700 1 -- 172.21.15.116:0/2372792433 shutdown_connections 2020-11-20T12:20:40.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a6b53700 1 --2- 172.21.15.116:0/2372792433 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1a0061a40 0x7fe1a00ad010 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a6b53700 1 --2- 172.21.15.116:0/2372792433 >> v2:172.21.15.173:3300/0 conn(0x7fe1a01518c0 0x7fe1a0151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a6b53700 1 --2- 172.21.15.116:0/2372792433 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1a00c9010 0x7fe1a0061550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a6b53700 1 -- 172.21.15.116:0/2372792433 >> 172.21.15.116:0/2372792433 conn(0x7fe1a0068370 msgr2=0x7fe1a00c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:40.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a6b53700 1 -- 172.21.15.116:0/2372792433 shutdown_connections 2020-11-20T12:20:40.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.376+0000 7fe1a6b53700 1 -- 172.21.15.116:0/2372792433 wait complete. 2020-11-20T12:20:40.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.380+0000 7fe1a6b53700 1 Processor -- start 2020-11-20T12:20:40.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.380+0000 7fe1a6b53700 1 -- start start 2020-11-20T12:20:40.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.380+0000 7fe1a6b53700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1a0061a40 0x7fe1a0165f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.380+0000 7fe1a6b53700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1a00c9010 0x7fe1a0064bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.380+0000 7fe1a6b53700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1a01518c0 0x7fe1a00c3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.380+0000 7fe1a6b53700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a0065b70 con 0x7fe1a0061a40 2020-11-20T12:20:40.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.380+0000 7fe1a6b53700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00650a0 con 0x7fe1a00c9010 2020-11-20T12:20:40.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.380+0000 7fe1a6b53700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a00c3bf0 con 0x7fe1a01518c0 2020-11-20T12:20:40.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a48ef700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1a0061a40 0x7fe1a0165f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a48ef700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1a0061a40 0x7fe1a0165f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49828/0 (socket says 172.21.15.116:49828) 2020-11-20T12:20:40.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a48ef700 1 -- 172.21.15.116:0/655562034 learned_addr learned my addr 172.21.15.116:0/655562034 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:40.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a50f0700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1a01518c0 0x7fe1a00c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe19ffff700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1a00c9010 0x7fe1a0064bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a48ef700 1 -- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1a01518c0 msgr2=0x7fe1a00c3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a48ef700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1a01518c0 0x7fe1a00c3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a48ef700 1 -- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1a00c9010 msgr2=0x7fe1a0064bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a48ef700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1a00c9010 0x7fe1a0064bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a48ef700 1 -- 172.21.15.116:0/655562034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00c3e90 con 0x7fe1a0061a40 2020-11-20T12:20:40.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a48ef700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1a0061a40 0x7fe1a0165f50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fe19400ebd0 tx=0x7fe194008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:40.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe19cff9700 1 -- 172.21.15.116:0/655562034 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19400c820 con 0x7fe1a0061a40 2020-11-20T12:20:40.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe19cff9700 1 -- 172.21.15.116:0/655562034 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe19400c980 con 0x7fe1a0061a40 2020-11-20T12:20:40.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a6b53700 1 -- 172.21.15.116:0/655562034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a00c4130 con 0x7fe1a0061a40 2020-11-20T12:20:40.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe19cff9700 1 -- 172.21.15.116:0/655562034 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe19400cd00 con 0x7fe1a0061a40 2020-11-20T12:20:40.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a6b53700 1 -- 172.21.15.116:0/655562034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a00c3810 con 0x7fe1a0061a40 2020-11-20T12:20:40.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.384+0000 7fe1a6b53700 1 -- 172.21.15.116:0/655562034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1a0069950 con 0x7fe1a0061a40 2020-11-20T12:20:40.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.388+0000 7fe19cff9700 1 -- 172.21.15.116:0/655562034 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe194022070 con 0x7fe1a0061a40 2020-11-20T12:20:40.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.388+0000 7fe19cff9700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1780557f0 0x7fe178057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.388+0000 7fe19cff9700 1 -- 172.21.15.116:0/655562034 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe19406e990 con 0x7fe1a0061a40 2020-11-20T12:20:40.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.388+0000 7fe19ffff700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1780557f0 0x7fe178057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.392+0000 7fe19ffff700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1780557f0 0x7fe178057cb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fe1a00aaa10 tx=0x7fe18c006f90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:40.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.396+0000 7fe19cff9700 1 -- 172.21.15.116:0/655562034 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe194044830 con 0x7fe1a0061a40 2020-11-20T12:20:40.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.556+0000 7fe1a6b53700 1 -- 172.21.15.116:0/655562034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1a00608c0 con 0x7fe1a0061a40 2020-11-20T12:20:40.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe19cff9700 1 -- 172.21.15.116:0/655562034 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe194016070 con 0x7fe1a0061a40 2020-11-20T12:20:40.556 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:40.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 -- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1780557f0 msgr2=0x7fe178057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1780557f0 0x7fe178057cb0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fe1a00aaa10 tx=0x7fe18c006f90).stop 2020-11-20T12:20:40.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 -- 172.21.15.116:0/655562034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1a0061a40 msgr2=0x7fe1a0165f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1a0061a40 0x7fe1a0165f50 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7fe19400ebd0 tx=0x7fe194008040).stop 2020-11-20T12:20:40.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 -- 172.21.15.116:0/655562034 shutdown_connections 2020-11-20T12:20:40.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1780557f0 0x7fe178057cb0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1a01518c0 0x7fe1a00c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1a0061a40 0x7fe1a0165f50 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 --2- 172.21.15.116:0/655562034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1a00c9010 0x7fe1a0064bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 -- 172.21.15.116:0/655562034 >> 172.21.15.116:0/655562034 conn(0x7fe1a0068370 msgr2=0x7fe1a00a8b30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:40.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 -- 172.21.15.116:0/655562034 shutdown_connections 2020-11-20T12:20:40.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.560+0000 7fe1a6b53700 1 -- 172.21.15.116:0/655562034 wait complete. 2020-11-20T12:20:40.581 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:20:40.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf359e700 1 Processor -- start 2020-11-20T12:20:40.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf359e700 1 -- start start 2020-11-20T12:20:40.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf359e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6cf414f880 0x7f6cf40b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf359e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6cf40a81f0 0x7f6cf4060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf4060660 0x7f6cf40a8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf359e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf40a8930 con 0x7f6cf40a81f0 2020-11-20T12:20:40.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf359e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cf40ba2a0 con 0x7f6cf414f880 2020-11-20T12:20:40.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf40a8690 con 0x7f6cf4060660 2020-11-20T12:20:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf1d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6cf40a81f0 0x7f6cf4060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf1d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6cf40a81f0 0x7f6cf4060170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49838/0 (socket says 172.21.15.116:49838) 2020-11-20T12:20:40.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf1d9b700 1 -- 172.21.15.116:0/2695050083 learned_addr learned my addr 172.21.15.116:0/2695050083 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf259c700 1 --2- 172.21.15.116:0/2695050083 >> v2:172.21.15.173:3300/0 conn(0x7f6cf414f880 0x7f6cf40b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.768+0000 7f6cf2d9d700 1 --2- 172.21.15.116:0/2695050083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf4060660 0x7f6cf40a8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf1d9b700 1 -- 172.21.15.116:0/2695050083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf4060660 msgr2=0x7f6cf40a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:40.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf1d9b700 1 --2- 172.21.15.116:0/2695050083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf4060660 0x7f6cf40a8ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf1d9b700 1 -- 172.21.15.116:0/2695050083 >> v2:172.21.15.173:3300/0 conn(0x7f6cf414f880 msgr2=0x7f6cf40b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:40.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf1d9b700 1 --2- 172.21.15.116:0/2695050083 >> v2:172.21.15.173:3300/0 conn(0x7f6cf414f880 0x7f6cf40b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf1d9b700 1 -- 172.21.15.116:0/2695050083 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf40b94d0 con 0x7f6cf40a81f0 2020-11-20T12:20:40.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf1d9b700 1 --2- 172.21.15.116:0/2695050083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf40a81f0 0x7f6cf4060170 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f6cf4065ab0 tx=0x7f6ce8006eb0).ready entity=mon.0 client_cookie=33aba52fc5299f5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:40.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf159a700 1 -- 172.21.15.116:0/2695050083 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8009810 con 0x7f6cf40a81f0 2020-11-20T12:20:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf159a700 1 -- 172.21.15.116:0/2695050083 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce8009970 con 0x7f6cf40a81f0 2020-11-20T12:20:40.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf159a700 1 -- 172.21.15.116:0/2695050083 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce8009cf0 con 0x7f6cf40a81f0 2020-11-20T12:20:40.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf359e700 1 -- 172.21.15.116:0/2695050083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf40a81f0 msgr2=0x7f6cf4060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf359e700 1 --2- 172.21.15.116:0/2695050083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf40a81f0 0x7f6cf4060170 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f6cf4065ab0 tx=0x7f6ce8006eb0).stop 2020-11-20T12:20:40.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf359e700 1 -- 172.21.15.116:0/2695050083 shutdown_connections 2020-11-20T12:20:40.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf359e700 1 --2- 172.21.15.116:0/2695050083 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf4060660 0x7f6cf40a8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf359e700 1 --2- 172.21.15.116:0/2695050083 >> v2:172.21.15.173:3300/0 conn(0x7f6cf414f880 0x7f6cf40b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf359e700 1 --2- 172.21.15.116:0/2695050083 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf40a81f0 0x7f6cf4060170 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf359e700 1 -- 172.21.15.116:0/2695050083 >> 172.21.15.116:0/2695050083 conn(0x7f6cf4068350 msgr2=0x7f6cf40627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf359e700 1 -- 172.21.15.116:0/2695050083 shutdown_connections 2020-11-20T12:20:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.772+0000 7f6cf359e700 1 -- 172.21.15.116:0/2695050083 wait complete. 2020-11-20T12:20:40.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.776+0000 7f6cf359e700 1 Processor -- start 2020-11-20T12:20:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.776+0000 7f6cf359e700 1 -- start start 2020-11-20T12:20:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.776+0000 7f6cf359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf4060660 0x7f6cf414dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.776+0000 7f6cf359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf40a81f0 0x7f6cf409fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.776+0000 7f6cf359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf414f880 0x7f6cf4151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.776+0000 7f6cf359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf40a0750 con 0x7f6cf414f880 2020-11-20T12:20:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.776+0000 7f6cf359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf40a09f0 con 0x7f6cf4060660 2020-11-20T12:20:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.776+0000 7f6cf359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cf40a0c90 con 0x7f6cf40a81f0 2020-11-20T12:20:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf2d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf414f880 0x7f6cf4151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf2d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf414f880 0x7f6cf4151e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49846/0 (socket says 172.21.15.116:49846) 2020-11-20T12:20:40.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf2d9d700 1 -- 172.21.15.116:0/1639247528 learned_addr learned my addr 172.21.15.116:0/1639247528 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf259c700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf4060660 0x7f6cf414dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf1d9b700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf40a81f0 0x7f6cf409fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf2d9d700 1 -- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf40a81f0 msgr2=0x7f6cf409fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf2d9d700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf40a81f0 0x7f6cf409fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf2d9d700 1 -- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf4060660 msgr2=0x7f6cf414dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf2d9d700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf4060660 0x7f6cf414dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf2d9d700 1 -- 172.21.15.116:0/1639247528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf40a0f30 con 0x7f6cf414f880 2020-11-20T12:20:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf2d9d700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf414f880 0x7f6cf4151e70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f6cec00d7c0 tx=0x7f6cec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6ce27fc700 1 -- 172.21.15.116:0/1639247528 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec00f5e0 con 0x7f6cf414f880 2020-11-20T12:20:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6ce27fc700 1 -- 172.21.15.116:0/1639247528 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cec00f740 con 0x7f6cf414f880 2020-11-20T12:20:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf359e700 1 -- 172.21.15.116:0/1639247528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cf40c3350 con 0x7f6cf414f880 2020-11-20T12:20:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6ce27fc700 1 -- 172.21.15.116:0/1639247528 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cec00fae0 con 0x7f6cf414f880 2020-11-20T12:20:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.780+0000 7f6cf359e700 1 -- 172.21.15.116:0/1639247528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cf40a0290 con 0x7f6cf414f880 2020-11-20T12:20:40.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.784+0000 7f6cf359e700 1 -- 172.21.15.116:0/1639247528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cf40c34e0 con 0x7f6cf414f880 2020-11-20T12:20:40.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.784+0000 7f6ce27fc700 1 -- 172.21.15.116:0/1639247528 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6cec01c460 con 0x7f6cf414f880 2020-11-20T12:20:40.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.784+0000 7f6ce27fc700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cc80557c0 0x7f6cc8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:40.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.784+0000 7f6ce27fc700 1 -- 172.21.15.116:0/1639247528 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6cec06e8c0 con 0x7f6cf414f880 2020-11-20T12:20:40.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.784+0000 7f6cf259c700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cc80557c0 0x7f6cc8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:40.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.788+0000 7f6cf259c700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cc80557c0 0x7f6cc8057c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f6ce4009d60 tx=0x7f6ce40091d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:40.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.788+0000 7f6ce27fc700 1 -- 172.21.15.116:0/1639247528 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6cec0445f0 con 0x7f6cf414f880 2020-11-20T12:20:40.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.948+0000 7f6cf359e700 1 -- 172.21.15.116:0/1639247528 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6cf405cf30 con 0x7f6cf414f880 2020-11-20T12:20:40.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6ce27fc700 1 -- 172.21.15.116:0/1639247528 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6cec047c10 con 0x7f6cf414f880 2020-11-20T12:20:40.947 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:40.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 -- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cc80557c0 msgr2=0x7f6cc8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cc80557c0 0x7f6cc8057c80 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f6ce4009d60 tx=0x7f6ce40091d0).stop 2020-11-20T12:20:40.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 -- 172.21.15.116:0/1639247528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf414f880 msgr2=0x7f6cf4151e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:40.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf414f880 0x7f6cf4151e70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f6cec00d7c0 tx=0x7f6cec008040).stop 2020-11-20T12:20:40.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 -- 172.21.15.116:0/1639247528 shutdown_connections 2020-11-20T12:20:40.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6cc80557c0 0x7f6cc8057c80 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6cf414f880 0x7f6cf4151e70 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6cf4060660 0x7f6cf414dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 --2- 172.21.15.116:0/1639247528 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6cf40a81f0 0x7f6cf409fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:40.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 -- 172.21.15.116:0/1639247528 >> 172.21.15.116:0/1639247528 conn(0x7f6cf4068350 msgr2=0x7f6cf4154390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:40.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 -- 172.21.15.116:0/1639247528 shutdown_connections 2020-11-20T12:20:40.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:40.952+0000 7f6cf359e700 1 -- 172.21.15.116:0/1639247528 wait complete. 2020-11-20T12:20:40.972 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:20:40.973 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:41.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.152+0000 7f0bac26a700 1 Processor -- start 2020-11-20T12:20:41.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.152+0000 7f0bac26a700 1 -- start start 2020-11-20T12:20:41.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0bac26a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ba40ba890 0x7f0ba40bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:41.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0bac26a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0ba4148a20 0x7f0ba4148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:41.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0bac26a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba4066140 con 0x7f0ba40c2230 2020-11-20T12:20:41.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0bac26a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba40c1d80 con 0x7f0ba4148a20 2020-11-20T12:20:41.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0bac26a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba41526c0 con 0x7f0ba40ba890 2020-11-20T12:20:41.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0ba9805700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f0ba40c2230 0x7f0ba40c1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59286/0 (socket says 172.21.15.116:59286) 2020-11-20T12:20:41.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0ba9805700 1 -- 172.21.15.116:0/373072145 learned_addr learned my addr 172.21.15.116:0/373072145 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:41.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0baa807700 1 --2- 172.21.15.116:0/373072145 >> v2:172.21.15.116:3300/0 conn(0x7f0ba4148a20 0x7f0ba4148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:41.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0baa006700 1 --2- 172.21.15.116:0/373072145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ba40ba890 0x7f0ba40bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:41.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0ba9004700 1 -- 172.21.15.116:0/373072145 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145708137 0 0) 0x7f0ba4066140 con 0x7f0ba40c2230 2020-11-20T12:20:41.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0ba9004700 1 -- 172.21.15.116:0/373072145 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b8c003240 con 0x7f0ba40c2230 2020-11-20T12:20:41.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0baa807700 1 -- 172.21.15.116:0/373072145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ba40ba890 msgr2=0x7f0ba40bac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:41.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0baa807700 1 --2- 172.21.15.116:0/373072145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ba40ba890 0x7f0ba40bac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0baa807700 1 -- 172.21.15.116:0/373072145 >> v1:172.21.15.116:6789/0 conn(0x7f0ba40c2230 legacy=0x7f0ba40c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:41.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0baa807700 1 -- 172.21.15.116:0/373072145 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba4152960 con 0x7f0ba4148a20 2020-11-20T12:20:41.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0baa807700 1 --2- 172.21.15.116:0/373072145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ba4148a20 0x7f0ba4148ed0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7f0b9c009c90 tx=0x7f0b9c0091d0).ready entity=mon.0 client_cookie=d43c28c4a243c07d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:41.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0ba9004700 1 -- 172.21.15.116:0/373072145 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c00b9c0 con 0x7f0ba4148a20 2020-11-20T12:20:41.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0ba9004700 1 -- 172.21.15.116:0/373072145 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9c00bb20 con 0x7f0ba4148a20 2020-11-20T12:20:41.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0ba9004700 1 -- 172.21.15.116:0/373072145 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9c00bea0 con 0x7f0ba4148a20 2020-11-20T12:20:41.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0bac26a700 1 -- 172.21.15.116:0/373072145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ba4148a20 msgr2=0x7f0ba4148ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:41.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.156+0000 7f0bac26a700 1 --2- 172.21.15.116:0/373072145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ba4148a20 0x7f0ba4148ed0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7f0b9c009c90 tx=0x7f0b9c0091d0).stop 2020-11-20T12:20:41.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.160+0000 7f0bac26a700 1 -- 172.21.15.116:0/373072145 shutdown_connections 2020-11-20T12:20:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.160+0000 7f0bac26a700 1 --2- 172.21.15.116:0/373072145 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ba4148a20 0x7f0ba4148ed0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.160+0000 7f0bac26a700 1 --2- 172.21.15.116:0/373072145 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ba40ba890 0x7f0ba40bac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.160+0000 7f0bac26a700 1 -- 172.21.15.116:0/373072145 >> 172.21.15.116:0/373072145 conn(0x7f0ba40c4f80 msgr2=0x7f0ba40c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.160+0000 7f0bac26a700 1 -- 172.21.15.116:0/373072145 shutdown_connections 2020-11-20T12:20:41.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.160+0000 7f0bac26a700 1 -- 172.21.15.116:0/373072145 wait complete. 2020-11-20T12:20:41.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.160+0000 7f0bac26a700 1 Processor -- start 2020-11-20T12:20:41.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.160+0000 7f0bac26a700 1 -- start start 2020-11-20T12:20:41.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0bac26a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ba40ba890 0x7f0ba4164840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:41.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0bac26a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ba40c2230 0x7f0ba4143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:41.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0bac26a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ba4148a20 0x7f0ba4143730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:41.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0bac26a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba4151490 con 0x7f0ba40ba890 2020-11-20T12:20:41.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0bac26a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40a76a0 con 0x7f0ba4148a20 2020-11-20T12:20:41.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0bac26a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba40a7940 con 0x7f0ba40c2230 2020-11-20T12:20:41.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0baa006700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ba40ba890 0x7f0ba4164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:41.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0baa006700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ba40ba890 0x7f0ba4164840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49856/0 (socket says 172.21.15.116:49856) 2020-11-20T12:20:41.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0baa006700 1 -- 172.21.15.116:0/1125859654 learned_addr learned my addr 172.21.15.116:0/1125859654 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:41.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0baa807700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ba4148a20 0x7f0ba4143730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:41.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0ba9805700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ba40c2230 0x7f0ba4143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:41.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0baa006700 1 -- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ba40c2230 msgr2=0x7f0ba4143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:41.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0baa006700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ba40c2230 0x7f0ba4143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0baa006700 1 -- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ba4148a20 msgr2=0x7f0ba4143730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:41.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0baa006700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ba4148a20 0x7f0ba4143730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0baa006700 1 -- 172.21.15.116:0/1125859654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba40a7be0 con 0x7f0ba40ba890 2020-11-20T12:20:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0baa006700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ba40ba890 0x7f0ba4164840 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7f0ba000ebd0 tx=0x7f0ba0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0b927fc700 1 -- 172.21.15.116:0/1125859654 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba000b8d0 con 0x7f0ba40ba890 2020-11-20T12:20:41.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0b927fc700 1 -- 172.21.15.116:0/1125859654 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba000ba30 con 0x7f0ba40ba890 2020-11-20T12:20:41.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0bac26a700 1 -- 172.21.15.116:0/1125859654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba4164d30 con 0x7f0ba40ba890 2020-11-20T12:20:41.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0b927fc700 1 -- 172.21.15.116:0/1125859654 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba000bdb0 con 0x7f0ba40ba890 2020-11-20T12:20:41.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.164+0000 7f0bac26a700 1 -- 172.21.15.116:0/1125859654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba41509a0 con 0x7f0ba40ba890 2020-11-20T12:20:41.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.168+0000 7f0bac26a700 1 -- 172.21.15.116:0/1125859654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba40a7d70 con 0x7f0ba40ba890 2020-11-20T12:20:41.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.168+0000 7f0b927fc700 1 -- 172.21.15.116:0/1125859654 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0ba0012600 con 0x7f0ba40ba890 2020-11-20T12:20:41.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.168+0000 7f0b927fc700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0b7c0557c0 0x7f0b7c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:41.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.168+0000 7f0b927fc700 1 -- 172.21.15.116:0/1125859654 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0ba006eab0 con 0x7f0ba40ba890 2020-11-20T12:20:41.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.168+0000 7f0ba9805700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0b7c0557c0 0x7f0b7c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:41.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.168+0000 7f0ba9805700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0b7c0557c0 0x7f0b7c057c80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f0ba4146330 tx=0x7f0b94009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:41.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.172+0000 7f0b927fc700 1 -- 172.21.15.116:0/1125859654 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0ba00447e0 con 0x7f0ba40ba890 2020-11-20T12:20:41.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.388+0000 7f0bac26a700 1 -- 172.21.15.116:0/1125859654 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ba405d2a0 con 0x7f0ba40ba890 2020-11-20T12:20:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.388+0000 7f0b927fc700 1 -- 172.21.15.116:0/1125859654 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2848 (secure 0 0 0) 0x7f0ba0047e00 con 0x7f0ba40ba890 2020-11-20T12:20:41.385 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:41.385 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3105911814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3105911814},{"type":"v1","addr":"172.21.15.173:6837","nonce":3105911814}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/3457547857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3457547857},{"type":"v1","addr":"172.21.15.116:6835","nonce":3457547857}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/81195738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":81195738},{"type":"v1","addr":"172.21.15.116:6833","nonce":81195738}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:20:34.449524+0000","modified":"2020-11-20T12:20:36.475264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/2361610151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2361610151},{"type":"v1","addr":"172.21.15.173:6835","nonce":2361610151}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-20T12:20:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.388+0000 7f0bac26a700 1 -- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0b7c0557c0 msgr2=0x7f0b7c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.388+0000 7f0bac26a700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0b7c0557c0 0x7f0b7c057c80 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f0ba4146330 tx=0x7f0b94009040).stop 2020-11-20T12:20:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.388+0000 7f0bac26a700 1 -- 172.21.15.116:0/1125859654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ba40ba890 msgr2=0x7f0ba4164840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.388+0000 7f0bac26a700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ba40ba890 0x7f0ba4164840 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7f0ba000ebd0 tx=0x7f0ba0006df0).stop 2020-11-20T12:20:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.392+0000 7f0bac26a700 1 -- 172.21.15.116:0/1125859654 shutdown_connections 2020-11-20T12:20:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.392+0000 7f0bac26a700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0b7c0557c0 0x7f0b7c057c80 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.392+0000 7f0bac26a700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0ba4148a20 0x7f0ba4143730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.392+0000 7f0bac26a700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0ba40ba890 0x7f0ba4164840 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.392+0000 7f0bac26a700 1 --2- 172.21.15.116:0/1125859654 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0ba40c2230 0x7f0ba4143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:41.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.392+0000 7f0bac26a700 1 -- 172.21.15.116:0/1125859654 >> 172.21.15.116:0/1125859654 conn(0x7f0ba40c4f80 msgr2=0x7f0ba40c2990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:41.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.392+0000 7f0bac26a700 1 -- 172.21.15.116:0/1125859654 shutdown_connections 2020-11-20T12:20:41.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:41.392+0000 7f0bac26a700 1 -- 172.21.15.116:0/1125859654 wait complete. 2020-11-20T12:20:41.390 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 54 2020-11-20T12:20:41.814 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:20:34.449524+0000', 'modified': '2020-11-20T12:20:36.475264+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7311}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6835/2361610151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2361610151}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2361610151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:20:41.815 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:20:41.816 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-20T12:20:42.010 INFO:teuthology.orchestra.run.smithi173.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":7311,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.5437663170000002,"mdsmap_epoch":54,"osdmap_epoch":96,"osdmap_epoch_barrier":96,"uptime":13.063374925} 2020-11-20T12:20:42.011 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 7311, "mdsmap_epoch": 54, "osdmap_epoch": 96, "osdmap_epoch_barrier": 96, "rank_uptime": 7.543766317, "state": "up:active", "uptime": 13.063374925, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:20:42.011 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:20:42.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:42.012 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-20T12:20:42.028 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:20:42.028 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:42.093 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:42.094 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:20:42.118 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:42.118 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:20:42.125 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:20:42.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:42.126 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-20T12:20:42.158 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:20:42.159 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:20:42.159 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-20T12:20:42.159 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-20T12:20:42.160 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:20:42.160 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:20:42.161 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:20:42.161 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:20:42.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:42.162 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:20:42.178 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:20:42.178 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:20:42.179 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:20:42.179 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:42.179 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:20:42.180 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:42.180 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:42.180 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:42.180 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:42.181 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:42.181 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:42.181 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:42.182 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:20:42.182 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:42.182 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:20:42.182 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:42.183 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:20:42.183 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:42.183 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:20:42.184 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:42.184 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:20:42.184 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:42.184 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:20:42.185 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:42.185 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:20:42.185 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:42.185 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:20:42.186 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:42.186 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:42.186 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:20:42.186 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:20:42.187 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:42.187 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:42.187 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:20:42.188 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:20:42.188 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:42.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:42.189 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:20:42.221 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:20:42.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:42.225 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:20:42.243 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:20:42.243 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:20:42.244 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:20:42.244 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:20:42.244 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:20:42.245 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:20:42.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:42.246 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:20:42.247 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:20:42.247 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:20:42.248 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:20:42.249 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:20:42.249 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:20:42.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:42.384 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:20:42.400 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:20:42.400 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:20:42.401 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:20:42.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:42.402 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2020-11-20T12:20:42.431 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:42.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.612+0000 7f8dd9658700 1 Processor -- start 2020-11-20T12:20:42.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.612+0000 7f8dd9658700 1 -- start start 2020-11-20T12:20:42.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.612+0000 7f8dd9658700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8dd414c0a0 0x7f8dd4062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:42.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.612+0000 7f8dd9658700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dd40b94e0 0x7f8dd40b9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.612+0000 7f8dd9658700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd40ba1b0 con 0x7f8dd4062880 2020-11-20T12:20:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.612+0000 7f8dd9658700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd4152e60 con 0x7f8dd414c0a0 2020-11-20T12:20:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.612+0000 7f8dd9658700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd41555a0 con 0x7f8dd40b94e0 2020-11-20T12:20:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.612+0000 7f8dd37fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8dd414c0a0 0x7f8dd4062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:42.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd37fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8dd414c0a0 0x7f8dd4062c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49870/0 (socket says 172.21.15.116:49870) 2020-11-20T12:20:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd37fe700 1 -- 172.21.15.116:0/4265167940 learned_addr learned my addr 172.21.15.116:0/4265167940 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd8e57700 1 --2- 172.21.15.116:0/4265167940 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dd40b94e0 0x7f8dd40b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd37fe700 1 -- 172.21.15.116:0/4265167940 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dd40b94e0 msgr2=0x7f8dd40b9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:42.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd37fe700 1 --2- 172.21.15.116:0/4265167940 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dd40b94e0 0x7f8dd40b9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd37fe700 1 -- 172.21.15.116:0/4265167940 >> v1:172.21.15.173:6789/0 conn(0x7f8dd4062880 legacy=0x7f8dd414b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:42.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd37fe700 1 -- 172.21.15.116:0/4265167940 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd4155840 con 0x7f8dd414c0a0 2020-11-20T12:20:42.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd37fe700 1 --2- 172.21.15.116:0/4265167940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dd414c0a0 0x7f8dd4062c60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f8dd4152fa0 tx=0x7f8dc8006eb0).ready entity=mon.0 client_cookie=29153e16beea2371 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:42.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd2ffd700 1 -- 172.21.15.116:0/4265167940 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80098d0 con 0x7f8dd414c0a0 2020-11-20T12:20:42.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd2ffd700 1 -- 172.21.15.116:0/4265167940 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc8009a30 con 0x7f8dd414c0a0 2020-11-20T12:20:42.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd2ffd700 1 -- 172.21.15.116:0/4265167940 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8009db0 con 0x7f8dd414c0a0 2020-11-20T12:20:42.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd9658700 1 -- 172.21.15.116:0/4265167940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dd414c0a0 msgr2=0x7f8dd4062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:42.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd9658700 1 --2- 172.21.15.116:0/4265167940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dd414c0a0 0x7f8dd4062c60 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f8dd4152fa0 tx=0x7f8dc8006eb0).stop 2020-11-20T12:20:42.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd9658700 1 -- 172.21.15.116:0/4265167940 shutdown_connections 2020-11-20T12:20:42.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd9658700 1 --2- 172.21.15.116:0/4265167940 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dd40b94e0 0x7f8dd40b9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd9658700 1 --2- 172.21.15.116:0/4265167940 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dd414c0a0 0x7f8dd4062c60 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd9658700 1 -- 172.21.15.116:0/4265167940 >> 172.21.15.116:0/4265167940 conn(0x7f8dd4060c40 msgr2=0x7f8dd40c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:42.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd9658700 1 -- 172.21.15.116:0/4265167940 shutdown_connections 2020-11-20T12:20:42.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.616+0000 7f8dd9658700 1 -- 172.21.15.116:0/4265167940 wait complete. 2020-11-20T12:20:42.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd9658700 1 Processor -- start 2020-11-20T12:20:42.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd9658700 1 -- start start 2020-11-20T12:20:42.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd9658700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dd4062880 0x7f8dd40a2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:42.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd9658700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dd40b94e0 0x7f8dd40a4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd9658700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dd41649b0 0x7f8dd4137330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd9658700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40a4570 con 0x7f8dd4062880 2020-11-20T12:20:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd9658700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40a2890 con 0x7f8dd40b94e0 2020-11-20T12:20:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd9658700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40a2b30 con 0x7f8dd41649b0 2020-11-20T12:20:42.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dd4062880 0x7f8dd40a2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:42.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dd4062880 0x7f8dd40a2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49874/0 (socket says 172.21.15.116:49874) 2020-11-20T12:20:42.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd3fff700 1 -- 172.21.15.116:0/666019754 learned_addr learned my addr 172.21.15.116:0/666019754 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:42.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd8e57700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dd41649b0 0x7f8dd4137330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:42.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.620+0000 7f8dd37fe700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dd40b94e0 0x7f8dd40a4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:42.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8dd3fff700 1 -- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dd41649b0 msgr2=0x7f8dd4137330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8dd3fff700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dd41649b0 0x7f8dd4137330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8dd3fff700 1 -- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dd40b94e0 msgr2=0x7f8dd40a4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8dd3fff700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dd40b94e0 0x7f8dd40a4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8dd3fff700 1 -- 172.21.15.116:0/666019754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40a2dd0 con 0x7f8dd4062880 2020-11-20T12:20:42.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8dd3fff700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dd4062880 0x7f8dd40a2220 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f8dc400c710 tx=0x7f8dc4007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8db3fff700 1 -- 172.21.15.116:0/666019754 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc400e560 con 0x7f8dd4062880 2020-11-20T12:20:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8db3fff700 1 -- 172.21.15.116:0/666019754 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc400e6c0 con 0x7f8dd4062880 2020-11-20T12:20:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8dd9658700 1 -- 172.21.15.116:0/666019754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd40a3070 con 0x7f8dd4062880 2020-11-20T12:20:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8db3fff700 1 -- 172.21.15.116:0/666019754 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc400ea60 con 0x7f8dd4062880 2020-11-20T12:20:42.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8dd9658700 1 -- 172.21.15.116:0/666019754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd4164e10 con 0x7f8dd4062880 2020-11-20T12:20:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.624+0000 7f8dd9658700 1 -- 172.21.15.116:0/666019754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd405cf30 con 0x7f8dd4062880 2020-11-20T12:20:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.628+0000 7f8db3fff700 1 -- 172.21.15.116:0/666019754 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8dc4010030 con 0x7f8dd4062880 2020-11-20T12:20:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.628+0000 7f8db3fff700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dac0557c0 0x7f8dac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:42.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.628+0000 7f8db3fff700 1 -- 172.21.15.116:0/666019754 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8dc406e450 con 0x7f8dd4062880 2020-11-20T12:20:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.628+0000 7f8dd37fe700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dac0557c0 0x7f8dac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:42.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.628+0000 7f8dd37fe700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dac0557c0 0x7f8dac057c80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f8dd4152fa0 tx=0x7f8dc8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:42.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.632+0000 7f8db3fff700 1 -- 172.21.15.116:0/666019754 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8dc40442f0 con 0x7f8dd4062880 2020-11-20T12:20:42.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.800+0000 7f8dd9658700 1 -- 172.21.15.116:0/666019754 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8dd4061230 con 0x7f8dd4062880 2020-11-20T12:20:42.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.800+0000 7f8db3fff700 1 -- 172.21.15.116:0/666019754 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f8dc4047910 con 0x7f8dd4062880 2020-11-20T12:20:42.797 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:42.798 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":96,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:20:35.459061+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:20:32.886768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-20T12:20:33.446960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:42.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 -- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dac0557c0 msgr2=0x7f8dac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:42.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dac0557c0 0x7f8dac057c80 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f8dd4152fa0 tx=0x7f8dc8015000).stop 2020-11-20T12:20:42.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 -- 172.21.15.116:0/666019754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dd4062880 msgr2=0x7f8dd40a2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:42.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dd4062880 0x7f8dd40a2220 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f8dc400c710 tx=0x7f8dc4007c10).stop 2020-11-20T12:20:42.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 -- 172.21.15.116:0/666019754 shutdown_connections 2020-11-20T12:20:42.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8dac0557c0 0x7f8dac057c80 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8dd41649b0 0x7f8dd4137330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8dd4062880 0x7f8dd40a2220 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 --2- 172.21.15.116:0/666019754 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8dd40b94e0 0x7f8dd40a4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:42.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 -- 172.21.15.116:0/666019754 >> 172.21.15.116:0/666019754 conn(0x7f8dd4060c40 msgr2=0x7f8dd40bf4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:42.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 -- 172.21.15.116:0/666019754 shutdown_connections 2020-11-20T12:20:42.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:42.804+0000 7f8dd9658700 1 -- 172.21.15.116:0/666019754 wait complete. 2020-11-20T12:20:42.827 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:20:32.886768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:20:33.446960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:20:42.828 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:20:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.012+0000 7ff84160f700 1 Processor -- start 2020-11-20T12:20:43.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.012+0000 7ff84160f700 1 -- start start 2020-11-20T12:20:43.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.016+0000 7ff84160f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff83c1518c0 0x7ff83c151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.016+0000 7ff84160f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff83c0c9010 0x7ff83c061550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.016+0000 7ff84160f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff83c0a92f0 con 0x7ff83c061a40 2020-11-20T12:20:43.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.016+0000 7ff84160f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff83c061ea0 con 0x7ff83c1518c0 2020-11-20T12:20:43.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.016+0000 7ff84160f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c0c9450 con 0x7ff83c0c9010 2020-11-20T12:20:43.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.016+0000 7ff83affd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff83c1518c0 0x7ff83c151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.016+0000 7ff83affd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff83c1518c0 0x7ff83c151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49886/0 (socket says 172.21.15.116:49886) 2020-11-20T12:20:43.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.016+0000 7ff83affd700 1 -- 172.21.15.116:0/677755558 learned_addr learned my addr 172.21.15.116:0/677755558 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:43.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.016+0000 7ff833fff700 1 --2- 172.21.15.116:0/677755558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff83c0c9010 0x7ff83c061550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff83affd700 1 -- 172.21.15.116:0/677755558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff83c0c9010 msgr2=0x7ff83c061550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:43.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff83affd700 1 --2- 172.21.15.116:0/677755558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff83c0c9010 0x7ff83c061550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff83affd700 1 -- 172.21.15.116:0/677755558 >> v1:172.21.15.173:6789/0 conn(0x7ff83c061a40 legacy=0x7ff83c0ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff83affd700 1 -- 172.21.15.116:0/677755558 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c0a9590 con 0x7ff83c1518c0 2020-11-20T12:20:43.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff83affd700 1 --2- 172.21.15.116:0/677755558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c1518c0 0x7ff83c151ca0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7ff82c00ba70 tx=0x7ff82c006e50).ready entity=mon.0 client_cookie=6867ab893ac063f0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:43.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff83a7fc700 1 -- 172.21.15.116:0/677755558 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c006030 con 0x7ff83c1518c0 2020-11-20T12:20:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff83a7fc700 1 -- 172.21.15.116:0/677755558 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c006190 con 0x7ff83c1518c0 2020-11-20T12:20:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff83a7fc700 1 -- 172.21.15.116:0/677755558 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c006510 con 0x7ff83c1518c0 2020-11-20T12:20:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff84160f700 1 -- 172.21.15.116:0/677755558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c1518c0 msgr2=0x7ff83c151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff84160f700 1 --2- 172.21.15.116:0/677755558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c1518c0 0x7ff83c151ca0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7ff82c00ba70 tx=0x7ff82c006e50).stop 2020-11-20T12:20:43.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff84160f700 1 -- 172.21.15.116:0/677755558 shutdown_connections 2020-11-20T12:20:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff84160f700 1 --2- 172.21.15.116:0/677755558 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c1518c0 0x7ff83c151ca0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff84160f700 1 --2- 172.21.15.116:0/677755558 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff83c0c9010 0x7ff83c061550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff84160f700 1 -- 172.21.15.116:0/677755558 >> 172.21.15.116:0/677755558 conn(0x7ff83c068370 msgr2=0x7ff83c0c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:43.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff84160f700 1 -- 172.21.15.116:0/677755558 shutdown_connections 2020-11-20T12:20:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.020+0000 7ff84160f700 1 -- 172.21.15.116:0/677755558 wait complete. 2020-11-20T12:20:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.024+0000 7ff84160f700 1 Processor -- start 2020-11-20T12:20:43.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.024+0000 7ff84160f700 1 -- start start 2020-11-20T12:20:43.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.024+0000 7ff84160f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c061a40 0x7ff83c164d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.024+0000 7ff84160f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff83c0c9010 0x7ff83c1554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.024+0000 7ff84160f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff83c1518c0 0x7ff83c1559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.024+0000 7ff84160f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c155ec0 con 0x7ff83c061a40 2020-11-20T12:20:43.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.024+0000 7ff84160f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c1560d0 con 0x7ff83c0c9010 2020-11-20T12:20:43.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.024+0000 7ff84160f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff83c156370 con 0x7ff83c1518c0 2020-11-20T12:20:43.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.024+0000 7ff83affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c061a40 0x7ff83c164d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff83affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c061a40 0x7ff83c164d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49888/0 (socket says 172.21.15.116:49888) 2020-11-20T12:20:43.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff83affd700 1 -- 172.21.15.116:0/2659513764 learned_addr learned my addr 172.21.15.116:0/2659513764 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:43.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff83b7fe700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff83c1518c0 0x7ff83c1559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff833fff700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff83c0c9010 0x7ff83c1554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff83affd700 1 -- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff83c1518c0 msgr2=0x7ff83c1559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff83affd700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff83c1518c0 0x7ff83c1559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff83affd700 1 -- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff83c0c9010 msgr2=0x7ff83c1554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff83affd700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff83c0c9010 0x7ff83c1554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff83affd700 1 -- 172.21.15.116:0/2659513764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff83c156610 con 0x7ff83c061a40 2020-11-20T12:20:43.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff83b7fe700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff83c1518c0 0x7ff83c1559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:43.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff83affd700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c061a40 0x7ff83c164d10 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7ff83c0c6400 tx=0x7ff82c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:43.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff832ffd700 1 -- 172.21.15.116:0/2659513764 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0059b0 con 0x7ff83c061a40 2020-11-20T12:20:43.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff832ffd700 1 -- 172.21.15.116:0/2659513764 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff82c005b10 con 0x7ff83c061a40 2020-11-20T12:20:43.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff84160f700 1 -- 172.21.15.116:0/2659513764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff83c1568b0 con 0x7ff83c061a40 2020-11-20T12:20:43.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff832ffd700 1 -- 172.21.15.116:0/2659513764 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff82c005e90 con 0x7ff83c061a40 2020-11-20T12:20:43.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff84160f700 1 -- 172.21.15.116:0/2659513764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff83c1653a0 con 0x7ff83c061a40 2020-11-20T12:20:43.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.028+0000 7ff84160f700 1 -- 172.21.15.116:0/2659513764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff83c069950 con 0x7ff83c061a40 2020-11-20T12:20:43.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.032+0000 7ff832ffd700 1 -- 172.21.15.116:0/2659513764 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff82c00f070 con 0x7ff83c061a40 2020-11-20T12:20:43.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.032+0000 7ff832ffd700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8180557c0 0x7ff818057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.032+0000 7ff832ffd700 1 -- 172.21.15.116:0/2659513764 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff82c0705e0 con 0x7ff83c061a40 2020-11-20T12:20:43.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.032+0000 7ff833fff700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8180557c0 0x7ff818057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.032+0000 7ff833fff700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8180557c0 0x7ff818057c80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7ff83c157780 tx=0x7ff824008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:43.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.036+0000 7ff832ffd700 1 -- 172.21.15.116:0/2659513764 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff82c046370 con 0x7ff83c061a40 2020-11-20T12:20:43.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.196+0000 7ff84160f700 1 -- 172.21.15.116:0/2659513764 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff83c0608c0 con 0x7ff83c061a40 2020-11-20T12:20:43.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.196+0000 7ff832ffd700 1 -- 172.21.15.116:0/2659513764 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v96) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff82c018e60 con 0x7ff83c061a40 2020-11-20T12:20:43.193 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:20:43.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 -- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8180557c0 msgr2=0x7ff818057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8180557c0 0x7ff818057c80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7ff83c157780 tx=0x7ff824008040).stop 2020-11-20T12:20:43.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 -- 172.21.15.116:0/2659513764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c061a40 msgr2=0x7ff83c164d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c061a40 0x7ff83c164d10 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7ff83c0c6400 tx=0x7ff82c015000).stop 2020-11-20T12:20:43.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 -- 172.21.15.116:0/2659513764 shutdown_connections 2020-11-20T12:20:43.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff8180557c0 0x7ff818057c80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff83c1518c0 0x7ff83c1559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff83c061a40 0x7ff83c164d10 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 --2- 172.21.15.116:0/2659513764 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff83c0c9010 0x7ff83c1554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 -- 172.21.15.116:0/2659513764 >> 172.21.15.116:0/2659513764 conn(0x7ff83c068370 msgr2=0x7ff83c0c6530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:43.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 -- 172.21.15.116:0/2659513764 shutdown_connections 2020-11-20T12:20:43.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.200+0000 7ff84160f700 1 -- 172.21.15.116:0/2659513764 wait complete. 2020-11-20T12:20:43.225 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:20:43.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.416+0000 7f531104b700 1 Processor -- start 2020-11-20T12:20:43.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.416+0000 7f531104b700 1 -- start start 2020-11-20T12:20:43.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.416+0000 7f531104b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f530c0b9db0 0x7f530c0c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.416+0000 7f531104b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c0a8930 con 0x7f530c14f880 2020-11-20T12:20:43.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.416+0000 7f531104b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c0a8690 con 0x7f530c0a81f0 2020-11-20T12:20:43.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.416+0000 7f531104b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c0b94d0 con 0x7f530c0b9db0 2020-11-20T12:20:43.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.416+0000 7f531084a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f530c0b9db0 0x7f530c0c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.416+0000 7f531084a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f530c0b9db0 0x7f530c0c15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49896/0 (socket says 172.21.15.116:49896) 2020-11-20T12:20:43.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.416+0000 7f531084a700 1 -- 172.21.15.116:0/2490825747 learned_addr learned my addr 172.21.15.116:0/2490825747 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:43.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f531084a700 1 -- 172.21.15.116:0/2490825747 >> v1:172.21.15.173:6789/0 conn(0x7f530c0a81f0 legacy=0x7f530c0c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:43.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f531084a700 1 -- 172.21.15.116:0/2490825747 >> v1:172.21.15.116:6789/0 conn(0x7f530c14f880 legacy=0x7f530c060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:43.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f531084a700 1 -- 172.21.15.116:0/2490825747 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0b9770 con 0x7f530c0b9db0 2020-11-20T12:20:43.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f531084a700 1 --2- 172.21.15.116:0/2490825747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530c0b9db0 0x7f530c0c15a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f5304009d50 tx=0x7f5304009290).ready entity=mon.0 client_cookie=8639a340cb35875e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:43.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f530affd700 1 -- 172.21.15.116:0/2490825747 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530400bc00 con 0x7f530c0b9db0 2020-11-20T12:20:43.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f530affd700 1 -- 172.21.15.116:0/2490825747 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530400bd60 con 0x7f530c0b9db0 2020-11-20T12:20:43.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f530affd700 1 -- 172.21.15.116:0/2490825747 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304011610 con 0x7f530c0b9db0 2020-11-20T12:20:43.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f531104b700 1 -- 172.21.15.116:0/2490825747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530c0b9db0 msgr2=0x7f530c0c15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f531104b700 1 --2- 172.21.15.116:0/2490825747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530c0b9db0 0x7f530c0c15a0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7f5304009d50 tx=0x7f5304009290).stop 2020-11-20T12:20:43.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f531104b700 1 -- 172.21.15.116:0/2490825747 shutdown_connections 2020-11-20T12:20:43.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f531104b700 1 --2- 172.21.15.116:0/2490825747 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530c0b9db0 0x7f530c0c15a0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f531104b700 1 -- 172.21.15.116:0/2490825747 >> 172.21.15.116:0/2490825747 conn(0x7f530c068350 msgr2=0x7f530c0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:43.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f531104b700 1 -- 172.21.15.116:0/2490825747 shutdown_connections 2020-11-20T12:20:43.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.420+0000 7f531104b700 1 -- 172.21.15.116:0/2490825747 wait complete. 2020-11-20T12:20:43.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.424+0000 7f531104b700 1 Processor -- start 2020-11-20T12:20:43.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.424+0000 7f531104b700 1 -- start start 2020-11-20T12:20:43.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.424+0000 7f531104b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f530c0a81f0 0x7f530c13c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.424+0000 7f531104b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530c0b9db0 0x7f530c162db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f531104b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f530c14f880 0x7f530c1632a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f531104b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0c35e0 con 0x7f530c0b9db0 2020-11-20T12:20:43.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f531104b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0c3340 con 0x7f530c14f880 2020-11-20T12:20:43.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f531104b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c09d4e0 con 0x7f530c0a81f0 2020-11-20T12:20:43.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f530b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530c0b9db0 0x7f530c162db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f530b7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530c0b9db0 0x7f530c162db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49904/0 (socket says 172.21.15.116:49904) 2020-11-20T12:20:43.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f530b7fe700 1 -- 172.21.15.116:0/1250713111 learned_addr learned my addr 172.21.15.116:0/1250713111 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:43.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f531084a700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f530c14f880 0x7f530c1632a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f530bfff700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f530c0a81f0 0x7f530c13c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f530b7fe700 1 -- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f530c0a81f0 msgr2=0x7f530c13c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f530b7fe700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f530c0a81f0 0x7f530c13c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f530b7fe700 1 -- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f530c14f880 msgr2=0x7f530c1632a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f530b7fe700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f530c14f880 0x7f530c1632a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f530b7fe700 1 -- 172.21.15.116:0/1250713111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c09d780 con 0x7f530c0b9db0 2020-11-20T12:20:43.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f530b7fe700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530c0b9db0 0x7f530c162db0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f530000c730 tx=0x7f5300007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:43.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f52ebfff700 1 -- 172.21.15.116:0/1250713111 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530000e490 con 0x7f530c0b9db0 2020-11-20T12:20:43.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f52ebfff700 1 -- 172.21.15.116:0/1250713111 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530000e610 con 0x7f530c0b9db0 2020-11-20T12:20:43.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f531104b700 1 -- 172.21.15.116:0/1250713111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530c09da20 con 0x7f530c0b9db0 2020-11-20T12:20:43.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f52ebfff700 1 -- 172.21.15.116:0/1250713111 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530000e9d0 con 0x7f530c0b9db0 2020-11-20T12:20:43.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f531104b700 1 -- 172.21.15.116:0/1250713111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c0a0070 con 0x7f530c0b9db0 2020-11-20T12:20:43.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.428+0000 7f531104b700 1 -- 172.21.15.116:0/1250713111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530c09dbb0 con 0x7f530c0b9db0 2020-11-20T12:20:43.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.432+0000 7f52ebfff700 1 -- 172.21.15.116:0/1250713111 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5300010030 con 0x7f530c0b9db0 2020-11-20T12:20:43.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.432+0000 7f52ebfff700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52e40557c0 0x7f52e4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.432+0000 7f52ebfff700 1 -- 172.21.15.116:0/1250713111 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f530006e4a0 con 0x7f530c0b9db0 2020-11-20T12:20:43.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.432+0000 7f530bfff700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52e40557c0 0x7f52e4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.432+0000 7f530bfff700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52e40557c0 0x7f52e4057c80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f52fc006fd0 tx=0x7f52fc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:43.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.440+0000 7f52ebfff700 1 -- 172.21.15.116:0/1250713111 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5300044340 con 0x7f530c0b9db0 2020-11-20T12:20:43.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.600+0000 7f531104b700 1 -- 172.21.15.116:0/1250713111 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f530c05cf30 con 0x7f530c0b9db0 2020-11-20T12:20:43.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.600+0000 7f52ebfff700 1 -- 172.21.15.116:0/1250713111 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v96) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5300047960 con 0x7f530c0b9db0 2020-11-20T12:20:43.598 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:43.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 -- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52e40557c0 msgr2=0x7f52e4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52e40557c0 0x7f52e4057c80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f52fc006fd0 tx=0x7f52fc008040).stop 2020-11-20T12:20:43.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 -- 172.21.15.116:0/1250713111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530c0b9db0 msgr2=0x7f530c162db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530c0b9db0 0x7f530c162db0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f530000c730 tx=0x7f5300007c30).stop 2020-11-20T12:20:43.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 -- 172.21.15.116:0/1250713111 shutdown_connections 2020-11-20T12:20:43.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52e40557c0 0x7f52e4057c80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f530c14f880 0x7f530c1632a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f530c0a81f0 0x7f530c13c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 --2- 172.21.15.116:0/1250713111 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f530c0b9db0 0x7f530c162db0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 -- 172.21.15.116:0/1250713111 >> 172.21.15.116:0/1250713111 conn(0x7f530c068350 msgr2=0x7f530c0a75e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:43.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 -- 172.21.15.116:0/1250713111 shutdown_connections 2020-11-20T12:20:43.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.604+0000 7f531104b700 1 -- 172.21.15.116:0/1250713111 wait complete. 2020-11-20T12:20:43.621 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:20:43.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.812+0000 7fd0a4b86700 1 Processor -- start 2020-11-20T12:20:43.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.812+0000 7fd0a4b86700 1 -- start start 2020-11-20T12:20:43.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd0a4b86700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd0a00a8400 0x7fd0a00aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd0a4b86700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd0a00ad540 0x7fd0a00ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd0a4b86700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0a00ace40 con 0x7fd0a00aa7d0 2020-11-20T12:20:43.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd0a4b86700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a01526c0 con 0x7fd0a00a8400 2020-11-20T12:20:43.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd0a4b86700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0a0152960 con 0x7fd0a00ad540 2020-11-20T12:20:43.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd09effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd0a00a8400 0x7fd0a00aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd09effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd0a00a8400 0x7fd0a00aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49912/0 (socket says 172.21.15.116:49912) 2020-11-20T12:20:43.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd09effd700 1 -- 172.21.15.116:0/2702918667 learned_addr learned my addr 172.21.15.116:0/2702918667 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:43.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd09ffff700 1 --2- 172.21.15.116:0/2702918667 >> v2:172.21.15.173:3300/0 conn(0x7fd0a00ad540 0x7fd0a00ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd09effd700 1 -- 172.21.15.116:0/2702918667 >> v2:172.21.15.173:3300/0 conn(0x7fd0a00ad540 msgr2=0x7fd0a00ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:43.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd09effd700 1 --2- 172.21.15.116:0/2702918667 >> v2:172.21.15.173:3300/0 conn(0x7fd0a00ad540 0x7fd0a00ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd09effd700 1 -- 172.21.15.116:0/2702918667 >> v1:172.21.15.173:6789/0 conn(0x7fd0a00aa7d0 legacy=0x7fd0a0151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.816+0000 7fd09effd700 1 -- 172.21.15.116:0/2702918667 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a0152c00 con 0x7fd0a00a8400 2020-11-20T12:20:43.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd09effd700 1 --2- 172.21.15.116:0/2702918667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0a00a8400 0x7fd0a00aabb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fd0a00a8230 tx=0x7fd094006eb0).ready entity=mon.0 client_cookie=39819db0f329e673 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:43.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd09e7fc700 1 -- 172.21.15.116:0/2702918667 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0940098d0 con 0x7fd0a00a8400 2020-11-20T12:20:43.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd09e7fc700 1 -- 172.21.15.116:0/2702918667 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd094009a30 con 0x7fd0a00a8400 2020-11-20T12:20:43.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd09e7fc700 1 -- 172.21.15.116:0/2702918667 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd094009db0 con 0x7fd0a00a8400 2020-11-20T12:20:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd0a4b86700 1 -- 172.21.15.116:0/2702918667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0a00a8400 msgr2=0x7fd0a00aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd0a4b86700 1 --2- 172.21.15.116:0/2702918667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0a00a8400 0x7fd0a00aabb0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7fd0a00a8230 tx=0x7fd094006eb0).stop 2020-11-20T12:20:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd0a4b86700 1 -- 172.21.15.116:0/2702918667 shutdown_connections 2020-11-20T12:20:43.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd0a4b86700 1 --2- 172.21.15.116:0/2702918667 >> v2:172.21.15.173:3300/0 conn(0x7fd0a00ad540 0x7fd0a00ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd0a4b86700 1 --2- 172.21.15.116:0/2702918667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0a00a8400 0x7fd0a00aabb0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd0a4b86700 1 -- 172.21.15.116:0/2702918667 >> 172.21.15.116:0/2702918667 conn(0x7fd0a00678a0 msgr2=0x7fd0a0067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:43.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd0a4b86700 1 -- 172.21.15.116:0/2702918667 shutdown_connections 2020-11-20T12:20:43.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.820+0000 7fd0a4b86700 1 -- 172.21.15.116:0/2702918667 wait complete. 2020-11-20T12:20:43.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd0a4b86700 1 Processor -- start 2020-11-20T12:20:43.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd0a4b86700 1 -- start start 2020-11-20T12:20:43.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd0a4b86700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0a00aa7d0 0x7fd0a0161c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd0a4b86700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a00ad540 0x7fd0a0160a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd0a4b86700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0a01575e0 0x7fd0a00bd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd0a4b86700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a01566c0 con 0x7fd0a01575e0 2020-11-20T12:20:43.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd0a4b86700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a0156960 con 0x7fd0a00ad540 2020-11-20T12:20:43.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd0a4b86700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0a0156c00 con 0x7fd0a00aa7d0 2020-11-20T12:20:43.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd09ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0a01575e0 0x7fd0a00bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd09ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0a01575e0 0x7fd0a00bd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49920/0 (socket says 172.21.15.116:49920) 2020-11-20T12:20:43.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd09effd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a00ad540 0x7fd0a0160a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd09f7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0a00aa7d0 0x7fd0a0161c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.824+0000 7fd09ffff700 1 -- 172.21.15.116:0/3579920138 learned_addr learned my addr 172.21.15.116:0/3579920138 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:43.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd09ffff700 1 -- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0a00aa7d0 msgr2=0x7fd0a0161c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd09ffff700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0a00aa7d0 0x7fd0a0161c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd09ffff700 1 -- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a00ad540 msgr2=0x7fd0a0160a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd09ffff700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a00ad540 0x7fd0a0160a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd09ffff700 1 -- 172.21.15.116:0/3579920138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0a0156ea0 con 0x7fd0a01575e0 2020-11-20T12:20:43.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd09ffff700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0a01575e0 0x7fd0a00bd700 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7fd09800db40 tx=0x7fd098008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:43.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd07f7fe700 1 -- 172.21.15.116:0/3579920138 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098009850 con 0x7fd0a01575e0 2020-11-20T12:20:43.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd07f7fe700 1 -- 172.21.15.116:0/3579920138 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0980099b0 con 0x7fd0a01575e0 2020-11-20T12:20:43.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd0a4b86700 1 -- 172.21.15.116:0/3579920138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0a0157140 con 0x7fd0a01575e0 2020-11-20T12:20:43.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd07f7fe700 1 -- 172.21.15.116:0/3579920138 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd098009d50 con 0x7fd0a01575e0 2020-11-20T12:20:43.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd0a4b86700 1 -- 172.21.15.116:0/3579920138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0a0160f60 con 0x7fd0a01575e0 2020-11-20T12:20:43.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.828+0000 7fd0a4b86700 1 -- 172.21.15.116:0/3579920138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0a01572d0 con 0x7fd0a01575e0 2020-11-20T12:20:43.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.832+0000 7fd07f7fe700 1 -- 172.21.15.116:0/3579920138 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd098013690 con 0x7fd0a01575e0 2020-11-20T12:20:43.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.832+0000 7fd07f7fe700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0780557f0 0x7fd078057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:43.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.832+0000 7fd07f7fe700 1 -- 172.21.15.116:0/3579920138 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd09806edf0 con 0x7fd0a01575e0 2020-11-20T12:20:43.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.832+0000 7fd09f7fe700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0780557f0 0x7fd078057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:43.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.832+0000 7fd09f7fe700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0780557f0 0x7fd078057cb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fd0900035f0 tx=0x7fd090007b90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:43.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.836+0000 7fd07f7fe700 1 -- 172.21.15.116:0/3579920138 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd098044c30 con 0x7fd0a01575e0 2020-11-20T12:20:43.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.996+0000 7fd0a4b86700 1 -- 172.21.15.116:0/3579920138 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd0a005cf30 con 0x7fd0a01575e0 2020-11-20T12:20:43.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:43.996+0000 7fd07f7fe700 1 -- 172.21.15.116:0/3579920138 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+10 (secure 0 0 0) 0x7fd098048250 con 0x7fd0a01575e0 2020-11-20T12:20:43.995 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:43.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 -- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0780557f0 msgr2=0x7fd078057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0780557f0 0x7fd078057cb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fd0900035f0 tx=0x7fd090007b90).stop 2020-11-20T12:20:43.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 -- 172.21.15.116:0/3579920138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0a01575e0 msgr2=0x7fd0a00bd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:43.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0a01575e0 0x7fd0a00bd700 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7fd09800db40 tx=0x7fd098008040).stop 2020-11-20T12:20:43.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 -- 172.21.15.116:0/3579920138 shutdown_connections 2020-11-20T12:20:43.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0780557f0 0x7fd078057cb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:43.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0a01575e0 0x7fd0a00bd700 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0a00aa7d0 0x7fd0a0161c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 --2- 172.21.15.116:0/3579920138 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0a00ad540 0x7fd0a0160a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 -- 172.21.15.116:0/3579920138 >> 172.21.15.116:0/3579920138 conn(0x7fd0a00678a0 msgr2=0x7fd0a00613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:44.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 -- 172.21.15.116:0/3579920138 shutdown_connections 2020-11-20T12:20:44.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.000+0000 7fd0a4b86700 1 -- 172.21.15.116:0/3579920138 wait complete. 2020-11-20T12:20:44.023 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:20:44.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.200+0000 7fb1e9caa700 1 Processor -- start 2020-11-20T12:20:44.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.200+0000 7fb1e9caa700 1 -- start start 2020-11-20T12:20:44.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.200+0000 7fb1e9caa700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb1e40aadb0 0x7fb1e4151ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.200+0000 7fb1e9caa700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb1e41523e0 0x7fb1e413aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.200+0000 7fb1e9caa700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1e40acfe0 con 0x7fb1e40aa9d0 2020-11-20T12:20:44.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.200+0000 7fb1e9caa700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e41529c0 con 0x7fb1e41523e0 2020-11-20T12:20:44.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.200+0000 7fb1e9caa700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1e4152c60 con 0x7fb1e40aadb0 2020-11-20T12:20:44.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e37fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb1e40aa9d0 0x7fb1e40a8570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59358/0 (socket says 172.21.15.116:59358) 2020-11-20T12:20:44.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e37fe700 1 -- 172.21.15.116:0/3441815595 learned_addr learned my addr 172.21.15.116:0/3441815595 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:44.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e3fff700 1 --2- 172.21.15.116:0/3441815595 >> v2:172.21.15.116:3300/0 conn(0x7fb1e41523e0 0x7fb1e413aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e2ffd700 1 --2- 172.21.15.116:0/3441815595 >> v2:172.21.15.173:3300/0 conn(0x7fb1e40aadb0 0x7fb1e4151ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e27fc700 1 -- 172.21.15.116:0/3441815595 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089461038 0 0) 0x7fb1e40acfe0 con 0x7fb1e40aa9d0 2020-11-20T12:20:44.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e27fc700 1 -- 172.21.15.116:0/3441815595 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1d0003240 con 0x7fb1e40aa9d0 2020-11-20T12:20:44.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e3fff700 1 -- 172.21.15.116:0/3441815595 >> v2:172.21.15.173:3300/0 conn(0x7fb1e40aadb0 msgr2=0x7fb1e4151ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:44.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e3fff700 1 --2- 172.21.15.116:0/3441815595 >> v2:172.21.15.173:3300/0 conn(0x7fb1e40aadb0 0x7fb1e4151ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e3fff700 1 -- 172.21.15.116:0/3441815595 >> v1:172.21.15.116:6789/0 conn(0x7fb1e40aa9d0 legacy=0x7fb1e40a8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e3fff700 1 -- 172.21.15.116:0/3441815595 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e40ad240 con 0x7fb1e41523e0 2020-11-20T12:20:44.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e3fff700 1 --2- 172.21.15.116:0/3441815595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e41523e0 0x7fb1e413aed0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7fb1e40aa5f0 tx=0x7fb1dc0091d0).ready entity=mon.0 client_cookie=99f08c2e174678be server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:44.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e27fc700 1 -- 172.21.15.116:0/3441815595 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc00ba70 con 0x7fb1e41523e0 2020-11-20T12:20:44.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e27fc700 1 -- 172.21.15.116:0/3441815595 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc00bbd0 con 0x7fb1e41523e0 2020-11-20T12:20:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e27fc700 1 -- 172.21.15.116:0/3441815595 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc011400 con 0x7fb1e41523e0 2020-11-20T12:20:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e9caa700 1 -- 172.21.15.116:0/3441815595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e41523e0 msgr2=0x7fb1e413aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e9caa700 1 --2- 172.21.15.116:0/3441815595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e41523e0 0x7fb1e413aed0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7fb1e40aa5f0 tx=0x7fb1dc0091d0).stop 2020-11-20T12:20:44.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e9caa700 1 -- 172.21.15.116:0/3441815595 shutdown_connections 2020-11-20T12:20:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e9caa700 1 --2- 172.21.15.116:0/3441815595 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e41523e0 0x7fb1e413aed0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e9caa700 1 --2- 172.21.15.116:0/3441815595 >> v2:172.21.15.173:3300/0 conn(0x7fb1e40aadb0 0x7fb1e4151ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e9caa700 1 -- 172.21.15.116:0/3441815595 >> 172.21.15.116:0/3441815595 conn(0x7fb1e40678a0 msgr2=0x7fb1e4067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e9caa700 1 -- 172.21.15.116:0/3441815595 shutdown_connections 2020-11-20T12:20:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.204+0000 7fb1e9caa700 1 -- 172.21.15.116:0/3441815595 wait complete. 2020-11-20T12:20:44.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.208+0000 7fb1e9caa700 1 Processor -- start 2020-11-20T12:20:44.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.208+0000 7fb1e9caa700 1 -- start start 2020-11-20T12:20:44.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e9caa700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1e40aa9d0 0x7fb1e413a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e9caa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e40aadb0 0x7fb1e4155be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e9caa700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e41523e0 0x7fb1e41560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e9caa700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4156c00 con 0x7fb1e41523e0 2020-11-20T12:20:44.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e9caa700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e41566c0 con 0x7fb1e40aadb0 2020-11-20T12:20:44.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e9caa700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4156960 con 0x7fb1e40aa9d0 2020-11-20T12:20:44.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e41523e0 0x7fb1e41560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e41523e0 0x7fb1e41560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49932/0 (socket says 172.21.15.116:49932) 2020-11-20T12:20:44.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e3fff700 1 -- 172.21.15.116:0/658478065 learned_addr learned my addr 172.21.15.116:0/658478065 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:44.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e37fe700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1e40aa9d0 0x7fb1e413a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e2ffd700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e40aadb0 0x7fb1e4155be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e3fff700 1 -- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1e40aa9d0 msgr2=0x7fb1e413a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e3fff700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1e40aa9d0 0x7fb1e413a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e3fff700 1 -- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e40aadb0 msgr2=0x7fb1e4155be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e3fff700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e40aadb0 0x7fb1e4155be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e3fff700 1 -- 172.21.15.116:0/658478065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1e4162050 con 0x7fb1e41523e0 2020-11-20T12:20:44.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e3fff700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e41523e0 0x7fb1e41560d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7fb1e40ad170 tx=0x7fb1dc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:44.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1c37fe700 1 -- 172.21.15.116:0/658478065 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc006b20 con 0x7fb1e41523e0 2020-11-20T12:20:44.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1c37fe700 1 -- 172.21.15.116:0/658478065 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1dc01c5f0 con 0x7fb1e41523e0 2020-11-20T12:20:44.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e9caa700 1 -- 172.21.15.116:0/658478065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e41622f0 con 0x7fb1e41523e0 2020-11-20T12:20:44.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1c37fe700 1 -- 172.21.15.116:0/658478065 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1dc01c970 con 0x7fb1e41523e0 2020-11-20T12:20:44.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.212+0000 7fb1e9caa700 1 -- 172.21.15.116:0/658478065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e413a620 con 0x7fb1e41523e0 2020-11-20T12:20:44.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.216+0000 7fb1e9caa700 1 -- 172.21.15.116:0/658478065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e4162480 con 0x7fb1e41523e0 2020-11-20T12:20:44.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.216+0000 7fb1c37fe700 1 -- 172.21.15.116:0/658478065 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb1dc017070 con 0x7fb1e41523e0 2020-11-20T12:20:44.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.216+0000 7fb1c37fe700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb1bc0557c0 0x7fb1bc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.216+0000 7fb1c37fe700 1 -- 172.21.15.116:0/658478065 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb1dc071490 con 0x7fb1e41523e0 2020-11-20T12:20:44.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.216+0000 7fb1e37fe700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb1bc0557c0 0x7fb1bc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.220+0000 7fb1e37fe700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb1bc0557c0 0x7fb1bc057c80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fb1d40055a0 tx=0x7fb1d4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:44.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.220+0000 7fb1c37fe700 1 -- 172.21.15.116:0/658478065 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb1dc047370 con 0x7fb1e41523e0 2020-11-20T12:20:44.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1e9caa700 1 -- 172.21.15.116:0/658478065 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1e405d2a0 con 0x7fb1e41523e0 2020-11-20T12:20:44.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1c37fe700 1 -- 172.21.15.116:0/658478065 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+10 (secure 0 0 0) 0x7fb1dc015020 con 0x7fb1e41523e0 2020-11-20T12:20:44.377 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:44.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1e9caa700 1 -- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb1bc0557c0 msgr2=0x7fb1bc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1e9caa700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb1bc0557c0 0x7fb1bc057c80 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fb1d40055a0 tx=0x7fb1d4009040).stop 2020-11-20T12:20:44.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1e9caa700 1 -- 172.21.15.116:0/658478065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e41523e0 msgr2=0x7fb1e41560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1e9caa700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e41523e0 0x7fb1e41560d0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7fb1e40ad170 tx=0x7fb1dc016000).stop 2020-11-20T12:20:44.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1e9caa700 1 -- 172.21.15.116:0/658478065 shutdown_connections 2020-11-20T12:20:44.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1e9caa700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb1bc0557c0 0x7fb1bc057c80 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1e9caa700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1e41523e0 0x7fb1e41560d0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1e9caa700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1e40aa9d0 0x7fb1e413a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1e9caa700 1 --2- 172.21.15.116:0/658478065 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1e40aadb0 0x7fb1e4155be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.380+0000 7fb1e9caa700 1 -- 172.21.15.116:0/658478065 >> 172.21.15.116:0/658478065 conn(0x7fb1e40678a0 msgr2=0x7fb1e4150490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.384+0000 7fb1e9caa700 1 -- 172.21.15.116:0/658478065 shutdown_connections 2020-11-20T12:20:44.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.384+0000 7fb1e9caa700 1 -- 172.21.15.116:0/658478065 wait complete. 2020-11-20T12:20:44.402 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:20:44.403 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:44.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff738dd4700 1 Processor -- start 2020-11-20T12:20:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff738dd4700 1 -- start start 2020-11-20T12:20:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff738dd4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff73414c0a0 0x7ff734062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff738dd4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7340b94e0 0x7ff7340b9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff738dd4700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7340ba1b0 con 0x7ff734062880 2020-11-20T12:20:44.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff738dd4700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff734152e60 con 0x7ff7340b94e0 2020-11-20T12:20:44.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff738dd4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7341555a0 con 0x7ff73414c0a0 2020-11-20T12:20:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff733fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7340b94e0 0x7ff7340b9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff732ffd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff73414c0a0 0x7ff734062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff733fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7340b94e0 0x7ff7340b9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49942/0 (socket says 172.21.15.116:49942) 2020-11-20T12:20:44.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff733fff700 1 -- 172.21.15.116:0/279232359 learned_addr learned my addr 172.21.15.116:0/279232359 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff7337fe700 1 --1- 172.21.15.116:0/279232359 >> v1:172.21.15.173:6789/0 conn(0x7ff734062880 0x7ff73414b990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:55604/0 (socket says 172.21.15.116:55604) 2020-11-20T12:20:44.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff733fff700 1 -- 172.21.15.116:0/279232359 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff73414c0a0 msgr2=0x7ff734062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:44.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff733fff700 1 --2- 172.21.15.116:0/279232359 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff73414c0a0 0x7ff734062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff733fff700 1 -- 172.21.15.116:0/279232359 >> v1:172.21.15.173:6789/0 conn(0x7ff734062880 legacy=0x7ff73414b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff733fff700 1 -- 172.21.15.116:0/279232359 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff734155840 con 0x7ff7340b94e0 2020-11-20T12:20:44.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.588+0000 7ff733fff700 1 --2- 172.21.15.116:0/279232359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7340b94e0 0x7ff7340b9990 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7ff72c009d50 tx=0x7ff72c009290).ready entity=mon.0 client_cookie=b05dedf08beb9b1e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:44.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.592+0000 7ff7327fc700 1 -- 172.21.15.116:0/279232359 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c00bbc0 con 0x7ff7340b94e0 2020-11-20T12:20:44.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.592+0000 7ff7327fc700 1 -- 172.21.15.116:0/279232359 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff72c00bd20 con 0x7ff7340b94e0 2020-11-20T12:20:44.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.592+0000 7ff7327fc700 1 -- 172.21.15.116:0/279232359 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff72c011610 con 0x7ff7340b94e0 2020-11-20T12:20:44.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.592+0000 7ff738dd4700 1 -- 172.21.15.116:0/279232359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7340b94e0 msgr2=0x7ff7340b9990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.592+0000 7ff738dd4700 1 --2- 172.21.15.116:0/279232359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7340b94e0 0x7ff7340b9990 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7ff72c009d50 tx=0x7ff72c009290).stop 2020-11-20T12:20:44.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.592+0000 7ff738dd4700 1 -- 172.21.15.116:0/279232359 shutdown_connections 2020-11-20T12:20:44.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.592+0000 7ff738dd4700 1 --2- 172.21.15.116:0/279232359 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7340b94e0 0x7ff7340b9990 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.592+0000 7ff738dd4700 1 --2- 172.21.15.116:0/279232359 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff73414c0a0 0x7ff734062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.592+0000 7ff738dd4700 1 -- 172.21.15.116:0/279232359 >> 172.21.15.116:0/279232359 conn(0x7ff734060c40 msgr2=0x7ff7340c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:44.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.592+0000 7ff738dd4700 1 -- 172.21.15.116:0/279232359 shutdown_connections 2020-11-20T12:20:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.592+0000 7ff738dd4700 1 -- 172.21.15.116:0/279232359 wait complete. 2020-11-20T12:20:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff738dd4700 1 Processor -- start 2020-11-20T12:20:44.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff738dd4700 1 -- start start 2020-11-20T12:20:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff738dd4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff734062880 0x7ff7340bbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff738dd4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7340b94e0 0x7ff734160750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff738dd4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff73414c0a0 0x7ff7340c3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff738dd4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff734165230 con 0x7ff734062880 2020-11-20T12:20:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff738dd4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340bb570 con 0x7ff73414c0a0 2020-11-20T12:20:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff738dd4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7340bb810 con 0x7ff7340b94e0 2020-11-20T12:20:44.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff7337fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff734062880 0x7ff7340bbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff7337fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff734062880 0x7ff7340bbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49944/0 (socket says 172.21.15.116:49944) 2020-11-20T12:20:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff7337fe700 1 -- 172.21.15.116:0/1646271703 learned_addr learned my addr 172.21.15.116:0/1646271703 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:44.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff732ffd700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7340b94e0 0x7ff734160750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff733fff700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff73414c0a0 0x7ff7340c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff7337fe700 1 -- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7340b94e0 msgr2=0x7ff734160750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff7337fe700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7340b94e0 0x7ff734160750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff7337fe700 1 -- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff73414c0a0 msgr2=0x7ff7340c3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff7337fe700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff73414c0a0 0x7ff7340c3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff7337fe700 1 -- 172.21.15.116:0/1646271703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7340bbab0 con 0x7ff734062880 2020-11-20T12:20:44.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff7337fe700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff734062880 0x7ff7340bbf50 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7ff72400c710 tx=0x7ff724007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:44.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff7137fe700 1 -- 172.21.15.116:0/1646271703 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7240072e0 con 0x7ff734062880 2020-11-20T12:20:44.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff7137fe700 1 -- 172.21.15.116:0/1646271703 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff724005530 con 0x7ff734062880 2020-11-20T12:20:44.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.596+0000 7ff738dd4700 1 -- 172.21.15.116:0/1646271703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff734160c40 con 0x7ff734062880 2020-11-20T12:20:44.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.600+0000 7ff7137fe700 1 -- 172.21.15.116:0/1646271703 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7240058d0 con 0x7ff734062880 2020-11-20T12:20:44.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.600+0000 7ff738dd4700 1 -- 172.21.15.116:0/1646271703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7340c3860 con 0x7ff734062880 2020-11-20T12:20:44.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.600+0000 7ff738dd4700 1 -- 172.21.15.116:0/1646271703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff73405cf30 con 0x7ff734062880 2020-11-20T12:20:44.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.600+0000 7ff7137fe700 1 -- 172.21.15.116:0/1646271703 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff724010070 con 0x7ff734062880 2020-11-20T12:20:44.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.600+0000 7ff7137fe700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff70c0557f0 0x7ff70c057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:44.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.600+0000 7ff7137fe700 1 -- 172.21.15.116:0/1646271703 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff72406db80 con 0x7ff734062880 2020-11-20T12:20:44.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.604+0000 7ff732ffd700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff70c0557f0 0x7ff70c057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:44.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.604+0000 7ff732ffd700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff70c0557f0 0x7ff70c057cb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7ff73414c510 tx=0x7ff728006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:44.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.608+0000 7ff7137fe700 1 -- 172.21.15.116:0/1646271703 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff724043a20 con 0x7ff734062880 2020-11-20T12:20:44.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.820+0000 7ff738dd4700 1 -- 172.21.15.116:0/1646271703 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff7340bfa60 con 0x7ff734062880 2020-11-20T12:20:44.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.820+0000 7ff7137fe700 1 -- 172.21.15.116:0/1646271703 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2848 (secure 0 0 0) 0x7ff724018030 con 0x7ff734062880 2020-11-20T12:20:44.818 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:44.818 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3105911814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3105911814},{"type":"v1","addr":"172.21.15.173:6837","nonce":3105911814}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/3457547857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3457547857},{"type":"v1","addr":"172.21.15.116:6835","nonce":3457547857}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/81195738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":81195738},{"type":"v1","addr":"172.21.15.116:6833","nonce":81195738}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:20:34.449524+0000","modified":"2020-11-20T12:20:36.475264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/2361610151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2361610151},{"type":"v1","addr":"172.21.15.173:6835","nonce":2361610151}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-20T12:20:44.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 -- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff70c0557f0 msgr2=0x7ff70c057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff70c0557f0 0x7ff70c057cb0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7ff73414c510 tx=0x7ff728006df0).stop 2020-11-20T12:20:44.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 -- 172.21.15.116:0/1646271703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff734062880 msgr2=0x7ff7340bbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:44.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff734062880 0x7ff7340bbf50 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7ff72400c710 tx=0x7ff724007bf0).stop 2020-11-20T12:20:44.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 -- 172.21.15.116:0/1646271703 shutdown_connections 2020-11-20T12:20:44.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff70c0557f0 0x7ff70c057cb0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff73414c0a0 0x7ff7340c3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff734062880 0x7ff7340bbf50 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 --2- 172.21.15.116:0/1646271703 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7340b94e0 0x7ff734160750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:44.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 -- 172.21.15.116:0/1646271703 >> 172.21.15.116:0/1646271703 conn(0x7ff734060c40 msgr2=0x7ff73414ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:44.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 -- 172.21.15.116:0/1646271703 shutdown_connections 2020-11-20T12:20:44.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:44.824+0000 7ff738dd4700 1 -- 172.21.15.116:0/1646271703 wait complete. 2020-11-20T12:20:44.823 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 54 2020-11-20T12:20:45.247 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:20:34.449524+0000', 'modified': '2020-11-20T12:20:36.475264+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7311}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6835/2361610151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2361610151}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2361610151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:20:45.247 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:20:45.248 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-20T12:20:45.441 INFO:teuthology.orchestra.run.smithi173.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":7311,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":10.975043583,"mdsmap_epoch":54,"osdmap_epoch":96,"osdmap_epoch_barrier":96,"uptime":16.494656892999998} 2020-11-20T12:20:45.441 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 7311, "mdsmap_epoch": 54, "osdmap_epoch": 96, "osdmap_epoch_barrier": 96, "rank_uptime": 10.975043583, "state": "up:active", "uptime": 16.494656893, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:20:45.442 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:20:45.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:45.443 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-20T12:20:45.457 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:20:45.458 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:45.529 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:45.529 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:20:45.552 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:45.553 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:20:45.559 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:20:45.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:45.560 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-20T12:20:45.590 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:20:45.590 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:20:45.591 INFO:tasks.cephfs.mount:self.client.name = client.2 2020-11-20T12:20:45.591 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2020-11-20T12:20:45.592 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:20:45.592 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:20:45.593 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:20:45.593 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:20:45.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:45.594 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:20:45.613 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:20:45.613 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:20:45.614 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:20:45.614 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:45.614 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:20:45.614 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:45.615 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:45.615 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:45.615 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:45.615 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:45.616 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:45.616 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:45.616 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:20:45.616 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:45.617 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:20:45.617 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:45.617 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:20:45.617 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:45.618 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:20:45.618 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:45.618 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:20:45.618 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:45.618 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:20:45.619 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:45.619 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:20:45.619 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:45.619 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:20:45.620 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:45.620 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:45.620 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:20:45.620 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:20:45.620 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:45.621 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:45.621 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:20:45.621 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:20:45.621 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:45.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:45.622 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:20:45.688 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:20:45.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:45.692 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:20:45.709 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:20:45.710 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:20:45.710 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:20:45.711 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:20:45.711 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:20:45.711 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:20:45.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:45.713 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:20:45.714 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:20:45.714 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:20:45.715 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:20:45.715 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:20:45.715 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:20:45.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:45.812 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:20:45.826 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:20:45.826 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:20:45.827 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:20:45.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:45.828 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2020-11-20T12:20:45.857 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:46.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.084+0000 7f522359e700 1 Processor -- start 2020-11-20T12:20:46.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.084+0000 7f522359e700 1 -- start start 2020-11-20T12:20:46.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.084+0000 7f522359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5224148040 0x7f5224148420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.084+0000 7f522359e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5224067190 con 0x7f5224066480 2020-11-20T12:20:46.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.084+0000 7f522359e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52240677a0 con 0x7f5224068420 2020-11-20T12:20:46.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.084+0000 7f522359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5224067a40 con 0x7f5224148040 2020-11-20T12:20:46.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f5222d9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5224066480 0x7f5224066930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59390/0 (socket says 172.21.15.116:59390) 2020-11-20T12:20:46.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f5222d9d700 1 -- 172.21.15.116:0/1751419328 learned_addr learned my addr 172.21.15.116:0/1751419328 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:46.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522259c700 1 --2- 172.21.15.116:0/1751419328 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5224148040 0x7f5224148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522159a700 1 -- 172.21.15.116:0/1751419328 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2789066507 0 0) 0x7f52240677a0 con 0x7f5224068420 2020-11-20T12:20:46.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522159a700 1 -- 172.21.15.116:0/1751419328 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5208003240 con 0x7f5224068420 2020-11-20T12:20:46.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522159a700 1 -- 172.21.15.116:0/1751419328 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2124950562 0 0) 0x7f5224067190 con 0x7f5224066480 2020-11-20T12:20:46.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522159a700 1 -- 172.21.15.116:0/1751419328 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52240677a0 con 0x7f5224066480 2020-11-20T12:20:46.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522259c700 1 -- 172.21.15.116:0/1751419328 >> v1:172.21.15.173:6789/0 conn(0x7f5224068420 legacy=0x7f52240626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522259c700 1 -- 172.21.15.116:0/1751419328 >> v1:172.21.15.116:6789/0 conn(0x7f5224066480 legacy=0x7f5224066930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.086 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522259c700 1 -- 172.21.15.116:0/1751419328 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5224067ce0 con 0x7f5224148040 2020-11-20T12:20:46.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522259c700 1 --2- 172.21.15.116:0/1751419328 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5224148040 0x7f5224148420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f521400b9d0 tx=0x7f5214006df0).ready entity=mon.2 client_cookie=50cba84311fa45d7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:46.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522159a700 1 -- 172.21.15.116:0/1751419328 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214009680 con 0x7f5224148040 2020-11-20T12:20:46.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522159a700 1 -- 172.21.15.116:0/1751419328 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52140097e0 con 0x7f5224148040 2020-11-20T12:20:46.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.088+0000 7f522159a700 1 -- 172.21.15.116:0/1751419328 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5214009b60 con 0x7f5224148040 2020-11-20T12:20:46.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.092+0000 7f522359e700 1 -- 172.21.15.116:0/1751419328 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5224148040 msgr2=0x7f5224148420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.088 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.092+0000 7f522359e700 1 --2- 172.21.15.116:0/1751419328 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5224148040 0x7f5224148420 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f521400b9d0 tx=0x7f5214006df0).stop 2020-11-20T12:20:46.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.092+0000 7f522359e700 1 -- 172.21.15.116:0/1751419328 shutdown_connections 2020-11-20T12:20:46.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.092+0000 7f522359e700 1 --2- 172.21.15.116:0/1751419328 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5224148040 0x7f5224148420 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.092+0000 7f522359e700 1 -- 172.21.15.116:0/1751419328 >> 172.21.15.116:0/1751419328 conn(0x7f5224060c40 msgr2=0x7f52240c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:46.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.092+0000 7f522359e700 1 -- 172.21.15.116:0/1751419328 shutdown_connections 2020-11-20T12:20:46.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.092+0000 7f522359e700 1 -- 172.21.15.116:0/1751419328 wait complete. 2020-11-20T12:20:46.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.092+0000 7f522359e700 1 Processor -- start 2020-11-20T12:20:46.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.092+0000 7f522359e700 1 -- start start 2020-11-20T12:20:46.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f522359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5224066480 0x7f52241619f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f522359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5224068420 0x7f52240a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f522359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5224148040 0x7f522413fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f522359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5224161ee0 con 0x7f5224148040 2020-11-20T12:20:46.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f522359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52240a4c10 con 0x7f5224068420 2020-11-20T12:20:46.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f522359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52240a4eb0 con 0x7f5224066480 2020-11-20T12:20:46.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f5222d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5224148040 0x7f522413fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f5222d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5224148040 0x7f522413fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49962/0 (socket says 172.21.15.116:49962) 2020-11-20T12:20:46.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f5222d9d700 1 -- 172.21.15.116:0/569086802 learned_addr learned my addr 172.21.15.116:0/569086802 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:46.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f522259c700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5224066480 0x7f52241619f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f5221d9b700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5224068420 0x7f52240a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f5222d9d700 1 -- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5224066480 msgr2=0x7f52241619f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f5222d9d700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5224066480 0x7f52241619f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f5222d9d700 1 -- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5224068420 msgr2=0x7f52240a74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f5222d9d700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5224068420 0x7f52240a74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f5222d9d700 1 -- 172.21.15.116:0/569086802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52240a5150 con 0x7f5224148040 2020-11-20T12:20:46.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f5222d9d700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5224148040 0x7f522413fd70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f521c00ca60 tx=0x7f521c007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:46.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f52127fc700 1 -- 172.21.15.116:0/569086802 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c005b80 con 0x7f5224148040 2020-11-20T12:20:46.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f52127fc700 1 -- 172.21.15.116:0/569086802 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521c005ce0 con 0x7f5224148040 2020-11-20T12:20:46.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f52127fc700 1 -- 172.21.15.116:0/569086802 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521c007320 con 0x7f5224148040 2020-11-20T12:20:46.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f522359e700 1 -- 172.21.15.116:0/569086802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52240a53f0 con 0x7f5224148040 2020-11-20T12:20:46.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.096+0000 7f522359e700 1 -- 172.21.15.116:0/569086802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52240a7ff0 con 0x7f5224148040 2020-11-20T12:20:46.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.100+0000 7f522359e700 1 -- 172.21.15.116:0/569086802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f522405cf30 con 0x7f5224148040 2020-11-20T12:20:46.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.100+0000 7f52127fc700 1 -- 172.21.15.116:0/569086802 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f521c010030 con 0x7f5224148040 2020-11-20T12:20:46.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.100+0000 7f52127fc700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51f80557c0 0x7f51f8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.100+0000 7f52127fc700 1 -- 172.21.15.116:0/569086802 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f521c06e6b0 con 0x7f5224148040 2020-11-20T12:20:46.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.104+0000 7f522259c700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51f80557c0 0x7f51f8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.104+0000 7f522259c700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51f80557c0 0x7f51f8057c80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f521400b150 tx=0x7f5214015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:46.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.108+0000 7f52127fc700 1 -- 172.21.15.116:0/569086802 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f521c044550 con 0x7f5224148040 2020-11-20T12:20:46.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.268+0000 7f522359e700 1 -- 172.21.15.116:0/569086802 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52240be950 con 0x7f5224148040 2020-11-20T12:20:46.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.272+0000 7f52127fc700 1 -- 172.21.15.116:0/569086802 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f521c047b70 con 0x7f5224148040 2020-11-20T12:20:46.270 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:46.270 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":96,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:20:35.459061+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:20:32.886768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-20T12:20:33.446960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:46.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.272+0000 7f522359e700 1 -- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51f80557c0 msgr2=0x7f51f8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.272+0000 7f522359e700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51f80557c0 0x7f51f8057c80 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f521400b150 tx=0x7f5214015000).stop 2020-11-20T12:20:46.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.272+0000 7f522359e700 1 -- 172.21.15.116:0/569086802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5224148040 msgr2=0x7f522413fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.272+0000 7f522359e700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5224148040 0x7f522413fd70 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7f521c00ca60 tx=0x7f521c007e50).stop 2020-11-20T12:20:46.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.276+0000 7f522359e700 1 -- 172.21.15.116:0/569086802 shutdown_connections 2020-11-20T12:20:46.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.276+0000 7f522359e700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f51f80557c0 0x7f51f8057c80 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.276+0000 7f522359e700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5224148040 0x7f522413fd70 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.276+0000 7f522359e700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5224066480 0x7f52241619f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.276+0000 7f522359e700 1 --2- 172.21.15.116:0/569086802 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5224068420 0x7f52240a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.276+0000 7f522359e700 1 -- 172.21.15.116:0/569086802 >> 172.21.15.116:0/569086802 conn(0x7f5224060c40 msgr2=0x7f52240c3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:46.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.276+0000 7f522359e700 1 -- 172.21.15.116:0/569086802 shutdown_connections 2020-11-20T12:20:46.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.276+0000 7f522359e700 1 -- 172.21.15.116:0/569086802 wait complete. 2020-11-20T12:20:46.293 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:20:32.886768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:20:33.446960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:20:46.294 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:20:46.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.476+0000 7feedf5f8700 1 Processor -- start 2020-11-20T12:20:46.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.476+0000 7feedf5f8700 1 -- start start 2020-11-20T12:20:46.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.476+0000 7feedf5f8700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feed80a8ab0 0x7feed80a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.476+0000 7feedf5f8700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7feed80a6bd0 0x7feed814f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.476+0000 7feedf5f8700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed80a6fd0 con 0x7feed8148cf0 2020-11-20T12:20:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.476+0000 7feedf5f8700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed80a8f60 con 0x7feed80a8ab0 2020-11-20T12:20:46.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.476+0000 7feedf5f8700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed8152710 con 0x7feed80a6bd0 2020-11-20T12:20:46.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.476+0000 7feedcb93700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feed80a8ab0 0x7feed80a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.476+0000 7feedd394700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7feed8148cf0 0x7feed80c4bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59400/0 (socket says 172.21.15.116:59400) 2020-11-20T12:20:46.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.476+0000 7feedd394700 1 -- 172.21.15.116:0/1111240250 learned_addr learned my addr 172.21.15.116:0/1111240250 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:46.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feeddb95700 1 --2- 172.21.15.116:0/1111240250 >> v2:172.21.15.173:3300/0 conn(0x7feed80a6bd0 0x7feed814f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feedcb93700 1 -- 172.21.15.116:0/1111240250 >> v2:172.21.15.173:3300/0 conn(0x7feed80a6bd0 msgr2=0x7feed814f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:46.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feedcb93700 1 --2- 172.21.15.116:0/1111240250 >> v2:172.21.15.173:3300/0 conn(0x7feed80a6bd0 0x7feed814f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feedcb93700 1 -- 172.21.15.116:0/1111240250 >> v1:172.21.15.116:6789/0 conn(0x7feed8148cf0 legacy=0x7feed80c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feedcb93700 1 -- 172.21.15.116:0/1111240250 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed81529b0 con 0x7feed80a8ab0 2020-11-20T12:20:46.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feeddb95700 1 --2- 172.21.15.116:0/1111240250 >> v2:172.21.15.173:3300/0 conn(0x7feed80a6bd0 0x7feed814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:46.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feedcb93700 1 --2- 172.21.15.116:0/1111240250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed80a8ab0 0x7feed80a66e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7feed805d100 tx=0x7feec8006df0).ready entity=mon.0 client_cookie=357b2a33fc338930 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:46.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feec7fff700 1 -- 172.21.15.116:0/1111240250 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec80097b0 con 0x7feed80a8ab0 2020-11-20T12:20:46.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feec7fff700 1 -- 172.21.15.116:0/1111240250 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec8009910 con 0x7feed80a8ab0 2020-11-20T12:20:46.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feec7fff700 1 -- 172.21.15.116:0/1111240250 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8009c90 con 0x7feed80a8ab0 2020-11-20T12:20:46.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feedf5f8700 1 -- 172.21.15.116:0/1111240250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed80a8ab0 msgr2=0x7feed80a66e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.480+0000 7feedf5f8700 1 --2- 172.21.15.116:0/1111240250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed80a8ab0 0x7feed80a66e0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7feed805d100 tx=0x7feec8006df0).stop 2020-11-20T12:20:46.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.484+0000 7feedf5f8700 1 -- 172.21.15.116:0/1111240250 shutdown_connections 2020-11-20T12:20:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.484+0000 7feedf5f8700 1 --2- 172.21.15.116:0/1111240250 >> v2:172.21.15.173:3300/0 conn(0x7feed80a6bd0 0x7feed814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.484+0000 7feedf5f8700 1 --2- 172.21.15.116:0/1111240250 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed80a8ab0 0x7feed80a66e0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.484+0000 7feedf5f8700 1 -- 172.21.15.116:0/1111240250 >> 172.21.15.116:0/1111240250 conn(0x7feed805ba80 msgr2=0x7feed80c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:46.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.484+0000 7feedf5f8700 1 -- 172.21.15.116:0/1111240250 shutdown_connections 2020-11-20T12:20:46.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.484+0000 7feedf5f8700 1 -- 172.21.15.116:0/1111240250 wait complete. 2020-11-20T12:20:46.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.484+0000 7feedf5f8700 1 Processor -- start 2020-11-20T12:20:46.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.484+0000 7feedf5f8700 1 -- start start 2020-11-20T12:20:46.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedf5f8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feed80a6bd0 0x7feed80bfdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedf5f8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed80a8ab0 0x7feed8161260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedf5f8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed8148cf0 0x7feed80bef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedf5f8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8161be0 con 0x7feed80a8ab0 2020-11-20T12:20:46.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedf5f8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8161e80 con 0x7feed8148cf0 2020-11-20T12:20:46.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedf5f8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed8162120 con 0x7feed80a6bd0 2020-11-20T12:20:46.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedcb93700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed80a8ab0 0x7feed8161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedcb93700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed80a8ab0 0x7feed8161260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49974/0 (socket says 172.21.15.116:49974) 2020-11-20T12:20:46.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedcb93700 1 -- 172.21.15.116:0/1248302301 learned_addr learned my addr 172.21.15.116:0/1248302301 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:46.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feeddb95700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed8148cf0 0x7feed80bef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedd394700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feed80a6bd0 0x7feed80bfdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedcb93700 1 -- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feed80a6bd0 msgr2=0x7feed80bfdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedcb93700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feed80a6bd0 0x7feed80bfdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedcb93700 1 -- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed8148cf0 msgr2=0x7feed80bef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedcb93700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed8148cf0 0x7feed80bef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedcb93700 1 -- 172.21.15.116:0/1248302301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed8156b90 con 0x7feed80a8ab0 2020-11-20T12:20:46.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedcb93700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed80a8ab0 0x7feed8161260 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7feec800c010 tx=0x7feec8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:46.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feec57fa700 1 -- 172.21.15.116:0/1248302301 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec801e460 con 0x7feed80a8ab0 2020-11-20T12:20:46.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feec57fa700 1 -- 172.21.15.116:0/1248302301 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec801e5c0 con 0x7feed80a8ab0 2020-11-20T12:20:46.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feec57fa700 1 -- 172.21.15.116:0/1248302301 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec801e940 con 0x7feed80a8ab0 2020-11-20T12:20:46.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedf5f8700 1 -- 172.21.15.116:0/1248302301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed8156810 con 0x7feed80a8ab0 2020-11-20T12:20:46.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.488+0000 7feedf5f8700 1 -- 172.21.15.116:0/1248302301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed8161750 con 0x7feed80a8ab0 2020-11-20T12:20:46.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.492+0000 7feedf5f8700 1 -- 172.21.15.116:0/1248302301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed81569a0 con 0x7feed80a8ab0 2020-11-20T12:20:46.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.492+0000 7feec57fa700 1 -- 172.21.15.116:0/1248302301 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7feec8025030 con 0x7feed80a8ab0 2020-11-20T12:20:46.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.492+0000 7feec57fa700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feeb00557c0 0x7feeb0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.492+0000 7feec57fa700 1 -- 172.21.15.116:0/1248302301 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feec806fd60 con 0x7feed80a8ab0 2020-11-20T12:20:46.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.492+0000 7feedd394700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feeb00557c0 0x7feeb0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.496+0000 7feedd394700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feeb00557c0 0x7feeb0057c80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7feed4000bc0 tx=0x7feed4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:46.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.496+0000 7feec57fa700 1 -- 172.21.15.116:0/1248302301 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7feec8045cb0 con 0x7feed80a8ab0 2020-11-20T12:20:46.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.660+0000 7feedf5f8700 1 -- 172.21.15.116:0/1248302301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7feed8156d20 con 0x7feed80a8ab0 2020-11-20T12:20:46.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.660+0000 7feec57fa700 1 -- 172.21.15.116:0/1248302301 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v96) v1 ==== 110+0+10 (secure 0 0 0) 0x7feec8023070 con 0x7feed80a8ab0 2020-11-20T12:20:46.657 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:20:46.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 -- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feeb00557c0 msgr2=0x7feeb0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feeb00557c0 0x7feeb0057c80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7feed4000bc0 tx=0x7feed4009040).stop 2020-11-20T12:20:46.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 -- 172.21.15.116:0/1248302301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed80a8ab0 msgr2=0x7feed8161260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed80a8ab0 0x7feed8161260 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7feec800c010 tx=0x7feec8015000).stop 2020-11-20T12:20:46.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 -- 172.21.15.116:0/1248302301 shutdown_connections 2020-11-20T12:20:46.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feeb00557c0 0x7feeb0057c80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feed8148cf0 0x7feed80bef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feed80a6bd0 0x7feed80bfdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 --2- 172.21.15.116:0/1248302301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feed80a8ab0 0x7feed8161260 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 -- 172.21.15.116:0/1248302301 >> 172.21.15.116:0/1248302301 conn(0x7feed805ba80 msgr2=0x7feed814dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:46.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 -- 172.21.15.116:0/1248302301 shutdown_connections 2020-11-20T12:20:46.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.664+0000 7feedf5f8700 1 -- 172.21.15.116:0/1248302301 wait complete. 2020-11-20T12:20:46.682 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:20:46.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.864+0000 7fea8cb56700 1 Processor -- start 2020-11-20T12:20:46.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.864+0000 7fea8cb56700 1 -- start start 2020-11-20T12:20:46.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.864+0000 7fea8cb56700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea88068500 0x7fea880a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.864+0000 7fea8cb56700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fea88151630 0x7fea88151a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.864+0000 7fea8cb56700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea880c49a0 con 0x7fea881558c0 2020-11-20T12:20:46.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8cb56700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea880c4700 con 0x7fea88151630 2020-11-20T12:20:46.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8cb56700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea880c9140 con 0x7fea88068500 2020-11-20T12:20:46.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8659c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fea881558c0 0x7fea88155ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59414/0 (socket says 172.21.15.116:59414) 2020-11-20T12:20:46.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8659c700 1 -- 172.21.15.116:0/2912149698 learned_addr learned my addr 172.21.15.116:0/2912149698 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:46.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea86d9d700 1 --2- 172.21.15.116:0/2912149698 >> v2:172.21.15.116:3300/0 conn(0x7fea88151630 0x7fea88151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea85d9b700 1 --2- 172.21.15.116:0/2912149698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea88068500 0x7fea880a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8559a700 1 -- 172.21.15.116:0/2912149698 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535286873 0 0) 0x7fea880c49a0 con 0x7fea881558c0 2020-11-20T12:20:46.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8559a700 1 -- 172.21.15.116:0/2912149698 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea6c003240 con 0x7fea881558c0 2020-11-20T12:20:46.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8559a700 1 -- 172.21.15.116:0/2912149698 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2567477238 0 0) 0x7fea6c003240 con 0x7fea881558c0 2020-11-20T12:20:46.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea86d9d700 1 -- 172.21.15.116:0/2912149698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea88068500 msgr2=0x7fea880a8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:46.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea86d9d700 1 --2- 172.21.15.116:0/2912149698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea88068500 0x7fea880a8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea86d9d700 1 -- 172.21.15.116:0/2912149698 >> v1:172.21.15.116:6789/0 conn(0x7fea881558c0 legacy=0x7fea88155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea86d9d700 1 -- 172.21.15.116:0/2912149698 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea880c93e0 con 0x7fea88151630 2020-11-20T12:20:46.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea86d9d700 1 --2- 172.21.15.116:0/2912149698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea88151630 0x7fea88151a80 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fea80009c90 tx=0x7fea800091d0).ready entity=mon.0 client_cookie=3ef36c3142bb5eae server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:46.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8559a700 1 -- 172.21.15.116:0/2912149698 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea780014e0 con 0x7fea88151630 2020-11-20T12:20:46.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8559a700 1 -- 172.21.15.116:0/2912149698 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8000b6b0 con 0x7fea88151630 2020-11-20T12:20:46.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8559a700 1 -- 172.21.15.116:0/2912149698 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8000ba30 con 0x7fea88151630 2020-11-20T12:20:46.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8cb56700 1 -- 172.21.15.116:0/2912149698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea88151630 msgr2=0x7fea88151a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8cb56700 1 --2- 172.21.15.116:0/2912149698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea88151630 0x7fea88151a80 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7fea80009c90 tx=0x7fea800091d0).stop 2020-11-20T12:20:46.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8cb56700 1 -- 172.21.15.116:0/2912149698 shutdown_connections 2020-11-20T12:20:46.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.868+0000 7fea8cb56700 1 --2- 172.21.15.116:0/2912149698 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea88151630 0x7fea88151a80 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 --2- 172.21.15.116:0/2912149698 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea88068500 0x7fea880a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 -- 172.21.15.116:0/2912149698 >> 172.21.15.116:0/2912149698 conn(0x7fea88068080 msgr2=0x7fea880c8920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:46.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 -- 172.21.15.116:0/2912149698 shutdown_connections 2020-11-20T12:20:46.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 -- 172.21.15.116:0/2912149698 wait complete. 2020-11-20T12:20:46.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 Processor -- start 2020-11-20T12:20:46.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 -- start start 2020-11-20T12:20:46.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea88068500 0x7fea88165070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea88151630 0x7fea88161000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea881558c0 0x7fea88095fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea880926d0 con 0x7fea88151630 2020-11-20T12:20:46.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea881617f0 con 0x7fea88068500 2020-11-20T12:20:46.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea8cb56700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea88161a90 con 0x7fea881558c0 2020-11-20T12:20:46.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.872+0000 7fea85d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea88151630 0x7fea88161000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea8659c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea88068500 0x7fea88165070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea85d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea88151630 0x7fea88161000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49990/0 (socket says 172.21.15.116:49990) 2020-11-20T12:20:46.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea86d9d700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea881558c0 0x7fea88095fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea85d9b700 1 -- 172.21.15.116:0/1129147267 learned_addr learned my addr 172.21.15.116:0/1129147267 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:46.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea85d9b700 1 -- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea881558c0 msgr2=0x7fea88095fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea85d9b700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea881558c0 0x7fea88095fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea85d9b700 1 -- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea88068500 msgr2=0x7fea88165070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:46.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea85d9b700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea88068500 0x7fea88165070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:46.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea85d9b700 1 -- 172.21.15.116:0/1129147267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea88161d30 con 0x7fea88151630 2020-11-20T12:20:46.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea85d9b700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea88151630 0x7fea88161000 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7fea7c00ebd0 tx=0x7fea7c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:46.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea767fc700 1 -- 172.21.15.116:0/1129147267 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c00b810 con 0x7fea88151630 2020-11-20T12:20:46.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea767fc700 1 -- 172.21.15.116:0/1129147267 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea7c00b970 con 0x7fea88151630 2020-11-20T12:20:46.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea8cb56700 1 -- 172.21.15.116:0/1129147267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea880928c0 con 0x7fea88151630 2020-11-20T12:20:46.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea767fc700 1 -- 172.21.15.116:0/1129147267 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea7c00bcf0 con 0x7fea88151630 2020-11-20T12:20:46.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea8cb56700 1 -- 172.21.15.116:0/1129147267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea880964a0 con 0x7fea88151630 2020-11-20T12:20:46.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.876+0000 7fea8cb56700 1 -- 172.21.15.116:0/1129147267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea88092a50 con 0x7fea88151630 2020-11-20T12:20:46.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.880+0000 7fea767fc700 1 -- 172.21.15.116:0/1129147267 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fea7c012600 con 0x7fea88151630 2020-11-20T12:20:46.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.880+0000 7fea767fc700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea5c0557c0 0x7fea5c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:46.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.880+0000 7fea767fc700 1 -- 172.21.15.116:0/1129147267 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fea7c06e900 con 0x7fea88151630 2020-11-20T12:20:46.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.880+0000 7fea8659c700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea5c0557c0 0x7fea5c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:46.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.880+0000 7fea8659c700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea5c0557c0 0x7fea5c057c80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fea780013f0 tx=0x7fea78001a40).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:46.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:46.888+0000 7fea767fc700 1 -- 172.21.15.116:0/1129147267 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fea7c044740 con 0x7fea88151630 2020-11-20T12:20:47.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.044+0000 7fea8cb56700 1 -- 172.21.15.116:0/1129147267 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fea8805d2a0 con 0x7fea88151630 2020-11-20T12:20:47.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.044+0000 7fea767fc700 1 -- 172.21.15.116:0/1129147267 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v96) v1 ==== 92+0+10 (secure 0 0 0) 0x7fea7c047d60 con 0x7fea88151630 2020-11-20T12:20:47.042 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:47.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 -- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea5c0557c0 msgr2=0x7fea5c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea5c0557c0 0x7fea5c057c80 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fea780013f0 tx=0x7fea78001a40).stop 2020-11-20T12:20:47.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 -- 172.21.15.116:0/1129147267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea88151630 msgr2=0x7fea88161000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea88151630 0x7fea88161000 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7fea7c00ebd0 tx=0x7fea7c006df0).stop 2020-11-20T12:20:47.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 -- 172.21.15.116:0/1129147267 shutdown_connections 2020-11-20T12:20:47.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea5c0557c0 0x7fea5c057c80 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea881558c0 0x7fea88095fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea88068500 0x7fea88165070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 --2- 172.21.15.116:0/1129147267 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea88151630 0x7fea88161000 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 -- 172.21.15.116:0/1129147267 >> 172.21.15.116:0/1129147267 conn(0x7fea88068080 msgr2=0x7fea880620e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:47.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 -- 172.21.15.116:0/1129147267 shutdown_connections 2020-11-20T12:20:47.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.048+0000 7fea8cb56700 1 -- 172.21.15.116:0/1129147267 wait complete. 2020-11-20T12:20:47.067 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:20:47.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.248+0000 7f71559bd700 1 Processor -- start 2020-11-20T12:20:47.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.248+0000 7f71559bd700 1 -- start start 2020-11-20T12:20:47.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.252+0000 7f71559bd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f71500aa1a0 0x7f71500c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.252+0000 7f71559bd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f71500c2de0 0x7f71500a8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.252+0000 7f71559bd700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7150151cf0 con 0x7f71500aa580 2020-11-20T12:20:47.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.252+0000 7f71559bd700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71500c7e00 con 0x7f71500aa1a0 2020-11-20T12:20:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.252+0000 7f71559bd700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71500c80a0 con 0x7f71500c2de0 2020-11-20T12:20:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.252+0000 7f71549bb700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f71500aa1a0 0x7f71500c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.252+0000 7f71549bb700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f71500aa1a0 0x7f71500c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:49994/0 (socket says 172.21.15.116:49994) 2020-11-20T12:20:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.252+0000 7f71549bb700 1 -- 172.21.15.116:0/3975497219 learned_addr learned my addr 172.21.15.116:0/3975497219 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.252+0000 7f71551bc700 1 --2- 172.21.15.116:0/3975497219 >> v2:172.21.15.173:3300/0 conn(0x7f71500c2de0 0x7f71500a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71549bb700 1 -- 172.21.15.116:0/3975497219 >> v2:172.21.15.173:3300/0 conn(0x7f71500c2de0 msgr2=0x7f71500a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71549bb700 1 --2- 172.21.15.116:0/3975497219 >> v2:172.21.15.173:3300/0 conn(0x7f71500c2de0 0x7f71500a8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71549bb700 1 -- 172.21.15.116:0/3975497219 >> v1:172.21.15.173:6789/0 conn(0x7f71500aa580 legacy=0x7f7150061460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f714f7fe700 1 -- 172.21.15.116:0/3975497219 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2987717497 0 0) 0x7f7150151cf0 con 0x7f71500aa580 2020-11-20T12:20:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71549bb700 1 -- 172.21.15.116:0/3975497219 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71500c34f0 con 0x7f71500aa1a0 2020-11-20T12:20:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71549bb700 1 --2- 172.21.15.116:0/3975497219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71500aa1a0 0x7f71500c37d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7f71500c4b60 tx=0x7f7144006eb0).ready entity=mon.0 client_cookie=b5df7d196fa0fd9e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f714f7fe700 1 -- 172.21.15.116:0/3975497219 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7144009810 con 0x7f71500aa1a0 2020-11-20T12:20:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f714f7fe700 1 -- 172.21.15.116:0/3975497219 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7144009970 con 0x7f71500aa1a0 2020-11-20T12:20:47.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f714f7fe700 1 -- 172.21.15.116:0/3975497219 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7144009cf0 con 0x7f71500aa1a0 2020-11-20T12:20:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71559bd700 1 -- 172.21.15.116:0/3975497219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71500aa1a0 msgr2=0x7f71500c37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71559bd700 1 --2- 172.21.15.116:0/3975497219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71500aa1a0 0x7f71500c37d0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7f71500c4b60 tx=0x7f7144006eb0).stop 2020-11-20T12:20:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71559bd700 1 -- 172.21.15.116:0/3975497219 shutdown_connections 2020-11-20T12:20:47.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71559bd700 1 --2- 172.21.15.116:0/3975497219 >> v2:172.21.15.173:3300/0 conn(0x7f71500c2de0 0x7f71500a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71559bd700 1 --2- 172.21.15.116:0/3975497219 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71500aa1a0 0x7f71500c37d0 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71559bd700 1 -- 172.21.15.116:0/3975497219 >> 172.21.15.116:0/3975497219 conn(0x7f7150068370 msgr2=0x7f71500c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71559bd700 1 -- 172.21.15.116:0/3975497219 shutdown_connections 2020-11-20T12:20:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.256+0000 7f71559bd700 1 -- 172.21.15.116:0/3975497219 wait complete. 2020-11-20T12:20:47.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71559bd700 1 Processor -- start 2020-11-20T12:20:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71559bd700 1 -- start start 2020-11-20T12:20:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71559bd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71500aa580 0x7f7150154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71559bd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71500c2de0 0x7f71501612b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71559bd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71500bf860 0x7f7150061510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71559bd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71501622b0 con 0x7f71500bf860 2020-11-20T12:20:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71559bd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7150161ad0 con 0x7f71500aa580 2020-11-20T12:20:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71559bd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7150161d70 con 0x7f71500c2de0 2020-11-20T12:20:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71551bc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71500bf860 0x7f7150061510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71551bc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71500bf860 0x7f7150061510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50004/0 (socket says 172.21.15.116:50004) 2020-11-20T12:20:47.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71551bc700 1 -- 172.21.15.116:0/2154164460 learned_addr learned my addr 172.21.15.116:0/2154164460 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f71549bb700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71500aa580 0x7f7150154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.260+0000 7f714ffff700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71500c2de0 0x7f71501612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f71549bb700 1 -- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71500c2de0 msgr2=0x7f71501612b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f71549bb700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71500c2de0 0x7f71501612b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f71549bb700 1 -- 172.21.15.116:0/2154164460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71500bf860 msgr2=0x7f7150061510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f71549bb700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71500bf860 0x7f7150061510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f71549bb700 1 -- 172.21.15.116:0/2154164460 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7150162010 con 0x7f71500aa580 2020-11-20T12:20:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f71549bb700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71500aa580 0x7f7150154430 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f7144012000 tx=0x7f7144015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f714cff9700 1 -- 172.21.15.116:0/2154164460 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7144011c80 con 0x7f71500aa580 2020-11-20T12:20:47.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f714cff9700 1 -- 172.21.15.116:0/2154164460 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7144011de0 con 0x7f71500aa580 2020-11-20T12:20:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f714cff9700 1 -- 172.21.15.116:0/2154164460 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714401e6b0 con 0x7f71500aa580 2020-11-20T12:20:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f71559bd700 1 -- 172.21.15.116:0/2154164460 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7150152dd0 con 0x7f71500aa580 2020-11-20T12:20:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f71559bd700 1 -- 172.21.15.116:0/2154164460 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7150061a00 con 0x7f71500aa580 2020-11-20T12:20:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.264+0000 7f71559bd700 1 -- 172.21.15.116:0/2154164460 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715005cf30 con 0x7f71500aa580 2020-11-20T12:20:47.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.268+0000 7f714cff9700 1 -- 172.21.15.116:0/2154164460 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f714401a040 con 0x7f71500aa580 2020-11-20T12:20:47.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.268+0000 7f714cff9700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71280557c0 0x7f7128057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.268+0000 7f714cff9700 1 -- 172.21.15.116:0/2154164460 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f714406f940 con 0x7f71500aa580 2020-11-20T12:20:47.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.268+0000 7f714ffff700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71280557c0 0x7f7128057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.268+0000 7f714ffff700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71280557c0 0x7f7128057c80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f713c007fd0 tx=0x7f713c007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:47.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.276+0000 7f714cff9700 1 -- 172.21.15.116:0/2154164460 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f71440457e0 con 0x7f71500aa580 2020-11-20T12:20:47.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.436+0000 7f71559bd700 1 -- 172.21.15.116:0/2154164460 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7150061bf0 con 0x7f71500aa580 2020-11-20T12:20:47.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.440+0000 7f714cff9700 1 -- 172.21.15.116:0/2154164460 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+10 (secure 0 0 0) 0x7f7144023020 con 0x7f71500aa580 2020-11-20T12:20:47.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.440+0000 7f71559bd700 1 -- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71280557c0 msgr2=0x7f7128057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.440+0000 7f71559bd700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71280557c0 0x7f7128057c80 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f713c007fd0 tx=0x7f713c007a00).stop 2020-11-20T12:20:47.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.440+0000 7f71559bd700 1 -- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71500aa580 msgr2=0x7f7150154430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.440+0000 7f71559bd700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71500aa580 0x7f7150154430 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f7144012000 tx=0x7f7144015000).stop 2020-11-20T12:20:47.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.440+0000 7f71559bd700 1 -- 172.21.15.116:0/2154164460 shutdown_connections 2020-11-20T12:20:47.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.440+0000 7f71559bd700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f71280557c0 0x7f7128057c80 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.440+0000 7f71559bd700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f71500bf860 0x7f7150061510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.440+0000 7f71559bd700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f71500aa580 0x7f7150154430 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.440+0000 7f71559bd700 1 --2- 172.21.15.116:0/2154164460 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f71500c2de0 0x7f71501612b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.440+0000 7f71559bd700 1 -- 172.21.15.116:0/2154164460 >> 172.21.15.116:0/2154164460 conn(0x7f7150068370 msgr2=0x7f71500c7420 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.444+0000 7f71559bd700 1 -- 172.21.15.116:0/2154164460 shutdown_connections 2020-11-20T12:20:47.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.444+0000 7f71559bd700 1 -- 172.21.15.116:0/2154164460 wait complete. 2020-11-20T12:20:47.460 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:47.467 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:20:47.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.652+0000 7f77ddaaa700 1 Processor -- start 2020-11-20T12:20:47.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.652+0000 7f77ddaaa700 1 -- start start 2020-11-20T12:20:47.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77ddaaa700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77d80aa7d0 0x7f77d80a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77ddaaa700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77d80aabb0 0x7f77d8151e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77ddaaa700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f77d80ad540 0x7f77d80ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77ddaaa700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d80ace40 con 0x7f77d80aabb0 2020-11-20T12:20:47.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77ddaaa700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d81526c0 con 0x7f77d80ad540 2020-11-20T12:20:47.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77ddaaa700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8152960 con 0x7f77d80aa7d0 2020-11-20T12:20:47.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77d7fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77d80aabb0 0x7f77d8151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77dd2a9700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f77d80ad540 0x7f77d80ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77dd2a9700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f77d80ad540 0x7f77d80ad9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:49064/0 (socket says 172.21.15.116:49064) 2020-11-20T12:20:47.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77dd2a9700 1 -- 172.21.15.116:0/2253188729 learned_addr learned my addr 172.21.15.116:0/2253188729 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:47.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77d7fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77d80aabb0 0x7f77d8151e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50010/0 (socket says 172.21.15.116:50010) 2020-11-20T12:20:47.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77dcaa8700 1 --2- 172.21.15.116:0/2253188729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77d80aa7d0 0x7f77d80a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77dd2a9700 1 -- 172.21.15.116:0/2253188729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77d80aa7d0 msgr2=0x7f77d80a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:47.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77dd2a9700 1 --2- 172.21.15.116:0/2253188729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77d80aa7d0 0x7f77d80a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77dd2a9700 1 -- 172.21.15.116:0/2253188729 >> v2:172.21.15.116:3300/0 conn(0x7f77d80aabb0 msgr2=0x7f77d8151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:47.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77dd2a9700 1 --2- 172.21.15.116:0/2253188729 >> v2:172.21.15.116:3300/0 conn(0x7f77d80aabb0 0x7f77d8151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77dd2a9700 1 -- 172.21.15.116:0/2253188729 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d8152c00 con 0x7f77d80ad540 2020-11-20T12:20:47.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77d7fff700 1 --2- 172.21.15.116:0/2253188729 >> v2:172.21.15.116:3300/0 conn(0x7f77d80aabb0 0x7f77d8151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:47.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.656+0000 7f77dd2a9700 1 --2- 172.21.15.116:0/2253188729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77d80ad540 0x7f77d80ad9f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f77d8081d40 tx=0x7f77d0009290).ready entity=mon.1 client_cookie=66d44d32dce0d9d1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:47.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77d77fe700 1 -- 172.21.15.116:0/2253188729 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d000bb10 con 0x7f77d80ad540 2020-11-20T12:20:47.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77d77fe700 1 -- 172.21.15.116:0/2253188729 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77d000bc70 con 0x7f77d80ad540 2020-11-20T12:20:47.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77d77fe700 1 -- 172.21.15.116:0/2253188729 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d0011400 con 0x7f77d80ad540 2020-11-20T12:20:47.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2253188729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77d80ad540 msgr2=0x7f77d80ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77ddaaa700 1 --2- 172.21.15.116:0/2253188729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77d80ad540 0x7f77d80ad9f0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f77d8081d40 tx=0x7f77d0009290).stop 2020-11-20T12:20:47.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2253188729 shutdown_connections 2020-11-20T12:20:47.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77ddaaa700 1 --2- 172.21.15.116:0/2253188729 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77d80ad540 0x7f77d80ad9f0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77ddaaa700 1 --2- 172.21.15.116:0/2253188729 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77d80aa7d0 0x7f77d80a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77ddaaa700 1 --2- 172.21.15.116:0/2253188729 >> v2:172.21.15.116:3300/0 conn(0x7f77d80aabb0 0x7f77d8151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2253188729 >> 172.21.15.116:0/2253188729 conn(0x7f77d80678a0 msgr2=0x7f77d8067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:47.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2253188729 shutdown_connections 2020-11-20T12:20:47.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.660+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2253188729 wait complete. 2020-11-20T12:20:47.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.664+0000 7f77ddaaa700 1 Processor -- start 2020-11-20T12:20:47.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.664+0000 7f77ddaaa700 1 -- start start 2020-11-20T12:20:47.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.664+0000 7f77ddaaa700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77d80aa7d0 0x7f77d8149710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.664+0000 7f77ddaaa700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77d80aabb0 0x7f77d8066f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.664+0000 7f77ddaaa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77d80ad540 0x7f77d80613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.664+0000 7f77ddaaa700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8166200 con 0x7f77d80aa7d0 2020-11-20T12:20:47.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.664+0000 7f77ddaaa700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8165070 con 0x7f77d80ad540 2020-11-20T12:20:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.664+0000 7f77ddaaa700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d8149c00 con 0x7f77d80aabb0 2020-11-20T12:20:47.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77dcaa8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77d80aa7d0 0x7f77d8149710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77dcaa8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77d80aa7d0 0x7f77d8149710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50014/0 (socket says 172.21.15.116:50014) 2020-11-20T12:20:47.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77dcaa8700 1 -- 172.21.15.116:0/2836747287 learned_addr learned my addr 172.21.15.116:0/2836747287 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:47.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77d7fff700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77d80aabb0 0x7f77d8066f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77dd2a9700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77d80ad540 0x7f77d80613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77dcaa8700 1 -- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77d80aabb0 msgr2=0x7f77d8066f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77dcaa8700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77d80aabb0 0x7f77d8066f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77dcaa8700 1 -- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77d80ad540 msgr2=0x7f77d80613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77dcaa8700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77d80ad540 0x7f77d80613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77dcaa8700 1 -- 172.21.15.116:0/2836747287 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d81652c0 con 0x7f77d80aa7d0 2020-11-20T12:20:47.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77dd2a9700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77d80ad540 0x7f77d80613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:47.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77d7fff700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77d80aabb0 0x7f77d8066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:47.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77dcaa8700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77d80aa7d0 0x7f77d8149710 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7f77cc00ec00 tx=0x7f77cc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:47.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77d4ff9700 1 -- 172.21.15.116:0/2836747287 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00b9a0 con 0x7f77d80aa7d0 2020-11-20T12:20:47.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77d4ff9700 1 -- 172.21.15.116:0/2836747287 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc00bb00 con 0x7f77d80aa7d0 2020-11-20T12:20:47.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77d4ff9700 1 -- 172.21.15.116:0/2836747287 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00be80 con 0x7f77d80aa7d0 2020-11-20T12:20:47.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2836747287 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d8165560 con 0x7f77d80aa7d0 2020-11-20T12:20:47.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.668+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2836747287 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d8165aa0 con 0x7f77d80aa7d0 2020-11-20T12:20:47.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.672+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2836747287 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d805cf30 con 0x7f77d80aa7d0 2020-11-20T12:20:47.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.672+0000 7f77d4ff9700 1 -- 172.21.15.116:0/2836747287 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f77cc012600 con 0x7f77d80aa7d0 2020-11-20T12:20:47.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.672+0000 7f77d4ff9700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77b00557c0 0x7f77b0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:47.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.672+0000 7f77d4ff9700 1 -- 172.21.15.116:0/2836747287 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f77cc06eb40 con 0x7f77d80aa7d0 2020-11-20T12:20:47.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.672+0000 7f77d7fff700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77b00557c0 0x7f77b0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:47.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.676+0000 7f77d7fff700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77b00557c0 0x7f77b0057c80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f77d8061f40 tx=0x7f77c4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:47.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.676+0000 7f77d4ff9700 1 -- 172.21.15.116:0/2836747287 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f77cc044870 con 0x7f77d80aa7d0 2020-11-20T12:20:47.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.832+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2836747287 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77d80601b0 con 0x7f77d80aa7d0 2020-11-20T12:20:47.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.836+0000 7f77d4ff9700 1 -- 172.21.15.116:0/2836747287 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+10 (secure 0 0 0) 0x7f77cc047e90 con 0x7f77d80aa7d0 2020-11-20T12:20:47.832 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:47.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.836+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77b00557c0 msgr2=0x7f77b0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.836+0000 7f77ddaaa700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77b00557c0 0x7f77b0057c80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f77d8061f40 tx=0x7f77c4008040).stop 2020-11-20T12:20:47.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.836+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2836747287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77d80aa7d0 msgr2=0x7f77d8149710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:47.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.836+0000 7f77ddaaa700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77d80aa7d0 0x7f77d8149710 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7f77cc00ec00 tx=0x7f77cc006df0).stop 2020-11-20T12:20:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.836+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2836747287 shutdown_connections 2020-11-20T12:20:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.836+0000 7f77ddaaa700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77b00557c0 0x7f77b0057c80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.836+0000 7f77ddaaa700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77d80ad540 0x7f77d80613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.836+0000 7f77ddaaa700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77d80aa7d0 0x7f77d8149710 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.836+0000 7f77ddaaa700 1 --2- 172.21.15.116:0/2836747287 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77d80aabb0 0x7f77d8066f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.836+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2836747287 >> 172.21.15.116:0/2836747287 conn(0x7f77d80678a0 msgr2=0x7f77d80a94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.840+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2836747287 shutdown_connections 2020-11-20T12:20:47.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:47.840+0000 7f77ddaaa700 1 -- 172.21.15.116:0/2836747287 wait complete. 2020-11-20T12:20:47.857 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:20:47.857 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:48.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.028+0000 7fd90cd5d700 1 Processor -- start 2020-11-20T12:20:48.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.028+0000 7fd90cd5d700 1 -- start start 2020-11-20T12:20:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.028+0000 7fd90cd5d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd90814baa0 0x7fd90814be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.028+0000 7fd90cd5d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd908065e80 0x7fd908060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.028+0000 7fd90cd5d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd908152810 con 0x7fd908065a20 2020-11-20T12:20:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.028+0000 7fd90cd5d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd908152ab0 con 0x7fd90814baa0 2020-11-20T12:20:48.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.028+0000 7fd90cd5d700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9080652e0 con 0x7fd908065e80 2020-11-20T12:20:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd9077fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd90814baa0 0x7fd90814be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd906ffd700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fd908065a20 0x7fd908151d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:55686/0 (socket says 172.21.15.116:55686) 2020-11-20T12:20:48.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd906ffd700 1 -- 172.21.15.116:0/341035312 learned_addr learned my addr 172.21.15.116:0/341035312 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:48.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd907fff700 1 --2- 172.21.15.116:0/341035312 >> v2:172.21.15.173:3300/0 conn(0x7fd908065e80 0x7fd908060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd9067fc700 1 -- 172.21.15.116:0/341035312 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648235806 0 0) 0x7fd908152810 con 0x7fd908065a20 2020-11-20T12:20:48.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd9067fc700 1 -- 172.21.15.116:0/341035312 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8f4003240 con 0x7fd908065a20 2020-11-20T12:20:48.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd9077fe700 1 -- 172.21.15.116:0/341035312 >> v2:172.21.15.173:3300/0 conn(0x7fd908065e80 msgr2=0x7fd908060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:48.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd9077fe700 1 --2- 172.21.15.116:0/341035312 >> v2:172.21.15.173:3300/0 conn(0x7fd908065e80 0x7fd908060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd9077fe700 1 -- 172.21.15.116:0/341035312 >> v1:172.21.15.173:6789/0 conn(0x7fd908065a20 legacy=0x7fd908151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:48.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd9077fe700 1 -- 172.21.15.116:0/341035312 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd908065580 con 0x7fd90814baa0 2020-11-20T12:20:48.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd9077fe700 1 --2- 172.21.15.116:0/341035312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd90814baa0 0x7fd90814be80 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7fd908062240 tx=0x7fd8f8006df0).ready entity=mon.0 client_cookie=1a8713f34a41bbc0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:48.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd9067fc700 1 -- 172.21.15.116:0/341035312 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8009640 con 0x7fd90814baa0 2020-11-20T12:20:48.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd9067fc700 1 -- 172.21.15.116:0/341035312 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f80097a0 con 0x7fd90814baa0 2020-11-20T12:20:48.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd9067fc700 1 -- 172.21.15.116:0/341035312 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f8009b20 con 0x7fd90814baa0 2020-11-20T12:20:48.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd90cd5d700 1 -- 172.21.15.116:0/341035312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd90814baa0 msgr2=0x7fd90814be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:48.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd90cd5d700 1 --2- 172.21.15.116:0/341035312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd90814baa0 0x7fd90814be80 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7fd908062240 tx=0x7fd8f8006df0).stop 2020-11-20T12:20:48.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd90cd5d700 1 -- 172.21.15.116:0/341035312 shutdown_connections 2020-11-20T12:20:48.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd90cd5d700 1 --2- 172.21.15.116:0/341035312 >> v2:172.21.15.173:3300/0 conn(0x7fd908065e80 0x7fd908060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd90cd5d700 1 --2- 172.21.15.116:0/341035312 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd90814baa0 0x7fd90814be80 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd90cd5d700 1 -- 172.21.15.116:0/341035312 >> 172.21.15.116:0/341035312 conn(0x7fd908061280 msgr2=0x7fd908061660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:48.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd90cd5d700 1 -- 172.21.15.116:0/341035312 shutdown_connections 2020-11-20T12:20:48.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.032+0000 7fd90cd5d700 1 -- 172.21.15.116:0/341035312 wait complete. 2020-11-20T12:20:48.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.036+0000 7fd90cd5d700 1 Processor -- start 2020-11-20T12:20:48.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.036+0000 7fd90cd5d700 1 -- start start 2020-11-20T12:20:48.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd90cd5d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd908065a20 0x7fd908148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd90cd5d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd908065e80 0x7fd908155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd90cd5d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd908155600 0x7fd908164d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd90cd5d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd908157a30 con 0x7fd908065e80 2020-11-20T12:20:48.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd90cd5d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd908157cd0 con 0x7fd908155600 2020-11-20T12:20:48.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd90cd5d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd908157f70 con 0x7fd908065a20 2020-11-20T12:20:48.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd906ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd908065e80 0x7fd908155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd906ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd908065e80 0x7fd908155110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50030/0 (socket says 172.21.15.116:50030) 2020-11-20T12:20:48.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd906ffd700 1 -- 172.21.15.116:0/1008380689 learned_addr learned my addr 172.21.15.116:0/1008380689 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:48.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd9077fe700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd908065a20 0x7fd908148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd907fff700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd908155600 0x7fd908164d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd906ffd700 1 -- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd908065a20 msgr2=0x7fd908148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:48.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd906ffd700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd908065a20 0x7fd908148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd906ffd700 1 -- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd908155600 msgr2=0x7fd908164d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:48.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd906ffd700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd908155600 0x7fd908164d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd906ffd700 1 -- 172.21.15.116:0/1008380689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90814d3f0 con 0x7fd908065e80 2020-11-20T12:20:48.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd906ffd700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd908065e80 0x7fd908155110 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7fd8fc00c6a0 tx=0x7fd8fc007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:48.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd8e77fe700 1 -- 172.21.15.116:0/1008380689 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc0072e0 con 0x7fd908065e80 2020-11-20T12:20:48.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd8e77fe700 1 -- 172.21.15.116:0/1008380689 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8fc007440 con 0x7fd908065e80 2020-11-20T12:20:48.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd90cd5d700 1 -- 172.21.15.116:0/1008380689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90814d100 con 0x7fd908065e80 2020-11-20T12:20:48.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd8e77fe700 1 -- 172.21.15.116:0/1008380689 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc005820 con 0x7fd908065e80 2020-11-20T12:20:48.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.040+0000 7fd90cd5d700 1 -- 172.21.15.116:0/1008380689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd908148b10 con 0x7fd908065e80 2020-11-20T12:20:48.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.044+0000 7fd90cd5d700 1 -- 172.21.15.116:0/1008380689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90814d580 con 0x7fd908065e80 2020-11-20T12:20:48.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.044+0000 7fd8e77fe700 1 -- 172.21.15.116:0/1008380689 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd8fc010070 con 0x7fd908065e80 2020-11-20T12:20:48.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.044+0000 7fd8e77fe700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8e00557c0 0x7fd8e0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:48.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.044+0000 7fd8e77fe700 1 -- 172.21.15.116:0/1008380689 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd8fc06dca0 con 0x7fd908065e80 2020-11-20T12:20:48.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.048+0000 7fd9077fe700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8e00557c0 0x7fd8e0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:48.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.048+0000 7fd9077fe700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8e00557c0 0x7fd8e0057c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fd9080a4ef0 tx=0x7fd8f8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:48.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.052+0000 7fd8e77fe700 1 -- 172.21.15.116:0/1008380689 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd8fc072070 con 0x7fd908065e80 2020-11-20T12:20:48.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.264+0000 7fd90cd5d700 1 -- 172.21.15.116:0/1008380689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd90805cf30 con 0x7fd908065e80 2020-11-20T12:20:48.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd8e77fe700 1 -- 172.21.15.116:0/1008380689 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2848 (secure 0 0 0) 0x7fd8fc018020 con 0x7fd908065e80 2020-11-20T12:20:48.263 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:48.264 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3105911814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3105911814},{"type":"v1","addr":"172.21.15.173:6837","nonce":3105911814}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/3457547857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3457547857},{"type":"v1","addr":"172.21.15.116:6835","nonce":3457547857}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/81195738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":81195738},{"type":"v1","addr":"172.21.15.116:6833","nonce":81195738}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:20:34.449524+0000","modified":"2020-11-20T12:20:36.475264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/2361610151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2361610151},{"type":"v1","addr":"172.21.15.173:6835","nonce":2361610151}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-20T12:20:48.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 -- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8e00557c0 msgr2=0x7fd8e0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:48.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8e00557c0 0x7fd8e0057c80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fd9080a4ef0 tx=0x7fd8f8015000).stop 2020-11-20T12:20:48.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 -- 172.21.15.116:0/1008380689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd908065e80 msgr2=0x7fd908155110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:48.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd908065e80 0x7fd908155110 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7fd8fc00c6a0 tx=0x7fd8fc007ac0).stop 2020-11-20T12:20:48.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 -- 172.21.15.116:0/1008380689 shutdown_connections 2020-11-20T12:20:48.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8e00557c0 0x7fd8e0057c80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd908155600 0x7fd908164d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd908065a20 0x7fd908148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 --2- 172.21.15.116:0/1008380689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd908065e80 0x7fd908155110 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:48.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 -- 172.21.15.116:0/1008380689 >> 172.21.15.116:0/1008380689 conn(0x7fd908061280 msgr2=0x7fd908150160 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:48.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 -- 172.21.15.116:0/1008380689 shutdown_connections 2020-11-20T12:20:48.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:48.268+0000 7fd90cd5d700 1 -- 172.21.15.116:0/1008380689 wait complete. 2020-11-20T12:20:48.270 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 54 2020-11-20T12:20:48.693 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:20:34.449524+0000', 'modified': '2020-11-20T12:20:36.475264+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7311}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6835/2361610151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2361610151}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2361610151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:20:48.693 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:20:48.694 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-20T12:20:48.889 INFO:teuthology.orchestra.run.smithi173.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":7311,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.423384276,"mdsmap_epoch":54,"osdmap_epoch":96,"osdmap_epoch_barrier":96,"uptime":19.942994465000002} 2020-11-20T12:20:48.890 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 7311, "mdsmap_epoch": 54, "osdmap_epoch": 96, "osdmap_epoch_barrier": 96, "rank_uptime": 14.423384276, "state": "up:active", "uptime": 19.942994465, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:20:48.891 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:20:48.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:48.892 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.2 2020-11-20T12:20:48.916 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:20:48.917 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:49.046 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:49.046 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:20:49.073 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:49.074 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:20:49.078 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:20:49.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:49.079 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2020-11-20T12:20:49.120 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:20:49.121 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:20:49.122 INFO:tasks.cephfs.mount:self.client.name = client.3 2020-11-20T12:20:49.122 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2020-11-20T12:20:49.123 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:20:49.123 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:20:49.124 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:20:49.124 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:20:49.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:49.126 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:20:49.147 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:20:49.147 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:20:49.147 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:20:49.148 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:49.148 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:20:49.148 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:49.148 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:49.149 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:49.149 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:49.149 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:49.149 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:20:49.150 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:49.150 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:20:49.150 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:49.150 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:20:49.150 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:49.151 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:20:49.151 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:49.151 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:20:49.151 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:49.151 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:20:49.152 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:49.152 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:20:49.152 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:20:49.152 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:20:49.152 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:49.153 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:20:49.153 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:49.153 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:49.153 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:20:49.154 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:20:49.154 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:49.154 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:20:49.154 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:20:49.154 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:20:49.155 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:20:49.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:49.156 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:20:49.237 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:20:49.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:49.242 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:20:49.262 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:20:49.263 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:20:49.263 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:20:49.263 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:20:49.264 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:20:49.264 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:20:49.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:49.265 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:20:49.266 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:20:49.266 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:20:49.267 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:20:49.267 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:20:49.267 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:20:49.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:49.415 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:20:49.440 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:20:49.441 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:20:49.442 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:20:49.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:49.444 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2020-11-20T12:20:49.518 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:49.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.700+0000 7fb9f6ba2700 1 Processor -- start 2020-11-20T12:20:49.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.700+0000 7fb9f6ba2700 1 -- start start 2020-11-20T12:20:49.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f6ba2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9f0062880 0x7fb9f0062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:49.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f6ba2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb9f0065d40 0x7fb9f00bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:49.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f6ba2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f00c0530 con 0x7fb9f00677a0 2020-11-20T12:20:49.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f6ba2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f00661a0 con 0x7fb9f0065d40 2020-11-20T12:20:49.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f6ba2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f00c0310 con 0x7fb9f0062880 2020-11-20T12:20:49.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f539f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb9f0065d40 0x7fb9f00bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:49.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f539f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb9f0065d40 0x7fb9f00bfe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50042/0 (socket says 172.21.15.116:50042) 2020-11-20T12:20:49.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f539f700 1 -- 172.21.15.116:0/4066372120 learned_addr learned my addr 172.21.15.116:0/4066372120 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:49.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f5ba0700 1 --2- 172.21.15.116:0/4066372120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9f0062880 0x7fb9f0062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:49.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f539f700 1 -- 172.21.15.116:0/4066372120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9f0062880 msgr2=0x7fb9f0062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:49.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f539f700 1 --2- 172.21.15.116:0/4066372120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9f0062880 0x7fb9f0062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f539f700 1 -- 172.21.15.116:0/4066372120 >> v1:172.21.15.173:6789/0 conn(0x7fb9f00677a0 legacy=0x7fb9f01556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:49.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f539f700 1 -- 172.21.15.116:0/4066372120 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f00c07d0 con 0x7fb9f0065d40 2020-11-20T12:20:49.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f539f700 1 --2- 172.21.15.116:0/4066372120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0065d40 0x7fb9f00bfe20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fb9e000ba20 tx=0x7fb9e0006eb0).ready entity=mon.0 client_cookie=7bca9f4d78187c20 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:49.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f4b9e700 1 -- 172.21.15.116:0/4066372120 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0009890 con 0x7fb9f0065d40 2020-11-20T12:20:49.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.704+0000 7fb9f4b9e700 1 -- 172.21.15.116:0/4066372120 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e00099f0 con 0x7fb9f0065d40 2020-11-20T12:20:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.708+0000 7fb9f4b9e700 1 -- 172.21.15.116:0/4066372120 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0009d70 con 0x7fb9f0065d40 2020-11-20T12:20:49.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.708+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/4066372120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0065d40 msgr2=0x7fb9f00bfe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:49.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.708+0000 7fb9f6ba2700 1 --2- 172.21.15.116:0/4066372120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0065d40 0x7fb9f00bfe20 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7fb9e000ba20 tx=0x7fb9e0006eb0).stop 2020-11-20T12:20:49.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.708+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/4066372120 shutdown_connections 2020-11-20T12:20:49.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.708+0000 7fb9f6ba2700 1 --2- 172.21.15.116:0/4066372120 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9f0062880 0x7fb9f0062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.708+0000 7fb9f6ba2700 1 --2- 172.21.15.116:0/4066372120 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0065d40 0x7fb9f00bfe20 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.708+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/4066372120 >> 172.21.15.116:0/4066372120 conn(0x7fb9f0060c40 msgr2=0x7fb9f00c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:49.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.708+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/4066372120 shutdown_connections 2020-11-20T12:20:49.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.708+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/4066372120 wait complete. 2020-11-20T12:20:49.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f6ba2700 1 Processor -- start 2020-11-20T12:20:49.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f6ba2700 1 -- start start 2020-11-20T12:20:49.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f6ba2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0062880 0x7fb9f01445c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:49.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f6ba2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f00677a0 0x7fb9f0155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:49.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f6ba2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9f01562a0 0x7fb9f00a65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:49.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f6ba2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f0156810 con 0x7fb9f0062880 2020-11-20T12:20:49.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f6ba2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f0156ab0 con 0x7fb9f00677a0 2020-11-20T12:20:49.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f6ba2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f0156d50 con 0x7fb9f01562a0 2020-11-20T12:20:49.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f5ba0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0062880 0x7fb9f01445c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:49.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f5ba0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0062880 0x7fb9f01445c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50046/0 (socket says 172.21.15.116:50046) 2020-11-20T12:20:49.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f5ba0700 1 -- 172.21.15.116:0/3453718633 learned_addr learned my addr 172.21.15.116:0/3453718633 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:49.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f539f700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f00677a0 0x7fb9f0155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:49.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.712+0000 7fb9f63a1700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9f01562a0 0x7fb9f00a65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:49.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9f5ba0700 1 -- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9f01562a0 msgr2=0x7fb9f00a65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:49.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9f5ba0700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9f01562a0 0x7fb9f00a65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9f5ba0700 1 -- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f00677a0 msgr2=0x7fb9f0155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:49.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9f5ba0700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f00677a0 0x7fb9f0155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9f5ba0700 1 -- 172.21.15.116:0/3453718633 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f00a7430 con 0x7fb9f0062880 2020-11-20T12:20:49.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9f539f700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f00677a0 0x7fb9f0155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:49.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9f63a1700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9f01562a0 0x7fb9f00a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:49.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9f5ba0700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0062880 0x7fb9f01445c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fb9ec00db60 tx=0x7fb9ec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9ddffb700 1 -- 172.21.15.116:0/3453718633 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec009790 con 0x7fb9f0062880 2020-11-20T12:20:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/3453718633 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f0144ab0 con 0x7fb9f0062880 2020-11-20T12:20:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9ddffb700 1 -- 172.21.15.116:0/3453718633 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9ec0098f0 con 0x7fb9f0062880 2020-11-20T12:20:49.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9ddffb700 1 -- 172.21.15.116:0/3453718633 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ec009c90 con 0x7fb9f0062880 2020-11-20T12:20:49.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/3453718633 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f00a9220 con 0x7fb9f0062880 2020-11-20T12:20:49.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.716+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/3453718633 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f00a75c0 con 0x7fb9f0062880 2020-11-20T12:20:49.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.720+0000 7fb9ddffb700 1 -- 172.21.15.116:0/3453718633 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb9ec013480 con 0x7fb9f0062880 2020-11-20T12:20:49.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.720+0000 7fb9ddffb700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9c80557f0 0x7fb9c8057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:49.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.720+0000 7fb9ddffb700 1 -- 172.21.15.116:0/3453718633 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb9ec06ed30 con 0x7fb9f0062880 2020-11-20T12:20:49.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.720+0000 7fb9f539f700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9c80557f0 0x7fb9c8057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:49.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.724+0000 7fb9f539f700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9c80557f0 0x7fb9c8057cb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fb9f0061cf0 tx=0x7fb9e0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:49.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.728+0000 7fb9ddffb700 1 -- 172.21.15.116:0/3453718633 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb9ec044b70 con 0x7fb9f0062880 2020-11-20T12:20:49.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.888+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/3453718633 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb9f0156f10 con 0x7fb9f0062880 2020-11-20T12:20:49.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.892+0000 7fb9ddffb700 1 -- 172.21.15.116:0/3453718633 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16877 (secure 0 0 0) 0x7fb9ec048190 con 0x7fb9f0062880 2020-11-20T12:20:49.888 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:49.888 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":96,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:20:35.459061+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:20:32.886768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-20T12:20:33.446960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:49.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.892+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9c80557f0 msgr2=0x7fb9c8057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:49.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.892+0000 7fb9f6ba2700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9c80557f0 0x7fb9c8057cb0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fb9f0061cf0 tx=0x7fb9e0015000).stop 2020-11-20T12:20:49.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.896+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/3453718633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0062880 msgr2=0x7fb9f01445c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:49.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.896+0000 7fb9f6ba2700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0062880 0x7fb9f01445c0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fb9ec00db60 tx=0x7fb9ec008040).stop 2020-11-20T12:20:49.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.896+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/3453718633 shutdown_connections 2020-11-20T12:20:49.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.896+0000 7fb9f6ba2700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9c80557f0 0x7fb9c8057cb0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.896+0000 7fb9f6ba2700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb9f01562a0 0x7fb9f00a65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.896+0000 7fb9f6ba2700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb9f0062880 0x7fb9f01445c0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.896+0000 7fb9f6ba2700 1 --2- 172.21.15.116:0/3453718633 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb9f00677a0 0x7fb9f0155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:49.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.896+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/3453718633 >> 172.21.15.116:0/3453718633 conn(0x7fb9f0060c40 msgr2=0x7fb9f0154b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:49.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.896+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/3453718633 shutdown_connections 2020-11-20T12:20:49.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:49.896+0000 7fb9f6ba2700 1 -- 172.21.15.116:0/3453718633 wait complete. 2020-11-20T12:20:49.916 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:20:32.886768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:20:33.446960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:20:49.917 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:20:50.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.104+0000 7f15fd104700 1 Processor -- start 2020-11-20T12:20:50.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.104+0000 7f15fd104700 1 -- start start 2020-11-20T12:20:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15fd104700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f15f8151ca0 0x7f15f80c9010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15fd104700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f80a92f0 con 0x7f15f80aa210 2020-11-20T12:20:50.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15fd104700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f80a9590 con 0x7f15f81518c0 2020-11-20T12:20:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15fd104700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f80a9830 con 0x7f15f8151ca0 2020-11-20T12:20:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f759e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f15f80aa210 0x7f15f80aa6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59492/0 (socket says 172.21.15.116:59492) 2020-11-20T12:20:50.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f759e700 1 -- 172.21.15.116:0/4203415999 learned_addr learned my addr 172.21.15.116:0/4203415999 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f659c700 1 --2- 172.21.15.116:0/4203415999 >> v2:172.21.15.173:3300/0 conn(0x7f15f8151ca0 0x7f15f80c9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1865711710 0 0) 0x7f15f80a92f0 con 0x7f15f80aa210 2020-11-20T12:20:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15e4003240 con 0x7f15f80aa210 2020-11-20T12:20:50.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2497188992 0 0) 0x7f15e4003240 con 0x7f15f80aa210 2020-11-20T12:20:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 >> v2:172.21.15.173:3300/0 conn(0x7f15f8151ca0 msgr2=0x7f15f80c9010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 --2- 172.21.15.116:0/4203415999 >> v2:172.21.15.173:3300/0 conn(0x7f15f8151ca0 0x7f15f80c9010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 >> v1:172.21.15.173:6789/0 conn(0x7f15f81518c0 legacy=0x7f15f8061550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f80a9ad0 con 0x7f15f80aa210 2020-11-20T12:20:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f15f0001860 con 0x7f15f80aa210 2020-11-20T12:20:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 >> v1:172.21.15.116:6789/0 conn(0x7f15f80aa210 legacy=0x7f15f80aa6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e4003780 0x7f15e4003ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 0x7f15e4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e4004e50 0x7f15e400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f80a9830 con 0x7f15e40044e0 2020-11-20T12:20:50.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15e40053b0 con 0x7f15e4004e50 2020-11-20T12:20:50.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15e400bca0 con 0x7f15e4003780 2020-11-20T12:20:50.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.108+0000 7f15f659c700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 0x7f15e4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f759e700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e4004e50 0x7f15e400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f659c700 1 -- 172.21.15.116:0/4203415999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e4003780 msgr2=0x7f15e4003ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:50.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f6d9d700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e4003780 0x7f15e4003ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f659c700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e4003780 0x7f15e4003ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f659c700 1 -- 172.21.15.116:0/4203415999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e4004e50 msgr2=0x7f15e400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:50.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f659c700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e4004e50 0x7f15e400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f659c700 1 -- 172.21.15.116:0/4203415999 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15e4003240 con 0x7f15e40044e0 2020-11-20T12:20:50.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f759e700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15e4004e50 0x7f15e400b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:50.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f759e700 1 -- 172.21.15.116:0/4203415999 reap_dead start 2020-11-20T12:20:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f6d9d700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15e4003780 0x7f15e4003ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f659c700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 0x7f15e4004960 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7f15ec00bc00 tx=0x7f15ec006df0).ready entity=mon.0 client_cookie=aeee8cf4a3189ccd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00b310 con 0x7f15e40044e0 2020-11-20T12:20:50.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec00eb40 con 0x7f15e40044e0 2020-11-20T12:20:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15f5d9b700 1 -- 172.21.15.116:0/4203415999 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec00eec0 con 0x7f15e40044e0 2020-11-20T12:20:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15fd104700 1 -- 172.21.15.116:0/4203415999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 msgr2=0x7f15e4004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15fd104700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 0x7f15e4004960 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7f15ec00bc00 tx=0x7f15ec006df0).stop 2020-11-20T12:20:50.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15fd104700 1 -- 172.21.15.116:0/4203415999 shutdown_connections 2020-11-20T12:20:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15fd104700 1 --2- 172.21.15.116:0/4203415999 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 0x7f15e4004960 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15fd104700 1 -- 172.21.15.116:0/4203415999 >> 172.21.15.116:0/4203415999 conn(0x7f15f8068370 msgr2=0x7f15f80c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15fd104700 1 -- 172.21.15.116:0/4203415999 shutdown_connections 2020-11-20T12:20:50.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.112+0000 7f15fd104700 1 -- 172.21.15.116:0/4203415999 wait complete. 2020-11-20T12:20:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15fd104700 1 Processor -- start 2020-11-20T12:20:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15fd104700 1 -- start start 2020-11-20T12:20:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15fd104700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 0x7f15f8150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15fd104700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15f8150820 0x7f15f814c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15fd104700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f814c9e0 0x7f15f8154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15fd104700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f814ce40 con 0x7f15e40044e0 2020-11-20T12:20:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15fd104700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f81610b0 con 0x7f15f8150820 2020-11-20T12:20:50.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15fd104700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f8161310 con 0x7f15f814c9e0 2020-11-20T12:20:50.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15f6d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 0x7f15f8150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15f6d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 0x7f15f8150330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50066/0 (socket says 172.21.15.116:50066) 2020-11-20T12:20:50.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15f6d9d700 1 -- 172.21.15.116:0/4149999106 learned_addr learned my addr 172.21.15.116:0/4149999106 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:50.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15f659c700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15f8150820 0x7f15f814c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15f759e700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f814c9e0 0x7f15f8154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15f6d9d700 1 -- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f814c9e0 msgr2=0x7f15f8154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15f6d9d700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f814c9e0 0x7f15f8154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15f6d9d700 1 -- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15f8150820 msgr2=0x7f15f814c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15f6d9d700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15f8150820 0x7f15f814c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15f6d9d700 1 -- 172.21.15.116:0/4149999106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f81615b0 con 0x7f15e40044e0 2020-11-20T12:20:50.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.116+0000 7f15f759e700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f814c9e0 0x7f15f8154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:50.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.120+0000 7f15f6d9d700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 0x7f15f8150330 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f15e800d780 tx=0x7f15e8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:50.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.120+0000 7f15deffd700 1 -- 172.21.15.116:0/4149999106 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e800f5d0 con 0x7f15e40044e0 2020-11-20T12:20:50.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.120+0000 7f15deffd700 1 -- 172.21.15.116:0/4149999106 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15e800f730 con 0x7f15e40044e0 2020-11-20T12:20:50.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.120+0000 7f15fd104700 1 -- 172.21.15.116:0/4149999106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f8161850 con 0x7f15e40044e0 2020-11-20T12:20:50.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.120+0000 7f15deffd700 1 -- 172.21.15.116:0/4149999106 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15e800fab0 con 0x7f15e40044e0 2020-11-20T12:20:50.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.120+0000 7f15fd104700 1 -- 172.21.15.116:0/4149999106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f8154990 con 0x7f15e40044e0 2020-11-20T12:20:50.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.120+0000 7f15fd104700 1 -- 172.21.15.116:0/4149999106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f81511a0 con 0x7f15e40044e0 2020-11-20T12:20:50.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.120+0000 7f15deffd700 1 -- 172.21.15.116:0/4149999106 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f15e801a030 con 0x7f15e40044e0 2020-11-20T12:20:50.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.124+0000 7f15deffd700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15d40557c0 0x7f15d4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.124+0000 7f15deffd700 1 -- 172.21.15.116:0/4149999106 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f15e806dee0 con 0x7f15e40044e0 2020-11-20T12:20:50.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.124+0000 7f15f659c700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15d40557c0 0x7f15d4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.124+0000 7f15f659c700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15d40557c0 0x7f15d4057c80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f15f8150c00 tx=0x7f15ec019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:50.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.128+0000 7f15deffd700 1 -- 172.21.15.116:0/4149999106 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f15e8043e30 con 0x7f15e40044e0 2020-11-20T12:20:50.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.288+0000 7f15fd104700 1 -- 172.21.15.116:0/4149999106 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f15f8069950 con 0x7f15e40044e0 2020-11-20T12:20:50.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.288+0000 7f15deffd700 1 -- 172.21.15.116:0/4149999106 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v96) v1 ==== 110+0+10 (secure 0 0 0) 0x7f15e8047450 con 0x7f15e40044e0 2020-11-20T12:20:50.287 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:20:50.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 -- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15d40557c0 msgr2=0x7f15d4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15d40557c0 0x7f15d4057c80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f15f8150c00 tx=0x7f15ec019000).stop 2020-11-20T12:20:50.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 -- 172.21.15.116:0/4149999106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 msgr2=0x7f15f8150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 0x7f15f8150330 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7f15e800d780 tx=0x7f15e8009040).stop 2020-11-20T12:20:50.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 -- 172.21.15.116:0/4149999106 shutdown_connections 2020-11-20T12:20:50.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15d40557c0 0x7f15d4057c80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15f814c9e0 0x7f15f8154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15e40044e0 0x7f15f8150330 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 --2- 172.21.15.116:0/4149999106 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15f8150820 0x7f15f814c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 -- 172.21.15.116:0/4149999106 >> 172.21.15.116:0/4149999106 conn(0x7f15f8068370 msgr2=0x7f15f80ad010 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:50.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 -- 172.21.15.116:0/4149999106 shutdown_connections 2020-11-20T12:20:50.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.292+0000 7f15fd104700 1 -- 172.21.15.116:0/4149999106 wait complete. 2020-11-20T12:20:50.324 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:20:50.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.508+0000 7efeb8920700 1 Processor -- start 2020-11-20T12:20:50.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.508+0000 7efeb8920700 1 -- start start 2020-11-20T12:20:50.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.508+0000 7efeb8920700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efeb4149df0 0x7efeb4153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.508+0000 7efeb8920700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7efeb414a1d0 0x7efeb40be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.508+0000 7efeb8920700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efeb40a9c20 con 0x7efeb40a9750 2020-11-20T12:20:50.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.508+0000 7efeb8920700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb40a9ec0 con 0x7efeb4149df0 2020-11-20T12:20:50.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.508+0000 7efeb8920700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efeb4153670 con 0x7efeb414a1d0 2020-11-20T12:20:50.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.508+0000 7efeb259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efeb4149df0 0x7efeb4153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.508+0000 7efeb259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efeb4149df0 0x7efeb4153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50074/0 (socket says 172.21.15.116:50074) 2020-11-20T12:20:50.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.508+0000 7efeb259c700 1 -- 172.21.15.116:0/3513222858 learned_addr learned my addr 172.21.15.116:0/3513222858 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:50.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.508+0000 7efeb2d9d700 1 --2- 172.21.15.116:0/3513222858 >> v2:172.21.15.173:3300/0 conn(0x7efeb414a1d0 0x7efeb40be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb259c700 1 -- 172.21.15.116:0/3513222858 >> v2:172.21.15.173:3300/0 conn(0x7efeb414a1d0 msgr2=0x7efeb40be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:50.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb259c700 1 --2- 172.21.15.116:0/3513222858 >> v2:172.21.15.173:3300/0 conn(0x7efeb414a1d0 0x7efeb40be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb259c700 1 -- 172.21.15.116:0/3513222858 >> v1:172.21.15.173:6789/0 conn(0x7efeb40a9750 legacy=0x7efeb40c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb259c700 1 -- 172.21.15.116:0/3513222858 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb4153910 con 0x7efeb4149df0 2020-11-20T12:20:50.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb159a700 1 -- 172.21.15.116:0/3513222858 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1966985265 0 0) 0x7efeb40a9c20 con 0x7efeb40a9750 2020-11-20T12:20:50.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb2d9d700 1 --2- 172.21.15.116:0/3513222858 >> v2:172.21.15.173:3300/0 conn(0x7efeb414a1d0 0x7efeb40be240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:50.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb259c700 1 --2- 172.21.15.116:0/3513222858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efeb4149df0 0x7efeb4153db0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7efeb40c2370 tx=0x7efea4006eb0).ready entity=mon.0 client_cookie=12ac1e19792814ad server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb159a700 1 -- 172.21.15.116:0/3513222858 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea40098d0 con 0x7efeb4149df0 2020-11-20T12:20:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb159a700 1 -- 172.21.15.116:0/3513222858 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea4009a30 con 0x7efeb4149df0 2020-11-20T12:20:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb159a700 1 -- 172.21.15.116:0/3513222858 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea4009db0 con 0x7efeb4149df0 2020-11-20T12:20:50.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb8920700 1 -- 172.21.15.116:0/3513222858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efeb4149df0 msgr2=0x7efeb4153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb8920700 1 --2- 172.21.15.116:0/3513222858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efeb4149df0 0x7efeb4153db0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7efeb40c2370 tx=0x7efea4006eb0).stop 2020-11-20T12:20:50.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb8920700 1 -- 172.21.15.116:0/3513222858 shutdown_connections 2020-11-20T12:20:50.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb8920700 1 --2- 172.21.15.116:0/3513222858 >> v2:172.21.15.173:3300/0 conn(0x7efeb414a1d0 0x7efeb40be240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb8920700 1 --2- 172.21.15.116:0/3513222858 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efeb4149df0 0x7efeb4153db0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb8920700 1 -- 172.21.15.116:0/3513222858 >> 172.21.15.116:0/3513222858 conn(0x7efeb405ba80 msgr2=0x7efeb40c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb8920700 1 -- 172.21.15.116:0/3513222858 shutdown_connections 2020-11-20T12:20:50.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.512+0000 7efeb8920700 1 -- 172.21.15.116:0/3513222858 wait complete. 2020-11-20T12:20:50.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb8920700 1 Processor -- start 2020-11-20T12:20:50.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb8920700 1 -- start start 2020-11-20T12:20:50.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb8920700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efeb40a9750 0x7efeb414acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb8920700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efeb4149df0 0x7efeb4135a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb8920700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeb414a1d0 0x7efeb40c3b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb8920700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb40bd700 con 0x7efeb4149df0 2020-11-20T12:20:50.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb8920700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb40bd9a0 con 0x7efeb414a1d0 2020-11-20T12:20:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb8920700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efeb40bdc40 con 0x7efeb40a9750 2020-11-20T12:20:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb1d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efeb4149df0 0x7efeb4135a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb1d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efeb4149df0 0x7efeb4135a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50082/0 (socket says 172.21.15.116:50082) 2020-11-20T12:20:50.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb1d9b700 1 -- 172.21.15.116:0/1910049786 learned_addr learned my addr 172.21.15.116:0/1910049786 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:50.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.516+0000 7efeb259c700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efeb40a9750 0x7efeb414acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efeb2d9d700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeb414a1d0 0x7efeb40c3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efeb1d9b700 1 -- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efeb40a9750 msgr2=0x7efeb414acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efeb1d9b700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efeb40a9750 0x7efeb414acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efeb1d9b700 1 -- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeb414a1d0 msgr2=0x7efeb40c3b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efeb1d9b700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeb414a1d0 0x7efeb40c3b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efeb1d9b700 1 -- 172.21.15.116:0/1910049786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeb40bdee0 con 0x7efeb4149df0 2020-11-20T12:20:50.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efeb1d9b700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efeb4149df0 0x7efeb4135a80 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7efea8007fd0 tx=0x7efea8007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:50.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efea27fc700 1 -- 172.21.15.116:0/1910049786 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea800e490 con 0x7efeb4149df0 2020-11-20T12:20:50.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efea27fc700 1 -- 172.21.15.116:0/1910049786 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efea800e610 con 0x7efeb4149df0 2020-11-20T12:20:50.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efeb8920700 1 -- 172.21.15.116:0/1910049786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb40be180 con 0x7efeb4149df0 2020-11-20T12:20:50.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efea27fc700 1 -- 172.21.15.116:0/1910049786 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efea800e9d0 con 0x7efeb4149df0 2020-11-20T12:20:50.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efeb8920700 1 -- 172.21.15.116:0/1910049786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb4135f70 con 0x7efeb4149df0 2020-11-20T12:20:50.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.520+0000 7efeb8920700 1 -- 172.21.15.116:0/1910049786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efeb405d2a0 con 0x7efeb4149df0 2020-11-20T12:20:50.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.524+0000 7efea27fc700 1 -- 172.21.15.116:0/1910049786 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7efea8019030 con 0x7efeb4149df0 2020-11-20T12:20:50.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.524+0000 7efea27fc700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe880557c0 0x7efe88057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.524+0000 7efea27fc700 1 -- 172.21.15.116:0/1910049786 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efea806dd00 con 0x7efeb4149df0 2020-11-20T12:20:50.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.524+0000 7efeb259c700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe880557c0 0x7efe88057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.524+0000 7efeb259c700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe880557c0 0x7efe88057c80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7efea400b1b0 tx=0x7efea4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:50.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.528+0000 7efea27fc700 1 -- 172.21.15.116:0/1910049786 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efea8043a90 con 0x7efeb4149df0 2020-11-20T12:20:50.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.688+0000 7efeb8920700 1 -- 172.21.15.116:0/1910049786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efeb4061380 con 0x7efeb4149df0 2020-11-20T12:20:50.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.688+0000 7efea27fc700 1 -- 172.21.15.116:0/1910049786 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v96) v1 ==== 92+0+10 (secure 0 0 0) 0x7efea80470b0 con 0x7efeb4149df0 2020-11-20T12:20:50.686 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:50.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 -- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe880557c0 msgr2=0x7efe88057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe880557c0 0x7efe88057c80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7efea400b1b0 tx=0x7efea4015000).stop 2020-11-20T12:20:50.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 -- 172.21.15.116:0/1910049786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efeb4149df0 msgr2=0x7efeb4135a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efeb4149df0 0x7efeb4135a80 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7efea8007fd0 tx=0x7efea8007ac0).stop 2020-11-20T12:20:50.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 -- 172.21.15.116:0/1910049786 shutdown_connections 2020-11-20T12:20:50.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efe880557c0 0x7efe88057c80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efeb414a1d0 0x7efeb40c3b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efeb40a9750 0x7efeb414acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 --2- 172.21.15.116:0/1910049786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efeb4149df0 0x7efeb4135a80 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 -- 172.21.15.116:0/1910049786 >> 172.21.15.116:0/1910049786 conn(0x7efeb405ba80 msgr2=0x7efeb40b9400 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:50.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 -- 172.21.15.116:0/1910049786 shutdown_connections 2020-11-20T12:20:50.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.692+0000 7efeb8920700 1 -- 172.21.15.116:0/1910049786 wait complete. 2020-11-20T12:20:50.711 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:20:50.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.892+0000 7fe234a7d700 1 Processor -- start 2020-11-20T12:20:50.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.892+0000 7fe234a7d700 1 -- start start 2020-11-20T12:20:50.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.892+0000 7fe234a7d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2300a8400 0x7fe2300aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.892+0000 7fe234a7d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2300ace40 con 0x7fe2300aa7d0 2020-11-20T12:20:50.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.892+0000 7fe234a7d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2301526c0 con 0x7fe2300ad540 2020-11-20T12:20:50.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.892+0000 7fe234a7d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe230152960 con 0x7fe2300a8400 2020-11-20T12:20:50.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.892+0000 7fe22effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2300a8400 0x7fe2300aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.892+0000 7fe22effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2300a8400 0x7fe2300aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50088/0 (socket says 172.21.15.116:50088) 2020-11-20T12:20:50.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.892+0000 7fe22effd700 1 -- 172.21.15.116:0/3828911648 learned_addr learned my addr 172.21.15.116:0/3828911648 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:50.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.892+0000 7fe22f7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe2300aa7d0 0x7fe230151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59524/0 (socket says 172.21.15.116:59524) 2020-11-20T12:20:50.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe22effd700 1 -- 172.21.15.116:0/3828911648 >> v1:172.21.15.173:6789/0 conn(0x7fe2300ad540 legacy=0x7fe2300ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe22effd700 1 -- 172.21.15.116:0/3828911648 >> v1:172.21.15.116:6789/0 conn(0x7fe2300aa7d0 legacy=0x7fe230151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe22effd700 1 -- 172.21.15.116:0/3828911648 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230152c00 con 0x7fe2300a8400 2020-11-20T12:20:50.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe22e7fc700 1 -- 172.21.15.116:0/3828911648 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552209427 0 0) 0x7fe2300ace40 con 0x7fe2300aa7d0 2020-11-20T12:20:50.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe22effd700 1 --2- 172.21.15.116:0/3828911648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2300a8400 0x7fe2300aabb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7fe224006fd0 tx=0x7fe224006eb0).ready entity=mon.0 client_cookie=433cead51bc82c23 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:50.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe22e7fc700 1 -- 172.21.15.116:0/3828911648 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2240098d0 con 0x7fe2300a8400 2020-11-20T12:20:50.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe22e7fc700 1 -- 172.21.15.116:0/3828911648 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe224009a30 con 0x7fe2300a8400 2020-11-20T12:20:50.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe22e7fc700 1 -- 172.21.15.116:0/3828911648 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224009db0 con 0x7fe2300a8400 2020-11-20T12:20:50.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe234a7d700 1 -- 172.21.15.116:0/3828911648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2300a8400 msgr2=0x7fe2300aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe234a7d700 1 --2- 172.21.15.116:0/3828911648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2300a8400 0x7fe2300aabb0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7fe224006fd0 tx=0x7fe224006eb0).stop 2020-11-20T12:20:50.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe234a7d700 1 -- 172.21.15.116:0/3828911648 shutdown_connections 2020-11-20T12:20:50.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe234a7d700 1 --2- 172.21.15.116:0/3828911648 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2300a8400 0x7fe2300aabb0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe234a7d700 1 -- 172.21.15.116:0/3828911648 >> 172.21.15.116:0/3828911648 conn(0x7fe2300678a0 msgr2=0x7fe230067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:50.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe234a7d700 1 -- 172.21.15.116:0/3828911648 shutdown_connections 2020-11-20T12:20:50.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.896+0000 7fe234a7d700 1 -- 172.21.15.116:0/3828911648 wait complete. 2020-11-20T12:20:50.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.900+0000 7fe234a7d700 1 Processor -- start 2020-11-20T12:20:50.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.900+0000 7fe234a7d700 1 -- start start 2020-11-20T12:20:50.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.900+0000 7fe234a7d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2300a8400 0x7fe230164610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.900+0000 7fe234a7d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2300aa7d0 0x7fe230162210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.900+0000 7fe234a7d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2300ad540 0x7fe2301539e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.900+0000 7fe234a7d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe230161000 con 0x7fe2300ad540 2020-11-20T12:20:50.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.900+0000 7fe234a7d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe230164bd0 con 0x7fe2300a8400 2020-11-20T12:20:50.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.900+0000 7fe234a7d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe230164e70 con 0x7fe2300aa7d0 2020-11-20T12:20:50.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.900+0000 7fe22ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2300ad540 0x7fe2301539e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.900+0000 7fe22ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2300ad540 0x7fe2301539e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50098/0 (socket says 172.21.15.116:50098) 2020-11-20T12:20:50.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.900+0000 7fe22ffff700 1 -- 172.21.15.116:0/3003283971 learned_addr learned my addr 172.21.15.116:0/3003283971 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:50.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe22f7fe700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2300a8400 0x7fe230164610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe22effd700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2300aa7d0 0x7fe230162210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe22ffff700 1 -- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2300aa7d0 msgr2=0x7fe230162210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe22ffff700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2300aa7d0 0x7fe230162210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe22ffff700 1 -- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2300a8400 msgr2=0x7fe230164610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:50.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe22ffff700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2300a8400 0x7fe230164610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:50.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe22ffff700 1 -- 172.21.15.116:0/3003283971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230165110 con 0x7fe2300ad540 2020-11-20T12:20:50.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe22f7fe700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2300a8400 0x7fe230164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:50.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe22ffff700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2300ad540 0x7fe2301539e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7fe22800ca90 tx=0x7fe228001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:50.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe20f7fe700 1 -- 172.21.15.116:0/3003283971 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800a730 con 0x7fe2300ad540 2020-11-20T12:20:50.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe20f7fe700 1 -- 172.21.15.116:0/3003283971 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22800a890 con 0x7fe2300ad540 2020-11-20T12:20:50.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe234a7d700 1 -- 172.21.15.116:0/3003283971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2301653b0 con 0x7fe2300ad540 2020-11-20T12:20:50.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe20f7fe700 1 -- 172.21.15.116:0/3003283971 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800ac30 con 0x7fe2300ad540 2020-11-20T12:20:50.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe234a7d700 1 -- 172.21.15.116:0/3003283971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe230153ed0 con 0x7fe2300ad540 2020-11-20T12:20:50.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.904+0000 7fe234a7d700 1 -- 172.21.15.116:0/3003283971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe230165540 con 0x7fe2300ad540 2020-11-20T12:20:50.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.908+0000 7fe20f7fe700 1 -- 172.21.15.116:0/3003283971 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe228010070 con 0x7fe2300ad540 2020-11-20T12:20:50.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.908+0000 7fe20f7fe700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2080557c0 0x7fe208057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:50.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.908+0000 7fe20f7fe700 1 -- 172.21.15.116:0/3003283971 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe22806e3b0 con 0x7fe2300ad540 2020-11-20T12:20:50.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.908+0000 7fe22f7fe700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2080557c0 0x7fe208057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:50.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.908+0000 7fe22f7fe700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2080557c0 0x7fe208057c80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fe2200011a0 tx=0x7fe220007120).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:50.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:50.912+0000 7fe20f7fe700 1 -- 172.21.15.116:0/3003283971 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe228044250 con 0x7fe2300ad540 2020-11-20T12:20:51.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.072+0000 7fe234a7d700 1 -- 172.21.15.116:0/3003283971 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe23005cf30 con 0x7fe2300ad540 2020-11-20T12:20:51.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.072+0000 7fe20f7fe700 1 -- 172.21.15.116:0/3003283971 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+10 (secure 0 0 0) 0x7fe228047870 con 0x7fe2300ad540 2020-11-20T12:20:51.069 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:51.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 -- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2080557c0 msgr2=0x7fe208057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2080557c0 0x7fe208057c80 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7fe2200011a0 tx=0x7fe220007120).stop 2020-11-20T12:20:51.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 -- 172.21.15.116:0/3003283971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2300ad540 msgr2=0x7fe2301539e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2300ad540 0x7fe2301539e0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7fe22800ca90 tx=0x7fe228001360).stop 2020-11-20T12:20:51.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 -- 172.21.15.116:0/3003283971 shutdown_connections 2020-11-20T12:20:51.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2080557c0 0x7fe208057c80 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2300ad540 0x7fe2301539e0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2300a8400 0x7fe230164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 --2- 172.21.15.116:0/3003283971 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2300aa7d0 0x7fe230162210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 -- 172.21.15.116:0/3003283971 >> 172.21.15.116:0/3003283971 conn(0x7fe2300678a0 msgr2=0x7fe230067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:51.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 -- 172.21.15.116:0/3003283971 shutdown_connections 2020-11-20T12:20:51.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.076+0000 7fe234a7d700 1 -- 172.21.15.116:0/3003283971 wait complete. 2020-11-20T12:20:51.098 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:20:51.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.272+0000 7f08ab59e700 1 Processor -- start 2020-11-20T12:20:51.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.272+0000 7f08ab59e700 1 -- start start 2020-11-20T12:20:51.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08ab59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ac060310 0x7f08ac0606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:51.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08ab59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f08ac0668e0 0x7f08ac0c1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:51.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08ab59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac066e00 con 0x7f08ac066480 2020-11-20T12:20:51.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08ab59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac0670a0 con 0x7f08ac0668e0 2020-11-20T12:20:51.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08ab59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac067340 con 0x7f08ac060310 2020-11-20T12:20:51.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08a9d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f08ac066480 0x7f08ac0bf5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59540/0 (socket says 172.21.15.116:59540) 2020-11-20T12:20:51.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08a9d9b700 1 -- 172.21.15.116:0/1579197887 learned_addr learned my addr 172.21.15.116:0/1579197887 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:51.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08aad9d700 1 --2- 172.21.15.116:0/1579197887 >> v2:172.21.15.173:3300/0 conn(0x7f08ac0668e0 0x7f08ac0c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:51.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08aa59c700 1 --2- 172.21.15.116:0/1579197887 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ac060310 0x7f08ac0606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:51.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08a959a700 1 -- 172.21.15.116:0/1579197887 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3884892897 0 0) 0x7f08ac066e00 con 0x7f08ac066480 2020-11-20T12:20:51.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08a959a700 1 -- 172.21.15.116:0/1579197887 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0890003240 con 0x7f08ac066480 2020-11-20T12:20:51.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08aad9d700 1 -- 172.21.15.116:0/1579197887 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ac060310 msgr2=0x7f08ac0606f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:51.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08aad9d700 1 --2- 172.21.15.116:0/1579197887 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ac060310 0x7f08ac0606f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08aad9d700 1 -- 172.21.15.116:0/1579197887 >> v1:172.21.15.116:6789/0 conn(0x7f08ac066480 legacy=0x7f08ac0bf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08aad9d700 1 -- 172.21.15.116:0/1579197887 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac0c0d30 con 0x7f08ac0668e0 2020-11-20T12:20:51.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.276+0000 7f08aad9d700 1 --2- 172.21.15.116:0/1579197887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac0668e0 0x7f08ac0c1470 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f08a400ab20 tx=0x7f08a400a2b0).ready entity=mon.1 client_cookie=191ddbec0a2bb1a2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:51.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.280+0000 7f08a959a700 1 -- 172.21.15.116:0/1579197887 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400b9f0 con 0x7f08ac0668e0 2020-11-20T12:20:51.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.280+0000 7f08a959a700 1 -- 172.21.15.116:0/1579197887 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400bb50 con 0x7f08ac0668e0 2020-11-20T12:20:51.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.280+0000 7f08a959a700 1 -- 172.21.15.116:0/1579197887 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400f400 con 0x7f08ac0668e0 2020-11-20T12:20:51.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.280+0000 7f08ab59e700 1 -- 172.21.15.116:0/1579197887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac0668e0 msgr2=0x7f08ac0c1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.280+0000 7f08ab59e700 1 --2- 172.21.15.116:0/1579197887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac0668e0 0x7f08ac0c1470 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f08a400ab20 tx=0x7f08a400a2b0).stop 2020-11-20T12:20:51.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.280+0000 7f08ab59e700 1 -- 172.21.15.116:0/1579197887 shutdown_connections 2020-11-20T12:20:51.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.280+0000 7f08ab59e700 1 --2- 172.21.15.116:0/1579197887 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac0668e0 0x7f08ac0c1470 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.280+0000 7f08ab59e700 1 --2- 172.21.15.116:0/1579197887 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ac060310 0x7f08ac0606f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.280+0000 7f08ab59e700 1 -- 172.21.15.116:0/1579197887 >> 172.21.15.116:0/1579197887 conn(0x7f08ac068350 msgr2=0x7f08ac0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:51.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.280+0000 7f08ab59e700 1 -- 172.21.15.116:0/1579197887 shutdown_connections 2020-11-20T12:20:51.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.280+0000 7f08ab59e700 1 -- 172.21.15.116:0/1579197887 wait complete. 2020-11-20T12:20:51.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08ab59e700 1 Processor -- start 2020-11-20T12:20:51.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08ab59e700 1 -- start start 2020-11-20T12:20:51.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08ab59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac060310 0x7f08ac14b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:51.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08ab59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08ac066480 0x7f08ac0b9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:51.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08ab59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ac0668e0 0x7f08ac144080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:51.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08ab59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac149a40 con 0x7f08ac066480 2020-11-20T12:20:51.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08ab59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac149500 con 0x7f08ac060310 2020-11-20T12:20:51.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08ab59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac1497a0 con 0x7f08ac0668e0 2020-11-20T12:20:51.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08a9d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08ac066480 0x7f08ac0b9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:51.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08a9d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08ac066480 0x7f08ac0b9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50110/0 (socket says 172.21.15.116:50110) 2020-11-20T12:20:51.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08a9d9b700 1 -- 172.21.15.116:0/2979679864 learned_addr learned my addr 172.21.15.116:0/2979679864 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:51.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08aad9d700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ac0668e0 0x7f08ac144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:51.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08aa59c700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac060310 0x7f08ac14b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:51.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08a9d9b700 1 -- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ac0668e0 msgr2=0x7f08ac144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08a9d9b700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ac0668e0 0x7f08ac144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08a9d9b700 1 -- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac060310 msgr2=0x7f08ac14b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08a9d9b700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac060310 0x7f08ac14b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08a9d9b700 1 -- 172.21.15.116:0/2979679864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac14bb10 con 0x7f08ac066480 2020-11-20T12:20:51.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.284+0000 7f08aad9d700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ac0668e0 0x7f08ac144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:51.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.288+0000 7f08a9d9b700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08ac066480 0x7f08ac0b9e90 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f08a000c710 tx=0x7f08a0007aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:51.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.288+0000 7f089a7fc700 1 -- 172.21.15.116:0/2979679864 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0007320 con 0x7f08ac066480 2020-11-20T12:20:51.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.288+0000 7f089a7fc700 1 -- 172.21.15.116:0/2979679864 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a0007480 con 0x7f08ac066480 2020-11-20T12:20:51.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.288+0000 7f08ab59e700 1 -- 172.21.15.116:0/2979679864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac149180 con 0x7f08ac066480 2020-11-20T12:20:51.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.288+0000 7f089a7fc700 1 -- 172.21.15.116:0/2979679864 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0005800 con 0x7f08ac066480 2020-11-20T12:20:51.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.288+0000 7f08ab59e700 1 -- 172.21.15.116:0/2979679864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac0ba380 con 0x7f08ac066480 2020-11-20T12:20:51.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.288+0000 7f08ab59e700 1 -- 172.21.15.116:0/2979679864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac14bca0 con 0x7f08ac066480 2020-11-20T12:20:51.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.288+0000 7f089a7fc700 1 -- 172.21.15.116:0/2979679864 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f08a0010030 con 0x7f08ac066480 2020-11-20T12:20:51.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.292+0000 7f089a7fc700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08800557c0 0x7f0880057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:51.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.292+0000 7f089a7fc700 1 -- 172.21.15.116:0/2979679864 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f08a006e400 con 0x7f08ac066480 2020-11-20T12:20:51.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.292+0000 7f08aa59c700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08800557c0 0x7f0880057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:51.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.292+0000 7f08aa59c700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08800557c0 0x7f0880057c80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f089c00dd90 tx=0x7f089c006e80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:51.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.300+0000 7f089a7fc700 1 -- 172.21.15.116:0/2979679864 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f08a00442a0 con 0x7f08ac066480 2020-11-20T12:20:51.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.468+0000 7f08ab59e700 1 -- 172.21.15.116:0/2979679864 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f08ac05cf30 con 0x7f08ac066480 2020-11-20T12:20:51.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.468+0000 7f089a7fc700 1 -- 172.21.15.116:0/2979679864 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+10 (secure 0 0 0) 0x7f08a00478c0 con 0x7f08ac066480 2020-11-20T12:20:51.465 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:51.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 -- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08800557c0 msgr2=0x7f0880057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08800557c0 0x7f0880057c80 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f089c00dd90 tx=0x7f089c006e80).stop 2020-11-20T12:20:51.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 -- 172.21.15.116:0/2979679864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08ac066480 msgr2=0x7f08ac0b9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08ac066480 0x7f08ac0b9e90 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7f08a000c710 tx=0x7f08a0007aa0).stop 2020-11-20T12:20:51.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 -- 172.21.15.116:0/2979679864 shutdown_connections 2020-11-20T12:20:51.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08800557c0 0x7f0880057c80 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08ac0668e0 0x7f08ac144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08ac060310 0x7f08ac14b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 --2- 172.21.15.116:0/2979679864 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08ac066480 0x7f08ac0b9e90 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 -- 172.21.15.116:0/2979679864 >> 172.21.15.116:0/2979679864 conn(0x7f08ac068350 msgr2=0x7f08ac0c11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:51.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 -- 172.21.15.116:0/2979679864 shutdown_connections 2020-11-20T12:20:51.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.472+0000 7f08ab59e700 1 -- 172.21.15.116:0/2979679864 wait complete. 2020-11-20T12:20:51.493 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:20:51.494 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:51.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.668+0000 7fadfb0da700 1 Processor -- start 2020-11-20T12:20:51.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.672+0000 7fadfb0da700 1 -- start start 2020-11-20T12:20:51.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.672+0000 7fadfb0da700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadf4148040 0x7fadf4148420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:51.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.672+0000 7fadfb0da700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf4067190 con 0x7fadf4068420 2020-11-20T12:20:51.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.672+0000 7fadfb0da700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadf40677a0 con 0x7fadf4066480 2020-11-20T12:20:51.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.672+0000 7fadfb0da700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf4067a40 con 0x7fadf4148040 2020-11-20T12:20:51.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.672+0000 7fadf98d7700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fadf4068420 0x7fadf40626e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59552/0 (socket says 172.21.15.116:59552) 2020-11-20T12:20:51.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.672+0000 7fadf98d7700 1 -- 172.21.15.116:0/182253482 learned_addr learned my addr 172.21.15.116:0/182253482 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:51.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.672+0000 7fadfa0d8700 1 --2- 172.21.15.116:0/182253482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadf4148040 0x7fadf4148420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:51.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.672+0000 7fadf90d6700 1 -- 172.21.15.116:0/182253482 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739894905 0 0) 0x7fadf4067190 con 0x7fadf4068420 2020-11-20T12:20:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadf90d6700 1 -- 172.21.15.116:0/182253482 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faddc003240 con 0x7fadf4068420 2020-11-20T12:20:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadf90d6700 1 -- 172.21.15.116:0/182253482 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2646560747 0 0) 0x7fadf40677a0 con 0x7fadf4066480 2020-11-20T12:20:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadfa0d8700 1 -- 172.21.15.116:0/182253482 >> v1:172.21.15.173:6789/0 conn(0x7fadf4066480 legacy=0x7fadf4066930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadfa0d8700 1 -- 172.21.15.116:0/182253482 >> v1:172.21.15.116:6789/0 conn(0x7fadf4068420 legacy=0x7fadf40626e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadfa0d8700 1 -- 172.21.15.116:0/182253482 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4067ce0 con 0x7fadf4148040 2020-11-20T12:20:51.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadfa0d8700 1 --2- 172.21.15.116:0/182253482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadf4148040 0x7fadf4148420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fadf000b9d0 tx=0x7fadf0006df0).ready entity=mon.2 client_cookie=4cd5edc72e84e5ef server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:51.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadf90d6700 1 -- 172.21.15.116:0/182253482 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00097b0 con 0x7fadf4148040 2020-11-20T12:20:51.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadf90d6700 1 -- 172.21.15.116:0/182253482 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0009910 con 0x7fadf4148040 2020-11-20T12:20:51.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadf90d6700 1 -- 172.21.15.116:0/182253482 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0009c90 con 0x7fadf4148040 2020-11-20T12:20:51.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadfb0da700 1 -- 172.21.15.116:0/182253482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadf4148040 msgr2=0x7fadf4148420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadfb0da700 1 --2- 172.21.15.116:0/182253482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadf4148040 0x7fadf4148420 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7fadf000b9d0 tx=0x7fadf0006df0).stop 2020-11-20T12:20:51.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadfb0da700 1 -- 172.21.15.116:0/182253482 shutdown_connections 2020-11-20T12:20:51.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadfb0da700 1 --2- 172.21.15.116:0/182253482 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadf4148040 0x7fadf4148420 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadfb0da700 1 -- 172.21.15.116:0/182253482 >> 172.21.15.116:0/182253482 conn(0x7fadf4060c40 msgr2=0x7fadf40c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:51.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadfb0da700 1 -- 172.21.15.116:0/182253482 shutdown_connections 2020-11-20T12:20:51.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.676+0000 7fadfb0da700 1 -- 172.21.15.116:0/182253482 wait complete. 2020-11-20T12:20:51.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.680+0000 7fadfb0da700 1 Processor -- start 2020-11-20T12:20:51.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.680+0000 7fadfb0da700 1 -- start start 2020-11-20T12:20:51.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfb0da700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadf4066480 0x7fadf41471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:51.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfb0da700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadf4068420 0x7fadf40a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:51.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfb0da700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf4148040 0x7fadf40a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:51.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfb0da700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf41549b0 con 0x7fadf4148040 2020-11-20T12:20:51.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfb0da700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadf4154c50 con 0x7fadf4068420 2020-11-20T12:20:51.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfb0da700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadf4154ef0 con 0x7fadf4066480 2020-11-20T12:20:51.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfa8d9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf4148040 0x7fadf40a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:51.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfa8d9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf4148040 0x7fadf40a5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50126/0 (socket says 172.21.15.116:50126) 2020-11-20T12:20:51.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfa8d9700 1 -- 172.21.15.116:0/3122752090 learned_addr learned my addr 172.21.15.116:0/3122752090 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:51.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfa0d8700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadf4066480 0x7fadf41471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:51.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadf98d7700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadf4068420 0x7fadf40a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:51.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfa8d9700 1 -- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadf4066480 msgr2=0x7fadf41471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfa8d9700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadf4066480 0x7fadf41471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfa8d9700 1 -- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadf4068420 msgr2=0x7fadf40a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfa8d9700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadf4068420 0x7fadf40a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfa8d9700 1 -- 172.21.15.116:0/3122752090 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadf4155190 con 0x7fadf4148040 2020-11-20T12:20:51.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfa8d9700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf4148040 0x7fadf40a5280 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7fadec00cae0 tx=0x7fadec007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:51.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fade27fc700 1 -- 172.21.15.116:0/3122752090 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec00a7c0 con 0x7fadf4148040 2020-11-20T12:20:51.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fade27fc700 1 -- 172.21.15.116:0/3122752090 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec00a940 con 0x7fadf4148040 2020-11-20T12:20:51.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfb0da700 1 -- 172.21.15.116:0/3122752090 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadf4155430 con 0x7fadf4148040 2020-11-20T12:20:51.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fade27fc700 1 -- 172.21.15.116:0/3122752090 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec00ad00 con 0x7fadf4148040 2020-11-20T12:20:51.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.684+0000 7fadfb0da700 1 -- 172.21.15.116:0/3122752090 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadf4154510 con 0x7fadf4148040 2020-11-20T12:20:51.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.688+0000 7fadfb0da700 1 -- 172.21.15.116:0/3122752090 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadf41555c0 con 0x7fadf4148040 2020-11-20T12:20:51.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.688+0000 7fade27fc700 1 -- 172.21.15.116:0/3122752090 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fadec019030 con 0x7fadf4148040 2020-11-20T12:20:51.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.688+0000 7fade27fc700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadcc0557c0 0x7fadcc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:51.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.688+0000 7fade27fc700 1 -- 172.21.15.116:0/3122752090 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fadec06dc30 con 0x7fadf4148040 2020-11-20T12:20:51.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.688+0000 7fadfa0d8700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadcc0557c0 0x7fadcc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:51.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.692+0000 7fadfa0d8700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadcc0557c0 0x7fadcc057c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fadf000b920 tx=0x7fadf0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:51.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.692+0000 7fade27fc700 1 -- 172.21.15.116:0/3122752090 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fadec043b80 con 0x7fadf4148040 2020-11-20T12:20:51.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.896+0000 7fadfb0da700 1 -- 172.21.15.116:0/3122752090 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fadf405cf30 con 0x7fadf4148040 2020-11-20T12:20:51.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fade27fc700 1 -- 172.21.15.116:0/3122752090 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2848 (secure 0 0 0) 0x7fadec0471a0 con 0x7fadf4148040 2020-11-20T12:20:51.896 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:51.896 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3105911814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3105911814},{"type":"v1","addr":"172.21.15.173:6837","nonce":3105911814}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/3457547857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3457547857},{"type":"v1","addr":"172.21.15.116:6835","nonce":3457547857}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/81195738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":81195738},{"type":"v1","addr":"172.21.15.116:6833","nonce":81195738}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:20:34.449524+0000","modified":"2020-11-20T12:20:36.475264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/2361610151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2361610151},{"type":"v1","addr":"172.21.15.173:6835","nonce":2361610151}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-20T12:20:51.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 -- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadcc0557c0 msgr2=0x7fadcc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadcc0557c0 0x7fadcc057c80 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fadf000b920 tx=0x7fadf0015000).stop 2020-11-20T12:20:51.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 -- 172.21.15.116:0/3122752090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf4148040 msgr2=0x7fadf40a5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:51.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf4148040 0x7fadf40a5280 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7fadec00cae0 tx=0x7fadec007f90).stop 2020-11-20T12:20:51.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 -- 172.21.15.116:0/3122752090 shutdown_connections 2020-11-20T12:20:51.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadcc0557c0 0x7fadcc057c80 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadf4148040 0x7fadf40a5280 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadf4066480 0x7fadf41471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 --2- 172.21.15.116:0/3122752090 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadf4068420 0x7fadf40a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:51.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 -- 172.21.15.116:0/3122752090 >> 172.21.15.116:0/3122752090 conn(0x7fadf4060c40 msgr2=0x7fadf40c42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:51.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 -- 172.21.15.116:0/3122752090 shutdown_connections 2020-11-20T12:20:51.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:51.900+0000 7fadfb0da700 1 -- 172.21.15.116:0/3122752090 wait complete. 2020-11-20T12:20:51.901 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 54 2020-11-20T12:20:52.327 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:20:34.449524+0000', 'modified': '2020-11-20T12:20:36.475264+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7311}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7311': {'gid': 7311, 'name': 'b', 'rank': 0, 'incarnation': 51, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.173:6835/2361610151', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.173:6834', 'nonce': 2361610151}, {'type': 'v1', 'addr': '172.21.15.173:6835', 'nonce': 2361610151}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:20:53.119 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:20:53.120 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.b.asok --format=json status 2020-11-20T12:20:53.317 INFO:teuthology.orchestra.run.smithi173.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":7311,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.846695881999999,"mdsmap_epoch":54,"osdmap_epoch":96,"osdmap_epoch_barrier":96,"uptime":24.366306658999999} 2020-11-20T12:20:53.318 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 7311, "mdsmap_epoch": 54, "osdmap_epoch": 96, "osdmap_epoch_barrier": 96, "rank_uptime": 18.846695882, "state": "up:active", "uptime": 24.366306659, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:20:53.319 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:20:53.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:53.320 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.3 2020-11-20T12:20:53.340 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:20:53.340 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:53.454 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:53.454 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:20:53.480 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:20:53.480 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:20:53.488 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:20:53.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:53.489 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2020-11-20T12:20:53.526 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:20:53.526 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:20:53.578 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:20:53.623 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:20:53.623 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-20T12:20:53.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:20:53.624 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-20T12:20:53.700 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:20:53.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:53.702 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:20:53.724 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-20T12:20:53.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:20:53.725 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 2020-11-20T12:20:53.809 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:20:53.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:53.811 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T12:20:53.832 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2020-11-20T12:20:53.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:20:53.833 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.2 2020-11-20T12:20:53.920 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:20:53.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:53.921 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T12:20:53.944 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2020-11-20T12:20:53.944 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:20:53.945 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.3 2020-11-20T12:20:54.037 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:20:54.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:20:54.038 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T12:20:54.067 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-20T12:20:54.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.259+0000 7f5df4c2d700 1 Processor -- start 2020-11-20T12:20:54.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.259+0000 7f5df4c2d700 1 -- start start 2020-11-20T12:20:54.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.259+0000 7f5df4c2d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5df00aa7d0 0x7f5df00a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.259+0000 7f5df4c2d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df00ad540 0x7f5df00ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.259+0000 7f5df4c2d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5df00ace40 con 0x7f5df00aabb0 2020-11-20T12:20:54.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.259+0000 7f5df4c2d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5df01526c0 con 0x7f5df00aa7d0 2020-11-20T12:20:54.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.259+0000 7f5df4c2d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df0152960 con 0x7f5df00ad540 2020-11-20T12:20:54.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5deeffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5df00aabb0 0x7f5df0151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59568/0 (socket says 172.21.15.116:59568) 2020-11-20T12:20:54.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5deeffd700 1 -- 172.21.15.116:0/699274633 learned_addr learned my addr 172.21.15.116:0/699274633 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:54.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5deffff700 1 --2- 172.21.15.116:0/699274633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df00ad540 0x7f5df00ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5def7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5df00aa7d0 0x7f5df00a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5deffff700 1 -- 172.21.15.116:0/699274633 >> v2:172.21.15.116:3300/0 conn(0x7f5df00aa7d0 msgr2=0x7f5df00a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:54.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5deffff700 1 --2- 172.21.15.116:0/699274633 >> v2:172.21.15.116:3300/0 conn(0x7f5df00aa7d0 0x7f5df00a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5deffff700 1 -- 172.21.15.116:0/699274633 >> v1:172.21.15.116:6789/0 conn(0x7f5df00aabb0 legacy=0x7f5df0151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5deffff700 1 -- 172.21.15.116:0/699274633 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df0152c00 con 0x7f5df00ad540 2020-11-20T12:20:54.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5def7fe700 1 --2- 172.21.15.116:0/699274633 >> v2:172.21.15.116:3300/0 conn(0x7f5df00aa7d0 0x7f5df00a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:54.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5deffff700 1 --2- 172.21.15.116:0/699274633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df00ad540 0x7f5df00ad9d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f5df00acfd0 tx=0x7f5de80091d0).ready entity=mon.2 client_cookie=83a1f9c9efa6a34f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:54.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5dee7fc700 1 -- 172.21.15.116:0/699274633 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de800ba70 con 0x7f5df00ad540 2020-11-20T12:20:54.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5dee7fc700 1 -- 172.21.15.116:0/699274633 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de800bbd0 con 0x7f5df00ad540 2020-11-20T12:20:54.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5dee7fc700 1 -- 172.21.15.116:0/699274633 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8011400 con 0x7f5df00ad540 2020-11-20T12:20:54.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5df4c2d700 1 -- 172.21.15.116:0/699274633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df00ad540 msgr2=0x7f5df00ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.263+0000 7f5df4c2d700 1 --2- 172.21.15.116:0/699274633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df00ad540 0x7f5df00ad9d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f5df00acfd0 tx=0x7f5de80091d0).stop 2020-11-20T12:20:54.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.267+0000 7f5df4c2d700 1 -- 172.21.15.116:0/699274633 shutdown_connections 2020-11-20T12:20:54.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.267+0000 7f5df4c2d700 1 --2- 172.21.15.116:0/699274633 >> v2:172.21.15.116:3300/0 conn(0x7f5df00aa7d0 0x7f5df00a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.267+0000 7f5df4c2d700 1 --2- 172.21.15.116:0/699274633 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df00ad540 0x7f5df00ad9d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.267+0000 7f5df4c2d700 1 -- 172.21.15.116:0/699274633 >> 172.21.15.116:0/699274633 conn(0x7f5df00678a0 msgr2=0x7f5df0067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:54.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.267+0000 7f5df4c2d700 1 -- 172.21.15.116:0/699274633 shutdown_connections 2020-11-20T12:20:54.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.267+0000 7f5df4c2d700 1 -- 172.21.15.116:0/699274633 wait complete. 2020-11-20T12:20:54.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.267+0000 7f5df4c2d700 1 Processor -- start 2020-11-20T12:20:54.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.267+0000 7f5df4c2d700 1 -- start start 2020-11-20T12:20:54.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5df4c2d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5df00aa7d0 0x7f5df013c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5df4c2d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df00aabb0 0x7f5df014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5df4c2d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5df00ad540 0x7f5df014faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5df4c2d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df01518a0 con 0x7f5df00ad540 2020-11-20T12:20:54.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5df4c2d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5df0156f10 con 0x7f5df00aa7d0 2020-11-20T12:20:54.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5df4c2d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5df01571b0 con 0x7f5df00aabb0 2020-11-20T12:20:54.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5deffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5df00ad540 0x7f5df014faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5deffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5df00ad540 0x7f5df014faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50140/0 (socket says 172.21.15.116:50140) 2020-11-20T12:20:54.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5deffff700 1 -- 172.21.15.116:0/78810038 learned_addr learned my addr 172.21.15.116:0/78810038 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:54.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5deeffd700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df00aabb0 0x7f5df014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5def7fe700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5df00aa7d0 0x7f5df013c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5deffff700 1 -- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df00aabb0 msgr2=0x7f5df014f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5deffff700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df00aabb0 0x7f5df014f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5deffff700 1 -- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5df00aa7d0 msgr2=0x7f5df013c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5deffff700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5df00aa7d0 0x7f5df013c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5deffff700 1 -- 172.21.15.116:0/78810038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5df0157450 con 0x7f5df00ad540 2020-11-20T12:20:54.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5deffff700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5df00ad540 0x7f5df014faa0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f5de800c010 tx=0x7f5de8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:54.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5dcf7fe700 1 -- 172.21.15.116:0/78810038 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de8006ae0 con 0x7f5df00ad540 2020-11-20T12:20:54.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5dcf7fe700 1 -- 172.21.15.116:0/78810038 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5de801c5f0 con 0x7f5df00ad540 2020-11-20T12:20:54.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5df4c2d700 1 -- 172.21.15.116:0/78810038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5df01576f0 con 0x7f5df00ad540 2020-11-20T12:20:54.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.271+0000 7f5dcf7fe700 1 -- 172.21.15.116:0/78810038 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5de801c990 con 0x7f5df00ad540 2020-11-20T12:20:54.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.275+0000 7f5df4c2d700 1 -- 172.21.15.116:0/78810038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5df013c640 con 0x7f5df00ad540 2020-11-20T12:20:54.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.275+0000 7f5df4c2d700 1 -- 172.21.15.116:0/78810038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5df0157d50 con 0x7f5df00ad540 2020-11-20T12:20:54.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.275+0000 7f5dcf7fe700 1 -- 172.21.15.116:0/78810038 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5de8017070 con 0x7f5df00ad540 2020-11-20T12:20:54.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.279+0000 7f5dcf7fe700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dc80557c0 0x7f5dc8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.279+0000 7f5dcf7fe700 1 -- 172.21.15.116:0/78810038 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5de8071290 con 0x7f5df00ad540 2020-11-20T12:20:54.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.279+0000 7f5def7fe700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dc80557c0 0x7f5dc8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.279+0000 7f5def7fe700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dc80557c0 0x7f5dc8057c80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f5de0006fd0 tx=0x7f5de0006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:54.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.279+0000 7f5dcf7fe700 1 -- 172.21.15.116:0/78810038 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5de804ab60 con 0x7f5df00ad540 2020-11-20T12:20:54.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.439+0000 7f5df4c2d700 1 -- 172.21.15.116:0/78810038 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5df00ad280 con 0x7f5df00ad540 2020-11-20T12:20:54.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.447+0000 7f5dcf7fe700 1 -- 172.21.15.116:0/78810038 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5df00ad280 con 0x7f5df00ad540 2020-11-20T12:20:54.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 -- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dc80557c0 msgr2=0x7f5dc8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dc80557c0 0x7f5dc8057c80 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f5de0006fd0 tx=0x7f5de0006df0).stop 2020-11-20T12:20:54.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 -- 172.21.15.116:0/78810038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5df00ad540 msgr2=0x7f5df014faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5df00ad540 0x7f5df014faa0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f5de800c010 tx=0x7f5de8016000).stop 2020-11-20T12:20:54.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 -- 172.21.15.116:0/78810038 shutdown_connections 2020-11-20T12:20:54.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5dc80557c0 0x7f5dc8057c80 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5df00ad540 0x7f5df014faa0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5df00aa7d0 0x7f5df013c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 --2- 172.21.15.116:0/78810038 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5df00aabb0 0x7f5df014f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 -- 172.21.15.116:0/78810038 >> 172.21.15.116:0/78810038 conn(0x7f5df00678a0 msgr2=0x7f5df00c87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:54.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 -- 172.21.15.116:0/78810038 shutdown_connections 2020-11-20T12:20:54.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.451+0000 7f5df4c2d700 1 -- 172.21.15.116:0/78810038 wait complete. 2020-11-20T12:20:54.476 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_evict_client' 2020-11-20T12:20:54.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb63beb1700 1 Processor -- start 2020-11-20T12:20:54.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb63beb1700 1 -- start start 2020-11-20T12:20:54.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb63beb1700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb6340c1670 0x7fb634152120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb63beb1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6340b98f0 0x7fb6340b9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb63beb1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb634152610 con 0x7fb6340a3750 2020-11-20T12:20:54.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb63beb1700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb63414f170 con 0x7fb6340c1670 2020-11-20T12:20:54.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb63beb1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63414e250 con 0x7fb6340b98f0 2020-11-20T12:20:54.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb63944c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb6340a3750 0x7fb6340bf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59580/0 (socket says 172.21.15.116:59580) 2020-11-20T12:20:54.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb63944c700 1 -- 172.21.15.116:0/577010183 learned_addr learned my addr 172.21.15.116:0/577010183 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:54.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb639c4d700 1 --2- 172.21.15.116:0/577010183 >> v2:172.21.15.173:3300/0 conn(0x7fb6340c1670 0x7fb634152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb63a44e700 1 --2- 172.21.15.116:0/577010183 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6340b98f0 0x7fb6340b9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb638c4b700 1 -- 172.21.15.116:0/577010183 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132797627 0 0) 0x7fb634152610 con 0x7fb6340a3750 2020-11-20T12:20:54.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.659+0000 7fb638c4b700 1 -- 172.21.15.116:0/577010183 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb61c003240 con 0x7fb6340a3750 2020-11-20T12:20:54.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb639c4d700 1 -- 172.21.15.116:0/577010183 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6340b98f0 msgr2=0x7fb6340b9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:54.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb639c4d700 1 --2- 172.21.15.116:0/577010183 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6340b98f0 0x7fb6340b9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb639c4d700 1 -- 172.21.15.116:0/577010183 >> v1:172.21.15.116:6789/0 conn(0x7fb6340a3750 legacy=0x7fb6340bf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb638c4b700 1 -- 172.21.15.116:0/577010183 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 621946928 0 0) 0x7fb61c003240 con 0x7fb6340a3750 2020-11-20T12:20:54.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb639c4d700 1 -- 172.21.15.116:0/577010183 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb63414e4f0 con 0x7fb6340c1670 2020-11-20T12:20:54.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb639c4d700 1 --2- 172.21.15.116:0/577010183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6340c1670 0x7fb634152120 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fb63000b9a0 tx=0x7fb630006df0).ready entity=mon.1 client_cookie=24f360fa38fa9f77 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:54.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb638c4b700 1 -- 172.21.15.116:0/577010183 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6240014e0 con 0x7fb6340c1670 2020-11-20T12:20:54.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb638c4b700 1 -- 172.21.15.116:0/577010183 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb630011400 con 0x7fb6340c1670 2020-11-20T12:20:54.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb638c4b700 1 -- 172.21.15.116:0/577010183 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb630011780 con 0x7fb6340c1670 2020-11-20T12:20:54.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb63beb1700 1 -- 172.21.15.116:0/577010183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6340c1670 msgr2=0x7fb634152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb63beb1700 1 --2- 172.21.15.116:0/577010183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6340c1670 0x7fb634152120 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fb63000b9a0 tx=0x7fb630006df0).stop 2020-11-20T12:20:54.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb63beb1700 1 -- 172.21.15.116:0/577010183 shutdown_connections 2020-11-20T12:20:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb63beb1700 1 --2- 172.21.15.116:0/577010183 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6340c1670 0x7fb634152120 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb63beb1700 1 --2- 172.21.15.116:0/577010183 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6340b98f0 0x7fb6340b9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb63beb1700 1 -- 172.21.15.116:0/577010183 >> 172.21.15.116:0/577010183 conn(0x7fb6340c4f50 msgr2=0x7fb6340c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb63beb1700 1 -- 172.21.15.116:0/577010183 shutdown_connections 2020-11-20T12:20:54.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.663+0000 7fb63beb1700 1 -- 172.21.15.116:0/577010183 wait complete. 2020-11-20T12:20:54.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63beb1700 1 Processor -- start 2020-11-20T12:20:54.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63beb1700 1 -- start start 2020-11-20T12:20:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63beb1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6340a3750 0x7fb63414c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63beb1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6340b98f0 0x7fb6340bc670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63beb1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6340a7470 0x7fb6340a78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63beb1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63414c5d0 con 0x7fb6340b98f0 2020-11-20T12:20:54.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63beb1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6340a7dc0 con 0x7fb6340a3750 2020-11-20T12:20:54.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63beb1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6341619c0 con 0x7fb6340a7470 2020-11-20T12:20:54.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63944c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6340b98f0 0x7fb6340bc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63944c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6340b98f0 0x7fb6340bc670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50150/0 (socket says 172.21.15.116:50150) 2020-11-20T12:20:54.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63944c700 1 -- 172.21.15.116:0/3702947807 learned_addr learned my addr 172.21.15.116:0/3702947807 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:54.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb63a44e700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6340a7470 0x7fb6340a78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.667+0000 7fb639c4d700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6340a3750 0x7fb63414c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb63944c700 1 -- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6340a7470 msgr2=0x7fb6340a78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb63944c700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6340a7470 0x7fb6340a78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb63944c700 1 -- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6340a3750 msgr2=0x7fb63414c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:54.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb63944c700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6340a3750 0x7fb63414c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:54.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb63944c700 1 -- 172.21.15.116:0/3702947807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb634161c60 con 0x7fb6340b98f0 2020-11-20T12:20:54.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb63944c700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6340b98f0 0x7fb6340bc670 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fb62400cae0 tx=0x7fb624001330).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:54.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb621ffb700 1 -- 172.21.15.116:0/3702947807 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb624007170 con 0x7fb6340b98f0 2020-11-20T12:20:54.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb621ffb700 1 -- 172.21.15.116:0/3702947807 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6240072d0 con 0x7fb6340b98f0 2020-11-20T12:20:54.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb621ffb700 1 -- 172.21.15.116:0/3702947807 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb62400aa40 con 0x7fb6340b98f0 2020-11-20T12:20:54.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb63beb1700 1 -- 172.21.15.116:0/3702947807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb634161f00 con 0x7fb6340b98f0 2020-11-20T12:20:54.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb63beb1700 1 -- 172.21.15.116:0/3702947807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb634162440 con 0x7fb6340b98f0 2020-11-20T12:20:54.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.671+0000 7fb63beb1700 1 -- 172.21.15.116:0/3702947807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6340aa600 con 0x7fb6340b98f0 2020-11-20T12:20:54.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.675+0000 7fb621ffb700 1 -- 172.21.15.116:0/3702947807 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb624010070 con 0x7fb6340b98f0 2020-11-20T12:20:54.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.675+0000 7fb621ffb700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb60c0557f0 0x7fb60c057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.675+0000 7fb621ffb700 1 -- 172.21.15.116:0/3702947807 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb62406e1d0 con 0x7fb6340b98f0 2020-11-20T12:20:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.675+0000 7fb639c4d700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb60c0557f0 0x7fb60c057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.675+0000 7fb639c4d700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb60c0557f0 0x7fb60c057cb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fb6340c0ec0 tx=0x7fb630013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:54.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.679+0000 7fb621ffb700 1 -- 172.21.15.116:0/3702947807 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb6340aa600 con 0x7fb6340b98f0 2020-11-20T12:20:54.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:54.891+0000 7fb63beb1700 1 -- 172.21.15.116:0/3702947807 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_evict_client"]} v 0) v1 -- 0x7fb6340aa600 con 0x7fb6340b98f0 2020-11-20T12:20:55.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.291+0000 7fb621ffb700 1 -- 172.21.15.116:0/3702947807 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_evict_client"]}]=0 v284) v1 ==== 145+0+0 (secure 0 0 0) 0x7fb6340aa600 con 0x7fb6340b98f0 2020-11-20T12:20:55.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 -- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb60c0557f0 msgr2=0x7fb60c057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:55.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb60c0557f0 0x7fb60c057cb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7fb6340c0ec0 tx=0x7fb630013000).stop 2020-11-20T12:20:55.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 -- 172.21.15.116:0/3702947807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6340b98f0 msgr2=0x7fb6340bc670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:55.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6340b98f0 0x7fb6340bc670 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7fb62400cae0 tx=0x7fb624001330).stop 2020-11-20T12:20:55.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 -- 172.21.15.116:0/3702947807 shutdown_connections 2020-11-20T12:20:55.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb60c0557f0 0x7fb60c057cb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:55.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb6340a7470 0x7fb6340a78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb6340a3750 0x7fb63414c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 --2- 172.21.15.116:0/3702947807 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb6340b98f0 0x7fb6340bc670 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 -- 172.21.15.116:0/3702947807 >> 172.21.15.116:0/3702947807 conn(0x7fb6340c4f50 msgr2=0x7fb6340b9d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:55.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 -- 172.21.15.116:0/3702947807 shutdown_connections 2020-11-20T12:20:55.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.295+0000 7fb63beb1700 1 -- 172.21.15.116:0/3702947807 wait complete. 2020-11-20T12:20:55.321 INFO:tasks.cephfs_test_runner:test_evict_client (tasks.cephfs.test_volume_client.TestVolumeClient) ... skipped 'Requires FUSE client to inject client metadata' 2020-11-20T12:20:55.322 INFO:tasks.cephfs_test_runner:Starting test: test_get_authorized_ids (tasks.cephfs.test_volume_client.TestVolumeClient) 2020-11-20T12:20:55.322 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_authorized_ids' 2020-11-20T12:20:55.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.503+0000 7ff5c1cdd700 1 Processor -- start 2020-11-20T12:20:55.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.507+0000 7ff5c1cdd700 1 -- start start 2020-11-20T12:20:55.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.507+0000 7ff5c1cdd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff5bc0c93a0 0x7ff5bc0c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:55.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.507+0000 7ff5c1cdd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5bc0a55f0 0x7ff5bc0c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:55.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.507+0000 7ff5c1cdd700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5bc0c35e0 con 0x7ff5bc0c3e10 2020-11-20T12:20:55.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.507+0000 7ff5c1cdd700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc0c3880 con 0x7ff5bc0a55f0 2020-11-20T12:20:55.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.507+0000 7ff5c1cdd700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc0c7280 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.507+0000 7ff5bbfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5bc0a55f0 0x7ff5bc0c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:55.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.507+0000 7ff5bbfff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff5bc0a55f0 0x7ff5bc0c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50162/0 (socket says 172.21.15.116:50162) 2020-11-20T12:20:55.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.507+0000 7ff5bbfff700 1 -- 172.21.15.116:0/1181021163 learned_addr learned my addr 172.21.15.116:0/1181021163 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:55.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.507+0000 7ff5baffd700 1 --2- 172.21.15.116:0/1181021163 >> v2:172.21.15.173:3300/0 conn(0x7ff5bc0c93a0 0x7ff5bc0c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:55.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5bbfff700 1 -- 172.21.15.116:0/1181021163 >> v2:172.21.15.173:3300/0 conn(0x7ff5bc0c93a0 msgr2=0x7ff5bc0c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:55.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5bbfff700 1 --2- 172.21.15.116:0/1181021163 >> v2:172.21.15.173:3300/0 conn(0x7ff5bc0c93a0 0x7ff5bc0c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:55.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5bbfff700 1 -- 172.21.15.116:0/1181021163 >> v1:172.21.15.173:6789/0 conn(0x7ff5bc0c3e10 legacy=0x7ff5bc0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:55.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5ba7fc700 1 -- 172.21.15.116:0/1181021163 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3022494380 0 0) 0x7ff5bc0c35e0 con 0x7ff5bc0c3e10 2020-11-20T12:20:55.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5bbfff700 1 -- 172.21.15.116:0/1181021163 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc0c7520 con 0x7ff5bc0a55f0 2020-11-20T12:20:55.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5bbfff700 1 --2- 172.21.15.116:0/1181021163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5bc0a55f0 0x7ff5bc0c80d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7ff5b4009d50 tx=0x7ff5b4009290).ready entity=mon.0 client_cookie=633ae9e7d4bac172 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:55.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5ba7fc700 1 -- 172.21.15.116:0/1181021163 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b400bb80 con 0x7ff5bc0a55f0 2020-11-20T12:20:55.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5ba7fc700 1 -- 172.21.15.116:0/1181021163 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b400bce0 con 0x7ff5bc0a55f0 2020-11-20T12:20:55.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5ba7fc700 1 -- 172.21.15.116:0/1181021163 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4011610 con 0x7ff5bc0a55f0 2020-11-20T12:20:55.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/1181021163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5bc0a55f0 msgr2=0x7ff5bc0c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:55.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5c1cdd700 1 --2- 172.21.15.116:0/1181021163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5bc0a55f0 0x7ff5bc0c80d0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7ff5b4009d50 tx=0x7ff5b4009290).stop 2020-11-20T12:20:55.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/1181021163 shutdown_connections 2020-11-20T12:20:55.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5c1cdd700 1 --2- 172.21.15.116:0/1181021163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5bc0a55f0 0x7ff5bc0c80d0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:55.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5c1cdd700 1 --2- 172.21.15.116:0/1181021163 >> v2:172.21.15.173:3300/0 conn(0x7ff5bc0c93a0 0x7ff5bc0c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:55.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/1181021163 >> 172.21.15.116:0/1181021163 conn(0x7ff5bc068080 msgr2=0x7ff5bc0c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:55.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/1181021163 shutdown_connections 2020-11-20T12:20:55.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.511+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/1181021163 wait complete. 2020-11-20T12:20:55.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.515+0000 7ff5c1cdd700 1 Processor -- start 2020-11-20T12:20:55.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.515+0000 7ff5c1cdd700 1 -- start start 2020-11-20T12:20:55.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5c1cdd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5bc0a55f0 0x7ff5bc1515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:55.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5c1cdd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5bc0c3e10 0x7ff5bc094930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:55.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5c1cdd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5bc0c93a0 0x7ff5bc0a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:55.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5c1cdd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc165590 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5c1cdd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc164670 con 0x7ff5bc0c3e10 2020-11-20T12:20:55.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5c1cdd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc164910 con 0x7ff5bc0a55f0 2020-11-20T12:20:55.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5bbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5bc0c93a0 0x7ff5bc0a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5bbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5bc0c93a0 0x7ff5bc0a63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50168/0 (socket says 172.21.15.116:50168) 2020-11-20T12:20:55.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5bbfff700 1 -- 172.21.15.116:0/115294163 learned_addr learned my addr 172.21.15.116:0/115294163 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:55.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5baffd700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5bc0c3e10 0x7ff5bc094930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:55.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5bb7fe700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5bc0a55f0 0x7ff5bc1515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:55.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5bbfff700 1 -- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5bc0a55f0 msgr2=0x7ff5bc1515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:55.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5bbfff700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5bc0a55f0 0x7ff5bc1515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:55.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5bbfff700 1 -- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5bc0c3e10 msgr2=0x7ff5bc094930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:55.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5bbfff700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5bc0c3e10 0x7ff5bc094930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:55.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5bbfff700 1 -- 172.21.15.116:0/115294163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc164bb0 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5bbfff700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5bc0c93a0 0x7ff5bc0a63e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7ff5bc0810b0 tx=0x7ff5b4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:55.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff59b7fe700 1 -- 172.21.15.116:0/115294163 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4011e30 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff59b7fe700 1 -- 172.21.15.116:0/115294163 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b400be50 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/115294163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc164e50 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff59b7fe700 1 -- 172.21.15.116:0/115294163 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4010cb0 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.519+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/115294163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc095220 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.523+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/115294163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5bc05d2a0 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.523+0000 7ff59b7fe700 1 -- 172.21.15.116:0/115294163 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff5b4017070 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.523+0000 7ff59b7fe700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5940557c0 0x7ff594057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:55.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.523+0000 7ff59b7fe700 1 -- 172.21.15.116:0/115294163 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff5b4070690 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.523+0000 7ff5bb7fe700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5940557c0 0x7ff594057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:55.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.523+0000 7ff5bb7fe700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5940557c0 0x7ff594057c80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7ff5ac007fd0 tx=0x7ff5ac007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:55.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.527+0000 7ff59b7fe700 1 -- 172.21.15.116:0/115294163 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff5b40465b0 con 0x7ff5bc0c93a0 2020-11-20T12:20:55.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:55.743+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/115294163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_authorized_ids"]} v 0) v1 -- 0x7ff5bc0ab380 con 0x7ff5bc0c93a0 2020-11-20T12:20:56.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.295+0000 7ff59b7fe700 1 -- 172.21.15.116:0/115294163 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_authorized_ids"]}]=0 v285) v1 ==== 154+0+0 (secure 0 0 0) 0x7ff5b4015020 con 0x7ff5bc0c93a0 2020-11-20T12:20:56.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5940557c0 msgr2=0x7ff594057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:56.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5940557c0 0x7ff594057c80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7ff5ac007fd0 tx=0x7ff5ac007a00).stop 2020-11-20T12:20:56.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/115294163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5bc0c93a0 msgr2=0x7ff5bc0a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:56.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5bc0c93a0 0x7ff5bc0a63e0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7ff5bc0810b0 tx=0x7ff5b4016000).stop 2020-11-20T12:20:56.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/115294163 shutdown_connections 2020-11-20T12:20:56.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff5940557c0 0x7ff594057c80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff5bc0c93a0 0x7ff5bc0a63e0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff5bc0a55f0 0x7ff5bc1515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 --2- 172.21.15.116:0/115294163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff5bc0c3e10 0x7ff5bc094930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/115294163 >> 172.21.15.116:0/115294163 conn(0x7ff5bc068080 msgr2=0x7ff5bc0621c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:56.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/115294163 shutdown_connections 2020-11-20T12:20:56.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.299+0000 7ff5c1cdd700 1 -- 172.21.15.116:0/115294163 wait complete. 2020-11-20T12:20:56.324 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-20T12:20:56.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.507+0000 7f6ec0838700 1 Processor -- start 2020-11-20T12:20:56.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.507+0000 7f6ec0838700 1 -- start start 2020-11-20T12:20:56.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.511+0000 7f6ec0838700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6ebc0c8fa0 0x7f6ebc152990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.511+0000 7f6ec0838700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc0abc10 con 0x7f6ebc14bee0 2020-11-20T12:20:56.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.511+0000 7f6ec0838700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ebc0acf60 con 0x7f6ebc0616b0 2020-11-20T12:20:56.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.511+0000 7f6ec0838700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ebc152e80 con 0x7f6ebc0c8fa0 2020-11-20T12:20:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.511+0000 7f6eba59c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6ebc14bee0 0x7f6ebc0adba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59606/0 (socket says 172.21.15.116:59606) 2020-11-20T12:20:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.511+0000 7f6eba59c700 1 -- 172.21.15.116:0/1595177041 learned_addr learned my addr 172.21.15.116:0/1595177041 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb9d9b700 1 --2- 172.21.15.116:0/1595177041 >> v2:172.21.15.173:3300/0 conn(0x7f6ebc0c8fa0 0x7f6ebc152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577233649 0 0) 0x7f6ebc0abc10 con 0x7f6ebc14bee0 2020-11-20T12:20:56.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea0003240 con 0x7f6ebc14bee0 2020-11-20T12:20:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1450030446 0 0) 0x7f6ebc0acf60 con 0x7f6ebc0616b0 2020-11-20T12:20:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ebc0abc10 con 0x7f6ebc0616b0 2020-11-20T12:20:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2778804254 0 0) 0x7f6ea0003240 con 0x7f6ebc14bee0 2020-11-20T12:20:56.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 >> v2:172.21.15.173:3300/0 conn(0x7f6ebc0c8fa0 msgr2=0x7f6ebc152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 --2- 172.21.15.116:0/1595177041 >> v2:172.21.15.173:3300/0 conn(0x7f6ebc0c8fa0 0x7f6ebc152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 >> v1:172.21.15.173:6789/0 conn(0x7f6ebc0616b0 legacy=0x7f6ebc153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc0c9520 con 0x7f6ebc14bee0 2020-11-20T12:20:56.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f6eac0014e0 con 0x7f6ebc14bee0 2020-11-20T12:20:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 >> v1:172.21.15.116:6789/0 conn(0x7f6ebc14bee0 legacy=0x7f6ebc0adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 --2- 172.21.15.116:0/1595177041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ea0003910 0x7f6ea0003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 --2- 172.21.15.116:0/1595177041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ea00044e0 0x7f6ea0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 --2- 172.21.15.116:0/1595177041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ea0004ea0 0x7f6ea000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc152e80 con 0x7f6ea0003910 2020-11-20T12:20:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea000bc50 con 0x7f6ea0004ea0 2020-11-20T12:20:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea000be10 con 0x7f6ea00044e0 2020-11-20T12:20:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eba59c700 1 --2- 172.21.15.116:0/1595177041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ea0003910 0x7f6ea0003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6ebad9d700 1 --2- 172.21.15.116:0/1595177041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ea0004ea0 0x7f6ea000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eb9d9b700 1 --2- 172.21.15.116:0/1595177041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ea00044e0 0x7f6ea0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eba59c700 1 -- 172.21.15.116:0/1595177041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ea00044e0 msgr2=0x7f6ea0004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eba59c700 1 --2- 172.21.15.116:0/1595177041 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ea00044e0 0x7f6ea0004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eba59c700 1 -- 172.21.15.116:0/1595177041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ea0004ea0 msgr2=0x7f6ea000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eba59c700 1 --2- 172.21.15.116:0/1595177041 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ea0004ea0 0x7f6ea000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6eba59c700 1 -- 172.21.15.116:0/1595177041 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea0003240 con 0x7f6ea0003910 2020-11-20T12:20:56.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.515+0000 7f6ebad9d700 1 -- 172.21.15.116:0/1595177041 reap_dead start 2020-11-20T12:20:56.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.519+0000 7f6eba59c700 1 --2- 172.21.15.116:0/1595177041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ea0003910 0x7f6ea0003d10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f6eac00c210 tx=0x7f6ebc0c96b0).ready entity=mon.0 client_cookie=a3ad1e0ae66cc39d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:56.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.519+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac00ce60 con 0x7f6ea0003910 2020-11-20T12:20:56.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.519+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eac0070d0 con 0x7f6ea0003910 2020-11-20T12:20:56.523 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.519+0000 7f6eb959a700 1 -- 172.21.15.116:0/1595177041 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eac007490 con 0x7f6ea0003910 2020-11-20T12:20:56.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.519+0000 7f6ec0838700 1 -- 172.21.15.116:0/1595177041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ea0003910 msgr2=0x7f6ea0003d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:56.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.519+0000 7f6ec0838700 1 --2- 172.21.15.116:0/1595177041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ea0003910 0x7f6ea0003d10 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f6eac00c210 tx=0x7f6ebc0c96b0).stop 2020-11-20T12:20:56.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.519+0000 7f6ec0838700 1 -- 172.21.15.116:0/1595177041 shutdown_connections 2020-11-20T12:20:56.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.519+0000 7f6ec0838700 1 --2- 172.21.15.116:0/1595177041 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ea0003910 0x7f6ea0003d10 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.519+0000 7f6ec0838700 1 -- 172.21.15.116:0/1595177041 >> 172.21.15.116:0/1595177041 conn(0x7f6ebc068370 msgr2=0x7f6ebc0c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:56.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.519+0000 7f6ec0838700 1 -- 172.21.15.116:0/1595177041 shutdown_connections 2020-11-20T12:20:56.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.519+0000 7f6ec0838700 1 -- 172.21.15.116:0/1595177041 wait complete. 2020-11-20T12:20:56.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ec0838700 1 Processor -- start 2020-11-20T12:20:56.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ec0838700 1 -- start start 2020-11-20T12:20:56.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ec0838700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ea0003910 0x7f6ebc137170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ec0838700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ebc0c5660 0x7f6ebc14aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ec0838700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ebc0c5ae0 0x7f6ebc146bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ec0838700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc149e70 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ec0838700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc14a110 con 0x7f6ea0003910 2020-11-20T12:20:56.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ec0838700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ebc14a3b0 con 0x7f6ebc0c5660 2020-11-20T12:20:56.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ebad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ebc0c5ae0 0x7f6ebc146bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ebad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ebc0c5ae0 0x7f6ebc146bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50188/0 (socket says 172.21.15.116:50188) 2020-11-20T12:20:56.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ebad9d700 1 -- 172.21.15.116:0/1575581851 learned_addr learned my addr 172.21.15.116:0/1575581851 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:56.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6eb9d9b700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ebc0c5660 0x7f6ebc14aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6eba59c700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ea0003910 0x7f6ebc137170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ebad9d700 1 -- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ebc0c5660 msgr2=0x7f6ebc14aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:56.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ebad9d700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ebc0c5660 0x7f6ebc14aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ebad9d700 1 -- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ea0003910 msgr2=0x7f6ebc137170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:56.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ebad9d700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ea0003910 0x7f6ebc137170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ebad9d700 1 -- 172.21.15.116:0/1575581851 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc14a650 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.523+0000 7f6ebad9d700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ebc0c5ae0 0x7f6ebc146bf0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f6ebc068820 tx=0x7f6eb4007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:56.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.527+0000 7f6eaa7fc700 1 -- 172.21.15.116:0/1575581851 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4006a80 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.527+0000 7f6eaa7fc700 1 -- 172.21.15.116:0/1575581851 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4006be0 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.527+0000 7f6eaa7fc700 1 -- 172.21.15.116:0/1575581851 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4002020 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.527+0000 7f6ec0838700 1 -- 172.21.15.116:0/1575581851 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ebc149b80 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.527+0000 7f6ec0838700 1 -- 172.21.15.116:0/1575581851 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ebc1493e0 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.527+0000 7f6ec0838700 1 -- 172.21.15.116:0/1575581851 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ebc062690 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.527+0000 7f6eaa7fc700 1 -- 172.21.15.116:0/1575581851 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6eb40023d0 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.527+0000 7f6eaa7fc700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e900557c0 0x7f6e90057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.527+0000 7f6eaa7fc700 1 -- 172.21.15.116:0/1575581851 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6eb406e4e0 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.531+0000 7f6eba59c700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e900557c0 0x7f6e90057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.531+0000 7f6eba59c700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e900557c0 0x7f6e90057c80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f6eac007970 tx=0x7f6eac013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:56.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.535+0000 7f6eaa7fc700 1 -- 172.21.15.116:0/1575581851 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6ebc062690 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.691+0000 7f6ec0838700 1 -- 172.21.15.116:0/1575581851 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f6ebc062690 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6eaa7fc700 1 -- 172.21.15.116:0/1575581851 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ebc062690 con 0x7f6ebc0c5ae0 2020-11-20T12:20:56.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 -- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e900557c0 msgr2=0x7f6e90057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:56.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e900557c0 0x7f6e90057c80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f6eac007970 tx=0x7f6eac013000).stop 2020-11-20T12:20:56.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 -- 172.21.15.116:0/1575581851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ebc0c5ae0 msgr2=0x7f6ebc146bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:56.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ebc0c5ae0 0x7f6ebc146bf0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f6ebc068820 tx=0x7f6eb4007f10).stop 2020-11-20T12:20:56.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 -- 172.21.15.116:0/1575581851 shutdown_connections 2020-11-20T12:20:56.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e900557c0 0x7f6e90057c80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ebc0c5ae0 0x7f6ebc146bf0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ea0003910 0x7f6ebc137170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 --2- 172.21.15.116:0/1575581851 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ebc0c5660 0x7f6ebc14aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 -- 172.21.15.116:0/1575581851 >> 172.21.15.116:0/1575581851 conn(0x7f6ebc068370 msgr2=0x7f6ebc14fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:56.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 -- 172.21.15.116:0/1575581851 shutdown_connections 2020-11-20T12:20:56.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.703+0000 7f6ec0838700 1 -- 172.21.15.116:0/1575581851 wait complete. 2020-11-20T12:20:56.730 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:20:56.731 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:20:56.769 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:20:56.807 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:20:56.808 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2020-11-20T12:20:56.809 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:56.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.983+0000 7f2e6a672700 1 Processor -- start 2020-11-20T12:20:56.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.987+0000 7f2e6a672700 1 -- start start 2020-11-20T12:20:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.987+0000 7f2e6a672700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2e64062880 0x7f2e64062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e6a672700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e640ba1b0 con 0x7f2e640b9bf0 2020-11-20T12:20:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e6a672700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e641555a0 con 0x7f2e6414b990 2020-11-20T12:20:56.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e6a672700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e64155840 con 0x7f2e64062880 2020-11-20T12:20:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e69e71700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2e640b9bf0 0x7f2e64151e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59630/0 (socket says 172.21.15.116:59630) 2020-11-20T12:20:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e69e71700 1 -- 172.21.15.116:0/516980384 learned_addr learned my addr 172.21.15.116:0/516980384 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e69670700 1 --2- 172.21.15.116:0/516980384 >> v2:172.21.15.173:3300/0 conn(0x7f2e64062880 0x7f2e64062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248772419 0 0) 0x7f2e640ba1b0 con 0x7f2e640b9bf0 2020-11-20T12:20:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e4c003240 con 0x7f2e640b9bf0 2020-11-20T12:20:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2813789485 0 0) 0x7f2e641555a0 con 0x7f2e6414b990 2020-11-20T12:20:56.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e640ba1b0 con 0x7f2e6414b990 2020-11-20T12:20:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3907101360 0 0) 0x7f2e4c003240 con 0x7f2e640b9bf0 2020-11-20T12:20:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 >> v2:172.21.15.173:3300/0 conn(0x7f2e64062880 msgr2=0x7f2e64062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:56.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 --2- 172.21.15.116:0/516980384 >> v2:172.21.15.173:3300/0 conn(0x7f2e64062880 0x7f2e64062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 >> v1:172.21.15.173:6789/0 conn(0x7f2e6414b990 legacy=0x7f2e640b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:56.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e64155ae0 con 0x7f2e640b9bf0 2020-11-20T12:20:56.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f2e5c001860 con 0x7f2e640b9bf0 2020-11-20T12:20:56.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 >> v1:172.21.15.116:6789/0 conn(0x7f2e640b9bf0 legacy=0x7f2e64151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:56.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0038c0 0x7f2e4c003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e4c0044e0 0x7f2e4c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e4c004e50 0x7f2e4c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:56.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64155840 con 0x7f2e4c0038c0 2020-11-20T12:20:56.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c00bc20 con 0x7f2e4c004e50 2020-11-20T12:20:56.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c00bde0 con 0x7f2e4c0044e0 2020-11-20T12:20:56.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e68e6f700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e4c0044e0 0x7f2e4c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e69670700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0038c0 0x7f2e4c003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e69e71700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e4c004e50 0x7f2e4c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:56.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e68e6f700 1 -- 172.21.15.116:0/516980384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e4c004e50 msgr2=0x7f2e4c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:56.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e68e6f700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e4c004e50 0x7f2e4c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e68e6f700 1 -- 172.21.15.116:0/516980384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0038c0 msgr2=0x7f2e4c003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:56.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e68e6f700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0038c0 0x7f2e4c003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:56.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e68e6f700 1 -- 172.21.15.116:0/516980384 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c003240 con 0x7f2e4c0044e0 2020-11-20T12:20:56.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.991+0000 7f2e69670700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0038c0 0x7f2e4c003cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e69e71700 1 -- 172.21.15.116:0/516980384 reap_dead start 2020-11-20T12:20:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e68e6f700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e4c0044e0 0x7f2e4c004960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f2e54001a80 tx=0x7f2e54007b90).ready entity=mon.2 client_cookie=75168194dcd2746b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e54007020 con 0x7f2e4c0044e0 2020-11-20T12:20:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e54007180 con 0x7f2e4c0044e0 2020-11-20T12:20:56.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e53fff700 1 -- 172.21.15.116:0/516980384 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e5400c890 con 0x7f2e4c0044e0 2020-11-20T12:20:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e6a672700 1 -- 172.21.15.116:0/516980384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e4c0044e0 msgr2=0x7f2e4c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e6a672700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e4c0044e0 0x7f2e4c004960 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f2e54001a80 tx=0x7f2e54007b90).stop 2020-11-20T12:20:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e6a672700 1 -- 172.21.15.116:0/516980384 shutdown_connections 2020-11-20T12:20:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e6a672700 1 --2- 172.21.15.116:0/516980384 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e4c0044e0 0x7f2e4c004960 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e6a672700 1 -- 172.21.15.116:0/516980384 >> 172.21.15.116:0/516980384 conn(0x7f2e64060c40 msgr2=0x7f2e640c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e6a672700 1 -- 172.21.15.116:0/516980384 shutdown_connections 2020-11-20T12:20:57.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.995+0000 7f2e6a672700 1 -- 172.21.15.116:0/516980384 wait complete. 2020-11-20T12:20:57.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.999+0000 7f2e6a672700 1 Processor -- start 2020-11-20T12:20:57.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.999+0000 7f2e6a672700 1 -- start start 2020-11-20T12:20:57.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.999+0000 7f2e6a672700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0044e0 0x7f2e64161580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.999+0000 7f2e6a672700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e640a49b0 0x7f2e64146780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.999+0000 7f2e6a672700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e640a6490 0x7f2e640a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.999+0000 7f2e6a672700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640a6de0 con 0x7f2e4c0044e0 2020-11-20T12:20:57.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.999+0000 7f2e6a672700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e64161a70 con 0x7f2e640a49b0 2020-11-20T12:20:57.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.999+0000 7f2e6a672700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e640a4e30 con 0x7f2e640a6490 2020-11-20T12:20:57.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.999+0000 7f2e69670700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0044e0 0x7f2e64161580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.999+0000 7f2e69670700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0044e0 0x7f2e64161580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50204/0 (socket says 172.21.15.116:50204) 2020-11-20T12:20:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:56.999+0000 7f2e69670700 1 -- 172.21.15.116:0/2382893719 learned_addr learned my addr 172.21.15.116:0/2382893719 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e69e71700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e640a6490 0x7f2e640a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e68e6f700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e640a49b0 0x7f2e64146780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e69670700 1 -- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e640a6490 msgr2=0x7f2e640a68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e69670700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e640a6490 0x7f2e640a68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e69670700 1 -- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e640a49b0 msgr2=0x7f2e64146780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e69670700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e640a49b0 0x7f2e64146780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e69670700 1 -- 172.21.15.116:0/2382893719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e640a50d0 con 0x7f2e4c0044e0 2020-11-20T12:20:57.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e69e71700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e640a6490 0x7f2e640a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:57.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e68e6f700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e640a49b0 0x7f2e64146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:57.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e69670700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0044e0 0x7f2e64161580 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f2e60011d20 tx=0x7f2e60006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:57.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e517fa700 1 -- 172.21.15.116:0/2382893719 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e6000fcd0 con 0x7f2e4c0044e0 2020-11-20T12:20:57.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e517fa700 1 -- 172.21.15.116:0/2382893719 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e6000fe30 con 0x7f2e4c0044e0 2020-11-20T12:20:57.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e6a672700 1 -- 172.21.15.116:0/2382893719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e640a5370 con 0x7f2e4c0044e0 2020-11-20T12:20:57.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e517fa700 1 -- 172.21.15.116:0/2382893719 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e60014650 con 0x7f2e4c0044e0 2020-11-20T12:20:57.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e6a672700 1 -- 172.21.15.116:0/2382893719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e640a58b0 con 0x7f2e4c0044e0 2020-11-20T12:20:57.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e6a672700 1 -- 172.21.15.116:0/2382893719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e6405cf30 con 0x7f2e4c0044e0 2020-11-20T12:20:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.003+0000 7f2e517fa700 1 -- 172.21.15.116:0/2382893719 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2e60014a40 con 0x7f2e4c0044e0 2020-11-20T12:20:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.007+0000 7f2e517fa700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e3c0557c0 0x7f2e3c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.007+0000 7f2e517fa700 1 -- 172.21.15.116:0/2382893719 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2e6006f3e0 con 0x7f2e4c0044e0 2020-11-20T12:20:57.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.007+0000 7f2e68e6f700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e3c0557c0 0x7f2e3c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.007+0000 7f2e68e6f700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e3c0557c0 0x7f2e3c057c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f2e540077f0 tx=0x7f2e54016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:57.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.011+0000 7f2e517fa700 1 -- 172.21.15.116:0/2382893719 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2e60045280 con 0x7f2e4c0044e0 2020-11-20T12:20:57.214 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.215+0000 7f2e6a672700 1 -- 172.21.15.116:0/2382893719 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2e64061230 con 0x7f2e4c0044e0 2020-11-20T12:20:57.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.215+0000 7f2e517fa700 1 -- 172.21.15.116:0/2382893719 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2848 (secure 0 0 0) 0x7f2e600488a0 con 0x7f2e4c0044e0 2020-11-20T12:20:57.216 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:57.216 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3105911814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3105911814},{"type":"v1","addr":"172.21.15.173:6837","nonce":3105911814}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/3457547857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3457547857},{"type":"v1","addr":"172.21.15.116:6835","nonce":3457547857}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/81195738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":81195738},{"type":"v1","addr":"172.21.15.116:6833","nonce":81195738}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:20:34.449524+0000","modified":"2020-11-20T12:20:36.475264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/2361610151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2361610151},{"type":"v1","addr":"172.21.15.173:6835","nonce":2361610151}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-20T12:20:57.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 -- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e3c0557c0 msgr2=0x7f2e3c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e3c0557c0 0x7f2e3c057c80 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f2e540077f0 tx=0x7f2e54016000).stop 2020-11-20T12:20:57.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 -- 172.21.15.116:0/2382893719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0044e0 msgr2=0x7f2e64161580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.218 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0044e0 0x7f2e64161580 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f2e60011d20 tx=0x7f2e60006df0).stop 2020-11-20T12:20:57.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 -- 172.21.15.116:0/2382893719 shutdown_connections 2020-11-20T12:20:57.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2e3c0557c0 0x7f2e3c057c80 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e640a6490 0x7f2e640a68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e4c0044e0 0x7f2e64161580 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 --2- 172.21.15.116:0/2382893719 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e640a49b0 0x7f2e64146780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 -- 172.21.15.116:0/2382893719 >> 172.21.15.116:0/2382893719 conn(0x7f2e64060c40 msgr2=0x7f2e640be9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:57.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 -- 172.21.15.116:0/2382893719 shutdown_connections 2020-11-20T12:20:57.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.219+0000 7f2e6a672700 1 -- 172.21.15.116:0/2382893719 wait complete. 2020-11-20T12:20:57.221 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 54 2020-11-20T12:20:57.242 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:57.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.419+0000 7f6948ebe700 1 Processor -- start 2020-11-20T12:20:57.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6948ebe700 1 -- start start 2020-11-20T12:20:57.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6948ebe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f69440c3e10 0x7f69440c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6948ebe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69440a55f0 0x7f69440c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6948ebe700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69440c35e0 con 0x7f69440c93a0 2020-11-20T12:20:57.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6948ebe700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69440c3880 con 0x7f69440a55f0 2020-11-20T12:20:57.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6948ebe700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69440c7280 con 0x7f69440c3e10 2020-11-20T12:20:57.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6942d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69440a55f0 0x7f69440c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6942d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f69440a55f0 0x7f69440c80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50216/0 (socket says 172.21.15.116:50216) 2020-11-20T12:20:57.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6942d9d700 1 -- 172.21.15.116:0/652992218 learned_addr learned my addr 172.21.15.116:0/652992218 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:57.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f694259c700 1 --2- 172.21.15.116:0/652992218 >> v2:172.21.15.173:3300/0 conn(0x7f69440c3e10 0x7f69440c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6942d9d700 1 -- 172.21.15.116:0/652992218 >> v2:172.21.15.173:3300/0 conn(0x7f69440c3e10 msgr2=0x7f69440c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:57.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6942d9d700 1 --2- 172.21.15.116:0/652992218 >> v2:172.21.15.173:3300/0 conn(0x7f69440c3e10 0x7f69440c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6942d9d700 1 -- 172.21.15.116:0/652992218 >> v1:172.21.15.173:6789/0 conn(0x7f69440c93a0 legacy=0x7f69440c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:57.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6942d9d700 1 -- 172.21.15.116:0/652992218 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440c7520 con 0x7f69440a55f0 2020-11-20T12:20:57.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f694259c700 1 --2- 172.21.15.116:0/652992218 >> v2:172.21.15.173:3300/0 conn(0x7f69440c3e10 0x7f69440c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:57.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.423+0000 7f6942d9d700 1 --2- 172.21.15.116:0/652992218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69440a55f0 0x7f69440c80d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f693c009d50 tx=0x7f693c009290).ready entity=mon.0 client_cookie=9adc3ab8f5b80ede server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:57.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.427+0000 7f6941d9b700 1 -- 172.21.15.116:0/652992218 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c00bc00 con 0x7f69440a55f0 2020-11-20T12:20:57.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.427+0000 7f6941d9b700 1 -- 172.21.15.116:0/652992218 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f693c00bd60 con 0x7f69440a55f0 2020-11-20T12:20:57.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.427+0000 7f6941d9b700 1 -- 172.21.15.116:0/652992218 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693c011610 con 0x7f69440a55f0 2020-11-20T12:20:57.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.427+0000 7f6948ebe700 1 -- 172.21.15.116:0/652992218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69440a55f0 msgr2=0x7f69440c80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.427+0000 7f6948ebe700 1 --2- 172.21.15.116:0/652992218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69440a55f0 0x7f69440c80d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7f693c009d50 tx=0x7f693c009290).stop 2020-11-20T12:20:57.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.427+0000 7f6948ebe700 1 -- 172.21.15.116:0/652992218 shutdown_connections 2020-11-20T12:20:57.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.427+0000 7f6948ebe700 1 --2- 172.21.15.116:0/652992218 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69440a55f0 0x7f69440c80d0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.427+0000 7f6948ebe700 1 --2- 172.21.15.116:0/652992218 >> v2:172.21.15.173:3300/0 conn(0x7f69440c3e10 0x7f69440c41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.427+0000 7f6948ebe700 1 -- 172.21.15.116:0/652992218 >> 172.21.15.116:0/652992218 conn(0x7f6944068080 msgr2=0x7f69440c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:57.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.427+0000 7f6948ebe700 1 -- 172.21.15.116:0/652992218 shutdown_connections 2020-11-20T12:20:57.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.427+0000 7f6948ebe700 1 -- 172.21.15.116:0/652992218 wait complete. 2020-11-20T12:20:57.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.431+0000 7f6948ebe700 1 Processor -- start 2020-11-20T12:20:57.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.431+0000 7f6948ebe700 1 -- start start 2020-11-20T12:20:57.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.431+0000 7f6948ebe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69440a55f0 0x7f69440a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.431+0000 7f6948ebe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69440c3e10 0x7f6944151870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.431+0000 7f6948ebe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69440c93a0 0x7f69440a63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.431+0000 7f6948ebe700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440a5f00 con 0x7f69440c3e10 2020-11-20T12:20:57.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.431+0000 7f6948ebe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69440a99d0 con 0x7f69440a55f0 2020-11-20T12:20:57.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.431+0000 7f6948ebe700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69440a9c70 con 0x7f69440c93a0 2020-11-20T12:20:57.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.431+0000 7f693bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69440c3e10 0x7f6944151870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.431+0000 7f693bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69440c3e10 0x7f6944151870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50220/0 (socket says 172.21.15.116:50220) 2020-11-20T12:20:57.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.431+0000 7f693bfff700 1 -- 172.21.15.116:0/1110536924 learned_addr learned my addr 172.21.15.116:0/1110536924 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:57.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f6942d9d700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69440c93a0 0x7f69440a63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f694259c700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69440a55f0 0x7f69440a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f693bfff700 1 -- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69440c93a0 msgr2=0x7f69440a63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f693bfff700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69440c93a0 0x7f69440a63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f693bfff700 1 -- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69440a55f0 msgr2=0x7f69440a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f693bfff700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69440a55f0 0x7f69440a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f693bfff700 1 -- 172.21.15.116:0/1110536924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440a9f10 con 0x7f69440c3e10 2020-11-20T12:20:57.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f693bfff700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69440c3e10 0x7f6944151870 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f692c0096e0 tx=0x7f692c006d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:57.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f693a7fc700 1 -- 172.21.15.116:0/1110536924 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c00d490 con 0x7f69440c3e10 2020-11-20T12:20:57.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f693a7fc700 1 -- 172.21.15.116:0/1110536924 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692c000e90 con 0x7f69440c3e10 2020-11-20T12:20:57.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f693a7fc700 1 -- 172.21.15.116:0/1110536924 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692c00d860 con 0x7f69440c3e10 2020-11-20T12:20:57.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f6948ebe700 1 -- 172.21.15.116:0/1110536924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69440aa1b0 con 0x7f69440c3e10 2020-11-20T12:20:57.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f6948ebe700 1 -- 172.21.15.116:0/1110536924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69440a96a0 con 0x7f69440c3e10 2020-11-20T12:20:57.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.435+0000 7f6948ebe700 1 -- 172.21.15.116:0/1110536924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f694405d2a0 con 0x7f69440c3e10 2020-11-20T12:20:57.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.439+0000 7f693a7fc700 1 -- 172.21.15.116:0/1110536924 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f692c00f030 con 0x7f69440c3e10 2020-11-20T12:20:57.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.439+0000 7f693a7fc700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69200557c0 0x7f6920057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.439+0000 7f693a7fc700 1 -- 172.21.15.116:0/1110536924 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f692c06e1e0 con 0x7f69440c3e10 2020-11-20T12:20:57.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.439+0000 7f694259c700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69200557c0 0x7f6920057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.443+0000 7f694259c700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69200557c0 0x7f6920057c80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f6944165650 tx=0x7f6934008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:57.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.443+0000 7f693a7fc700 1 -- 172.21.15.116:0/1110536924 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f692c044080 con 0x7f69440c3e10 2020-11-20T12:20:57.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.602+0000 7f6948ebe700 1 -- 172.21.15.116:0/1110536924 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6944060290 con 0x7f69440c3e10 2020-11-20T12:20:57.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.606+0000 7f693a7fc700 1 -- 172.21.15.116:0/1110536924 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f692c0476a0 con 0x7f69440c3e10 2020-11-20T12:20:57.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.606+0000 7f6948ebe700 1 -- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69200557c0 msgr2=0x7f6920057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.606+0000 7f6948ebe700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69200557c0 0x7f6920057c80 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f6944165650 tx=0x7f6934008040).stop 2020-11-20T12:20:57.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.606+0000 7f6948ebe700 1 -- 172.21.15.116:0/1110536924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69440c3e10 msgr2=0x7f6944151870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.606+0000 7f6948ebe700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69440c3e10 0x7f6944151870 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f692c0096e0 tx=0x7f692c006d40).stop 2020-11-20T12:20:57.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.606+0000 7f6948ebe700 1 -- 172.21.15.116:0/1110536924 shutdown_connections 2020-11-20T12:20:57.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.606+0000 7f6948ebe700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69200557c0 0x7f6920057c80 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.606+0000 7f6948ebe700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69440c93a0 0x7f69440a63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.606+0000 7f6948ebe700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69440a55f0 0x7f69440a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.606+0000 7f6948ebe700 1 --2- 172.21.15.116:0/1110536924 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f69440c3e10 0x7f6944151870 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.606+0000 7f6948ebe700 1 -- 172.21.15.116:0/1110536924 >> 172.21.15.116:0/1110536924 conn(0x7f6944068080 msgr2=0x7f6944062460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:57.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.610+0000 7f6948ebe700 1 -- 172.21.15.116:0/1110536924 shutdown_connections 2020-11-20T12:20:57.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.610+0000 7f6948ebe700 1 -- 172.21.15.116:0/1110536924 wait complete. 2020-11-20T12:20:57.614 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:57.614 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":96,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:20:35.459061+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:20:32.886768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-20T12:20:33.446960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:20:57.634 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:20:32.886768+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:20:33.446960+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:20:57.635 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:20:57.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.818+0000 7ff9dd010700 1 Processor -- start 2020-11-20T12:20:57.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.818+0000 7ff9dd010700 1 -- start start 2020-11-20T12:20:57.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.822+0000 7ff9dd010700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9d814f8f0 0x7ff9d81531d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.822+0000 7ff9dd010700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d80c41a0 con 0x7ff9d81536c0 2020-11-20T12:20:57.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.822+0000 7ff9dd010700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d80bd6e0 con 0x7ff9d814f510 2020-11-20T12:20:57.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.822+0000 7ff9dd010700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80c43d0 con 0x7ff9d814f8f0 2020-11-20T12:20:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.822+0000 7ff9dc80f700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff9d81536c0 0x7ff9d80c52f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59664/0 (socket says 172.21.15.116:59664) 2020-11-20T12:20:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.822+0000 7ff9dc80f700 1 -- 172.21.15.116:0/1699805777 learned_addr learned my addr 172.21.15.116:0/1699805777 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:57.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.822+0000 7ff9d77fe700 1 --2- 172.21.15.116:0/1699805777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9d814f8f0 0x7ff9d81531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.822+0000 7ff9d6ffd700 1 -- 172.21.15.116:0/1699805777 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2061024798 0 0) 0x7ff9d80c41a0 con 0x7ff9d81536c0 2020-11-20T12:20:57.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.822+0000 7ff9d6ffd700 1 -- 172.21.15.116:0/1699805777 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9c4003240 con 0x7ff9d81536c0 2020-11-20T12:20:57.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.822+0000 7ff9d6ffd700 1 -- 172.21.15.116:0/1699805777 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3088922484 0 0) 0x7ff9d80bd6e0 con 0x7ff9d814f510 2020-11-20T12:20:57.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.822+0000 7ff9d6ffd700 1 -- 172.21.15.116:0/1699805777 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d80c41a0 con 0x7ff9d814f510 2020-11-20T12:20:57.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9d77fe700 1 -- 172.21.15.116:0/1699805777 >> v1:172.21.15.173:6789/0 conn(0x7ff9d814f510 legacy=0x7ff9d8066a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9d77fe700 1 -- 172.21.15.116:0/1699805777 >> v1:172.21.15.116:6789/0 conn(0x7ff9d81536c0 legacy=0x7ff9d80c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9d77fe700 1 -- 172.21.15.116:0/1699805777 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d80c4670 con 0x7ff9d814f8f0 2020-11-20T12:20:57.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9d77fe700 1 --2- 172.21.15.116:0/1699805777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9d814f8f0 0x7ff9d81531d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7ff9cc00b9d0 tx=0x7ff9cc006df0).ready entity=mon.2 client_cookie=4d07e875d03ee9b9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:57.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9d6ffd700 1 -- 172.21.15.116:0/1699805777 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc009680 con 0x7ff9d814f8f0 2020-11-20T12:20:57.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9d6ffd700 1 -- 172.21.15.116:0/1699805777 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc0097e0 con 0x7ff9d814f8f0 2020-11-20T12:20:57.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9d6ffd700 1 -- 172.21.15.116:0/1699805777 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc009b60 con 0x7ff9d814f8f0 2020-11-20T12:20:57.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9dd010700 1 -- 172.21.15.116:0/1699805777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9d814f8f0 msgr2=0x7ff9d81531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9dd010700 1 --2- 172.21.15.116:0/1699805777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9d814f8f0 0x7ff9d81531d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7ff9cc00b9d0 tx=0x7ff9cc006df0).stop 2020-11-20T12:20:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9dd010700 1 -- 172.21.15.116:0/1699805777 shutdown_connections 2020-11-20T12:20:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9dd010700 1 --2- 172.21.15.116:0/1699805777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9d814f8f0 0x7ff9d81531d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9dd010700 1 -- 172.21.15.116:0/1699805777 >> 172.21.15.116:0/1699805777 conn(0x7ff9d8060c40 msgr2=0x7ff9d80c6640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:57.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9dd010700 1 -- 172.21.15.116:0/1699805777 shutdown_connections 2020-11-20T12:20:57.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.826+0000 7ff9dd010700 1 -- 172.21.15.116:0/1699805777 wait complete. 2020-11-20T12:20:57.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.830+0000 7ff9dd010700 1 Processor -- start 2020-11-20T12:20:57.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.830+0000 7ff9dd010700 1 -- start start 2020-11-20T12:20:57.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9dd010700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d814f510 0x7ff9d81647f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9dd010700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9d814f8f0 0x7ff9d80a6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9dd010700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9d81536c0 0x7ff9d80a7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9dd010700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80a78e0 con 0x7ff9d814f510 2020-11-20T12:20:57.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9dd010700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80a7b80 con 0x7ff9d814f8f0 2020-11-20T12:20:57.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9dd010700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d80a7e20 con 0x7ff9d81536c0 2020-11-20T12:20:57.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9d7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d814f510 0x7ff9d81647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9d7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d814f510 0x7ff9d81647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50232/0 (socket says 172.21.15.116:50232) 2020-11-20T12:20:57.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9d7fff700 1 -- 172.21.15.116:0/3194178478 learned_addr learned my addr 172.21.15.116:0/3194178478 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:57.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9dc80f700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9d81536c0 0x7ff9d80a7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9d77fe700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9d814f8f0 0x7ff9d80a6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9d7fff700 1 -- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9d81536c0 msgr2=0x7ff9d80a7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9d7fff700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9d81536c0 0x7ff9d80a7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9d7fff700 1 -- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9d814f8f0 msgr2=0x7ff9d80a6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:57.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9d7fff700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9d814f8f0 0x7ff9d80a6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:57.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9d7fff700 1 -- 172.21.15.116:0/3194178478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d8061400 con 0x7ff9d814f510 2020-11-20T12:20:57.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9d7fff700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d814f510 0x7ff9d81647f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7ff9c800c6d0 tx=0x7ff9c8007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:57.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9b7fff700 1 -- 172.21.15.116:0/3194178478 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c800e490 con 0x7ff9d814f510 2020-11-20T12:20:57.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9b7fff700 1 -- 172.21.15.116:0/3194178478 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9c800e5f0 con 0x7ff9d814f510 2020-11-20T12:20:57.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9b7fff700 1 -- 172.21.15.116:0/3194178478 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c800e990 con 0x7ff9d814f510 2020-11-20T12:20:57.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9dd010700 1 -- 172.21.15.116:0/3194178478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d81652b0 con 0x7ff9d814f510 2020-11-20T12:20:57.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.834+0000 7ff9dd010700 1 -- 172.21.15.116:0/3194178478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d8164f00 con 0x7ff9d814f510 2020-11-20T12:20:57.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.838+0000 7ff9dd010700 1 -- 172.21.15.116:0/3194178478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d80a7fe0 con 0x7ff9d814f510 2020-11-20T12:20:57.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.838+0000 7ff9b7fff700 1 -- 172.21.15.116:0/3194178478 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff9c8019030 con 0x7ff9d814f510 2020-11-20T12:20:57.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.838+0000 7ff9b7fff700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9b00557c0 0x7ff9b0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:57.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.838+0000 7ff9b7fff700 1 -- 172.21.15.116:0/3194178478 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff9c806da70 con 0x7ff9d814f510 2020-11-20T12:20:57.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.838+0000 7ff9d77fe700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9b00557c0 0x7ff9b0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:57.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.842+0000 7ff9d77fe700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9b00557c0 0x7ff9b0057c80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff9cc00b150 tx=0x7ff9cc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:57.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:57.846+0000 7ff9b7fff700 1 -- 172.21.15.116:0/3194178478 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff9c80439c0 con 0x7ff9d814f510 2020-11-20T12:20:58.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.006+0000 7ff9dd010700 1 -- 172.21.15.116:0/3194178478 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ff9d805cf30 con 0x7ff9d814f510 2020-11-20T12:20:58.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.006+0000 7ff9b7fff700 1 -- 172.21.15.116:0/3194178478 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v96) v1 ==== 110+0+10 (secure 0 0 0) 0x7ff9c806de60 con 0x7ff9d814f510 2020-11-20T12:20:58.008 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:20:58.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 -- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9b00557c0 msgr2=0x7ff9b0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9b00557c0 0x7ff9b0057c80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff9cc00b150 tx=0x7ff9cc015000).stop 2020-11-20T12:20:58.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 -- 172.21.15.116:0/3194178478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d814f510 msgr2=0x7ff9d81647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d814f510 0x7ff9d81647f0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7ff9c800c6d0 tx=0x7ff9c8007ae0).stop 2020-11-20T12:20:58.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 -- 172.21.15.116:0/3194178478 shutdown_connections 2020-11-20T12:20:58.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff9b00557c0 0x7ff9b0057c80 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff9d81536c0 0x7ff9d80a7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff9d814f510 0x7ff9d81647f0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 --2- 172.21.15.116:0/3194178478 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff9d814f8f0 0x7ff9d80a6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 -- 172.21.15.116:0/3194178478 >> 172.21.15.116:0/3194178478 conn(0x7ff9d8060c40 msgr2=0x7ff9d80bcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 -- 172.21.15.116:0/3194178478 shutdown_connections 2020-11-20T12:20:58.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.010+0000 7ff9dd010700 1 -- 172.21.15.116:0/3194178478 wait complete. 2020-11-20T12:20:58.034 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:20:58.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.214+0000 7f94a4c05700 1 Processor -- start 2020-11-20T12:20:58.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.214+0000 7f94a4c05700 1 -- start start 2020-11-20T12:20:58.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.218+0000 7f94a4c05700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f94a00a50d0 0x7f94a0060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.218+0000 7f94a4c05700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0060660 0x7f94a00a6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.218+0000 7f94a4c05700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a00a7fb0 con 0x7f94a0151960 2020-11-20T12:20:58.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.218+0000 7f94a4c05700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a00c5320 con 0x7f94a00a50d0 2020-11-20T12:20:58.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.218+0000 7f94a4c05700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a00a7a70 con 0x7f94a0060660 2020-11-20T12:20:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.218+0000 7f949effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f94a00a50d0 0x7f94a0060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f94a00a50d0 0x7f94a0060170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50242/0 (socket says 172.21.15.116:50242) 2020-11-20T12:20:58.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949effd700 1 -- 172.21.15.116:0/2827467928 learned_addr learned my addr 172.21.15.116:0/2827467928 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:58.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949ffff700 1 --2- 172.21.15.116:0/2827467928 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0060660 0x7f94a00a6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949e7fc700 1 -- 172.21.15.116:0/2827467928 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 885059959 0 0) 0x7f94a00a7fb0 con 0x7f94a0151960 2020-11-20T12:20:58.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949e7fc700 1 -- 172.21.15.116:0/2827467928 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f948c003240 con 0x7f94a0151960 2020-11-20T12:20:58.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949effd700 1 -- 172.21.15.116:0/2827467928 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0060660 msgr2=0x7f94a00a6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:58.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949effd700 1 --2- 172.21.15.116:0/2827467928 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0060660 0x7f94a00a6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949effd700 1 -- 172.21.15.116:0/2827467928 >> v1:172.21.15.173:6789/0 conn(0x7f94a0151960 legacy=0x7f94a00c5dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949effd700 1 -- 172.21.15.116:0/2827467928 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a00a7d10 con 0x7f94a00a50d0 2020-11-20T12:20:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949ffff700 1 --2- 172.21.15.116:0/2827467928 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0060660 0x7f94a00a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949effd700 1 --2- 172.21.15.116:0/2827467928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a00a50d0 0x7f94a0060170 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f9494006fd0 tx=0x7f9494006eb0).ready entity=mon.0 client_cookie=1ef95ae18d964cc1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949e7fc700 1 -- 172.21.15.116:0/2827467928 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494009810 con 0x7f94a00a50d0 2020-11-20T12:20:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949e7fc700 1 -- 172.21.15.116:0/2827467928 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9494009970 con 0x7f94a00a50d0 2020-11-20T12:20:58.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f949e7fc700 1 -- 172.21.15.116:0/2827467928 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9494009cf0 con 0x7f94a00a50d0 2020-11-20T12:20:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f94a4c05700 1 -- 172.21.15.116:0/2827467928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a00a50d0 msgr2=0x7f94a0060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f94a4c05700 1 --2- 172.21.15.116:0/2827467928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a00a50d0 0x7f94a0060170 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f9494006fd0 tx=0x7f9494006eb0).stop 2020-11-20T12:20:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f94a4c05700 1 -- 172.21.15.116:0/2827467928 shutdown_connections 2020-11-20T12:20:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f94a4c05700 1 --2- 172.21.15.116:0/2827467928 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0060660 0x7f94a00a6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f94a4c05700 1 --2- 172.21.15.116:0/2827467928 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a00a50d0 0x7f94a0060170 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f94a4c05700 1 -- 172.21.15.116:0/2827467928 >> 172.21.15.116:0/2827467928 conn(0x7f94a0061a60 msgr2=0x7f94a00c54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f94a4c05700 1 -- 172.21.15.116:0/2827467928 shutdown_connections 2020-11-20T12:20:58.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.222+0000 7f94a4c05700 1 -- 172.21.15.116:0/2827467928 wait complete. 2020-11-20T12:20:58.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f94a4c05700 1 Processor -- start 2020-11-20T12:20:58.228 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f94a4c05700 1 -- start start 2020-11-20T12:20:58.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f94a4c05700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0060660 0x7f94a0154cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f94a4c05700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0151960 0x7f94a00c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f94a4c05700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a00c1440 0x7f94a0161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f94a4c05700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a01614d0 con 0x7f94a0060660 2020-11-20T12:20:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f94a4c05700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0152970 con 0x7f94a00c1440 2020-11-20T12:20:58.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f94a4c05700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a0152c10 con 0x7f94a0151960 2020-11-20T12:20:58.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0060660 0x7f94a0154cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0060660 0x7f94a0154cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50246/0 (socket says 172.21.15.116:50246) 2020-11-20T12:20:58.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949f7fe700 1 -- 172.21.15.116:0/1476868386 learned_addr learned my addr 172.21.15.116:0/1476868386 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:58.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949ffff700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a00c1440 0x7f94a0161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949effd700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0151960 0x7f94a00c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949f7fe700 1 -- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0151960 msgr2=0x7f94a00c0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949f7fe700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0151960 0x7f94a00c0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949f7fe700 1 -- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a00c1440 msgr2=0x7f94a0161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949f7fe700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a00c1440 0x7f94a0161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949f7fe700 1 -- 172.21.15.116:0/1476868386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a0161730 con 0x7f94a0060660 2020-11-20T12:20:58.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949ffff700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a00c1440 0x7f94a0161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:58.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.226+0000 7f949effd700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0151960 0x7f94a00c0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:20:58.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.230+0000 7f949f7fe700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0060660 0x7f94a0154cb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f949000c710 tx=0x7f9490007a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.230+0000 7f947f7fe700 1 -- 172.21.15.116:0/1476868386 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949000e560 con 0x7f94a0060660 2020-11-20T12:20:58.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.230+0000 7f947f7fe700 1 -- 172.21.15.116:0/1476868386 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949000e6c0 con 0x7f94a0060660 2020-11-20T12:20:58.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.230+0000 7f947f7fe700 1 -- 172.21.15.116:0/1476868386 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949000ea60 con 0x7f94a0060660 2020-11-20T12:20:58.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.230+0000 7f94a4c05700 1 -- 172.21.15.116:0/1476868386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a01526a0 con 0x7f94a0060660 2020-11-20T12:20:58.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.230+0000 7f94a4c05700 1 -- 172.21.15.116:0/1476868386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a01620c0 con 0x7f94a0060660 2020-11-20T12:20:58.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.230+0000 7f94a4c05700 1 -- 172.21.15.116:0/1476868386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a0162370 con 0x7f94a0060660 2020-11-20T12:20:58.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.230+0000 7f947f7fe700 1 -- 172.21.15.116:0/1476868386 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9490010030 con 0x7f94a0060660 2020-11-20T12:20:58.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.234+0000 7f947f7fe700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94780557c0 0x7f9478057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.234+0000 7f947f7fe700 1 -- 172.21.15.116:0/1476868386 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f949006e580 con 0x7f94a0060660 2020-11-20T12:20:58.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.234+0000 7f949effd700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94780557c0 0x7f9478057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.234+0000 7f949effd700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94780557c0 0x7f9478057c80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f94a005c9f0 tx=0x7f9494015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.238+0000 7f947f7fe700 1 -- 172.21.15.116:0/1476868386 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9490044420 con 0x7f94a0060660 2020-11-20T12:20:58.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.398+0000 7f94a4c05700 1 -- 172.21.15.116:0/1476868386 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f94a0161300 con 0x7f94a0060660 2020-11-20T12:20:58.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.398+0000 7f947f7fe700 1 -- 172.21.15.116:0/1476868386 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v96) v1 ==== 92+0+10 (secure 0 0 0) 0x7f9490047a40 con 0x7f94a0060660 2020-11-20T12:20:58.398 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:58.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.398+0000 7f94a4c05700 1 -- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94780557c0 msgr2=0x7f9478057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.398+0000 7f94a4c05700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94780557c0 0x7f9478057c80 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f94a005c9f0 tx=0x7f9494015000).stop 2020-11-20T12:20:58.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.398+0000 7f94a4c05700 1 -- 172.21.15.116:0/1476868386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0060660 msgr2=0x7f94a0154cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.398+0000 7f94a4c05700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0060660 0x7f94a0154cb0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f949000c710 tx=0x7f9490007a90).stop 2020-11-20T12:20:58.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.402+0000 7f94a4c05700 1 -- 172.21.15.116:0/1476868386 shutdown_connections 2020-11-20T12:20:58.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.402+0000 7f94a4c05700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f94780557c0 0x7f9478057c80 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.402+0000 7f94a4c05700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f94a00c1440 0x7f94a0161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.402+0000 7f94a4c05700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f94a0060660 0x7f94a0154cb0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.402+0000 7f94a4c05700 1 --2- 172.21.15.116:0/1476868386 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f94a0151960 0x7f94a00c0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.402+0000 7f94a4c05700 1 -- 172.21.15.116:0/1476868386 >> 172.21.15.116:0/1476868386 conn(0x7f94a0061a60 msgr2=0x7f94a01509c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.402+0000 7f94a4c05700 1 -- 172.21.15.116:0/1476868386 shutdown_connections 2020-11-20T12:20:58.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.402+0000 7f94a4c05700 1 -- 172.21.15.116:0/1476868386 wait complete. 2020-11-20T12:20:58.428 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:20:58.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.610+0000 7f4a04926700 1 Processor -- start 2020-11-20T12:20:58.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.610+0000 7f4a04926700 1 -- start start 2020-11-20T12:20:58.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f4a04926700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4a0014e590 0x7f4a00063100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f4a04926700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a000635f0 con 0x7f4a000bff10 2020-11-20T12:20:58.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f4a04926700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a000c0690 con 0x7f4a000a66e0 2020-11-20T12:20:58.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f4a04926700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a000c0930 con 0x7f4a0014e590 2020-11-20T12:20:58.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49feffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4a000bff10 0x7f4a0014f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59692/0 (socket says 172.21.15.116:59692) 2020-11-20T12:20:58.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49feffd700 1 -- 172.21.15.116:0/599149757 learned_addr learned my addr 172.21.15.116:0/599149757 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:58.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49ff7fe700 1 --2- 172.21.15.116:0/599149757 >> v2:172.21.15.173:3300/0 conn(0x7f4a0014e590 0x7f4a00063100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984506041 0 0) 0x7f4a000635f0 con 0x7f4a000bff10 2020-11-20T12:20:58.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49ec003240 con 0x7f4a000bff10 2020-11-20T12:20:58.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 109409415 0 0) 0x7f49ec003240 con 0x7f4a000bff10 2020-11-20T12:20:58.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 >> v2:172.21.15.173:3300/0 conn(0x7f4a0014e590 msgr2=0x7f4a00063100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:58.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 --2- 172.21.15.116:0/599149757 >> v2:172.21.15.173:3300/0 conn(0x7f4a0014e590 0x7f4a00063100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 >> v1:172.21.15.173:6789/0 conn(0x7f4a000a66e0 legacy=0x7f4a00066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a000c0bd0 con 0x7f4a000bff10 2020-11-20T12:20:58.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f49f40014e0 con 0x7f4a000bff10 2020-11-20T12:20:58.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 >> v1:172.21.15.116:6789/0 conn(0x7f4a000bff10 legacy=0x7f4a0014f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 --2- 172.21.15.116:0/599149757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec003730 0x7f49ec003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 --2- 172.21.15.116:0/599149757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49ec0044e0 0x7f49ec004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 --2- 172.21.15.116:0/599149757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49ec004e50 0x7f49ec00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000c0930 con 0x7f49ec004e50 2020-11-20T12:20:58.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec005360 con 0x7f49ec003730 2020-11-20T12:20:58.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49ec00bc50 con 0x7f49ec0044e0 2020-11-20T12:20:58.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fffff700 1 --2- 172.21.15.116:0/599149757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49ec004e50 0x7f49ec00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49ff7fe700 1 --2- 172.21.15.116:0/599149757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec003730 0x7f49ec003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49feffd700 1 --2- 172.21.15.116:0/599149757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49ec0044e0 0x7f49ec004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fffff700 1 -- 172.21.15.116:0/599149757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49ec0044e0 msgr2=0x7f49ec004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:58.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fffff700 1 --2- 172.21.15.116:0/599149757 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49ec0044e0 0x7f49ec004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fffff700 1 -- 172.21.15.116:0/599149757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec003730 msgr2=0x7f49ec003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:58.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fffff700 1 --2- 172.21.15.116:0/599149757 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec003730 0x7f49ec003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.614+0000 7f49fffff700 1 -- 172.21.15.116:0/599149757 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ec003240 con 0x7f49ec004e50 2020-11-20T12:20:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f49fffff700 1 -- 172.21.15.116:0/599149757 reap_dead start 2020-11-20T12:20:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f49fffff700 1 --2- 172.21.15.116:0/599149757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49ec004e50 0x7f49ec00b730 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f49f8001e70 tx=0x7f49f8007f10).ready entity=mon.0 client_cookie=116ecff33e720a6c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f8001540 con 0x7f49ec004e50 2020-11-20T12:20:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f800ca70 con 0x7f49ec004e50 2020-11-20T12:20:58.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f49fe7fc700 1 -- 172.21.15.116:0/599149757 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f800ce10 con 0x7f49ec004e50 2020-11-20T12:20:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f4a04926700 1 -- 172.21.15.116:0/599149757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49ec004e50 msgr2=0x7f49ec00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f4a04926700 1 --2- 172.21.15.116:0/599149757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49ec004e50 0x7f49ec00b730 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f49f8001e70 tx=0x7f49f8007f10).stop 2020-11-20T12:20:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f4a04926700 1 -- 172.21.15.116:0/599149757 shutdown_connections 2020-11-20T12:20:58.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f4a04926700 1 --2- 172.21.15.116:0/599149757 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49ec004e50 0x7f49ec00b730 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f4a04926700 1 -- 172.21.15.116:0/599149757 >> 172.21.15.116:0/599149757 conn(0x7f4a00068350 msgr2=0x7f4a000627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f4a04926700 1 -- 172.21.15.116:0/599149757 shutdown_connections 2020-11-20T12:20:58.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f4a04926700 1 -- 172.21.15.116:0/599149757 wait complete. 2020-11-20T12:20:58.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f4a04926700 1 Processor -- start 2020-11-20T12:20:58.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.618+0000 7f4a04926700 1 -- start start 2020-11-20T12:20:58.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f4a04926700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec004e50 0x7f4a00160770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f4a04926700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a00160c60 0x7f4a000a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f4a04926700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a000a9440 0x7f4a000c31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f4a04926700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00150430 con 0x7f4a000a9440 2020-11-20T12:20:58.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f4a04926700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00150620 con 0x7f49ec004e50 2020-11-20T12:20:58.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f4a04926700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a001508c0 con 0x7f4a00160c60 2020-11-20T12:20:58.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49fffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a000a9440 0x7f4a000c31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49fffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a000a9440 0x7f4a000c31a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50270/0 (socket says 172.21.15.116:50270) 2020-11-20T12:20:58.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49fffff700 1 -- 172.21.15.116:0/3585976052 learned_addr learned my addr 172.21.15.116:0/3585976052 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:58.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49ff7fe700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec004e50 0x7f4a00160770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49feffd700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a00160c60 0x7f4a000a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49fffff700 1 -- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a00160c60 msgr2=0x7f4a000a8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49fffff700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a00160c60 0x7f4a000a8f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49fffff700 1 -- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec004e50 msgr2=0x7f4a00160770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49fffff700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec004e50 0x7f4a00160770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49fffff700 1 -- 172.21.15.116:0/3585976052 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00150b60 con 0x7f4a000a9440 2020-11-20T12:20:58.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49fffff700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a000a9440 0x7f4a000c31a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f49f8001310 tx=0x7f49f8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49df7fe700 1 -- 172.21.15.116:0/3585976052 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f801f400 con 0x7f4a000a9440 2020-11-20T12:20:58.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49df7fe700 1 -- 172.21.15.116:0/3585976052 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f801f560 con 0x7f4a000a9440 2020-11-20T12:20:58.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f49df7fe700 1 -- 172.21.15.116:0/3585976052 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f801f920 con 0x7f4a000a9440 2020-11-20T12:20:58.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f4a04926700 1 -- 172.21.15.116:0/3585976052 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a00150e00 con 0x7f4a000a9440 2020-11-20T12:20:58.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.622+0000 7f4a04926700 1 -- 172.21.15.116:0/3585976052 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a00151540 con 0x7f4a000a9440 2020-11-20T12:20:58.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.626+0000 7f4a04926700 1 -- 172.21.15.116:0/3585976052 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0005cf30 con 0x7f4a000a9440 2020-11-20T12:20:58.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.626+0000 7f49df7fe700 1 -- 172.21.15.116:0/3585976052 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f49f801d070 con 0x7f4a000a9440 2020-11-20T12:20:58.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.626+0000 7f49df7fe700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49d80557c0 0x7f49d8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:58.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.626+0000 7f49df7fe700 1 -- 172.21.15.116:0/3585976052 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f49f806f8e0 con 0x7f4a000a9440 2020-11-20T12:20:58.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.626+0000 7f49ff7fe700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49d80557c0 0x7f49d8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:58.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.626+0000 7f49ff7fe700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49d80557c0 0x7f49d8057c80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f49f0006d00 tx=0x7f49f0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:58.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.630+0000 7f49df7fe700 1 -- 172.21.15.116:0/3585976052 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f49f8045780 con 0x7f4a000a9440 2020-11-20T12:20:58.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.794+0000 7f4a04926700 1 -- 172.21.15.116:0/3585976052 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4a00066220 con 0x7f4a000a9440 2020-11-20T12:20:58.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.794+0000 7f49df7fe700 1 -- 172.21.15.116:0/3585976052 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+10 (secure 0 0 0) 0x7f49f8024020 con 0x7f4a000a9440 2020-11-20T12:20:58.795 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:58.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.794+0000 7f4a04926700 1 -- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49d80557c0 msgr2=0x7f49d8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.794+0000 7f4a04926700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49d80557c0 0x7f49d8057c80 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f49f0006d00 tx=0x7f49f0008040).stop 2020-11-20T12:20:58.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.794+0000 7f4a04926700 1 -- 172.21.15.116:0/3585976052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a000a9440 msgr2=0x7f4a000c31a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:58.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.794+0000 7f4a04926700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a000a9440 0x7f4a000c31a0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f49f8001310 tx=0x7f49f8015000).stop 2020-11-20T12:20:58.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.794+0000 7f4a04926700 1 -- 172.21.15.116:0/3585976052 shutdown_connections 2020-11-20T12:20:58.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.794+0000 7f4a04926700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49d80557c0 0x7f49d8057c80 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.794+0000 7f4a04926700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4a000a9440 0x7f4a000c31a0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.794+0000 7f4a04926700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49ec004e50 0x7f4a00160770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.798+0000 7f4a04926700 1 --2- 172.21.15.116:0/3585976052 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4a00160c60 0x7f4a000a8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:58.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.798+0000 7f4a04926700 1 -- 172.21.15.116:0/3585976052 >> 172.21.15.116:0/3585976052 conn(0x7f4a00068350 msgr2=0x7f4a000a7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:58.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.798+0000 7f4a04926700 1 -- 172.21.15.116:0/3585976052 shutdown_connections 2020-11-20T12:20:58.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:58.798+0000 7f4a04926700 1 -- 172.21.15.116:0/3585976052 wait complete. 2020-11-20T12:20:58.823 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:20:59.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.014+0000 7f369ca5f700 1 Processor -- start 2020-11-20T12:20:59.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.014+0000 7f369ca5f700 1 -- start start 2020-11-20T12:20:59.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f369ca5f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f36980b9db0 0x7f36980c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f369ca5f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36980a8930 con 0x7f369814f880 2020-11-20T12:20:59.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f369ca5f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36980a8690 con 0x7f36980a81f0 2020-11-20T12:20:59.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f369ca5f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36980b94d0 con 0x7f36980b9db0 2020-11-20T12:20:59.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f3697fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f36980b9db0 0x7f36980c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f36977fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f369814f880 0x7f3698060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59708/0 (socket says 172.21.15.116:59708) 2020-11-20T12:20:59.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f36977fe700 1 -- 172.21.15.116:0/994062606 learned_addr learned my addr 172.21.15.116:0/994062606 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:20:59.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f36967fc700 1 -- 172.21.15.116:0/994062606 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2590914919 0 0) 0x7f36980a8930 con 0x7f369814f880 2020-11-20T12:20:59.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f36967fc700 1 -- 172.21.15.116:0/994062606 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3684003240 con 0x7f369814f880 2020-11-20T12:20:59.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f36967fc700 1 -- 172.21.15.116:0/994062606 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819313337 0 0) 0x7f36980a8690 con 0x7f36980a81f0 2020-11-20T12:20:59.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f3697fff700 1 -- 172.21.15.116:0/994062606 >> v1:172.21.15.173:6789/0 conn(0x7f36980a81f0 legacy=0x7f36980c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f3697fff700 1 -- 172.21.15.116:0/994062606 >> v1:172.21.15.116:6789/0 conn(0x7f369814f880 legacy=0x7f3698060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f3697fff700 1 -- 172.21.15.116:0/994062606 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36980b9770 con 0x7f36980b9db0 2020-11-20T12:20:59.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f3697fff700 1 --2- 172.21.15.116:0/994062606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36980b9db0 0x7f36980c15a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f3690009c90 tx=0x7f36900091d0).ready entity=mon.0 client_cookie=ac7c94799bffd5bd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:59.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.018+0000 7f36967fc700 1 -- 172.21.15.116:0/994062606 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369000ba70 con 0x7f36980b9db0 2020-11-20T12:20:59.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.022+0000 7f36967fc700 1 -- 172.21.15.116:0/994062606 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369000bbd0 con 0x7f36980b9db0 2020-11-20T12:20:59.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.022+0000 7f36967fc700 1 -- 172.21.15.116:0/994062606 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690011400 con 0x7f36980b9db0 2020-11-20T12:20:59.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.022+0000 7f369ca5f700 1 -- 172.21.15.116:0/994062606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36980b9db0 msgr2=0x7f36980c15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.022+0000 7f369ca5f700 1 --2- 172.21.15.116:0/994062606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36980b9db0 0x7f36980c15a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f3690009c90 tx=0x7f36900091d0).stop 2020-11-20T12:20:59.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.022+0000 7f369ca5f700 1 -- 172.21.15.116:0/994062606 shutdown_connections 2020-11-20T12:20:59.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.022+0000 7f369ca5f700 1 --2- 172.21.15.116:0/994062606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36980b9db0 0x7f36980c15a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.022+0000 7f369ca5f700 1 -- 172.21.15.116:0/994062606 >> 172.21.15.116:0/994062606 conn(0x7f3698068350 msgr2=0x7f36980627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:59.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.022+0000 7f369ca5f700 1 -- 172.21.15.116:0/994062606 shutdown_connections 2020-11-20T12:20:59.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.022+0000 7f369ca5f700 1 -- 172.21.15.116:0/994062606 wait complete. 2020-11-20T12:20:59.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f369ca5f700 1 Processor -- start 2020-11-20T12:20:59.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f369ca5f700 1 -- start start 2020-11-20T12:20:59.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f369ca5f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36980a81f0 0x7f36981616c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f369ca5f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36980b9db0 0x7f369814d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f369ca5f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f369814f880 0x7f3698156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f369ca5f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3698161bb0 con 0x7f369814f880 2020-11-20T12:20:59.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f369ca5f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36981555e0 con 0x7f36980a81f0 2020-11-20T12:20:59.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f369ca5f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3698155880 con 0x7f36980b9db0 2020-11-20T12:20:59.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f3697fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f369814f880 0x7f3698156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f3697fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f369814f880 0x7f3698156a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50284/0 (socket says 172.21.15.116:50284) 2020-11-20T12:20:59.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f3697fff700 1 -- 172.21.15.116:0/2384114579 learned_addr learned my addr 172.21.15.116:0/2384114579 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:59.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f36977fe700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36980a81f0 0x7f36981616c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f3696ffd700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36980b9db0 0x7f369814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f3697fff700 1 -- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36980b9db0 msgr2=0x7f369814d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f3697fff700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36980b9db0 0x7f369814d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f3697fff700 1 -- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36980a81f0 msgr2=0x7f36981616c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f3697fff700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36980a81f0 0x7f36981616c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f3697fff700 1 -- 172.21.15.116:0/2384114579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3698155b20 con 0x7f369814f880 2020-11-20T12:20:59.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f36977fe700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36980a81f0 0x7f36981616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:59.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f3697fff700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f369814f880 0x7f3698156a70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f369000b770 tx=0x7f3690016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:59.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f36777fe700 1 -- 172.21.15.116:0/2384114579 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690021070 con 0x7f369814f880 2020-11-20T12:20:59.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f36777fe700 1 -- 172.21.15.116:0/2384114579 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3690010780 con 0x7f369814f880 2020-11-20T12:20:59.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f36777fe700 1 -- 172.21.15.116:0/2384114579 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3690006ea0 con 0x7f369814f880 2020-11-20T12:20:59.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.026+0000 7f369ca5f700 1 -- 172.21.15.116:0/2384114579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3698155dc0 con 0x7f369814f880 2020-11-20T12:20:59.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.030+0000 7f369ca5f700 1 -- 172.21.15.116:0/2384114579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369814d800 con 0x7f369814f880 2020-11-20T12:20:59.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.030+0000 7f369ca5f700 1 -- 172.21.15.116:0/2384114579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f369805cf30 con 0x7f369814f880 2020-11-20T12:20:59.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.030+0000 7f36777fe700 1 -- 172.21.15.116:0/2384114579 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3690017070 con 0x7f369814f880 2020-11-20T12:20:59.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.030+0000 7f36777fe700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36700557c0 0x7f3670057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.030+0000 7f36777fe700 1 -- 172.21.15.116:0/2384114579 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f36900708c0 con 0x7f369814f880 2020-11-20T12:20:59.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.030+0000 7f36977fe700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36700557c0 0x7f3670057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.034+0000 7f36977fe700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36700557c0 0x7f3670057c80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f36880055a0 tx=0x7f3688009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:59.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.038+0000 7f36777fe700 1 -- 172.21.15.116:0/2384114579 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f36900466e0 con 0x7f369814f880 2020-11-20T12:20:59.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.198+0000 7f369ca5f700 1 -- 172.21.15.116:0/2384114579 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f36980639c0 con 0x7f369814f880 2020-11-20T12:20:59.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.198+0000 7f36777fe700 1 -- 172.21.15.116:0/2384114579 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+10 (secure 0 0 0) 0x7f3690015070 con 0x7f369814f880 2020-11-20T12:20:59.199 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:20:59.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.198+0000 7f369ca5f700 1 -- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36700557c0 msgr2=0x7f3670057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.198+0000 7f369ca5f700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36700557c0 0x7f3670057c80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f36880055a0 tx=0x7f3688009040).stop 2020-11-20T12:20:59.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.198+0000 7f369ca5f700 1 -- 172.21.15.116:0/2384114579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f369814f880 msgr2=0x7f3698156a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.198+0000 7f369ca5f700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f369814f880 0x7f3698156a70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f369000b770 tx=0x7f3690016000).stop 2020-11-20T12:20:59.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.202+0000 7f369ca5f700 1 -- 172.21.15.116:0/2384114579 shutdown_connections 2020-11-20T12:20:59.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.202+0000 7f369ca5f700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36700557c0 0x7f3670057c80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.202+0000 7f369ca5f700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f369814f880 0x7f3698156a70 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.202+0000 7f369ca5f700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36980a81f0 0x7f36981616c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.202+0000 7f369ca5f700 1 --2- 172.21.15.116:0/2384114579 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36980b9db0 0x7f369814d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.202+0000 7f369ca5f700 1 -- 172.21.15.116:0/2384114579 >> 172.21.15.116:0/2384114579 conn(0x7f3698068350 msgr2=0x7f3698066650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:59.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.202+0000 7f369ca5f700 1 -- 172.21.15.116:0/2384114579 shutdown_connections 2020-11-20T12:20:59.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.202+0000 7f369ca5f700 1 -- 172.21.15.116:0/2384114579 wait complete. 2020-11-20T12:20:59.224 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:20:59.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.414+0000 7ff44b863700 1 Processor -- start 2020-11-20T12:20:59.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.414+0000 7ff44b863700 1 -- start start 2020-11-20T12:20:59.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff44b863700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff444151e90 0x7ff4440c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff44b863700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff4440bf020 0x7ff4440a5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff44b863700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4440c3600 con 0x7ff4440c3c90 2020-11-20T12:20:59.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff44b863700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4440c38a0 con 0x7ff4440bf020 2020-11-20T12:20:59.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff44b863700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4440c72a0 con 0x7ff444151e90 2020-11-20T12:20:59.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff449e00700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff4440bf020 0x7ff4440a5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff449e00700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff4440bf020 0x7ff4440a5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50292/0 (socket says 172.21.15.116:50292) 2020-11-20T12:20:59.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff449e00700 1 -- 172.21.15.116:0/2111063752 learned_addr learned my addr 172.21.15.116:0/2111063752 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:59.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff448dfe700 1 --2- 172.21.15.116:0/2111063752 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff444151e90 0x7ff4440c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff449e00700 1 -- 172.21.15.116:0/2111063752 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff444151e90 msgr2=0x7ff4440c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:59.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff449e00700 1 --2- 172.21.15.116:0/2111063752 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff444151e90 0x7ff4440c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff449e00700 1 -- 172.21.15.116:0/2111063752 >> v1:172.21.15.173:6789/0 conn(0x7ff4440c3c90 legacy=0x7ff4440c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff449e00700 1 -- 172.21.15.116:0/2111063752 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4440c7540 con 0x7ff4440bf020 2020-11-20T12:20:59.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff449e00700 1 --2- 172.21.15.116:0/2111063752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4440bf020 0x7ff4440a5610 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7ff43c009d50 tx=0x7ff43c009290).ready entity=mon.0 client_cookie=6bce7eedd4a631a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:59.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff43bfff700 1 -- 172.21.15.116:0/2111063752 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c00bbc0 con 0x7ff4440bf020 2020-11-20T12:20:59.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff43bfff700 1 -- 172.21.15.116:0/2111063752 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c00bd20 con 0x7ff4440bf020 2020-11-20T12:20:59.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.418+0000 7ff43bfff700 1 -- 172.21.15.116:0/2111063752 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c011610 con 0x7ff4440bf020 2020-11-20T12:20:59.423 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.422+0000 7ff44b863700 1 -- 172.21.15.116:0/2111063752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4440bf020 msgr2=0x7ff4440a5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.422+0000 7ff44b863700 1 --2- 172.21.15.116:0/2111063752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4440bf020 0x7ff4440a5610 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7ff43c009d50 tx=0x7ff43c009290).stop 2020-11-20T12:20:59.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.422+0000 7ff44b863700 1 -- 172.21.15.116:0/2111063752 shutdown_connections 2020-11-20T12:20:59.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.422+0000 7ff44b863700 1 --2- 172.21.15.116:0/2111063752 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff4440bf020 0x7ff4440a5610 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.424 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.422+0000 7ff44b863700 1 --2- 172.21.15.116:0/2111063752 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff444151e90 0x7ff4440c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.422+0000 7ff44b863700 1 -- 172.21.15.116:0/2111063752 >> 172.21.15.116:0/2111063752 conn(0x7ff444068080 msgr2=0x7ff4440c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:59.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.422+0000 7ff44b863700 1 -- 172.21.15.116:0/2111063752 shutdown_connections 2020-11-20T12:20:59.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.422+0000 7ff44b863700 1 -- 172.21.15.116:0/2111063752 wait complete. 2020-11-20T12:20:59.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.422+0000 7ff44b863700 1 Processor -- start 2020-11-20T12:20:59.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff44b863700 1 -- start start 2020-11-20T12:20:59.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff44b863700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4440bf020 0x7ff4441651d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff44b863700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4440c3c90 0x7ff444161c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff44b863700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff444151e90 0x7ff4440c0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff44b863700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4440ace60 con 0x7ff444151e90 2020-11-20T12:20:59.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff44b863700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4440ad100 con 0x7ff4440c3c90 2020-11-20T12:20:59.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff44b863700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4441647f0 con 0x7ff4440bf020 2020-11-20T12:20:59.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff449e00700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff444151e90 0x7ff4440c0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff449e00700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff444151e90 0x7ff4440c0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50298/0 (socket says 172.21.15.116:50298) 2020-11-20T12:20:59.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff449e00700 1 -- 172.21.15.116:0/413513396 learned_addr learned my addr 172.21.15.116:0/413513396 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:59.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff4495ff700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4440bf020 0x7ff4441651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff448dfe700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4440c3c90 0x7ff444161c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff449e00700 1 -- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4440bf020 msgr2=0x7ff4441651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff449e00700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4440bf020 0x7ff4441651d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff449e00700 1 -- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4440c3c90 msgr2=0x7ff444161c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff449e00700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4440c3c90 0x7ff444161c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.426+0000 7ff449e00700 1 -- 172.21.15.116:0/413513396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff444164a90 con 0x7ff444151e90 2020-11-20T12:20:59.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.430+0000 7ff449e00700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff444151e90 0x7ff4440c0c20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7ff43c0092d0 tx=0x7ff43c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:59.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.430+0000 7ff4397fa700 1 -- 172.21.15.116:0/413513396 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c00bbc0 con 0x7ff444151e90 2020-11-20T12:20:59.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.430+0000 7ff4397fa700 1 -- 172.21.15.116:0/413513396 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43c00be90 con 0x7ff444151e90 2020-11-20T12:20:59.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.430+0000 7ff44b863700 1 -- 172.21.15.116:0/413513396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff444164d30 con 0x7ff444151e90 2020-11-20T12:20:59.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.430+0000 7ff4397fa700 1 -- 172.21.15.116:0/413513396 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43c010cf0 con 0x7ff444151e90 2020-11-20T12:20:59.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.430+0000 7ff44b863700 1 -- 172.21.15.116:0/413513396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff444162180 con 0x7ff444151e90 2020-11-20T12:20:59.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.430+0000 7ff44b863700 1 -- 172.21.15.116:0/413513396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff444164ec0 con 0x7ff444151e90 2020-11-20T12:20:59.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.430+0000 7ff4397fa700 1 -- 172.21.15.116:0/413513396 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff43c017070 con 0x7ff444151e90 2020-11-20T12:20:59.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.434+0000 7ff4397fa700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff41c0557c0 0x7ff41c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.434+0000 7ff4397fa700 1 -- 172.21.15.116:0/413513396 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff43c0707b0 con 0x7ff444151e90 2020-11-20T12:20:59.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.434+0000 7ff4495ff700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff41c0557c0 0x7ff41c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.434+0000 7ff4495ff700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff41c0557c0 0x7ff41c057c80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7ff440003760 tx=0x7ff440007b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:59.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.438+0000 7ff4397fa700 1 -- 172.21.15.116:0/413513396 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff43c0466d0 con 0x7ff444151e90 2020-11-20T12:20:59.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.642+0000 7ff44b863700 1 -- 172.21.15.116:0/413513396 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff44405d2a0 con 0x7ff444151e90 2020-11-20T12:20:59.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.646+0000 7ff4397fa700 1 -- 172.21.15.116:0/413513396 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+2848 (secure 0 0 0) 0x7ff43c015020 con 0x7ff444151e90 2020-11-20T12:20:59.646 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:20:59.646 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6323,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3105911814","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3105911814},{"type":"v1","addr":"172.21.15.173:6837","nonce":3105911814}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7231,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/3457547857","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":3457547857},{"type":"v1","addr":"172.21.15.116:6835","nonce":3457547857}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":50},{"gid":7302,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.116:6833/81195738","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":81195738},{"type":"v1","addr":"172.21.15.116:6833","nonce":81195738}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:20:34.449524+0000","modified":"2020-11-20T12:20:36.475264+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7311},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7311":{"gid":7311,"name":"b","rank":0,"incarnation":51,"state":"up:active","state_seq":3,"addr":"172.21.15.173:6835/2361610151","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":2361610151},{"type":"v1","addr":"172.21.15.173:6835","nonce":2361610151}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2020-11-20T12:20:59.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 -- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff41c0557c0 msgr2=0x7ff41c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff41c0557c0 0x7ff41c057c80 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7ff440003760 tx=0x7ff440007b80).stop 2020-11-20T12:20:59.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 -- 172.21.15.116:0/413513396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff444151e90 msgr2=0x7ff4440c0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff444151e90 0x7ff4440c0c20 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7ff43c0092d0 tx=0x7ff43c016000).stop 2020-11-20T12:20:59.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 -- 172.21.15.116:0/413513396 shutdown_connections 2020-11-20T12:20:59.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff41c0557c0 0x7ff41c057c80 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff444151e90 0x7ff4440c0c20 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff4440bf020 0x7ff4441651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 --2- 172.21.15.116:0/413513396 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff4440c3c90 0x7ff444161c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 -- 172.21.15.116:0/413513396 >> 172.21.15.116:0/413513396 conn(0x7ff444068080 msgr2=0x7ff4440c7900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:59.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 -- 172.21.15.116:0/413513396 shutdown_connections 2020-11-20T12:20:59.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.650+0000 7ff44b863700 1 -- 172.21.15.116:0/413513396 wait complete. 2020-11-20T12:20:59.654 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 54 2020-11-20T12:20:59.678 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:20:59.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.858+0000 7f0c65803700 1 Processor -- start 2020-11-20T12:20:59.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.858+0000 7f0c65803700 1 -- start start 2020-11-20T12:20:59.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c65803700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c6014bfa0 0x7f0c600a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c65803700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0c6014eb30 0x7f0c600a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c65803700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0c60062220 0x7f0c60152100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c65803700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6014f0b0 con 0x7f0c6014eb30 2020-11-20T12:20:59.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c65803700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6014f350 con 0x7f0c60062220 2020-11-20T12:20:59.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c65803700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6014f5f0 con 0x7f0c6014bfa0 2020-11-20T12:20:59.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c5e7fc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0c6014eb30 0x7f0c600a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c5ffff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0c60062220 0x7f0c60152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c5e7fc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0c6014eb30 0x7f0c600a8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50306/0 (socket says 172.21.15.116:50306) 2020-11-20T12:20:59.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c5ffff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f0c60062220 0x7f0c60152100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:49356/0 (socket says 172.21.15.116:49356) 2020-11-20T12:20:59.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c5effd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c6014bfa0 0x7f0c600a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c5e7fc700 1 -- 172.21.15.116:0/961814639 learned_addr learned my addr 172.21.15.116:0/961814639 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:59.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.862+0000 7f0c5e7fc700 1 -- 172.21.15.116:0/961814639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c6014bfa0 msgr2=0x7f0c600a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:59.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c5e7fc700 1 --2- 172.21.15.116:0/961814639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c6014bfa0 0x7f0c600a89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c5e7fc700 1 -- 172.21.15.116:0/961814639 >> v2:172.21.15.173:3300/0 conn(0x7f0c60062220 msgr2=0x7f0c60152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:20:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c5e7fc700 1 --2- 172.21.15.116:0/961814639 >> v2:172.21.15.173:3300/0 conn(0x7f0c60062220 0x7f0c60152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c5e7fc700 1 -- 172.21.15.116:0/961814639 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6014f890 con 0x7f0c6014eb30 2020-11-20T12:20:59.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c5ffff700 1 --2- 172.21.15.116:0/961814639 >> v2:172.21.15.173:3300/0 conn(0x7f0c60062220 0x7f0c60152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:20:59.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c5e7fc700 1 --2- 172.21.15.116:0/961814639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6014eb30 0x7f0c600a8e90 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f0c600628d0 tx=0x7f0c54006eb0).ready entity=mon.0 client_cookie=e68d50c3753121a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c5dffb700 1 -- 172.21.15.116:0/961814639 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54009720 con 0x7f0c6014eb30 2020-11-20T12:20:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c5dffb700 1 -- 172.21.15.116:0/961814639 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54009880 con 0x7f0c6014eb30 2020-11-20T12:20:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c5dffb700 1 -- 172.21.15.116:0/961814639 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54009c00 con 0x7f0c6014eb30 2020-11-20T12:20:59.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c65803700 1 -- 172.21.15.116:0/961814639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6014eb30 msgr2=0x7f0c600a8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c65803700 1 --2- 172.21.15.116:0/961814639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6014eb30 0x7f0c600a8e90 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7f0c600628d0 tx=0x7f0c54006eb0).stop 2020-11-20T12:20:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c65803700 1 -- 172.21.15.116:0/961814639 shutdown_connections 2020-11-20T12:20:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c65803700 1 --2- 172.21.15.116:0/961814639 >> v2:172.21.15.173:3300/0 conn(0x7f0c60062220 0x7f0c60152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c65803700 1 --2- 172.21.15.116:0/961814639 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c6014bfa0 0x7f0c600a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c65803700 1 --2- 172.21.15.116:0/961814639 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6014eb30 0x7f0c600a8e90 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c65803700 1 -- 172.21.15.116:0/961814639 >> 172.21.15.116:0/961814639 conn(0x7f0c600c63f0 msgr2=0x7f0c600c23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:20:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c65803700 1 -- 172.21.15.116:0/961814639 shutdown_connections 2020-11-20T12:20:59.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.866+0000 7f0c65803700 1 -- 172.21.15.116:0/961814639 wait complete. 2020-11-20T12:20:59.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.870+0000 7f0c65803700 1 Processor -- start 2020-11-20T12:20:59.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.870+0000 7f0c65803700 1 -- start start 2020-11-20T12:20:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c65803700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c60062220 0x7f0c60143e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c65803700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c6014bfa0 0x7f0c600aab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c65803700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6014ad90 0x7f0c6012cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c65803700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600a9c00 con 0x7f0c6014ad90 2020-11-20T12:20:59.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c65803700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600a9ea0 con 0x7f0c60062220 2020-11-20T12:20:59.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c65803700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600aa140 con 0x7f0c6014bfa0 2020-11-20T12:20:59.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6014ad90 0x7f0c6012cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6014ad90 0x7f0c6012cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50312/0 (socket says 172.21.15.116:50312) 2020-11-20T12:20:59.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5ffff700 1 -- 172.21.15.116:0/3039694303 learned_addr learned my addr 172.21.15.116:0/3039694303 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:20:59.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5e7fc700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c6014bfa0 0x7f0c600aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5effd700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c60062220 0x7f0c60143e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5ffff700 1 -- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c6014bfa0 msgr2=0x7f0c600aab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5ffff700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c6014bfa0 0x7f0c600aab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5ffff700 1 -- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c60062220 msgr2=0x7f0c60143e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:20:59.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5ffff700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c60062220 0x7f0c60143e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:20:59.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5ffff700 1 -- 172.21.15.116:0/3039694303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c600aa3e0 con 0x7f0c6014ad90 2020-11-20T12:20:59.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5effd700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c60062220 0x7f0c60143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:20:59.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c5ffff700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6014ad90 0x7f0c6012cf10 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7f0c5800ef00 tx=0x7f0c580091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:59.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c46ffd700 1 -- 172.21.15.116:0/3039694303 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5800cdd0 con 0x7f0c6014ad90 2020-11-20T12:20:59.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c46ffd700 1 -- 172.21.15.116:0/3039694303 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c580123f0 con 0x7f0c6014ad90 2020-11-20T12:20:59.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c46ffd700 1 -- 172.21.15.116:0/3039694303 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58012770 con 0x7f0c6014ad90 2020-11-20T12:20:59.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c65803700 1 -- 172.21.15.116:0/3039694303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c600aa680 con 0x7f0c6014ad90 2020-11-20T12:20:59.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.874+0000 7f0c65803700 1 -- 172.21.15.116:0/3039694303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c600a97d0 con 0x7f0c6014ad90 2020-11-20T12:20:59.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.878+0000 7f0c65803700 1 -- 172.21.15.116:0/3039694303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6005d2a0 con 0x7f0c6014ad90 2020-11-20T12:20:59.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.878+0000 7f0c46ffd700 1 -- 172.21.15.116:0/3039694303 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0c58022070 con 0x7f0c6014ad90 2020-11-20T12:20:59.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.878+0000 7f0c46ffd700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c3c0557c0 0x7f0c3c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:20:59.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.878+0000 7f0c46ffd700 1 -- 172.21.15.116:0/3039694303 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0c5806f090 con 0x7f0c6014ad90 2020-11-20T12:20:59.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.878+0000 7f0c5effd700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c3c0557c0 0x7f0c3c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:20:59.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.882+0000 7f0c5effd700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c3c0557c0 0x7f0c3c057c80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f0c50006fd0 tx=0x7f0c50008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:20:59.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:20:59.886+0000 7f0c46ffd700 1 -- 172.21.15.116:0/3039694303 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0c58048e60 con 0x7f0c6014ad90 2020-11-20T12:20:59.983 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:20:59.992 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:21:00.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.046+0000 7f0c65803700 1 -- 172.21.15.116:0/3039694303 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c600aa810 con 0x7f0c6014ad90 2020-11-20T12:21:00.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c46ffd700 1 -- 172.21.15.116:0/3039694303 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16877 (secure 0 0 0) 0x7f0c58044f30 con 0x7f0c6014ad90 2020-11-20T12:21:00.051 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:00.051 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":96,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:20:35.459061+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:20:32.886768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2020-11-20T12:20:33.446960+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:00.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 -- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c3c0557c0 msgr2=0x7f0c3c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:00.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c3c0557c0 0x7f0c3c057c80 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f0c50006fd0 tx=0x7f0c50008040).stop 2020-11-20T12:21:00.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 -- 172.21.15.116:0/3039694303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6014ad90 msgr2=0x7f0c6012cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:00.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6014ad90 0x7f0c6012cf10 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7f0c5800ef00 tx=0x7f0c580091d0).stop 2020-11-20T12:21:00.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 -- 172.21.15.116:0/3039694303 shutdown_connections 2020-11-20T12:21:00.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0c3c0557c0 0x7f0c3c057c80 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:00.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0c6014ad90 0x7f0c6012cf10 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:00.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0c60062220 0x7f0c60143e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:00.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 --2- 172.21.15.116:0/3039694303 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0c6014bfa0 0x7f0c600aab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:00.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 -- 172.21.15.116:0/3039694303 >> 172.21.15.116:0/3039694303 conn(0x7f0c600c63f0 msgr2=0x7f0c600a64d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:00.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 -- 172.21.15.116:0/3039694303 shutdown_connections 2020-11-20T12:21:00.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.050+0000 7f0c65803700 1 -- 172.21.15.116:0/3039694303 wait complete. 2020-11-20T12:21:00.074 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-20T12:21:00.074 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-20T12:21:00.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f5238f8e700 1 Processor -- start 2020-11-20T12:21:00.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f5238f8e700 1 -- start start 2020-11-20T12:21:00.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f5238f8e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f523414e530 0x7f523414e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:00.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f5238f8e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f52340a5990 0x7f52340a6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:00.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f5238f8e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52340a5df0 0x7f52340a6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f5238f8e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52340a4ff0 con 0x7f523414e530 2020-11-20T12:21:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f5238f8e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5234152ff0 con 0x7f52340a5990 2020-11-20T12:21:00.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f5238f8e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52340a5250 con 0x7f52340a5df0 2020-11-20T12:21:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f523259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f523414e530 0x7f523414e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f523259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f523414e530 0x7f523414e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50316/0 (socket says 172.21.15.116:50316) 2020-11-20T12:21:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f523259c700 1 -- 172.21.15.116:0/712398405 learned_addr learned my addr 172.21.15.116:0/712398405 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:00.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f5231d9b700 1 --2- 172.21.15.116:0/712398405 >> v2:172.21.15.173:3300/0 conn(0x7f52340a5990 0x7f52340a6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f5232d9d700 1 --2- 172.21.15.116:0/712398405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52340a5df0 0x7f52340a6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f523259c700 1 -- 172.21.15.116:0/712398405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52340a5df0 msgr2=0x7f52340a6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f523259c700 1 --2- 172.21.15.116:0/712398405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52340a5df0 0x7f52340a6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f523259c700 1 -- 172.21.15.116:0/712398405 >> v2:172.21.15.173:3300/0 conn(0x7f52340a5990 msgr2=0x7f52340a6610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:00.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f523259c700 1 --2- 172.21.15.116:0/712398405 >> v2:172.21.15.173:3300/0 conn(0x7f52340a5990 0x7f52340a6610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f523259c700 1 -- 172.21.15.116:0/712398405 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52340a54f0 con 0x7f523414e530 2020-11-20T12:21:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f523259c700 1 --2- 172.21.15.116:0/712398405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f523414e530 0x7f523414e910 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7f5234066370 tx=0x7f5224006eb0).ready entity=mon.0 client_cookie=6ba1ac1e57d112f7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.286+0000 7f523159a700 1 -- 172.21.15.116:0/712398405 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240098d0 con 0x7f523414e530 2020-11-20T12:21:00.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f523159a700 1 -- 172.21.15.116:0/712398405 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5224009a30 con 0x7f523414e530 2020-11-20T12:21:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f523159a700 1 -- 172.21.15.116:0/712398405 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224009db0 con 0x7f523414e530 2020-11-20T12:21:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f5238f8e700 1 -- 172.21.15.116:0/712398405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f523414e530 msgr2=0x7f523414e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:00.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f5238f8e700 1 --2- 172.21.15.116:0/712398405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f523414e530 0x7f523414e910 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7f5234066370 tx=0x7f5224006eb0).stop 2020-11-20T12:21:00.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f5238f8e700 1 -- 172.21.15.116:0/712398405 shutdown_connections 2020-11-20T12:21:00.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f5238f8e700 1 --2- 172.21.15.116:0/712398405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f52340a5df0 0x7f52340a6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f5238f8e700 1 --2- 172.21.15.116:0/712398405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f523414e530 0x7f523414e910 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f5238f8e700 1 --2- 172.21.15.116:0/712398405 >> v2:172.21.15.173:3300/0 conn(0x7f52340a5990 0x7f52340a6610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f5238f8e700 1 -- 172.21.15.116:0/712398405 >> 172.21.15.116:0/712398405 conn(0x7f5234060170 msgr2=0x7f5234060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:00.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f5238f8e700 1 -- 172.21.15.116:0/712398405 shutdown_connections 2020-11-20T12:21:00.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f5238f8e700 1 -- 172.21.15.116:0/712398405 wait complete. 2020-11-20T12:21:00.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f5238f8e700 1 Processor -- start 2020-11-20T12:21:00.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.290+0000 7f5238f8e700 1 -- start start 2020-11-20T12:21:00.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f5238f8e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52340a5990 0x7f5234151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:00.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f5238f8e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52340a5df0 0x7f52340be900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:00.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f5238f8e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f523414e530 0x7f523414d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:00.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f5238f8e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5234152bf0 con 0x7f52340a5990 2020-11-20T12:21:00.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f5238f8e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5234092010 con 0x7f52340a5df0 2020-11-20T12:21:00.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f5238f8e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5234150f60 con 0x7f523414e530 2020-11-20T12:21:00.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f523259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52340a5990 0x7f5234151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:00.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f523259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52340a5990 0x7f5234151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50322/0 (socket says 172.21.15.116:50322) 2020-11-20T12:21:00.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f523259c700 1 -- 172.21.15.116:0/2211036773 learned_addr learned my addr 172.21.15.116:0/2211036773 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:00.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f5232d9d700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f523414e530 0x7f523414d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:00.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f5231d9b700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52340a5df0 0x7f52340be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:00.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f523259c700 1 -- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f523414e530 msgr2=0x7f523414d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:00.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f523259c700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f523414e530 0x7f523414d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:00.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f523259c700 1 -- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52340a5df0 msgr2=0x7f52340be900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:00.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f523259c700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52340a5df0 0x7f52340be900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:00.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f523259c700 1 -- 172.21.15.116:0/2211036773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5234151200 con 0x7f52340a5990 2020-11-20T12:21:00.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f523259c700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52340a5990 0x7f5234151940 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f523405c930 tx=0x7f5224017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:00.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f52227fc700 1 -- 172.21.15.116:0/2211036773 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224010e40 con 0x7f52340a5990 2020-11-20T12:21:00.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f52227fc700 1 -- 172.21.15.116:0/2211036773 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5224018400 con 0x7f52340a5990 2020-11-20T12:21:00.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f52227fc700 1 -- 172.21.15.116:0/2211036773 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52240187a0 con 0x7f52340a5990 2020-11-20T12:21:00.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f5238f8e700 1 -- 172.21.15.116:0/2211036773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52341514a0 con 0x7f52340a5990 2020-11-20T12:21:00.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.294+0000 7f5238f8e700 1 -- 172.21.15.116:0/2211036773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52340bedf0 con 0x7f52340a5990 2020-11-20T12:21:00.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.298+0000 7f5238f8e700 1 -- 172.21.15.116:0/2211036773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52340bf070 con 0x7f52340a5990 2020-11-20T12:21:00.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.298+0000 7f52227fc700 1 -- 172.21.15.116:0/2211036773 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f522402b070 con 0x7f52340a5990 2020-11-20T12:21:00.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.298+0000 7f52227fc700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 0x7f5208057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:00.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.298+0000 7f52227fc700 1 -- 172.21.15.116:0/2211036773 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f522406fcb0 con 0x7f52340a5990 2020-11-20T12:21:00.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.298+0000 7f5231d9b700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 0x7f5208057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:00.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.302+0000 7f5231d9b700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 0x7f5208057c80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f523414ff60 tx=0x7f5228008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:00.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.306+0000 7f52227fc700 1 -- 172.21.15.116:0/2211036773 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5224045b50 con 0x7f52340a5990 2020-11-20T12:21:00.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:00.518+0000 7f5238f8e700 1 -- 172.21.15.116:0/2211036773 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f523405d2a0 con 0x7f52340a5990 2020-11-20T12:21:01.328 INFO:tasks.ceph.mon.a.smithi116.stderr:2020-11-20T12:21:01.326+0000 7fc111d1b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-20T12:21:01.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f52227fc700 1 -- 172.21.15.116:0/2211036773 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5224029070 con 0x7f52340a5990 2020-11-20T12:21:01.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f5238f8e700 1 -- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 msgr2=0x7f5208057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:01.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f5238f8e700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 0x7f5208057c80 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f523414ff60 tx=0x7f5228008040).stop 2020-11-20T12:21:01.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f5238f8e700 1 -- 172.21.15.116:0/2211036773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52340a5990 msgr2=0x7f5234151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:01.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f5238f8e700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52340a5990 0x7f5234151940 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f523405c930 tx=0x7f5224017000).stop 2020-11-20T12:21:01.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f5238f8e700 1 -- 172.21.15.116:0/2211036773 shutdown_connections 2020-11-20T12:21:01.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f5238f8e700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f52080557c0 0x7f5208057c80 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:01.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f5238f8e700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f523414e530 0x7f523414d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:01.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f5238f8e700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f52340a5990 0x7f5234151940 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:01.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f5238f8e700 1 --2- 172.21.15.116:0/2211036773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f52340a5df0 0x7f52340be900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:01.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f5238f8e700 1 -- 172.21.15.116:0/2211036773 >> 172.21.15.116:0/2211036773 conn(0x7f5234060170 msgr2=0x7f52340c3680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:01.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.342+0000 7f5238f8e700 1 -- 172.21.15.116:0/2211036773 shutdown_connections 2020-11-20T12:21:01.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.346+0000 7f5238f8e700 1 -- 172.21.15.116:0/2211036773 wait complete. 2020-11-20T12:21:01.348 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-20T12:21:01.348 INFO:tasks.ceph.mds.b.smithi173.stderr: -1250> 2020-11-20T12:20:38.818+0000 7f14ba2ca700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:21:01.372 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-20T12:21:01.386 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2020-11-20T12:21:01.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.562+0000 7f6112bf5700 1 Processor -- start 2020-11-20T12:21:01.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.562+0000 7f6112bf5700 1 -- start start 2020-11-20T12:21:01.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f6112bf5700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f610c14a1d0 0x7f610c0be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:01.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f6112bf5700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f610c0a9c20 con 0x7f610c149df0 2020-11-20T12:21:01.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f6112bf5700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f610c0a9ec0 con 0x7f610c0a9750 2020-11-20T12:21:01.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f6112bf5700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f610c153670 con 0x7f610c14a1d0 2020-11-20T12:21:01.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f6110991700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f610c149df0 0x7f610c153db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59764/0 (socket says 172.21.15.116:59764) 2020-11-20T12:21:01.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f6110991700 1 -- 172.21.15.116:0/1060902692 learned_addr learned my addr 172.21.15.116:0/1060902692 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:01.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f6111192700 1 --2- 172.21.15.116:0/1060902692 >> v2:172.21.15.173:3300/0 conn(0x7f610c14a1d0 0x7f610c0be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:01.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f610b7fe700 1 -- 172.21.15.116:0/1060902692 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205368090 0 0) 0x7f610c0a9c20 con 0x7f610c149df0 2020-11-20T12:21:01.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f610b7fe700 1 -- 172.21.15.116:0/1060902692 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60f0003240 con 0x7f610c149df0 2020-11-20T12:21:01.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f610b7fe700 1 -- 172.21.15.116:0/1060902692 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3259585593 0 0) 0x7f610c0a9ec0 con 0x7f610c0a9750 2020-11-20T12:21:01.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f6111192700 1 -- 172.21.15.116:0/1060902692 >> v1:172.21.15.173:6789/0 conn(0x7f610c0a9750 legacy=0x7f610c0c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:01.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f6111192700 1 -- 172.21.15.116:0/1060902692 >> v1:172.21.15.116:6789/0 conn(0x7f610c149df0 legacy=0x7f610c153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:01.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f6111192700 1 -- 172.21.15.116:0/1060902692 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610c153910 con 0x7f610c14a1d0 2020-11-20T12:21:01.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f6111192700 1 --2- 172.21.15.116:0/1060902692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f610c14a1d0 0x7f610c0be240 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f6104009c90 tx=0x7f61040091d0).ready entity=mon.1 client_cookie=216a9efa382c87d7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:01.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f610b7fe700 1 -- 172.21.15.116:0/1060902692 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610400ba70 con 0x7f610c14a1d0 2020-11-20T12:21:01.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f610b7fe700 1 -- 172.21.15.116:0/1060902692 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610400bbd0 con 0x7f610c14a1d0 2020-11-20T12:21:01.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.566+0000 7f610b7fe700 1 -- 172.21.15.116:0/1060902692 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6104011400 con 0x7f610c14a1d0 2020-11-20T12:21:01.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.570+0000 7f6112bf5700 1 -- 172.21.15.116:0/1060902692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f610c14a1d0 msgr2=0x7f610c0be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:01.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.570+0000 7f6112bf5700 1 --2- 172.21.15.116:0/1060902692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f610c14a1d0 0x7f610c0be240 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f6104009c90 tx=0x7f61040091d0).stop 2020-11-20T12:21:01.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.570+0000 7f6112bf5700 1 -- 172.21.15.116:0/1060902692 shutdown_connections 2020-11-20T12:21:01.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.570+0000 7f6112bf5700 1 --2- 172.21.15.116:0/1060902692 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f610c14a1d0 0x7f610c0be240 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:01.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.570+0000 7f6112bf5700 1 -- 172.21.15.116:0/1060902692 >> 172.21.15.116:0/1060902692 conn(0x7f610c05ba80 msgr2=0x7f610c0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:01.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.570+0000 7f6112bf5700 1 -- 172.21.15.116:0/1060902692 shutdown_connections 2020-11-20T12:21:01.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.570+0000 7f6112bf5700 1 -- 172.21.15.116:0/1060902692 wait complete. 2020-11-20T12:21:01.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.570+0000 7f6112bf5700 1 Processor -- start 2020-11-20T12:21:01.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.570+0000 7f6112bf5700 1 -- start start 2020-11-20T12:21:01.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6112bf5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f610c0a9750 0x7f610c164f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:01.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6112bf5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f610c149df0 0x7f610c1481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:01.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6112bf5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f610c14a1d0 0x7f610c0a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:01.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6112bf5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610c165440 con 0x7f610c14a1d0 2020-11-20T12:21:01.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6112bf5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f610c1486d0 con 0x7f610c0a9750 2020-11-20T12:21:01.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6112bf5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f610c1438a0 con 0x7f610c149df0 2020-11-20T12:21:01.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6111192700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f610c14a1d0 0x7f610c0a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:01.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6111192700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f610c14a1d0 0x7f610c0a7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50340/0 (socket says 172.21.15.116:50340) 2020-11-20T12:21:01.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6111192700 1 -- 172.21.15.116:0/1777608781 learned_addr learned my addr 172.21.15.116:0/1777608781 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:01.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f610bfff700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f610c149df0 0x7f610c1481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:01.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6110991700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f610c0a9750 0x7f610c164f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:01.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6111192700 1 -- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f610c149df0 msgr2=0x7f610c1481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:01.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6111192700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f610c149df0 0x7f610c1481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:01.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6111192700 1 -- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f610c0a9750 msgr2=0x7f610c164f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:01.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6111192700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f610c0a9750 0x7f610c164f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:01.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6111192700 1 -- 172.21.15.116:0/1777608781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f610c0c22b0 con 0x7f610c14a1d0 2020-11-20T12:21:01.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f610bfff700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f610c149df0 0x7f610c1481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:01.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.574+0000 7f6111192700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f610c14a1d0 0x7f610c0a7470 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7f610400c010 tx=0x7f6104016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:01.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.578+0000 7f6108ff9700 1 -- 172.21.15.116:0/1777608781 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610401c5f0 con 0x7f610c14a1d0 2020-11-20T12:21:01.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.578+0000 7f6108ff9700 1 -- 172.21.15.116:0/1777608781 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f610401c750 con 0x7f610c14a1d0 2020-11-20T12:21:01.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.578+0000 7f6108ff9700 1 -- 172.21.15.116:0/1777608781 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f610401caf0 con 0x7f610c14a1d0 2020-11-20T12:21:01.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.578+0000 7f6112bf5700 1 -- 172.21.15.116:0/1777608781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f610c0a7b80 con 0x7f610c14a1d0 2020-11-20T12:21:01.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.578+0000 7f6112bf5700 1 -- 172.21.15.116:0/1777608781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f610c0a80c0 con 0x7f610c14a1d0 2020-11-20T12:21:01.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.578+0000 7f6112bf5700 1 -- 172.21.15.116:0/1777608781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f610c0a83c0 con 0x7f610c14a1d0 2020-11-20T12:21:01.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.578+0000 7f6108ff9700 1 -- 172.21.15.116:0/1777608781 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6104024030 con 0x7f610c14a1d0 2020-11-20T12:21:01.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.578+0000 7f6108ff9700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60e40557c0 0x7f60e4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:01.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.578+0000 7f6108ff9700 1 -- 172.21.15.116:0/1777608781 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f61040700e0 con 0x7f610c14a1d0 2020-11-20T12:21:01.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.582+0000 7f6110991700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60e40557c0 0x7f60e4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:01.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.582+0000 7f6110991700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60e40557c0 0x7f60e4057c80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f61000055a0 tx=0x7f6100009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:01.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.586+0000 7f6108ff9700 1 -- 172.21.15.116:0/1777608781 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f610c0a83c0 con 0x7f610c14a1d0 2020-11-20T12:21:01.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:01.790+0000 7f6112bf5700 1 -- 172.21.15.116:0/1777608781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f610c0a83c0 con 0x7f610c14a1d0 2020-11-20T12:21:02.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6108ff9700 1 -- 172.21.15.116:0/1777608781 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7f610c0a83c0 con 0x7f610c14a1d0 2020-11-20T12:21:03.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 -- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60e40557c0 msgr2=0x7f60e4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:03.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60e40557c0 0x7f60e4057c80 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f61000055a0 tx=0x7f6100009040).stop 2020-11-20T12:21:03.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 -- 172.21.15.116:0/1777608781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f610c14a1d0 msgr2=0x7f610c0a7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:03.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f610c14a1d0 0x7f610c0a7470 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7f610400c010 tx=0x7f6104016000).stop 2020-11-20T12:21:03.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 -- 172.21.15.116:0/1777608781 shutdown_connections 2020-11-20T12:21:03.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60e40557c0 0x7f60e4057c80 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:03.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f610c14a1d0 0x7f610c0a7470 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:03.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f610c0a9750 0x7f610c164f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:03.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 --2- 172.21.15.116:0/1777608781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f610c149df0 0x7f610c1481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:03.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 -- 172.21.15.116:0/1777608781 >> 172.21.15.116:0/1777608781 conn(0x7f610c05ba80 msgr2=0x7f610c0c0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:03.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 -- 172.21.15.116:0/1777608781 shutdown_connections 2020-11-20T12:21:03.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:02.350+0000 7f6112bf5700 1 -- 172.21.15.116:0/1777608781 wait complete. 2020-11-20T12:21:03.269 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-20T12:21:03.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.469+0000 7ff81d859700 1 Processor -- start 2020-11-20T12:21:03.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.469+0000 7ff81d859700 1 -- start start 2020-11-20T12:21:03.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff81d859700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8180aa5f0 0x7ff81813b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:03.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff81d859700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff8180615f0 0x7ff818061a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:03.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff81d859700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8180ae1c0 con 0x7ff81813af90 2020-11-20T12:21:03.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff81d859700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff818061f50 con 0x7ff8180615f0 2020-11-20T12:21:03.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff81d859700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8180aaa90 con 0x7ff8180aa5f0 2020-11-20T12:21:03.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff816ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff81813af90 0x7ff8180adab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59778/0 (socket says 172.21.15.116:59778) 2020-11-20T12:21:03.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff816ffd700 1 -- 172.21.15.116:0/1878862234 learned_addr learned my addr 172.21.15.116:0/1878862234 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:03.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff8177fe700 1 --2- 172.21.15.116:0/1878862234 >> v2:172.21.15.116:3300/0 conn(0x7ff8180615f0 0x7ff818061a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:03.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff8167fc700 1 --2- 172.21.15.116:0/1878862234 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8180aa5f0 0x7ff81813b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:03.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff815ffb700 1 -- 172.21.15.116:0/1878862234 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879306275 0 0) 0x7ff8080010d0 con 0x7ff81813af90 2020-11-20T12:21:03.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff815ffb700 1 -- 172.21.15.116:0/1878862234 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff804003240 con 0x7ff81813af90 2020-11-20T12:21:03.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff8177fe700 1 -- 172.21.15.116:0/1878862234 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8180aa5f0 msgr2=0x7ff81813b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:03.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff8177fe700 1 --2- 172.21.15.116:0/1878862234 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8180aa5f0 0x7ff81813b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:03.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff8177fe700 1 -- 172.21.15.116:0/1878862234 >> v1:172.21.15.116:6789/0 conn(0x7ff81813af90 legacy=0x7ff8180adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:03.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff8177fe700 1 -- 172.21.15.116:0/1878862234 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8180aad30 con 0x7ff8180615f0 2020-11-20T12:21:03.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff8177fe700 1 --2- 172.21.15.116:0/1878862234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8180615f0 0x7ff818061a60 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7ff810009c90 tx=0x7ff8100091d0).ready entity=mon.0 client_cookie=b939a1f8088d84df server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:03.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff815ffb700 1 -- 172.21.15.116:0/1878862234 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81000b9c0 con 0x7ff8180615f0 2020-11-20T12:21:03.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff815ffb700 1 -- 172.21.15.116:0/1878862234 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81000bb20 con 0x7ff8180615f0 2020-11-20T12:21:03.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff815ffb700 1 -- 172.21.15.116:0/1878862234 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81000bea0 con 0x7ff8180615f0 2020-11-20T12:21:03.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff81d859700 1 -- 172.21.15.116:0/1878862234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8180615f0 msgr2=0x7ff818061a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:03.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.473+0000 7ff81d859700 1 --2- 172.21.15.116:0/1878862234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8180615f0 0x7ff818061a60 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7ff810009c90 tx=0x7ff8100091d0).stop 2020-11-20T12:21:03.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 -- 172.21.15.116:0/1878862234 shutdown_connections 2020-11-20T12:21:03.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 --2- 172.21.15.116:0/1878862234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8180615f0 0x7ff818061a60 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:03.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 --2- 172.21.15.116:0/1878862234 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff8180aa5f0 0x7ff81813b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:03.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 -- 172.21.15.116:0/1878862234 >> 172.21.15.116:0/1878862234 conn(0x7ff8180678a0 msgr2=0x7ff818067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:03.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 -- 172.21.15.116:0/1878862234 shutdown_connections 2020-11-20T12:21:03.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 -- 172.21.15.116:0/1878862234 wait complete. 2020-11-20T12:21:03.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 Processor -- start 2020-11-20T12:21:03.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 -- start start 2020-11-20T12:21:03.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8180615f0 0x7ff818154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:03.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8180aa5f0 0x7ff81813a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:03.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff81813af90 0x7ff818152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:03.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff818066660 con 0x7ff8180aa5f0 2020-11-20T12:21:03.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff818066900 con 0x7ff8180615f0 2020-11-20T12:21:03.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.477+0000 7ff81d859700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff818066ba0 con 0x7ff81813af90 2020-11-20T12:21:03.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff8167fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8180aa5f0 0x7ff81813a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:03.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff8167fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8180aa5f0 0x7ff81813a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50352/0 (socket says 172.21.15.116:50352) 2020-11-20T12:21:03.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff8167fc700 1 -- 172.21.15.116:0/918039209 learned_addr learned my addr 172.21.15.116:0/918039209 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:03.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff816ffd700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8180615f0 0x7ff818154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:03.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff8177fe700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff81813af90 0x7ff818152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:03.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff8167fc700 1 -- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff81813af90 msgr2=0x7ff818152580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:03.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff8167fc700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff81813af90 0x7ff818152580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:03.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff8167fc700 1 -- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8180615f0 msgr2=0x7ff818154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:03.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff8167fc700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8180615f0 0x7ff818154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:03.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff8167fc700 1 -- 172.21.15.116:0/918039209 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff818066e40 con 0x7ff8180aa5f0 2020-11-20T12:21:03.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff816ffd700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8180615f0 0x7ff818154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:03.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff8167fc700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8180aa5f0 0x7ff81813a3c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7ff80c00eba0 tx=0x7ff80c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:03.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff7feffd700 1 -- 172.21.15.116:0/918039209 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c00c8e0 con 0x7ff8180aa5f0 2020-11-20T12:21:03.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff7feffd700 1 -- 172.21.15.116:0/918039209 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80c00ca40 con 0x7ff8180aa5f0 2020-11-20T12:21:03.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff7feffd700 1 -- 172.21.15.116:0/918039209 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c00cdc0 con 0x7ff8180aa5f0 2020-11-20T12:21:03.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff81d859700 1 -- 172.21.15.116:0/918039209 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8180670e0 con 0x7ff8180aa5f0 2020-11-20T12:21:03.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff81d859700 1 -- 172.21.15.116:0/918039209 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8181554d0 con 0x7ff8180aa5f0 2020-11-20T12:21:03.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.481+0000 7ff81d859700 1 -- 172.21.15.116:0/918039209 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff818067270 con 0x7ff8180aa5f0 2020-11-20T12:21:03.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.485+0000 7ff7feffd700 1 -- 172.21.15.116:0/918039209 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff80c012600 con 0x7ff8180aa5f0 2020-11-20T12:21:03.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.485+0000 7ff7feffd700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7f40557c0 0x7ff7f4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:03.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.485+0000 7ff7feffd700 1 -- 172.21.15.116:0/918039209 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff80c06eaf0 con 0x7ff8180aa5f0 2020-11-20T12:21:03.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.485+0000 7ff816ffd700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7f40557c0 0x7ff7f4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:03.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.485+0000 7ff816ffd700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7f40557c0 0x7ff7f4057c80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7ff808001230 tx=0x7ff8080078c0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:03.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.489+0000 7ff7feffd700 1 -- 172.21.15.116:0/918039209 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff80c044810 con 0x7ff8180aa5f0 2020-11-20T12:21:03.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:03.645+0000 7ff81d859700 1 -- 172.21.15.116:0/918039209 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff81805cf30 con 0x7ff8180aa5f0 2020-11-20T12:21:04.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.365+0000 7ff7feffd700 1 -- 172.21.15.116:0/918039209 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v98) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff80c047e30 con 0x7ff8180aa5f0 2020-11-20T12:21:04.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.369+0000 7ff81d859700 1 -- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7f40557c0 msgr2=0x7ff7f4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:04.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.369+0000 7ff81d859700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7f40557c0 0x7ff7f4057c80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7ff808001230 tx=0x7ff8080078c0).stop 2020-11-20T12:21:04.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.369+0000 7ff81d859700 1 -- 172.21.15.116:0/918039209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8180aa5f0 msgr2=0x7ff81813a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:04.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.369+0000 7ff81d859700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8180aa5f0 0x7ff81813a3c0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7ff80c00eba0 tx=0x7ff80c006df0).stop 2020-11-20T12:21:04.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.369+0000 7ff81d859700 1 -- 172.21.15.116:0/918039209 shutdown_connections 2020-11-20T12:21:04.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.369+0000 7ff81d859700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7f40557c0 0x7ff7f4057c80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:04.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.369+0000 7ff81d859700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff81813af90 0x7ff818152580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:04.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.369+0000 7ff81d859700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8180615f0 0x7ff818154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:04.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.369+0000 7ff81d859700 1 --2- 172.21.15.116:0/918039209 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8180aa5f0 0x7ff81813a3c0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:04.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.369+0000 7ff81d859700 1 -- 172.21.15.116:0/918039209 >> 172.21.15.116:0/918039209 conn(0x7ff8180678a0 msgr2=0x7ff8180c6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:04.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.373+0000 7ff81d859700 1 -- 172.21.15.116:0/918039209 shutdown_connections 2020-11-20T12:21:04.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.373+0000 7ff81d859700 1 -- 172.21.15.116:0/918039209 wait complete. 2020-11-20T12:21:04.377 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2020-11-20T12:21:04.400 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-20T12:21:04.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.593+0000 7fc4ce592700 1 Processor -- start 2020-11-20T12:21:04.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.593+0000 7fc4ce592700 1 -- start start 2020-11-20T12:21:04.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4ce592700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc4c80a7830 0x7fc4c80a7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:04.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4ce592700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4c80c4c50 0x7fc4c80be110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:04.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4ce592700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c80c3d30 con 0x7fc4c80be600 2020-11-20T12:21:04.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4ce592700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c80bea50 con 0x7fc4c80a7830 2020-11-20T12:21:04.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4ce592700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80c3fd0 con 0x7fc4c80c4c50 2020-11-20T12:21:04.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c7fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc4c80a7830 0x7fc4c80a7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:04.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c7fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc4c80a7830 0x7fc4c80a7c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:49410/0 (socket says 172.21.15.116:49410) 2020-11-20T12:21:04.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c7fff700 1 -- 172.21.15.116:0/4083670043 learned_addr learned my addr 172.21.15.116:0/4083670043 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:04.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c77fe700 1 --2- 172.21.15.116:0/4083670043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4c80c4c50 0x7fc4c80be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:04.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c7fff700 1 -- 172.21.15.116:0/4083670043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4c80c4c50 msgr2=0x7fc4c80be110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:04.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c7fff700 1 --2- 172.21.15.116:0/4083670043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4c80c4c50 0x7fc4c80be110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:04.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c7fff700 1 -- 172.21.15.116:0/4083670043 >> v1:172.21.15.173:6789/0 conn(0x7fc4c80be600 legacy=0x7fc4c81515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:04.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c7fff700 1 -- 172.21.15.116:0/4083670043 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c80c4270 con 0x7fc4c80a7830 2020-11-20T12:21:04.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c7fff700 1 --2- 172.21.15.116:0/4083670043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4c80a7830 0x7fc4c80a7c10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fc4b8009d50 tx=0x7fc4b8009290).ready entity=mon.1 client_cookie=14c51bcd5e3488dc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:04.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c6ffd700 1 -- 172.21.15.116:0/4083670043 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b800bbc0 con 0x7fc4c80a7830 2020-11-20T12:21:04.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c6ffd700 1 -- 172.21.15.116:0/4083670043 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b800bd20 con 0x7fc4c80a7830 2020-11-20T12:21:04.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.597+0000 7fc4c6ffd700 1 -- 172.21.15.116:0/4083670043 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8011610 con 0x7fc4c80a7830 2020-11-20T12:21:04.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.601+0000 7fc4ce592700 1 -- 172.21.15.116:0/4083670043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4c80a7830 msgr2=0x7fc4c80a7c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:04.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.601+0000 7fc4ce592700 1 --2- 172.21.15.116:0/4083670043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4c80a7830 0x7fc4c80a7c10 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7fc4b8009d50 tx=0x7fc4b8009290).stop 2020-11-20T12:21:04.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.601+0000 7fc4ce592700 1 -- 172.21.15.116:0/4083670043 shutdown_connections 2020-11-20T12:21:04.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.601+0000 7fc4ce592700 1 --2- 172.21.15.116:0/4083670043 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4c80a7830 0x7fc4c80a7c10 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:04.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.601+0000 7fc4ce592700 1 --2- 172.21.15.116:0/4083670043 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4c80c4c50 0x7fc4c80be110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:04.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.601+0000 7fc4ce592700 1 -- 172.21.15.116:0/4083670043 >> 172.21.15.116:0/4083670043 conn(0x7fc4c8068320 msgr2=0x7fc4c80c8870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:04.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.601+0000 7fc4ce592700 1 -- 172.21.15.116:0/4083670043 shutdown_connections 2020-11-20T12:21:04.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.601+0000 7fc4ce592700 1 -- 172.21.15.116:0/4083670043 wait complete. 2020-11-20T12:21:04.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.601+0000 7fc4ce592700 1 Processor -- start 2020-11-20T12:21:04.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.601+0000 7fc4ce592700 1 -- start start 2020-11-20T12:21:04.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4ce592700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4c80be600 0x7fc4c814e730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:04.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4ce592700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c80c4c50 0x7fc4c8140f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:04.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4ce592700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4c814ec20 0x7fc4c8141460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:04.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4ce592700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c8161960 con 0x7fc4c80c4c50 2020-11-20T12:21:04.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4ce592700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80adaa0 con 0x7fc4c80be600 2020-11-20T12:21:04.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4ce592700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c80ad2c0 con 0x7fc4c814ec20 2020-11-20T12:21:04.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4c77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c80c4c50 0x7fc4c8140f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:04.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4c77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c80c4c50 0x7fc4c8140f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50368/0 (socket says 172.21.15.116:50368) 2020-11-20T12:21:04.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4c77fe700 1 -- 172.21.15.116:0/1147567205 learned_addr learned my addr 172.21.15.116:0/1147567205 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:04.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4c7fff700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4c80be600 0x7fc4c814e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:04.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4ccb2f700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4c814ec20 0x7fc4c8141460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:04.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4c77fe700 1 -- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4c814ec20 msgr2=0x7fc4c8141460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:04.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4c77fe700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4c814ec20 0x7fc4c8141460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:04.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4c77fe700 1 -- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4c80be600 msgr2=0x7fc4c814e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:04.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4c77fe700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4c80be600 0x7fc4c814e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:04.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4c77fe700 1 -- 172.21.15.116:0/1147567205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c80ad560 con 0x7fc4c80c4c50 2020-11-20T12:21:04.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4ccb2f700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4c814ec20 0x7fc4c8141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:04.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4c7fff700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4c80be600 0x7fc4c814e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:04.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4c77fe700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c80c4c50 0x7fc4c8140f70 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7fc4bc00ebd0 tx=0x7fc4bc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:04.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4a7fff700 1 -- 172.21.15.116:0/1147567205 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00b9a0 con 0x7fc4c80c4c50 2020-11-20T12:21:04.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4a7fff700 1 -- 172.21.15.116:0/1147567205 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc00bb00 con 0x7fc4c80c4c50 2020-11-20T12:21:04.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4ce592700 1 -- 172.21.15.116:0/1147567205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c80ad800 con 0x7fc4c80c4c50 2020-11-20T12:21:04.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4a7fff700 1 -- 172.21.15.116:0/1147567205 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc00be80 con 0x7fc4c80c4c50 2020-11-20T12:21:04.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.605+0000 7fc4ce592700 1 -- 172.21.15.116:0/1147567205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c80ac5d0 con 0x7fc4c80c4c50 2020-11-20T12:21:04.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.609+0000 7fc4ce592700 1 -- 172.21.15.116:0/1147567205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c805d2a0 con 0x7fc4c80c4c50 2020-11-20T12:21:04.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.609+0000 7fc4a7fff700 1 -- 172.21.15.116:0/1147567205 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc4bc012600 con 0x7fc4c80c4c50 2020-11-20T12:21:04.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.609+0000 7fc4a7fff700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4a00557c0 0x7fc4a0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:04.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.609+0000 7fc4a7fff700 1 -- 172.21.15.116:0/1147567205 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5721+0+0 (secure 0 0 0) 0x7fc4bc06e920 con 0x7fc4c80c4c50 2020-11-20T12:21:04.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.609+0000 7fc4c7fff700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4a00557c0 0x7fc4a0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:04.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.613+0000 7fc4c7fff700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4a00557c0 0x7fc4a0057c80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fc4c805cf80 tx=0x7fc4b8016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:04.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.617+0000 7fc4a7fff700 1 -- 172.21.15.116:0/1147567205 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc4bc0448c0 con 0x7fc4c80c4c50 2020-11-20T12:21:04.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:04.777+0000 7fc4ce592700 1 -- 172.21.15.116:0/1147567205 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc4c80a8130 con 0x7fc4c80c4c50 2020-11-20T12:21:05.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.377+0000 7fc4a7fff700 1 -- 172.21.15.116:0/1147567205 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v99) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc4bc047ee0 con 0x7fc4c80c4c50 2020-11-20T12:21:05.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 -- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4a00557c0 msgr2=0x7fc4a0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:05.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4a00557c0 0x7fc4a0057c80 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7fc4c805cf80 tx=0x7fc4b8016000).stop 2020-11-20T12:21:05.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 -- 172.21.15.116:0/1147567205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c80c4c50 msgr2=0x7fc4c8140f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:05.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c80c4c50 0x7fc4c8140f70 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7fc4bc00ebd0 tx=0x7fc4bc006df0).stop 2020-11-20T12:21:05.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 -- 172.21.15.116:0/1147567205 shutdown_connections 2020-11-20T12:21:05.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4a00557c0 0x7fc4a0057c80 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:05.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4c814ec20 0x7fc4c8141460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:05.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc4c80be600 0x7fc4c814e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:05.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 --2- 172.21.15.116:0/1147567205 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc4c80c4c50 0x7fc4c8140f70 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:05.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 -- 172.21.15.116:0/1147567205 >> 172.21.15.116:0/1147567205 conn(0x7fc4c8068320 msgr2=0x7fc4c80a9580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:05.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 -- 172.21.15.116:0/1147567205 shutdown_connections 2020-11-20T12:21:05.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.381+0000 7fc4ce592700 1 -- 172.21.15.116:0/1147567205 wait complete. 2020-11-20T12:21:05.388 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2020-11-20T12:21:05.409 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T12:21:05.409 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-20T12:21:05.410 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-20T12:21:05.410 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:21:05.410 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-20T12:21:05.411 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-20T12:21:05.411 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-20T12:21:05.411 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:21:05.412 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T12:21:05.412 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-20T12:21:05.412 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-20T12:21:05.412 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:21:05.413 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-20T12:21:05.413 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-20T12:21:05.413 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-20T12:21:05.414 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:21:05.438 INFO:tasks.ceph.mds.c:Stopped 2020-11-20T12:21:05.439 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-20T12:21:05.440 INFO:tasks.ceph.mds.d:Stopped 2020-11-20T12:21:05.440 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-20T12:21:05.441 INFO:tasks.ceph.mds.b:Stopped 2020-11-20T12:21:05.441 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T12:21:05.442 INFO:tasks.ceph.mds.a:Stopped 2020-11-20T12:21:05.443 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T12:21:05.450 INFO:tasks.ceph.mds.d:Started 2020-11-20T12:21:05.450 DEBUG:teuthology.parallel:result is None 2020-11-20T12:21:05.453 INFO:tasks.ceph.mds.b:Started 2020-11-20T12:21:05.454 DEBUG:teuthology.parallel:result is None 2020-11-20T12:21:05.495 INFO:tasks.ceph.mds.c:Started 2020-11-20T12:21:05.496 DEBUG:teuthology.parallel:result is None 2020-11-20T12:21:05.499 INFO:tasks.ceph.mds.a:Started 2020-11-20T12:21:05.499 DEBUG:teuthology.parallel:result is None 2020-11-20T12:21:05.500 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-20T12:21:05.558 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2020-11-20T12:21:05.559 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2020-11-20T12:21:05.602 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:21:05.602 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:21:05.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.681+0000 7f44a3024700 1 Processor -- start 2020-11-20T12:21:05.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.681+0000 7f44a3024700 1 -- start start 2020-11-20T12:21:05.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.681+0000 7f44a3024700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0c3f90 0x7f449c0c5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:05.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.681+0000 7f44a3024700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c155580 con 0x7f449c151440 2020-11-20T12:21:05.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.681+0000 7f44a3024700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f449c1557b0 con 0x7f449c0c5420 2020-11-20T12:21:05.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.681+0000 7f44a3024700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c155a50 con 0x7f449c0c3f90 2020-11-20T12:21:05.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f44a15c1700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f449c151440 0x7f449c1518f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59834/0 (socket says 172.21.15.116:59834) 2020-11-20T12:21:05.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f44a15c1700 1 -- 172.21.15.116:0/2208102717 learned_addr learned my addr 172.21.15.116:0/2208102717 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:05.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f4493fff700 1 --2- 172.21.15.116:0/2208102717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0c3f90 0x7f449c0c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:05.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f44937fe700 1 -- 172.21.15.116:0/2208102717 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812143807 0 0) 0x7f449c155580 con 0x7f449c151440 2020-11-20T12:21:05.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f44937fe700 1 -- 172.21.15.116:0/2208102717 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4488003240 con 0x7f449c151440 2020-11-20T12:21:05.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f44937fe700 1 -- 172.21.15.116:0/2208102717 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578218227 0 0) 0x7f449c1557b0 con 0x7f449c0c5420 2020-11-20T12:21:05.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f44937fe700 1 -- 172.21.15.116:0/2208102717 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f449c155580 con 0x7f449c0c5420 2020-11-20T12:21:05.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f4493fff700 1 -- 172.21.15.116:0/2208102717 >> v1:172.21.15.173:6789/0 conn(0x7f449c0c5420 legacy=0x7f449c0c3880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:05.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f4493fff700 1 -- 172.21.15.116:0/2208102717 >> v1:172.21.15.116:6789/0 conn(0x7f449c151440 legacy=0x7f449c1518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f4493fff700 1 -- 172.21.15.116:0/2208102717 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c155cf0 con 0x7f449c0c3f90 2020-11-20T12:21:05.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f4493fff700 1 --2- 172.21.15.116:0/2208102717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0c3f90 0x7f449c0c5800 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f448c00b9d0 tx=0x7f448c006df0).ready entity=mon.2 client_cookie=98fb10eeb23f9a8d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f44937fe700 1 -- 172.21.15.116:0/2208102717 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c009680 con 0x7f449c0c3f90 2020-11-20T12:21:05.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f44937fe700 1 -- 172.21.15.116:0/2208102717 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f448c0097e0 con 0x7f449c0c3f90 2020-11-20T12:21:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.685+0000 7f44937fe700 1 -- 172.21.15.116:0/2208102717 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f448c009b60 con 0x7f449c0c3f90 2020-11-20T12:21:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.689+0000 7f44a3024700 1 -- 172.21.15.116:0/2208102717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0c3f90 msgr2=0x7f449c0c5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:05.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.689+0000 7f44a3024700 1 --2- 172.21.15.116:0/2208102717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0c3f90 0x7f449c0c5800 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f448c00b9d0 tx=0x7f448c006df0).stop 2020-11-20T12:21:05.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.689+0000 7f44a3024700 1 -- 172.21.15.116:0/2208102717 shutdown_connections 2020-11-20T12:21:05.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.689+0000 7f44a3024700 1 --2- 172.21.15.116:0/2208102717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0c3f90 0x7f449c0c5800 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:05.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.689+0000 7f44a3024700 1 -- 172.21.15.116:0/2208102717 >> 172.21.15.116:0/2208102717 conn(0x7f449c0678a0 msgr2=0x7f449c067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:05.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.689+0000 7f44a3024700 1 -- 172.21.15.116:0/2208102717 shutdown_connections 2020-11-20T12:21:05.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.689+0000 7f44a3024700 1 -- 172.21.15.116:0/2208102717 wait complete. 2020-11-20T12:21:05.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.689+0000 7f44a3024700 1 Processor -- start 2020-11-20T12:21:05.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f44a3024700 1 -- start start 2020-11-20T12:21:05.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f44a3024700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c0c3f90 0x7f449c0c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:05.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f44a3024700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0c5420 0x7f449c14a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:05.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f44a3024700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f449c151440 0x7f449c14a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:05.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f44a3024700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c168d90 con 0x7f449c151440 2020-11-20T12:21:05.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f44a3024700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f449c169030 con 0x7f449c0c3f90 2020-11-20T12:21:05.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f44a3024700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f449c1692d0 con 0x7f449c0c5420 2020-11-20T12:21:05.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f44a15c1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f449c151440 0x7f449c14a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:05.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f44a15c1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f449c151440 0x7f449c14a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50406/0 (socket says 172.21.15.116:50406) 2020-11-20T12:21:05.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f44a15c1700 1 -- 172.21.15.116:0/3498843615 learned_addr learned my addr 172.21.15.116:0/3498843615 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:05.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f4493fff700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0c5420 0x7f449c14a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:05.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.693+0000 7f44a0dc0700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c0c3f90 0x7f449c0c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:05.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f44a15c1700 1 -- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0c5420 msgr2=0x7f449c14a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:05.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f44a15c1700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0c5420 0x7f449c14a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:05.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f44a15c1700 1 -- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c0c3f90 msgr2=0x7f449c0c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:05.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f44a15c1700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c0c3f90 0x7f449c0c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:05.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f44a15c1700 1 -- 172.21.15.116:0/3498843615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c169570 con 0x7f449c151440 2020-11-20T12:21:05.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f44a15c1700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f449c151440 0x7f449c14a850 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f449400ca60 tx=0x7f4494007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:05.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f4490ff9700 1 -- 172.21.15.116:0/3498843615 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449400ce40 con 0x7f449c151440 2020-11-20T12:21:05.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f4490ff9700 1 -- 172.21.15.116:0/3498843615 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4494005b80 con 0x7f449c151440 2020-11-20T12:21:05.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f44a3024700 1 -- 172.21.15.116:0/3498843615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f449c169810 con 0x7f449c151440 2020-11-20T12:21:05.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f4490ff9700 1 -- 172.21.15.116:0/3498843615 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4494007110 con 0x7f449c151440 2020-11-20T12:21:05.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f44a3024700 1 -- 172.21.15.116:0/3498843615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f449c0c8510 con 0x7f449c151440 2020-11-20T12:21:05.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f44a3024700 1 -- 172.21.15.116:0/3498843615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f449c169bd0 con 0x7f449c151440 2020-11-20T12:21:05.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.697+0000 7f4490ff9700 1 -- 172.21.15.116:0/3498843615 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4494010030 con 0x7f449c151440 2020-11-20T12:21:05.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.701+0000 7f4490ff9700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44740557f0 0x7f4474057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:05.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.701+0000 7f4490ff9700 1 -- 172.21.15.116:0/3498843615 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f449406e350 con 0x7f449c151440 2020-11-20T12:21:05.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.701+0000 7f44a0dc0700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44740557f0 0x7f4474057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:05.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.701+0000 7f44a0dc0700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44740557f0 0x7f4474057cb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f4498000bc0 tx=0x7f4498007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:05.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.705+0000 7f4490ff9700 1 -- 172.21.15.116:0/3498843615 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f44940443c0 con 0x7f449c151440 2020-11-20T12:21:05.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:05.881+0000 7f44a3024700 1 -- 172.21.15.116:0/3498843615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f449c1699a0 con 0x7f449c151440 2020-11-20T12:21:06.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.389+0000 7f4490ff9700 1 -- 172.21.15.116:0/3498843615 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v100) v1 ==== 97+0+0 (secure 0 0 0) 0x7f44940479e0 con 0x7f449c151440 2020-11-20T12:21:06.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 -- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44740557f0 msgr2=0x7f4474057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44740557f0 0x7f4474057cb0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f4498000bc0 tx=0x7f4498007a00).stop 2020-11-20T12:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 -- 172.21.15.116:0/3498843615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f449c151440 msgr2=0x7f449c14a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f449c151440 0x7f449c14a850 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f449400ca60 tx=0x7f4494007e20).stop 2020-11-20T12:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 -- 172.21.15.116:0/3498843615 shutdown_connections 2020-11-20T12:21:06.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44740557f0 0x7f4474057cb0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f449c151440 0x7f449c14a850 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f449c0c3f90 0x7f449c0c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 --2- 172.21.15.116:0/3498843615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f449c0c5420 0x7f449c14a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 -- 172.21.15.116:0/3498843615 >> 172.21.15.116:0/3498843615 conn(0x7f449c0678a0 msgr2=0x7f449c0a9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 -- 172.21.15.116:0/3498843615 shutdown_connections 2020-11-20T12:21:06.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.393+0000 7f44a3024700 1 -- 172.21.15.116:0/3498843615 wait complete. 2020-11-20T12:21:06.398 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2020-11-20T12:21:06.420 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-20T12:21:06.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.597+0000 7f777ee1b700 1 Processor -- start 2020-11-20T12:21:06.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.597+0000 7f777ee1b700 1 -- start start 2020-11-20T12:21:06.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777ee1b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77780ba890 0x7f77780bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:06.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777ee1b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7778148a20 0x7f7778148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:06.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777ee1b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7778066140 con 0x7f77780c2230 2020-11-20T12:21:06.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777ee1b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77780c1d80 con 0x7f77780ba890 2020-11-20T12:21:06.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777ee1b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77781526c0 con 0x7f7778148a20 2020-11-20T12:21:06.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777cbb7700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77780ba890 0x7f77780bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:06.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f7777fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f77780c2230 0x7f77780c1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59846/0 (socket says 172.21.15.116:59846) 2020-11-20T12:21:06.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f7777fff700 1 -- 172.21.15.116:0/5221783 learned_addr learned my addr 172.21.15.116:0/5221783 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:06.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777d3b8700 1 --2- 172.21.15.116:0/5221783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7778148a20 0x7f7778148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:06.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777cbb7700 1 -- 172.21.15.116:0/5221783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7778148a20 msgr2=0x7f7778148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:06.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777cbb7700 1 --2- 172.21.15.116:0/5221783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7778148a20 0x7f7778148ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777cbb7700 1 -- 172.21.15.116:0/5221783 >> v1:172.21.15.116:6789/0 conn(0x7f77780c2230 legacy=0x7f77780c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:06.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777cbb7700 1 -- 172.21.15.116:0/5221783 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7778152960 con 0x7f77780ba890 2020-11-20T12:21:06.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f77777fe700 1 -- 172.21.15.116:0/5221783 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1712014007 0 0) 0x7f7778066140 con 0x7f77780c2230 2020-11-20T12:21:06.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f777cbb7700 1 --2- 172.21.15.116:0/5221783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77780ba890 0x7f77780bac70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f776c00b9a0 tx=0x7f776c006df0).ready entity=mon.0 client_cookie=8848e30a3142163a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:06.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f77777fe700 1 -- 172.21.15.116:0/5221783 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c0097b0 con 0x7f77780ba890 2020-11-20T12:21:06.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f77777fe700 1 -- 172.21.15.116:0/5221783 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f776c009910 con 0x7f77780ba890 2020-11-20T12:21:06.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.601+0000 7f77777fe700 1 -- 172.21.15.116:0/5221783 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f776c009c90 con 0x7f77780ba890 2020-11-20T12:21:06.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.605+0000 7f777ee1b700 1 -- 172.21.15.116:0/5221783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77780ba890 msgr2=0x7f77780bac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:06.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.605+0000 7f777ee1b700 1 --2- 172.21.15.116:0/5221783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77780ba890 0x7f77780bac70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f776c00b9a0 tx=0x7f776c006df0).stop 2020-11-20T12:21:06.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.605+0000 7f777ee1b700 1 -- 172.21.15.116:0/5221783 shutdown_connections 2020-11-20T12:21:06.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.605+0000 7f777ee1b700 1 --2- 172.21.15.116:0/5221783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77780ba890 0x7f77780bac70 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.605+0000 7f777ee1b700 1 --2- 172.21.15.116:0/5221783 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7778148a20 0x7f7778148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.605+0000 7f777ee1b700 1 -- 172.21.15.116:0/5221783 >> 172.21.15.116:0/5221783 conn(0x7f77780c4f80 msgr2=0x7f77780c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:06.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.605+0000 7f777ee1b700 1 -- 172.21.15.116:0/5221783 shutdown_connections 2020-11-20T12:21:06.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.605+0000 7f777ee1b700 1 -- 172.21.15.116:0/5221783 wait complete. 2020-11-20T12:21:06.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.605+0000 7f777ee1b700 1 Processor -- start 2020-11-20T12:21:06.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.605+0000 7f777ee1b700 1 -- start start 2020-11-20T12:21:06.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f777ee1b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77780ba890 0x7f77781477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:06.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f777ee1b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77780c2230 0x7f77780bde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:06.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f777ee1b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7778148a20 0x7f7778164660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:06.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f777ee1b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7778150d00 con 0x7f77780c2230 2020-11-20T12:21:06.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f777ee1b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7778150fa0 con 0x7f77780ba890 2020-11-20T12:21:06.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f777ee1b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7778151240 con 0x7f7778148a20 2020-11-20T12:21:06.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7777fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77780c2230 0x7f77780bde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:06.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7777fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77780c2230 0x7f77780bde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50418/0 (socket says 172.21.15.116:50418) 2020-11-20T12:21:06.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7777fff700 1 -- 172.21.15.116:0/3205152886 learned_addr learned my addr 172.21.15.116:0/3205152886 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:06.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f777cbb7700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77780ba890 0x7f77781477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:06.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f777d3b8700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7778148a20 0x7f7778164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7777fff700 1 -- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7778148a20 msgr2=0x7f7778164660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7777fff700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7778148a20 0x7f7778164660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7777fff700 1 -- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77780ba890 msgr2=0x7f77781477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7777fff700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77780ba890 0x7f77781477d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7777fff700 1 -- 172.21.15.116:0/3205152886 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77781514e0 con 0x7f77780c2230 2020-11-20T12:21:06.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7777fff700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77780c2230 0x7f77780bde00 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f776400c6e0 tx=0x7f7764007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:06.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7774ff9700 1 -- 172.21.15.116:0/3205152886 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77640072e0 con 0x7f77780c2230 2020-11-20T12:21:06.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7774ff9700 1 -- 172.21.15.116:0/3205152886 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7764007440 con 0x7f77780c2230 2020-11-20T12:21:06.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f777ee1b700 1 -- 172.21.15.116:0/3205152886 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7778151780 con 0x7f77780c2230 2020-11-20T12:21:06.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f7774ff9700 1 -- 172.21.15.116:0/3205152886 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7764005800 con 0x7f77780c2230 2020-11-20T12:21:06.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.609+0000 7f777ee1b700 1 -- 172.21.15.116:0/3205152886 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7778147cc0 con 0x7f77780c2230 2020-11-20T12:21:06.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.613+0000 7f777ee1b700 1 -- 172.21.15.116:0/3205152886 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7778151910 con 0x7f77780c2230 2020-11-20T12:21:06.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.613+0000 7f7774ff9700 1 -- 172.21.15.116:0/3205152886 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7764010070 con 0x7f77780c2230 2020-11-20T12:21:06.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.613+0000 7f7774ff9700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77500557c0 0x7f7750057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:06.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.613+0000 7f7774ff9700 1 -- 172.21.15.116:0/3205152886 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f776406db50 con 0x7f77780c2230 2020-11-20T12:21:06.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.617+0000 7f777cbb7700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77500557c0 0x7f7750057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:06.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.617+0000 7f777cbb7700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77500557c0 0x7f7750057c80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f776c006e30 tx=0x7f776c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:06.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.621+0000 7f7774ff9700 1 -- 172.21.15.116:0/3205152886 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7764043c10 con 0x7f77780c2230 2020-11-20T12:21:06.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.833+0000 7f777ee1b700 1 -- 172.21.15.116:0/3205152886 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f777805d2a0 con 0x7f77780c2230 2020-11-20T12:21:06.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.833+0000 7f7774ff9700 1 -- 172.21.15.116:0/3205152886 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+5138 (secure 0 0 0) 0x7f7764047230 con 0x7f77780c2230 2020-11-20T12:21:06.837 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:06.837 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:21:06.837 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2020-11-20T12:21:06.838 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.838 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2020-11-20T12:21:06.838 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw==", 2020-11-20T12:21:06.838 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.838 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:06.839 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.839 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.839 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.839 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.840 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.840 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.840 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2020-11-20T12:21:06.840 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ==", 2020-11-20T12:21:06.840 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.841 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:06.841 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.841 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.841 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.841 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.842 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.842 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.842 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2020-11-20T12:21:06.842 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag==", 2020-11-20T12:21:06.842 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.843 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:06.843 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.843 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.843 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.843 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.844 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.844 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.844 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2020-11-20T12:21:06.844 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ==", 2020-11-20T12:21:06.844 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.845 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:06.845 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.845 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.845 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.845 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.846 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.846 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.846 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2020-11-20T12:21:06.846 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ==", 2020-11-20T12:21:06.847 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.847 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.847 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.847 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.847 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.848 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.848 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.848 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2020-11-20T12:21:06.848 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q==", 2020-11-20T12:21:06.848 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.849 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.849 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.849 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.849 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.849 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.850 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.850 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2020-11-20T12:21:06.850 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw==", 2020-11-20T12:21:06.850 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.850 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.851 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.851 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.851 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.851 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.851 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.852 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2020-11-20T12:21:06.852 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w==", 2020-11-20T12:21:06.852 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.852 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.852 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.853 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.853 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.853 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.853 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.854 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2020-11-20T12:21:06.854 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA==", 2020-11-20T12:21:06.854 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.854 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.854 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.855 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.855 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.855 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.855 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.855 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2020-11-20T12:21:06.856 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ==", 2020-11-20T12:21:06.856 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.856 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.856 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.856 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.857 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.857 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.857 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.857 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2020-11-20T12:21:06.858 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg==", 2020-11-20T12:21:06.858 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.858 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.858 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.858 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.859 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.859 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.859 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.859 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2020-11-20T12:21:06.859 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg==", 2020-11-20T12:21:06.860 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.860 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.860 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.860 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.860 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.861 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.861 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.861 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2020-11-20T12:21:06.861 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA==", 2020-11-20T12:21:06.861 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.862 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:06.862 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2020-11-20T12:21:06.862 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-20T12:21:06.862 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.862 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.863 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.863 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2020-11-20T12:21:06.863 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow==", 2020-11-20T12:21:06.863 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.864 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:06.864 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:21:06.864 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:21:06.864 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:21:06.864 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.865 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.865 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.865 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2020-11-20T12:21:06.865 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ==", 2020-11-20T12:21:06.865 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.866 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:06.866 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:21:06.866 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:21:06.866 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:21:06.866 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.867 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.867 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.867 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2020-11-20T12:21:06.867 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ==", 2020-11-20T12:21:06.867 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.868 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:06.868 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:21:06.868 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:21:06.868 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:21:06.868 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.869 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.869 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.869 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2020-11-20T12:21:06.869 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBDs7dfh+KKOBAAZvcPXWly9/mxRE7NWuR85A==", 2020-11-20T12:21:06.869 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.870 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:21:06.870 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:06.870 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:06.870 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.870 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.871 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.871 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.871 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2020-11-20T12:21:06.871 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A==", 2020-11-20T12:21:06.871 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.872 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:21:06.872 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:21:06.873 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.874 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.874 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:06.874 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:06.874 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2020-11-20T12:21:06.875 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA==", 2020-11-20T12:21:06.875 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:06.875 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:21:06.875 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:21:06.875 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:06.876 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.876 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:06.876 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:21:06.876 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:21:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 -- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77500557c0 msgr2=0x7f7750057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77500557c0 0x7f7750057c80 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f776c006e30 tx=0x7f776c015000).stop 2020-11-20T12:21:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 -- 172.21.15.116:0/3205152886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77780c2230 msgr2=0x7f77780bde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77780c2230 0x7f77780bde00 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f776400c6e0 tx=0x7f7764007a60).stop 2020-11-20T12:21:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 -- 172.21.15.116:0/3205152886 shutdown_connections 2020-11-20T12:21:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f77500557c0 0x7f7750057c80 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7778148a20 0x7f7778164660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77780ba890 0x7f77781477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 --2- 172.21.15.116:0/3205152886 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77780c2230 0x7f77780bde00 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 -- 172.21.15.116:0/3205152886 >> 172.21.15.116:0/3205152886 conn(0x7f77780c4f80 msgr2=0x7f77780664c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 -- 172.21.15.116:0/3205152886 shutdown_connections 2020-11-20T12:21:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:06.837+0000 7f777ee1b700 1 -- 172.21.15.116:0/3205152886 wait complete. 2020-11-20T12:21:06.881 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:07.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.053+0000 7fd0513d6700 1 Processor -- start 2020-11-20T12:21:07.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.053+0000 7fd0513d6700 1 -- start start 2020-11-20T12:21:07.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd0513d6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd04c153450 0x7fd04c062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd0513d6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c0677a0 0x7fd04c0b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd0513d6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd04c1543a0 con 0x7fd04c062880 2020-11-20T12:21:07.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd0513d6700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd04c067c00 con 0x7fd04c153450 2020-11-20T12:21:07.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd0513d6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c153920 con 0x7fd04c0677a0 2020-11-20T12:21:07.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04bfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd04c062880 0x7fd04c0c4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59858/0 (socket says 172.21.15.116:59858) 2020-11-20T12:21:07.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04bfff700 1 -- 172.21.15.116:0/3439870380 learned_addr learned my addr 172.21.15.116:0/3439870380 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:07.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd050bd5700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c0677a0 0x7fd04c0b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd043fff700 1 --2- 172.21.15.116:0/3439870380 >> v2:172.21.15.173:3300/0 conn(0x7fd04c153450 0x7fd04c062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1783595577 0 0) 0x7fd04c1543a0 con 0x7fd04c062880 2020-11-20T12:21:07.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd030003240 con 0x7fd04c062880 2020-11-20T12:21:07.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1126570196 0 0) 0x7fd030003240 con 0x7fd04c062880 2020-11-20T12:21:07.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c0677a0 msgr2=0x7fd04c0b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:07.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c0677a0 0x7fd04c0b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 >> v2:172.21.15.173:3300/0 conn(0x7fd04c153450 msgr2=0x7fd04c062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:07.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 --2- 172.21.15.116:0/3439870380 >> v2:172.21.15.173:3300/0 conn(0x7fd04c153450 0x7fd04c062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c153bc0 con 0x7fd04c062880 2020-11-20T12:21:07.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fd03c0014e0 con 0x7fd04c062880 2020-11-20T12:21:07.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 >> v1:172.21.15.116:6789/0 conn(0x7fd04c062880 legacy=0x7fd04c0c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c0677a0 0x7fd030003730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd030003c40 0x7fd0300044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0300049d0 0x7fd030004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c067c00 con 0x7fd0300049d0 2020-11-20T12:21:07.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c153920 con 0x7fd030003c40 2020-11-20T12:21:07.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.057+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd030004200 con 0x7fd04c0677a0 2020-11-20T12:21:07.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd043fff700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd030003c40 0x7fd0300044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd050bd5700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0300049d0 0x7fd030004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd04bfff700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c0677a0 0x7fd030003730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd043fff700 1 -- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c0677a0 msgr2=0x7fd030003730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:07.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd043fff700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c0677a0 0x7fd030003730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd043fff700 1 -- 172.21.15.116:0/3439870380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0300049d0 msgr2=0x7fd030004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:07.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd043fff700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0300049d0 0x7fd030004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd043fff700 1 -- 172.21.15.116:0/3439870380 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd030003240 con 0x7fd030003c40 2020-11-20T12:21:07.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd04bfff700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c0677a0 0x7fd030003730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:07.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd043fff700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd030003c40 0x7fd0300044e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fd04c14daa0 tx=0x7fd034006df0).ready entity=mon.1 client_cookie=6cf2ca0dbb950866 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:07.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03400b330 con 0x7fd030003c40 2020-11-20T12:21:07.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd03400ea30 con 0x7fd030003c40 2020-11-20T12:21:07.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd04b7fe700 1 -- 172.21.15.116:0/3439870380 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd03400edb0 con 0x7fd030003c40 2020-11-20T12:21:07.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd0513d6700 1 -- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd030003c40 msgr2=0x7fd0300044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd0513d6700 1 --2- 172.21.15.116:0/3439870380 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd030003c40 0x7fd0300044e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7fd04c14daa0 tx=0x7fd034006df0).stop 2020-11-20T12:21:07.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd050bd5700 1 -- 172.21.15.116:0/3439870380 reap_dead start 2020-11-20T12:21:07.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd0513d6700 1 -- 172.21.15.116:0/3439870380 shutdown_connections 2020-11-20T12:21:07.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd0513d6700 1 -- 172.21.15.116:0/3439870380 >> 172.21.15.116:0/3439870380 conn(0x7fd04c060c40 msgr2=0x7fd04c0c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:07.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd0513d6700 1 -- 172.21.15.116:0/3439870380 shutdown_connections 2020-11-20T12:21:07.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.061+0000 7fd0513d6700 1 -- 172.21.15.116:0/3439870380 wait complete. 2020-11-20T12:21:07.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.065+0000 7fd0513d6700 1 Processor -- start 2020-11-20T12:21:07.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.065+0000 7fd0513d6700 1 -- start start 2020-11-20T12:21:07.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd0513d6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c146b20 0x7fd04c146f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd0513d6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd04c14e180 0x7fd04c1454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd0513d6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd04c14e600 0x7fd04c1459d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd0513d6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c15d2d0 con 0x7fd04c14e600 2020-11-20T12:21:07.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd0513d6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c15d570 con 0x7fd04c14e180 2020-11-20T12:21:07.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd0513d6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04c15d810 con 0x7fd04c146b20 2020-11-20T12:21:07.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd050bd5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd04c14e600 0x7fd04c1459d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd050bd5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd04c14e600 0x7fd04c1459d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50440/0 (socket says 172.21.15.116:50440) 2020-11-20T12:21:07.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd050bd5700 1 -- 172.21.15.116:0/3124527615 learned_addr learned my addr 172.21.15.116:0/3124527615 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:07.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd04bfff700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c146b20 0x7fd04c146f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd043fff700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd04c14e180 0x7fd04c1454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd050bd5700 1 -- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c146b20 msgr2=0x7fd04c146f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd050bd5700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c146b20 0x7fd04c146f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd050bd5700 1 -- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd04c14e180 msgr2=0x7fd04c1454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd050bd5700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd04c14e180 0x7fd04c1454e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd050bd5700 1 -- 172.21.15.116:0/3124527615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd04c15dab0 con 0x7fd04c14e600 2020-11-20T12:21:07.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd050bd5700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd04c14e600 0x7fd04c1459d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7fd04c153830 tx=0x7fd0440091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:07.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd048ff9700 1 -- 172.21.15.116:0/3124527615 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd044019070 con 0x7fd04c14e600 2020-11-20T12:21:07.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd048ff9700 1 -- 172.21.15.116:0/3124527615 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd04400f970 con 0x7fd04c14e600 2020-11-20T12:21:07.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd0513d6700 1 -- 172.21.15.116:0/3124527615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd04c15dd50 con 0x7fd04c14e600 2020-11-20T12:21:07.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd048ff9700 1 -- 172.21.15.116:0/3124527615 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd04400fd10 con 0x7fd04c14e600 2020-11-20T12:21:07.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd0513d6700 1 -- 172.21.15.116:0/3124527615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd04c14ea70 con 0x7fd04c14e600 2020-11-20T12:21:07.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.069+0000 7fd0513d6700 1 -- 172.21.15.116:0/3124527615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd04c15dee0 con 0x7fd04c14e600 2020-11-20T12:21:07.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.073+0000 7fd048ff9700 1 -- 172.21.15.116:0/3124527615 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd044014650 con 0x7fd04c14e600 2020-11-20T12:21:07.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.073+0000 7fd048ff9700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd01c0557c0 0x7fd01c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.073+0000 7fd048ff9700 1 -- 172.21.15.116:0/3124527615 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd04406ec60 con 0x7fd04c14e600 2020-11-20T12:21:07.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.073+0000 7fd04bfff700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd01c0557c0 0x7fd01c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.073+0000 7fd04bfff700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd01c0557c0 0x7fd01c057c80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fd03c001e90 tx=0x7fd03c001610).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:07.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.077+0000 7fd048ff9700 1 -- 172.21.15.116:0/3124527615 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd044044d50 con 0x7fd04c14e600 2020-11-20T12:21:07.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.233+0000 7fd0513d6700 1 -- 172.21.15.116:0/3124527615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd04c05cf30 con 0x7fd04c14e600 2020-11-20T12:21:07.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.237+0000 7fd048ff9700 1 -- 172.21.15.116:0/3124527615 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+13725 (secure 0 0 0) 0x7fd044048370 con 0x7fd04c14e600 2020-11-20T12:21:07.240 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:07.240 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":100,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:06.380869+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:07.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.237+0000 7fd0513d6700 1 -- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd01c0557c0 msgr2=0x7fd01c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.237+0000 7fd0513d6700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd01c0557c0 0x7fd01c057c80 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fd03c001e90 tx=0x7fd03c001610).stop 2020-11-20T12:21:07.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.237+0000 7fd0513d6700 1 -- 172.21.15.116:0/3124527615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd04c14e600 msgr2=0x7fd04c1459d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.237+0000 7fd0513d6700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd04c14e600 0x7fd04c1459d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7fd04c153830 tx=0x7fd0440091d0).stop 2020-11-20T12:21:07.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.237+0000 7fd0513d6700 1 -- 172.21.15.116:0/3124527615 shutdown_connections 2020-11-20T12:21:07.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.237+0000 7fd0513d6700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd01c0557c0 0x7fd01c057c80 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.237+0000 7fd0513d6700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd04c14e600 0x7fd04c1459d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.237+0000 7fd0513d6700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd04c146b20 0x7fd04c146f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.237+0000 7fd0513d6700 1 --2- 172.21.15.116:0/3124527615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd04c14e180 0x7fd04c1454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.237+0000 7fd0513d6700 1 -- 172.21.15.116:0/3124527615 >> 172.21.15.116:0/3124527615 conn(0x7fd04c060c40 msgr2=0x7fd04c067da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:07.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.241+0000 7fd0513d6700 1 -- 172.21.15.116:0/3124527615 shutdown_connections 2020-11-20T12:21:07.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.241+0000 7fd0513d6700 1 -- 172.21.15.116:0/3124527615 wait complete. 2020-11-20T12:21:07.269 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T12:21:07.271 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:21:07.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.445+0000 7f57d6eea700 1 Processor -- start 2020-11-20T12:21:07.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.449+0000 7f57d6eea700 1 -- start start 2020-11-20T12:21:07.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.449+0000 7f57d6eea700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d0151d50 0x7f57d00c4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.449+0000 7f57d6eea700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d00b9e80 con 0x7f57d0151970 2020-11-20T12:21:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.449+0000 7f57d6eea700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57d00c4b10 con 0x7f57d00ba820 2020-11-20T12:21:07.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.449+0000 7f57d6eea700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d00615b0 con 0x7f57d0151d50 2020-11-20T12:21:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.449+0000 7f57d4c86700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f57d0151970 0x7f57d00c2ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59878/0 (socket says 172.21.15.116:59878) 2020-11-20T12:21:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.449+0000 7f57d4c86700 1 -- 172.21.15.116:0/1762384531 learned_addr learned my addr 172.21.15.116:0/1762384531 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:07.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.449+0000 7f57cffff700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d0151d50 0x7f57d00c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.449+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3746587540 0 0) 0x7f57d00b9e80 con 0x7f57d0151970 2020-11-20T12:21:07.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57bc003240 con 0x7f57d0151970 2020-11-20T12:21:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 351520147 0 0) 0x7f57bc003240 con 0x7f57d0151970 2020-11-20T12:21:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d0151d50 msgr2=0x7f57d00c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d0151d50 0x7f57d00c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 >> v1:172.21.15.173:6789/0 conn(0x7f57d00ba820 legacy=0x7f57d00bacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d0061850 con 0x7f57d0151970 2020-11-20T12:21:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f57c40014e0 con 0x7f57d0151970 2020-11-20T12:21:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 >> v1:172.21.15.116:6789/0 conn(0x7f57d0151970 legacy=0x7f57d00c2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d0151d50 0x7f57bc003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57bc003c70 0x7f57bc0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57bc0049d0 0x7f57bc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d00615b0 con 0x7f57bc003c70 2020-11-20T12:21:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc004260 con 0x7f57bc0049d0 2020-11-20T12:21:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57bc00b730 con 0x7f57d0151d50 2020-11-20T12:21:07.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cffff700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57bc003c70 0x7f57bc0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57d4c86700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d0151d50 0x7f57bc003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57d5487700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57bc0049d0 0x7f57bc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cffff700 1 -- 172.21.15.116:0/1762384531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d0151d50 msgr2=0x7f57bc003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:07.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cffff700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d0151d50 0x7f57bc003760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cffff700 1 -- 172.21.15.116:0/1762384531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57bc0049d0 msgr2=0x7f57bc004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cffff700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57bc0049d0 0x7f57bc004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cffff700 1 -- 172.21.15.116:0/1762384531 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc003240 con 0x7f57bc003c70 2020-11-20T12:21:07.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cffff700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57bc003c70 0x7f57bc0044e0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f57c000bc00 tx=0x7f57c0006df0).ready entity=mon.0 client_cookie=2e2783310f8eb1ce server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c000b310 con 0x7f57bc003c70 2020-11-20T12:21:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c000ea30 con 0x7f57bc003c70 2020-11-20T12:21:07.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.453+0000 7f57cf7fe700 1 -- 172.21.15.116:0/1762384531 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c000edb0 con 0x7f57bc003c70 2020-11-20T12:21:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.457+0000 7f57d6eea700 1 -- 172.21.15.116:0/1762384531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57bc003c70 msgr2=0x7f57bc0044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.457+0000 7f57d6eea700 1 --2- 172.21.15.116:0/1762384531 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57bc003c70 0x7f57bc0044e0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7f57c000bc00 tx=0x7f57c0006df0).stop 2020-11-20T12:21:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.457+0000 7f57d5487700 1 -- 172.21.15.116:0/1762384531 reap_dead start 2020-11-20T12:21:07.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.457+0000 7f57d6eea700 1 -- 172.21.15.116:0/1762384531 shutdown_connections 2020-11-20T12:21:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.457+0000 7f57d6eea700 1 -- 172.21.15.116:0/1762384531 >> 172.21.15.116:0/1762384531 conn(0x7f57d00c1800 msgr2=0x7f57d00c1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.457+0000 7f57d6eea700 1 -- 172.21.15.116:0/1762384531 shutdown_connections 2020-11-20T12:21:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.457+0000 7f57d6eea700 1 -- 172.21.15.116:0/1762384531 wait complete. 2020-11-20T12:21:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.457+0000 7f57d6eea700 1 Processor -- start 2020-11-20T12:21:07.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.457+0000 7f57d6eea700 1 -- start start 2020-11-20T12:21:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d6eea700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d01428a0 0x7f57d00a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d6eea700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57d0142c80 0x7f57d01522c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d6eea700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d01527b0 0x7f57d0164840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d6eea700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d0161580 con 0x7f57d01428a0 2020-11-20T12:21:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d6eea700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57d0161820 con 0x7f57d0142c80 2020-11-20T12:21:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d6eea700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57d0161ac0 con 0x7f57d01527b0 2020-11-20T12:21:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d4c86700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d01428a0 0x7f57d00a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d4c86700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d01428a0 0x7f57d00a72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50456/0 (socket says 172.21.15.116:50456) 2020-11-20T12:21:07.467 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d4c86700 1 -- 172.21.15.116:0/3960315453 learned_addr learned my addr 172.21.15.116:0/3960315453 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57cffff700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57d0142c80 0x7f57d01522c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.468 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d5487700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d01527b0 0x7f57d0164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d4c86700 1 -- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d01527b0 msgr2=0x7f57d0164840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d4c86700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d01527b0 0x7f57d0164840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d4c86700 1 -- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57d0142c80 msgr2=0x7f57d01522c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d4c86700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57d0142c80 0x7f57d01522c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d4c86700 1 -- 172.21.15.116:0/3960315453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d0161d60 con 0x7f57d01428a0 2020-11-20T12:21:07.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d4c86700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d01428a0 0x7f57d00a72c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f57c4004be0 tx=0x7f57d00619e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:07.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57ccff9700 1 -- 172.21.15.116:0/3960315453 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c400f8a0 con 0x7f57d01428a0 2020-11-20T12:21:07.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57ccff9700 1 -- 172.21.15.116:0/3960315453 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c400fa00 con 0x7f57d01428a0 2020-11-20T12:21:07.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d6eea700 1 -- 172.21.15.116:0/3960315453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57d0162000 con 0x7f57d01428a0 2020-11-20T12:21:07.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57ccff9700 1 -- 172.21.15.116:0/3960315453 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c400fda0 con 0x7f57d01428a0 2020-11-20T12:21:07.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.461+0000 7f57d6eea700 1 -- 172.21.15.116:0/3960315453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57d00bed70 con 0x7f57d01428a0 2020-11-20T12:21:07.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.465+0000 7f57d6eea700 1 -- 172.21.15.116:0/3960315453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57d0152b90 con 0x7f57d01428a0 2020-11-20T12:21:07.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.465+0000 7f57ccff9700 1 -- 172.21.15.116:0/3960315453 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f57c401b030 con 0x7f57d01428a0 2020-11-20T12:21:07.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.465+0000 7f57ccff9700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57a80557c0 0x7f57a8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.465+0000 7f57ccff9700 1 -- 172.21.15.116:0/3960315453 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f57c406db90 con 0x7f57d01428a0 2020-11-20T12:21:07.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.465+0000 7f57cffff700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57a80557c0 0x7f57a8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.465+0000 7f57cffff700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57a80557c0 0x7f57a8057c80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f57d005e250 tx=0x7f57c0019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:07.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.469+0000 7f57ccff9700 1 -- 172.21.15.116:0/3960315453 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f57c4043d00 con 0x7f57d01428a0 2020-11-20T12:21:07.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.629+0000 7f57d6eea700 1 -- 172.21.15.116:0/3960315453 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f57d0069950 con 0x7f57d01428a0 2020-11-20T12:21:07.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.629+0000 7f57ccff9700 1 -- 172.21.15.116:0/3960315453 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v100) v1 ==== 110+0+10 (secure 0 0 0) 0x7f57c4047320 con 0x7f57d01428a0 2020-11-20T12:21:07.632 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:21:07.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 -- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57a80557c0 msgr2=0x7f57a8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57a80557c0 0x7f57a8057c80 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f57d005e250 tx=0x7f57c0019000).stop 2020-11-20T12:21:07.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 -- 172.21.15.116:0/3960315453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d01428a0 msgr2=0x7f57d00a72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d01428a0 0x7f57d00a72c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7f57c4004be0 tx=0x7f57d00619e0).stop 2020-11-20T12:21:07.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 -- 172.21.15.116:0/3960315453 shutdown_connections 2020-11-20T12:21:07.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f57a80557c0 0x7f57a8057c80 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f57d01527b0 0x7f57d0164840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f57d01428a0 0x7f57d00a72c0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 --2- 172.21.15.116:0/3960315453 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f57d0142c80 0x7f57d01522c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 -- 172.21.15.116:0/3960315453 >> 172.21.15.116:0/3960315453 conn(0x7f57d00c1800 msgr2=0x7f57d00a9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:07.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 -- 172.21.15.116:0/3960315453 shutdown_connections 2020-11-20T12:21:07.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.633+0000 7f57d6eea700 1 -- 172.21.15.116:0/3960315453 wait complete. 2020-11-20T12:21:07.659 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.829+0000 7f8b88c9c700 1 Processor -- start 2020-11-20T12:21:07.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.829+0000 7f8b88c9c700 1 -- start start 2020-11-20T12:21:07.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b88c9c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8b84149df0 0x7f8b84153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b88c9c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b8414a1d0 0x7f8b840be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b88c9c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b840a9c20 con 0x7f8b840a9750 2020-11-20T12:21:07.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b88c9c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b840a9ec0 con 0x7f8b84149df0 2020-11-20T12:21:07.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b88c9c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b84153670 con 0x7f8b8414a1d0 2020-11-20T12:21:07.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b81d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8b840a9750 0x7f8b840c0700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59898/0 (socket says 172.21.15.116:59898) 2020-11-20T12:21:07.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b81d9b700 1 -- 172.21.15.116:0/1479479670 learned_addr learned my addr 172.21.15.116:0/1479479670 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:07.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b8259c700 1 --2- 172.21.15.116:0/1479479670 >> v2:172.21.15.116:3300/0 conn(0x7f8b84149df0 0x7f8b84153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b82d9d700 1 --2- 172.21.15.116:0/1479479670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b8414a1d0 0x7f8b840be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b8159a700 1 -- 172.21.15.116:0/1479479670 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853686319 0 0) 0x7f8b840a9c20 con 0x7f8b840a9750 2020-11-20T12:21:07.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b8159a700 1 -- 172.21.15.116:0/1479479670 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b68003240 con 0x7f8b840a9750 2020-11-20T12:21:07.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b8259c700 1 -- 172.21.15.116:0/1479479670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b8414a1d0 msgr2=0x7f8b840be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:07.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b8259c700 1 --2- 172.21.15.116:0/1479479670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b8414a1d0 0x7f8b840be240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b8259c700 1 -- 172.21.15.116:0/1479479670 >> v1:172.21.15.116:6789/0 conn(0x7f8b840a9750 legacy=0x7f8b840c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b8259c700 1 -- 172.21.15.116:0/1479479670 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b84153910 con 0x7f8b84149df0 2020-11-20T12:21:07.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.833+0000 7f8b8259c700 1 --2- 172.21.15.116:0/1479479670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b84149df0 0x7f8b84153db0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f8b7400b9a0 tx=0x7f8b74006df0).ready entity=mon.0 client_cookie=67fb1fca0fdd414b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:07.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b8159a700 1 -- 172.21.15.116:0/1479479670 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b740097b0 con 0x7f8b84149df0 2020-11-20T12:21:07.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b8159a700 1 -- 172.21.15.116:0/1479479670 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b74009910 con 0x7f8b84149df0 2020-11-20T12:21:07.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b8159a700 1 -- 172.21.15.116:0/1479479670 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74009c90 con 0x7f8b84149df0 2020-11-20T12:21:07.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b88c9c700 1 -- 172.21.15.116:0/1479479670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b84149df0 msgr2=0x7f8b84153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b88c9c700 1 --2- 172.21.15.116:0/1479479670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b84149df0 0x7f8b84153db0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f8b7400b9a0 tx=0x7f8b74006df0).stop 2020-11-20T12:21:07.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b88c9c700 1 -- 172.21.15.116:0/1479479670 shutdown_connections 2020-11-20T12:21:07.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b88c9c700 1 --2- 172.21.15.116:0/1479479670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b84149df0 0x7f8b84153db0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b88c9c700 1 --2- 172.21.15.116:0/1479479670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b8414a1d0 0x7f8b840be240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b88c9c700 1 -- 172.21.15.116:0/1479479670 >> 172.21.15.116:0/1479479670 conn(0x7f8b8405ba80 msgr2=0x7f8b840c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:07.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b88c9c700 1 -- 172.21.15.116:0/1479479670 shutdown_connections 2020-11-20T12:21:07.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b88c9c700 1 -- 172.21.15.116:0/1479479670 wait complete. 2020-11-20T12:21:07.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b88c9c700 1 Processor -- start 2020-11-20T12:21:07.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.837+0000 7f8b88c9c700 1 -- start start 2020-11-20T12:21:07.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b88c9c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b840a9750 0x7f8b84139850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b88c9c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b84149df0 0x7f8b841521a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b88c9c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b8414a1d0 0x7f8b84138a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b88c9c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8414b470 con 0x7f8b840a9750 2020-11-20T12:21:07.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b88c9c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8414b710 con 0x7f8b8414a1d0 2020-11-20T12:21:07.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b88c9c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8414b9b0 con 0x7f8b84149df0 2020-11-20T12:21:07.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b8259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b840a9750 0x7f8b84139850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b8259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b840a9750 0x7f8b84139850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50470/0 (socket says 172.21.15.116:50470) 2020-11-20T12:21:07.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b8259c700 1 -- 172.21.15.116:0/3048767270 learned_addr learned my addr 172.21.15.116:0/3048767270 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:07.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b81d9b700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b84149df0 0x7f8b841521a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b82d9d700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b8414a1d0 0x7f8b84138a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b8259c700 1 -- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b84149df0 msgr2=0x7f8b841521a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b8259c700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b84149df0 0x7f8b841521a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b8259c700 1 -- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b8414a1d0 msgr2=0x7f8b84138a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:07.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b8259c700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b8414a1d0 0x7f8b84138a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:07.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b8259c700 1 -- 172.21.15.116:0/3048767270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b8414bc50 con 0x7f8b840a9750 2020-11-20T12:21:07.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b82d9d700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b8414a1d0 0x7f8b84138a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:07.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b8259c700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b840a9750 0x7f8b84139850 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f8b74006fd0 tx=0x7f8b74015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:07.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b727fc700 1 -- 172.21.15.116:0/3048767270 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b74011dd0 con 0x7f8b840a9750 2020-11-20T12:21:07.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b727fc700 1 -- 172.21.15.116:0/3048767270 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7401e460 con 0x7f8b840a9750 2020-11-20T12:21:07.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b727fc700 1 -- 172.21.15.116:0/3048767270 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7401e7e0 con 0x7f8b840a9750 2020-11-20T12:21:07.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b88c9c700 1 -- 172.21.15.116:0/3048767270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8414bef0 con 0x7f8b840a9750 2020-11-20T12:21:07.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b88c9c700 1 -- 172.21.15.116:0/3048767270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b841528b0 con 0x7f8b840a9750 2020-11-20T12:21:07.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b88c9c700 1 -- 172.21.15.116:0/3048767270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8405d2a0 con 0x7f8b840a9750 2020-11-20T12:21:07.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b727fc700 1 -- 172.21.15.116:0/3048767270 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8b74025030 con 0x7f8b840a9750 2020-11-20T12:21:07.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b727fc700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b580557c0 0x7f8b58057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:07.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b727fc700 1 -- 172.21.15.116:0/3048767270 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f8b7406f9e0 con 0x7f8b840a9750 2020-11-20T12:21:07.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.841+0000 7f8b81d9b700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b580557c0 0x7f8b58057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:07.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.845+0000 7f8b81d9b700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b580557c0 0x7f8b58057c80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f8b840c19b0 tx=0x7f8b78009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:07.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:07.845+0000 7f8b727fc700 1 -- 172.21.15.116:0/3048767270 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8b74045c60 con 0x7f8b840a9750 2020-11-20T12:21:08.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b88c9c700 1 -- 172.21.15.116:0/3048767270 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8b84061380 con 0x7f8b840a9750 2020-11-20T12:21:08.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b727fc700 1 -- 172.21.15.116:0/3048767270 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v100) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8b74023030 con 0x7f8b840a9750 2020-11-20T12:21:08.012 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:08.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b88c9c700 1 -- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b580557c0 msgr2=0x7f8b58057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:08.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b88c9c700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b580557c0 0x7f8b58057c80 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f8b840c19b0 tx=0x7f8b78009040).stop 2020-11-20T12:21:08.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b88c9c700 1 -- 172.21.15.116:0/3048767270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b840a9750 msgr2=0x7f8b84139850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:08.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b88c9c700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b840a9750 0x7f8b84139850 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f8b74006fd0 tx=0x7f8b74015000).stop 2020-11-20T12:21:08.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b88c9c700 1 -- 172.21.15.116:0/3048767270 shutdown_connections 2020-11-20T12:21:08.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b88c9c700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b580557c0 0x7f8b58057c80 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:08.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b88c9c700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b8414a1d0 0x7f8b84138a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:08.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b88c9c700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b840a9750 0x7f8b84139850 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:08.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b88c9c700 1 --2- 172.21.15.116:0/3048767270 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b84149df0 0x7f8b841521a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:08.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.013+0000 7f8b88c9c700 1 -- 172.21.15.116:0/3048767270 >> 172.21.15.116:0/3048767270 conn(0x7f8b8405ba80 msgr2=0x7f8b84062880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:08.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.017+0000 7f8b88c9c700 1 -- 172.21.15.116:0/3048767270 shutdown_connections 2020-11-20T12:21:08.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.017+0000 7f8b88c9c700 1 -- 172.21.15.116:0/3048767270 wait complete. 2020-11-20T12:21:08.728 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-20T12:21:08.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.912+0000 7fb5f41c8700 1 Processor -- start 2020-11-20T12:21:08.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.912+0000 7fb5f41c8700 1 -- start start 2020-11-20T12:21:08.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.912+0000 7fb5f41c8700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb5ec0c93a0 0x7fb5ec0c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:08.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.912+0000 7fb5f41c8700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ec0c35e0 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.912+0000 7fb5f41c8700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ec0c3880 con 0x7fb5ec0a55f0 2020-11-20T12:21:08.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.912+0000 7fb5f41c8700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec0c7280 con 0x7fb5ec0c93a0 2020-11-20T12:21:08.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.912+0000 7fb5f1f64700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb5ec0c3e10 0x7fb5ec0c79c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59912/0 (socket says 172.21.15.116:59912) 2020-11-20T12:21:08.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.912+0000 7fb5f1f64700 1 -- 172.21.15.116:0/568108981 learned_addr learned my addr 172.21.15.116:0/568108981 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:08.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.912+0000 7fb5f1763700 1 --2- 172.21.15.116:0/568108981 >> v2:172.21.15.116:3300/0 conn(0x7fb5ec0c93a0 0x7fb5ec0c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:08.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f0f62700 1 -- 172.21.15.116:0/568108981 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064174911 0 0) 0x7fb5ec0c35e0 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f0f62700 1 -- 172.21.15.116:0/568108981 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d4003240 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f1763700 1 -- 172.21.15.116:0/568108981 >> v1:172.21.15.173:6789/0 conn(0x7fb5ec0a55f0 legacy=0x7fb5ec0c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:08.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f1763700 1 -- 172.21.15.116:0/568108981 >> v1:172.21.15.116:6789/0 conn(0x7fb5ec0c3e10 legacy=0x7fb5ec0c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:08.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f1763700 1 -- 172.21.15.116:0/568108981 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0c7520 con 0x7fb5ec0c93a0 2020-11-20T12:21:08.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f1763700 1 --2- 172.21.15.116:0/568108981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5ec0c93a0 0x7fb5ec0c41f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fb5dc00b9d0 tx=0x7fb5dc006df0).ready entity=mon.0 client_cookie=1cbadad66bff64a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:08.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f0f62700 1 -- 172.21.15.116:0/568108981 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc009770 con 0x7fb5ec0c93a0 2020-11-20T12:21:08.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f0f62700 1 -- 172.21.15.116:0/568108981 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc0098d0 con 0x7fb5ec0c93a0 2020-11-20T12:21:08.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f0f62700 1 -- 172.21.15.116:0/568108981 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc009c50 con 0x7fb5ec0c93a0 2020-11-20T12:21:08.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f41c8700 1 -- 172.21.15.116:0/568108981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5ec0c93a0 msgr2=0x7fb5ec0c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:08.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f41c8700 1 --2- 172.21.15.116:0/568108981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5ec0c93a0 0x7fb5ec0c41f0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7fb5dc00b9d0 tx=0x7fb5dc006df0).stop 2020-11-20T12:21:08.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f41c8700 1 -- 172.21.15.116:0/568108981 shutdown_connections 2020-11-20T12:21:08.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f41c8700 1 --2- 172.21.15.116:0/568108981 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5ec0c93a0 0x7fb5ec0c41f0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:08.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f41c8700 1 -- 172.21.15.116:0/568108981 >> 172.21.15.116:0/568108981 conn(0x7fb5ec068080 msgr2=0x7fb5ec0c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:08.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f41c8700 1 -- 172.21.15.116:0/568108981 shutdown_connections 2020-11-20T12:21:08.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.916+0000 7fb5f41c8700 1 -- 172.21.15.116:0/568108981 wait complete. 2020-11-20T12:21:08.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f41c8700 1 Processor -- start 2020-11-20T12:21:08.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f41c8700 1 -- start start 2020-11-20T12:21:08.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f41c8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5ec0a55f0 0x7fb5ec0c1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:08.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f41c8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5ec0c3e10 0x7fb5ec061f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f41c8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5ec062420 0x7fb5ec143040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f41c8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0c0320 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f41c8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0c0580 con 0x7fb5ec062420 2020-11-20T12:21:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f41c8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0c0820 con 0x7fb5ec0a55f0 2020-11-20T12:21:08.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f1763700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5ec0c3e10 0x7fb5ec061f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:08.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f1763700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5ec0c3e10 0x7fb5ec061f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50484/0 (socket says 172.21.15.116:50484) 2020-11-20T12:21:08.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f1763700 1 -- 172.21.15.116:0/255950778 learned_addr learned my addr 172.21.15.116:0/255950778 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:08.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f1f64700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5ec0a55f0 0x7fb5ec0c1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:08.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f2765700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5ec062420 0x7fb5ec143040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:08.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f1763700 1 -- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5ec0a55f0 msgr2=0x7fb5ec0c1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:08.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f1763700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5ec0a55f0 0x7fb5ec0c1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:08.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f1763700 1 -- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5ec062420 msgr2=0x7fb5ec143040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:08.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f1763700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5ec062420 0x7fb5ec143040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:08.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f1763700 1 -- 172.21.15.116:0/255950778 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0c0ac0 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.920+0000 7fb5f2765700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5ec062420 0x7fb5ec143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:08.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.924+0000 7fb5f1763700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5ec0c3e10 0x7fb5ec061f30 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fb5dc010000 tx=0x7fb5dc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:08.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.924+0000 7fb5d9ffb700 1 -- 172.21.15.116:0/255950778 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc011d80 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.924+0000 7fb5d9ffb700 1 -- 172.21.15.116:0/255950778 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc01e460 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.924+0000 7fb5f41c8700 1 -- 172.21.15.116:0/255950778 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec0c0d60 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.924+0000 7fb5d9ffb700 1 -- 172.21.15.116:0/255950778 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc01e7e0 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.924+0000 7fb5f41c8700 1 -- 172.21.15.116:0/255950778 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec143530 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.924+0000 7fb5f41c8700 1 -- 172.21.15.116:0/255950778 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ec0c3a60 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.924+0000 7fb5d9ffb700 1 -- 172.21.15.116:0/255950778 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb5dc01a040 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.928+0000 7fb5d9ffb700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb5c40557c0 0x7fb5c4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:08.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.928+0000 7fb5d9ffb700 1 -- 172.21.15.116:0/255950778 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fb5dc06f830 con 0x7fb5ec0c3e10 2020-11-20T12:21:08.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.928+0000 7fb5f1f64700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb5c40557c0 0x7fb5c4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:08.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.928+0000 7fb5f1f64700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb5c40557c0 0x7fb5c4057c80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fb5e8001320 tx=0x7fb5e8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:08.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:08.932+0000 7fb5d9ffb700 1 -- 172.21.15.116:0/255950778 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb5dc045a00 con 0x7fb5ec0c3e10 2020-11-20T12:21:09.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.088+0000 7fb5f41c8700 1 -- 172.21.15.116:0/255950778 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb5ec05d2a0 con 0x7fb5ec0c3e10 2020-11-20T12:21:09.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5d9ffb700 1 -- 172.21.15.116:0/255950778 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v100) v1 ==== 84+0+143 (secure 0 0 0) 0x7fb5dc023030 con 0x7fb5ec0c3e10 2020-11-20T12:21:09.091 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:09.091 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2020-11-20T12:21:09.091 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:09.092 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2020-11-20T12:21:09.092 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "device_health_metrics" 2020-11-20T12:21:09.092 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:09.092 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:09.092 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 2, 2020-11-20T12:21:09.093 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "rbd" 2020-11-20T12:21:09.093 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:09.093 INFO:teuthology.orchestra.run.smithi116.stdout:] 2020-11-20T12:21:09.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 -- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb5c40557c0 msgr2=0x7fb5c4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb5c40557c0 0x7fb5c4057c80 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7fb5e8001320 tx=0x7fb5e8009040).stop 2020-11-20T12:21:09.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 -- 172.21.15.116:0/255950778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5ec0c3e10 msgr2=0x7fb5ec061f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5ec0c3e10 0x7fb5ec061f30 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7fb5dc010000 tx=0x7fb5dc015000).stop 2020-11-20T12:21:09.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 -- 172.21.15.116:0/255950778 shutdown_connections 2020-11-20T12:21:09.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb5c40557c0 0x7fb5c4057c80 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb5ec062420 0x7fb5ec143040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb5ec0a55f0 0x7fb5ec0c1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 --2- 172.21.15.116:0/255950778 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb5ec0c3e10 0x7fb5ec061f30 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 -- 172.21.15.116:0/255950778 >> 172.21.15.116:0/255950778 conn(0x7fb5ec068080 msgr2=0x7fb5ec0a5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:09.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 -- 172.21.15.116:0/255950778 shutdown_connections 2020-11-20T12:21:09.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.092+0000 7fb5f41c8700 1 -- 172.21.15.116:0/255950778 wait complete. 2020-11-20T12:21:09.116 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T12:21:09.117 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-20T12:21:09.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.300+0000 7fa72759e700 1 Processor -- start 2020-11-20T12:21:09.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.300+0000 7fa72759e700 1 -- start start 2020-11-20T12:21:09.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.304+0000 7fa72759e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa72814e6d0 0x7fa728061280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.304+0000 7fa72759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa728062db0 0x7fa728063240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.304+0000 7fa72759e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7280c14f0 con 0x7fa7280ba680 2020-11-20T12:21:09.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.304+0000 7fa72759e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72814eab0 con 0x7fa72814e6d0 2020-11-20T12:21:09.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.304+0000 7fa72759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa728061770 con 0x7fa728062db0 2020-11-20T12:21:09.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.304+0000 7fa72659c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa72814e6d0 0x7fa728061280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.304+0000 7fa72659c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa72814e6d0 0x7fa728061280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:49544/0 (socket says 172.21.15.116:49544) 2020-11-20T12:21:09.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.304+0000 7fa72659c700 1 -- 172.21.15.116:0/2975813448 learned_addr learned my addr 172.21.15.116:0/2975813448 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:09.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.304+0000 7fa725d9b700 1 --1- 172.21.15.116:0/2975813448 >> v1:172.21.15.173:6789/0 conn(0x7fa7280ba680 0x7fa7280bb090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:56160/0 (socket says 172.21.15.116:56160) 2020-11-20T12:21:09.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.304+0000 7fa726d9d700 1 --2- 172.21.15.116:0/2975813448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa728062db0 0x7fa728063240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72659c700 1 -- 172.21.15.116:0/2975813448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa728062db0 msgr2=0x7fa728063240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:09.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72659c700 1 --2- 172.21.15.116:0/2975813448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa728062db0 0x7fa728063240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72659c700 1 -- 172.21.15.116:0/2975813448 >> v1:172.21.15.173:6789/0 conn(0x7fa7280ba680 legacy=0x7fa7280bb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72659c700 1 -- 172.21.15.116:0/2975813448 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7280c1250 con 0x7fa72814e6d0 2020-11-20T12:21:09.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72559a700 1 -- 172.21.15.116:0/2975813448 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2237910310 0 0) 0x7fa7280c14f0 con 0x7fa7280ba680 2020-11-20T12:21:09.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72659c700 1 --2- 172.21.15.116:0/2975813448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa72814e6d0 0x7fa728061280 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fa718006fd0 tx=0x7fa718006eb0).ready entity=mon.1 client_cookie=d3f4b8297efde266 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:09.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72559a700 1 -- 172.21.15.116:0/2975813448 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718009810 con 0x7fa72814e6d0 2020-11-20T12:21:09.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72559a700 1 -- 172.21.15.116:0/2975813448 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa718009970 con 0x7fa72814e6d0 2020-11-20T12:21:09.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72559a700 1 -- 172.21.15.116:0/2975813448 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718009cf0 con 0x7fa72814e6d0 2020-11-20T12:21:09.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72759e700 1 -- 172.21.15.116:0/2975813448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa72814e6d0 msgr2=0x7fa728061280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72759e700 1 --2- 172.21.15.116:0/2975813448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa72814e6d0 0x7fa728061280 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fa718006fd0 tx=0x7fa718006eb0).stop 2020-11-20T12:21:09.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72759e700 1 -- 172.21.15.116:0/2975813448 shutdown_connections 2020-11-20T12:21:09.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72759e700 1 --2- 172.21.15.116:0/2975813448 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa728062db0 0x7fa728063240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72759e700 1 --2- 172.21.15.116:0/2975813448 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa72814e6d0 0x7fa728061280 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72759e700 1 -- 172.21.15.116:0/2975813448 >> 172.21.15.116:0/2975813448 conn(0x7fa7280c5700 msgr2=0x7fa7280c16d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:09.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72759e700 1 -- 172.21.15.116:0/2975813448 shutdown_connections 2020-11-20T12:21:09.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.308+0000 7fa72759e700 1 -- 172.21.15.116:0/2975813448 wait complete. 2020-11-20T12:21:09.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.312+0000 7fa72759e700 1 Processor -- start 2020-11-20T12:21:09.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.312+0000 7fa72759e700 1 -- start start 2020-11-20T12:21:09.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa72759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa728062db0 0x7fa72813ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa72759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7280ba680 0x7fa72813c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa72759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa72814e6d0 0x7fa7280c03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa72759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7280a94d0 con 0x7fa7280ba680 2020-11-20T12:21:09.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa72759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7280bcd90 con 0x7fa728062db0 2020-11-20T12:21:09.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa72759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7280bd030 con 0x7fa72814e6d0 2020-11-20T12:21:09.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa725d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7280ba680 0x7fa72813c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa725d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7280ba680 0x7fa72813c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50500/0 (socket says 172.21.15.116:50500) 2020-11-20T12:21:09.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa725d9b700 1 -- 172.21.15.116:0/1363052590 learned_addr learned my addr 172.21.15.116:0/1363052590 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:09.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa72659c700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa728062db0 0x7fa72813ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa726d9d700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa72814e6d0 0x7fa7280c03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa725d9b700 1 -- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa72814e6d0 msgr2=0x7fa7280c03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa725d9b700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa72814e6d0 0x7fa7280c03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa725d9b700 1 -- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa728062db0 msgr2=0x7fa72813ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa725d9b700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa728062db0 0x7fa72813ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa725d9b700 1 -- 172.21.15.116:0/1363052590 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7280bd2d0 con 0x7fa7280ba680 2020-11-20T12:21:09.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa725d9b700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7280ba680 0x7fa72813c140 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fa71c00c710 tx=0x7fa71c007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:09.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa7167fc700 1 -- 172.21.15.116:0/1363052590 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0072e0 con 0x7fa7280ba680 2020-11-20T12:21:09.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa7167fc700 1 -- 172.21.15.116:0/1363052590 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa71c005530 con 0x7fa7280ba680 2020-11-20T12:21:09.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa72759e700 1 -- 172.21.15.116:0/1363052590 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7280bd570 con 0x7fa7280ba680 2020-11-20T12:21:09.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa7167fc700 1 -- 172.21.15.116:0/1363052590 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71c0058b0 con 0x7fa7280ba680 2020-11-20T12:21:09.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.316+0000 7fa72759e700 1 -- 172.21.15.116:0/1363052590 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72813c630 con 0x7fa7280ba680 2020-11-20T12:21:09.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.320+0000 7fa72759e700 1 -- 172.21.15.116:0/1363052590 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7280bace0 con 0x7fa7280ba680 2020-11-20T12:21:09.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.320+0000 7fa7167fc700 1 -- 172.21.15.116:0/1363052590 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa71c010070 con 0x7fa7280ba680 2020-11-20T12:21:09.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.324+0000 7fa7167fc700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6fc0557f0 0x7fa6fc057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.324+0000 7fa7167fc700 1 -- 172.21.15.116:0/1363052590 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa71c06d9e0 con 0x7fa7280ba680 2020-11-20T12:21:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.324+0000 7fa72659c700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6fc0557f0 0x7fa6fc057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.324+0000 7fa72659c700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6fc0557f0 0x7fa6fc057cb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fa71800beb0 tx=0x7fa718015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:09.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.328+0000 7fa7167fc700 1 -- 172.21.15.116:0/1363052590 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa7280bace0 con 0x7fa7280ba680 2020-11-20T12:21:09.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.492+0000 7fa72759e700 1 -- 172.21.15.116:0/1363052590 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fa7280bace0 con 0x7fa7280ba680 2020-11-20T12:21:09.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.576+0000 7fa7167fc700 1 -- 172.21.15.116:0/1363052590 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v101) v1 ==== 133+0+0 (secure 0 0 0) 0x7fa7280bace0 con 0x7fa7280ba680 2020-11-20T12:21:09.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 -- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6fc0557f0 msgr2=0x7fa6fc057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6fc0557f0 0x7fa6fc057cb0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fa71800beb0 tx=0x7fa718015000).stop 2020-11-20T12:21:09.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 -- 172.21.15.116:0/1363052590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7280ba680 msgr2=0x7fa72813c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7280ba680 0x7fa72813c140 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fa71c00c710 tx=0x7fa71c007ae0).stop 2020-11-20T12:21:09.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 -- 172.21.15.116:0/1363052590 shutdown_connections 2020-11-20T12:21:09.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa6fc0557f0 0x7fa6fc057cb0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa72814e6d0 0x7fa7280c03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa728062db0 0x7fa72813ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 --2- 172.21.15.116:0/1363052590 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa7280ba680 0x7fa72813c140 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 -- 172.21.15.116:0/1363052590 >> 172.21.15.116:0/1363052590 conn(0x7fa7280c5700 msgr2=0x7fa728060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:09.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 -- 172.21.15.116:0/1363052590 shutdown_connections 2020-11-20T12:21:09.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.580+0000 7fa72759e700 1 -- 172.21.15.116:0/1363052590 wait complete. 2020-11-20T12:21:09.583 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2020-11-20T12:21:09.604 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-20T12:21:09.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.780+0000 7f7b443d7700 1 Processor -- start 2020-11-20T12:21:09.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.780+0000 7f7b443d7700 1 -- start start 2020-11-20T12:21:09.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.780+0000 7f7b443d7700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f7b3c14f030 0x7f7b3c14f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.780+0000 7f7b443d7700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3c0a8f60 con 0x7f7b3c0a8ab0 2020-11-20T12:21:09.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.780+0000 7f7b443d7700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b3c152710 con 0x7f7b3c148cf0 2020-11-20T12:21:09.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.780+0000 7f7b443d7700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b3c1529b0 con 0x7f7b3c14f030 2020-11-20T12:21:09.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.780+0000 7f7b41972700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7b3c0a8ab0 0x7f7b3c0a66e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59940/0 (socket says 172.21.15.116:59940) 2020-11-20T12:21:09.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.780+0000 7f7b41972700 1 -- 172.21.15.116:0/2895646115 learned_addr learned my addr 172.21.15.116:0/2895646115 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:09.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.780+0000 7f7b42974700 1 --2- 172.21.15.116:0/2895646115 >> v2:172.21.15.116:3300/0 conn(0x7f7b3c14f030 0x7f7b3c14f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b41171700 1 -- 172.21.15.116:0/2895646115 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496908102 0 0) 0x7f7b3c0a8f60 con 0x7f7b3c0a8ab0 2020-11-20T12:21:09.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b41171700 1 -- 172.21.15.116:0/2895646115 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b24003240 con 0x7f7b3c0a8ab0 2020-11-20T12:21:09.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b42974700 1 -- 172.21.15.116:0/2895646115 >> v1:172.21.15.173:6789/0 conn(0x7f7b3c148cf0 legacy=0x7f7b3c0c4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b42974700 1 -- 172.21.15.116:0/2895646115 >> v1:172.21.15.116:6789/0 conn(0x7f7b3c0a8ab0 legacy=0x7f7b3c0a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b41171700 1 -- 172.21.15.116:0/2895646115 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020521322 0 0) 0x7f7b3c152710 con 0x7f7b3c148cf0 2020-11-20T12:21:09.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b42974700 1 -- 172.21.15.116:0/2895646115 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c152c50 con 0x7f7b3c14f030 2020-11-20T12:21:09.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b42974700 1 --2- 172.21.15.116:0/2895646115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b3c14f030 0x7f7b3c14f430 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f7b34009c90 tx=0x7f7b340091d0).ready entity=mon.0 client_cookie=b7a6d1c4f3933b84 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:09.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b41171700 1 -- 172.21.15.116:0/2895646115 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c0014e0 con 0x7f7b3c14f030 2020-11-20T12:21:09.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b41171700 1 -- 172.21.15.116:0/2895646115 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b3400b760 con 0x7f7b3c14f030 2020-11-20T12:21:09.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b41171700 1 -- 172.21.15.116:0/2895646115 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b3400bae0 con 0x7f7b3c14f030 2020-11-20T12:21:09.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b443d7700 1 -- 172.21.15.116:0/2895646115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b3c14f030 msgr2=0x7f7b3c14f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b443d7700 1 --2- 172.21.15.116:0/2895646115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b3c14f030 0x7f7b3c14f430 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f7b34009c90 tx=0x7f7b340091d0).stop 2020-11-20T12:21:09.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b443d7700 1 -- 172.21.15.116:0/2895646115 shutdown_connections 2020-11-20T12:21:09.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b443d7700 1 --2- 172.21.15.116:0/2895646115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b3c14f030 0x7f7b3c14f430 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b443d7700 1 -- 172.21.15.116:0/2895646115 >> 172.21.15.116:0/2895646115 conn(0x7f7b3c05ba80 msgr2=0x7f7b3c0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:09.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b443d7700 1 -- 172.21.15.116:0/2895646115 shutdown_connections 2020-11-20T12:21:09.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.784+0000 7f7b443d7700 1 -- 172.21.15.116:0/2895646115 wait complete. 2020-11-20T12:21:09.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b443d7700 1 Processor -- start 2020-11-20T12:21:09.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b443d7700 1 -- start start 2020-11-20T12:21:09.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b443d7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b3c0a8ab0 0x7f7b3c144080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b443d7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b3c148cf0 0x7f7b3c165030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b443d7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b3c14f030 0x7f7b3c0bd810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b443d7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c144570 con 0x7f7b3c148cf0 2020-11-20T12:21:09.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b443d7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c0a6e90 con 0x7f7b3c0a8ab0 2020-11-20T12:21:09.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b443d7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c1613a0 con 0x7f7b3c14f030 2020-11-20T12:21:09.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b41972700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b3c148cf0 0x7f7b3c165030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b41972700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b3c148cf0 0x7f7b3c165030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50514/0 (socket says 172.21.15.116:50514) 2020-11-20T12:21:09.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b41972700 1 -- 172.21.15.116:0/1898715034 learned_addr learned my addr 172.21.15.116:0/1898715034 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:09.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b42173700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b3c0a8ab0 0x7f7b3c144080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b42974700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b3c14f030 0x7f7b3c0bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b41972700 1 -- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b3c14f030 msgr2=0x7f7b3c0bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b41972700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b3c14f030 0x7f7b3c0bd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b41972700 1 -- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b3c0a8ab0 msgr2=0x7f7b3c144080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:09.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b41972700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b3c0a8ab0 0x7f7b3c144080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:09.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b41972700 1 -- 172.21.15.116:0/1898715034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c161640 con 0x7f7b3c148cf0 2020-11-20T12:21:09.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b42173700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b3c0a8ab0 0x7f7b3c144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:09.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.788+0000 7f7b41972700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b3c148cf0 0x7f7b3c165030 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f7b2c00cb40 tx=0x7f7b2c001c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:09.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.792+0000 7f7b2a7fc700 1 -- 172.21.15.116:0/1898715034 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c007170 con 0x7f7b3c148cf0 2020-11-20T12:21:09.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.792+0000 7f7b2a7fc700 1 -- 172.21.15.116:0/1898715034 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b2c0072d0 con 0x7f7b3c148cf0 2020-11-20T12:21:09.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.792+0000 7f7b443d7700 1 -- 172.21.15.116:0/1898715034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3c1618e0 con 0x7f7b3c148cf0 2020-11-20T12:21:09.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.792+0000 7f7b2a7fc700 1 -- 172.21.15.116:0/1898715034 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b2c00ac20 con 0x7f7b3c148cf0 2020-11-20T12:21:09.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.792+0000 7f7b443d7700 1 -- 172.21.15.116:0/1898715034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3c161050 con 0x7f7b3c148cf0 2020-11-20T12:21:09.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.792+0000 7f7b443d7700 1 -- 172.21.15.116:0/1898715034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3c05d2a0 con 0x7f7b3c148cf0 2020-11-20T12:21:09.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.792+0000 7f7b2a7fc700 1 -- 172.21.15.116:0/1898715034 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7b2c010070 con 0x7f7b3c148cf0 2020-11-20T12:21:09.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.796+0000 7f7b2a7fc700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7b140557f0 0x7f7b14057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:09.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.796+0000 7f7b2a7fc700 1 -- 172.21.15.116:0/1898715034 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f7b2c06e5c0 con 0x7f7b3c148cf0 2020-11-20T12:21:09.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.796+0000 7f7b42173700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7b140557f0 0x7f7b14057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:09.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.796+0000 7f7b42173700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7b140557f0 0x7f7b14057cb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f7b380011a0 tx=0x7f7b38009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:09.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.800+0000 7f7b2a7fc700 1 -- 172.21.15.116:0/1898715034 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7b2c044630 con 0x7f7b3c148cf0 2020-11-20T12:21:09.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:09.956+0000 7f7b443d7700 1 -- 172.21.15.116:0/1898715034 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f7b3c061380 con 0x7f7b3c148cf0 2020-11-20T12:21:10.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.656+0000 7f7b2a7fc700 1 -- 172.21.15.116:0/1898715034 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v105) v1 ==== 125+0+0 (secure 0 0 0) 0x7f7b2c047c50 con 0x7f7b3c148cf0 2020-11-20T12:21:10.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 -- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7b140557f0 msgr2=0x7f7b14057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7b140557f0 0x7f7b14057cb0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f7b380011a0 tx=0x7f7b38009040).stop 2020-11-20T12:21:10.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 -- 172.21.15.116:0/1898715034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b3c148cf0 msgr2=0x7f7b3c165030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b3c148cf0 0x7f7b3c165030 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f7b2c00cb40 tx=0x7f7b2c001c80).stop 2020-11-20T12:21:10.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 -- 172.21.15.116:0/1898715034 shutdown_connections 2020-11-20T12:21:10.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7b140557f0 0x7f7b14057cb0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7b3c14f030 0x7f7b3c0bd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7b3c0a8ab0 0x7f7b3c144080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 --2- 172.21.15.116:0/1898715034 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7b3c148cf0 0x7f7b3c165030 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 -- 172.21.15.116:0/1898715034 >> 172.21.15.116:0/1898715034 conn(0x7f7b3c05ba80 msgr2=0x7f7b3c14d880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:10.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 -- 172.21.15.116:0/1898715034 shutdown_connections 2020-11-20T12:21:10.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.664+0000 7f7b443d7700 1 -- 172.21.15.116:0/1898715034 wait complete. 2020-11-20T12:21:10.666 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2020-11-20T12:21:10.686 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T12:21:10.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.876+0000 7f01bbd78700 1 Processor -- start 2020-11-20T12:21:10.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.876+0000 7f01bbd78700 1 -- start start 2020-11-20T12:21:10.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01bbd78700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f01b4156190 0x7f01b40b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01bbd78700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01b40bab60 0x7f01b4060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01bbd78700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f01b40bafe0 0x7f01b4060660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01bbd78700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b40b9cc0 con 0x7f01b40bafe0 2020-11-20T12:21:10.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01bbd78700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b40b9f60 con 0x7f01b4156190 2020-11-20T12:21:10.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01bbd78700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40ba200 con 0x7f01b40bab60 2020-11-20T12:21:10.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01ba315700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f01b40bafe0 0x7f01b4060660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01ba315700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f01b40bafe0 0x7f01b4060660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50528/0 (socket says 172.21.15.116:50528) 2020-11-20T12:21:10.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01ba315700 1 -- 172.21.15.116:0/1335543017 learned_addr learned my addr 172.21.15.116:0/1335543017 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:10.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01b9313700 1 --2- 172.21.15.116:0/1335543017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01b40bab60 0x7f01b4060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01b9b14700 1 --2- 172.21.15.116:0/1335543017 >> v2:172.21.15.173:3300/0 conn(0x7f01b4156190 0x7f01b40b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01ba315700 1 -- 172.21.15.116:0/1335543017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01b40bab60 msgr2=0x7f01b4060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:10.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01ba315700 1 --2- 172.21.15.116:0/1335543017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01b40bab60 0x7f01b4060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01ba315700 1 -- 172.21.15.116:0/1335543017 >> v2:172.21.15.173:3300/0 conn(0x7f01b4156190 msgr2=0x7f01b40b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:10.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01ba315700 1 --2- 172.21.15.116:0/1335543017 >> v2:172.21.15.173:3300/0 conn(0x7f01b4156190 0x7f01b40b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.880+0000 7f01ba315700 1 -- 172.21.15.116:0/1335543017 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b40ba4a0 con 0x7f01b40bafe0 2020-11-20T12:21:10.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01ba315700 1 --2- 172.21.15.116:0/1335543017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01b40bafe0 0x7f01b4060660 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f01b40c1f80 tx=0x7f01ac009290).ready entity=mon.0 client_cookie=aaf8cb4ebbf97ef server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01b8b12700 1 -- 172.21.15.116:0/1335543017 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00bbc0 con 0x7f01b40bafe0 2020-11-20T12:21:10.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01b8b12700 1 -- 172.21.15.116:0/1335543017 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac00bd20 con 0x7f01b40bafe0 2020-11-20T12:21:10.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01b8b12700 1 -- 172.21.15.116:0/1335543017 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac011610 con 0x7f01b40bafe0 2020-11-20T12:21:10.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01bbd78700 1 -- 172.21.15.116:0/1335543017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01b40bafe0 msgr2=0x7f01b4060660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01bbd78700 1 --2- 172.21.15.116:0/1335543017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01b40bafe0 0x7f01b4060660 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f01b40c1f80 tx=0x7f01ac009290).stop 2020-11-20T12:21:10.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01bbd78700 1 -- 172.21.15.116:0/1335543017 shutdown_connections 2020-11-20T12:21:10.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01bbd78700 1 --2- 172.21.15.116:0/1335543017 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01b40bafe0 0x7f01b4060660 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01bbd78700 1 --2- 172.21.15.116:0/1335543017 >> v2:172.21.15.173:3300/0 conn(0x7f01b4156190 0x7f01b40b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01bbd78700 1 --2- 172.21.15.116:0/1335543017 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01b40bab60 0x7f01b4060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01bbd78700 1 -- 172.21.15.116:0/1335543017 >> 172.21.15.116:0/1335543017 conn(0x7f01b40682c0 msgr2=0x7f01b40c4470 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:10.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01bbd78700 1 -- 172.21.15.116:0/1335543017 shutdown_connections 2020-11-20T12:21:10.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.884+0000 7f01bbd78700 1 -- 172.21.15.116:0/1335543017 wait complete. 2020-11-20T12:21:10.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01bbd78700 1 Processor -- start 2020-11-20T12:21:10.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01bbd78700 1 -- start start 2020-11-20T12:21:10.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01bbd78700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b40bab60 0x7f01b40a9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01bbd78700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01b40bafe0 0x7f01b4157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01bbd78700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01b4156190 0x7f01b4153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01bbd78700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b415acf0 con 0x7f01b4156190 2020-11-20T12:21:10.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01bbd78700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b41581b0 con 0x7f01b40bab60 2020-11-20T12:21:10.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01bbd78700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b4157ca0 con 0x7f01b40bafe0 2020-11-20T12:21:10.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01ba315700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01b4156190 0x7f01b4153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01ba315700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01b4156190 0x7f01b4153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50534/0 (socket says 172.21.15.116:50534) 2020-11-20T12:21:10.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01ba315700 1 -- 172.21.15.116:0/3321446998 learned_addr learned my addr 172.21.15.116:0/3321446998 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:10.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01b9b14700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b40bab60 0x7f01b40a9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01b9313700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01b40bafe0 0x7f01b4157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01ba315700 1 -- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01b40bafe0 msgr2=0x7f01b4157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01ba315700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01b40bafe0 0x7f01b4157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01ba315700 1 -- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b40bab60 msgr2=0x7f01b40a9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:10.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01ba315700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b40bab60 0x7f01b40a9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:10.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.888+0000 7f01ba315700 1 -- 172.21.15.116:0/3321446998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4157f10 con 0x7f01b4156190 2020-11-20T12:21:10.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.892+0000 7f01ba315700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01b4156190 0x7f01b4153cf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f01ac0106d0 tx=0x7f01ac016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.892+0000 7f01a1ffb700 1 -- 172.21.15.116:0/3321446998 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00bbc0 con 0x7f01b4156190 2020-11-20T12:21:10.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.892+0000 7f01a1ffb700 1 -- 172.21.15.116:0/3321446998 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac00be90 con 0x7f01b4156190 2020-11-20T12:21:10.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.892+0000 7f01a1ffb700 1 -- 172.21.15.116:0/3321446998 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac010e40 con 0x7f01b4156190 2020-11-20T12:21:10.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.892+0000 7f01bbd78700 1 -- 172.21.15.116:0/3321446998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b415af40 con 0x7f01b4156190 2020-11-20T12:21:10.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.892+0000 7f01bbd78700 1 -- 172.21.15.116:0/3321446998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b40aa1f0 con 0x7f01b4156190 2020-11-20T12:21:10.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.892+0000 7f01bbd78700 1 -- 172.21.15.116:0/3321446998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b405d2a0 con 0x7f01b4156190 2020-11-20T12:21:10.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.892+0000 7f01a1ffb700 1 -- 172.21.15.116:0/3321446998 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f01ac017070 con 0x7f01b4156190 2020-11-20T12:21:10.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.896+0000 7f01a1ffb700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f018c0557f0 0x7f018c057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:10.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.896+0000 7f01a1ffb700 1 -- 172.21.15.116:0/3321446998 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f01ac0705b0 con 0x7f01b4156190 2020-11-20T12:21:10.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.896+0000 7f01b9b14700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f018c0557f0 0x7f018c057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:10.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.896+0000 7f01b9b14700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f018c0557f0 0x7f018c057cb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f01b000d990 tx=0x7f01b0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:10.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:10.900+0000 7f01a1ffb700 1 -- 172.21.15.116:0/3321446998 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f01ac046540 con 0x7f01b4156190 2020-11-20T12:21:11.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.104+0000 7f01bbd78700 1 -- 172.21.15.116:0/3321446998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f01b4062330 con 0x7f01b4156190 2020-11-20T12:21:11.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.120+0000 7f01a1ffb700 1 -- 172.21.15.116:0/3321446998 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 19 and data pool 20 v58) v1 ==== 189+0+0 (secure 0 0 0) 0x7f01ac015070 con 0x7f01b4156190 2020-11-20T12:21:11.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.124+0000 7f01bbd78700 1 -- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f018c0557f0 msgr2=0x7f018c057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.124+0000 7f01bbd78700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f018c0557f0 0x7f018c057cb0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f01b000d990 tx=0x7f01b0008040).stop 2020-11-20T12:21:11.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.124+0000 7f01bbd78700 1 -- 172.21.15.116:0/3321446998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01b4156190 msgr2=0x7f01b4153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.124+0000 7f01bbd78700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01b4156190 0x7f01b4153cf0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f01ac0106d0 tx=0x7f01ac016000).stop 2020-11-20T12:21:11.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.124+0000 7f01bbd78700 1 -- 172.21.15.116:0/3321446998 shutdown_connections 2020-11-20T12:21:11.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.124+0000 7f01bbd78700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f018c0557f0 0x7f018c057cb0 secure :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0x7f01b000d990 tx=0x7f01b0008040).stop 2020-11-20T12:21:11.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.124+0000 7f01bbd78700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01b4156190 0x7f01b4153cf0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.124+0000 7f01bbd78700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01b40bab60 0x7f01b40a9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.124+0000 7f01bbd78700 1 --2- 172.21.15.116:0/3321446998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01b40bafe0 0x7f01b4157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.124+0000 7f01bbd78700 1 -- 172.21.15.116:0/3321446998 >> 172.21.15.116:0/3321446998 conn(0x7f01b40682c0 msgr2=0x7f01b40c2620 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:11.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.128+0000 7f01bbd78700 1 -- 172.21.15.116:0/3321446998 shutdown_connections 2020-11-20T12:21:11.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.128+0000 7f01bbd78700 1 -- 172.21.15.116:0/3321446998 wait complete. 2020-11-20T12:21:11.133 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 19 and data pool 20 2020-11-20T12:21:11.154 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:11.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.344+0000 7f79c14db700 1 Processor -- start 2020-11-20T12:21:11.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.344+0000 7f79c14db700 1 -- start start 2020-11-20T12:21:11.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79c14db700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79bc0618e0 0x7f79bc061cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79c14db700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f79bc13b440 0x7f79bc0ade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79c14db700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79bc0aaa20 con 0x7f79bc0aa5c0 2020-11-20T12:21:11.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79c14db700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79bc0ae320 con 0x7f79bc13b440 2020-11-20T12:21:11.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79c14db700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0aac60 con 0x7f79bc0618e0 2020-11-20T12:21:11.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79b3fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f79bc0aa5c0 0x7f79bc13ad30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:59986/0 (socket says 172.21.15.116:59986) 2020-11-20T12:21:11.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79b3fff700 1 -- 172.21.15.116:0/2926378208 learned_addr learned my addr 172.21.15.116:0/2926378208 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:11.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79c0cda700 1 --2- 172.21.15.116:0/2926378208 >> v2:172.21.15.173:3300/0 conn(0x7f79bc13b440 0x7f79bc0ade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3107039562 0 0) 0x7f79a40010d0 con 0x7f79bc0aa5c0 2020-11-20T12:21:11.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79bbfff700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79bc0618e0 0x7f79bc061cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f799c003240 con 0x7f79bc0aa5c0 2020-11-20T12:21:11.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 33011920 0 0) 0x7f799c003240 con 0x7f79bc0aa5c0 2020-11-20T12:21:11.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79bc0618e0 msgr2=0x7f79bc061cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:11.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79bb7fe700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79bc0618e0 0x7f79bc061cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 >> v2:172.21.15.173:3300/0 conn(0x7f79bc13b440 msgr2=0x7f79bc0ade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:11.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79bb7fe700 1 --2- 172.21.15.116:0/2926378208 >> v2:172.21.15.173:3300/0 conn(0x7f79bc13b440 0x7f79bc0ade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc0aaf00 con 0x7f79bc0aa5c0 2020-11-20T12:21:11.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f79a4001670 con 0x7f79bc0aa5c0 2020-11-20T12:21:11.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.348+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 >> v1:172.21.15.116:6789/0 conn(0x7f79bc0aa5c0 legacy=0x7f79bc13ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79bb7fe700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f799c0037b0 0x7f799c003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79bb7fe700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f799c0044e0 0x7f799c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79bb7fe700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f799c004e50 0x7f799c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0ae320 con 0x7f799c004e50 2020-11-20T12:21:11.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc0aac60 con 0x7f799c0037b0 2020-11-20T12:21:11.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f799c005350 con 0x7f799c0044e0 2020-11-20T12:21:11.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79b3fff700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f799c0044e0 0x7f799c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79c0cda700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f799c004e50 0x7f799c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79bbfff700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f799c0037b0 0x7f799c003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79b3fff700 1 -- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f799c0037b0 msgr2=0x7f799c003bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:11.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79b3fff700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f799c0037b0 0x7f799c003bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79b3fff700 1 -- 172.21.15.116:0/2926378208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f799c004e50 msgr2=0x7f799c00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:11.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79b3fff700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f799c004e50 0x7f799c00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79b3fff700 1 -- 172.21.15.116:0/2926378208 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f799c003240 con 0x7f799c0044e0 2020-11-20T12:21:11.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79b3fff700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f799c0044e0 0x7f799c004960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f79a400c440 tx=0x7f79bc0ab090).ready entity=mon.2 client_cookie=a007d11fc24842f2 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:11.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a400ce40 con 0x7f799c0044e0 2020-11-20T12:21:11.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a40072e0 con 0x7f799c0044e0 2020-11-20T12:21:11.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79bb7fe700 1 -- 172.21.15.116:0/2926378208 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4005820 con 0x7f799c0044e0 2020-11-20T12:21:11.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79c14db700 1 -- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f799c0044e0 msgr2=0x7f799c004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79c14db700 1 --2- 172.21.15.116:0/2926378208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f799c0044e0 0x7f799c004960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f79a400c440 tx=0x7f79bc0ab090).stop 2020-11-20T12:21:11.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79c0cda700 1 -- 172.21.15.116:0/2926378208 reap_dead start 2020-11-20T12:21:11.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79c14db700 1 -- 172.21.15.116:0/2926378208 shutdown_connections 2020-11-20T12:21:11.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79c14db700 1 -- 172.21.15.116:0/2926378208 >> 172.21.15.116:0/2926378208 conn(0x7f79bc0678a0 msgr2=0x7f79bc067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:11.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79c14db700 1 -- 172.21.15.116:0/2926378208 shutdown_connections 2020-11-20T12:21:11.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.352+0000 7f79c14db700 1 -- 172.21.15.116:0/2926378208 wait complete. 2020-11-20T12:21:11.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.356+0000 7f79c14db700 1 Processor -- start 2020-11-20T12:21:11.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.356+0000 7f79c14db700 1 -- start start 2020-11-20T12:21:11.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79c14db700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79bc0a1700 0x7f79bc0a1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79c14db700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79bc0a3e70 0x7f79bc136b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79c14db700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79bc0a42f0 0x7f79bc137000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79c14db700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc09f600 con 0x7f79bc0a3e70 2020-11-20T12:21:11.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79c14db700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc09f8a0 con 0x7f79bc0a1700 2020-11-20T12:21:11.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79c14db700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79bc09fb40 con 0x7f79bc0a42f0 2020-11-20T12:21:11.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79bc0a3e70 0x7f79bc136b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79bc0a3e70 0x7f79bc136b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50564/0 (socket says 172.21.15.116:50564) 2020-11-20T12:21:11.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b3fff700 1 -- 172.21.15.116:0/401003987 learned_addr learned my addr 172.21.15.116:0/401003987 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:11.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79bbfff700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79bc0a1700 0x7f79bc0a1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79c0cda700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79bc0a42f0 0x7f79bc137000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b3fff700 1 -- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79bc0a42f0 msgr2=0x7f79bc137000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b3fff700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79bc0a42f0 0x7f79bc137000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b3fff700 1 -- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79bc0a1700 msgr2=0x7f79bc0a1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b3fff700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79bc0a1700 0x7f79bc0a1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b3fff700 1 -- 172.21.15.116:0/401003987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc09fde0 con 0x7f79bc0a3e70 2020-11-20T12:21:11.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79bbfff700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79bc0a1700 0x7f79bc0a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:11.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b3fff700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79bc0a3e70 0x7f79bc136b10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f79a400c410 tx=0x7f79a4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:11.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b8ff9700 1 -- 172.21.15.116:0/401003987 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a4001880 con 0x7f79bc0a3e70 2020-11-20T12:21:11.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b8ff9700 1 -- 172.21.15.116:0/401003987 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a4013450 con 0x7f79bc0a3e70 2020-11-20T12:21:11.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79c14db700 1 -- 172.21.15.116:0/401003987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79bc0a0080 con 0x7f79bc0a3e70 2020-11-20T12:21:11.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79b8ff9700 1 -- 172.21.15.116:0/401003987 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a40137d0 con 0x7f79bc0a3e70 2020-11-20T12:21:11.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79c14db700 1 -- 172.21.15.116:0/401003987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79bc0a0520 con 0x7f79bc0a3e70 2020-11-20T12:21:11.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.360+0000 7f79c14db700 1 -- 172.21.15.116:0/401003987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79bc05cf30 con 0x7f79bc0a3e70 2020-11-20T12:21:11.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.364+0000 7f79b8ff9700 1 -- 172.21.15.116:0/401003987 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f79a401d070 con 0x7f79bc0a3e70 2020-11-20T12:21:11.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.364+0000 7f79b8ff9700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79980557f0 0x7f7998057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.364+0000 7f79b8ff9700 1 -- 172.21.15.116:0/401003987 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f79a406f5d0 con 0x7f79bc0a3e70 2020-11-20T12:21:11.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.364+0000 7f79bbfff700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79980557f0 0x7f7998057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.368+0000 7f79bbfff700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79980557f0 0x7f7998057cb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f79bc081610 tx=0x7f79ac008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:11.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.372+0000 7f79b8ff9700 1 -- 172.21.15.116:0/401003987 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f79a40453f0 con 0x7f79bc0a3e70 2020-11-20T12:21:11.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.532+0000 7f79c14db700 1 -- 172.21.15.116:0/401003987 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79bc0aaa20 con 0x7f79bc0a3e70 2020-11-20T12:21:11.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.532+0000 7f79b8ff9700 1 -- 172.21.15.116:0/401003987 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f79a4031220 con 0x7f79bc0a3e70 2020-11-20T12:21:11.535 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:11.535 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":106,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:11.109209+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:09.571842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:10.034013+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:11.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 -- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79980557f0 msgr2=0x7f7998057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79980557f0 0x7f7998057cb0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f79bc081610 tx=0x7f79ac008040).stop 2020-11-20T12:21:11.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 -- 172.21.15.116:0/401003987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79bc0a3e70 msgr2=0x7f79bc136b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79bc0a3e70 0x7f79bc136b10 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f79a400c410 tx=0x7f79a4015000).stop 2020-11-20T12:21:11.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 -- 172.21.15.116:0/401003987 shutdown_connections 2020-11-20T12:21:11.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f79980557f0 0x7f7998057cb0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f79bc0a42f0 0x7f79bc137000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f79bc0a1700 0x7f79bc0a1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 --2- 172.21.15.116:0/401003987 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f79bc0a3e70 0x7f79bc136b10 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 -- 172.21.15.116:0/401003987 >> 172.21.15.116:0/401003987 conn(0x7f79bc0678a0 msgr2=0x7f79bc151390 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:11.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 -- 172.21.15.116:0/401003987 shutdown_connections 2020-11-20T12:21:11.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.536+0000 7f79c14db700 1 -- 172.21.15.116:0/401003987 wait complete. 2020-11-20T12:21:11.562 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:11.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.744+0000 7f3a8da07700 1 Processor -- start 2020-11-20T12:21:11.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.744+0000 7f3a8da07700 1 -- start start 2020-11-20T12:21:11.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.744+0000 7f3a8da07700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813afa0 0x7f3a880aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.744+0000 7f3a8da07700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3a8813b380 0x7f3a880aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.744+0000 7f3a8da07700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3a880acd30 0x7f3a880ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.744+0000 7f3a8da07700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a880ada00 con 0x7f3a880acd30 2020-11-20T12:21:11.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.744+0000 7f3a8da07700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a880adca0 con 0x7f3a8813b380 2020-11-20T12:21:11.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.744+0000 7f3a8da07700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a880adf40 con 0x7f3a8813afa0 2020-11-20T12:21:11.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8d206700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3a880acd30 0x7f3a880ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8d206700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3a880acd30 0x7f3a880ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50572/0 (socket says 172.21.15.116:50572) 2020-11-20T12:21:11.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8d206700 1 -- 172.21.15.116:0/3946271297 learned_addr learned my addr 172.21.15.116:0/3946271297 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:11.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a7ffff700 1 --2- 172.21.15.116:0/3946271297 >> v2:172.21.15.173:3300/0 conn(0x7f3a8813b380 0x7f3a880aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8ca05700 1 --2- 172.21.15.116:0/3946271297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813afa0 0x7f3a880aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8d206700 1 -- 172.21.15.116:0/3946271297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813afa0 msgr2=0x7f3a880aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:11.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8d206700 1 --2- 172.21.15.116:0/3946271297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813afa0 0x7f3a880aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8d206700 1 -- 172.21.15.116:0/3946271297 >> v2:172.21.15.173:3300/0 conn(0x7f3a8813b380 msgr2=0x7f3a880aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:11.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8d206700 1 --2- 172.21.15.116:0/3946271297 >> v2:172.21.15.173:3300/0 conn(0x7f3a8813b380 0x7f3a880aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8d206700 1 -- 172.21.15.116:0/3946271297 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a880ae1e0 con 0x7f3a880acd30 2020-11-20T12:21:11.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a7ffff700 1 --2- 172.21.15.116:0/3946271297 >> v2:172.21.15.173:3300/0 conn(0x7f3a8813b380 0x7f3a880aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:11.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8ca05700 1 --2- 172.21.15.116:0/3946271297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813afa0 0x7f3a880aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:11.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8d206700 1 --2- 172.21.15.116:0/3946271297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a880acd30 0x7f3a880ad1e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f3a88081a10 tx=0x7f3a80009290).ready entity=mon.0 client_cookie=ce9262109e8376f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:11.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a7f7fe700 1 -- 172.21.15.116:0/3946271297 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000bbc0 con 0x7f3a880acd30 2020-11-20T12:21:11.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a7f7fe700 1 -- 172.21.15.116:0/3946271297 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8000bd20 con 0x7f3a880acd30 2020-11-20T12:21:11.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a7f7fe700 1 -- 172.21.15.116:0/3946271297 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80011610 con 0x7f3a880acd30 2020-11-20T12:21:11.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8da07700 1 -- 172.21.15.116:0/3946271297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a880acd30 msgr2=0x7f3a880ad1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8da07700 1 --2- 172.21.15.116:0/3946271297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a880acd30 0x7f3a880ad1e0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f3a88081a10 tx=0x7f3a80009290).stop 2020-11-20T12:21:11.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8da07700 1 -- 172.21.15.116:0/3946271297 shutdown_connections 2020-11-20T12:21:11.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8da07700 1 --2- 172.21.15.116:0/3946271297 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a880acd30 0x7f3a880ad1e0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.759 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8da07700 1 --2- 172.21.15.116:0/3946271297 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813afa0 0x7f3a880aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8da07700 1 --2- 172.21.15.116:0/3946271297 >> v2:172.21.15.173:3300/0 conn(0x7f3a8813b380 0x7f3a880aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8da07700 1 -- 172.21.15.116:0/3946271297 >> 172.21.15.116:0/3946271297 conn(0x7f3a880678a0 msgr2=0x7f3a88067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:11.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8da07700 1 -- 172.21.15.116:0/3946271297 shutdown_connections 2020-11-20T12:21:11.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.748+0000 7f3a8da07700 1 -- 172.21.15.116:0/3946271297 wait complete. 2020-11-20T12:21:11.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8da07700 1 Processor -- start 2020-11-20T12:21:11.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8da07700 1 -- start start 2020-11-20T12:21:11.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8da07700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a880acd30 0x7f3a880a56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8da07700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8813afa0 0x7f3a88164760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8da07700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813b380 0x7f3a880a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8da07700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a88164c50 con 0x7f3a880acd30 2020-11-20T12:21:11.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8da07700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a88164ee0 con 0x7f3a8813afa0 2020-11-20T12:21:11.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8da07700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a880a5d50 con 0x7f3a8813b380 2020-11-20T12:21:11.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8ca05700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a880acd30 0x7f3a880a56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8d206700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813b380 0x7f3a880a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a7ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8813afa0 0x7f3a88164760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8ca05700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a880acd30 0x7f3a880a56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50580/0 (socket says 172.21.15.116:50580) 2020-11-20T12:21:11.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.752+0000 7f3a8ca05700 1 -- 172.21.15.116:0/3900421667 learned_addr learned my addr 172.21.15.116:0/3900421667 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:11.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a8d206700 1 -- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8813afa0 msgr2=0x7f3a88164760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a8d206700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8813afa0 0x7f3a88164760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a8d206700 1 -- 172.21.15.116:0/3900421667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a880acd30 msgr2=0x7f3a880a56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a8d206700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a880acd30 0x7f3a880a56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a8d206700 1 -- 172.21.15.116:0/3900421667 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a880a5ff0 con 0x7f3a8813b380 2020-11-20T12:21:11.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a7ffff700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8813afa0 0x7f3a88164760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:11.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a8ca05700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a880acd30 0x7f3a880a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:11.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a8d206700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813b380 0x7f3a880a69d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f3a88081a10 tx=0x7f3a80016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:11.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a7cff9700 1 -- 172.21.15.116:0/3900421667 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a8000bbc0 con 0x7f3a8813b380 2020-11-20T12:21:11.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a7cff9700 1 -- 172.21.15.116:0/3900421667 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a8000be90 con 0x7f3a8813b380 2020-11-20T12:21:11.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a8da07700 1 -- 172.21.15.116:0/3900421667 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a880a6290 con 0x7f3a8813b380 2020-11-20T12:21:11.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a7cff9700 1 -- 172.21.15.116:0/3900421667 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a80010cf0 con 0x7f3a8813b380 2020-11-20T12:21:11.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.756+0000 7f3a8da07700 1 -- 172.21.15.116:0/3900421667 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a880a6ec0 con 0x7f3a8813b380 2020-11-20T12:21:11.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.760+0000 7f3a8da07700 1 -- 172.21.15.116:0/3900421667 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8805cf30 con 0x7f3a8813b380 2020-11-20T12:21:11.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.760+0000 7f3a7cff9700 1 -- 172.21.15.116:0/3900421667 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3a80017070 con 0x7f3a8813b380 2020-11-20T12:21:11.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.760+0000 7f3a7cff9700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a600557c0 0x7f3a60057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:11.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.760+0000 7f3a7cff9700 1 -- 172.21.15.116:0/3900421667 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3a80070b00 con 0x7f3a8813b380 2020-11-20T12:21:11.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.760+0000 7f3a8ca05700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a600557c0 0x7f3a60057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:11.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.764+0000 7f3a8ca05700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a600557c0 0x7f3a60057c80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f3a8400af60 tx=0x7f3a84008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:11.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.768+0000 7f3a7cff9700 1 -- 172.21.15.116:0/3900421667 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3a80046910 con 0x7f3a8813b380 2020-11-20T12:21:11.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.932+0000 7f3a8da07700 1 -- 172.21.15.116:0/3900421667 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a88060170 con 0x7f3a8813b380 2020-11-20T12:21:11.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a7cff9700 1 -- 172.21.15.116:0/3900421667 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f3a80015030 con 0x7f3a8813b380 2020-11-20T12:21:11.935 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:11.936 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":106,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:11.109209+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:09.571842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:10.034013+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:11.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 -- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a600557c0 msgr2=0x7f3a60057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a600557c0 0x7f3a60057c80 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f3a8400af60 tx=0x7f3a84008040).stop 2020-11-20T12:21:11.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 -- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813b380 msgr2=0x7f3a880a69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:11.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813b380 0x7f3a880a69d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f3a88081a10 tx=0x7f3a80016000).stop 2020-11-20T12:21:11.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 -- 172.21.15.116:0/3900421667 shutdown_connections 2020-11-20T12:21:11.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3a600557c0 0x7f3a60057c80 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3a8813b380 0x7f3a880a69d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3a880acd30 0x7f3a880a56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 --2- 172.21.15.116:0/3900421667 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3a8813afa0 0x7f3a88164760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:11.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 -- 172.21.15.116:0/3900421667 >> 172.21.15.116:0/3900421667 conn(0x7f3a880678a0 msgr2=0x7f3a880a9b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:11.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 -- 172.21.15.116:0/3900421667 shutdown_connections 2020-11-20T12:21:11.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:11.936+0000 7f3a8da07700 1 -- 172.21.15.116:0/3900421667 wait complete. 2020-11-20T12:21:11.961 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-20T12:21:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.136+0000 7f9d08987700 1 Processor -- start 2020-11-20T12:21:12.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.136+0000 7f9d08987700 1 -- start start 2020-11-20T12:21:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d08987700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d0414f880 0x7f9d040b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:12.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d08987700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d040a81f0 0x7f9d04060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d08987700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d040a8930 con 0x7f9d04060660 2020-11-20T12:21:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d08987700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d040ba2a0 con 0x7f9d0414f880 2020-11-20T12:21:12.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d08987700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040a8690 con 0x7f9d040a81f0 2020-11-20T12:21:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d037fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9d0414f880 0x7f9d040b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:12.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d03fff700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f9d04060660 0x7f9d040a8ae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:56252/0 (socket says 172.21.15.116:56252) 2020-11-20T12:21:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d03fff700 1 -- 172.21.15.116:0/1440305837 learned_addr learned my addr 172.21.15.116:0/1440305837 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d02ffd700 1 --2- 172.21.15.116:0/1440305837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d040a81f0 0x7f9d04060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d02ffd700 1 -- 172.21.15.116:0/1440305837 >> v2:172.21.15.116:3300/0 conn(0x7f9d0414f880 msgr2=0x7f9d040b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:12.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d02ffd700 1 --2- 172.21.15.116:0/1440305837 >> v2:172.21.15.116:3300/0 conn(0x7f9d0414f880 0x7f9d040b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d027fc700 1 -- 172.21.15.116:0/1440305837 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930804605 0 0) 0x7f9d040a8930 con 0x7f9d04060660 2020-11-20T12:21:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d02ffd700 1 -- 172.21.15.116:0/1440305837 >> v1:172.21.15.173:6789/0 conn(0x7f9d04060660 legacy=0x7f9d040a8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:12.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d02ffd700 1 -- 172.21.15.116:0/1440305837 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d040b94d0 con 0x7f9d040a81f0 2020-11-20T12:21:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d02ffd700 1 --2- 172.21.15.116:0/1440305837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d040a81f0 0x7f9d04060170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9cf800b9a0 tx=0x7f9cf8006df0).ready entity=mon.2 client_cookie=25105a719502e2a0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:12.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.140+0000 7f9d027fc700 1 -- 172.21.15.116:0/1440305837 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf80097b0 con 0x7f9d040a81f0 2020-11-20T12:21:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.144+0000 7f9d027fc700 1 -- 172.21.15.116:0/1440305837 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf8009910 con 0x7f9d040a81f0 2020-11-20T12:21:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.144+0000 7f9d027fc700 1 -- 172.21.15.116:0/1440305837 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8009c90 con 0x7f9d040a81f0 2020-11-20T12:21:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.144+0000 7f9d08987700 1 -- 172.21.15.116:0/1440305837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d040a81f0 msgr2=0x7f9d04060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.144+0000 7f9d08987700 1 --2- 172.21.15.116:0/1440305837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d040a81f0 0x7f9d04060170 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f9cf800b9a0 tx=0x7f9cf8006df0).stop 2020-11-20T12:21:12.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.144+0000 7f9d08987700 1 -- 172.21.15.116:0/1440305837 shutdown_connections 2020-11-20T12:21:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.144+0000 7f9d08987700 1 --2- 172.21.15.116:0/1440305837 >> v2:172.21.15.116:3300/0 conn(0x7f9d0414f880 0x7f9d040b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.144+0000 7f9d08987700 1 --2- 172.21.15.116:0/1440305837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d040a81f0 0x7f9d04060170 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.144+0000 7f9d08987700 1 -- 172.21.15.116:0/1440305837 >> 172.21.15.116:0/1440305837 conn(0x7f9d04068350 msgr2=0x7f9d040627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:12.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.144+0000 7f9d08987700 1 -- 172.21.15.116:0/1440305837 shutdown_connections 2020-11-20T12:21:12.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.144+0000 7f9d08987700 1 -- 172.21.15.116:0/1440305837 wait complete. 2020-11-20T12:21:12.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d08987700 1 Processor -- start 2020-11-20T12:21:12.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d08987700 1 -- start start 2020-11-20T12:21:12.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d08987700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d04060660 0x7f9d040c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:12.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d08987700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d040a81f0 0x7f9d040a5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:12.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d08987700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0414f880 0x7f9d04145af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:12.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d08987700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040a50f0 con 0x7f9d040a81f0 2020-11-20T12:21:12.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d08987700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040a1960 con 0x7f9d0414f880 2020-11-20T12:21:12.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d08987700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d040a1c00 con 0x7f9d04060660 2020-11-20T12:21:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d02ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d040a81f0 0x7f9d040a5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d02ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d040a81f0 0x7f9d040a5390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50592/0 (socket says 172.21.15.116:50592) 2020-11-20T12:21:12.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d02ffd700 1 -- 172.21.15.116:0/307682693 learned_addr learned my addr 172.21.15.116:0/307682693 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d03fff700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0414f880 0x7f9d04145af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d037fe700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d04060660 0x7f9d040c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d02ffd700 1 -- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d04060660 msgr2=0x7f9d040c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:12.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d02ffd700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d04060660 0x7f9d040c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d02ffd700 1 -- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0414f880 msgr2=0x7f9d04145af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d02ffd700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0414f880 0x7f9d04145af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.148+0000 7f9d02ffd700 1 -- 172.21.15.116:0/307682693 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d040a1ea0 con 0x7f9d040a81f0 2020-11-20T12:21:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.152+0000 7f9d02ffd700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d040a81f0 0x7f9d040a5390 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f9cf8006fd0 tx=0x7f9cf8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:12.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.152+0000 7f9ce37fe700 1 -- 172.21.15.116:0/307682693 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf8011c80 con 0x7f9d040a81f0 2020-11-20T12:21:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.152+0000 7f9ce37fe700 1 -- 172.21.15.116:0/307682693 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf8011de0 con 0x7f9d040a81f0 2020-11-20T12:21:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.152+0000 7f9d08987700 1 -- 172.21.15.116:0/307682693 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d040c3590 con 0x7f9d040a81f0 2020-11-20T12:21:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.152+0000 7f9ce37fe700 1 -- 172.21.15.116:0/307682693 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf801e690 con 0x7f9d040a81f0 2020-11-20T12:21:12.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.152+0000 7f9d08987700 1 -- 172.21.15.116:0/307682693 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d040a5aa0 con 0x7f9d040a81f0 2020-11-20T12:21:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.152+0000 7f9d08987700 1 -- 172.21.15.116:0/307682693 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d0405cf30 con 0x7f9d040a81f0 2020-11-20T12:21:12.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.152+0000 7f9ce37fe700 1 -- 172.21.15.116:0/307682693 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9cf8025030 con 0x7f9d040a81f0 2020-11-20T12:21:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.152+0000 7f9ce37fe700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cdc0557c0 0x7f9cdc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.152+0000 7f9ce37fe700 1 -- 172.21.15.116:0/307682693 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9cf806fd40 con 0x7f9d040a81f0 2020-11-20T12:21:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.156+0000 7f9d037fe700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cdc0557c0 0x7f9cdc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:12.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.156+0000 7f9d037fe700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cdc0557c0 0x7f9cdc057c80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f9cf4006fd0 tx=0x7f9cf4006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:12.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.160+0000 7f9ce37fe700 1 -- 172.21.15.116:0/307682693 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9cf8045ad0 con 0x7f9d040a81f0 2020-11-20T12:21:12.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:12.372+0000 7f9d08987700 1 -- 172.21.15.116:0/307682693 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9d04067080 con 0x7f9d040a81f0 2020-11-20T12:21:13.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.124+0000 7f9ce37fe700 1 -- 172.21.15.116:0/307682693 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9cf8023070 con 0x7f9d040a81f0 2020-11-20T12:21:13.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 -- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cdc0557c0 msgr2=0x7f9cdc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cdc0557c0 0x7f9cdc057c80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f9cf4006fd0 tx=0x7f9cf4006df0).stop 2020-11-20T12:21:13.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 -- 172.21.15.116:0/307682693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d040a81f0 msgr2=0x7f9d040a5390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d040a81f0 0x7f9d040a5390 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f9cf8006fd0 tx=0x7f9cf8015000).stop 2020-11-20T12:21:13.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 -- 172.21.15.116:0/307682693 shutdown_connections 2020-11-20T12:21:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cdc0557c0 0x7f9cdc057c80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9d0414f880 0x7f9d04145af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9d04060660 0x7f9d040c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 --2- 172.21.15.116:0/307682693 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9d040a81f0 0x7f9d040a5390 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 -- 172.21.15.116:0/307682693 >> 172.21.15.116:0/307682693 conn(0x7f9d04068350 msgr2=0x7f9d040bc670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:13.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 -- 172.21.15.116:0/307682693 shutdown_connections 2020-11-20T12:21:13.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.128+0000 7f9d08987700 1 -- 172.21.15.116:0/307682693 wait complete. 2020-11-20T12:21:13.152 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:21:13.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.324+0000 7f3c72bc1700 1 Processor -- start 2020-11-20T12:21:13.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.324+0000 7f3c72bc1700 1 -- start start 2020-11-20T12:21:13.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c72bc1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 0x7f3c6c14ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:13.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c72bc1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3c6c152a30 0x7f3c6c148a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:13.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c72bc1700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6c148f40 con 0x7f3c6c148670 2020-11-20T12:21:13.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c72bc1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c6c0a3ab0 con 0x7f3c6c152a30 2020-11-20T12:21:13.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c72bc1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c14ef40 con 0x7f3c6c0b9290 2020-11-20T12:21:13.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c7115e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3c6c152a30 0x7f3c6c148a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:13.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c6bfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 0x7f3c6c14ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:13.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c6bfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 0x7f3c6c14ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:59550/0 (socket says 172.21.15.116:59550) 2020-11-20T12:21:13.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c7115e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3c6c152a30 0x7f3c6c148a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50602/0 (socket says 172.21.15.116:50602) 2020-11-20T12:21:13.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c6bfff700 1 -- 172.21.15.116:0/3875822962 learned_addr learned my addr 172.21.15.116:0/3875822962 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:13.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c7095d700 1 --1- 172.21.15.116:0/3875822962 >> v1:172.21.15.173:6789/0 conn(0x7f3c6c148670 0x7f3c6c0677e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:56262/0 (socket says 172.21.15.116:56262) 2020-11-20T12:21:13.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c6bfff700 1 -- 172.21.15.116:0/3875822962 >> v2:172.21.15.116:3300/0 conn(0x7f3c6c152a30 msgr2=0x7f3c6c148a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:13.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c6b7fe700 1 -- 172.21.15.116:0/3875822962 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1654497204 0 0) 0x7f3c600010d0 con 0x7f3c6c148670 2020-11-20T12:21:13.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c6bfff700 1 --2- 172.21.15.116:0/3875822962 >> v2:172.21.15.116:3300/0 conn(0x7f3c6c152a30 0x7f3c6c148a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c6bfff700 1 -- 172.21.15.116:0/3875822962 >> v1:172.21.15.173:6789/0 conn(0x7f3c6c148670 legacy=0x7f3c6c0677e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c6bfff700 1 -- 172.21.15.116:0/3875822962 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c14f130 con 0x7f3c6c0b9290 2020-11-20T12:21:13.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c7115e700 1 --2- 172.21.15.116:0/3875822962 >> v2:172.21.15.116:3300/0 conn(0x7f3c6c152a30 0x7f3c6c148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:13.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.328+0000 7f3c6bfff700 1 --2- 172.21.15.116:0/3875822962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 0x7f3c6c14ea50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f3c58006fd0 tx=0x7f3c58006eb0).ready entity=mon.2 client_cookie=9408efeedd5f0b7a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:13.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.332+0000 7f3c6b7fe700 1 -- 172.21.15.116:0/3875822962 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58009810 con 0x7f3c6c0b9290 2020-11-20T12:21:13.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.332+0000 7f3c6b7fe700 1 -- 172.21.15.116:0/3875822962 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c58009970 con 0x7f3c6c0b9290 2020-11-20T12:21:13.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.332+0000 7f3c6b7fe700 1 -- 172.21.15.116:0/3875822962 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58009cf0 con 0x7f3c6c0b9290 2020-11-20T12:21:13.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.332+0000 7f3c72bc1700 1 -- 172.21.15.116:0/3875822962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 msgr2=0x7f3c6c14ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.332+0000 7f3c72bc1700 1 --2- 172.21.15.116:0/3875822962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 0x7f3c6c14ea50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f3c58006fd0 tx=0x7f3c58006eb0).stop 2020-11-20T12:21:13.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.332+0000 7f3c72bc1700 1 -- 172.21.15.116:0/3875822962 shutdown_connections 2020-11-20T12:21:13.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.332+0000 7f3c72bc1700 1 --2- 172.21.15.116:0/3875822962 >> v2:172.21.15.116:3300/0 conn(0x7f3c6c152a30 0x7f3c6c148a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.332+0000 7f3c72bc1700 1 --2- 172.21.15.116:0/3875822962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 0x7f3c6c14ea50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.332+0000 7f3c72bc1700 1 -- 172.21.15.116:0/3875822962 >> 172.21.15.116:0/3875822962 conn(0x7f3c6c0c4330 msgr2=0x7f3c6c0c4740 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:13.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.332+0000 7f3c72bc1700 1 -- 172.21.15.116:0/3875822962 shutdown_connections 2020-11-20T12:21:13.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.332+0000 7f3c72bc1700 1 -- 172.21.15.116:0/3875822962 wait complete. 2020-11-20T12:21:13.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.336+0000 7f3c72bc1700 1 Processor -- start 2020-11-20T12:21:13.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.336+0000 7f3c72bc1700 1 -- start start 2020-11-20T12:21:13.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.336+0000 7f3c72bc1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 0x7f3c6c147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:13.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.336+0000 7f3c72bc1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c6c148670 0x7f3c6c14d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:13.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.336+0000 7f3c72bc1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c6c152a30 0x7f3c6c14da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:13.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.336+0000 7f3c72bc1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c0a7e80 con 0x7f3c6c148670 2020-11-20T12:21:13.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.336+0000 7f3c72bc1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c161460 con 0x7f3c6c152a30 2020-11-20T12:21:13.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.336+0000 7f3c72bc1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6c0a6520 con 0x7f3c6c0b9290 2020-11-20T12:21:13.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.336+0000 7f3c6bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c6c148670 0x7f3c6c14d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:13.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c6bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c6c148670 0x7f3c6c14d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50606/0 (socket says 172.21.15.116:50606) 2020-11-20T12:21:13.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c6bfff700 1 -- 172.21.15.116:0/1468193277 learned_addr learned my addr 172.21.15.116:0/1468193277 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:13.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c7115e700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c6c152a30 0x7f3c6c14da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:13.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c7095d700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 0x7f3c6c147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:13.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c6bfff700 1 -- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 msgr2=0x7f3c6c147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c6bfff700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 0x7f3c6c147df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c6bfff700 1 -- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c6c152a30 msgr2=0x7f3c6c14da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c6bfff700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c6c152a30 0x7f3c6c14da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c6bfff700 1 -- 172.21.15.116:0/1468193277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6c161650 con 0x7f3c6c148670 2020-11-20T12:21:13.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c7115e700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c6c152a30 0x7f3c6c14da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:13.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c7095d700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 0x7f3c6c147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:13.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c6bfff700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c6c148670 0x7f3c6c14d510 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f3c6c0617c0 tx=0x7f3c58015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:13.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c68ff9700 1 -- 172.21.15.116:0/1468193277 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c58011d50 con 0x7f3c6c148670 2020-11-20T12:21:13.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c68ff9700 1 -- 172.21.15.116:0/1468193277 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c5801e460 con 0x7f3c6c148670 2020-11-20T12:21:13.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c68ff9700 1 -- 172.21.15.116:0/1468193277 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c5801e7e0 con 0x7f3c6c148670 2020-11-20T12:21:13.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c72bc1700 1 -- 172.21.15.116:0/1468193277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6c1618f0 con 0x7f3c6c148670 2020-11-20T12:21:13.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c72bc1700 1 -- 172.21.15.116:0/1468193277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6c161e30 con 0x7f3c6c148670 2020-11-20T12:21:13.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.340+0000 7f3c72bc1700 1 -- 172.21.15.116:0/1468193277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6c05cf30 con 0x7f3c6c148670 2020-11-20T12:21:13.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.344+0000 7f3c68ff9700 1 -- 172.21.15.116:0/1468193277 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3c58025030 con 0x7f3c6c148670 2020-11-20T12:21:13.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.344+0000 7f3c68ff9700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3c440557c0 0x7f3c44057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:13.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.344+0000 7f3c68ff9700 1 -- 172.21.15.116:0/1468193277 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3c5806fba0 con 0x7f3c6c148670 2020-11-20T12:21:13.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.344+0000 7f3c7095d700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3c440557c0 0x7f3c44057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:13.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.344+0000 7f3c7095d700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3c440557c0 0x7f3c44057c80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f3c60000bc0 tx=0x7f3c60009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:13.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.352+0000 7f3c68ff9700 1 -- 172.21.15.116:0/1468193277 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3c58045b70 con 0x7f3c6c148670 2020-11-20T12:21:13.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.560+0000 7f3c72bc1700 1 -- 172.21.15.116:0/1468193277 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3c6c0a6cd0 con 0x7f3c6c148670 2020-11-20T12:21:13.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.560+0000 7f3c68ff9700 1 -- 172.21.15.116:0/1468193277 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2851 (secure 0 0 0) 0x7f3c58023030 con 0x7f3c6c148670 2020-11-20T12:21:13.562 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:13.563 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1638614527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1638614527},{"type":"v1","addr":"172.21.15.173:6837","nonce":1638614527}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3682335131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3682335131},{"type":"v1","addr":"172.21.15.173:6836","nonce":3682335131}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7725,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/3975405805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3975405805},{"type":"v1","addr":"172.21.15.116:6834","nonce":3975405805}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:11.109567+0000","modified":"2020-11-20T12:21:13.118616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/267899152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":267899152},{"type":"v1","addr":"172.21.15.116:6835","nonce":267899152}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-20T12:21:13.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 -- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3c440557c0 msgr2=0x7f3c44057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3c440557c0 0x7f3c44057c80 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f3c60000bc0 tx=0x7f3c60009040).stop 2020-11-20T12:21:13.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 -- 172.21.15.116:0/1468193277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c6c148670 msgr2=0x7f3c6c14d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c6c148670 0x7f3c6c14d510 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f3c6c0617c0 tx=0x7f3c58015000).stop 2020-11-20T12:21:13.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 -- 172.21.15.116:0/1468193277 shutdown_connections 2020-11-20T12:21:13.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3c440557c0 0x7f3c44057c80 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3c6c152a30 0x7f3c6c14da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3c6c0b9290 0x7f3c6c147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 --2- 172.21.15.116:0/1468193277 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3c6c148670 0x7f3c6c14d510 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 -- 172.21.15.116:0/1468193277 >> 172.21.15.116:0/1468193277 conn(0x7f3c6c0c4330 msgr2=0x7f3c6c14c890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:13.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 -- 172.21.15.116:0/1468193277 shutdown_connections 2020-11-20T12:21:13.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.564+0000 7f3c72bc1700 1 -- 172.21.15.116:0/1468193277 wait complete. 2020-11-20T12:21:13.569 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 60 2020-11-20T12:21:13.591 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:13.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.764+0000 7fe1dac51700 1 Processor -- start 2020-11-20T12:21:13.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.764+0000 7fe1dac51700 1 -- start start 2020-11-20T12:21:13.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1dac51700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe1d413afa0 0x7fe1d40aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:13.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1dac51700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d40ada00 con 0x7fe1d413b380 2020-11-20T12:21:13.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1dac51700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1d40adca0 con 0x7fe1d40ad440 2020-11-20T12:21:13.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1dac51700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1d40adf40 con 0x7fe1d413afa0 2020-11-20T12:21:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1d9c4f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe1d413afa0 0x7fe1d40aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:13.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1d944e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe1d413b380 0x7fe1d40acd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60048/0 (socket says 172.21.15.116:60048) 2020-11-20T12:21:13.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1d944e700 1 -- 172.21.15.116:0/2376059271 learned_addr learned my addr 172.21.15.116:0/2376059271 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:13.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1d8c4d700 1 -- 172.21.15.116:0/2376059271 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229344442 0 0) 0x7fe1d40ada00 con 0x7fe1d413b380 2020-11-20T12:21:13.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1d8c4d700 1 -- 172.21.15.116:0/2376059271 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1bc003240 con 0x7fe1d413b380 2020-11-20T12:21:13.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1d9c4f700 1 -- 172.21.15.116:0/2376059271 >> v1:172.21.15.173:6789/0 conn(0x7fe1d40ad440 legacy=0x7fe1d40a8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1d8c4d700 1 -- 172.21.15.116:0/2376059271 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2610629308 0 0) 0x7fe1d40adca0 con 0x7fe1d40ad440 2020-11-20T12:21:13.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.768+0000 7fe1d9c4f700 1 -- 172.21.15.116:0/2376059271 >> v1:172.21.15.116:6789/0 conn(0x7fe1d413b380 legacy=0x7fe1d40acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1d9c4f700 1 -- 172.21.15.116:0/2376059271 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d40ae1e0 con 0x7fe1d413afa0 2020-11-20T12:21:13.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1d9c4f700 1 --2- 172.21.15.116:0/2376059271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1d413afa0 0x7fe1d40aa5b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7fe1d000b9a0 tx=0x7fe1d0006df0).ready entity=mon.0 client_cookie=81ec779e8f930021 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1d8c4d700 1 -- 172.21.15.116:0/2376059271 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d00097b0 con 0x7fe1d413afa0 2020-11-20T12:21:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1d8c4d700 1 -- 172.21.15.116:0/2376059271 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d0009910 con 0x7fe1d413afa0 2020-11-20T12:21:13.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1d8c4d700 1 -- 172.21.15.116:0/2376059271 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d0009c90 con 0x7fe1d413afa0 2020-11-20T12:21:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1dac51700 1 -- 172.21.15.116:0/2376059271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1d413afa0 msgr2=0x7fe1d40aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1dac51700 1 --2- 172.21.15.116:0/2376059271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1d413afa0 0x7fe1d40aa5b0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7fe1d000b9a0 tx=0x7fe1d0006df0).stop 2020-11-20T12:21:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1dac51700 1 -- 172.21.15.116:0/2376059271 shutdown_connections 2020-11-20T12:21:13.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1dac51700 1 --2- 172.21.15.116:0/2376059271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1d413afa0 0x7fe1d40aa5b0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1dac51700 1 -- 172.21.15.116:0/2376059271 >> 172.21.15.116:0/2376059271 conn(0x7fe1d40678a0 msgr2=0x7fe1d4067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:13.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1dac51700 1 -- 172.21.15.116:0/2376059271 shutdown_connections 2020-11-20T12:21:13.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.772+0000 7fe1dac51700 1 -- 172.21.15.116:0/2376059271 wait complete. 2020-11-20T12:21:13.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.776+0000 7fe1dac51700 1 Processor -- start 2020-11-20T12:21:13.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.776+0000 7fe1dac51700 1 -- start start 2020-11-20T12:21:13.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.776+0000 7fe1dac51700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d40ad440 0x7fe1d40bd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:13.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.776+0000 7fe1dac51700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d413afa0 0x7fe1d40a04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:13.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.776+0000 7fe1dac51700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1d413b380 0x7fe1d40a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:13.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.776+0000 7fe1dac51700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40a0fb0 con 0x7fe1d413b380 2020-11-20T12:21:13.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.776+0000 7fe1dac51700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40a1250 con 0x7fe1d413afa0 2020-11-20T12:21:13.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.776+0000 7fe1dac51700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d40a14f0 con 0x7fe1d40ad440 2020-11-20T12:21:13.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.776+0000 7fe1da450700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1d413b380 0x7fe1d40a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:13.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.776+0000 7fe1da450700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1d413b380 0x7fe1d40a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50618/0 (socket says 172.21.15.116:50618) 2020-11-20T12:21:13.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1da450700 1 -- 172.21.15.116:0/1517824759 learned_addr learned my addr 172.21.15.116:0/1517824759 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:13.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1d944e700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d413afa0 0x7fe1d40a04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:13.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1d9c4f700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d40ad440 0x7fe1d40bd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:13.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1da450700 1 -- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d40ad440 msgr2=0x7fe1d40bd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1da450700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d40ad440 0x7fe1d40bd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1da450700 1 -- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d413afa0 msgr2=0x7fe1d40a04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1da450700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d413afa0 0x7fe1d40a04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1da450700 1 -- 172.21.15.116:0/1517824759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d40a1790 con 0x7fe1d413b380 2020-11-20T12:21:13.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1d9c4f700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d40ad440 0x7fe1d40bd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:13.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1da450700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1d413b380 0x7fe1d40a0990 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7fe1cc00cab0 tx=0x7fe1cc007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:13.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1c1ffb700 1 -- 172.21.15.116:0/1517824759 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00a8c0 con 0x7fe1d413b380 2020-11-20T12:21:13.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1c1ffb700 1 -- 172.21.15.116:0/1517824759 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1cc00aa20 con 0x7fe1d413b380 2020-11-20T12:21:13.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1dac51700 1 -- 172.21.15.116:0/1517824759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1d40a1a30 con 0x7fe1d413b380 2020-11-20T12:21:13.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1c1ffb700 1 -- 172.21.15.116:0/1517824759 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1cc00adc0 con 0x7fe1d413b380 2020-11-20T12:21:13.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1dac51700 1 -- 172.21.15.116:0/1517824759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1d40be3a0 con 0x7fe1d413b380 2020-11-20T12:21:13.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.780+0000 7fe1dac51700 1 -- 172.21.15.116:0/1517824759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1d40a1bc0 con 0x7fe1d413b380 2020-11-20T12:21:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.784+0000 7fe1c1ffb700 1 -- 172.21.15.116:0/1517824759 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe1cc019030 con 0x7fe1d413b380 2020-11-20T12:21:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.784+0000 7fe1c1ffb700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1ac0557c0 0x7fe1ac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.784+0000 7fe1c1ffb700 1 -- 172.21.15.116:0/1517824759 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe1cc06ea00 con 0x7fe1d413b380 2020-11-20T12:21:13.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.784+0000 7fe1d9c4f700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1ac0557c0 0x7fe1ac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:13.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.788+0000 7fe1d9c4f700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1ac0557c0 0x7fe1ac057c80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fe1d000be60 tx=0x7fe1d0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:13.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.788+0000 7fe1c1ffb700 1 -- 172.21.15.116:0/1517824759 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe1cc0448c0 con 0x7fe1d413b380 2020-11-20T12:21:13.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.948+0000 7fe1dac51700 1 -- 172.21.15.116:0/1517824759 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1d405cf30 con 0x7fe1d413b380 2020-11-20T12:21:13.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.948+0000 7fe1c1ffb700 1 -- 172.21.15.116:0/1517824759 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16888 (secure 0 0 0) 0x7fe1cc021070 con 0x7fe1d413b380 2020-11-20T12:21:13.951 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:13.952 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":107,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:12.114283+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:09.571842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:10.034013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:13.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 -- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1ac0557c0 msgr2=0x7fe1ac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1ac0557c0 0x7fe1ac057c80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fe1d000be60 tx=0x7fe1d0015000).stop 2020-11-20T12:21:13.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 -- 172.21.15.116:0/1517824759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1d413b380 msgr2=0x7fe1d40a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:13.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1d413b380 0x7fe1d40a0990 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7fe1cc00cab0 tx=0x7fe1cc007dc0).stop 2020-11-20T12:21:13.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 -- 172.21.15.116:0/1517824759 shutdown_connections 2020-11-20T12:21:13.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1ac0557c0 0x7fe1ac057c80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe1d413b380 0x7fe1d40a0990 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe1d40ad440 0x7fe1d40bd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 --2- 172.21.15.116:0/1517824759 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe1d413afa0 0x7fe1d40a04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 -- 172.21.15.116:0/1517824759 >> 172.21.15.116:0/1517824759 conn(0x7fe1d40678a0 msgr2=0x7fe1d4067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 -- 172.21.15.116:0/1517824759 shutdown_connections 2020-11-20T12:21:13.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:13.952+0000 7fe1dac51700 1 -- 172.21.15.116:0/1517824759 wait complete. 2020-11-20T12:21:13.977 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-20T12:21:14.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.156+0000 7fe5d5c35700 1 Processor -- start 2020-11-20T12:21:14.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.156+0000 7fe5d5c35700 1 -- start start 2020-11-20T12:21:14.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.160+0000 7fe5d5c35700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe5d00aa7d0 0x7fe5d00a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.160+0000 7fe5d5c35700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5d00aabb0 0x7fe5d0151e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.160+0000 7fe5d5c35700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d00ace40 con 0x7fe5d00ad540 2020-11-20T12:21:14.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.160+0000 7fe5d5c35700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d01526c0 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.160+0000 7fe5d5c35700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d0152960 con 0x7fe5d00aabb0 2020-11-20T12:21:14.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.160+0000 7fe5d5434700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe5d00ad540 0x7fe5d00ad9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60064/0 (socket says 172.21.15.116:60064) 2020-11-20T12:21:14.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.160+0000 7fe5d5434700 1 -- 172.21.15.116:0/81587924 learned_addr learned my addr 172.21.15.116:0/81587924 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:14.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.160+0000 7fe5cffff700 1 --2- 172.21.15.116:0/81587924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5d00aabb0 0x7fe5d0151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.160+0000 7fe5d4c33700 1 --2- 172.21.15.116:0/81587924 >> v2:172.21.15.173:3300/0 conn(0x7fe5d00aa7d0 0x7fe5d00a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.160+0000 7fe5cf7fe700 1 -- 172.21.15.116:0/81587924 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 184478382 0 0) 0x7fe5d00ace40 con 0x7fe5d00ad540 2020-11-20T12:21:14.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5cf7fe700 1 -- 172.21.15.116:0/81587924 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5b4003240 con 0x7fe5d00ad540 2020-11-20T12:21:14.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5cffff700 1 -- 172.21.15.116:0/81587924 >> v2:172.21.15.173:3300/0 conn(0x7fe5d00aa7d0 msgr2=0x7fe5d00a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:14.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5cffff700 1 --2- 172.21.15.116:0/81587924 >> v2:172.21.15.173:3300/0 conn(0x7fe5d00aa7d0 0x7fe5d00a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5cffff700 1 -- 172.21.15.116:0/81587924 >> v1:172.21.15.116:6789/0 conn(0x7fe5d00ad540 legacy=0x7fe5d00ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5cffff700 1 -- 172.21.15.116:0/81587924 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d0152c00 con 0x7fe5d00aabb0 2020-11-20T12:21:14.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5cffff700 1 --2- 172.21.15.116:0/81587924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5d00aabb0 0x7fe5d0151e00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe5bc00b9a0 tx=0x7fe5bc006df0).ready entity=mon.2 client_cookie=e1d4782900ec35a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:14.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5cf7fe700 1 -- 172.21.15.116:0/81587924 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc009680 con 0x7fe5d00aabb0 2020-11-20T12:21:14.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5cf7fe700 1 -- 172.21.15.116:0/81587924 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0097e0 con 0x7fe5d00aabb0 2020-11-20T12:21:14.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5cf7fe700 1 -- 172.21.15.116:0/81587924 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc009b60 con 0x7fe5d00aabb0 2020-11-20T12:21:14.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5d5c35700 1 -- 172.21.15.116:0/81587924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5d00aabb0 msgr2=0x7fe5d0151e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5d5c35700 1 --2- 172.21.15.116:0/81587924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5d00aabb0 0x7fe5d0151e00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fe5bc00b9a0 tx=0x7fe5bc006df0).stop 2020-11-20T12:21:14.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5d5c35700 1 -- 172.21.15.116:0/81587924 shutdown_connections 2020-11-20T12:21:14.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5d5c35700 1 --2- 172.21.15.116:0/81587924 >> v2:172.21.15.173:3300/0 conn(0x7fe5d00aa7d0 0x7fe5d00a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5d5c35700 1 --2- 172.21.15.116:0/81587924 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5d00aabb0 0x7fe5d0151e00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5d5c35700 1 -- 172.21.15.116:0/81587924 >> 172.21.15.116:0/81587924 conn(0x7fe5d00678a0 msgr2=0x7fe5d0067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:14.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5d5c35700 1 -- 172.21.15.116:0/81587924 shutdown_connections 2020-11-20T12:21:14.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.164+0000 7fe5d5c35700 1 -- 172.21.15.116:0/81587924 wait complete. 2020-11-20T12:21:14.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.168+0000 7fe5d5c35700 1 Processor -- start 2020-11-20T12:21:14.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.168+0000 7fe5d5c35700 1 -- start start 2020-11-20T12:21:14.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d5c35700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5d00aa7d0 0x7fe5d0164560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d5c35700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5d00aabb0 0x7fe5d0160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d5c35700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5d00ad540 0x7fe5d01612a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d5c35700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d00a05d0 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d5c35700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d00a0870 con 0x7fe5d00ad540 2020-11-20T12:21:14.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d5c35700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d00a0b10 con 0x7fe5d00aabb0 2020-11-20T12:21:14.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d4c33700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5d00aa7d0 0x7fe5d0164560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d4c33700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5d00aa7d0 0x7fe5d0164560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50632/0 (socket says 172.21.15.116:50632) 2020-11-20T12:21:14.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d4c33700 1 -- 172.21.15.116:0/145195568 learned_addr learned my addr 172.21.15.116:0/145195568 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:14.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d5434700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5d00ad540 0x7fe5d01612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5cffff700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5d00aabb0 0x7fe5d0160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d4c33700 1 -- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5d00aabb0 msgr2=0x7fe5d0160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d4c33700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5d00aabb0 0x7fe5d0160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d4c33700 1 -- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5d00ad540 msgr2=0x7fe5d01612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d4c33700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5d00ad540 0x7fe5d01612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d4c33700 1 -- 172.21.15.116:0/145195568 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d00a0db0 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d4c33700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5d00aa7d0 0x7fe5d0164560 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7fe5c400eba0 tx=0x7fe5c4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:14.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5ccff9700 1 -- 172.21.15.116:0/145195568 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400c9b0 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5ccff9700 1 -- 172.21.15.116:0/145195568 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c400cb10 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d5c35700 1 -- 172.21.15.116:0/145195568 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d00a1050 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5ccff9700 1 -- 172.21.15.116:0/145195568 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c400ce90 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d5c35700 1 -- 172.21.15.116:0/145195568 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d00a0050 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.172+0000 7fe5d5c35700 1 -- 172.21.15.116:0/145195568 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d005cf30 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.176+0000 7fe5ccff9700 1 -- 172.21.15.116:0/145195568 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe5c4012600 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.176+0000 7fe5ccff9700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5a80557c0 0x7fe5a8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.176+0000 7fe5ccff9700 1 -- 172.21.15.116:0/145195568 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe5c406eb80 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.176+0000 7fe5cffff700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5a80557c0 0x7fe5a8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.180+0000 7fe5cffff700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5a80557c0 0x7fe5a8057c80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fe5bc00b150 tx=0x7fe5bc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:14.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.180+0000 7fe5ccff9700 1 -- 172.21.15.116:0/145195568 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe5c40448b0 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.396+0000 7fe5d5c35700 1 -- 172.21.15.116:0/145195568 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fe5d00a0300 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.404+0000 7fe5ccff9700 1 -- 172.21.15.116:0/145195568 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v26) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe5c406ee60 con 0x7fe5d00aa7d0 2020-11-20T12:21:14.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 -- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5a80557c0 msgr2=0x7fe5a8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5a80557c0 0x7fe5a8057c80 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7fe5bc00b150 tx=0x7fe5bc015000).stop 2020-11-20T12:21:14.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 -- 172.21.15.116:0/145195568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5d00aa7d0 msgr2=0x7fe5d0164560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5d00aa7d0 0x7fe5d0164560 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7fe5c400eba0 tx=0x7fe5c4006df0).stop 2020-11-20T12:21:14.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 -- 172.21.15.116:0/145195568 shutdown_connections 2020-11-20T12:21:14.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe5a80557c0 0x7fe5a8057c80 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5d00ad540 0x7fe5d01612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5d00aa7d0 0x7fe5d0164560 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 --2- 172.21.15.116:0/145195568 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe5d00aabb0 0x7fe5d0160db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 -- 172.21.15.116:0/145195568 >> 172.21.15.116:0/145195568 conn(0x7fe5d00678a0 msgr2=0x7fe5d0067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:14.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 -- 172.21.15.116:0/145195568 shutdown_connections 2020-11-20T12:21:14.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.408+0000 7fe5d5c35700 1 -- 172.21.15.116:0/145195568 wait complete. 2020-11-20T12:21:14.411 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2020-11-20T12:21:14.432 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:21:14.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.608+0000 7f8e6243b700 1 Processor -- start 2020-11-20T12:21:14.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.608+0000 7f8e6243b700 1 -- start start 2020-11-20T12:21:14.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e6243b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c0bfa80 0x7f8e5c0bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e6243b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e5c0bf100 0x7f8e5c0bac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e6243b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8e5c066e20 0x7f8e5c14db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e6243b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c0bb0f0 con 0x7f8e5c0bf100 2020-11-20T12:21:14.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e6243b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b9d20 con 0x7f8e5c066e20 2020-11-20T12:21:14.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e6243b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0b9f80 con 0x7f8e5c0bfa80 2020-11-20T12:21:14.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e53fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e5c0bf100 0x7f8e5c0bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e53fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8e5c0bf100 0x7f8e5c0bac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50640/0 (socket says 172.21.15.116:50640) 2020-11-20T12:21:14.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e53fff700 1 -- 172.21.15.116:0/888677717 learned_addr learned my addr 172.21.15.116:0/888677717 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:14.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e609d8700 1 --2- 172.21.15.116:0/888677717 >> v2:172.21.15.173:3300/0 conn(0x7f8e5c066e20 0x7f8e5c14db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e53fff700 1 -- 172.21.15.116:0/888677717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c0bfa80 msgr2=0x7f8e5c0bfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:14.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e53fff700 1 --2- 172.21.15.116:0/888677717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c0bfa80 0x7f8e5c0bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:14.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e53fff700 1 -- 172.21.15.116:0/888677717 >> v2:172.21.15.173:3300/0 conn(0x7f8e5c066e20 msgr2=0x7f8e5c14db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:14.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e53fff700 1 --2- 172.21.15.116:0/888677717 >> v2:172.21.15.173:3300/0 conn(0x7f8e5c066e20 0x7f8e5c14db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e53fff700 1 -- 172.21.15.116:0/888677717 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c0ba220 con 0x7f8e5c0bf100 2020-11-20T12:21:14.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e609d8700 1 --2- 172.21.15.116:0/888677717 >> v2:172.21.15.173:3300/0 conn(0x7f8e5c066e20 0x7f8e5c14db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:14.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e53fff700 1 --2- 172.21.15.116:0/888677717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0bf100 0x7f8e5c0bac00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f8e5c0a71c0 tx=0x7f8e44006eb0).ready entity=mon.0 client_cookie=4fcb9cea4228b3bc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:14.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e5b7fe700 1 -- 172.21.15.116:0/888677717 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e440098d0 con 0x7f8e5c0bf100 2020-11-20T12:21:14.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e5b7fe700 1 -- 172.21.15.116:0/888677717 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e44009a30 con 0x7f8e5c0bf100 2020-11-20T12:21:14.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.612+0000 7f8e5b7fe700 1 -- 172.21.15.116:0/888677717 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44009db0 con 0x7f8e5c0bf100 2020-11-20T12:21:14.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.616+0000 7f8e6243b700 1 -- 172.21.15.116:0/888677717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0bf100 msgr2=0x7f8e5c0bac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.616+0000 7f8e6243b700 1 --2- 172.21.15.116:0/888677717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0bf100 0x7f8e5c0bac00 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f8e5c0a71c0 tx=0x7f8e44006eb0).stop 2020-11-20T12:21:14.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.616+0000 7f8e6243b700 1 -- 172.21.15.116:0/888677717 shutdown_connections 2020-11-20T12:21:14.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.616+0000 7f8e6243b700 1 --2- 172.21.15.116:0/888677717 >> v2:172.21.15.173:3300/0 conn(0x7f8e5c066e20 0x7f8e5c14db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.616+0000 7f8e6243b700 1 --2- 172.21.15.116:0/888677717 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c0bfa80 0x7f8e5c0bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:14.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.616+0000 7f8e6243b700 1 --2- 172.21.15.116:0/888677717 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0bf100 0x7f8e5c0bac00 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.616+0000 7f8e6243b700 1 -- 172.21.15.116:0/888677717 >> 172.21.15.116:0/888677717 conn(0x7f8e5c0c4f80 msgr2=0x7f8e5c0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:14.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.616+0000 7f8e6243b700 1 -- 172.21.15.116:0/888677717 shutdown_connections 2020-11-20T12:21:14.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.616+0000 7f8e6243b700 1 -- 172.21.15.116:0/888677717 wait complete. 2020-11-20T12:21:14.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e6243b700 1 Processor -- start 2020-11-20T12:21:14.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e6243b700 1 -- start start 2020-11-20T12:21:14.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e6243b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c066e20 0x7f8e5c143590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e6243b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0bf100 0x7f8e5c144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e6243b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c0bfa80 0x7f8e5c145310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e6243b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c145800 con 0x7f8e5c0bf100 2020-11-20T12:21:14.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e6243b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c143a80 con 0x7f8e5c066e20 2020-11-20T12:21:14.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e6243b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0be140 con 0x7f8e5c0bfa80 2020-11-20T12:21:14.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e53fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0bf100 0x7f8e5c144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e53fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0bf100 0x7f8e5c144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50648/0 (socket says 172.21.15.116:50648) 2020-11-20T12:21:14.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e53fff700 1 -- 172.21.15.116:0/3657328502 learned_addr learned my addr 172.21.15.116:0/3657328502 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:14.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e5bfff700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c066e20 0x7f8e5c143590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e609d8700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c0bfa80 0x7f8e5c145310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e53fff700 1 -- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c0bfa80 msgr2=0x7f8e5c145310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e53fff700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c0bfa80 0x7f8e5c145310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e53fff700 1 -- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c066e20 msgr2=0x7f8e5c143590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e53fff700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c066e20 0x7f8e5c143590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e53fff700 1 -- 172.21.15.116:0/3657328502 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c0be3e0 con 0x7f8e5c0bf100 2020-11-20T12:21:14.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.620+0000 7f8e609d8700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c0bfa80 0x7f8e5c145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:14.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.624+0000 7f8e53fff700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0bf100 0x7f8e5c144e20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f8e44010000 tx=0x7f8e44015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:14.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.624+0000 7f8e58ff9700 1 -- 172.21.15.116:0/3657328502 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e44011e50 con 0x7f8e5c0bf100 2020-11-20T12:21:14.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.624+0000 7f8e58ff9700 1 -- 172.21.15.116:0/3657328502 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4401e460 con 0x7f8e5c0bf100 2020-11-20T12:21:14.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.624+0000 7f8e6243b700 1 -- 172.21.15.116:0/3657328502 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c0a82d0 con 0x7f8e5c0bf100 2020-11-20T12:21:14.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.624+0000 7f8e58ff9700 1 -- 172.21.15.116:0/3657328502 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4401e800 con 0x7f8e5c0bf100 2020-11-20T12:21:14.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.624+0000 7f8e6243b700 1 -- 172.21.15.116:0/3657328502 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c0a8810 con 0x7f8e5c0bf100 2020-11-20T12:21:14.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.624+0000 7f8e6243b700 1 -- 172.21.15.116:0/3657328502 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c0a89a0 con 0x7f8e5c0bf100 2020-11-20T12:21:14.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.624+0000 7f8e58ff9700 1 -- 172.21.15.116:0/3657328502 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8e44025030 con 0x7f8e5c0bf100 2020-11-20T12:21:14.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.628+0000 7f8e58ff9700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e380557c0 0x7f8e38057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:14.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.628+0000 7f8e58ff9700 1 -- 172.21.15.116:0/3657328502 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8e4406fe80 con 0x7f8e5c0bf100 2020-11-20T12:21:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.628+0000 7f8e5bfff700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e380557c0 0x7f8e38057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:14.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.628+0000 7f8e5bfff700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e380557c0 0x7f8e38057c80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f8e4c00b6c0 tx=0x7f8e4c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:14.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.632+0000 7f8e58ff9700 1 -- 172.21.15.116:0/3657328502 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8e44045cc0 con 0x7f8e5c0bf100 2020-11-20T12:21:14.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.840+0000 7f8e6243b700 1 -- 172.21.15.116:0/3657328502 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e5c05d2a0 con 0x7f8e5c0bf100 2020-11-20T12:21:14.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.840+0000 7f8e58ff9700 1 -- 172.21.15.116:0/3657328502 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2851 (secure 0 0 0) 0x7f8e44023030 con 0x7f8e5c0bf100 2020-11-20T12:21:14.842 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:14.842 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1638614527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1638614527},{"type":"v1","addr":"172.21.15.173:6837","nonce":1638614527}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3682335131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3682335131},{"type":"v1","addr":"172.21.15.173:6836","nonce":3682335131}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7725,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/3975405805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3975405805},{"type":"v1","addr":"172.21.15.116:6834","nonce":3975405805}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:11.109567+0000","modified":"2020-11-20T12:21:13.118616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/267899152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":267899152},{"type":"v1","addr":"172.21.15.116:6835","nonce":267899152}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-20T12:21:14.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 -- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e380557c0 msgr2=0x7f8e38057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e380557c0 0x7f8e38057c80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f8e4c00b6c0 tx=0x7f8e4c008040).stop 2020-11-20T12:21:14.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 -- 172.21.15.116:0/3657328502 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0bf100 msgr2=0x7f8e5c144e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:14.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0bf100 0x7f8e5c144e20 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f8e44010000 tx=0x7f8e44015000).stop 2020-11-20T12:21:14.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 -- 172.21.15.116:0/3657328502 shutdown_connections 2020-11-20T12:21:14.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e380557c0 0x7f8e38057c80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c0bfa80 0x7f8e5c145310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c066e20 0x7f8e5c143590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 --2- 172.21.15.116:0/3657328502 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0bf100 0x7f8e5c144e20 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:14.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 -- 172.21.15.116:0/3657328502 >> 172.21.15.116:0/3657328502 conn(0x7f8e5c0c4f80 msgr2=0x7f8e5c0a6090 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:14.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 -- 172.21.15.116:0/3657328502 shutdown_connections 2020-11-20T12:21:14.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:14.844+0000 7f8e6243b700 1 -- 172.21.15.116:0/3657328502 wait complete. 2020-11-20T12:21:14.848 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 60 2020-11-20T12:21:15.273 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:21:11.109567+0000', 'modified': '2020-11-20T12:21:13.118616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7728': {'gid': 7728, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/267899152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 267899152}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 267899152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:21:15.276 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:21:15.278 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:21:15.442 INFO:tasks.ceph.mds.c.smithi116.stderr:2020-11-20T12:21:15.439+0000 7f5704183700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:21:15.475 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":7728,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.3447004329999999,"mdsmap_epoch":60,"osdmap_epoch":107,"osdmap_epoch_barrier":107,"uptime":9.8497582450000003} 2020-11-20T12:21:15.476 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 7728, "mdsmap_epoch": 60, "osdmap_epoch": 107, "osdmap_epoch_barrier": 107, "rank_uptime": 4.344700433, "state": "up:active", "uptime": 9.849758245, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:21:15.476 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:21:15.477 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:21:15.477 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:21:15.477 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:21:15.478 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:21:15.478 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:21:15.479 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:21:15.479 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:21:15.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:15.480 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:21:15.498 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:21:15.498 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:21:15.498 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:21:15.499 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:15.499 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:21:15.499 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:15.499 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:15.499 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:15.500 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:15.500 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:15.500 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:15.500 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:15.501 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:21:15.501 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:15.501 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:21:15.501 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:15.501 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:21:15.502 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:15.502 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:21:15.502 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:15.502 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:21:15.503 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:15.505 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:21:15.505 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:15.505 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:21:15.506 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:15.506 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:21:15.506 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:15.506 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:21:15.506 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:21:15.507 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:21:15.507 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:15.507 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:21:15.507 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:21:15.507 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:21:15.508 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:15.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:15.509 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:21:15.538 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:21:15.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:15.542 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:21:15.605 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:21:15.605 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:21:15.605 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:21:15.606 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:21:15.606 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:21:15.606 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:21:15.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:15.608 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:21:15.608 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:21:15.608 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:21:15.609 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:21:15.609 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:21:15.610 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:21:15.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:15.704 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:21:15.720 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:21:15.720 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:21:15.721 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:21:15.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:15.722 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2020-11-20T12:21:15.798 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:15.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.971+0000 7fd0ccbd7700 1 Processor -- start 2020-11-20T12:21:15.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.971+0000 7fd0ccbd7700 1 -- start start 2020-11-20T12:21:15.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0ccbd7700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd0c80c5310 0x7fd0c80c56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:15.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0ccbd7700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd0c80c3a90 0x7fd0c814b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:15.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0ccbd7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0c814beb0 0x7fd0c8152040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:15.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0ccbd7700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c80c2e50 con 0x7fd0c80c3a90 2020-11-20T12:21:15.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0ccbd7700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c8152d50 con 0x7fd0c80c5310 2020-11-20T12:21:15.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0ccbd7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c80c30b0 con 0x7fd0c814beb0 2020-11-20T12:21:15.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0c6ffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd0c80c3a90 0x7fd0c814b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:15.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0c6ffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fd0c80c3a90 0x7fd0c814b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50654/0 (socket says 172.21.15.116:50654) 2020-11-20T12:21:15.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0c6ffd700 1 -- 172.21.15.116:0/1247226802 learned_addr learned my addr 172.21.15.116:0/1247226802 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:15.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0c77fe700 1 --2- 172.21.15.116:0/1247226802 >> v2:172.21.15.173:3300/0 conn(0x7fd0c80c5310 0x7fd0c80c56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:15.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0c7fff700 1 --2- 172.21.15.116:0/1247226802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0c814beb0 0x7fd0c8152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:15.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0c6ffd700 1 -- 172.21.15.116:0/1247226802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0c814beb0 msgr2=0x7fd0c8152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:15.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0c6ffd700 1 --2- 172.21.15.116:0/1247226802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0c814beb0 0x7fd0c8152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:15.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0c6ffd700 1 -- 172.21.15.116:0/1247226802 >> v2:172.21.15.173:3300/0 conn(0x7fd0c80c5310 msgr2=0x7fd0c80c56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:15.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0c6ffd700 1 --2- 172.21.15.116:0/1247226802 >> v2:172.21.15.173:3300/0 conn(0x7fd0c80c5310 0x7fd0c80c56f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:15.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0c6ffd700 1 -- 172.21.15.116:0/1247226802 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c80c3350 con 0x7fd0c80c3a90 2020-11-20T12:21:15.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.975+0000 7fd0c6ffd700 1 --2- 172.21.15.116:0/1247226802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c80c3a90 0x7fd0c814b9c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7fd0c80688b0 tx=0x7fd0bc006eb0).ready entity=mon.0 client_cookie=a88adccac7222c1b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:15.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0c67fc700 1 -- 172.21.15.116:0/1247226802 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc009720 con 0x7fd0c80c3a90 2020-11-20T12:21:15.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0c67fc700 1 -- 172.21.15.116:0/1247226802 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0bc009880 con 0x7fd0c80c3a90 2020-11-20T12:21:15.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0c67fc700 1 -- 172.21.15.116:0/1247226802 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc009c00 con 0x7fd0c80c3a90 2020-11-20T12:21:15.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/1247226802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c80c3a90 msgr2=0x7fd0c814b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:15.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0ccbd7700 1 --2- 172.21.15.116:0/1247226802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c80c3a90 0x7fd0c814b9c0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7fd0c80688b0 tx=0x7fd0bc006eb0).stop 2020-11-20T12:21:15.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/1247226802 shutdown_connections 2020-11-20T12:21:15.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0ccbd7700 1 --2- 172.21.15.116:0/1247226802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0c814beb0 0x7fd0c8152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:15.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0ccbd7700 1 --2- 172.21.15.116:0/1247226802 >> v2:172.21.15.173:3300/0 conn(0x7fd0c80c5310 0x7fd0c80c56f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:15.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0ccbd7700 1 --2- 172.21.15.116:0/1247226802 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c80c3a90 0x7fd0c814b9c0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:15.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/1247226802 >> 172.21.15.116:0/1247226802 conn(0x7fd0c80c8740 msgr2=0x7fd0c80c8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:15.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/1247226802 shutdown_connections 2020-11-20T12:21:15.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.979+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/1247226802 wait complete. 2020-11-20T12:21:15.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0ccbd7700 1 Processor -- start 2020-11-20T12:21:15.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0ccbd7700 1 -- start start 2020-11-20T12:21:15.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0ccbd7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0c80c5310 0x7fd0c8161b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:15.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0ccbd7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c814beb0 0x7fd0c80c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:15.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0ccbd7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0c80c7d60 0x7fd0c8164800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:15.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0ccbd7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c8161230 con 0x7fd0c814beb0 2020-11-20T12:21:15.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0ccbd7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c8165260 con 0x7fd0c80c5310 2020-11-20T12:21:15.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0ccbd7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c8165500 con 0x7fd0c80c7d60 2020-11-20T12:21:15.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c814beb0 0x7fd0c80c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:15.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c6ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c814beb0 0x7fd0c80c7870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50660/0 (socket says 172.21.15.116:50660) 2020-11-20T12:21:15.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c6ffd700 1 -- 172.21.15.116:0/3728989425 learned_addr learned my addr 172.21.15.116:0/3728989425 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:15.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c77fe700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0c80c5310 0x7fd0c8161b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:15.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c7fff700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0c80c7d60 0x7fd0c8164800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:15.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c6ffd700 1 -- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0c80c7d60 msgr2=0x7fd0c8164800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:15.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c6ffd700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0c80c7d60 0x7fd0c8164800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:15.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c6ffd700 1 -- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0c80c5310 msgr2=0x7fd0c8161b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:15.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c6ffd700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0c80c5310 0x7fd0c8161b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:15.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c6ffd700 1 -- 172.21.15.116:0/3728989425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c8161420 con 0x7fd0c814beb0 2020-11-20T12:21:15.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c77fe700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0c80c5310 0x7fd0c8161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:15.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0c6ffd700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c814beb0 0x7fd0c80c7870 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7fd0bc006ef0 tx=0x7fd0bc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:15.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0a77fe700 1 -- 172.21.15.116:0/3728989425 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc011c60 con 0x7fd0c814beb0 2020-11-20T12:21:15.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0a77fe700 1 -- 172.21.15.116:0/3728989425 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0bc011dc0 con 0x7fd0c814beb0 2020-11-20T12:21:15.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/3728989425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c81616c0 con 0x7fd0c814beb0 2020-11-20T12:21:15.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0a77fe700 1 -- 172.21.15.116:0/3728989425 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc01e690 con 0x7fd0c814beb0 2020-11-20T12:21:15.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.983+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/3728989425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c8162050 con 0x7fd0c814beb0 2020-11-20T12:21:15.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.987+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/3728989425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c81656c0 con 0x7fd0c814beb0 2020-11-20T12:21:15.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.987+0000 7fd0a77fe700 1 -- 172.21.15.116:0/3728989425 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd0bc025030 con 0x7fd0c814beb0 2020-11-20T12:21:15.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.987+0000 7fd0a77fe700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0a00557c0 0x7fd0a0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:15.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.987+0000 7fd0a77fe700 1 -- 172.21.15.116:0/3728989425 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd0bc06fd50 con 0x7fd0c814beb0 2020-11-20T12:21:15.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.991+0000 7fd0c77fe700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0a00557c0 0x7fd0a0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:15.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.991+0000 7fd0c77fe700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0a00557c0 0x7fd0a0057c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fd0c80ae6f0 tx=0x7fd0b8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:15.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:15.995+0000 7fd0a77fe700 1 -- 172.21.15.116:0/3728989425 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd0bc045bf0 con 0x7fd0c814beb0 2020-11-20T12:21:16.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.155+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/3728989425 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0c8161850 con 0x7fd0c814beb0 2020-11-20T12:21:16.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.155+0000 7fd0a77fe700 1 -- 172.21.15.116:0/3728989425 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16888 (secure 0 0 0) 0x7fd0bc023020 con 0x7fd0c814beb0 2020-11-20T12:21:16.159 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:16.159 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":107,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:12.114283+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:09.571842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:10.034013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:16.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0a00557c0 msgr2=0x7fd0a0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0a00557c0 0x7fd0a0057c80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fd0c80ae6f0 tx=0x7fd0b8008040).stop 2020-11-20T12:21:16.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/3728989425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c814beb0 msgr2=0x7fd0c80c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c814beb0 0x7fd0c80c7870 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7fd0bc006ef0 tx=0x7fd0bc015000).stop 2020-11-20T12:21:16.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/3728989425 shutdown_connections 2020-11-20T12:21:16.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd0a00557c0 0x7fd0a0057c80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd0c80c7d60 0x7fd0c8164800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd0c80c5310 0x7fd0c8161b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 --2- 172.21.15.116:0/3728989425 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd0c814beb0 0x7fd0c80c7870 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/3728989425 >> 172.21.15.116:0/3728989425 conn(0x7fd0c80c8740 msgr2=0x7fd0c814b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:16.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/3728989425 shutdown_connections 2020-11-20T12:21:16.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.159+0000 7fd0ccbd7700 1 -- 172.21.15.116:0/3728989425 wait complete. 2020-11-20T12:21:16.185 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:21:09.571842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:21:10.034013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:21:16.186 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:21:16.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.363+0000 7f9316de2700 1 Processor -- start 2020-11-20T12:21:16.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.363+0000 7f9316de2700 1 -- start start 2020-11-20T12:21:16.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9316de2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93100c3610 0x7f9310065120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:16.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9316de2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f93100bcef0 0x7f93100bbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:16.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9316de2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93100bd3a0 con 0x7f93100bca90 2020-11-20T12:21:16.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9316de2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f931014fcf0 con 0x7f93100bcef0 2020-11-20T12:21:16.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9316de2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93100c1eb0 con 0x7f93100c3610 2020-11-20T12:21:16.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9315de0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93100c3610 0x7f9310065120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:16.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f93165e1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f93100bcef0 0x7f93100bbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:16.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9315de0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93100c3610 0x7f9310065120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:59618/0 (socket says 172.21.15.116:59618) 2020-11-20T12:21:16.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9315de0700 1 -- 172.21.15.116:0/1674192123 learned_addr learned my addr 172.21.15.116:0/1674192123 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:16.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f93165e1700 1 -- 172.21.15.116:0/1674192123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93100c3610 msgr2=0x7f9310065120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:16.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f93165e1700 1 --2- 172.21.15.116:0/1674192123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93100c3610 0x7f9310065120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f93165e1700 1 -- 172.21.15.116:0/1674192123 >> v1:172.21.15.173:6789/0 conn(0x7f93100bca90 legacy=0x7f93100bb5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9314dde700 1 -- 172.21.15.116:0/1674192123 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629730046 0 0) 0x7f93100bd3a0 con 0x7f93100bca90 2020-11-20T12:21:16.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f93165e1700 1 -- 172.21.15.116:0/1674192123 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931014fee0 con 0x7f93100bcef0 2020-11-20T12:21:16.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9315de0700 1 --2- 172.21.15.116:0/1674192123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93100c3610 0x7f9310065120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:16.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f93165e1700 1 --2- 172.21.15.116:0/1674192123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100bcef0 0x7f93100bbcc0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f9308009c90 tx=0x7f93080091d0).ready entity=mon.0 client_cookie=411baaf47a7b8fe9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:16.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9314dde700 1 -- 172.21.15.116:0/1674192123 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930800ba70 con 0x7f93100bcef0 2020-11-20T12:21:16.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9314dde700 1 -- 172.21.15.116:0/1674192123 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f930800bbd0 con 0x7f93100bcef0 2020-11-20T12:21:16.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.367+0000 7f9314dde700 1 -- 172.21.15.116:0/1674192123 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308011400 con 0x7f93100bcef0 2020-11-20T12:21:16.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.371+0000 7f9316de2700 1 -- 172.21.15.116:0/1674192123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100bcef0 msgr2=0x7f93100bbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.371+0000 7f9316de2700 1 --2- 172.21.15.116:0/1674192123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100bcef0 0x7f93100bbcc0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f9308009c90 tx=0x7f93080091d0).stop 2020-11-20T12:21:16.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.371+0000 7f9316de2700 1 -- 172.21.15.116:0/1674192123 shutdown_connections 2020-11-20T12:21:16.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.371+0000 7f9316de2700 1 --2- 172.21.15.116:0/1674192123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100bcef0 0x7f93100bbcc0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.371+0000 7f9316de2700 1 --2- 172.21.15.116:0/1674192123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93100c3610 0x7f9310065120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.371+0000 7f9316de2700 1 -- 172.21.15.116:0/1674192123 >> 172.21.15.116:0/1674192123 conn(0x7f931007ff70 msgr2=0x7f9310080350 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:16.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.371+0000 7f9316de2700 1 -- 172.21.15.116:0/1674192123 shutdown_connections 2020-11-20T12:21:16.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.371+0000 7f9316de2700 1 -- 172.21.15.116:0/1674192123 wait complete. 2020-11-20T12:21:16.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f9316de2700 1 Processor -- start 2020-11-20T12:21:16.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f9316de2700 1 -- start start 2020-11-20T12:21:16.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f9316de2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93100bca90 0x7f93100be3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:16.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f9316de2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93100bcef0 0x7f931014ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:16.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f9316de2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100c3610 0x7f931014d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:16.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f9316de2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93100be8d0 con 0x7f93100c3610 2020-11-20T12:21:16.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f9316de2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93100bf110 con 0x7f93100bcef0 2020-11-20T12:21:16.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f9316de2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93100bf3b0 con 0x7f93100bca90 2020-11-20T12:21:16.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f93165e1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100c3610 0x7f931014d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:16.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f93165e1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100c3610 0x7f931014d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50674/0 (socket says 172.21.15.116:50674) 2020-11-20T12:21:16.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f93165e1700 1 -- 172.21.15.116:0/4208527813 learned_addr learned my addr 172.21.15.116:0/4208527813 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:16.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f93165e1700 1 -- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93100bca90 msgr2=0x7f93100be3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f93165e1700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93100bca90 0x7f93100be3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:16.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f93165e1700 1 -- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93100bcef0 msgr2=0x7f931014ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f93155df700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93100bcef0 0x7f931014ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:16.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f93165e1700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93100bcef0 0x7f931014ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f93165e1700 1 -- 172.21.15.116:0/4208527813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93100bf650 con 0x7f93100c3610 2020-11-20T12:21:16.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f93155df700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93100bcef0 0x7f931014ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:16.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f93165e1700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100c3610 0x7f931014d630 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f9310067fa0 tx=0x7f9308016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:16.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f92fdffb700 1 -- 172.21.15.116:0/4208527813 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308021070 con 0x7f93100c3610 2020-11-20T12:21:16.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f92fdffb700 1 -- 172.21.15.116:0/4208527813 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9308010800 con 0x7f93100c3610 2020-11-20T12:21:16.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f9316de2700 1 -- 172.21.15.116:0/4208527813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93100bf8f0 con 0x7f93100c3610 2020-11-20T12:21:16.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f92fdffb700 1 -- 172.21.15.116:0/4208527813 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308010ba0 con 0x7f93100c3610 2020-11-20T12:21:16.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.375+0000 7f9316de2700 1 -- 172.21.15.116:0/4208527813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93100c00b0 con 0x7f93100c3610 2020-11-20T12:21:16.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.379+0000 7f9316de2700 1 -- 172.21.15.116:0/4208527813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f931005cf30 con 0x7f93100c3610 2020-11-20T12:21:16.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.379+0000 7f92fdffb700 1 -- 172.21.15.116:0/4208527813 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9308017070 con 0x7f93100c3610 2020-11-20T12:21:16.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.379+0000 7f92fdffb700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92e80557c0 0x7f92e8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:16.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.379+0000 7f92fdffb700 1 -- 172.21.15.116:0/4208527813 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f93080706f0 con 0x7f93100c3610 2020-11-20T12:21:16.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.379+0000 7f9315de0700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92e80557c0 0x7f92e8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:16.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.383+0000 7f9315de0700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92e80557c0 0x7f92e8057c80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f930c006fd0 tx=0x7f930c006eb0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:16.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.387+0000 7f92fdffb700 1 -- 172.21.15.116:0/4208527813 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9308046590 con 0x7f93100c3610 2020-11-20T12:21:16.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.547+0000 7f9316de2700 1 -- 172.21.15.116:0/4208527813 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f93100a87f0 con 0x7f93100c3610 2020-11-20T12:21:16.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f92fdffb700 1 -- 172.21.15.116:0/4208527813 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v107) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9308015070 con 0x7f93100c3610 2020-11-20T12:21:16.551 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:21:16.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 -- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92e80557c0 msgr2=0x7f92e8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92e80557c0 0x7f92e8057c80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f930c006fd0 tx=0x7f930c006eb0).stop 2020-11-20T12:21:16.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 -- 172.21.15.116:0/4208527813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100c3610 msgr2=0x7f931014d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100c3610 0x7f931014d630 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f9310067fa0 tx=0x7f9308016000).stop 2020-11-20T12:21:16.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 -- 172.21.15.116:0/4208527813 shutdown_connections 2020-11-20T12:21:16.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f92e80557c0 0x7f92e8057c80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f93100c3610 0x7f931014d630 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f93100bca90 0x7f93100be3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:16.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 --2- 172.21.15.116:0/4208527813 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f93100bcef0 0x7f931014ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 -- 172.21.15.116:0/4208527813 >> 172.21.15.116:0/4208527813 conn(0x7f931007ff70 msgr2=0x7f9310153250 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:16.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 -- 172.21.15.116:0/4208527813 shutdown_connections 2020-11-20T12:21:16.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.551+0000 7f9316de2700 1 -- 172.21.15.116:0/4208527813 wait complete. 2020-11-20T12:21:16.577 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:21:16.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.759+0000 7f25cef8f700 1 Processor -- start 2020-11-20T12:21:16.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.759+0000 7f25cef8f700 1 -- start start 2020-11-20T12:21:16.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.763+0000 7f25cef8f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25c80617c0 0x7f25c80631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:16.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.763+0000 7f25cef8f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c80c2050 con 0x7f25c8149f10 2020-11-20T12:21:16.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.763+0000 7f25cef8f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c80c22f0 con 0x7f25c8061360 2020-11-20T12:21:16.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.763+0000 7f25cef8f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80c2590 con 0x7f25c80617c0 2020-11-20T12:21:16.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.763+0000 7f25cdf8d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f25c8149f10 0x7f25c80ba5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60118/0 (socket says 172.21.15.116:60118) 2020-11-20T12:21:16.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.763+0000 7f25cdf8d700 1 -- 172.21.15.116:0/2699941944 learned_addr learned my addr 172.21.15.116:0/2699941944 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:16.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.763+0000 7f25ce78e700 1 --2- 172.21.15.116:0/2699941944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25c80617c0 0x7f25c80631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:16.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.763+0000 7f25ccf8b700 1 -- 172.21.15.116:0/2699941944 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4010138413 0 0) 0x7f25c80c2050 con 0x7f25c8149f10 2020-11-20T12:21:16.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25ccf8b700 1 -- 172.21.15.116:0/2699941944 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b0003240 con 0x7f25c8149f10 2020-11-20T12:21:16.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25ccf8b700 1 -- 172.21.15.116:0/2699941944 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675699632 0 0) 0x7f25c80c22f0 con 0x7f25c8061360 2020-11-20T12:21:16.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25ccf8b700 1 -- 172.21.15.116:0/2699941944 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25c80c2050 con 0x7f25c8061360 2020-11-20T12:21:16.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25ccf8b700 1 -- 172.21.15.116:0/2699941944 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 92184076 0 0) 0x7f25b0003240 con 0x7f25c8149f10 2020-11-20T12:21:16.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25ce78e700 1 -- 172.21.15.116:0/2699941944 >> v1:172.21.15.173:6789/0 conn(0x7f25c8061360 legacy=0x7f25c8153490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25ce78e700 1 -- 172.21.15.116:0/2699941944 >> v1:172.21.15.116:6789/0 conn(0x7f25c8149f10 legacy=0x7f25c80ba5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25ce78e700 1 -- 172.21.15.116:0/2699941944 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c8060c20 con 0x7f25c80617c0 2020-11-20T12:21:16.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25ce78e700 1 --2- 172.21.15.116:0/2699941944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25c80617c0 0x7f25c80631f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f25c0009cc0 tx=0x7f25c00091d0).ready entity=mon.2 client_cookie=ab2beb98f445abf server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:16.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25ccf8b700 1 -- 172.21.15.116:0/2699941944 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c40014e0 con 0x7f25c80617c0 2020-11-20T12:21:16.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25ccf8b700 1 -- 172.21.15.116:0/2699941944 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25c000b710 con 0x7f25c80617c0 2020-11-20T12:21:16.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25ccf8b700 1 -- 172.21.15.116:0/2699941944 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25c000ba90 con 0x7f25c80617c0 2020-11-20T12:21:16.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25cef8f700 1 -- 172.21.15.116:0/2699941944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25c80617c0 msgr2=0x7f25c80631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25cef8f700 1 --2- 172.21.15.116:0/2699941944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25c80617c0 0x7f25c80631f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f25c0009cc0 tx=0x7f25c00091d0).stop 2020-11-20T12:21:16.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25cef8f700 1 -- 172.21.15.116:0/2699941944 shutdown_connections 2020-11-20T12:21:16.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25cef8f700 1 --2- 172.21.15.116:0/2699941944 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25c80617c0 0x7f25c80631f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25cef8f700 1 -- 172.21.15.116:0/2699941944 >> 172.21.15.116:0/2699941944 conn(0x7f25c80c5740 msgr2=0x7f25c80c1710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:16.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25cef8f700 1 -- 172.21.15.116:0/2699941944 shutdown_connections 2020-11-20T12:21:16.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.767+0000 7f25cef8f700 1 -- 172.21.15.116:0/2699941944 wait complete. 2020-11-20T12:21:16.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.771+0000 7f25cef8f700 1 Processor -- start 2020-11-20T12:21:16.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.771+0000 7f25cef8f700 1 -- start start 2020-11-20T12:21:16.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cef8f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c8061360 0x7f25c8149820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:16.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cef8f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25c80617c0 0x7f25c80bcf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:16.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cef8f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25c8149f10 0x7f25c80c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:16.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cef8f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c8161320 con 0x7f25c80617c0 2020-11-20T12:21:16.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cef8f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c81615c0 con 0x7f25c8061360 2020-11-20T12:21:16.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cef8f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c8161860 con 0x7f25c8149f10 2020-11-20T12:21:16.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cd78c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25c80617c0 0x7f25c80bcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:16.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cd78c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25c80617c0 0x7f25c80bcf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50688/0 (socket says 172.21.15.116:50688) 2020-11-20T12:21:16.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cd78c700 1 -- 172.21.15.116:0/509358199 learned_addr learned my addr 172.21.15.116:0/509358199 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:16.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cdf8d700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c8061360 0x7f25c8149820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:16.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25ce78e700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25c8149f10 0x7f25c80c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:16.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cd78c700 1 -- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25c8149f10 msgr2=0x7f25c80c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cd78c700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25c8149f10 0x7f25c80c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cd78c700 1 -- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c8061360 msgr2=0x7f25c8149820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cd78c700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c8061360 0x7f25c8149820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cd78c700 1 -- 172.21.15.116:0/509358199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c8161b00 con 0x7f25c80617c0 2020-11-20T12:21:16.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cd78c700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25c80617c0 0x7f25c80bcf80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f25b800c6d0 tx=0x7f25b8007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:16.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25b5ffb700 1 -- 172.21.15.116:0/509358199 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b800e560 con 0x7f25c80617c0 2020-11-20T12:21:16.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25b5ffb700 1 -- 172.21.15.116:0/509358199 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b800e6c0 con 0x7f25c80617c0 2020-11-20T12:21:16.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25b5ffb700 1 -- 172.21.15.116:0/509358199 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b800ea60 con 0x7f25c80617c0 2020-11-20T12:21:16.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.775+0000 7f25cef8f700 1 -- 172.21.15.116:0/509358199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c8149d10 con 0x7f25c80617c0 2020-11-20T12:21:16.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.779+0000 7f25cef8f700 1 -- 172.21.15.116:0/509358199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c80bd9e0 con 0x7f25c80617c0 2020-11-20T12:21:16.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.779+0000 7f25cef8f700 1 -- 172.21.15.116:0/509358199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c8161c90 con 0x7f25c80617c0 2020-11-20T12:21:16.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.779+0000 7f25b5ffb700 1 -- 172.21.15.116:0/509358199 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f25b8010070 con 0x7f25c80617c0 2020-11-20T12:21:16.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.779+0000 7f25b5ffb700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f25a00557c0 0x7f25a0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:16.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.779+0000 7f25b5ffb700 1 -- 172.21.15.116:0/509358199 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f25b806e2f0 con 0x7f25c80617c0 2020-11-20T12:21:16.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.783+0000 7f25cdf8d700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f25a00557c0 0x7f25a0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:16.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.783+0000 7f25cdf8d700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f25a00557c0 0x7f25a0057c80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f25c4000bc0 tx=0x7f25c40014a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:16.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.787+0000 7f25b5ffb700 1 -- 172.21.15.116:0/509358199 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f25b8044190 con 0x7f25c80617c0 2020-11-20T12:21:16.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25cef8f700 1 -- 172.21.15.116:0/509358199 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f25c805cf30 con 0x7f25c80617c0 2020-11-20T12:21:16.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25b5ffb700 1 -- 172.21.15.116:0/509358199 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7f25b80477b0 con 0x7f25c80617c0 2020-11-20T12:21:16.949 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:16.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25cef8f700 1 -- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f25a00557c0 msgr2=0x7f25a0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25cef8f700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f25a00557c0 0x7f25a0057c80 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f25c4000bc0 tx=0x7f25c40014a0).stop 2020-11-20T12:21:16.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25cef8f700 1 -- 172.21.15.116:0/509358199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25c80617c0 msgr2=0x7f25c80bcf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:16.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25cef8f700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25c80617c0 0x7f25c80bcf80 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f25b800c6d0 tx=0x7f25b8007ae0).stop 2020-11-20T12:21:16.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25cef8f700 1 -- 172.21.15.116:0/509358199 shutdown_connections 2020-11-20T12:21:16.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25cef8f700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f25a00557c0 0x7f25a0057c80 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25cef8f700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f25c8149f10 0x7f25c80c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25cef8f700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f25c8061360 0x7f25c8149820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25cef8f700 1 --2- 172.21.15.116:0/509358199 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f25c80617c0 0x7f25c80bcf80 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:16.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.947+0000 7f25cef8f700 1 -- 172.21.15.116:0/509358199 >> 172.21.15.116:0/509358199 conn(0x7f25c80c5740 msgr2=0x7f25c80a71f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:16.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.951+0000 7f25cef8f700 1 -- 172.21.15.116:0/509358199 shutdown_connections 2020-11-20T12:21:16.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:16.951+0000 7f25cef8f700 1 -- 172.21.15.116:0/509358199 wait complete. 2020-11-20T12:21:16.973 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:21:17.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.147+0000 7fec709a2700 1 Processor -- start 2020-11-20T12:21:17.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.147+0000 7fec709a2700 1 -- start start 2020-11-20T12:21:17.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.151+0000 7fec709a2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec6c14f880 0x7fec6c0b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.151+0000 7fec709a2700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6c0a8930 con 0x7fec6c0c0e90 2020-11-20T12:21:17.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.151+0000 7fec709a2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec6c0ba2a0 con 0x7fec6c0a81f0 2020-11-20T12:21:17.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.151+0000 7fec709a2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0a8690 con 0x7fec6c14f880 2020-11-20T12:21:17.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.151+0000 7fec6b59e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fec6c0c0e90 0x7fec6c0c1300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60134/0 (socket says 172.21.15.116:60134) 2020-11-20T12:21:17.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.151+0000 7fec6b59e700 1 -- 172.21.15.116:0/2693497395 learned_addr learned my addr 172.21.15.116:0/2693497395 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:17.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.151+0000 7fec6ad9d700 1 --2- 172.21.15.116:0/2693497395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec6c14f880 0x7fec6c0b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.151+0000 7fec69d9b700 1 -- 172.21.15.116:0/2693497395 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3286008369 0 0) 0x7fec6c0a8930 con 0x7fec6c0c0e90 2020-11-20T12:21:17.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.151+0000 7fec69d9b700 1 -- 172.21.15.116:0/2693497395 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec58003240 con 0x7fec6c0c0e90 2020-11-20T12:21:17.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec6ad9d700 1 -- 172.21.15.116:0/2693497395 >> v1:172.21.15.173:6789/0 conn(0x7fec6c0a81f0 legacy=0x7fec6c060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec6ad9d700 1 -- 172.21.15.116:0/2693497395 >> v1:172.21.15.116:6789/0 conn(0x7fec6c0c0e90 legacy=0x7fec6c0c1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec6ad9d700 1 -- 172.21.15.116:0/2693497395 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0b94d0 con 0x7fec6c14f880 2020-11-20T12:21:17.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec6ad9d700 1 --2- 172.21.15.116:0/2693497395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec6c14f880 0x7fec6c0b9db0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fec5c00b9d0 tx=0x7fec5c006df0).ready entity=mon.2 client_cookie=e19efea694488932 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:17.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec69d9b700 1 -- 172.21.15.116:0/2693497395 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c0097b0 con 0x7fec6c14f880 2020-11-20T12:21:17.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec69d9b700 1 -- 172.21.15.116:0/2693497395 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c009910 con 0x7fec6c14f880 2020-11-20T12:21:17.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec69d9b700 1 -- 172.21.15.116:0/2693497395 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c009c90 con 0x7fec6c14f880 2020-11-20T12:21:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec709a2700 1 -- 172.21.15.116:0/2693497395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec6c14f880 msgr2=0x7fec6c0b9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec709a2700 1 --2- 172.21.15.116:0/2693497395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec6c14f880 0x7fec6c0b9db0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fec5c00b9d0 tx=0x7fec5c006df0).stop 2020-11-20T12:21:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec709a2700 1 -- 172.21.15.116:0/2693497395 shutdown_connections 2020-11-20T12:21:17.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec709a2700 1 --2- 172.21.15.116:0/2693497395 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec6c14f880 0x7fec6c0b9db0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec709a2700 1 -- 172.21.15.116:0/2693497395 >> 172.21.15.116:0/2693497395 conn(0x7fec6c068350 msgr2=0x7fec6c0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:17.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec709a2700 1 -- 172.21.15.116:0/2693497395 shutdown_connections 2020-11-20T12:21:17.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.155+0000 7fec709a2700 1 -- 172.21.15.116:0/2693497395 wait complete. 2020-11-20T12:21:17.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec709a2700 1 Processor -- start 2020-11-20T12:21:17.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec709a2700 1 -- start start 2020-11-20T12:21:17.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec709a2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec6c0a81f0 0x7fec6c140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec709a2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec6c0c0e90 0x7fec6c151c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec709a2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec6c14f880 0x7fec6c152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec709a2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0bec00 con 0x7fec6c14f880 2020-11-20T12:21:17.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec709a2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0beea0 con 0x7fec6c0a81f0 2020-11-20T12:21:17.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec709a2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec6c0bf140 con 0x7fec6c0c0e90 2020-11-20T12:21:17.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec6b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec6c14f880 0x7fec6c152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec6b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec6c14f880 0x7fec6c152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50706/0 (socket says 172.21.15.116:50706) 2020-11-20T12:21:17.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec6b59e700 1 -- 172.21.15.116:0/3631868208 learned_addr learned my addr 172.21.15.116:0/3631868208 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:17.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec6ad9d700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec6c0a81f0 0x7fec6c140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.159+0000 7fec6a59c700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec6c0c0e90 0x7fec6c151c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec6b59e700 1 -- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec6c0c0e90 msgr2=0x7fec6c151c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec6b59e700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec6c0c0e90 0x7fec6c151c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec6b59e700 1 -- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec6c0a81f0 msgr2=0x7fec6c140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec6b59e700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec6c0a81f0 0x7fec6c140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec6b59e700 1 -- 172.21.15.116:0/3631868208 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c0bf3e0 con 0x7fec6c14f880 2020-11-20T12:21:17.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec6ad9d700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec6c0a81f0 0x7fec6c140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec6b59e700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec6c14f880 0x7fec6c152160 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7fec64001420 tx=0x7fec64007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec52ffd700 1 -- 172.21.15.116:0/3631868208 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64005b80 con 0x7fec6c14f880 2020-11-20T12:21:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec52ffd700 1 -- 172.21.15.116:0/3631868208 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec64005ce0 con 0x7fec6c14f880 2020-11-20T12:21:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec709a2700 1 -- 172.21.15.116:0/3631868208 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6c0bf680 con 0x7fec6c14f880 2020-11-20T12:21:17.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec52ffd700 1 -- 172.21.15.116:0/3631868208 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64007320 con 0x7fec6c14f880 2020-11-20T12:21:17.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec709a2700 1 -- 172.21.15.116:0/3631868208 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec6c141aa0 con 0x7fec6c14f880 2020-11-20T12:21:17.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec709a2700 1 -- 172.21.15.116:0/3631868208 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec6c0bf810 con 0x7fec6c14f880 2020-11-20T12:21:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.163+0000 7fec52ffd700 1 -- 172.21.15.116:0/3631868208 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fec64010030 con 0x7fec6c14f880 2020-11-20T12:21:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.167+0000 7fec52ffd700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec480557c0 0x7fec48057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.167+0000 7fec52ffd700 1 -- 172.21.15.116:0/3631868208 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec6406e700 con 0x7fec6c14f880 2020-11-20T12:21:17.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.167+0000 7fec6ad9d700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec480557c0 0x7fec48057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.167+0000 7fec6ad9d700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec480557c0 0x7fec48057c80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fec5c00b920 tx=0x7fec5c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:17.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.171+0000 7fec52ffd700 1 -- 172.21.15.116:0/3631868208 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fec640445a0 con 0x7fec6c14f880 2020-11-20T12:21:17.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.331+0000 7fec709a2700 1 -- 172.21.15.116:0/3631868208 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fec6c05cf30 con 0x7fec6c14f880 2020-11-20T12:21:17.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.331+0000 7fec52ffd700 1 -- 172.21.15.116:0/3631868208 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+10 (secure 0 0 0) 0x7fec64047bc0 con 0x7fec6c14f880 2020-11-20T12:21:17.333 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:17.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.331+0000 7fec709a2700 1 -- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec480557c0 msgr2=0x7fec48057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.331+0000 7fec709a2700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec480557c0 0x7fec48057c80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fec5c00b920 tx=0x7fec5c015000).stop 2020-11-20T12:21:17.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.335+0000 7fec709a2700 1 -- 172.21.15.116:0/3631868208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec6c14f880 msgr2=0x7fec6c152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.335+0000 7fec709a2700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec6c14f880 0x7fec6c152160 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7fec64001420 tx=0x7fec64007e20).stop 2020-11-20T12:21:17.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.335+0000 7fec709a2700 1 -- 172.21.15.116:0/3631868208 shutdown_connections 2020-11-20T12:21:17.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.335+0000 7fec709a2700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec480557c0 0x7fec48057c80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.335+0000 7fec709a2700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec6c14f880 0x7fec6c152160 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.335+0000 7fec709a2700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec6c0a81f0 0x7fec6c140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.335+0000 7fec709a2700 1 --2- 172.21.15.116:0/3631868208 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec6c0c0e90 0x7fec6c151c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.335+0000 7fec709a2700 1 -- 172.21.15.116:0/3631868208 >> 172.21.15.116:0/3631868208 conn(0x7fec6c068350 msgr2=0x7fec6c14d870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.335+0000 7fec709a2700 1 -- 172.21.15.116:0/3631868208 shutdown_connections 2020-11-20T12:21:17.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.335+0000 7fec709a2700 1 -- 172.21.15.116:0/3631868208 wait complete. 2020-11-20T12:21:17.360 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:21:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.547+0000 7fc1317cf700 1 Processor -- start 2020-11-20T12:21:17.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.551+0000 7fc1317cf700 1 -- start start 2020-11-20T12:21:17.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.551+0000 7fc1317cf700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c0bd880 0x7fc12c0c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.551+0000 7fc1317cf700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc12c151b30 0x7fc12c0c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc1317cf700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc12c155850 con 0x7fc12c151750 2020-11-20T12:21:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc1317cf700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc12c0c4280 con 0x7fc12c151b30 2020-11-20T12:21:17.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc1317cf700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c155af0 con 0x7fc12c0bd880 2020-11-20T12:21:17.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc12affd700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fc12c151750 0x7fc12c0c5230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:56374/0 (socket says 172.21.15.116:56374) 2020-11-20T12:21:17.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc12affd700 1 -- 172.21.15.116:0/294758167 learned_addr learned my addr 172.21.15.116:0/294758167 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:17.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc12a7fc700 1 --2- 172.21.15.116:0/294758167 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c0bd880 0x7fc12c0c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc12b7fe700 1 --2- 172.21.15.116:0/294758167 >> v2:172.21.15.173:3300/0 conn(0x7fc12c151b30 0x7fc12c0c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc129ffb700 1 -- 172.21.15.116:0/294758167 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740492289 0 0) 0x7fc12c155850 con 0x7fc12c151750 2020-11-20T12:21:17.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc129ffb700 1 -- 172.21.15.116:0/294758167 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc118003240 con 0x7fc12c151750 2020-11-20T12:21:17.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc12a7fc700 1 -- 172.21.15.116:0/294758167 >> v2:172.21.15.173:3300/0 conn(0x7fc12c151b30 msgr2=0x7fc12c0c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc12a7fc700 1 --2- 172.21.15.116:0/294758167 >> v2:172.21.15.173:3300/0 conn(0x7fc12c151b30 0x7fc12c0c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc12a7fc700 1 -- 172.21.15.116:0/294758167 >> v1:172.21.15.173:6789/0 conn(0x7fc12c151750 legacy=0x7fc12c0c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc12a7fc700 1 -- 172.21.15.116:0/294758167 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c155d90 con 0x7fc12c0bd880 2020-11-20T12:21:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.555+0000 7fc12a7fc700 1 --2- 172.21.15.116:0/294758167 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c0bd880 0x7fc12c0c38a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fc12000b9a0 tx=0x7fc120006df0).ready entity=mon.2 client_cookie=68821f17cb87b6bc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:17.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc129ffb700 1 -- 172.21.15.116:0/294758167 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120009680 con 0x7fc12c0bd880 2020-11-20T12:21:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc129ffb700 1 -- 172.21.15.116:0/294758167 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1200097e0 con 0x7fc12c0bd880 2020-11-20T12:21:17.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc129ffb700 1 -- 172.21.15.116:0/294758167 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120009b60 con 0x7fc12c0bd880 2020-11-20T12:21:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc1317cf700 1 -- 172.21.15.116:0/294758167 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c0bd880 msgr2=0x7fc12c0c38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc1317cf700 1 --2- 172.21.15.116:0/294758167 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c0bd880 0x7fc12c0c38a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fc12000b9a0 tx=0x7fc120006df0).stop 2020-11-20T12:21:17.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc1317cf700 1 -- 172.21.15.116:0/294758167 shutdown_connections 2020-11-20T12:21:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc1317cf700 1 --2- 172.21.15.116:0/294758167 >> v2:172.21.15.173:3300/0 conn(0x7fc12c151b30 0x7fc12c0c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc1317cf700 1 --2- 172.21.15.116:0/294758167 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c0bd880 0x7fc12c0c38a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc1317cf700 1 -- 172.21.15.116:0/294758167 >> 172.21.15.116:0/294758167 conn(0x7fc12c0678a0 msgr2=0x7fc12c067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:17.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc1317cf700 1 -- 172.21.15.116:0/294758167 shutdown_connections 2020-11-20T12:21:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc1317cf700 1 -- 172.21.15.116:0/294758167 wait complete. 2020-11-20T12:21:17.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc1317cf700 1 Processor -- start 2020-11-20T12:21:17.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.559+0000 7fc1317cf700 1 -- start start 2020-11-20T12:21:17.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc1317cf700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc12c0bd880 0x7fc12c1646a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc1317cf700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc12c151750 0x7fc12c0ac1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc1317cf700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c151b30 0x7fc12c0acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc1317cf700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c0ac6b0 con 0x7fc12c151750 2020-11-20T12:21:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc1317cf700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c164cd0 con 0x7fc12c0bd880 2020-11-20T12:21:17.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc1317cf700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12c164f70 con 0x7fc12c151b30 2020-11-20T12:21:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12a7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc12c151750 0x7fc12c0ac1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12a7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc12c151750 0x7fc12c0ac1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50720/0 (socket says 172.21.15.116:50720) 2020-11-20T12:21:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12a7fc700 1 -- 172.21.15.116:0/20599640 learned_addr learned my addr 172.21.15.116:0/20599640 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12b7fe700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c151b30 0x7fc12c0acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12affd700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc12c0bd880 0x7fc12c1646a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12a7fc700 1 -- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c151b30 msgr2=0x7fc12c0acb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12a7fc700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c151b30 0x7fc12c0acb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12a7fc700 1 -- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc12c0bd880 msgr2=0x7fc12c1646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12a7fc700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc12c0bd880 0x7fc12c1646a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12a7fc700 1 -- 172.21.15.116:0/20599640 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc12c165210 con 0x7fc12c151750 2020-11-20T12:21:17.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12b7fe700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c151b30 0x7fc12c0acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:17.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12affd700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc12c0bd880 0x7fc12c1646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:17.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc12a7fc700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc12c151750 0x7fc12c0ac1c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7fc12c1530f0 tx=0x7fc120015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:17.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc112ffd700 1 -- 172.21.15.116:0/20599640 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120011c40 con 0x7fc12c151750 2020-11-20T12:21:17.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc112ffd700 1 -- 172.21.15.116:0/20599640 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc120011da0 con 0x7fc12c151750 2020-11-20T12:21:17.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc112ffd700 1 -- 172.21.15.116:0/20599640 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12001e670 con 0x7fc12c151750 2020-11-20T12:21:17.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc1317cf700 1 -- 172.21.15.116:0/20599640 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12c1654b0 con 0x7fc12c151750 2020-11-20T12:21:17.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.563+0000 7fc1317cf700 1 -- 172.21.15.116:0/20599640 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12c0ad060 con 0x7fc12c151750 2020-11-20T12:21:17.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.567+0000 7fc1317cf700 1 -- 172.21.15.116:0/20599640 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc12c1612d0 con 0x7fc12c151750 2020-11-20T12:21:17.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.567+0000 7fc112ffd700 1 -- 172.21.15.116:0/20599640 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc120025030 con 0x7fc12c151750 2020-11-20T12:21:17.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.567+0000 7fc112ffd700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1080557c0 0x7fc108057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.567+0000 7fc112ffd700 1 -- 172.21.15.116:0/20599640 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc12006fbe0 con 0x7fc12c151750 2020-11-20T12:21:17.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.567+0000 7fc12affd700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1080557c0 0x7fc108057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.567+0000 7fc12affd700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1080557c0 0x7fc108057c80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fc11c0055a0 tx=0x7fc11c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:17.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.571+0000 7fc112ffd700 1 -- 172.21.15.116:0/20599640 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc12c1612d0 con 0x7fc12c151750 2020-11-20T12:21:17.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.735+0000 7fc1317cf700 1 -- 172.21.15.116:0/20599640 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc12c1612d0 con 0x7fc12c151750 2020-11-20T12:21:17.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.735+0000 7fc112ffd700 1 -- 172.21.15.116:0/20599640 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+10 (secure 0 0 0) 0x7fc12c1612d0 con 0x7fc12c151750 2020-11-20T12:21:17.739 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:17.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 -- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1080557c0 msgr2=0x7fc108057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1080557c0 0x7fc108057c80 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fc11c0055a0 tx=0x7fc11c009040).stop 2020-11-20T12:21:17.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 -- 172.21.15.116:0/20599640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc12c151750 msgr2=0x7fc12c0ac1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc12c151750 0x7fc12c0ac1c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7fc12c1530f0 tx=0x7fc120015000).stop 2020-11-20T12:21:17.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 -- 172.21.15.116:0/20599640 shutdown_connections 2020-11-20T12:21:17.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc1080557c0 0x7fc108057c80 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc12c151b30 0x7fc12c0acb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc12c0bd880 0x7fc12c1646a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 --2- 172.21.15.116:0/20599640 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc12c151750 0x7fc12c0ac1c0 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 -- 172.21.15.116:0/20599640 >> 172.21.15.116:0/20599640 conn(0x7fc12c0678a0 msgr2=0x7fc12c0c51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:17.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 -- 172.21.15.116:0/20599640 shutdown_connections 2020-11-20T12:21:17.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.739+0000 7fc1317cf700 1 -- 172.21.15.116:0/20599640 wait complete. 2020-11-20T12:21:17.765 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:21:17.766 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:21:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.939+0000 7f984c305700 1 Processor -- start 2020-11-20T12:21:17.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.943+0000 7f984c305700 1 -- start start 2020-11-20T12:21:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.943+0000 7f984c305700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f98440ba890 0x7f98440bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.943+0000 7f984c305700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9844148a20 0x7f9844148e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.943+0000 7f984c305700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9844066140 con 0x7f98440c2230 2020-11-20T12:21:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.943+0000 7f984c305700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9844149380 con 0x7f9844148a20 2020-11-20T12:21:17.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.943+0000 7f984c305700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98440c1d80 con 0x7f98440ba890 2020-11-20T12:21:17.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.943+0000 7f984a8a2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9844148a20 0x7f9844148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.943+0000 7f984a8a2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9844148a20 0x7f9844148e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50728/0 (socket says 172.21.15.116:50728) 2020-11-20T12:21:17.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.943+0000 7f984a8a2700 1 -- 172.21.15.116:0/742880116 learned_addr learned my addr 172.21.15.116:0/742880116 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:17.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.943+0000 7f984a0a1700 1 --2- 172.21.15.116:0/742880116 >> v2:172.21.15.173:3300/0 conn(0x7f98440ba890 0x7f98440bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984a8a2700 1 -- 172.21.15.116:0/742880116 >> v2:172.21.15.173:3300/0 conn(0x7f98440ba890 msgr2=0x7f98440bac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984a8a2700 1 --2- 172.21.15.116:0/742880116 >> v2:172.21.15.173:3300/0 conn(0x7f98440ba890 0x7f98440bac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984a8a2700 1 -- 172.21.15.116:0/742880116 >> v1:172.21.15.173:6789/0 conn(0x7f98440c2230 legacy=0x7f98440c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984a8a2700 1 -- 172.21.15.116:0/742880116 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98441526c0 con 0x7f9844148a20 2020-11-20T12:21:17.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984909f700 1 -- 172.21.15.116:0/742880116 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245955323 0 0) 0x7f9844066140 con 0x7f98440c2230 2020-11-20T12:21:17.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984a8a2700 1 --2- 172.21.15.116:0/742880116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9844148a20 0x7f9844148e90 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f984405c830 tx=0x7f983c009290).ready entity=mon.0 client_cookie=a4645de3e1114413 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:17.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984909f700 1 -- 172.21.15.116:0/742880116 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c00bbc0 con 0x7f9844148a20 2020-11-20T12:21:17.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984909f700 1 -- 172.21.15.116:0/742880116 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983c00bd20 con 0x7f9844148a20 2020-11-20T12:21:17.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984909f700 1 -- 172.21.15.116:0/742880116 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c011610 con 0x7f9844148a20 2020-11-20T12:21:17.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984c305700 1 -- 172.21.15.116:0/742880116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9844148a20 msgr2=0x7f9844148e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984c305700 1 --2- 172.21.15.116:0/742880116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9844148a20 0x7f9844148e90 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f984405c830 tx=0x7f983c009290).stop 2020-11-20T12:21:17.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984c305700 1 -- 172.21.15.116:0/742880116 shutdown_connections 2020-11-20T12:21:17.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984c305700 1 --2- 172.21.15.116:0/742880116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9844148a20 0x7f9844148e90 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984c305700 1 --2- 172.21.15.116:0/742880116 >> v2:172.21.15.173:3300/0 conn(0x7f98440ba890 0x7f98440bac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984c305700 1 -- 172.21.15.116:0/742880116 >> 172.21.15.116:0/742880116 conn(0x7f98440c4f80 msgr2=0x7f98440c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:17.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984c305700 1 -- 172.21.15.116:0/742880116 shutdown_connections 2020-11-20T12:21:17.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.947+0000 7f984c305700 1 -- 172.21.15.116:0/742880116 wait complete. 2020-11-20T12:21:17.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984c305700 1 Processor -- start 2020-11-20T12:21:17.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984c305700 1 -- start start 2020-11-20T12:21:17.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984c305700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98440ba890 0x7f9844142080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984c305700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98440c2230 0x7f9844164f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984c305700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9844148a20 0x7f98440c04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984c305700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984414ef90 con 0x7f98440c2230 2020-11-20T12:21:17.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984c305700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984414f230 con 0x7f9844148a20 2020-11-20T12:21:17.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984c305700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f984414f4d0 con 0x7f98440ba890 2020-11-20T12:21:17.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f98498a0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98440c2230 0x7f9844164f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f98498a0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98440c2230 0x7f9844164f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50732/0 (socket says 172.21.15.116:50732) 2020-11-20T12:21:17.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f98498a0700 1 -- 172.21.15.116:0/1244669661 learned_addr learned my addr 172.21.15.116:0/1244669661 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:17.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984a0a1700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98440ba890 0x7f9844142080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984a8a2700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9844148a20 0x7f98440c04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f98498a0700 1 -- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98440ba890 msgr2=0x7f9844142080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f98498a0700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98440ba890 0x7f9844142080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f98498a0700 1 -- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9844148a20 msgr2=0x7f98440c04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:17.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f98498a0700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9844148a20 0x7f98440c04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:17.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f98498a0700 1 -- 172.21.15.116:0/1244669661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984414f770 con 0x7f98440c2230 2020-11-20T12:21:17.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984a0a1700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98440ba890 0x7f9844142080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:17.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f984a8a2700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9844148a20 0x7f98440c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:17.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.951+0000 7f98498a0700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98440c2230 0x7f9844164f10 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f983400c730 tx=0x7f9834007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:17.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.955+0000 7f98327fc700 1 -- 172.21.15.116:0/1244669661 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400e560 con 0x7f98440c2230 2020-11-20T12:21:17.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.955+0000 7f98327fc700 1 -- 172.21.15.116:0/1244669661 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f983400e6e0 con 0x7f98440c2230 2020-11-20T12:21:17.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.955+0000 7f984c305700 1 -- 172.21.15.116:0/1244669661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984414fa10 con 0x7f98440c2230 2020-11-20T12:21:17.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.955+0000 7f98327fc700 1 -- 172.21.15.116:0/1244669661 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983400eaa0 con 0x7f98440c2230 2020-11-20T12:21:17.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.955+0000 7f984c305700 1 -- 172.21.15.116:0/1244669661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98441648d0 con 0x7f98440c2230 2020-11-20T12:21:17.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.955+0000 7f984c305700 1 -- 172.21.15.116:0/1244669661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984414fba0 con 0x7f98440c2230 2020-11-20T12:21:17.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.955+0000 7f98327fc700 1 -- 172.21.15.116:0/1244669661 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9834019030 con 0x7f98440c2230 2020-11-20T12:21:17.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.959+0000 7f98327fc700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f981c0557c0 0x7f981c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:17.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.959+0000 7f98327fc700 1 -- 172.21.15.116:0/1244669661 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f983406dd10 con 0x7f98440c2230 2020-11-20T12:21:17.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.959+0000 7f984a0a1700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f981c0557c0 0x7f981c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:17.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.959+0000 7f984a0a1700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f981c0557c0 0x7f981c057c80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f984414e690 tx=0x7f9840008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:17.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:17.963+0000 7f98327fc700 1 -- 172.21.15.116:0/1244669661 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9834043bb0 con 0x7f98440c2230 2020-11-20T12:21:18.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.171+0000 7f984c305700 1 -- 172.21.15.116:0/1244669661 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f984405d2a0 con 0x7f98440c2230 2020-11-20T12:21:18.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.171+0000 7f98327fc700 1 -- 172.21.15.116:0/1244669661 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2851 (secure 0 0 0) 0x7f98340471d0 con 0x7f98440c2230 2020-11-20T12:21:18.174 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:18.174 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1638614527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1638614527},{"type":"v1","addr":"172.21.15.173:6837","nonce":1638614527}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3682335131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3682335131},{"type":"v1","addr":"172.21.15.173:6836","nonce":3682335131}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7725,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/3975405805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3975405805},{"type":"v1","addr":"172.21.15.116:6834","nonce":3975405805}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:11.109567+0000","modified":"2020-11-20T12:21:13.118616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/267899152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":267899152},{"type":"v1","addr":"172.21.15.116:6835","nonce":267899152}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-20T12:21:18.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 -- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f981c0557c0 msgr2=0x7f981c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:18.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f981c0557c0 0x7f981c057c80 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f984414e690 tx=0x7f9840008040).stop 2020-11-20T12:21:18.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 -- 172.21.15.116:0/1244669661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98440c2230 msgr2=0x7f9844164f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:18.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98440c2230 0x7f9844164f10 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f983400c730 tx=0x7f9834007a60).stop 2020-11-20T12:21:18.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 -- 172.21.15.116:0/1244669661 shutdown_connections 2020-11-20T12:21:18.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f981c0557c0 0x7f981c057c80 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:18.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9844148a20 0x7f98440c04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:18.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98440ba890 0x7f9844142080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:18.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 --2- 172.21.15.116:0/1244669661 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98440c2230 0x7f9844164f10 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:18.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 -- 172.21.15.116:0/1244669661 >> 172.21.15.116:0/1244669661 conn(0x7f98440c4f80 msgr2=0x7f98440c1020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:18.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 -- 172.21.15.116:0/1244669661 shutdown_connections 2020-11-20T12:21:18.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:18.175+0000 7f984c305700 1 -- 172.21.15.116:0/1244669661 wait complete. 2020-11-20T12:21:18.179 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 60 2020-11-20T12:21:18.602 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:21:11.109567+0000', 'modified': '2020-11-20T12:21:13.118616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7728': {'gid': 7728, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/267899152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 267899152}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 267899152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:21:18.602 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:21:18.603 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:21:18.797 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":7728,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.6681641279999999,"mdsmap_epoch":60,"osdmap_epoch":107,"osdmap_epoch_barrier":107,"uptime":13.173210864} 2020-11-20T12:21:18.798 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 7728, "mdsmap_epoch": 60, "osdmap_epoch": 107, "osdmap_epoch_barrier": 107, "rank_uptime": 7.668164128, "state": "up:active", "uptime": 13.173210864, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:21:18.798 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:21:18.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:18.799 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-20T12:21:18.816 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:21:18.816 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:18.885 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:18.885 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:21:18.911 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:18.912 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:21:18.919 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:21:18.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:18.920 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-20T12:21:18.945 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:21:18.945 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:21:18.945 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-20T12:21:18.946 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-20T12:21:18.946 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:21:18.946 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:21:18.947 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:21:18.947 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:21:18.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:18.948 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:21:18.964 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:21:18.965 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:21:18.965 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:21:18.965 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:18.965 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:21:18.966 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:18.966 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:18.966 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:18.966 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:18.966 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:18.967 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:18.967 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:18.967 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:21:18.967 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:18.967 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:21:18.968 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:18.968 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:21:18.969 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:18.970 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:21:18.970 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:18.970 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:21:18.970 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:18.970 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:21:18.971 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:18.971 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:21:18.971 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:18.971 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:21:18.971 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:18.972 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:21:18.972 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:21:18.972 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:21:18.972 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:18.972 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:21:18.973 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:21:18.973 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:21:18.973 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:18.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:18.974 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:21:19.043 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:21:19.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:19.048 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:21:19.066 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:21:19.066 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:21:19.066 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:21:19.066 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:21:19.067 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:21:19.067 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:21:19.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:19.068 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:21:19.069 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:21:19.069 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:21:19.070 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:21:19.070 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:21:19.071 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:21:19.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:19.172 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:21:19.189 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:21:19.190 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:21:19.190 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:21:19.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:19.191 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2020-11-20T12:21:19.221 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:19.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.399+0000 7f08d52a8700 1 Processor -- start 2020-11-20T12:21:19.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.399+0000 7f08d52a8700 1 -- start start 2020-11-20T12:21:19.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08d52a8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08d014baa0 0x7f08d014be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:19.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08d52a8700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d0152810 con 0x7f08d0065a20 2020-11-20T12:21:19.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08d52a8700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d0152ab0 con 0x7f08d0065e80 2020-11-20T12:21:19.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08d52a8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d00652e0 con 0x7f08d014baa0 2020-11-20T12:21:19.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08c7fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f08d0065a20 0x7f08d0151d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60180/0 (socket says 172.21.15.116:60180) 2020-11-20T12:21:19.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08c7fff700 1 -- 172.21.15.116:0/3873200993 learned_addr learned my addr 172.21.15.116:0/3873200993 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:19.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08cffff700 1 --2- 172.21.15.116:0/3873200993 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08d014baa0 0x7f08d014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:19.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08cf7fe700 1 -- 172.21.15.116:0/3873200993 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2943829936 0 0) 0x7f08d0152810 con 0x7f08d0065a20 2020-11-20T12:21:19.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08cf7fe700 1 -- 172.21.15.116:0/3873200993 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08b0003240 con 0x7f08d0065a20 2020-11-20T12:21:19.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08cffff700 1 -- 172.21.15.116:0/3873200993 >> v1:172.21.15.173:6789/0 conn(0x7f08d0065e80 legacy=0x7f08d0060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08cffff700 1 -- 172.21.15.116:0/3873200993 >> v1:172.21.15.116:6789/0 conn(0x7f08d0065a20 legacy=0x7f08d0151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08cffff700 1 -- 172.21.15.116:0/3873200993 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d0065580 con 0x7f08d014baa0 2020-11-20T12:21:19.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08cffff700 1 --2- 172.21.15.116:0/3873200993 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08d014baa0 0x7f08d014be80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f08c000b9d0 tx=0x7f08c0006df0).ready entity=mon.2 client_cookie=8eeddf3589fd441e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:19.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08cf7fe700 1 -- 172.21.15.116:0/3873200993 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c00097b0 con 0x7f08d014baa0 2020-11-20T12:21:19.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08cf7fe700 1 -- 172.21.15.116:0/3873200993 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c0009910 con 0x7f08d014baa0 2020-11-20T12:21:19.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.403+0000 7f08cf7fe700 1 -- 172.21.15.116:0/3873200993 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c0009c90 con 0x7f08d014baa0 2020-11-20T12:21:19.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.407+0000 7f08d52a8700 1 -- 172.21.15.116:0/3873200993 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08d014baa0 msgr2=0x7f08d014be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.407+0000 7f08d52a8700 1 --2- 172.21.15.116:0/3873200993 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08d014baa0 0x7f08d014be80 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f08c000b9d0 tx=0x7f08c0006df0).stop 2020-11-20T12:21:19.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.407+0000 7f08d52a8700 1 -- 172.21.15.116:0/3873200993 shutdown_connections 2020-11-20T12:21:19.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.407+0000 7f08d52a8700 1 --2- 172.21.15.116:0/3873200993 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08d014baa0 0x7f08d014be80 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.407+0000 7f08d52a8700 1 -- 172.21.15.116:0/3873200993 >> 172.21.15.116:0/3873200993 conn(0x7f08d0061280 msgr2=0x7f08d0061660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:19.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.407+0000 7f08d52a8700 1 -- 172.21.15.116:0/3873200993 shutdown_connections 2020-11-20T12:21:19.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.407+0000 7f08d52a8700 1 -- 172.21.15.116:0/3873200993 wait complete. 2020-11-20T12:21:19.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.407+0000 7f08d52a8700 1 Processor -- start 2020-11-20T12:21:19.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.407+0000 7f08d52a8700 1 -- start start 2020-11-20T12:21:19.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d52a8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08d0065a20 0x7f08d0162180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:19.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d52a8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08d0065e80 0x7f08d0147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:19.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d52a8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08d0164d00 0x7f08d0148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:19.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d52a8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d0165280 con 0x7f08d0164d00 2020-11-20T12:21:19.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d52a8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d0165520 con 0x7f08d0065e80 2020-11-20T12:21:19.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d52a8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d01657c0 con 0x7f08d0065a20 2020-11-20T12:21:19.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d4aa7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08d0164d00 0x7f08d0148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:19.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d4aa7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08d0164d00 0x7f08d0148ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50752/0 (socket says 172.21.15.116:50752) 2020-11-20T12:21:19.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d4aa7700 1 -- 172.21.15.116:0/543978898 learned_addr learned my addr 172.21.15.116:0/543978898 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:19.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08c7fff700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08d0065e80 0x7f08d0147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:19.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08cffff700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08d0065a20 0x7f08d0162180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:19.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d4aa7700 1 -- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08d0065a20 msgr2=0x7f08d0162180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d4aa7700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08d0065a20 0x7f08d0162180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d4aa7700 1 -- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08d0065e80 msgr2=0x7f08d0147d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d4aa7700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08d0065e80 0x7f08d0147d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d4aa7700 1 -- 172.21.15.116:0/543978898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d0165a60 con 0x7f08d0164d00 2020-11-20T12:21:19.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08c7fff700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08d0065e80 0x7f08d0147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:19.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d4aa7700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08d0164d00 0x7f08d0148ea0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f08c800ca90 tx=0x7f08c8007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:19.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08ccff9700 1 -- 172.21.15.116:0/543978898 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800a730 con 0x7f08d0164d00 2020-11-20T12:21:19.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08ccff9700 1 -- 172.21.15.116:0/543978898 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c800a890 con 0x7f08d0164d00 2020-11-20T12:21:19.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08ccff9700 1 -- 172.21.15.116:0/543978898 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c800ac10 con 0x7f08d0164d00 2020-11-20T12:21:19.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.411+0000 7f08d52a8700 1 -- 172.21.15.116:0/543978898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d0162670 con 0x7f08d0164d00 2020-11-20T12:21:19.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.415+0000 7f08d52a8700 1 -- 172.21.15.116:0/543978898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d0149390 con 0x7f08d0164d00 2020-11-20T12:21:19.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.415+0000 7f08d52a8700 1 -- 172.21.15.116:0/543978898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d0165bf0 con 0x7f08d0164d00 2020-11-20T12:21:19.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.415+0000 7f08ccff9700 1 -- 172.21.15.116:0/543978898 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f08c8019030 con 0x7f08d0164d00 2020-11-20T12:21:19.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.415+0000 7f08ccff9700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08ac0557c0 0x7f08ac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:19.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.415+0000 7f08ccff9700 1 -- 172.21.15.116:0/543978898 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f08c806de30 con 0x7f08d0164d00 2020-11-20T12:21:19.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.419+0000 7f08cffff700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08ac0557c0 0x7f08ac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:19.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.419+0000 7f08cffff700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08ac0557c0 0x7f08ac057c80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f08d00ab630 tx=0x7f08c0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:19.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.423+0000 7f08ccff9700 1 -- 172.21.15.116:0/543978898 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f08c8043d80 con 0x7f08d0164d00 2020-11-20T12:21:19.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.587+0000 7f08d52a8700 1 -- 172.21.15.116:0/543978898 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08d005cf30 con 0x7f08d0164d00 2020-11-20T12:21:19.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08ccff9700 1 -- 172.21.15.116:0/543978898 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f08c8025210 con 0x7f08d0164d00 2020-11-20T12:21:19.592 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:19.593 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":107,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:12.114283+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:09.571842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:10.034013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:19.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 -- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08ac0557c0 msgr2=0x7f08ac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08ac0557c0 0x7f08ac057c80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f08d00ab630 tx=0x7f08c0015000).stop 2020-11-20T12:21:19.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 -- 172.21.15.116:0/543978898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08d0164d00 msgr2=0x7f08d0148ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08d0164d00 0x7f08d0148ea0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f08c800ca90 tx=0x7f08c8007f70).stop 2020-11-20T12:21:19.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 -- 172.21.15.116:0/543978898 shutdown_connections 2020-11-20T12:21:19.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f08ac0557c0 0x7f08ac057c80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f08d0164d00 0x7f08d0148ea0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f08d0065a20 0x7f08d0162180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 --2- 172.21.15.116:0/543978898 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f08d0065e80 0x7f08d0147d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 -- 172.21.15.116:0/543978898 >> 172.21.15.116:0/543978898 conn(0x7f08d0061280 msgr2=0x7f08d0149cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:19.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 -- 172.21.15.116:0/543978898 shutdown_connections 2020-11-20T12:21:19.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.591+0000 7f08d52a8700 1 -- 172.21.15.116:0/543978898 wait complete. 2020-11-20T12:21:19.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:21:09.571842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:21:10.034013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:21:19.620 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:21:19.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.799+0000 7fad3f668700 1 Processor -- start 2020-11-20T12:21:19.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.799+0000 7fad3f668700 1 -- start start 2020-11-20T12:21:19.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3f668700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fad381518c0 0x7fad38151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:19.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3f668700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fad380c9010 0x7fad38061550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:19.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3f668700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad380a92f0 con 0x7fad38061a40 2020-11-20T12:21:19.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3f668700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad380c9450 con 0x7fad381518c0 2020-11-20T12:21:19.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3f668700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad380a9590 con 0x7fad380c9010 2020-11-20T12:21:19.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3d404700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fad381518c0 0x7fad38151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:19.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3cc03700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fad380c9010 0x7fad38061550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:19.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3dc05700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fad38061a40 0x7fad380ad010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:56424/0 (socket says 172.21.15.116:56424) 2020-11-20T12:21:19.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3dc05700 1 -- 172.21.15.116:0/2445590185 learned_addr learned my addr 172.21.15.116:0/2445590185 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:19.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3d404700 1 -- 172.21.15.116:0/2445590185 >> v2:172.21.15.173:3300/0 conn(0x7fad380c9010 msgr2=0x7fad38061550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:19.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3d404700 1 --2- 172.21.15.116:0/2445590185 >> v2:172.21.15.173:3300/0 conn(0x7fad380c9010 0x7fad38061550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3d404700 1 -- 172.21.15.116:0/2445590185 >> v1:172.21.15.173:6789/0 conn(0x7fad38061a40 legacy=0x7fad380ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3d404700 1 -- 172.21.15.116:0/2445590185 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad380a9830 con 0x7fad381518c0 2020-11-20T12:21:19.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.803+0000 7fad3d404700 1 --2- 172.21.15.116:0/2445590185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad381518c0 0x7fad38151ca0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7fad3400b990 tx=0x7fad34006df0).ready entity=mon.0 client_cookie=f992f9de5a5ce7f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:19.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.807+0000 7fad27fff700 1 -- 172.21.15.116:0/2445590185 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34006e60 con 0x7fad381518c0 2020-11-20T12:21:19.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.807+0000 7fad27fff700 1 -- 172.21.15.116:0/2445590185 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad34009770 con 0x7fad381518c0 2020-11-20T12:21:19.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.807+0000 7fad27fff700 1 -- 172.21.15.116:0/2445590185 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad34009af0 con 0x7fad381518c0 2020-11-20T12:21:19.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.807+0000 7fad3f668700 1 -- 172.21.15.116:0/2445590185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad381518c0 msgr2=0x7fad38151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.807+0000 7fad3f668700 1 --2- 172.21.15.116:0/2445590185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad381518c0 0x7fad38151ca0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7fad3400b990 tx=0x7fad34006df0).stop 2020-11-20T12:21:19.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.807+0000 7fad3f668700 1 -- 172.21.15.116:0/2445590185 shutdown_connections 2020-11-20T12:21:19.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.807+0000 7fad3f668700 1 --2- 172.21.15.116:0/2445590185 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad381518c0 0x7fad38151ca0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.807+0000 7fad3f668700 1 --2- 172.21.15.116:0/2445590185 >> v2:172.21.15.173:3300/0 conn(0x7fad380c9010 0x7fad38061550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.807+0000 7fad3f668700 1 -- 172.21.15.116:0/2445590185 >> 172.21.15.116:0/2445590185 conn(0x7fad38068370 msgr2=0x7fad380c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:19.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.807+0000 7fad3f668700 1 -- 172.21.15.116:0/2445590185 shutdown_connections 2020-11-20T12:21:19.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.807+0000 7fad3f668700 1 -- 172.21.15.116:0/2445590185 wait complete. 2020-11-20T12:21:19.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3f668700 1 Processor -- start 2020-11-20T12:21:19.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3f668700 1 -- start start 2020-11-20T12:21:19.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3f668700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad38061a40 0x7fad380bd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:19.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3f668700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad380c9010 0x7fad380a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:19.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3f668700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad381518c0 0x7fad380c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:19.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3f668700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380c0050 con 0x7fad381518c0 2020-11-20T12:21:19.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3f668700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad380c0280 con 0x7fad380c9010 2020-11-20T12:21:19.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3f668700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad380c0520 con 0x7fad38061a40 2020-11-20T12:21:19.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3dc05700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad381518c0 0x7fad380c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:19.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3dc05700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad381518c0 0x7fad380c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50766/0 (socket says 172.21.15.116:50766) 2020-11-20T12:21:19.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3dc05700 1 -- 172.21.15.116:0/2443584488 learned_addr learned my addr 172.21.15.116:0/2443584488 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:19.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3cc03700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad380c9010 0x7fad380a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:19.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.811+0000 7fad3d404700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad38061a40 0x7fad380bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:19.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad3cc03700 1 -- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad38061a40 msgr2=0x7fad380bd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad3cc03700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad38061a40 0x7fad380bd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad3cc03700 1 -- 172.21.15.116:0/2443584488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad381518c0 msgr2=0x7fad380c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad3cc03700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad381518c0 0x7fad380c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad3cc03700 1 -- 172.21.15.116:0/2443584488 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad380c07c0 con 0x7fad380c9010 2020-11-20T12:21:19.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad3dc05700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad381518c0 0x7fad380c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:19.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad3cc03700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad380c9010 0x7fad380a8ac0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fad2800eba0 tx=0x7fad28006df0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:19.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad257fa700 1 -- 172.21.15.116:0/2443584488 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2800b810 con 0x7fad380c9010 2020-11-20T12:21:19.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad257fa700 1 -- 172.21.15.116:0/2443584488 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2800b970 con 0x7fad380c9010 2020-11-20T12:21:19.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad3f668700 1 -- 172.21.15.116:0/2443584488 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad380bdd50 con 0x7fad380c9010 2020-11-20T12:21:19.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad257fa700 1 -- 172.21.15.116:0/2443584488 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2800bcf0 con 0x7fad380c9010 2020-11-20T12:21:19.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad3f668700 1 -- 172.21.15.116:0/2443584488 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad380a8fb0 con 0x7fad380c9010 2020-11-20T12:21:19.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.815+0000 7fad3f668700 1 -- 172.21.15.116:0/2443584488 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad38069950 con 0x7fad380c9010 2020-11-20T12:21:19.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.819+0000 7fad257fa700 1 -- 172.21.15.116:0/2443584488 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fad28033750 con 0x7fad380c9010 2020-11-20T12:21:19.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.819+0000 7fad257fa700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad100557c0 0x7fad10057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:19.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.819+0000 7fad257fa700 1 -- 172.21.15.116:0/2443584488 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fad2806eb40 con 0x7fad380c9010 2020-11-20T12:21:19.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.819+0000 7fad3d404700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad100557c0 0x7fad10057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:19.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.823+0000 7fad3d404700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad100557c0 0x7fad10057c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fad3400be50 tx=0x7fad34015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:19.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.827+0000 7fad257fa700 1 -- 172.21.15.116:0/2443584488 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fad280449e0 con 0x7fad380c9010 2020-11-20T12:21:19.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.987+0000 7fad3f668700 1 -- 172.21.15.116:0/2443584488 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fad380608c0 con 0x7fad380c9010 2020-11-20T12:21:19.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad257fa700 1 -- 172.21.15.116:0/2443584488 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v107) v1 ==== 110+0+10 (secure 0 0 0) 0x7fad28048000 con 0x7fad380c9010 2020-11-20T12:21:19.992 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:21:19.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 -- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad100557c0 msgr2=0x7fad10057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad100557c0 0x7fad10057c80 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fad3400be50 tx=0x7fad34015000).stop 2020-11-20T12:21:19.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 -- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad380c9010 msgr2=0x7fad380a8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:19.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad380c9010 0x7fad380a8ac0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7fad2800eba0 tx=0x7fad28006df0).stop 2020-11-20T12:21:19.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 -- 172.21.15.116:0/2443584488 shutdown_connections 2020-11-20T12:21:19.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad100557c0 0x7fad10057c80 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad381518c0 0x7fad380c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:19.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad38061a40 0x7fad380bd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 --2- 172.21.15.116:0/2443584488 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad380c9010 0x7fad380a8ac0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 -- 172.21.15.116:0/2443584488 >> 172.21.15.116:0/2443584488 conn(0x7fad38068370 msgr2=0x7fad380c7970 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:20.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 -- 172.21.15.116:0/2443584488 shutdown_connections 2020-11-20T12:21:20.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:19.991+0000 7fad3f668700 1 -- 172.21.15.116:0/2443584488 wait complete. 2020-11-20T12:21:20.018 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:21:20.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.183+0000 7f68a95aa700 1 Processor -- start 2020-11-20T12:21:20.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.183+0000 7f68a95aa700 1 -- start start 2020-11-20T12:21:20.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a95aa700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f68a40aa7d0 0x7f68a40a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a95aa700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a40ad540 0x7f68a40ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a95aa700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a40ace40 con 0x7f68a40aabb0 2020-11-20T12:21:20.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a95aa700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a41526c0 con 0x7f68a40aa7d0 2020-11-20T12:21:20.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a95aa700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a4152960 con 0x7f68a40ad540 2020-11-20T12:21:20.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a3fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f68a40aa7d0 0x7f68a40a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a8da9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a40ad540 0x7f68a40ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a3fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f68a40aa7d0 0x7f68a40a8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:49824/0 (socket says 172.21.15.116:49824) 2020-11-20T12:21:20.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a3fff700 1 -- 172.21.15.116:0/1415748811 learned_addr learned my addr 172.21.15.116:0/1415748811 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:20.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a37fe700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f68a40aabb0 0x7f68a4151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:56434/0 (socket says 172.21.15.116:56434) 2020-11-20T12:21:20.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a3fff700 1 -- 172.21.15.116:0/1415748811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a40ad540 msgr2=0x7f68a40ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:20.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a3fff700 1 --2- 172.21.15.116:0/1415748811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a40ad540 0x7f68a40ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a3fff700 1 -- 172.21.15.116:0/1415748811 >> v1:172.21.15.173:6789/0 conn(0x7f68a40aabb0 legacy=0x7f68a4151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a3fff700 1 -- 172.21.15.116:0/1415748811 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4152c00 con 0x7f68a40aa7d0 2020-11-20T12:21:20.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a2ffd700 1 -- 172.21.15.116:0/1415748811 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3949493765 0 0) 0x7f68a40ace40 con 0x7f68a40aabb0 2020-11-20T12:21:20.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.187+0000 7f68a3fff700 1 --2- 172.21.15.116:0/1415748811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68a40aa7d0 0x7f68a40a8400 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f68a40acfd0 tx=0x7f6894006eb0).ready entity=mon.1 client_cookie=ea9ddde21a308c1f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:20.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a2ffd700 1 -- 172.21.15.116:0/1415748811 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894009810 con 0x7f68a40aa7d0 2020-11-20T12:21:20.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a2ffd700 1 -- 172.21.15.116:0/1415748811 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6894009970 con 0x7f68a40aa7d0 2020-11-20T12:21:20.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a2ffd700 1 -- 172.21.15.116:0/1415748811 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894009cf0 con 0x7f68a40aa7d0 2020-11-20T12:21:20.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a95aa700 1 -- 172.21.15.116:0/1415748811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68a40aa7d0 msgr2=0x7f68a40a8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a95aa700 1 --2- 172.21.15.116:0/1415748811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68a40aa7d0 0x7f68a40a8400 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7f68a40acfd0 tx=0x7f6894006eb0).stop 2020-11-20T12:21:20.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a95aa700 1 -- 172.21.15.116:0/1415748811 shutdown_connections 2020-11-20T12:21:20.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a95aa700 1 --2- 172.21.15.116:0/1415748811 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a40ad540 0x7f68a40ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a95aa700 1 --2- 172.21.15.116:0/1415748811 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68a40aa7d0 0x7f68a40a8400 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a95aa700 1 -- 172.21.15.116:0/1415748811 >> 172.21.15.116:0/1415748811 conn(0x7f68a40678a0 msgr2=0x7f68a4067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:20.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a95aa700 1 -- 172.21.15.116:0/1415748811 shutdown_connections 2020-11-20T12:21:20.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a95aa700 1 -- 172.21.15.116:0/1415748811 wait complete. 2020-11-20T12:21:20.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a95aa700 1 Processor -- start 2020-11-20T12:21:20.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.191+0000 7f68a95aa700 1 -- start start 2020-11-20T12:21:20.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a95aa700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68a40aabb0 0x7f68a41653f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a95aa700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68a40ad540 0x7f68a414cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.204 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a95aa700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a40a69c0 0x7f68a409f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a95aa700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a4162270 con 0x7f68a40aabb0 2020-11-20T12:21:20.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a95aa700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a4161390 con 0x7f68a40ad540 2020-11-20T12:21:20.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a95aa700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a41615f0 con 0x7f68a40a69c0 2020-11-20T12:21:20.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68a40aabb0 0x7f68a41653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68a40aabb0 0x7f68a41653f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50778/0 (socket says 172.21.15.116:50778) 2020-11-20T12:21:20.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a3fff700 1 -- 172.21.15.116:0/3564867995 learned_addr learned my addr 172.21.15.116:0/3564867995 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:20.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a37fe700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68a40ad540 0x7f68a414cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a8da9700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a40a69c0 0x7f68a409f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a3fff700 1 -- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a40a69c0 msgr2=0x7f68a409f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a3fff700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a40a69c0 0x7f68a409f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a3fff700 1 -- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68a40ad540 msgr2=0x7f68a414cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a3fff700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68a40ad540 0x7f68a414cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a3fff700 1 -- 172.21.15.116:0/3564867995 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4161890 con 0x7f68a40aabb0 2020-11-20T12:21:20.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a3fff700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68a40aabb0 0x7f68a41653f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f68a40c6350 tx=0x7f6894015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:20.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f6883fff700 1 -- 172.21.15.116:0/3564867995 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894011c80 con 0x7f68a40aabb0 2020-11-20T12:21:20.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f6883fff700 1 -- 172.21.15.116:0/3564867995 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6894011de0 con 0x7f68a40aabb0 2020-11-20T12:21:20.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a95aa700 1 -- 172.21.15.116:0/3564867995 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a4161b30 con 0x7f68a40aabb0 2020-11-20T12:21:20.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f6883fff700 1 -- 172.21.15.116:0/3564867995 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689401e690 con 0x7f68a40aabb0 2020-11-20T12:21:20.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.195+0000 7f68a95aa700 1 -- 172.21.15.116:0/3564867995 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a414d060 con 0x7f68a40aabb0 2020-11-20T12:21:20.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.199+0000 7f68a95aa700 1 -- 172.21.15.116:0/3564867995 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a405cf30 con 0x7f68a40aabb0 2020-11-20T12:21:20.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.199+0000 7f6883fff700 1 -- 172.21.15.116:0/3564867995 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6894025030 con 0x7f68a40aabb0 2020-11-20T12:21:20.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.199+0000 7f6883fff700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f687c0557c0 0x7f687c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.199+0000 7f6883fff700 1 -- 172.21.15.116:0/3564867995 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f689406fdb0 con 0x7f68a40aabb0 2020-11-20T12:21:20.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.199+0000 7f68a37fe700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f687c0557c0 0x7f687c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.203+0000 7f68a37fe700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f687c0557c0 0x7f687c057c80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f68a4151590 tx=0x7f6898007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:20.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.203+0000 7f6883fff700 1 -- 172.21.15.116:0/3564867995 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6894074070 con 0x7f68a40aabb0 2020-11-20T12:21:20.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.359+0000 7f68a95aa700 1 -- 172.21.15.116:0/3564867995 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f68a40601b0 con 0x7f68a40aabb0 2020-11-20T12:21:20.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f6883fff700 1 -- 172.21.15.116:0/3564867995 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6894023030 con 0x7f68a40aabb0 2020-11-20T12:21:20.364 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:20.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 -- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f687c0557c0 msgr2=0x7f687c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f687c0557c0 0x7f687c057c80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f68a4151590 tx=0x7f6898007a00).stop 2020-11-20T12:21:20.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 -- 172.21.15.116:0/3564867995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68a40aabb0 msgr2=0x7f68a41653f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68a40aabb0 0x7f68a41653f0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f68a40c6350 tx=0x7f6894015000).stop 2020-11-20T12:21:20.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 -- 172.21.15.116:0/3564867995 shutdown_connections 2020-11-20T12:21:20.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f687c0557c0 0x7f687c057c80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f68a40a69c0 0x7f68a409f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f68a40aabb0 0x7f68a41653f0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 --2- 172.21.15.116:0/3564867995 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f68a40ad540 0x7f68a414cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 -- 172.21.15.116:0/3564867995 >> 172.21.15.116:0/3564867995 conn(0x7f68a40678a0 msgr2=0x7f68a4152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:20.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 -- 172.21.15.116:0/3564867995 shutdown_connections 2020-11-20T12:21:20.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.363+0000 7f68a95aa700 1 -- 172.21.15.116:0/3564867995 wait complete. 2020-11-20T12:21:20.391 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:21:20.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.571+0000 7fa306617700 1 Processor -- start 2020-11-20T12:21:20.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.571+0000 7fa306617700 1 -- start start 2020-11-20T12:21:20.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.571+0000 7fa306617700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa300155c80 0x7fa300062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.571+0000 7fa306617700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa300156060 0x7fa3000c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.571+0000 7fa306617700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3000b93c0 con 0x7fa3000c12c0 2020-11-20T12:21:20.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.571+0000 7fa306617700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3000ba580 con 0x7fa300156060 2020-11-20T12:21:20.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.571+0000 7fa306617700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000b9660 con 0x7fa300155c80 2020-11-20T12:21:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa305e16700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa3000c12c0 0x7fa300060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60218/0 (socket says 172.21.15.116:60218) 2020-11-20T12:21:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa305e16700 1 -- 172.21.15.116:0/3452585781 learned_addr learned my addr 172.21.15.116:0/3452585781 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:20.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa304e14700 1 --2- 172.21.15.116:0/3452585781 >> v2:172.21.15.173:3300/0 conn(0x7fa300156060 0x7fa3000c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3718673490 0 0) 0x7fa3000b93c0 con 0x7fa3000c12c0 2020-11-20T12:21:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e8003240 con 0x7fa3000c12c0 2020-11-20T12:21:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa305615700 1 --2- 172.21.15.116:0/3452585781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa300155c80 0x7fa300062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 587069600 0 0) 0x7fa2e8003240 con 0x7fa3000c12c0 2020-11-20T12:21:20.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa300155c80 msgr2=0x7fa300062ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:20.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 --2- 172.21.15.116:0/3452585781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa300155c80 0x7fa300062ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 >> v2:172.21.15.173:3300/0 conn(0x7fa300156060 msgr2=0x7fa3000c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:20.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 --2- 172.21.15.116:0/3452585781 >> v2:172.21.15.173:3300/0 conn(0x7fa300156060 0x7fa3000c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000b9900 con 0x7fa3000c12c0 2020-11-20T12:21:20.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fa2f8001860 con 0x7fa3000c12c0 2020-11-20T12:21:20.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 >> v1:172.21.15.116:6789/0 conn(0x7fa3000c12c0 legacy=0x7fa300060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 --2- 172.21.15.116:0/3452585781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2e8003730 0x7fa2e8003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 --2- 172.21.15.116:0/3452585781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2e80040b0 0x7fa2e8004580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 --2- 172.21.15.116:0/3452585781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2e8004c20 0x7fa2e800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000ba580 con 0x7fa2e8004c20 2020-11-20T12:21:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000b9660 con 0x7fa2e80040b0 2020-11-20T12:21:20.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e8005310 con 0x7fa2e8003730 2020-11-20T12:21:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa305e16700 1 --2- 172.21.15.116:0/3452585781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2e8004c20 0x7fa2e800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa304e14700 1 --2- 172.21.15.116:0/3452585781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2e80040b0 0x7fa2e8004580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa305e16700 1 -- 172.21.15.116:0/3452585781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2e8003730 msgr2=0x7fa2e8003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:20.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa305e16700 1 --2- 172.21.15.116:0/3452585781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa2e8003730 0x7fa2e8003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:20.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa305e16700 1 -- 172.21.15.116:0/3452585781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2e80040b0 msgr2=0x7fa2e8004580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:20.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa305e16700 1 --2- 172.21.15.116:0/3452585781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2e80040b0 0x7fa2e8004580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa305e16700 1 -- 172.21.15.116:0/3452585781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e8003240 con 0x7fa2e8004c20 2020-11-20T12:21:20.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa305e16700 1 --2- 172.21.15.116:0/3452585781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2e8004c20 0x7fa2e800b730 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7fa2f800c840 tx=0x7fa2f8007b80).ready entity=mon.0 client_cookie=b60d99329555e807 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:20.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8005b80 con 0x7fa2e8004c20 2020-11-20T12:21:20.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f8005ce0 con 0x7fa2e8004c20 2020-11-20T12:21:20.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa2effff700 1 -- 172.21.15.116:0/3452585781 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f8007320 con 0x7fa2e8004c20 2020-11-20T12:21:20.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa306617700 1 -- 172.21.15.116:0/3452585781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2e8004c20 msgr2=0x7fa2e800b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa306617700 1 --2- 172.21.15.116:0/3452585781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2e8004c20 0x7fa2e800b730 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7fa2f800c840 tx=0x7fa2f8007b80).stop 2020-11-20T12:21:20.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.575+0000 7fa305e16700 1 -- 172.21.15.116:0/3452585781 reap_dead start 2020-11-20T12:21:20.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.579+0000 7fa306617700 1 -- 172.21.15.116:0/3452585781 shutdown_connections 2020-11-20T12:21:20.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.579+0000 7fa306617700 1 -- 172.21.15.116:0/3452585781 >> 172.21.15.116:0/3452585781 conn(0x7fa300068350 msgr2=0x7fa3000627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:20.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.579+0000 7fa306617700 1 -- 172.21.15.116:0/3452585781 shutdown_connections 2020-11-20T12:21:20.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.579+0000 7fa306617700 1 -- 172.21.15.116:0/3452585781 wait complete. 2020-11-20T12:21:20.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.579+0000 7fa306617700 1 Processor -- start 2020-11-20T12:21:20.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.579+0000 7fa306617700 1 -- start start 2020-11-20T12:21:20.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa306617700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3001692b0 0x7fa300168b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa306617700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa300169690 0x7fa3000a8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa306617700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa300159930 0x7fa300159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa306617700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000a85f0 con 0x7fa300169690 2020-11-20T12:21:20.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa306617700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa300169af0 con 0x7fa300159930 2020-11-20T12:21:20.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa306617700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3000a8810 con 0x7fa3001692b0 2020-11-20T12:21:20.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa304e14700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa300169690 0x7fa3000a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa304e14700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa300169690 0x7fa3000a8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50796/0 (socket says 172.21.15.116:50796) 2020-11-20T12:21:20.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa304e14700 1 -- 172.21.15.116:0/2414664115 learned_addr learned my addr 172.21.15.116:0/2414664115 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:20.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa305e16700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa300159930 0x7fa300159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa305615700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3001692b0 0x7fa300168b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa304e14700 1 -- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3001692b0 msgr2=0x7fa300168b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa304e14700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3001692b0 0x7fa300168b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa304e14700 1 -- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa300159930 msgr2=0x7fa300159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa304e14700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa300159930 0x7fa300159db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa304e14700 1 -- 172.21.15.116:0/2414664115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3000a8ab0 con 0x7fa300169690 2020-11-20T12:21:20.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa304e14700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa300169690 0x7fa3000a8100 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7fa2f0010920 tx=0x7fa2f0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:20.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa2ed7fa700 1 -- 172.21.15.116:0/2414664115 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f00126d0 con 0x7fa300169690 2020-11-20T12:21:20.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa2ed7fa700 1 -- 172.21.15.116:0/2414664115 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f0012830 con 0x7fa300169690 2020-11-20T12:21:20.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa306617700 1 -- 172.21.15.116:0/2414664115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3000a8d50 con 0x7fa300169690 2020-11-20T12:21:20.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa2ed7fa700 1 -- 172.21.15.116:0/2414664115 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0012bb0 con 0x7fa300169690 2020-11-20T12:21:20.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.583+0000 7fa306617700 1 -- 172.21.15.116:0/2414664115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3000a9290 con 0x7fa300169690 2020-11-20T12:21:20.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.587+0000 7fa306617700 1 -- 172.21.15.116:0/2414664115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30005cf30 con 0x7fa300169690 2020-11-20T12:21:20.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.587+0000 7fa2ed7fa700 1 -- 172.21.15.116:0/2414664115 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa2f0014470 con 0x7fa300169690 2020-11-20T12:21:20.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.587+0000 7fa2ed7fa700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2d80557f0 0x7fa2d8057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:20.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.587+0000 7fa2ed7fa700 1 -- 172.21.15.116:0/2414664115 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa2f006eca0 con 0x7fa300169690 2020-11-20T12:21:20.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.587+0000 7fa305615700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2d80557f0 0x7fa2d8057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:20.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.591+0000 7fa305615700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2d80557f0 0x7fa2d8057cb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fa2fc006fd0 tx=0x7fa2fc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:20.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.591+0000 7fa2ed7fa700 1 -- 172.21.15.116:0/2414664115 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa2f0073070 con 0x7fa300169690 2020-11-20T12:21:20.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 -- 172.21.15.116:0/2414664115 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3000639c0 con 0x7fa300169690 2020-11-20T12:21:20.758 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa2ed7fa700 1 -- 172.21.15.116:0/2414664115 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa2f0044ae0 con 0x7fa300169690 2020-11-20T12:21:20.759 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:20.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 -- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2d80557f0 msgr2=0x7fa2d8057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2d80557f0 0x7fa2d8057cb0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7fa2fc006fd0 tx=0x7fa2fc008040).stop 2020-11-20T12:21:20.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 -- 172.21.15.116:0/2414664115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa300169690 msgr2=0x7fa3000a8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa300169690 0x7fa3000a8100 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7fa2f0010920 tx=0x7fa2f0008040).stop 2020-11-20T12:21:20.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 -- 172.21.15.116:0/2414664115 shutdown_connections 2020-11-20T12:21:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2d80557f0 0x7fa2d8057cb0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa300159930 0x7fa300159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa3001692b0 0x7fa300168b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 --2- 172.21.15.116:0/2414664115 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa300169690 0x7fa3000a8100 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:20.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 -- 172.21.15.116:0/2414664115 >> 172.21.15.116:0/2414664115 conn(0x7fa300068350 msgr2=0x7fa3000c2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:20.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 -- 172.21.15.116:0/2414664115 shutdown_connections 2020-11-20T12:21:20.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.755+0000 7fa306617700 1 -- 172.21.15.116:0/2414664115 wait complete. 2020-11-20T12:21:20.787 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:21:20.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.967+0000 7f4ee6f6b700 1 Processor -- start 2020-11-20T12:21:20.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.967+0000 7f4ee6f6b700 1 -- start start 2020-11-20T12:21:20.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4ee6f6b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f4ee00bab50 0x7f4ee00baf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4ee6f6b700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee00c0740 con 0x7f4ee00bfee0 2020-11-20T12:21:21.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4ee6f6b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ee00c09e0 con 0x7f4ee0152190 2020-11-20T12:21:21.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4ee6f6b700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ee00c0c80 con 0x7f4ee00bab50 2020-11-20T12:21:21.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edffff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4ee00bfee0 0x7f4ee0151a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60238/0 (socket says 172.21.15.116:60238) 2020-11-20T12:21:21.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edffff700 1 -- 172.21.15.116:0/3347935532 learned_addr learned my addr 172.21.15.116:0/3347935532 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:21.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4ee4d07700 1 --2- 172.21.15.116:0/3347935532 >> v2:172.21.15.173:3300/0 conn(0x7f4ee00bab50 0x7f4ee00baf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731035315 0 0) 0x7f4ee00c0740 con 0x7f4ee00bfee0 2020-11-20T12:21:21.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec4003240 con 0x7f4ee00bfee0 2020-11-20T12:21:21.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992955697 0 0) 0x7f4ee00c09e0 con 0x7f4ee0152190 2020-11-20T12:21:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ee00c0740 con 0x7f4ee0152190 2020-11-20T12:21:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2866288887 0 0) 0x7f4ec4003240 con 0x7f4ee00bfee0 2020-11-20T12:21:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 >> v2:172.21.15.173:3300/0 conn(0x7f4ee00bab50 msgr2=0x7f4ee00baf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:21.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 --2- 172.21.15.116:0/3347935532 >> v2:172.21.15.173:3300/0 conn(0x7f4ee00bab50 0x7f4ee00baf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 >> v1:172.21.15.173:6789/0 conn(0x7f4ee0152190 legacy=0x7f4ee00637b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee00c0f20 con 0x7f4ee00bfee0 2020-11-20T12:21:21.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f4ecc0014e0 con 0x7f4ee00bfee0 2020-11-20T12:21:21.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 >> v1:172.21.15.116:6789/0 conn(0x7f4ee00bfee0 legacy=0x7f4ee0151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 --2- 172.21.15.116:0/3347935532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec40038c0 0x7f4ec4003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 --2- 172.21.15.116:0/3347935532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ec40044e0 0x7f4ec4004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 --2- 172.21.15.116:0/3347935532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ec4004e50 0x7f4ec400b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee00c0c80 con 0x7f4ec40038c0 2020-11-20T12:21:21.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec400bc20 con 0x7f4ec40044e0 2020-11-20T12:21:21.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec400bde0 con 0x7f4ec4004e50 2020-11-20T12:21:21.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.971+0000 7f4ee4d07700 1 --2- 172.21.15.116:0/3347935532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec40038c0 0x7f4ec4003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4edffff700 1 --2- 172.21.15.116:0/3347935532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ec40044e0 0x7f4ec4004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee5508700 1 --2- 172.21.15.116:0/3347935532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ec4004e50 0x7f4ec400b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee4d07700 1 -- 172.21.15.116:0/3347935532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ec4004e50 msgr2=0x7f4ec400b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:21.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee4d07700 1 --2- 172.21.15.116:0/3347935532 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ec4004e50 0x7f4ec400b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee4d07700 1 -- 172.21.15.116:0/3347935532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ec40044e0 msgr2=0x7f4ec4004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:21.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee4d07700 1 --2- 172.21.15.116:0/3347935532 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ec40044e0 0x7f4ec4004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee4d07700 1 -- 172.21.15.116:0/3347935532 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec4003240 con 0x7f4ec40038c0 2020-11-20T12:21:21.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee5508700 1 -- 172.21.15.116:0/3347935532 reap_dead start 2020-11-20T12:21:21.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee4d07700 1 --2- 172.21.15.116:0/3347935532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec40038c0 0x7f4ec4003cc0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7f4ed400bc00 tx=0x7f4ed4006df0).ready entity=mon.0 client_cookie=19c0c4b396611274 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:21.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400b310 con 0x7f4ec40038c0 2020-11-20T12:21:21.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed400eb40 con 0x7f4ec40038c0 2020-11-20T12:21:21.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4edf7fe700 1 -- 172.21.15.116:0/3347935532 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400eec0 con 0x7f4ec40038c0 2020-11-20T12:21:21.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/3347935532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec40038c0 msgr2=0x7f4ec4003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee6f6b700 1 --2- 172.21.15.116:0/3347935532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec40038c0 0x7f4ec4003cc0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7f4ed400bc00 tx=0x7f4ed4006df0).stop 2020-11-20T12:21:21.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/3347935532 shutdown_connections 2020-11-20T12:21:21.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee6f6b700 1 --2- 172.21.15.116:0/3347935532 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ec40038c0 0x7f4ec4003cc0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/3347935532 >> 172.21.15.116:0/3347935532 conn(0x7f4ee00c5880 msgr2=0x7f4ee00c1850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:21.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/3347935532 shutdown_connections 2020-11-20T12:21:21.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.975+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/3347935532 wait complete. 2020-11-20T12:21:21.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.979+0000 7f4ee6f6b700 1 Processor -- start 2020-11-20T12:21:21.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.979+0000 7f4ee6f6b700 1 -- start start 2020-11-20T12:21:21.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.979+0000 7f4ee6f6b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ec40038c0 0x7f4ee00a4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.979+0000 7f4ee6f6b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ee014bac0 0x7f4ee00a5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.979+0000 7f4ee6f6b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ee0149f40 0x7f4ee00aa390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.979+0000 7f4ee6f6b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee014a800 con 0x7f4ee0149f40 2020-11-20T12:21:21.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.979+0000 7f4ee6f6b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee014aaa0 con 0x7f4ec40038c0 2020-11-20T12:21:21.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.979+0000 7f4ee6f6b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ee014ad40 con 0x7f4ee014bac0 2020-11-20T12:21:21.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.979+0000 7f4ee5508700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ee0149f40 0x7f4ee00aa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.979+0000 7f4ee5508700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ee0149f40 0x7f4ee00aa390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50820/0 (socket says 172.21.15.116:50820) 2020-11-20T12:21:21.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4ee5508700 1 -- 172.21.15.116:0/2219212689 learned_addr learned my addr 172.21.15.116:0/2219212689 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:21.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4ee4d07700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ec40038c0 0x7f4ee00a4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4edffff700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ee014bac0 0x7f4ee00a5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4ee5508700 1 -- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ee014bac0 msgr2=0x7f4ee00a5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4ee5508700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ee014bac0 0x7f4ee00a5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4ee5508700 1 -- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ec40038c0 msgr2=0x7f4ee00a4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4ee5508700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ec40038c0 0x7f4ee00a4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4ee5508700 1 -- 172.21.15.116:0/2219212689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee014afe0 con 0x7f4ee0149f40 2020-11-20T12:21:21.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4ee5508700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ee0149f40 0x7f4ee00aa390 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f4ee00bfe30 tx=0x7f4ed8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:21.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4edcff9700 1 -- 172.21.15.116:0/2219212689 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8006a80 con 0x7f4ee0149f40 2020-11-20T12:21:21.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4edcff9700 1 -- 172.21.15.116:0/2219212689 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed8006be0 con 0x7f4ee0149f40 2020-11-20T12:21:21.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/2219212689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee0148d20 con 0x7f4ee0149f40 2020-11-20T12:21:21.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4edcff9700 1 -- 172.21.15.116:0/2219212689 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8003030 con 0x7f4ee0149f40 2020-11-20T12:21:21.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/2219212689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ee00a5920 con 0x7f4ee0149f40 2020-11-20T12:21:21.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.983+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/2219212689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ee0061a10 con 0x7f4ee0149f40 2020-11-20T12:21:21.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.987+0000 7f4edcff9700 1 -- 172.21.15.116:0/2219212689 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4ed801c060 con 0x7f4ee0149f40 2020-11-20T12:21:21.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.987+0000 7f4edcff9700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4eb80557c0 0x7f4eb8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.987+0000 7f4edcff9700 1 -- 172.21.15.116:0/2219212689 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4ed806dea0 con 0x7f4ee0149f40 2020-11-20T12:21:21.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.987+0000 7f4ee4d07700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4eb80557c0 0x7f4eb8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.987+0000 7f4ee4d07700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4eb80557c0 0x7f4eb8057c80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f4ed4006e30 tx=0x7f4ed4019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:21.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:20.991+0000 7f4edcff9700 1 -- 172.21.15.116:0/2219212689 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4ee0061a10 con 0x7f4ee0149f40 2020-11-20T12:21:21.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.147+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/2219212689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4ee0061a10 con 0x7f4ee0149f40 2020-11-20T12:21:21.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.147+0000 7f4edcff9700 1 -- 172.21.15.116:0/2219212689 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4ee0061a10 con 0x7f4ee0149f40 2020-11-20T12:21:21.152 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:21.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4eb80557c0 msgr2=0x7f4eb8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4eb80557c0 0x7f4eb8057c80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f4ed4006e30 tx=0x7f4ed4019000).stop 2020-11-20T12:21:21.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/2219212689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ee0149f40 msgr2=0x7f4ee00aa390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ee0149f40 0x7f4ee00aa390 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f4ee00bfe30 tx=0x7f4ed8009040).stop 2020-11-20T12:21:21.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/2219212689 shutdown_connections 2020-11-20T12:21:21.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4eb80557c0 0x7f4eb8057c80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4ee0149f40 0x7f4ee00aa390 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4ec40038c0 0x7f4ee00a4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 --2- 172.21.15.116:0/2219212689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4ee014bac0 0x7f4ee00a5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/2219212689 >> 172.21.15.116:0/2219212689 conn(0x7f4ee00c5880 msgr2=0x7f4ee0064590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:21.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/2219212689 shutdown_connections 2020-11-20T12:21:21.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.151+0000 7f4ee6f6b700 1 -- 172.21.15.116:0/2219212689 wait complete. 2020-11-20T12:21:21.186 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:21:21.187 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:21:21.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.355+0000 7fe2bf076700 1 Processor -- start 2020-11-20T12:21:21.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.355+0000 7fe2bf076700 1 -- start start 2020-11-20T12:21:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2bf076700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe2b813afa0 0x7fe2b80aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2bf076700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2b80ad440 0x7fe2b80aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2bf076700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2b80ada00 con 0x7fe2b813b380 2020-11-20T12:21:21.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2bf076700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b80adca0 con 0x7fe2b80ad440 2020-11-20T12:21:21.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2bf076700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2b80adf40 con 0x7fe2b813afa0 2020-11-20T12:21:21.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2bd873700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe2b813b380 0x7fe2b80acd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60258/0 (socket says 172.21.15.116:60258) 2020-11-20T12:21:21.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2bd873700 1 -- 172.21.15.116:0/647694234 learned_addr learned my addr 172.21.15.116:0/647694234 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:21.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2be875700 1 --2- 172.21.15.116:0/647694234 >> v2:172.21.15.116:3300/0 conn(0x7fe2b80ad440 0x7fe2b80aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2be074700 1 --2- 172.21.15.116:0/647694234 >> v2:172.21.15.173:3300/0 conn(0x7fe2b813afa0 0x7fe2b80aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2bd072700 1 -- 172.21.15.116:0/647694234 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268315796 0 0) 0x7fe2b80ada00 con 0x7fe2b813b380 2020-11-20T12:21:21.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2bd072700 1 -- 172.21.15.116:0/647694234 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a0003240 con 0x7fe2b813b380 2020-11-20T12:21:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2be875700 1 -- 172.21.15.116:0/647694234 >> v2:172.21.15.173:3300/0 conn(0x7fe2b813afa0 msgr2=0x7fe2b80aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2be875700 1 --2- 172.21.15.116:0/647694234 >> v2:172.21.15.173:3300/0 conn(0x7fe2b813afa0 0x7fe2b80aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2be875700 1 -- 172.21.15.116:0/647694234 >> v1:172.21.15.116:6789/0 conn(0x7fe2b813b380 legacy=0x7fe2b80acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2be875700 1 -- 172.21.15.116:0/647694234 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b80ae1e0 con 0x7fe2b80ad440 2020-11-20T12:21:21.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.359+0000 7fe2be074700 1 --2- 172.21.15.116:0/647694234 >> v2:172.21.15.173:3300/0 conn(0x7fe2b813afa0 0x7fe2b80aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:21.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2be875700 1 --2- 172.21.15.116:0/647694234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2b80ad440 0x7fe2b80aaaa0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7fe2b80818e0 tx=0x7fe2b00091d0).ready entity=mon.0 client_cookie=c017acbca43fb984 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:21.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bd072700 1 -- 172.21.15.116:0/647694234 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b000ba70 con 0x7fe2b80ad440 2020-11-20T12:21:21.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bd072700 1 -- 172.21.15.116:0/647694234 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b000bbd0 con 0x7fe2b80ad440 2020-11-20T12:21:21.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bd072700 1 -- 172.21.15.116:0/647694234 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0011400 con 0x7fe2b80ad440 2020-11-20T12:21:21.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bf076700 1 -- 172.21.15.116:0/647694234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2b80ad440 msgr2=0x7fe2b80aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bf076700 1 --2- 172.21.15.116:0/647694234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2b80ad440 0x7fe2b80aaaa0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7fe2b80818e0 tx=0x7fe2b00091d0).stop 2020-11-20T12:21:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bf076700 1 -- 172.21.15.116:0/647694234 shutdown_connections 2020-11-20T12:21:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bf076700 1 --2- 172.21.15.116:0/647694234 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2b80ad440 0x7fe2b80aaaa0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bf076700 1 --2- 172.21.15.116:0/647694234 >> v2:172.21.15.173:3300/0 conn(0x7fe2b813afa0 0x7fe2b80aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bf076700 1 -- 172.21.15.116:0/647694234 >> 172.21.15.116:0/647694234 conn(0x7fe2b80678a0 msgr2=0x7fe2b8067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bf076700 1 -- 172.21.15.116:0/647694234 shutdown_connections 2020-11-20T12:21:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bf076700 1 -- 172.21.15.116:0/647694234 wait complete. 2020-11-20T12:21:21.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bf076700 1 Processor -- start 2020-11-20T12:21:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.363+0000 7fe2bf076700 1 -- start start 2020-11-20T12:21:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bf076700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2b80ad440 0x7fe2b813a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bf076700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2b813afa0 0x7fe2b80a2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bf076700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2b813b380 0x7fe2b80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bf076700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80a5e20 con 0x7fe2b813afa0 2020-11-20T12:21:21.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bf076700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80a2410 con 0x7fe2b80ad440 2020-11-20T12:21:21.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bf076700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b80a26b0 con 0x7fe2b813b380 2020-11-20T12:21:21.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bd873700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2b813afa0 0x7fe2b80a2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bd873700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2b813afa0 0x7fe2b80a2df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50832/0 (socket says 172.21.15.116:50832) 2020-11-20T12:21:21.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bd873700 1 -- 172.21.15.116:0/3981917014 learned_addr learned my addr 172.21.15.116:0/3981917014 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:21.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2be074700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2b80ad440 0x7fe2b813a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2be875700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2b813b380 0x7fe2b80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bd873700 1 -- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2b813b380 msgr2=0x7fe2b80a54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bd873700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2b813b380 0x7fe2b80a54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bd873700 1 -- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2b80ad440 msgr2=0x7fe2b813a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bd873700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2b80ad440 0x7fe2b813a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bd873700 1 -- 172.21.15.116:0/3981917014 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b80a2950 con 0x7fe2b813afa0 2020-11-20T12:21:21.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bd873700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2b813afa0 0x7fe2b80a2df0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7fe2a800c710 tx=0x7fe2a8007a60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:21.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2a67fc700 1 -- 172.21.15.116:0/3981917014 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80072e0 con 0x7fe2b813afa0 2020-11-20T12:21:21.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2a67fc700 1 -- 172.21.15.116:0/3981917014 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2a8007440 con 0x7fe2b813afa0 2020-11-20T12:21:21.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bf076700 1 -- 172.21.15.116:0/3981917014 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b80a32e0 con 0x7fe2b813afa0 2020-11-20T12:21:21.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2a67fc700 1 -- 172.21.15.116:0/3981917014 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8005820 con 0x7fe2b813afa0 2020-11-20T12:21:21.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.367+0000 7fe2bf076700 1 -- 172.21.15.116:0/3981917014 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b80a5b80 con 0x7fe2b813afa0 2020-11-20T12:21:21.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.371+0000 7fe2bf076700 1 -- 172.21.15.116:0/3981917014 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b805cf30 con 0x7fe2b813afa0 2020-11-20T12:21:21.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.371+0000 7fe2a67fc700 1 -- 172.21.15.116:0/3981917014 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe2a8010070 con 0x7fe2b813afa0 2020-11-20T12:21:21.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.371+0000 7fe2a67fc700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2900557c0 0x7fe290057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:21.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.371+0000 7fe2a67fc700 1 -- 172.21.15.116:0/3981917014 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe2a806e220 con 0x7fe2b813afa0 2020-11-20T12:21:21.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.375+0000 7fe2be074700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2900557c0 0x7fe290057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:21.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.375+0000 7fe2be074700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2900557c0 0x7fe290057c80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fe2b80a4de0 tx=0x7fe2b4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:21.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.379+0000 7fe2a67fc700 1 -- 172.21.15.116:0/3981917014 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe2a8044140 con 0x7fe2b813afa0 2020-11-20T12:21:21.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.583+0000 7fe2bf076700 1 -- 172.21.15.116:0/3981917014 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2b8060170 con 0x7fe2b813afa0 2020-11-20T12:21:21.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.583+0000 7fe2a67fc700 1 -- 172.21.15.116:0/3981917014 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2851 (secure 0 0 0) 0x7fe2a8047760 con 0x7fe2b813afa0 2020-11-20T12:21:21.585 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:21.586 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1638614527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1638614527},{"type":"v1","addr":"172.21.15.173:6837","nonce":1638614527}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3682335131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3682335131},{"type":"v1","addr":"172.21.15.173:6836","nonce":3682335131}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7725,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/3975405805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3975405805},{"type":"v1","addr":"172.21.15.116:6834","nonce":3975405805}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:11.109567+0000","modified":"2020-11-20T12:21:13.118616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/267899152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":267899152},{"type":"v1","addr":"172.21.15.116:6835","nonce":267899152}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-20T12:21:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.583+0000 7fe2bf076700 1 -- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2900557c0 msgr2=0x7fe290057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.583+0000 7fe2bf076700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2900557c0 0x7fe290057c80 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7fe2b80a4de0 tx=0x7fe2b4008040).stop 2020-11-20T12:21:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.583+0000 7fe2bf076700 1 -- 172.21.15.116:0/3981917014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2b813afa0 msgr2=0x7fe2b80a2df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:21.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.583+0000 7fe2bf076700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2b813afa0 0x7fe2b80a2df0 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7fe2a800c710 tx=0x7fe2a8007a60).stop 2020-11-20T12:21:21.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.587+0000 7fe2bf076700 1 -- 172.21.15.116:0/3981917014 shutdown_connections 2020-11-20T12:21:21.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.587+0000 7fe2bf076700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe2900557c0 0x7fe290057c80 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.587+0000 7fe2bf076700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2b813b380 0x7fe2b80a54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.587+0000 7fe2bf076700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2b80ad440 0x7fe2b813a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.587+0000 7fe2bf076700 1 --2- 172.21.15.116:0/3981917014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2b813afa0 0x7fe2b80a2df0 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:21.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.587+0000 7fe2bf076700 1 -- 172.21.15.116:0/3981917014 >> 172.21.15.116:0/3981917014 conn(0x7fe2b80678a0 msgr2=0x7fe2b8155890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:21.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.587+0000 7fe2bf076700 1 -- 172.21.15.116:0/3981917014 shutdown_connections 2020-11-20T12:21:21.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:21.587+0000 7fe2bf076700 1 -- 172.21.15.116:0/3981917014 wait complete. 2020-11-20T12:21:21.591 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 60 2020-11-20T12:21:22.018 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:21:11.109567+0000', 'modified': '2020-11-20T12:21:13.118616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7728': {'gid': 7728, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/267899152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 267899152}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 267899152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:21:22.019 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:21:22.019 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:21:22.228 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":7728,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.095308418,"mdsmap_epoch":60,"osdmap_epoch":107,"osdmap_epoch_barrier":107,"uptime":16.600351992} 2020-11-20T12:21:22.228 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 7728, "mdsmap_epoch": 60, "osdmap_epoch": 107, "osdmap_epoch_barrier": 107, "rank_uptime": 11.095308418, "state": "up:active", "uptime": 16.600351992, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:21:22.229 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:21:22.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:22.230 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-20T12:21:22.251 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:21:22.252 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:22.369 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:22.369 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:21:22.394 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:22.394 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:21:22.400 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:21:22.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:22.402 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-20T12:21:22.437 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:21:22.437 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:21:22.438 INFO:tasks.cephfs.mount:self.client.name = client.2 2020-11-20T12:21:22.438 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2020-11-20T12:21:22.438 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:21:22.439 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:21:22.439 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:21:22.439 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:21:22.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:22.440 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:21:22.461 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:21:22.461 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:21:22.461 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:21:22.462 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:22.462 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:21:22.462 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:22.462 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:22.462 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:22.463 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:22.463 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:22.463 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:22.463 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:22.463 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:21:22.464 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:22.464 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:21:22.464 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:22.464 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:21:22.464 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:22.465 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:21:22.465 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:22.465 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:21:22.465 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:22.466 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:21:22.466 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:22.466 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:21:22.466 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:22.466 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:21:22.467 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:22.467 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:21:22.467 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:21:22.467 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:21:22.468 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:22.468 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:21:22.468 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:21:22.468 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:21:22.468 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:22.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:22.469 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:21:22.508 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:21:22.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:22.511 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:21:22.588 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:21:22.588 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:21:22.588 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:21:22.589 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:21:22.589 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:21:22.589 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:21:22.590 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:22.591 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:21:22.591 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:21:22.591 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:21:22.592 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:21:22.592 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:21:22.593 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:21:22.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:22.753 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:21:22.778 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:21:22.778 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:21:22.779 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:21:22.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:22.780 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2020-11-20T12:21:22.860 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:23.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.094+0000 7fb709171700 1 Processor -- start 2020-11-20T12:21:23.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.094+0000 7fb709171700 1 -- start start 2020-11-20T12:21:23.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.094+0000 7fb709171700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb70414e480 0x7fb7041529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.094+0000 7fb709171700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb704066800 0x7fb7040677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.094+0000 7fb709171700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7040c5010 con 0x7fb7040663a0 2020-11-20T12:21:23.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.094+0000 7fb709171700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb704067340 con 0x7fb70414e480 2020-11-20T12:21:23.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.094+0000 7fb709171700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb704066e00 con 0x7fb704066800 2020-11-20T12:21:23.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.094+0000 7fb703fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb70414e480 0x7fb7041529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.094+0000 7fb703fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fb70414e480 0x7fb7041529e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50838/0 (socket says 172.21.15.116:50838) 2020-11-20T12:21:23.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.094+0000 7fb6fbfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb7040663a0 0x7fb7040a6480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60274/0 (socket says 172.21.15.116:60274) 2020-11-20T12:21:23.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.094+0000 7fb703fff700 1 -- 172.21.15.116:0/96992125 learned_addr learned my addr 172.21.15.116:0/96992125 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:23.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb708970700 1 --2- 172.21.15.116:0/96992125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb704066800 0x7fb7040677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb703fff700 1 -- 172.21.15.116:0/96992125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb704066800 msgr2=0x7fb7040677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:23.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb703fff700 1 --2- 172.21.15.116:0/96992125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb704066800 0x7fb7040677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb703fff700 1 -- 172.21.15.116:0/96992125 >> v1:172.21.15.116:6789/0 conn(0x7fb7040663a0 legacy=0x7fb7040a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb703fff700 1 -- 172.21.15.116:0/96992125 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7040670a0 con 0x7fb70414e480 2020-11-20T12:21:23.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb703fff700 1 --2- 172.21.15.116:0/96992125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb70414e480 0x7fb7041529e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7fb6f4006fd0 tx=0x7fb6f4006eb0).ready entity=mon.0 client_cookie=5a79b83cf077dc09 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:23.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb7037fe700 1 -- 172.21.15.116:0/96992125 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f40098d0 con 0x7fb70414e480 2020-11-20T12:21:23.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb7037fe700 1 -- 172.21.15.116:0/96992125 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f4009a30 con 0x7fb70414e480 2020-11-20T12:21:23.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb7037fe700 1 -- 172.21.15.116:0/96992125 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f4009db0 con 0x7fb70414e480 2020-11-20T12:21:23.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb709171700 1 -- 172.21.15.116:0/96992125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb70414e480 msgr2=0x7fb7041529e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb709171700 1 --2- 172.21.15.116:0/96992125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb70414e480 0x7fb7041529e0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7fb6f4006fd0 tx=0x7fb6f4006eb0).stop 2020-11-20T12:21:23.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb709171700 1 -- 172.21.15.116:0/96992125 shutdown_connections 2020-11-20T12:21:23.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb709171700 1 --2- 172.21.15.116:0/96992125 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb70414e480 0x7fb7041529e0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb709171700 1 --2- 172.21.15.116:0/96992125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb704066800 0x7fb7040677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb709171700 1 -- 172.21.15.116:0/96992125 >> 172.21.15.116:0/96992125 conn(0x7fb704060c40 msgr2=0x7fb7040c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:23.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb709171700 1 -- 172.21.15.116:0/96992125 shutdown_connections 2020-11-20T12:21:23.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.098+0000 7fb709171700 1 -- 172.21.15.116:0/96992125 wait complete. 2020-11-20T12:21:23.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.102+0000 7fb709171700 1 Processor -- start 2020-11-20T12:21:23.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.102+0000 7fb709171700 1 -- start start 2020-11-20T12:21:23.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.102+0000 7fb709171700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7040663a0 0x7fb7040bbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.102+0000 7fb709171700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb704066800 0x7fb7041511a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.102+0000 7fb709171700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb70414e480 0x7fb704150430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.102+0000 7fb709171700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb704151690 con 0x7fb704066800 2020-11-20T12:21:23.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.102+0000 7fb709171700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb704150920 con 0x7fb70414e480 2020-11-20T12:21:23.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.102+0000 7fb709171700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7040a7aa0 con 0x7fb7040663a0 2020-11-20T12:21:23.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb6fbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb704066800 0x7fb7041511a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb703fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7040663a0 0x7fb7040bbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb6fbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb704066800 0x7fb7041511a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50846/0 (socket says 172.21.15.116:50846) 2020-11-20T12:21:23.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb703fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7040663a0 0x7fb7040bbfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:59794/0 (socket says 172.21.15.116:59794) 2020-11-20T12:21:23.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb6fbfff700 1 -- 172.21.15.116:0/623591398 learned_addr learned my addr 172.21.15.116:0/623591398 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:23.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb708970700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb70414e480 0x7fb704150430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb6fbfff700 1 -- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7040663a0 msgr2=0x7fb7040bbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb6fbfff700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7040663a0 0x7fb7040bbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb6fbfff700 1 -- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb70414e480 msgr2=0x7fb704150430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb6fbfff700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb70414e480 0x7fb704150430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb6fbfff700 1 -- 172.21.15.116:0/623591398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7040a7d40 con 0x7fb704066800 2020-11-20T12:21:23.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb6fbfff700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb704066800 0x7fb7041511a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7fb6ec00c710 tx=0x7fb6ec007c40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:23.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb700ff9700 1 -- 172.21.15.116:0/623591398 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0072e0 con 0x7fb704066800 2020-11-20T12:21:23.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb700ff9700 1 -- 172.21.15.116:0/623591398 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6ec005530 con 0x7fb704066800 2020-11-20T12:21:23.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb700ff9700 1 -- 172.21.15.116:0/623591398 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6ec0058b0 con 0x7fb704066800 2020-11-20T12:21:23.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb709171700 1 -- 172.21.15.116:0/623591398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7040a7fe0 con 0x7fb704066800 2020-11-20T12:21:23.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.106+0000 7fb709171700 1 -- 172.21.15.116:0/623591398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70414e860 con 0x7fb704066800 2020-11-20T12:21:23.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.110+0000 7fb709171700 1 -- 172.21.15.116:0/623591398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7040a3540 con 0x7fb704066800 2020-11-20T12:21:23.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.110+0000 7fb700ff9700 1 -- 172.21.15.116:0/623591398 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb6ec010070 con 0x7fb704066800 2020-11-20T12:21:23.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.110+0000 7fb700ff9700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6e00557c0 0x7fb6e0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.110+0000 7fb700ff9700 1 -- 172.21.15.116:0/623591398 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb6ec06dc90 con 0x7fb704066800 2020-11-20T12:21:23.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.110+0000 7fb703fff700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6e00557c0 0x7fb6e0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.114+0000 7fb703fff700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6e00557c0 0x7fb6e0057c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fb6f400b1b0 tx=0x7fb6f4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:23.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.118+0000 7fb700ff9700 1 -- 172.21.15.116:0/623591398 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb6ec043be0 con 0x7fb704066800 2020-11-20T12:21:23.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.278+0000 7fb709171700 1 -- 172.21.15.116:0/623591398 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb704061230 con 0x7fb704066800 2020-11-20T12:21:23.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.278+0000 7fb700ff9700 1 -- 172.21.15.116:0/623591398 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16888 (secure 0 0 0) 0x7fb6ec047200 con 0x7fb704066800 2020-11-20T12:21:23.284 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:23.284 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":107,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:12.114283+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:09.571842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:10.034013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:23.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 -- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6e00557c0 msgr2=0x7fb6e0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6e00557c0 0x7fb6e0057c80 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fb6f400b1b0 tx=0x7fb6f4015000).stop 2020-11-20T12:21:23.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 -- 172.21.15.116:0/623591398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb704066800 msgr2=0x7fb7041511a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb704066800 0x7fb7041511a0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7fb6ec00c710 tx=0x7fb6ec007c40).stop 2020-11-20T12:21:23.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 -- 172.21.15.116:0/623591398 shutdown_connections 2020-11-20T12:21:23.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb6e00557c0 0x7fb6e0057c80 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb70414e480 0x7fb704150430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb7040663a0 0x7fb7040bbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 --2- 172.21.15.116:0/623591398 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb704066800 0x7fb7041511a0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 -- 172.21.15.116:0/623591398 >> 172.21.15.116:0/623591398 conn(0x7fb704060c40 msgr2=0x7fb7040c0fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:23.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 -- 172.21.15.116:0/623591398 shutdown_connections 2020-11-20T12:21:23.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.282+0000 7fb709171700 1 -- 172.21.15.116:0/623591398 wait complete. 2020-11-20T12:21:23.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:21:09.571842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:21:10.034013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:21:23.313 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:21:23.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.490+0000 7fd925016700 1 Processor -- start 2020-11-20T12:21:23.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.490+0000 7fd925016700 1 -- start start 2020-11-20T12:21:23.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd925016700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd9200a81f0 0x7fd9200b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd925016700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9200c0e90 0x7fd9200c1340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd925016700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9200a8930 con 0x7fd92014f880 2020-11-20T12:21:23.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd925016700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9200ba2a0 con 0x7fd9200a81f0 2020-11-20T12:21:23.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd925016700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9200a8690 con 0x7fd9200c0e90 2020-11-20T12:21:23.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd91f7fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd9200a81f0 0x7fd9200b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd91f7fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd9200a81f0 0x7fd9200b9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:49906/0 (socket says 172.21.15.116:49906) 2020-11-20T12:21:23.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd924815700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9200c0e90 0x7fd9200c1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd91f7fe700 1 -- 172.21.15.116:0/3169346067 learned_addr learned my addr 172.21.15.116:0/3169346067 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:23.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd924815700 1 -- 172.21.15.116:0/3169346067 >> v2:172.21.15.173:3300/0 conn(0x7fd9200a81f0 msgr2=0x7fd9200b9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:23.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd924815700 1 --2- 172.21.15.116:0/3169346067 >> v2:172.21.15.173:3300/0 conn(0x7fd9200a81f0 0x7fd9200b9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd924815700 1 -- 172.21.15.116:0/3169346067 >> v1:172.21.15.173:6789/0 conn(0x7fd92014f880 legacy=0x7fd920060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd924815700 1 -- 172.21.15.116:0/3169346067 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9200b94d0 con 0x7fd9200c0e90 2020-11-20T12:21:23.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd91f7fe700 1 --2- 172.21.15.116:0/3169346067 >> v2:172.21.15.173:3300/0 conn(0x7fd9200a81f0 0x7fd9200b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:23.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.494+0000 7fd924815700 1 --2- 172.21.15.116:0/3169346067 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9200c0e90 0x7fd9200c1340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fd918009c90 tx=0x7fd9180091d0).ready entity=mon.2 client_cookie=7cc92a0b02ab906c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:23.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.498+0000 7fd91effd700 1 -- 172.21.15.116:0/3169346067 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91800ba70 con 0x7fd9200c0e90 2020-11-20T12:21:23.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.498+0000 7fd91effd700 1 -- 172.21.15.116:0/3169346067 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91800bbd0 con 0x7fd9200c0e90 2020-11-20T12:21:23.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.498+0000 7fd91effd700 1 -- 172.21.15.116:0/3169346067 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918011400 con 0x7fd9200c0e90 2020-11-20T12:21:23.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.498+0000 7fd925016700 1 -- 172.21.15.116:0/3169346067 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9200c0e90 msgr2=0x7fd9200c1340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.498+0000 7fd925016700 1 --2- 172.21.15.116:0/3169346067 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9200c0e90 0x7fd9200c1340 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fd918009c90 tx=0x7fd9180091d0).stop 2020-11-20T12:21:23.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.498+0000 7fd925016700 1 -- 172.21.15.116:0/3169346067 shutdown_connections 2020-11-20T12:21:23.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.498+0000 7fd925016700 1 --2- 172.21.15.116:0/3169346067 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9200c0e90 0x7fd9200c1340 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.498+0000 7fd925016700 1 --2- 172.21.15.116:0/3169346067 >> v2:172.21.15.173:3300/0 conn(0x7fd9200a81f0 0x7fd9200b9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.498+0000 7fd925016700 1 -- 172.21.15.116:0/3169346067 >> 172.21.15.116:0/3169346067 conn(0x7fd920068350 msgr2=0x7fd9200627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:23.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.498+0000 7fd925016700 1 -- 172.21.15.116:0/3169346067 shutdown_connections 2020-11-20T12:21:23.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.498+0000 7fd925016700 1 -- 172.21.15.116:0/3169346067 wait complete. 2020-11-20T12:21:23.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd925016700 1 Processor -- start 2020-11-20T12:21:23.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd925016700 1 -- start start 2020-11-20T12:21:23.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd925016700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9200a81f0 0x7fd92015cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd925016700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9200c0e90 0x7fd92015de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd925016700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd92014f880 0x7fd9200c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd925016700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92015d5f0 con 0x7fd9200a81f0 2020-11-20T12:21:23.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd925016700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92015d890 con 0x7fd92014f880 2020-11-20T12:21:23.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd925016700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92015db30 con 0x7fd9200c0e90 2020-11-20T12:21:23.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd91ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9200a81f0 0x7fd92015cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd91ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9200a81f0 0x7fd92015cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50858/0 (socket says 172.21.15.116:50858) 2020-11-20T12:21:23.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd91ffff700 1 -- 172.21.15.116:0/550992511 learned_addr learned my addr 172.21.15.116:0/550992511 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:23.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd91f7fe700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9200c0e90 0x7fd92015de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd91ffff700 1 -- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9200c0e90 msgr2=0x7fd92015de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd924815700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd92014f880 0x7fd9200c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd91ffff700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9200c0e90 0x7fd92015de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd91ffff700 1 -- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd92014f880 msgr2=0x7fd9200c3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd91ffff700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd92014f880 0x7fd9200c3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd91ffff700 1 -- 172.21.15.116:0/550992511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92009dec0 con 0x7fd9200a81f0 2020-11-20T12:21:23.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd924815700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd92014f880 0x7fd9200c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:23.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd91ffff700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9200a81f0 0x7fd92015cce0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7fd91000c6e0 tx=0x7fd910007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:23.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.502+0000 7fd8fffff700 1 -- 172.21.15.116:0/550992511 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910007020 con 0x7fd9200a81f0 2020-11-20T12:21:23.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.506+0000 7fd8fffff700 1 -- 172.21.15.116:0/550992511 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd910007180 con 0x7fd9200a81f0 2020-11-20T12:21:23.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.506+0000 7fd925016700 1 -- 172.21.15.116:0/550992511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92009db40 con 0x7fd9200a81f0 2020-11-20T12:21:23.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.506+0000 7fd8fffff700 1 -- 172.21.15.116:0/550992511 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91000e560 con 0x7fd9200a81f0 2020-11-20T12:21:23.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.506+0000 7fd925016700 1 -- 172.21.15.116:0/550992511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92015e300 con 0x7fd9200a81f0 2020-11-20T12:21:23.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.506+0000 7fd925016700 1 -- 172.21.15.116:0/550992511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92009dcd0 con 0x7fd9200a81f0 2020-11-20T12:21:23.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.506+0000 7fd8fffff700 1 -- 172.21.15.116:0/550992511 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd910010070 con 0x7fd9200a81f0 2020-11-20T12:21:23.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.506+0000 7fd8fffff700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8f80557c0 0x7fd8f8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.510+0000 7fd8fffff700 1 -- 172.21.15.116:0/550992511 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd91006da20 con 0x7fd9200a81f0 2020-11-20T12:21:23.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.510+0000 7fd91f7fe700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8f80557c0 0x7fd8f8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.510+0000 7fd91f7fe700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8f80557c0 0x7fd8f8057c80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fd9200a7050 tx=0x7fd914008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:23.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.510+0000 7fd8fffff700 1 -- 172.21.15.116:0/550992511 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd910042fd0 con 0x7fd9200a81f0 2020-11-20T12:21:23.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.670+0000 7fd925016700 1 -- 172.21.15.116:0/550992511 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd92005cf30 con 0x7fd9200a81f0 2020-11-20T12:21:23.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.674+0000 7fd8fffff700 1 -- 172.21.15.116:0/550992511 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v107) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd9100438c0 con 0x7fd9200a81f0 2020-11-20T12:21:23.677 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:21:23.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.674+0000 7fd925016700 1 -- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8f80557c0 msgr2=0x7fd8f8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.674+0000 7fd925016700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8f80557c0 0x7fd8f8057c80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fd9200a7050 tx=0x7fd914008040).stop 2020-11-20T12:21:23.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.678+0000 7fd925016700 1 -- 172.21.15.116:0/550992511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9200a81f0 msgr2=0x7fd92015cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.678+0000 7fd925016700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9200a81f0 0x7fd92015cce0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7fd91000c6e0 tx=0x7fd910007c50).stop 2020-11-20T12:21:23.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.678+0000 7fd925016700 1 -- 172.21.15.116:0/550992511 shutdown_connections 2020-11-20T12:21:23.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.678+0000 7fd925016700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd8f80557c0 0x7fd8f8057c80 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.678+0000 7fd925016700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd92014f880 0x7fd9200c3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.678+0000 7fd925016700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd9200a81f0 0x7fd92015cce0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.678+0000 7fd925016700 1 --2- 172.21.15.116:0/550992511 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd9200c0e90 0x7fd92015de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.678+0000 7fd925016700 1 -- 172.21.15.116:0/550992511 >> 172.21.15.116:0/550992511 conn(0x7fd920068350 msgr2=0x7fd9200a7680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:23.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.678+0000 7fd925016700 1 -- 172.21.15.116:0/550992511 shutdown_connections 2020-11-20T12:21:23.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.678+0000 7fd925016700 1 -- 172.21.15.116:0/550992511 wait complete. 2020-11-20T12:21:23.707 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:21:23.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.890+0000 7f3b41a38700 1 Processor -- start 2020-11-20T12:21:23.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.890+0000 7f3b41a38700 1 -- start start 2020-11-20T12:21:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.890+0000 7f3b41a38700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3b3c0a8400 0x7f3b3c0aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.890+0000 7f3b41a38700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b3c0ad540 0x7f3b3c0ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.890+0000 7f3b41a38700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3c0ace40 con 0x7f3b3c0aa7d0 2020-11-20T12:21:23.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.890+0000 7f3b41a38700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b3c1526c0 con 0x7f3b3c0a8400 2020-11-20T12:21:23.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.890+0000 7f3b41a38700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c152960 con 0x7f3b3c0ad540 2020-11-20T12:21:23.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b41237700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b3c0ad540 0x7f3b3c0ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b33fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3b3c0a8400 0x7f3b3c0aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b33fff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3b3c0a8400 0x7f3b3c0aabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:49918/0 (socket says 172.21.15.116:49918) 2020-11-20T12:21:23.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b33fff700 1 -- 172.21.15.116:0/3045348574 learned_addr learned my addr 172.21.15.116:0/3045348574 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:23.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b33fff700 1 -- 172.21.15.116:0/3045348574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b3c0ad540 msgr2=0x7f3b3c0ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:23.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b33fff700 1 --2- 172.21.15.116:0/3045348574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b3c0ad540 0x7f3b3c0ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b33fff700 1 -- 172.21.15.116:0/3045348574 >> v1:172.21.15.173:6789/0 conn(0x7f3b3c0aa7d0 legacy=0x7f3b3c151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b33fff700 1 -- 172.21.15.116:0/3045348574 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c152c00 con 0x7f3b3c0a8400 2020-11-20T12:21:23.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b337fe700 1 -- 172.21.15.116:0/3045348574 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954819204 0 0) 0x7f3b3c0ace40 con 0x7f3b3c0aa7d0 2020-11-20T12:21:23.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b41237700 1 --2- 172.21.15.116:0/3045348574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b3c0ad540 0x7f3b3c0ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:23.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b33fff700 1 --2- 172.21.15.116:0/3045348574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c0a8400 0x7f3b3c0aabb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f3b2c006fd0 tx=0x7f3b2c006eb0).ready entity=mon.1 client_cookie=fca79bfccd49636e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:23.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b337fe700 1 -- 172.21.15.116:0/3045348574 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c009810 con 0x7f3b3c0a8400 2020-11-20T12:21:23.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b337fe700 1 -- 172.21.15.116:0/3045348574 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c009970 con 0x7f3b3c0a8400 2020-11-20T12:21:23.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b337fe700 1 -- 172.21.15.116:0/3045348574 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c009cf0 con 0x7f3b3c0a8400 2020-11-20T12:21:23.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b41a38700 1 -- 172.21.15.116:0/3045348574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c0a8400 msgr2=0x7f3b3c0aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b41a38700 1 --2- 172.21.15.116:0/3045348574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c0a8400 0x7f3b3c0aabb0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f3b2c006fd0 tx=0x7f3b2c006eb0).stop 2020-11-20T12:21:23.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b41a38700 1 -- 172.21.15.116:0/3045348574 shutdown_connections 2020-11-20T12:21:23.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b41a38700 1 --2- 172.21.15.116:0/3045348574 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b3c0ad540 0x7f3b3c0ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b41a38700 1 --2- 172.21.15.116:0/3045348574 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c0a8400 0x7f3b3c0aabb0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.894+0000 7f3b41a38700 1 -- 172.21.15.116:0/3045348574 >> 172.21.15.116:0/3045348574 conn(0x7f3b3c0678a0 msgr2=0x7f3b3c067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:23.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.898+0000 7f3b41a38700 1 -- 172.21.15.116:0/3045348574 shutdown_connections 2020-11-20T12:21:23.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.898+0000 7f3b41a38700 1 -- 172.21.15.116:0/3045348574 wait complete. 2020-11-20T12:21:23.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.898+0000 7f3b41a38700 1 Processor -- start 2020-11-20T12:21:23.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.898+0000 7f3b41a38700 1 -- start start 2020-11-20T12:21:23.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.898+0000 7f3b41a38700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b3c0aa7d0 0x7f3b3c14d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.898+0000 7f3b41a38700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b3c0ad540 0x7f3b3c14f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.898+0000 7f3b41a38700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c1611a0 0x7f3b3c163de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.898+0000 7f3b41a38700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c14fe20 con 0x7f3b3c0ad540 2020-11-20T12:21:23.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.898+0000 7f3b41a38700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c150070 con 0x7f3b3c1611a0 2020-11-20T12:21:23.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.898+0000 7f3b41a38700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c156340 con 0x7f3b3c0aa7d0 2020-11-20T12:21:23.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b33fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b3c0ad540 0x7f3b3c14f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b40a36700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b3c0aa7d0 0x7f3b3c14d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b33fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b3c0ad540 0x7f3b3c14f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50874/0 (socket says 172.21.15.116:50874) 2020-11-20T12:21:23.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b33fff700 1 -- 172.21.15.116:0/2928673421 learned_addr learned my addr 172.21.15.116:0/2928673421 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:23.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b41237700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c1611a0 0x7f3b3c163de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b33fff700 1 -- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b3c0aa7d0 msgr2=0x7f3b3c14d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b33fff700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b3c0aa7d0 0x7f3b3c14d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b33fff700 1 -- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c1611a0 msgr2=0x7f3b3c163de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:23.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b33fff700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c1611a0 0x7f3b3c163de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:23.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b33fff700 1 -- 172.21.15.116:0/2928673421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c1565e0 con 0x7f3b3c0ad540 2020-11-20T12:21:23.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b33fff700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b3c0ad540 0x7f3b3c14f930 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f3b2c010000 tx=0x7f3b2c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:23.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b30ff9700 1 -- 172.21.15.116:0/2928673421 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c011d00 con 0x7f3b3c0ad540 2020-11-20T12:21:23.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b41a38700 1 -- 172.21.15.116:0/2928673421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c156880 con 0x7f3b3c0ad540 2020-11-20T12:21:23.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b30ff9700 1 -- 172.21.15.116:0/2928673421 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c011e60 con 0x7f3b3c0ad540 2020-11-20T12:21:23.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b30ff9700 1 -- 172.21.15.116:0/2928673421 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c01e6b0 con 0x7f3b3c0ad540 2020-11-20T12:21:23.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.902+0000 7f3b41a38700 1 -- 172.21.15.116:0/2928673421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c161600 con 0x7f3b3c0ad540 2020-11-20T12:21:23.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.906+0000 7f3b41a38700 1 -- 172.21.15.116:0/2928673421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3c05cf30 con 0x7f3b3c0ad540 2020-11-20T12:21:23.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.906+0000 7f3b30ff9700 1 -- 172.21.15.116:0/2928673421 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3b2c025030 con 0x7f3b3c0ad540 2020-11-20T12:21:23.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.906+0000 7f3b30ff9700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b140557c0 0x7f3b14057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:23.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.906+0000 7f3b30ff9700 1 -- 172.21.15.116:0/2928673421 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3b2c06fae0 con 0x7f3b3c0ad540 2020-11-20T12:21:23.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.906+0000 7f3b40a36700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b140557c0 0x7f3b14057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:23.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.910+0000 7f3b40a36700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b140557c0 0x7f3b14057c80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f3b38007fd0 tx=0x7f3b38007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:23.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:23.914+0000 7f3b30ff9700 1 -- 172.21.15.116:0/2928673421 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3b2c045a30 con 0x7f3b3c0ad540 2020-11-20T12:21:24.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.070+0000 7f3b41a38700 1 -- 172.21.15.116:0/2928673421 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3b3c0601b0 con 0x7f3b3c0ad540 2020-11-20T12:21:24.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.070+0000 7f3b30ff9700 1 -- 172.21.15.116:0/2928673421 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3b2c023030 con 0x7f3b3c0ad540 2020-11-20T12:21:24.076 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:24.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 -- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b140557c0 msgr2=0x7f3b14057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b140557c0 0x7f3b14057c80 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f3b38007fd0 tx=0x7f3b38007a00).stop 2020-11-20T12:21:24.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 -- 172.21.15.116:0/2928673421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b3c0ad540 msgr2=0x7f3b3c14f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b3c0ad540 0x7f3b3c14f930 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f3b2c010000 tx=0x7f3b2c015000).stop 2020-11-20T12:21:24.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 -- 172.21.15.116:0/2928673421 shutdown_connections 2020-11-20T12:21:24.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3b140557c0 0x7f3b14057c80 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3b3c1611a0 0x7f3b3c163de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3b3c0aa7d0 0x7f3b3c14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 --2- 172.21.15.116:0/2928673421 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3b3c0ad540 0x7f3b3c14f930 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 -- 172.21.15.116:0/2928673421 >> 172.21.15.116:0/2928673421 conn(0x7f3b3c0678a0 msgr2=0x7f3b3c152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:24.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 -- 172.21.15.116:0/2928673421 shutdown_connections 2020-11-20T12:21:24.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.074+0000 7f3b41a38700 1 -- 172.21.15.116:0/2928673421 wait complete. 2020-11-20T12:21:24.103 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:21:24.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.274+0000 7fae196c8700 1 Processor -- start 2020-11-20T12:21:24.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.274+0000 7fae196c8700 1 -- start start 2020-11-20T12:21:24.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.274+0000 7fae196c8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae14156440 0x7fae140aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.274+0000 7fae196c8700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae140ace40 con 0x7fae140a8400 2020-11-20T12:21:24.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.274+0000 7fae196c8700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae141526c0 con 0x7fae140aa7d0 2020-11-20T12:21:24.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.274+0000 7fae196c8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae14152960 con 0x7fae14156440 2020-11-20T12:21:24.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.274+0000 7fae137fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fae140a8400 0x7fae140ad540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60318/0 (socket says 172.21.15.116:60318) 2020-11-20T12:21:24.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.274+0000 7fae137fe700 1 -- 172.21.15.116:0/3254776798 learned_addr learned my addr 172.21.15.116:0/3254776798 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:24.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.274+0000 7fae18ec7700 1 --2- 172.21.15.116:0/3254776798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae14156440 0x7fae140aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae12ffd700 1 -- 172.21.15.116:0/3254776798 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806328288 0 0) 0x7fae140ace40 con 0x7fae140a8400 2020-11-20T12:21:24.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae12ffd700 1 -- 172.21.15.116:0/3254776798 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae00003240 con 0x7fae140a8400 2020-11-20T12:21:24.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae18ec7700 1 -- 172.21.15.116:0/3254776798 >> v1:172.21.15.173:6789/0 conn(0x7fae140aa7d0 legacy=0x7fae14151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae18ec7700 1 -- 172.21.15.116:0/3254776798 >> v1:172.21.15.116:6789/0 conn(0x7fae140a8400 legacy=0x7fae140ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae18ec7700 1 -- 172.21.15.116:0/3254776798 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae14152c00 con 0x7fae14156440 2020-11-20T12:21:24.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae18ec7700 1 --2- 172.21.15.116:0/3254776798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae14156440 0x7fae140aabb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fae0c009cc0 tx=0x7fae0c0091d0).ready entity=mon.2 client_cookie=dfffed803686dee7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:24.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae12ffd700 1 -- 172.21.15.116:0/3254776798 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c00bad0 con 0x7fae14156440 2020-11-20T12:21:24.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae12ffd700 1 -- 172.21.15.116:0/3254776798 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c00bc30 con 0x7fae14156440 2020-11-20T12:21:24.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae12ffd700 1 -- 172.21.15.116:0/3254776798 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c011400 con 0x7fae14156440 2020-11-20T12:21:24.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae196c8700 1 -- 172.21.15.116:0/3254776798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae14156440 msgr2=0x7fae140aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae196c8700 1 --2- 172.21.15.116:0/3254776798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae14156440 0x7fae140aabb0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fae0c009cc0 tx=0x7fae0c0091d0).stop 2020-11-20T12:21:24.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae196c8700 1 -- 172.21.15.116:0/3254776798 shutdown_connections 2020-11-20T12:21:24.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae196c8700 1 --2- 172.21.15.116:0/3254776798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae14156440 0x7fae140aabb0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae196c8700 1 -- 172.21.15.116:0/3254776798 >> 172.21.15.116:0/3254776798 conn(0x7fae140678a0 msgr2=0x7fae14067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:24.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae196c8700 1 -- 172.21.15.116:0/3254776798 shutdown_connections 2020-11-20T12:21:24.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.278+0000 7fae196c8700 1 -- 172.21.15.116:0/3254776798 wait complete. 2020-11-20T12:21:24.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae196c8700 1 Processor -- start 2020-11-20T12:21:24.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae196c8700 1 -- start start 2020-11-20T12:21:24.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae196c8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae140a8400 0x7fae1414e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae196c8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae140aa7d0 0x7fae141644e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae196c8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae14156440 0x7fae14149dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae196c8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1414ec00 con 0x7fae14156440 2020-11-20T12:21:24.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae196c8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae14160f80 con 0x7fae140a8400 2020-11-20T12:21:24.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae196c8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae141649d0 con 0x7fae140aa7d0 2020-11-20T12:21:24.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae18ec7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae14156440 0x7fae14149dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae18ec7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae14156440 0x7fae14149dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50890/0 (socket says 172.21.15.116:50890) 2020-11-20T12:21:24.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae18ec7700 1 -- 172.21.15.116:0/2224843371 learned_addr learned my addr 172.21.15.116:0/2224843371 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:24.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae137fe700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae140aa7d0 0x7fae141644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae13fff700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae140a8400 0x7fae1414e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae18ec7700 1 -- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae140aa7d0 msgr2=0x7fae141644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae18ec7700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae140aa7d0 0x7fae141644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae18ec7700 1 -- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae140a8400 msgr2=0x7fae1414e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae18ec7700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae140a8400 0x7fae1414e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.282+0000 7fae18ec7700 1 -- 172.21.15.116:0/2224843371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae14164c60 con 0x7fae14156440 2020-11-20T12:21:24.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.286+0000 7fae18ec7700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae14156440 0x7fae14149dd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7fae0c011000 tx=0x7fae0c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:24.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.286+0000 7fadf3fff700 1 -- 172.21.15.116:0/2224843371 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c006ea0 con 0x7fae14156440 2020-11-20T12:21:24.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.286+0000 7fadf3fff700 1 -- 172.21.15.116:0/2224843371 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c01c5f0 con 0x7fae14156440 2020-11-20T12:21:24.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.286+0000 7fae196c8700 1 -- 172.21.15.116:0/2224843371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae14164f00 con 0x7fae14156440 2020-11-20T12:21:24.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.286+0000 7fadf3fff700 1 -- 172.21.15.116:0/2224843371 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c01c990 con 0x7fae14156440 2020-11-20T12:21:24.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.286+0000 7fae196c8700 1 -- 172.21.15.116:0/2224843371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae141611a0 con 0x7fae14156440 2020-11-20T12:21:24.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.286+0000 7fae196c8700 1 -- 172.21.15.116:0/2224843371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae14161330 con 0x7fae14156440 2020-11-20T12:21:24.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.286+0000 7fadf3fff700 1 -- 172.21.15.116:0/2224843371 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fae0c017070 con 0x7fae14156440 2020-11-20T12:21:24.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.290+0000 7fadf3fff700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadec0557c0 0x7fadec057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.290+0000 7fadf3fff700 1 -- 172.21.15.116:0/2224843371 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fae0c0708d0 con 0x7fae14156440 2020-11-20T12:21:24.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.290+0000 7fae13fff700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadec0557c0 0x7fadec057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.290+0000 7fae13fff700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadec0557c0 0x7fadec057c80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fae04003610 tx=0x7fae04000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:24.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.294+0000 7fadf3fff700 1 -- 172.21.15.116:0/2224843371 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fae0c046770 con 0x7fae14156440 2020-11-20T12:21:24.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fae196c8700 1 -- 172.21.15.116:0/2224843371 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae1405cf30 con 0x7fae14156440 2020-11-20T12:21:24.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fadf3fff700 1 -- 172.21.15.116:0/2224843371 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+10 (secure 0 0 0) 0x7fae0c049d90 con 0x7fae14156440 2020-11-20T12:21:24.458 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fae196c8700 1 -- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadec0557c0 msgr2=0x7fadec057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fae196c8700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadec0557c0 0x7fadec057c80 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fae04003610 tx=0x7fae04000b80).stop 2020-11-20T12:21:24.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fae196c8700 1 -- 172.21.15.116:0/2224843371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae14156440 msgr2=0x7fae14149dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fae196c8700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae14156440 0x7fae14149dd0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7fae0c011000 tx=0x7fae0c016000).stop 2020-11-20T12:21:24.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fae196c8700 1 -- 172.21.15.116:0/2224843371 shutdown_connections 2020-11-20T12:21:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fae196c8700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fadec0557c0 0x7fadec057c80 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fae196c8700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fae14156440 0x7fae14149dd0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fae196c8700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fae140a8400 0x7fae1414e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fae196c8700 1 --2- 172.21.15.116:0/2224843371 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fae140aa7d0 0x7fae141644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.454+0000 7fae196c8700 1 -- 172.21.15.116:0/2224843371 >> 172.21.15.116:0/2224843371 conn(0x7fae140678a0 msgr2=0x7fae1414bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.458+0000 7fae196c8700 1 -- 172.21.15.116:0/2224843371 shutdown_connections 2020-11-20T12:21:24.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.458+0000 7fae196c8700 1 -- 172.21.15.116:0/2224843371 wait complete. 2020-11-20T12:21:24.483 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:21:24.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.654+0000 7fcf68a1f700 1 Processor -- start 2020-11-20T12:21:24.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.654+0000 7fcf68a1f700 1 -- start start 2020-11-20T12:21:24.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.654+0000 7fcf68a1f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf640606f0 0x7fcf640bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.654+0000 7fcf68a1f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcf640bfaa0 0x7fcf640c1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.654+0000 7fcf68a1f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf64066b90 con 0x7fcf64060310 2020-11-20T12:21:24.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.654+0000 7fcf68a1f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf64066e00 con 0x7fcf640bfaa0 2020-11-20T12:21:24.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.654+0000 7fcf68a1f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640670a0 con 0x7fcf640606f0 2020-11-20T12:21:24.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf63fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcf640bfaa0 0x7fcf640c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.654+0000 7fcf637fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fcf64060310 0x7fcf64066480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60330/0 (socket says 172.21.15.116:60330) 2020-11-20T12:21:24.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf637fe700 1 -- 172.21.15.116:0/1136714889 learned_addr learned my addr 172.21.15.116:0/1136714889 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:24.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf62ffd700 1 --2- 172.21.15.116:0/1136714889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf640606f0 0x7fcf640bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf63fff700 1 -- 172.21.15.116:0/1136714889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf640606f0 msgr2=0x7fcf640bf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:24.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf63fff700 1 --2- 172.21.15.116:0/1136714889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf640606f0 0x7fcf640bf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf63fff700 1 -- 172.21.15.116:0/1136714889 >> v1:172.21.15.116:6789/0 conn(0x7fcf64060310 legacy=0x7fcf64066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf63fff700 1 -- 172.21.15.116:0/1136714889 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf64067340 con 0x7fcf640bfaa0 2020-11-20T12:21:24.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf627fc700 1 -- 172.21.15.116:0/1136714889 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803872322 0 0) 0x7fcf64066b90 con 0x7fcf64060310 2020-11-20T12:21:24.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf63fff700 1 --2- 172.21.15.116:0/1136714889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf640bfaa0 0x7fcf640c1470 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fcf5c009c90 tx=0x7fcf5c0091d0).ready entity=mon.0 client_cookie=f21c7f469161a85c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:24.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf627fc700 1 -- 172.21.15.116:0/1136714889 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c00ba70 con 0x7fcf640bfaa0 2020-11-20T12:21:24.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf627fc700 1 -- 172.21.15.116:0/1136714889 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5c00bbd0 con 0x7fcf640bfaa0 2020-11-20T12:21:24.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf627fc700 1 -- 172.21.15.116:0/1136714889 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5c011400 con 0x7fcf640bfaa0 2020-11-20T12:21:24.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf68a1f700 1 -- 172.21.15.116:0/1136714889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf640bfaa0 msgr2=0x7fcf640c1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf68a1f700 1 --2- 172.21.15.116:0/1136714889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf640bfaa0 0x7fcf640c1470 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7fcf5c009c90 tx=0x7fcf5c0091d0).stop 2020-11-20T12:21:24.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf68a1f700 1 -- 172.21.15.116:0/1136714889 shutdown_connections 2020-11-20T12:21:24.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf68a1f700 1 --2- 172.21.15.116:0/1136714889 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf640bfaa0 0x7fcf640c1470 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf68a1f700 1 --2- 172.21.15.116:0/1136714889 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf640606f0 0x7fcf640bf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf68a1f700 1 -- 172.21.15.116:0/1136714889 >> 172.21.15.116:0/1136714889 conn(0x7fcf64068350 msgr2=0x7fcf640627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:24.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf68a1f700 1 -- 172.21.15.116:0/1136714889 shutdown_connections 2020-11-20T12:21:24.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.658+0000 7fcf68a1f700 1 -- 172.21.15.116:0/1136714889 wait complete. 2020-11-20T12:21:24.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.662+0000 7fcf68a1f700 1 Processor -- start 2020-11-20T12:21:24.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.662+0000 7fcf68a1f700 1 -- start start 2020-11-20T12:21:24.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf68a1f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf64060310 0x7fcf640bc870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf68a1f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf640606f0 0x7fcf640c2210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf68a1f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf640bfaa0 0x7fcf641646b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf68a1f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640bb950 con 0x7fcf640606f0 2020-11-20T12:21:24.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf68a1f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640bbbf0 con 0x7fcf64060310 2020-11-20T12:21:24.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf68a1f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640bbe90 con 0x7fcf640bfaa0 2020-11-20T12:21:24.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf62ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf640606f0 0x7fcf640c2210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf62ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf640606f0 0x7fcf640c2210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50902/0 (socket says 172.21.15.116:50902) 2020-11-20T12:21:24.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf62ffd700 1 -- 172.21.15.116:0/3034618812 learned_addr learned my addr 172.21.15.116:0/3034618812 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:24.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf637fe700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf64060310 0x7fcf640bc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf63fff700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf640bfaa0 0x7fcf641646b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf62ffd700 1 -- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf640bfaa0 msgr2=0x7fcf641646b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf62ffd700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf640bfaa0 0x7fcf641646b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf62ffd700 1 -- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf64060310 msgr2=0x7fcf640bc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf62ffd700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf64060310 0x7fcf640bc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf62ffd700 1 -- 172.21.15.116:0/3034618812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf640bc130 con 0x7fcf640606f0 2020-11-20T12:21:24.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf62ffd700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf640606f0 0x7fcf640c2210 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fcf5800edc0 tx=0x7fcf58008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:24.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf437fe700 1 -- 172.21.15.116:0/3034618812 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800ab70 con 0x7fcf640606f0 2020-11-20T12:21:24.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf437fe700 1 -- 172.21.15.116:0/3034618812 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5800acd0 con 0x7fcf640606f0 2020-11-20T12:21:24.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf437fe700 1 -- 172.21.15.116:0/3034618812 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800ae40 con 0x7fcf640606f0 2020-11-20T12:21:24.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf68a1f700 1 -- 172.21.15.116:0/3034618812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf640bc3d0 con 0x7fcf640606f0 2020-11-20T12:21:24.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.666+0000 7fcf68a1f700 1 -- 172.21.15.116:0/3034618812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf640bb520 con 0x7fcf640606f0 2020-11-20T12:21:24.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.670+0000 7fcf68a1f700 1 -- 172.21.15.116:0/3034618812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf640bc560 con 0x7fcf640606f0 2020-11-20T12:21:24.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.670+0000 7fcf437fe700 1 -- 172.21.15.116:0/3034618812 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fcf58020070 con 0x7fcf640606f0 2020-11-20T12:21:24.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.670+0000 7fcf437fe700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcf3c0557c0 0x7fcf3c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:24.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.670+0000 7fcf437fe700 1 -- 172.21.15.116:0/3034618812 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fcf5806e680 con 0x7fcf640606f0 2020-11-20T12:21:24.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.670+0000 7fcf637fe700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcf3c0557c0 0x7fcf3c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:24.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.674+0000 7fcf637fe700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcf3c0557c0 0x7fcf3c057c80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fcf54006780 tx=0x7fcf54007c20).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:24.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.674+0000 7fcf437fe700 1 -- 172.21.15.116:0/3034618812 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcf58044410 con 0x7fcf640606f0 2020-11-20T12:21:24.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf68a1f700 1 -- 172.21.15.116:0/3034618812 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcf6405cf30 con 0x7fcf640606f0 2020-11-20T12:21:24.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf437fe700 1 -- 172.21.15.116:0/3034618812 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+10 (secure 0 0 0) 0x7fcf5801e070 con 0x7fcf640606f0 2020-11-20T12:21:24.842 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:24.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf68a1f700 1 -- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcf3c0557c0 msgr2=0x7fcf3c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf68a1f700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcf3c0557c0 0x7fcf3c057c80 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fcf54006780 tx=0x7fcf54007c20).stop 2020-11-20T12:21:24.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf68a1f700 1 -- 172.21.15.116:0/3034618812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf640606f0 msgr2=0x7fcf640c2210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:24.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf68a1f700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf640606f0 0x7fcf640c2210 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7fcf5800edc0 tx=0x7fcf58008040).stop 2020-11-20T12:21:24.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf68a1f700 1 -- 172.21.15.116:0/3034618812 shutdown_connections 2020-11-20T12:21:24.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf68a1f700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcf3c0557c0 0x7fcf3c057c80 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf68a1f700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcf640bfaa0 0x7fcf641646b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf68a1f700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcf64060310 0x7fcf640bc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf68a1f700 1 --2- 172.21.15.116:0/3034618812 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcf640606f0 0x7fcf640c2210 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:24.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.838+0000 7fcf68a1f700 1 -- 172.21.15.116:0/3034618812 >> 172.21.15.116:0/3034618812 conn(0x7fcf64068350 msgr2=0x7fcf640bffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:24.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.842+0000 7fcf68a1f700 1 -- 172.21.15.116:0/3034618812 shutdown_connections 2020-11-20T12:21:24.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:24.842+0000 7fcf68a1f700 1 -- 172.21.15.116:0/3034618812 wait complete. 2020-11-20T12:21:24.866 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:21:24.867 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:21:25.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.034+0000 7f1545787700 1 Processor -- start 2020-11-20T12:21:25.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.034+0000 7f1545787700 1 -- start start 2020-11-20T12:21:25.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f1545787700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f154014baa0 0x7f154014be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:25.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f1545787700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1540065a20 0x7f1540151d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:25.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f1545787700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1540152810 con 0x7f1540065e80 2020-11-20T12:21:25.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f1545787700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1540152ab0 con 0x7f154014baa0 2020-11-20T12:21:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f1545787700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15400652e0 con 0x7f1540065a20 2020-11-20T12:21:25.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f153ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f154014baa0 0x7f154014be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f153ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f154014baa0 0x7f154014be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50912/0 (socket says 172.21.15.116:50912) 2020-11-20T12:21:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f153ffff700 1 -- 172.21.15.116:0/2510619161 learned_addr learned my addr 172.21.15.116:0/2510619161 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:25.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f153f7fe700 1 --2- 172.21.15.116:0/2510619161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1540065a20 0x7f1540151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f153ffff700 1 -- 172.21.15.116:0/2510619161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1540065a20 msgr2=0x7f1540151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:25.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f153ffff700 1 --2- 172.21.15.116:0/2510619161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1540065a20 0x7f1540151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:25.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f153ffff700 1 -- 172.21.15.116:0/2510619161 >> v1:172.21.15.173:6789/0 conn(0x7f1540065e80 legacy=0x7f1540060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:25.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f153effd700 1 -- 172.21.15.116:0/2510619161 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4267637342 0 0) 0x7f1540152810 con 0x7f1540065e80 2020-11-20T12:21:25.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f153ffff700 1 -- 172.21.15.116:0/2510619161 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540065580 con 0x7f154014baa0 2020-11-20T12:21:25.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.038+0000 7f153ffff700 1 --2- 172.21.15.116:0/2510619161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154014baa0 0x7f154014be80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f1530006fd0 tx=0x7f1530006eb0).ready entity=mon.0 client_cookie=f8e698f2f2f060b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:25.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f153effd700 1 -- 172.21.15.116:0/2510619161 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530009810 con 0x7f154014baa0 2020-11-20T12:21:25.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f153effd700 1 -- 172.21.15.116:0/2510619161 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1530009970 con 0x7f154014baa0 2020-11-20T12:21:25.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f153effd700 1 -- 172.21.15.116:0/2510619161 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1530009cf0 con 0x7f154014baa0 2020-11-20T12:21:25.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f1545787700 1 -- 172.21.15.116:0/2510619161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154014baa0 msgr2=0x7f154014be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:25.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f1545787700 1 --2- 172.21.15.116:0/2510619161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154014baa0 0x7f154014be80 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f1530006fd0 tx=0x7f1530006eb0).stop 2020-11-20T12:21:25.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f1545787700 1 -- 172.21.15.116:0/2510619161 shutdown_connections 2020-11-20T12:21:25.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f1545787700 1 --2- 172.21.15.116:0/2510619161 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f154014baa0 0x7f154014be80 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:25.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f1545787700 1 --2- 172.21.15.116:0/2510619161 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1540065a20 0x7f1540151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:25.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f1545787700 1 -- 172.21.15.116:0/2510619161 >> 172.21.15.116:0/2510619161 conn(0x7f1540061280 msgr2=0x7f1540061660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:25.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f1545787700 1 -- 172.21.15.116:0/2510619161 shutdown_connections 2020-11-20T12:21:25.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f1545787700 1 -- 172.21.15.116:0/2510619161 wait complete. 2020-11-20T12:21:25.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.042+0000 7f1545787700 1 Processor -- start 2020-11-20T12:21:25.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f1545787700 1 -- start start 2020-11-20T12:21:25.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f1545787700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1540065a20 0x7f1540161db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:25.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f1545787700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1540065e80 0x7f1540148e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:25.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f1545787700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f154014baa0 0x7f154014ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:25.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f1545787700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15401622a0 con 0x7f1540065e80 2020-11-20T12:21:25.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f1545787700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15400a8900 con 0x7f1540065a20 2020-11-20T12:21:25.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f1545787700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15400a8ba0 con 0x7f154014baa0 2020-11-20T12:21:25.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f153ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1540065a20 0x7f1540161db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:25.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f153f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1540065e80 0x7f1540148e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:25.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f153f7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1540065e80 0x7f1540148e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50916/0 (socket says 172.21.15.116:50916) 2020-11-20T12:21:25.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f153ffff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1540065a20 0x7f1540161db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:49966/0 (socket says 172.21.15.116:49966) 2020-11-20T12:21:25.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f153f7fe700 1 -- 172.21.15.116:0/799914844 learned_addr learned my addr 172.21.15.116:0/799914844 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:25.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f1544f86700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f154014baa0 0x7f154014ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:25.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f153f7fe700 1 -- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f154014baa0 msgr2=0x7f154014ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:25.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f153f7fe700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f154014baa0 0x7f154014ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:25.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f153f7fe700 1 -- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1540065a20 msgr2=0x7f1540161db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:25.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f153f7fe700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1540065a20 0x7f1540161db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:25.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f153f7fe700 1 -- 172.21.15.116:0/799914844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15400a8e40 con 0x7f1540065e80 2020-11-20T12:21:25.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.046+0000 7f1544f86700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f154014baa0 0x7f154014ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:25.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.050+0000 7f153f7fe700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1540065e80 0x7f1540148e80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f153400eba0 tx=0x7f1534006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:25.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.050+0000 7f151ffff700 1 -- 172.21.15.116:0/799914844 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400c9b0 con 0x7f1540065e80 2020-11-20T12:21:25.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.050+0000 7f1545787700 1 -- 172.21.15.116:0/799914844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15400a90e0 con 0x7f1540065e80 2020-11-20T12:21:25.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.050+0000 7f151ffff700 1 -- 172.21.15.116:0/799914844 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f153400cb10 con 0x7f1540065e80 2020-11-20T12:21:25.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.050+0000 7f151ffff700 1 -- 172.21.15.116:0/799914844 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400ce90 con 0x7f1540065e80 2020-11-20T12:21:25.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.050+0000 7f1545787700 1 -- 172.21.15.116:0/799914844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15400a9820 con 0x7f1540065e80 2020-11-20T12:21:25.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.050+0000 7f1545787700 1 -- 172.21.15.116:0/799914844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1540069bc0 con 0x7f1540065e80 2020-11-20T12:21:25.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.050+0000 7f151ffff700 1 -- 172.21.15.116:0/799914844 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1534012600 con 0x7f1540065e80 2020-11-20T12:21:25.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.050+0000 7f151ffff700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15180557c0 0x7f1518057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:25.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.054+0000 7f151ffff700 1 -- 172.21.15.116:0/799914844 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f153406eb50 con 0x7f1540065e80 2020-11-20T12:21:25.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.054+0000 7f153ffff700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15180557c0 0x7f1518057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:25.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.054+0000 7f153ffff700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15180557c0 0x7f1518057c80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f153000b1b0 tx=0x7f1530015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:25.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.058+0000 7f151ffff700 1 -- 172.21.15.116:0/799914844 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1540069bc0 con 0x7f1540065e80 2020-11-20T12:21:25.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.266+0000 7f1545787700 1 -- 172.21.15.116:0/799914844 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1540069bc0 con 0x7f1540065e80 2020-11-20T12:21:25.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f151ffff700 1 -- 172.21.15.116:0/799914844 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2851 (secure 0 0 0) 0x7f1540069bc0 con 0x7f1540065e80 2020-11-20T12:21:25.273 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:25.273 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1638614527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1638614527},{"type":"v1","addr":"172.21.15.173:6837","nonce":1638614527}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3682335131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3682335131},{"type":"v1","addr":"172.21.15.173:6836","nonce":3682335131}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7725,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/3975405805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3975405805},{"type":"v1","addr":"172.21.15.116:6834","nonce":3975405805}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:11.109567+0000","modified":"2020-11-20T12:21:13.118616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/267899152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":267899152},{"type":"v1","addr":"172.21.15.116:6835","nonce":267899152}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-20T12:21:25.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 -- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15180557c0 msgr2=0x7f1518057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:25.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15180557c0 0x7f1518057c80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f153000b1b0 tx=0x7f1530015000).stop 2020-11-20T12:21:25.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 -- 172.21.15.116:0/799914844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1540065e80 msgr2=0x7f1540148e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:25.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1540065e80 0x7f1540148e80 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f153400eba0 tx=0x7f1534006df0).stop 2020-11-20T12:21:25.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 -- 172.21.15.116:0/799914844 shutdown_connections 2020-11-20T12:21:25.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15180557c0 0x7f1518057c80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:25.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f154014baa0 0x7f154014ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:25.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1540065a20 0x7f1540161db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:25.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 --2- 172.21.15.116:0/799914844 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1540065e80 0x7f1540148e80 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:25.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 -- 172.21.15.116:0/799914844 >> 172.21.15.116:0/799914844 conn(0x7f1540061280 msgr2=0x7f15401500a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:25.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 -- 172.21.15.116:0/799914844 shutdown_connections 2020-11-20T12:21:25.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:25.270+0000 7f1545787700 1 -- 172.21.15.116:0/799914844 wait complete. 2020-11-20T12:21:25.277 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 60 2020-11-20T12:21:25.702 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:21:11.109567+0000', 'modified': '2020-11-20T12:21:13.118616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7728': {'gid': 7728, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/267899152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 267899152}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 267899152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:21:25.966 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:21:25.967 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:21:26.164 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":7728,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.033020738999999,"mdsmap_epoch":60,"osdmap_epoch":107,"osdmap_epoch_barrier":107,"uptime":20.538060258000002} 2020-11-20T12:21:26.165 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 7728, "mdsmap_epoch": 60, "osdmap_epoch": 107, "osdmap_epoch_barrier": 107, "rank_uptime": 15.033020739, "state": "up:active", "uptime": 20.538060258, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:21:26.165 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:21:26.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:26.166 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.2 2020-11-20T12:21:26.184 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:21:26.185 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:26.253 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:26.254 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:21:26.276 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:26.276 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:21:26.281 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:21:26.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:26.282 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2020-11-20T12:21:26.309 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:21:26.310 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:21:26.310 INFO:tasks.cephfs.mount:self.client.name = client.3 2020-11-20T12:21:26.311 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2020-11-20T12:21:26.311 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:21:26.311 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:21:26.312 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:21:26.312 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:21:26.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:26.313 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:21:26.330 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:21:26.331 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:21:26.331 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:21:26.331 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:26.332 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:21:26.332 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:26.332 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:26.332 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:26.332 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:26.333 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:26.333 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:21:26.333 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:26.333 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:21:26.333 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:26.334 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:21:26.334 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:26.334 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:21:26.334 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:26.335 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:21:26.335 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:26.335 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:21:26.335 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:26.335 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:21:26.336 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:21:26.336 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:21:26.336 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:26.336 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:21:26.337 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:26.337 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:21:26.337 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:21:26.337 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:21:26.338 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:26.338 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:21:26.338 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:21:26.338 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:21:26.338 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:21:26.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:26.340 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:21:26.375 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:21:26.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:26.379 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:21:26.394 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:21:26.394 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:21:26.394 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:21:26.395 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:21:26.395 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:21:26.395 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:21:26.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:26.396 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:21:26.396 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:21:26.397 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:21:26.397 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:21:26.398 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:21:26.398 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:21:26.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:26.540 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:21:26.555 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:21:26.556 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:21:26.556 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:21:26.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:26.557 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2020-11-20T12:21:26.587 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:26.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.770+0000 7f1fbdc56700 1 Processor -- start 2020-11-20T12:21:26.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.770+0000 7f1fbdc56700 1 -- start start 2020-11-20T12:21:26.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fbdc56700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1fb80c3780 0x7f1fb80c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fbdc56700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fb80c1970 0x7f1fb8150560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fbdc56700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb80a6950 con 0x7f1fb80a6280 2020-11-20T12:21:26.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fbdc56700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb814fed0 con 0x7f1fb80c3780 2020-11-20T12:21:26.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fbdc56700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb80a6730 con 0x7f1fb80c1970 2020-11-20T12:21:26.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fb77fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1fb80c3780 0x7f1fb80c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fb77fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1fb80c3780 0x7f1fb80c3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50926/0 (socket says 172.21.15.116:50926) 2020-11-20T12:21:26.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fb77fe700 1 -- 172.21.15.116:0/3795926569 learned_addr learned my addr 172.21.15.116:0/3795926569 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:26.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fb7fff700 1 --2- 172.21.15.116:0/3795926569 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fb80c1970 0x7f1fb8150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fb77fe700 1 -- 172.21.15.116:0/3795926569 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fb80c1970 msgr2=0x7f1fb8150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:26.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fb77fe700 1 --2- 172.21.15.116:0/3795926569 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fb80c1970 0x7f1fb8150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fb77fe700 1 -- 172.21.15.116:0/3795926569 >> v1:172.21.15.173:6789/0 conn(0x7f1fb80a6280 legacy=0x7f1fb80c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fb77fe700 1 -- 172.21.15.116:0/3795926569 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb80a6bf0 con 0x7f1fb80c3780 2020-11-20T12:21:26.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fb77fe700 1 --2- 172.21.15.116:0/3795926569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fb80c3780 0x7f1fb80c3b60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f1fa800ba20 tx=0x7f1fa8006eb0).ready entity=mon.0 client_cookie=e45f4e3ec89c633 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.774+0000 7f1fb67fc700 1 -- 172.21.15.116:0/3795926569 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80097d0 con 0x7f1fb80c3780 2020-11-20T12:21:26.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.778+0000 7f1fb67fc700 1 -- 172.21.15.116:0/3795926569 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa8009930 con 0x7f1fb80c3780 2020-11-20T12:21:26.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.778+0000 7f1fb67fc700 1 -- 172.21.15.116:0/3795926569 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8009cb0 con 0x7f1fb80c3780 2020-11-20T12:21:26.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.778+0000 7f1fbdc56700 1 -- 172.21.15.116:0/3795926569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fb80c3780 msgr2=0x7f1fb80c3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.778+0000 7f1fbdc56700 1 --2- 172.21.15.116:0/3795926569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fb80c3780 0x7f1fb80c3b60 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f1fa800ba20 tx=0x7f1fa8006eb0).stop 2020-11-20T12:21:26.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.778+0000 7f1fbdc56700 1 -- 172.21.15.116:0/3795926569 shutdown_connections 2020-11-20T12:21:26.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.778+0000 7f1fbdc56700 1 --2- 172.21.15.116:0/3795926569 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fb80c1970 0x7f1fb8150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.778+0000 7f1fbdc56700 1 --2- 172.21.15.116:0/3795926569 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fb80c3780 0x7f1fb80c3b60 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.778+0000 7f1fbdc56700 1 -- 172.21.15.116:0/3795926569 >> 172.21.15.116:0/3795926569 conn(0x7f1fb80c4f80 msgr2=0x7f1fb80c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:26.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.778+0000 7f1fbdc56700 1 -- 172.21.15.116:0/3795926569 shutdown_connections 2020-11-20T12:21:26.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.778+0000 7f1fbdc56700 1 -- 172.21.15.116:0/3795926569 wait complete. 2020-11-20T12:21:26.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.782+0000 7f1fbdc56700 1 Processor -- start 2020-11-20T12:21:26.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.782+0000 7f1fbdc56700 1 -- start start 2020-11-20T12:21:26.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.782+0000 7f1fbdc56700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fb80a6280 0x7f1fb8162170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.782+0000 7f1fbdc56700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fb80c1970 0x7f1fb814c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.782+0000 7f1fbdc56700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fb80c3780 0x7f1fb80bdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.782+0000 7f1fbdc56700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb80be5f0 con 0x7f1fb80c3780 2020-11-20T12:21:26.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.782+0000 7f1fbdc56700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb8161790 con 0x7f1fb80a6280 2020-11-20T12:21:26.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.782+0000 7f1fbdc56700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb8161a30 con 0x7f1fb80c1970 2020-11-20T12:21:26.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fb7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fb80c3780 0x7f1fb80bdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fb77fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fb80a6280 0x7f1fb8162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fb7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fb80c3780 0x7f1fb80bdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50936/0 (socket says 172.21.15.116:50936) 2020-11-20T12:21:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fb7fff700 1 -- 172.21.15.116:0/4098890821 learned_addr learned my addr 172.21.15.116:0/4098890821 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fb6ffd700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fb80c1970 0x7f1fb814c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fb7fff700 1 -- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fb80c1970 msgr2=0x7f1fb814c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fb7fff700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fb80c1970 0x7f1fb814c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fb7fff700 1 -- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fb80a6280 msgr2=0x7f1fb8162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fb7fff700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fb80a6280 0x7f1fb8162170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fb7fff700 1 -- 172.21.15.116:0/4098890821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb8161cd0 con 0x7f1fb80c3780 2020-11-20T12:21:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fb7fff700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fb80c3780 0x7f1fb80bdb20 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f1fb000ef50 tx=0x7f1fb00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1f977fe700 1 -- 172.21.15.116:0/4098890821 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb000cbf0 con 0x7f1fb80c3780 2020-11-20T12:21:26.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1f977fe700 1 -- 172.21.15.116:0/4098890821 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fb000cd50 con 0x7f1fb80c3780 2020-11-20T12:21:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fbdc56700 1 -- 172.21.15.116:0/4098890821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb814c870 con 0x7f1fb80c3780 2020-11-20T12:21:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1f977fe700 1 -- 172.21.15.116:0/4098890821 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fb0012620 con 0x7f1fb80c3780 2020-11-20T12:21:26.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fbdc56700 1 -- 172.21.15.116:0/4098890821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb814f3c0 con 0x7f1fb80c3780 2020-11-20T12:21:26.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.786+0000 7f1fbdc56700 1 -- 172.21.15.116:0/4098890821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb81615e0 con 0x7f1fb80c3780 2020-11-20T12:21:26.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.790+0000 7f1f977fe700 1 -- 172.21.15.116:0/4098890821 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1fb0022070 con 0x7f1fb80c3780 2020-11-20T12:21:26.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.790+0000 7f1f977fe700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f900557c0 0x7f1f90057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:26.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.790+0000 7f1f977fe700 1 -- 172.21.15.116:0/4098890821 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1fb006f0d0 con 0x7f1fb80c3780 2020-11-20T12:21:26.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.790+0000 7f1fb77fe700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f900557c0 0x7f1f90057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:26.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.794+0000 7f1fb77fe700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f900557c0 0x7f1f90057c80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f1fa8006ef0 tx=0x7f1fa8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:26.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.794+0000 7f1f977fe700 1 -- 172.21.15.116:0/4098890821 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1fb0044e60 con 0x7f1fb80c3780 2020-11-20T12:21:26.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.954+0000 7f1fbdc56700 1 -- 172.21.15.116:0/4098890821 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fb80a6dd0 con 0x7f1fb80c3780 2020-11-20T12:21:26.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.958+0000 7f1f977fe700 1 -- 172.21.15.116:0/4098890821 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f1fb0016070 con 0x7f1fb80c3780 2020-11-20T12:21:26.962 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:26.962 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":107,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:12.114283+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:09.571842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:10.034013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:26.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.958+0000 7f1fbdc56700 1 -- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f900557c0 msgr2=0x7f1f90057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.958+0000 7f1fbdc56700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f900557c0 0x7f1f90057c80 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f1fa8006ef0 tx=0x7f1fa8015000).stop 2020-11-20T12:21:26.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.958+0000 7f1fbdc56700 1 -- 172.21.15.116:0/4098890821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fb80c3780 msgr2=0x7f1fb80bdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:26.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.958+0000 7f1fbdc56700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fb80c3780 0x7f1fb80bdb20 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f1fb000ef50 tx=0x7f1fb00091d0).stop 2020-11-20T12:21:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.958+0000 7f1fbdc56700 1 -- 172.21.15.116:0/4098890821 shutdown_connections 2020-11-20T12:21:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.958+0000 7f1fbdc56700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1f900557c0 0x7f1f90057c80 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.958+0000 7f1fbdc56700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1fb80c3780 0x7f1fb80bdb20 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.958+0000 7f1fbdc56700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1fb80a6280 0x7f1fb8162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.958+0000 7f1fbdc56700 1 --2- 172.21.15.116:0/4098890821 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1fb80c1970 0x7f1fb814c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:26.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.958+0000 7f1fbdc56700 1 -- 172.21.15.116:0/4098890821 >> 172.21.15.116:0/4098890821 conn(0x7f1fb80c4f80 msgr2=0x7f1fb814e190 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:26.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.962+0000 7f1fbdc56700 1 -- 172.21.15.116:0/4098890821 shutdown_connections 2020-11-20T12:21:26.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:26.962+0000 7f1fbdc56700 1 -- 172.21.15.116:0/4098890821 wait complete. 2020-11-20T12:21:26.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:21:09.571842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:21:10.034013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:21:26.990 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:21:27.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.162+0000 7fb80aedb700 1 Processor -- start 2020-11-20T12:21:27.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.162+0000 7fb80aedb700 1 -- start start 2020-11-20T12:21:27.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.166+0000 7fb80aedb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb80414e590 0x7fb804063100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.166+0000 7fb80aedb700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb8040a66e0 0x7fb8040a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.166+0000 7fb80aedb700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8040635f0 con 0x7fb8040bff10 2020-11-20T12:21:27.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.166+0000 7fb80aedb700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8040c0690 con 0x7fb8040a66e0 2020-11-20T12:21:27.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.166+0000 7fb80aedb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8040c0930 con 0x7fb80414e590 2020-11-20T12:21:27.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.166+0000 7fb8096d8700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fb8040bff10 0x7fb80414f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:56606/0 (socket says 172.21.15.116:56606) 2020-11-20T12:21:27.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.166+0000 7fb8096d8700 1 -- 172.21.15.116:0/1566021121 learned_addr learned my addr 172.21.15.116:0/1566021121 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:27.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.166+0000 7fb809ed9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb80414e590 0x7fb804063100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.166+0000 7fb80a6da700 1 --2- 172.21.15.116:0/1566021121 >> v2:172.21.15.173:3300/0 conn(0x7fb8040a66e0 0x7fb8040a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb809ed9700 1 -- 172.21.15.116:0/1566021121 >> v2:172.21.15.173:3300/0 conn(0x7fb8040a66e0 msgr2=0x7fb8040a6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:27.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb809ed9700 1 --2- 172.21.15.116:0/1566021121 >> v2:172.21.15.173:3300/0 conn(0x7fb8040a66e0 0x7fb8040a6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.166+0000 7fb808ed7700 1 -- 172.21.15.116:0/1566021121 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895913100 0 0) 0x7fb8040635f0 con 0x7fb8040bff10 2020-11-20T12:21:27.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb809ed9700 1 -- 172.21.15.116:0/1566021121 >> v1:172.21.15.173:6789/0 conn(0x7fb8040bff10 legacy=0x7fb80414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb809ed9700 1 -- 172.21.15.116:0/1566021121 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8040c0bd0 con 0x7fb80414e590 2020-11-20T12:21:27.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb809ed9700 1 --2- 172.21.15.116:0/1566021121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb80414e590 0x7fb804063100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fb80000b9a0 tx=0x7fb800006df0).ready entity=mon.2 client_cookie=eb8170014fbea6fe server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:27.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb808ed7700 1 -- 172.21.15.116:0/1566021121 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8000097b0 con 0x7fb80414e590 2020-11-20T12:21:27.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb808ed7700 1 -- 172.21.15.116:0/1566021121 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb800009910 con 0x7fb80414e590 2020-11-20T12:21:27.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb808ed7700 1 -- 172.21.15.116:0/1566021121 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb800009c90 con 0x7fb80414e590 2020-11-20T12:21:27.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb80aedb700 1 -- 172.21.15.116:0/1566021121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb80414e590 msgr2=0x7fb804063100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb80aedb700 1 --2- 172.21.15.116:0/1566021121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb80414e590 0x7fb804063100 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7fb80000b9a0 tx=0x7fb800006df0).stop 2020-11-20T12:21:27.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb80aedb700 1 -- 172.21.15.116:0/1566021121 shutdown_connections 2020-11-20T12:21:27.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb80aedb700 1 --2- 172.21.15.116:0/1566021121 >> v2:172.21.15.173:3300/0 conn(0x7fb8040a66e0 0x7fb8040a6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb80aedb700 1 --2- 172.21.15.116:0/1566021121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb80414e590 0x7fb804063100 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb80aedb700 1 -- 172.21.15.116:0/1566021121 >> 172.21.15.116:0/1566021121 conn(0x7fb804068350 msgr2=0x7fb8040627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:27.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb80aedb700 1 -- 172.21.15.116:0/1566021121 shutdown_connections 2020-11-20T12:21:27.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.170+0000 7fb80aedb700 1 -- 172.21.15.116:0/1566021121 wait complete. 2020-11-20T12:21:27.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.174+0000 7fb80aedb700 1 Processor -- start 2020-11-20T12:21:27.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.174+0000 7fb80aedb700 1 -- start start 2020-11-20T12:21:27.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80aedb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040a66e0 0x7fb80415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80aedb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8040bff10 0x7fb804160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80aedb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80414e590 0x7fb80414d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80aedb700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb80415cfb0 con 0x7fb80414e590 2020-11-20T12:21:27.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80aedb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8041614c0 con 0x7fb8040bff10 2020-11-20T12:21:27.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80aedb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb80414ad10 con 0x7fb8040a66e0 2020-11-20T12:21:27.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80a6da700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80414e590 0x7fb80414d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80a6da700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80414e590 0x7fb80414d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50950/0 (socket says 172.21.15.116:50950) 2020-11-20T12:21:27.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80a6da700 1 -- 172.21.15.116:0/2402943163 learned_addr learned my addr 172.21.15.116:0/2402943163 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:27.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb809ed9700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040a66e0 0x7fb80415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb8096d8700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8040bff10 0x7fb804160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80a6da700 1 -- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040a66e0 msgr2=0x7fb80415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80a6da700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040a66e0 0x7fb80415dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80a6da700 1 -- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8040bff10 msgr2=0x7fb804160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80a6da700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8040bff10 0x7fb804160fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80a6da700 1 -- 172.21.15.116:0/2402943163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb80415d1e0 con 0x7fb80414e590 2020-11-20T12:21:27.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb8096d8700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8040bff10 0x7fb804160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:27.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80a6da700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80414e590 0x7fb80414d8e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7fb7fc00ef50 tx=0x7fb7fc0091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:27.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/2402943163 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc00cbf0 con 0x7fb80414e590 2020-11-20T12:21:27.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/2402943163 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc00cd50 con 0x7fb80414e590 2020-11-20T12:21:27.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80aedb700 1 -- 172.21.15.116:0/2402943163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb80415d480 con 0x7fb80414e590 2020-11-20T12:21:27.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/2402943163 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc012620 con 0x7fb80414e590 2020-11-20T12:21:27.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.178+0000 7fb80aedb700 1 -- 172.21.15.116:0/2402943163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb80414ddd0 con 0x7fb80414e590 2020-11-20T12:21:27.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.182+0000 7fb80aedb700 1 -- 172.21.15.116:0/2402943163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb80405cf30 con 0x7fb80414e590 2020-11-20T12:21:27.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.182+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/2402943163 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb7fc022070 con 0x7fb80414e590 2020-11-20T12:21:27.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.182+0000 7fb7f1ffb700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 0x7fb7dc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.182+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/2402943163 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb7fc06ef60 con 0x7fb80414e590 2020-11-20T12:21:27.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.182+0000 7fb809ed9700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 0x7fb7dc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.186+0000 7fb809ed9700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 0x7fb7dc057c80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fb80000b150 tx=0x7fb800015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:27.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.190+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/2402943163 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb7fc044e00 con 0x7fb80414e590 2020-11-20T12:21:27.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.350+0000 7fb80aedb700 1 -- 172.21.15.116:0/2402943163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb804066220 con 0x7fb80414e590 2020-11-20T12:21:27.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.350+0000 7fb7f1ffb700 1 -- 172.21.15.116:0/2402943163 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v107) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb7fc016070 con 0x7fb80414e590 2020-11-20T12:21:27.354 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:21:27.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 -- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 msgr2=0x7fb7dc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 0x7fb7dc057c80 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fb80000b150 tx=0x7fb800015000).stop 2020-11-20T12:21:27.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 -- 172.21.15.116:0/2402943163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80414e590 msgr2=0x7fb80414d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80414e590 0x7fb80414d8e0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7fb7fc00ef50 tx=0x7fb7fc0091d0).stop 2020-11-20T12:21:27.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 -- 172.21.15.116:0/2402943163 shutdown_connections 2020-11-20T12:21:27.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb7dc0557c0 0x7fb7dc057c80 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb80414e590 0x7fb80414d8e0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb8040a66e0 0x7fb80415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 --2- 172.21.15.116:0/2402943163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb8040bff10 0x7fb804160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 -- 172.21.15.116:0/2402943163 >> 172.21.15.116:0/2402943163 conn(0x7fb804068350 msgr2=0x7fb8040c2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:27.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 -- 172.21.15.116:0/2402943163 shutdown_connections 2020-11-20T12:21:27.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.354+0000 7fb80aedb700 1 -- 172.21.15.116:0/2402943163 wait complete. 2020-11-20T12:21:27.382 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:21:27.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.550+0000 7ff1ff480700 1 Processor -- start 2020-11-20T12:21:27.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.554+0000 7ff1ff480700 1 -- start start 2020-11-20T12:21:27.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.554+0000 7ff1ff480700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f81518f0 0x7ff1f80bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.554+0000 7ff1ff480700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1f80b93c0 0x7ff1f80c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.554+0000 7ff1ff480700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f80bf510 con 0x7ff1f8151cd0 2020-11-20T12:21:27.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.554+0000 7ff1ff480700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f80bf7b0 con 0x7ff1f80b93c0 2020-11-20T12:21:27.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.554+0000 7ff1ff480700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f80bfa50 con 0x7ff1f81518f0 2020-11-20T12:21:27.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.554+0000 7ff1fec7f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff1f80b93c0 0x7ff1f80c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.554+0000 7ff1fe47e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f81518f0 0x7ff1f80bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.554+0000 7ff1fe47e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f81518f0 0x7ff1f80bed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:59904/0 (socket says 172.21.15.116:59904) 2020-11-20T12:21:27.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.554+0000 7ff1fe47e700 1 -- 172.21.15.116:0/4291791830 learned_addr learned my addr 172.21.15.116:0/4291791830 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:27.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1fd47c700 1 -- 172.21.15.116:0/4291791830 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689949337 0 0) 0x7ff1f80bf510 con 0x7ff1f8151cd0 2020-11-20T12:21:27.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1fd47c700 1 -- 172.21.15.116:0/4291791830 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e0003240 con 0x7ff1f8151cd0 2020-11-20T12:21:27.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1fec7f700 1 -- 172.21.15.116:0/4291791830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f81518f0 msgr2=0x7ff1f80bed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:27.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1fec7f700 1 --2- 172.21.15.116:0/4291791830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f81518f0 0x7ff1f80bed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1fec7f700 1 -- 172.21.15.116:0/4291791830 >> v1:172.21.15.173:6789/0 conn(0x7ff1f8151cd0 legacy=0x7ff1f8066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1fec7f700 1 -- 172.21.15.116:0/4291791830 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f80bfcf0 con 0x7ff1f80b93c0 2020-11-20T12:21:27.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1fe47e700 1 --2- 172.21.15.116:0/4291791830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f81518f0 0x7ff1f80bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:27.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1fec7f700 1 --2- 172.21.15.116:0/4291791830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1f80b93c0 0x7ff1f80c0430 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7ff1f0009c90 tx=0x7ff1f00091d0).ready entity=mon.0 client_cookie=7945f7c65c05f785 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:27.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1fd47c700 1 -- 172.21.15.116:0/4291791830 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000b9c0 con 0x7ff1f80b93c0 2020-11-20T12:21:27.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1fd47c700 1 -- 172.21.15.116:0/4291791830 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f000bb20 con 0x7ff1f80b93c0 2020-11-20T12:21:27.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1fd47c700 1 -- 172.21.15.116:0/4291791830 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f000bea0 con 0x7ff1f80b93c0 2020-11-20T12:21:27.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1ff480700 1 -- 172.21.15.116:0/4291791830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1f80b93c0 msgr2=0x7ff1f80c0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1ff480700 1 --2- 172.21.15.116:0/4291791830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1f80b93c0 0x7ff1f80c0430 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7ff1f0009c90 tx=0x7ff1f00091d0).stop 2020-11-20T12:21:27.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1ff480700 1 -- 172.21.15.116:0/4291791830 shutdown_connections 2020-11-20T12:21:27.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1ff480700 1 --2- 172.21.15.116:0/4291791830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1f80b93c0 0x7ff1f80c0430 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1ff480700 1 --2- 172.21.15.116:0/4291791830 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f81518f0 0x7ff1f80bed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1ff480700 1 -- 172.21.15.116:0/4291791830 >> 172.21.15.116:0/4291791830 conn(0x7ff1f8068350 msgr2=0x7ff1f80627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:27.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1ff480700 1 -- 172.21.15.116:0/4291791830 shutdown_connections 2020-11-20T12:21:27.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.558+0000 7ff1ff480700 1 -- 172.21.15.116:0/4291791830 wait complete. 2020-11-20T12:21:27.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.562+0000 7ff1ff480700 1 Processor -- start 2020-11-20T12:21:27.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.562+0000 7ff1ff480700 1 -- start start 2020-11-20T12:21:27.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.562+0000 7ff1ff480700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b93c0 0x7ff1f8154660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.562+0000 7ff1ff480700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1f81518f0 0x7ff1f813ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.562+0000 7ff1ff480700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1f8151cd0 0x7ff1f80bd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.562+0000 7ff1ff480700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f81520d0 con 0x7ff1f81518f0 2020-11-20T12:21:27.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.562+0000 7ff1ff480700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f80bc4b0 con 0x7ff1f8151cd0 2020-11-20T12:21:27.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.562+0000 7ff1ff480700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f80bc750 con 0x7ff1f80b93c0 2020-11-20T12:21:27.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1fdc7d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1f81518f0 0x7ff1f813ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1fdc7d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1f81518f0 0x7ff1f813ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50964/0 (socket says 172.21.15.116:50964) 2020-11-20T12:21:27.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1fdc7d700 1 -- 172.21.15.116:0/1028086289 learned_addr learned my addr 172.21.15.116:0/1028086289 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:27.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1fe47e700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b93c0 0x7ff1f8154660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1fec7f700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1f8151cd0 0x7ff1f80bd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1fdc7d700 1 -- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b93c0 msgr2=0x7ff1f8154660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1fdc7d700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b93c0 0x7ff1f8154660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1fdc7d700 1 -- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1f8151cd0 msgr2=0x7ff1f80bd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1fdc7d700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1f8151cd0 0x7ff1f80bd080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1fdc7d700 1 -- 172.21.15.116:0/1028086289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f80bc9f0 con 0x7ff1f81518f0 2020-11-20T12:21:27.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1fdc7d700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1f81518f0 0x7ff1f813ddd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7ff1e800c6e0 tx=0x7ff1e8007a20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:27.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1e67fc700 1 -- 172.21.15.116:0/1028086289 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800f470 con 0x7ff1f81518f0 2020-11-20T12:21:27.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1e67fc700 1 -- 172.21.15.116:0/1028086289 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e800f5d0 con 0x7ff1f81518f0 2020-11-20T12:21:27.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1ff480700 1 -- 172.21.15.116:0/1028086289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f81522c0 con 0x7ff1f81518f0 2020-11-20T12:21:27.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1e67fc700 1 -- 172.21.15.116:0/1028086289 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e800f950 con 0x7ff1f81518f0 2020-11-20T12:21:27.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.566+0000 7ff1ff480700 1 -- 172.21.15.116:0/1028086289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f813e2c0 con 0x7ff1f81518f0 2020-11-20T12:21:27.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.570+0000 7ff1ff480700 1 -- 172.21.15.116:0/1028086289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f805cf30 con 0x7ff1f81518f0 2020-11-20T12:21:27.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.570+0000 7ff1e67fc700 1 -- 172.21.15.116:0/1028086289 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff1e8019030 con 0x7ff1f81518f0 2020-11-20T12:21:27.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.570+0000 7ff1e67fc700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1d00557f0 0x7ff1d0057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.570+0000 7ff1e67fc700 1 -- 172.21.15.116:0/1028086289 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff1e806d8a0 con 0x7ff1f81518f0 2020-11-20T12:21:27.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.570+0000 7ff1fe47e700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1d00557f0 0x7ff1d0057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.574+0000 7ff1fe47e700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1d00557f0 0x7ff1d0057cb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7ff1f4006fd0 tx=0x7ff1f4006eb0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:27.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.578+0000 7ff1e67fc700 1 -- 172.21.15.116:0/1028086289 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff1e80437f0 con 0x7ff1f81518f0 2020-11-20T12:21:27.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.734+0000 7ff1ff480700 1 -- 172.21.15.116:0/1028086289 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff1f80639c0 con 0x7ff1f81518f0 2020-11-20T12:21:27.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.734+0000 7ff1e67fc700 1 -- 172.21.15.116:0/1028086289 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff1e8046e10 con 0x7ff1f81518f0 2020-11-20T12:21:27.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.734+0000 7ff1ff480700 1 -- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1d00557f0 msgr2=0x7ff1d0057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.734+0000 7ff1ff480700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1d00557f0 0x7ff1d0057cb0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7ff1f4006fd0 tx=0x7ff1f4006eb0).stop 2020-11-20T12:21:27.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.734+0000 7ff1ff480700 1 -- 172.21.15.116:0/1028086289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1f81518f0 msgr2=0x7ff1f813ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.734+0000 7ff1ff480700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1f81518f0 0x7ff1f813ddd0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7ff1e800c6e0 tx=0x7ff1e8007a20).stop 2020-11-20T12:21:27.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.734+0000 7ff1ff480700 1 -- 172.21.15.116:0/1028086289 shutdown_connections 2020-11-20T12:21:27.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.734+0000 7ff1ff480700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1d00557f0 0x7ff1d0057cb0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.738+0000 7ff1ff480700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1f8151cd0 0x7ff1f80bd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.738+0000 7ff1ff480700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1f80b93c0 0x7ff1f8154660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.738+0000 7ff1ff480700 1 --2- 172.21.15.116:0/1028086289 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1f81518f0 0x7ff1f813ddd0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.738+0000 7ff1ff480700 1 -- 172.21.15.116:0/1028086289 >> 172.21.15.116:0/1028086289 conn(0x7ff1f8068350 msgr2=0x7ff1f814ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:27.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.738+0000 7ff1ff480700 1 -- 172.21.15.116:0/1028086289 shutdown_connections 2020-11-20T12:21:27.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.738+0000 7ff1ff480700 1 -- 172.21.15.116:0/1028086289 wait complete. 2020-11-20T12:21:27.749 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:27.767 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:21:27.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.950+0000 7ff1e681c700 1 Processor -- start 2020-11-20T12:21:27.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.950+0000 7ff1e681c700 1 -- start start 2020-11-20T12:21:27.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1e681c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff1e00c38e0 0x7ff1e00804f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1e681c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e0153620 con 0x7ff1e0080110 2020-11-20T12:21:27.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1e681c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e00c6560 con 0x7ff1e01499b0 2020-11-20T12:21:27.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1e681c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e00a60d0 con 0x7ff1e00c38e0 2020-11-20T12:21:27.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1dffff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff1e0080110 0x7ff1e00c5d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60404/0 (socket says 172.21.15.116:60404) 2020-11-20T12:21:27.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1dffff700 1 -- 172.21.15.116:0/2431904220 learned_addr learned my addr 172.21.15.116:0/2431904220 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:27.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1df7fe700 1 --2- 172.21.15.116:0/2431904220 >> v2:172.21.15.173:3300/0 conn(0x7ff1e00c38e0 0x7ff1e00804f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1deffd700 1 -- 172.21.15.116:0/2431904220 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723971204 0 0) 0x7ff1e00c6560 con 0x7ff1e01499b0 2020-11-20T12:21:27.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1deffd700 1 -- 172.21.15.116:0/2431904220 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1cc003240 con 0x7ff1e01499b0 2020-11-20T12:21:27.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1deffd700 1 -- 172.21.15.116:0/2431904220 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 672878692 0 0) 0x7ff1e0153620 con 0x7ff1e0080110 2020-11-20T12:21:27.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1deffd700 1 -- 172.21.15.116:0/2431904220 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1e00c6560 con 0x7ff1e0080110 2020-11-20T12:21:27.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1df7fe700 1 -- 172.21.15.116:0/2431904220 >> v1:172.21.15.173:6789/0 conn(0x7ff1e01499b0 legacy=0x7ff1e0149e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1df7fe700 1 -- 172.21.15.116:0/2431904220 >> v1:172.21.15.116:6789/0 conn(0x7ff1e0080110 legacy=0x7ff1e00c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1df7fe700 1 -- 172.21.15.116:0/2431904220 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e0153880 con 0x7ff1e00c38e0 2020-11-20T12:21:27.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.954+0000 7ff1df7fe700 1 --2- 172.21.15.116:0/2431904220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1e00c38e0 0x7ff1e00804f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7ff1d400b9d0 tx=0x7ff1d4006df0).ready entity=mon.1 client_cookie=c3b7aa6f8f51abfa server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:27.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1deffd700 1 -- 172.21.15.116:0/2431904220 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4009640 con 0x7ff1e00c38e0 2020-11-20T12:21:27.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1deffd700 1 -- 172.21.15.116:0/2431904220 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d40097a0 con 0x7ff1e00c38e0 2020-11-20T12:21:27.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1deffd700 1 -- 172.21.15.116:0/2431904220 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4009b20 con 0x7ff1e00c38e0 2020-11-20T12:21:27.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1e681c700 1 -- 172.21.15.116:0/2431904220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1e00c38e0 msgr2=0x7ff1e00804f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1e681c700 1 --2- 172.21.15.116:0/2431904220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1e00c38e0 0x7ff1e00804f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7ff1d400b9d0 tx=0x7ff1d4006df0).stop 2020-11-20T12:21:27.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1e681c700 1 -- 172.21.15.116:0/2431904220 shutdown_connections 2020-11-20T12:21:27.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1e681c700 1 --2- 172.21.15.116:0/2431904220 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1e00c38e0 0x7ff1e00804f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1e681c700 1 -- 172.21.15.116:0/2431904220 >> 172.21.15.116:0/2431904220 conn(0x7ff1e00c7200 msgr2=0x7ff1e00c31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:27.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1e681c700 1 -- 172.21.15.116:0/2431904220 shutdown_connections 2020-11-20T12:21:27.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1e681c700 1 -- 172.21.15.116:0/2431904220 wait complete. 2020-11-20T12:21:27.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1e681c700 1 Processor -- start 2020-11-20T12:21:27.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.958+0000 7ff1e681c700 1 -- start start 2020-11-20T12:21:27.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1e681c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1e0080110 0x7ff1e014de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1e681c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1e00c38e0 0x7ff1e00a71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1e681c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1e01499b0 0x7ff1e01648d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1e681c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00a76d0 con 0x7ff1e00c38e0 2020-11-20T12:21:27.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1e681c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00a78f0 con 0x7ff1e0080110 2020-11-20T12:21:27.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1e681c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e00a7b90 con 0x7ff1e01499b0 2020-11-20T12:21:27.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1e00c38e0 0x7ff1e00a71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1df7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1e00c38e0 0x7ff1e00a71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50974/0 (socket says 172.21.15.116:50974) 2020-11-20T12:21:27.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1df7fe700 1 -- 172.21.15.116:0/835161606 learned_addr learned my addr 172.21.15.116:0/835161606 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:27.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1dffff700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1e0080110 0x7ff1e014de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1e4db9700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1e01499b0 0x7ff1e01648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1df7fe700 1 -- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1e01499b0 msgr2=0x7ff1e01648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1df7fe700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1e01499b0 0x7ff1e01648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1df7fe700 1 -- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1e0080110 msgr2=0x7ff1e014de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:27.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1df7fe700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1e0080110 0x7ff1e014de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:27.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1df7fe700 1 -- 172.21.15.116:0/835161606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e00a7e30 con 0x7ff1e00c38e0 2020-11-20T12:21:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1dffff700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1e0080110 0x7ff1e014de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1df7fe700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1e00c38e0 0x7ff1e00a71e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7ff1d4006b80 tx=0x7ff1d4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:27.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1bffff700 1 -- 172.21.15.116:0/835161606 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4011e20 con 0x7ff1e00c38e0 2020-11-20T12:21:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1bffff700 1 -- 172.21.15.116:0/835161606 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d401e460 con 0x7ff1e00c38e0 2020-11-20T12:21:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1e681c700 1 -- 172.21.15.116:0/835161606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e0161530 con 0x7ff1e00c38e0 2020-11-20T12:21:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1bffff700 1 -- 172.21.15.116:0/835161606 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d401e7e0 con 0x7ff1e00c38e0 2020-11-20T12:21:27.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.962+0000 7ff1e681c700 1 -- 172.21.15.116:0/835161606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e0164fe0 con 0x7ff1e00c38e0 2020-11-20T12:21:27.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.966+0000 7ff1e681c700 1 -- 172.21.15.116:0/835161606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e01616c0 con 0x7ff1e00c38e0 2020-11-20T12:21:27.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.966+0000 7ff1bffff700 1 -- 172.21.15.116:0/835161606 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff1d4025030 con 0x7ff1e00c38e0 2020-11-20T12:21:27.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.966+0000 7ff1bffff700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1b80557c0 0x7ff1b8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:27.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.966+0000 7ff1bffff700 1 -- 172.21.15.116:0/835161606 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff1d406fd70 con 0x7ff1e00c38e0 2020-11-20T12:21:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.970+0000 7ff1dffff700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1b80557c0 0x7ff1b8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.970+0000 7ff1dffff700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1b80557c0 0x7ff1b8057c80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7ff1d00055a0 tx=0x7ff1d0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:27.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:27.974+0000 7ff1bffff700 1 -- 172.21.15.116:0/835161606 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff1d4045c10 con 0x7ff1e00c38e0 2020-11-20T12:21:28.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.130+0000 7ff1e681c700 1 -- 172.21.15.116:0/835161606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1e0069950 con 0x7ff1e00c38e0 2020-11-20T12:21:28.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1bffff700 1 -- 172.21.15.116:0/835161606 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff1d40230a0 con 0x7ff1e00c38e0 2020-11-20T12:21:28.137 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:28.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 -- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1b80557c0 msgr2=0x7ff1b8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1b80557c0 0x7ff1b8057c80 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7ff1d00055a0 tx=0x7ff1d0009040).stop 2020-11-20T12:21:28.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 -- 172.21.15.116:0/835161606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1e00c38e0 msgr2=0x7ff1e00a71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1e00c38e0 0x7ff1e00a71e0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7ff1d4006b80 tx=0x7ff1d4015000).stop 2020-11-20T12:21:28.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 -- 172.21.15.116:0/835161606 shutdown_connections 2020-11-20T12:21:28.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff1b80557c0 0x7ff1b8057c80 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff1e01499b0 0x7ff1e01648d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff1e0080110 0x7ff1e014de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 --2- 172.21.15.116:0/835161606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff1e00c38e0 0x7ff1e00a71e0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 -- 172.21.15.116:0/835161606 >> 172.21.15.116:0/835161606 conn(0x7ff1e00c7200 msgr2=0x7ff1e0062d50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:28.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 -- 172.21.15.116:0/835161606 shutdown_connections 2020-11-20T12:21:28.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.134+0000 7ff1e681c700 1 -- 172.21.15.116:0/835161606 wait complete. 2020-11-20T12:21:28.164 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:21:28.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.342+0000 7efce0ebc700 1 Processor -- start 2020-11-20T12:21:28.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.342+0000 7efce0ebc700 1 -- start start 2020-11-20T12:21:28.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efce0ebc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efcdc0a73e0 0x7efcdc060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:28.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efce0ebc700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcdc0a5cc0 con 0x7efcdc0a3da0 2020-11-20T12:21:28.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efce0ebc700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcdc0a5a20 con 0x7efcdc1518d0 2020-11-20T12:21:28.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efce0ebc700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcdc063cc0 con 0x7efcdc0a73e0 2020-11-20T12:21:28.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efcd3fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7efcdc0a73e0 0x7efcdc060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:28.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efcdb7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7efcdc0a3da0 0x7efcdc0a51f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60416/0 (socket says 172.21.15.116:60416) 2020-11-20T12:21:28.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efcdb7fe700 1 -- 172.21.15.116:0/280395085 learned_addr learned my addr 172.21.15.116:0/280395085 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:28.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efcd3fff700 1 -- 172.21.15.116:0/280395085 >> v1:172.21.15.173:6789/0 conn(0x7efcdc1518d0 legacy=0x7efcdc0646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:28.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efcd3fff700 1 -- 172.21.15.116:0/280395085 >> v1:172.21.15.116:6789/0 conn(0x7efcdc0a3da0 legacy=0x7efcdc0a51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efcd3fff700 1 -- 172.21.15.116:0/280395085 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc063f60 con 0x7efcdc0a73e0 2020-11-20T12:21:28.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efcd3fff700 1 --2- 172.21.15.116:0/280395085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efcdc0a73e0 0x7efcdc060170 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7efcc400b9a0 tx=0x7efcc4006df0).ready entity=mon.0 client_cookie=95a00a6325960ea3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:28.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.346+0000 7efcdaffd700 1 -- 172.21.15.116:0/280395085 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4009770 con 0x7efcdc0a73e0 2020-11-20T12:21:28.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.350+0000 7efcdaffd700 1 -- 172.21.15.116:0/280395085 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc40098d0 con 0x7efcdc0a73e0 2020-11-20T12:21:28.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.350+0000 7efcdaffd700 1 -- 172.21.15.116:0/280395085 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4009c50 con 0x7efcdc0a73e0 2020-11-20T12:21:28.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.350+0000 7efce0ebc700 1 -- 172.21.15.116:0/280395085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efcdc0a73e0 msgr2=0x7efcdc060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.350+0000 7efce0ebc700 1 --2- 172.21.15.116:0/280395085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efcdc0a73e0 0x7efcdc060170 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7efcc400b9a0 tx=0x7efcc4006df0).stop 2020-11-20T12:21:28.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.350+0000 7efce0ebc700 1 -- 172.21.15.116:0/280395085 shutdown_connections 2020-11-20T12:21:28.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.350+0000 7efce0ebc700 1 --2- 172.21.15.116:0/280395085 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efcdc0a73e0 0x7efcdc060170 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.350+0000 7efce0ebc700 1 -- 172.21.15.116:0/280395085 >> 172.21.15.116:0/280395085 conn(0x7efcdc0c1710 msgr2=0x7efcdc0c1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:28.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.350+0000 7efce0ebc700 1 -- 172.21.15.116:0/280395085 shutdown_connections 2020-11-20T12:21:28.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.350+0000 7efce0ebc700 1 -- 172.21.15.116:0/280395085 wait complete. 2020-11-20T12:21:28.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.350+0000 7efce0ebc700 1 Processor -- start 2020-11-20T12:21:28.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.350+0000 7efce0ebc700 1 -- start start 2020-11-20T12:21:28.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efce0ebc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efcdc0a3da0 0x7efcdc0ba980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:28.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efce0ebc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcdc0a73e0 0x7efcdc0a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:28.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efce0ebc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efcdc1518d0 0x7efcdc150320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:28.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efce0ebc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0aa360 con 0x7efcdc1518d0 2020-11-20T12:21:28.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efce0ebc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0a98e0 con 0x7efcdc0a73e0 2020-11-20T12:21:28.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efce0ebc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcdc0a9b80 con 0x7efcdc0a3da0 2020-11-20T12:21:28.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcdbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efcdc1518d0 0x7efcdc150320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:28.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcdbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efcdc1518d0 0x7efcdc150320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:50988/0 (socket says 172.21.15.116:50988) 2020-11-20T12:21:28.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcdbfff700 1 -- 172.21.15.116:0/1055977829 learned_addr learned my addr 172.21.15.116:0/1055977829 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:28.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcdb7fe700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efcdc0a3da0 0x7efcdc0ba980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:28.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcd3fff700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcdc0a73e0 0x7efcdc0a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:28.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcdbfff700 1 -- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efcdc0a3da0 msgr2=0x7efcdc0ba980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcdbfff700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efcdc0a3da0 0x7efcdc0ba980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcdbfff700 1 -- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcdc0a73e0 msgr2=0x7efcdc0a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcdbfff700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcdc0a73e0 0x7efcdc0a8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcdbfff700 1 -- 172.21.15.116:0/1055977829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcdc0a9e20 con 0x7efcdc1518d0 2020-11-20T12:21:28.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcd3fff700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcdc0a73e0 0x7efcdc0a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:28.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcdbfff700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efcdc1518d0 0x7efcdc150320 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7efcd400cab0 tx=0x7efcd40023b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:28.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcd37fe700 1 -- 172.21.15.116:0/1055977829 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd400a830 con 0x7efcdc1518d0 2020-11-20T12:21:28.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcd37fe700 1 -- 172.21.15.116:0/1055977829 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcd400a990 con 0x7efcdc1518d0 2020-11-20T12:21:28.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efce0ebc700 1 -- 172.21.15.116:0/1055977829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcdc0aa0c0 con 0x7efcdc1518d0 2020-11-20T12:21:28.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efcd37fe700 1 -- 172.21.15.116:0/1055977829 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcd400ad30 con 0x7efcdc1518d0 2020-11-20T12:21:28.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.354+0000 7efce0ebc700 1 -- 172.21.15.116:0/1055977829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcdc0b9cf0 con 0x7efcdc1518d0 2020-11-20T12:21:28.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.358+0000 7efce0ebc700 1 -- 172.21.15.116:0/1055977829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcdc05cf30 con 0x7efcdc1518d0 2020-11-20T12:21:28.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.358+0000 7efcd37fe700 1 -- 172.21.15.116:0/1055977829 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7efcd4019030 con 0x7efcdc1518d0 2020-11-20T12:21:28.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.358+0000 7efcd37fe700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efcb80557f0 0x7efcb8057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:28.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.358+0000 7efcd37fe700 1 -- 172.21.15.116:0/1055977829 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7efcd406dfc0 con 0x7efcdc1518d0 2020-11-20T12:21:28.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.358+0000 7efcdb7fe700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efcb80557f0 0x7efcb8057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:28.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.362+0000 7efcdb7fe700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efcb80557f0 0x7efcb8057cb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7efccc001430 tx=0x7efccc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:28.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.366+0000 7efcd37fe700 1 -- 172.21.15.116:0/1055977829 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efcd4043f10 con 0x7efcdc1518d0 2020-11-20T12:21:28.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.522+0000 7efce0ebc700 1 -- 172.21.15.116:0/1055977829 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcdc062250 con 0x7efcdc1518d0 2020-11-20T12:21:28.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.522+0000 7efcd37fe700 1 -- 172.21.15.116:0/1055977829 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+10 (secure 0 0 0) 0x7efcd4047530 con 0x7efcdc1518d0 2020-11-20T12:21:28.528 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:28.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 -- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efcb80557f0 msgr2=0x7efcb8057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efcb80557f0 0x7efcb8057cb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7efccc001430 tx=0x7efccc009040).stop 2020-11-20T12:21:28.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 -- 172.21.15.116:0/1055977829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efcdc1518d0 msgr2=0x7efcdc150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efcdc1518d0 0x7efcdc150320 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7efcd400cab0 tx=0x7efcd40023b0).stop 2020-11-20T12:21:28.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 -- 172.21.15.116:0/1055977829 shutdown_connections 2020-11-20T12:21:28.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efcb80557f0 0x7efcb8057cb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efcdc1518d0 0x7efcdc150320 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efcdc0a3da0 0x7efcdc0ba980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 --2- 172.21.15.116:0/1055977829 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efcdc0a73e0 0x7efcdc0a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 -- 172.21.15.116:0/1055977829 >> 172.21.15.116:0/1055977829 conn(0x7efcdc0c1710 msgr2=0x7efcdc14d650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:28.532 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 -- 172.21.15.116:0/1055977829 shutdown_connections 2020-11-20T12:21:28.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.526+0000 7efce0ebc700 1 -- 172.21.15.116:0/1055977829 wait complete. 2020-11-20T12:21:28.555 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:21:28.556 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:21:28.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.734+0000 7f88c897a700 1 Processor -- start 2020-11-20T12:21:28.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.734+0000 7f88c897a700 1 -- start start 2020-11-20T12:21:28.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.734+0000 7f88c897a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f88c4062880 0x7f88c4062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:28.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.734+0000 7f88c897a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88c4065d40 0x7f88c40bfe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:28.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.734+0000 7f88c897a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88c40c0530 con 0x7f88c40677a0 2020-11-20T12:21:28.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.734+0000 7f88c897a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88c40661a0 con 0x7f88c4062880 2020-11-20T12:21:28.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.734+0000 7f88c897a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c40c0310 con 0x7f88c4065d40 2020-11-20T12:21:28.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.734+0000 7f88c2d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f88c4062880 0x7f88c4062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:28.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.734+0000 7f88c359e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f88c40677a0 0x7f88c41556e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60432/0 (socket says 172.21.15.116:60432) 2020-11-20T12:21:28.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.734+0000 7f88c359e700 1 -- 172.21.15.116:0/913779995 learned_addr learned my addr 172.21.15.116:0/913779995 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:28.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.734+0000 7f88c259c700 1 --2- 172.21.15.116:0/913779995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88c4065d40 0x7f88c40bfe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:28.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c2d9d700 1 -- 172.21.15.116:0/913779995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88c4065d40 msgr2=0x7f88c40bfe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:28.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c2d9d700 1 --2- 172.21.15.116:0/913779995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88c4065d40 0x7f88c40bfe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c1d9b700 1 -- 172.21.15.116:0/913779995 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152423213 0 0) 0x7f88c40c0530 con 0x7f88c40677a0 2020-11-20T12:21:28.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c2d9d700 1 -- 172.21.15.116:0/913779995 >> v1:172.21.15.116:6789/0 conn(0x7f88c40677a0 legacy=0x7f88c41556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c2d9d700 1 -- 172.21.15.116:0/913779995 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c40c07d0 con 0x7f88c4062880 2020-11-20T12:21:28.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c2d9d700 1 --2- 172.21.15.116:0/913779995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88c4062880 0x7f88c4062c60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f88b400b9d0 tx=0x7f88b4006df0).ready entity=mon.0 client_cookie=afac586f7915601e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:28.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c1d9b700 1 -- 172.21.15.116:0/913779995 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4009770 con 0x7f88c4062880 2020-11-20T12:21:28.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c1d9b700 1 -- 172.21.15.116:0/913779995 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88b40098d0 con 0x7f88c4062880 2020-11-20T12:21:28.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c1d9b700 1 -- 172.21.15.116:0/913779995 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b4009c50 con 0x7f88c4062880 2020-11-20T12:21:28.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c897a700 1 -- 172.21.15.116:0/913779995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88c4062880 msgr2=0x7f88c4062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c897a700 1 --2- 172.21.15.116:0/913779995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88c4062880 0x7f88c4062c60 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f88b400b9d0 tx=0x7f88b4006df0).stop 2020-11-20T12:21:28.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c897a700 1 -- 172.21.15.116:0/913779995 shutdown_connections 2020-11-20T12:21:28.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c897a700 1 --2- 172.21.15.116:0/913779995 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88c4062880 0x7f88c4062c60 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c897a700 1 --2- 172.21.15.116:0/913779995 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88c4065d40 0x7f88c40bfe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c897a700 1 -- 172.21.15.116:0/913779995 >> 172.21.15.116:0/913779995 conn(0x7f88c4060c40 msgr2=0x7f88c40c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:28.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c897a700 1 -- 172.21.15.116:0/913779995 shutdown_connections 2020-11-20T12:21:28.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.738+0000 7f88c897a700 1 -- 172.21.15.116:0/913779995 wait complete. 2020-11-20T12:21:28.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c897a700 1 Processor -- start 2020-11-20T12:21:28.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c897a700 1 -- start start 2020-11-20T12:21:28.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c897a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88c4065d40 0x7f88c4169500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:28.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c897a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88c40677a0 0x7f88c4152740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:28.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c897a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88c41653d0 0x7f88c40a5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:28.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c897a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c4165c60 con 0x7f88c41653d0 2020-11-20T12:21:28.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c897a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88c4165f00 con 0x7f88c40677a0 2020-11-20T12:21:28.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c897a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88c41661a0 con 0x7f88c4065d40 2020-11-20T12:21:28.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88c41653d0 0x7f88c40a5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:28.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88c41653d0 0x7f88c40a5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51000/0 (socket says 172.21.15.116:51000) 2020-11-20T12:21:28.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c359e700 1 -- 172.21.15.116:0/3603941839 learned_addr learned my addr 172.21.15.116:0/3603941839 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c2d9d700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88c4065d40 0x7f88c4169500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c359e700 1 -- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88c4065d40 msgr2=0x7f88c4169500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c359e700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88c4065d40 0x7f88c4169500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c359e700 1 -- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88c40677a0 msgr2=0x7f88c4152740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.742+0000 7f88c259c700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88c40677a0 0x7f88c4152740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88c359e700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88c40677a0 0x7f88c4152740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88c359e700 1 -- 172.21.15.116:0/3603941839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c40a74a0 con 0x7f88c41653d0 2020-11-20T12:21:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88c259c700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88c40677a0 0x7f88c4152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88c2d9d700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88c4065d40 0x7f88c4169500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88c359e700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88c41653d0 0x7f88c40a5440 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f88bc00cab0 tx=0x7f88bc007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88aaffd700 1 -- 172.21.15.116:0/3603941839 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc005c50 con 0x7f88c41653d0 2020-11-20T12:21:28.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88aaffd700 1 -- 172.21.15.116:0/3603941839 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88bc005db0 con 0x7f88c41653d0 2020-11-20T12:21:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88aaffd700 1 -- 172.21.15.116:0/3603941839 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88bc007340 con 0x7f88c41653d0 2020-11-20T12:21:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88c897a700 1 -- 172.21.15.116:0/3603941839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88c4152c30 con 0x7f88c41653d0 2020-11-20T12:21:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88c897a700 1 -- 172.21.15.116:0/3603941839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88c40a5930 con 0x7f88c41653d0 2020-11-20T12:21:28.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88c897a700 1 -- 172.21.15.116:0/3603941839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88c40c09b0 con 0x7f88c41653d0 2020-11-20T12:21:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.746+0000 7f88aaffd700 1 -- 172.21.15.116:0/3603941839 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f88bc010030 con 0x7f88c41653d0 2020-11-20T12:21:28.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.750+0000 7f88aaffd700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88a00557c0 0x7f88a0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.750+0000 7f88aaffd700 1 -- 172.21.15.116:0/3603941839 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f88bc06e9c0 con 0x7f88c41653d0 2020-11-20T12:21:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.750+0000 7f88c2d9d700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88a00557c0 0x7f88a0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:28.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.750+0000 7f88c2d9d700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88a00557c0 0x7f88a0057c80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f88c40637e0 tx=0x7f88b4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:28.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.754+0000 7f88aaffd700 1 -- 172.21.15.116:0/3603941839 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f88bc044750 con 0x7f88c41653d0 2020-11-20T12:21:28.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.966+0000 7f88c897a700 1 -- 172.21.15.116:0/3603941839 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f88c40a7630 con 0x7f88c41653d0 2020-11-20T12:21:28.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.966+0000 7f88aaffd700 1 -- 172.21.15.116:0/3603941839 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2851 (secure 0 0 0) 0x7f88bc047d70 con 0x7f88c41653d0 2020-11-20T12:21:28.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 -- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88a00557c0 msgr2=0x7f88a0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88a00557c0 0x7f88a0057c80 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f88c40637e0 tx=0x7f88b4015000).stop 2020-11-20T12:21:28.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 -- 172.21.15.116:0/3603941839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88c41653d0 msgr2=0x7f88c40a5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:28.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88c41653d0 0x7f88c40a5440 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f88bc00cab0 tx=0x7f88bc007de0).stop 2020-11-20T12:21:28.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 -- 172.21.15.116:0/3603941839 shutdown_connections 2020-11-20T12:21:28.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88a00557c0 0x7f88a0057c80 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f88c41653d0 0x7f88c40a5440 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88c4065d40 0x7f88c4169500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 --2- 172.21.15.116:0/3603941839 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88c40677a0 0x7f88c4152740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:28.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 -- 172.21.15.116:0/3603941839 >> 172.21.15.116:0/3603941839 conn(0x7f88c4060c40 msgr2=0x7f88c4154800 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:28.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 -- 172.21.15.116:0/3603941839 shutdown_connections 2020-11-20T12:21:28.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:28.970+0000 7f88c897a700 1 -- 172.21.15.116:0/3603941839 wait complete. 2020-11-20T12:21:28.980 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 60 2020-11-20T12:21:28.980 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:28.981 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1638614527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1638614527},{"type":"v1","addr":"172.21.15.173:6837","nonce":1638614527}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3682335131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3682335131},{"type":"v1","addr":"172.21.15.173:6836","nonce":3682335131}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7725,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/3975405805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3975405805},{"type":"v1","addr":"172.21.15.116:6834","nonce":3975405805}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:11.109567+0000","modified":"2020-11-20T12:21:13.118616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/267899152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":267899152},{"type":"v1","addr":"172.21.15.116:6835","nonce":267899152}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-20T12:21:29.402 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 60, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:21:11.109567+0000', 'modified': '2020-11-20T12:21:13.118616+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7728}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7728': {'gid': 7728, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/267899152', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 267899152}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 267899152}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:21:29.402 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:21:29.403 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:21:29.596 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":7728,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.464054665999999,"mdsmap_epoch":60,"osdmap_epoch":107,"osdmap_epoch_barrier":107,"uptime":23.969092925000002} 2020-11-20T12:21:29.596 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 7728, "mdsmap_epoch": 60, "osdmap_epoch": 107, "osdmap_epoch_barrier": 107, "rank_uptime": 18.464054666, "state": "up:active", "uptime": 23.969092925, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:21:29.597 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:21:29.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:29.598 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.3 2020-11-20T12:21:29.615 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:21:29.616 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:29.728 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:29.729 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:21:29.754 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:21:29.755 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:21:29.760 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:21:29.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:29.761 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2020-11-20T12:21:29.790 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-20T12:21:29.790 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:21:29.791 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 2020-11-20T12:21:29.861 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:21:29.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:29.862 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T12:21:29.880 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.manila mds 'allow *' osd 'allow rw' mon 'allow *' 2020-11-20T12:21:30.057 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:21:30.063 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:21:30.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.062+0000 7f9dcf25e700 1 Processor -- start 2020-11-20T12:21:30.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.062+0000 7f9dcf25e700 1 -- start start 2020-11-20T12:21:30.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.062+0000 7f9dcf25e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9dc8063100 0x7f9dc80bff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:30.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.062+0000 7f9dcf25e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc80c0400 con 0x7f9dc80a66e0 2020-11-20T12:21:30.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.062+0000 7f9dcf25e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc80c0690 con 0x7f9dc814e590 2020-11-20T12:21:30.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.062+0000 7f9dcf25e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc80c0930 con 0x7f9dc8063100 2020-11-20T12:21:30.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.062+0000 7f9dcea5d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9dc80a66e0 0x7f9dc8066480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60446/0 (socket says 172.21.15.116:60446) 2020-11-20T12:21:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.062+0000 7f9dcea5d700 1 -- 172.21.15.116:0/3920640334 learned_addr learned my addr 172.21.15.116:0/3920640334 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.062+0000 7f9dce25c700 1 --1- 172.21.15.116:0/3920640334 >> v1:172.21.15.173:6789/0 conn(0x7f9dc814e590 0x7f9dc814f470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:56672/0 (socket says 172.21.15.116:56672) 2020-11-20T12:21:30.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.062+0000 7f9dcda5b700 1 --2- 172.21.15.116:0/3920640334 >> v2:172.21.15.173:3300/0 conn(0x7f9dc8063100 0x7f9dc80bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcd25a700 1 -- 172.21.15.116:0/3920640334 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2798116328 0 0) 0x7f9dc80c0690 con 0x7f9dc814e590 2020-11-20T12:21:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcd25a700 1 -- 172.21.15.116:0/3920640334 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9db0003240 con 0x7f9dc814e590 2020-11-20T12:21:30.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcd25a700 1 -- 172.21.15.116:0/3920640334 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741826386 0 0) 0x7f9dc80c0400 con 0x7f9dc80a66e0 2020-11-20T12:21:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcda5b700 1 -- 172.21.15.116:0/3920640334 >> v1:172.21.15.173:6789/0 conn(0x7f9dc814e590 legacy=0x7f9dc814f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcda5b700 1 -- 172.21.15.116:0/3920640334 >> v1:172.21.15.116:6789/0 conn(0x7f9dc80a66e0 legacy=0x7f9dc8066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcda5b700 1 -- 172.21.15.116:0/3920640334 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80c0bd0 con 0x7f9dc8063100 2020-11-20T12:21:30.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcda5b700 1 --2- 172.21.15.116:0/3920640334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc8063100 0x7f9dc80bff10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f9db8009c90 tx=0x7f9db80091d0).ready entity=mon.1 client_cookie=8c62991c08dbf1a3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcd25a700 1 -- 172.21.15.116:0/3920640334 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db800ba30 con 0x7f9dc8063100 2020-11-20T12:21:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcd25a700 1 -- 172.21.15.116:0/3920640334 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db800bb90 con 0x7f9dc8063100 2020-11-20T12:21:30.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcd25a700 1 -- 172.21.15.116:0/3920640334 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8011400 con 0x7f9dc8063100 2020-11-20T12:21:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcf25e700 1 -- 172.21.15.116:0/3920640334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc8063100 msgr2=0x7f9dc80bff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcf25e700 1 --2- 172.21.15.116:0/3920640334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc8063100 0x7f9dc80bff10 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f9db8009c90 tx=0x7f9db80091d0).stop 2020-11-20T12:21:30.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcf25e700 1 -- 172.21.15.116:0/3920640334 shutdown_connections 2020-11-20T12:21:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcf25e700 1 --2- 172.21.15.116:0/3920640334 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc8063100 0x7f9dc80bff10 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcf25e700 1 -- 172.21.15.116:0/3920640334 >> 172.21.15.116:0/3920640334 conn(0x7f9dc8068350 msgr2=0x7f9dc80627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcf25e700 1 -- 172.21.15.116:0/3920640334 shutdown_connections 2020-11-20T12:21:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.066+0000 7f9dcf25e700 1 -- 172.21.15.116:0/3920640334 wait complete. 2020-11-20T12:21:30.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcf25e700 1 Processor -- start 2020-11-20T12:21:30.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcf25e700 1 -- start start 2020-11-20T12:21:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcf25e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc8063100 0x7f9dc814f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcf25e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dc80a66e0 0x7f9dc8160e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcf25e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dc814e590 0x7f9dc81491d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:30.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcf25e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80bd210 con 0x7f9dc80a66e0 2020-11-20T12:21:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcf25e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc814f880 con 0x7f9dc8063100 2020-11-20T12:21:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcf25e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc8161330 con 0x7f9dc814e590 2020-11-20T12:21:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcda5b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dc80a66e0 0x7f9dc8160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcda5b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dc80a66e0 0x7f9dc8160e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51016/0 (socket says 172.21.15.116:51016) 2020-11-20T12:21:30.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcda5b700 1 -- 172.21.15.116:0/1488694797 learned_addr learned my addr 172.21.15.116:0/1488694797 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dce25c700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc8063100 0x7f9dc814f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcea5d700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dc814e590 0x7f9dc81491d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcda5b700 1 -- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dc814e590 msgr2=0x7f9dc81491d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:30.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcda5b700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dc814e590 0x7f9dc81491d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcda5b700 1 -- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc8063100 msgr2=0x7f9dc814f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcda5b700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc8063100 0x7f9dc814f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcda5b700 1 -- 172.21.15.116:0/1488694797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc80bc8d0 con 0x7f9dc80a66e0 2020-11-20T12:21:30.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dce25c700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc8063100 0x7f9dc814f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.070+0000 7f9dcda5b700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dc80a66e0 0x7f9dc8160e40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f9db800c010 tx=0x7f9db8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.074+0000 7f9db67fc700 1 -- 172.21.15.116:0/1488694797 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db801c5f0 con 0x7f9dc80a66e0 2020-11-20T12:21:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.074+0000 7f9db67fc700 1 -- 172.21.15.116:0/1488694797 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db801c770 con 0x7f9dc80a66e0 2020-11-20T12:21:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.074+0000 7f9dcf25e700 1 -- 172.21.15.116:0/1488694797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc815d420 con 0x7f9dc80a66e0 2020-11-20T12:21:30.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.074+0000 7f9db67fc700 1 -- 172.21.15.116:0/1488694797 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db801cb30 con 0x7f9dc80a66e0 2020-11-20T12:21:30.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.074+0000 7f9dcf25e700 1 -- 172.21.15.116:0/1488694797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc815d960 con 0x7f9dc80a66e0 2020-11-20T12:21:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.074+0000 7f9dcf25e700 1 -- 172.21.15.116:0/1488694797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc814a830 con 0x7f9dc80a66e0 2020-11-20T12:21:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.074+0000 7f9db67fc700 1 -- 172.21.15.116:0/1488694797 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9db8017070 con 0x7f9dc80a66e0 2020-11-20T12:21:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.078+0000 7f9db67fc700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9da00557c0 0x7f9da0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.078+0000 7f9db67fc700 1 -- 172.21.15.116:0/1488694797 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9db8070850 con 0x7f9dc80a66e0 2020-11-20T12:21:30.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.078+0000 7f9dce25c700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9da00557c0 0x7f9da0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:30.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.078+0000 7f9dce25c700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9da00557c0 0x7f9da0057c80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f9dc4005630 tx=0x7f9dc4007100).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:30.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.082+0000 7f9db67fc700 1 -- 172.21.15.116:0/1488694797 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9dc814a830 con 0x7f9dc80a66e0 2020-11-20T12:21:30.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.294+0000 7f9dcf25e700 1 -- 172.21.15.116:0/1488694797 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]} v 0) v1 -- 0x7f9dc814a830 con 0x7f9dc80a66e0 2020-11-20T12:21:30.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.302+0000 7f9db67fc700 1 -- 172.21.15.116:0/1488694797 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]}]=0 v27) v1 ==== 158+0+64 (secure 0 0 0) 0x7f9dc814a830 con 0x7f9dc80a66e0 2020-11-20T12:21:30.306 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:21:30.307 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQDKtLdfEEDJERAAgK2ZvLfum7GYKOMoYIjP8Q== 2020-11-20T12:21:30.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 -- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9da00557c0 msgr2=0x7f9da0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:30.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9da00557c0 0x7f9da0057c80 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f9dc4005630 tx=0x7f9dc4007100).stop 2020-11-20T12:21:30.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 -- 172.21.15.116:0/1488694797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dc80a66e0 msgr2=0x7f9dc8160e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:30.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dc80a66e0 0x7f9dc8160e40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f9db800c010 tx=0x7f9db8016000).stop 2020-11-20T12:21:30.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 -- 172.21.15.116:0/1488694797 shutdown_connections 2020-11-20T12:21:30.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9da00557c0 0x7f9da0057c80 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:30.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9dc814e590 0x7f9dc81491d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:30.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9dc8063100 0x7f9dc814f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:30.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 --2- 172.21.15.116:0/1488694797 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9dc80a66e0 0x7f9dc8160e40 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:30.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 -- 172.21.15.116:0/1488694797 >> 172.21.15.116:0/1488694797 conn(0x7f9dc8068350 msgr2=0x7f9dc80c31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:30.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 -- 172.21.15.116:0/1488694797 shutdown_connections 2020-11-20T12:21:30.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:30.306+0000 7f9dcf25e700 1 -- 172.21.15.116:0/1488694797 wait complete. 2020-11-20T12:21:30.333 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:21:30.334 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:21:30.344 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:21:30.357 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:21:30.357 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:21:30.358 INFO:teuthology.orchestra.run.smithi173.stderr:64 bytes copied, 0.000333114 s, 192 kB/s 2020-11-20T12:21:30.361 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:21:30.368 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:21:30.432 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:21:30.432 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:21:30.433 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:21:30.433 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:21:30.433 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.434 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:21:30.434 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.434 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:21:30.435 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.435 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:21:30.435 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:21:30.435 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.436 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:21:30.436 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:21:30.436 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.436 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:21:30.437 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:21:30.437 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.437 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:21:30.437 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:21:30.438 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.438 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:21:30.438 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:21:30.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:21:30.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:21:30.439 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:21:30.440 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:21:30.440 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:21:30.440 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:21:30.440 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:21:30.441 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:21:30.441 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:21:30.441 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.442 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:21:30.442 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.442 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:21:30.443 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.443 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:21:30.443 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.443 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:21:30.444 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:21:30.444 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:21:30.444 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:21:30.445 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:21:30.445 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:21:30.445 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.446 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:21:30.446 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:21:30.446 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.447 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:21:30.447 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:21:30.447 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.447 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:21:30.448 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:21:30.448 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.448 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:21:30.448 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:21:30.449 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:21:30.449 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.449 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:21:30.450 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:21:30.450 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.451 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:21:30.451 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.451 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:21:30.451 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.452 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:21:30.452 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.452 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:21:30.452 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:21:30.453 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.453 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:21:30.453 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:21:30.454 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:21:30.454 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:21:30.454 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:21:30.454 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:21:30.455 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:21:30.455 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:21:30.455 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:21:30.455 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:21:30.456 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:21:30.456 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:21:30.456 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:21:30.456 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:21:30.457 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:21:30.457 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:21:30.457 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:21:30.458 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:21:30.458 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:21:30.458 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:21:30.459 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.459 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:21:30.460 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:21:30.460 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:21:30.460 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:21:30.460 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:21:30.461 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:21:30.461 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:21:30.461 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:21:30.461 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.462 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:21:30.462 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:21:30.462 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:21:30.462 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:21:30.463 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:21:30.463 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:21:30.463 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:21:30.463 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:21:30.464 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:21:30.464 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:21:30.464 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:21:30.464 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:21:30.465 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:30.465 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:21:30.465 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:21:30.466 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:21:30.466 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:21:30.466 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:21:30.466 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:21:30.467 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:21:30.467 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:21:30.467 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:21:30.468 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:21:30.468 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:21:30.468 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:21:30.468 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:21:30.469 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:21:30.469 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:21:30.469 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:21:30.470 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:21:30.470 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:21:30.470 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:21:30.471 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:21:30.471 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:21:30.471 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:21:30.471 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:21:30.472 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:21:30.472 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:21:30.472 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:21:30.472 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:21:30.473 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:21:30.473 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:21:30.474 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:21:30.474 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:21:30.474 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:21:30.474 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:21:30.475 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:21:30.475 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:21:30.475 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:21:30.476 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:21:30.478 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:21:30.478 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:21:30.479 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:21:30.479 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:21:30.479 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.480 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:21:30.480 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.480 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:21:30.481 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.481 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:21:30.481 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:21:30.482 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.482 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:21:30.482 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:21:30.483 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.483 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:21:30.483 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:21:30.483 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.484 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:21:30.484 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:21:30.484 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.484 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:21:30.485 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.485 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:21:30.485 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:21:30.485 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:21:30.486 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:21:30.486 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:21:30.486 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:21:30.487 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:21:30.487 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:21:30.487 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:21:30.488 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:21:30.488 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.488 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:21:30.488 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.489 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:21:30.489 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.489 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:21:30.489 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.490 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:21:30.490 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:21:30.490 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:21:30.490 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:21:30.491 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:21:30.491 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:21:30.491 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.491 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:21:30.492 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:21:30.492 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.492 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:21:30.492 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:21:30.492 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.493 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:21:30.493 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:21:30.493 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.493 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:21:30.494 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:21:30.494 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:21:30.494 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.495 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:21:30.495 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:21:30.495 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.495 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:21:30.496 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.496 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:21:30.496 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.496 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:21:30.497 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.497 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:21:30.497 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:21:30.497 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.497 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:21:30.498 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:21:30.498 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:21:30.498 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:21:30.499 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:21:30.499 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:21:30.499 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:21:30.499 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:21:30.500 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:21:30.500 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:21:30.500 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:21:30.500 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:21:30.501 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:21:30.501 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:21:30.501 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:21:30.501 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:21:30.501 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:21:30.502 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:21:30.502 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:21:30.502 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:21:30.502 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.503 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:21:30.503 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:21:30.503 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:21:30.504 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:21:30.504 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:21:30.504 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:21:30.504 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:21:30.505 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:21:30.505 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.505 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:21:30.505 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:21:30.506 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:21:30.506 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:21:30.506 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:21:30.506 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:21:30.507 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:21:30.507 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:21:30.507 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:21:30.508 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:21:30.508 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:21:30.508 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:21:30.508 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:30.509 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:21:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:21:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:21:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:21:30.509 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:21:30.510 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:21:30.510 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:21:30.510 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:21:30.511 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:21:30.511 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:21:30.511 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:21:30.512 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:21:30.512 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:21:30.512 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:21:30.512 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:21:30.513 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:21:30.513 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:21:30.513 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:21:30.514 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:21:30.514 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:21:30.514 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:21:30.515 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:21:30.515 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:21:30.515 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:21:30.515 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:21:30.516 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:21:30.516 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:21:30.516 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:21:30.516 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:21:30.517 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:21:30.517 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:21:30.517 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:21:30.517 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:21:30.518 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:21:30.518 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:21:30.518 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:21:30.518 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:21:30.519 INFO:tasks.cephfs.test_volume_client:print(group ID: grpid) 2020-11-20T12:21:30.519 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:21:30.520 INFO:teuthology.orchestra.run.smithi173:> from __future__ import print_function 2020-11-20T12:21:30.520 INFO:teuthology.orchestra.run.smithi173:> from ceph_volume_client import CephFSVolumeClient, VolumePath 2020-11-20T12:21:30.521 INFO:teuthology.orchestra.run.smithi173:> from sys import version_info as sys_version_info 2020-11-20T12:21:30.521 INFO:teuthology.orchestra.run.smithi173:> from rados import OSError as rados_OSError 2020-11-20T12:21:30.522 INFO:teuthology.orchestra.run.smithi173:> import logging 2020-11-20T12:21:30.522 INFO:teuthology.orchestra.run.smithi173:> log = logging.getLogger("ceph_volume_client") 2020-11-20T12:21:30.523 INFO:teuthology.orchestra.run.smithi173:> log.addHandler(logging.StreamHandler()) 2020-11-20T12:21:30.523 INFO:teuthology.orchestra.run.smithi173:> log.setLevel(logging.DEBUG) 2020-11-20T12:21:30.523 INFO:teuthology.orchestra.run.smithi173:> vc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None) 2020-11-20T12:21:30.524 INFO:teuthology.orchestra.run.smithi173:> vc.connect() 2020-11-20T12:21:30.524 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:21:30.525 INFO:teuthology.orchestra.run.smithi173:> vp = VolumePath("grpid", "volid") 2020-11-20T12:21:30.525 INFO:teuthology.orchestra.run.smithi173:> vc.create_volume(vp, 1024*1024*10) 2020-11-20T12:21:30.526 INFO:teuthology.orchestra.run.smithi173:> auths = vc.get_authorized_ids(vp) 2020-11-20T12:21:30.526 INFO:teuthology.orchestra.run.smithi173:> print(auths) 2020-11-20T12:21:30.527 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:21:30.527 INFO:teuthology.orchestra.run.smithi173:> vc.disconnect() 2020-11-20T12:21:30.528 INFO:teuthology.orchestra.run.smithi173:> ' 2020-11-20T12:21:30.707 INFO:teuthology.orchestra.run.smithi173.stderr:Configuring to RADOS with config /etc/ceph/ceph.conf... 2020-11-20T12:21:30.712 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to RADOS... 2020-11-20T12:21:30.722 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to RADOS complete 2020-11-20T12:21:30.722 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to cephfs... 2020-11-20T12:21:30.723 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS initializing... 2020-11-20T12:21:30.730 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS mounting... 2020-11-20T12:21:30.740 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to cephfs complete 2020-11-20T12:21:30.740 INFO:teuthology.orchestra.run.smithi173.stderr:Recovering from partial auth updates (if any)... 2020-11-20T12:21:30.741 INFO:teuthology.orchestra.run.smithi173.stderr:Nothing to recover. No auth meta files. 2020-11-20T12:21:30.741 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: /volumes/grpid/volid 2020-11-20T12:21:30.762 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: grpid/volid, using rados namespace fsvolumens_volid to isolate data. 2020-11-20T12:21:30.780 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:21:30.781 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs... 2020-11-20T12:21:30.794 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs complete 2020-11-20T12:21:30.795 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados... 2020-11-20T12:21:30.798 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados complete 2020-11-20T12:21:30.799 INFO:teuthology.orchestra.run.smithi173.stdout:None 2020-11-20T12:21:30.802 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:21:31.030 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:21:31.031 INFO:teuthology.orchestra.run.smithi173:> from __future__ import print_function 2020-11-20T12:21:31.032 INFO:teuthology.orchestra.run.smithi173:> from ceph_volume_client import CephFSVolumeClient, VolumePath 2020-11-20T12:21:31.032 INFO:teuthology.orchestra.run.smithi173:> from sys import version_info as sys_version_info 2020-11-20T12:21:31.032 INFO:teuthology.orchestra.run.smithi173:> from rados import OSError as rados_OSError 2020-11-20T12:21:31.033 INFO:teuthology.orchestra.run.smithi173:> import logging 2020-11-20T12:21:31.034 INFO:teuthology.orchestra.run.smithi173:> log = logging.getLogger("ceph_volume_client") 2020-11-20T12:21:31.035 INFO:teuthology.orchestra.run.smithi173:> log.addHandler(logging.StreamHandler()) 2020-11-20T12:21:31.035 INFO:teuthology.orchestra.run.smithi173:> log.setLevel(logging.DEBUG) 2020-11-20T12:21:31.035 INFO:teuthology.orchestra.run.smithi173:> vc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None) 2020-11-20T12:21:31.036 INFO:teuthology.orchestra.run.smithi173:> vc.connect() 2020-11-20T12:21:31.036 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:21:31.036 INFO:teuthology.orchestra.run.smithi173:> vp = VolumePath("grpid", "volid") 2020-11-20T12:21:31.037 INFO:teuthology.orchestra.run.smithi173:> vc.authorize(vp, "guest1", readonly=False) 2020-11-20T12:21:31.037 INFO:teuthology.orchestra.run.smithi173:> vc.authorize(vp, "guest2", readonly=True) 2020-11-20T12:21:31.038 INFO:teuthology.orchestra.run.smithi173:> auths = vc.get_authorized_ids(vp) 2020-11-20T12:21:31.038 INFO:teuthology.orchestra.run.smithi173:> print(auths) 2020-11-20T12:21:31.038 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:21:31.039 INFO:teuthology.orchestra.run.smithi173:> vc.disconnect() 2020-11-20T12:21:31.039 INFO:teuthology.orchestra.run.smithi173:> ' 2020-11-20T12:21:31.253 INFO:teuthology.orchestra.run.smithi173.stderr:Configuring to RADOS with config /etc/ceph/ceph.conf... 2020-11-20T12:21:31.258 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to RADOS... 2020-11-20T12:21:31.269 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to RADOS complete 2020-11-20T12:21:31.269 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to cephfs... 2020-11-20T12:21:31.270 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS initializing... 2020-11-20T12:21:31.277 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS mounting... 2020-11-20T12:21:31.287 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to cephfs complete 2020-11-20T12:21:31.287 INFO:teuthology.orchestra.run.smithi173.stderr:Recovering from partial auth updates (if any)... 2020-11-20T12:21:31.290 INFO:teuthology.orchestra.run.smithi173.stderr:Recovered from partial auth updates (if any). 2020-11-20T12:21:31.306 INFO:teuthology.orchestra.run.smithi173.stderr:Creating meta for ID guest1 with tenant None 2020-11-20T12:21:31.307 INFO:teuthology.orchestra.run.smithi173.stderr:Authorize: no existing meta 2020-11-20T12:21:31.327 INFO:teuthology.orchestra.run.smithi173.stderr:Authorizing Ceph id 'guest1' for path '/volumes/grpid/volid' 2020-11-20T12:21:31.396 INFO:teuthology.orchestra.run.smithi173.stderr:Creating meta for ID guest2 with tenant None 2020-11-20T12:21:31.396 INFO:teuthology.orchestra.run.smithi173.stderr:Authorize: no existing meta 2020-11-20T12:21:31.432 INFO:teuthology.orchestra.run.smithi173.stderr:Authorizing Ceph id 'guest2' for path '/volumes/grpid/volid' 2020-11-20T12:21:31.491 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:21:31.492 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs... 2020-11-20T12:21:31.504 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs complete 2020-11-20T12:21:31.505 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados... 2020-11-20T12:21:31.508 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados complete 2020-11-20T12:21:31.509 INFO:teuthology.orchestra.run.smithi173.stdout:[('guest1', 'rw'), ('guest2', 'r')] 2020-11-20T12:21:31.510 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:21:31.575 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:21:31.576 INFO:teuthology.orchestra.run.smithi173:> from __future__ import print_function 2020-11-20T12:21:31.577 INFO:teuthology.orchestra.run.smithi173:> from ceph_volume_client import CephFSVolumeClient, VolumePath 2020-11-20T12:21:31.578 INFO:teuthology.orchestra.run.smithi173:> from sys import version_info as sys_version_info 2020-11-20T12:21:31.578 INFO:teuthology.orchestra.run.smithi173:> from rados import OSError as rados_OSError 2020-11-20T12:21:31.579 INFO:teuthology.orchestra.run.smithi173:> import logging 2020-11-20T12:21:31.580 INFO:teuthology.orchestra.run.smithi173:> log = logging.getLogger("ceph_volume_client") 2020-11-20T12:21:31.580 INFO:teuthology.orchestra.run.smithi173:> log.addHandler(logging.StreamHandler()) 2020-11-20T12:21:31.581 INFO:teuthology.orchestra.run.smithi173:> log.setLevel(logging.DEBUG) 2020-11-20T12:21:31.581 INFO:teuthology.orchestra.run.smithi173:> vc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None) 2020-11-20T12:21:31.582 INFO:teuthology.orchestra.run.smithi173:> vc.connect() 2020-11-20T12:21:31.583 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:21:31.583 INFO:teuthology.orchestra.run.smithi173:> vp = VolumePath("grpid", "volid") 2020-11-20T12:21:31.584 INFO:teuthology.orchestra.run.smithi173:> vc.deauthorize(vp, "guest1") 2020-11-20T12:21:31.584 INFO:teuthology.orchestra.run.smithi173:> vc.deauthorize(vp, "guest2") 2020-11-20T12:21:31.585 INFO:teuthology.orchestra.run.smithi173:> auths = vc.get_authorized_ids(vp) 2020-11-20T12:21:31.585 INFO:teuthology.orchestra.run.smithi173:> print(auths) 2020-11-20T12:21:31.586 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:21:31.587 INFO:teuthology.orchestra.run.smithi173:> vc.disconnect() 2020-11-20T12:21:31.587 INFO:teuthology.orchestra.run.smithi173:> ' 2020-11-20T12:21:31.766 INFO:teuthology.orchestra.run.smithi173.stderr:Configuring to RADOS with config /etc/ceph/ceph.conf... 2020-11-20T12:21:31.771 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to RADOS... 2020-11-20T12:21:31.783 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to RADOS complete 2020-11-20T12:21:31.783 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to cephfs... 2020-11-20T12:21:31.784 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS initializing... 2020-11-20T12:21:31.790 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS mounting... 2020-11-20T12:21:31.801 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to cephfs complete 2020-11-20T12:21:31.802 INFO:teuthology.orchestra.run.smithi173.stderr:Recovering from partial auth updates (if any)... 2020-11-20T12:21:31.824 INFO:teuthology.orchestra.run.smithi173.stderr:Recovered from partial auth updates (if any). 2020-11-20T12:21:32.000 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:21:32.000 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs... 2020-11-20T12:21:32.017 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs complete 2020-11-20T12:21:32.017 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados... 2020-11-20T12:21:32.020 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados complete 2020-11-20T12:21:32.021 INFO:teuthology.orchestra.run.smithi173.stdout:None 2020-11-20T12:21:32.024 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:21:32.091 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:21:32.092 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:21:32.145 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:21:32.184 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:21:32.185 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-20T12:21:32.185 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:21:32.186 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-20T12:21:32.277 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:21:32.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:32.278 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:21:32.300 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2020-11-20T12:21:32.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:21:32.301 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.2 2020-11-20T12:21:32.428 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:21:32.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:32.429 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T12:21:32.450 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2020-11-20T12:21:32.451 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:21:32.451 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.3 2020-11-20T12:21:32.533 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:21:32.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:21:32.534 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T12:21:32.557 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:21:32.565 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:21:32.630 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:21:32.630 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:21:32.631 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:21:32.631 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:21:32.631 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.632 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:21:32.632 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.632 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:21:32.633 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.633 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:21:32.633 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:21:32.633 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.634 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:21:32.634 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:21:32.634 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.634 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:21:32.635 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:21:32.635 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.635 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:21:32.635 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:21:32.635 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.636 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:21:32.636 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:21:32.636 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:21:32.637 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:21:32.637 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:21:32.637 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:21:32.637 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:21:32.638 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:21:32.638 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:21:32.638 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:21:32.638 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:21:32.639 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.639 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:21:32.639 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.639 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:21:32.640 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.640 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:21:32.640 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.641 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:21:32.641 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:21:32.641 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:21:32.642 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:21:32.642 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:21:32.642 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:21:32.642 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.643 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:21:32.643 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:21:32.643 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.644 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:21:32.644 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:21:32.644 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.645 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:21:32.645 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:21:32.645 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.645 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:21:32.646 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:21:32.646 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:21:32.646 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.646 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:21:32.647 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:21:32.647 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.647 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:21:32.648 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.648 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:21:32.648 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.648 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:21:32.648 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.649 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:21:32.649 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:21:32.649 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.649 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:21:32.650 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:21:32.650 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:21:32.650 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:21:32.651 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:21:32.651 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:21:32.652 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:21:32.652 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:21:32.652 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:21:32.652 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:21:32.653 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:21:32.653 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:21:32.653 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:21:32.654 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:21:32.654 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:21:32.654 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:21:32.655 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:21:32.655 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:21:32.655 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:21:32.656 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:21:32.656 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.656 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:21:32.657 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:21:32.657 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:21:32.657 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:21:32.658 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:21:32.658 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:21:32.658 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:21:32.659 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:21:32.659 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.659 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:21:32.660 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:21:32.660 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:21:32.660 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:21:32.660 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:21:32.661 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:21:32.661 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:21:32.661 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:21:32.662 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:21:32.662 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:21:32.662 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:21:32.663 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:21:32.663 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:32.663 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:21:32.664 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:21:32.664 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:21:32.664 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:21:32.665 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:21:32.665 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:21:32.665 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:21:32.666 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:21:32.666 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:21:32.666 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:21:32.667 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:21:32.667 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:21:32.667 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:21:32.668 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:21:32.668 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:21:32.668 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:21:32.669 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:21:32.669 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:21:32.669 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:21:32.670 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:21:32.670 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:21:32.670 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:21:32.671 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:21:32.671 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:21:32.671 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:21:32.672 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:21:32.672 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:21:32.672 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:21:32.673 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:21:32.673 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:21:32.673 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:21:32.674 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:21:32.674 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:21:32.674 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:21:32.675 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:21:32.675 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:21:32.676 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:21:32.677 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:21:32.677 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:21:32.677 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:21:32.677 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.678 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:21:32.678 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.678 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:21:32.678 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.679 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:21:32.679 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:21:32.679 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.680 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:21:32.680 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:21:32.680 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.681 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:21:32.681 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:21:32.681 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.682 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:21:32.682 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:21:32.682 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.683 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:21:32.683 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.683 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:21:32.684 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:21:32.684 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:21:32.685 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:21:32.685 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:21:32.685 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:21:32.686 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:21:32.686 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:21:32.686 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:21:32.687 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:21:32.687 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.687 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:21:32.688 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.688 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:21:32.688 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.689 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:21:32.689 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.689 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:21:32.690 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:21:32.690 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:21:32.690 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:21:32.691 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:21:32.691 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:21:32.691 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.692 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:21:32.692 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:21:32.692 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.693 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:21:32.693 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:21:32.694 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.694 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:21:32.694 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:21:32.695 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.695 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:21:32.695 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:21:32.695 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:21:32.696 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.696 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:21:32.696 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:21:32.697 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.697 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:21:32.697 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.698 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:21:32.698 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.698 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:21:32.699 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.699 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:21:32.700 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:21:32.700 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.700 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:21:32.701 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:21:32.701 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:21:32.701 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:21:32.702 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:21:32.702 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:21:32.702 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:21:32.703 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:21:32.703 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:21:32.703 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:21:32.703 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:21:32.704 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:21:32.704 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:21:32.704 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:21:32.705 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:21:32.705 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:21:32.705 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:21:32.706 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:21:32.706 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:21:32.706 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:21:32.707 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.707 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:21:32.707 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:21:32.707 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:21:32.708 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:21:32.708 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:21:32.708 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:21:32.709 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:21:32.709 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:21:32.709 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.710 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:21:32.710 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:21:32.710 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:21:32.711 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:21:32.711 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:21:32.712 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:21:32.712 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:21:32.712 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:21:32.713 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:21:32.713 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:21:32.713 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:21:32.714 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:21:32.714 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:21:32.714 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:21:32.715 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:21:32.715 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:21:32.715 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:21:32.716 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:21:32.716 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:21:32.716 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:21:32.717 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:21:32.717 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:21:32.717 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:21:32.718 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:21:32.718 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:21:32.718 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:21:32.719 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:21:32.719 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:21:32.719 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:21:32.720 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:21:32.720 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:21:32.721 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:21:32.721 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:21:32.721 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:21:32.722 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:21:32.722 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:21:32.722 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:21:32.723 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:21:32.723 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:21:32.723 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:21:32.724 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:21:32.724 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:21:32.724 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:21:32.725 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:21:32.725 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:21:32.725 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:21:32.726 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:21:32.726 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:21:32.726 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:21:32.728 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-20T12:21:32.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.906+0000 7fbc79b6b700 1 Processor -- start 2020-11-20T12:21:32.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.906+0000 7fbc79b6b700 1 -- start start 2020-11-20T12:21:32.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc79b6b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbc74154dd0 0x7fbc74156160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:32.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc79b6b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc740bd810 0x7fbc740be340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:32.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc79b6b700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc740bdfc0 con 0x7fbc741551b0 2020-11-20T12:21:32.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc79b6b700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc740bdd20 con 0x7fbc74154dd0 2020-11-20T12:21:32.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc79b6b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74155780 con 0x7fbc740bd810 2020-11-20T12:21:32.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc737fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbc74154dd0 0x7fbc74156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:32.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc737fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fbc74154dd0 0x7fbc74156160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51024/0 (socket says 172.21.15.116:51024) 2020-11-20T12:21:32.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc737fe700 1 -- 172.21.15.116:0/52604468 learned_addr learned my addr 172.21.15.116:0/52604468 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:32.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc737fe700 1 -- 172.21.15.116:0/52604468 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc740bd810 msgr2=0x7fbc740be340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:32.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc737fe700 1 --2- 172.21.15.116:0/52604468 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc740bd810 0x7fbc740be340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:32.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc737fe700 1 -- 172.21.15.116:0/52604468 >> v1:172.21.15.173:6789/0 conn(0x7fbc741551b0 legacy=0x7fbc740a6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:32.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc737fe700 1 -- 172.21.15.116:0/52604468 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc74155a20 con 0x7fbc74154dd0 2020-11-20T12:21:32.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc737fe700 1 --2- 172.21.15.116:0/52604468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc74154dd0 0x7fbc74156160 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7fbc64006fd0 tx=0x7fbc64006eb0).ready entity=mon.0 client_cookie=954488abb7c1ea60 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:32.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc72ffd700 1 -- 172.21.15.116:0/52604468 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64009890 con 0x7fbc74154dd0 2020-11-20T12:21:32.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc72ffd700 1 -- 172.21.15.116:0/52604468 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc640099f0 con 0x7fbc74154dd0 2020-11-20T12:21:32.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.910+0000 7fbc72ffd700 1 -- 172.21.15.116:0/52604468 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64009d70 con 0x7fbc74154dd0 2020-11-20T12:21:32.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.914+0000 7fbc79b6b700 1 -- 172.21.15.116:0/52604468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc74154dd0 msgr2=0x7fbc74156160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:32.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.914+0000 7fbc79b6b700 1 --2- 172.21.15.116:0/52604468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc74154dd0 0x7fbc74156160 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7fbc64006fd0 tx=0x7fbc64006eb0).stop 2020-11-20T12:21:32.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.914+0000 7fbc79b6b700 1 -- 172.21.15.116:0/52604468 shutdown_connections 2020-11-20T12:21:32.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.914+0000 7fbc79b6b700 1 --2- 172.21.15.116:0/52604468 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc74154dd0 0x7fbc74156160 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:32.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.914+0000 7fbc79b6b700 1 --2- 172.21.15.116:0/52604468 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc740bd810 0x7fbc740be340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:32.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.914+0000 7fbc79b6b700 1 -- 172.21.15.116:0/52604468 >> 172.21.15.116:0/52604468 conn(0x7fbc7405ba80 msgr2=0x7fbc740c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:32.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.914+0000 7fbc79b6b700 1 -- 172.21.15.116:0/52604468 shutdown_connections 2020-11-20T12:21:32.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.914+0000 7fbc79b6b700 1 -- 172.21.15.116:0/52604468 wait complete. 2020-11-20T12:21:32.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.918+0000 7fbc79b6b700 1 Processor -- start 2020-11-20T12:21:32.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.918+0000 7fbc79b6b700 1 -- start start 2020-11-20T12:21:32.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.918+0000 7fbc79b6b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc740bd810 0x7fbc740c4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:32.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.918+0000 7fbc79b6b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc74154dd0 0x7fbc74152770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:32.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc79b6b700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc741551b0 0x7fbc74158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:32.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc79b6b700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74165e30 con 0x7fbc740bd810 2020-11-20T12:21:32.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc79b6b700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74169950 con 0x7fbc741551b0 2020-11-20T12:21:32.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc79b6b700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc74169bf0 con 0x7fbc74154dd0 2020-11-20T12:21:32.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc737fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc740bd810 0x7fbc740c4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:32.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc737fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc740bd810 0x7fbc740c4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51030/0 (socket says 172.21.15.116:51030) 2020-11-20T12:21:32.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc737fe700 1 -- 172.21.15.116:0/888121514 learned_addr learned my addr 172.21.15.116:0/888121514 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:32.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc73fff700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc741551b0 0x7fbc74158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:32.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc6bfff700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc74154dd0 0x7fbc74152770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:32.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc737fe700 1 -- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc74154dd0 msgr2=0x7fbc74152770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:32.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc737fe700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc74154dd0 0x7fbc74152770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:32.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc737fe700 1 -- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc741551b0 msgr2=0x7fbc74158d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:32.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc737fe700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc741551b0 0x7fbc74158d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:32.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc737fe700 1 -- 172.21.15.116:0/888121514 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc74169e90 con 0x7fbc740bd810 2020-11-20T12:21:32.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc737fe700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc740bd810 0x7fbc740c4cc0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7fbc6400c010 tx=0x7fbc64015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:32.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc6b7fe700 1 -- 172.21.15.116:0/888121514 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc64011dd0 con 0x7fbc740bd810 2020-11-20T12:21:32.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc6b7fe700 1 -- 172.21.15.116:0/888121514 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc6401e460 con 0x7fbc740bd810 2020-11-20T12:21:32.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc6b7fe700 1 -- 172.21.15.116:0/888121514 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc6401e800 con 0x7fbc740bd810 2020-11-20T12:21:32.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc79b6b700 1 -- 172.21.15.116:0/888121514 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc7416a130 con 0x7fbc740bd810 2020-11-20T12:21:32.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.922+0000 7fbc79b6b700 1 -- 172.21.15.116:0/888121514 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc74152c60 con 0x7fbc740bd810 2020-11-20T12:21:32.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.926+0000 7fbc79b6b700 1 -- 172.21.15.116:0/888121514 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc74165fc0 con 0x7fbc740bd810 2020-11-20T12:21:32.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.926+0000 7fbc6b7fe700 1 -- 172.21.15.116:0/888121514 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fbc64025030 con 0x7fbc740bd810 2020-11-20T12:21:32.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.926+0000 7fbc6b7fe700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc500557c0 0x7fbc50057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:32.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.926+0000 7fbc6b7fe700 1 -- 172.21.15.116:0/888121514 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fbc640701a0 con 0x7fbc740bd810 2020-11-20T12:21:32.933 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.930+0000 7fbc6bfff700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc500557c0 0x7fbc50057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:32.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.930+0000 7fbc6bfff700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc500557c0 0x7fbc50057c80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fbc740a8260 tx=0x7fbc5c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:32.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:32.930+0000 7fbc6b7fe700 1 -- 172.21.15.116:0/888121514 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fbc64045fb0 con 0x7fbc740bd810 2020-11-20T12:21:33.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.094+0000 7fbc79b6b700 1 -- 172.21.15.116:0/888121514 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbc7405d2a0 con 0x7fbc740bd810 2020-11-20T12:21:33.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc6b7fe700 1 -- 172.21.15.116:0/888121514 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbc640495d0 con 0x7fbc740bd810 2020-11-20T12:21:33.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 -- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc500557c0 msgr2=0x7fbc50057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc500557c0 0x7fbc50057c80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fbc740a8260 tx=0x7fbc5c008040).stop 2020-11-20T12:21:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 -- 172.21.15.116:0/888121514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc740bd810 msgr2=0x7fbc740c4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc740bd810 0x7fbc740c4cc0 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7fbc6400c010 tx=0x7fbc64015000).stop 2020-11-20T12:21:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 -- 172.21.15.116:0/888121514 shutdown_connections 2020-11-20T12:21:33.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fbc500557c0 0x7fbc50057c80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fbc741551b0 0x7fbc74158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fbc740bd810 0x7fbc740c4cc0 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 --2- 172.21.15.116:0/888121514 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fbc74154dd0 0x7fbc74152770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:33.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 -- 172.21.15.116:0/888121514 >> 172.21.15.116:0/888121514 conn(0x7fbc7405ba80 msgr2=0x7fbc740bfdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:33.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 -- 172.21.15.116:0/888121514 shutdown_connections 2020-11-20T12:21:33.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.102+0000 7fbc79b6b700 1 -- 172.21.15.116:0/888121514 wait complete. 2020-11-20T12:21:33.133 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_authorized_ids' 2020-11-20T12:21:33.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.306+0000 7f3701a5c700 1 Processor -- start 2020-11-20T12:21:33.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.306+0000 7f3701a5c700 1 -- start start 2020-11-20T12:21:33.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.306+0000 7f3701a5c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f36fc062880 0x7f36fc062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:33.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.306+0000 7f3701a5c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36fc0ba1b0 con 0x7f36fc14b990 2020-11-20T12:21:33.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.306+0000 7f3701a5c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36fc1555a0 con 0x7f36fc0b9bf0 2020-11-20T12:21:33.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.306+0000 7f3701a5c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36fc155840 con 0x7f36fc062880 2020-11-20T12:21:33.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.306+0000 7f36fbfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f36fc14b990 0x7f36fc0b94e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60476/0 (socket says 172.21.15.116:60476) 2020-11-20T12:21:33.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.306+0000 7f36fbfff700 1 -- 172.21.15.116:0/480246580 learned_addr learned my addr 172.21.15.116:0/480246580 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:33.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.306+0000 7f3700a5a700 1 --2- 172.21.15.116:0/480246580 >> v2:172.21.15.173:3300/0 conn(0x7f36fc062880 0x7f36fc062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:33.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.306+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3735801440 0 0) 0x7f36fc0ba1b0 con 0x7f36fc14b990 2020-11-20T12:21:33.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.306+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36e0003240 con 0x7f36fc14b990 2020-11-20T12:21:33.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835376572 0 0) 0x7f36fc1555a0 con 0x7f36fc0b9bf0 2020-11-20T12:21:33.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36fc0ba1b0 con 0x7f36fc0b9bf0 2020-11-20T12:21:33.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3555027620 0 0) 0x7f36e0003240 con 0x7f36fc14b990 2020-11-20T12:21:33.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 >> v2:172.21.15.173:3300/0 conn(0x7f36fc062880 msgr2=0x7f36fc062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:33.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 --2- 172.21.15.116:0/480246580 >> v2:172.21.15.173:3300/0 conn(0x7f36fc062880 0x7f36fc062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:33.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 >> v1:172.21.15.173:6789/0 conn(0x7f36fc0b9bf0 legacy=0x7f36fc151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:33.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc155ae0 con 0x7f36fc14b990 2020-11-20T12:21:33.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f36e80014e0 con 0x7f36fc14b990 2020-11-20T12:21:33.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 >> v1:172.21.15.116:6789/0 conn(0x7f36fc14b990 legacy=0x7f36fc0b94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:33.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 --2- 172.21.15.116:0/480246580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36e0003910 0x7f36e0003d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:33.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 --2- 172.21.15.116:0/480246580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36e00044e0 0x7f36e0004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:33.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 --2- 172.21.15.116:0/480246580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36e0004ea0 0x7f36e000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:33.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc155840 con 0x7f36e00044e0 2020-11-20T12:21:33.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e000bc50 con 0x7f36e0004ea0 2020-11-20T12:21:33.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e000be10 con 0x7f36e0003910 2020-11-20T12:21:33.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fbfff700 1 --2- 172.21.15.116:0/480246580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36e00044e0 0x7f36e0004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:33.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f370125b700 1 --2- 172.21.15.116:0/480246580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36e0004ea0 0x7f36e000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:33.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f3700a5a700 1 --2- 172.21.15.116:0/480246580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36e0003910 0x7f36e0003d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:33.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fbfff700 1 -- 172.21.15.116:0/480246580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36e0003910 msgr2=0x7f36e0003d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:33.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fbfff700 1 --2- 172.21.15.116:0/480246580 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36e0003910 0x7f36e0003d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:33.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fbfff700 1 -- 172.21.15.116:0/480246580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36e0004ea0 msgr2=0x7f36e000b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:33.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fbfff700 1 --2- 172.21.15.116:0/480246580 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36e0004ea0 0x7f36e000b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:33.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fbfff700 1 -- 172.21.15.116:0/480246580 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e0003240 con 0x7f36e00044e0 2020-11-20T12:21:33.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f370125b700 1 -- 172.21.15.116:0/480246580 reap_dead start 2020-11-20T12:21:33.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fbfff700 1 --2- 172.21.15.116:0/480246580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36e00044e0 0x7f36e0004960 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7f36e800c160 tx=0x7f36fc155d80).ready entity=mon.0 client_cookie=f5fbefc8868ab7e9 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:33.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e800ce60 con 0x7f36e00044e0 2020-11-20T12:21:33.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e8007750 con 0x7f36e00044e0 2020-11-20T12:21:33.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.310+0000 7f36fb7fe700 1 -- 172.21.15.116:0/480246580 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8007320 con 0x7f36e00044e0 2020-11-20T12:21:33.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.314+0000 7f3701a5c700 1 -- 172.21.15.116:0/480246580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36e00044e0 msgr2=0x7f36e0004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:33.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.314+0000 7f3701a5c700 1 --2- 172.21.15.116:0/480246580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36e00044e0 0x7f36e0004960 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7f36e800c160 tx=0x7f36fc155d80).stop 2020-11-20T12:21:33.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.314+0000 7f3701a5c700 1 -- 172.21.15.116:0/480246580 shutdown_connections 2020-11-20T12:21:33.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.314+0000 7f3701a5c700 1 --2- 172.21.15.116:0/480246580 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36e00044e0 0x7f36e0004960 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:33.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.314+0000 7f3701a5c700 1 -- 172.21.15.116:0/480246580 >> 172.21.15.116:0/480246580 conn(0x7f36fc060c40 msgr2=0x7f36fc0c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:33.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.314+0000 7f3701a5c700 1 -- 172.21.15.116:0/480246580 shutdown_connections 2020-11-20T12:21:33.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.314+0000 7f3701a5c700 1 -- 172.21.15.116:0/480246580 wait complete. 2020-11-20T12:21:33.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.314+0000 7f3701a5c700 1 Processor -- start 2020-11-20T12:21:33.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f3701a5c700 1 -- start start 2020-11-20T12:21:33.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f3701a5c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36e00044e0 0x7f36fc1692c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:33.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f3701a5c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36fc146ae0 0x7f36fc0a8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:33.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f3701a5c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36fc1686d0 0x7f36fc168b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:33.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f3701a5c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc1489e0 con 0x7f36fc146ae0 2020-11-20T12:21:33.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f3701a5c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc169020 con 0x7f36e00044e0 2020-11-20T12:21:33.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f3701a5c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36fc146f60 con 0x7f36fc1686d0 2020-11-20T12:21:33.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f36fbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36fc146ae0 0x7f36fc0a8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:33.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f36fbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36fc146ae0 0x7f36fc0a8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51052/0 (socket says 172.21.15.116:51052) 2020-11-20T12:21:33.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f36fbfff700 1 -- 172.21.15.116:0/650155309 learned_addr learned my addr 172.21.15.116:0/650155309 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:33.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f3700a5a700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36e00044e0 0x7f36fc1692c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:33.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f370125b700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36fc1686d0 0x7f36fc168b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:33.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f36fbfff700 1 -- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36fc1686d0 msgr2=0x7f36fc168b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:33.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f36fbfff700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36fc1686d0 0x7f36fc168b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:33.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f36fbfff700 1 -- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36e00044e0 msgr2=0x7f36fc1692c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:33.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f36fbfff700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36e00044e0 0x7f36fc1692c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:33.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f36fbfff700 1 -- 172.21.15.116:0/650155309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc0a96a0 con 0x7f36fc146ae0 2020-11-20T12:21:33.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f370125b700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36fc1686d0 0x7f36fc168b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:33.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.318+0000 7f36fbfff700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36fc146ae0 0x7f36fc0a8e80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f36e8000f50 tx=0x7f36e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:33.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.322+0000 7f36f8ff9700 1 -- 172.21.15.116:0/650155309 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8006c70 con 0x7f36fc146ae0 2020-11-20T12:21:33.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.322+0000 7f36f8ff9700 1 -- 172.21.15.116:0/650155309 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36e8006dd0 con 0x7f36fc146ae0 2020-11-20T12:21:33.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.322+0000 7f36f8ff9700 1 -- 172.21.15.116:0/650155309 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36e8013680 con 0x7f36fc146ae0 2020-11-20T12:21:33.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.322+0000 7f3701a5c700 1 -- 172.21.15.116:0/650155309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36fc0a9940 con 0x7f36fc146ae0 2020-11-20T12:21:33.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.322+0000 7f3701a5c700 1 -- 172.21.15.116:0/650155309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36fc148e70 con 0x7f36fc146ae0 2020-11-20T12:21:33.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.322+0000 7f3701a5c700 1 -- 172.21.15.116:0/650155309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36fc149030 con 0x7f36fc146ae0 2020-11-20T12:21:33.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.322+0000 7f36f8ff9700 1 -- 172.21.15.116:0/650155309 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f36e8026020 con 0x7f36fc146ae0 2020-11-20T12:21:33.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.322+0000 7f36f8ff9700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36d40557c0 0x7f36d4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:33.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.322+0000 7f36f8ff9700 1 -- 172.21.15.116:0/650155309 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f36e806fa40 con 0x7f36fc146ae0 2020-11-20T12:21:33.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.326+0000 7f3700a5a700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36d40557c0 0x7f36d4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:33.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.326+0000 7f3700a5a700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36d40557c0 0x7f36d4057c80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f36fc05f2f0 tx=0x7f36f0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:33.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.330+0000 7f36f8ff9700 1 -- 172.21.15.116:0/650155309 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f36e8045920 con 0x7f36fc146ae0 2020-11-20T12:21:33.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:33.545+0000 7f3701a5c700 1 -- 172.21.15.116:0/650155309 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_authorized_ids"]} v 0) v1 -- 0x7f36fc05cf30 con 0x7f36fc146ae0 2020-11-20T12:21:34.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.253+0000 7f36f8ff9700 1 -- 172.21.15.116:0/650155309 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_authorized_ids"]}]=0 v320) v1 ==== 151+0+0 (secure 0 0 0) 0x7f36e8048f00 con 0x7f36fc146ae0 2020-11-20T12:21:34.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 -- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36d40557c0 msgr2=0x7f36d4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:34.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36d40557c0 0x7f36d4057c80 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f36fc05f2f0 tx=0x7f36f0008040).stop 2020-11-20T12:21:34.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 -- 172.21.15.116:0/650155309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36fc146ae0 msgr2=0x7f36fc0a8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:34.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36fc146ae0 0x7f36fc0a8e80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f36e8000f50 tx=0x7f36e8015000).stop 2020-11-20T12:21:34.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 -- 172.21.15.116:0/650155309 shutdown_connections 2020-11-20T12:21:34.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f36d40557c0 0x7f36d4057c80 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:34.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36fc1686d0 0x7f36fc168b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:34.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36e00044e0 0x7f36fc1692c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:34.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 --2- 172.21.15.116:0/650155309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36fc146ae0 0x7f36fc0a8e80 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:34.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 -- 172.21.15.116:0/650155309 >> 172.21.15.116:0/650155309 conn(0x7f36fc060c40 msgr2=0x7f36fc0c2180 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:34.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 -- 172.21.15.116:0/650155309 shutdown_connections 2020-11-20T12:21:34.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.257+0000 7f3701a5c700 1 -- 172.21.15.116:0/650155309 wait complete. 2020-11-20T12:21:34.289 INFO:tasks.cephfs_test_runner:test_get_authorized_ids (tasks.cephfs.test_volume_client.TestVolumeClient) ... ok 2020-11-20T12:21:34.289 INFO:tasks.cephfs_test_runner:Starting test: test_get_object (tasks.cephfs.test_volume_client.TestVolumeClient) 2020-11-20T12:21:34.290 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_object' 2020-11-20T12:21:34.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.461+0000 7ff6808fe700 1 Processor -- start 2020-11-20T12:21:34.466 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.461+0000 7ff6808fe700 1 -- start start 2020-11-20T12:21:34.470 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.461+0000 7ff6808fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff67c061490 0x7ff67c152150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:34.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.461+0000 7ff6808fe700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c156250 con 0x7ff67c14bbd0 2020-11-20T12:21:34.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.461+0000 7ff6808fe700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c1557d0 con 0x7ff67c14bfb0 2020-11-20T12:21:34.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.461+0000 7ff6808fe700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c0a70e0 con 0x7ff67c061490 2020-11-20T12:21:34.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff67ad9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7ff67c14bbd0 0x7ff67c0a8960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60492/0 (socket says 172.21.15.116:60492) 2020-11-20T12:21:34.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff67ad9d700 1 -- 172.21.15.116:0/2512670664 learned_addr learned my addr 172.21.15.116:0/2512670664 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:34.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff67b59e700 1 --2- 172.21.15.116:0/2512670664 >> v2:172.21.15.173:3300/0 conn(0x7ff67c061490 0x7ff67c152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:34.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff679d9b700 1 -- 172.21.15.116:0/2512670664 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752132900 0 0) 0x7ff67c156250 con 0x7ff67c14bbd0 2020-11-20T12:21:34.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff679d9b700 1 -- 172.21.15.116:0/2512670664 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff668003240 con 0x7ff67c14bbd0 2020-11-20T12:21:34.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff67b59e700 1 -- 172.21.15.116:0/2512670664 >> v1:172.21.15.173:6789/0 conn(0x7ff67c14bfb0 legacy=0x7ff67c064a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:34.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff67b59e700 1 -- 172.21.15.116:0/2512670664 >> v1:172.21.15.116:6789/0 conn(0x7ff67c14bbd0 legacy=0x7ff67c0a8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:34.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff67b59e700 1 -- 172.21.15.116:0/2512670664 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c155a70 con 0x7ff67c061490 2020-11-20T12:21:34.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff67b59e700 1 --2- 172.21.15.116:0/2512670664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff67c061490 0x7ff67c152150 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff674009c90 tx=0x7ff6740091d0).ready entity=mon.1 client_cookie=33d17b202c8fafb3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:34.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff679d9b700 1 -- 172.21.15.116:0/2512670664 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67400ba70 con 0x7ff67c061490 2020-11-20T12:21:34.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff679d9b700 1 -- 172.21.15.116:0/2512670664 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67400bbd0 con 0x7ff67c061490 2020-11-20T12:21:34.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff679d9b700 1 -- 172.21.15.116:0/2512670664 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674011400 con 0x7ff67c061490 2020-11-20T12:21:34.478 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff6808fe700 1 -- 172.21.15.116:0/2512670664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff67c061490 msgr2=0x7ff67c152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:34.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.465+0000 7ff6808fe700 1 --2- 172.21.15.116:0/2512670664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff67c061490 0x7ff67c152150 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7ff674009c90 tx=0x7ff6740091d0).stop 2020-11-20T12:21:34.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.469+0000 7ff6808fe700 1 -- 172.21.15.116:0/2512670664 shutdown_connections 2020-11-20T12:21:34.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.469+0000 7ff6808fe700 1 --2- 172.21.15.116:0/2512670664 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff67c061490 0x7ff67c152150 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:34.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.469+0000 7ff6808fe700 1 -- 172.21.15.116:0/2512670664 >> 172.21.15.116:0/2512670664 conn(0x7ff67c0c6560 msgr2=0x7ff67c0c6940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:34.479 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.469+0000 7ff6808fe700 1 -- 172.21.15.116:0/2512670664 shutdown_connections 2020-11-20T12:21:34.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.469+0000 7ff6808fe700 1 -- 172.21.15.116:0/2512670664 wait complete. 2020-11-20T12:21:34.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.469+0000 7ff6808fe700 1 Processor -- start 2020-11-20T12:21:34.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.469+0000 7ff6808fe700 1 -- start start 2020-11-20T12:21:34.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff6808fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff67c061490 0x7ff67c161720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:34.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff6808fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff67c14bbd0 0x7ff67c14eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:34.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff6808fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff67c14bfb0 0x7ff67c164430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:34.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff6808fe700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c151d70 con 0x7ff67c061490 2020-11-20T12:21:34.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff6808fe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c1652f0 con 0x7ff67c14bbd0 2020-11-20T12:21:34.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff6808fe700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c164db0 con 0x7ff67c14bfb0 2020-11-20T12:21:34.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff67c061490 0x7ff67c161720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:34.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff67c061490 0x7ff67c161720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51064/0 (socket says 172.21.15.116:51064) 2020-11-20T12:21:34.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67ad9d700 1 -- 172.21.15.116:0/2427767708 learned_addr learned my addr 172.21.15.116:0/2427767708 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:34.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67b59e700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff67c14bfb0 0x7ff67c164430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:34.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67a59c700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff67c14bbd0 0x7ff67c14eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:34.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67ad9d700 1 -- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff67c14bfb0 msgr2=0x7ff67c164430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:34.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67ad9d700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff67c14bfb0 0x7ff67c164430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:34.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67ad9d700 1 -- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff67c14bbd0 msgr2=0x7ff67c14eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:34.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67ad9d700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff67c14bbd0 0x7ff67c14eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:34.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67ad9d700 1 -- 172.21.15.116:0/2427767708 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c165050 con 0x7ff67c061490 2020-11-20T12:21:34.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67a59c700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff67c14bbd0 0x7ff67c14eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:34.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67b59e700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff67c14bfb0 0x7ff67c164430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:34.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff67ad9d700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff67c061490 0x7ff67c161720 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7ff66c00c6f0 tx=0x7ff66c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:34.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff662ffd700 1 -- 172.21.15.116:0/2427767708 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c007320 con 0x7ff67c061490 2020-11-20T12:21:34.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff662ffd700 1 -- 172.21.15.116:0/2427767708 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c007480 con 0x7ff67c061490 2020-11-20T12:21:34.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff6808fe700 1 -- 172.21.15.116:0/2427767708 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c160fe0 con 0x7ff67c061490 2020-11-20T12:21:34.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff662ffd700 1 -- 172.21.15.116:0/2427767708 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c005800 con 0x7ff67c061490 2020-11-20T12:21:34.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff6808fe700 1 -- 172.21.15.116:0/2427767708 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c161c10 con 0x7ff67c061490 2020-11-20T12:21:34.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.473+0000 7ff6808fe700 1 -- 172.21.15.116:0/2427767708 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c05cf30 con 0x7ff67c061490 2020-11-20T12:21:34.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.477+0000 7ff662ffd700 1 -- 172.21.15.116:0/2427767708 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff66c010030 con 0x7ff67c061490 2020-11-20T12:21:34.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.477+0000 7ff662ffd700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6580557c0 0x7ff658057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:34.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.477+0000 7ff662ffd700 1 -- 172.21.15.116:0/2427767708 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff66c06e420 con 0x7ff67c061490 2020-11-20T12:21:34.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.477+0000 7ff67a59c700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6580557c0 0x7ff658057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:34.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.477+0000 7ff67a59c700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6580557c0 0x7ff658057c80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7ff67c157c40 tx=0x7ff670009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:34.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.481+0000 7ff662ffd700 1 -- 172.21.15.116:0/2427767708 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff66c044260 con 0x7ff67c061490 2020-11-20T12:21:34.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:34.693+0000 7ff6808fe700 1 -- 172.21.15.116:0/2427767708 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_object"]} v 0) v1 -- 0x7ff67c063490 con 0x7ff67c061490 2020-11-20T12:21:35.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.265+0000 7ff662ffd700 1 -- 172.21.15.116:0/2427767708 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_object"]}]=0 v321) v1 ==== 146+0+0 (secure 0 0 0) 0x7ff66c047880 con 0x7ff67c061490 2020-11-20T12:21:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 -- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6580557c0 msgr2=0x7ff658057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6580557c0 0x7ff658057c80 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7ff67c157c40 tx=0x7ff670009040).stop 2020-11-20T12:21:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 -- 172.21.15.116:0/2427767708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff67c061490 msgr2=0x7ff67c161720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff67c061490 0x7ff67c161720 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7ff66c00c6f0 tx=0x7ff66c007ad0).stop 2020-11-20T12:21:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 -- 172.21.15.116:0/2427767708 shutdown_connections 2020-11-20T12:21:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6580557c0 0x7ff658057c80 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff67c14bfb0 0x7ff67c164430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff67c061490 0x7ff67c161720 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 --2- 172.21.15.116:0/2427767708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff67c14bbd0 0x7ff67c14eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 -- 172.21.15.116:0/2427767708 >> 172.21.15.116:0/2427767708 conn(0x7ff67c0c6560 msgr2=0x7ff67c148fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 -- 172.21.15.116:0/2427767708 shutdown_connections 2020-11-20T12:21:35.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.269+0000 7ff6808fe700 1 -- 172.21.15.116:0/2427767708 wait complete. 2020-11-20T12:21:35.297 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-20T12:21:35.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.477+0000 7f471f112700 1 Processor -- start 2020-11-20T12:21:35.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.477+0000 7f471f112700 1 -- start start 2020-11-20T12:21:35.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471f112700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f47180aa7d0 0x7f47180a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471f112700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f47180ad540 0x7f47180ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471f112700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47180ace40 con 0x7f47180aabb0 2020-11-20T12:21:35.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471f112700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47181526c0 con 0x7f47180ad540 2020-11-20T12:21:35.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471f112700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4718152960 con 0x7f47180aa7d0 2020-11-20T12:21:35.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471e911700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f47180ad540 0x7f47180ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471e110700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f47180aa7d0 0x7f47180a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471e911700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f47180ad540 0x7f47180ad980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51076/0 (socket says 172.21.15.116:51076) 2020-11-20T12:21:35.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471d90f700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f47180aabb0 0x7f4718151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:56738/0 (socket says 172.21.15.116:56738) 2020-11-20T12:21:35.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471e911700 1 -- 172.21.15.116:0/3263773941 learned_addr learned my addr 172.21.15.116:0/3263773941 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:35.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471e911700 1 -- 172.21.15.116:0/3263773941 >> v2:172.21.15.173:3300/0 conn(0x7f47180aa7d0 msgr2=0x7f47180a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471e911700 1 --2- 172.21.15.116:0/3263773941 >> v2:172.21.15.173:3300/0 conn(0x7f47180aa7d0 0x7f47180a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471d10e700 1 -- 172.21.15.116:0/3263773941 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 589106143 0 0) 0x7f47180ace40 con 0x7f47180aabb0 2020-11-20T12:21:35.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471e911700 1 -- 172.21.15.116:0/3263773941 >> v1:172.21.15.173:6789/0 conn(0x7f47180aabb0 legacy=0x7f4718151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471e911700 1 -- 172.21.15.116:0/3263773941 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4718152c00 con 0x7f47180ad540 2020-11-20T12:21:35.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471e911700 1 --2- 172.21.15.116:0/3263773941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47180ad540 0x7f47180ad980 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f4718081950 tx=0x7f4710009290).ready entity=mon.0 client_cookie=505419ed41956026 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:35.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471d10e700 1 -- 172.21.15.116:0/3263773941 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471000bbc0 con 0x7f47180ad540 2020-11-20T12:21:35.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471d10e700 1 -- 172.21.15.116:0/3263773941 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471000bd20 con 0x7f47180ad540 2020-11-20T12:21:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471d10e700 1 -- 172.21.15.116:0/3263773941 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710011610 con 0x7f47180ad540 2020-11-20T12:21:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471f112700 1 -- 172.21.15.116:0/3263773941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47180ad540 msgr2=0x7f47180ad980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.481+0000 7f471f112700 1 --2- 172.21.15.116:0/3263773941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47180ad540 0x7f47180ad980 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f4718081950 tx=0x7f4710009290).stop 2020-11-20T12:21:35.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.485+0000 7f471f112700 1 -- 172.21.15.116:0/3263773941 shutdown_connections 2020-11-20T12:21:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.485+0000 7f471f112700 1 --2- 172.21.15.116:0/3263773941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47180ad540 0x7f47180ad980 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.485+0000 7f471f112700 1 --2- 172.21.15.116:0/3263773941 >> v2:172.21.15.173:3300/0 conn(0x7f47180aa7d0 0x7f47180a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.485+0000 7f471f112700 1 -- 172.21.15.116:0/3263773941 >> 172.21.15.116:0/3263773941 conn(0x7f47180678a0 msgr2=0x7f4718067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:35.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.485+0000 7f471f112700 1 -- 172.21.15.116:0/3263773941 shutdown_connections 2020-11-20T12:21:35.494 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.485+0000 7f471f112700 1 -- 172.21.15.116:0/3263773941 wait complete. 2020-11-20T12:21:35.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.485+0000 7f471f112700 1 Processor -- start 2020-11-20T12:21:35.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.485+0000 7f471f112700 1 -- start start 2020-11-20T12:21:35.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471f112700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47180aa7d0 0x7f471814ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471f112700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47180aabb0 0x7f47180a29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471f112700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47180ad540 0x7f47180a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471f112700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180a3420 con 0x7f47180aa7d0 2020-11-20T12:21:35.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471f112700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180a36c0 con 0x7f47180aabb0 2020-11-20T12:21:35.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471f112700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47180a3960 con 0x7f47180ad540 2020-11-20T12:21:35.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471e110700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47180aa7d0 0x7f471814ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471e110700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47180aa7d0 0x7f471814ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51080/0 (socket says 172.21.15.116:51080) 2020-11-20T12:21:35.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471e110700 1 -- 172.21.15.116:0/2659952871 learned_addr learned my addr 172.21.15.116:0/2659952871 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:35.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471d90f700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47180aabb0 0x7f47180a29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471e911700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47180ad540 0x7f47180a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471e110700 1 -- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47180ad540 msgr2=0x7f47180a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471e110700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47180ad540 0x7f47180a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471e110700 1 -- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47180aabb0 msgr2=0x7f47180a29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471e110700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47180aabb0 0x7f47180a29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471e110700 1 -- 172.21.15.116:0/2659952871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471809f4a0 con 0x7f47180aa7d0 2020-11-20T12:21:35.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471d90f700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47180aabb0 0x7f47180a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:35.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471e110700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47180aa7d0 0x7f471814ae00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f471400eba0 tx=0x7f4714006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:35.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f47067fc700 1 -- 172.21.15.116:0/2659952871 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471400ca60 con 0x7f47180aa7d0 2020-11-20T12:21:35.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f47067fc700 1 -- 172.21.15.116:0/2659952871 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471400cbc0 con 0x7f47180aa7d0 2020-11-20T12:21:35.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471f112700 1 -- 172.21.15.116:0/2659952871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471814b2f0 con 0x7f47180aa7d0 2020-11-20T12:21:35.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f47067fc700 1 -- 172.21.15.116:0/2659952871 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47140123f0 con 0x7f47180aa7d0 2020-11-20T12:21:35.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.489+0000 7f471f112700 1 -- 172.21.15.116:0/2659952871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471814e720 con 0x7f47180aa7d0 2020-11-20T12:21:35.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.493+0000 7f471f112700 1 -- 172.21.15.116:0/2659952871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47181517d0 con 0x7f47180aa7d0 2020-11-20T12:21:35.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.493+0000 7f47067fc700 1 -- 172.21.15.116:0/2659952871 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f47140127a0 con 0x7f47180aa7d0 2020-11-20T12:21:35.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.493+0000 7f47067fc700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46f00557f0 0x7f46f0057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.493+0000 7f47067fc700 1 -- 172.21.15.116:0/2659952871 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f471406ebe0 con 0x7f47180aa7d0 2020-11-20T12:21:35.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.493+0000 7f471d90f700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46f00557f0 0x7f46f0057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.497+0000 7f471d90f700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46f00557f0 0x7f46f0057cb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f471814b960 tx=0x7f4708007120).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:35.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.501+0000 7f47067fc700 1 -- 172.21.15.116:0/2659952871 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4714044970 con 0x7f47180aa7d0 2020-11-20T12:21:35.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.661+0000 7f471f112700 1 -- 172.21.15.116:0/2659952871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f47180a3b20 con 0x7f47180aa7d0 2020-11-20T12:21:35.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.669+0000 7f47067fc700 1 -- 172.21.15.116:0/2659952871 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4714047f90 con 0x7f47180aa7d0 2020-11-20T12:21:35.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 -- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46f00557f0 msgr2=0x7f46f0057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46f00557f0 0x7f46f0057cb0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7f471814b960 tx=0x7f4708007120).stop 2020-11-20T12:21:35.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 -- 172.21.15.116:0/2659952871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47180aa7d0 msgr2=0x7f471814ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47180aa7d0 0x7f471814ae00 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f471400eba0 tx=0x7f4714006df0).stop 2020-11-20T12:21:35.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 -- 172.21.15.116:0/2659952871 shutdown_connections 2020-11-20T12:21:35.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46f00557f0 0x7f46f0057cb0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f47180ad540 0x7f47180a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f47180aa7d0 0x7f471814ae00 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 --2- 172.21.15.116:0/2659952871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f47180aabb0 0x7f47180a29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 -- 172.21.15.116:0/2659952871 >> 172.21.15.116:0/2659952871 conn(0x7f47180678a0 msgr2=0x7f47180a9790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:35.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 -- 172.21.15.116:0/2659952871 shutdown_connections 2020-11-20T12:21:35.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.673+0000 7f471f112700 1 -- 172.21.15.116:0/2659952871 wait complete. 2020-11-20T12:21:35.700 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:21:35.701 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:21:35.738 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:21:35.774 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:21:35.775 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2020-11-20T12:21:35.775 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:21:35.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.941+0000 7fd5d359e700 1 Processor -- start 2020-11-20T12:21:35.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.941+0000 7fd5d359e700 1 -- start start 2020-11-20T12:21:35.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d413afa0 0x7fd5d40aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d359e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d40ada00 con 0x7fd5d40ad440 2020-11-20T12:21:35.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d359e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5d40adca0 con 0x7fd5d413b380 2020-11-20T12:21:35.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40adf40 con 0x7fd5d413afa0 2020-11-20T12:21:35.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d2d9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd5d40ad440 0x7fd5d40a8790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60524/0 (socket says 172.21.15.116:60524) 2020-11-20T12:21:35.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d2d9d700 1 -- 172.21.15.116:0/1314622798 learned_addr learned my addr 172.21.15.116:0/1314622798 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:35.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d259c700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d413afa0 0x7fd5d40aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 890682267 0 0) 0x7fd5d40ada00 con 0x7fd5d40ad440 2020-11-20T12:21:35.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5b4003240 con 0x7fd5d40ad440 2020-11-20T12:21:35.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 936594925 0 0) 0x7fd5d40adca0 con 0x7fd5d413b380 2020-11-20T12:21:35.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5d40ada00 con 0x7fd5d413b380 2020-11-20T12:21:35.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3709621697 0 0) 0x7fd5b4003240 con 0x7fd5d40ad440 2020-11-20T12:21:35.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d413afa0 msgr2=0x7fd5d40aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:35.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d413afa0 0x7fd5d40aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 >> v1:172.21.15.173:6789/0 conn(0x7fd5d413b380 legacy=0x7fd5d40acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d40ae1e0 con 0x7fd5d40ad440 2020-11-20T12:21:35.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fd5cc001860 con 0x7fd5d40ad440 2020-11-20T12:21:35.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 >> v1:172.21.15.116:6789/0 conn(0x7fd5d40ad440 legacy=0x7fd5d40a8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d413afa0 0x7fd5b4004950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b4004e40 0x7fd5b4003480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b4003970 0x7fd5b4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d40adf40 con 0x7fd5b4003970 2020-11-20T12:21:35.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4004310 con 0x7fd5b4004e40 2020-11-20T12:21:35.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b400b780 con 0x7fd5d413afa0 2020-11-20T12:21:35.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5c9d9b700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b4004e40 0x7fd5b4003480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d2d9d700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b4003970 0x7fd5b4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.945+0000 7fd5d259c700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d413afa0 0x7fd5b4004950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5c9d9b700 1 -- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d413afa0 msgr2=0x7fd5b4004950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:35.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5c9d9b700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d413afa0 0x7fd5b4004950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5c9d9b700 1 -- 172.21.15.116:0/1314622798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b4003970 msgr2=0x7fd5b4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:35.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5c9d9b700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b4003970 0x7fd5b4003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5c9d9b700 1 -- 172.21.15.116:0/1314622798 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b4003240 con 0x7fd5b4004e40 2020-11-20T12:21:35.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5d2d9d700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5b4003970 0x7fd5b4003df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:35.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5c9d9b700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b4004e40 0x7fd5b4003480 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fd5bc001a50 tx=0x7fd5bc007b90).ready entity=mon.1 client_cookie=9a30b5f57f33a32b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:35.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc007020 con 0x7fd5b4004e40 2020-11-20T12:21:35.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc007180 con 0x7fd5b4004e40 2020-11-20T12:21:35.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5d1d9b700 1 -- 172.21.15.116:0/1314622798 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc00c830 con 0x7fd5b4004e40 2020-11-20T12:21:35.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5d359e700 1 -- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b4004e40 msgr2=0x7fd5b4003480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5d359e700 1 --2- 172.21.15.116:0/1314622798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5b4004e40 0x7fd5b4003480 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7fd5bc001a50 tx=0x7fd5bc007b90).stop 2020-11-20T12:21:35.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5d2d9d700 1 -- 172.21.15.116:0/1314622798 reap_dead start 2020-11-20T12:21:35.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5d359e700 1 -- 172.21.15.116:0/1314622798 shutdown_connections 2020-11-20T12:21:35.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5d359e700 1 -- 172.21.15.116:0/1314622798 >> 172.21.15.116:0/1314622798 conn(0x7fd5d40678a0 msgr2=0x7fd5d4067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:35.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5d359e700 1 -- 172.21.15.116:0/1314622798 shutdown_connections 2020-11-20T12:21:35.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.949+0000 7fd5d359e700 1 -- 172.21.15.116:0/1314622798 wait complete. 2020-11-20T12:21:35.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.953+0000 7fd5d359e700 1 Processor -- start 2020-11-20T12:21:35.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.953+0000 7fd5d359e700 1 -- start start 2020-11-20T12:21:35.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.953+0000 7fd5d359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5d41541f0 0x7fd5d41545d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.953+0000 7fd5d359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d409fbe0 0x7fd5d4160c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.953+0000 7fd5d359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5d4161170 0x7fd5d4137150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.953+0000 7fd5d359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d4161c50 con 0x7fd5d41541f0 2020-11-20T12:21:35.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.953+0000 7fd5d359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d4161710 con 0x7fd5d4161170 2020-11-20T12:21:35.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.953+0000 7fd5d359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d41619b0 con 0x7fd5d409fbe0 2020-11-20T12:21:35.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.953+0000 7fd5d259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5d41541f0 0x7fd5d41545d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d259c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5d41541f0 0x7fd5d41545d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51098/0 (socket says 172.21.15.116:51098) 2020-11-20T12:21:35.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d259c700 1 -- 172.21.15.116:0/197827837 learned_addr learned my addr 172.21.15.116:0/197827837 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:35.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5c9d9b700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d409fbe0 0x7fd5d4160c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d2d9d700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5d4161170 0x7fd5d4137150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d259c700 1 -- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d409fbe0 msgr2=0x7fd5d4160c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d259c700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d409fbe0 0x7fd5d4160c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d259c700 1 -- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5d4161170 msgr2=0x7fd5d4137150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:35.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d259c700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5d4161170 0x7fd5d4137150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:35.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d259c700 1 -- 172.21.15.116:0/197827837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d409f4a0 con 0x7fd5d41541f0 2020-11-20T12:21:35.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d2d9d700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5d4161170 0x7fd5d4137150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:35.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d259c700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5d41541f0 0x7fd5d41545d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7fd5c4011cc0 tx=0x7fd5c4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:35.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5caffd700 1 -- 172.21.15.116:0/197827837 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400f9e0 con 0x7fd5d41541f0 2020-11-20T12:21:35.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5caffd700 1 -- 172.21.15.116:0/197827837 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c400fb40 con 0x7fd5d41541f0 2020-11-20T12:21:35.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d359e700 1 -- 172.21.15.116:0/197827837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d409f740 con 0x7fd5d41541f0 2020-11-20T12:21:35.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5caffd700 1 -- 172.21.15.116:0/197827837 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c400fec0 con 0x7fd5d41541f0 2020-11-20T12:21:35.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d359e700 1 -- 172.21.15.116:0/197827837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d40a0060 con 0x7fd5d41541f0 2020-11-20T12:21:35.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.957+0000 7fd5d359e700 1 -- 172.21.15.116:0/197827837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d405cf30 con 0x7fd5d41541f0 2020-11-20T12:21:35.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.961+0000 7fd5caffd700 1 -- 172.21.15.116:0/197827837 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd5c4014650 con 0x7fd5d41541f0 2020-11-20T12:21:35.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.961+0000 7fd5caffd700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5b00557c0 0x7fd5b0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:35.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.961+0000 7fd5caffd700 1 -- 172.21.15.116:0/197827837 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd5c406f060 con 0x7fd5d41541f0 2020-11-20T12:21:35.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.961+0000 7fd5c9d9b700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5b00557c0 0x7fd5b0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:35.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.965+0000 7fd5c9d9b700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5b00557c0 0x7fd5b0057c80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fd5bc009000 tx=0x7fd5bc016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:35.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:35.969+0000 7fd5caffd700 1 -- 172.21.15.116:0/197827837 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd5c4044ea0 con 0x7fd5d41541f0 2020-11-20T12:21:36.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.177+0000 7fd5d359e700 1 -- 172.21.15.116:0/197827837 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5d4060170 con 0x7fd5d41541f0 2020-11-20T12:21:36.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.177+0000 7fd5caffd700 1 -- 172.21.15.116:0/197827837 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2851 (secure 0 0 0) 0x7fd5c4018020 con 0x7fd5d41541f0 2020-11-20T12:21:36.183 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:36.183 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1638614527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1638614527},{"type":"v1","addr":"172.21.15.173:6837","nonce":1638614527}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3682335131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3682335131},{"type":"v1","addr":"172.21.15.173:6836","nonce":3682335131}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7725,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/3975405805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3975405805},{"type":"v1","addr":"172.21.15.116:6834","nonce":3975405805}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:11.109567+0000","modified":"2020-11-20T12:21:13.118616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/267899152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":267899152},{"type":"v1","addr":"172.21.15.116:6835","nonce":267899152}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-20T12:21:36.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 -- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5b00557c0 msgr2=0x7fd5b0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:36.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5b00557c0 0x7fd5b0057c80 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fd5bc009000 tx=0x7fd5bc016000).stop 2020-11-20T12:21:36.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 -- 172.21.15.116:0/197827837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5d41541f0 msgr2=0x7fd5d41545d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:36.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5d41541f0 0x7fd5d41545d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7fd5c4011cc0 tx=0x7fd5c4006df0).stop 2020-11-20T12:21:36.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 -- 172.21.15.116:0/197827837 shutdown_connections 2020-11-20T12:21:36.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd5b00557c0 0x7fd5b0057c80 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd5d4161170 0x7fd5d4137150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd5d41541f0 0x7fd5d41545d0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 --2- 172.21.15.116:0/197827837 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd5d409fbe0 0x7fd5d4160c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 -- 172.21.15.116:0/197827837 >> 172.21.15.116:0/197827837 conn(0x7fd5d40678a0 msgr2=0x7fd5d413a490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:36.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 -- 172.21.15.116:0/197827837 shutdown_connections 2020-11-20T12:21:36.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.181+0000 7fd5d359e700 1 -- 172.21.15.116:0/197827837 wait complete. 2020-11-20T12:21:36.189 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 60 2020-11-20T12:21:36.209 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:36.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.389+0000 7f59a70c7700 1 Processor -- start 2020-11-20T12:21:36.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.389+0000 7f59a70c7700 1 -- start start 2020-11-20T12:21:36.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a70c7700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f59a00c3e10 0x7f59a00c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a70c7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a00a55f0 0x7f59a00c80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a70c7700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59a00c35e0 con 0x7f59a00c93a0 2020-11-20T12:21:36.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a70c7700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59a00c3880 con 0x7f59a00c3e10 2020-11-20T12:21:36.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a70c7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00c7280 con 0x7f59a00a55f0 2020-11-20T12:21:36.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a4e63700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f59a00c3e10 0x7f59a00c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a4e63700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f59a00c3e10 0x7f59a00c41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51106/0 (socket says 172.21.15.116:51106) 2020-11-20T12:21:36.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a4e63700 1 -- 172.21.15.116:0/1993655112 learned_addr learned my addr 172.21.15.116:0/1993655112 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:36.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a5664700 1 --2- 172.21.15.116:0/1993655112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a00a55f0 0x7f59a00c80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a4e63700 1 -- 172.21.15.116:0/1993655112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a00a55f0 msgr2=0x7f59a00c80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:36.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a4e63700 1 --2- 172.21.15.116:0/1993655112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a00a55f0 0x7f59a00c80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a4e63700 1 -- 172.21.15.116:0/1993655112 >> v1:172.21.15.116:6789/0 conn(0x7f59a00c93a0 legacy=0x7f59a00c79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:36.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.393+0000 7f59a4e63700 1 -- 172.21.15.116:0/1993655112 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a00c7520 con 0x7f59a00c3e10 2020-11-20T12:21:36.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f59a4e63700 1 --2- 172.21.15.116:0/1993655112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a00c3e10 0x7f59a00c41f0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f599400ba70 tx=0x7f5994006e50).ready entity=mon.0 client_cookie=a1d40d31ca0ed7fe server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:36.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f599f7fe700 1 -- 172.21.15.116:0/1993655112 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994006030 con 0x7f59a00c3e10 2020-11-20T12:21:36.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f599f7fe700 1 -- 172.21.15.116:0/1993655112 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5994006190 con 0x7f59a00c3e10 2020-11-20T12:21:36.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f599f7fe700 1 -- 172.21.15.116:0/1993655112 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994006510 con 0x7f59a00c3e10 2020-11-20T12:21:36.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f59a70c7700 1 -- 172.21.15.116:0/1993655112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a00c3e10 msgr2=0x7f59a00c41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:36.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f59a70c7700 1 --2- 172.21.15.116:0/1993655112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a00c3e10 0x7f59a00c41f0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7f599400ba70 tx=0x7f5994006e50).stop 2020-11-20T12:21:36.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f59a70c7700 1 -- 172.21.15.116:0/1993655112 shutdown_connections 2020-11-20T12:21:36.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f59a70c7700 1 --2- 172.21.15.116:0/1993655112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a00c3e10 0x7f59a00c41f0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f59a70c7700 1 --2- 172.21.15.116:0/1993655112 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a00a55f0 0x7f59a00c80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f59a70c7700 1 -- 172.21.15.116:0/1993655112 >> 172.21.15.116:0/1993655112 conn(0x7f59a0068080 msgr2=0x7f59a00c88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:36.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f59a70c7700 1 -- 172.21.15.116:0/1993655112 shutdown_connections 2020-11-20T12:21:36.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f59a70c7700 1 -- 172.21.15.116:0/1993655112 wait complete. 2020-11-20T12:21:36.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f59a70c7700 1 Processor -- start 2020-11-20T12:21:36.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.397+0000 7f59a70c7700 1 -- start start 2020-11-20T12:21:36.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a70c7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a00a55f0 0x7f59a00aaa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a70c7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a00c93a0 0x7f59a0161d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a70c7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a0162200 0x7f59a00ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a70c7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00abbf0 con 0x7f59a0162200 2020-11-20T12:21:36.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a70c7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00abe90 con 0x7f59a00c93a0 2020-11-20T12:21:36.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a70c7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59a00ac130 con 0x7f59a00a55f0 2020-11-20T12:21:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a5664700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a0162200 0x7f59a00ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a4e63700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a00a55f0 0x7f59a00aaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a5664700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a0162200 0x7f59a00ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51116/0 (socket says 172.21.15.116:51116) 2020-11-20T12:21:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a5664700 1 -- 172.21.15.116:0/4207559998 learned_addr learned my addr 172.21.15.116:0/4207559998 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:36.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f599ffff700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a00c93a0 0x7f59a0161d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a5664700 1 -- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a00a55f0 msgr2=0x7f59a00aaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:36.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a5664700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a00a55f0 0x7f59a00aaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a5664700 1 -- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a00c93a0 msgr2=0x7f59a0161d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:36.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a5664700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a00c93a0 0x7f59a0161d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a5664700 1 -- 172.21.15.116:0/4207559998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a00ac3d0 con 0x7f59a0162200 2020-11-20T12:21:36.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a4e63700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a00a55f0 0x7f59a00aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:36.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a5664700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a0162200 0x7f59a00ad710 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f599800eb70 tx=0x7f5998008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:36.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f599cff9700 1 -- 172.21.15.116:0/4207559998 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599800c700 con 0x7f59a0162200 2020-11-20T12:21:36.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f599cff9700 1 -- 172.21.15.116:0/4207559998 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f599800c860 con 0x7f59a0162200 2020-11-20T12:21:36.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a70c7700 1 -- 172.21.15.116:0/4207559998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59a00ac670 con 0x7f59a0162200 2020-11-20T12:21:36.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f599cff9700 1 -- 172.21.15.116:0/4207559998 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599800cc00 con 0x7f59a0162200 2020-11-20T12:21:36.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.401+0000 7f59a70c7700 1 -- 172.21.15.116:0/4207559998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59a00aaf80 con 0x7f59a0162200 2020-11-20T12:21:36.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.405+0000 7f59a70c7700 1 -- 172.21.15.116:0/4207559998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59a00ac830 con 0x7f59a0162200 2020-11-20T12:21:36.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.405+0000 7f599cff9700 1 -- 172.21.15.116:0/4207559998 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f599801d460 con 0x7f59a0162200 2020-11-20T12:21:36.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.405+0000 7f599cff9700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59780557c0 0x7f5978057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.405+0000 7f599cff9700 1 -- 172.21.15.116:0/4207559998 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f599806e640 con 0x7f59a0162200 2020-11-20T12:21:36.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.409+0000 7f59a4e63700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59780557c0 0x7f5978057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.409+0000 7f59a4e63700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59780557c0 0x7f5978057c80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f59a00811f0 tx=0x7f5994015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:36.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.413+0000 7f599cff9700 1 -- 172.21.15.116:0/4207559998 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5998044480 con 0x7f59a0162200 2020-11-20T12:21:36.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.573+0000 7f59a70c7700 1 -- 172.21.15.116:0/4207559998 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59a005d2a0 con 0x7f59a0162200 2020-11-20T12:21:36.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f599cff9700 1 -- 172.21.15.116:0/4207559998 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f5998047aa0 con 0x7f59a0162200 2020-11-20T12:21:36.582 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:36.583 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":107,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:12.114283+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:09.571842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:10.034013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:36.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 -- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59780557c0 msgr2=0x7f5978057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:36.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59780557c0 0x7f5978057c80 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f59a00811f0 tx=0x7f5994015000).stop 2020-11-20T12:21:36.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 -- 172.21.15.116:0/4207559998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a0162200 msgr2=0x7f59a00ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:36.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a0162200 0x7f59a00ad710 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f599800eb70 tx=0x7f5998008040).stop 2020-11-20T12:21:36.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 -- 172.21.15.116:0/4207559998 shutdown_connections 2020-11-20T12:21:36.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f59780557c0 0x7f5978057c80 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f59a0162200 0x7f59a00ad710 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f59a00a55f0 0x7f59a00aaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 --2- 172.21.15.116:0/4207559998 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f59a00c93a0 0x7f59a0161d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 -- 172.21.15.116:0/4207559998 >> 172.21.15.116:0/4207559998 conn(0x7f59a0068080 msgr2=0x7f59a014e080 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:36.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 -- 172.21.15.116:0/4207559998 shutdown_connections 2020-11-20T12:21:36.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.577+0000 7f59a70c7700 1 -- 172.21.15.116:0/4207559998 wait complete. 2020-11-20T12:21:36.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:21:09.571842+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:21:10.034013+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:21:36.609 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:21:36.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.781+0000 7f49d91bd700 1 Processor -- start 2020-11-20T12:21:36.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.785+0000 7f49d91bd700 1 -- start start 2020-11-20T12:21:36.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.785+0000 7f49d91bd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f49d40a6cc0 0x7f49d40a70a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.785+0000 7f49d91bd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f49d40c40c0 0x7f49d40c4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.785+0000 7f49d91bd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d414f500 0x7f49d414f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.785+0000 7f49d91bd700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d4061130 con 0x7f49d40c40c0 2020-11-20T12:21:36.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.785+0000 7f49d91bd700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d40c2ad0 con 0x7f49d40a6cc0 2020-11-20T12:21:36.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.785+0000 7f49d91bd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d4060210 con 0x7f49d414f500 2020-11-20T12:21:36.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f49d40c40c0 0x7f49d40c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d414f500 0x7f49d414f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f49d40c40c0 0x7f49d40c4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51122/0 (socket says 172.21.15.116:51122) 2020-11-20T12:21:36.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d259c700 1 -- 172.21.15.116:0/3223145923 learned_addr learned my addr 172.21.15.116:0/3223145923 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:36.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d2d9d700 1 --2- 172.21.15.116:0/3223145923 >> v2:172.21.15.173:3300/0 conn(0x7f49d40a6cc0 0x7f49d40a70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d359e700 1 -- 172.21.15.116:0/3223145923 >> v2:172.21.15.173:3300/0 conn(0x7f49d40a6cc0 msgr2=0x7f49d40a70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:36.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d359e700 1 --2- 172.21.15.116:0/3223145923 >> v2:172.21.15.173:3300/0 conn(0x7f49d40a6cc0 0x7f49d40a70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d359e700 1 -- 172.21.15.116:0/3223145923 >> v2:172.21.15.116:3300/0 conn(0x7f49d40c40c0 msgr2=0x7f49d40c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:36.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d359e700 1 --2- 172.21.15.116:0/3223145923 >> v2:172.21.15.116:3300/0 conn(0x7f49d40c40c0 0x7f49d40c4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d359e700 1 -- 172.21.15.116:0/3223145923 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40604b0 con 0x7f49d414f500 2020-11-20T12:21:36.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d259c700 1 --2- 172.21.15.116:0/3223145923 >> v2:172.21.15.116:3300/0 conn(0x7f49d40c40c0 0x7f49d40c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:36.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d359e700 1 --2- 172.21.15.116:0/3223145923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d414f500 0x7f49d414f990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f49d4066860 tx=0x7f49cc0091d0).ready entity=mon.2 client_cookie=b369fde199908ce8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:36.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d1d9b700 1 -- 172.21.15.116:0/3223145923 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00b9c0 con 0x7f49d414f500 2020-11-20T12:21:36.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d1d9b700 1 -- 172.21.15.116:0/3223145923 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc00bb20 con 0x7f49d414f500 2020-11-20T12:21:36.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d1d9b700 1 -- 172.21.15.116:0/3223145923 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc00bea0 con 0x7f49d414f500 2020-11-20T12:21:36.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d91bd700 1 -- 172.21.15.116:0/3223145923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d414f500 msgr2=0x7f49d414f990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:36.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d91bd700 1 --2- 172.21.15.116:0/3223145923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d414f500 0x7f49d414f990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f49d4066860 tx=0x7f49cc0091d0).stop 2020-11-20T12:21:36.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d91bd700 1 -- 172.21.15.116:0/3223145923 shutdown_connections 2020-11-20T12:21:36.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d91bd700 1 --2- 172.21.15.116:0/3223145923 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d414f500 0x7f49d414f990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d91bd700 1 --2- 172.21.15.116:0/3223145923 >> v2:172.21.15.173:3300/0 conn(0x7f49d40a6cc0 0x7f49d40a70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d91bd700 1 --2- 172.21.15.116:0/3223145923 >> v2:172.21.15.116:3300/0 conn(0x7f49d40c40c0 0x7f49d40c4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d91bd700 1 -- 172.21.15.116:0/3223145923 >> 172.21.15.116:0/3223145923 conn(0x7f49d40c1800 msgr2=0x7f49d40c1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:36.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d91bd700 1 -- 172.21.15.116:0/3223145923 shutdown_connections 2020-11-20T12:21:36.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.789+0000 7f49d91bd700 1 -- 172.21.15.116:0/3223145923 wait complete. 2020-11-20T12:21:36.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.793+0000 7f49d91bd700 1 Processor -- start 2020-11-20T12:21:36.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.793+0000 7f49d91bd700 1 -- start start 2020-11-20T12:21:36.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d91bd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d40a6cc0 0x7f49d4161980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d91bd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d40c40c0 0x7f49d415e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d91bd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d414f500 0x7f49d40a1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d91bd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d415e990 con 0x7f49d414f500 2020-11-20T12:21:36.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d91bd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40a1af0 con 0x7f49d40c40c0 2020-11-20T12:21:36.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d91bd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d40a3e00 con 0x7f49d40a6cc0 2020-11-20T12:21:36.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d414f500 0x7f49d40a1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d414f500 0x7f49d40a1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51130/0 (socket says 172.21.15.116:51130) 2020-11-20T12:21:36.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d359e700 1 -- 172.21.15.116:0/2831165603 learned_addr learned my addr 172.21.15.116:0/2831165603 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:36.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d259c700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d40c40c0 0x7f49d415e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d2d9d700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d40a6cc0 0x7f49d4161980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d359e700 1 -- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d40a6cc0 msgr2=0x7f49d4161980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:36.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d359e700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d40a6cc0 0x7f49d4161980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d359e700 1 -- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d40c40c0 msgr2=0x7f49d415e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:36.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d359e700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d40c40c0 0x7f49d415e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:36.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d359e700 1 -- 172.21.15.116:0/2831165603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40a40a0 con 0x7f49d414f500 2020-11-20T12:21:36.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d2d9d700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d40a6cc0 0x7f49d4161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:36.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d359e700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d414f500 0x7f49d40a1d60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f49cc011da0 tx=0x7f49cc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:36.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49baffd700 1 -- 172.21.15.116:0/2831165603 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc021070 con 0x7f49d414f500 2020-11-20T12:21:36.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49baffd700 1 -- 172.21.15.116:0/2831165603 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49cc006b20 con 0x7f49d414f500 2020-11-20T12:21:36.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49baffd700 1 -- 172.21.15.116:0/2831165603 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49cc01c820 con 0x7f49d414f500 2020-11-20T12:21:36.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d91bd700 1 -- 172.21.15.116:0/2831165603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d40a33e0 con 0x7f49d414f500 2020-11-20T12:21:36.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.797+0000 7f49d91bd700 1 -- 172.21.15.116:0/2831165603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d40a2250 con 0x7f49d414f500 2020-11-20T12:21:36.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.801+0000 7f49d91bd700 1 -- 172.21.15.116:0/2831165603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d40a3570 con 0x7f49d414f500 2020-11-20T12:21:36.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.801+0000 7f49baffd700 1 -- 172.21.15.116:0/2831165603 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f49cc017070 con 0x7f49d414f500 2020-11-20T12:21:36.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.801+0000 7f49baffd700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49b00557c0 0x7f49b0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:36.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.801+0000 7f49baffd700 1 -- 172.21.15.116:0/2831165603 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f49cc070560 con 0x7f49d414f500 2020-11-20T12:21:36.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.801+0000 7f49d2d9d700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49b00557c0 0x7f49b0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:36.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.801+0000 7f49d2d9d700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49b00557c0 0x7f49b0057c80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f49d40a4480 tx=0x7f49c4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:36.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.809+0000 7f49baffd700 1 -- 172.21.15.116:0/2831165603 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f49cc046400 con 0x7f49d414f500 2020-11-20T12:21:36.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.969+0000 7f49d91bd700 1 -- 172.21.15.116:0/2831165603 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f49d4069950 con 0x7f49d414f500 2020-11-20T12:21:37.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.969+0000 7f49baffd700 1 -- 172.21.15.116:0/2831165603 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v107) v1 ==== 110+0+10 (secure 0 0 0) 0x7f49cc015020 con 0x7f49d414f500 2020-11-20T12:21:37.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 -- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49b00557c0 msgr2=0x7f49b0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49b00557c0 0x7f49b0057c80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f49d40a4480 tx=0x7f49c4008040).stop 2020-11-20T12:21:37.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 -- 172.21.15.116:0/2831165603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d414f500 msgr2=0x7f49d40a1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d414f500 0x7f49d40a1d60 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f49cc011da0 tx=0x7f49cc016000).stop 2020-11-20T12:21:37.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 -- 172.21.15.116:0/2831165603 shutdown_connections 2020-11-20T12:21:37.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f49b00557c0 0x7f49b0057c80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f49d414f500 0x7f49d40a1d60 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f49d40a6cc0 0x7f49d4161980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 --2- 172.21.15.116:0/2831165603 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f49d40c40c0 0x7f49d415e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 -- 172.21.15.116:0/2831165603 >> 172.21.15.116:0/2831165603 conn(0x7f49d40c1800 msgr2=0x7f49d40c0990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:37.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 -- 172.21.15.116:0/2831165603 shutdown_connections 2020-11-20T12:21:37.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:36.973+0000 7f49d91bd700 1 -- 172.21.15.116:0/2831165603 wait complete. 2020-11-20T12:21:37.298 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:21:37.299 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:21:37.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.477+0000 7f1de69f0700 1 Processor -- start 2020-11-20T12:21:37.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.477+0000 7f1de69f0700 1 -- start start 2020-11-20T12:21:37.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.477+0000 7f1de69f0700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1de01518f0 0x7f1de00bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:37.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.477+0000 7f1de69f0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1de0151cd0 0x7f1de0066480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:37.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.477+0000 7f1de69f0700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de00bf510 con 0x7f1de00b93c0 2020-11-20T12:21:37.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.477+0000 7f1de69f0700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de00bf7b0 con 0x7f1de01518f0 2020-11-20T12:21:37.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de69f0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de00bfa50 con 0x7f1de0151cd0 2020-11-20T12:21:37.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de59ee700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1de01518f0 0x7f1de00bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:37.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de59ee700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1de01518f0 0x7f1de00bed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51134/0 (socket says 172.21.15.116:51134) 2020-11-20T12:21:37.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de59ee700 1 -- 172.21.15.116:0/256991830 learned_addr learned my addr 172.21.15.116:0/256991830 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:37.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de51ed700 1 --2- 172.21.15.116:0/256991830 >> v2:172.21.15.173:3300/0 conn(0x7f1de0151cd0 0x7f1de0066480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:37.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de59ee700 1 -- 172.21.15.116:0/256991830 >> v2:172.21.15.173:3300/0 conn(0x7f1de0151cd0 msgr2=0x7f1de0066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:37.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de59ee700 1 --2- 172.21.15.116:0/256991830 >> v2:172.21.15.173:3300/0 conn(0x7f1de0151cd0 0x7f1de0066480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de59ee700 1 -- 172.21.15.116:0/256991830 >> v1:172.21.15.173:6789/0 conn(0x7f1de00b93c0 legacy=0x7f1de0060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de59ee700 1 -- 172.21.15.116:0/256991830 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de00bfcf0 con 0x7f1de01518f0 2020-11-20T12:21:37.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de59ee700 1 --2- 172.21.15.116:0/256991830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1de01518f0 0x7f1de00bed20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f1de00624e0 tx=0x7f1ddc006eb0).ready entity=mon.0 client_cookie=a59cebc92c06ba2b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:37.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de49ec700 1 -- 172.21.15.116:0/256991830 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc009810 con 0x7f1de01518f0 2020-11-20T12:21:37.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de49ec700 1 -- 172.21.15.116:0/256991830 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ddc009970 con 0x7f1de01518f0 2020-11-20T12:21:37.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de49ec700 1 -- 172.21.15.116:0/256991830 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc009cf0 con 0x7f1de01518f0 2020-11-20T12:21:37.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de69f0700 1 -- 172.21.15.116:0/256991830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1de01518f0 msgr2=0x7f1de00bed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de69f0700 1 --2- 172.21.15.116:0/256991830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1de01518f0 0x7f1de00bed20 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f1de00624e0 tx=0x7f1ddc006eb0).stop 2020-11-20T12:21:37.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de69f0700 1 -- 172.21.15.116:0/256991830 shutdown_connections 2020-11-20T12:21:37.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de69f0700 1 --2- 172.21.15.116:0/256991830 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1de01518f0 0x7f1de00bed20 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de69f0700 1 --2- 172.21.15.116:0/256991830 >> v2:172.21.15.173:3300/0 conn(0x7f1de0151cd0 0x7f1de0066480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.481+0000 7f1de69f0700 1 -- 172.21.15.116:0/256991830 >> 172.21.15.116:0/256991830 conn(0x7f1de0068350 msgr2=0x7f1de00627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:37.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.485+0000 7f1de69f0700 1 -- 172.21.15.116:0/256991830 shutdown_connections 2020-11-20T12:21:37.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.485+0000 7f1de69f0700 1 -- 172.21.15.116:0/256991830 wait complete. 2020-11-20T12:21:37.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.485+0000 7f1de69f0700 1 Processor -- start 2020-11-20T12:21:37.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de69f0700 1 -- start start 2020-11-20T12:21:37.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de69f0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1de00b93c0 0x7f1de014c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:37.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de69f0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1de01518f0 0x7f1de0169630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:37.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de69f0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1de0151cd0 0x7f1de014f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:37.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de69f0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0165990 con 0x7f1de00b93c0 2020-11-20T12:21:37.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de69f0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0165c30 con 0x7f1de0151cd0 2020-11-20T12:21:37.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de69f0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de0165ed0 con 0x7f1de01518f0 2020-11-20T12:21:37.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de59ee700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1de00b93c0 0x7f1de014c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:37.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de59ee700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1de00b93c0 0x7f1de014c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51144/0 (socket says 172.21.15.116:51144) 2020-11-20T12:21:37.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de59ee700 1 -- 172.21.15.116:0/1997517781 learned_addr learned my addr 172.21.15.116:0/1997517781 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:37.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de61ef700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1de0151cd0 0x7f1de014f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:37.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de51ed700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1de01518f0 0x7f1de0169630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:37.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de59ee700 1 -- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1de01518f0 msgr2=0x7f1de0169630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de59ee700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1de01518f0 0x7f1de0169630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de59ee700 1 -- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1de0151cd0 msgr2=0x7f1de014f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de59ee700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1de0151cd0 0x7f1de014f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de59ee700 1 -- 172.21.15.116:0/1997517781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de0166170 con 0x7f1de00b93c0 2020-11-20T12:21:37.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de59ee700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1de00b93c0 0x7f1de014c330 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f1ddc00beb0 tx=0x7f1ddc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:37.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1dcdffb700 1 -- 172.21.15.116:0/1997517781 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc011dc0 con 0x7f1de00b93c0 2020-11-20T12:21:37.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1dcdffb700 1 -- 172.21.15.116:0/1997517781 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ddc01e460 con 0x7f1de00b93c0 2020-11-20T12:21:37.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1de69f0700 1 -- 172.21.15.116:0/1997517781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de0166410 con 0x7f1de00b93c0 2020-11-20T12:21:37.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.489+0000 7f1dcdffb700 1 -- 172.21.15.116:0/1997517781 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ddc01e820 con 0x7f1de00b93c0 2020-11-20T12:21:37.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.493+0000 7f1de69f0700 1 -- 172.21.15.116:0/1997517781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de0169b20 con 0x7f1de00b93c0 2020-11-20T12:21:37.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.493+0000 7f1de69f0700 1 -- 172.21.15.116:0/1997517781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de01665a0 con 0x7f1de00b93c0 2020-11-20T12:21:37.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.493+0000 7f1dcdffb700 1 -- 172.21.15.116:0/1997517781 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1ddc025030 con 0x7f1de00b93c0 2020-11-20T12:21:37.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.493+0000 7f1dcdffb700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1db80557c0 0x7f1db8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:37.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.493+0000 7f1dcdffb700 1 -- 172.21.15.116:0/1997517781 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1ddc06fbf0 con 0x7f1de00b93c0 2020-11-20T12:21:37.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.497+0000 7f1de51ed700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1db80557c0 0x7f1db8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:37.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.497+0000 7f1de51ed700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1db80557c0 0x7f1db8057c80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f1de00ba2a0 tx=0x7f1dd0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:37.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.501+0000 7f1dcdffb700 1 -- 172.21.15.116:0/1997517781 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1ddc045b40 con 0x7f1de00b93c0 2020-11-20T12:21:37.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.661+0000 7f1de69f0700 1 -- 172.21.15.116:0/1997517781 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f1de005cf30 con 0x7f1de00b93c0 2020-11-20T12:21:37.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.661+0000 7f1dcdffb700 1 -- 172.21.15.116:0/1997517781 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v107) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1ddc023030 con 0x7f1de00b93c0 2020-11-20T12:21:37.669 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:37.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.665+0000 7f1de69f0700 1 -- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1db80557c0 msgr2=0x7f1db8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.665+0000 7f1de69f0700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1db80557c0 0x7f1db8057c80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f1de00ba2a0 tx=0x7f1dd0008040).stop 2020-11-20T12:21:37.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.665+0000 7f1de69f0700 1 -- 172.21.15.116:0/1997517781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1de00b93c0 msgr2=0x7f1de014c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.665+0000 7f1de69f0700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1de00b93c0 0x7f1de014c330 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f1ddc00beb0 tx=0x7f1ddc015000).stop 2020-11-20T12:21:37.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.665+0000 7f1de69f0700 1 -- 172.21.15.116:0/1997517781 shutdown_connections 2020-11-20T12:21:37.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.665+0000 7f1de69f0700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1db80557c0 0x7f1db8057c80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.665+0000 7f1de69f0700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1de0151cd0 0x7f1de014f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.665+0000 7f1de69f0700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1de00b93c0 0x7f1de014c330 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.665+0000 7f1de69f0700 1 --2- 172.21.15.116:0/1997517781 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1de01518f0 0x7f1de0169630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.665+0000 7f1de69f0700 1 -- 172.21.15.116:0/1997517781 >> 172.21.15.116:0/1997517781 conn(0x7f1de0068350 msgr2=0x7f1de014cb50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.669+0000 7f1de69f0700 1 -- 172.21.15.116:0/1997517781 shutdown_connections 2020-11-20T12:21:37.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.669+0000 7f1de69f0700 1 -- 172.21.15.116:0/1997517781 wait complete. 2020-11-20T12:21:37.696 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:21:37.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.865+0000 7f131561e700 1 Processor -- start 2020-11-20T12:21:37.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.865+0000 7f131561e700 1 -- start start 2020-11-20T12:21:37.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f131561e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1310151630 0x7f1310062170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:37.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f131561e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13100abbf0 con 0x7f13100ae250 2020-11-20T12:21:37.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f131561e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13100aac10 con 0x7f1310151a10 2020-11-20T12:21:37.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f131561e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13100aaeb0 con 0x7f1310151630 2020-11-20T12:21:37.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f130e7fc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f13100ae250 0x7f1310152640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60584/0 (socket says 172.21.15.116:60584) 2020-11-20T12:21:37.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f130e7fc700 1 -- 172.21.15.116:0/3316951979 learned_addr learned my addr 172.21.15.116:0/3316951979 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:37.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f130effd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1310151630 0x7f1310062170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:37.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f130dffb700 1 -- 172.21.15.116:0/3316951979 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3814174370 0 0) 0x7f1304004850 con 0x7f13100ae250 2020-11-20T12:21:37.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f130effd700 1 -- 172.21.15.116:0/3316951979 >> v1:172.21.15.173:6789/0 conn(0x7f1310151a10 legacy=0x7f1310061490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f130effd700 1 -- 172.21.15.116:0/3316951979 >> v1:172.21.15.116:6789/0 conn(0x7f13100ae250 legacy=0x7f1310152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f130effd700 1 -- 172.21.15.116:0/3316951979 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13100ab150 con 0x7f1310151630 2020-11-20T12:21:37.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f130effd700 1 --2- 172.21.15.116:0/3316951979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151630 0x7f1310062170 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f130000b9a0 tx=0x7f1300006df0).ready entity=mon.0 client_cookie=9a2f79c5dd0beeef server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:37.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f130dffb700 1 -- 172.21.15.116:0/3316951979 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13000097b0 con 0x7f1310151630 2020-11-20T12:21:37.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f130dffb700 1 -- 172.21.15.116:0/3316951979 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1300009910 con 0x7f1310151630 2020-11-20T12:21:37.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f130dffb700 1 -- 172.21.15.116:0/3316951979 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300009c90 con 0x7f1310151630 2020-11-20T12:21:37.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f131561e700 1 -- 172.21.15.116:0/3316951979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151630 msgr2=0x7f1310062170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f131561e700 1 --2- 172.21.15.116:0/3316951979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151630 0x7f1310062170 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f130000b9a0 tx=0x7f1300006df0).stop 2020-11-20T12:21:37.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f131561e700 1 -- 172.21.15.116:0/3316951979 shutdown_connections 2020-11-20T12:21:37.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f131561e700 1 --2- 172.21.15.116:0/3316951979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151630 0x7f1310062170 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f131561e700 1 -- 172.21.15.116:0/3316951979 >> 172.21.15.116:0/3316951979 conn(0x7f1310068370 msgr2=0x7f13100c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:37.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.869+0000 7f131561e700 1 -- 172.21.15.116:0/3316951979 shutdown_connections 2020-11-20T12:21:37.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.873+0000 7f131561e700 1 -- 172.21.15.116:0/3316951979 wait complete. 2020-11-20T12:21:37.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.873+0000 7f131561e700 1 Processor -- start 2020-11-20T12:21:37.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.873+0000 7f131561e700 1 -- start start 2020-11-20T12:21:37.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.873+0000 7f131561e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13100ae250 0x7f13100bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:37.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.873+0000 7f131561e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151630 0x7f13100c0a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:37.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.873+0000 7f131561e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1310151a10 0x7f1310154420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:37.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.873+0000 7f131561e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13100c1c20 con 0x7f1310151630 2020-11-20T12:21:37.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.873+0000 7f131561e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13100c1440 con 0x7f1310151a10 2020-11-20T12:21:37.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.873+0000 7f131561e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13100c16e0 con 0x7f13100ae250 2020-11-20T12:21:37.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130effd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13100ae250 0x7f13100bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:37.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130e7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151630 0x7f13100c0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:37.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130effd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13100ae250 0x7f13100bf0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:60104/0 (socket says 172.21.15.116:60104) 2020-11-20T12:21:37.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130effd700 1 -- 172.21.15.116:0/3143203644 learned_addr learned my addr 172.21.15.116:0/3143203644 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:37.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130f7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1310151a10 0x7f1310154420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:37.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130effd700 1 -- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1310151a10 msgr2=0x7f1310154420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130effd700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1310151a10 0x7f1310154420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130effd700 1 -- 172.21.15.116:0/3143203644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151630 msgr2=0x7f13100c0a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:37.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130effd700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151630 0x7f13100c0a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:37.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130effd700 1 -- 172.21.15.116:0/3143203644 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13100c1980 con 0x7f13100ae250 2020-11-20T12:21:37.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130e7fc700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151630 0x7f13100c0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:37.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f130effd700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13100ae250 0x7f13100bf0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f1300006fd0 tx=0x7f1300015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:37.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f12f6ffd700 1 -- 172.21.15.116:0/3143203644 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1300011e50 con 0x7f13100ae250 2020-11-20T12:21:37.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f131561e700 1 -- 172.21.15.116:0/3143203644 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13100bd6c0 con 0x7f13100ae250 2020-11-20T12:21:37.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f12f6ffd700 1 -- 172.21.15.116:0/3143203644 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1300006760 con 0x7f13100ae250 2020-11-20T12:21:37.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f12f6ffd700 1 -- 172.21.15.116:0/3143203644 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f130001e670 con 0x7f13100ae250 2020-11-20T12:21:37.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f131561e700 1 -- 172.21.15.116:0/3143203644 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13100bf730 con 0x7f13100ae250 2020-11-20T12:21:37.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.877+0000 7f131561e700 1 -- 172.21.15.116:0/3143203644 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131005cf30 con 0x7f13100ae250 2020-11-20T12:21:37.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.881+0000 7f12f6ffd700 1 -- 172.21.15.116:0/3143203644 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1300025030 con 0x7f13100ae250 2020-11-20T12:21:37.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.881+0000 7f12f6ffd700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12ec0557c0 0x7f12ec057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:37.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.881+0000 7f12f6ffd700 1 -- 172.21.15.116:0/3143203644 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f130006fe90 con 0x7f13100ae250 2020-11-20T12:21:37.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.881+0000 7f130e7fc700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12ec0557c0 0x7f12ec057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:37.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.885+0000 7f130e7fc700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12ec0557c0 0x7f12ec057c80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f1310154fa0 tx=0x7f1304007800).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:37.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:37.889+0000 7f12f6ffd700 1 -- 172.21.15.116:0/3143203644 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1300045cd0 con 0x7f13100ae250 2020-11-20T12:21:38.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.049+0000 7f131561e700 1 -- 172.21.15.116:0/3143203644 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f13100607e0 con 0x7f13100ae250 2020-11-20T12:21:38.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.049+0000 7f12f6ffd700 1 -- 172.21.15.116:0/3143203644 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v107) v1 ==== 104+0+10 (secure 0 0 0) 0x7f1300023070 con 0x7f13100ae250 2020-11-20T12:21:38.056 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:38.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.049+0000 7f131561e700 1 -- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12ec0557c0 msgr2=0x7f12ec057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.049+0000 7f131561e700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12ec0557c0 0x7f12ec057c80 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f1310154fa0 tx=0x7f1304007800).stop 2020-11-20T12:21:38.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.053+0000 7f131561e700 1 -- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13100ae250 msgr2=0x7f13100bf0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.053+0000 7f131561e700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13100ae250 0x7f13100bf0a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7f1300006fd0 tx=0x7f1300015000).stop 2020-11-20T12:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.053+0000 7f131561e700 1 -- 172.21.15.116:0/3143203644 shutdown_connections 2020-11-20T12:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.053+0000 7f131561e700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12ec0557c0 0x7f12ec057c80 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.053+0000 7f131561e700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1310151a10 0x7f1310154420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.053+0000 7f131561e700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f13100ae250 0x7f13100bf0a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.053+0000 7f131561e700 1 --2- 172.21.15.116:0/3143203644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1310151630 0x7f13100c0a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.053+0000 7f131561e700 1 -- 172.21.15.116:0/3143203644 >> 172.21.15.116:0/3143203644 conn(0x7f1310068370 msgr2=0x7f131014ba80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:38.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.053+0000 7f131561e700 1 -- 172.21.15.116:0/3143203644 shutdown_connections 2020-11-20T12:21:38.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.053+0000 7f131561e700 1 -- 172.21.15.116:0/3143203644 wait complete. 2020-11-20T12:21:38.079 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:21:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.265+0000 7f7c3759e700 1 Processor -- start 2020-11-20T12:21:38.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.265+0000 7f7c3759e700 1 -- start start 2020-11-20T12:21:38.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.269+0000 7f7c3759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c38063100 0x7f7c380bff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.269+0000 7f7c3759e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c380c0400 con 0x7f7c380a66e0 2020-11-20T12:21:38.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.269+0000 7f7c3759e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c380c0690 con 0x7f7c3814e590 2020-11-20T12:21:38.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.269+0000 7f7c3759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380c0930 con 0x7f7c38063100 2020-11-20T12:21:38.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.269+0000 7f7c36d9d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f7c380a66e0 0x7f7c38066480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60600/0 (socket says 172.21.15.116:60600) 2020-11-20T12:21:38.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.269+0000 7f7c36d9d700 1 -- 172.21.15.116:0/2636166721 learned_addr learned my addr 172.21.15.116:0/2636166721 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:38.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.269+0000 7f7c35d9b700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c38063100 0x7f7c380bff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.269+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700272757 0 0) 0x7f7c380c0400 con 0x7f7c380a66e0 2020-11-20T12:21:38.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.269+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c1c003240 con 0x7f7c380a66e0 2020-11-20T12:21:38.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.269+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3774881886 0 0) 0x7f7c380c0690 con 0x7f7c3814e590 2020-11-20T12:21:38.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c380c0400 con 0x7f7c3814e590 2020-11-20T12:21:38.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2406627711 0 0) 0x7f7c1c003240 con 0x7f7c380a66e0 2020-11-20T12:21:38.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c38063100 msgr2=0x7f7c380bff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:38.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c38063100 0x7f7c380bff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 >> v1:172.21.15.173:6789/0 conn(0x7f7c3814e590 legacy=0x7f7c3814f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380c0bd0 con 0x7f7c380a66e0 2020-11-20T12:21:38.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f7c30001860 con 0x7f7c380a66e0 2020-11-20T12:21:38.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 >> v1:172.21.15.116:6789/0 conn(0x7f7c380a66e0 legacy=0x7f7c38066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c1c004950 0x7f7c1c004d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c38063100 0x7f7c1c003480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c1c003970 0x7f7c1c003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380c0930 con 0x7f7c1c004950 2020-11-20T12:21:38.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c00d940 con 0x7f7c1c003970 2020-11-20T12:21:38.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c1c00da80 con 0x7f7c38063100 2020-11-20T12:21:38.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3659c700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c1c004950 0x7f7c1c004d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c35d9b700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c38063100 0x7f7c1c003480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c36d9d700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c1c003970 0x7f7c1c003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3659c700 1 -- 172.21.15.116:0/2636166721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c38063100 msgr2=0x7f7c1c003480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:38.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3659c700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c38063100 0x7f7c1c003480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3659c700 1 -- 172.21.15.116:0/2636166721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c1c003970 msgr2=0x7f7c1c003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:38.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3659c700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c1c003970 0x7f7c1c003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3659c700 1 -- 172.21.15.116:0/2636166721 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c003240 con 0x7f7c1c004950 2020-11-20T12:21:38.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3659c700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c1c004950 0x7f7c1c004d50 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7f7c28001a80 tx=0x7f7c28007b90).ready entity=mon.0 client_cookie=e982ad3dde708f95 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:38.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.273+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28007020 con 0x7f7c1c004950 2020-11-20T12:21:38.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.277+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c28007180 con 0x7f7c1c004950 2020-11-20T12:21:38.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.277+0000 7f7c3559a700 1 -- 172.21.15.116:0/2636166721 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2800c7d0 con 0x7f7c1c004950 2020-11-20T12:21:38.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.277+0000 7f7c3759e700 1 -- 172.21.15.116:0/2636166721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c1c004950 msgr2=0x7f7c1c004d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.277+0000 7f7c3759e700 1 --2- 172.21.15.116:0/2636166721 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c1c004950 0x7f7c1c004d50 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7f7c28001a80 tx=0x7f7c28007b90).stop 2020-11-20T12:21:38.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.277+0000 7f7c36d9d700 1 -- 172.21.15.116:0/2636166721 reap_dead start 2020-11-20T12:21:38.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.277+0000 7f7c3759e700 1 -- 172.21.15.116:0/2636166721 shutdown_connections 2020-11-20T12:21:38.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.277+0000 7f7c3759e700 1 -- 172.21.15.116:0/2636166721 >> 172.21.15.116:0/2636166721 conn(0x7f7c38068350 msgr2=0x7f7c380627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:38.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.277+0000 7f7c3759e700 1 -- 172.21.15.116:0/2636166721 shutdown_connections 2020-11-20T12:21:38.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.277+0000 7f7c3759e700 1 -- 172.21.15.116:0/2636166721 wait complete. 2020-11-20T12:21:38.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.277+0000 7f7c3759e700 1 Processor -- start 2020-11-20T12:21:38.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c3759e700 1 -- start start 2020-11-20T12:21:38.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c3759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c1c004950 0x7f7c3814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c3759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c380bc1c0 0x7f7c38137df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c3759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c380c30a0 0x7f7c380c3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c3759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380c17c0 con 0x7f7c380bc1c0 2020-11-20T12:21:38.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c3759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380bc640 con 0x7f7c380c30a0 2020-11-20T12:21:38.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c3759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380c1150 con 0x7f7c1c004950 2020-11-20T12:21:38.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c35d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c380bc1c0 0x7f7c38137df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c35d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c380bc1c0 0x7f7c38137df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51176/0 (socket says 172.21.15.116:51176) 2020-11-20T12:21:38.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c35d9b700 1 -- 172.21.15.116:0/2774102230 learned_addr learned my addr 172.21.15.116:0/2774102230 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:38.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c3659c700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c1c004950 0x7f7c3814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c36d9d700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c380c30a0 0x7f7c380c3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c35d9b700 1 -- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c1c004950 msgr2=0x7f7c3814fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c35d9b700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c1c004950 0x7f7c3814fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c35d9b700 1 -- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c380c30a0 msgr2=0x7f7c380c3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c35d9b700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c380c30a0 0x7f7c380c3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c35d9b700 1 -- 172.21.15.116:0/2774102230 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c380c19f0 con 0x7f7c380bc1c0 2020-11-20T12:21:38.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.281+0000 7f7c35d9b700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c380bc1c0 0x7f7c38137df0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f7c2c011cc0 tx=0x7f7c2c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:38.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.285+0000 7f7c267fc700 1 -- 172.21.15.116:0/2774102230 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c011e70 con 0x7f7c380bc1c0 2020-11-20T12:21:38.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.285+0000 7f7c267fc700 1 -- 172.21.15.116:0/2774102230 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c2c00fd30 con 0x7f7c380bc1c0 2020-11-20T12:21:38.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.285+0000 7f7c3759e700 1 -- 172.21.15.116:0/2774102230 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c380bcb70 con 0x7f7c380bc1c0 2020-11-20T12:21:38.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.285+0000 7f7c267fc700 1 -- 172.21.15.116:0/2774102230 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c2c014650 con 0x7f7c380bc1c0 2020-11-20T12:21:38.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.285+0000 7f7c3759e700 1 -- 172.21.15.116:0/2774102230 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c380c1c90 con 0x7f7c380bc1c0 2020-11-20T12:21:38.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.285+0000 7f7c3759e700 1 -- 172.21.15.116:0/2774102230 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3805cf30 con 0x7f7c380bc1c0 2020-11-20T12:21:38.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.289+0000 7f7c267fc700 1 -- 172.21.15.116:0/2774102230 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7c2c014a40 con 0x7f7c380bc1c0 2020-11-20T12:21:38.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.289+0000 7f7c267fc700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c0c0557c0 0x7f7c0c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.289+0000 7f7c267fc700 1 -- 172.21.15.116:0/2774102230 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f7c2c06f250 con 0x7f7c380bc1c0 2020-11-20T12:21:38.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.289+0000 7f7c3659c700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c0c0557c0 0x7f7c0c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.289+0000 7f7c3659c700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c0c0557c0 0x7f7c0c057c80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f7c28008fd0 tx=0x7f7c28016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:38.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.293+0000 7f7c267fc700 1 -- 172.21.15.116:0/2774102230 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7c2c0450f0 con 0x7f7c380bc1c0 2020-11-20T12:21:38.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.453+0000 7f7c3759e700 1 -- 172.21.15.116:0/2774102230 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7c38066220 con 0x7f7c380bc1c0 2020-11-20T12:21:38.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.453+0000 7f7c267fc700 1 -- 172.21.15.116:0/2774102230 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v107) v1 ==== 100+0+10 (secure 0 0 0) 0x7f7c2c018020 con 0x7f7c380bc1c0 2020-11-20T12:21:38.459 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:38.461 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 -- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c0c0557c0 msgr2=0x7f7c0c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c0c0557c0 0x7f7c0c057c80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f7c28008fd0 tx=0x7f7c28016000).stop 2020-11-20T12:21:38.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 -- 172.21.15.116:0/2774102230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c380bc1c0 msgr2=0x7f7c38137df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.462 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c380bc1c0 0x7f7c38137df0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7f7c2c011cc0 tx=0x7f7c2c006df0).stop 2020-11-20T12:21:38.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 -- 172.21.15.116:0/2774102230 shutdown_connections 2020-11-20T12:21:38.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7c0c0557c0 0x7f7c0c057c80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7c380c30a0 0x7f7c380c3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7c1c004950 0x7f7c3814fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 --2- 172.21.15.116:0/2774102230 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7c380bc1c0 0x7f7c38137df0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 -- 172.21.15.116:0/2774102230 >> 172.21.15.116:0/2774102230 conn(0x7f7c38068350 msgr2=0x7f7c38060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:38.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 -- 172.21.15.116:0/2774102230 shutdown_connections 2020-11-20T12:21:38.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.457+0000 7f7c3759e700 1 -- 172.21.15.116:0/2774102230 wait complete. 2020-11-20T12:21:38.493 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.653+0000 7f46e8b57700 1 Processor -- start 2020-11-20T12:21:38.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.653+0000 7f46e8b57700 1 -- start start 2020-11-20T12:21:38.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e8b57700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f46e413afa0 0x7f46e40aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e8b57700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f46e40ad440 0x7f46e40aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e8b57700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46e40ada00 con 0x7f46e413b380 2020-11-20T12:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e8b57700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e40adca0 con 0x7f46e40ad440 2020-11-20T12:21:38.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e8b57700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46e40adf40 con 0x7f46e413afa0 2020-11-20T12:21:38.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e3fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f46e40ad440 0x7f46e40aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46daffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f46e413b380 0x7f46e40acd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60620/0 (socket says 172.21.15.116:60620) 2020-11-20T12:21:38.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46daffd700 1 -- 172.21.15.116:0/3664685180 learned_addr learned my addr 172.21.15.116:0/3664685180 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:38.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e37fe700 1 --2- 172.21.15.116:0/3664685180 >> v2:172.21.15.173:3300/0 conn(0x7f46e413afa0 0x7f46e40aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e3fff700 1 -- 172.21.15.116:0/3664685180 >> v2:172.21.15.173:3300/0 conn(0x7f46e413afa0 msgr2=0x7f46e40aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:38.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e2ffd700 1 -- 172.21.15.116:0/3664685180 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921439591 0 0) 0x7f46e40ada00 con 0x7f46e413b380 2020-11-20T12:21:38.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e3fff700 1 --2- 172.21.15.116:0/3664685180 >> v2:172.21.15.173:3300/0 conn(0x7f46e413afa0 0x7f46e40aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e3fff700 1 -- 172.21.15.116:0/3664685180 >> v1:172.21.15.116:6789/0 conn(0x7f46e413b380 legacy=0x7f46e40acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e3fff700 1 -- 172.21.15.116:0/3664685180 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e40ae1e0 con 0x7f46e40ad440 2020-11-20T12:21:38.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e3fff700 1 --2- 172.21.15.116:0/3664685180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46e40ad440 0x7f46e40aaaa0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f46e40818e0 tx=0x7f46dc0091d0).ready entity=mon.0 client_cookie=af6db8235af425e6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:38.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e2ffd700 1 -- 172.21.15.116:0/3664685180 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc00bbc0 con 0x7f46e40ad440 2020-11-20T12:21:38.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e2ffd700 1 -- 172.21.15.116:0/3664685180 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46dc00bd20 con 0x7f46e40ad440 2020-11-20T12:21:38.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.657+0000 7f46e2ffd700 1 -- 172.21.15.116:0/3664685180 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46dc011610 con 0x7f46e40ad440 2020-11-20T12:21:38.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.661+0000 7f46e8b57700 1 -- 172.21.15.116:0/3664685180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46e40ad440 msgr2=0x7f46e40aaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.661+0000 7f46e8b57700 1 --2- 172.21.15.116:0/3664685180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46e40ad440 0x7f46e40aaaa0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f46e40818e0 tx=0x7f46dc0091d0).stop 2020-11-20T12:21:38.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.661+0000 7f46e8b57700 1 -- 172.21.15.116:0/3664685180 shutdown_connections 2020-11-20T12:21:38.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.661+0000 7f46e8b57700 1 --2- 172.21.15.116:0/3664685180 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46e40ad440 0x7f46e40aaaa0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.661+0000 7f46e8b57700 1 --2- 172.21.15.116:0/3664685180 >> v2:172.21.15.173:3300/0 conn(0x7f46e413afa0 0x7f46e40aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.661+0000 7f46e8b57700 1 -- 172.21.15.116:0/3664685180 >> 172.21.15.116:0/3664685180 conn(0x7f46e40678a0 msgr2=0x7f46e4067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:38.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.661+0000 7f46e8b57700 1 -- 172.21.15.116:0/3664685180 shutdown_connections 2020-11-20T12:21:38.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.661+0000 7f46e8b57700 1 -- 172.21.15.116:0/3664685180 wait complete. 2020-11-20T12:21:38.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.661+0000 7f46e8b57700 1 Processor -- start 2020-11-20T12:21:38.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.661+0000 7f46e8b57700 1 -- start start 2020-11-20T12:21:38.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e8b57700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46e40ad440 0x7f46e413a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e8b57700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f46e413afa0 0x7f46e4161660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e8b57700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46e413b380 0x7f46e4164370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e8b57700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e4164dd0 con 0x7f46e40ad440 2020-11-20T12:21:38.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e8b57700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46e4165070 con 0x7f46e413b380 2020-11-20T12:21:38.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e8b57700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46e4160f20 con 0x7f46e413afa0 2020-11-20T12:21:38.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e37fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46e40ad440 0x7f46e413a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e37fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46e40ad440 0x7f46e413a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51188/0 (socket says 172.21.15.116:51188) 2020-11-20T12:21:38.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e37fe700 1 -- 172.21.15.116:0/332520776 learned_addr learned my addr 172.21.15.116:0/332520776 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:38.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e3fff700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46e413b380 0x7f46e4164370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e37fe700 1 -- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f46e413afa0 msgr2=0x7f46e4161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e37fe700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f46e413afa0 0x7f46e4161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:38.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e37fe700 1 -- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46e413b380 msgr2=0x7f46e4164370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e37fe700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46e413b380 0x7f46e4164370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e37fe700 1 -- 172.21.15.116:0/332520776 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46e41611c0 con 0x7f46e40ad440 2020-11-20T12:21:38.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e37fe700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46e40ad440 0x7f46e413a020 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7f46d400db60 tx=0x7f46d4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:38.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46dbfff700 1 -- 172.21.15.116:0/332520776 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4009790 con 0x7f46e40ad440 2020-11-20T12:21:38.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46dbfff700 1 -- 172.21.15.116:0/332520776 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46d40098f0 con 0x7f46e40ad440 2020-11-20T12:21:38.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e8b57700 1 -- 172.21.15.116:0/332520776 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46e40a1590 con 0x7f46e40ad440 2020-11-20T12:21:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46dbfff700 1 -- 172.21.15.116:0/332520776 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46d4009c70 con 0x7f46e40ad440 2020-11-20T12:21:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e8b57700 1 -- 172.21.15.116:0/332520776 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46e413a6b0 con 0x7f46e40ad440 2020-11-20T12:21:38.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46e8b57700 1 -- 172.21.15.116:0/332520776 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46e4160d70 con 0x7f46e40ad440 2020-11-20T12:21:38.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.665+0000 7f46dbfff700 1 -- 172.21.15.116:0/332520776 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f46d4013480 con 0x7f46e40ad440 2020-11-20T12:21:38.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.669+0000 7f46dbfff700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46c00557c0 0x7f46c0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:38.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.669+0000 7f46dbfff700 1 -- 172.21.15.116:0/332520776 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f46d406ed50 con 0x7f46e40ad440 2020-11-20T12:21:38.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.669+0000 7f46daffd700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46c00557c0 0x7f46c0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:38.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.669+0000 7f46daffd700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46c00557c0 0x7f46c0057c80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f46e40a0350 tx=0x7f46cc0078a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:38.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.673+0000 7f46dbfff700 1 -- 172.21.15.116:0/332520776 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f46d4044b90 con 0x7f46e40ad440 2020-11-20T12:21:38.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.881+0000 7f46e8b57700 1 -- 172.21.15.116:0/332520776 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46e4161350 con 0x7f46e40ad440 2020-11-20T12:21:38.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.881+0000 7f46dbfff700 1 -- 172.21.15.116:0/332520776 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+2851 (secure 0 0 0) 0x7f46d40481b0 con 0x7f46e40ad440 2020-11-20T12:21:38.888 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:38.889 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":60,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6581,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/1638614527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6835","nonce":1638614527},{"type":"v1","addr":"172.21.15.173:6837","nonce":1638614527}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7612,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6836/3682335131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3682335131},{"type":"v1","addr":"172.21.15.173:6836","nonce":3682335131}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57},{"gid":7725,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/3975405805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":3975405805},{"type":"v1","addr":"172.21.15.116:6834","nonce":3975405805}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":57}],"filesystems":[{"mdsmap":{"epoch":60,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:11.109567+0000","modified":"2020-11-20T12:21:13.118616+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7728},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7728":{"gid":7728,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/267899152","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":267899152},{"type":"v1","addr":"172.21.15.116:6835","nonce":267899152}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2020-11-20T12:21:38.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 -- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46c00557c0 msgr2=0x7f46c0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46c00557c0 0x7f46c0057c80 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f46e40a0350 tx=0x7f46cc0078a0).stop 2020-11-20T12:21:38.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 -- 172.21.15.116:0/332520776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46e40ad440 msgr2=0x7f46e413a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:38.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46e40ad440 0x7f46e413a020 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7f46d400db60 tx=0x7f46d4008040).stop 2020-11-20T12:21:38.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 -- 172.21.15.116:0/332520776 shutdown_connections 2020-11-20T12:21:38.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f46c00557c0 0x7f46c0057c80 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f46e413b380 0x7f46e4164370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f46e40ad440 0x7f46e413a020 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:38.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 --2- 172.21.15.116:0/332520776 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f46e413afa0 0x7f46e4161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:21:38.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 -- 172.21.15.116:0/332520776 >> 172.21.15.116:0/332520776 conn(0x7f46e40678a0 msgr2=0x7f46e4155920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:38.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 -- 172.21.15.116:0/332520776 shutdown_connections 2020-11-20T12:21:38.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:38.885+0000 7f46e8b57700 1 -- 172.21.15.116:0/332520776 wait complete. 2020-11-20T12:21:38.894 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 60 2020-11-20T12:21:38.915 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:39.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.109+0000 7f07ac1ae700 1 Processor -- start 2020-11-20T12:21:39.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.109+0000 7f07ac1ae700 1 -- start start 2020-11-20T12:21:39.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07ac1ae700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f07a4150680 0x7f07a4066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:39.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07ac1ae700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07a4156f20 0x7f07a40bf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:39.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07ac1ae700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07a41573d0 con 0x7f07a40bf560 2020-11-20T12:21:39.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07ac1ae700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07a40bf9d0 con 0x7f07a4150680 2020-11-20T12:21:39.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07ac1ae700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a4150a60 con 0x7f07a4156f20 2020-11-20T12:21:39.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07a9f4a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f07a4150680 0x7f07a4066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:39.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07aa74b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f07a40bf560 0x7f07a4159aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60634/0 (socket says 172.21.15.116:60634) 2020-11-20T12:21:39.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07aa74b700 1 -- 172.21.15.116:0/3307905273 learned_addr learned my addr 172.21.15.116:0/3307905273 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:39.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07a9749700 1 --2- 172.21.15.116:0/3307905273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07a4156f20 0x7f07a40bf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:39.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07a9f4a700 1 -- 172.21.15.116:0/3307905273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07a4156f20 msgr2=0x7f07a40bf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:39.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07a9f4a700 1 --2- 172.21.15.116:0/3307905273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07a4156f20 0x7f07a40bf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07a9f4a700 1 -- 172.21.15.116:0/3307905273 >> v1:172.21.15.116:6789/0 conn(0x7f07a40bf560 legacy=0x7f07a4159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:39.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07a9f4a700 1 -- 172.21.15.116:0/3307905273 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a4066af0 con 0x7f07a4150680 2020-11-20T12:21:39.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.113+0000 7f07a9f4a700 1 --2- 172.21.15.116:0/3307905273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07a4150680 0x7f07a4066450 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f07a000b9a0 tx=0x7f07a0006df0).ready entity=mon.0 client_cookie=1925efe12eb81722 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:39.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07a8f48700 1 -- 172.21.15.116:0/3307905273 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a00097b0 con 0x7f07a4150680 2020-11-20T12:21:39.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07a8f48700 1 -- 172.21.15.116:0/3307905273 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a0009910 con 0x7f07a4150680 2020-11-20T12:21:39.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07a8f48700 1 -- 172.21.15.116:0/3307905273 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0009c90 con 0x7f07a4150680 2020-11-20T12:21:39.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07ac1ae700 1 -- 172.21.15.116:0/3307905273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07a4150680 msgr2=0x7f07a4066450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:39.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07ac1ae700 1 --2- 172.21.15.116:0/3307905273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07a4150680 0x7f07a4066450 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f07a000b9a0 tx=0x7f07a0006df0).stop 2020-11-20T12:21:39.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07ac1ae700 1 -- 172.21.15.116:0/3307905273 shutdown_connections 2020-11-20T12:21:39.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07ac1ae700 1 --2- 172.21.15.116:0/3307905273 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07a4150680 0x7f07a4066450 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07ac1ae700 1 --2- 172.21.15.116:0/3307905273 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07a4156f20 0x7f07a40bf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07ac1ae700 1 -- 172.21.15.116:0/3307905273 >> 172.21.15.116:0/3307905273 conn(0x7f07a40c4f80 msgr2=0x7f07a40c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:39.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07ac1ae700 1 -- 172.21.15.116:0/3307905273 shutdown_connections 2020-11-20T12:21:39.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07ac1ae700 1 -- 172.21.15.116:0/3307905273 wait complete. 2020-11-20T12:21:39.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07ac1ae700 1 Processor -- start 2020-11-20T12:21:39.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.117+0000 7f07ac1ae700 1 -- start start 2020-11-20T12:21:39.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07ac1ae700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07a40bf560 0x7f07a414d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:39.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07ac1ae700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07a4150680 0x7f07a41536c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:39.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07ac1ae700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07a4156f20 0x7f07a4063390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:39.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07ac1ae700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a40a7910 con 0x7f07a40bf560 2020-11-20T12:21:39.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07ac1ae700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a40a7bb0 con 0x7f07a4150680 2020-11-20T12:21:39.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07ac1ae700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07a40a7e50 con 0x7f07a4156f20 2020-11-20T12:21:39.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07a9f4a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07a40bf560 0x7f07a414d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:39.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07a9f4a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07a40bf560 0x7f07a414d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51204/0 (socket says 172.21.15.116:51204) 2020-11-20T12:21:39.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07a9f4a700 1 -- 172.21.15.116:0/2805852952 learned_addr learned my addr 172.21.15.116:0/2805852952 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:39.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07a9749700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07a4150680 0x7f07a41536c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:39.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07aa74b700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07a4156f20 0x7f07a4063390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:39.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07a9f4a700 1 -- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07a4156f20 msgr2=0x7f07a4063390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:39.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07a9f4a700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07a4156f20 0x7f07a4063390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07a9f4a700 1 -- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07a4150680 msgr2=0x7f07a41536c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:39.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07a9f4a700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07a4150680 0x7f07a41536c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07a9f4a700 1 -- 172.21.15.116:0/2805852952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a40a80f0 con 0x7f07a40bf560 2020-11-20T12:21:39.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07a9f4a700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07a40bf560 0x7f07a414d820 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7f07a0006e30 tx=0x7f07a0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:39.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f0791ffb700 1 -- 172.21.15.116:0/2805852952 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a0011c80 con 0x7f07a40bf560 2020-11-20T12:21:39.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f0791ffb700 1 -- 172.21.15.116:0/2805852952 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07a0011de0 con 0x7f07a40bf560 2020-11-20T12:21:39.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07ac1ae700 1 -- 172.21.15.116:0/2805852952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a40a8390 con 0x7f07a40bf560 2020-11-20T12:21:39.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f0791ffb700 1 -- 172.21.15.116:0/2805852952 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07a001e670 con 0x7f07a40bf560 2020-11-20T12:21:39.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.121+0000 7f07ac1ae700 1 -- 172.21.15.116:0/2805852952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a4153bb0 con 0x7f07a40bf560 2020-11-20T12:21:39.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.125+0000 7f07ac1ae700 1 -- 172.21.15.116:0/2805852952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a40a8550 con 0x7f07a40bf560 2020-11-20T12:21:39.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.125+0000 7f0791ffb700 1 -- 172.21.15.116:0/2805852952 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f07a0025030 con 0x7f07a40bf560 2020-11-20T12:21:39.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.125+0000 7f0791ffb700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07800557c0 0x7f0780057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:39.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.125+0000 7f0791ffb700 1 -- 172.21.15.116:0/2805852952 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f07a006fd80 con 0x7f07a40bf560 2020-11-20T12:21:39.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.125+0000 7f07a9749700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07800557c0 0x7f0780057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:39.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.129+0000 7f07a9749700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07800557c0 0x7f0780057c80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f07a415b8c0 tx=0x7f0794008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:39.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.129+0000 7f0791ffb700 1 -- 172.21.15.116:0/2805852952 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f07a0045bc0 con 0x7f07a40bf560 2020-11-20T12:21:39.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.289+0000 7f07ac1ae700 1 -- 172.21.15.116:0/2805852952 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07a405d2a0 con 0x7f07a40bf560 2020-11-20T12:21:39.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.293+0000 7f0791ffb700 1 -- 172.21.15.116:0/2805852952 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f07a0031210 con 0x7f07a40bf560 2020-11-20T12:21:39.300 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:39.300 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":107,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:12.114283+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:09.571842+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:10.034013+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:39.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 -- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07800557c0 msgr2=0x7f0780057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:39.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07800557c0 0x7f0780057c80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f07a415b8c0 tx=0x7f0794008040).stop 2020-11-20T12:21:39.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 -- 172.21.15.116:0/2805852952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07a40bf560 msgr2=0x7f07a414d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:39.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07a40bf560 0x7f07a414d820 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7f07a0006e30 tx=0x7f07a0015000).stop 2020-11-20T12:21:39.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 -- 172.21.15.116:0/2805852952 shutdown_connections 2020-11-20T12:21:39.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f07800557c0 0x7f0780057c80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f07a4156f20 0x7f07a4063390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f07a40bf560 0x7f07a414d820 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 --2- 172.21.15.116:0/2805852952 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f07a4150680 0x7f07a41536c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 -- 172.21.15.116:0/2805852952 >> 172.21.15.116:0/2805852952 conn(0x7f07a40c4f80 msgr2=0x7f07a40a6940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:39.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 -- 172.21.15.116:0/2805852952 shutdown_connections 2020-11-20T12:21:39.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.297+0000 7f07ac1ae700 1 -- 172.21.15.116:0/2805852952 wait complete. 2020-11-20T12:21:39.326 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-20T12:21:39.327 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-20T12:21:39.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fdca46700 1 Processor -- start 2020-11-20T12:21:39.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fdca46700 1 -- start start 2020-11-20T12:21:39.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fdca46700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fd80a6c20 0x7f6fd814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fdca46700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd8150340 con 0x7f6fd80a6840 2020-11-20T12:21:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fdca46700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd80677e0 con 0x7f6fd8066cb0 2020-11-20T12:21:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fdca46700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd8067a80 con 0x7f6fd80a6c20 2020-11-20T12:21:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fd659c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6fd80a6840 0x7f6fd80bff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60646/0 (socket says 172.21.15.116:60646) 2020-11-20T12:21:39.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fd659c700 1 -- 172.21.15.116:0/335390649 learned_addr learned my addr 172.21.15.116:0/335390649 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:39.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fcffff700 1 --2- 172.21.15.116:0/335390649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fd80a6c20 0x7f6fd814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:39.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fd5d9b700 1 -- 172.21.15.116:0/335390649 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619153102 0 0) 0x7f6fd8150340 con 0x7f6fd80a6840 2020-11-20T12:21:39.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fd5d9b700 1 -- 172.21.15.116:0/335390649 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fb8003240 con 0x7f6fd80a6840 2020-11-20T12:21:39.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fcffff700 1 -- 172.21.15.116:0/335390649 >> v1:172.21.15.173:6789/0 conn(0x7f6fd8066cb0 legacy=0x7f6fd8153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:39.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fcffff700 1 -- 172.21.15.116:0/335390649 >> v1:172.21.15.116:6789/0 conn(0x7f6fd80a6840 legacy=0x7f6fd80bff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:39.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fcffff700 1 -- 172.21.15.116:0/335390649 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd8067d20 con 0x7f6fd80a6c20 2020-11-20T12:21:39.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.497+0000 7f6fcffff700 1 --2- 172.21.15.116:0/335390649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fd80a6c20 0x7f6fd814fe50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f6fc000b9d0 tx=0x7f6fc0006df0).ready entity=mon.2 client_cookie=e9e4228e59a4aabe server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:39.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.501+0000 7f6fd5d9b700 1 -- 172.21.15.116:0/335390649 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc00097b0 con 0x7f6fd80a6c20 2020-11-20T12:21:39.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.501+0000 7f6fd5d9b700 1 -- 172.21.15.116:0/335390649 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fc0009910 con 0x7f6fd80a6c20 2020-11-20T12:21:39.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.501+0000 7f6fd5d9b700 1 -- 172.21.15.116:0/335390649 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0009c90 con 0x7f6fd80a6c20 2020-11-20T12:21:39.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.501+0000 7f6fdca46700 1 -- 172.21.15.116:0/335390649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fd80a6c20 msgr2=0x7f6fd814fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:39.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.501+0000 7f6fdca46700 1 --2- 172.21.15.116:0/335390649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fd80a6c20 0x7f6fd814fe50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f6fc000b9d0 tx=0x7f6fc0006df0).stop 2020-11-20T12:21:39.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.501+0000 7f6fdca46700 1 -- 172.21.15.116:0/335390649 shutdown_connections 2020-11-20T12:21:39.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.501+0000 7f6fdca46700 1 --2- 172.21.15.116:0/335390649 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fd80a6c20 0x7f6fd814fe50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.501+0000 7f6fdca46700 1 -- 172.21.15.116:0/335390649 >> 172.21.15.116:0/335390649 conn(0x7f6fd8060170 msgr2=0x7f6fd8060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:39.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.501+0000 7f6fdca46700 1 -- 172.21.15.116:0/335390649 shutdown_connections 2020-11-20T12:21:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.501+0000 7f6fdca46700 1 -- 172.21.15.116:0/335390649 wait complete. 2020-11-20T12:21:39.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fdca46700 1 Processor -- start 2020-11-20T12:21:39.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fdca46700 1 -- start start 2020-11-20T12:21:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fdca46700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fd8066cb0 0x7f6fd80a3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fdca46700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fd80a6840 0x7f6fd814b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fdca46700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd80a6c20 0x7f6fd808f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fdca46700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd808e700 con 0x7f6fd80a6c20 2020-11-20T12:21:39.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fdca46700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd808e9a0 con 0x7f6fd80a6840 2020-11-20T12:21:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fdca46700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd808ec40 con 0x7f6fd8066cb0 2020-11-20T12:21:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fd6d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd80a6c20 0x7f6fd808f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fd6d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd80a6c20 0x7f6fd808f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51220/0 (socket says 172.21.15.116:51220) 2020-11-20T12:21:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fd6d9d700 1 -- 172.21.15.116:0/4130261329 learned_addr learned my addr 172.21.15.116:0/4130261329 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:39.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fd659c700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fd8066cb0 0x7f6fd80a3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fcffff700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fd80a6840 0x7f6fd814b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fd6d9d700 1 -- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fd8066cb0 msgr2=0x7f6fd80a3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fd6d9d700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fd8066cb0 0x7f6fd80a3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fd6d9d700 1 -- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fd80a6840 msgr2=0x7f6fd814b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:39.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fd6d9d700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fd80a6840 0x7f6fd814b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:39.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fd6d9d700 1 -- 172.21.15.116:0/4130261329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd808eee0 con 0x7f6fd80a6c20 2020-11-20T12:21:39.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.505+0000 7f6fd6d9d700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd80a6c20 0x7f6fd808f620 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f6fd000ca90 tx=0x7f6fd0007f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:39.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.509+0000 7f6fce7fc700 1 -- 172.21.15.116:0/4130261329 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000a7f0 con 0x7f6fd80a6c20 2020-11-20T12:21:39.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.509+0000 7f6fce7fc700 1 -- 172.21.15.116:0/4130261329 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd000a950 con 0x7f6fd80a6c20 2020-11-20T12:21:39.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.509+0000 7f6fce7fc700 1 -- 172.21.15.116:0/4130261329 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd000acd0 con 0x7f6fd80a6c20 2020-11-20T12:21:39.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.509+0000 7f6fdca46700 1 -- 172.21.15.116:0/4130261329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd808f180 con 0x7f6fd80a6c20 2020-11-20T12:21:39.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.509+0000 7f6fdca46700 1 -- 172.21.15.116:0/4130261329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd814bac0 con 0x7f6fd80a6c20 2020-11-20T12:21:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.509+0000 7f6fdca46700 1 -- 172.21.15.116:0/4130261329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd808f310 con 0x7f6fd80a6c20 2020-11-20T12:21:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.509+0000 7f6fce7fc700 1 -- 172.21.15.116:0/4130261329 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6fd0019030 con 0x7f6fd80a6c20 2020-11-20T12:21:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.513+0000 7f6fce7fc700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb40557f0 0x7f6fb4057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:39.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.513+0000 7f6fce7fc700 1 -- 172.21.15.116:0/4130261329 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6fd006df70 con 0x7f6fd80a6c20 2020-11-20T12:21:39.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.513+0000 7f6fd659c700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb40557f0 0x7f6fb4057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:39.519 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.513+0000 7f6fd659c700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb40557f0 0x7f6fb4057cb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f6fc80055a0 tx=0x7f6fc8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:39.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.517+0000 7f6fce7fc700 1 -- 172.21.15.116:0/4130261329 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6fd0043ec0 con 0x7f6fd80a6c20 2020-11-20T12:21:39.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:39.725+0000 7f6fdca46700 1 -- 172.21.15.116:0/4130261329 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6fd805d2a0 con 0x7f6fd80a6c20 2020-11-20T12:21:40.302 INFO:tasks.ceph.mon.a.smithi116.stderr:2020-11-20T12:21:40.297+0000 7fc111d1b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-20T12:21:40.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.309+0000 7f6fce7fc700 1 -- 172.21.15.116:0/4130261329 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v61) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6fd0017030 con 0x7f6fd80a6c20 2020-11-20T12:21:40.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 -- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb40557f0 msgr2=0x7f6fb4057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:40.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb40557f0 0x7f6fb4057cb0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f6fc80055a0 tx=0x7f6fc8009040).stop 2020-11-20T12:21:40.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 -- 172.21.15.116:0/4130261329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd80a6c20 msgr2=0x7f6fd808f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:40.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd80a6c20 0x7f6fd808f620 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f6fd000ca90 tx=0x7f6fd0007f00).stop 2020-11-20T12:21:40.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 -- 172.21.15.116:0/4130261329 shutdown_connections 2020-11-20T12:21:40.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb40557f0 0x7f6fb4057cb0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:40.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fd80a6c20 0x7f6fd808f620 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:40.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fd8066cb0 0x7f6fd80a3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:40.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 --2- 172.21.15.116:0/4130261329 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fd80a6840 0x7f6fd814b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:40.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 -- 172.21.15.116:0/4130261329 >> 172.21.15.116:0/4130261329 conn(0x7f6fd8060170 msgr2=0x7f6fd80bf720 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:40.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 -- 172.21.15.116:0/4130261329 shutdown_connections 2020-11-20T12:21:40.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.313+0000 7f6fdca46700 1 -- 172.21.15.116:0/4130261329 wait complete. 2020-11-20T12:21:40.321 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-20T12:21:40.345 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-20T12:21:40.387 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:21:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.513+0000 7f15b756a700 1 Processor -- start 2020-11-20T12:21:40.521 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.513+0000 7f15b756a700 1 -- start start 2020-11-20T12:21:40.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.517+0000 7f15b756a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f15b00b9db0 0x7f15b00c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:40.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.517+0000 7f15b756a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b00a8930 con 0x7f15b00a81f0 2020-11-20T12:21:40.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.517+0000 7f15b756a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b00a8690 con 0x7f15b014f880 2020-11-20T12:21:40.524 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.517+0000 7f15b756a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b00b94d0 con 0x7f15b00b9db0 2020-11-20T12:21:40.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.517+0000 7f15b5d67700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f15b00a81f0 0x7f15b00c0e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60672/0 (socket says 172.21.15.116:60672) 2020-11-20T12:21:40.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.517+0000 7f15b5d67700 1 -- 172.21.15.116:0/2328763705 learned_addr learned my addr 172.21.15.116:0/2328763705 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:40.525 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.517+0000 7f15b6d69700 1 --2- 172.21.15.116:0/2328763705 >> v2:172.21.15.116:3300/0 conn(0x7f15b00b9db0 0x7f15b00c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.517+0000 7f15b5566700 1 -- 172.21.15.116:0/2328763705 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138075350 0 0) 0x7f15b00a8930 con 0x7f15b00a81f0 2020-11-20T12:21:40.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.517+0000 7f15b5566700 1 -- 172.21.15.116:0/2328763705 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1598003240 con 0x7f15b00a81f0 2020-11-20T12:21:40.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b5566700 1 -- 172.21.15.116:0/2328763705 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140115023 0 0) 0x7f15b00a8690 con 0x7f15b014f880 2020-11-20T12:21:40.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b6d69700 1 -- 172.21.15.116:0/2328763705 >> v1:172.21.15.173:6789/0 conn(0x7f15b014f880 legacy=0x7f15b0060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:40.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b6d69700 1 -- 172.21.15.116:0/2328763705 >> v1:172.21.15.116:6789/0 conn(0x7f15b00a81f0 legacy=0x7f15b00c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:40.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b6d69700 1 -- 172.21.15.116:0/2328763705 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b00b9770 con 0x7f15b00b9db0 2020-11-20T12:21:40.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b6d69700 1 --2- 172.21.15.116:0/2328763705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00b9db0 0x7f15b00c15a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f15a8009c90 tx=0x7f15a80091d0).ready entity=mon.0 client_cookie=212783fccdf701cb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:40.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b5566700 1 -- 172.21.15.116:0/2328763705 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a800ba70 con 0x7f15b00b9db0 2020-11-20T12:21:40.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b5566700 1 -- 172.21.15.116:0/2328763705 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a800bbd0 con 0x7f15b00b9db0 2020-11-20T12:21:40.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b5566700 1 -- 172.21.15.116:0/2328763705 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a8011400 con 0x7f15b00b9db0 2020-11-20T12:21:40.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b756a700 1 -- 172.21.15.116:0/2328763705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00b9db0 msgr2=0x7f15b00c15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:40.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b756a700 1 --2- 172.21.15.116:0/2328763705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00b9db0 0x7f15b00c15a0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7f15a8009c90 tx=0x7f15a80091d0).stop 2020-11-20T12:21:40.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b756a700 1 -- 172.21.15.116:0/2328763705 shutdown_connections 2020-11-20T12:21:40.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b756a700 1 --2- 172.21.15.116:0/2328763705 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00b9db0 0x7f15b00c15a0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:40.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b756a700 1 -- 172.21.15.116:0/2328763705 >> 172.21.15.116:0/2328763705 conn(0x7f15b0068350 msgr2=0x7f15b00627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:40.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b756a700 1 -- 172.21.15.116:0/2328763705 shutdown_connections 2020-11-20T12:21:40.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.521+0000 7f15b756a700 1 -- 172.21.15.116:0/2328763705 wait complete. 2020-11-20T12:21:40.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b756a700 1 Processor -- start 2020-11-20T12:21:40.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b756a700 1 -- start start 2020-11-20T12:21:40.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b756a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15b00a81f0 0x7f15b0161a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:40.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b756a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00b9db0 0x7f15b0164e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:40.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b756a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15b014f880 0x7f15b00c46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:40.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b756a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0165310 con 0x7f15b00b9db0 2020-11-20T12:21:40.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b756a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b0160df0 con 0x7f15b014f880 2020-11-20T12:21:40.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b756a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b00c1c20 con 0x7f15b00a81f0 2020-11-20T12:21:40.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b5d67700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00b9db0 0x7f15b0164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:40.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b5d67700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00b9db0 0x7f15b0164e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51246/0 (socket says 172.21.15.116:51246) 2020-11-20T12:21:40.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b5d67700 1 -- 172.21.15.116:0/2263769469 learned_addr learned my addr 172.21.15.116:0/2263769469 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:40.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b6568700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15b00a81f0 0x7f15b0161a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:40.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b6d69700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15b014f880 0x7f15b00c46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:40.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b5d67700 1 -- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15b00a81f0 msgr2=0x7f15b0161a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:40.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b5d67700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15b00a81f0 0x7f15b0161a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:40.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b5d67700 1 -- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15b014f880 msgr2=0x7f15b00c46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:40.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b5d67700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15b014f880 0x7f15b00c46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:40.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b5d67700 1 -- 172.21.15.116:0/2263769469 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b0161030 con 0x7f15b00b9db0 2020-11-20T12:21:40.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b6568700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15b00a81f0 0x7f15b0161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:40.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.525+0000 7f15b6d69700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15b014f880 0x7f15b00c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:40.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.529+0000 7f15b5d67700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00b9db0 0x7f15b0164e20 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f15a000c6a0 tx=0x7f15a0007ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.529+0000 7f159e7fc700 1 -- 172.21.15.116:0/2263769469 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a00072e0 con 0x7f15b00b9db0 2020-11-20T12:21:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.529+0000 7f159e7fc700 1 -- 172.21.15.116:0/2263769469 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a0007440 con 0x7f15b00b9db0 2020-11-20T12:21:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.529+0000 7f159e7fc700 1 -- 172.21.15.116:0/2263769469 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a0005820 con 0x7f15b00b9db0 2020-11-20T12:21:40.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.529+0000 7f15b756a700 1 -- 172.21.15.116:0/2263769469 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b01612d0 con 0x7f15b00b9db0 2020-11-20T12:21:40.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.529+0000 7f15b756a700 1 -- 172.21.15.116:0/2263769469 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b00c4bb0 con 0x7f15b00b9db0 2020-11-20T12:21:40.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.529+0000 7f15b756a700 1 -- 172.21.15.116:0/2263769469 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b005cf30 con 0x7f15b00b9db0 2020-11-20T12:21:40.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.529+0000 7f159e7fc700 1 -- 172.21.15.116:0/2263769469 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f15a0010070 con 0x7f15b00b9db0 2020-11-20T12:21:40.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.529+0000 7f159e7fc700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15880557c0 0x7f1588057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:40.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.533+0000 7f159e7fc700 1 -- 172.21.15.116:0/2263769469 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f15a006dda0 con 0x7f15b00b9db0 2020-11-20T12:21:40.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.533+0000 7f15b6568700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15880557c0 0x7f1588057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:40.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.533+0000 7f15b6568700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15880557c0 0x7f1588057c80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f15b0152e50 tx=0x7f15ac009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:40.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.537+0000 7f159e7fc700 1 -- 172.21.15.116:0/2263769469 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f15a0043bf0 con 0x7f15b00b9db0 2020-11-20T12:21:40.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:40.745+0000 7f15b756a700 1 -- 172.21.15.116:0/2263769469 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f15b00639c0 con 0x7f15b00b9db0 2020-11-20T12:21:41.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.317+0000 7f159e7fc700 1 -- 172.21.15.116:0/2263769469 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v62) v1 ==== 104+0+0 (secure 0 0 0) 0x7f15a0047210 con 0x7f15b00b9db0 2020-11-20T12:21:41.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 -- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15880557c0 msgr2=0x7f1588057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:41.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15880557c0 0x7f1588057c80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f15b0152e50 tx=0x7f15ac009040).stop 2020-11-20T12:21:41.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 -- 172.21.15.116:0/2263769469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00b9db0 msgr2=0x7f15b0164e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:41.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00b9db0 0x7f15b0164e20 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7f15a000c6a0 tx=0x7f15a0007ac0).stop 2020-11-20T12:21:41.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 -- 172.21.15.116:0/2263769469 shutdown_connections 2020-11-20T12:21:41.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f15880557c0 0x7f1588057c80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:41.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f15b014f880 0x7f15b00c46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:41.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f15b00a81f0 0x7f15b0161a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:41.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 --2- 172.21.15.116:0/2263769469 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f15b00b9db0 0x7f15b0164e20 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:41.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 -- 172.21.15.116:0/2263769469 >> 172.21.15.116:0/2263769469 conn(0x7f15b0068350 msgr2=0x7f15b00bd580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:41.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 -- 172.21.15.116:0/2263769469 shutdown_connections 2020-11-20T12:21:41.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.321+0000 7f15b756a700 1 -- 172.21.15.116:0/2263769469 wait complete. 2020-11-20T12:21:41.354 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-20T12:21:41.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.581+0000 7f66fc7b4700 1 Processor -- start 2020-11-20T12:21:41.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.585+0000 7f66fc7b4700 1 -- start start 2020-11-20T12:21:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.585+0000 7f66fc7b4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66f41517f0 0x7f66f40bbd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.585+0000 7f66fc7b4700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f66f40c0140 0x7f66f40c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.585+0000 7f66fc7b4700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f40ba5a0 con 0x7f66f4151bd0 2020-11-20T12:21:41.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.585+0000 7f66fc7b4700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f40c0aa0 con 0x7f66f40c0140 2020-11-20T12:21:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.585+0000 7f66fc7b4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4156650 con 0x7f66f41517f0 2020-11-20T12:21:41.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.585+0000 7f66f9d4f700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f66f4151bd0 0x7f66f4060b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60686/0 (socket says 172.21.15.116:60686) 2020-11-20T12:21:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.585+0000 7f66f9d4f700 1 -- 172.21.15.116:0/2333964420 learned_addr learned my addr 172.21.15.116:0/2333964420 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.585+0000 7f66fad51700 1 --2- 172.21.15.116:0/2333964420 >> v2:172.21.15.173:3300/0 conn(0x7f66f40c0140 0x7f66f40c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.585+0000 7f66fa550700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66f41517f0 0x7f66f40bbd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:41.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033897953 0 0) 0x7f66f40ba5a0 con 0x7f66f4151bd0 2020-11-20T12:21:41.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66dc003240 con 0x7f66f4151bd0 2020-11-20T12:21:41.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1074300450 0 0) 0x7f66dc003240 con 0x7f66f4151bd0 2020-11-20T12:21:41.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66f41517f0 msgr2=0x7f66f40bbd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:41.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66f41517f0 0x7f66f40bbd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:41.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 >> v2:172.21.15.173:3300/0 conn(0x7f66f40c0140 msgr2=0x7f66f40c05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:41.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 --2- 172.21.15.116:0/2333964420 >> v2:172.21.15.173:3300/0 conn(0x7f66f40c0140 0x7f66f40c05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:41.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4068560 con 0x7f66f4151bd0 2020-11-20T12:21:41.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f66e40014e0 con 0x7f66f4151bd0 2020-11-20T12:21:41.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 >> v1:172.21.15.116:6789/0 conn(0x7f66f4151bd0 legacy=0x7f66f4060b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:41.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66dc003730 0x7f66dc003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:41.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 0x7f66dc004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:41.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66dc004e50 0x7f66dc00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:41.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40c0aa0 con 0x7f66dc0044e0 2020-11-20T12:21:41.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4156650 con 0x7f66dc003730 2020-11-20T12:21:41.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66dc004210 con 0x7f66dc004e50 2020-11-20T12:21:41.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f9d4f700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 0x7f66dc004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:41.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66fad51700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66dc004e50 0x7f66dc00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:41.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66fa550700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66dc003730 0x7f66dc003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:41.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f9d4f700 1 -- 172.21.15.116:0/2333964420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66dc004e50 msgr2=0x7f66dc00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:41.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f9d4f700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66dc004e50 0x7f66dc00b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:41.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f9d4f700 1 -- 172.21.15.116:0/2333964420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66dc003730 msgr2=0x7f66dc003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:41.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f9d4f700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66dc003730 0x7f66dc003b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:41.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f9d4f700 1 -- 172.21.15.116:0/2333964420 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66dc003240 con 0x7f66dc0044e0 2020-11-20T12:21:41.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.589+0000 7f66f9d4f700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 0x7f66dc004960 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7f66e400c410 tx=0x7f66e4007800).ready entity=mon.0 client_cookie=4983f1fdc4f9fdc8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:41.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.593+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e400ce80 con 0x7f66dc0044e0 2020-11-20T12:21:41.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.593+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e40072e0 con 0x7f66dc0044e0 2020-11-20T12:21:41.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.593+0000 7f66f954e700 1 -- 172.21.15.116:0/2333964420 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4005800 con 0x7f66dc0044e0 2020-11-20T12:21:41.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.593+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2333964420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 msgr2=0x7f66dc004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:41.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.593+0000 7f66fc7b4700 1 --2- 172.21.15.116:0/2333964420 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 0x7f66dc004960 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7f66e400c410 tx=0x7f66e4007800).stop 2020-11-20T12:21:41.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.593+0000 7f66fad51700 1 -- 172.21.15.116:0/2333964420 reap_dead start 2020-11-20T12:21:41.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.593+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2333964420 shutdown_connections 2020-11-20T12:21:41.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.593+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2333964420 >> 172.21.15.116:0/2333964420 conn(0x7f66f40c2b00 msgr2=0x7f66f4061280 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:41.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.593+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2333964420 shutdown_connections 2020-11-20T12:21:41.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.593+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2333964420 wait complete. 2020-11-20T12:21:41.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fc7b4700 1 Processor -- start 2020-11-20T12:21:41.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fc7b4700 1 -- start start 2020-11-20T12:21:41.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fc7b4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 0x7f66f41498f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:41.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fc7b4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f40a8540 0x7f66f40aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:41.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fc7b4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66f40a89c0 0x7f66f40aa550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:41.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fc7b4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40aaa40 con 0x7f66dc0044e0 2020-11-20T12:21:41.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fc7b4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4152b10 con 0x7f66f40a8540 2020-11-20T12:21:41.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fc7b4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4152870 con 0x7f66f40a89c0 2020-11-20T12:21:41.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fa550700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 0x7f66f41498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:41.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fa550700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 0x7f66f41498f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51262/0 (socket says 172.21.15.116:51262) 2020-11-20T12:21:41.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fa550700 1 -- 172.21.15.116:0/2645193101 learned_addr learned my addr 172.21.15.116:0/2645193101 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:41.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fad51700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66f40a89c0 0x7f66f40aa550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:41.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66f9d4f700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f40a8540 0x7f66f40aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:41.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fa550700 1 -- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66f40a89c0 msgr2=0x7f66f40aa550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:41.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fa550700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66f40a89c0 0x7f66f40aa550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:41.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fa550700 1 -- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f40a8540 msgr2=0x7f66f40aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:41.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fa550700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f40a8540 0x7f66f40aa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:41.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fa550700 1 -- 172.21.15.116:0/2645193101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4152d60 con 0x7f66dc0044e0 2020-11-20T12:21:41.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66f9d4f700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f40a8540 0x7f66f40aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:41.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fa550700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 0x7f66f41498f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f66f000f840 tx=0x7f66f0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:41.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66e27fc700 1 -- 172.21.15.116:0/2645193101 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0017070 con 0x7f66dc0044e0 2020-11-20T12:21:41.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66e27fc700 1 -- 172.21.15.116:0/2645193101 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0011590 con 0x7f66dc0044e0 2020-11-20T12:21:41.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66e27fc700 1 -- 172.21.15.116:0/2645193101 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0011910 con 0x7f66dc0044e0 2020-11-20T12:21:41.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.597+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2645193101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f4153000 con 0x7f66dc0044e0 2020-11-20T12:21:41.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.601+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2645193101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f4153540 con 0x7f66dc0044e0 2020-11-20T12:21:41.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.601+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2645193101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f405d2a0 con 0x7f66dc0044e0 2020-11-20T12:21:41.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.601+0000 7f66e27fc700 1 -- 172.21.15.116:0/2645193101 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f66f0011c80 con 0x7f66dc0044e0 2020-11-20T12:21:41.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.601+0000 7f66e27fc700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66cc0557c0 0x7f66cc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:41.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.601+0000 7f66e27fc700 1 -- 172.21.15.116:0/2645193101 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f66f006e490 con 0x7f66dc0044e0 2020-11-20T12:21:41.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.601+0000 7f66f9d4f700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66cc0557c0 0x7f66cc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:41.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.605+0000 7f66f9d4f700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66cc0557c0 0x7f66cc057c80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f66e400c410 tx=0x7f66e4015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:41.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.605+0000 7f66e27fc700 1 -- 172.21.15.116:0/2645193101 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f66f00442e0 con 0x7f66dc0044e0 2020-11-20T12:21:41.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:41.765+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2645193101 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f66f4063440 con 0x7f66dc0044e0 2020-11-20T12:21:42.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.321+0000 7f66e27fc700 1 -- 172.21.15.116:0/2645193101 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v109) v1 ==== 181+0+0 (secure 0 0 0) 0x7f66f0047900 con 0x7f66dc0044e0 2020-11-20T12:21:42.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66cc0557c0 msgr2=0x7f66cc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:42.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66cc0557c0 0x7f66cc057c80 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f66e400c410 tx=0x7f66e4015000).stop 2020-11-20T12:21:42.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2645193101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 msgr2=0x7f66f41498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:42.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 0x7f66f41498f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f66f000f840 tx=0x7f66f0008040).stop 2020-11-20T12:21:42.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2645193101 shutdown_connections 2020-11-20T12:21:42.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f66cc0557c0 0x7f66cc057c80 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:42.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f66f40a89c0 0x7f66f40aa550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:42.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f66dc0044e0 0x7f66f41498f0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:42.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 --2- 172.21.15.116:0/2645193101 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f66f40a8540 0x7f66f40aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:42.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2645193101 >> 172.21.15.116:0/2645193101 conn(0x7f66f40c2b00 msgr2=0x7f66f40611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:42.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2645193101 shutdown_connections 2020-11-20T12:21:42.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.325+0000 7f66fc7b4700 1 -- 172.21.15.116:0/2645193101 wait complete. 2020-11-20T12:21:42.335 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2020-11-20T12:21:42.357 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-20T12:21:42.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.529+0000 7f36a5957700 1 Processor -- start 2020-11-20T12:21:42.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.529+0000 7f36a5957700 1 -- start start 2020-11-20T12:21:42.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.533+0000 7f36a5957700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a00aa000 0x7f36a00aa3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:42.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.533+0000 7f36a5957700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f36a0152580 0x7f36a0152a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:42.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.533+0000 7f36a5957700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36a00ac120 con 0x7f36a00abac0 2020-11-20T12:21:42.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.533+0000 7f36a5957700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36a00abf40 con 0x7f36a0152580 2020-11-20T12:21:42.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.533+0000 7f36a5957700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a00ac3c0 con 0x7f36a00aa000 2020-11-20T12:21:42.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.533+0000 7f369effd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a00aa000 0x7f36a00aa3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:42.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.533+0000 7f369e7fc700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f36a00abac0 0x7f36a00c7090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:56936/0 (socket says 172.21.15.116:56936) 2020-11-20T12:21:42.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.533+0000 7f369e7fc700 1 -- 172.21.15.116:0/4087129523 learned_addr learned my addr 172.21.15.116:0/4087129523 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:42.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.533+0000 7f369ffff700 1 --2- 172.21.15.116:0/4087129523 >> v2:172.21.15.173:3300/0 conn(0x7f36a0152580 0x7f36a0152a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:42.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f369dffb700 1 -- 172.21.15.116:0/4087129523 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3020551881 0 0) 0x7f36a00ac120 con 0x7f36a00abac0 2020-11-20T12:21:42.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f369dffb700 1 -- 172.21.15.116:0/4087129523 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f368c003240 con 0x7f36a00abac0 2020-11-20T12:21:42.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f369ffff700 1 -- 172.21.15.116:0/4087129523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a00aa000 msgr2=0x7f36a00aa3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:42.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f369ffff700 1 --2- 172.21.15.116:0/4087129523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a00aa000 0x7f36a00aa3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:42.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f369ffff700 1 -- 172.21.15.116:0/4087129523 >> v1:172.21.15.173:6789/0 conn(0x7f36a00abac0 legacy=0x7f36a00c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:42.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f369ffff700 1 -- 172.21.15.116:0/4087129523 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a00ac660 con 0x7f36a0152580 2020-11-20T12:21:42.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f369ffff700 1 --2- 172.21.15.116:0/4087129523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a0152580 0x7f36a0152a30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f3698009c90 tx=0x7f36980091d0).ready entity=mon.1 client_cookie=fc8753a63ef036cc server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:42.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f369dffb700 1 -- 172.21.15.116:0/4087129523 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369800b9c0 con 0x7f36a0152580 2020-11-20T12:21:42.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f369dffb700 1 -- 172.21.15.116:0/4087129523 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369800bb20 con 0x7f36a0152580 2020-11-20T12:21:42.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f369dffb700 1 -- 172.21.15.116:0/4087129523 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369800bea0 con 0x7f36a0152580 2020-11-20T12:21:42.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f36a5957700 1 -- 172.21.15.116:0/4087129523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a0152580 msgr2=0x7f36a0152a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:42.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f36a5957700 1 --2- 172.21.15.116:0/4087129523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a0152580 0x7f36a0152a30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f3698009c90 tx=0x7f36980091d0).stop 2020-11-20T12:21:42.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f36a5957700 1 -- 172.21.15.116:0/4087129523 shutdown_connections 2020-11-20T12:21:42.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f36a5957700 1 --2- 172.21.15.116:0/4087129523 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a0152580 0x7f36a0152a30 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:42.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f36a5957700 1 --2- 172.21.15.116:0/4087129523 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a00aa000 0x7f36a00aa3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:42.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f36a5957700 1 -- 172.21.15.116:0/4087129523 >> 172.21.15.116:0/4087129523 conn(0x7f36a00ca7c0 msgr2=0x7f36a00c6790 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:42.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f36a5957700 1 -- 172.21.15.116:0/4087129523 shutdown_connections 2020-11-20T12:21:42.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.537+0000 7f36a5957700 1 -- 172.21.15.116:0/4087129523 wait complete. 2020-11-20T12:21:42.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.541+0000 7f36a5957700 1 Processor -- start 2020-11-20T12:21:42.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.541+0000 7f36a5957700 1 -- start start 2020-11-20T12:21:42.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f36a5957700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36a00aa000 0x7f36a014df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:42.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f36a5957700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a00abac0 0x7f36a00adc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:42.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f36a5957700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a0152580 0x7f36a00ae160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:42.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f36a5957700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0166310 con 0x7f36a00aa000 2020-11-20T12:21:42.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f36a5957700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0165690 con 0x7f36a0152580 2020-11-20T12:21:42.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f36a5957700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0165930 con 0x7f36a00abac0 2020-11-20T12:21:42.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369effd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36a00aa000 0x7f36a014df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:42.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369e7fc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a00abac0 0x7f36a00adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:42.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369effd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36a00aa000 0x7f36a014df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51276/0 (socket says 172.21.15.116:51276) 2020-11-20T12:21:42.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369effd700 1 -- 172.21.15.116:0/678873769 learned_addr learned my addr 172.21.15.116:0/678873769 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:42.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369ffff700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a0152580 0x7f36a00ae160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:42.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369effd700 1 -- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a00abac0 msgr2=0x7f36a00adc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:42.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369effd700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a00abac0 0x7f36a00adc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:42.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369effd700 1 -- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a0152580 msgr2=0x7f36a00ae160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:42.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369effd700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a0152580 0x7f36a00ae160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:42.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369effd700 1 -- 172.21.15.116:0/678873769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a0165bd0 con 0x7f36a00aa000 2020-11-20T12:21:42.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369ffff700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a0152580 0x7f36a00ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:42.554 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f369effd700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36a00aa000 0x7f36a014df80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f369000eba0 tx=0x7f3690006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:42.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f3686ffd700 1 -- 172.21.15.116:0/678873769 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369000c9b0 con 0x7f36a00aa000 2020-11-20T12:21:42.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f3686ffd700 1 -- 172.21.15.116:0/678873769 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f369000cb10 con 0x7f36a00aa000 2020-11-20T12:21:42.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f36a5957700 1 -- 172.21.15.116:0/678873769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a0165e70 con 0x7f36a00aa000 2020-11-20T12:21:42.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f3686ffd700 1 -- 172.21.15.116:0/678873769 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f369000ce90 con 0x7f36a00aa000 2020-11-20T12:21:42.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.545+0000 7f36a5957700 1 -- 172.21.15.116:0/678873769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a014e470 con 0x7f36a00aa000 2020-11-20T12:21:42.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.549+0000 7f36a5957700 1 -- 172.21.15.116:0/678873769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a005d2a0 con 0x7f36a00aa000 2020-11-20T12:21:42.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.549+0000 7f3686ffd700 1 -- 172.21.15.116:0/678873769 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3690012600 con 0x7f36a00aa000 2020-11-20T12:21:42.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.553+0000 7f3686ffd700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f367c0557c0 0x7f367c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:42.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.553+0000 7f3686ffd700 1 -- 172.21.15.116:0/678873769 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 5721+0+0 (secure 0 0 0) 0x7f369006e910 con 0x7f36a00aa000 2020-11-20T12:21:42.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.553+0000 7f369e7fc700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f367c0557c0 0x7f367c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:42.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.553+0000 7f369e7fc700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f367c0557c0 0x7f367c057c80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f36a00c5ed0 tx=0x7f3694009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:42.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.557+0000 7f3686ffd700 1 -- 172.21.15.116:0/678873769 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f36900448b0 con 0x7f36a00aa000 2020-11-20T12:21:42.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:42.713+0000 7f36a5957700 1 -- 172.21.15.116:0/678873769 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f36a0166000 con 0x7f36a00aa000 2020-11-20T12:21:43.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.337+0000 7f3686ffd700 1 -- 172.21.15.116:0/678873769 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v110) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3690047ed0 con 0x7f36a00aa000 2020-11-20T12:21:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 -- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f367c0557c0 msgr2=0x7f367c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f367c0557c0 0x7f367c057c80 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f36a00c5ed0 tx=0x7f3694009040).stop 2020-11-20T12:21:43.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 -- 172.21.15.116:0/678873769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36a00aa000 msgr2=0x7f36a014df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36a00aa000 0x7f36a014df80 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f369000eba0 tx=0x7f3690006df0).stop 2020-11-20T12:21:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 -- 172.21.15.116:0/678873769 shutdown_connections 2020-11-20T12:21:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f367c0557c0 0x7f367c057c80 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:43.347 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f36a0152580 0x7f36a00ae160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f36a00aa000 0x7f36a014df80 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 --2- 172.21.15.116:0/678873769 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f36a00abac0 0x7f36a00adc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:43.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 -- 172.21.15.116:0/678873769 >> 172.21.15.116:0/678873769 conn(0x7f36a00ca7c0 msgr2=0x7f36a00c7fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 -- 172.21.15.116:0/678873769 shutdown_connections 2020-11-20T12:21:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:43.341+0000 7f36a5957700 1 -- 172.21.15.116:0/678873769 wait complete. 2020-11-20T12:21:43.349 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2020-11-20T12:21:43.371 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T12:21:43.371 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-20T12:21:43.372 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-20T12:21:43.372 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:21:43.373 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-20T12:21:43.373 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-20T12:21:43.374 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-20T12:21:43.374 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:21:43.375 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T12:21:43.375 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-20T12:21:43.375 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-20T12:21:43.376 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:21:43.376 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-20T12:21:43.377 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-20T12:21:43.377 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-20T12:21:43.378 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:21:43.395 INFO:tasks.ceph.mds.a:Stopped 2020-11-20T12:21:43.395 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T12:21:43.397 INFO:tasks.ceph.mds.c:Stopped 2020-11-20T12:21:43.398 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-20T12:21:43.399 INFO:tasks.ceph.mds.d:Stopped 2020-11-20T12:21:43.400 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-20T12:21:43.447 INFO:tasks.ceph.mds.a:Started 2020-11-20T12:21:43.448 DEBUG:teuthology.parallel:result is None 2020-11-20T12:21:43.450 INFO:tasks.ceph.mds.d:Started 2020-11-20T12:21:43.450 DEBUG:teuthology.parallel:result is None 2020-11-20T12:21:43.451 INFO:tasks.ceph.mds.c:Started 2020-11-20T12:21:43.452 DEBUG:teuthology.parallel:result is None 2020-11-20T12:21:43.551 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:21:43.552 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:21:43.558 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2020-11-20T12:21:44.213 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2020-11-20T12:21:49.403 INFO:tasks.ceph.mds.b:Stopped 2020-11-20T12:21:49.404 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T12:21:49.416 INFO:tasks.ceph.mds.b:Started 2020-11-20T12:21:49.418 DEBUG:teuthology.parallel:result is None 2020-11-20T12:21:49.419 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-20T12:21:49.520 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2020-11-20T12:21:49.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.604+0000 7f102aff1700 1 Processor -- start 2020-11-20T12:21:49.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.604+0000 7f102aff1700 1 -- start start 2020-11-20T12:21:49.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f102aff1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f10240a67f0 0x7f10240a4640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:49.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f102aff1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10241564d0 con 0x7f10241559d0 2020-11-20T12:21:49.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f102aff1700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1024156770 con 0x7f1024155db0 2020-11-20T12:21:49.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f102aff1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1024156a10 con 0x7f10240a67f0 2020-11-20T12:21:49.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f10297ee700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f1024155db0 0x7f10240a3f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:56976/0 (socket says 172.21.15.116:56976) 2020-11-20T12:21:49.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f1029fef700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f10241559d0 0x7f1024062750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60748/0 (socket says 172.21.15.116:60748) 2020-11-20T12:21:49.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f10297ee700 1 -- 172.21.15.116:0/3651675946 learned_addr learned my addr 172.21.15.116:0/3651675946 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:49.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f102a7f0700 1 --2- 172.21.15.116:0/3651675946 >> v2:172.21.15.116:3300/0 conn(0x7f10240a67f0 0x7f10240a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:49.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f1028fed700 1 -- 172.21.15.116:0/3651675946 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437836834 0 0) 0x7f1024156770 con 0x7f1024155db0 2020-11-20T12:21:49.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f1028fed700 1 -- 172.21.15.116:0/3651675946 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f100c003240 con 0x7f1024155db0 2020-11-20T12:21:49.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f1028fed700 1 -- 172.21.15.116:0/3651675946 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698728685 0 0) 0x7f10241564d0 con 0x7f10241559d0 2020-11-20T12:21:49.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f1028fed700 1 -- 172.21.15.116:0/3651675946 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1024156770 con 0x7f10241559d0 2020-11-20T12:21:49.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f102a7f0700 1 -- 172.21.15.116:0/3651675946 >> v1:172.21.15.173:6789/0 conn(0x7f1024155db0 legacy=0x7f10240a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:49.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f102a7f0700 1 -- 172.21.15.116:0/3651675946 >> v1:172.21.15.116:6789/0 conn(0x7f10241559d0 legacy=0x7f1024062750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:49.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.608+0000 7f102a7f0700 1 -- 172.21.15.116:0/3651675946 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1024156cb0 con 0x7f10240a67f0 2020-11-20T12:21:49.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.612+0000 7f102a7f0700 1 --2- 172.21.15.116:0/3651675946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10240a67f0 0x7f10240a4640 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f101c00b9a0 tx=0x7f101c006df0).ready entity=mon.0 client_cookie=3b29a8286b181531 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:49.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.612+0000 7f1028fed700 1 -- 172.21.15.116:0/3651675946 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c009680 con 0x7f10240a67f0 2020-11-20T12:21:49.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.612+0000 7f1028fed700 1 -- 172.21.15.116:0/3651675946 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101c0097e0 con 0x7f10240a67f0 2020-11-20T12:21:49.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.612+0000 7f1028fed700 1 -- 172.21.15.116:0/3651675946 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101c009b60 con 0x7f10240a67f0 2020-11-20T12:21:49.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.612+0000 7f102aff1700 1 -- 172.21.15.116:0/3651675946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10240a67f0 msgr2=0x7f10240a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:49.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.612+0000 7f102aff1700 1 --2- 172.21.15.116:0/3651675946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10240a67f0 0x7f10240a4640 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f101c00b9a0 tx=0x7f101c006df0).stop 2020-11-20T12:21:49.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.612+0000 7f102aff1700 1 -- 172.21.15.116:0/3651675946 shutdown_connections 2020-11-20T12:21:49.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.612+0000 7f102aff1700 1 --2- 172.21.15.116:0/3651675946 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10240a67f0 0x7f10240a4640 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:49.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.612+0000 7f102aff1700 1 -- 172.21.15.116:0/3651675946 >> 172.21.15.116:0/3651675946 conn(0x7f1024060170 msgr2=0x7f1024060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:49.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.612+0000 7f102aff1700 1 -- 172.21.15.116:0/3651675946 shutdown_connections 2020-11-20T12:21:49.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.612+0000 7f102aff1700 1 -- 172.21.15.116:0/3651675946 wait complete. 2020-11-20T12:21:49.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f102aff1700 1 Processor -- start 2020-11-20T12:21:49.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f102aff1700 1 -- start start 2020-11-20T12:21:49.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f102aff1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10240a67f0 0x7f10240c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:49.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f102aff1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10241559d0 0x7f102414f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:49.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f102aff1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1024155db0 0x7f102415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:49.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f102aff1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1024154dd0 con 0x7f10240a67f0 2020-11-20T12:21:49.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f102aff1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1024155070 con 0x7f1024155db0 2020-11-20T12:21:49.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f102aff1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10240c3c80 con 0x7f10241559d0 2020-11-20T12:21:49.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f1029fef700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10240a67f0 0x7f10240c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:49.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f1029fef700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10240a67f0 0x7f10240c43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51318/0 (socket says 172.21.15.116:51318) 2020-11-20T12:21:49.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f1029fef700 1 -- 172.21.15.116:0/992925123 learned_addr learned my addr 172.21.15.116:0/992925123 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:49.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.616+0000 7f102a7f0700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1024155db0 0x7f102415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:49.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f10297ee700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10241559d0 0x7f102414f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:49.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f1029fef700 1 -- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10241559d0 msgr2=0x7f102414f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:49.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f1029fef700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10241559d0 0x7f102414f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:49.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f1029fef700 1 -- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1024155db0 msgr2=0x7f102415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:49.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f1029fef700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1024155db0 0x7f102415a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:49.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f1029fef700 1 -- 172.21.15.116:0/992925123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10240c3f20 con 0x7f10240a67f0 2020-11-20T12:21:49.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f102a7f0700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1024155db0 0x7f102415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:49.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f1029fef700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10240a67f0 0x7f10240c43c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f102000ca60 tx=0x7f1020007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:49.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f1011ffb700 1 -- 172.21.15.116:0/992925123 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020005ac0 con 0x7f10240a67f0 2020-11-20T12:21:49.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f1011ffb700 1 -- 172.21.15.116:0/992925123 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1020005c20 con 0x7f10240a67f0 2020-11-20T12:21:49.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f102aff1700 1 -- 172.21.15.116:0/992925123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10240c48b0 con 0x7f10240a67f0 2020-11-20T12:21:49.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f1011ffb700 1 -- 172.21.15.116:0/992925123 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020007110 con 0x7f10240a67f0 2020-11-20T12:21:49.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.620+0000 7f102aff1700 1 -- 172.21.15.116:0/992925123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1024159a40 con 0x7f10240a67f0 2020-11-20T12:21:49.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.624+0000 7f102aff1700 1 -- 172.21.15.116:0/992925123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1024155270 con 0x7f10240a67f0 2020-11-20T12:21:49.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.624+0000 7f1011ffb700 1 -- 172.21.15.116:0/992925123 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1020010030 con 0x7f10240a67f0 2020-11-20T12:21:49.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.624+0000 7f1011ffb700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ffc0557c0 0x7f0ffc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:49.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.624+0000 7f1011ffb700 1 -- 172.21.15.116:0/992925123 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f102006e3c0 con 0x7f10240a67f0 2020-11-20T12:21:49.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.624+0000 7f10297ee700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ffc0557c0 0x7f0ffc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:49.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.624+0000 7f10297ee700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ffc0557c0 0x7f0ffc057c80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f10240c4d10 tx=0x7f1014009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:49.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.632+0000 7f1011ffb700 1 -- 172.21.15.116:0/992925123 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1020044540 con 0x7f10240a67f0 2020-11-20T12:21:49.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:49.808+0000 7f102aff1700 1 -- 172.21.15.116:0/992925123 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f10240c3ac0 con 0x7f10240a67f0 2020-11-20T12:21:50.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.396+0000 7f1011ffb700 1 -- 172.21.15.116:0/992925123 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v113) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1020047b60 con 0x7f10240a67f0 2020-11-20T12:21:50.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 -- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ffc0557c0 msgr2=0x7f0ffc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:50.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ffc0557c0 0x7f0ffc057c80 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f10240c4d10 tx=0x7f1014009040).stop 2020-11-20T12:21:50.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 -- 172.21.15.116:0/992925123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10240a67f0 msgr2=0x7f10240c43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:50.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10240a67f0 0x7f10240c43c0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f102000ca60 tx=0x7f1020007e50).stop 2020-11-20T12:21:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 -- 172.21.15.116:0/992925123 shutdown_connections 2020-11-20T12:21:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0ffc0557c0 0x7f0ffc057c80 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1024155db0 0x7f102415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f10240a67f0 0x7f10240c43c0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 --2- 172.21.15.116:0/992925123 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f10241559d0 0x7f102414f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 -- 172.21.15.116:0/992925123 >> 172.21.15.116:0/992925123 conn(0x7f1024060170 msgr2=0x7f1024067460 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 -- 172.21.15.116:0/992925123 shutdown_connections 2020-11-20T12:21:50.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.400+0000 7f102aff1700 1 -- 172.21.15.116:0/992925123 wait complete. 2020-11-20T12:21:50.406 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2020-11-20T12:21:50.429 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-20T12:21:50.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a6759e700 1 Processor -- start 2020-11-20T12:21:50.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a6759e700 1 -- start start 2020-11-20T12:21:50.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a6759e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1a6813afa0 0x7f1a680aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:50.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a6759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6813b380 0x7f1a680aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:50.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a6759e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1a680acd30 0x7f1a680ad1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:50.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a6759e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a680ada00 con 0x7f1a680acd30 2020-11-20T12:21:50.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a6759e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a680adca0 con 0x7f1a6813afa0 2020-11-20T12:21:50.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a6759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680adf40 con 0x7f1a6813b380 2020-11-20T12:21:50.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a66d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1a680acd30 0x7f1a680ad1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:50.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a66d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1a680acd30 0x7f1a680ad1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51328/0 (socket says 172.21.15.116:51328) 2020-11-20T12:21:50.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a66d9d700 1 -- 172.21.15.116:0/1451808675 learned_addr learned my addr 172.21.15.116:0/1451808675 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:50.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a5dd9b700 1 --2- 172.21.15.116:0/1451808675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6813b380 0x7f1a680aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:50.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.600+0000 7f1a6659c700 1 --2- 172.21.15.116:0/1451808675 >> v2:172.21.15.173:3300/0 conn(0x7f1a6813afa0 0x7f1a680aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:50.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a66d9d700 1 -- 172.21.15.116:0/1451808675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6813b380 msgr2=0x7f1a680aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:50.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a66d9d700 1 --2- 172.21.15.116:0/1451808675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6813b380 0x7f1a680aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a66d9d700 1 -- 172.21.15.116:0/1451808675 >> v2:172.21.15.173:3300/0 conn(0x7f1a6813afa0 msgr2=0x7f1a680aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:50.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a66d9d700 1 --2- 172.21.15.116:0/1451808675 >> v2:172.21.15.173:3300/0 conn(0x7f1a6813afa0 0x7f1a680aa5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a66d9d700 1 -- 172.21.15.116:0/1451808675 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680ae1e0 con 0x7f1a680acd30 2020-11-20T12:21:50.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a66d9d700 1 --2- 172.21.15.116:0/1451808675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680acd30 0x7f1a680ad1c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f1a680818e0 tx=0x7f1a60006eb0).ready entity=mon.0 client_cookie=75785d050c66332f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:50.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a65d9b700 1 -- 172.21.15.116:0/1451808675 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60009810 con 0x7f1a680acd30 2020-11-20T12:21:50.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a65d9b700 1 -- 172.21.15.116:0/1451808675 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a60009970 con 0x7f1a680acd30 2020-11-20T12:21:50.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a65d9b700 1 -- 172.21.15.116:0/1451808675 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a60009cf0 con 0x7f1a680acd30 2020-11-20T12:21:50.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a6759e700 1 -- 172.21.15.116:0/1451808675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680acd30 msgr2=0x7f1a680ad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:50.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a6759e700 1 --2- 172.21.15.116:0/1451808675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680acd30 0x7f1a680ad1c0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f1a680818e0 tx=0x7f1a60006eb0).stop 2020-11-20T12:21:50.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a6759e700 1 -- 172.21.15.116:0/1451808675 shutdown_connections 2020-11-20T12:21:50.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a6759e700 1 --2- 172.21.15.116:0/1451808675 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680acd30 0x7f1a680ad1c0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a6759e700 1 --2- 172.21.15.116:0/1451808675 >> v2:172.21.15.173:3300/0 conn(0x7f1a6813afa0 0x7f1a680aa5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a6759e700 1 --2- 172.21.15.116:0/1451808675 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6813b380 0x7f1a680aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a6759e700 1 -- 172.21.15.116:0/1451808675 >> 172.21.15.116:0/1451808675 conn(0x7f1a680678a0 msgr2=0x7f1a68067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:50.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a6759e700 1 -- 172.21.15.116:0/1451808675 shutdown_connections 2020-11-20T12:21:50.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.604+0000 7f1a6759e700 1 -- 172.21.15.116:0/1451808675 wait complete. 2020-11-20T12:21:50.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.608+0000 7f1a6759e700 1 Processor -- start 2020-11-20T12:21:50.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.608+0000 7f1a6759e700 1 -- start start 2020-11-20T12:21:50.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.608+0000 7f1a6759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680acd30 0x7f1a680a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:50.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.608+0000 7f1a6759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6813afa0 0x7f1a68151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:50.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.608+0000 7f1a6759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a6813b380 0x7f1a680c7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:50.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.608+0000 7f1a6759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680a4a70 con 0x7f1a680acd30 2020-11-20T12:21:50.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.608+0000 7f1a6759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a68161c60 con 0x7f1a6813b380 2020-11-20T12:21:50.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.608+0000 7f1a6759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a680a4c60 con 0x7f1a6813afa0 2020-11-20T12:21:50.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.608+0000 7f1a6659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680acd30 0x7f1a680a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:50.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.608+0000 7f1a6659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680acd30 0x7f1a680a58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51332/0 (socket says 172.21.15.116:51332) 2020-11-20T12:21:50.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a6659c700 1 -- 172.21.15.116:0/226102871 learned_addr learned my addr 172.21.15.116:0/226102871 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:50.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a5dd9b700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6813afa0 0x7f1a68151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:50.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a66d9d700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a6813b380 0x7f1a680c7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:50.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a6659c700 1 -- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6813afa0 msgr2=0x7f1a68151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:50.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a6659c700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6813afa0 0x7f1a68151300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a6659c700 1 -- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a6813b380 msgr2=0x7f1a680c7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:50.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a6659c700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a6813b380 0x7f1a680c7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a6659c700 1 -- 172.21.15.116:0/226102871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a680a4f00 con 0x7f1a680acd30 2020-11-20T12:21:50.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a6659c700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680acd30 0x7f1a680a58e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f1a5800ec90 tx=0x7f1a58008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:50.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a5effd700 1 -- 172.21.15.116:0/226102871 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5800c9d0 con 0x7f1a680acd30 2020-11-20T12:21:50.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a5effd700 1 -- 172.21.15.116:0/226102871 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5800cb30 con 0x7f1a680acd30 2020-11-20T12:21:50.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a5effd700 1 -- 172.21.15.116:0/226102871 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a58013430 con 0x7f1a680acd30 2020-11-20T12:21:50.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a6759e700 1 -- 172.21.15.116:0/226102871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a680a51a0 con 0x7f1a680acd30 2020-11-20T12:21:50.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a6759e700 1 -- 172.21.15.116:0/226102871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a68151870 con 0x7f1a680acd30 2020-11-20T12:21:50.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.612+0000 7f1a6759e700 1 -- 172.21.15.116:0/226102871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a680a5640 con 0x7f1a680acd30 2020-11-20T12:21:50.621 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.616+0000 7f1a5effd700 1 -- 172.21.15.116:0/226102871 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1a580137a0 con 0x7f1a680acd30 2020-11-20T12:21:50.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.616+0000 7f1a5effd700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a440557c0 0x7f1a44057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:50.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.616+0000 7f1a5effd700 1 -- 172.21.15.116:0/226102871 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f1a5806e5f0 con 0x7f1a680acd30 2020-11-20T12:21:50.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.616+0000 7f1a5dd9b700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a440557c0 0x7f1a44057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:50.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.616+0000 7f1a5dd9b700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a440557c0 0x7f1a44057c80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f1a68060e50 tx=0x7f1a50008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:50.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.620+0000 7f1a5effd700 1 -- 172.21.15.116:0/226102871 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1a680a5640 con 0x7f1a680acd30 2020-11-20T12:21:50.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.828+0000 7f1a6759e700 1 -- 172.21.15.116:0/226102871 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1a680a5640 con 0x7f1a680acd30 2020-11-20T12:21:50.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a5effd700 1 -- 172.21.15.116:0/226102871 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v31) v1 ==== 80+0+5399 (secure 0 0 0) 0x7f1a680a5640 con 0x7f1a680acd30 2020-11-20T12:21:50.835 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:50.835 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:21:50.835 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2020-11-20T12:21:50.835 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.836 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2020-11-20T12:21:50.836 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw==", 2020-11-20T12:21:50.836 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.836 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:50.837 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.837 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.837 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.837 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.838 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.838 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.838 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2020-11-20T12:21:50.838 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ==", 2020-11-20T12:21:50.839 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.839 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:50.839 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.839 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.839 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.840 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.840 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.840 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.840 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2020-11-20T12:21:50.841 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag==", 2020-11-20T12:21:50.841 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.841 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:50.841 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.842 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.842 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.842 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.842 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.842 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.843 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2020-11-20T12:21:50.843 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ==", 2020-11-20T12:21:50.843 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.843 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:50.843 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.844 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.844 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.844 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.844 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.845 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.845 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2020-11-20T12:21:50.845 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ==", 2020-11-20T12:21:50.845 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.845 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.846 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.846 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.846 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.846 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.847 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.847 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2020-11-20T12:21:50.847 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q==", 2020-11-20T12:21:50.853 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.853 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.854 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.854 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.854 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.854 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.854 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.855 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2020-11-20T12:21:50.855 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw==", 2020-11-20T12:21:50.855 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.855 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.856 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.856 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.856 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.856 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.856 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.857 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2020-11-20T12:21:50.857 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w==", 2020-11-20T12:21:50.857 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.857 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.858 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.858 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.858 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.858 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.858 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.859 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2020-11-20T12:21:50.859 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA==", 2020-11-20T12:21:50.859 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.859 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.860 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.860 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.860 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.860 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.860 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.861 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2020-11-20T12:21:50.861 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ==", 2020-11-20T12:21:50.861 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.861 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.862 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.862 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.862 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.862 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.862 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.863 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2020-11-20T12:21:50.863 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg==", 2020-11-20T12:21:50.863 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.863 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.863 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.864 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.864 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.864 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.864 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.865 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2020-11-20T12:21:50.865 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg==", 2020-11-20T12:21:50.865 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.865 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.865 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.866 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.866 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.866 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.866 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.866 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2020-11-20T12:21:50.867 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA==", 2020-11-20T12:21:50.867 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.867 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:50.867 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2020-11-20T12:21:50.868 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-20T12:21:50.868 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.868 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.868 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.868 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2020-11-20T12:21:50.869 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow==", 2020-11-20T12:21:50.869 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.869 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:50.869 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:21:50.869 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:21:50.870 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:21:50.870 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.870 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.870 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.871 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2020-11-20T12:21:50.871 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ==", 2020-11-20T12:21:50.871 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.871 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:50.871 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:21:50.872 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:21:50.872 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:21:50.872 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.872 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.873 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.873 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2020-11-20T12:21:50.873 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ==", 2020-11-20T12:21:50.873 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.873 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:21:50.874 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:21:50.874 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:21:50.874 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:21:50.874 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.874 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.875 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.875 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2020-11-20T12:21:50.875 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBDs7dfh+KKOBAAZvcPXWly9/mxRE7NWuR85A==", 2020-11-20T12:21:50.875 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.876 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:21:50.876 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:21:50.876 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.876 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.876 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.877 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.877 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.877 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.manila", 2020-11-20T12:21:50.877 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQDKtLdfEEDJERAAgK2ZvLfum7GYKOMoYIjP8Q==", 2020-11-20T12:21:50.877 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.878 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:21:50.878 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:21:50.878 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw" 2020-11-20T12:21:50.878 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.878 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.879 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.879 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2020-11-20T12:21:50.879 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A==", 2020-11-20T12:21:50.879 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.880 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:21:50.880 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:21:50.880 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.880 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.880 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:50.881 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:50.881 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2020-11-20T12:21:50.881 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA==", 2020-11-20T12:21:50.881 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:21:50.882 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:21:50.882 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:21:50.882 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:21:50.882 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.883 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:50.883 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:21:50.883 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:21:50.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 -- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a440557c0 msgr2=0x7f1a44057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:50.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a440557c0 0x7f1a44057c80 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f1a68060e50 tx=0x7f1a50008040).stop 2020-11-20T12:21:50.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 -- 172.21.15.116:0/226102871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680acd30 msgr2=0x7f1a680a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:50.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680acd30 0x7f1a680a58e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f1a5800ec90 tx=0x7f1a58008040).stop 2020-11-20T12:21:50.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 -- 172.21.15.116:0/226102871 shutdown_connections 2020-11-20T12:21:50.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1a440557c0 0x7f1a44057c80 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1a6813b380 0x7f1a680c7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1a680acd30 0x7f1a680a58e0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 --2- 172.21.15.116:0/226102871 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1a6813afa0 0x7f1a68151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:50.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 -- 172.21.15.116:0/226102871 >> 172.21.15.116:0/226102871 conn(0x7f1a680678a0 msgr2=0x7f1a680a9c20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:50.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 -- 172.21.15.116:0/226102871 shutdown_connections 2020-11-20T12:21:50.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:50.832+0000 7f1a6759e700 1 -- 172.21.15.116:0/226102871 wait complete. 2020-11-20T12:21:50.888 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.manila 2020-11-20T12:21:50.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2020-11-20T12:21:51.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.064+0000 7f355a5a2700 1 Processor -- start 2020-11-20T12:21:51.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.064+0000 7f355a5a2700 1 -- start start 2020-11-20T12:21:51.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f355a5a2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3554155db0 0x7f35540a67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f355a5a2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f35540a3f30 0x7f35540a4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f355a5a2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35540a6ce0 con 0x7f35541559d0 2020-11-20T12:21:51.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f355a5a2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35541564d0 con 0x7f35540a3f30 2020-11-20T12:21:51.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f355a5a2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3554156770 con 0x7f3554155db0 2020-11-20T12:21:51.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f3559da1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f35540a3f30 0x7f35540a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f3558d9f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3554155db0 0x7f35540a67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f3559da1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f35540a3f30 0x7f35540a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51344/0 (socket says 172.21.15.116:51344) 2020-11-20T12:21:51.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f3559da1700 1 -- 172.21.15.116:0/969213225 learned_addr learned my addr 172.21.15.116:0/969213225 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:51.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f35595a0700 1 --1- 172.21.15.116:0/969213225 >> v1:172.21.15.173:6789/0 conn(0x7f35541559d0 0x7f3554062750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:57004/0 (socket says 172.21.15.116:57004) 2020-11-20T12:21:51.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f3559da1700 1 -- 172.21.15.116:0/969213225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3554155db0 msgr2=0x7f35540a67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:51.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f3559da1700 1 --2- 172.21.15.116:0/969213225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3554155db0 0x7f35540a67f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f3559da1700 1 -- 172.21.15.116:0/969213225 >> v1:172.21.15.173:6789/0 conn(0x7f35541559d0 legacy=0x7f3554062750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.068+0000 7f3559da1700 1 -- 172.21.15.116:0/969213225 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3554156a10 con 0x7f35540a3f30 2020-11-20T12:21:51.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f3559da1700 1 --2- 172.21.15.116:0/969213225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35540a3f30 0x7f35540a4390 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7f354c009d50 tx=0x7f354c009290).ready entity=mon.0 client_cookie=e5ad334405505b62 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f3543fff700 1 -- 172.21.15.116:0/969213225 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c00bbc0 con 0x7f35540a3f30 2020-11-20T12:21:51.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f3543fff700 1 -- 172.21.15.116:0/969213225 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354c00bd20 con 0x7f35540a3f30 2020-11-20T12:21:51.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f3543fff700 1 -- 172.21.15.116:0/969213225 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354c011610 con 0x7f35540a3f30 2020-11-20T12:21:51.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f355a5a2700 1 -- 172.21.15.116:0/969213225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35540a3f30 msgr2=0x7f35540a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f355a5a2700 1 --2- 172.21.15.116:0/969213225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35540a3f30 0x7f35540a4390 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7f354c009d50 tx=0x7f354c009290).stop 2020-11-20T12:21:51.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f355a5a2700 1 -- 172.21.15.116:0/969213225 shutdown_connections 2020-11-20T12:21:51.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f355a5a2700 1 --2- 172.21.15.116:0/969213225 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35540a3f30 0x7f35540a4390 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f355a5a2700 1 --2- 172.21.15.116:0/969213225 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3554155db0 0x7f35540a67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f355a5a2700 1 -- 172.21.15.116:0/969213225 >> 172.21.15.116:0/969213225 conn(0x7f3554060170 msgr2=0x7f3554060580 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:51.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f355a5a2700 1 -- 172.21.15.116:0/969213225 shutdown_connections 2020-11-20T12:21:51.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.072+0000 7f355a5a2700 1 -- 172.21.15.116:0/969213225 wait complete. 2020-11-20T12:21:51.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.076+0000 7f355a5a2700 1 Processor -- start 2020-11-20T12:21:51.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.076+0000 7f355a5a2700 1 -- start start 2020-11-20T12:21:51.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.076+0000 7f355a5a2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35540a3f30 0x7f3554066740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.076+0000 7f355a5a2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35541559d0 0x7f355414f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.090 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.076+0000 7f355a5a2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3554155db0 0x7f355414fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.076+0000 7f355a5a2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35540c4770 con 0x7f35541559d0 2020-11-20T12:21:51.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.076+0000 7f355a5a2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35540c44d0 con 0x7f3554155db0 2020-11-20T12:21:51.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.076+0000 7f355a5a2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3554159740 con 0x7f35540a3f30 2020-11-20T12:21:51.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.076+0000 7f3558d9f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35541559d0 0x7f355414f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.076+0000 7f3558d9f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35541559d0 0x7f355414f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51348/0 (socket says 172.21.15.116:51348) 2020-11-20T12:21:51.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.076+0000 7f3558d9f700 1 -- 172.21.15.116:0/1231233394 learned_addr learned my addr 172.21.15.116:0/1231233394 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:51.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f3559da1700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3554155db0 0x7f355414fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f35595a0700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35540a3f30 0x7f3554066740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f3558d9f700 1 -- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35540a3f30 msgr2=0x7f3554066740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f3558d9f700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35540a3f30 0x7f3554066740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f3558d9f700 1 -- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3554155db0 msgr2=0x7f355414fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f3558d9f700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3554155db0 0x7f355414fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f3558d9f700 1 -- 172.21.15.116:0/1231233394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35541599e0 con 0x7f35541559d0 2020-11-20T12:21:51.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f3558d9f700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35541559d0 0x7f355414f780 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f354400eba0 tx=0x7f3544006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:51.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f35417fa700 1 -- 172.21.15.116:0/1231233394 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354400c8e0 con 0x7f35541559d0 2020-11-20T12:21:51.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f35417fa700 1 -- 172.21.15.116:0/1231233394 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f354400ca40 con 0x7f35541559d0 2020-11-20T12:21:51.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f355a5a2700 1 -- 172.21.15.116:0/1231233394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3554159c80 con 0x7f35541559d0 2020-11-20T12:21:51.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f35417fa700 1 -- 172.21.15.116:0/1231233394 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f354400cdc0 con 0x7f35541559d0 2020-11-20T12:21:51.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f355a5a2700 1 -- 172.21.15.116:0/1231233394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35540c4030 con 0x7f35541559d0 2020-11-20T12:21:51.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.080+0000 7f355a5a2700 1 -- 172.21.15.116:0/1231233394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3554159e10 con 0x7f35541559d0 2020-11-20T12:21:51.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.084+0000 7f35417fa700 1 -- 172.21.15.116:0/1231233394 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3544012600 con 0x7f35541559d0 2020-11-20T12:21:51.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.084+0000 7f35417fa700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f352c0557c0 0x7f352c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.084+0000 7f35417fa700 1 -- 172.21.15.116:0/1231233394 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f354406e830 con 0x7f35541559d0 2020-11-20T12:21:51.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.084+0000 7f35595a0700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f352c0557c0 0x7f352c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.084+0000 7f35595a0700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f352c0557c0 0x7f352c057c80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f35500045d0 tx=0x7f3550007b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:51.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.092+0000 7f35417fa700 1 -- 172.21.15.116:0/1231233394 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3544044890 con 0x7f35541559d0 2020-11-20T12:21:51.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.308+0000 7f355a5a2700 1 -- 172.21.15.116:0/1231233394 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.manila"} v 0) v1 -- 0x7f355405cd40 con 0x7f35541559d0 2020-11-20T12:21:51.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.316+0000 7f35417fa700 1 -- 172.21.15.116:0/1231233394 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.manila"}]=0 updated v32) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3544047eb0 con 0x7f35541559d0 2020-11-20T12:21:51.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.316+0000 7f355a5a2700 1 -- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f352c0557c0 msgr2=0x7f352c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.316+0000 7f355a5a2700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f352c0557c0 0x7f352c057c80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f35500045d0 tx=0x7f3550007b80).stop 2020-11-20T12:21:51.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.316+0000 7f355a5a2700 1 -- 172.21.15.116:0/1231233394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35541559d0 msgr2=0x7f355414f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.316+0000 7f355a5a2700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35541559d0 0x7f355414f780 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f354400eba0 tx=0x7f3544006df0).stop 2020-11-20T12:21:51.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.320+0000 7f355a5a2700 1 -- 172.21.15.116:0/1231233394 shutdown_connections 2020-11-20T12:21:51.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.320+0000 7f355a5a2700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f352c0557c0 0x7f352c057c80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.320+0000 7f355a5a2700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3554155db0 0x7f355414fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.320+0000 7f355a5a2700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35540a3f30 0x7f3554066740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.320+0000 7f355a5a2700 1 --2- 172.21.15.116:0/1231233394 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35541559d0 0x7f355414f780 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.320+0000 7f355a5a2700 1 -- 172.21.15.116:0/1231233394 >> 172.21.15.116:0/1231233394 conn(0x7f3554060170 msgr2=0x7f3554158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:51.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.320+0000 7f355a5a2700 1 -- 172.21.15.116:0/1231233394 shutdown_connections 2020-11-20T12:21:51.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.320+0000 7f355a5a2700 1 -- 172.21.15.116:0/1231233394 wait complete. 2020-11-20T12:21:51.329 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:21:51.350 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:51.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.520+0000 7f8e616b9700 1 Processor -- start 2020-11-20T12:21:51.526 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e616b9700 1 -- start start 2020-11-20T12:21:51.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e616b9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c14e480 0x7f8e5c1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.527 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e616b9700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8e5c066800 0x7f8e5c0677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e616b9700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e5c0c5010 con 0x7f8e5c0663a0 2020-11-20T12:21:51.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e616b9700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e5c067340 con 0x7f8e5c066800 2020-11-20T12:21:51.528 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e616b9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c066e00 con 0x7f8e5c14e480 2020-11-20T12:21:51.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e60eb8700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8e5c066800 0x7f8e5c0677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e5bfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c14e480 0x7f8e5c1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e60eb8700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8e5c066800 0x7f8e5c0677a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:50408/0 (socket says 172.21.15.116:50408) 2020-11-20T12:21:51.529 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e60eb8700 1 -- 172.21.15.116:0/1327805512 learned_addr learned my addr 172.21.15.116:0/1327805512 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:51.530 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e60eb8700 1 -- 172.21.15.116:0/1327805512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c14e480 msgr2=0x7f8e5c1529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:51.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.524+0000 7f8e60eb8700 1 --2- 172.21.15.116:0/1327805512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c14e480 0x7f8e5c1529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e60eb8700 1 -- 172.21.15.116:0/1327805512 >> v1:172.21.15.173:6789/0 conn(0x7f8e5c0663a0 legacy=0x7f8e5c0a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e5affd700 1 -- 172.21.15.116:0/1327805512 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1508158416 0 0) 0x7f8e5c0c5010 con 0x7f8e5c0663a0 2020-11-20T12:21:51.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e60eb8700 1 -- 172.21.15.116:0/1327805512 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c0670a0 con 0x7f8e5c066800 2020-11-20T12:21:51.531 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e60eb8700 1 --2- 172.21.15.116:0/1327805512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c066800 0x7f8e5c0677a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f8e5400d2f0 tx=0x7f8e5400b530).ready entity=mon.1 client_cookie=432f86dfbdc2a69e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:51.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e5affd700 1 -- 172.21.15.116:0/1327805512 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54010470 con 0x7f8e5c066800 2020-11-20T12:21:51.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e5affd700 1 -- 172.21.15.116:0/1327805512 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e540105d0 con 0x7f8e5c066800 2020-11-20T12:21:51.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e5affd700 1 -- 172.21.15.116:0/1327805512 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e54010950 con 0x7f8e5c066800 2020-11-20T12:21:51.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e616b9700 1 -- 172.21.15.116:0/1327805512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c066800 msgr2=0x7f8e5c0677a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e616b9700 1 --2- 172.21.15.116:0/1327805512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c066800 0x7f8e5c0677a0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f8e5400d2f0 tx=0x7f8e5400b530).stop 2020-11-20T12:21:51.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e616b9700 1 -- 172.21.15.116:0/1327805512 shutdown_connections 2020-11-20T12:21:51.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e616b9700 1 --2- 172.21.15.116:0/1327805512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c066800 0x7f8e5c0677a0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e616b9700 1 --2- 172.21.15.116:0/1327805512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c14e480 0x7f8e5c1529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e616b9700 1 -- 172.21.15.116:0/1327805512 >> 172.21.15.116:0/1327805512 conn(0x7f8e5c060c40 msgr2=0x7f8e5c0c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:51.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e616b9700 1 -- 172.21.15.116:0/1327805512 shutdown_connections 2020-11-20T12:21:51.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.528+0000 7f8e616b9700 1 -- 172.21.15.116:0/1327805512 wait complete. 2020-11-20T12:21:51.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.532+0000 7f8e616b9700 1 Processor -- start 2020-11-20T12:21:51.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.532+0000 7f8e616b9700 1 -- start start 2020-11-20T12:21:51.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.532+0000 7f8e616b9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0663a0 0x7f8e5c15d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.532+0000 7f8e616b9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c066800 0x7f8e5c0c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e616b9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c14e480 0x7f8e5c160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e616b9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0a9550 con 0x7f8e5c0663a0 2020-11-20T12:21:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e616b9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0a97f0 con 0x7f8e5c14e480 2020-11-20T12:21:51.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e616b9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e5c0a9a90 con 0x7f8e5c066800 2020-11-20T12:21:51.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e5bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0663a0 0x7f8e5c15d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e5bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0663a0 0x7f8e5c15d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51360/0 (socket says 172.21.15.116:51360) 2020-11-20T12:21:51.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e5bfff700 1 -- 172.21.15.116:0/2520005763 learned_addr learned my addr 172.21.15.116:0/2520005763 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:51.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e60eb8700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c14e480 0x7f8e5c160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e5b7fe700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c066800 0x7f8e5c0c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e5bfff700 1 -- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c066800 msgr2=0x7f8e5c0c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e5bfff700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c066800 0x7f8e5c0c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e5bfff700 1 -- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c14e480 msgr2=0x7f8e5c160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e5bfff700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c14e480 0x7f8e5c160220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e5bfff700 1 -- 172.21.15.116:0/2520005763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5c0a9d30 con 0x7f8e5c0663a0 2020-11-20T12:21:51.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e5b7fe700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c066800 0x7f8e5c0c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:51.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e60eb8700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c14e480 0x7f8e5c160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:51.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e5bfff700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0663a0 0x7f8e5c15d6d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f8e4c00eba0 tx=0x7f8e4c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:51.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e3bfff700 1 -- 172.21.15.116:0/2520005763 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00b9a0 con 0x7f8e5c0663a0 2020-11-20T12:21:51.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e3bfff700 1 -- 172.21.15.116:0/2520005763 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c00bb00 con 0x7f8e5c0663a0 2020-11-20T12:21:51.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e616b9700 1 -- 172.21.15.116:0/2520005763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e5c0a9fd0 con 0x7f8e5c0663a0 2020-11-20T12:21:51.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e3bfff700 1 -- 172.21.15.116:0/2520005763 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c00bea0 con 0x7f8e5c0663a0 2020-11-20T12:21:51.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e616b9700 1 -- 172.21.15.116:0/2520005763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e5c0a7550 con 0x7f8e5c0663a0 2020-11-20T12:21:51.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.536+0000 7f8e616b9700 1 -- 172.21.15.116:0/2520005763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5c05cf30 con 0x7f8e5c0663a0 2020-11-20T12:21:51.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.540+0000 7f8e3bfff700 1 -- 172.21.15.116:0/2520005763 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8e4c012600 con 0x7f8e5c0663a0 2020-11-20T12:21:51.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.540+0000 7f8e3bfff700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e340557f0 0x7f8e34057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.540+0000 7f8e3bfff700 1 -- 172.21.15.116:0/2520005763 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f8e4c06e8c0 con 0x7f8e5c0663a0 2020-11-20T12:21:51.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.540+0000 7f8e5b7fe700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e340557f0 0x7f8e34057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.544+0000 7f8e5b7fe700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e340557f0 0x7f8e34057cb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f8e5c0c3ce0 tx=0x7f8e50009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:51.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.548+0000 7f8e3bfff700 1 -- 172.21.15.116:0/2520005763 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8e4c044980 con 0x7f8e5c0663a0 2020-11-20T12:21:51.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.708+0000 7f8e616b9700 1 -- 172.21.15.116:0/2520005763 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e5c061230 con 0x7f8e5c0663a0 2020-11-20T12:21:51.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.708+0000 7f8e3bfff700 1 -- 172.21.15.116:0/2520005763 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+13733 (secure 0 0 0) 0x7f8e4c047fa0 con 0x7f8e5c0663a0 2020-11-20T12:21:51.712 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:51.713 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":113,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:50.388421+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":101,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:51.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 -- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e340557f0 msgr2=0x7f8e34057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e340557f0 0x7f8e34057cb0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7f8e5c0c3ce0 tx=0x7f8e50009040).stop 2020-11-20T12:21:51.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 -- 172.21.15.116:0/2520005763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0663a0 msgr2=0x7f8e5c15d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0663a0 0x7f8e5c15d6d0 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f8e4c00eba0 tx=0x7f8e4c006df0).stop 2020-11-20T12:21:51.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 -- 172.21.15.116:0/2520005763 shutdown_connections 2020-11-20T12:21:51.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8e340557f0 0x7f8e34057cb0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8e5c14e480 0x7f8e5c160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8e5c0663a0 0x7f8e5c15d6d0 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 --2- 172.21.15.116:0/2520005763 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8e5c066800 0x7f8e5c0c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 -- 172.21.15.116:0/2520005763 >> 172.21.15.116:0/2520005763 conn(0x7f8e5c060c40 msgr2=0x7f8e5c150870 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:51.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 -- 172.21.15.116:0/2520005763 shutdown_connections 2020-11-20T12:21:51.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.712+0000 7f8e616b9700 1 -- 172.21.15.116:0/2520005763 wait complete. 2020-11-20T12:21:51.742 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T12:21:51.743 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:21:51.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.904+0000 7f38111fd700 1 Processor -- start 2020-11-20T12:21:51.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.904+0000 7f38111fd700 1 -- start start 2020-11-20T12:21:51.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f38111fd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f380c060310 0x7f380c0606f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f38111fd700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380c0a67e0 con 0x7f380c066d90 2020-11-20T12:21:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f38111fd700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f380c0a5fd0 con 0x7f380c0aa0d0 2020-11-20T12:21:51.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f38111fd700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f380c0a6270 con 0x7f380c060310 2020-11-20T12:21:51.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f380ad9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f380c060310 0x7f380c0606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f380a59c700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f380c0aa0d0 0x7f380c066680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:57032/0 (socket says 172.21.15.116:57032) 2020-11-20T12:21:51.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f380a59c700 1 -- 172.21.15.116:0/916817121 learned_addr learned my addr 172.21.15.116:0/916817121 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:51.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f380ad9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f380c060310 0x7f380c0606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51370/0 (socket says 172.21.15.116:51370) 2020-11-20T12:21:51.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f380ad9d700 1 -- 172.21.15.116:0/916817121 >> v1:172.21.15.173:6789/0 conn(0x7f380c0aa0d0 legacy=0x7f380c066680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f380ad9d700 1 -- 172.21.15.116:0/916817121 >> v1:172.21.15.116:6789/0 conn(0x7f380c066d90 legacy=0x7f380c159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f380ad9d700 1 -- 172.21.15.116:0/916817121 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c0a6a40 con 0x7f380c060310 2020-11-20T12:21:51.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.908+0000 7f380ad9d700 1 --2- 172.21.15.116:0/916817121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c060310 0x7f380c0606f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f37fc00b9a0 tx=0x7f37fc006df0).ready entity=mon.0 client_cookie=9701f072a49cc0af server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:51.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.912+0000 7f3809d9b700 1 -- 172.21.15.116:0/916817121 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc0097b0 con 0x7f380c060310 2020-11-20T12:21:51.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.912+0000 7f3809d9b700 1 -- 172.21.15.116:0/916817121 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37fc009910 con 0x7f380c060310 2020-11-20T12:21:51.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.912+0000 7f3809d9b700 1 -- 172.21.15.116:0/916817121 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37fc009c90 con 0x7f380c060310 2020-11-20T12:21:51.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.912+0000 7f38111fd700 1 -- 172.21.15.116:0/916817121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c060310 msgr2=0x7f380c0606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.912+0000 7f38111fd700 1 --2- 172.21.15.116:0/916817121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c060310 0x7f380c0606f0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f37fc00b9a0 tx=0x7f37fc006df0).stop 2020-11-20T12:21:51.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.912+0000 7f38111fd700 1 -- 172.21.15.116:0/916817121 shutdown_connections 2020-11-20T12:21:51.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.912+0000 7f38111fd700 1 --2- 172.21.15.116:0/916817121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c060310 0x7f380c0606f0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.912+0000 7f38111fd700 1 -- 172.21.15.116:0/916817121 >> 172.21.15.116:0/916817121 conn(0x7f380c05ba80 msgr2=0x7f380c0c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:51.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.912+0000 7f38111fd700 1 -- 172.21.15.116:0/916817121 shutdown_connections 2020-11-20T12:21:51.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.912+0000 7f38111fd700 1 -- 172.21.15.116:0/916817121 wait complete. 2020-11-20T12:21:51.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.916+0000 7f38111fd700 1 Processor -- start 2020-11-20T12:21:51.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.916+0000 7f38111fd700 1 -- start start 2020-11-20T12:21:51.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.916+0000 7f38111fd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f380c060310 0x7f380c158c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.916+0000 7f38111fd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c066d90 0x7f380c0c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.916+0000 7f38111fd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380c0aa0d0 0x7f380c150110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.916+0000 7f38111fd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c159460 con 0x7f380c066d90 2020-11-20T12:21:51.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.916+0000 7f38111fd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f380c165a00 con 0x7f380c0aa0d0 2020-11-20T12:21:51.931 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.916+0000 7f38111fd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f380c165ca0 con 0x7f380c060310 2020-11-20T12:21:51.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f380a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c066d90 0x7f380c0c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.932 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f380a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c066d90 0x7f380c0c4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51378/0 (socket says 172.21.15.116:51378) 2020-11-20T12:21:51.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f380b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380c0aa0d0 0x7f380c150110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f380a59c700 1 -- 172.21.15.116:0/1135274223 learned_addr learned my addr 172.21.15.116:0/1135274223 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:51.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f380ad9d700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f380c060310 0x7f380c158c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.934 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f380a59c700 1 -- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f380c060310 msgr2=0x7f380c158c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f380a59c700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f380c060310 0x7f380c158c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f380a59c700 1 -- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380c0aa0d0 msgr2=0x7f380c150110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:51.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f380a59c700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380c0aa0d0 0x7f380c150110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:51.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f380a59c700 1 -- 172.21.15.116:0/1135274223 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f380c165f40 con 0x7f380c066d90 2020-11-20T12:21:51.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f380a59c700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c066d90 0x7f380c0c4100 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f380000c710 tx=0x7f3800007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:51.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f37f2ffd700 1 -- 172.21.15.116:0/1135274223 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38000072e0 con 0x7f380c066d90 2020-11-20T12:21:51.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f37f2ffd700 1 -- 172.21.15.116:0/1135274223 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800007440 con 0x7f380c066d90 2020-11-20T12:21:51.936 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f37f2ffd700 1 -- 172.21.15.116:0/1135274223 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800005800 con 0x7f380c066d90 2020-11-20T12:21:51.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f38111fd700 1 -- 172.21.15.116:0/1135274223 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f380c1661e0 con 0x7f380c066d90 2020-11-20T12:21:51.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f38111fd700 1 -- 172.21.15.116:0/1135274223 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f380c166680 con 0x7f380c066d90 2020-11-20T12:21:51.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.920+0000 7f38111fd700 1 -- 172.21.15.116:0/1135274223 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f380c05d2a0 con 0x7f380c066d90 2020-11-20T12:21:51.938 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.924+0000 7f37f2ffd700 1 -- 172.21.15.116:0/1135274223 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3800010070 con 0x7f380c066d90 2020-11-20T12:21:51.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.924+0000 7f37f2ffd700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37e80557c0 0x7f37e8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:51.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.924+0000 7f37f2ffd700 1 -- 172.21.15.116:0/1135274223 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f380006d900 con 0x7f380c066d90 2020-11-20T12:21:51.939 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.924+0000 7f380ad9d700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37e80557c0 0x7f37e8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:51.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.928+0000 7f380ad9d700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37e80557c0 0x7f37e8057c80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f37fc00bf10 tx=0x7f37fc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:51.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:51.928+0000 7f37f2ffd700 1 -- 172.21.15.116:0/1135274223 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3800043ad0 con 0x7f380c066d90 2020-11-20T12:21:52.091 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.088+0000 7f38111fd700 1 -- 172.21.15.116:0/1135274223 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f380c061380 con 0x7f380c066d90 2020-11-20T12:21:52.092 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.088+0000 7f37f2ffd700 1 -- 172.21.15.116:0/1135274223 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v113) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3800007db0 con 0x7f380c066d90 2020-11-20T12:21:52.093 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:21:52.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 -- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37e80557c0 msgr2=0x7f37e8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37e80557c0 0x7f37e8057c80 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f37fc00bf10 tx=0x7f37fc015000).stop 2020-11-20T12:21:52.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 -- 172.21.15.116:0/1135274223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c066d90 msgr2=0x7f380c0c4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c066d90 0x7f380c0c4100 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f380000c710 tx=0x7f3800007c70).stop 2020-11-20T12:21:52.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 -- 172.21.15.116:0/1135274223 shutdown_connections 2020-11-20T12:21:52.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f37e80557c0 0x7f37e8057c80 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f380c0aa0d0 0x7f380c150110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f380c060310 0x7f380c158c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 --2- 172.21.15.116:0/1135274223 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f380c066d90 0x7f380c0c4100 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 -- 172.21.15.116:0/1135274223 >> 172.21.15.116:0/1135274223 conn(0x7f380c05ba80 msgr2=0x7f380c152240 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:52.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 -- 172.21.15.116:0/1135274223 shutdown_connections 2020-11-20T12:21:52.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.092+0000 7f38111fd700 1 -- 172.21.15.116:0/1135274223 wait complete. 2020-11-20T12:21:52.119 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:21:52.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.292+0000 7f7f2a8d0700 1 Processor -- start 2020-11-20T12:21:52.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.292+0000 7f7f2a8d0700 1 -- start start 2020-11-20T12:21:52.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.296+0000 7f7f2a8d0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 0x7f7f240bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:52.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.296+0000 7f7f2a8d0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7f24151cd0 0x7f7f24066480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:52.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.296+0000 7f7f2a8d0700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f240bf510 con 0x7f7f240b93c0 2020-11-20T12:21:52.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.296+0000 7f7f2a8d0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f240bf7b0 con 0x7f7f24151cd0 2020-11-20T12:21:52.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.296+0000 7f7f2a8d0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f240bfa50 con 0x7f7f241518f0 2020-11-20T12:21:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.296+0000 7f7f290cd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7f24151cd0 0x7f7f24066480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:52.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.296+0000 7f7f298ce700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 0x7f7f240bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:52.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.296+0000 7f7f298ce700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 0x7f7f240bed20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:60332/0 (socket says 172.21.15.116:60332) 2020-11-20T12:21:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.296+0000 7f7f290cd700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f7f24151cd0 0x7f7f24066480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:50436/0 (socket says 172.21.15.116:50436) 2020-11-20T12:21:52.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.296+0000 7f7f298ce700 1 -- 172.21.15.116:0/2794675895 learned_addr learned my addr 172.21.15.116:0/2794675895 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f298ce700 1 -- 172.21.15.116:0/2794675895 >> v2:172.21.15.173:3300/0 conn(0x7f7f24151cd0 msgr2=0x7f7f24066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:52.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f298ce700 1 --2- 172.21.15.116:0/2794675895 >> v2:172.21.15.173:3300/0 conn(0x7f7f24151cd0 0x7f7f24066480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f298ce700 1 -- 172.21.15.116:0/2794675895 >> v1:172.21.15.173:6789/0 conn(0x7f7f240b93c0 legacy=0x7f7f24060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f298ce700 1 -- 172.21.15.116:0/2794675895 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f240bfcf0 con 0x7f7f241518f0 2020-11-20T12:21:52.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f298ce700 1 --2- 172.21.15.116:0/2794675895 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 0x7f7f240bed20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f7f20006fd0 tx=0x7f7f20006eb0).ready entity=mon.2 client_cookie=eabe4a233af51a0a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:52.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f288cc700 1 -- 172.21.15.116:0/2794675895 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20009810 con 0x7f7f241518f0 2020-11-20T12:21:52.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f288cc700 1 -- 172.21.15.116:0/2794675895 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f20009970 con 0x7f7f241518f0 2020-11-20T12:21:52.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f288cc700 1 -- 172.21.15.116:0/2794675895 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20009cf0 con 0x7f7f241518f0 2020-11-20T12:21:52.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/2794675895 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 msgr2=0x7f7f240bed20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f2a8d0700 1 --2- 172.21.15.116:0/2794675895 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 0x7f7f240bed20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f7f20006fd0 tx=0x7f7f20006eb0).stop 2020-11-20T12:21:52.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/2794675895 shutdown_connections 2020-11-20T12:21:52.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f2a8d0700 1 --2- 172.21.15.116:0/2794675895 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 0x7f7f240bed20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f2a8d0700 1 --2- 172.21.15.116:0/2794675895 >> v2:172.21.15.173:3300/0 conn(0x7f7f24151cd0 0x7f7f24066480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.300+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/2794675895 >> 172.21.15.116:0/2794675895 conn(0x7f7f24068350 msgr2=0x7f7f240627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.304+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/2794675895 shutdown_connections 2020-11-20T12:21:52.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.304+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/2794675895 wait complete. 2020-11-20T12:21:52.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.304+0000 7f7f2a8d0700 1 Processor -- start 2020-11-20T12:21:52.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.304+0000 7f7f2a8d0700 1 -- start start 2020-11-20T12:21:52.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f2a8d0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f240b93c0 0x7f7f240a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:52.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f2a8d0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 0x7f7f24166880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:52.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f2a8d0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f24151cd0 0x7f7f240a7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:52.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f2a8d0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f241530a0 con 0x7f7f240b93c0 2020-11-20T12:21:52.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f2a8d0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f241532d0 con 0x7f7f24151cd0 2020-11-20T12:21:52.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f2a8d0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f24153570 con 0x7f7f241518f0 2020-11-20T12:21:52.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f298ce700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f240b93c0 0x7f7f240a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:52.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f298ce700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f240b93c0 0x7f7f240a9c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51388/0 (socket says 172.21.15.116:51388) 2020-11-20T12:21:52.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f298ce700 1 -- 172.21.15.116:0/3876963660 learned_addr learned my addr 172.21.15.116:0/3876963660 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:52.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f290cd700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 0x7f7f24166880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:52.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f2a0cf700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f24151cd0 0x7f7f240a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:52.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f298ce700 1 -- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 msgr2=0x7f7f24166880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f298ce700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 0x7f7f24166880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f298ce700 1 -- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f24151cd0 msgr2=0x7f7f240a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f298ce700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f24151cd0 0x7f7f240a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f298ce700 1 -- 172.21.15.116:0/3876963660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f24153810 con 0x7f7f240b93c0 2020-11-20T12:21:52.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f298ce700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f240b93c0 0x7f7f240a9c10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f7f240bfb90 tx=0x7f7f20015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:52.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f11ffb700 1 -- 172.21.15.116:0/3876963660 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f20011e50 con 0x7f7f240b93c0 2020-11-20T12:21:52.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f11ffb700 1 -- 172.21.15.116:0/3876963660 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f20006b40 con 0x7f7f240b93c0 2020-11-20T12:21:52.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f11ffb700 1 -- 172.21.15.116:0/3876963660 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2001e6b0 con 0x7f7f240b93c0 2020-11-20T12:21:52.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/3876963660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f24153ab0 con 0x7f7f240b93c0 2020-11-20T12:21:52.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.308+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/3876963660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f24166d70 con 0x7f7f240b93c0 2020-11-20T12:21:52.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.312+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/3876963660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2405cf30 con 0x7f7f240b93c0 2020-11-20T12:21:52.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.312+0000 7f7f11ffb700 1 -- 172.21.15.116:0/3876963660 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7f20025030 con 0x7f7f240b93c0 2020-11-20T12:21:52.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.312+0000 7f7f11ffb700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7efc0557c0 0x7f7efc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:52.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.312+0000 7f7f11ffb700 1 -- 172.21.15.116:0/3876963660 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f7f2006fb00 con 0x7f7f240b93c0 2020-11-20T12:21:52.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.312+0000 7f7f290cd700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7efc0557c0 0x7f7efc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:52.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.316+0000 7f7f290cd700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7efc0557c0 0x7f7efc057c80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f7f24168c80 tx=0x7f7f14008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:52.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.320+0000 7f7f11ffb700 1 -- 172.21.15.116:0/3876963660 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7f20071e60 con 0x7f7f240b93c0 2020-11-20T12:21:52.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.476+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/3876963660 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7f240639c0 con 0x7f7f240b93c0 2020-11-20T12:21:52.483 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.480+0000 7f7f11ffb700 1 -- 172.21.15.116:0/3876963660 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7f20023030 con 0x7f7f240b93c0 2020-11-20T12:21:52.483 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:21:52.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7efc0557c0 msgr2=0x7f7efc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7efc0557c0 0x7f7efc057c80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f7f24168c80 tx=0x7f7f14008040).stop 2020-11-20T12:21:52.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/3876963660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f240b93c0 msgr2=0x7f7f240a9c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.488 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f240b93c0 0x7f7f240a9c10 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f7f240bfb90 tx=0x7f7f20015000).stop 2020-11-20T12:21:52.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/3876963660 shutdown_connections 2020-11-20T12:21:52.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f7efc0557c0 0x7f7efc057c80 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.489 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f7f24151cd0 0x7f7f240a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7f240b93c0 0x7f7f240a9c10 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 --2- 172.21.15.116:0/3876963660 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f7f241518f0 0x7f7f24166880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/3876963660 >> 172.21.15.116:0/3876963660 conn(0x7f7f24068350 msgr2=0x7f7f240b9960 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:52.490 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/3876963660 shutdown_connections 2020-11-20T12:21:52.491 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.484+0000 7f7f2a8d0700 1 -- 172.21.15.116:0/3876963660 wait complete. 2020-11-20T12:21:52.514 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-20T12:21:52.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.700+0000 7f8cc2bc1700 1 Processor -- start 2020-11-20T12:21:52.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.700+0000 7f8cc2bc1700 1 -- start start 2020-11-20T12:21:52.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.700+0000 7f8cc2bc1700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8cbc0c3c90 0x7f8cbc0c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:52.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.700+0000 7f8cc2bc1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc0c3600 con 0x7f8cbc151e90 2020-11-20T12:21:52.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.700+0000 7f8cc2bc1700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cbc0c38a0 con 0x7f8cbc0bf020 2020-11-20T12:21:52.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.700+0000 7f8cc2bc1700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cbc0c72a0 con 0x7f8cbc0c3c90 2020-11-20T12:21:52.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc095d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8cbc0c3c90 0x7f8cbc0c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:52.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cbbfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f8cbc151e90 0x7f8cbc0c79e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60834/0 (socket says 172.21.15.116:60834) 2020-11-20T12:21:52.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cbbfff700 1 -- 172.21.15.116:0/2271321391 learned_addr learned my addr 172.21.15.116:0/2271321391 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:52.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cbb7fe700 1 -- 172.21.15.116:0/2271321391 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1509326436 0 0) 0x7f8cbc0c3600 con 0x7f8cbc151e90 2020-11-20T12:21:52.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cbb7fe700 1 -- 172.21.15.116:0/2271321391 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ca0003240 con 0x7f8cbc151e90 2020-11-20T12:21:52.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc095d700 1 -- 172.21.15.116:0/2271321391 >> v1:172.21.15.173:6789/0 conn(0x7f8cbc0bf020 legacy=0x7f8cbc0c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc095d700 1 -- 172.21.15.116:0/2271321391 >> v1:172.21.15.116:6789/0 conn(0x7f8cbc151e90 legacy=0x7f8cbc0c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc095d700 1 -- 172.21.15.116:0/2271321391 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc0c7540 con 0x7f8cbc0c3c90 2020-11-20T12:21:52.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc095d700 1 --2- 172.21.15.116:0/2271321391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cbc0c3c90 0x7f8cbc0c9310 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f8cb000b990 tx=0x7f8cb0006d90).ready entity=mon.1 client_cookie=41d89d0fe0fd9f1e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:52.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cbb7fe700 1 -- 172.21.15.116:0/2271321391 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0006030 con 0x7f8cbc0c3c90 2020-11-20T12:21:52.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cbb7fe700 1 -- 172.21.15.116:0/2271321391 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb0006190 con 0x7f8cbc0c3c90 2020-11-20T12:21:52.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cbb7fe700 1 -- 172.21.15.116:0/2271321391 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0006510 con 0x7f8cbc0c3c90 2020-11-20T12:21:52.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/2271321391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cbc0c3c90 msgr2=0x7f8cbc0c9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc2bc1700 1 --2- 172.21.15.116:0/2271321391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cbc0c3c90 0x7f8cbc0c9310 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f8cb000b990 tx=0x7f8cb0006d90).stop 2020-11-20T12:21:52.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/2271321391 shutdown_connections 2020-11-20T12:21:52.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc2bc1700 1 --2- 172.21.15.116:0/2271321391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cbc0c3c90 0x7f8cbc0c9310 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/2271321391 >> 172.21.15.116:0/2271321391 conn(0x7f8cbc068080 msgr2=0x7f8cbc0c88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:52.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/2271321391 shutdown_connections 2020-11-20T12:21:52.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.704+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/2271321391 wait complete. 2020-11-20T12:21:52.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.708+0000 7f8cc2bc1700 1 Processor -- start 2020-11-20T12:21:52.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.708+0000 7f8cc2bc1700 1 -- start start 2020-11-20T12:21:52.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc2bc1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cbc0bf020 0x7f8cbc0935e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:52.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc2bc1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cbc0c3c90 0x7f8cbc0ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:52.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc2bc1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cbc151e90 0x7f8cbc14f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:52.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc2bc1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0aded0 con 0x7f8cbc0bf020 2020-11-20T12:21:52.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc2bc1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0ae0f0 con 0x7f8cbc0c3c90 2020-11-20T12:21:52.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc2bc1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0ae390 con 0x7f8cbc151e90 2020-11-20T12:21:52.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc095d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cbc0bf020 0x7f8cbc0935e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:52.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc095d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cbc0bf020 0x7f8cbc0935e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51402/0 (socket says 172.21.15.116:51402) 2020-11-20T12:21:52.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc095d700 1 -- 172.21.15.116:0/4029863735 learned_addr learned my addr 172.21.15.116:0/4029863735 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:52.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cbbfff700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cbc0c3c90 0x7f8cbc0ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:52.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc115e700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cbc151e90 0x7f8cbc14f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:52.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc095d700 1 -- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cbc151e90 msgr2=0x7f8cbc14f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc095d700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cbc151e90 0x7f8cbc14f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc095d700 1 -- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cbc0c3c90 msgr2=0x7f8cbc0ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc095d700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cbc0c3c90 0x7f8cbc0ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc095d700 1 -- 172.21.15.116:0/4029863735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc0ae630 con 0x7f8cbc0bf020 2020-11-20T12:21:52.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc115e700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cbc151e90 0x7f8cbc14f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:21:52.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.712+0000 7f8cc095d700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cbc0bf020 0x7f8cbc0935e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f8cbc081230 tx=0x7f8cb0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:52.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.716+0000 7f8cb8ff9700 1 -- 172.21.15.116:0/4029863735 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0021070 con 0x7f8cbc0bf020 2020-11-20T12:21:52.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.716+0000 7f8cb8ff9700 1 -- 172.21.15.116:0/4029863735 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb0005ab0 con 0x7f8cbc0bf020 2020-11-20T12:21:52.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.716+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/4029863735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc168f40 con 0x7f8cbc0bf020 2020-11-20T12:21:52.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.716+0000 7f8cb8ff9700 1 -- 172.21.15.116:0/4029863735 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0005e30 con 0x7f8cbc0bf020 2020-11-20T12:21:52.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.716+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/4029863735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc1693e0 con 0x7f8cbc0bf020 2020-11-20T12:21:52.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.716+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/4029863735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cbc05d2a0 con 0x7f8cbc0bf020 2020-11-20T12:21:52.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.720+0000 7f8cb8ff9700 1 -- 172.21.15.116:0/4029863735 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8cb000f070 con 0x7f8cbc0bf020 2020-11-20T12:21:52.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.720+0000 7f8cb8ff9700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c940557c0 0x7f8c94057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:52.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.720+0000 7f8cb8ff9700 1 -- 172.21.15.116:0/4029863735 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f8cb0070a60 con 0x7f8cbc0bf020 2020-11-20T12:21:52.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.720+0000 7f8cbbfff700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c940557c0 0x7f8c94057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:52.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.724+0000 7f8cbbfff700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c940557c0 0x7f8c94057c80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f8cbc0aead0 tx=0x7f8ca8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:52.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.724+0000 7f8cb8ff9700 1 -- 172.21.15.116:0/4029863735 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8cb004a5d0 con 0x7f8cbc0bf020 2020-11-20T12:21:52.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.888+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/4029863735 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8cbc060290 con 0x7f8cbc0bf020 2020-11-20T12:21:52.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.888+0000 7f8cb8ff9700 1 -- 172.21.15.116:0/4029863735 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) v1 ==== 84+0+143 (secure 0 0 0) 0x7f8cb004a1c0 con 0x7f8cbc0bf020 2020-11-20T12:21:52.892 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:52.892 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2020-11-20T12:21:52.893 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:52.893 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2020-11-20T12:21:52.893 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "device_health_metrics" 2020-11-20T12:21:52.893 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:21:52.894 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:21:52.894 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 2, 2020-11-20T12:21:52.894 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "rbd" 2020-11-20T12:21:52.894 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:21:52.895 INFO:teuthology.orchestra.run.smithi116.stdout:] 2020-11-20T12:21:52.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c940557c0 msgr2=0x7f8c94057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c940557c0 0x7f8c94057c80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f8cbc0aead0 tx=0x7f8ca8009040).stop 2020-11-20T12:21:52.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/4029863735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cbc0bf020 msgr2=0x7f8cbc0935e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:52.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cbc0bf020 0x7f8cbc0935e0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f8cbc081230 tx=0x7f8cb0015000).stop 2020-11-20T12:21:52.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/4029863735 shutdown_connections 2020-11-20T12:21:52.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8c940557c0 0x7f8c94057c80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8cbc151e90 0x7f8cbc14f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8cbc0bf020 0x7f8cbc0935e0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 --2- 172.21.15.116:0/4029863735 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8cbc0c3c90 0x7f8cbc0ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:52.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/4029863735 >> 172.21.15.116:0/4029863735 conn(0x7f8cbc068080 msgr2=0x7f8cbc061fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:52.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/4029863735 shutdown_connections 2020-11-20T12:21:52.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:52.892+0000 7f8cc2bc1700 1 -- 172.21.15.116:0/4029863735 wait complete. 2020-11-20T12:21:52.919 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T12:21:52.919 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-20T12:21:53.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.092+0000 7fca83b25700 1 Processor -- start 2020-11-20T12:21:53.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.092+0000 7fca83b25700 1 -- start start 2020-11-20T12:21:53.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca83b25700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c14bee0 0x7fca7c152990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca83b25700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca7c0abc10 con 0x7fca7c0616b0 2020-11-20T12:21:53.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca83b25700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca7c0acf60 con 0x7fca7c0c8fa0 2020-11-20T12:21:53.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca83b25700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c152e80 con 0x7fca7c14bee0 2020-11-20T12:21:53.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca820c2700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fca7c0616b0 0x7fca7c153070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60848/0 (socket says 172.21.15.116:60848) 2020-11-20T12:21:53.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca820c2700 1 -- 172.21.15.116:0/3236552419 learned_addr learned my addr 172.21.15.116:0/3236552419 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:53.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca818c1700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c14bee0 0x7fca7c152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3722016564 0 0) 0x7fca7c0abc10 con 0x7fca7c0616b0 2020-11-20T12:21:53.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca64003240 con 0x7fca7c0616b0 2020-11-20T12:21:53.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025389265 0 0) 0x7fca7c0acf60 con 0x7fca7c0c8fa0 2020-11-20T12:21:53.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca7c0abc10 con 0x7fca7c0c8fa0 2020-11-20T12:21:53.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 340165309 0 0) 0x7fca64003240 con 0x7fca7c0616b0 2020-11-20T12:21:53.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c14bee0 msgr2=0x7fca7c152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:53.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c14bee0 0x7fca7c152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 >> v1:172.21.15.173:6789/0 conn(0x7fca7c0c8fa0 legacy=0x7fca7c0adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:53.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c0c9520 con 0x7fca7c0616b0 2020-11-20T12:21:53.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fca74001860 con 0x7fca7c0616b0 2020-11-20T12:21:53.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 >> v1:172.21.15.116:6789/0 conn(0x7fca7c0616b0 legacy=0x7fca7c153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:53.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca640038f0 0x7fca64003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca640044e0 0x7fca64004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c14bee0 0x7fca64004e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c152e80 con 0x7fca640038f0 2020-11-20T12:21:53.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca640053b0 con 0x7fca640044e0 2020-11-20T12:21:53.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6400b7b0 con 0x7fca7c14bee0 2020-11-20T12:21:53.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca818c1700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca640038f0 0x7fca64003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca820c2700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c14bee0 0x7fca64004e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca810c0700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca640044e0 0x7fca64004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca818c1700 1 -- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c14bee0 msgr2=0x7fca64004e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:53.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca818c1700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c14bee0 0x7fca64004e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca818c1700 1 -- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca640044e0 msgr2=0x7fca64004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:53.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.096+0000 7fca818c1700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca640044e0 0x7fca64004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca818c1700 1 -- 172.21.15.116:0/3236552419 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca64003240 con 0x7fca640038f0 2020-11-20T12:21:53.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca820c2700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c14bee0 0x7fca64004e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:53.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca810c0700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca640044e0 0x7fca64004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:53.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca818c1700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca640038f0 0x7fca64003cf0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7fca7800bc00 tx=0x7fca78006df0).ready entity=mon.0 client_cookie=da166b2c8356308e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:53.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7800b310 con 0x7fca640038f0 2020-11-20T12:21:53.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca7800eb40 con 0x7fca640038f0 2020-11-20T12:21:53.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca808bf700 1 -- 172.21.15.116:0/3236552419 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca7800eec0 con 0x7fca640038f0 2020-11-20T12:21:53.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca83b25700 1 -- 172.21.15.116:0/3236552419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca640038f0 msgr2=0x7fca64003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:53.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca83b25700 1 --2- 172.21.15.116:0/3236552419 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca640038f0 0x7fca64003cf0 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7fca7800bc00 tx=0x7fca78006df0).stop 2020-11-20T12:21:53.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca820c2700 1 -- 172.21.15.116:0/3236552419 reap_dead start 2020-11-20T12:21:53.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca83b25700 1 -- 172.21.15.116:0/3236552419 shutdown_connections 2020-11-20T12:21:53.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca83b25700 1 -- 172.21.15.116:0/3236552419 >> 172.21.15.116:0/3236552419 conn(0x7fca7c068370 msgr2=0x7fca7c0c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:53.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca83b25700 1 -- 172.21.15.116:0/3236552419 shutdown_connections 2020-11-20T12:21:53.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.100+0000 7fca83b25700 1 -- 172.21.15.116:0/3236552419 wait complete. 2020-11-20T12:21:53.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca83b25700 1 Processor -- start 2020-11-20T12:21:53.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca83b25700 1 -- start start 2020-11-20T12:21:53.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca83b25700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c13a1a0 0x7fca7c14d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca83b25700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca7c13a580 0x7fca7c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca83b25700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca7c14de10 0x7fca7c0c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca83b25700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0c5e10 con 0x7fca7c13a580 2020-11-20T12:21:53.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca83b25700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0c60b0 con 0x7fca7c14de10 2020-11-20T12:21:53.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca83b25700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c0c6350 con 0x7fca7c13a1a0 2020-11-20T12:21:53.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca810c0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca7c13a580 0x7fca7c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca810c0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca7c13a580 0x7fca7c14d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51422/0 (socket says 172.21.15.116:51422) 2020-11-20T12:21:53.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca810c0700 1 -- 172.21.15.116:0/1821921991 learned_addr learned my addr 172.21.15.116:0/1821921991 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:53.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca820c2700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca7c14de10 0x7fca7c0c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca818c1700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c13a1a0 0x7fca7c14d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca810c0700 1 -- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c13a1a0 msgr2=0x7fca7c14d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:53.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca810c0700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c13a1a0 0x7fca7c14d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca810c0700 1 -- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca7c14de10 msgr2=0x7fca7c0c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:53.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca810c0700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca7c14de10 0x7fca7c0c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca810c0700 1 -- 172.21.15.116:0/1821921991 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7c0c65f0 con 0x7fca7c13a580 2020-11-20T12:21:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.104+0000 7fca820c2700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca7c14de10 0x7fca7c0c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.108+0000 7fca810c0700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca7c13a580 0x7fca7c14d920 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7fca6c00e830 tx=0x7fca6c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.108+0000 7fca69ffb700 1 -- 172.21.15.116:0/1821921991 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c007030 con 0x7fca7c13a580 2020-11-20T12:21:53.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.108+0000 7fca69ffb700 1 -- 172.21.15.116:0/1821921991 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c007190 con 0x7fca7c13a580 2020-11-20T12:21:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.108+0000 7fca83b25700 1 -- 172.21.15.116:0/1821921991 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c0c6890 con 0x7fca7c13a580 2020-11-20T12:21:53.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.108+0000 7fca69ffb700 1 -- 172.21.15.116:0/1821921991 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c00c610 con 0x7fca7c13a580 2020-11-20T12:21:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.108+0000 7fca83b25700 1 -- 172.21.15.116:0/1821921991 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c151900 con 0x7fca7c13a580 2020-11-20T12:21:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.108+0000 7fca83b25700 1 -- 172.21.15.116:0/1821921991 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c0c6a20 con 0x7fca7c13a580 2020-11-20T12:21:53.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.108+0000 7fca69ffb700 1 -- 172.21.15.116:0/1821921991 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fca6c01a030 con 0x7fca7c13a580 2020-11-20T12:21:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.112+0000 7fca69ffb700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fca540557c0 0x7fca54057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.125 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.112+0000 7fca69ffb700 1 -- 172.21.15.116:0/1821921991 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fca6c06dd20 con 0x7fca7c13a580 2020-11-20T12:21:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.112+0000 7fca818c1700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fca540557c0 0x7fca54057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.112+0000 7fca818c1700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fca540557c0 0x7fca54057c80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fca7c05f2f0 tx=0x7fca78019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:53.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.116+0000 7fca69ffb700 1 -- 172.21.15.116:0/1821921991 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fca6c072070 con 0x7fca7c13a580 2020-11-20T12:21:53.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.280+0000 7fca83b25700 1 -- 172.21.15.116:0/1821921991 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7fca7c05d2a0 con 0x7fca7c13a580 2020-11-20T12:21:53.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.436+0000 7fca69ffb700 1 -- 172.21.15.116:0/1821921991 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v114) v1 ==== 133+0+0 (secure 0 0 0) 0x7fca6c043f20 con 0x7fca7c13a580 2020-11-20T12:21:53.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 -- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fca540557c0 msgr2=0x7fca54057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:53.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fca540557c0 0x7fca54057c80 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7fca7c05f2f0 tx=0x7fca78019000).stop 2020-11-20T12:21:53.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 -- 172.21.15.116:0/1821921991 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca7c13a580 msgr2=0x7fca7c14d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:53.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca7c13a580 0x7fca7c14d920 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7fca6c00e830 tx=0x7fca6c009040).stop 2020-11-20T12:21:53.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 -- 172.21.15.116:0/1821921991 shutdown_connections 2020-11-20T12:21:53.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fca540557c0 0x7fca54057c80 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fca7c14de10 0x7fca7c0c6d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fca7c13a1a0 0x7fca7c14d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 --2- 172.21.15.116:0/1821921991 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fca7c13a580 0x7fca7c14d920 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 -- 172.21.15.116:0/1821921991 >> 172.21.15.116:0/1821921991 conn(0x7fca7c068370 msgr2=0x7fca7c154b50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:53.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 -- 172.21.15.116:0/1821921991 shutdown_connections 2020-11-20T12:21:53.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.444+0000 7fca83b25700 1 -- 172.21.15.116:0/1821921991 wait complete. 2020-11-20T12:21:53.450 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2020-11-20T12:21:53.471 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-20T12:21:53.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9776bf5700 1 Processor -- start 2020-11-20T12:21:53.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9776bf5700 1 -- start start 2020-11-20T12:21:53.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9776bf5700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f97700c0220 0x7f97700c0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9776bf5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9770063490 0x7f9770060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9776bf5700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9770060660 0x7f97700664f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9776bf5700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9770062880 con 0x7f97700c0220 2020-11-20T12:21:53.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9776bf5700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9770067200 con 0x7f9770060660 2020-11-20T12:21:53.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9776bf5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9770062ab0 con 0x7f9770063490 2020-11-20T12:21:53.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9774991700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f97700c0220 0x7f97700c0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9774991700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f97700c0220 0x7f97700c0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51430/0 (socket says 172.21.15.116:51430) 2020-11-20T12:21:53.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9774991700 1 -- 172.21.15.116:0/187611907 learned_addr learned my addr 172.21.15.116:0/187611907 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:53.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.656+0000 7f9775192700 1 --2- 172.21.15.116:0/187611907 >> v2:172.21.15.173:3300/0 conn(0x7f9770060660 0x7f97700664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f976ffff700 1 --2- 172.21.15.116:0/187611907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9770063490 0x7f9770060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9774991700 1 -- 172.21.15.116:0/187611907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9770063490 msgr2=0x7f9770060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:53.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9774991700 1 --2- 172.21.15.116:0/187611907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9770063490 0x7f9770060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9774991700 1 -- 172.21.15.116:0/187611907 >> v2:172.21.15.173:3300/0 conn(0x7f9770060660 msgr2=0x7f97700664f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:53.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9774991700 1 --2- 172.21.15.116:0/187611907 >> v2:172.21.15.173:3300/0 conn(0x7f9770060660 0x7f97700664f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9774991700 1 -- 172.21.15.116:0/187611907 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9770062d50 con 0x7f97700c0220 2020-11-20T12:21:53.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9774991700 1 --2- 172.21.15.116:0/187611907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97700c0220 0x7f97700c0600 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f97700a6f80 tx=0x7f9764006eb0).ready entity=mon.0 client_cookie=b6204ba32c0d4fc5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:53.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f976f7fe700 1 -- 172.21.15.116:0/187611907 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764014070 con 0x7f97700c0220 2020-11-20T12:21:53.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f976f7fe700 1 -- 172.21.15.116:0/187611907 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9764009720 con 0x7f97700c0220 2020-11-20T12:21:53.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f976f7fe700 1 -- 172.21.15.116:0/187611907 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9764009aa0 con 0x7f97700c0220 2020-11-20T12:21:53.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9776bf5700 1 -- 172.21.15.116:0/187611907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97700c0220 msgr2=0x7f97700c0600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:53.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9776bf5700 1 --2- 172.21.15.116:0/187611907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97700c0220 0x7f97700c0600 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7f97700a6f80 tx=0x7f9764006eb0).stop 2020-11-20T12:21:53.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9776bf5700 1 -- 172.21.15.116:0/187611907 shutdown_connections 2020-11-20T12:21:53.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9776bf5700 1 --2- 172.21.15.116:0/187611907 >> v2:172.21.15.173:3300/0 conn(0x7f9770060660 0x7f97700664f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9776bf5700 1 --2- 172.21.15.116:0/187611907 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f97700c0220 0x7f97700c0600 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9776bf5700 1 --2- 172.21.15.116:0/187611907 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9770063490 0x7f9770060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9776bf5700 1 -- 172.21.15.116:0/187611907 >> 172.21.15.116:0/187611907 conn(0x7f977005ba80 msgr2=0x7f97700c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:53.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9776bf5700 1 -- 172.21.15.116:0/187611907 shutdown_connections 2020-11-20T12:21:53.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.660+0000 7f9776bf5700 1 -- 172.21.15.116:0/187611907 wait complete. 2020-11-20T12:21:53.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.664+0000 7f9776bf5700 1 Processor -- start 2020-11-20T12:21:53.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.664+0000 7f9776bf5700 1 -- start start 2020-11-20T12:21:53.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f9776bf5700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9770060660 0x7f9770146910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f9776bf5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9770063490 0x7f977014bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f9776bf5700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97700c0220 0x7f977014d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f9776bf5700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977014c680 con 0x7f9770063490 2020-11-20T12:21:53.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f9776bf5700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f977014c920 con 0x7f97700c0220 2020-11-20T12:21:53.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f9776bf5700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977014cbc0 con 0x7f9770060660 2020-11-20T12:21:53.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9770063490 0x7f977014bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9770063490 0x7f977014bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51438/0 (socket says 172.21.15.116:51438) 2020-11-20T12:21:53.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976ffff700 1 -- 172.21.15.116:0/1461908710 learned_addr learned my addr 172.21.15.116:0/1461908710 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:53.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f9774991700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9770060660 0x7f9770146910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f9775192700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97700c0220 0x7f977014d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976ffff700 1 -- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9770060660 msgr2=0x7f9770146910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:53.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976ffff700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9770060660 0x7f9770146910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976ffff700 1 -- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97700c0220 msgr2=0x7f977014d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:53.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976ffff700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97700c0220 0x7f977014d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:53.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976ffff700 1 -- 172.21.15.116:0/1461908710 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977014ce60 con 0x7f9770063490 2020-11-20T12:21:53.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976ffff700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9770063490 0x7f977014bde0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f975c00ebd0 tx=0x7f975c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:53.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976cff9700 1 -- 172.21.15.116:0/1461908710 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00b810 con 0x7f9770063490 2020-11-20T12:21:53.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f9776bf5700 1 -- 172.21.15.116:0/1461908710 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977014d100 con 0x7f9770063490 2020-11-20T12:21:53.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976cff9700 1 -- 172.21.15.116:0/1461908710 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c00b970 con 0x7f9770063490 2020-11-20T12:21:53.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f976cff9700 1 -- 172.21.15.116:0/1461908710 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00bcf0 con 0x7f9770063490 2020-11-20T12:21:53.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.668+0000 7f9776bf5700 1 -- 172.21.15.116:0/1461908710 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f977014c2d0 con 0x7f9770063490 2020-11-20T12:21:53.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.672+0000 7f9776bf5700 1 -- 172.21.15.116:0/1461908710 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f977014d2e0 con 0x7f9770063490 2020-11-20T12:21:53.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.672+0000 7f976cff9700 1 -- 172.21.15.116:0/1461908710 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f975c012600 con 0x7f9770063490 2020-11-20T12:21:53.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.672+0000 7f976cff9700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97480557c0 0x7f9748057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:53.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.672+0000 7f976cff9700 1 -- 172.21.15.116:0/1461908710 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f975c06f080 con 0x7f9770063490 2020-11-20T12:21:53.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.676+0000 7f9774991700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97480557c0 0x7f9748057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:53.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.676+0000 7f9774991700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97480557c0 0x7f9748057c80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f976400b1b0 tx=0x7f9764016000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:53.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.676+0000 7f976cff9700 1 -- 172.21.15.116:0/1461908710 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f975c045090 con 0x7f9770063490 2020-11-20T12:21:53.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:53.832+0000 7f9776bf5700 1 -- 172.21.15.116:0/1461908710 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f977014d290 con 0x7f9770063490 2020-11-20T12:21:54.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.584+0000 7f976cff9700 1 -- 172.21.15.116:0/1461908710 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v116) v1 ==== 125+0+0 (secure 0 0 0) 0x7f975c0486b0 con 0x7f9770063490 2020-11-20T12:21:56.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 -- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97480557c0 msgr2=0x7f9748057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97480557c0 0x7f9748057c80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f976400b1b0 tx=0x7f9764016000).stop 2020-11-20T12:21:56.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 -- 172.21.15.116:0/1461908710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9770063490 msgr2=0x7f977014bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9770063490 0x7f977014bde0 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7f975c00ebd0 tx=0x7f975c006df0).stop 2020-11-20T12:21:56.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 -- 172.21.15.116:0/1461908710 shutdown_connections 2020-11-20T12:21:56.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f97480557c0 0x7f9748057c80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f97700c0220 0x7f977014d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9770060660 0x7f9770146910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 --2- 172.21.15.116:0/1461908710 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9770063490 0x7f977014bde0 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 -- 172.21.15.116:0/1461908710 >> 172.21.15.116:0/1461908710 conn(0x7f977005ba80 msgr2=0x7f97700c3210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:56.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 -- 172.21.15.116:0/1461908710 shutdown_connections 2020-11-20T12:21:56.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:54.592+0000 7f9776bf5700 1 -- 172.21.15.116:0/1461908710 wait complete. 2020-11-20T12:21:56.150 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2020-11-20T12:21:56.150 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T12:21:56.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.340+0000 7f01ce395700 1 Processor -- start 2020-11-20T12:21:56.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.340+0000 7f01ce395700 1 -- start start 2020-11-20T12:21:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.340+0000 7f01ce395700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f01c80a75b0 0x7f01c80a9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.340+0000 7f01ce395700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c8060ed0 con 0x7f01c81516b0 2020-11-20T12:21:56.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.340+0000 7f01ce395700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c80a7a50 con 0x7f01c8151a90 2020-11-20T12:21:56.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.340+0000 7f01ce395700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01c80a7cf0 con 0x7f01c80a75b0 2020-11-20T12:21:56.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.340+0000 7f01cd393700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f01c81516b0 0x7f01c8155e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60878/0 (socket says 172.21.15.116:60878) 2020-11-20T12:21:56.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.340+0000 7f01cd393700 1 -- 172.21.15.116:0/3982819454 learned_addr learned my addr 172.21.15.116:0/3982819454 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:56.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01ccb92700 1 --2- 172.21.15.116:0/3982819454 >> v2:172.21.15.173:3300/0 conn(0x7f01c80a75b0 0x7f01c80a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169134301 0 0) 0x7f01c8060ed0 con 0x7f01c81516b0 2020-11-20T12:21:56.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ac003240 con 0x7f01c81516b0 2020-11-20T12:21:56.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2021125246 0 0) 0x7f01ac003240 con 0x7f01c81516b0 2020-11-20T12:21:56.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 >> v2:172.21.15.173:3300/0 conn(0x7f01c80a75b0 msgr2=0x7f01c80a9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:56.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 --2- 172.21.15.116:0/3982819454 >> v2:172.21.15.173:3300/0 conn(0x7f01c80a75b0 0x7f01c80a9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 >> v1:172.21.15.173:6789/0 conn(0x7f01c8151a90 legacy=0x7f01c8066610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c80a7f90 con 0x7f01c81516b0 2020-11-20T12:21:56.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f01c40014e0 con 0x7f01c81516b0 2020-11-20T12:21:56.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 >> v1:172.21.15.116:6789/0 conn(0x7f01c81516b0 legacy=0x7f01c8155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 --2- 172.21.15.116:0/3982819454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01ac003730 0x7f01ac003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 --2- 172.21.15.116:0/3982819454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac0044e0 0x7f01ac004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 --2- 172.21.15.116:0/3982819454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01ac004e50 0x7f01ac00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c80a7cf0 con 0x7f01ac0044e0 2020-11-20T12:21:56.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac004240 con 0x7f01ac003730 2020-11-20T12:21:56.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac00bc20 con 0x7f01ac004e50 2020-11-20T12:21:56.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01ccb92700 1 --2- 172.21.15.116:0/3982819454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac0044e0 0x7f01ac004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01cd393700 1 --2- 172.21.15.116:0/3982819454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01ac003730 0x7f01ac003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01cdb94700 1 --2- 172.21.15.116:0/3982819454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01ac004e50 0x7f01ac00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01ccb92700 1 -- 172.21.15.116:0/3982819454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01ac004e50 msgr2=0x7f01ac00b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:56.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01ccb92700 1 --2- 172.21.15.116:0/3982819454 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01ac004e50 0x7f01ac00b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01ccb92700 1 -- 172.21.15.116:0/3982819454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01ac003730 msgr2=0x7f01ac003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:56.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01ccb92700 1 --2- 172.21.15.116:0/3982819454 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01ac003730 0x7f01ac003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01ccb92700 1 -- 172.21.15.116:0/3982819454 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ac003240 con 0x7f01ac0044e0 2020-11-20T12:21:56.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01cdb94700 1 -- 172.21.15.116:0/3982819454 reap_dead start 2020-11-20T12:21:56.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.344+0000 7f01ccb92700 1 --2- 172.21.15.116:0/3982819454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac0044e0 0x7f01ac004960 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f01b800bc30 tx=0x7f01b8006df0).ready entity=mon.0 client_cookie=dbdedf4a1ab31d0f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:56.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.348+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b800b310 con 0x7f01ac0044e0 2020-11-20T12:21:56.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.348+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b800ea30 con 0x7f01ac0044e0 2020-11-20T12:21:56.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.348+0000 7f01bffff700 1 -- 172.21.15.116:0/3982819454 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b800edb0 con 0x7f01ac0044e0 2020-11-20T12:21:56.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.348+0000 7f01ce395700 1 -- 172.21.15.116:0/3982819454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac0044e0 msgr2=0x7f01ac004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.348+0000 7f01ce395700 1 --2- 172.21.15.116:0/3982819454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac0044e0 0x7f01ac004960 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f01b800bc30 tx=0x7f01b8006df0).stop 2020-11-20T12:21:56.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.348+0000 7f01ce395700 1 -- 172.21.15.116:0/3982819454 shutdown_connections 2020-11-20T12:21:56.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.348+0000 7f01ce395700 1 --2- 172.21.15.116:0/3982819454 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01ac0044e0 0x7f01ac004960 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.348+0000 7f01ce395700 1 -- 172.21.15.116:0/3982819454 >> 172.21.15.116:0/3982819454 conn(0x7f01c80682e0 msgr2=0x7f01c8060170 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:56.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.348+0000 7f01ce395700 1 -- 172.21.15.116:0/3982819454 shutdown_connections 2020-11-20T12:21:56.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.348+0000 7f01ce395700 1 -- 172.21.15.116:0/3982819454 wait complete. 2020-11-20T12:21:56.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01ce395700 1 Processor -- start 2020-11-20T12:21:56.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01ce395700 1 -- start start 2020-11-20T12:21:56.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01ce395700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01c814b1f0 0x7f01c8161b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01ce395700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c8153e10 0x7f01c8154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01ce395700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01c8155ef0 0x7f01c8156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01ce395700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c8162020 con 0x7f01c8155ef0 2020-11-20T12:21:56.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01ce395700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c8164ab0 con 0x7f01c8153e10 2020-11-20T12:21:56.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01ce395700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c8164d50 con 0x7f01c814b1f0 2020-11-20T12:21:56.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01cdb94700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01c8155ef0 0x7f01c8156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01cdb94700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01c8155ef0 0x7f01c8156370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51460/0 (socket says 172.21.15.116:51460) 2020-11-20T12:21:56.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01cdb94700 1 -- 172.21.15.116:0/3776531623 learned_addr learned my addr 172.21.15.116:0/3776531623 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:56.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01cd393700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01c814b1f0 0x7f01c8161b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01ccb92700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c8153e10 0x7f01c8154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01cdb94700 1 -- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01c814b1f0 msgr2=0x7f01c8161b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01cdb94700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01c814b1f0 0x7f01c8161b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01cdb94700 1 -- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c8153e10 msgr2=0x7f01c8154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01cdb94700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c8153e10 0x7f01c8154290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01cdb94700 1 -- 172.21.15.116:0/3776531623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c8164ff0 con 0x7f01c8155ef0 2020-11-20T12:21:56.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01cd393700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01c814b1f0 0x7f01c8161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01ccb92700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c8153e10 0x7f01c8154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:21:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.352+0000 7f01cdb94700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01c8155ef0 0x7f01c8156370 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f01c8061ef0 tx=0x7f01c0007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.356+0000 7f01bd7fa700 1 -- 172.21.15.116:0/3776531623 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c000c7b0 con 0x7f01c8155ef0 2020-11-20T12:21:56.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.356+0000 7f01bd7fa700 1 -- 172.21.15.116:0/3776531623 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c000c910 con 0x7f01c8155ef0 2020-11-20T12:21:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.356+0000 7f01ce395700 1 -- 172.21.15.116:0/3776531623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c8165290 con 0x7f01c8155ef0 2020-11-20T12:21:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.356+0000 7f01bd7fa700 1 -- 172.21.15.116:0/3776531623 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c000ccf0 con 0x7f01c8155ef0 2020-11-20T12:21:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.356+0000 7f01ce395700 1 -- 172.21.15.116:0/3776531623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c814b5d0 con 0x7f01c8155ef0 2020-11-20T12:21:56.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.356+0000 7f01aaffd700 1 -- 172.21.15.116:0/3776531623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c805cf30 con 0x7f01c8155ef0 2020-11-20T12:21:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.356+0000 7f01bd7fa700 1 -- 172.21.15.116:0/3776531623 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f01c0013630 con 0x7f01c8155ef0 2020-11-20T12:21:56.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.360+0000 7f01bd7fa700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01a00557c0 0x7f01a0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.360+0000 7f01bd7fa700 1 -- 172.21.15.116:0/3776531623 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f01c006e980 con 0x7f01c8155ef0 2020-11-20T12:21:56.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.360+0000 7f01cd393700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01a00557c0 0x7f01a0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.360+0000 7f01cd393700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01a00557c0 0x7f01a0057c80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f01c805f7f0 tx=0x7f01c4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:56.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.364+0000 7f01bd7fa700 1 -- 172.21.15.116:0/3776531623 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f01c0044720 con 0x7f01c8155ef0 2020-11-20T12:21:56.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.568+0000 7f01aaffd700 1 -- 172.21.15.116:0/3776531623 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7f01c80a62b0 con 0x7f01c8155ef0 2020-11-20T12:21:56.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.576+0000 7f01bd7fa700 1 -- 172.21.15.116:0/3776531623 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 21 and data pool 22 v65) v1 ==== 189+0+0 (secure 0 0 0) 0x7f01c0047d40 con 0x7f01c8155ef0 2020-11-20T12:21:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.580+0000 7f01aaffd700 1 -- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01a00557c0 msgr2=0x7f01a0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.580+0000 7f01aaffd700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01a00557c0 0x7f01a0057c80 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f01c805f7f0 tx=0x7f01c4009040).stop 2020-11-20T12:21:56.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.580+0000 7f01aaffd700 1 -- 172.21.15.116:0/3776531623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01c8155ef0 msgr2=0x7f01c8156370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.580+0000 7f01aaffd700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01c8155ef0 0x7f01c8156370 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7f01c8061ef0 tx=0x7f01c0007ef0).stop 2020-11-20T12:21:56.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.584+0000 7f01aaffd700 1 -- 172.21.15.116:0/3776531623 shutdown_connections 2020-11-20T12:21:56.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.584+0000 7f01aaffd700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f01a00557c0 0x7f01a0057c80 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.584+0000 7f01aaffd700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f01c8155ef0 0x7f01c8156370 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.584+0000 7f01aaffd700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f01c814b1f0 0x7f01c8161b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.584+0000 7f01aaffd700 1 --2- 172.21.15.116:0/3776531623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f01c8153e10 0x7f01c8154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.584+0000 7f01aaffd700 1 -- 172.21.15.116:0/3776531623 >> 172.21.15.116:0/3776531623 conn(0x7f01c80682e0 msgr2=0x7f01c814ceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:56.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.584+0000 7f01aaffd700 1 -- 172.21.15.116:0/3776531623 shutdown_connections 2020-11-20T12:21:56.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.584+0000 7f01aaffd700 1 -- 172.21.15.116:0/3776531623 wait complete. 2020-11-20T12:21:56.592 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 21 and data pool 22 2020-11-20T12:21:56.613 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:56.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.776+0000 7fc5f187c700 1 Processor -- start 2020-11-20T12:21:56.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.776+0000 7fc5f187c700 1 -- start start 2020-11-20T12:21:56.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.776+0000 7fc5f187c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc5ec0615d0 0x7fc5ec065310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.776+0000 7fc5f187c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5ec066070 0x7fc5ec065800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.776+0000 7fc5f187c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5ec060950 con 0x7fc5ec151940 2020-11-20T12:21:56.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.776+0000 7fc5f187c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5ec0c4a90 con 0x7fc5ec0615d0 2020-11-20T12:21:56.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.776+0000 7fc5f187c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec060bf0 con 0x7fc5ec066070 2020-11-20T12:21:56.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.776+0000 7fc5eaffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc5ec151940 0x7fc5ec0c4380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60910/0 (socket says 172.21.15.116:60910) 2020-11-20T12:21:56.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.776+0000 7fc5eaffd700 1 -- 172.21.15.116:0/1176892584 learned_addr learned my addr 172.21.15.116:0/1176892584 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:56.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5ebfff700 1 --2- 172.21.15.116:0/1176892584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5ec066070 0x7fc5ec065800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5ea7fc700 1 --2- 172.21.15.116:0/1176892584 >> v2:172.21.15.173:3300/0 conn(0x7fc5ec0615d0 0x7fc5ec065310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5e9ffb700 1 -- 172.21.15.116:0/1176892584 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 922341197 0 0) 0x7fc5ec060950 con 0x7fc5ec151940 2020-11-20T12:21:56.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5e9ffb700 1 -- 172.21.15.116:0/1176892584 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5d8003240 con 0x7fc5ec151940 2020-11-20T12:21:56.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5ebfff700 1 -- 172.21.15.116:0/1176892584 >> v2:172.21.15.173:3300/0 conn(0x7fc5ec0615d0 msgr2=0x7fc5ec065310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:56.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5ebfff700 1 --2- 172.21.15.116:0/1176892584 >> v2:172.21.15.173:3300/0 conn(0x7fc5ec0615d0 0x7fc5ec065310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5e9ffb700 1 -- 172.21.15.116:0/1176892584 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 191108378 0 0) 0x7fc5d8003240 con 0x7fc5ec151940 2020-11-20T12:21:56.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5ebfff700 1 -- 172.21.15.116:0/1176892584 >> v1:172.21.15.116:6789/0 conn(0x7fc5ec151940 legacy=0x7fc5ec0c4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5ebfff700 1 -- 172.21.15.116:0/1176892584 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec060e90 con 0x7fc5ec066070 2020-11-20T12:21:56.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5ebfff700 1 --2- 172.21.15.116:0/1176892584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5ec066070 0x7fc5ec065800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fc5e4009c90 tx=0x7fc5e40091d0).ready entity=mon.2 client_cookie=a7943d8bcc5ae5e8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:56.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5e9ffb700 1 -- 172.21.15.116:0/1176892584 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc0014e0 con 0x7fc5ec066070 2020-11-20T12:21:56.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5e9ffb700 1 -- 172.21.15.116:0/1176892584 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e400b6b0 con 0x7fc5ec066070 2020-11-20T12:21:56.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5e9ffb700 1 -- 172.21.15.116:0/1176892584 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e400ba30 con 0x7fc5ec066070 2020-11-20T12:21:56.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5f187c700 1 -- 172.21.15.116:0/1176892584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5ec066070 msgr2=0x7fc5ec065800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5f187c700 1 --2- 172.21.15.116:0/1176892584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5ec066070 0x7fc5ec065800 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fc5e4009c90 tx=0x7fc5e40091d0).stop 2020-11-20T12:21:56.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5f187c700 1 -- 172.21.15.116:0/1176892584 shutdown_connections 2020-11-20T12:21:56.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5f187c700 1 --2- 172.21.15.116:0/1176892584 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5ec066070 0x7fc5ec065800 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5f187c700 1 --2- 172.21.15.116:0/1176892584 >> v2:172.21.15.173:3300/0 conn(0x7fc5ec0615d0 0x7fc5ec065310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.780+0000 7fc5f187c700 1 -- 172.21.15.116:0/1176892584 >> 172.21.15.116:0/1176892584 conn(0x7fc5ec0ca850 msgr2=0x7fc5ec0c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:56.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.784+0000 7fc5f187c700 1 -- 172.21.15.116:0/1176892584 shutdown_connections 2020-11-20T12:21:56.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.784+0000 7fc5f187c700 1 -- 172.21.15.116:0/1176892584 wait complete. 2020-11-20T12:21:56.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.784+0000 7fc5f187c700 1 Processor -- start 2020-11-20T12:21:56.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.784+0000 7fc5f187c700 1 -- start start 2020-11-20T12:21:56.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5f187c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5ec0615d0 0x7fc5ec161840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5f187c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5ec066070 0x7fc5ec0acd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5f187c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5ec151940 0x7fc5ec0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5f187c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec161d30 con 0x7fc5ec0615d0 2020-11-20T12:21:56.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5f187c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec0ae850 con 0x7fc5ec151940 2020-11-20T12:21:56.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5f187c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5ec0aeaf0 con 0x7fc5ec066070 2020-11-20T12:21:56.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5eaffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5ec0615d0 0x7fc5ec161840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5eaffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5ec0615d0 0x7fc5ec161840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51482/0 (socket says 172.21.15.116:51482) 2020-11-20T12:21:56.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5eaffd700 1 -- 172.21.15.116:0/691097405 learned_addr learned my addr 172.21.15.116:0/691097405 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:56.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5ebfff700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5ec151940 0x7fc5ec0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5ea7fc700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5ec066070 0x7fc5ec0acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5eaffd700 1 -- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5ec066070 msgr2=0x7fc5ec0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5eaffd700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5ec066070 0x7fc5ec0acd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5eaffd700 1 -- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5ec151940 msgr2=0x7fc5ec0ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5eaffd700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5ec151940 0x7fc5ec0ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5eaffd700 1 -- 172.21.15.116:0/691097405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec0aed90 con 0x7fc5ec0615d0 2020-11-20T12:21:56.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.788+0000 7fc5eaffd700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5ec0615d0 0x7fc5ec161840 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7fc5dc00cae0 tx=0x7fc5dc001a70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:56.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.792+0000 7fc5d2ffd700 1 -- 172.21.15.116:0/691097405 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc007170 con 0x7fc5ec0615d0 2020-11-20T12:21:56.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.792+0000 7fc5d2ffd700 1 -- 172.21.15.116:0/691097405 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5dc0072d0 con 0x7fc5ec0615d0 2020-11-20T12:21:56.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.792+0000 7fc5f187c700 1 -- 172.21.15.116:0/691097405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5ec0af030 con 0x7fc5ec0615d0 2020-11-20T12:21:56.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.792+0000 7fc5d2ffd700 1 -- 172.21.15.116:0/691097405 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5dc00a8e0 con 0x7fc5ec0615d0 2020-11-20T12:21:56.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.792+0000 7fc5f187c700 1 -- 172.21.15.116:0/691097405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5ec0adaf0 con 0x7fc5ec0615d0 2020-11-20T12:21:56.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.792+0000 7fc5f187c700 1 -- 172.21.15.116:0/691097405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5ec05d2a0 con 0x7fc5ec0615d0 2020-11-20T12:21:56.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.792+0000 7fc5d2ffd700 1 -- 172.21.15.116:0/691097405 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc5dc01b060 con 0x7fc5ec0615d0 2020-11-20T12:21:56.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.796+0000 7fc5d2ffd700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5c80557c0 0x7fc5c8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:56.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.796+0000 7fc5d2ffd700 1 -- 172.21.15.116:0/691097405 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc5dc06e210 con 0x7fc5ec0615d0 2020-11-20T12:21:56.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.796+0000 7fc5ea7fc700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5c80557c0 0x7fc5c8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:56.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.796+0000 7fc5ea7fc700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5c80557c0 0x7fc5c8057c80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fc5ec164330 tx=0x7fc5e0006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:56.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.800+0000 7fc5d2ffd700 1 -- 172.21.15.116:0/691097405 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc5dc0440b0 con 0x7fc5ec0615d0 2020-11-20T12:21:56.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.964+0000 7fc5f187c700 1 -- 172.21.15.116:0/691097405 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc5ec0628d0 con 0x7fc5ec0615d0 2020-11-20T12:21:56.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5d2ffd700 1 -- 172.21.15.116:0/691097405 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16888 (secure 0 0 0) 0x7fc5dc019030 con 0x7fc5ec0615d0 2020-11-20T12:21:56.971 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:56.971 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":118,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:56.569994+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:53.361384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:53.918177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:56.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 -- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5c80557c0 msgr2=0x7fc5c8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5c80557c0 0x7fc5c8057c80 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fc5ec164330 tx=0x7fc5e0006df0).stop 2020-11-20T12:21:56.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 -- 172.21.15.116:0/691097405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5ec0615d0 msgr2=0x7fc5ec161840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:56.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5ec0615d0 0x7fc5ec161840 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7fc5dc00cae0 tx=0x7fc5dc001a70).stop 2020-11-20T12:21:56.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 -- 172.21.15.116:0/691097405 shutdown_connections 2020-11-20T12:21:56.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc5c80557c0 0x7fc5c8057c80 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc5ec151940 0x7fc5ec0ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc5ec0615d0 0x7fc5ec161840 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 --2- 172.21.15.116:0/691097405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc5ec066070 0x7fc5ec0acd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:56.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 -- 172.21.15.116:0/691097405 >> 172.21.15.116:0/691097405 conn(0x7fc5ec0ca850 msgr2=0x7fc5ec0c6560 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:56.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 -- 172.21.15.116:0/691097405 shutdown_connections 2020-11-20T12:21:56.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:56.968+0000 7fc5f187c700 1 -- 172.21.15.116:0/691097405 wait complete. 2020-11-20T12:21:56.995 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:57.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.156+0000 7f9d013f4700 1 Processor -- start 2020-11-20T12:21:57.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.156+0000 7f9d013f4700 1 -- start start 2020-11-20T12:21:57.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.160+0000 7f9d013f4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cfc0ba890 0x7f9cfc0bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.160+0000 7f9d013f4700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9cfc148a20 0x7f9cfc148ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.160+0000 7f9d013f4700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cfc066140 con 0x7f9cfc0c2230 2020-11-20T12:21:57.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.160+0000 7f9d013f4700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cfc0c1d80 con 0x7f9cfc148a20 2020-11-20T12:21:57.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.160+0000 7f9d013f4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc1526c0 con 0x7f9cfc0ba890 2020-11-20T12:21:57.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.160+0000 7f9cfaffd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cfc0ba890 0x7f9cfc0bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.160+0000 7f9cf3fff700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f9cfc0c2230 0x7f9cfc0c1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:57156/0 (socket says 172.21.15.116:57156) 2020-11-20T12:21:57.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.160+0000 7f9cf3fff700 1 -- 172.21.15.116:0/2337938471 learned_addr learned my addr 172.21.15.116:0/2337938471 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:57.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.160+0000 7f9cfb7fe700 1 --2- 172.21.15.116:0/2337938471 >> v2:172.21.15.173:3300/0 conn(0x7f9cfc148a20 0x7f9cfc148ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9cfa7fc700 1 -- 172.21.15.116:0/2337938471 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153976300 0 0) 0x7f9cfc066140 con 0x7f9cfc0c2230 2020-11-20T12:21:57.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9cfa7fc700 1 -- 172.21.15.116:0/2337938471 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cdc003240 con 0x7f9cfc0c2230 2020-11-20T12:21:57.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9cfaffd700 1 -- 172.21.15.116:0/2337938471 >> v2:172.21.15.173:3300/0 conn(0x7f9cfc148a20 msgr2=0x7f9cfc148ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:57.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9cfaffd700 1 --2- 172.21.15.116:0/2337938471 >> v2:172.21.15.173:3300/0 conn(0x7f9cfc148a20 0x7f9cfc148ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9cfaffd700 1 -- 172.21.15.116:0/2337938471 >> v1:172.21.15.173:6789/0 conn(0x7f9cfc0c2230 legacy=0x7f9cfc0c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:57.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9cfaffd700 1 -- 172.21.15.116:0/2337938471 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc152960 con 0x7f9cfc0ba890 2020-11-20T12:21:57.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9cfaffd700 1 --2- 172.21.15.116:0/2337938471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cfc0ba890 0x7f9cfc0bac70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f9cec00b9a0 tx=0x7f9cec006df0).ready entity=mon.2 client_cookie=a93ee2d09a40fc90 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9cfa7fc700 1 -- 172.21.15.116:0/2337938471 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec009680 con 0x7f9cfc0ba890 2020-11-20T12:21:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9cfa7fc700 1 -- 172.21.15.116:0/2337938471 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cec0097e0 con 0x7f9cfc0ba890 2020-11-20T12:21:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9cfa7fc700 1 -- 172.21.15.116:0/2337938471 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cec009b60 con 0x7f9cfc0ba890 2020-11-20T12:21:57.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9d013f4700 1 -- 172.21.15.116:0/2337938471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cfc0ba890 msgr2=0x7f9cfc0bac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9d013f4700 1 --2- 172.21.15.116:0/2337938471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cfc0ba890 0x7f9cfc0bac70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f9cec00b9a0 tx=0x7f9cec006df0).stop 2020-11-20T12:21:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9d013f4700 1 -- 172.21.15.116:0/2337938471 shutdown_connections 2020-11-20T12:21:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9d013f4700 1 --2- 172.21.15.116:0/2337938471 >> v2:172.21.15.173:3300/0 conn(0x7f9cfc148a20 0x7f9cfc148ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9d013f4700 1 --2- 172.21.15.116:0/2337938471 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cfc0ba890 0x7f9cfc0bac70 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.164+0000 7f9d013f4700 1 -- 172.21.15.116:0/2337938471 >> 172.21.15.116:0/2337938471 conn(0x7f9cfc0c4f80 msgr2=0x7f9cfc0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.168+0000 7f9d013f4700 1 -- 172.21.15.116:0/2337938471 shutdown_connections 2020-11-20T12:21:57.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.168+0000 7f9d013f4700 1 -- 172.21.15.116:0/2337938471 wait complete. 2020-11-20T12:21:57.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.168+0000 7f9d013f4700 1 Processor -- start 2020-11-20T12:21:57.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.168+0000 7f9d013f4700 1 -- start start 2020-11-20T12:21:57.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9d013f4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cfc0ba890 0x7f9cfc1696b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9d013f4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cfc0c2230 0x7f9cfc166750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9d013f4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cfc148a20 0x7f9cfc137640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9d013f4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc0a91c0 con 0x7f9cfc148a20 2020-11-20T12:21:57.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9d013f4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc165de0 con 0x7f9cfc0ba890 2020-11-20T12:21:57.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9d013f4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cfc166080 con 0x7f9cfc0c2230 2020-11-20T12:21:57.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cfb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cfc148a20 0x7f9cfc137640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cfb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cfc148a20 0x7f9cfc137640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51500/0 (socket says 172.21.15.116:51500) 2020-11-20T12:21:57.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cfb7fe700 1 -- 172.21.15.116:0/1758810170 learned_addr learned my addr 172.21.15.116:0/1758810170 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:57.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cf3fff700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cfc0c2230 0x7f9cfc166750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cfaffd700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cfc0ba890 0x7f9cfc1696b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cfb7fe700 1 -- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cfc0c2230 msgr2=0x7f9cfc166750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:57.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cfb7fe700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cfc0c2230 0x7f9cfc166750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cfb7fe700 1 -- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cfc0ba890 msgr2=0x7f9cfc1696b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:57.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cfb7fe700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cfc0ba890 0x7f9cfc1696b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cfb7fe700 1 -- 172.21.15.116:0/1758810170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc166320 con 0x7f9cfc148a20 2020-11-20T12:21:57.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cfaffd700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cfc0ba890 0x7f9cfc1696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:21:57.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.172+0000 7f9cfb7fe700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cfc148a20 0x7f9cfc137640 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f9cf400ed20 tx=0x7f9cf40091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:57.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.176+0000 7f9cf2ffd700 1 -- 172.21.15.116:0/1758810170 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400cbb0 con 0x7f9cfc148a20 2020-11-20T12:21:57.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.176+0000 7f9cf2ffd700 1 -- 172.21.15.116:0/1758810170 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf400cd10 con 0x7f9cfc148a20 2020-11-20T12:21:57.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.176+0000 7f9d013f4700 1 -- 172.21.15.116:0/1758810170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cfc0a9420 con 0x7f9cfc148a20 2020-11-20T12:21:57.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.176+0000 7f9cf2ffd700 1 -- 172.21.15.116:0/1758810170 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4012620 con 0x7f9cfc148a20 2020-11-20T12:21:57.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.176+0000 7f9d013f4700 1 -- 172.21.15.116:0/1758810170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cfc165580 con 0x7f9cfc148a20 2020-11-20T12:21:57.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.176+0000 7f9d013f4700 1 -- 172.21.15.116:0/1758810170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cfc0a95b0 con 0x7f9cfc148a20 2020-11-20T12:21:57.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.176+0000 7f9cf2ffd700 1 -- 172.21.15.116:0/1758810170 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9cf4022070 con 0x7f9cfc148a20 2020-11-20T12:21:57.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.180+0000 7f9cf2ffd700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cd80557c0 0x7f9cd8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.180+0000 7f9cf2ffd700 1 -- 172.21.15.116:0/1758810170 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9cf406eec0 con 0x7f9cfc148a20 2020-11-20T12:21:57.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.180+0000 7f9cfaffd700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cd80557c0 0x7f9cd8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.180+0000 7f9cfaffd700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cd80557c0 0x7f9cd8057c80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9cec006e30 tx=0x7f9cec015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:57.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.184+0000 7f9cf2ffd700 1 -- 172.21.15.116:0/1758810170 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9cf4044d60 con 0x7f9cfc148a20 2020-11-20T12:21:57.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.344+0000 7f9d013f4700 1 -- 172.21.15.116:0/1758810170 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9cfc05d2a0 con 0x7f9cfc148a20 2020-11-20T12:21:57.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9cf2ffd700 1 -- 172.21.15.116:0/1758810170 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f9cf4016030 con 0x7f9cfc148a20 2020-11-20T12:21:57.351 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:57.351 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":118,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:56.569994+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:53.361384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:53.918177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:57.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 -- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cd80557c0 msgr2=0x7f9cd8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:57.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cd80557c0 0x7f9cd8057c80 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9cec006e30 tx=0x7f9cec015000).stop 2020-11-20T12:21:57.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 -- 172.21.15.116:0/1758810170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cfc148a20 msgr2=0x7f9cfc137640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:57.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cfc148a20 0x7f9cfc137640 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f9cf400ed20 tx=0x7f9cf40091d0).stop 2020-11-20T12:21:57.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 -- 172.21.15.116:0/1758810170 shutdown_connections 2020-11-20T12:21:57.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9cd80557c0 0x7f9cd8057c80 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9cfc148a20 0x7f9cfc137640 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9cfc0ba890 0x7f9cfc1696b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 --2- 172.21.15.116:0/1758810170 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9cfc0c2230 0x7f9cfc166750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 -- 172.21.15.116:0/1758810170 >> 172.21.15.116:0/1758810170 conn(0x7f9cfc0c4f80 msgr2=0x7f9cfc063070 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:57.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 -- 172.21.15.116:0/1758810170 shutdown_connections 2020-11-20T12:21:57.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.348+0000 7f9d013f4700 1 -- 172.21.15.116:0/1758810170 wait complete. 2020-11-20T12:21:57.378 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-20T12:21:57.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.560+0000 7fe088bf1700 1 Processor -- start 2020-11-20T12:21:57.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.560+0000 7fe088bf1700 1 -- start start 2020-11-20T12:21:57.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe088bf1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0840a8400 0x7fe0840aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe088bf1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0840ad540 0x7fe0840ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe088bf1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0840ace40 con 0x7fe0840aa7d0 2020-11-20T12:21:57.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe088bf1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0841526c0 con 0x7fe0840ad540 2020-11-20T12:21:57.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe088bf1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe084152960 con 0x7fe0840a8400 2020-11-20T12:21:57.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe0837fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe0840aa7d0 0x7fe084151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60938/0 (socket says 172.21.15.116:60938) 2020-11-20T12:21:57.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe0837fe700 1 -- 172.21.15.116:0/3420925587 learned_addr learned my addr 172.21.15.116:0/3420925587 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:57.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe083fff700 1 --2- 172.21.15.116:0/3420925587 >> v2:172.21.15.116:3300/0 conn(0x7fe0840ad540 0x7fe0840ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe082ffd700 1 --2- 172.21.15.116:0/3420925587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0840a8400 0x7fe0840aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe0827fc700 1 -- 172.21.15.116:0/3420925587 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1490723122 0 0) 0x7fe0840ace40 con 0x7fe0840aa7d0 2020-11-20T12:21:57.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe0827fc700 1 -- 172.21.15.116:0/3420925587 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe070003240 con 0x7fe0840aa7d0 2020-11-20T12:21:57.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe083fff700 1 -- 172.21.15.116:0/3420925587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0840a8400 msgr2=0x7fe0840aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:57.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe083fff700 1 --2- 172.21.15.116:0/3420925587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0840a8400 0x7fe0840aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe083fff700 1 -- 172.21.15.116:0/3420925587 >> v1:172.21.15.116:6789/0 conn(0x7fe0840aa7d0 legacy=0x7fe084151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:57.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe083fff700 1 -- 172.21.15.116:0/3420925587 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe084152c00 con 0x7fe0840ad540 2020-11-20T12:21:57.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe083fff700 1 --2- 172.21.15.116:0/3420925587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0840ad540 0x7fe0840ad9f0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7fe07c009c90 tx=0x7fe07c0091d0).ready entity=mon.0 client_cookie=b5943fecd99388f4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:57.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe0827fc700 1 -- 172.21.15.116:0/3420925587 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00b9c0 con 0x7fe0840ad540 2020-11-20T12:21:57.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe0827fc700 1 -- 172.21.15.116:0/3420925587 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c00bb20 con 0x7fe0840ad540 2020-11-20T12:21:57.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.564+0000 7fe0827fc700 1 -- 172.21.15.116:0/3420925587 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00bea0 con 0x7fe0840ad540 2020-11-20T12:21:57.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.568+0000 7fe088bf1700 1 -- 172.21.15.116:0/3420925587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0840ad540 msgr2=0x7fe0840ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:57.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.568+0000 7fe088bf1700 1 --2- 172.21.15.116:0/3420925587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0840ad540 0x7fe0840ad9f0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7fe07c009c90 tx=0x7fe07c0091d0).stop 2020-11-20T12:21:57.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.568+0000 7fe088bf1700 1 -- 172.21.15.116:0/3420925587 shutdown_connections 2020-11-20T12:21:57.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.568+0000 7fe088bf1700 1 --2- 172.21.15.116:0/3420925587 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0840ad540 0x7fe0840ad9f0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.568+0000 7fe088bf1700 1 --2- 172.21.15.116:0/3420925587 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0840a8400 0x7fe0840aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.568+0000 7fe088bf1700 1 -- 172.21.15.116:0/3420925587 >> 172.21.15.116:0/3420925587 conn(0x7fe0840678a0 msgr2=0x7fe084067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:57.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.568+0000 7fe088bf1700 1 -- 172.21.15.116:0/3420925587 shutdown_connections 2020-11-20T12:21:57.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.568+0000 7fe088bf1700 1 -- 172.21.15.116:0/3420925587 wait complete. 2020-11-20T12:21:57.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe088bf1700 1 Processor -- start 2020-11-20T12:21:57.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe088bf1700 1 -- start start 2020-11-20T12:21:57.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe088bf1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0840a8400 0x7fe084160ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe088bf1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0840aa7d0 0x7fe08414b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe088bf1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0840ad540 0x7fe0840bd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe088bf1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe084161400 con 0x7fe0840ad540 2020-11-20T12:21:57.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe088bf1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0841616a0 con 0x7fe0840a8400 2020-11-20T12:21:57.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe088bf1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe084161940 con 0x7fe0840aa7d0 2020-11-20T12:21:57.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe083fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0840ad540 0x7fe0840bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe083fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0840ad540 0x7fe0840bd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51514/0 (socket says 172.21.15.116:51514) 2020-11-20T12:21:57.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe083fff700 1 -- 172.21.15.116:0/2369860324 learned_addr learned my addr 172.21.15.116:0/2369860324 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:57.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe0837fe700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0840a8400 0x7fe084160ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe082ffd700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0840aa7d0 0x7fe08414b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe083fff700 1 -- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0840aa7d0 msgr2=0x7fe08414b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:57.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe083fff700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0840aa7d0 0x7fe08414b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe083fff700 1 -- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0840a8400 msgr2=0x7fe084160ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe083fff700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0840a8400 0x7fe084160ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe083fff700 1 -- 172.21.15.116:0/2369860324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840c0f90 con 0x7fe0840ad540 2020-11-20T12:21:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe083fff700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0840ad540 0x7fe0840bd700 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7fe07c00b6c0 tx=0x7fe07c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:57.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe0637fe700 1 -- 172.21.15.116:0/2369860324 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c01c5f0 con 0x7fe0840ad540 2020-11-20T12:21:57.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe0637fe700 1 -- 172.21.15.116:0/2369860324 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c01c750 con 0x7fe0840ad540 2020-11-20T12:21:57.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe0637fe700 1 -- 172.21.15.116:0/2369860324 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c01caf0 con 0x7fe0840ad540 2020-11-20T12:21:57.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.572+0000 7fe088bf1700 1 -- 172.21.15.116:0/2369860324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0840c0c80 con 0x7fe0840ad540 2020-11-20T12:21:57.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.576+0000 7fe088bf1700 1 -- 172.21.15.116:0/2369860324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08414b620 con 0x7fe0840ad540 2020-11-20T12:21:57.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.576+0000 7fe088bf1700 1 -- 172.21.15.116:0/2369860324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0840c1120 con 0x7fe0840ad540 2020-11-20T12:21:57.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.576+0000 7fe0637fe700 1 -- 172.21.15.116:0/2369860324 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe07c024030 con 0x7fe0840ad540 2020-11-20T12:21:57.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.580+0000 7fe0637fe700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe05c0557c0 0x7fe05c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:57.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.580+0000 7fe0837fe700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe05c0557c0 0x7fe05c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:57.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.580+0000 7fe0637fe700 1 -- 172.21.15.116:0/2369860324 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe07c048180 con 0x7fe0840ad540 2020-11-20T12:21:57.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.580+0000 7fe0837fe700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe05c0557c0 0x7fe05c057c80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fe0740055a0 tx=0x7fe074009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:57.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.584+0000 7fe0637fe700 1 -- 172.21.15.116:0/2369860324 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe07c046380 con 0x7fe0840ad540 2020-11-20T12:21:57.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:57.796+0000 7fe088bf1700 1 -- 172.21.15.116:0/2369860324 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe08405cf30 con 0x7fe0840ad540 2020-11-20T12:21:58.386 INFO:tasks.ceph.mds.a.smithi116.stderr:2020-11-20T12:21:58.384+0000 7f01f5c20700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:21:58.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.596+0000 7fe0637fe700 1 -- 172.21.15.116:0/2369860324 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe07c022030 con 0x7fe0840ad540 2020-11-20T12:21:58.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.596+0000 7fe088bf1700 1 -- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe05c0557c0 msgr2=0x7fe05c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:58.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.596+0000 7fe088bf1700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe05c0557c0 0x7fe05c057c80 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fe0740055a0 tx=0x7fe074009040).stop 2020-11-20T12:21:58.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.596+0000 7fe088bf1700 1 -- 172.21.15.116:0/2369860324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0840ad540 msgr2=0x7fe0840bd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:58.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.596+0000 7fe088bf1700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0840ad540 0x7fe0840bd700 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7fe07c00b6c0 tx=0x7fe07c016000).stop 2020-11-20T12:21:58.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.600+0000 7fe088bf1700 1 -- 172.21.15.116:0/2369860324 shutdown_connections 2020-11-20T12:21:58.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.600+0000 7fe088bf1700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe05c0557c0 0x7fe05c057c80 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:58.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.600+0000 7fe088bf1700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0840ad540 0x7fe0840bd700 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:58.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.600+0000 7fe088bf1700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0840a8400 0x7fe084160ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:58.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.600+0000 7fe088bf1700 1 --2- 172.21.15.116:0/2369860324 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0840aa7d0 0x7fe08414b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:58.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.600+0000 7fe088bf1700 1 -- 172.21.15.116:0/2369860324 >> 172.21.15.116:0/2369860324 conn(0x7fe0840678a0 msgr2=0x7fe084067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:58.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.600+0000 7fe088bf1700 1 -- 172.21.15.116:0/2369860324 shutdown_connections 2020-11-20T12:21:58.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.600+0000 7fe088bf1700 1 -- 172.21.15.116:0/2369860324 wait complete. 2020-11-20T12:21:58.630 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:21:58.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.824+0000 7f5393d3c700 1 Processor -- start 2020-11-20T12:21:58.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.824+0000 7f5393d3c700 1 -- start start 2020-11-20T12:21:58.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.824+0000 7f5393d3c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c0bf100 0x7f538c0bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:58.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.824+0000 7f5393d3c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f538c0bac00 0x7f538c14db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:58.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.824+0000 7f5393d3c700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f538c0b9d20 con 0x7f538c0bfa80 2020-11-20T12:21:58.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.824+0000 7f5393d3c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f538c0bb0b0 con 0x7f538c0bac00 2020-11-20T12:21:58.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.824+0000 7f5393d3c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538c0b9f80 con 0x7f538c0bf100 2020-11-20T12:21:58.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5391ad8700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f538c0bfa80 0x7f538c066e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60952/0 (socket says 172.21.15.116:60952) 2020-11-20T12:21:58.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5391ad8700 1 -- 172.21.15.116:0/3623661086 learned_addr learned my addr 172.21.15.116:0/3623661086 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:58.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f53912d7700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c0bf100 0x7f538c0bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:58.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f53922d9700 1 --2- 172.21.15.116:0/3623661086 >> v2:172.21.15.173:3300/0 conn(0x7f538c0bac00 0x7f538c14db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:58.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728899217 0 0) 0x7f538c0b9d20 con 0x7f538c0bfa80 2020-11-20T12:21:58.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5374004350 con 0x7f538c0bfa80 2020-11-20T12:21:58.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3250606428 0 0) 0x7f5374004350 con 0x7f538c0bfa80 2020-11-20T12:21:58.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c0bf100 msgr2=0x7f538c0bfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:58.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c0bf100 0x7f538c0bfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:58.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 >> v2:172.21.15.173:3300/0 conn(0x7f538c0bac00 msgr2=0x7f538c14db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:58.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 --2- 172.21.15.116:0/3623661086 >> v2:172.21.15.173:3300/0 conn(0x7f538c0bac00 0x7f538c14db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:58.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c0ba220 con 0x7f538c0bfa80 2020-11-20T12:21:58.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f53880014e0 con 0x7f538c0bfa80 2020-11-20T12:21:58.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 >> v1:172.21.15.116:6789/0 conn(0x7f538c0bfa80 legacy=0x7f538c066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:58.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53740048c0 0x7f5374004ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:58.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c0bf100 0x7f53740055f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:58.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5374005ae0 0x7f5374005f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:58.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c0bb0b0 con 0x7f5374005ae0 2020-11-20T12:21:58.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c0b9f80 con 0x7f53740048c0 2020-11-20T12:21:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53740053b0 con 0x7f538c0bf100 2020-11-20T12:21:58.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f53922d9700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5374005ae0 0x7f5374005f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:58.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f53912d7700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c0bf100 0x7f53740055f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:58.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.828+0000 7f5391ad8700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53740048c0 0x7f5374004ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:58.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f53922d9700 1 -- 172.21.15.116:0/3623661086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c0bf100 msgr2=0x7f53740055f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:58.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f53922d9700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c0bf100 0x7f53740055f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:58.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f53922d9700 1 -- 172.21.15.116:0/3623661086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53740048c0 msgr2=0x7f5374004ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:58.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f53922d9700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f53740048c0 0x7f5374004ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:58.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f53922d9700 1 -- 172.21.15.116:0/3623661086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5374004350 con 0x7f5374005ae0 2020-11-20T12:21:58.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f53922d9700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5374005ae0 0x7f5374005f60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f5384009ea0 tx=0x7f53840091d0).ready entity=mon.0 client_cookie=dc3845b573f37958 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:58.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53840095c0 con 0x7f5374005ae0 2020-11-20T12:21:58.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538400ecf0 con 0x7f5374005ae0 2020-11-20T12:21:58.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f5390ad6700 1 -- 172.21.15.116:0/3623661086 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5384012600 con 0x7f5374005ae0 2020-11-20T12:21:58.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f5393d3c700 1 -- 172.21.15.116:0/3623661086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5374005ae0 msgr2=0x7f5374005f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:58.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f5393d3c700 1 --2- 172.21.15.116:0/3623661086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5374005ae0 0x7f5374005f60 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f5384009ea0 tx=0x7f53840091d0).stop 2020-11-20T12:21:58.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f53922d9700 1 -- 172.21.15.116:0/3623661086 reap_dead start 2020-11-20T12:21:58.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f5393d3c700 1 -- 172.21.15.116:0/3623661086 shutdown_connections 2020-11-20T12:21:58.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f5393d3c700 1 -- 172.21.15.116:0/3623661086 >> 172.21.15.116:0/3623661086 conn(0x7f538c0c4f80 msgr2=0x7f538c0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:58.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f5393d3c700 1 -- 172.21.15.116:0/3623661086 shutdown_connections 2020-11-20T12:21:58.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.832+0000 7f5393d3c700 1 -- 172.21.15.116:0/3623661086 wait complete. 2020-11-20T12:21:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f5393d3c700 1 Processor -- start 2020-11-20T12:21:58.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f5393d3c700 1 -- start start 2020-11-20T12:21:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f5393d3c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f538c151ef0 0x7f538c14aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f5393d3c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c14a350 0x7f538c0bb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f5393d3c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f538c1522d0 0x7f538c0bba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f5393d3c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c141980 con 0x7f538c1522d0 2020-11-20T12:21:58.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f5393d3c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f538c14b320 con 0x7f538c151ef0 2020-11-20T12:21:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f5393d3c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f538c140fa0 con 0x7f538c14a350 2020-11-20T12:21:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f53922d9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f538c1522d0 0x7f538c0bba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f53922d9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f538c1522d0 0x7f538c0bba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51534/0 (socket says 172.21.15.116:51534) 2020-11-20T12:21:58.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f53922d9700 1 -- 172.21.15.116:0/1935791941 learned_addr learned my addr 172.21.15.116:0/1935791941 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f53912d7700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c14a350 0x7f538c0bb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.836+0000 7f5391ad8700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f538c151ef0 0x7f538c14aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f53922d9700 1 -- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c14a350 msgr2=0x7f538c0bb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f53922d9700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c14a350 0x7f538c0bb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:58.866 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f53922d9700 1 -- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f538c151ef0 msgr2=0x7f538c14aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f53922d9700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f538c151ef0 0x7f538c14aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f53922d9700 1 -- 172.21.15.116:0/1935791941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f538c141240 con 0x7f538c1522d0 2020-11-20T12:21:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f53922d9700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f538c1522d0 0x7f538c0bba30 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f5384009e70 tx=0x7f538401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:58.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f5379ffb700 1 -- 172.21.15.116:0/1935791941 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53840095c0 con 0x7f538c1522d0 2020-11-20T12:21:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f5379ffb700 1 -- 172.21.15.116:0/1935791941 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f538400ee60 con 0x7f538c1522d0 2020-11-20T12:21:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f5379ffb700 1 -- 172.21.15.116:0/1935791941 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53840245f0 con 0x7f538c1522d0 2020-11-20T12:21:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f5393d3c700 1 -- 172.21.15.116:0/1935791941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f538c1414e0 con 0x7f538c1522d0 2020-11-20T12:21:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f5393d3c700 1 -- 172.21.15.116:0/1935791941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f538c14af70 con 0x7f538c1522d0 2020-11-20T12:21:58.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f5393d3c700 1 -- 172.21.15.116:0/1935791941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f538c141670 con 0x7f538c1522d0 2020-11-20T12:21:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.840+0000 7f5379ffb700 1 -- 172.21.15.116:0/1935791941 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f538402c070 con 0x7f538c1522d0 2020-11-20T12:21:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.844+0000 7f5379ffb700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53640557c0 0x7f5364057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.844+0000 7f5379ffb700 1 -- 172.21.15.116:0/1935791941 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f53840709b0 con 0x7f538c1522d0 2020-11-20T12:21:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.844+0000 7f5391ad8700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53640557c0 0x7f5364057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:58.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.844+0000 7f5391ad8700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53640557c0 0x7f5364057c80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f53880013b0 tx=0x7f5388009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:58.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:58.848+0000 7f5379ffb700 1 -- 172.21.15.116:0/1935791941 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5384046850 con 0x7f538c1522d0 2020-11-20T12:21:59.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5393d3c700 1 -- 172.21.15.116:0/1935791941 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f538c05d2a0 con 0x7f538c1522d0 2020-11-20T12:21:59.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5379ffb700 1 -- 172.21.15.116:0/1935791941 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2854 (secure 0 0 0) 0x7f5384016030 con 0x7f538c1522d0 2020-11-20T12:21:59.059 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:59.059 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/1612795957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1612795957},{"type":"v1","addr":"172.21.15.116:6835","nonce":1612795957}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/3255680781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3255680781},{"type":"v1","addr":"172.21.15.173:6835","nonce":3255680781}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/4058046293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":4058046293},{"type":"v1","addr":"172.21.15.173:6837","nonce":4058046293}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:56.570302+0000","modified":"2020-11-20T12:21:58.589498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8148":{"gid":8148,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2775705840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2775705840},{"type":"v1","addr":"172.21.15.116:6833","nonce":2775705840}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-20T12:21:59.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5393d3c700 1 -- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53640557c0 msgr2=0x7f5364057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:59.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5393d3c700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53640557c0 0x7f5364057c80 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f53880013b0 tx=0x7f5388009040).stop 2020-11-20T12:21:59.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5393d3c700 1 -- 172.21.15.116:0/1935791941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f538c1522d0 msgr2=0x7f538c0bba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:59.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5393d3c700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f538c1522d0 0x7f538c0bba30 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f5384009e70 tx=0x7f538401a000).stop 2020-11-20T12:21:59.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5393d3c700 1 -- 172.21.15.116:0/1935791941 shutdown_connections 2020-11-20T12:21:59.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5393d3c700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f53640557c0 0x7f5364057c80 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5393d3c700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f538c1522d0 0x7f538c0bba30 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5393d3c700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f538c151ef0 0x7f538c14aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5393d3c700 1 --2- 172.21.15.116:0/1935791941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f538c14a350 0x7f538c0bb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.056+0000 7f5393d3c700 1 -- 172.21.15.116:0/1935791941 >> 172.21.15.116:0/1935791941 conn(0x7f538c0c4f80 msgr2=0x7f538c062990 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:59.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.060+0000 7f5393d3c700 1 -- 172.21.15.116:0/1935791941 shutdown_connections 2020-11-20T12:21:59.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.060+0000 7f5393d3c700 1 -- 172.21.15.116:0/1935791941 wait complete. 2020-11-20T12:21:59.065 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 68 2020-11-20T12:21:59.086 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:21:59.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.260+0000 7f5d04bfe700 1 Processor -- start 2020-11-20T12:21:59.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.260+0000 7f5d04bfe700 1 -- start start 2020-11-20T12:21:59.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.260+0000 7f5d04bfe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5d00062880 0x7f5d00062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:59.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.260+0000 7f5d04bfe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5d00153450 0x7f5d000677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:59.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.260+0000 7f5d04bfe700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d001543a0 con 0x7f5d000c4c10 2020-11-20T12:21:59.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.260+0000 7f5d04bfe700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d00153920 con 0x7f5d00062880 2020-11-20T12:21:59.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.260+0000 7f5d04bfe700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d00153bc0 con 0x7f5d00153450 2020-11-20T12:21:59.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cff7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f5d00062880 0x7f5d00062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:59.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cfffff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5d000c4c10 0x7f5d000b9710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:60976/0 (socket says 172.21.15.116:60976) 2020-11-20T12:21:59.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cfffff700 1 -- 172.21.15.116:0/1370134157 learned_addr learned my addr 172.21.15.116:0/1370134157 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:21:59.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cfeffd700 1 --2- 172.21.15.116:0/1370134157 >> v2:172.21.15.173:3300/0 conn(0x7f5d00153450 0x7f5d000677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:59.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cff7fe700 1 -- 172.21.15.116:0/1370134157 >> v2:172.21.15.173:3300/0 conn(0x7f5d00153450 msgr2=0x7f5d000677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:21:59.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cff7fe700 1 --2- 172.21.15.116:0/1370134157 >> v2:172.21.15.173:3300/0 conn(0x7f5d00153450 0x7f5d000677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cff7fe700 1 -- 172.21.15.116:0/1370134157 >> v1:172.21.15.116:6789/0 conn(0x7f5d000c4c10 legacy=0x7f5d000b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:59.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cff7fe700 1 -- 172.21.15.116:0/1370134157 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d00153e60 con 0x7f5d00062880 2020-11-20T12:21:59.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cfe7fc700 1 -- 172.21.15.116:0/1370134157 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451713914 0 0) 0x7f5d001543a0 con 0x7f5d000c4c10 2020-11-20T12:21:59.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cff7fe700 1 --2- 172.21.15.116:0/1370134157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d00062880 0x7f5d00062c60 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f5d00064b70 tx=0x7f5cf0006df0).ready entity=mon.0 client_cookie=ff5da54f3cad4f8f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:59.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cfe7fc700 1 -- 172.21.15.116:0/1370134157 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00097b0 con 0x7f5d00062880 2020-11-20T12:21:59.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cfe7fc700 1 -- 172.21.15.116:0/1370134157 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf0009910 con 0x7f5d00062880 2020-11-20T12:21:59.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5cfe7fc700 1 -- 172.21.15.116:0/1370134157 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0009c90 con 0x7f5d00062880 2020-11-20T12:21:59.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5d04bfe700 1 -- 172.21.15.116:0/1370134157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d00062880 msgr2=0x7f5d00062c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:59.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5d04bfe700 1 --2- 172.21.15.116:0/1370134157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d00062880 0x7f5d00062c60 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f5d00064b70 tx=0x7f5cf0006df0).stop 2020-11-20T12:21:59.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5d04bfe700 1 -- 172.21.15.116:0/1370134157 shutdown_connections 2020-11-20T12:21:59.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5d04bfe700 1 --2- 172.21.15.116:0/1370134157 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d00062880 0x7f5d00062c60 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5d04bfe700 1 --2- 172.21.15.116:0/1370134157 >> v2:172.21.15.173:3300/0 conn(0x7f5d00153450 0x7f5d000677a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5d04bfe700 1 -- 172.21.15.116:0/1370134157 >> 172.21.15.116:0/1370134157 conn(0x7f5d00060c40 msgr2=0x7f5d000c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:59.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5d04bfe700 1 -- 172.21.15.116:0/1370134157 shutdown_connections 2020-11-20T12:21:59.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.264+0000 7f5d04bfe700 1 -- 172.21.15.116:0/1370134157 wait complete. 2020-11-20T12:21:59.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.268+0000 7f5d04bfe700 1 Processor -- start 2020-11-20T12:21:59.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.268+0000 7f5d04bfe700 1 -- start start 2020-11-20T12:21:59.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5d04bfe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d00062880 0x7f5d0014a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:59.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5d04bfe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d000c4c10 0x7f5d000c3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:59.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5d04bfe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d00153450 0x7f5d000c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:59.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5d04bfe700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d000a1b30 con 0x7f5d000c4c10 2020-11-20T12:21:59.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5d04bfe700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00161fb0 con 0x7f5d00062880 2020-11-20T12:21:59.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5d04bfe700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d00161330 con 0x7f5d00153450 2020-11-20T12:21:59.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cfeffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d000c4c10 0x7f5d000c3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:59.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cff7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d00062880 0x7f5d0014a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:59.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cfeffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d000c4c10 0x7f5d000c3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51546/0 (socket says 172.21.15.116:51546) 2020-11-20T12:21:59.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cfeffd700 1 -- 172.21.15.116:0/2995310030 learned_addr learned my addr 172.21.15.116:0/2995310030 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:21:59.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cfffff700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d00153450 0x7f5d000c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:59.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cfeffd700 1 -- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d00153450 msgr2=0x7f5d000c3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:59.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cfeffd700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d00153450 0x7f5d000c3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.278 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cfeffd700 1 -- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d00062880 msgr2=0x7f5d0014a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:59.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cfeffd700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d00062880 0x7f5d0014a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cfeffd700 1 -- 172.21.15.116:0/2995310030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d001615d0 con 0x7f5d000c4c10 2020-11-20T12:21:59.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cfeffd700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d000c4c10 0x7f5d000c3720 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f5cf400eba0 tx=0x7f5cf4006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:59.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cdf7fe700 1 -- 172.21.15.116:0/2995310030 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400b8a0 con 0x7f5d000c4c10 2020-11-20T12:21:59.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cdf7fe700 1 -- 172.21.15.116:0/2995310030 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cf400ba00 con 0x7f5d000c4c10 2020-11-20T12:21:59.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5d04bfe700 1 -- 172.21.15.116:0/2995310030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d00161870 con 0x7f5d000c4c10 2020-11-20T12:21:59.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5cdf7fe700 1 -- 172.21.15.116:0/2995310030 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf400bd80 con 0x7f5d000c4c10 2020-11-20T12:21:59.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.272+0000 7f5d04bfe700 1 -- 172.21.15.116:0/2995310030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d0014aee0 con 0x7f5d000c4c10 2020-11-20T12:21:59.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.276+0000 7f5d04bfe700 1 -- 172.21.15.116:0/2995310030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0005cf30 con 0x7f5d000c4c10 2020-11-20T12:21:59.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.276+0000 7f5cdf7fe700 1 -- 172.21.15.116:0/2995310030 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5cf4012600 con 0x7f5d000c4c10 2020-11-20T12:21:59.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.276+0000 7f5cdf7fe700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5cd80557c0 0x7f5cd8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:21:59.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.276+0000 7f5cdf7fe700 1 -- 172.21.15.116:0/2995310030 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5cf406f3e0 con 0x7f5d000c4c10 2020-11-20T12:21:59.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.280+0000 7f5cff7fe700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5cd80557c0 0x7f5cd8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:21:59.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.280+0000 7f5cdf7fe700 1 -- 172.21.15.116:0/2995310030 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5cf40452a0 con 0x7f5d000c4c10 2020-11-20T12:21:59.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.280+0000 7f5cff7fe700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5cd80557c0 0x7f5cd8057c80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f5cf0006e30 tx=0x7f5cf0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:21:59.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.436+0000 7f5d04bfe700 1 -- 172.21.15.116:0/2995310030 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d000c0b80 con 0x7f5d000c4c10 2020-11-20T12:21:59.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.436+0000 7f5cdf7fe700 1 -- 172.21.15.116:0/2995310030 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f5cf40488c0 con 0x7f5d000c4c10 2020-11-20T12:21:59.442 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:21:59.442 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":118,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:56.569994+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:53.361384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:53.918177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:21:59.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 -- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5cd80557c0 msgr2=0x7f5cd8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:59.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5cd80557c0 0x7f5cd8057c80 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7f5cf0006e30 tx=0x7f5cf0015000).stop 2020-11-20T12:21:59.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 -- 172.21.15.116:0/2995310030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d000c4c10 msgr2=0x7f5d000c3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:21:59.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d000c4c10 0x7f5d000c3720 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f5cf400eba0 tx=0x7f5cf4006df0).stop 2020-11-20T12:21:59.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 -- 172.21.15.116:0/2995310030 shutdown_connections 2020-11-20T12:21:59.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5cd80557c0 0x7f5cd8057c80 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5d00153450 0x7f5d000c3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5d00062880 0x7f5d0014a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 --2- 172.21.15.116:0/2995310030 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5d000c4c10 0x7f5d000c3720 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:21:59.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 -- 172.21.15.116:0/2995310030 >> 172.21.15.116:0/2995310030 conn(0x7f5d00060c40 msgr2=0x7f5d0014ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:21:59.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 -- 172.21.15.116:0/2995310030 shutdown_connections 2020-11-20T12:21:59.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.440+0000 7f5d04bfe700 1 -- 172.21.15.116:0/2995310030 wait complete. 2020-11-20T12:21:59.468 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-20T12:21:59.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.660+0000 7ff87759e700 1 Processor -- start 2020-11-20T12:22:01.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.660+0000 7ff87759e700 1 -- start start 2020-11-20T12:22:01.288 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:22:01.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.660+0000 7ff87759e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff87814f880 0x7ff8780b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87759e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff8780c0e90 0x7ff8780c12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87759e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8780a8930 con 0x7ff8780a81f0 2020-11-20T12:22:01.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87759e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8780ba2a0 con 0x7ff87814f880 2020-11-20T12:22:01.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87759e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8780a8690 con 0x7ff8780c0e90 2020-11-20T12:22:01.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87659c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff87814f880 0x7ff8780b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87659c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff87814f880 0x7ff8780b9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51552/0 (socket says 172.21.15.116:51552) 2020-11-20T12:22:01.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff875d9b700 1 --1- 172.21.15.116:0/2700816311 >> v1:172.21.15.173:6789/0 conn(0x7ff8780a81f0 0x7ff878060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:57218/0 (socket says 172.21.15.116:57218) 2020-11-20T12:22:01.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87659c700 1 -- 172.21.15.116:0/2700816311 learned_addr learned my addr 172.21.15.116:0/2700816311 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:01.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff876d9d700 1 --2- 172.21.15.116:0/2700816311 >> v2:172.21.15.173:3300/0 conn(0x7ff8780c0e90 0x7ff8780c12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87659c700 1 -- 172.21.15.116:0/2700816311 >> v2:172.21.15.173:3300/0 conn(0x7ff8780c0e90 msgr2=0x7ff8780c12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:01.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87559a700 1 -- 172.21.15.116:0/2700816311 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2786265581 0 0) 0x7ff8780a8930 con 0x7ff8780a81f0 2020-11-20T12:22:01.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87659c700 1 --2- 172.21.15.116:0/2700816311 >> v2:172.21.15.173:3300/0 conn(0x7ff8780c0e90 0x7ff8780c12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87659c700 1 -- 172.21.15.116:0/2700816311 >> v1:172.21.15.173:6789/0 conn(0x7ff8780a81f0 legacy=0x7ff878060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87659c700 1 -- 172.21.15.116:0/2700816311 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8780b94d0 con 0x7ff87814f880 2020-11-20T12:22:01.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87659c700 1 --2- 172.21.15.116:0/2700816311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff87814f880 0x7ff8780b9db0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7ff878063860 tx=0x7ff868006eb0).ready entity=mon.0 client_cookie=d4311296f2b60ecb server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:01.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87559a700 1 -- 172.21.15.116:0/2700816311 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868009810 con 0x7ff87814f880 2020-11-20T12:22:01.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87559a700 1 -- 172.21.15.116:0/2700816311 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff868009970 con 0x7ff87814f880 2020-11-20T12:22:01.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87559a700 1 -- 172.21.15.116:0/2700816311 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868009cf0 con 0x7ff87814f880 2020-11-20T12:22:01.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87759e700 1 -- 172.21.15.116:0/2700816311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff87814f880 msgr2=0x7ff8780b9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.664+0000 7ff87759e700 1 --2- 172.21.15.116:0/2700816311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff87814f880 0x7ff8780b9db0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7ff878063860 tx=0x7ff868006eb0).stop 2020-11-20T12:22:01.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 -- 172.21.15.116:0/2700816311 shutdown_connections 2020-11-20T12:22:01.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 --2- 172.21.15.116:0/2700816311 >> v2:172.21.15.173:3300/0 conn(0x7ff8780c0e90 0x7ff8780c12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 --2- 172.21.15.116:0/2700816311 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff87814f880 0x7ff8780b9db0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 -- 172.21.15.116:0/2700816311 >> 172.21.15.116:0/2700816311 conn(0x7ff878068350 msgr2=0x7ff8780627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:01.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 -- 172.21.15.116:0/2700816311 shutdown_connections 2020-11-20T12:22:01.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 -- 172.21.15.116:0/2700816311 wait complete. 2020-11-20T12:22:01.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 Processor -- start 2020-11-20T12:22:01.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 -- start start 2020-11-20T12:22:01.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8780a81f0 0x7ff8780bea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8780c0e90 0x7ff878161350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff87814f880 0x7ff878150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff878161b10 con 0x7ff8780a81f0 2020-11-20T12:22:01.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff878161db0 con 0x7ff8780c0e90 2020-11-20T12:22:01.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.668+0000 7ff87759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff878162050 con 0x7ff87814f880 2020-11-20T12:22:01.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8780a81f0 0x7ff8780bea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8780a81f0 0x7ff8780bea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51558/0 (socket says 172.21.15.116:51558) 2020-11-20T12:22:01.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87659c700 1 -- 172.21.15.116:0/1951539629 learned_addr learned my addr 172.21.15.116:0/1951539629 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:01.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff875d9b700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8780c0e90 0x7ff878161350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.298 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff876d9d700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff87814f880 0x7ff878150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87659c700 1 -- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff87814f880 msgr2=0x7ff878150dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87659c700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff87814f880 0x7ff878150dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87659c700 1 -- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8780c0e90 msgr2=0x7ff878161350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87659c700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8780c0e90 0x7ff878161350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87659c700 1 -- 172.21.15.116:0/1951539629 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8781622f0 con 0x7ff8780a81f0 2020-11-20T12:22:01.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87659c700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8780a81f0 0x7ff8780bea80 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7ff86800b1b0 tx=0x7ff868015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:01.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff8667fc700 1 -- 172.21.15.116:0/1951539629 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff868011d50 con 0x7ff8780a81f0 2020-11-20T12:22:01.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff8667fc700 1 -- 172.21.15.116:0/1951539629 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff86801e460 con 0x7ff8780a81f0 2020-11-20T12:22:01.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87759e700 1 -- 172.21.15.116:0/1951539629 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8780c3ef0 con 0x7ff8780a81f0 2020-11-20T12:22:01.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff8667fc700 1 -- 172.21.15.116:0/1951539629 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff86801e820 con 0x7ff8780a81f0 2020-11-20T12:22:01.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87759e700 1 -- 172.21.15.116:0/1951539629 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8781512c0 con 0x7ff8780a81f0 2020-11-20T12:22:01.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.672+0000 7ff87759e700 1 -- 172.21.15.116:0/1951539629 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff87805cf30 con 0x7ff8780a81f0 2020-11-20T12:22:01.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.676+0000 7ff8667fc700 1 -- 172.21.15.116:0/1951539629 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff868025030 con 0x7ff8780a81f0 2020-11-20T12:22:01.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.676+0000 7ff8667fc700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff84c0557f0 0x7ff84c057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.676+0000 7ff8667fc700 1 -- 172.21.15.116:0/1951539629 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff86806fde0 con 0x7ff8780a81f0 2020-11-20T12:22:01.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.676+0000 7ff875d9b700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff84c0557f0 0x7ff84c057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.676+0000 7ff875d9b700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff84c0557f0 0x7ff84c057cb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7ff8780c3b80 tx=0x7ff86c007ba0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:01.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.680+0000 7ff8667fc700 1 -- 172.21.15.116:0/1951539629 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff868045b70 con 0x7ff8780a81f0 2020-11-20T12:22:01.302 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.892+0000 7ff87759e700 1 -- 172.21.15.116:0/1951539629 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7ff8780639c0 con 0x7ff8780a81f0 2020-11-20T12:22:01.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff8667fc700 1 -- 172.21.15.116:0/1951539629 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v33) v1 ==== 184+0+0 (secure 0 0 0) 0x7ff868023020 con 0x7ff8780a81f0 2020-11-20T12:22:01.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 -- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff84c0557f0 msgr2=0x7ff84c057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff84c0557f0 0x7ff84c057cb0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7ff8780c3b80 tx=0x7ff86c007ba0).stop 2020-11-20T12:22:01.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 -- 172.21.15.116:0/1951539629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8780a81f0 msgr2=0x7ff8780bea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8780a81f0 0x7ff8780bea80 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7ff86800b1b0 tx=0x7ff868015000).stop 2020-11-20T12:22:01.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 -- 172.21.15.116:0/1951539629 shutdown_connections 2020-11-20T12:22:01.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff84c0557f0 0x7ff84c057cb0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff87814f880 0x7ff878150dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff8780a81f0 0x7ff8780bea80 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 --2- 172.21.15.116:0/1951539629 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff8780c0e90 0x7ff878161350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 -- 172.21.15.116:0/1951539629 >> 172.21.15.116:0/1951539629 conn(0x7ff878068350 msgr2=0x7ff8780bb520 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 -- 172.21.15.116:0/1951539629 shutdown_connections 2020-11-20T12:22:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:21:59.904+0000 7ff87759e700 1 -- 172.21.15.116:0/1951539629 wait complete. 2020-11-20T12:22:01.307 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2020-11-20T12:22:01.308 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:01.317 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:22:01.492 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.488+0000 7fea3605f700 1 Processor -- start 2020-11-20T12:22:01.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.488+0000 7fea3605f700 1 -- start start 2020-11-20T12:22:01.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3605f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea300618e0 0x7fea30061cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3605f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fea300adac0 0x7fea300a83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3605f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea300ace80 con 0x7fea300adf20 2020-11-20T12:22:01.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3605f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea30152b20 con 0x7fea300adac0 2020-11-20T12:22:01.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3605f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea300ad0e0 con 0x7fea300618e0 2020-11-20T12:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3485c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fea300adac0 0x7fea300a83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3485c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fea300adac0 0x7fea300a83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51568/0 (socket says 172.21.15.116:51568) 2020-11-20T12:22:01.497 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3485c700 1 -- 172.21.15.116:0/3032288309 learned_addr learned my addr 172.21.15.116:0/3032288309 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3505d700 1 --2- 172.21.15.116:0/3032288309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea300618e0 0x7fea30061cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3485c700 1 -- 172.21.15.116:0/3032288309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea300618e0 msgr2=0x7fea30061cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3485c700 1 --2- 172.21.15.116:0/3032288309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea300618e0 0x7fea30061cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3485c700 1 -- 172.21.15.116:0/3032288309 >> v1:172.21.15.116:6789/0 conn(0x7fea300adf20 legacy=0x7fea300aa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.498 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3485c700 1 -- 172.21.15.116:0/3032288309 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea300ad380 con 0x7fea300adac0 2020-11-20T12:22:01.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea3485c700 1 --2- 172.21.15.116:0/3032288309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300adac0 0x7fea300a83d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7fea2000ba20 tx=0x7fea20006eb0).ready entity=mon.0 client_cookie=95c876f260d2dec8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:01.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea1ffff700 1 -- 172.21.15.116:0/3032288309 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea200097d0 con 0x7fea300adac0 2020-11-20T12:22:01.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea1ffff700 1 -- 172.21.15.116:0/3032288309 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea20009930 con 0x7fea300adac0 2020-11-20T12:22:01.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.492+0000 7fea1ffff700 1 -- 172.21.15.116:0/3032288309 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20009cb0 con 0x7fea300adac0 2020-11-20T12:22:01.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.496+0000 7fea3605f700 1 -- 172.21.15.116:0/3032288309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300adac0 msgr2=0x7fea300a83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.496+0000 7fea3605f700 1 --2- 172.21.15.116:0/3032288309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300adac0 0x7fea300a83d0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7fea2000ba20 tx=0x7fea20006eb0).stop 2020-11-20T12:22:01.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.496+0000 7fea3605f700 1 -- 172.21.15.116:0/3032288309 shutdown_connections 2020-11-20T12:22:01.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.496+0000 7fea3605f700 1 --2- 172.21.15.116:0/3032288309 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea300618e0 0x7fea30061cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.496+0000 7fea3605f700 1 --2- 172.21.15.116:0/3032288309 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300adac0 0x7fea300a83d0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.496+0000 7fea3605f700 1 -- 172.21.15.116:0/3032288309 >> 172.21.15.116:0/3032288309 conn(0x7fea300678a0 msgr2=0x7fea30067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:01.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.496+0000 7fea3605f700 1 -- 172.21.15.116:0/3032288309 shutdown_connections 2020-11-20T12:22:01.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.496+0000 7fea3605f700 1 -- 172.21.15.116:0/3032288309 wait complete. 2020-11-20T12:22:01.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3605f700 1 Processor -- start 2020-11-20T12:22:01.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3605f700 1 -- start start 2020-11-20T12:22:01.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3605f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea300618e0 0x7fea300c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3605f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300adf20 0x7fea3014c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3605f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea30150e70 0x7fea3009f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3605f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea30165930 con 0x7fea300adf20 2020-11-20T12:22:01.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3605f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea30164eb0 con 0x7fea30150e70 2020-11-20T12:22:01.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3605f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea30165150 con 0x7fea300618e0 2020-11-20T12:22:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3485c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300adf20 0x7fea3014c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3485c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300adf20 0x7fea3014c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51572/0 (socket says 172.21.15.116:51572) 2020-11-20T12:22:01.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3485c700 1 -- 172.21.15.116:0/3051445888 learned_addr learned my addr 172.21.15.116:0/3051445888 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:01.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3505d700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea300618e0 0x7fea300c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.500+0000 7fea3585e700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea30150e70 0x7fea3009f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea3485c700 1 -- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea300618e0 msgr2=0x7fea300c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea3485c700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea300618e0 0x7fea300c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.510 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea3485c700 1 -- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea30150e70 msgr2=0x7fea3009f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea3485c700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea30150e70 0x7fea3009f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea3485c700 1 -- 172.21.15.116:0/3051445888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea301653f0 con 0x7fea300adf20 2020-11-20T12:22:01.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea3485c700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300adf20 0x7fea3014c0c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7fea2000c010 tx=0x7fea20015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:01.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea1d7fa700 1 -- 172.21.15.116:0/3051445888 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20011c80 con 0x7fea300adf20 2020-11-20T12:22:01.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea1d7fa700 1 -- 172.21.15.116:0/3051445888 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea20011de0 con 0x7fea300adf20 2020-11-20T12:22:01.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea1d7fa700 1 -- 172.21.15.116:0/3051445888 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2001e690 con 0x7fea300adf20 2020-11-20T12:22:01.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea3605f700 1 -- 172.21.15.116:0/3051445888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea30165690 con 0x7fea300adf20 2020-11-20T12:22:01.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea3605f700 1 -- 172.21.15.116:0/3051445888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3014c5b0 con 0x7fea300adf20 2020-11-20T12:22:01.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.504+0000 7fea3605f700 1 -- 172.21.15.116:0/3051445888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3005cf30 con 0x7fea300adf20 2020-11-20T12:22:01.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.508+0000 7fea1d7fa700 1 -- 172.21.15.116:0/3051445888 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fea2001a040 con 0x7fea300adf20 2020-11-20T12:22:01.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.508+0000 7fea1d7fa700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea080557c0 0x7fea08057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.508+0000 7fea1d7fa700 1 -- 172.21.15.116:0/3051445888 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fea2006f910 con 0x7fea300adf20 2020-11-20T12:22:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.508+0000 7fea3505d700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea080557c0 0x7fea08057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:01.517 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.508+0000 7fea3505d700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea080557c0 0x7fea08057c80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fea2c006fd0 tx=0x7fea2c006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:01.518 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.512+0000 7fea1d7fa700 1 -- 172.21.15.116:0/3051445888 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fea20045830 con 0x7fea300adf20 2020-11-20T12:22:01.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.724+0000 7fea3605f700 1 -- 172.21.15.116:0/3051445888 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea30060170 con 0x7fea300adf20 2020-11-20T12:22:01.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.724+0000 7fea1d7fa700 1 -- 172.21.15.116:0/3051445888 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2854 (secure 0 0 0) 0x7fea20023090 con 0x7fea300adf20 2020-11-20T12:22:01.728 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:01.729 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/1612795957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1612795957},{"type":"v1","addr":"172.21.15.116:6835","nonce":1612795957}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/3255680781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3255680781},{"type":"v1","addr":"172.21.15.173:6835","nonce":3255680781}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/4058046293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":4058046293},{"type":"v1","addr":"172.21.15.173:6837","nonce":4058046293}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:56.570302+0000","modified":"2020-11-20T12:21:58.589498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8148":{"gid":8148,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2775705840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2775705840},{"type":"v1","addr":"172.21.15.116:6833","nonce":2775705840}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-20T12:22:01.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 -- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea080557c0 msgr2=0x7fea08057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea080557c0 0x7fea08057c80 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fea2c006fd0 tx=0x7fea2c006df0).stop 2020-11-20T12:22:01.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 -- 172.21.15.116:0/3051445888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300adf20 msgr2=0x7fea3014c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:01.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300adf20 0x7fea3014c0c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7fea2000c010 tx=0x7fea20015000).stop 2020-11-20T12:22:01.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 -- 172.21.15.116:0/3051445888 shutdown_connections 2020-11-20T12:22:01.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fea080557c0 0x7fea08057c80 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fea30150e70 0x7fea3009f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fea300618e0 0x7fea300c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 --2- 172.21.15.116:0/3051445888 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fea300adf20 0x7fea3014c0c0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:01.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 -- 172.21.15.116:0/3051445888 >> 172.21.15.116:0/3051445888 conn(0x7fea300678a0 msgr2=0x7fea3014df60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:01.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 -- 172.21.15.116:0/3051445888 shutdown_connections 2020-11-20T12:22:01.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:01.728+0000 7fea3605f700 1 -- 172.21.15.116:0/3051445888 wait complete. 2020-11-20T12:22:01.734 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 68 2020-11-20T12:22:02.157 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:21:56.570302+0000', 'modified': '2020-11-20T12:21:58.589498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8148}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8148': {'gid': 8148, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.116:6833/2775705840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 2775705840}, {'type': 'v1', 'addr': '172.21.15.116:6833', 'nonce': 2775705840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:22:02.157 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:22:02.158 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:22:02.358 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":8148,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":5.7637620949999997,"mdsmap_epoch":68,"osdmap_epoch":118,"osdmap_epoch_barrier":118,"uptime":18.783819684000001} 2020-11-20T12:22:02.359 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 8148, "mdsmap_epoch": 68, "osdmap_epoch": 118, "osdmap_epoch_barrier": 118, "rank_uptime": 5.763762095, "state": "up:active", "uptime": 18.783819684, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:22:02.359 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:22:02.360 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:22:02.360 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:22:02.360 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:22:02.361 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:22:02.361 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:22:02.361 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:22:02.362 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:22:02.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:02.363 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:22:02.377 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:22:02.378 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:22:02.378 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:22:02.378 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:02.378 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:22:02.379 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:02.379 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:02.379 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:02.379 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:02.379 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:02.380 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:02.380 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:02.380 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:22:02.380 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:02.380 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:22:02.381 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:02.381 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:22:02.381 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:02.381 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:22:02.382 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:02.382 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:22:02.382 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:02.382 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:22:02.382 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:02.383 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:22:02.383 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:02.383 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:22:02.383 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:02.383 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:02.384 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:22:02.384 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:22:02.384 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:02.384 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:02.384 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:22:02.385 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:22:02.385 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:02.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:02.386 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:22:02.464 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:22:02.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:02.468 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:22:02.485 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:22:02.486 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:22:02.486 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:22:02.486 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:22:02.486 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:22:02.487 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:22:02.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:02.489 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:22:02.489 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:22:02.490 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:22:02.490 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:22:02.490 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:22:02.491 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:22:02.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:02.626 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:22:02.643 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:22:02.644 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:22:02.644 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:22:02.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:02.646 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2020-11-20T12:22:02.676 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:02.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.900+0000 7f16bef51700 1 Processor -- start 2020-11-20T12:22:02.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.900+0000 7f16bef51700 1 -- start start 2020-11-20T12:22:02.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bef51700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f16b80bf100 0x7f16b80bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:02.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bef51700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f16b80bac00 0x7f16b814db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:02.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bef51700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16b80b9d20 con 0x7f16b80bfa80 2020-11-20T12:22:02.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bef51700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b80b9f80 con 0x7f16b80bac00 2020-11-20T12:22:02.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bef51700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16b80ba220 con 0x7f16b80bf100 2020-11-20T12:22:02.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bd4ee700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f16b80bac00 0x7f16b814db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:02.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bd4ee700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f16b80bac00 0x7f16b814db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51584/0 (socket says 172.21.15.116:51584) 2020-11-20T12:22:02.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bd4ee700 1 -- 172.21.15.116:0/3674726742 learned_addr learned my addr 172.21.15.116:0/3674726742 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:02.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16b7fff700 1 --2- 172.21.15.116:0/3674726742 >> v2:172.21.15.173:3300/0 conn(0x7f16b80bf100 0x7f16b80bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:02.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bd4ee700 1 -- 172.21.15.116:0/3674726742 >> v2:172.21.15.173:3300/0 conn(0x7f16b80bf100 msgr2=0x7f16b80bfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:02.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bd4ee700 1 --2- 172.21.15.116:0/3674726742 >> v2:172.21.15.173:3300/0 conn(0x7f16b80bf100 0x7f16b80bfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:02.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bd4ee700 1 -- 172.21.15.116:0/3674726742 >> v1:172.21.15.173:6789/0 conn(0x7f16b80bfa80 legacy=0x7f16b8066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:02.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bd4ee700 1 -- 172.21.15.116:0/3674726742 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b80ba4c0 con 0x7f16b80bac00 2020-11-20T12:22:02.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16bd4ee700 1 --2- 172.21.15.116:0/3674726742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b80bac00 0x7f16b814db10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7f16b8065c70 tx=0x7f16b000a430).ready entity=mon.0 client_cookie=2c6609610c4c09e1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:02.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16b77fe700 1 -- 172.21.15.116:0/3674726742 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b000ce60 con 0x7f16b80bac00 2020-11-20T12:22:02.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16b77fe700 1 -- 172.21.15.116:0/3674726742 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b0010410 con 0x7f16b80bac00 2020-11-20T12:22:02.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.904+0000 7f16b77fe700 1 -- 172.21.15.116:0/3674726742 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0010790 con 0x7f16b80bac00 2020-11-20T12:22:02.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.908+0000 7f16bef51700 1 -- 172.21.15.116:0/3674726742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b80bac00 msgr2=0x7f16b814db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:02.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.908+0000 7f16bef51700 1 --2- 172.21.15.116:0/3674726742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b80bac00 0x7f16b814db10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7f16b8065c70 tx=0x7f16b000a430).stop 2020-11-20T12:22:02.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.908+0000 7f16bef51700 1 -- 172.21.15.116:0/3674726742 shutdown_connections 2020-11-20T12:22:02.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.908+0000 7f16bef51700 1 --2- 172.21.15.116:0/3674726742 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b80bac00 0x7f16b814db10 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:02.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.908+0000 7f16bef51700 1 --2- 172.21.15.116:0/3674726742 >> v2:172.21.15.173:3300/0 conn(0x7f16b80bf100 0x7f16b80bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:02.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.908+0000 7f16bef51700 1 -- 172.21.15.116:0/3674726742 >> 172.21.15.116:0/3674726742 conn(0x7f16b80c4f80 msgr2=0x7f16b80c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:02.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.908+0000 7f16bef51700 1 -- 172.21.15.116:0/3674726742 shutdown_connections 2020-11-20T12:22:02.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.908+0000 7f16bef51700 1 -- 172.21.15.116:0/3674726742 wait complete. 2020-11-20T12:22:02.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16bef51700 1 Processor -- start 2020-11-20T12:22:02.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16bef51700 1 -- start start 2020-11-20T12:22:02.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16bef51700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b80bac00 0x7f16b8162430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:02.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16bef51700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b80bf100 0x7f16b815f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:02.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16bef51700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b80bfa80 0x7f16b80a9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:02.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16bef51700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8162d70 con 0x7f16b80bf100 2020-11-20T12:22:02.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16bef51700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b8163010 con 0x7f16b80bac00 2020-11-20T12:22:02.917 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16bef51700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16b81632b0 con 0x7f16b80bfa80 2020-11-20T12:22:02.918 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16b7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b80bf100 0x7f16b815f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:02.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16b7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b80bf100 0x7f16b815f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51588/0 (socket says 172.21.15.116:51588) 2020-11-20T12:22:02.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16b7fff700 1 -- 172.21.15.116:0/1718740414 learned_addr learned my addr 172.21.15.116:0/1718740414 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:02.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16bd4ee700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b80bfa80 0x7f16b80a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:02.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16bcced700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b80bac00 0x7f16b8162430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:02.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16b7fff700 1 -- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b80bfa80 msgr2=0x7f16b80a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:02.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16b7fff700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b80bfa80 0x7f16b80a9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:02.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16b7fff700 1 -- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b80bac00 msgr2=0x7f16b8162430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:02.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16b7fff700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b80bac00 0x7f16b8162430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:02.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.912+0000 7f16b7fff700 1 -- 172.21.15.116:0/1718740414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b815f170 con 0x7f16b80bf100 2020-11-20T12:22:02.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.916+0000 7f16b7fff700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b80bf100 0x7f16b815f610 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f16a800d900 tx=0x7f16a8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:02.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.916+0000 7f16b4ff9700 1 -- 172.21.15.116:0/1718740414 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8009820 con 0x7f16b80bf100 2020-11-20T12:22:02.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.916+0000 7f16b4ff9700 1 -- 172.21.15.116:0/1718740414 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16a8009980 con 0x7f16b80bf100 2020-11-20T12:22:02.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.916+0000 7f16b4ff9700 1 -- 172.21.15.116:0/1718740414 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16a8009d20 con 0x7f16b80bf100 2020-11-20T12:22:02.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.916+0000 7f16bef51700 1 -- 172.21.15.116:0/1718740414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b815ee20 con 0x7f16b80bf100 2020-11-20T12:22:02.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.916+0000 7f16bef51700 1 -- 172.21.15.116:0/1718740414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b8162b40 con 0x7f16b80bf100 2020-11-20T12:22:02.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.916+0000 7f16bef51700 1 -- 172.21.15.116:0/1718740414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16b805d2a0 con 0x7f16b80bf100 2020-11-20T12:22:02.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.916+0000 7f16b4ff9700 1 -- 172.21.15.116:0/1718740414 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f16a8012630 con 0x7f16b80bf100 2020-11-20T12:22:02.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.920+0000 7f16b4ff9700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16940557c0 0x7f1694057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:02.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.920+0000 7f16b4ff9700 1 -- 172.21.15.116:0/1718740414 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f16a806e7f0 con 0x7f16b80bf100 2020-11-20T12:22:02.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.920+0000 7f16bcced700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16940557c0 0x7f1694057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:02.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.920+0000 7f16bcced700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16940557c0 0x7f1694057c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f16b815ca80 tx=0x7f16a0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:02.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:02.924+0000 7f16b4ff9700 1 -- 172.21.15.116:0/1718740414 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f16a8044690 con 0x7f16b80bf100 2020-11-20T12:22:03.087 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.084+0000 7f16bef51700 1 -- 172.21.15.116:0/1718740414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16b8061380 con 0x7f16b80bf100 2020-11-20T12:22:03.089 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.084+0000 7f16b4ff9700 1 -- 172.21.15.116:0/1718740414 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f16a8047cb0 con 0x7f16b80bf100 2020-11-20T12:22:03.093 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:03.093 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":118,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:56.569994+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:53.361384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:53.918177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:03.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 -- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16940557c0 msgr2=0x7f1694057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16940557c0 0x7f1694057c80 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f16b815ca80 tx=0x7f16a0009040).stop 2020-11-20T12:22:03.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 -- 172.21.15.116:0/1718740414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b80bf100 msgr2=0x7f16b815f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b80bf100 0x7f16b815f610 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7f16a800d900 tx=0x7f16a8008040).stop 2020-11-20T12:22:03.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 -- 172.21.15.116:0/1718740414 shutdown_connections 2020-11-20T12:22:03.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f16940557c0 0x7f1694057c80 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f16b80bfa80 0x7f16b80a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f16b80bac00 0x7f16b8162430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 --2- 172.21.15.116:0/1718740414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f16b80bf100 0x7f16b815f610 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 -- 172.21.15.116:0/1718740414 >> 172.21.15.116:0/1718740414 conn(0x7f16b80c4f80 msgr2=0x7f16b80c2000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:03.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 -- 172.21.15.116:0/1718740414 shutdown_connections 2020-11-20T12:22:03.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.088+0000 7f16bef51700 1 -- 172.21.15.116:0/1718740414 wait complete. 2020-11-20T12:22:03.116 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:21:53.361384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:21:53.918177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:22:03.117 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:22:03.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.292+0000 7f84d9196700 1 Processor -- start 2020-11-20T12:22:03.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.292+0000 7f84d9196700 1 -- start start 2020-11-20T12:22:03.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d9196700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f84d40664f0 0x7f84d4066930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d9196700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d4062880 con 0x7f84d40c0600 2020-11-20T12:22:03.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d9196700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d4067200 con 0x7f84d40c0220 2020-11-20T12:22:03.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d9196700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d4062ab0 con 0x7f84d40664f0 2020-11-20T12:22:03.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d259c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f84d40c0600 0x7f84d4060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:32798/0 (socket says 172.21.15.116:32798) 2020-11-20T12:22:03.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d259c700 1 -- 172.21.15.116:0/3132355793 learned_addr learned my addr 172.21.15.116:0/3132355793 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:03.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d359e700 1 --2- 172.21.15.116:0/3132355793 >> v2:172.21.15.173:3300/0 conn(0x7f84d40664f0 0x7f84d4066930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1121387285 0 0) 0x7f84d4062880 con 0x7f84d40c0600 2020-11-20T12:22:03.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84c0005450 con 0x7f84d40c0600 2020-11-20T12:22:03.308 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2348542261 0 0) 0x7f84c0005450 con 0x7f84d40c0600 2020-11-20T12:22:03.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 >> v2:172.21.15.173:3300/0 conn(0x7f84d40664f0 msgr2=0x7f84d4066930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:03.309 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 --2- 172.21.15.116:0/3132355793 >> v2:172.21.15.173:3300/0 conn(0x7f84d40664f0 0x7f84d4066930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 >> v1:172.21.15.173:6789/0 conn(0x7f84d40c0220 legacy=0x7f84d4063490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d4062d50 con 0x7f84d40c0600 2020-11-20T12:22:03.310 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f84c80014e0 con 0x7f84d40c0600 2020-11-20T12:22:03.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 >> v1:172.21.15.116:6789/0 conn(0x7f84d40c0600 legacy=0x7f84d4060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 --2- 172.21.15.116:0/3132355793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 0x7f84c0005d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 --2- 172.21.15.116:0/3132355793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84c00066f0 0x7f84c0006b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 --2- 172.21.15.116:0/3132355793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84c0007060 0x7f84c000d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d4062ab0 con 0x7f84c0005940 2020-11-20T12:22:03.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c0007570 con 0x7f84c00066f0 2020-11-20T12:22:03.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84c000de60 con 0x7f84c0007060 2020-11-20T12:22:03.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d2d9d700 1 --2- 172.21.15.116:0/3132355793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 0x7f84c0005d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d259c700 1 --2- 172.21.15.116:0/3132355793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84c00066f0 0x7f84c0006b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.296+0000 7f84d359e700 1 --2- 172.21.15.116:0/3132355793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84c0007060 0x7f84c000d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d2d9d700 1 -- 172.21.15.116:0/3132355793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84c0007060 msgr2=0x7f84c000d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:03.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d2d9d700 1 --2- 172.21.15.116:0/3132355793 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84c0007060 0x7f84c000d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d2d9d700 1 -- 172.21.15.116:0/3132355793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84c00066f0 msgr2=0x7f84c0006b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:03.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d2d9d700 1 --2- 172.21.15.116:0/3132355793 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84c00066f0 0x7f84c0006b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d2d9d700 1 -- 172.21.15.116:0/3132355793 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84c0005450 con 0x7f84c0005940 2020-11-20T12:22:03.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d359e700 1 -- 172.21.15.116:0/3132355793 reap_dead start 2020-11-20T12:22:03.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d2d9d700 1 --2- 172.21.15.116:0/3132355793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 0x7f84c0005d60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f84c4001920 tx=0x7f84c4007b90).ready entity=mon.0 client_cookie=4f52f81be2f3640f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:03.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4007020 con 0x7f84c0005940 2020-11-20T12:22:03.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c4007180 con 0x7f84c0005940 2020-11-20T12:22:03.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d1d9b700 1 -- 172.21.15.116:0/3132355793 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c400c830 con 0x7f84c0005940 2020-11-20T12:22:03.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d9196700 1 -- 172.21.15.116:0/3132355793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 msgr2=0x7f84c0005d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d9196700 1 --2- 172.21.15.116:0/3132355793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 0x7f84c0005d60 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7f84c4001920 tx=0x7f84c4007b90).stop 2020-11-20T12:22:03.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d9196700 1 -- 172.21.15.116:0/3132355793 shutdown_connections 2020-11-20T12:22:03.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d9196700 1 --2- 172.21.15.116:0/3132355793 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 0x7f84c0005d60 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d9196700 1 -- 172.21.15.116:0/3132355793 >> 172.21.15.116:0/3132355793 conn(0x7f84d405ba80 msgr2=0x7f84d40c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:03.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d9196700 1 -- 172.21.15.116:0/3132355793 shutdown_connections 2020-11-20T12:22:03.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.300+0000 7f84d9196700 1 -- 172.21.15.116:0/3132355793 wait complete. 2020-11-20T12:22:03.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.304+0000 7f84d9196700 1 Processor -- start 2020-11-20T12:22:03.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.304+0000 7f84d9196700 1 -- start start 2020-11-20T12:22:03.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d9196700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 0x7f84d413dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d9196700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84d40bc200 0x7f84d41501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d9196700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d40bc680 0x7f84d41506e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d9196700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d40bde20 con 0x7f84c0005940 2020-11-20T12:22:03.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d9196700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d40be0c0 con 0x7f84d40bc200 2020-11-20T12:22:03.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d9196700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d40be360 con 0x7f84d40bc680 2020-11-20T12:22:03.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d2d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 0x7f84d413dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d2d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 0x7f84d413dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51606/0 (socket says 172.21.15.116:51606) 2020-11-20T12:22:03.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d2d9d700 1 -- 172.21.15.116:0/331574116 learned_addr learned my addr 172.21.15.116:0/331574116 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:03.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d2d9d700 1 -- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d40bc680 msgr2=0x7f84d41506e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d259c700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84d40bc200 0x7f84d41501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d359e700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d40bc680 0x7f84d41506e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d2d9d700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d40bc680 0x7f84d41506e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d2d9d700 1 -- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84d40bc200 msgr2=0x7f84d41501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d2d9d700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84d40bc200 0x7f84d41501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d2d9d700 1 -- 172.21.15.116:0/331574116 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d40be600 con 0x7f84c0005940 2020-11-20T12:22:03.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d259c700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84d40bc200 0x7f84d41501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:03.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d2d9d700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 0x7f84d413dfa0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7f84c4013a40 tx=0x7f84c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:03.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84baffd700 1 -- 172.21.15.116:0/331574116 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4020440 con 0x7f84c0005940 2020-11-20T12:22:03.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84baffd700 1 -- 172.21.15.116:0/331574116 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c40205a0 con 0x7f84c0005940 2020-11-20T12:22:03.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d9196700 1 -- 172.21.15.116:0/331574116 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d40be8a0 con 0x7f84c0005940 2020-11-20T12:22:03.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84baffd700 1 -- 172.21.15.116:0/331574116 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c4020940 con 0x7f84c0005940 2020-11-20T12:22:03.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d9196700 1 -- 172.21.15.116:0/331574116 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d413e490 con 0x7f84c0005940 2020-11-20T12:22:03.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.308+0000 7f84d9196700 1 -- 172.21.15.116:0/331574116 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d40bea30 con 0x7f84c0005940 2020-11-20T12:22:03.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.312+0000 7f84baffd700 1 -- 172.21.15.116:0/331574116 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f84c4026020 con 0x7f84c0005940 2020-11-20T12:22:03.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.312+0000 7f84baffd700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84b00557f0 0x7f84b0057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.312+0000 7f84baffd700 1 -- 172.21.15.116:0/331574116 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f84c4070570 con 0x7f84c0005940 2020-11-20T12:22:03.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.312+0000 7f84d259c700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84b00557f0 0x7f84b0057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.312+0000 7f84d259c700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84b00557f0 0x7f84b0057cb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f84d4164360 tx=0x7f84c8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:03.334 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.316+0000 7f84baffd700 1 -- 172.21.15.116:0/331574116 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f84c4046300 con 0x7f84c0005940 2020-11-20T12:22:03.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.476+0000 7f84d9196700 1 -- 172.21.15.116:0/331574116 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f84d40bde20 con 0x7f84c0005940 2020-11-20T12:22:03.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.476+0000 7f84baffd700 1 -- 172.21.15.116:0/331574116 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v118) v1 ==== 110+0+10 (secure 0 0 0) 0x7f84c4049920 con 0x7f84c0005940 2020-11-20T12:22:03.481 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:22:03.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 -- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84b00557f0 msgr2=0x7f84b0057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84b00557f0 0x7f84b0057cb0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f84d4164360 tx=0x7f84c8009040).stop 2020-11-20T12:22:03.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 -- 172.21.15.116:0/331574116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 msgr2=0x7f84d413dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 0x7f84d413dfa0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7f84c4013a40 tx=0x7f84c4015000).stop 2020-11-20T12:22:03.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 -- 172.21.15.116:0/331574116 shutdown_connections 2020-11-20T12:22:03.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f84b00557f0 0x7f84b0057cb0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f84d40bc680 0x7f84d41506e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f84c0005940 0x7f84d413dfa0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 --2- 172.21.15.116:0/331574116 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f84d40bc200 0x7f84d41501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 -- 172.21.15.116:0/331574116 >> 172.21.15.116:0/331574116 conn(0x7f84d405ba80 msgr2=0x7f84d40c3020 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:03.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 -- 172.21.15.116:0/331574116 shutdown_connections 2020-11-20T12:22:03.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.480+0000 7f84d9196700 1 -- 172.21.15.116:0/331574116 wait complete. 2020-11-20T12:22:03.511 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:22:03.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.684+0000 7f8870289700 1 Processor -- start 2020-11-20T12:22:03.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.684+0000 7f8870289700 1 -- start start 2020-11-20T12:22:03.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f8870289700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8868148cf0 0x7f88680a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f8870289700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88680c5060 0x7f886814f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f8870289700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88680a8fa0 con 0x7f88680c4bb0 2020-11-20T12:22:03.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f8870289700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8868152710 con 0x7f8868148cf0 2020-11-20T12:22:03.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f8870289700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88681529b0 con 0x7f88680c5060 2020-11-20T12:22:03.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886e025700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8868148cf0 0x7f88680a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886e025700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8868148cf0 0x7f88680a8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51614/0 (socket says 172.21.15.116:51614) 2020-11-20T12:22:03.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886d824700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f88680c4bb0 0x7f88680a66e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:32816/0 (socket says 172.21.15.116:32816) 2020-11-20T12:22:03.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886e025700 1 -- 172.21.15.116:0/2853257452 learned_addr learned my addr 172.21.15.116:0/2853257452 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:03.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886e826700 1 --2- 172.21.15.116:0/2853257452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88680c5060 0x7f886814f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886d023700 1 -- 172.21.15.116:0/2853257452 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1916122395 0 0) 0x7f88580010d0 con 0x7f88680c4bb0 2020-11-20T12:22:03.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886d023700 1 -- 172.21.15.116:0/2853257452 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8850003240 con 0x7f88680c4bb0 2020-11-20T12:22:03.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886e025700 1 -- 172.21.15.116:0/2853257452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88680c5060 msgr2=0x7f886814f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:03.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886e025700 1 --2- 172.21.15.116:0/2853257452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88680c5060 0x7f886814f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886e025700 1 -- 172.21.15.116:0/2853257452 >> v1:172.21.15.116:6789/0 conn(0x7f88680c4bb0 legacy=0x7f88680a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886e025700 1 -- 172.21.15.116:0/2853257452 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8868152c50 con 0x7f8868148cf0 2020-11-20T12:22:03.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886e025700 1 --2- 172.21.15.116:0/2853257452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8868148cf0 0x7f88680a8ab0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f8864006fd0 tx=0x7f8864006eb0).ready entity=mon.0 client_cookie=9e4f956a7c791e37 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:03.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886d023700 1 -- 172.21.15.116:0/2853257452 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864009720 con 0x7f8868148cf0 2020-11-20T12:22:03.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886d023700 1 -- 172.21.15.116:0/2853257452 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8864009880 con 0x7f8868148cf0 2020-11-20T12:22:03.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.688+0000 7f886d023700 1 -- 172.21.15.116:0/2853257452 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864009c00 con 0x7f8868148cf0 2020-11-20T12:22:03.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.692+0000 7f8870289700 1 -- 172.21.15.116:0/2853257452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8868148cf0 msgr2=0x7f88680a8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.692+0000 7f8870289700 1 --2- 172.21.15.116:0/2853257452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8868148cf0 0x7f88680a8ab0 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7f8864006fd0 tx=0x7f8864006eb0).stop 2020-11-20T12:22:03.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.692+0000 7f8870289700 1 -- 172.21.15.116:0/2853257452 shutdown_connections 2020-11-20T12:22:03.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.692+0000 7f8870289700 1 --2- 172.21.15.116:0/2853257452 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8868148cf0 0x7f88680a8ab0 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.692+0000 7f8870289700 1 --2- 172.21.15.116:0/2853257452 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88680c5060 0x7f886814f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.692+0000 7f8870289700 1 -- 172.21.15.116:0/2853257452 >> 172.21.15.116:0/2853257452 conn(0x7f886805ba80 msgr2=0x7f88680c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:03.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.692+0000 7f8870289700 1 -- 172.21.15.116:0/2853257452 shutdown_connections 2020-11-20T12:22:03.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.692+0000 7f8870289700 1 -- 172.21.15.116:0/2853257452 wait complete. 2020-11-20T12:22:03.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.692+0000 7f8870289700 1 Processor -- start 2020-11-20T12:22:03.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.692+0000 7f8870289700 1 -- start start 2020-11-20T12:22:03.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f8870289700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88680c4bb0 0x7f88680a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f8870289700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88680c5060 0x7f886814afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f8870289700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8868148cf0 0x7f886814b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f8870289700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886814bf60 con 0x7f8868148cf0 2020-11-20T12:22:03.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f8870289700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886814bcc0 con 0x7f88680c5060 2020-11-20T12:22:03.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f8870289700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f886814ef80 con 0x7f88680c4bb0 2020-11-20T12:22:03.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f886e826700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8868148cf0 0x7f886814b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f886e826700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8868148cf0 0x7f886814b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51624/0 (socket says 172.21.15.116:51624) 2020-11-20T12:22:03.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f886e826700 1 -- 172.21.15.116:0/2125492820 learned_addr learned my addr 172.21.15.116:0/2125492820 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:03.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f886e025700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88680c4bb0 0x7f88680a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f886d824700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88680c5060 0x7f886814afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f886e826700 1 -- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88680c4bb0 msgr2=0x7f88680a92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f886e826700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88680c4bb0 0x7f88680a92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f886e826700 1 -- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88680c5060 msgr2=0x7f886814afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:03.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f886e826700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88680c5060 0x7f886814afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:03.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f886e826700 1 -- 172.21.15.116:0/2125492820 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886814f220 con 0x7f8868148cf0 2020-11-20T12:22:03.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f886e826700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8868148cf0 0x7f886814b4a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f886000ed20 tx=0x7f88600091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:03.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f88567fc700 1 -- 172.21.15.116:0/2125492820 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886000cb00 con 0x7f8868148cf0 2020-11-20T12:22:03.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f88567fc700 1 -- 172.21.15.116:0/2125492820 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886000cc60 con 0x7f8868148cf0 2020-11-20T12:22:03.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f8870289700 1 -- 172.21.15.116:0/2125492820 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f886814f4c0 con 0x7f8868148cf0 2020-11-20T12:22:03.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f88567fc700 1 -- 172.21.15.116:0/2125492820 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88600123f0 con 0x7f8868148cf0 2020-11-20T12:22:03.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.696+0000 7f8870289700 1 -- 172.21.15.116:0/2125492820 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88680a97d0 con 0x7f8868148cf0 2020-11-20T12:22:03.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.700+0000 7f8870289700 1 -- 172.21.15.116:0/2125492820 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886805d2a0 con 0x7f8868148cf0 2020-11-20T12:22:03.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.700+0000 7f88567fc700 1 -- 172.21.15.116:0/2125492820 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8860022070 con 0x7f8868148cf0 2020-11-20T12:22:03.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.700+0000 7f88567fc700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88400557c0 0x7f8840057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:03.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.700+0000 7f88567fc700 1 -- 172.21.15.116:0/2125492820 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f886006edd0 con 0x7f8868148cf0 2020-11-20T12:22:03.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.700+0000 7f886e025700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88400557c0 0x7f8840057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:03.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.704+0000 7f886e025700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88400557c0 0x7f8840057c80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f8864009420 tx=0x7f8864015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:03.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.708+0000 7f88567fc700 1 -- 172.21.15.116:0/2125492820 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8860044c70 con 0x7f8868148cf0 2020-11-20T12:22:03.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.864+0000 7f8870289700 1 -- 172.21.15.116:0/2125492820 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f88680c4220 con 0x7f8868148cf0 2020-11-20T12:22:04.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f88567fc700 1 -- 172.21.15.116:0/2125492820 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v118) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8860016070 con 0x7f8868148cf0 2020-11-20T12:22:04.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 -- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88400557c0 msgr2=0x7f8840057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88400557c0 0x7f8840057c80 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f8864009420 tx=0x7f8864015000).stop 2020-11-20T12:22:04.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 -- 172.21.15.116:0/2125492820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8868148cf0 msgr2=0x7f886814b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8868148cf0 0x7f886814b4a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7f886000ed20 tx=0x7f88600091d0).stop 2020-11-20T12:22:04.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 -- 172.21.15.116:0/2125492820 shutdown_connections 2020-11-20T12:22:04.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f88400557c0 0x7f8840057c80 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8868148cf0 0x7f886814b4a0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f88680c4bb0 0x7f88680a92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 --2- 172.21.15.116:0/2125492820 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f88680c5060 0x7f886814afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 -- 172.21.15.116:0/2125492820 >> 172.21.15.116:0/2125492820 conn(0x7f886805ba80 msgr2=0x7f8868062880 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:04.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 -- 172.21.15.116:0/2125492820 shutdown_connections 2020-11-20T12:22:04.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:03.868+0000 7f8870289700 1 -- 172.21.15.116:0/2125492820 wait complete. 2020-11-20T12:22:04.256 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:04.257 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:22:04.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.432+0000 7f2e1d523700 1 Processor -- start 2020-11-20T12:22:04.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.432+0000 7f2e1d523700 1 -- start start 2020-11-20T12:22:04.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e1d523700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2e18065770 0x7f2e18065b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:04.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e1d523700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e180ca810 0x7f2e181557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:04.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e1d523700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e18155eb0 con 0x7f2e180a8d20 2020-11-20T12:22:04.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e1d523700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e180cac90 con 0x7f2e180ca810 2020-11-20T12:22:04.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e1d523700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e18155c90 con 0x7f2e18065770 2020-11-20T12:22:04.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e0ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e180ca810 0x7f2e181557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:04.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e0ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f2e180ca810 0x7f2e181557a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51630/0 (socket says 172.21.15.116:51630) 2020-11-20T12:22:04.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e0ffff700 1 -- 172.21.15.116:0/1521498890 learned_addr learned my addr 172.21.15.116:0/1521498890 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:04.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e17fff700 1 --1- 172.21.15.116:0/1521498890 >> v1:172.21.15.173:6789/0 conn(0x7f2e180a8d20 0x7f2e181521e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:57296/0 (socket says 172.21.15.116:57296) 2020-11-20T12:22:04.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e16ffd700 1 --2- 172.21.15.116:0/1521498890 >> v2:172.21.15.173:3300/0 conn(0x7f2e18065770 0x7f2e18065b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:04.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e167fc700 1 -- 172.21.15.116:0/1521498890 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1218953161 0 0) 0x7f2e18155eb0 con 0x7f2e180a8d20 2020-11-20T12:22:04.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e167fc700 1 -- 172.21.15.116:0/1521498890 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2df8003240 con 0x7f2e180a8d20 2020-11-20T12:22:04.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e0ffff700 1 -- 172.21.15.116:0/1521498890 >> v2:172.21.15.173:3300/0 conn(0x7f2e18065770 msgr2=0x7f2e18065b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:04.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e0ffff700 1 --2- 172.21.15.116:0/1521498890 >> v2:172.21.15.173:3300/0 conn(0x7f2e18065770 0x7f2e18065b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e0ffff700 1 -- 172.21.15.116:0/1521498890 >> v1:172.21.15.173:6789/0 conn(0x7f2e180a8d20 legacy=0x7f2e181521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e0ffff700 1 -- 172.21.15.116:0/1521498890 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18156150 con 0x7f2e180ca810 2020-11-20T12:22:04.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e16ffd700 1 --2- 172.21.15.116:0/1521498890 >> v2:172.21.15.173:3300/0 conn(0x7f2e18065770 0x7f2e18065b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:04.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e0ffff700 1 --2- 172.21.15.116:0/1521498890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e180ca810 0x7f2e181557a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f2e18066cc0 tx=0x7f2e00006eb0).ready entity=mon.0 client_cookie=1704ac0a0b633227 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:04.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e167fc700 1 -- 172.21.15.116:0/1521498890 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e000097d0 con 0x7f2e180ca810 2020-11-20T12:22:04.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e167fc700 1 -- 172.21.15.116:0/1521498890 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e00009930 con 0x7f2e180ca810 2020-11-20T12:22:04.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.436+0000 7f2e167fc700 1 -- 172.21.15.116:0/1521498890 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e00009cb0 con 0x7f2e180ca810 2020-11-20T12:22:04.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.440+0000 7f2e1d523700 1 -- 172.21.15.116:0/1521498890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e180ca810 msgr2=0x7f2e181557a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.440+0000 7f2e1d523700 1 --2- 172.21.15.116:0/1521498890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e180ca810 0x7f2e181557a0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f2e18066cc0 tx=0x7f2e00006eb0).stop 2020-11-20T12:22:04.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.440+0000 7f2e1d523700 1 -- 172.21.15.116:0/1521498890 shutdown_connections 2020-11-20T12:22:04.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.440+0000 7f2e1d523700 1 --2- 172.21.15.116:0/1521498890 >> v2:172.21.15.173:3300/0 conn(0x7f2e18065770 0x7f2e18065b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.440+0000 7f2e1d523700 1 --2- 172.21.15.116:0/1521498890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e180ca810 0x7f2e181557a0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.440+0000 7f2e1d523700 1 -- 172.21.15.116:0/1521498890 >> 172.21.15.116:0/1521498890 conn(0x7f2e180647f0 msgr2=0x7f2e180c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:04.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.440+0000 7f2e1d523700 1 -- 172.21.15.116:0/1521498890 shutdown_connections 2020-11-20T12:22:04.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.440+0000 7f2e1d523700 1 -- 172.21.15.116:0/1521498890 wait complete. 2020-11-20T12:22:04.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.440+0000 7f2e1d523700 1 Processor -- start 2020-11-20T12:22:04.448 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.440+0000 7f2e1d523700 1 -- start start 2020-11-20T12:22:04.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e1d523700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18065770 0x7f2e180c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:04.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e1d523700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e180a8d20 0x7f2e1814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:04.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e1d523700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e180ca810 0x7f2e18154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:04.449 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e1d523700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e180add20 con 0x7f2e180ca810 2020-11-20T12:22:04.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e1d523700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1814fc00 con 0x7f2e18065770 2020-11-20T12:22:04.450 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e1d523700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e1814fea0 con 0x7f2e180a8d20 2020-11-20T12:22:04.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e180ca810 0x7f2e18154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:04.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e17fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e180ca810 0x7f2e18154110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51638/0 (socket says 172.21.15.116:51638) 2020-11-20T12:22:04.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e17fff700 1 -- 172.21.15.116:0/612993777 learned_addr learned my addr 172.21.15.116:0/612993777 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:04.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e16ffd700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18065770 0x7f2e180c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:04.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e0ffff700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e180a8d20 0x7f2e1814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:04.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e17fff700 1 -- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e180a8d20 msgr2=0x7f2e1814dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e17fff700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e180a8d20 0x7f2e1814dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e17fff700 1 -- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18065770 msgr2=0x7f2e180c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e17fff700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18065770 0x7f2e180c9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.453 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e17fff700 1 -- 172.21.15.116:0/612993777 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e18150140 con 0x7f2e180ca810 2020-11-20T12:22:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e0ffff700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e180a8d20 0x7f2e1814dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e17fff700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e180ca810 0x7f2e18154110 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f2e1000ca90 tx=0x7f2e10007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e0effd700 1 -- 172.21.15.116:0/612993777 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e10005ac0 con 0x7f2e180ca810 2020-11-20T12:22:04.454 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e0effd700 1 -- 172.21.15.116:0/612993777 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e10005c20 con 0x7f2e180ca810 2020-11-20T12:22:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e1d523700 1 -- 172.21.15.116:0/612993777 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e181503e0 con 0x7f2e180ca810 2020-11-20T12:22:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e0effd700 1 -- 172.21.15.116:0/612993777 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e10007110 con 0x7f2e180ca810 2020-11-20T12:22:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.444+0000 7f2e1d523700 1 -- 172.21.15.116:0/612993777 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e1814e1d0 con 0x7f2e180ca810 2020-11-20T12:22:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.448+0000 7f2e1d523700 1 -- 172.21.15.116:0/612993777 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e1805d2a0 con 0x7f2e180ca810 2020-11-20T12:22:04.455 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.448+0000 7f2e0effd700 1 -- 172.21.15.116:0/612993777 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2e10010030 con 0x7f2e180ca810 2020-11-20T12:22:04.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.448+0000 7f2e0effd700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2df40557c0 0x7f2df4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.448+0000 7f2e0effd700 1 -- 172.21.15.116:0/612993777 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2e1006e7a0 con 0x7f2e180ca810 2020-11-20T12:22:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.448+0000 7f2e16ffd700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2df40557c0 0x7f2df4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.452+0000 7f2e16ffd700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2df40557c0 0x7f2df4057c80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f2e0800ba80 tx=0x7f2e08008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:04.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.452+0000 7f2e0effd700 1 -- 172.21.15.116:0/612993777 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2e10044530 con 0x7f2e180ca810 2020-11-20T12:22:04.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.616+0000 7f2e1d523700 1 -- 172.21.15.116:0/612993777 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e18063850 con 0x7f2e180ca810 2020-11-20T12:22:04.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.616+0000 7f2e0effd700 1 -- 172.21.15.116:0/612993777 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+10 (secure 0 0 0) 0x7f2e10047b50 con 0x7f2e180ca810 2020-11-20T12:22:04.620 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:04.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 -- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2df40557c0 msgr2=0x7f2df4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2df40557c0 0x7f2df4057c80 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f2e0800ba80 tx=0x7f2e08008040).stop 2020-11-20T12:22:04.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 -- 172.21.15.116:0/612993777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e180ca810 msgr2=0x7f2e18154110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e180ca810 0x7f2e18154110 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f2e1000ca90 tx=0x7f2e10007de0).stop 2020-11-20T12:22:04.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 -- 172.21.15.116:0/612993777 shutdown_connections 2020-11-20T12:22:04.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2df40557c0 0x7f2df4057c80 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2e180ca810 0x7f2e18154110 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2e18065770 0x7f2e180c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 --2- 172.21.15.116:0/612993777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2e180a8d20 0x7f2e1814dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 -- 172.21.15.116:0/612993777 >> 172.21.15.116:0/612993777 conn(0x7f2e180647f0 msgr2=0x7f2e180c5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:04.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 -- 172.21.15.116:0/612993777 shutdown_connections 2020-11-20T12:22:04.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.620+0000 7f2e1d523700 1 -- 172.21.15.116:0/612993777 wait complete. 2020-11-20T12:22:04.647 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:22:04.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.816+0000 7f98e0de1700 1 Processor -- start 2020-11-20T12:22:04.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.816+0000 7f98e0de1700 1 -- start start 2020-11-20T12:22:04.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.816+0000 7f98e0de1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f98dc0aa7d0 0x7f98dc0a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:04.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.816+0000 7f98e0de1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc0ace40 con 0x7f98dc0aabb0 2020-11-20T12:22:04.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.816+0000 7f98e0de1700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98dc1526c0 con 0x7f98dc0ad540 2020-11-20T12:22:04.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.816+0000 7f98e0de1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98dc152960 con 0x7f98dc0aa7d0 2020-11-20T12:22:04.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98db7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f98dc0aa7d0 0x7f98dc0a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:04.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98daffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f98dc0aabb0 0x7f98dc151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:32848/0 (socket says 172.21.15.116:32848) 2020-11-20T12:22:04.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98daffd700 1 -- 172.21.15.116:0/2768358937 learned_addr learned my addr 172.21.15.116:0/2768358937 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:04.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98db7fe700 1 -- 172.21.15.116:0/2768358937 >> v1:172.21.15.173:6789/0 conn(0x7f98dc0ad540 legacy=0x7f98dc0ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98db7fe700 1 -- 172.21.15.116:0/2768358937 >> v1:172.21.15.116:6789/0 conn(0x7f98dc0aabb0 legacy=0x7f98dc151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98db7fe700 1 -- 172.21.15.116:0/2768358937 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc152c00 con 0x7f98dc0aa7d0 2020-11-20T12:22:04.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98da7fc700 1 -- 172.21.15.116:0/2768358937 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1391628130 0 0) 0x7f98dc1526c0 con 0x7f98dc0ad540 2020-11-20T12:22:04.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98db7fe700 1 --2- 172.21.15.116:0/2768358937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98dc0aa7d0 0x7f98dc0a8400 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f98cc00b9a0 tx=0x7f98cc006df0).ready entity=mon.0 client_cookie=eeadfc1e86524347 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:04.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98da7fc700 1 -- 172.21.15.116:0/2768358937 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc0097b0 con 0x7f98dc0aa7d0 2020-11-20T12:22:04.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98da7fc700 1 -- 172.21.15.116:0/2768358937 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98cc009910 con 0x7f98dc0aa7d0 2020-11-20T12:22:04.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98da7fc700 1 -- 172.21.15.116:0/2768358937 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98cc009c90 con 0x7f98dc0aa7d0 2020-11-20T12:22:04.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98e0de1700 1 -- 172.21.15.116:0/2768358937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98dc0aa7d0 msgr2=0x7f98dc0a8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98e0de1700 1 --2- 172.21.15.116:0/2768358937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98dc0aa7d0 0x7f98dc0a8400 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7f98cc00b9a0 tx=0x7f98cc006df0).stop 2020-11-20T12:22:04.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98e0de1700 1 -- 172.21.15.116:0/2768358937 shutdown_connections 2020-11-20T12:22:04.827 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98e0de1700 1 --2- 172.21.15.116:0/2768358937 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98dc0aa7d0 0x7f98dc0a8400 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98e0de1700 1 -- 172.21.15.116:0/2768358937 >> 172.21.15.116:0/2768358937 conn(0x7f98dc0678a0 msgr2=0x7f98dc067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:04.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.820+0000 7f98e0de1700 1 -- 172.21.15.116:0/2768358937 shutdown_connections 2020-11-20T12:22:04.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98e0de1700 1 -- 172.21.15.116:0/2768358937 wait complete. 2020-11-20T12:22:04.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98e0de1700 1 Processor -- start 2020-11-20T12:22:04.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98e0de1700 1 -- start start 2020-11-20T12:22:04.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98e0de1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98dc0aabb0 0x7f98dc14dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:04.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98e0de1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98dc0ad540 0x7f98dc165440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:04.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98e0de1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98dc0c5120 0x7f98dc150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:04.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98e0de1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc142b50 con 0x7f98dc0c5120 2020-11-20T12:22:04.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98e0de1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc14e0d0 con 0x7f98dc0ad540 2020-11-20T12:22:04.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98e0de1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98dc165930 con 0x7f98dc0aabb0 2020-11-20T12:22:04.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98dbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98dc0c5120 0x7f98dc150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:04.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98daffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98dc0ad540 0x7f98dc165440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:04.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98dbfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98dc0c5120 0x7f98dc150de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51652/0 (socket says 172.21.15.116:51652) 2020-11-20T12:22:04.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98dbfff700 1 -- 172.21.15.116:0/2794754025 learned_addr learned my addr 172.21.15.116:0/2794754025 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:04.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.824+0000 7f98db7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98dc0aabb0 0x7f98dc14dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:04.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98dbfff700 1 -- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98dc0aabb0 msgr2=0x7f98dc14dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98dbfff700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98dc0aabb0 0x7f98dc14dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98dbfff700 1 -- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98dc0ad540 msgr2=0x7f98dc165440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:04.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98dbfff700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98dc0ad540 0x7f98dc165440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:04.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98dbfff700 1 -- 172.21.15.116:0/2794754025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98dc158000 con 0x7f98dc0c5120 2020-11-20T12:22:04.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98daffd700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98dc0ad540 0x7f98dc165440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:04.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98dbfff700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98dc0c5120 0x7f98dc150de0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f98d400c860 tx=0x7f98d400cb70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:04.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98bb7fe700 1 -- 172.21.15.116:0/2794754025 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d400a650 con 0x7f98dc0c5120 2020-11-20T12:22:04.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98bb7fe700 1 -- 172.21.15.116:0/2794754025 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d400a7b0 con 0x7f98dc0c5120 2020-11-20T12:22:04.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98bb7fe700 1 -- 172.21.15.116:0/2794754025 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d400ab50 con 0x7f98dc0c5120 2020-11-20T12:22:04.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98e0de1700 1 -- 172.21.15.116:0/2794754025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98dc1582a0 con 0x7f98dc0c5120 2020-11-20T12:22:04.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98e0de1700 1 -- 172.21.15.116:0/2794754025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98dc1506a0 con 0x7f98dc0c5120 2020-11-20T12:22:04.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.828+0000 7f98e0de1700 1 -- 172.21.15.116:0/2794754025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98dc05cf30 con 0x7f98dc0c5120 2020-11-20T12:22:04.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.832+0000 7f98bb7fe700 1 -- 172.21.15.116:0/2794754025 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f98d4010070 con 0x7f98dc0c5120 2020-11-20T12:22:04.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.832+0000 7f98bb7fe700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98b40557c0 0x7f98b4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:04.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.832+0000 7f98bb7fe700 1 -- 172.21.15.116:0/2794754025 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f98d406e4d0 con 0x7f98dc0c5120 2020-11-20T12:22:04.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.832+0000 7f98db7fe700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98b40557c0 0x7f98b4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:04.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.832+0000 7f98db7fe700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98b40557c0 0x7f98b4057c80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f98dc060680 tx=0x7f98cc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:04.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.836+0000 7f98bb7fe700 1 -- 172.21.15.116:0/2794754025 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f98d40442f0 con 0x7f98dc0c5120 2020-11-20T12:22:04.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.992+0000 7f98e0de1700 1 -- 172.21.15.116:0/2794754025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98dc153390 con 0x7f98dc0c5120 2020-11-20T12:22:04.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.992+0000 7f98bb7fe700 1 -- 172.21.15.116:0/2794754025 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+10 (secure 0 0 0) 0x7f98d4047910 con 0x7f98dc0c5120 2020-11-20T12:22:04.998 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:05.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 -- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98b40557c0 msgr2=0x7f98b4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98b40557c0 0x7f98b4057c80 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f98dc060680 tx=0x7f98cc015000).stop 2020-11-20T12:22:05.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 -- 172.21.15.116:0/2794754025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98dc0c5120 msgr2=0x7f98dc150de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98dc0c5120 0x7f98dc150de0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f98d400c860 tx=0x7f98d400cb70).stop 2020-11-20T12:22:05.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 -- 172.21.15.116:0/2794754025 shutdown_connections 2020-11-20T12:22:05.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f98b40557c0 0x7f98b4057c80 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f98dc0c5120 0x7f98dc150de0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f98dc0aabb0 0x7f98dc14dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 --2- 172.21.15.116:0/2794754025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98dc0ad540 0x7f98dc165440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 -- 172.21.15.116:0/2794754025 >> 172.21.15.116:0/2794754025 conn(0x7f98dc0678a0 msgr2=0x7f98dc153d90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:05.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 -- 172.21.15.116:0/2794754025 shutdown_connections 2020-11-20T12:22:05.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:04.996+0000 7f98e0de1700 1 -- 172.21.15.116:0/2794754025 wait complete. 2020-11-20T12:22:05.029 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:22:05.030 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:05.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.212+0000 7ff66759e700 1 Processor -- start 2020-11-20T12:22:05.217 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.212+0000 7ff66759e700 1 -- start start 2020-11-20T12:22:05.219 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 0x7ff6680aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66759e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff66813b380 0x7ff6680aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66759e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff6680acd30 0x7ff6680ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.220 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66759e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6680ada00 con 0x7ff6680acd30 2020-11-20T12:22:05.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66759e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6680adca0 con 0x7ff66813b380 2020-11-20T12:22:05.221 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6680adf40 con 0x7ff66813afa0 2020-11-20T12:22:05.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff666d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff6680acd30 0x7ff6680ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66659c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 0x7ff6680aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.222 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff665d9b700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff66813b380 0x7ff6680aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66659c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 0x7ff6680aa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:60606/0 (socket says 172.21.15.116:60606) 2020-11-20T12:22:05.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66659c700 1 -- 172.21.15.116:0/274726536 learned_addr learned my addr 172.21.15.116:0/274726536 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:05.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66659c700 1 -- 172.21.15.116:0/274726536 >> v2:172.21.15.173:3300/0 conn(0x7ff66813b380 msgr2=0x7ff6680aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:05.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66659c700 1 --2- 172.21.15.116:0/274726536 >> v2:172.21.15.173:3300/0 conn(0x7ff66813b380 0x7ff6680aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.223 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66659c700 1 -- 172.21.15.116:0/274726536 >> v2:172.21.15.116:3300/0 conn(0x7ff6680acd30 msgr2=0x7ff6680ad1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:05.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66659c700 1 --2- 172.21.15.116:0/274726536 >> v2:172.21.15.116:3300/0 conn(0x7ff6680acd30 0x7ff6680ad1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66659c700 1 -- 172.21.15.116:0/274726536 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6680ae1e0 con 0x7ff66813afa0 2020-11-20T12:22:05.224 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.216+0000 7ff66659c700 1 --2- 172.21.15.116:0/274726536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 0x7ff6680aa5b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7ff668081a10 tx=0x7ff658006eb0).ready entity=mon.2 client_cookie=103a39d17c6b3d5b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66559a700 1 -- 172.21.15.116:0/274726536 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658009720 con 0x7ff66813afa0 2020-11-20T12:22:05.225 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66559a700 1 -- 172.21.15.116:0/274726536 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff658009880 con 0x7ff66813afa0 2020-11-20T12:22:05.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66559a700 1 -- 172.21.15.116:0/274726536 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658009c00 con 0x7ff66813afa0 2020-11-20T12:22:05.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66759e700 1 -- 172.21.15.116:0/274726536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 msgr2=0x7ff6680aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66759e700 1 --2- 172.21.15.116:0/274726536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 0x7ff6680aa5b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7ff668081a10 tx=0x7ff658006eb0).stop 2020-11-20T12:22:05.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66759e700 1 -- 172.21.15.116:0/274726536 shutdown_connections 2020-11-20T12:22:05.226 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66759e700 1 --2- 172.21.15.116:0/274726536 >> v2:172.21.15.116:3300/0 conn(0x7ff6680acd30 0x7ff6680ad1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66759e700 1 --2- 172.21.15.116:0/274726536 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 0x7ff6680aa5b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66759e700 1 --2- 172.21.15.116:0/274726536 >> v2:172.21.15.173:3300/0 conn(0x7ff66813b380 0x7ff6680aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66759e700 1 -- 172.21.15.116:0/274726536 >> 172.21.15.116:0/274726536 conn(0x7ff6680678a0 msgr2=0x7ff668067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:05.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66759e700 1 -- 172.21.15.116:0/274726536 shutdown_connections 2020-11-20T12:22:05.227 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.220+0000 7ff66759e700 1 -- 172.21.15.116:0/274726536 wait complete. 2020-11-20T12:22:05.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff66759e700 1 Processor -- start 2020-11-20T12:22:05.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff66759e700 1 -- start start 2020-11-20T12:22:05.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff66759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6680acd30 0x7ff668164c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff66759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 0x7ff66809fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff66759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff66813b380 0x7ff668153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff66759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff668161b60 con 0x7ff66813b380 2020-11-20T12:22:05.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff66759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66809f4a0 con 0x7ff6680acd30 2020-11-20T12:22:05.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff66759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff66809f740 con 0x7ff66813afa0 2020-11-20T12:22:05.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff666d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff66813b380 0x7ff668153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff666d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff66813b380 0x7ff668153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51666/0 (socket says 172.21.15.116:51666) 2020-11-20T12:22:05.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff666d9d700 1 -- 172.21.15.116:0/2611712636 learned_addr learned my addr 172.21.15.116:0/2611712636 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:05.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff665d9b700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 0x7ff66809fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff66659c700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6680acd30 0x7ff668164c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff666d9d700 1 -- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 msgr2=0x7ff66809fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff666d9d700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 0x7ff66809fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff666d9d700 1 -- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6680acd30 msgr2=0x7ff668164c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff666d9d700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6680acd30 0x7ff668164c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.224+0000 7ff666d9d700 1 -- 172.21.15.116:0/2611712636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66809f9e0 con 0x7ff66813b380 2020-11-20T12:22:05.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.228+0000 7ff666d9d700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff66813b380 0x7ff668153d60 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7ff66000ef50 tx=0x7ff6600091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.228+0000 7ff6567fc700 1 -- 172.21.15.116:0/2611712636 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66000cb40 con 0x7ff66813b380 2020-11-20T12:22:05.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.228+0000 7ff6567fc700 1 -- 172.21.15.116:0/2611712636 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66000cca0 con 0x7ff66813b380 2020-11-20T12:22:05.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.228+0000 7ff66759e700 1 -- 172.21.15.116:0/2611712636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff668165160 con 0x7ff66813b380 2020-11-20T12:22:05.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.228+0000 7ff6567fc700 1 -- 172.21.15.116:0/2611712636 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6600123f0 con 0x7ff66813b380 2020-11-20T12:22:05.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.228+0000 7ff66759e700 1 -- 172.21.15.116:0/2611712636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff668154250 con 0x7ff66813b380 2020-11-20T12:22:05.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.228+0000 7ff66759e700 1 -- 172.21.15.116:0/2611712636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66809fb70 con 0x7ff66813b380 2020-11-20T12:22:05.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.228+0000 7ff6567fc700 1 -- 172.21.15.116:0/2611712636 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff660022070 con 0x7ff66813b380 2020-11-20T12:22:05.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.232+0000 7ff6567fc700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff63c0557c0 0x7ff63c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:05.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.232+0000 7ff6567fc700 1 -- 172.21.15.116:0/2611712636 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff66006ef40 con 0x7ff66813b380 2020-11-20T12:22:05.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.232+0000 7ff66659c700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff63c0557c0 0x7ff63c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:05.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.232+0000 7ff66659c700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff63c0557c0 0x7ff63c057c80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7ff658006ef0 tx=0x7ff658015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:05.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.236+0000 7ff6567fc700 1 -- 172.21.15.116:0/2611712636 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff660044de0 con 0x7ff66813b380 2020-11-20T12:22:05.451 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.448+0000 7ff66759e700 1 -- 172.21.15.116:0/2611712636 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff66805cf30 con 0x7ff66813b380 2020-11-20T12:22:05.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.448+0000 7ff6567fc700 1 -- 172.21.15.116:0/2611712636 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2854 (secure 0 0 0) 0x7ff660016030 con 0x7ff66813b380 2020-11-20T12:22:05.452 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:05.453 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/1612795957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1612795957},{"type":"v1","addr":"172.21.15.116:6835","nonce":1612795957}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/3255680781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3255680781},{"type":"v1","addr":"172.21.15.173:6835","nonce":3255680781}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/4058046293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":4058046293},{"type":"v1","addr":"172.21.15.173:6837","nonce":4058046293}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:56.570302+0000","modified":"2020-11-20T12:21:58.589498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8148":{"gid":8148,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2775705840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2775705840},{"type":"v1","addr":"172.21.15.116:6833","nonce":2775705840}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-20T12:22:05.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 -- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff63c0557c0 msgr2=0x7ff63c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff63c0557c0 0x7ff63c057c80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7ff658006ef0 tx=0x7ff658015000).stop 2020-11-20T12:22:05.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 -- 172.21.15.116:0/2611712636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff66813b380 msgr2=0x7ff668153d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:05.456 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff66813b380 0x7ff668153d60 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7ff66000ef50 tx=0x7ff6600091d0).stop 2020-11-20T12:22:05.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 -- 172.21.15.116:0/2611712636 shutdown_connections 2020-11-20T12:22:05.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff63c0557c0 0x7ff63c057c80 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff66813b380 0x7ff668153d60 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.457 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6680acd30 0x7ff668164c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 --2- 172.21.15.116:0/2611712636 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff66813afa0 0x7ff66809fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:05.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 -- 172.21.15.116:0/2611712636 >> 172.21.15.116:0/2611712636 conn(0x7ff6680678a0 msgr2=0x7ff668067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:05.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 -- 172.21.15.116:0/2611712636 shutdown_connections 2020-11-20T12:22:05.458 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:05.452+0000 7ff66759e700 1 -- 172.21.15.116:0/2611712636 wait complete. 2020-11-20T12:22:05.459 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 68 2020-11-20T12:22:05.884 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:21:56.570302+0000', 'modified': '2020-11-20T12:21:58.589498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8148}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8148': {'gid': 8148, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.116:6833/2775705840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 2775705840}, {'type': 'v1', 'addr': '172.21.15.116:6833', 'nonce': 2775705840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:22:05.885 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:22:05.885 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:22:06.080 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":8148,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":9.4866479899999998,"mdsmap_epoch":68,"osdmap_epoch":118,"osdmap_epoch_barrier":118,"uptime":22.506683967000001} 2020-11-20T12:22:06.081 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 8148, "mdsmap_epoch": 68, "osdmap_epoch": 118, "osdmap_epoch_barrier": 118, "rank_uptime": 9.48664799, "state": "up:active", "uptime": 22.506683967, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:22:06.081 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:22:06.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:06.082 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-20T12:22:06.110 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:22:06.111 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:06.234 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:06.234 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:22:06.264 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:06.264 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:22:06.270 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:22:06.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:06.271 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-20T12:22:06.306 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:22:06.307 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:22:06.307 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-20T12:22:06.308 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-20T12:22:06.308 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:22:06.308 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:22:06.309 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:22:06.309 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:22:06.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:06.310 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:22:06.334 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:22:06.335 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:22:06.336 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:22:06.336 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:06.337 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:22:06.338 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:06.338 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:06.339 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:06.339 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:06.340 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:06.340 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:06.341 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:06.342 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:22:06.342 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:06.343 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:22:06.343 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:06.344 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:22:06.344 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:06.344 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:22:06.345 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:06.346 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:22:06.347 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:06.347 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:22:06.347 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:06.348 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:22:06.348 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:06.348 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:22:06.348 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:06.348 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:06.349 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:22:06.349 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:22:06.349 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:06.349 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:06.350 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:22:06.350 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:22:06.350 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:06.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:06.351 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:22:06.416 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:22:06.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:06.421 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:22:06.441 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:22:06.442 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:22:06.442 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:22:06.442 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:22:06.442 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:22:06.443 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:22:06.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:06.444 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:22:06.445 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:22:06.445 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:22:06.446 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:22:06.446 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:22:06.446 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:22:06.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:06.589 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:22:06.614 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:22:06.614 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:22:06.616 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:22:06.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:06.618 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2020-11-20T12:22:06.687 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:06.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.863+0000 7f868dc08700 1 Processor -- start 2020-11-20T12:22:06.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.863+0000 7f868dc08700 1 -- start start 2020-11-20T12:22:06.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.863+0000 7f868dc08700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8688150680 0x7f8688066450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:06.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.863+0000 7f868dc08700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688156f20 0x7f86880bf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:06.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.863+0000 7f868dc08700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86881573d0 con 0x7f86880bf560 2020-11-20T12:22:06.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.863+0000 7f868dc08700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86880bf9d0 con 0x7f8688150680 2020-11-20T12:22:06.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.863+0000 7f868dc08700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8688150a60 con 0x7f8688156f20 2020-11-20T12:22:06.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f8686ffd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688156f20 0x7f86880bf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:06.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f8686ffd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688156f20 0x7f86880bf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:60628/0 (socket says 172.21.15.116:60628) 2020-11-20T12:22:06.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f8686ffd700 1 -- 172.21.15.116:0/4281797802 learned_addr learned my addr 172.21.15.116:0/4281797802 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:06.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f86877fe700 1 --2- 172.21.15.116:0/4281797802 >> v2:172.21.15.173:3300/0 conn(0x7f8688150680 0x7f8688066450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f8686ffd700 1 -- 172.21.15.116:0/4281797802 >> v2:172.21.15.173:3300/0 conn(0x7f8688150680 msgr2=0x7f8688066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:06.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f8686ffd700 1 --2- 172.21.15.116:0/4281797802 >> v2:172.21.15.173:3300/0 conn(0x7f8688150680 0x7f8688066450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f8686ffd700 1 -- 172.21.15.116:0/4281797802 >> v1:172.21.15.173:6789/0 conn(0x7f86880bf560 legacy=0x7f8688159aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f8686ffd700 1 -- 172.21.15.116:0/4281797802 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8688066af0 con 0x7f8688156f20 2020-11-20T12:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f8686ffd700 1 --2- 172.21.15.116:0/4281797802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688156f20 0x7f86880bf070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f867c006fd0 tx=0x7f867c006eb0).ready entity=mon.2 client_cookie=c694ae12fc1166ad server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:06.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f86867fc700 1 -- 172.21.15.116:0/4281797802 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c009810 con 0x7f8688156f20 2020-11-20T12:22:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f86867fc700 1 -- 172.21.15.116:0/4281797802 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f867c009970 con 0x7f8688156f20 2020-11-20T12:22:06.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.867+0000 7f86867fc700 1 -- 172.21.15.116:0/4281797802 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f867c009cf0 con 0x7f8688156f20 2020-11-20T12:22:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.871+0000 7f868dc08700 1 -- 172.21.15.116:0/4281797802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688156f20 msgr2=0x7f86880bf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.871+0000 7f868dc08700 1 --2- 172.21.15.116:0/4281797802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688156f20 0x7f86880bf070 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f867c006fd0 tx=0x7f867c006eb0).stop 2020-11-20T12:22:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.871+0000 7f868dc08700 1 -- 172.21.15.116:0/4281797802 shutdown_connections 2020-11-20T12:22:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.871+0000 7f868dc08700 1 --2- 172.21.15.116:0/4281797802 >> v2:172.21.15.173:3300/0 conn(0x7f8688150680 0x7f8688066450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:06.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.871+0000 7f868dc08700 1 --2- 172.21.15.116:0/4281797802 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688156f20 0x7f86880bf070 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.871+0000 7f868dc08700 1 -- 172.21.15.116:0/4281797802 >> 172.21.15.116:0/4281797802 conn(0x7f86880c4f80 msgr2=0x7f86880c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.871+0000 7f868dc08700 1 -- 172.21.15.116:0/4281797802 shutdown_connections 2020-11-20T12:22:06.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.871+0000 7f868dc08700 1 -- 172.21.15.116:0/4281797802 wait complete. 2020-11-20T12:22:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.871+0000 7f868dc08700 1 Processor -- start 2020-11-20T12:22:06.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.871+0000 7f868dc08700 1 -- start start 2020-11-20T12:22:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f868dc08700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86880bf560 0x7f8688169a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f868dc08700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688150680 0x7f86880a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:06.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f868dc08700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8688156f20 0x7f868814c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f868dc08700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86880a6fd0 con 0x7f8688156f20 2020-11-20T12:22:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f868dc08700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8688169f10 con 0x7f86880bf560 2020-11-20T12:22:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f868dc08700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8688165b90 con 0x7f8688150680 2020-11-20T12:22:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f8687fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8688156f20 0x7f868814c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:06.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f8687fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8688156f20 0x7f868814c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51682/0 (socket says 172.21.15.116:51682) 2020-11-20T12:22:06.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f8687fff700 1 -- 172.21.15.116:0/3145973003 learned_addr learned my addr 172.21.15.116:0/3145973003 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:06.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f8686ffd700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688150680 0x7f86880a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:06.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f8687fff700 1 -- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688150680 msgr2=0x7f86880a63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:06.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f8687fff700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688150680 0x7f86880a63d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:06.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f8687fff700 1 -- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86880bf560 msgr2=0x7f8688169a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:06.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f86877fe700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86880bf560 0x7f8688169a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:06.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f8687fff700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86880bf560 0x7f8688169a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:06.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f8687fff700 1 -- 172.21.15.116:0/3145973003 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8688165e30 con 0x7f8688156f20 2020-11-20T12:22:06.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f8686ffd700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688150680 0x7f86880a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:06.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f86877fe700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86880bf560 0x7f8688169a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:06.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f8687fff700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8688156f20 0x7f868814c140 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f868000ca90 tx=0x7f8680001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:06.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f86677fe700 1 -- 172.21.15.116:0/3145973003 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868000a730 con 0x7f8688156f20 2020-11-20T12:22:06.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f86677fe700 1 -- 172.21.15.116:0/3145973003 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f868000a890 con 0x7f8688156f20 2020-11-20T12:22:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f868dc08700 1 -- 172.21.15.116:0/3145973003 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86881660d0 con 0x7f8688156f20 2020-11-20T12:22:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f86677fe700 1 -- 172.21.15.116:0/3145973003 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f868000ac30 con 0x7f8688156f20 2020-11-20T12:22:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.875+0000 7f868dc08700 1 -- 172.21.15.116:0/3145973003 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8688166610 con 0x7f8688156f20 2020-11-20T12:22:06.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.879+0000 7f868dc08700 1 -- 172.21.15.116:0/3145973003 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8688166810 con 0x7f8688156f20 2020-11-20T12:22:06.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.879+0000 7f86677fe700 1 -- 172.21.15.116:0/3145973003 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8680010070 con 0x7f8688156f20 2020-11-20T12:22:06.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.879+0000 7f86677fe700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86600557c0 0x7f8660057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:06.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.879+0000 7f86677fe700 1 -- 172.21.15.116:0/3145973003 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f868006e4a0 con 0x7f8688156f20 2020-11-20T12:22:06.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.879+0000 7f86877fe700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86600557c0 0x7f8660057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:06.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.879+0000 7f86877fe700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86600557c0 0x7f8660057c80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f8678006bd0 tx=0x7f8678008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:06.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:06.883+0000 7f86677fe700 1 -- 172.21.15.116:0/3145973003 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8688166810 con 0x7f8688156f20 2020-11-20T12:22:07.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.043+0000 7f868dc08700 1 -- 172.21.15.116:0/3145973003 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8688166810 con 0x7f8688156f20 2020-11-20T12:22:07.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.043+0000 7f86677fe700 1 -- 172.21.15.116:0/3145973003 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f8688166810 con 0x7f8688156f20 2020-11-20T12:22:07.048 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:07.049 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":118,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:56.569994+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:53.361384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:53.918177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:07.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 -- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86600557c0 msgr2=0x7f8660057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86600557c0 0x7f8660057c80 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f8678006bd0 tx=0x7f8678008040).stop 2020-11-20T12:22:07.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 -- 172.21.15.116:0/3145973003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8688156f20 msgr2=0x7f868814c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8688156f20 0x7f868814c140 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f868000ca90 tx=0x7f8680001360).stop 2020-11-20T12:22:07.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 -- 172.21.15.116:0/3145973003 shutdown_connections 2020-11-20T12:22:07.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86600557c0 0x7f8660057c80 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8688156f20 0x7f868814c140 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f86880bf560 0x7f8688169a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 --2- 172.21.15.116:0/3145973003 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8688150680 0x7f86880a63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 -- 172.21.15.116:0/3145973003 >> 172.21.15.116:0/3145973003 conn(0x7f86880c4f80 msgr2=0x7f868814d360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:07.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 -- 172.21.15.116:0/3145973003 shutdown_connections 2020-11-20T12:22:07.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.047+0000 7f868dc08700 1 -- 172.21.15.116:0/3145973003 wait complete. 2020-11-20T12:22:07.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:21:53.361384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:21:53.918177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:22:07.079 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:22:07.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.251+0000 7fac4de43700 1 Processor -- start 2020-11-20T12:22:07.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.251+0000 7fac4de43700 1 -- start start 2020-11-20T12:22:07.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.251+0000 7fac4de43700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fac48149df0 0x7fac48153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.251+0000 7fac4de43700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fac480a9750 0x7fac480c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.251+0000 7fac4de43700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac480a9c20 con 0x7fac4814a1d0 2020-11-20T12:22:07.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.251+0000 7fac4de43700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac480a9ec0 con 0x7fac48149df0 2020-11-20T12:22:07.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.251+0000 7fac4de43700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac48153670 con 0x7fac480a9750 2020-11-20T12:22:07.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.251+0000 7fac477fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fac48149df0 0x7fac48153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac477fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fac48149df0 0x7fac48153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51688/0 (socket says 172.21.15.116:51688) 2020-11-20T12:22:07.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac477fe700 1 -- 172.21.15.116:0/1550042879 learned_addr learned my addr 172.21.15.116:0/1550042879 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:07.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac46ffd700 1 --2- 172.21.15.116:0/1550042879 >> v2:172.21.15.173:3300/0 conn(0x7fac480a9750 0x7fac480c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac477fe700 1 -- 172.21.15.116:0/1550042879 >> v2:172.21.15.173:3300/0 conn(0x7fac480a9750 msgr2=0x7fac480c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:07.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac477fe700 1 --2- 172.21.15.116:0/1550042879 >> v2:172.21.15.173:3300/0 conn(0x7fac480a9750 0x7fac480c0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac477fe700 1 -- 172.21.15.116:0/1550042879 >> v1:172.21.15.173:6789/0 conn(0x7fac4814a1d0 legacy=0x7fac480be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac477fe700 1 -- 172.21.15.116:0/1550042879 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac48153910 con 0x7fac48149df0 2020-11-20T12:22:07.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac46ffd700 1 --2- 172.21.15.116:0/1550042879 >> v2:172.21.15.173:3300/0 conn(0x7fac480a9750 0x7fac480c0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:07.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac477fe700 1 --2- 172.21.15.116:0/1550042879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac48149df0 0x7fac48153db0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7fac480c4dc0 tx=0x7fac38006eb0).ready entity=mon.0 client_cookie=1520f700f9ed409d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:07.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac467fc700 1 -- 172.21.15.116:0/1550042879 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac380098d0 con 0x7fac48149df0 2020-11-20T12:22:07.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac467fc700 1 -- 172.21.15.116:0/1550042879 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac38009a30 con 0x7fac48149df0 2020-11-20T12:22:07.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac467fc700 1 -- 172.21.15.116:0/1550042879 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38009db0 con 0x7fac48149df0 2020-11-20T12:22:07.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac4de43700 1 -- 172.21.15.116:0/1550042879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac48149df0 msgr2=0x7fac48153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac4de43700 1 --2- 172.21.15.116:0/1550042879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac48149df0 0x7fac48153db0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7fac480c4dc0 tx=0x7fac38006eb0).stop 2020-11-20T12:22:07.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac4de43700 1 -- 172.21.15.116:0/1550042879 shutdown_connections 2020-11-20T12:22:07.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac4de43700 1 --2- 172.21.15.116:0/1550042879 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac48149df0 0x7fac48153db0 secure :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 rx=0x7fac480c4dc0 tx=0x7fac38006eb0).stop 2020-11-20T12:22:07.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac4de43700 1 --2- 172.21.15.116:0/1550042879 >> v2:172.21.15.173:3300/0 conn(0x7fac480a9750 0x7fac480c0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac4de43700 1 -- 172.21.15.116:0/1550042879 >> 172.21.15.116:0/1550042879 conn(0x7fac4805ba80 msgr2=0x7fac480c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:07.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac4de43700 1 -- 172.21.15.116:0/1550042879 shutdown_connections 2020-11-20T12:22:07.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.255+0000 7fac4de43700 1 -- 172.21.15.116:0/1550042879 wait complete. 2020-11-20T12:22:07.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.259+0000 7fac4de43700 1 Processor -- start 2020-11-20T12:22:07.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.259+0000 7fac4de43700 1 -- start start 2020-11-20T12:22:07.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.259+0000 7fac4de43700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480a9750 0x7fac4814ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac4de43700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac4814a1d0 0x7fac48152000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac4de43700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac481524f0 0x7fac48146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac4de43700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac48152950 con 0x7fac480a9750 2020-11-20T12:22:07.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac4de43700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac48147c60 con 0x7fac481524f0 2020-11-20T12:22:07.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac4de43700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac48139800 con 0x7fac4814a1d0 2020-11-20T12:22:07.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac477fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480a9750 0x7fac4814ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac477fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480a9750 0x7fac4814ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51698/0 (socket says 172.21.15.116:51698) 2020-11-20T12:22:07.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac477fe700 1 -- 172.21.15.116:0/1055920414 learned_addr learned my addr 172.21.15.116:0/1055920414 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:07.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac46ffd700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac4814a1d0 0x7fac48152000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac47fff700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac481524f0 0x7fac48146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac477fe700 1 -- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac4814a1d0 msgr2=0x7fac48152000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac477fe700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac4814a1d0 0x7fac48152000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac477fe700 1 -- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac481524f0 msgr2=0x7fac48146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac477fe700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac481524f0 0x7fac48146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac477fe700 1 -- 172.21.15.116:0/1055920414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac48162350 con 0x7fac480a9750 2020-11-20T12:22:07.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac47fff700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac481524f0 0x7fac48146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:07.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac477fe700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480a9750 0x7fac4814ae80 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7fac480611d0 tx=0x7fac38015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:07.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac277fe700 1 -- 172.21.15.116:0/1055920414 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac38011af0 con 0x7fac480a9750 2020-11-20T12:22:07.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac277fe700 1 -- 172.21.15.116:0/1055920414 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac38011c50 con 0x7fac480a9750 2020-11-20T12:22:07.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac277fe700 1 -- 172.21.15.116:0/1055920414 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac3801e460 con 0x7fac480a9750 2020-11-20T12:22:07.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac4de43700 1 -- 172.21.15.116:0/1055920414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac481616d0 con 0x7fac480a9750 2020-11-20T12:22:07.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.263+0000 7fac4de43700 1 -- 172.21.15.116:0/1055920414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac48161eb0 con 0x7fac480a9750 2020-11-20T12:22:07.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.267+0000 7fac4de43700 1 -- 172.21.15.116:0/1055920414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac48162040 con 0x7fac480a9750 2020-11-20T12:22:07.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.267+0000 7fac277fe700 1 -- 172.21.15.116:0/1055920414 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fac38025030 con 0x7fac480a9750 2020-11-20T12:22:07.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.267+0000 7fac277fe700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac200557c0 0x7fac20057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.267+0000 7fac277fe700 1 -- 172.21.15.116:0/1055920414 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fac3806fcb0 con 0x7fac480a9750 2020-11-20T12:22:07.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.267+0000 7fac46ffd700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac200557c0 0x7fac20057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.271+0000 7fac46ffd700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac200557c0 0x7fac20057c80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fac480c1480 tx=0x7fac3c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:07.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.271+0000 7fac277fe700 1 -- 172.21.15.116:0/1055920414 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fac38045b50 con 0x7fac480a9750 2020-11-20T12:22:07.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.435+0000 7fac4de43700 1 -- 172.21.15.116:0/1055920414 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fac4805d2a0 con 0x7fac480a9750 2020-11-20T12:22:07.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.435+0000 7fac277fe700 1 -- 172.21.15.116:0/1055920414 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v118) v1 ==== 110+0+10 (secure 0 0 0) 0x7fac38023070 con 0x7fac480a9750 2020-11-20T12:22:07.441 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:22:07.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 -- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac200557c0 msgr2=0x7fac20057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac200557c0 0x7fac20057c80 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7fac480c1480 tx=0x7fac3c008040).stop 2020-11-20T12:22:07.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 -- 172.21.15.116:0/1055920414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480a9750 msgr2=0x7fac4814ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480a9750 0x7fac4814ae80 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7fac480611d0 tx=0x7fac38015000).stop 2020-11-20T12:22:07.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 -- 172.21.15.116:0/1055920414 shutdown_connections 2020-11-20T12:22:07.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fac200557c0 0x7fac20057c80 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fac481524f0 0x7fac48146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fac480a9750 0x7fac4814ae80 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 --2- 172.21.15.116:0/1055920414 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fac4814a1d0 0x7fac48152000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 -- 172.21.15.116:0/1055920414 >> 172.21.15.116:0/1055920414 conn(0x7fac4805ba80 msgr2=0x7fac480c14f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:07.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 -- 172.21.15.116:0/1055920414 shutdown_connections 2020-11-20T12:22:07.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.439+0000 7fac4de43700 1 -- 172.21.15.116:0/1055920414 wait complete. 2020-11-20T12:22:07.467 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:22:07.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.651+0000 7f45b13a7700 1 Processor -- start 2020-11-20T12:22:07.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.651+0000 7f45b13a7700 1 -- start start 2020-11-20T12:22:07.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45b13a7700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f45ac0616c0 0x7f45ac060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.659 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45b13a7700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac0c1530 con 0x7f45ac0c1150 2020-11-20T12:22:07.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45b13a7700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45ac0607a0 con 0x7f45ac0a3e40 2020-11-20T12:22:07.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45b13a7700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45ac060a40 con 0x7f45ac0616c0 2020-11-20T12:22:07.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45abfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f45ac0c1150 0x7f45ac0a3730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:32904/0 (socket says 172.21.15.116:32904) 2020-11-20T12:22:07.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45abfff700 1 -- 172.21.15.116:0/3025962342 learned_addr learned my addr 172.21.15.116:0/3025962342 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:07.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 354188137 0 0) 0x7f45ac0c1530 con 0x7f45ac0c1150 2020-11-20T12:22:07.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4598003240 con 0x7f45ac0c1150 2020-11-20T12:22:07.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45b0ba6700 1 --2- 172.21.15.116:0/3025962342 >> v2:172.21.15.173:3300/0 conn(0x7f45ac0616c0 0x7f45ac060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2211222147 0 0) 0x7f4598003240 con 0x7f45ac0c1150 2020-11-20T12:22:07.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 >> v2:172.21.15.173:3300/0 conn(0x7f45ac0616c0 msgr2=0x7f45ac060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:07.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 --2- 172.21.15.116:0/3025962342 >> v2:172.21.15.173:3300/0 conn(0x7f45ac0616c0 0x7f45ac060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 >> v1:172.21.15.173:6789/0 conn(0x7f45ac0a3e40 legacy=0x7f45ac063380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac060ce0 con 0x7f45ac0c1150 2020-11-20T12:22:07.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f459c0014e0 con 0x7f45ac0c1150 2020-11-20T12:22:07.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 >> v1:172.21.15.116:6789/0 conn(0x7f45ac0c1150 legacy=0x7f45ac0a3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4598003730 0x7f4598003b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45980044e0 0x7f4598004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4598004e50 0x7f459800b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac060a40 con 0x7f45980044e0 2020-11-20T12:22:07.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4598005330 con 0x7f4598004e50 2020-11-20T12:22:07.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f459800bc20 con 0x7f4598003730 2020-11-20T12:22:07.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45ab7fe700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45980044e0 0x7f4598004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45b0ba6700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4598004e50 0x7f459800b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.655+0000 7f45abfff700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4598003730 0x7f4598003b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45ab7fe700 1 -- 172.21.15.116:0/3025962342 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4598003730 msgr2=0x7f4598003b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:07.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45ab7fe700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4598003730 0x7f4598003b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45ab7fe700 1 -- 172.21.15.116:0/3025962342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4598004e50 msgr2=0x7f459800b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:07.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45ab7fe700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4598004e50 0x7f459800b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45ab7fe700 1 -- 172.21.15.116:0/3025962342 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4598003240 con 0x7f45980044e0 2020-11-20T12:22:07.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45b0ba6700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4598004e50 0x7f459800b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:07.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45abfff700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4598003730 0x7f4598003b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:07.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45b0ba6700 1 -- 172.21.15.116:0/3025962342 reap_dead start 2020-11-20T12:22:07.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45ab7fe700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45980044e0 0x7f4598004960 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f45a0001a00 tx=0x7f45a0007b70).ready entity=mon.0 client_cookie=1e258222686fa60c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:07.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a000c830 con 0x7f45980044e0 2020-11-20T12:22:07.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a000c990 con 0x7f45980044e0 2020-11-20T12:22:07.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45aaffd700 1 -- 172.21.15.116:0/3025962342 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a000cd50 con 0x7f45980044e0 2020-11-20T12:22:07.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45b13a7700 1 -- 172.21.15.116:0/3025962342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45980044e0 msgr2=0x7f4598004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45b13a7700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45980044e0 0x7f4598004960 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f45a0001a00 tx=0x7f45a0007b70).stop 2020-11-20T12:22:07.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45b13a7700 1 -- 172.21.15.116:0/3025962342 shutdown_connections 2020-11-20T12:22:07.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45b13a7700 1 --2- 172.21.15.116:0/3025962342 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45980044e0 0x7f4598004960 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45b13a7700 1 -- 172.21.15.116:0/3025962342 >> 172.21.15.116:0/3025962342 conn(0x7f45ac0c1710 msgr2=0x7f45ac0c1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:07.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45b13a7700 1 -- 172.21.15.116:0/3025962342 shutdown_connections 2020-11-20T12:22:07.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.659+0000 7f45b13a7700 1 -- 172.21.15.116:0/3025962342 wait complete. 2020-11-20T12:22:07.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.663+0000 7f45b13a7700 1 Processor -- start 2020-11-20T12:22:07.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.663+0000 7f45b13a7700 1 -- start start 2020-11-20T12:22:07.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.663+0000 7f45b13a7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45980044e0 0x7f45ac14f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.663+0000 7f45b13a7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45ac151f10 0x7f45ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.663+0000 7f45b13a7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45ac133210 0x7f45ac1341b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.663+0000 7f45b13a7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac152490 con 0x7f45ac151f10 2020-11-20T12:22:07.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.663+0000 7f45b13a7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac152730 con 0x7f45980044e0 2020-11-20T12:22:07.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.663+0000 7f45b13a7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ac1529d0 con 0x7f45ac133210 2020-11-20T12:22:07.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45ab7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45ac151f10 0x7f45ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45abfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45980044e0 0x7f45ac14f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45ab7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45ac151f10 0x7f45ac0a5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51716/0 (socket says 172.21.15.116:51716) 2020-11-20T12:22:07.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45ab7fe700 1 -- 172.21.15.116:0/3717781541 learned_addr learned my addr 172.21.15.116:0/3717781541 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:07.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45b0ba6700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45ac133210 0x7f45ac1341b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45ab7fe700 1 -- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45ac133210 msgr2=0x7f45ac1341b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45ab7fe700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45ac133210 0x7f45ac1341b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45ab7fe700 1 -- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45980044e0 msgr2=0x7f45ac14f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45ab7fe700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45980044e0 0x7f45ac14f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45ab7fe700 1 -- 172.21.15.116:0/3717781541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ac152c70 con 0x7f45ac151f10 2020-11-20T12:22:07.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45ab7fe700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45ac151f10 0x7f45ac0a5700 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f4598005300 tx=0x7f45a0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:07.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f458bfff700 1 -- 172.21.15.116:0/3717781541 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a0007560 con 0x7f45ac151f10 2020-11-20T12:22:07.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f458bfff700 1 -- 172.21.15.116:0/3717781541 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45a00125c0 con 0x7f45ac151f10 2020-11-20T12:22:07.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45b13a7700 1 -- 172.21.15.116:0/3717781541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45ac163fb0 con 0x7f45ac151f10 2020-11-20T12:22:07.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f458bfff700 1 -- 172.21.15.116:0/3717781541 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45a0012940 con 0x7f45ac151f10 2020-11-20T12:22:07.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45b13a7700 1 -- 172.21.15.116:0/3717781541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45ac0a5bf0 con 0x7f45ac151f10 2020-11-20T12:22:07.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.667+0000 7f45b13a7700 1 -- 172.21.15.116:0/3717781541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45ac0c2da0 con 0x7f45ac151f10 2020-11-20T12:22:07.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.671+0000 7f458bfff700 1 -- 172.21.15.116:0/3717781541 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f45a0010030 con 0x7f45ac151f10 2020-11-20T12:22:07.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.671+0000 7f458bfff700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45840557c0 0x7f4584057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:07.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.671+0000 7f458bfff700 1 -- 172.21.15.116:0/3717781541 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f45a0070060 con 0x7f45ac151f10 2020-11-20T12:22:07.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.671+0000 7f45abfff700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45840557c0 0x7f4584057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:07.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.671+0000 7f45abfff700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45840557c0 0x7f4584057c80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f459c000fb0 tx=0x7f459c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:07.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.675+0000 7f458bfff700 1 -- 172.21.15.116:0/3717781541 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f45ac0c2da0 con 0x7f45ac151f10 2020-11-20T12:22:07.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.835+0000 7f45b13a7700 1 -- 172.21.15.116:0/3717781541 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f45ac0c2da0 con 0x7f45ac151f10 2020-11-20T12:22:07.841 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.835+0000 7f458bfff700 1 -- 172.21.15.116:0/3717781541 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v118) v1 ==== 92+0+10 (secure 0 0 0) 0x7f45ac0c2da0 con 0x7f45ac151f10 2020-11-20T12:22:07.841 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:07.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 -- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45840557c0 msgr2=0x7f4584057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45840557c0 0x7f4584057c80 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f459c000fb0 tx=0x7f459c009040).stop 2020-11-20T12:22:07.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 -- 172.21.15.116:0/3717781541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45ac151f10 msgr2=0x7f45ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:07.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45ac151f10 0x7f45ac0a5700 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f4598005300 tx=0x7f45a0016000).stop 2020-11-20T12:22:07.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 -- 172.21.15.116:0/3717781541 shutdown_connections 2020-11-20T12:22:07.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f45840557c0 0x7f4584057c80 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f45ac133210 0x7f45ac1341b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f45980044e0 0x7f45ac14f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 --2- 172.21.15.116:0/3717781541 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f45ac151f10 0x7f45ac0a5700 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:07.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 -- 172.21.15.116:0/3717781541 >> 172.21.15.116:0/3717781541 conn(0x7f45ac0c1710 msgr2=0x7f45ac0c2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:07.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 -- 172.21.15.116:0/3717781541 shutdown_connections 2020-11-20T12:22:07.846 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:07.839+0000 7f45b13a7700 1 -- 172.21.15.116:0/3717781541 wait complete. 2020-11-20T12:22:07.868 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:22:08.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.043+0000 7f9c7bca7700 1 Processor -- start 2020-11-20T12:22:08.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.043+0000 7f9c7bca7700 1 -- start start 2020-11-20T12:22:08.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c7bca7700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9c74151940 0x7f9c740a6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c7bca7700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c740633a0 con 0x7f9c74151d20 2020-11-20T12:22:08.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c7bca7700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c740a7220 con 0x7f9c74062f10 2020-11-20T12:22:08.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c7bca7700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c74062530 con 0x7f9c74151940 2020-11-20T12:22:08.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c79242700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9c74151d20 0x7f9c74064a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:32926/0 (socket says 172.21.15.116:32926) 2020-11-20T12:22:08.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c79242700 1 -- 172.21.15.116:0/17513537 learned_addr learned my addr 172.21.15.116:0/17513537 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:08.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c79a43700 1 --2- 172.21.15.116:0/17513537 >> v2:172.21.15.173:3300/0 conn(0x7f9c74151940 0x7f9c740a6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c78a41700 1 -- 172.21.15.116:0/17513537 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3785298588 0 0) 0x7f9c740633a0 con 0x7f9c74151d20 2020-11-20T12:22:08.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c78a41700 1 -- 172.21.15.116:0/17513537 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c5c003240 con 0x7f9c74151d20 2020-11-20T12:22:08.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c79a43700 1 -- 172.21.15.116:0/17513537 >> v1:172.21.15.173:6789/0 conn(0x7f9c74062f10 legacy=0x7f9c740bfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c79a43700 1 -- 172.21.15.116:0/17513537 >> v1:172.21.15.116:6789/0 conn(0x7f9c74151d20 legacy=0x7f9c74064a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.047+0000 7f9c79a43700 1 -- 172.21.15.116:0/17513537 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c740627d0 con 0x7f9c74151940 2020-11-20T12:22:08.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.051+0000 7f9c79a43700 1 --2- 172.21.15.116:0/17513537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c74151940 0x7f9c740a6b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f9c7000b9a0 tx=0x7f9c70006df0).ready entity=mon.1 client_cookie=652ee7a7401b1b4e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:08.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.051+0000 7f9c78a41700 1 -- 172.21.15.116:0/17513537 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c700097b0 con 0x7f9c74151940 2020-11-20T12:22:08.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.051+0000 7f9c78a41700 1 -- 172.21.15.116:0/17513537 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c70009910 con 0x7f9c74151940 2020-11-20T12:22:08.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.051+0000 7f9c78a41700 1 -- 172.21.15.116:0/17513537 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c70009c90 con 0x7f9c74151940 2020-11-20T12:22:08.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.051+0000 7f9c7bca7700 1 -- 172.21.15.116:0/17513537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c74151940 msgr2=0x7f9c740a6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.051+0000 7f9c7bca7700 1 --2- 172.21.15.116:0/17513537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c74151940 0x7f9c740a6b80 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f9c7000b9a0 tx=0x7f9c70006df0).stop 2020-11-20T12:22:08.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.051+0000 7f9c7bca7700 1 -- 172.21.15.116:0/17513537 shutdown_connections 2020-11-20T12:22:08.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.051+0000 7f9c7bca7700 1 --2- 172.21.15.116:0/17513537 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c74151940 0x7f9c740a6b80 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.051+0000 7f9c7bca7700 1 -- 172.21.15.116:0/17513537 >> 172.21.15.116:0/17513537 conn(0x7f9c74060c00 msgr2=0x7f9c740c66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:08.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.051+0000 7f9c7bca7700 1 -- 172.21.15.116:0/17513537 shutdown_connections 2020-11-20T12:22:08.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.051+0000 7f9c7bca7700 1 -- 172.21.15.116:0/17513537 wait complete. 2020-11-20T12:22:08.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.055+0000 7f9c7bca7700 1 Processor -- start 2020-11-20T12:22:08.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.055+0000 7f9c7bca7700 1 -- start start 2020-11-20T12:22:08.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.055+0000 7f9c7bca7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c74062f10 0x7f9c740a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.055+0000 7f9c7bca7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c74151940 0x7f9c7414d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.055+0000 7f9c7bca7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c74151d20 0x7f9c740abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.055+0000 7f9c7bca7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740c04a0 con 0x7f9c74151940 2020-11-20T12:22:08.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.055+0000 7f9c7bca7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740c0740 con 0x7f9c74062f10 2020-11-20T12:22:08.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.055+0000 7f9c7bca7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c740c09e0 con 0x7f9c74151d20 2020-11-20T12:22:08.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.055+0000 7f9c79242700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c74151940 0x7f9c7414d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c79242700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c74151940 0x7f9c7414d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51732/0 (socket says 172.21.15.116:51732) 2020-11-20T12:22:08.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c79a43700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c74062f10 0x7f9c740a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c79242700 1 -- 172.21.15.116:0/1244850288 learned_addr learned my addr 172.21.15.116:0/1244850288 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:08.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c7a244700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c74151d20 0x7f9c740abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c79242700 1 -- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c74151d20 msgr2=0x7f9c740abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c79242700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c74151d20 0x7f9c740abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c79242700 1 -- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c74062f10 msgr2=0x7f9c740a96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c79242700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c74062f10 0x7f9c740a96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c79242700 1 -- 172.21.15.116:0/1244850288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c740c0c80 con 0x7f9c74151940 2020-11-20T12:22:08.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c79242700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c74151940 0x7f9c7414d820 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f9c6400c710 tx=0x7f9c64007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:08.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c61ffb700 1 -- 172.21.15.116:0/1244850288 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c640072e0 con 0x7f9c74151940 2020-11-20T12:22:08.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c61ffb700 1 -- 172.21.15.116:0/1244850288 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c64007440 con 0x7f9c74151940 2020-11-20T12:22:08.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c61ffb700 1 -- 172.21.15.116:0/1244850288 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c64005800 con 0x7f9c74151940 2020-11-20T12:22:08.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c7bca7700 1 -- 172.21.15.116:0/1244850288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c740c0f20 con 0x7f9c74151940 2020-11-20T12:22:08.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c7bca7700 1 -- 172.21.15.116:0/1244850288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7414deb0 con 0x7f9c74151940 2020-11-20T12:22:08.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.059+0000 7f9c7bca7700 1 -- 172.21.15.116:0/1244850288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7405d2a0 con 0x7f9c74151940 2020-11-20T12:22:08.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.063+0000 7f9c61ffb700 1 -- 172.21.15.116:0/1244850288 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9c64010070 con 0x7f9c74151940 2020-11-20T12:22:08.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.063+0000 7f9c61ffb700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c4c0557c0 0x7f9c4c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.063+0000 7f9c61ffb700 1 -- 172.21.15.116:0/1244850288 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9c6406dc70 con 0x7f9c74151940 2020-11-20T12:22:08.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.063+0000 7f9c79a43700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c4c0557c0 0x7f9c4c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.063+0000 7f9c79a43700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c4c0557c0 0x7f9c4c057c80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f9c7000be60 tx=0x7f9c70015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:08.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.067+0000 7f9c61ffb700 1 -- 172.21.15.116:0/1244850288 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9c64043b10 con 0x7f9c74151940 2020-11-20T12:22:08.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c7bca7700 1 -- 172.21.15.116:0/1244850288 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9c74150c10 con 0x7f9c74151940 2020-11-20T12:22:08.231 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c61ffb700 1 -- 172.21.15.116:0/1244850288 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+10 (secure 0 0 0) 0x7f9c64047130 con 0x7f9c74151940 2020-11-20T12:22:08.232 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:08.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c7bca7700 1 -- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c4c0557c0 msgr2=0x7f9c4c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c7bca7700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c4c0557c0 0x7f9c4c057c80 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7f9c7000be60 tx=0x7f9c70015000).stop 2020-11-20T12:22:08.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c7bca7700 1 -- 172.21.15.116:0/1244850288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c74151940 msgr2=0x7f9c7414d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c7bca7700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c74151940 0x7f9c7414d820 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7f9c6400c710 tx=0x7f9c64007ad0).stop 2020-11-20T12:22:08.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c7bca7700 1 -- 172.21.15.116:0/1244850288 shutdown_connections 2020-11-20T12:22:08.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c7bca7700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9c4c0557c0 0x7f9c4c057c80 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c7bca7700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9c74151d20 0x7f9c740abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c7bca7700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9c74062f10 0x7f9c740a96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c7bca7700 1 --2- 172.21.15.116:0/1244850288 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9c74151940 0x7f9c7414d820 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.227+0000 7f9c7bca7700 1 -- 172.21.15.116:0/1244850288 >> 172.21.15.116:0/1244850288 conn(0x7f9c74060c00 msgr2=0x7f9c740c5930 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:08.236 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.231+0000 7f9c7bca7700 1 -- 172.21.15.116:0/1244850288 shutdown_connections 2020-11-20T12:22:08.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.231+0000 7f9c7bca7700 1 -- 172.21.15.116:0/1244850288 wait complete. 2020-11-20T12:22:08.256 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:22:08.425 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.419+0000 7f865b59e700 1 Processor -- start 2020-11-20T12:22:08.426 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.419+0000 7f865b59e700 1 -- start start 2020-11-20T12:22:08.427 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865b59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f865c14f880 0x7f865c0b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f865c0c0e90 0x7f865c0c1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865b59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865c0a8930 con 0x7f865c0a81f0 2020-11-20T12:22:08.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865b59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865c0ba2a0 con 0x7f865c14f880 2020-11-20T12:22:08.428 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0a8690 con 0x7f865c0c0e90 2020-11-20T12:22:08.429 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f865c14f880 0x7f865c0b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f865c14f880 0x7f865c0b9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51736/0 (socket says 172.21.15.116:51736) 2020-11-20T12:22:08.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865a59c700 1 -- 172.21.15.116:0/1789865143 learned_addr learned my addr 172.21.15.116:0/1789865143 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:08.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865ad9d700 1 --2- 172.21.15.116:0/1789865143 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f865c0c0e90 0x7f865c0c1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865a59c700 1 -- 172.21.15.116:0/1789865143 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f865c0c0e90 msgr2=0x7f865c0c1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:08.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865a59c700 1 --2- 172.21.15.116:0/1789865143 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f865c0c0e90 0x7f865c0c1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865a59c700 1 -- 172.21.15.116:0/1789865143 >> v1:172.21.15.173:6789/0 conn(0x7f865c0a81f0 legacy=0x7f865c060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865a59c700 1 -- 172.21.15.116:0/1789865143 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c0b94d0 con 0x7f865c14f880 2020-11-20T12:22:08.431 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.423+0000 7f865a59c700 1 --2- 172.21.15.116:0/1789865143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f865c14f880 0x7f865c0b9db0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f864c006fd0 tx=0x7f864c006eb0).ready entity=mon.0 client_cookie=a77754dbf8edd490 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:08.432 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.427+0000 7f865959a700 1 -- 172.21.15.116:0/1789865143 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c009810 con 0x7f865c14f880 2020-11-20T12:22:08.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.427+0000 7f865959a700 1 -- 172.21.15.116:0/1789865143 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c009970 con 0x7f865c14f880 2020-11-20T12:22:08.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.427+0000 7f865959a700 1 -- 172.21.15.116:0/1789865143 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c009cf0 con 0x7f865c14f880 2020-11-20T12:22:08.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.427+0000 7f865b59e700 1 -- 172.21.15.116:0/1789865143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f865c14f880 msgr2=0x7f865c0b9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.427+0000 7f865b59e700 1 --2- 172.21.15.116:0/1789865143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f865c14f880 0x7f865c0b9db0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f864c006fd0 tx=0x7f864c006eb0).stop 2020-11-20T12:22:08.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.427+0000 7f865b59e700 1 -- 172.21.15.116:0/1789865143 shutdown_connections 2020-11-20T12:22:08.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.427+0000 7f865b59e700 1 --2- 172.21.15.116:0/1789865143 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f865c0c0e90 0x7f865c0c1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.427+0000 7f865b59e700 1 --2- 172.21.15.116:0/1789865143 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f865c14f880 0x7f865c0b9db0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.427+0000 7f865b59e700 1 -- 172.21.15.116:0/1789865143 >> 172.21.15.116:0/1789865143 conn(0x7f865c068350 msgr2=0x7f865c0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:08.434 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.427+0000 7f865b59e700 1 -- 172.21.15.116:0/1789865143 shutdown_connections 2020-11-20T12:22:08.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.427+0000 7f865b59e700 1 -- 172.21.15.116:0/1789865143 wait complete. 2020-11-20T12:22:08.435 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.431+0000 7f865b59e700 1 Processor -- start 2020-11-20T12:22:08.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.431+0000 7f865b59e700 1 -- start start 2020-11-20T12:22:08.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.431+0000 7f865b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f865c0a81f0 0x7f865c13c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.431+0000 7f865b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f865c0c0e90 0x7f865c13bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.431+0000 7f865b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f865c14f880 0x7f865c0c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.431+0000 7f865b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c150df0 con 0x7f865c14f880 2020-11-20T12:22:08.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.431+0000 7f865b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f865c0c3150 con 0x7f865c0c0e90 2020-11-20T12:22:08.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.431+0000 7f865b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f865c151050 con 0x7f865c0a81f0 2020-11-20T12:22:08.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f865c14f880 0x7f865c0c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865ad9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f865c14f880 0x7f865c0c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51746/0 (socket says 172.21.15.116:51746) 2020-11-20T12:22:08.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865ad9d700 1 -- 172.21.15.116:0/1537614624 learned_addr learned my addr 172.21.15.116:0/1537614624 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:08.440 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865a59c700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f865c0a81f0 0x7f865c13c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865ad9d700 1 -- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f865c0a81f0 msgr2=0x7f865c13c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865ad9d700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f865c0a81f0 0x7f865c13c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865ad9d700 1 -- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f865c0c0e90 msgr2=0x7f865c13bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865ad9d700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f865c0c0e90 0x7f865c13bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:08.441 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865ad9d700 1 -- 172.21.15.116:0/1537614624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c1512f0 con 0x7f865c14f880 2020-11-20T12:22:08.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865ad9d700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f865c14f880 0x7f865c0c26a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f865400ed50 tx=0x7f86540091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:08.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f864a7fc700 1 -- 172.21.15.116:0/1537614624 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865400cbb0 con 0x7f865c14f880 2020-11-20T12:22:08.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f864a7fc700 1 -- 172.21.15.116:0/1537614624 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865400cd10 con 0x7f865c14f880 2020-11-20T12:22:08.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f864a7fc700 1 -- 172.21.15.116:0/1537614624 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654012600 con 0x7f865c14f880 2020-11-20T12:22:08.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865b59e700 1 -- 172.21.15.116:0/1537614624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865c151590 con 0x7f865c14f880 2020-11-20T12:22:08.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865b59e700 1 -- 172.21.15.116:0/1537614624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865c0c1a00 con 0x7f865c14f880 2020-11-20T12:22:08.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.435+0000 7f865b59e700 1 -- 172.21.15.116:0/1537614624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f865c0ba9c0 con 0x7f865c14f880 2020-11-20T12:22:08.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.439+0000 7f864a7fc700 1 -- 172.21.15.116:0/1537614624 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8654022070 con 0x7f865c14f880 2020-11-20T12:22:08.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.439+0000 7f864a7fc700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86300557c0 0x7f8630057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.439+0000 7f864a7fc700 1 -- 172.21.15.116:0/1537614624 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f865406ef50 con 0x7f865c14f880 2020-11-20T12:22:08.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.439+0000 7f865a59c700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86300557c0 0x7f8630057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.443+0000 7f865a59c700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86300557c0 0x7f8630057c80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f864c00b1b0 tx=0x7f864c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:08.452 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.447+0000 7f864a7fc700 1 -- 172.21.15.116:0/1537614624 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f865c0ba9c0 con 0x7f865c14f880 2020-11-20T12:22:08.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.603+0000 7f865b59e700 1 -- 172.21.15.116:0/1537614624 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f865c0ba9c0 con 0x7f865c14f880 2020-11-20T12:22:08.611 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f864a7fc700 1 -- 172.21.15.116:0/1537614624 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+10 (secure 0 0 0) 0x7f865c0ba9c0 con 0x7f865c14f880 2020-11-20T12:22:08.611 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 -- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86300557c0 msgr2=0x7f8630057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86300557c0 0x7f8630057c80 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7f864c00b1b0 tx=0x7f864c015000).stop 2020-11-20T12:22:08.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 -- 172.21.15.116:0/1537614624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f865c14f880 msgr2=0x7f865c0c26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f865c14f880 0x7f865c0c26a0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7f865400ed50 tx=0x7f86540091d0).stop 2020-11-20T12:22:08.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 -- 172.21.15.116:0/1537614624 shutdown_connections 2020-11-20T12:22:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f86300557c0 0x7f8630057c80 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f865c14f880 0x7f865c0c26a0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f865c0a81f0 0x7f865c13c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 --2- 172.21.15.116:0/1537614624 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f865c0c0e90 0x7f865c13bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 -- 172.21.15.116:0/1537614624 >> 172.21.15.116:0/1537614624 conn(0x7f865c068350 msgr2=0x7f865c14d680 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 -- 172.21.15.116:0/1537614624 shutdown_connections 2020-11-20T12:22:08.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.607+0000 7f865b59e700 1 -- 172.21.15.116:0/1537614624 wait complete. 2020-11-20T12:22:08.640 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:22:08.641 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:08.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.807+0000 7f67e5564700 1 Processor -- start 2020-11-20T12:22:08.815 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67e5564700 1 -- start start 2020-11-20T12:22:08.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67e5564700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f67e00bfa80 0x7f67e00bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67e5564700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f67e00bf100 0x7f67e00bac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67e5564700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67e00bb0f0 con 0x7f67e0066e20 2020-11-20T12:22:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67e5564700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e00b9d20 con 0x7f67e00bfa80 2020-11-20T12:22:08.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67e5564700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67e00b9f80 con 0x7f67e00bf100 2020-11-20T12:22:08.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67deffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f67e00bfa80 0x7f67e00bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67deffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f67e00bfa80 0x7f67e00bfe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51750/0 (socket says 172.21.15.116:51750) 2020-11-20T12:22:08.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67deffd700 1 -- 172.21.15.116:0/369986061 learned_addr learned my addr 172.21.15.116:0/369986061 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:08.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67df7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f67e0066e20 0x7f67e014db10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:32956/0 (socket says 172.21.15.116:32956) 2020-11-20T12:22:08.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67d7fff700 1 --2- 172.21.15.116:0/369986061 >> v2:172.21.15.173:3300/0 conn(0x7f67e00bf100 0x7f67e00bac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67deffd700 1 -- 172.21.15.116:0/369986061 >> v2:172.21.15.173:3300/0 conn(0x7f67e00bf100 msgr2=0x7f67e00bac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:08.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.811+0000 7f67deffd700 1 --2- 172.21.15.116:0/369986061 >> v2:172.21.15.173:3300/0 conn(0x7f67e00bf100 0x7f67e00bac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67deffd700 1 -- 172.21.15.116:0/369986061 >> v1:172.21.15.116:6789/0 conn(0x7f67e0066e20 legacy=0x7f67e014db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67de7fc700 1 -- 172.21.15.116:0/369986061 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724898047 0 0) 0x7f67d8001450 con 0x7f67e0066e20 2020-11-20T12:22:08.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67deffd700 1 -- 172.21.15.116:0/369986061 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e00ba220 con 0x7f67e00bfa80 2020-11-20T12:22:08.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67d7fff700 1 --2- 172.21.15.116:0/369986061 >> v2:172.21.15.173:3300/0 conn(0x7f67e00bf100 0x7f67e00bac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:08.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67deffd700 1 --2- 172.21.15.116:0/369986061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67e00bfa80 0x7f67e00bfe60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f67e0065c70 tx=0x7f67d0006eb0).ready entity=mon.0 client_cookie=f050bdffc894b4a4 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:08.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67de7fc700 1 -- 172.21.15.116:0/369986061 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d00098d0 con 0x7f67e00bfa80 2020-11-20T12:22:08.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67de7fc700 1 -- 172.21.15.116:0/369986061 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d0009a30 con 0x7f67e00bfa80 2020-11-20T12:22:08.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67de7fc700 1 -- 172.21.15.116:0/369986061 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0009db0 con 0x7f67e00bfa80 2020-11-20T12:22:08.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67e5564700 1 -- 172.21.15.116:0/369986061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67e00bfa80 msgr2=0x7f67e00bfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67e5564700 1 --2- 172.21.15.116:0/369986061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67e00bfa80 0x7f67e00bfe60 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f67e0065c70 tx=0x7f67d0006eb0).stop 2020-11-20T12:22:08.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67e5564700 1 -- 172.21.15.116:0/369986061 shutdown_connections 2020-11-20T12:22:08.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67e5564700 1 --2- 172.21.15.116:0/369986061 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67e00bfa80 0x7f67e00bfe60 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67e5564700 1 --2- 172.21.15.116:0/369986061 >> v2:172.21.15.173:3300/0 conn(0x7f67e00bf100 0x7f67e00bac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67e5564700 1 -- 172.21.15.116:0/369986061 >> 172.21.15.116:0/369986061 conn(0x7f67e00c4f80 msgr2=0x7f67e00c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:08.825 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67e5564700 1 -- 172.21.15.116:0/369986061 shutdown_connections 2020-11-20T12:22:08.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.815+0000 7f67e5564700 1 -- 172.21.15.116:0/369986061 wait complete. 2020-11-20T12:22:08.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67e5564700 1 Processor -- start 2020-11-20T12:22:08.826 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67e5564700 1 -- start start 2020-11-20T12:22:08.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67e5564700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67e0066e20 0x7f67e0140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.828 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67e5564700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67e00bf100 0x7f67e00a2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67e5564700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e013e6b0 0x7f67e00a5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67e5564700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e01619a0 con 0x7f67e00bf100 2020-11-20T12:22:08.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67e5564700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0161c40 con 0x7f67e013e6b0 2020-11-20T12:22:08.829 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67e5564700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0161ee0 con 0x7f67e0066e20 2020-11-20T12:22:08.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67d7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67e00bf100 0x7f67e00a2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67d7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67e00bf100 0x7f67e00a2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51758/0 (socket says 172.21.15.116:51758) 2020-11-20T12:22:08.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67d7fff700 1 -- 172.21.15.116:0/3876369457 learned_addr learned my addr 172.21.15.116:0/3876369457 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:08.830 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67df7fe700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e013e6b0 0x7f67e00a5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67deffd700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67e0066e20 0x7f67e0140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67d7fff700 1 -- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67e0066e20 msgr2=0x7f67e0140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.819+0000 7f67d7fff700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67e0066e20 0x7f67e0140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.831 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67d7fff700 1 -- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e013e6b0 msgr2=0x7f67e00a5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:08.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67d7fff700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e013e6b0 0x7f67e00a5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:08.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67d7fff700 1 -- 172.21.15.116:0/3876369457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e0162180 con 0x7f67e00bf100 2020-11-20T12:22:08.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67df7fe700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e013e6b0 0x7f67e00a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:08.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67d7fff700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67e00bf100 0x7f67e00a2f80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f67c800d8e0 tx=0x7f67c8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:08.832 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67d6ffd700 1 -- 172.21.15.116:0/3876369457 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8009790 con 0x7f67e00bf100 2020-11-20T12:22:08.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67d6ffd700 1 -- 172.21.15.116:0/3876369457 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c80098f0 con 0x7f67e00bf100 2020-11-20T12:22:08.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67e5564700 1 -- 172.21.15.116:0/3876369457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e0162420 con 0x7f67e00bf100 2020-11-20T12:22:08.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67d6ffd700 1 -- 172.21.15.116:0/3876369457 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c8009c70 con 0x7f67e00bf100 2020-11-20T12:22:08.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67e5564700 1 -- 172.21.15.116:0/3876369457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e00a39b0 con 0x7f67e00bf100 2020-11-20T12:22:08.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67e5564700 1 -- 172.21.15.116:0/3876369457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e005d2a0 con 0x7f67e00bf100 2020-11-20T12:22:08.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.823+0000 7f67d6ffd700 1 -- 172.21.15.116:0/3876369457 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f67c8013690 con 0x7f67e00bf100 2020-11-20T12:22:08.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.827+0000 7f67d6ffd700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67bc0557c0 0x7f67bc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:08.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.827+0000 7f67d6ffd700 1 -- 172.21.15.116:0/3876369457 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f67c806e9e0 con 0x7f67e00bf100 2020-11-20T12:22:08.835 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.827+0000 7f67deffd700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67bc0557c0 0x7f67bc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:08.836 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.827+0000 7f67deffd700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67bc0557c0 0x7f67bc057c80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f67e00c0ec0 tx=0x7f67d0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:08.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:08.831+0000 7f67d6ffd700 1 -- 172.21.15.116:0/3876369457 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f67c8044880 con 0x7f67e00bf100 2020-11-20T12:22:09.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.039+0000 7f67e5564700 1 -- 172.21.15.116:0/3876369457 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f67e0061380 con 0x7f67e00bf100 2020-11-20T12:22:09.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.039+0000 7f67d6ffd700 1 -- 172.21.15.116:0/3876369457 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2854 (secure 0 0 0) 0x7f67c8047ea0 con 0x7f67e00bf100 2020-11-20T12:22:09.045 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:09.046 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/1612795957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1612795957},{"type":"v1","addr":"172.21.15.116:6835","nonce":1612795957}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/3255680781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3255680781},{"type":"v1","addr":"172.21.15.173:6835","nonce":3255680781}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/4058046293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":4058046293},{"type":"v1","addr":"172.21.15.173:6837","nonce":4058046293}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:56.570302+0000","modified":"2020-11-20T12:21:58.589498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8148":{"gid":8148,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2775705840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2775705840},{"type":"v1","addr":"172.21.15.116:6833","nonce":2775705840}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-20T12:22:09.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 -- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67bc0557c0 msgr2=0x7f67bc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:09.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67bc0557c0 0x7f67bc057c80 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f67e00c0ec0 tx=0x7f67d0015000).stop 2020-11-20T12:22:09.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 -- 172.21.15.116:0/3876369457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67e00bf100 msgr2=0x7f67e00a2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:09.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67e00bf100 0x7f67e00a2f80 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f67c800d8e0 tx=0x7f67c8008040).stop 2020-11-20T12:22:09.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 -- 172.21.15.116:0/3876369457 shutdown_connections 2020-11-20T12:22:09.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f67bc0557c0 0x7f67bc057c80 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:09.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f67e013e6b0 0x7f67e00a5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:09.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f67e0066e20 0x7f67e0140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:09.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 --2- 172.21.15.116:0/3876369457 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f67e00bf100 0x7f67e00a2f80 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:09.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 -- 172.21.15.116:0/3876369457 >> 172.21.15.116:0/3876369457 conn(0x7f67e00c4f80 msgr2=0x7f67e014c9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:09.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 -- 172.21.15.116:0/3876369457 shutdown_connections 2020-11-20T12:22:09.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:09.043+0000 7f67e5564700 1 -- 172.21.15.116:0/3876369457 wait complete. 2020-11-20T12:22:09.051 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 68 2020-11-20T12:22:09.475 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:21:56.570302+0000', 'modified': '2020-11-20T12:21:58.589498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8148}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8148': {'gid': 8148, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.116:6833/2775705840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 2775705840}, {'type': 'v1', 'addr': '172.21.15.116:6833', 'nonce': 2775705840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:22:09.798 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:22:09.799 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:22:09.995 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":8148,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.40196143,"mdsmap_epoch":68,"osdmap_epoch":118,"osdmap_epoch_barrier":118,"uptime":26.421999893999999} 2020-11-20T12:22:09.996 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 8148, "mdsmap_epoch": 68, "osdmap_epoch": 118, "osdmap_epoch_barrier": 118, "rank_uptime": 13.40196143, "state": "up:active", "uptime": 26.421999894, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:22:09.997 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:22:09.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:09.998 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-20T12:22:10.017 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:22:10.017 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:10.089 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:10.090 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:22:10.115 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:10.115 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:22:10.121 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:22:10.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:10.123 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-20T12:22:10.153 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:22:10.154 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:22:10.154 INFO:tasks.cephfs.mount:self.client.name = client.2 2020-11-20T12:22:10.154 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2020-11-20T12:22:10.155 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:22:10.155 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:22:10.156 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:22:10.156 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:22:10.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:10.157 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:22:10.171 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:22:10.172 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:22:10.172 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:22:10.172 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:10.172 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:22:10.172 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:10.173 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:10.173 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:10.173 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:10.173 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:10.174 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:10.174 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:10.175 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:22:10.176 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:10.176 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:22:10.176 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:10.177 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:22:10.177 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:10.177 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:22:10.177 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:10.178 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:22:10.178 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:10.178 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:22:10.179 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:10.179 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:22:10.179 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:10.179 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:22:10.179 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:10.180 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:10.180 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:22:10.180 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:22:10.181 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:10.181 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:10.181 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:22:10.181 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:22:10.181 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:10.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:10.183 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:22:10.261 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:22:10.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:10.266 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:22:10.284 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:22:10.284 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:22:10.284 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:22:10.285 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:22:10.285 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:22:10.285 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:22:10.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:10.287 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:22:10.287 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:22:10.288 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:22:10.288 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:22:10.289 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:22:10.289 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:22:10.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:10.384 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:22:10.428 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:22:10.429 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:22:10.429 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:22:10.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:10.431 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2020-11-20T12:22:10.478 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:10.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.695+0000 7fd8b58d6700 1 Processor -- start 2020-11-20T12:22:10.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.695+0000 7fd8b58d6700 1 -- start start 2020-11-20T12:22:10.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8b58d6700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd8b00a54d0 0x7fd8b0060c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:10.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8b58d6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b00a5970 con 0x7fd8b0155ba0 2020-11-20T12:22:10.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8b58d6700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b0060280 con 0x7fd8b00623a0 2020-11-20T12:22:10.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8b58d6700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b0060520 con 0x7fd8b00a54d0 2020-11-20T12:22:10.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8aeffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd8b0155ba0 0x7fd8b0065330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:32966/0 (socket says 172.21.15.116:32966) 2020-11-20T12:22:10.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8aeffd700 1 -- 172.21.15.116:0/3231590622 learned_addr learned my addr 172.21.15.116:0/3231590622 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:10.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8ae7fc700 1 --2- 172.21.15.116:0/3231590622 >> v2:172.21.15.173:3300/0 conn(0x7fd8b00a54d0 0x7fd8b0060c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:10.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4258378104 0 0) 0x7fd8b00a5970 con 0x7fd8b0155ba0 2020-11-20T12:22:10.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd89c003240 con 0x7fd8b0155ba0 2020-11-20T12:22:10.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231872486 0 0) 0x7fd8b0060280 con 0x7fd8b00623a0 2020-11-20T12:22:10.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8b00a5970 con 0x7fd8b00623a0 2020-11-20T12:22:10.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3906720917 0 0) 0x7fd89c003240 con 0x7fd8b0155ba0 2020-11-20T12:22:10.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 >> v2:172.21.15.173:3300/0 conn(0x7fd8b00a54d0 msgr2=0x7fd8b0060c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:10.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 --2- 172.21.15.116:0/3231590622 >> v2:172.21.15.173:3300/0 conn(0x7fd8b00a54d0 0x7fd8b0060c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:10.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 >> v1:172.21.15.173:6789/0 conn(0x7fd8b00623a0 legacy=0x7fd8b0061150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:10.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b00607c0 con 0x7fd8b0155ba0 2020-11-20T12:22:10.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fd8a0001860 con 0x7fd8b0155ba0 2020-11-20T12:22:10.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 >> v1:172.21.15.116:6789/0 conn(0x7fd8b0155ba0 legacy=0x7fd8b0065330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:10.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 --2- 172.21.15.116:0/3231590622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd89c0038c0 0x7fd89c003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:10.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 --2- 172.21.15.116:0/3231590622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd89c0044e0 0x7fd89c004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:10.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 --2- 172.21.15.116:0/3231590622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd89c004e50 0x7fd89c00b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:10.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0060520 con 0x7fd89c004e50 2020-11-20T12:22:10.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c00bc20 con 0x7fd89c0038c0 2020-11-20T12:22:10.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c00bd60 con 0x7fd89c0044e0 2020-11-20T12:22:10.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.699+0000 7fd8affff700 1 --2- 172.21.15.116:0/3231590622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd89c004e50 0x7fd89c00b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:10.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8aeffd700 1 --2- 172.21.15.116:0/3231590622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd89c0038c0 0x7fd89c003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:10.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8ae7fc700 1 --2- 172.21.15.116:0/3231590622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd89c0044e0 0x7fd89c004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:10.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8affff700 1 -- 172.21.15.116:0/3231590622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd89c0044e0 msgr2=0x7fd89c004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:10.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8affff700 1 --2- 172.21.15.116:0/3231590622 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd89c0044e0 0x7fd89c004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:10.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8affff700 1 -- 172.21.15.116:0/3231590622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd89c0038c0 msgr2=0x7fd89c003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:10.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8affff700 1 --2- 172.21.15.116:0/3231590622 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd89c0038c0 0x7fd89c003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:10.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8affff700 1 -- 172.21.15.116:0/3231590622 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c003240 con 0x7fd89c004e50 2020-11-20T12:22:10.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8affff700 1 -- 172.21.15.116:0/3231590622 reap_dead start 2020-11-20T12:22:10.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8affff700 1 --2- 172.21.15.116:0/3231590622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd89c004e50 0x7fd89c00b730 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7fd8a8001ab0 tx=0x7fd8a8007b90).ready entity=mon.0 client_cookie=93d5735475b9f71c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:10.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a8007020 con 0x7fd89c004e50 2020-11-20T12:22:10.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a8007180 con 0x7fd89c004e50 2020-11-20T12:22:10.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8adffb700 1 -- 172.21.15.116:0/3231590622 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a800c8b0 con 0x7fd89c004e50 2020-11-20T12:22:10.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3231590622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd89c004e50 msgr2=0x7fd89c00b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:10.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8b58d6700 1 --2- 172.21.15.116:0/3231590622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd89c004e50 0x7fd89c00b730 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7fd8a8001ab0 tx=0x7fd8a8007b90).stop 2020-11-20T12:22:10.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3231590622 shutdown_connections 2020-11-20T12:22:10.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8b58d6700 1 --2- 172.21.15.116:0/3231590622 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd89c004e50 0x7fd89c00b730 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:10.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3231590622 >> 172.21.15.116:0/3231590622 conn(0x7fd8b00c63f0 msgr2=0x7fd8b00c23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:10.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3231590622 shutdown_connections 2020-11-20T12:22:10.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.703+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3231590622 wait complete. 2020-11-20T12:22:10.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.707+0000 7fd8b58d6700 1 Processor -- start 2020-11-20T12:22:10.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.707+0000 7fd8b58d6700 1 -- start start 2020-11-20T12:22:10.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.707+0000 7fd8b58d6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd89c004e50 0x7fd8b0157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:10.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.707+0000 7fd8b58d6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b0156150 0x7fd8b0158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:10.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.707+0000 7fd8b58d6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b0158ff0 0x7fd8b00c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:10.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.707+0000 7fd8b58d6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b0159410 con 0x7fd8b0156150 2020-11-20T12:22:10.730 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.707+0000 7fd8b58d6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b01686b0 con 0x7fd89c004e50 2020-11-20T12:22:10.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.707+0000 7fd8b58d6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b01688e0 con 0x7fd8b0158ff0 2020-11-20T12:22:10.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.707+0000 7fd8ae7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b0156150 0x7fd8b0158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:10.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.707+0000 7fd8ae7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b0156150 0x7fd8b0158b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51778/0 (socket says 172.21.15.116:51778) 2020-11-20T12:22:10.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.707+0000 7fd8ae7fc700 1 -- 172.21.15.116:0/3923665850 learned_addr learned my addr 172.21.15.116:0/3923665850 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:10.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd8aeffd700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd89c004e50 0x7fd8b0157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:10.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd8affff700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b0158ff0 0x7fd8b00c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:10.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd8ae7fc700 1 -- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b0158ff0 msgr2=0x7fd8b00c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:10.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd8ae7fc700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b0158ff0 0x7fd8b00c22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:10.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd8ae7fc700 1 -- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd89c004e50 msgr2=0x7fd8b0157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:10.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd8ae7fc700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd89c004e50 0x7fd8b0157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:10.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd8ae7fc700 1 -- 172.21.15.116:0/3923665850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0168b80 con 0x7fd8b0156150 2020-11-20T12:22:10.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd8ae7fc700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b0156150 0x7fd8b0158b00 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fd8a4010a20 tx=0x7fd8a4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:10.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd896ffd700 1 -- 172.21.15.116:0/3923665850 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400e9f0 con 0x7fd8b0156150 2020-11-20T12:22:10.733 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd896ffd700 1 -- 172.21.15.116:0/3923665850 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a400eb50 con 0x7fd8b0156150 2020-11-20T12:22:10.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3923665850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b0168e20 con 0x7fd8b0156150 2020-11-20T12:22:10.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd896ffd700 1 -- 172.21.15.116:0/3923665850 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400eed0 con 0x7fd8b0156150 2020-11-20T12:22:10.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3923665850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b0169360 con 0x7fd8b0156150 2020-11-20T12:22:10.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3923665850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b01694f0 con 0x7fd8b0156150 2020-11-20T12:22:10.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.711+0000 7fd896ffd700 1 -- 172.21.15.116:0/3923665850 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd8a4014680 con 0x7fd8b0156150 2020-11-20T12:22:10.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.715+0000 7fd896ffd700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd88c0557c0 0x7fd88c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:10.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.715+0000 7fd896ffd700 1 -- 172.21.15.116:0/3923665850 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd8a406ead0 con 0x7fd8b0156150 2020-11-20T12:22:10.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.715+0000 7fd8aeffd700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd88c0557c0 0x7fd88c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:10.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.715+0000 7fd8aeffd700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd88c0557c0 0x7fd88c057c80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fd8a0007fd0 tx=0x7fd8a0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:10.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.719+0000 7fd896ffd700 1 -- 172.21.15.116:0/3923665850 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd8a4044970 con 0x7fd8b0156150 2020-11-20T12:22:10.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.875+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3923665850 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8b005d2a0 con 0x7fd8b0156150 2020-11-20T12:22:10.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.879+0000 7fd896ffd700 1 -- 172.21.15.116:0/3923665850 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16888 (secure 0 0 0) 0x7fd8a4047f90 con 0x7fd8b0156150 2020-11-20T12:22:10.884 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:10.885 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":118,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:56.569994+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:53.361384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:53.918177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:10.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd88c0557c0 msgr2=0x7fd88c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:10.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd88c0557c0 0x7fd88c057c80 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fd8a0007fd0 tx=0x7fd8a0009040).stop 2020-11-20T12:22:10.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3923665850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b0156150 msgr2=0x7fd8b0158b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:10.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b0156150 0x7fd8b0158b00 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fd8a4010a20 tx=0x7fd8a4008040).stop 2020-11-20T12:22:10.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3923665850 shutdown_connections 2020-11-20T12:22:10.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd88c0557c0 0x7fd88c057c80 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:10.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd8b0158ff0 0x7fd8b00c22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:10.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd89c004e50 0x7fd8b0157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:10.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 --2- 172.21.15.116:0/3923665850 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd8b0156150 0x7fd8b0158b00 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:10.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3923665850 >> 172.21.15.116:0/3923665850 conn(0x7fd8b00c63f0 msgr2=0x7fd8b01523e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:10.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3923665850 shutdown_connections 2020-11-20T12:22:10.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:10.883+0000 7fd8b58d6700 1 -- 172.21.15.116:0/3923665850 wait complete. 2020-11-20T12:22:10.914 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:21:53.361384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:21:53.918177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:22:10.915 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:22:11.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.087+0000 7f3bd5a54700 1 Processor -- start 2020-11-20T12:22:11.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.087+0000 7f3bd5a54700 1 -- start start 2020-11-20T12:22:11.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.091+0000 7f3bd5a54700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3bd0064bf0 0x7f3bd00ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.091+0000 7f3bd5a54700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd00a8ba0 con 0x7f3bd0064fd0 2020-11-20T12:22:11.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.091+0000 7f3bd5a54700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bd00a8e40 con 0x7f3bd00680c0 2020-11-20T12:22:11.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.091+0000 7f3bd5a54700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bd0068570 con 0x7f3bd0064bf0 2020-11-20T12:22:11.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.091+0000 7f3bceffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3bd0064bf0 0x7f3bd00ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.091+0000 7f3bceffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f3bd0064bf0 0x7f3bd00ac3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51784/0 (socket says 172.21.15.116:51784) 2020-11-20T12:22:11.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.091+0000 7f3bceffd700 1 -- 172.21.15.116:0/3752219329 learned_addr learned my addr 172.21.15.116:0/3752219329 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:11.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.091+0000 7f3bceffd700 1 -- 172.21.15.116:0/3752219329 >> v1:172.21.15.173:6789/0 conn(0x7f3bd00680c0 legacy=0x7f3bd014f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.091+0000 7f3bceffd700 1 -- 172.21.15.116:0/3752219329 >> v1:172.21.15.116:6789/0 conn(0x7f3bd0064fd0 legacy=0x7f3bd00a62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.091+0000 7f3bceffd700 1 -- 172.21.15.116:0/3752219329 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00a90e0 con 0x7f3bd0064bf0 2020-11-20T12:22:11.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.091+0000 7f3bceffd700 1 --2- 172.21.15.116:0/3752219329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bd0064bf0 0x7f3bd00ac3f0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f3bb8006fd0 tx=0x7f3bb8006eb0).ready entity=mon.0 client_cookie=6344ea9875f987a7 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:11.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.095+0000 7f3bcdffb700 1 -- 172.21.15.116:0/3752219329 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb80098d0 con 0x7f3bd0064bf0 2020-11-20T12:22:11.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.095+0000 7f3bcdffb700 1 -- 172.21.15.116:0/3752219329 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb8009a30 con 0x7f3bd0064bf0 2020-11-20T12:22:11.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.095+0000 7f3bcdffb700 1 -- 172.21.15.116:0/3752219329 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8009db0 con 0x7f3bd0064bf0 2020-11-20T12:22:11.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.095+0000 7f3bd5a54700 1 -- 172.21.15.116:0/3752219329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bd0064bf0 msgr2=0x7f3bd00ac3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.095+0000 7f3bd5a54700 1 --2- 172.21.15.116:0/3752219329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bd0064bf0 0x7f3bd00ac3f0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f3bb8006fd0 tx=0x7f3bb8006eb0).stop 2020-11-20T12:22:11.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.095+0000 7f3bd5a54700 1 -- 172.21.15.116:0/3752219329 shutdown_connections 2020-11-20T12:22:11.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.095+0000 7f3bd5a54700 1 --2- 172.21.15.116:0/3752219329 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bd0064bf0 0x7f3bd00ac3f0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.095+0000 7f3bd5a54700 1 -- 172.21.15.116:0/3752219329 >> 172.21.15.116:0/3752219329 conn(0x7f3bd0060c00 msgr2=0x7f3bd00c66f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:11.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.095+0000 7f3bd5a54700 1 -- 172.21.15.116:0/3752219329 shutdown_connections 2020-11-20T12:22:11.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.095+0000 7f3bd5a54700 1 -- 172.21.15.116:0/3752219329 wait complete. 2020-11-20T12:22:11.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.099+0000 7f3bd5a54700 1 Processor -- start 2020-11-20T12:22:11.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.099+0000 7f3bd5a54700 1 -- start start 2020-11-20T12:22:11.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.099+0000 7f3bd5a54700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bd0064bf0 0x7f3bd00c1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.099+0000 7f3bd5a54700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bd0064fd0 0x7f3bd0165320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.099+0000 7f3bd5a54700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bd00680c0 0x7f3bd00c4b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.099+0000 7f3bd5a54700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd01649b0 con 0x7f3bd0064fd0 2020-11-20T12:22:11.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.099+0000 7f3bd5a54700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00c24c0 con 0x7f3bd0064bf0 2020-11-20T12:22:11.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.099+0000 7f3bd5a54700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd00bfde0 con 0x7f3bd00680c0 2020-11-20T12:22:11.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bd0064fd0 0x7f3bd0165320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bce7fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bd0064fd0 0x7f3bd0165320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51792/0 (socket says 172.21.15.116:51792) 2020-11-20T12:22:11.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bceffd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bd0064bf0 0x7f3bd00c1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bce7fc700 1 -- 172.21.15.116:0/28035615 learned_addr learned my addr 172.21.15.116:0/28035615 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:11.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bcf7fe700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bd00680c0 0x7f3bd00c4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bce7fc700 1 -- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bd00680c0 msgr2=0x7f3bd00c4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bce7fc700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bd00680c0 0x7f3bd00c4b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bce7fc700 1 -- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bd0064bf0 msgr2=0x7f3bd00c1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bce7fc700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bd0064bf0 0x7f3bd00c1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bce7fc700 1 -- 172.21.15.116:0/28035615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd00c0080 con 0x7f3bd0064fd0 2020-11-20T12:22:11.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bcf7fe700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bd00680c0 0x7f3bd00c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:11.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bce7fc700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bd0064fd0 0x7f3bd0165320 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f3bc000c710 tx=0x7f3bc0007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:11.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bc6ffd700 1 -- 172.21.15.116:0/28035615 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc000e490 con 0x7f3bd0064fd0 2020-11-20T12:22:11.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bc6ffd700 1 -- 172.21.15.116:0/28035615 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc000e5f0 con 0x7f3bd0064fd0 2020-11-20T12:22:11.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bd5a54700 1 -- 172.21.15.116:0/28035615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bd0164be0 con 0x7f3bd0064fd0 2020-11-20T12:22:11.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bc6ffd700 1 -- 172.21.15.116:0/28035615 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc000e990 con 0x7f3bd0064fd0 2020-11-20T12:22:11.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.103+0000 7f3bd5a54700 1 -- 172.21.15.116:0/28035615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bd0165890 con 0x7f3bd0064fd0 2020-11-20T12:22:11.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.107+0000 7f3bd5a54700 1 -- 172.21.15.116:0/28035615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bd005d2a0 con 0x7f3bd0064fd0 2020-11-20T12:22:11.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.107+0000 7f3bc6ffd700 1 -- 172.21.15.116:0/28035615 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3bc0010070 con 0x7f3bd0064fd0 2020-11-20T12:22:11.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.107+0000 7f3bc6ffd700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bac0557c0 0x7f3bac057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.107+0000 7f3bc6ffd700 1 -- 172.21.15.116:0/28035615 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3bc006dab0 con 0x7f3bd0064fd0 2020-11-20T12:22:11.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.107+0000 7f3bceffd700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bac0557c0 0x7f3bac057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.111+0000 7f3bceffd700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bac0557c0 0x7f3bac057c80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f3bb800bf60 tx=0x7f3bb8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:11.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.115+0000 7f3bc6ffd700 1 -- 172.21.15.116:0/28035615 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3bc0042fd0 con 0x7f3bd0064fd0 2020-11-20T12:22:11.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.275+0000 7f3bd5a54700 1 -- 172.21.15.116:0/28035615 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f3bd00635b0 con 0x7f3bd0064fd0 2020-11-20T12:22:11.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.275+0000 7f3bc6ffd700 1 -- 172.21.15.116:0/28035615 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v118) v1 ==== 110+0+10 (secure 0 0 0) 0x7f3bc0043950 con 0x7f3bd0064fd0 2020-11-20T12:22:11.281 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:22:11.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.275+0000 7f3bd5a54700 1 -- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bac0557c0 msgr2=0x7f3bac057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.275+0000 7f3bd5a54700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bac0557c0 0x7f3bac057c80 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f3bb800bf60 tx=0x7f3bb8015000).stop 2020-11-20T12:22:11.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.275+0000 7f3bd5a54700 1 -- 172.21.15.116:0/28035615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bd0064fd0 msgr2=0x7f3bd0165320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.275+0000 7f3bd5a54700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bd0064fd0 0x7f3bd0165320 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7f3bc000c710 tx=0x7f3bc0007bd0).stop 2020-11-20T12:22:11.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.279+0000 7f3bd5a54700 1 -- 172.21.15.116:0/28035615 shutdown_connections 2020-11-20T12:22:11.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.279+0000 7f3bd5a54700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3bac0557c0 0x7f3bac057c80 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.279+0000 7f3bd5a54700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3bd00680c0 0x7f3bd00c4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.279+0000 7f3bd5a54700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3bd0064bf0 0x7f3bd00c1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.279+0000 7f3bd5a54700 1 --2- 172.21.15.116:0/28035615 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3bd0064fd0 0x7f3bd0165320 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.279+0000 7f3bd5a54700 1 -- 172.21.15.116:0/28035615 >> 172.21.15.116:0/28035615 conn(0x7f3bd0060c00 msgr2=0x7f3bd014dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:11.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.279+0000 7f3bd5a54700 1 -- 172.21.15.116:0/28035615 shutdown_connections 2020-11-20T12:22:11.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.279+0000 7f3bd5a54700 1 -- 172.21.15.116:0/28035615 wait complete. 2020-11-20T12:22:11.304 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:22:11.495 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.491+0000 7ff6ed5ee700 1 Processor -- start 2020-11-20T12:22:11.496 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.491+0000 7ff6ed5ee700 1 -- start start 2020-11-20T12:22:11.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6ed5ee700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7ff6e80aa7d0 0x7ff6e80a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6ed5ee700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff6e80aabb0 0x7ff6e8151e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6ed5ee700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e80ace40 con 0x7ff6e80ad540 2020-11-20T12:22:11.499 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6ed5ee700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e81526c0 con 0x7ff6e80aabb0 2020-11-20T12:22:11.500 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6ed5ee700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e8152960 con 0x7ff6e80aa7d0 2020-11-20T12:22:11.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e77fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff6e80aabb0 0x7ff6e8151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e77fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff6e80aabb0 0x7ff6e8151e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51800/0 (socket says 172.21.15.116:51800) 2020-11-20T12:22:11.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e77fe700 1 -- 172.21.15.116:0/1152227682 learned_addr learned my addr 172.21.15.116:0/1152227682 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:11.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e7fff700 1 --2- 172.21.15.116:0/1152227682 >> v2:172.21.15.173:3300/0 conn(0x7ff6e80aa7d0 0x7ff6e80a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e77fe700 1 -- 172.21.15.116:0/1152227682 >> v2:172.21.15.173:3300/0 conn(0x7ff6e80aa7d0 msgr2=0x7ff6e80a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:11.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e77fe700 1 --2- 172.21.15.116:0/1152227682 >> v2:172.21.15.173:3300/0 conn(0x7ff6e80aa7d0 0x7ff6e80a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e77fe700 1 -- 172.21.15.116:0/1152227682 >> v1:172.21.15.173:6789/0 conn(0x7ff6e80ad540 legacy=0x7ff6e80ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e77fe700 1 -- 172.21.15.116:0/1152227682 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e8152c00 con 0x7ff6e80aabb0 2020-11-20T12:22:11.502 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e77fe700 1 --2- 172.21.15.116:0/1152227682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e80aabb0 0x7ff6e8151e00 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7ff6e8081950 tx=0x7ff6dc006eb0).ready entity=mon.0 client_cookie=d525b897ea62190 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:11.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e6ffd700 1 -- 172.21.15.116:0/1152227682 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc0098d0 con 0x7ff6e80aabb0 2020-11-20T12:22:11.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e6ffd700 1 -- 172.21.15.116:0/1152227682 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc009a30 con 0x7ff6e80aabb0 2020-11-20T12:22:11.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.495+0000 7ff6e6ffd700 1 -- 172.21.15.116:0/1152227682 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc009db0 con 0x7ff6e80aabb0 2020-11-20T12:22:11.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.499+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/1152227682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e80aabb0 msgr2=0x7ff6e8151e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.504 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.499+0000 7ff6ed5ee700 1 --2- 172.21.15.116:0/1152227682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e80aabb0 0x7ff6e8151e00 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7ff6e8081950 tx=0x7ff6dc006eb0).stop 2020-11-20T12:22:11.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.499+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/1152227682 shutdown_connections 2020-11-20T12:22:11.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.499+0000 7ff6ed5ee700 1 --2- 172.21.15.116:0/1152227682 >> v2:172.21.15.173:3300/0 conn(0x7ff6e80aa7d0 0x7ff6e80a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.499+0000 7ff6ed5ee700 1 --2- 172.21.15.116:0/1152227682 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e80aabb0 0x7ff6e8151e00 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.499+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/1152227682 >> 172.21.15.116:0/1152227682 conn(0x7ff6e80678a0 msgr2=0x7ff6e8067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:11.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.499+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/1152227682 shutdown_connections 2020-11-20T12:22:11.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.499+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/1152227682 wait complete. 2020-11-20T12:22:11.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.499+0000 7ff6ed5ee700 1 Processor -- start 2020-11-20T12:22:11.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6ed5ee700 1 -- start start 2020-11-20T12:22:11.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6ed5ee700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6e80aa7d0 0x7ff6e8161c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6ed5ee700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e80aabb0 0x7ff6e80c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6ed5ee700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e80ad540 0x7ff6e814ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6ed5ee700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e80be300 con 0x7ff6e80aabb0 2020-11-20T12:22:11.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6ed5ee700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e8162140 con 0x7ff6e80ad540 2020-11-20T12:22:11.509 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6ed5ee700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e8164bd0 con 0x7ff6e80aa7d0 2020-11-20T12:22:11.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6e77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e80aabb0 0x7ff6e80c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6e77fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e80aabb0 0x7ff6e80c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51806/0 (socket says 172.21.15.116:51806) 2020-11-20T12:22:11.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6e77fe700 1 -- 172.21.15.116:0/2933493499 learned_addr learned my addr 172.21.15.116:0/2933493499 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:11.511 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6e7fff700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6e80aa7d0 0x7ff6e8161c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6ecded700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e80ad540 0x7ff6e814ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6e77fe700 1 -- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6e80aa7d0 msgr2=0x7ff6e8161c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6e77fe700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6e80aa7d0 0x7ff6e8161c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6e77fe700 1 -- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e80ad540 msgr2=0x7ff6e814ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.512 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6e77fe700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e80ad540 0x7ff6e814ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6e77fe700 1 -- 172.21.15.116:0/2933493499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e8164e70 con 0x7ff6e80aabb0 2020-11-20T12:22:11.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.503+0000 7ff6e77fe700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e80aabb0 0x7ff6e80c4f10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7ff6e80acfd0 tx=0x7ff6dc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:11.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.507+0000 7ff6c7fff700 1 -- 172.21.15.116:0/2933493499 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc011dd0 con 0x7ff6e80aabb0 2020-11-20T12:22:11.513 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.507+0000 7ff6c7fff700 1 -- 172.21.15.116:0/2933493499 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6dc01e460 con 0x7ff6e80aabb0 2020-11-20T12:22:11.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.507+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/2933493499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e8165110 con 0x7ff6e80aabb0 2020-11-20T12:22:11.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.507+0000 7ff6c7fff700 1 -- 172.21.15.116:0/2933493499 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6dc01e800 con 0x7ff6e80aabb0 2020-11-20T12:22:11.514 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.507+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/2933493499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e80bd9a0 con 0x7ff6e80aabb0 2020-11-20T12:22:11.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.507+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/2933493499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e80bf5e0 con 0x7ff6e80aabb0 2020-11-20T12:22:11.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.507+0000 7ff6c7fff700 1 -- 172.21.15.116:0/2933493499 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff6dc025030 con 0x7ff6e80aabb0 2020-11-20T12:22:11.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.507+0000 7ff6c7fff700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6c00557c0 0x7ff6c0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.515 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.511+0000 7ff6c7fff700 1 -- 172.21.15.116:0/2933493499 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff6dc06fca0 con 0x7ff6e80aabb0 2020-11-20T12:22:11.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.511+0000 7ff6e7fff700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6c00557c0 0x7ff6c0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.516 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.511+0000 7ff6e7fff700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6c00557c0 0x7ff6c0057c80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7ff6d800ba00 tx=0x7ff6d8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:11.520 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.511+0000 7ff6c7fff700 1 -- 172.21.15.116:0/2933493499 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff6dc074070 con 0x7ff6e80aabb0 2020-11-20T12:22:11.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.675+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/2933493499 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff6e80c5bd0 con 0x7ff6e80aabb0 2020-11-20T12:22:11.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.675+0000 7ff6c7fff700 1 -- 172.21.15.116:0/2933493499 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v118) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff6dc023030 con 0x7ff6e80aabb0 2020-11-20T12:22:11.683 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:11.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6c00557c0 msgr2=0x7ff6c0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6c00557c0 0x7ff6c0057c80 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7ff6d800ba00 tx=0x7ff6d8008040).stop 2020-11-20T12:22:11.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/2933493499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e80aabb0 msgr2=0x7ff6e80c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e80aabb0 0x7ff6e80c4f10 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7ff6e80acfd0 tx=0x7ff6dc015000).stop 2020-11-20T12:22:11.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/2933493499 shutdown_connections 2020-11-20T12:22:11.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff6c00557c0 0x7ff6c0057c80 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff6e80ad540 0x7ff6e814ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff6e80aa7d0 0x7ff6e8161c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 --2- 172.21.15.116:0/2933493499 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff6e80aabb0 0x7ff6e80c4f10 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/2933493499 >> 172.21.15.116:0/2933493499 conn(0x7ff6e80678a0 msgr2=0x7ff6e8154650 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:11.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/2933493499 shutdown_connections 2020-11-20T12:22:11.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.679+0000 7ff6ed5ee700 1 -- 172.21.15.116:0/2933493499 wait complete. 2020-11-20T12:22:11.712 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:22:11.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.887+0000 7f4f0ddcc700 1 Processor -- start 2020-11-20T12:22:11.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.887+0000 7f4f0ddcc700 1 -- start start 2020-11-20T12:22:11.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f0ddcc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f08156440 0x7f4f080aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f0ddcc700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f080ace40 con 0x7f4f080aa7d0 2020-11-20T12:22:11.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f0ddcc700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f081526c0 con 0x7f4f080a8400 2020-11-20T12:22:11.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f0ddcc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08152960 con 0x7f4f08156440 2020-11-20T12:22:11.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f0cdca700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4f080aa7d0 0x7f4f08151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33014/0 (socket says 172.21.15.116:33014) 2020-11-20T12:22:11.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f0cdca700 1 -- 172.21.15.116:0/1545771011 learned_addr learned my addr 172.21.15.116:0/1545771011 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:11.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f0d5cb700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f08156440 0x7f4f080aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1258635679 0 0) 0x7f4f080ace40 con 0x7f4f080aa7d0 2020-11-20T12:22:11.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4eec003240 con 0x7f4f080aa7d0 2020-11-20T12:22:11.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 863463854 0 0) 0x7f4eec003240 con 0x7f4f080aa7d0 2020-11-20T12:22:11.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f08156440 msgr2=0x7f4f080aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:11.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f08156440 0x7f4f080aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 >> v1:172.21.15.173:6789/0 conn(0x7f4f080a8400 legacy=0x7f4f080ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f08152c00 con 0x7f4f080aa7d0 2020-11-20T12:22:11.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f4efc0014e0 con 0x7f4f080aa7d0 2020-11-20T12:22:11.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 >> v1:172.21.15.116:6789/0 conn(0x7f4f080aa7d0 legacy=0x7f4f08151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4eec003760 0x7f4eec003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4eec0044e0 0x7f4eec004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f08156440 0x7f4eec004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08152960 con 0x7f4eec003760 2020-11-20T12:22:11.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec005340 con 0x7f4eec0044e0 2020-11-20T12:22:11.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec00b730 con 0x7f4f08156440 2020-11-20T12:22:11.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f0d5cb700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f08156440 0x7f4eec004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.891+0000 7f4f0cdca700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4eec003760 0x7f4eec003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f07fff700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4eec0044e0 0x7f4eec004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0d5cb700 1 -- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4eec0044e0 msgr2=0x7f4eec004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:11.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0d5cb700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4eec0044e0 0x7f4eec004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0d5cb700 1 -- 172.21.15.116:0/1545771011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4eec003760 msgr2=0x7f4eec003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:11.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0d5cb700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4eec003760 0x7f4eec003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0d5cb700 1 -- 172.21.15.116:0/1545771011 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec003240 con 0x7f4f08156440 2020-11-20T12:22:11.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0cdca700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4eec003760 0x7f4eec003b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:11.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0d5cb700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f08156440 0x7f4eec004e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f4f00009e70 tx=0x7f4f000091d0).ready entity=mon.2 client_cookie=a05585349be51054 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:11.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f000095c0 con 0x7f4f08156440 2020-11-20T12:22:11.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0000ec40 con 0x7f4f08156440 2020-11-20T12:22:11.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f077fe700 1 -- 172.21.15.116:0/1545771011 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f000123f0 con 0x7f4f08156440 2020-11-20T12:22:11.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f08156440 msgr2=0x7f4eec004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0ddcc700 1 --2- 172.21.15.116:0/1545771011 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f08156440 0x7f4eec004e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f4f00009e70 tx=0x7f4f000091d0).stop 2020-11-20T12:22:11.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0d5cb700 1 -- 172.21.15.116:0/1545771011 reap_dead start 2020-11-20T12:22:11.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/1545771011 shutdown_connections 2020-11-20T12:22:11.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/1545771011 >> 172.21.15.116:0/1545771011 conn(0x7f4f080678a0 msgr2=0x7f4f08067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:11.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/1545771011 shutdown_connections 2020-11-20T12:22:11.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.895+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/1545771011 wait complete. 2020-11-20T12:22:11.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.899+0000 7f4f0ddcc700 1 Processor -- start 2020-11-20T12:22:11.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.899+0000 7f4f0ddcc700 1 -- start start 2020-11-20T12:22:11.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.899+0000 7f4f0ddcc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f080becc0 0x7f4f08162020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.899+0000 7f4f0ddcc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f0815ea30 0x7f4f080bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.899+0000 7f4f0ddcc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0815eeb0 0x7f4f08164440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.899+0000 7f4f0ddcc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08162810 con 0x7f4f080becc0 2020-11-20T12:22:11.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.899+0000 7f4f0ddcc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08162ab0 con 0x7f4f0815eeb0 2020-11-20T12:22:11.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.899+0000 7f4f0ddcc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08162d50 con 0x7f4f0815ea30 2020-11-20T12:22:11.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.899+0000 7f4f0cdca700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f080becc0 0x7f4f08162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0cdca700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f080becc0 0x7f4f08162020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51824/0 (socket says 172.21.15.116:51824) 2020-11-20T12:22:11.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0cdca700 1 -- 172.21.15.116:0/2733242708 learned_addr learned my addr 172.21.15.116:0/2733242708 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:11.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f07fff700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f0815ea30 0x7f4f080bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0d5cb700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0815eeb0 0x7f4f08164440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0cdca700 1 -- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f0815ea30 msgr2=0x7f4f080bf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0cdca700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f0815ea30 0x7f4f080bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.924 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0cdca700 1 -- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0815eeb0 msgr2=0x7f4f08164440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:11.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0cdca700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0815eeb0 0x7f4f08164440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:11.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0cdca700 1 -- 172.21.15.116:0/2733242708 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f08162ff0 con 0x7f4f080becc0 2020-11-20T12:22:11.925 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0d5cb700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0815eeb0 0x7f4f08164440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:11.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0cdca700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f080becc0 0x7f4f08162020 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f4efc0077f0 tx=0x7f4f080acd90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:11.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f04ff9700 1 -- 172.21.15.116:0/2733242708 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc000f20 con 0x7f4f080becc0 2020-11-20T12:22:11.926 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f04ff9700 1 -- 172.21.15.116:0/2733242708 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4efc00fcc0 con 0x7f4f080becc0 2020-11-20T12:22:11.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/2733242708 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f08163290 con 0x7f4f080becc0 2020-11-20T12:22:11.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f04ff9700 1 -- 172.21.15.116:0/2733242708 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4efc012350 con 0x7f4f080becc0 2020-11-20T12:22:11.927 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/2733242708 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f080bf590 con 0x7f4f080becc0 2020-11-20T12:22:11.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/2733242708 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f080bf840 con 0x7f4f080becc0 2020-11-20T12:22:11.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.903+0000 7f4f04ff9700 1 -- 172.21.15.116:0/2733242708 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4efc01b030 con 0x7f4f080becc0 2020-11-20T12:22:11.928 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.907+0000 7f4f04ff9700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ee00557c0 0x7f4ee0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:11.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.907+0000 7f4f04ff9700 1 -- 172.21.15.116:0/2733242708 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4efc06e1c0 con 0x7f4f080becc0 2020-11-20T12:22:11.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.907+0000 7f4f07fff700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ee00557c0 0x7f4ee0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:11.929 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.907+0000 7f4f07fff700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ee00557c0 0x7f4ee0057c80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f4ef4000f20 tx=0x7f4ef4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:11.930 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:11.911+0000 7f4f04ff9700 1 -- 172.21.15.116:0/2733242708 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4f080bf840 con 0x7f4f080becc0 2020-11-20T12:22:12.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.063+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/2733242708 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4f080bf840 con 0x7f4f080becc0 2020-11-20T12:22:12.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f04ff9700 1 -- 172.21.15.116:0/2733242708 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4f080bf840 con 0x7f4f080becc0 2020-11-20T12:22:12.071 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:12.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ee00557c0 msgr2=0x7f4ee0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ee00557c0 0x7f4ee0057c80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f4ef4000f20 tx=0x7f4ef4009040).stop 2020-11-20T12:22:12.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/2733242708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f080becc0 msgr2=0x7f4f08162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f080becc0 0x7f4f08162020 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f4efc0077f0 tx=0x7f4f080acd90).stop 2020-11-20T12:22:12.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/2733242708 shutdown_connections 2020-11-20T12:22:12.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4ee00557c0 0x7f4ee0057c80 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4f0815eeb0 0x7f4f08164440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4f080becc0 0x7f4f08162020 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 --2- 172.21.15.116:0/2733242708 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4f0815ea30 0x7f4f080bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/2733242708 >> 172.21.15.116:0/2733242708 conn(0x7f4f080678a0 msgr2=0x7f4f080a93d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:12.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/2733242708 shutdown_connections 2020-11-20T12:22:12.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.067+0000 7f4f0ddcc700 1 -- 172.21.15.116:0/2733242708 wait complete. 2020-11-20T12:22:12.096 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:22:12.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.271+0000 7fccd12f3700 1 Processor -- start 2020-11-20T12:22:12.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.271+0000 7fccd12f3700 1 -- start start 2020-11-20T12:22:12.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fccd12f3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc155c80 0x7fcccc062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fccd12f3700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fcccc156060 0x7fcccc0c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fccd12f3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcccc0c12c0 0x7fcccc060170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.279 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fccd12f3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcccc0ba580 con 0x7fcccc0c12c0 2020-11-20T12:22:12.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fccd12f3700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcccc0b93c0 con 0x7fcccc156060 2020-11-20T12:22:12.280 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fccd12f3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc0b9660 con 0x7fcccc155c80 2020-11-20T12:22:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fccd0af2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcccc0c12c0 0x7fcccc060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.281 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fccd0af2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fcccc0c12c0 0x7fcccc060170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51836/0 (socket says 172.21.15.116:51836) 2020-11-20T12:22:12.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fcccbfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc155c80 0x7fcccc062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fcccb7fe700 1 --2- 172.21.15.116:0/4174802134 >> v2:172.21.15.173:3300/0 conn(0x7fcccc156060 0x7fcccc0c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fccd0af2700 1 -- 172.21.15.116:0/4174802134 learned_addr learned my addr 172.21.15.116:0/4174802134 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:12.282 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fcccbfff700 1 -- 172.21.15.116:0/4174802134 >> v2:172.21.15.173:3300/0 conn(0x7fcccc156060 msgr2=0x7fcccc0c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:12.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fcccbfff700 1 --2- 172.21.15.116:0/4174802134 >> v2:172.21.15.173:3300/0 conn(0x7fcccc156060 0x7fcccc0c0dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fcccbfff700 1 -- 172.21.15.116:0/4174802134 >> v2:172.21.15.116:3300/0 conn(0x7fcccc0c12c0 msgr2=0x7fcccc060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:12.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fcccbfff700 1 --2- 172.21.15.116:0/4174802134 >> v2:172.21.15.116:3300/0 conn(0x7fcccc0c12c0 0x7fcccc060170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fcccbfff700 1 -- 172.21.15.116:0/4174802134 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc0b9900 con 0x7fcccc155c80 2020-11-20T12:22:12.283 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fccd0af2700 1 --2- 172.21.15.116:0/4174802134 >> v2:172.21.15.116:3300/0 conn(0x7fcccc0c12c0 0x7fcccc060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:12.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fcccbfff700 1 --2- 172.21.15.116:0/4174802134 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc155c80 0x7fcccc062ed0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fcccc0c53f0 tx=0x7fccbc006df0).ready entity=mon.2 client_cookie=bdd3602a09525bb1 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:12.284 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fcccaffd700 1 -- 172.21.15.116:0/4174802134 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc009640 con 0x7fcccc155c80 2020-11-20T12:22:12.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fcccaffd700 1 -- 172.21.15.116:0/4174802134 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccbc0097a0 con 0x7fcccc155c80 2020-11-20T12:22:12.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.275+0000 7fcccaffd700 1 -- 172.21.15.116:0/4174802134 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccbc009b20 con 0x7fcccc155c80 2020-11-20T12:22:12.285 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.279+0000 7fccd12f3700 1 -- 172.21.15.116:0/4174802134 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc155c80 msgr2=0x7fcccc062ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.279+0000 7fccd12f3700 1 --2- 172.21.15.116:0/4174802134 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc155c80 0x7fcccc062ed0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7fcccc0c53f0 tx=0x7fccbc006df0).stop 2020-11-20T12:22:12.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.279+0000 7fccd12f3700 1 -- 172.21.15.116:0/4174802134 shutdown_connections 2020-11-20T12:22:12.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.279+0000 7fccd12f3700 1 --2- 172.21.15.116:0/4174802134 >> v2:172.21.15.116:3300/0 conn(0x7fcccc0c12c0 0x7fcccc060170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.279+0000 7fccd12f3700 1 --2- 172.21.15.116:0/4174802134 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc155c80 0x7fcccc062ed0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.286 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.279+0000 7fccd12f3700 1 --2- 172.21.15.116:0/4174802134 >> v2:172.21.15.173:3300/0 conn(0x7fcccc156060 0x7fcccc0c0dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.279+0000 7fccd12f3700 1 -- 172.21.15.116:0/4174802134 >> 172.21.15.116:0/4174802134 conn(0x7fcccc068350 msgr2=0x7fcccc0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:12.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.279+0000 7fccd12f3700 1 -- 172.21.15.116:0/4174802134 shutdown_connections 2020-11-20T12:22:12.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.279+0000 7fccd12f3700 1 -- 172.21.15.116:0/4174802134 wait complete. 2020-11-20T12:22:12.287 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.279+0000 7fccd12f3700 1 Processor -- start 2020-11-20T12:22:12.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.279+0000 7fccd12f3700 1 -- start start 2020-11-20T12:22:12.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccd12f3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc0c12c0 0x7fcccc1506f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccd12f3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcccc155c80 0x7fcccc0a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccd12f3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcccc156060 0x7fcccc150fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccd12f3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc166570 con 0x7fcccc155c80 2020-11-20T12:22:12.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccd12f3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc165650 con 0x7fcccc156060 2020-11-20T12:22:12.290 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccd12f3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc1658f0 con 0x7fcccc0c12c0 2020-11-20T12:22:12.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fcccb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcccc155c80 0x7fcccc0a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fcccb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcccc155c80 0x7fcccc0a9b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51840/0 (socket says 172.21.15.116:51840) 2020-11-20T12:22:12.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fcccb7fe700 1 -- 172.21.15.116:0/584856491 learned_addr learned my addr 172.21.15.116:0/584856491 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:12.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fcccbfff700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc0c12c0 0x7fcccc1506f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.291 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccd0af2700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcccc156060 0x7fcccc150fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fcccb7fe700 1 -- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc0c12c0 msgr2=0x7fcccc1506f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fcccb7fe700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc0c12c0 0x7fcccc1506f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fcccb7fe700 1 -- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcccc156060 msgr2=0x7fcccc150fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fcccb7fe700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcccc156060 0x7fcccc150fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.292 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fcccb7fe700 1 -- 172.21.15.116:0/584856491 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc165b90 con 0x7fcccc155c80 2020-11-20T12:22:12.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccd0af2700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcccc156060 0x7fcccc150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:12.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fcccbfff700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc0c12c0 0x7fcccc1506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:12.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fcccb7fe700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcccc155c80 0x7fcccc0a9b50 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fccc000ebd0 tx=0x7fccc0006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:12.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccabfff700 1 -- 172.21.15.116:0/584856491 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc000c980 con 0x7fcccc155c80 2020-11-20T12:22:12.293 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccabfff700 1 -- 172.21.15.116:0/584856491 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc000cae0 con 0x7fcccc155c80 2020-11-20T12:22:12.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccd12f3700 1 -- 172.21.15.116:0/584856491 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc165e30 con 0x7fcccc155c80 2020-11-20T12:22:12.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccabfff700 1 -- 172.21.15.116:0/584856491 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc000ce60 con 0x7fcccc155c80 2020-11-20T12:22:12.294 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.283+0000 7fccd12f3700 1 -- 172.21.15.116:0/584856491 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc150c60 con 0x7fcccc155c80 2020-11-20T12:22:12.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.287+0000 7fccd12f3700 1 -- 172.21.15.116:0/584856491 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcccc1662d0 con 0x7fcccc155c80 2020-11-20T12:22:12.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.287+0000 7fccabfff700 1 -- 172.21.15.116:0/584856491 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fccc001a030 con 0x7fcccc155c80 2020-11-20T12:22:12.295 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.287+0000 7fccabfff700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcca40557c0 0x7fcca4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.287+0000 7fccabfff700 1 -- 172.21.15.116:0/584856491 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fccc006e380 con 0x7fcccc155c80 2020-11-20T12:22:12.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.287+0000 7fcccbfff700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcca40557c0 0x7fcca4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.291+0000 7fcccbfff700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcca40557c0 0x7fcca4057c80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fccbc009340 tx=0x7fccbc015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:12.299 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.295+0000 7fccabfff700 1 -- 172.21.15.116:0/584856491 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fcccc1662d0 con 0x7fcccc155c80 2020-11-20T12:22:12.459 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.455+0000 7fccd12f3700 1 -- 172.21.15.116:0/584856491 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcccc1662d0 con 0x7fcccc155c80 2020-11-20T12:22:12.460 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.455+0000 7fccabfff700 1 -- 172.21.15.116:0/584856491 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+10 (secure 0 0 0) 0x7fcccc1662d0 con 0x7fcccc155c80 2020-11-20T12:22:12.460 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 -- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcca40557c0 msgr2=0x7fcca4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcca40557c0 0x7fcca4057c80 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7fccbc009340 tx=0x7fccbc015000).stop 2020-11-20T12:22:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 -- 172.21.15.116:0/584856491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcccc155c80 msgr2=0x7fcccc0a9b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.463 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcccc155c80 0x7fcccc0a9b50 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fccc000ebd0 tx=0x7fccc0006df0).stop 2020-11-20T12:22:12.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 -- 172.21.15.116:0/584856491 shutdown_connections 2020-11-20T12:22:12.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fcca40557c0 0x7fcca4057c80 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fcccc156060 0x7fcccc150fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fcccc0c12c0 0x7fcccc1506f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.464 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 --2- 172.21.15.116:0/584856491 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fcccc155c80 0x7fcccc0a9b50 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 -- 172.21.15.116:0/584856491 >> 172.21.15.116:0/584856491 conn(0x7fcccc068350 msgr2=0x7fcccc0c2750 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:12.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 -- 172.21.15.116:0/584856491 shutdown_connections 2020-11-20T12:22:12.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.459+0000 7fccd12f3700 1 -- 172.21.15.116:0/584856491 wait complete. 2020-11-20T12:22:12.487 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:22:12.487 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:12.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.659+0000 7f35d56d6700 1 Processor -- start 2020-11-20T12:22:12.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.659+0000 7f35d56d6700 1 -- start start 2020-11-20T12:22:12.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.659+0000 7f35d56d6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d00a40a0 0x7f35d0152880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.659+0000 7f35d56d6700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d0152d70 con 0x7f35d0060170 2020-11-20T12:22:12.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.659+0000 7f35d56d6700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35d0060610 con 0x7f35d00a3cc0 2020-11-20T12:22:12.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.659+0000 7f35d56d6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00608b0 con 0x7f35d00a40a0 2020-11-20T12:22:12.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35cf7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f35d0060170 0x7f35d00c2a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33050/0 (socket says 172.21.15.116:33050) 2020-11-20T12:22:12.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35cf7fe700 1 -- 172.21.15.116:0/2631894049 learned_addr learned my addr 172.21.15.116:0/2631894049 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:12.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35d4ed5700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d00a40a0 0x7f35d0152880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553979307 0 0) 0x7f35d0152d70 con 0x7f35d0060170 2020-11-20T12:22:12.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35bc003240 con 0x7f35d0060170 2020-11-20T12:22:12.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636562580 0 0) 0x7f35d0060610 con 0x7f35d00a3cc0 2020-11-20T12:22:12.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d0152d70 con 0x7f35d00a3cc0 2020-11-20T12:22:12.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1704021082 0 0) 0x7f35bc003240 con 0x7f35d0060170 2020-11-20T12:22:12.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d00a40a0 msgr2=0x7f35d0152880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:12.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d00a40a0 0x7f35d0152880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 >> v1:172.21.15.173:6789/0 conn(0x7f35d00a3cc0 legacy=0x7f35d00c0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0060b50 con 0x7f35d0060170 2020-11-20T12:22:12.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f35c40014e0 con 0x7f35d0060170 2020-11-20T12:22:12.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 >> v1:172.21.15.116:6789/0 conn(0x7f35d0060170 legacy=0x7f35d00c2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d00a40a0 0x7f35bc0038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35bc003de0 0x7f35bc0044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35bc0049d0 0x7f35bc004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d00608b0 con 0x7f35bc0049d0 2020-11-20T12:22:12.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc005370 con 0x7f35bc003de0 2020-11-20T12:22:12.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35bc00b780 con 0x7f35d00a40a0 2020-11-20T12:22:12.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35d4ed5700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35bc0049d0 0x7f35bc004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35cf7fe700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35bc003de0 0x7f35bc0044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35d4ed5700 1 -- 172.21.15.116:0/2631894049 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d00a40a0 msgr2=0x7f35bc0038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:12.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35d4ed5700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d00a40a0 0x7f35bc0038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:12.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35d4ed5700 1 -- 172.21.15.116:0/2631894049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35bc003de0 msgr2=0x7f35bc0044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:12.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35d4ed5700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35bc003de0 0x7f35bc0044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35d4ed5700 1 -- 172.21.15.116:0/2631894049 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35bc003240 con 0x7f35bc0049d0 2020-11-20T12:22:12.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35cf7fe700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35bc003de0 0x7f35bc0044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:12.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35d4ed5700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35bc0049d0 0x7f35bc004e50 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f35c8009ed0 tx=0x7f35c80091d0).ready entity=mon.0 client_cookie=c7e34d5398497229 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:12.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.663+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c80095c0 con 0x7f35bc0049d0 2020-11-20T12:22:12.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.667+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c800ee20 con 0x7f35bc0049d0 2020-11-20T12:22:12.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.667+0000 7f35ceffd700 1 -- 172.21.15.116:0/2631894049 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c8012600 con 0x7f35bc0049d0 2020-11-20T12:22:12.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.667+0000 7f35d56d6700 1 -- 172.21.15.116:0/2631894049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35bc0049d0 msgr2=0x7f35bc004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.667+0000 7f35d56d6700 1 --2- 172.21.15.116:0/2631894049 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35bc0049d0 0x7f35bc004e50 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f35c8009ed0 tx=0x7f35c80091d0).stop 2020-11-20T12:22:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.667+0000 7f35d4ed5700 1 -- 172.21.15.116:0/2631894049 reap_dead start 2020-11-20T12:22:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.667+0000 7f35d56d6700 1 -- 172.21.15.116:0/2631894049 shutdown_connections 2020-11-20T12:22:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.667+0000 7f35d56d6700 1 -- 172.21.15.116:0/2631894049 >> 172.21.15.116:0/2631894049 conn(0x7f35d00c1610 msgr2=0x7f35d00c1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:12.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.667+0000 7f35d56d6700 1 -- 172.21.15.116:0/2631894049 shutdown_connections 2020-11-20T12:22:12.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.667+0000 7f35d56d6700 1 -- 172.21.15.116:0/2631894049 wait complete. 2020-11-20T12:22:12.683 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.667+0000 7f35d56d6700 1 Processor -- start 2020-11-20T12:22:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.667+0000 7f35d56d6700 1 -- start start 2020-11-20T12:22:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35d56d6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35d00a6210 0x7f35d0157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.684 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35d56d6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d0161870 0x7f35d0155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35d56d6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d0156280 0x7f35d0164400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35d56d6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d014b510 con 0x7f35d00a6210 2020-11-20T12:22:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35d56d6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d01582e0 con 0x7f35d0161870 2020-11-20T12:22:12.685 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35d56d6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35d0156700 con 0x7f35d0156280 2020-11-20T12:22:12.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35cffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35d00a6210 0x7f35d0157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35cffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35d00a6210 0x7f35d0157720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51860/0 (socket says 172.21.15.116:51860) 2020-11-20T12:22:12.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35cffff700 1 -- 172.21.15.116:0/3852092426 learned_addr learned my addr 172.21.15.116:0/3852092426 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:12.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35cf7fe700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d0161870 0x7f35d0155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35d4ed5700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d0156280 0x7f35d0164400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.687 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35cffff700 1 -- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d0156280 msgr2=0x7f35d0164400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35cffff700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d0156280 0x7f35d0164400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35cffff700 1 -- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d0161870 msgr2=0x7f35d0155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35cffff700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d0161870 0x7f35d0155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.688 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35cffff700 1 -- 172.21.15.116:0/3852092426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35d0157da0 con 0x7f35d00a6210 2020-11-20T12:22:12.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35cffff700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35d00a6210 0x7f35d0157720 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f35c000c7a0 tx=0x7f35c0007b40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:12.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35affff700 1 -- 172.21.15.116:0/3852092426 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c000e560 con 0x7f35d00a6210 2020-11-20T12:22:12.689 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35affff700 1 -- 172.21.15.116:0/3852092426 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35c000e6c0 con 0x7f35d00a6210 2020-11-20T12:22:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35d56d6700 1 -- 172.21.15.116:0/3852092426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d0158040 con 0x7f35d00a6210 2020-11-20T12:22:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35affff700 1 -- 172.21.15.116:0/3852092426 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35c000ea60 con 0x7f35d00a6210 2020-11-20T12:22:12.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.671+0000 7f35d56d6700 1 -- 172.21.15.116:0/3852092426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d01558f0 con 0x7f35d00a6210 2020-11-20T12:22:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.675+0000 7f35d56d6700 1 -- 172.21.15.116:0/3852092426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d005cf30 con 0x7f35d00a6210 2020-11-20T12:22:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.675+0000 7f35affff700 1 -- 172.21.15.116:0/3852092426 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f35c0019030 con 0x7f35d00a6210 2020-11-20T12:22:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.675+0000 7f35affff700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35a80557c0 0x7f35a8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:12.691 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.675+0000 7f35affff700 1 -- 172.21.15.116:0/3852092426 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f35c006dc20 con 0x7f35d00a6210 2020-11-20T12:22:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.675+0000 7f35cf7fe700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35a80557c0 0x7f35a8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:12.692 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.679+0000 7f35cf7fe700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35a80557c0 0x7f35a8057c80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f35d01554d0 tx=0x7f35c4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:12.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.683+0000 7f35affff700 1 -- 172.21.15.116:0/3852092426 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f35c0043ac0 con 0x7f35d00a6210 2020-11-20T12:22:12.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.887+0000 7f35d56d6700 1 -- 172.21.15.116:0/3852092426 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35d0064ab0 con 0x7f35d00a6210 2020-11-20T12:22:12.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.887+0000 7f35affff700 1 -- 172.21.15.116:0/3852092426 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2854 (secure 0 0 0) 0x7f35c00470e0 con 0x7f35d00a6210 2020-11-20T12:22:12.894 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:12.894 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/1612795957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1612795957},{"type":"v1","addr":"172.21.15.116:6835","nonce":1612795957}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/3255680781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3255680781},{"type":"v1","addr":"172.21.15.173:6835","nonce":3255680781}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/4058046293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":4058046293},{"type":"v1","addr":"172.21.15.173:6837","nonce":4058046293}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:56.570302+0000","modified":"2020-11-20T12:21:58.589498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8148":{"gid":8148,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2775705840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2775705840},{"type":"v1","addr":"172.21.15.116:6833","nonce":2775705840}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-20T12:22:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 -- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35a80557c0 msgr2=0x7f35a8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35a80557c0 0x7f35a8057c80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f35d01554d0 tx=0x7f35c4009040).stop 2020-11-20T12:22:12.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 -- 172.21.15.116:0/3852092426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35d00a6210 msgr2=0x7f35d0157720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35d00a6210 0x7f35d0157720 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f35c000c7a0 tx=0x7f35c0007b40).stop 2020-11-20T12:22:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 -- 172.21.15.116:0/3852092426 shutdown_connections 2020-11-20T12:22:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35a80557c0 0x7f35a8057c80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f35d0156280 0x7f35d0164400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f35d00a6210 0x7f35d0157720 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 --2- 172.21.15.116:0/3852092426 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f35d0161870 0x7f35d0155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:12.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 -- 172.21.15.116:0/3852092426 >> 172.21.15.116:0/3852092426 conn(0x7f35d00c1610 msgr2=0x7f35d014ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:12.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 -- 172.21.15.116:0/3852092426 shutdown_connections 2020-11-20T12:22:12.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:12.891+0000 7f35d56d6700 1 -- 172.21.15.116:0/3852092426 wait complete. 2020-11-20T12:22:12.899 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 68 2020-11-20T12:22:13.324 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:21:56.570302+0000', 'modified': '2020-11-20T12:21:58.589498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8148}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8148': {'gid': 8148, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.116:6833/2775705840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 2775705840}, {'type': 'v1', 'addr': '172.21.15.116:6833', 'nonce': 2775705840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:22:13.324 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:22:13.325 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:22:13.517 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":8148,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":16.923710044,"mdsmap_epoch":68,"osdmap_epoch":118,"osdmap_epoch_barrier":118,"uptime":29.943744224} 2020-11-20T12:22:13.518 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 8148, "mdsmap_epoch": 68, "osdmap_epoch": 118, "osdmap_epoch_barrier": 118, "rank_uptime": 16.923710044, "state": "up:active", "uptime": 29.943744224, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:22:13.519 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:22:13.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:13.519 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.2 2020-11-20T12:22:13.537 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:22:13.538 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:13.648 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:13.649 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:22:13.673 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:13.674 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:22:13.681 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:22:13.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:13.682 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2020-11-20T12:22:13.710 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:22:13.710 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:22:13.711 INFO:tasks.cephfs.mount:self.client.name = client.3 2020-11-20T12:22:13.711 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2020-11-20T12:22:13.711 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:22:13.712 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:22:13.712 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:22:13.713 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:22:13.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:13.714 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:22:13.738 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:22:13.738 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:22:13.738 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:22:13.738 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:13.739 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:22:13.739 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:13.739 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:13.739 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:13.739 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:13.740 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:13.740 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:13.740 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:13.740 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:22:13.740 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:13.741 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:22:13.741 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:13.741 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:22:13.741 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:13.741 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:22:13.742 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:13.742 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:22:13.742 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:13.742 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:22:13.742 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:13.743 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:22:13.743 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:13.743 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:22:13.743 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:13.744 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:13.744 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:22:13.744 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:22:13.744 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:13.744 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:13.745 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:22:13.745 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:22:13.745 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:13.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:13.746 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:22:13.836 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:22:13.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:13.840 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:22:13.918 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:22:13.918 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:22:13.918 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:22:13.918 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:22:13.919 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:22:13.919 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:22:13.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:13.921 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:22:13.921 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:22:13.925 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:22:13.926 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:22:13.926 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:22:13.927 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:22:14.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:14.049 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:22:14.071 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:22:14.072 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:22:14.072 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:22:14.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:14.073 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2020-11-20T12:22:14.150 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:14.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.315+0000 7f6ebc993700 1 Processor -- start 2020-11-20T12:22:14.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.315+0000 7f6ebc993700 1 -- start start 2020-11-20T12:22:14.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.315+0000 7f6ebc993700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6eb8062880 0x7f6eb8062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:14.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.315+0000 7f6ebc993700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6eb8153450 0x7f6eb80677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:14.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.315+0000 7f6ebc993700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6eb80c4c10 0x7f6eb80b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:14.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6ebc993700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb81543a0 con 0x7f6eb8153450 2020-11-20T12:22:14.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6ebc993700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eb8153920 con 0x7f6eb80c4c10 2020-11-20T12:22:14.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6ebc993700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb8153bc0 con 0x7f6eb8062880 2020-11-20T12:22:14.324 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eae59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6eb8153450 0x7f6eb80677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:14.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eae59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f6eb8153450 0x7f6eb80677a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51874/0 (socket says 172.21.15.116:51874) 2020-11-20T12:22:14.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eb759e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6eb80c4c10 0x7f6eb80b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:14.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eae59c700 1 -- 172.21.15.116:0/4118575029 learned_addr learned my addr 172.21.15.116:0/4118575029 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:14.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eb6d9d700 1 --2- 172.21.15.116:0/4118575029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6eb8062880 0x7f6eb8062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:14.325 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eae59c700 1 -- 172.21.15.116:0/4118575029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6eb8062880 msgr2=0x7f6eb8062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:14.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eae59c700 1 --2- 172.21.15.116:0/4118575029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6eb8062880 0x7f6eb8062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:14.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eae59c700 1 -- 172.21.15.116:0/4118575029 >> v2:172.21.15.173:3300/0 conn(0x7f6eb80c4c10 msgr2=0x7f6eb80b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:14.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eae59c700 1 --2- 172.21.15.116:0/4118575029 >> v2:172.21.15.173:3300/0 conn(0x7f6eb80c4c10 0x7f6eb80b9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:14.326 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eae59c700 1 -- 172.21.15.116:0/4118575029 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb8153e60 con 0x7f6eb8153450 2020-11-20T12:22:14.327 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eae59c700 1 --2- 172.21.15.116:0/4118575029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eb8153450 0x7f6eb80677a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f6eb80a53a0 tx=0x7f6ea0006eb0).ready entity=mon.0 client_cookie=98d2f45ca008a230 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:14.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eb659c700 1 -- 172.21.15.116:0/4118575029 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0009810 con 0x7f6eb8153450 2020-11-20T12:22:14.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eb659c700 1 -- 172.21.15.116:0/4118575029 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea0009970 con 0x7f6eb8153450 2020-11-20T12:22:14.328 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6eb659c700 1 -- 172.21.15.116:0/4118575029 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea0009cf0 con 0x7f6eb8153450 2020-11-20T12:22:14.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6ebc993700 1 -- 172.21.15.116:0/4118575029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eb8153450 msgr2=0x7f6eb80677a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:14.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6ebc993700 1 --2- 172.21.15.116:0/4118575029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eb8153450 0x7f6eb80677a0 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f6eb80a53a0 tx=0x7f6ea0006eb0).stop 2020-11-20T12:22:14.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6ebc993700 1 -- 172.21.15.116:0/4118575029 shutdown_connections 2020-11-20T12:22:14.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6ebc993700 1 --2- 172.21.15.116:0/4118575029 >> v2:172.21.15.173:3300/0 conn(0x7f6eb80c4c10 0x7f6eb80b9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:14.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6ebc993700 1 --2- 172.21.15.116:0/4118575029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6eb8062880 0x7f6eb8062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:14.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6ebc993700 1 --2- 172.21.15.116:0/4118575029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eb8153450 0x7f6eb80677a0 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:14.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.319+0000 7f6ebc993700 1 -- 172.21.15.116:0/4118575029 >> 172.21.15.116:0/4118575029 conn(0x7f6eb8060c40 msgr2=0x7f6eb80c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:14.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.323+0000 7f6ebc993700 1 -- 172.21.15.116:0/4118575029 shutdown_connections 2020-11-20T12:22:14.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.323+0000 7f6ebc993700 1 -- 172.21.15.116:0/4118575029 wait complete. 2020-11-20T12:22:14.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.323+0000 7f6ebc993700 1 Processor -- start 2020-11-20T12:22:14.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.323+0000 7f6ebc993700 1 -- start start 2020-11-20T12:22:14.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6ebc993700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eb8062880 0x7f6eb81631f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:14.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6ebc993700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6eb80c4c10 0x7f6eb814d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:14.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6ebc993700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eb80a7d00 0x7f6eb8146e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:14.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6ebc993700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80a81a0 con 0x7f6eb8062880 2020-11-20T12:22:14.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6ebc993700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb80a8440 con 0x7f6eb80a7d00 2020-11-20T12:22:14.333 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6ebc993700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb8162810 con 0x7f6eb80c4c10 2020-11-20T12:22:14.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eb6d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eb8062880 0x7f6eb81631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:14.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eb6d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eb8062880 0x7f6eb81631f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51878/0 (socket says 172.21.15.116:51878) 2020-11-20T12:22:14.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eb6d9d700 1 -- 172.21.15.116:0/3135980628 learned_addr learned my addr 172.21.15.116:0/3135980628 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:14.335 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eb759e700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eb80a7d00 0x7f6eb8146e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:14.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eae59c700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6eb80c4c10 0x7f6eb814d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:14.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eb6d9d700 1 -- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6eb80c4c10 msgr2=0x7f6eb814d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:14.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eb6d9d700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6eb80c4c10 0x7f6eb814d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:14.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eb6d9d700 1 -- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eb80a7d00 msgr2=0x7f6eb8146e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:14.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eb6d9d700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eb80a7d00 0x7f6eb8146e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:14.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eb6d9d700 1 -- 172.21.15.116:0/3135980628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb8162ab0 con 0x7f6eb8062880 2020-11-20T12:22:14.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eb6d9d700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eb8062880 0x7f6eb81631f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f6ea800db60 tx=0x7f6ea8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:14.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eaf7fe700 1 -- 172.21.15.116:0/3135980628 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8009790 con 0x7f6eb8062880 2020-11-20T12:22:14.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eaf7fe700 1 -- 172.21.15.116:0/3135980628 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea80098f0 con 0x7f6eb8062880 2020-11-20T12:22:14.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6ebc993700 1 -- 172.21.15.116:0/3135980628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb8162d50 con 0x7f6eb8062880 2020-11-20T12:22:14.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6eaf7fe700 1 -- 172.21.15.116:0/3135980628 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea8009c90 con 0x7f6eb8062880 2020-11-20T12:22:14.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.327+0000 7f6ebc993700 1 -- 172.21.15.116:0/3135980628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb8147300 con 0x7f6eb8062880 2020-11-20T12:22:14.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.331+0000 7f6ebc993700 1 -- 172.21.15.116:0/3135980628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eb805cf30 con 0x7f6eb8062880 2020-11-20T12:22:14.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.331+0000 7f6eaf7fe700 1 -- 172.21.15.116:0/3135980628 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6ea8013480 con 0x7f6eb8062880 2020-11-20T12:22:14.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.331+0000 7f6eaf7fe700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e940557c0 0x7f6e94057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:14.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.331+0000 7f6eaf7fe700 1 -- 172.21.15.116:0/3135980628 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6ea806eda0 con 0x7f6eb8062880 2020-11-20T12:22:14.339 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.335+0000 7f6eae59c700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e940557c0 0x7f6e94057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:14.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.335+0000 7f6eae59c700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e940557c0 0x7f6e94057c80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f6eb80bb3c0 tx=0x7f6ea0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:14.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.335+0000 7f6eaf7fe700 1 -- 172.21.15.116:0/3135980628 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6ea8044be0 con 0x7f6eb8062880 2020-11-20T12:22:14.501 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.495+0000 7f6ebc993700 1 -- 172.21.15.116:0/3135980628 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6eb80c0b80 con 0x7f6eb8062880 2020-11-20T12:22:14.503 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6eaf7fe700 1 -- 172.21.15.116:0/3135980628 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f6ea8048200 con 0x7f6eb8062880 2020-11-20T12:22:14.504 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:14.504 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":118,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:56.569994+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:53.361384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:53.918177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:14.505 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 -- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e940557c0 msgr2=0x7f6e94057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:14.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e940557c0 0x7f6e94057c80 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f6eb80bb3c0 tx=0x7f6ea0015000).stop 2020-11-20T12:22:14.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 -- 172.21.15.116:0/3135980628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eb8062880 msgr2=0x7f6eb81631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:14.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eb8062880 0x7f6eb81631f0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f6ea800db60 tx=0x7f6ea8008040).stop 2020-11-20T12:22:14.506 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 -- 172.21.15.116:0/3135980628 shutdown_connections 2020-11-20T12:22:14.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6e940557c0 0x7f6e94057c80 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:14.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eb80a7d00 0x7f6eb8146e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:14.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eb8062880 0x7f6eb81631f0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:14.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 --2- 172.21.15.116:0/3135980628 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6eb80c4c10 0x7f6eb814d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:14.507 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 -- 172.21.15.116:0/3135980628 >> 172.21.15.116:0/3135980628 conn(0x7f6eb8060c40 msgr2=0x7f6eb80c0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:14.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 -- 172.21.15.116:0/3135980628 shutdown_connections 2020-11-20T12:22:14.508 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.499+0000 7f6ebc993700 1 -- 172.21.15.116:0/3135980628 wait complete. 2020-11-20T12:22:14.530 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:21:53.361384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:21:53.918177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:22:14.531 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:22:14.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.723+0000 7fb109ee7700 1 Processor -- start 2020-11-20T12:22:15.085 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.723+0000 7fb109ee7700 1 -- start start 2020-11-20T12:22:15.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb109ee7700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fb1040aa7d0 0x7fb1040a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb109ee7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040aabb0 0x7fb104151e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.093 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb109ee7700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1040ace40 con 0x7fb1040ad540 2020-11-20T12:22:15.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb109ee7700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1041526c0 con 0x7fb1040aa7d0 2020-11-20T12:22:15.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb109ee7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb104152960 con 0x7fb1040aabb0 2020-11-20T12:22:15.094 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1096e6700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fb1040ad540 0x7fb1040ad9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33090/0 (socket says 172.21.15.116:33090) 2020-11-20T12:22:15.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1096e6700 1 -- 172.21.15.116:0/1490550670 learned_addr learned my addr 172.21.15.116:0/1490550670 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:15.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb108ee5700 1 --2- 172.21.15.116:0/1490550670 >> v2:172.21.15.173:3300/0 conn(0x7fb1040aa7d0 0x7fb1040a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb103fff700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040aabb0 0x7fb104151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.095 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3151763075 0 0) 0x7fb1040ace40 con 0x7fb1040ad540 2020-11-20T12:22:15.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e8003240 con 0x7fb1040ad540 2020-11-20T12:22:15.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1426220838 0 0) 0x7fb0e8003240 con 0x7fb1040ad540 2020-11-20T12:22:15.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040aabb0 msgr2=0x7fb104151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:15.096 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040aabb0 0x7fb104151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 >> v2:172.21.15.173:3300/0 conn(0x7fb1040aa7d0 msgr2=0x7fb1040a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:15.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 --2- 172.21.15.116:0/1490550670 >> v2:172.21.15.173:3300/0 conn(0x7fb1040aa7d0 0x7fb1040a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.097 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104152c00 con 0x7fb1040ad540 2020-11-20T12:22:15.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fb0fc001860 con 0x7fb1040ad540 2020-11-20T12:22:15.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 >> v1:172.21.15.116:6789/0 conn(0x7fb1040ad540 legacy=0x7fb1040ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0e80037b0 0x7fb0e8003bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.098 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0e80044e0 0x7fb0e8004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040aabb0 0x7fb0e8004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1041526c0 con 0x7fb0e80037b0 2020-11-20T12:22:15.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104152960 con 0x7fb0e80044e0 2020-11-20T12:22:15.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e8005370 con 0x7fb1040aabb0 2020-11-20T12:22:15.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb108ee5700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0e80037b0 0x7fb0e8003bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb1096e6700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040aabb0 0x7fb0e8004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.100 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.727+0000 7fb103fff700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0e80044e0 0x7fb0e8004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb108ee5700 1 -- 172.21.15.116:0/1490550670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040aabb0 msgr2=0x7fb0e8004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:15.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb108ee5700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040aabb0 0x7fb0e8004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb108ee5700 1 -- 172.21.15.116:0/1490550670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0e80044e0 msgr2=0x7fb0e8004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:15.101 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb108ee5700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb0e80044e0 0x7fb0e8004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb108ee5700 1 -- 172.21.15.116:0/1490550670 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e8003240 con 0x7fb0e80037b0 2020-11-20T12:22:15.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb108ee5700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0e80037b0 0x7fb0e8003bd0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7fb0f800bc30 tx=0x7fb0f8006df0).ready entity=mon.0 client_cookie=32c21322fbe5287b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:15.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800b310 con 0x7fb0e80037b0 2020-11-20T12:22:15.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f800ea30 con 0x7fb0e80037b0 2020-11-20T12:22:15.102 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb1037fe700 1 -- 172.21.15.116:0/1490550670 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f800edb0 con 0x7fb0e80037b0 2020-11-20T12:22:15.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb109ee7700 1 -- 172.21.15.116:0/1490550670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0e80037b0 msgr2=0x7fb0e8003bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb109ee7700 1 --2- 172.21.15.116:0/1490550670 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb0e80037b0 0x7fb0e8003bd0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7fb0f800bc30 tx=0x7fb0f8006df0).stop 2020-11-20T12:22:15.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb1096e6700 1 -- 172.21.15.116:0/1490550670 reap_dead start 2020-11-20T12:22:15.103 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb109ee7700 1 -- 172.21.15.116:0/1490550670 shutdown_connections 2020-11-20T12:22:15.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb109ee7700 1 -- 172.21.15.116:0/1490550670 >> 172.21.15.116:0/1490550670 conn(0x7fb1040678a0 msgr2=0x7fb104067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:15.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb109ee7700 1 -- 172.21.15.116:0/1490550670 shutdown_connections 2020-11-20T12:22:15.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.731+0000 7fb109ee7700 1 -- 172.21.15.116:0/1490550670 wait complete. 2020-11-20T12:22:15.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.735+0000 7fb109ee7700 1 Processor -- start 2020-11-20T12:22:15.104 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.735+0000 7fb109ee7700 1 -- start start 2020-11-20T12:22:15.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.735+0000 7fb109ee7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040c84e0 0x7fb1040c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.735+0000 7fb109ee7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1040a2ad0 0x7fb1040a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.105 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.735+0000 7fb109ee7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040a5a30 0x7fb1040bda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.735+0000 7fb109ee7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104164570 con 0x7fb1040a5a30 2020-11-20T12:22:15.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.735+0000 7fb109ee7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb104164810 con 0x7fb1040a2ad0 2020-11-20T12:22:15.106 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.735+0000 7fb109ee7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb104164ab0 con 0x7fb1040c84e0 2020-11-20T12:22:15.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb1096e6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040a5a30 0x7fb1040bda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb1096e6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040a5a30 0x7fb1040bda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51900/0 (socket says 172.21.15.116:51900) 2020-11-20T12:22:15.107 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb1096e6700 1 -- 172.21.15.116:0/1449332727 learned_addr learned my addr 172.21.15.116:0/1449332727 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:15.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb103fff700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1040a2ad0 0x7fb1040a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb108ee5700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040c84e0 0x7fb1040c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb1096e6700 1 -- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040c84e0 msgr2=0x7fb1040c88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.108 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb1096e6700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040c84e0 0x7fb1040c88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb1096e6700 1 -- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1040a2ad0 msgr2=0x7fb1040a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb1096e6700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1040a2ad0 0x7fb1040a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb1096e6700 1 -- 172.21.15.116:0/1449332727 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104164d50 con 0x7fb1040a5a30 2020-11-20T12:22:15.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb1096e6700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040a5a30 0x7fb1040bda60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7fb1040ad920 tx=0x7fb1040acd90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:15.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb100ff9700 1 -- 172.21.15.116:0/1449332727 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00fc40 con 0x7fb1040a5a30 2020-11-20T12:22:15.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb100ff9700 1 -- 172.21.15.116:0/1449332727 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc00fda0 con 0x7fb1040a5a30 2020-11-20T12:22:15.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb109ee7700 1 -- 172.21.15.116:0/1449332727 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb104164ff0 con 0x7fb1040a5a30 2020-11-20T12:22:15.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb100ff9700 1 -- 172.21.15.116:0/1449332727 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc012580 con 0x7fb1040a5a30 2020-11-20T12:22:15.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb109ee7700 1 -- 172.21.15.116:0/1449332727 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1040bdf50 con 0x7fb1040a5a30 2020-11-20T12:22:15.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.739+0000 7fb109ee7700 1 -- 172.21.15.116:0/1449332727 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1040a2eb0 con 0x7fb1040a5a30 2020-11-20T12:22:15.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.743+0000 7fb100ff9700 1 -- 172.21.15.116:0/1449332727 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fb0fc01b030 con 0x7fb1040a5a30 2020-11-20T12:22:15.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.743+0000 7fb100ff9700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0dc0557c0 0x7fb0dc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.747+0000 7fb100ff9700 1 -- 172.21.15.116:0/1449332727 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fb0fc06e320 con 0x7fb1040a5a30 2020-11-20T12:22:15.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.747+0000 7fb108ee5700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0dc0557c0 0x7fb0dc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.747+0000 7fb108ee5700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0dc0557c0 0x7fb0dc057c80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fb10405f2f0 tx=0x7fb0f8019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:15.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.747+0000 7fb100ff9700 1 -- 172.21.15.116:0/1449332727 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fb0fc044130 con 0x7fb1040a5a30 2020-11-20T12:22:15.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.907+0000 7fb109ee7700 1 -- 172.21.15.116:0/1449332727 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb104165180 con 0x7fb1040a5a30 2020-11-20T12:22:15.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.907+0000 7fb100ff9700 1 -- 172.21.15.116:0/1449332727 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v118) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb104165180 con 0x7fb1040a5a30 2020-11-20T12:22:15.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 -- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0dc0557c0 msgr2=0x7fb0dc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0dc0557c0 0x7fb0dc057c80 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fb10405f2f0 tx=0x7fb0f8019000).stop 2020-11-20T12:22:15.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 -- 172.21.15.116:0/1449332727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040a5a30 msgr2=0x7fb1040bda60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040a5a30 0x7fb1040bda60 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7fb1040ad920 tx=0x7fb1040acd90).stop 2020-11-20T12:22:15.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 -- 172.21.15.116:0/1449332727 shutdown_connections 2020-11-20T12:22:15.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb0dc0557c0 0x7fb0dc057c80 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fb1040a5a30 0x7fb1040bda60 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fb1040c84e0 0x7fb1040c88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 --2- 172.21.15.116:0/1449332727 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fb1040a2ad0 0x7fb1040a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 -- 172.21.15.116:0/1449332727 >> 172.21.15.116:0/1449332727 conn(0x7fb1040678a0 msgr2=0x7fb104151360 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:15.116 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 -- 172.21.15.116:0/1449332727 shutdown_connections 2020-11-20T12:22:15.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:14.911+0000 7fb109ee7700 1 -- 172.21.15.116:0/1449332727 wait complete. 2020-11-20T12:22:15.117 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:22:15.118 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:22:15.296 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.291+0000 7f91205c6700 1 Processor -- start 2020-11-20T12:22:15.297 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.291+0000 7f91205c6700 1 -- start start 2020-11-20T12:22:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f91205c6700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f91181558c0 0x7f91180a8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f91205c6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9118068500 0x7f9118155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.300 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f91205c6700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91180c49a0 con 0x7f9118156190 2020-11-20T12:22:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f91205c6700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91180c4700 con 0x7f91181558c0 2020-11-20T12:22:15.301 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f91205c6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91180c9140 con 0x7f9118068500 2020-11-20T12:22:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f911e362700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f91181558c0 0x7f91180a8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f911db61700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9118068500 0x7f9118155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.303 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f911db61700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9118068500 0x7f9118155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:60858/0 (socket says 172.21.15.116:60858) 2020-11-20T12:22:15.304 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f911e362700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f91181558c0 0x7f91180a8ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51904/0 (socket says 172.21.15.116:51904) 2020-11-20T12:22:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f911db61700 1 -- 172.21.15.116:0/3565338285 learned_addr learned my addr 172.21.15.116:0/3565338285 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f911db61700 1 -- 172.21.15.116:0/3565338285 >> v2:172.21.15.116:3300/0 conn(0x7f91181558c0 msgr2=0x7f91180a8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:15.305 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f911db61700 1 --2- 172.21.15.116:0/3565338285 >> v2:172.21.15.116:3300/0 conn(0x7f91181558c0 0x7f91180a8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f911db61700 1 -- 172.21.15.116:0/3565338285 >> v1:172.21.15.173:6789/0 conn(0x7f9118156190 legacy=0x7f9118151630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f911db61700 1 -- 172.21.15.116:0/3565338285 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91180c93e0 con 0x7f9118068500 2020-11-20T12:22:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f911db61700 1 --2- 172.21.15.116:0/3565338285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9118068500 0x7f9118155ca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f9108006fd0 tx=0x7f9108006eb0).ready entity=mon.2 client_cookie=c765bb4db24429ec server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:15.306 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.295+0000 7f911d360700 1 -- 172.21.15.116:0/3565338285 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91080097d0 con 0x7f9118068500 2020-11-20T12:22:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f911d360700 1 -- 172.21.15.116:0/3565338285 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9108009930 con 0x7f9118068500 2020-11-20T12:22:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f911d360700 1 -- 172.21.15.116:0/3565338285 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108009cb0 con 0x7f9118068500 2020-11-20T12:22:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f91205c6700 1 -- 172.21.15.116:0/3565338285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9118068500 msgr2=0x7f9118155ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.307 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f91205c6700 1 --2- 172.21.15.116:0/3565338285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9118068500 0x7f9118155ca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f9108006fd0 tx=0x7f9108006eb0).stop 2020-11-20T12:22:15.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f91205c6700 1 -- 172.21.15.116:0/3565338285 shutdown_connections 2020-11-20T12:22:15.311 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f91205c6700 1 --2- 172.21.15.116:0/3565338285 >> v2:172.21.15.116:3300/0 conn(0x7f91181558c0 0x7f91180a8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f91205c6700 1 --2- 172.21.15.116:0/3565338285 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9118068500 0x7f9118155ca0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f91205c6700 1 -- 172.21.15.116:0/3565338285 >> 172.21.15.116:0/3565338285 conn(0x7f9118068080 msgr2=0x7f91180c8920 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f91205c6700 1 -- 172.21.15.116:0/3565338285 shutdown_connections 2020-11-20T12:22:15.312 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f91205c6700 1 -- 172.21.15.116:0/3565338285 wait complete. 2020-11-20T12:22:15.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f91205c6700 1 Processor -- start 2020-11-20T12:22:15.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.299+0000 7f91205c6700 1 -- start start 2020-11-20T12:22:15.313 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f91205c6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9118068500 0x7f91181548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f91205c6700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91181558c0 0x7f91180ad6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f91205c6700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9118156190 0x7f91180adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.314 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f91205c6700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9118154da0 con 0x7f9118068500 2020-11-20T12:22:15.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f91205c6700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91180aba50 con 0x7f9118156190 2020-11-20T12:22:15.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f91205c6700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91180abcf0 con 0x7f91181558c0 2020-11-20T12:22:15.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f911e362700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9118068500 0x7f91181548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.315 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f911e362700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9118068500 0x7f91181548b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51912/0 (socket says 172.21.15.116:51912) 2020-11-20T12:22:15.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f911e362700 1 -- 172.21.15.116:0/558460979 learned_addr learned my addr 172.21.15.116:0/558460979 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:15.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f911db61700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91181558c0 0x7f91180ad6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f911eb63700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9118156190 0x7f91180adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.316 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f911e362700 1 -- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91181558c0 msgr2=0x7f91180ad6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f911e362700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91181558c0 0x7f91180ad6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f911e362700 1 -- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9118156190 msgr2=0x7f91180adbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f911e362700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9118156190 0x7f91180adbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f911e362700 1 -- 172.21.15.116:0/558460979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91180abf90 con 0x7f9118068500 2020-11-20T12:22:15.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f911e362700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9118068500 0x7f91181548b0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f911400eba0 tx=0x7f9114006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f91067fc700 1 -- 172.21.15.116:0/558460979 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911400c9b0 con 0x7f9118068500 2020-11-20T12:22:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f91205c6700 1 -- 172.21.15.116:0/558460979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91180ac230 con 0x7f9118068500 2020-11-20T12:22:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f91067fc700 1 -- 172.21.15.116:0/558460979 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911400cb10 con 0x7f9118068500 2020-11-20T12:22:15.319 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f91067fc700 1 -- 172.21.15.116:0/558460979 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911400ce90 con 0x7f9118068500 2020-11-20T12:22:15.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.303+0000 7f91205c6700 1 -- 172.21.15.116:0/558460979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9118161ab0 con 0x7f9118068500 2020-11-20T12:22:15.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.307+0000 7f91205c6700 1 -- 172.21.15.116:0/558460979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911805d2a0 con 0x7f9118068500 2020-11-20T12:22:15.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.307+0000 7f91067fc700 1 -- 172.21.15.116:0/558460979 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9114012600 con 0x7f9118068500 2020-11-20T12:22:15.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.307+0000 7f91067fc700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90f00557c0 0x7f90f0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.307+0000 7f91067fc700 1 -- 172.21.15.116:0/558460979 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f911406ebf0 con 0x7f9118068500 2020-11-20T12:22:15.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.307+0000 7f911db61700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90f00557c0 0x7f90f0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.311+0000 7f911db61700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90f00557c0 0x7f90f0057c80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f9118154840 tx=0x7f9108015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:15.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.311+0000 7f91067fc700 1 -- 172.21.15.116:0/558460979 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9114044a90 con 0x7f9118068500 2020-11-20T12:22:15.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.471+0000 7f91205c6700 1 -- 172.21.15.116:0/558460979 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f91180c4ee0 con 0x7f9118068500 2020-11-20T12:22:15.477 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.471+0000 7f91067fc700 1 -- 172.21.15.116:0/558460979 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v118) v1 ==== 92+0+10 (secure 0 0 0) 0x7f91140480b0 con 0x7f9118068500 2020-11-20T12:22:15.477 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:15.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 -- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90f00557c0 msgr2=0x7f90f0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.480 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90f00557c0 0x7f90f0057c80 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f9118154840 tx=0x7f9108015000).stop 2020-11-20T12:22:15.481 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 -- 172.21.15.116:0/558460979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9118068500 msgr2=0x7f91181548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.482 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9118068500 0x7f91181548b0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f911400eba0 tx=0x7f9114006df0).stop 2020-11-20T12:22:15.484 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 -- 172.21.15.116:0/558460979 shutdown_connections 2020-11-20T12:22:15.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90f00557c0 0x7f90f0057c80 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.485 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9118156190 0x7f91180adbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9118068500 0x7f91181548b0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 --2- 172.21.15.116:0/558460979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f91181558c0 0x7f91180ad6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.486 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 -- 172.21.15.116:0/558460979 >> 172.21.15.116:0/558460979 conn(0x7f9118068080 msgr2=0x7f9118061260 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:15.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 -- 172.21.15.116:0/558460979 shutdown_connections 2020-11-20T12:22:15.487 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.475+0000 7f91205c6700 1 -- 172.21.15.116:0/558460979 wait complete. 2020-11-20T12:22:15.507 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:22:15.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.703+0000 7f4d9b59e700 1 Processor -- start 2020-11-20T12:22:15.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.703+0000 7f4d9b59e700 1 -- start start 2020-11-20T12:22:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c0aa7d0 0x7f4d9c0a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9b59e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c0ace40 con 0x7f4d9c0aabb0 2020-11-20T12:22:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9b59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d9c1526c0 con 0x7f4d9c0ad540 2020-11-20T12:22:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c152960 con 0x7f4d9c0aa7d0 2020-11-20T12:22:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d99d9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4d9c0aabb0 0x7f4d9c151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33120/0 (socket says 172.21.15.116:33120) 2020-11-20T12:22:15.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d99d9b700 1 -- 172.21.15.116:0/2656783100 learned_addr learned my addr 172.21.15.116:0/2656783100 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:15.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9a59c700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c0aa7d0 0x7f4d9c0a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 907265458 0 0) 0x7f4d9c0ace40 con 0x7f4d9c0aabb0 2020-11-20T12:22:15.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d80003240 con 0x7f4d9c0aabb0 2020-11-20T12:22:15.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2238038704 0 0) 0x7f4d80003240 con 0x7f4d9c0aabb0 2020-11-20T12:22:15.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c0aa7d0 msgr2=0x7f4d9c0a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:15.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c0aa7d0 0x7f4d9c0a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 >> v1:172.21.15.173:6789/0 conn(0x7f4d9c0ad540 legacy=0x7f4d9c0ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c152c00 con 0x7f4d9c0aabb0 2020-11-20T12:22:15.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f4d900014e0 con 0x7f4d9c0aabb0 2020-11-20T12:22:15.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 >> v1:172.21.15.116:6789/0 conn(0x7f4d9c0aabb0 legacy=0x7f4d9c151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d80003760 0x7f4d80003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d800044e0 0x7f4d80004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 0x7f4d8000b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c152960 con 0x7f4d80004e50 2020-11-20T12:22:15.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800052d0 con 0x7f4d800044e0 2020-11-20T12:22:15.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8000bc20 con 0x7f4d80003760 2020-11-20T12:22:15.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9ad9d700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 0x7f4d8000b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9a59c700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d80003760 0x7f4d80003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.718 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d99d9b700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d800044e0 0x7f4d80004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.719 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.707+0000 7f4d9ad9d700 1 -- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d80003760 msgr2=0x7f4d80003b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:15.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d9ad9d700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d80003760 0x7f4d80003b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d9ad9d700 1 -- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d800044e0 msgr2=0x7f4d80004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:15.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d9ad9d700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d800044e0 0x7f4d80004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d9ad9d700 1 -- 172.21.15.116:0/2656783100 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d80003240 con 0x7f4d80004e50 2020-11-20T12:22:15.720 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d99d9b700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d800044e0 0x7f4d80004960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:15.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d9ad9d700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 0x7f4d8000b730 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f4d94001e70 tx=0x7f4d94007f10).ready entity=mon.0 client_cookie=ceb2f3dfd2898343 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:15.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d94001540 con 0x7f4d80004e50 2020-11-20T12:22:15.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d94003030 con 0x7f4d80004e50 2020-11-20T12:22:15.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d9959a700 1 -- 172.21.15.116:0/2656783100 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d9400cd90 con 0x7f4d80004e50 2020-11-20T12:22:15.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2656783100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 msgr2=0x7f4d8000b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d9b59e700 1 --2- 172.21.15.116:0/2656783100 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 0x7f4d8000b730 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f4d94001e70 tx=0x7f4d94007f10).stop 2020-11-20T12:22:15.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.711+0000 7f4d9ad9d700 1 -- 172.21.15.116:0/2656783100 reap_dead start 2020-11-20T12:22:15.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.715+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2656783100 shutdown_connections 2020-11-20T12:22:15.722 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.715+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2656783100 >> 172.21.15.116:0/2656783100 conn(0x7f4d9c0678a0 msgr2=0x7f4d9c067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:15.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.715+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2656783100 shutdown_connections 2020-11-20T12:22:15.723 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.715+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2656783100 wait complete. 2020-11-20T12:22:15.724 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.715+0000 7f4d9b59e700 1 Processor -- start 2020-11-20T12:22:15.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.715+0000 7f4d9b59e700 1 -- start start 2020-11-20T12:22:15.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 0x7f4d9c150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c1616d0 0x7f4d9c0a6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.725 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d9c14b050 0x7f4d9c14b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c09fd60 con 0x7f4d80004e50 2020-11-20T12:22:15.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c14b9a0 con 0x7f4d9c14b050 2020-11-20T12:22:15.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c14e670 con 0x7f4d9c1616d0 2020-11-20T12:22:15.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 0x7f4d9c150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.726 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9a59c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 0x7f4d9c150e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51930/0 (socket says 172.21.15.116:51930) 2020-11-20T12:22:15.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9a59c700 1 -- 172.21.15.116:0/2122963535 learned_addr learned my addr 172.21.15.116:0/2122963535 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:15.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9a59c700 1 -- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c1616d0 msgr2=0x7f4d9c0a6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9a59c700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c1616d0 0x7f4d9c0a6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:15.727 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9a59c700 1 -- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d9c14b050 msgr2=0x7f4d9c14b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9a59c700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d9c14b050 0x7f4d9c14b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:15.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9a59c700 1 -- 172.21.15.116:0/2122963535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d9c09ff50 con 0x7f4d80004e50 2020-11-20T12:22:15.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9a59c700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 0x7f4d9c150e10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f4d8c010950 tx=0x7f4d8c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:15.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d8a7fc700 1 -- 172.21.15.116:0/2122963535 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00b7f0 con 0x7f4d80004e50 2020-11-20T12:22:15.728 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d8a7fc700 1 -- 172.21.15.116:0/2122963535 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c00b950 con 0x7f4d80004e50 2020-11-20T12:22:15.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2122963535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c0a01f0 con 0x7f4d80004e50 2020-11-20T12:22:15.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d8a7fc700 1 -- 172.21.15.116:0/2122963535 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00bcd0 con 0x7f4d80004e50 2020-11-20T12:22:15.729 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2122963535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c0a0490 con 0x7f4d80004e50 2020-11-20T12:22:15.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.719+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2122963535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9c05cf30 con 0x7f4d80004e50 2020-11-20T12:22:15.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.723+0000 7f4d8a7fc700 1 -- 172.21.15.116:0/2122963535 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4d8c024070 con 0x7f4d80004e50 2020-11-20T12:22:15.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.723+0000 7f4d8a7fc700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d700557c0 0x7f4d70057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:15.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.723+0000 7f4d8a7fc700 1 -- 172.21.15.116:0/2122963535 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4d8c06ee90 con 0x7f4d80004e50 2020-11-20T12:22:15.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.723+0000 7f4d99d9b700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d700557c0 0x7f4d70057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:15.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.723+0000 7f4d99d9b700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d700557c0 0x7f4d70057c80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f4d9c0c7f50 tx=0x7f4d90009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:15.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.727+0000 7f4d8a7fc700 1 -- 172.21.15.116:0/2122963535 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4d8c044c20 con 0x7f4d80004e50 2020-11-20T12:22:15.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.887+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2122963535 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4d9c0601b0 con 0x7f4d80004e50 2020-11-20T12:22:15.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.887+0000 7f4d8a7fc700 1 -- 172.21.15.116:0/2122963535 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+10 (secure 0 0 0) 0x7f4d8c018020 con 0x7f4d80004e50 2020-11-20T12:22:15.893 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:15.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d700557c0 msgr2=0x7f4d70057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d700557c0 0x7f4d70057c80 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f4d9c0c7f50 tx=0x7f4d90009040).stop 2020-11-20T12:22:15.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2122963535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 msgr2=0x7f4d9c150e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:15.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 0x7f4d9c150e10 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f4d8c010950 tx=0x7f4d8c008040).stop 2020-11-20T12:22:15.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2122963535 shutdown_connections 2020-11-20T12:22:15.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4d700557c0 0x7f4d70057c80 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4d9c14b050 0x7f4d9c14b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:15.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4d80004e50 0x7f4d9c150e10 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:15.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 --2- 172.21.15.116:0/2122963535 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4d9c1616d0 0x7f4d9c0a6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:15.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2122963535 >> 172.21.15.116:0/2122963535 conn(0x7f4d9c0678a0 msgr2=0x7f4d9c1545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:15.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2122963535 shutdown_connections 2020-11-20T12:22:15.909 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:15.891+0000 7f4d9b59e700 1 -- 172.21.15.116:0/2122963535 wait complete. 2020-11-20T12:22:15.921 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:22:16.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.131+0000 7fe52f0f8700 1 Processor -- start 2020-11-20T12:22:16.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.131+0000 7fe52f0f8700 1 -- start start 2020-11-20T12:22:16.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52f0f8700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe528148cf0 0x7fe5280a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52f0f8700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe5280c4bb0 0x7fe5280a66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52f0f8700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5280a8fa0 con 0x7fe5280c5060 2020-11-20T12:22:16.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52f0f8700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe528152710 con 0x7fe528148cf0 2020-11-20T12:22:16.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52f0f8700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5281529b0 con 0x7fe5280c4bb0 2020-11-20T12:22:16.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52ce94700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe528148cf0 0x7fe5280a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52ce94700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe528148cf0 0x7fe5280a8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51940/0 (socket says 172.21.15.116:51940) 2020-11-20T12:22:16.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52ce94700 1 -- 172.21.15.116:0/2990133013 learned_addr learned my addr 172.21.15.116:0/2990133013 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:16.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe527fff700 1 --2- 172.21.15.116:0/2990133013 >> v2:172.21.15.173:3300/0 conn(0x7fe5280c4bb0 0x7fe5280a66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe5277fe700 1 -- 172.21.15.116:0/2990133013 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 257571415 0 0) 0x7fe520001450 con 0x7fe5280c5060 2020-11-20T12:22:16.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe5277fe700 1 -- 172.21.15.116:0/2990133013 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe50c003240 con 0x7fe5280c5060 2020-11-20T12:22:16.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52ce94700 1 -- 172.21.15.116:0/2990133013 >> v2:172.21.15.173:3300/0 conn(0x7fe5280c4bb0 msgr2=0x7fe5280a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:16.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52ce94700 1 --2- 172.21.15.116:0/2990133013 >> v2:172.21.15.173:3300/0 conn(0x7fe5280c4bb0 0x7fe5280a66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52ce94700 1 -- 172.21.15.116:0/2990133013 >> v1:172.21.15.173:6789/0 conn(0x7fe5280c5060 legacy=0x7fe52814f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52ce94700 1 -- 172.21.15.116:0/2990133013 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe528152c50 con 0x7fe528148cf0 2020-11-20T12:22:16.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe52ce94700 1 --2- 172.21.15.116:0/2990133013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe528148cf0 0x7fe5280a8ab0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7fe5280c0ef0 tx=0x7fe51c006eb0).ready entity=mon.0 client_cookie=f3ca65208a44a919 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:16.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe5277fe700 1 -- 172.21.15.116:0/2990133013 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c009720 con 0x7fe528148cf0 2020-11-20T12:22:16.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe5277fe700 1 -- 172.21.15.116:0/2990133013 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c009880 con 0x7fe528148cf0 2020-11-20T12:22:16.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.135+0000 7fe5277fe700 1 -- 172.21.15.116:0/2990133013 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c009c00 con 0x7fe528148cf0 2020-11-20T12:22:16.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.139+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2990133013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe528148cf0 msgr2=0x7fe5280a8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.139+0000 7fe52f0f8700 1 --2- 172.21.15.116:0/2990133013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe528148cf0 0x7fe5280a8ab0 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7fe5280c0ef0 tx=0x7fe51c006eb0).stop 2020-11-20T12:22:16.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.139+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2990133013 shutdown_connections 2020-11-20T12:22:16.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.139+0000 7fe52f0f8700 1 --2- 172.21.15.116:0/2990133013 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe528148cf0 0x7fe5280a8ab0 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.139+0000 7fe52f0f8700 1 --2- 172.21.15.116:0/2990133013 >> v2:172.21.15.173:3300/0 conn(0x7fe5280c4bb0 0x7fe5280a66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.139+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2990133013 >> 172.21.15.116:0/2990133013 conn(0x7fe52805ba80 msgr2=0x7fe5280c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:16.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.139+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2990133013 shutdown_connections 2020-11-20T12:22:16.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.139+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2990133013 wait complete. 2020-11-20T12:22:16.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.139+0000 7fe52f0f8700 1 Processor -- start 2020-11-20T12:22:16.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.139+0000 7fe52f0f8700 1 -- start start 2020-11-20T12:22:16.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52f0f8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5280c4bb0 0x7fe528151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52f0f8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5280c5060 0x7fe5281393f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52f0f8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe52814e120 0x7fe52814e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52f0f8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe528150b10 con 0x7fe5280c5060 2020-11-20T12:22:16.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52f0f8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe528150db0 con 0x7fe5280c4bb0 2020-11-20T12:22:16.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52f0f8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe528151050 con 0x7fe52814e120 2020-11-20T12:22:16.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe527fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5280c5060 0x7fe5281393f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe527fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5280c5060 0x7fe5281393f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51946/0 (socket says 172.21.15.116:51946) 2020-11-20T12:22:16.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe527fff700 1 -- 172.21.15.116:0/2968277539 learned_addr learned my addr 172.21.15.116:0/2968277539 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:16.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52ce94700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5280c4bb0 0x7fe528151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52d695700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe52814e120 0x7fe52814e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52ce94700 1 -- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe52814e120 msgr2=0x7fe52814e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52ce94700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe52814e120 0x7fe52814e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52ce94700 1 -- 172.21.15.116:0/2968277539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5280c5060 msgr2=0x7fe5281393f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52ce94700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5280c5060 0x7fe5281393f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52ce94700 1 -- 172.21.15.116:0/2968277539 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5281512f0 con 0x7fe5280c4bb0 2020-11-20T12:22:16.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe527fff700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5280c5060 0x7fe5281393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:16.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52ce94700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5280c4bb0 0x7fe528151a30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fe51c012000 tx=0x7fe51c015000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:16.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe524ff9700 1 -- 172.21.15.116:0/2968277539 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c011af0 con 0x7fe5280c4bb0 2020-11-20T12:22:16.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe524ff9700 1 -- 172.21.15.116:0/2968277539 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51c011c50 con 0x7fe5280c4bb0 2020-11-20T12:22:16.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2968277539 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe528151590 con 0x7fe5280c4bb0 2020-11-20T12:22:16.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe524ff9700 1 -- 172.21.15.116:0/2968277539 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51c01e460 con 0x7fe5280c4bb0 2020-11-20T12:22:16.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.143+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2968277539 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5281398e0 con 0x7fe5280c4bb0 2020-11-20T12:22:16.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.147+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2968277539 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe528151720 con 0x7fe5280c4bb0 2020-11-20T12:22:16.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.147+0000 7fe524ff9700 1 -- 172.21.15.116:0/2968277539 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe51c025030 con 0x7fe5280c4bb0 2020-11-20T12:22:16.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.147+0000 7fe524ff9700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe500055810 0x7fe500057cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.147+0000 7fe524ff9700 1 -- 172.21.15.116:0/2968277539 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fe51c06fbf0 con 0x7fe5280c4bb0 2020-11-20T12:22:16.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.151+0000 7fe527fff700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe500055810 0x7fe500057cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.151+0000 7fe527fff700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe500055810 0x7fe500057cd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fe5281448f0 tx=0x7fe514006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:16.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.155+0000 7fe524ff9700 1 -- 172.21.15.116:0/2968277539 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe51c045ab0 con 0x7fe5280c4bb0 2020-11-20T12:22:16.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.315+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2968277539 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe52805d2a0 con 0x7fe5280c4bb0 2020-11-20T12:22:16.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe524ff9700 1 -- 172.21.15.116:0/2968277539 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+10 (secure 0 0 0) 0x7fe51c023070 con 0x7fe5280c4bb0 2020-11-20T12:22:16.329 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe500055810 msgr2=0x7fe500057cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe500055810 0x7fe500057cd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7fe5281448f0 tx=0x7fe514006df0).stop 2020-11-20T12:22:16.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5280c4bb0 msgr2=0x7fe528151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5280c4bb0 0x7fe528151a30 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fe51c012000 tx=0x7fe51c015000).stop 2020-11-20T12:22:16.330 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2968277539 shutdown_connections 2020-11-20T12:22:16.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe500055810 0x7fe500057cd0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe52814e120 0x7fe52814e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe5280c4bb0 0x7fe528151a30 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 --2- 172.21.15.116:0/2968277539 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe5280c5060 0x7fe5281393f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.331 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2968277539 >> 172.21.15.116:0/2968277539 conn(0x7fe52805ba80 msgr2=0x7fe5280a6830 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:16.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2968277539 shutdown_connections 2020-11-20T12:22:16.332 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.319+0000 7fe52f0f8700 1 -- 172.21.15.116:0/2968277539 wait complete. 2020-11-20T12:22:16.332 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:16.350 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:22:16.351 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:16.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.531+0000 7f2ad07d2700 1 Processor -- start 2020-11-20T12:22:16.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.531+0000 7f2ad07d2700 1 -- start start 2020-11-20T12:22:16.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.531+0000 7f2ad07d2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac80bf100 0x7f2ac80bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.531+0000 7f2ad07d2700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2ac80bac00 0x7f2ac814db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.531+0000 7f2ad07d2700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac80b9d20 con 0x7f2ac80bfa80 2020-11-20T12:22:16.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.531+0000 7f2ad07d2700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac80bb0b0 con 0x7f2ac80bac00 2020-11-20T12:22:16.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.531+0000 7f2ad07d2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac80b9f80 con 0x7f2ac80bf100 2020-11-20T12:22:16.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.531+0000 7f2ace56e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f2ac80bfa80 0x7f2ac8066e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33154/0 (socket says 172.21.15.116:33154) 2020-11-20T12:22:16.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.531+0000 7f2ace56e700 1 -- 172.21.15.116:0/1226819591 learned_addr learned my addr 172.21.15.116:0/1226819591 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:16.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2aced6f700 1 --2- 172.21.15.116:0/1226819591 >> v2:172.21.15.173:3300/0 conn(0x7f2ac80bac00 0x7f2ac814db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2acdd6d700 1 --2- 172.21.15.116:0/1226819591 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac80bf100 0x7f2ac80bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2acd56c700 1 -- 172.21.15.116:0/1226819591 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1980132798 0 0) 0x7f2ac80b9d20 con 0x7f2ac80bfa80 2020-11-20T12:22:16.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2acd56c700 1 -- 172.21.15.116:0/1226819591 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ab0003240 con 0x7f2ac80bfa80 2020-11-20T12:22:16.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2aced6f700 1 -- 172.21.15.116:0/1226819591 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac80bf100 msgr2=0x7f2ac80bfe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:16.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2aced6f700 1 --2- 172.21.15.116:0/1226819591 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac80bf100 0x7f2ac80bfe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2aced6f700 1 -- 172.21.15.116:0/1226819591 >> v1:172.21.15.116:6789/0 conn(0x7f2ac80bfa80 legacy=0x7f2ac8066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2aced6f700 1 -- 172.21.15.116:0/1226819591 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac80ba220 con 0x7f2ac80bac00 2020-11-20T12:22:16.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2aced6f700 1 --2- 172.21.15.116:0/1226819591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac80bac00 0x7f2ac814db10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f2ac0009c90 tx=0x7f2ac00091d0).ready entity=mon.1 client_cookie=8ae838f77c497812 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:16.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2acd56c700 1 -- 172.21.15.116:0/1226819591 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000b9c0 con 0x7f2ac80bac00 2020-11-20T12:22:16.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2acd56c700 1 -- 172.21.15.116:0/1226819591 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac000bb20 con 0x7f2ac80bac00 2020-11-20T12:22:16.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2acd56c700 1 -- 172.21.15.116:0/1226819591 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac000bea0 con 0x7f2ac80bac00 2020-11-20T12:22:16.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2ad07d2700 1 -- 172.21.15.116:0/1226819591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac80bac00 msgr2=0x7f2ac814db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.535+0000 7f2ad07d2700 1 --2- 172.21.15.116:0/1226819591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac80bac00 0x7f2ac814db10 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f2ac0009c90 tx=0x7f2ac00091d0).stop 2020-11-20T12:22:16.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 -- 172.21.15.116:0/1226819591 shutdown_connections 2020-11-20T12:22:16.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 --2- 172.21.15.116:0/1226819591 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac80bac00 0x7f2ac814db10 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 --2- 172.21.15.116:0/1226819591 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac80bf100 0x7f2ac80bfe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 -- 172.21.15.116:0/1226819591 >> 172.21.15.116:0/1226819591 conn(0x7f2ac80c4f80 msgr2=0x7f2ac80c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:16.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 -- 172.21.15.116:0/1226819591 shutdown_connections 2020-11-20T12:22:16.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 -- 172.21.15.116:0/1226819591 wait complete. 2020-11-20T12:22:16.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 Processor -- start 2020-11-20T12:22:16.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 -- start start 2020-11-20T12:22:16.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac80bac00 0x7f2ac8160fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac80bf100 0x7f2ac8163d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac80bfa80 0x7f2ac80a5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac8164990 con 0x7f2ac80bfa80 2020-11-20T12:22:16.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac8164c30 con 0x7f2ac80bf100 2020-11-20T12:22:16.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.539+0000 7f2ad07d2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac8164ed0 con 0x7f2ac80bac00 2020-11-20T12:22:16.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2aced6f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac80bfa80 0x7f2ac80a5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2aced6f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac80bfa80 0x7f2ac80a5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51962/0 (socket says 172.21.15.116:51962) 2020-11-20T12:22:16.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2aced6f700 1 -- 172.21.15.116:0/3386857816 learned_addr learned my addr 172.21.15.116:0/3386857816 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:16.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2ace56e700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac80bac00 0x7f2ac8160fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2acdd6d700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac80bf100 0x7f2ac8163d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2aced6f700 1 -- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac80bac00 msgr2=0x7f2ac8160fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2aced6f700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac80bac00 0x7f2ac8160fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2aced6f700 1 -- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac80bf100 msgr2=0x7f2ac8163d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2aced6f700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac80bf100 0x7f2ac8163d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2aced6f700 1 -- 172.21.15.116:0/3386857816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac8165170 con 0x7f2ac80bfa80 2020-11-20T12:22:16.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2aced6f700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac80bfa80 0x7f2ac80a5a90 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f2ac80b9eb0 tx=0x7f2ac0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:16.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2ab67fc700 1 -- 172.21.15.116:0/3386857816 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac001c6c0 con 0x7f2ac80bfa80 2020-11-20T12:22:16.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2ab67fc700 1 -- 172.21.15.116:0/3386857816 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ac001c820 con 0x7f2ac80bfa80 2020-11-20T12:22:16.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2ad07d2700 1 -- 172.21.15.116:0/3386857816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac8165410 con 0x7f2ac80bfa80 2020-11-20T12:22:16.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2ab67fc700 1 -- 172.21.15.116:0/3386857816 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac001cba0 con 0x7f2ac80bfa80 2020-11-20T12:22:16.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2ad07d2700 1 -- 172.21.15.116:0/3386857816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac8164560 con 0x7f2ac80bfa80 2020-11-20T12:22:16.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.543+0000 7f2aa7fff700 1 -- 172.21.15.116:0/3386857816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a9c004f90 con 0x7f2ac80bfa80 2020-11-20T12:22:16.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.547+0000 7f2ab67fc700 1 -- 172.21.15.116:0/3386857816 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2ac0024030 con 0x7f2ac80bfa80 2020-11-20T12:22:16.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.547+0000 7f2ab67fc700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2aa00557c0 0x7f2aa0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:16.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.547+0000 7f2ab67fc700 1 -- 172.21.15.116:0/3386857816 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f2ac0070390 con 0x7f2ac80bfa80 2020-11-20T12:22:16.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.547+0000 7f2ace56e700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2aa00557c0 0x7f2aa0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:16.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.547+0000 7f2ace56e700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2aa00557c0 0x7f2aa0057c80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f2ac40055a0 tx=0x7f2ac4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:16.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.551+0000 7f2ab67fc700 1 -- 172.21.15.116:0/3386857816 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2ac00461d0 con 0x7f2ac80bfa80 2020-11-20T12:22:16.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.759+0000 7f2aa7fff700 1 -- 172.21.15.116:0/3386857816 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a9c001660 con 0x7f2ac80bfa80 2020-11-20T12:22:16.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.759+0000 7f2ab67fc700 1 -- 172.21.15.116:0/3386857816 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2854 (secure 0 0 0) 0x7f2ac0022020 con 0x7f2ac80bfa80 2020-11-20T12:22:16.764 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:16.765 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/1612795957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1612795957},{"type":"v1","addr":"172.21.15.116:6835","nonce":1612795957}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/3255680781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3255680781},{"type":"v1","addr":"172.21.15.173:6835","nonce":3255680781}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/4058046293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":4058046293},{"type":"v1","addr":"172.21.15.173:6837","nonce":4058046293}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:56.570302+0000","modified":"2020-11-20T12:21:58.589498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8148":{"gid":8148,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2775705840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2775705840},{"type":"v1","addr":"172.21.15.116:6833","nonce":2775705840}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-20T12:22:16.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.759+0000 7f2aa7fff700 1 -- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2aa00557c0 msgr2=0x7f2aa0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.759+0000 7f2aa7fff700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2aa00557c0 0x7f2aa0057c80 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f2ac40055a0 tx=0x7f2ac4009040).stop 2020-11-20T12:22:16.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.759+0000 7f2aa7fff700 1 -- 172.21.15.116:0/3386857816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac80bfa80 msgr2=0x7f2ac80a5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:16.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.759+0000 7f2aa7fff700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac80bfa80 0x7f2ac80a5a90 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f2ac80b9eb0 tx=0x7f2ac0016000).stop 2020-11-20T12:22:16.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.763+0000 7f2aa7fff700 1 -- 172.21.15.116:0/3386857816 shutdown_connections 2020-11-20T12:22:16.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.763+0000 7f2aa7fff700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2aa00557c0 0x7f2aa0057c80 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.763+0000 7f2aa7fff700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac80bfa80 0x7f2ac80a5a90 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.763+0000 7f2aa7fff700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac80bac00 0x7f2ac8160fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.763+0000 7f2aa7fff700 1 --2- 172.21.15.116:0/3386857816 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac80bf100 0x7f2ac8163d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:16.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.763+0000 7f2aa7fff700 1 -- 172.21.15.116:0/3386857816 >> 172.21.15.116:0/3386857816 conn(0x7f2ac80c4f80 msgr2=0x7f2ac8063910 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:16.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.763+0000 7f2aa7fff700 1 -- 172.21.15.116:0/3386857816 shutdown_connections 2020-11-20T12:22:16.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:16.763+0000 7f2aa7fff700 1 -- 172.21.15.116:0/3386857816 wait complete. 2020-11-20T12:22:16.769 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 68 2020-11-20T12:22:17.193 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 68, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:21:56.570302+0000', 'modified': '2020-11-20T12:21:58.589498+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8148}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8148': {'gid': 8148, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.116:6833/2775705840', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6832', 'nonce': 2775705840}, {'type': 'v1', 'addr': '172.21.15.116:6833', 'nonce': 2775705840}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:22:17.194 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:22:17.194 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2020-11-20T12:22:17.401 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":8148,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":20.806892138999999,"mdsmap_epoch":68,"osdmap_epoch":118,"osdmap_epoch_barrier":118,"uptime":33.826922261999997} 2020-11-20T12:22:17.402 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 8148, "mdsmap_epoch": 68, "osdmap_epoch": 118, "osdmap_epoch_barrier": 118, "rank_uptime": 20.806892139, "state": "up:active", "uptime": 33.826922262, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:22:17.402 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:22:17.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:17.403 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.3 2020-11-20T12:22:17.422 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:22:17.423 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:17.538 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:17.538 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:22:17.565 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:17.565 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:22:17.572 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:22:17.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:17.573 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2020-11-20T12:22:17.600 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-20T12:22:17.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:22:17.601 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 2020-11-20T12:22:17.684 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:22:17.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:17.685 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T12:22:17.702 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.manila mds 'allow *' osd 'allow rw' mon 'allow *' 2020-11-20T12:22:17.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.883+0000 7f1299664700 1 Processor -- start 2020-11-20T12:22:17.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.883+0000 7f1299664700 1 -- start start 2020-11-20T12:22:17.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.883+0000 7f1299664700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1294148af0 0x7f12940c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.883+0000 7f1299664700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f12940bfc50 0x7f12940c00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.883+0000 7f1299664700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129414ee00 con 0x7f1294152ba0 2020-11-20T12:22:17.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.883+0000 7f1299664700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129414ebe0 con 0x7f12940bfc50 2020-11-20T12:22:17.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.883+0000 7f1299664700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129414f0a0 con 0x7f1294148af0 2020-11-20T12:22:17.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f12937fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f12940bfc50 0x7f12940c00a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f128bfff700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f1294152ba0 0x7f12940ba7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:57632/0 (socket says 172.21.15.116:57632) 2020-11-20T12:22:17.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f128bfff700 1 -- 172.21.15.116:0/2056477874 learned_addr learned my addr 172.21.15.116:0/2056477874 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:17.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f1292ffd700 1 --2- 172.21.15.116:0/2056477874 >> v2:172.21.15.173:3300/0 conn(0x7f1294148af0 0x7f12940c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f12927fc700 1 -- 172.21.15.116:0/2056477874 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2525398689 0 0) 0x7f129414ee00 con 0x7f1294152ba0 2020-11-20T12:22:17.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f12927fc700 1 -- 172.21.15.116:0/2056477874 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1274003240 con 0x7f1294152ba0 2020-11-20T12:22:17.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f12937fe700 1 -- 172.21.15.116:0/2056477874 >> v2:172.21.15.173:3300/0 conn(0x7f1294148af0 msgr2=0x7f12940c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:17.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f12937fe700 1 --2- 172.21.15.116:0/2056477874 >> v2:172.21.15.173:3300/0 conn(0x7f1294148af0 0x7f12940c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f12937fe700 1 -- 172.21.15.116:0/2056477874 >> v1:172.21.15.173:6789/0 conn(0x7f1294152ba0 legacy=0x7f12940ba7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f12937fe700 1 -- 172.21.15.116:0/2056477874 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129414f340 con 0x7f12940bfc50 2020-11-20T12:22:17.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f12937fe700 1 --2- 172.21.15.116:0/2056477874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12940bfc50 0x7f12940c00a0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7f1294065af0 tx=0x7f128c006df0).ready entity=mon.0 client_cookie=aebc952f82dd2ea8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:17.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f12927fc700 1 -- 172.21.15.116:0/2056477874 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c009640 con 0x7f12940bfc50 2020-11-20T12:22:17.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f12927fc700 1 -- 172.21.15.116:0/2056477874 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c0097a0 con 0x7f12940bfc50 2020-11-20T12:22:17.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f12927fc700 1 -- 172.21.15.116:0/2056477874 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c009b20 con 0x7f12940bfc50 2020-11-20T12:22:17.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f1299664700 1 -- 172.21.15.116:0/2056477874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12940bfc50 msgr2=0x7f12940c00a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f1299664700 1 --2- 172.21.15.116:0/2056477874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12940bfc50 0x7f12940c00a0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7f1294065af0 tx=0x7f128c006df0).stop 2020-11-20T12:22:17.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f1299664700 1 -- 172.21.15.116:0/2056477874 shutdown_connections 2020-11-20T12:22:17.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.887+0000 7f1299664700 1 --2- 172.21.15.116:0/2056477874 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f12940bfc50 0x7f12940c00a0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 --2- 172.21.15.116:0/2056477874 >> v2:172.21.15.173:3300/0 conn(0x7f1294148af0 0x7f12940c15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 -- 172.21.15.116:0/2056477874 >> 172.21.15.116:0/2056477874 conn(0x7f12940c4ec0 msgr2=0x7f12940c0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:17.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 -- 172.21.15.116:0/2056477874 shutdown_connections 2020-11-20T12:22:17.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 -- 172.21.15.116:0/2056477874 wait complete. 2020-11-20T12:22:17.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 Processor -- start 2020-11-20T12:22:17.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 -- start start 2020-11-20T12:22:17.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12940bfc50 0x7f12940a4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1294148af0 0x7f1294147e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1294152ba0 0x7f12940a9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12940a7a60 con 0x7f1294152ba0 2020-11-20T12:22:17.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12941549c0 con 0x7f1294148af0 2020-11-20T12:22:17.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.891+0000 7f1299664700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12940a7520 con 0x7f12940bfc50 2020-11-20T12:22:17.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f12937fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1294152ba0 0x7f12940a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f12937fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1294152ba0 0x7f12940a9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51976/0 (socket says 172.21.15.116:51976) 2020-11-20T12:22:17.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f12937fe700 1 -- 172.21.15.116:0/2751978165 learned_addr learned my addr 172.21.15.116:0/2751978165 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:17.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f128bfff700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1294148af0 0x7f1294147e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f1292ffd700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12940bfc50 0x7f12940a4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f12937fe700 1 -- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12940bfc50 msgr2=0x7f12940a4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f12937fe700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12940bfc50 0x7f12940a4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f12937fe700 1 -- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1294148af0 msgr2=0x7f1294147e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:17.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f12937fe700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1294148af0 0x7f1294147e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:17.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f12937fe700 1 -- 172.21.15.116:0/2751978165 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12940a77c0 con 0x7f1294152ba0 2020-11-20T12:22:17.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f12937fe700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1294152ba0 0x7f12940a9160 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f1294061c80 tx=0x7f128c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:17.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f128affd700 1 -- 172.21.15.116:0/2751978165 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c009640 con 0x7f1294152ba0 2020-11-20T12:22:17.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f128affd700 1 -- 172.21.15.116:0/2751978165 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128c01e460 con 0x7f1294152ba0 2020-11-20T12:22:17.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f1299664700 1 -- 172.21.15.116:0/2751978165 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12940a8c90 con 0x7f1294152ba0 2020-11-20T12:22:17.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f128affd700 1 -- 172.21.15.116:0/2751978165 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128c006b20 con 0x7f1294152ba0 2020-11-20T12:22:17.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.895+0000 7f1299664700 1 -- 172.21.15.116:0/2751978165 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1294154640 con 0x7f1294152ba0 2020-11-20T12:22:17.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.899+0000 7f1299664700 1 -- 172.21.15.116:0/2751978165 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129405cf30 con 0x7f1294152ba0 2020-11-20T12:22:17.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.899+0000 7f128affd700 1 -- 172.21.15.116:0/2751978165 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f128c025030 con 0x7f1294152ba0 2020-11-20T12:22:17.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.899+0000 7f128affd700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12700557c0 0x7f1270057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:17.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.899+0000 7f128affd700 1 -- 172.21.15.116:0/2751978165 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f128c06fc40 con 0x7f1294152ba0 2020-11-20T12:22:17.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.899+0000 7f1292ffd700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12700557c0 0x7f1270057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:17.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.903+0000 7f1292ffd700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12700557c0 0x7f1270057c80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f1284009e00 tx=0x7f12840091d0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:17.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:17.907+0000 7f128affd700 1 -- 172.21.15.116:0/2751978165 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f128c045b90 con 0x7f1294152ba0 2020-11-20T12:22:18.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.119+0000 7f1299664700 1 -- 172.21.15.116:0/2751978165 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]} v 0) v1 -- 0x7f1294064390 con 0x7f1294152ba0 2020-11-20T12:22:18.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.127+0000 7f128affd700 1 -- 172.21.15.116:0/2751978165 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]}]=0 v34) v1 ==== 158+0+64 (secure 0 0 0) 0x7f128c023070 con 0x7f1294152ba0 2020-11-20T12:22:18.133 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:22:18.133 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQD6tLdfuyhFBxAAL3B+R6VAZsxb6g9xsF8+wA== 2020-11-20T12:22:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 -- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12700557c0 msgr2=0x7f1270057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12700557c0 0x7f1270057c80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f1284009e00 tx=0x7f12840091d0).stop 2020-11-20T12:22:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 -- 172.21.15.116:0/2751978165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1294152ba0 msgr2=0x7f12940a9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1294152ba0 0x7f12940a9160 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f1294061c80 tx=0x7f128c015000).stop 2020-11-20T12:22:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 -- 172.21.15.116:0/2751978165 shutdown_connections 2020-11-20T12:22:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f12700557c0 0x7f1270057c80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1294152ba0 0x7f12940a9160 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f12940bfc50 0x7f12940a4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 --2- 172.21.15.116:0/2751978165 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1294148af0 0x7f1294147e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 -- 172.21.15.116:0/2751978165 >> 172.21.15.116:0/2751978165 conn(0x7f12940c4ec0 msgr2=0x7f12941519b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 -- 172.21.15.116:0/2751978165 shutdown_connections 2020-11-20T12:22:18.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.131+0000 7f1299664700 1 -- 172.21.15.116:0/2751978165 wait complete. 2020-11-20T12:22:18.159 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:22:18.160 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:22:18.170 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:22:18.186 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:22:18.186 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:22:18.186 INFO:teuthology.orchestra.run.smithi173.stderr:64 bytes copied, 0.000301365 s, 212 kB/s 2020-11-20T12:22:18.190 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:22:18.196 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:22:18.262 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:22:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:22:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:22:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:22:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.263 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:22:18.264 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.264 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:22:18.264 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.264 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:22:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:22:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:22:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:22:18.265 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:22:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:22:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:22:18.266 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:22:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:22:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:22:18.267 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:22:18.268 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:22:18.268 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:22:18.268 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:22:18.268 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:22:18.269 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:22:18.269 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:22:18.269 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:22:18.270 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:22:18.270 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.270 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:22:18.270 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.270 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:22:18.271 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.271 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:22:18.271 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.271 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:22:18.271 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:22:18.272 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:22:18.272 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:22:18.272 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:22:18.272 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:22:18.272 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.273 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:22:18.273 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:22:18.273 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.273 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:22:18.274 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:22:18.274 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.274 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:22:18.274 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:22:18.274 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.275 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:22:18.275 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:22:18.275 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:22:18.275 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.275 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:22:18.276 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:22:18.276 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.276 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:22:18.276 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.276 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:22:18.277 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.277 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:22:18.277 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.277 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:22:18.278 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:22:18.278 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.278 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:22:18.278 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:22:18.278 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:22:18.279 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:22:18.279 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:22:18.279 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:22:18.279 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:22:18.279 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:22:18.280 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:22:18.280 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:22:18.280 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:22:18.280 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:22:18.280 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:22:18.281 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:22:18.281 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:22:18.281 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:22:18.281 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:22:18.281 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:22:18.282 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:22:18.282 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:22:18.282 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.282 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:22:18.282 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:18.283 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:22:18.283 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:22:18.283 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:22:18.283 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:18.283 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:18.284 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:22:18.284 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.284 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:22:18.284 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:18.285 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:22:18.285 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:22:18.285 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:22:18.285 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:22:18.285 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:22:18.286 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:18.286 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:22:18.286 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:22:18.286 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:18.287 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:22:18.287 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:18.287 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:22:18.287 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:22:18.287 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:22:18.288 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:22:18.288 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:22:18.288 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:22:18.288 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:22:18.288 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:22:18.289 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:18.289 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:22:18.289 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:22:18.289 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:22:18.290 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:22:18.290 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:22:18.290 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:22:18.290 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:22:18.290 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:22:18.291 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:22:18.291 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:22:18.291 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:22:18.291 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:22:18.291 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:18.292 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:22:18.292 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:22:18.292 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:22:18.292 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:22:18.292 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:22:18.293 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:22:18.293 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:22:18.293 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:22:18.293 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:22:18.293 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:22:18.294 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:22:18.294 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:22:18.294 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:22:18.294 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:22:18.294 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:22:18.296 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:22:18.296 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:22:18.296 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:22:18.297 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:22:18.297 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.297 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:22:18.297 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.297 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:22:18.298 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.298 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:22:18.298 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:22:18.298 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.298 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:22:18.299 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:22:18.299 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.299 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:22:18.299 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:22:18.300 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.300 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:22:18.300 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:22:18.300 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.300 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:22:18.301 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.301 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:22:18.301 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:22:18.301 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:22:18.301 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:22:18.302 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:22:18.302 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:22:18.302 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:22:18.302 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:22:18.302 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:22:18.303 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:22:18.303 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.303 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:22:18.303 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.303 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:22:18.304 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.304 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:22:18.304 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.304 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:22:18.304 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:22:18.305 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:22:18.305 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:22:18.305 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:22:18.305 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:22:18.305 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.306 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:22:18.306 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:22:18.306 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.306 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:22:18.306 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:22:18.307 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.307 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:22:18.307 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:22:18.307 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.307 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:22:18.308 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:22:18.308 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:22:18.308 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.308 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:22:18.308 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:22:18.309 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.309 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:22:18.309 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.309 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:22:18.310 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.310 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:22:18.310 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.310 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:22:18.310 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:22:18.311 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.311 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:22:18.311 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:22:18.311 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:22:18.311 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:22:18.312 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:22:18.312 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:22:18.312 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:22:18.312 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:22:18.312 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:22:18.313 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:22:18.313 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:22:18.313 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:22:18.313 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:22:18.313 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:22:18.314 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:22:18.314 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:22:18.314 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:22:18.314 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:22:18.314 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:22:18.315 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:22:18.315 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.315 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:22:18.315 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:18.315 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:22:18.316 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:22:18.316 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:22:18.316 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:18.316 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:18.316 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:22:18.317 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.317 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:22:18.317 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:18.317 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:22:18.317 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:22:18.318 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:22:18.318 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:22:18.318 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:22:18.318 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:18.318 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:22:18.319 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:22:18.319 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:18.319 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:22:18.319 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:18.319 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:22:18.320 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:22:18.320 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:22:18.320 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:22:18.320 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:22:18.320 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:22:18.321 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:22:18.321 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:22:18.321 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:18.321 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:22:18.321 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:22:18.322 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:22:18.322 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:22:18.322 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:22:18.322 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:22:18.322 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:22:18.323 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:22:18.323 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:22:18.323 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:22:18.323 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:22:18.323 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:22:18.324 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:18.324 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:22:18.324 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:22:18.324 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:22:18.324 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:22:18.325 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:22:18.325 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:22:18.325 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:22:18.325 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:22:18.325 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:22:18.326 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:22:18.326 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:22:18.326 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:22:18.326 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:22:18.326 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:22:18.327 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:22:18.327 INFO:teuthology.orchestra.run.smithi116:> rados -p cephfs_data put test_vc_ob_2 - 2020-11-20T12:22:18.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.371+0000 7f52aef7ac40 1 Processor -- start 2020-11-20T12:22:18.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.371+0000 7f52aef7ac40 1 -- start start 2020-11-20T12:22:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.371+0000 7f52aef7ac40 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x561db5f5e660 0x561db5fa0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.371+0000 7f52aef7ac40 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f60690 0x561db6033040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.371+0000 7f52aef7ac40 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561db6034550 0x561db5f8acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.371+0000 7f52aef7ac40 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db5edabe0 con 0x561db5f60690 2020-11-20T12:22:18.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.371+0000 7f52aef7ac40 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db5fa6cd0 con 0x561db6034550 2020-11-20T12:22:18.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.371+0000 7f52aef7ac40 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x561db5fa6f70 con 0x561db5f5e660 2020-11-20T12:22:18.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f5297fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f60690 0x561db6033040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f5297fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f60690 0x561db6033040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51988/0 (socket says 172.21.15.116:51988) 2020-11-20T12:22:18.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f5297fff700 1 -- 172.21.15.116:0/1349014610 learned_addr learned my addr 172.21.15.116:0/1349014610 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f52a0580700 1 --2- 172.21.15.116:0/1349014610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x561db5f5e660 0x561db5fa0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f52a0d81700 1 --2- 172.21.15.116:0/1349014610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561db6034550 0x561db5f8acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f5297fff700 1 -- 172.21.15.116:0/1349014610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x561db5f5e660 msgr2=0x561db5fa0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:18.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f5297fff700 1 --2- 172.21.15.116:0/1349014610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x561db5f5e660 0x561db5fa0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f5297fff700 1 -- 172.21.15.116:0/1349014610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561db6034550 msgr2=0x561db5f8acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f5297fff700 1 --2- 172.21.15.116:0/1349014610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561db6034550 0x561db5f8acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f5297fff700 1 -- 172.21.15.116:0/1349014610 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561db5f03480 con 0x561db5f60690 2020-11-20T12:22:18.382 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f5297fff700 1 --2- 172.21.15.116:0/1349014610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f60690 0x561db6033040 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7f5288005b10 tx=0x7f5288015000).ready entity=mon.0 client_cookie=7e668b8103092030 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f529f57e700 1 -- 172.21.15.116:0/1349014610 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288008e40 con 0x561db5f60690 2020-11-20T12:22:18.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f529f57e700 1 -- 172.21.15.116:0/1349014610 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5288012600 con 0x561db5f60690 2020-11-20T12:22:18.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f529f57e700 1 -- 172.21.15.116:0/1349014610 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5288012980 con 0x561db5f60690 2020-11-20T12:22:18.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1349014610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f60690 msgr2=0x561db6033040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f52aef7ac40 1 --2- 172.21.15.116:0/1349014610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f60690 0x561db6033040 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7f5288005b10 tx=0x7f5288015000).stop 2020-11-20T12:22:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1349014610 shutdown_connections 2020-11-20T12:22:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f52aef7ac40 1 --2- 172.21.15.116:0/1349014610 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561db6034550 0x561db5f8acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f52aef7ac40 1 --2- 172.21.15.116:0/1349014610 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x561db5f5e660 0x561db5fa0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f52aef7ac40 1 --2- 172.21.15.116:0/1349014610 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f60690 0x561db6033040 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1349014610 >> 172.21.15.116:0/1349014610 conn(0x561db5fa5920 msgr2=0x561db5f5d490 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:18.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1349014610 shutdown_connections 2020-11-20T12:22:18.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.375+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1349014610 wait complete. 2020-11-20T12:22:18.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f52aef7ac40 1 Processor -- start 2020-11-20T12:22:18.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f52aef7ac40 1 -- start start 2020-11-20T12:22:18.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f52aef7ac40 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f5e660 0x561db6032730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f52aef7ac40 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x561db5f60690 0x561db5f8c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f52aef7ac40 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561db6034550 0x561db5f8c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f52aef7ac40 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db5f8a020 con 0x561db5f5e660 2020-11-20T12:22:18.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f52aef7ac40 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x561db5f8ed20 con 0x561db6034550 2020-11-20T12:22:18.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f52aef7ac40 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x561db5fa16c0 con 0x561db5f60690 2020-11-20T12:22:18.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f52a0580700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f5e660 0x561db6032730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f52a0580700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f5e660 0x561db6032730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:51992/0 (socket says 172.21.15.116:51992) 2020-11-20T12:22:18.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f52a0580700 1 -- 172.21.15.116:0/1028223485 learned_addr learned my addr 172.21.15.116:0/1028223485 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:18.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.379+0000 7f5297fff700 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x561db5f60690 0x561db5f8c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f52a0d81700 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561db6034550 0x561db5f8c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f52a0580700 1 -- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x561db5f60690 msgr2=0x561db5f8c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f52a0580700 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x561db5f60690 0x561db5f8c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f52a0580700 1 -- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561db6034550 msgr2=0x561db5f8c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f52a0580700 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x561db6034550 0x561db5f8c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f52a0580700 1 -- 172.21.15.116:0/1028223485 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561db5f048a0 con 0x561db5f5e660 2020-11-20T12:22:18.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f5297fff700 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x561db5f60690 0x561db5f8c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:18.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f52a0580700 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f5e660 0x561db6032730 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f5290008e20 tx=0x7f529000c040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f529dd7b700 1 -- 172.21.15.116:0/1028223485 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f529000f040 con 0x561db5f5e660 2020-11-20T12:22:18.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f529dd7b700 1 -- 172.21.15.116:0/1028223485 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52900095b0 con 0x561db5f5e660 2020-11-20T12:22:18.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f529dd7b700 1 -- 172.21.15.116:0/1028223485 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5290015470 con 0x561db5f5e660 2020-11-20T12:22:18.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1028223485 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561db5f00280 con 0x561db5f5e660 2020-11-20T12:22:18.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1028223485 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561db5ede620 con 0x561db5f5e660 2020-11-20T12:22:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.383+0000 7f529dd7b700 1 -- 172.21.15.116:0/1028223485 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5290015860 con 0x561db5f5e660 2020-11-20T12:22:18.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.387+0000 7f529dd7b700 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5280054470 0x7f5280056930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.387+0000 7f529dd7b700 1 -- 172.21.15.116:0/1028223485 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f529006e3d0 con 0x561db5f5e660 2020-11-20T12:22:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.387+0000 7f52aef7ac40 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] conn(0x561db5f8b380 0x561db5fa0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.387+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1028223485 --> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] -- osd_op(unknown.0.0:1 22.0 22:06dc0443:::test_vc_ob_2:head [writefull 0~9 in=9b] snapc 0=[] ondisk+write+known_if_redirected e118) v8 -- 0x561db6034db0 con 0x561db5f8b380 2020-11-20T12:22:18.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.387+0000 7f5297fff700 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5280054470 0x7f5280056930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.387+0000 7f52a0d81700 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] conn(0x561db5f8b380 0x561db5fa0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:18.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.387+0000 7f5297fff700 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5280054470 0x7f5280056930 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f5288020000 tx=0x7f528801c000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.387+0000 7f52a0d81700 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] conn(0x561db5f8b380 0x561db5fa0fa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:18.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52a0d81700 1 -- 172.21.15.116:0/1028223485 <== osd.4 v2:172.21.15.173:6802/14206 1 ==== osd_op_reply(1 test_vc_ob_2 [writefull 0~9] v118'1 uv1 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f529800dbe0 con 0x561db5f8b380 2020-11-20T12:22:18.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] conn(0x561db5f8b380 msgr2=0x561db5fa0fa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52aef7ac40 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:6802/14206,v1:172.21.15.173:6803/14206] conn(0x561db5f8b380 0x561db5fa0fa0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:18.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5280054470 msgr2=0x7f5280056930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52aef7ac40 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5280054470 0x7f5280056930 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f5288020000 tx=0x7f528801c000).stop 2020-11-20T12:22:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1028223485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f5e660 msgr2=0x561db6032730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52aef7ac40 1 --2- 172.21.15.116:0/1028223485 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x561db5f5e660 0x561db6032730 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f5290008e20 tx=0x7f529000c040).stop 2020-11-20T12:22:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52a0d81700 1 -- 172.21.15.116:0/1028223485 reap_dead start 2020-11-20T12:22:18.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1028223485 shutdown_connections 2020-11-20T12:22:18.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1028223485 >> 172.21.15.116:0/1028223485 conn(0x561db5fa5920 msgr2=0x561db5f8ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:18.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1028223485 shutdown_connections 2020-11-20T12:22:18.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:18.391+0000 7f52aef7ac40 1 -- 172.21.15.116:0/1028223485 wait complete. 2020-11-20T12:22:18.406 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:22:18.406 INFO:teuthology.orchestra.run.smithi173:> from __future__ import print_function 2020-11-20T12:22:18.407 INFO:teuthology.orchestra.run.smithi173:> from ceph_volume_client import CephFSVolumeClient, VolumePath 2020-11-20T12:22:18.407 INFO:teuthology.orchestra.run.smithi173:> from sys import version_info as sys_version_info 2020-11-20T12:22:18.407 INFO:teuthology.orchestra.run.smithi173:> from rados import OSError as rados_OSError 2020-11-20T12:22:18.408 INFO:teuthology.orchestra.run.smithi173:> import logging 2020-11-20T12:22:18.408 INFO:teuthology.orchestra.run.smithi173:> log = logging.getLogger("ceph_volume_client") 2020-11-20T12:22:18.409 INFO:teuthology.orchestra.run.smithi173:> log.addHandler(logging.StreamHandler()) 2020-11-20T12:22:18.409 INFO:teuthology.orchestra.run.smithi173:> log.setLevel(logging.DEBUG) 2020-11-20T12:22:18.409 INFO:teuthology.orchestra.run.smithi173:> vc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None) 2020-11-20T12:22:18.410 INFO:teuthology.orchestra.run.smithi173:> vc.connect() 2020-11-20T12:22:18.410 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:22:18.410 INFO:teuthology.orchestra.run.smithi173:> data_read = vc.get_object("cephfs_data", "test_vc_ob_2") 2020-11-20T12:22:18.411 INFO:teuthology.orchestra.run.smithi173:> assert data_read == b"test_data" 2020-11-20T12:22:18.411 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:22:18.412 INFO:teuthology.orchestra.run.smithi173:> vc.disconnect() 2020-11-20T12:22:18.412 INFO:teuthology.orchestra.run.smithi173:> ' 2020-11-20T12:22:18.597 INFO:teuthology.orchestra.run.smithi173.stderr:Configuring to RADOS with config /etc/ceph/ceph.conf... 2020-11-20T12:22:18.602 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to RADOS... 2020-11-20T12:22:18.614 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to RADOS complete 2020-11-20T12:22:18.615 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to cephfs... 2020-11-20T12:22:18.615 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS initializing... 2020-11-20T12:22:18.622 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS mounting... 2020-11-20T12:22:18.632 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to cephfs complete 2020-11-20T12:22:18.632 INFO:teuthology.orchestra.run.smithi173.stderr:Recovering from partial auth updates (if any)... 2020-11-20T12:22:18.632 INFO:teuthology.orchestra.run.smithi173.stderr:Nothing to recover. No auth meta files. 2020-11-20T12:22:18.634 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:22:18.635 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs... 2020-11-20T12:22:18.641 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting cephfs complete 2020-11-20T12:22:18.642 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados... 2020-11-20T12:22:18.645 INFO:teuthology.orchestra.run.smithi173.stderr:Disconnecting rados complete 2020-11-20T12:22:18.648 INFO:teuthology.orchestra.run.smithi173.stderr:disconnect 2020-11-20T12:22:18.716 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:22:18.717 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:22:18.760 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:22:18.806 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:22:18.806 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2020-11-20T12:22:18.807 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:22:18.807 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.0 2020-11-20T12:22:18.891 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:22:18.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:18.892 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2020-11-20T12:22:18.916 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2020-11-20T12:22:18.918 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:22:18.919 INFO:teuthology.orchestra.run.smithi116:> sudo umount /home/ubuntu/cephtest/mnt.2 2020-11-20T12:22:19.043 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi116.front.sepia.ceph.com 2020-11-20T12:22:19.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:19.044 INFO:teuthology.orchestra.run.smithi116:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2020-11-20T12:22:19.064 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2020-11-20T12:22:19.065 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:22:19.066 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.3 2020-11-20T12:22:19.152 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:22:19.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:19.153 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2020-11-20T12:22:19.172 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:22:19.178 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:22:19.244 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:22:19.244 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:22:19.245 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:22:19.245 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:22:19.245 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.246 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:22:19.246 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.246 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:22:19.246 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.247 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:22:19.247 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:22:19.247 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.247 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:22:19.248 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:22:19.248 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.248 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:22:19.248 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:22:19.249 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.249 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:22:19.249 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:22:19.249 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.250 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:22:19.250 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.250 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:22:19.250 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:22:19.250 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:22:19.251 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:22:19.251 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:22:19.251 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:22:19.251 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:22:19.252 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:22:19.252 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:22:19.252 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:22:19.252 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.253 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:22:19.253 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.254 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:22:19.254 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.254 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:22:19.254 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.255 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:22:19.255 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:22:19.255 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:22:19.255 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:22:19.256 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:22:19.256 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:22:19.256 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.256 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:22:19.257 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:22:19.257 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.257 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:22:19.258 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:22:19.258 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.258 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:22:19.258 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:22:19.258 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.259 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:22:19.259 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:22:19.259 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:22:19.259 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.260 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:22:19.260 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:22:19.260 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.260 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:22:19.261 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.261 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:22:19.261 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.261 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:22:19.262 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.262 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:22:19.262 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:22:19.263 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.263 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:22:19.263 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:22:19.263 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:22:19.264 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:22:19.264 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:22:19.264 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:22:19.265 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:22:19.265 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:22:19.265 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:22:19.265 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:22:19.266 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:22:19.266 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:22:19.266 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:22:19.266 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:22:19.267 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:22:19.267 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:22:19.267 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:22:19.267 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:22:19.268 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:22:19.268 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:22:19.268 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.268 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:22:19.269 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:19.269 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:22:19.269 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:22:19.269 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:22:19.270 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:19.270 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:19.270 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:22:19.271 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.271 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:22:19.271 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:19.271 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:22:19.272 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:22:19.272 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:22:19.272 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:22:19.272 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:22:19.272 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:19.273 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:22:19.273 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:22:19.273 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:19.273 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:22:19.274 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:19.274 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:22:19.274 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:22:19.274 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:22:19.275 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:22:19.275 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:22:19.275 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:22:19.276 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:22:19.277 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:22:19.277 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:19.277 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:22:19.277 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:22:19.277 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:22:19.278 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:22:19.278 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:22:19.278 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:22:19.278 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:22:19.279 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:22:19.279 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:22:19.279 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:22:19.279 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:22:19.280 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:22:19.280 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:19.280 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:22:19.280 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:22:19.281 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:22:19.281 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:22:19.281 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:22:19.281 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:22:19.282 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:22:19.282 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:22:19.282 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:22:19.283 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:22:19.283 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:22:19.283 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:22:19.284 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:22:19.284 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:22:19.286 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:22:19.286 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:22:19.286 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:22:19.286 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:22:19.287 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.287 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:22:19.287 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.287 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:22:19.288 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.288 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:22:19.288 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:22:19.288 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.289 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:22:19.289 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:22:19.289 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.289 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:22:19.290 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:22:19.290 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.290 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:22:19.290 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:22:19.291 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.291 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:22:19.291 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.291 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:22:19.291 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:22:19.292 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:22:19.292 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:22:19.292 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:22:19.292 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:22:19.292 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:22:19.293 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:22:19.293 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:22:19.293 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:22:19.293 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.293 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:22:19.294 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.294 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:22:19.294 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.294 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:22:19.294 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.295 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:22:19.295 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:22:19.295 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:22:19.295 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:22:19.295 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:22:19.296 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:22:19.296 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.296 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:22:19.296 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:22:19.296 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.297 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:22:19.297 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:22:19.297 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.297 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:22:19.297 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:22:19.298 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.298 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:22:19.298 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:22:19.298 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:22:19.298 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.299 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:22:19.299 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:22:19.299 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.299 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:22:19.299 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.300 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:22:19.300 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.300 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:22:19.300 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.300 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:22:19.301 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:22:19.301 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.301 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:22:19.301 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:22:19.301 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:22:19.302 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:22:19.302 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:22:19.302 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:22:19.302 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:22:19.302 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:22:19.303 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:22:19.303 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:22:19.303 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:22:19.303 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:22:19.303 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:22:19.303 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:22:19.304 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:22:19.304 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:22:19.304 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:22:19.304 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:22:19.305 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:22:19.305 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:22:19.305 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.305 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:22:19.305 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:19.306 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:22:19.306 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:22:19.306 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:22:19.306 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:19.306 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:19.307 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:22:19.307 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.307 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:22:19.307 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:19.307 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:22:19.308 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:22:19.308 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:22:19.308 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:22:19.308 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:22:19.308 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:19.309 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:22:19.309 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:22:19.309 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:19.309 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:22:19.309 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:19.310 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:22:19.310 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:22:19.310 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:22:19.310 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:22:19.310 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:22:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:22:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:22:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:22:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:19.311 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:22:19.312 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:22:19.312 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:22:19.312 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:22:19.312 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:22:19.312 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:22:19.312 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:22:19.313 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:22:19.313 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:22:19.313 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:22:19.313 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:22:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:22:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:22:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:22:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:22:19.314 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:22:19.315 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:22:19.315 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:22:19.315 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:22:19.315 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:22:19.315 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:22:19.316 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:22:19.316 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:22:19.316 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:22:19.316 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:22:19.316 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:22:19.317 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2020-11-20T12:22:19.493 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.487+0000 7f1e4359e700 1 Processor -- start 2020-11-20T12:22:19.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.487+0000 7f1e4359e700 1 -- start start 2020-11-20T12:22:19.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4359e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1e4414e590 0x7f1e44063100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e440a66e0 0x7f1e440a6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4359e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e440635f0 con 0x7f1e440bff10 2020-11-20T12:22:19.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4359e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e440a7040 con 0x7f1e4414e590 2020-11-20T12:22:19.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e440c0690 con 0x7f1e440a66e0 2020-11-20T12:22:19.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1e4414e590 0x7f1e44063100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4259c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1e4414e590 0x7f1e44063100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52002/0 (socket says 172.21.15.116:52002) 2020-11-20T12:22:19.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4259c700 1 -- 172.21.15.116:0/4208925775 learned_addr learned my addr 172.21.15.116:0/4208925775 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:19.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e42d9d700 1 --2- 172.21.15.116:0/4208925775 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e440a66e0 0x7f1e440a6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4259c700 1 -- 172.21.15.116:0/4208925775 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e440a66e0 msgr2=0x7f1e440a6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:19.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4259c700 1 --2- 172.21.15.116:0/4208925775 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e440a66e0 0x7f1e440a6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4259c700 1 -- 172.21.15.116:0/4208925775 >> v1:172.21.15.116:6789/0 conn(0x7f1e440bff10 legacy=0x7f1e4414f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.491+0000 7f1e4259c700 1 -- 172.21.15.116:0/4208925775 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440c0930 con 0x7f1e4414e590 2020-11-20T12:22:19.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4259c700 1 --2- 172.21.15.116:0/4208925775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e4414e590 0x7f1e44063100 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f1e34006fd0 tx=0x7f1e34006eb0).ready entity=mon.0 client_cookie=451c2020de933861 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:19.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4159a700 1 -- 172.21.15.116:0/4208925775 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34009810 con 0x7f1e4414e590 2020-11-20T12:22:19.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4159a700 1 -- 172.21.15.116:0/4208925775 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e34009970 con 0x7f1e4414e590 2020-11-20T12:22:19.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4159a700 1 -- 172.21.15.116:0/4208925775 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e34009cf0 con 0x7f1e4414e590 2020-11-20T12:22:19.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4359e700 1 -- 172.21.15.116:0/4208925775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e4414e590 msgr2=0x7f1e44063100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4359e700 1 --2- 172.21.15.116:0/4208925775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e4414e590 0x7f1e44063100 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f1e34006fd0 tx=0x7f1e34006eb0).stop 2020-11-20T12:22:19.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4359e700 1 -- 172.21.15.116:0/4208925775 shutdown_connections 2020-11-20T12:22:19.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4359e700 1 --2- 172.21.15.116:0/4208925775 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e4414e590 0x7f1e44063100 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4359e700 1 --2- 172.21.15.116:0/4208925775 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e440a66e0 0x7f1e440a6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4359e700 1 -- 172.21.15.116:0/4208925775 >> 172.21.15.116:0/4208925775 conn(0x7f1e44068350 msgr2=0x7f1e440627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:19.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4359e700 1 -- 172.21.15.116:0/4208925775 shutdown_connections 2020-11-20T12:22:19.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.495+0000 7f1e4359e700 1 -- 172.21.15.116:0/4208925775 wait complete. 2020-11-20T12:22:19.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e4359e700 1 Processor -- start 2020-11-20T12:22:19.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e4359e700 1 -- start start 2020-11-20T12:22:19.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e4359e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e440a66e0 0x7f1e44151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e4359e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e440bff10 0x7f1e4414b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e4359e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e4414e590 0x7f1e4414a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e4359e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4414aaf0 con 0x7f1e4414e590 2020-11-20T12:22:19.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e4359e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4414ad90 con 0x7f1e440a66e0 2020-11-20T12:22:19.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e4359e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4414b030 con 0x7f1e440bff10 2020-11-20T12:22:19.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e42d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e4414e590 0x7f1e4414a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e42d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e4414e590 0x7f1e4414a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52012/0 (socket says 172.21.15.116:52012) 2020-11-20T12:22:19.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e42d9d700 1 -- 172.21.15.116:0/2725183275 learned_addr learned my addr 172.21.15.116:0/2725183275 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:19.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e4259c700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e440a66e0 0x7f1e44151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e41d9b700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e440bff10 0x7f1e4414b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e42d9d700 1 -- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e440bff10 msgr2=0x7f1e4414b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e42d9d700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e440bff10 0x7f1e4414b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e42d9d700 1 -- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e440a66e0 msgr2=0x7f1e44151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e42d9d700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e440a66e0 0x7f1e44151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.499+0000 7f1e42d9d700 1 -- 172.21.15.116:0/2725183275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4414b2d0 con 0x7f1e4414e590 2020-11-20T12:22:19.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.503+0000 7f1e42d9d700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e4414e590 0x7f1e4414a300 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f1e3c00bf90 tx=0x7f1e3c00b780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:19.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.503+0000 7f1e327fc700 1 -- 172.21.15.116:0/2725183275 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c010040 con 0x7f1e4414e590 2020-11-20T12:22:19.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.503+0000 7f1e327fc700 1 -- 172.21.15.116:0/2725183275 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3c015480 con 0x7f1e4414e590 2020-11-20T12:22:19.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.503+0000 7f1e4359e700 1 -- 172.21.15.116:0/2725183275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4414a7f0 con 0x7f1e4414e590 2020-11-20T12:22:19.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.503+0000 7f1e327fc700 1 -- 172.21.15.116:0/2725183275 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c015840 con 0x7f1e4414e590 2020-11-20T12:22:19.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.503+0000 7f1e4359e700 1 -- 172.21.15.116:0/2725183275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e44067050 con 0x7f1e4414e590 2020-11-20T12:22:19.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.503+0000 7f1e4359e700 1 -- 172.21.15.116:0/2725183275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4405cf30 con 0x7f1e4414e590 2020-11-20T12:22:19.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.503+0000 7f1e327fc700 1 -- 172.21.15.116:0/2725183275 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1e3c015bb0 con 0x7f1e4414e590 2020-11-20T12:22:19.656 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.507+0000 7f1e327fc700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1e180557c0 0x7f1e18057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.507+0000 7f1e327fc700 1 -- 172.21.15.116:0/2725183275 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1e3c06ef00 con 0x7f1e4414e590 2020-11-20T12:22:19.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.507+0000 7f1e4259c700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1e180557c0 0x7f1e18057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.507+0000 7f1e4259c700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1e180557c0 0x7f1e18057c80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f1e440a5c20 tx=0x7f1e34015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:19.657 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.511+0000 7f1e327fc700 1 -- 172.21.15.116:0/2725183275 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1e3c044c30 con 0x7f1e4414e590 2020-11-20T12:22:19.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.675+0000 7f1e4359e700 1 -- 172.21.15.116:0/2725183275 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1e440639c0 con 0x7f1e4414e590 2020-11-20T12:22:19.690 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.683+0000 7f1e327fc700 1 -- 172.21.15.116:0/2725183275 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1e3c048250 con 0x7f1e4414e590 2020-11-20T12:22:19.693 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 -- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1e180557c0 msgr2=0x7f1e18057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1e180557c0 0x7f1e18057c80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f1e440a5c20 tx=0x7f1e34015000).stop 2020-11-20T12:22:19.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 -- 172.21.15.116:0/2725183275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e4414e590 msgr2=0x7f1e4414a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e4414e590 0x7f1e4414a300 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f1e3c00bf90 tx=0x7f1e3c00b780).stop 2020-11-20T12:22:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 -- 172.21.15.116:0/2725183275 shutdown_connections 2020-11-20T12:22:19.695 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1e180557c0 0x7f1e18057c80 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1e4414e590 0x7f1e4414a300 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1e440a66e0 0x7f1e44151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 --2- 172.21.15.116:0/2725183275 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1e440bff10 0x7f1e4414b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 -- 172.21.15.116:0/2725183275 >> 172.21.15.116:0/2725183275 conn(0x7f1e44068350 msgr2=0x7f1e440c2210 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 -- 172.21.15.116:0/2725183275 shutdown_connections 2020-11-20T12:22:19.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.687+0000 7f1e4359e700 1 -- 172.21.15.116:0/2725183275 wait complete. 2020-11-20T12:22:19.716 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_object' 2020-11-20T12:22:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.891+0000 7f4e751b3700 1 Processor -- start 2020-11-20T12:22:19.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.891+0000 7f4e751b3700 1 -- start start 2020-11-20T12:22:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.891+0000 7f4e751b3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e70151980 0x7f4e700b9420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.891+0000 7f4e751b3700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e70151d60 0x7f4e700bf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.891+0000 7f4e751b3700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e700c23b0 con 0x7f4e700bac10 2020-11-20T12:22:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.891+0000 7f4e751b3700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e700ba850 con 0x7f4e70151d60 2020-11-20T12:22:19.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.891+0000 7f4e751b3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700c2650 con 0x7f4e70151980 2020-11-20T12:22:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.891+0000 7f4e6e59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e70151d60 0x7f4e700bf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.891+0000 7f4e6e59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4e70151d60 0x7f4e700bf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52016/0 (socket says 172.21.15.116:52016) 2020-11-20T12:22:19.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.891+0000 7f4e6e59c700 1 -- 172.21.15.116:0/3874514856 learned_addr learned my addr 172.21.15.116:0/3874514856 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.891+0000 7f4e6ed9d700 1 --2- 172.21.15.116:0/3874514856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e70151980 0x7f4e700b9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e6e59c700 1 -- 172.21.15.116:0/3874514856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e70151980 msgr2=0x7f4e700b9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e6e59c700 1 --2- 172.21.15.116:0/3874514856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e70151980 0x7f4e700b9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e6dd9b700 1 -- 172.21.15.116:0/3874514856 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1431864654 0 0) 0x7f4e700c23b0 con 0x7f4e700bac10 2020-11-20T12:22:19.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e6e59c700 1 -- 172.21.15.116:0/3874514856 >> v1:172.21.15.116:6789/0 conn(0x7f4e700bac10 legacy=0x7f4e70066450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e6e59c700 1 -- 172.21.15.116:0/3874514856 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e700c28f0 con 0x7f4e70151d60 2020-11-20T12:22:19.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e6ed9d700 1 --2- 172.21.15.116:0/3874514856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e70151980 0x7f4e700b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:19.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e6e59c700 1 --2- 172.21.15.116:0/3874514856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e70151d60 0x7f4e700bf070 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f4e5c006fd0 tx=0x7f4e5c006eb0).ready entity=mon.0 client_cookie=5836bffe84141c26 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:19.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e6dd9b700 1 -- 172.21.15.116:0/3874514856 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c009810 con 0x7f4e70151d60 2020-11-20T12:22:19.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e6dd9b700 1 -- 172.21.15.116:0/3874514856 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5c009970 con 0x7f4e70151d60 2020-11-20T12:22:19.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e6dd9b700 1 -- 172.21.15.116:0/3874514856 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5c009cf0 con 0x7f4e70151d60 2020-11-20T12:22:19.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e751b3700 1 -- 172.21.15.116:0/3874514856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e70151d60 msgr2=0x7f4e700bf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e751b3700 1 --2- 172.21.15.116:0/3874514856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e70151d60 0x7f4e700bf070 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f4e5c006fd0 tx=0x7f4e5c006eb0).stop 2020-11-20T12:22:19.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e751b3700 1 -- 172.21.15.116:0/3874514856 shutdown_connections 2020-11-20T12:22:19.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e751b3700 1 --2- 172.21.15.116:0/3874514856 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e70151980 0x7f4e700b9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e751b3700 1 --2- 172.21.15.116:0/3874514856 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e70151d60 0x7f4e700bf070 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.904 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e751b3700 1 -- 172.21.15.116:0/3874514856 >> 172.21.15.116:0/3874514856 conn(0x7f4e700c4f80 msgr2=0x7f4e700c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:19.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e751b3700 1 -- 172.21.15.116:0/3874514856 shutdown_connections 2020-11-20T12:22:19.905 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.895+0000 7f4e751b3700 1 -- 172.21.15.116:0/3874514856 wait complete. 2020-11-20T12:22:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.899+0000 7f4e751b3700 1 Processor -- start 2020-11-20T12:22:19.906 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.899+0000 7f4e751b3700 1 -- start start 2020-11-20T12:22:19.907 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e751b3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e700bac10 0x7f4e70165800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e751b3700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e70151980 0x7f4e700a7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e751b3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e700a8030 0x7f4e700bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e751b3700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e700a7470 con 0x7f4e700a8030 2020-11-20T12:22:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e751b3700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e70166060 con 0x7f4e70151980 2020-11-20T12:22:19.908 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e751b3700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e70166300 con 0x7f4e700bac10 2020-11-20T12:22:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6f59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e700a8030 0x7f4e700bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6f59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e700a8030 0x7f4e700bd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52026/0 (socket says 172.21.15.116:52026) 2020-11-20T12:22:19.910 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6f59e700 1 -- 172.21.15.116:0/2351283028 learned_addr learned my addr 172.21.15.116:0/2351283028 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6e59c700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e70151980 0x7f4e700a7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6ed9d700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e700bac10 0x7f4e70165800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6f59e700 1 -- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e700bac10 msgr2=0x7f4e70165800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.911 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6f59e700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e700bac10 0x7f4e70165800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6f59e700 1 -- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e70151980 msgr2=0x7f4e700a7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:19.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6f59e700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e70151980 0x7f4e700a7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:19.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6f59e700 1 -- 172.21.15.116:0/2351283028 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e701665a0 con 0x7f4e700a8030 2020-11-20T12:22:19.912 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6e59c700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e70151980 0x7f4e700a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e6f59e700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e700a8030 0x7f4e700bd6c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f4e6800cab0 tx=0x7f4e68007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e66ffd700 1 -- 172.21.15.116:0/2351283028 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800a8c0 con 0x7f4e700a8030 2020-11-20T12:22:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e66ffd700 1 -- 172.21.15.116:0/2351283028 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6800aa20 con 0x7f4e700a8030 2020-11-20T12:22:19.913 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e751b3700 1 -- 172.21.15.116:0/2351283028 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e700a76a0 con 0x7f4e700a8030 2020-11-20T12:22:19.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e66ffd700 1 -- 172.21.15.116:0/2351283028 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6800adc0 con 0x7f4e700a8030 2020-11-20T12:22:19.914 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.903+0000 7f4e751b3700 1 -- 172.21.15.116:0/2351283028 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e700bdbb0 con 0x7f4e700a8030 2020-11-20T12:22:19.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.907+0000 7f4e751b3700 1 -- 172.21.15.116:0/2351283028 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e700a7830 con 0x7f4e700a8030 2020-11-20T12:22:19.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.907+0000 7f4e66ffd700 1 -- 172.21.15.116:0/2351283028 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4e68019030 con 0x7f4e700a8030 2020-11-20T12:22:19.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.907+0000 7f4e66ffd700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e440557c0 0x7f4e44057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:19.915 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.907+0000 7f4e66ffd700 1 -- 172.21.15.116:0/2351283028 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4e6806e4c0 con 0x7f4e700a8030 2020-11-20T12:22:19.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.907+0000 7f4e6ed9d700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e440557c0 0x7f4e44057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:19.916 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.911+0000 7f4e6ed9d700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e440557c0 0x7f4e44057c80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f4e60006fd0 tx=0x7f4e60008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:19.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:19.911+0000 7f4e66ffd700 1 -- 172.21.15.116:0/2351283028 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4e68044250 con 0x7f4e700a8030 2020-11-20T12:22:20.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.127+0000 7f4e751b3700 1 -- 172.21.15.116:0/2351283028 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_object"]} v 0) v1 -- 0x7f4e7005d2a0 con 0x7f4e700a8030 2020-11-20T12:22:20.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.391+0000 7f4e66ffd700 1 -- 172.21.15.116:0/2351283028 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volume_client.TestVolumeClient.test_get_object"]}]=0 v363) v1 ==== 143+0+0 (secure 0 0 0) 0x7f4e68047870 con 0x7f4e700a8030 2020-11-20T12:22:20.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 -- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e440557c0 msgr2=0x7f4e44057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:20.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e440557c0 0x7f4e44057c80 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f4e60006fd0 tx=0x7f4e60008040).stop 2020-11-20T12:22:20.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 -- 172.21.15.116:0/2351283028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e700a8030 msgr2=0x7f4e700bd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:20.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e700a8030 0x7f4e700bd6c0 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f4e6800cab0 tx=0x7f4e68007d90).stop 2020-11-20T12:22:20.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 -- 172.21.15.116:0/2351283028 shutdown_connections 2020-11-20T12:22:20.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4e440557c0 0x7f4e44057c80 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:20.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4e700a8030 0x7f4e700bd6c0 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:20.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4e700bac10 0x7f4e70165800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:20.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 --2- 172.21.15.116:0/2351283028 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4e70151980 0x7f4e700a7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:20.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 -- 172.21.15.116:0/2351283028 >> 172.21.15.116:0/2351283028 conn(0x7f4e700c4f80 msgr2=0x7f4e700bed70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:20.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 -- 172.21.15.116:0/2351283028 shutdown_connections 2020-11-20T12:22:20.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.395+0000 7f4e751b3700 1 -- 172.21.15.116:0/2351283028 wait complete. 2020-11-20T12:22:20.425 INFO:tasks.cephfs_test_runner:test_get_object (tasks.cephfs.test_volume_client.TestVolumeClient) ... ok 2020-11-20T12:22:20.426 INFO:tasks.cephfs_test_runner:Starting test: test_idempotency (tasks.cephfs.test_volume_client.TestVolumeClient) 2020-11-20T12:22:20.427 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_idempotency' 2020-11-20T12:22:20.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.599+0000 7fc9a8326700 1 Processor -- start 2020-11-20T12:22:20.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.599+0000 7fc9a8326700 1 -- start start 2020-11-20T12:22:20.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a8326700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc9a00ba890 0x7fc9a00bac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:20.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a8326700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc9a0148a20 0x7fc9a0148e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:20.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a8326700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a0066140 con 0x7fc9a00c2230 2020-11-20T12:22:20.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a8326700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a0149380 con 0x7fc9a00ba890 2020-11-20T12:22:20.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a8326700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a00c1d80 con 0x7fc9a0148a20 2020-11-20T12:22:20.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a60c2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc9a00ba890 0x7fc9a00bac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:20.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a58c1700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc9a00c2230 0x7fc9a00c1670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33234/0 (socket says 172.21.15.116:33234) 2020-11-20T12:22:20.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a58c1700 1 -- 172.21.15.116:0/3994649258 learned_addr learned my addr 172.21.15.116:0/3994649258 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:20.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a68c3700 1 --2- 172.21.15.116:0/3994649258 >> v2:172.21.15.173:3300/0 conn(0x7fc9a0148a20 0x7fc9a0148e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:20.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a50c0700 1 -- 172.21.15.116:0/3994649258 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2831719171 0 0) 0x7fc9a0066140 con 0x7fc9a00c2230 2020-11-20T12:22:20.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a50c0700 1 -- 172.21.15.116:0/3994649258 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc988003240 con 0x7fc9a00c2230 2020-11-20T12:22:20.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a60c2700 1 -- 172.21.15.116:0/3994649258 >> v2:172.21.15.173:3300/0 conn(0x7fc9a0148a20 msgr2=0x7fc9a0148e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:20.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a60c2700 1 --2- 172.21.15.116:0/3994649258 >> v2:172.21.15.173:3300/0 conn(0x7fc9a0148a20 0x7fc9a0148e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:20.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a60c2700 1 -- 172.21.15.116:0/3994649258 >> v1:172.21.15.116:6789/0 conn(0x7fc9a00c2230 legacy=0x7fc9a00c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:20.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a60c2700 1 -- 172.21.15.116:0/3994649258 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a01526c0 con 0x7fc9a00ba890 2020-11-20T12:22:20.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a68c3700 1 --2- 172.21.15.116:0/3994649258 >> v2:172.21.15.173:3300/0 conn(0x7fc9a0148a20 0x7fc9a0148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:20.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a60c2700 1 --2- 172.21.15.116:0/3994649258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9a00ba890 0x7fc9a00bac70 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7fc9a00ba400 tx=0x7fc99c006df0).ready entity=mon.0 client_cookie=a4ff8f2378214133 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:20.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.603+0000 7fc9a50c0700 1 -- 172.21.15.116:0/3994649258 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c0097b0 con 0x7fc9a00ba890 2020-11-20T12:22:20.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a50c0700 1 -- 172.21.15.116:0/3994649258 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c009910 con 0x7fc9a00ba890 2020-11-20T12:22:20.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a50c0700 1 -- 172.21.15.116:0/3994649258 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c009c90 con 0x7fc9a00ba890 2020-11-20T12:22:20.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a8326700 1 -- 172.21.15.116:0/3994649258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9a00ba890 msgr2=0x7fc9a00bac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:20.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a8326700 1 --2- 172.21.15.116:0/3994649258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9a00ba890 0x7fc9a00bac70 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7fc9a00ba400 tx=0x7fc99c006df0).stop 2020-11-20T12:22:20.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a8326700 1 -- 172.21.15.116:0/3994649258 shutdown_connections 2020-11-20T12:22:20.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a8326700 1 --2- 172.21.15.116:0/3994649258 >> v2:172.21.15.173:3300/0 conn(0x7fc9a0148a20 0x7fc9a0148e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:20.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a8326700 1 --2- 172.21.15.116:0/3994649258 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9a00ba890 0x7fc9a00bac70 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:20.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a8326700 1 -- 172.21.15.116:0/3994649258 >> 172.21.15.116:0/3994649258 conn(0x7fc9a00c4f80 msgr2=0x7fc9a00c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:20.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a8326700 1 -- 172.21.15.116:0/3994649258 shutdown_connections 2020-11-20T12:22:20.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a8326700 1 -- 172.21.15.116:0/3994649258 wait complete. 2020-11-20T12:22:20.622 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a8326700 1 Processor -- start 2020-11-20T12:22:20.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.607+0000 7fc9a8326700 1 -- start start 2020-11-20T12:22:20.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a8326700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9a00ba890 0x7fc9a0150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:20.623 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a8326700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a00c2230 0x7fc9a0062830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:20.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a8326700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9a0148a20 0x7fc9a0062d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:20.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a8326700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a01418a0 con 0x7fc9a0148a20 2020-11-20T12:22:20.624 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a8326700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a015ce30 con 0x7fc9a00c2230 2020-11-20T12:22:20.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a8326700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a015d060 con 0x7fc9a00ba890 2020-11-20T12:22:20.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a68c3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9a0148a20 0x7fc9a0062d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:20.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a68c3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9a0148a20 0x7fc9a0062d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52040/0 (socket says 172.21.15.116:52040) 2020-11-20T12:22:20.625 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a68c3700 1 -- 172.21.15.116:0/476794565 learned_addr learned my addr 172.21.15.116:0/476794565 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:20.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a58c1700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a00c2230 0x7fc9a0062830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:20.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a60c2700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9a00ba890 0x7fc9a0150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:20.626 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a68c3700 1 -- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9a00ba890 msgr2=0x7fc9a0150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:20.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a68c3700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9a00ba890 0x7fc9a0150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:20.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a68c3700 1 -- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a00c2230 msgr2=0x7fc9a0062830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:20.627 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a68c3700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a00c2230 0x7fc9a0062830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:20.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a68c3700 1 -- 172.21.15.116:0/476794565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a015d300 con 0x7fc9a0148a20 2020-11-20T12:22:20.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a58c1700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a00c2230 0x7fc9a0062830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:20.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a60c2700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9a00ba890 0x7fc9a0150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:20.628 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a68c3700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9a0148a20 0x7fc9a0062d20 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7fc99800db60 tx=0x7fc998008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:20.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc98e7fc700 1 -- 172.21.15.116:0/476794565 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99800b9a0 con 0x7fc9a0148a20 2020-11-20T12:22:20.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc98e7fc700 1 -- 172.21.15.116:0/476794565 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99800bb00 con 0x7fc9a0148a20 2020-11-20T12:22:20.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a8326700 1 -- 172.21.15.116:0/476794565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a015d5a0 con 0x7fc9a0148a20 2020-11-20T12:22:20.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc98e7fc700 1 -- 172.21.15.116:0/476794565 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99800bea0 con 0x7fc9a0148a20 2020-11-20T12:22:20.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.611+0000 7fc9a8326700 1 -- 172.21.15.116:0/476794565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a01611c0 con 0x7fc9a0148a20 2020-11-20T12:22:20.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.615+0000 7fc9a8326700 1 -- 172.21.15.116:0/476794565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a015d730 con 0x7fc9a0148a20 2020-11-20T12:22:20.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.615+0000 7fc98e7fc700 1 -- 172.21.15.116:0/476794565 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc998012630 con 0x7fc9a0148a20 2020-11-20T12:22:20.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.615+0000 7fc98e7fc700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9780557c0 0x7fc978057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:20.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.615+0000 7fc98e7fc700 1 -- 172.21.15.116:0/476794565 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc99806ebb0 con 0x7fc9a0148a20 2020-11-20T12:22:20.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.615+0000 7fc9a60c2700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9780557c0 0x7fc978057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:20.632 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.619+0000 7fc9a60c2700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9780557c0 0x7fc978057c80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fc99c006e30 tx=0x7fc99c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:20.633 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.623+0000 7fc98e7fc700 1 -- 172.21.15.116:0/476794565 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc998044940 con 0x7fc9a0148a20 2020-11-20T12:22:20.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:20.839+0000 7fc9a8326700 1 -- 172.21.15.116:0/476794565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_idempotency"]} v 0) v1 -- 0x7fc9a005d2a0 con 0x7fc9a0148a20 2020-11-20T12:22:21.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.403+0000 7fc98e7fc700 1 -- 172.21.15.116:0/476794565 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volume_client.TestVolumeClient.test_idempotency"]}]=0 v364) v1 ==== 147+0+0 (secure 0 0 0) 0x7fc998047f60 con 0x7fc9a0148a20 2020-11-20T12:22:21.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 -- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9780557c0 msgr2=0x7fc978057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:21.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9780557c0 0x7fc978057c80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7fc99c006e30 tx=0x7fc99c015000).stop 2020-11-20T12:22:21.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 -- 172.21.15.116:0/476794565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9a0148a20 msgr2=0x7fc9a0062d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:21.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9a0148a20 0x7fc9a0062d20 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7fc99800db60 tx=0x7fc998008040).stop 2020-11-20T12:22:21.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 -- 172.21.15.116:0/476794565 shutdown_connections 2020-11-20T12:22:21.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc9780557c0 0x7fc978057c80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc9a0148a20 0x7fc9a0062d20 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc9a00ba890 0x7fc9a0150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 --2- 172.21.15.116:0/476794565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc9a00c2230 0x7fc9a0062830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 -- 172.21.15.116:0/476794565 >> 172.21.15.116:0/476794565 conn(0x7fc9a00c4f80 msgr2=0x7fc9a0141b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:21.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 -- 172.21.15.116:0/476794565 shutdown_connections 2020-11-20T12:22:21.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.407+0000 7fc9a8326700 1 -- 172.21.15.116:0/476794565 wait complete. 2020-11-20T12:22:21.436 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2020-11-20T12:22:21.630 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.623+0000 7f267ce3d700 1 Processor -- start 2020-11-20T12:22:21.631 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.623+0000 7f267ce3d700 1 -- start start 2020-11-20T12:22:21.634 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f267ce3d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f26780aa1a0 0x7f26780c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f267ce3d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f26780aa580 0x7f26780c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f267ce3d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2678061460 0x7f26780618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:21.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f267ce3d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2678061dc0 con 0x7f26780aa580 2020-11-20T12:22:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f267ce3d700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26780c32d0 con 0x7f26780aa1a0 2020-11-20T12:22:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f267ce3d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2678151cf0 con 0x7f2678061460 2020-11-20T12:22:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f2676ffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f26780aa580 0x7f26780c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f2676ffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f26780aa580 0x7f26780c2de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52046/0 (socket says 172.21.15.116:52046) 2020-11-20T12:22:21.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f2677fff700 1 --2- 172.21.15.116:0/151645129 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2678061460 0x7f26780618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f2676ffd700 1 -- 172.21.15.116:0/151645129 learned_addr learned my addr 172.21.15.116:0/151645129 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f26777fe700 1 --2- 172.21.15.116:0/151645129 >> v2:172.21.15.173:3300/0 conn(0x7f26780aa1a0 0x7f26780c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f2676ffd700 1 -- 172.21.15.116:0/151645129 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2678061460 msgr2=0x7f26780618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f2676ffd700 1 --2- 172.21.15.116:0/151645129 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2678061460 0x7f26780618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f2676ffd700 1 -- 172.21.15.116:0/151645129 >> v2:172.21.15.173:3300/0 conn(0x7f26780aa1a0 msgr2=0x7f26780c37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f2676ffd700 1 --2- 172.21.15.116:0/151645129 >> v2:172.21.15.173:3300/0 conn(0x7f26780aa1a0 0x7f26780c37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f2676ffd700 1 -- 172.21.15.116:0/151645129 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780c34f0 con 0x7f26780aa580 2020-11-20T12:22:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f2676ffd700 1 --2- 172.21.15.116:0/151645129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26780aa580 0x7f26780c2de0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f2678044d90 tx=0x7f266c006eb0).ready entity=mon.0 client_cookie=32163dfadc26123b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:21.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.627+0000 7f26767fc700 1 -- 172.21.15.116:0/151645129 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c0096e0 con 0x7f26780aa580 2020-11-20T12:22:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.631+0000 7f26767fc700 1 -- 172.21.15.116:0/151645129 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c009840 con 0x7f26780aa580 2020-11-20T12:22:21.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.631+0000 7f26767fc700 1 -- 172.21.15.116:0/151645129 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c009bc0 con 0x7f26780aa580 2020-11-20T12:22:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.631+0000 7f267ce3d700 1 -- 172.21.15.116:0/151645129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26780aa580 msgr2=0x7f26780c2de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:21.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.631+0000 7f267ce3d700 1 --2- 172.21.15.116:0/151645129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26780aa580 0x7f26780c2de0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f2678044d90 tx=0x7f266c006eb0).stop 2020-11-20T12:22:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.631+0000 7f267ce3d700 1 -- 172.21.15.116:0/151645129 shutdown_connections 2020-11-20T12:22:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.631+0000 7f267ce3d700 1 --2- 172.21.15.116:0/151645129 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2678061460 0x7f26780618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.631+0000 7f267ce3d700 1 --2- 172.21.15.116:0/151645129 >> v2:172.21.15.173:3300/0 conn(0x7f26780aa1a0 0x7f26780c37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.631+0000 7f267ce3d700 1 --2- 172.21.15.116:0/151645129 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26780aa580 0x7f26780c2de0 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.631+0000 7f267ce3d700 1 -- 172.21.15.116:0/151645129 >> 172.21.15.116:0/151645129 conn(0x7f2678068370 msgr2=0x7f26780c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.631+0000 7f267ce3d700 1 -- 172.21.15.116:0/151645129 shutdown_connections 2020-11-20T12:22:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.631+0000 7f267ce3d700 1 -- 172.21.15.116:0/151645129 wait complete. 2020-11-20T12:22:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f267ce3d700 1 Processor -- start 2020-11-20T12:22:21.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f267ce3d700 1 -- start start 2020-11-20T12:22:21.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f267ce3d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2678061460 0x7f26780c2a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:21.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f267ce3d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26780aa1a0 0x7f2678155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:21.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f267ce3d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26780aa580 0x7f26780c1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:21.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f267ce3d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26780c2f40 con 0x7f26780aa1a0 2020-11-20T12:22:21.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f267ce3d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678161230 con 0x7f2678061460 2020-11-20T12:22:21.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f267ce3d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26780c0380 con 0x7f26780aa580 2020-11-20T12:22:21.645 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f2676ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26780aa1a0 0x7f2678155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:21.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f2676ffd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26780aa1a0 0x7f2678155cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52052/0 (socket says 172.21.15.116:52052) 2020-11-20T12:22:21.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f2676ffd700 1 -- 172.21.15.116:0/3821073000 learned_addr learned my addr 172.21.15.116:0/3821073000 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:21.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f26777fe700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2678061460 0x7f26780c2a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:21.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.635+0000 7f2677fff700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26780aa580 0x7f26780c1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:21.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f2676ffd700 1 -- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26780aa580 msgr2=0x7f26780c1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:21.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f2676ffd700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26780aa580 0x7f26780c1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f2676ffd700 1 -- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2678061460 msgr2=0x7f26780c2a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:21.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f2676ffd700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2678061460 0x7f26780c2a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f2676ffd700 1 -- 172.21.15.116:0/3821073000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26780c0620 con 0x7f26780aa1a0 2020-11-20T12:22:21.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f26777fe700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2678061460 0x7f26780c2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:21.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f2676ffd700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26780aa1a0 0x7f2678155cb0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f26780447c0 tx=0x7f266c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:21.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f26577fe700 1 -- 172.21.15.116:0/3821073000 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c011da0 con 0x7f26780aa1a0 2020-11-20T12:22:21.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f26577fe700 1 -- 172.21.15.116:0/3821073000 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f266c01d450 con 0x7f26780aa1a0 2020-11-20T12:22:21.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f267ce3d700 1 -- 172.21.15.116:0/3821073000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26781561a0 con 0x7f26780aa1a0 2020-11-20T12:22:21.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f26577fe700 1 -- 172.21.15.116:0/3821073000 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f266c01d7f0 con 0x7f26780aa1a0 2020-11-20T12:22:21.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f267ce3d700 1 -- 172.21.15.116:0/3821073000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26780c21a0 con 0x7f26780aa1a0 2020-11-20T12:22:21.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f267ce3d700 1 -- 172.21.15.116:0/3821073000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267805cf30 con 0x7f26780aa1a0 2020-11-20T12:22:21.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.639+0000 7f26577fe700 1 -- 172.21.15.116:0/3821073000 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f266c00f070 con 0x7f26780aa1a0 2020-11-20T12:22:21.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.643+0000 7f26577fe700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26500557c0 0x7f2650057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:21.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.643+0000 7f26577fe700 1 -- 172.21.15.116:0/3821073000 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f266c0703f0 con 0x7f26780aa1a0 2020-11-20T12:22:21.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.643+0000 7f26777fe700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26500557c0 0x7f2650057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:21.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.643+0000 7f26777fe700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26500557c0 0x7f2650057c80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f2668006fd0 tx=0x7f2668008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:21.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.647+0000 7f26577fe700 1 -- 172.21.15.116:0/3821073000 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f266c046250 con 0x7f26780aa1a0 2020-11-20T12:22:21.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.803+0000 7f267ce3d700 1 -- 172.21.15.116:0/3821073000 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"} v 0) v1 -- 0x7f26780c23c0 con 0x7f26780aa1a0 2020-11-20T12:22:21.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.811+0000 7f26577fe700 1 -- 172.21.15.116:0/3821073000 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "config set", "who": "mon", "name": "mon_allow_pool_delete", "value": "True"}]=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f266c049830 con 0x7f26780aa1a0 2020-11-20T12:22:21.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 -- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26500557c0 msgr2=0x7f2650057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:21.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26500557c0 0x7f2650057c80 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f2668006fd0 tx=0x7f2668008040).stop 2020-11-20T12:22:21.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 -- 172.21.15.116:0/3821073000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26780aa1a0 msgr2=0x7f2678155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:21.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26780aa1a0 0x7f2678155cb0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f26780447c0 tx=0x7f266c015000).stop 2020-11-20T12:22:21.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 -- 172.21.15.116:0/3821073000 shutdown_connections 2020-11-20T12:22:21.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f26500557c0 0x7f2650057c80 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f26780aa580 0x7f26780c1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2678061460 0x7f26780c2a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 --2- 172.21.15.116:0/3821073000 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f26780aa1a0 0x7f2678155cb0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:21.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 -- 172.21.15.116:0/3821073000 >> 172.21.15.116:0/3821073000 conn(0x7f2678068370 msgr2=0x7f26780a9850 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:21.823 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 -- 172.21.15.116:0/3821073000 shutdown_connections 2020-11-20T12:22:21.824 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:21.815+0000 7f267ce3d700 1 -- 172.21.15.116:0/3821073000 wait complete. 2020-11-20T12:22:21.855 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2020-11-20T12:22:21.855 INFO:teuthology.orchestra.run.smithi116:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:22:21.900 INFO:teuthology.orchestra.run.smithi173:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2020-11-20T12:22:21.946 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2020-11-20T12:22:21.946 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2020-11-20T12:22:21.947 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:22.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.123+0000 7fd2a08c0700 1 Processor -- start 2020-11-20T12:22:22.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.123+0000 7fd2a08c0700 1 -- start start 2020-11-20T12:22:22.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.127+0000 7fd2a08c0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd29c13afa0 0x7fd29c0aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.127+0000 7fd2a08c0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29c0ada00 con 0x7fd29c0ad440 2020-11-20T12:22:22.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.127+0000 7fd2a08c0700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd29c0adca0 con 0x7fd29c13b380 2020-11-20T12:22:22.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.127+0000 7fd2a08c0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd29c0adf40 con 0x7fd29c13afa0 2020-11-20T12:22:22.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.127+0000 7fd29b59e700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fd29c0ad440 0x7fd29c0a8790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33264/0 (socket says 172.21.15.116:33264) 2020-11-20T12:22:22.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.131+0000 7fd29b59e700 1 -- 172.21.15.116:0/178107258 learned_addr learned my addr 172.21.15.116:0/178107258 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:22.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.127+0000 7fd29ad9d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd29c13afa0 0x7fd29c0aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.131+0000 7fd299d9b700 1 -- 172.21.15.116:0/178107258 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007654437 0 0) 0x7fd29c0ada00 con 0x7fd29c0ad440 2020-11-20T12:22:22.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.131+0000 7fd299d9b700 1 -- 172.21.15.116:0/178107258 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd288003240 con 0x7fd29c0ad440 2020-11-20T12:22:22.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.131+0000 7fd29ad9d700 1 -- 172.21.15.116:0/178107258 >> v1:172.21.15.173:6789/0 conn(0x7fd29c13b380 legacy=0x7fd29c0acd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.131+0000 7fd29ad9d700 1 -- 172.21.15.116:0/178107258 >> v1:172.21.15.116:6789/0 conn(0x7fd29c0ad440 legacy=0x7fd29c0a8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.131+0000 7fd299d9b700 1 -- 172.21.15.116:0/178107258 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 172552247 0 0) 0x7fd288003240 con 0x7fd29c0ad440 2020-11-20T12:22:22.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.131+0000 7fd29ad9d700 1 -- 172.21.15.116:0/178107258 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c0ae1e0 con 0x7fd29c13afa0 2020-11-20T12:22:22.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.131+0000 7fd29ad9d700 1 --2- 172.21.15.116:0/178107258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd29c13afa0 0x7fd29c0aa5b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fd28c00b9a0 tx=0x7fd28c006df0).ready entity=mon.1 client_cookie=f3e8b6e396df1496 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.131+0000 7fd299d9b700 1 -- 172.21.15.116:0/178107258 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294001860 con 0x7fd29c13afa0 2020-11-20T12:22:22.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.131+0000 7fd299d9b700 1 -- 172.21.15.116:0/178107258 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd28c0094a0 con 0x7fd29c13afa0 2020-11-20T12:22:22.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.131+0000 7fd299d9b700 1 -- 172.21.15.116:0/178107258 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd28c009820 con 0x7fd29c13afa0 2020-11-20T12:22:22.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 -- 172.21.15.116:0/178107258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd29c13afa0 msgr2=0x7fd29c0aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 --2- 172.21.15.116:0/178107258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd29c13afa0 0x7fd29c0aa5b0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fd28c00b9a0 tx=0x7fd28c006df0).stop 2020-11-20T12:22:22.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 -- 172.21.15.116:0/178107258 shutdown_connections 2020-11-20T12:22:22.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 --2- 172.21.15.116:0/178107258 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd29c13afa0 0x7fd29c0aa5b0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 -- 172.21.15.116:0/178107258 >> 172.21.15.116:0/178107258 conn(0x7fd29c0678a0 msgr2=0x7fd29c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:22.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 -- 172.21.15.116:0/178107258 shutdown_connections 2020-11-20T12:22:22.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 -- 172.21.15.116:0/178107258 wait complete. 2020-11-20T12:22:22.146 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 Processor -- start 2020-11-20T12:22:22.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 -- start start 2020-11-20T12:22:22.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd29c0ad440 0x7fd29c154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd29c13afa0 0x7fd29c063630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd29c13b380 0x7fd29c139fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c065450 con 0x7fd29c13b380 2020-11-20T12:22:22.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c152e40 con 0x7fd29c13afa0 2020-11-20T12:22:22.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.135+0000 7fd2a08c0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c1530e0 con 0x7fd29c0ad440 2020-11-20T12:22:22.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd29b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd29c13b380 0x7fd29c139fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd29ad9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd29c0ad440 0x7fd29c154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd29b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd29c13b380 0x7fd29c139fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52068/0 (socket says 172.21.15.116:52068) 2020-11-20T12:22:22.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd29b59e700 1 -- 172.21.15.116:0/1192182563 learned_addr learned my addr 172.21.15.116:0/1192182563 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:22.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd29a59c700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd29c13afa0 0x7fd29c063630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.149 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd29b59e700 1 -- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd29c0ad440 msgr2=0x7fd29c154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd29b59e700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd29c0ad440 0x7fd29c154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd29b59e700 1 -- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd29c13afa0 msgr2=0x7fd29c063630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd29b59e700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd29c13afa0 0x7fd29c063630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd29b59e700 1 -- 172.21.15.116:0/1192182563 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c153380 con 0x7fd29c13b380 2020-11-20T12:22:22.150 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd29b59e700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd29c13b380 0x7fd29c139fe0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7fd29400ce60 tx=0x7fd294001dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd282ffd700 1 -- 172.21.15.116:0/1192182563 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294004a90 con 0x7fd29c13b380 2020-11-20T12:22:22.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd2a08c0700 1 -- 172.21.15.116:0/1192182563 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29c15aca0 con 0x7fd29c13b380 2020-11-20T12:22:22.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd282ffd700 1 -- 172.21.15.116:0/1192182563 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd294004bf0 con 0x7fd29c13b380 2020-11-20T12:22:22.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd282ffd700 1 -- 172.21.15.116:0/1192182563 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd294007170 con 0x7fd29c13b380 2020-11-20T12:22:22.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.139+0000 7fd2a08c0700 1 -- 172.21.15.116:0/1192182563 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c1550b0 con 0x7fd29c13b380 2020-11-20T12:22:22.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.143+0000 7fd2a08c0700 1 -- 172.21.15.116:0/1192182563 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29c05cf30 con 0x7fd29c13b380 2020-11-20T12:22:22.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.143+0000 7fd282ffd700 1 -- 172.21.15.116:0/1192182563 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd294010070 con 0x7fd29c13b380 2020-11-20T12:22:22.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.143+0000 7fd282ffd700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2780557c0 0x7fd278057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.143+0000 7fd282ffd700 1 -- 172.21.15.116:0/1192182563 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fd29406e470 con 0x7fd29c13b380 2020-11-20T12:22:22.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.143+0000 7fd29ad9d700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2780557c0 0x7fd278057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.147+0000 7fd29ad9d700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2780557c0 0x7fd278057c80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fd28c006e30 tx=0x7fd28c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.151+0000 7fd282ffd700 1 -- 172.21.15.116:0/1192182563 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd294044310 con 0x7fd29c13b380 2020-11-20T12:22:22.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.363+0000 7fd2a08c0700 1 -- 172.21.15.116:0/1192182563 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd29c060170 con 0x7fd29c13b380 2020-11-20T12:22:22.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.367+0000 7fd282ffd700 1 -- 172.21.15.116:0/1192182563 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2854 (secure 0 0 0) 0x7fd294047930 con 0x7fd29c13b380 2020-11-20T12:22:22.373 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:22.373 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/1612795957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1612795957},{"type":"v1","addr":"172.21.15.116:6835","nonce":1612795957}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/3255680781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3255680781},{"type":"v1","addr":"172.21.15.173:6835","nonce":3255680781}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/4058046293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":4058046293},{"type":"v1","addr":"172.21.15.173:6837","nonce":4058046293}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:56.570302+0000","modified":"2020-11-20T12:21:58.589498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8148":{"gid":8148,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2775705840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2775705840},{"type":"v1","addr":"172.21.15.116:6833","nonce":2775705840}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-20T12:22:22.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 -- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2780557c0 msgr2=0x7fd278057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2780557c0 0x7fd278057c80 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fd28c006e30 tx=0x7fd28c015000).stop 2020-11-20T12:22:22.375 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 -- 172.21.15.116:0/1192182563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd29c13b380 msgr2=0x7fd29c139fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd29c13b380 0x7fd29c139fe0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7fd29400ce60 tx=0x7fd294001dc0).stop 2020-11-20T12:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 -- 172.21.15.116:0/1192182563 shutdown_connections 2020-11-20T12:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2780557c0 0x7fd278057c80 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd29c13b380 0x7fd29c139fe0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd29c0ad440 0x7fd29c154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 --2- 172.21.15.116:0/1192182563 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd29c13afa0 0x7fd29c063630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 -- 172.21.15.116:0/1192182563 >> 172.21.15.116:0/1192182563 conn(0x7fd29c0678a0 msgr2=0x7fd29c0a86b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 -- 172.21.15.116:0/1192182563 shutdown_connections 2020-11-20T12:22:22.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.371+0000 7fd2a08c0700 1 -- 172.21.15.116:0/1192182563 wait complete. 2020-11-20T12:22:22.378 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 68 2020-11-20T12:22:22.400 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:22.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.571+0000 7f99f83c0700 1 Processor -- start 2020-11-20T12:22:22.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.571+0000 7f99f83c0700 1 -- start start 2020-11-20T12:22:22.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.575+0000 7f99f83c0700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f99f00c3780 0x7f99f00c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.575+0000 7f99f83c0700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f00a6950 con 0x7f99f00a6280 2020-11-20T12:22:22.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.575+0000 7f99f83c0700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f00a6bf0 con 0x7f99f00c1970 2020-11-20T12:22:22.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.575+0000 7f99f83c0700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f014fed0 con 0x7f99f00c3780 2020-11-20T12:22:22.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.575+0000 7f99f595b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f99f00a6280 0x7f99f00c22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33276/0 (socket says 172.21.15.116:33276) 2020-11-20T12:22:22.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.575+0000 7f99f595b700 1 -- 172.21.15.116:0/2638521216 learned_addr learned my addr 172.21.15.116:0/2638521216 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:22.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.575+0000 7f99f615c700 1 --2- 172.21.15.116:0/2638521216 >> v2:172.21.15.173:3300/0 conn(0x7f99f00c3780 0x7f99f00c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f515a700 1 -- 172.21.15.116:0/2638521216 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1522506709 0 0) 0x7f99f00a6950 con 0x7f99f00a6280 2020-11-20T12:22:22.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f515a700 1 -- 172.21.15.116:0/2638521216 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99d8003240 con 0x7f99f00a6280 2020-11-20T12:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f515a700 1 -- 172.21.15.116:0/2638521216 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 65113239 0 0) 0x7f99f00a6bf0 con 0x7f99f00c1970 2020-11-20T12:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f515a700 1 -- 172.21.15.116:0/2638521216 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99f00a6950 con 0x7f99f00c1970 2020-11-20T12:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f615c700 1 -- 172.21.15.116:0/2638521216 >> v1:172.21.15.173:6789/0 conn(0x7f99f00c1970 legacy=0x7f99f0150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f615c700 1 -- 172.21.15.116:0/2638521216 >> v1:172.21.15.116:6789/0 conn(0x7f99f00a6280 legacy=0x7f99f00c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f615c700 1 -- 172.21.15.116:0/2638521216 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f00a6e90 con 0x7f99f00c3780 2020-11-20T12:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f615c700 1 --2- 172.21.15.116:0/2638521216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f00c3780 0x7f99f00c3b60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f99ec00b9a0 tx=0x7f99ec006df0).ready entity=mon.1 client_cookie=ed5993d872904bae server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f515a700 1 -- 172.21.15.116:0/2638521216 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec009680 con 0x7f99f00c3780 2020-11-20T12:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f515a700 1 -- 172.21.15.116:0/2638521216 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec0097e0 con 0x7f99f00c3780 2020-11-20T12:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f515a700 1 -- 172.21.15.116:0/2638521216 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec009b60 con 0x7f99f00c3780 2020-11-20T12:22:22.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f83c0700 1 -- 172.21.15.116:0/2638521216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f00c3780 msgr2=0x7f99f00c3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f83c0700 1 --2- 172.21.15.116:0/2638521216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f00c3780 0x7f99f00c3b60 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f99ec00b9a0 tx=0x7f99ec006df0).stop 2020-11-20T12:22:22.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f83c0700 1 -- 172.21.15.116:0/2638521216 shutdown_connections 2020-11-20T12:22:22.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f83c0700 1 --2- 172.21.15.116:0/2638521216 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f00c3780 0x7f99f00c3b60 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f83c0700 1 -- 172.21.15.116:0/2638521216 >> 172.21.15.116:0/2638521216 conn(0x7f99f00c4f80 msgr2=0x7f99f00c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:22.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f83c0700 1 -- 172.21.15.116:0/2638521216 shutdown_connections 2020-11-20T12:22:22.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.579+0000 7f99f83c0700 1 -- 172.21.15.116:0/2638521216 wait complete. 2020-11-20T12:22:22.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f83c0700 1 Processor -- start 2020-11-20T12:22:22.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f83c0700 1 -- start start 2020-11-20T12:22:22.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f83c0700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99f00a6280 0x7f99f0154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f83c0700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f00c1970 0x7f99f013bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f83c0700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99f00c3780 0x7f99f00bbc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f83c0700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0154a40 con 0x7f99f00a6280 2020-11-20T12:22:22.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f83c0700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f0153630 con 0x7f99f00c1970 2020-11-20T12:22:22.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f83c0700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01538d0 con 0x7f99f00c3780 2020-11-20T12:22:22.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f615c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99f00a6280 0x7f99f0154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f615c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99f00a6280 0x7f99f0154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52078/0 (socket says 172.21.15.116:52078) 2020-11-20T12:22:22.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f615c700 1 -- 172.21.15.116:0/2384155391 learned_addr learned my addr 172.21.15.116:0/2384155391 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:22.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f595b700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f00c1970 0x7f99f013bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f695d700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99f00c3780 0x7f99f00bbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f615c700 1 -- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99f00c3780 msgr2=0x7f99f00bbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f615c700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99f00c3780 0x7f99f00bbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f615c700 1 -- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f00c1970 msgr2=0x7f99f013bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f615c700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f00c1970 0x7f99f013bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.583+0000 7f99f615c700 1 -- 172.21.15.116:0/2384155391 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f0153b70 con 0x7f99f00a6280 2020-11-20T12:22:22.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.587+0000 7f99f615c700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99f00a6280 0x7f99f0154550 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f99ec006b80 tx=0x7f99ec015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.587+0000 7f99de7fc700 1 -- 172.21.15.116:0/2384155391 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec011e60 con 0x7f99f00a6280 2020-11-20T12:22:22.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.587+0000 7f99de7fc700 1 -- 172.21.15.116:0/2384155391 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec01e460 con 0x7f99f00a6280 2020-11-20T12:22:22.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.587+0000 7f99de7fc700 1 -- 172.21.15.116:0/2384155391 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec01e7e0 con 0x7f99f00a6280 2020-11-20T12:22:22.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.587+0000 7f99f83c0700 1 -- 172.21.15.116:0/2384155391 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f0153e10 con 0x7f99f00a6280 2020-11-20T12:22:22.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.587+0000 7f99f83c0700 1 -- 172.21.15.116:0/2384155391 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f013c400 con 0x7f99f00a6280 2020-11-20T12:22:22.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.587+0000 7f99f83c0700 1 -- 172.21.15.116:0/2384155391 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f005d2a0 con 0x7f99f00a6280 2020-11-20T12:22:22.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.587+0000 7f99de7fc700 1 -- 172.21.15.116:0/2384155391 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f99ec025030 con 0x7f99f00a6280 2020-11-20T12:22:22.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.591+0000 7f99de7fc700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99c80557c0 0x7f99c8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.591+0000 7f99de7fc700 1 -- 172.21.15.116:0/2384155391 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f99ec06fd70 con 0x7f99f00a6280 2020-11-20T12:22:22.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.591+0000 7f99f595b700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99c80557c0 0x7f99c8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.591+0000 7f99f595b700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99c80557c0 0x7f99c8057c80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f99f00c09b0 tx=0x7f99e0009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.595+0000 7f99de7fc700 1 -- 172.21.15.116:0/2384155391 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f99ec045cc0 con 0x7f99f00a6280 2020-11-20T12:22:22.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.755+0000 7f99f83c0700 1 -- 172.21.15.116:0/2384155391 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99f00649f0 con 0x7f99f00a6280 2020-11-20T12:22:22.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.755+0000 7f99de7fc700 1 -- 172.21.15.116:0/2384155391 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f99ec031210 con 0x7f99f00a6280 2020-11-20T12:22:22.763 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:22.763 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":118,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:56.569994+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:53.361384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:53.918177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:22.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 -- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99c80557c0 msgr2=0x7f99c8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99c80557c0 0x7f99c8057c80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f99f00c09b0 tx=0x7f99e0009040).stop 2020-11-20T12:22:22.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 -- 172.21.15.116:0/2384155391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99f00a6280 msgr2=0x7f99f0154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99f00a6280 0x7f99f0154550 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f99ec006b80 tx=0x7f99ec015000).stop 2020-11-20T12:22:22.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 -- 172.21.15.116:0/2384155391 shutdown_connections 2020-11-20T12:22:22.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f99c80557c0 0x7f99c8057c80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f99f00c3780 0x7f99f00bbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f99f00a6280 0x7f99f0154550 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 --2- 172.21.15.116:0/2384155391 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f99f00c1970 0x7f99f013bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 -- 172.21.15.116:0/2384155391 >> 172.21.15.116:0/2384155391 conn(0x7f99f00c4f80 msgr2=0x7f99f00c1000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:22.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 -- 172.21.15.116:0/2384155391 shutdown_connections 2020-11-20T12:22:22.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.759+0000 7f99f83c0700 1 -- 172.21.15.116:0/2384155391 wait complete. 2020-11-20T12:22:22.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:21:53.361384+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:21:53.918177+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '118', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:22:22.787 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:22:22.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.959+0000 7feef759e700 1 Processor -- start 2020-11-20T12:22:22.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.959+0000 7feef759e700 1 -- start start 2020-11-20T12:22:22.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef759e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feef8061360 0x7feef8153490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.968 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feef80617c0 0x7feef8153980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef759e700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef80c2050 con 0x7feef8149f10 2020-11-20T12:22:22.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef759e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef80c22f0 con 0x7feef8061360 2020-11-20T12:22:22.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef80c2590 con 0x7feef80617c0 2020-11-20T12:22:22.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef5d9b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7feef8061360 0x7feef8153490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef659c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7feef8149f10 0x7feef80ba5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33288/0 (socket says 172.21.15.116:33288) 2020-11-20T12:22:22.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef659c700 1 -- 172.21.15.116:0/1536991415 learned_addr learned my addr 172.21.15.116:0/1536991415 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:22.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef6d9d700 1 --2- 172.21.15.116:0/1536991415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feef80617c0 0x7feef8153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef5d9b700 1 -- 172.21.15.116:0/1536991415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feef80617c0 msgr2=0x7feef8153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:22.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef5d9b700 1 --2- 172.21.15.116:0/1536991415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feef80617c0 0x7feef8153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef5d9b700 1 -- 172.21.15.116:0/1536991415 >> v1:172.21.15.116:6789/0 conn(0x7feef8149f10 legacy=0x7feef80ba5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef5d9b700 1 -- 172.21.15.116:0/1536991415 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef8060c20 con 0x7feef8061360 2020-11-20T12:22:22.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef5d9b700 1 --2- 172.21.15.116:0/1536991415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef8061360 0x7feef8153490 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7feeec00b9a0 tx=0x7feeec006df0).ready entity=mon.0 client_cookie=dd91f8663e51b4e5 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef559a700 1 -- 172.21.15.116:0/1536991415 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0097b0 con 0x7feef8061360 2020-11-20T12:22:22.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef559a700 1 -- 172.21.15.116:0/1536991415 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec009910 con 0x7feef8061360 2020-11-20T12:22:22.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.963+0000 7feef559a700 1 -- 172.21.15.116:0/1536991415 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec009c90 con 0x7feef8061360 2020-11-20T12:22:22.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.967+0000 7feef759e700 1 -- 172.21.15.116:0/1536991415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef8061360 msgr2=0x7feef8153490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.967+0000 7feef759e700 1 --2- 172.21.15.116:0/1536991415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef8061360 0x7feef8153490 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7feeec00b9a0 tx=0x7feeec006df0).stop 2020-11-20T12:22:22.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.967+0000 7feef759e700 1 -- 172.21.15.116:0/1536991415 shutdown_connections 2020-11-20T12:22:22.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.967+0000 7feef759e700 1 --2- 172.21.15.116:0/1536991415 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feef80617c0 0x7feef8153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.967+0000 7feef759e700 1 --2- 172.21.15.116:0/1536991415 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef8061360 0x7feef8153490 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.967+0000 7feef759e700 1 -- 172.21.15.116:0/1536991415 >> 172.21.15.116:0/1536991415 conn(0x7feef80c5740 msgr2=0x7feef80c1710 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:22.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.967+0000 7feef759e700 1 -- 172.21.15.116:0/1536991415 shutdown_connections 2020-11-20T12:22:22.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.967+0000 7feef759e700 1 -- 172.21.15.116:0/1536991415 wait complete. 2020-11-20T12:22:22.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.967+0000 7feef759e700 1 Processor -- start 2020-11-20T12:22:22.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef759e700 1 -- start start 2020-11-20T12:22:22.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feef8061360 0x7feef813e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feef80617c0 0x7feef80be510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef8149f10 0x7feef8060170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef81659e0 con 0x7feef8149f10 2020-11-20T12:22:22.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef8164ac0 con 0x7feef80617c0 2020-11-20T12:22:22.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef8164d60 con 0x7feef8061360 2020-11-20T12:22:22.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef6d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef8149f10 0x7feef8060170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef6d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef8149f10 0x7feef8060170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52096/0 (socket says 172.21.15.116:52096) 2020-11-20T12:22:22.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef6d9d700 1 -- 172.21.15.116:0/1114986055 learned_addr learned my addr 172.21.15.116:0/1114986055 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:22.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef659c700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feef8061360 0x7feef813e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef5d9b700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feef80617c0 0x7feef80be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef6d9d700 1 -- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feef8061360 msgr2=0x7feef813e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef6d9d700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feef8061360 0x7feef813e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef6d9d700 1 -- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feef80617c0 msgr2=0x7feef80be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:22.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef6d9d700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feef80617c0 0x7feef80be510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:22.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef6d9d700 1 -- 172.21.15.116:0/1114986055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef8165000 con 0x7feef8149f10 2020-11-20T12:22:22.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef5d9b700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feef80617c0 0x7feef80be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:22.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feef6d9d700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef8149f10 0x7feef8060170 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7feef000dae0 tx=0x7feef0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feee67fc700 1 -- 172.21.15.116:0/1114986055 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0009850 con 0x7feef8149f10 2020-11-20T12:22:22.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.971+0000 7feee67fc700 1 -- 172.21.15.116:0/1114986055 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef00099b0 con 0x7feef8149f10 2020-11-20T12:22:22.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.975+0000 7feef759e700 1 -- 172.21.15.116:0/1114986055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef81652a0 con 0x7feef8149f10 2020-11-20T12:22:22.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.975+0000 7feee67fc700 1 -- 172.21.15.116:0/1114986055 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef0009d50 con 0x7feef8149f10 2020-11-20T12:22:22.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.975+0000 7feef759e700 1 -- 172.21.15.116:0/1114986055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef80a56e0 con 0x7feef8149f10 2020-11-20T12:22:22.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.975+0000 7feef759e700 1 -- 172.21.15.116:0/1114986055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef805cf30 con 0x7feef8149f10 2020-11-20T12:22:22.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.975+0000 7feee67fc700 1 -- 172.21.15.116:0/1114986055 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7feef0012630 con 0x7feef8149f10 2020-11-20T12:22:22.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.975+0000 7feee67fc700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feecc0557c0 0x7feecc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:22.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.975+0000 7feee67fc700 1 -- 172.21.15.116:0/1114986055 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7feef006e960 con 0x7feef8149f10 2020-11-20T12:22:22.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.979+0000 7feef659c700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feecc0557c0 0x7feecc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:22.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.979+0000 7feef659c700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feecc0557c0 0x7feecc057c80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7feef8152710 tx=0x7feee8007b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:22.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:22.983+0000 7feee67fc700 1 -- 172.21.15.116:0/1114986055 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7feef0044800 con 0x7feef8149f10 2020-11-20T12:22:23.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feef759e700 1 -- 172.21.15.116:0/1114986055 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7feef805bc80 con 0x7feef8149f10 2020-11-20T12:22:23.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feee67fc700 1 -- 172.21.15.116:0/1114986055 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v118) v1 ==== 110+0+10 (secure 0 0 0) 0x7feef0047e20 con 0x7feef8149f10 2020-11-20T12:22:23.148 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:22:23.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feef759e700 1 -- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feecc0557c0 msgr2=0x7feecc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feef759e700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feecc0557c0 0x7feecc057c80 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7feef8152710 tx=0x7feee8007b80).stop 2020-11-20T12:22:23.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feef759e700 1 -- 172.21.15.116:0/1114986055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef8149f10 msgr2=0x7feef8060170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feef759e700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef8149f10 0x7feef8060170 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7feef000dae0 tx=0x7feef0008040).stop 2020-11-20T12:22:23.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feef759e700 1 -- 172.21.15.116:0/1114986055 shutdown_connections 2020-11-20T12:22:23.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feef759e700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7feecc0557c0 0x7feecc057c80 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feef759e700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7feef8149f10 0x7feef8060170 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feef759e700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7feef8061360 0x7feef813e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feef759e700 1 --2- 172.21.15.116:0/1114986055 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7feef80617c0 0x7feef80be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.143+0000 7feef759e700 1 -- 172.21.15.116:0/1114986055 >> 172.21.15.116:0/1114986055 conn(0x7feef80c5740 msgr2=0x7feef80c42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:23.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.147+0000 7feef759e700 1 -- 172.21.15.116:0/1114986055 shutdown_connections 2020-11-20T12:22:23.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.147+0000 7feef759e700 1 -- 172.21.15.116:0/1114986055 wait complete. 2020-11-20T12:22:23.174 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:22:23.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.343+0000 7f983aa64700 1 Processor -- start 2020-11-20T12:22:23.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.343+0000 7f983aa64700 1 -- start start 2020-11-20T12:22:23.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f983aa64700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9834155c80 0x7f9834062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:23.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f983aa64700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f98340bed20 0x7f98340bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:23.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f983aa64700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98340b93c0 con 0x7f9834156060 2020-11-20T12:22:23.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f983aa64700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98340ba580 con 0x7f9834155c80 2020-11-20T12:22:23.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f983aa64700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98340b9660 con 0x7f98340bed20 2020-11-20T12:22:23.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f9839a62700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9834155c80 0x7f9834062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:23.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f983a263700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f98340bed20 0x7f98340bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:23.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f9839261700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f9834156060 0x7f98340c0dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:57764/0 (socket says 172.21.15.116:57764) 2020-11-20T12:22:23.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f9839261700 1 -- 172.21.15.116:0/136892572 learned_addr learned my addr 172.21.15.116:0/136892572 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:23.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f9839a62700 1 --2- 172.21.15.116:0/136892572 >> v2:172.21.15.116:3300/0 conn(0x7f9834155c80 0x7f9834062ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52102/0 (socket says 172.21.15.116:52102) 2020-11-20T12:22:23.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f9838a60700 1 -- 172.21.15.116:0/136892572 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4235986868 0 0) 0x7f98340b93c0 con 0x7f9834156060 2020-11-20T12:22:23.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f983a263700 1 -- 172.21.15.116:0/136892572 >> v2:172.21.15.116:3300/0 conn(0x7f9834155c80 msgr2=0x7f9834062ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:23.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f983a263700 1 --2- 172.21.15.116:0/136892572 >> v2:172.21.15.116:3300/0 conn(0x7f9834155c80 0x7f9834062ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f983a263700 1 -- 172.21.15.116:0/136892572 >> v1:172.21.15.173:6789/0 conn(0x7f9834156060 legacy=0x7f98340c0dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.347+0000 7f983a263700 1 -- 172.21.15.116:0/136892572 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340b9900 con 0x7f98340bed20 2020-11-20T12:22:23.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f983a263700 1 --2- 172.21.15.116:0/136892572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98340bed20 0x7f98340bf1b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f983405c5c0 tx=0x7f982c0091d0).ready entity=mon.1 client_cookie=367fd88eacc6818a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:23.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f9838a60700 1 -- 172.21.15.116:0/136892572 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c00ba70 con 0x7f98340bed20 2020-11-20T12:22:23.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f9838a60700 1 -- 172.21.15.116:0/136892572 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c00bbd0 con 0x7f98340bed20 2020-11-20T12:22:23.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f9838a60700 1 -- 172.21.15.116:0/136892572 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c011400 con 0x7f98340bed20 2020-11-20T12:22:23.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f983aa64700 1 -- 172.21.15.116:0/136892572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98340bed20 msgr2=0x7f98340bf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f983aa64700 1 --2- 172.21.15.116:0/136892572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98340bed20 0x7f98340bf1b0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f983405c5c0 tx=0x7f982c0091d0).stop 2020-11-20T12:22:23.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f983aa64700 1 -- 172.21.15.116:0/136892572 shutdown_connections 2020-11-20T12:22:23.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f983aa64700 1 --2- 172.21.15.116:0/136892572 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98340bed20 0x7f98340bf1b0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f983aa64700 1 --2- 172.21.15.116:0/136892572 >> v2:172.21.15.116:3300/0 conn(0x7f9834155c80 0x7f9834062ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f983aa64700 1 -- 172.21.15.116:0/136892572 >> 172.21.15.116:0/136892572 conn(0x7f9834068350 msgr2=0x7f98340627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:23.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f983aa64700 1 -- 172.21.15.116:0/136892572 shutdown_connections 2020-11-20T12:22:23.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.351+0000 7f983aa64700 1 -- 172.21.15.116:0/136892572 wait complete. 2020-11-20T12:22:23.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f983aa64700 1 Processor -- start 2020-11-20T12:22:23.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f983aa64700 1 -- start start 2020-11-20T12:22:23.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f983aa64700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98340bed20 0x7f98341424c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:23.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f983aa64700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9834155c80 0x7f98340a87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:23.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f983aa64700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9834156060 0x7f98340c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:23.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f983aa64700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9834158bf0 con 0x7f9834156060 2020-11-20T12:22:23.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f983aa64700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9834165e10 con 0x7f98340bed20 2020-11-20T12:22:23.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f983aa64700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98341660b0 con 0x7f9834155c80 2020-11-20T12:22:23.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f983a263700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9834156060 0x7f98340c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:23.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f983a263700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9834156060 0x7f98340c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52110/0 (socket says 172.21.15.116:52110) 2020-11-20T12:22:23.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f983a263700 1 -- 172.21.15.116:0/845447728 learned_addr learned my addr 172.21.15.116:0/845447728 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:23.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f9839a62700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98340bed20 0x7f98341424c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:23.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.355+0000 7f9839261700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9834155c80 0x7f98340a87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:23.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f983a263700 1 -- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9834155c80 msgr2=0x7f98340a87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f983a263700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9834155c80 0x7f98340a87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f983a263700 1 -- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98340bed20 msgr2=0x7f98341424c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f983a263700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98340bed20 0x7f98341424c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f983a263700 1 -- 172.21.15.116:0/845447728 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98340a8080 con 0x7f9834156060 2020-11-20T12:22:23.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f9839261700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9834155c80 0x7f98340a87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:23.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f983a263700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9834156060 0x7f98340c2e80 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f98340b97a0 tx=0x7f982c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:23.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f9821ffb700 1 -- 172.21.15.116:0/845447728 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c006b20 con 0x7f9834156060 2020-11-20T12:22:23.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f9821ffb700 1 -- 172.21.15.116:0/845447728 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f982c01c6c0 con 0x7f9834156060 2020-11-20T12:22:23.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f983aa64700 1 -- 172.21.15.116:0/845447728 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98340a8320 con 0x7f9834156060 2020-11-20T12:22:23.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f9821ffb700 1 -- 172.21.15.116:0/845447728 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f982c01ca40 con 0x7f9834156060 2020-11-20T12:22:23.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f983aa64700 1 -- 172.21.15.116:0/845447728 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98341429b0 con 0x7f9834156060 2020-11-20T12:22:23.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.359+0000 7f983aa64700 1 -- 172.21.15.116:0/845447728 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f983405cf30 con 0x7f9834156060 2020-11-20T12:22:23.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.363+0000 7f9821ffb700 1 -- 172.21.15.116:0/845447728 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f982c017070 con 0x7f9834156060 2020-11-20T12:22:23.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.363+0000 7f9821ffb700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f980c0557c0 0x7f980c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:23.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.363+0000 7f9821ffb700 1 -- 172.21.15.116:0/845447728 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f982c071530 con 0x7f9834156060 2020-11-20T12:22:23.370 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.363+0000 7f9839a62700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f980c0557c0 0x7f980c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:23.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.367+0000 7f9839a62700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f980c0557c0 0x7f980c057c80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f9834158830 tx=0x7f9830006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:23.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.367+0000 7f9821ffb700 1 -- 172.21.15.116:0/845447728 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f982c04acf0 con 0x7f9834156060 2020-11-20T12:22:23.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.527+0000 7f983aa64700 1 -- 172.21.15.116:0/845447728 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f98340639c0 con 0x7f9834156060 2020-11-20T12:22:23.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.527+0000 7f9821ffb700 1 -- 172.21.15.116:0/845447728 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v118) v1 ==== 92+0+10 (secure 0 0 0) 0x7f982c015070 con 0x7f9834156060 2020-11-20T12:22:23.534 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:23.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 -- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f980c0557c0 msgr2=0x7f980c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f980c0557c0 0x7f980c057c80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7f9834158830 tx=0x7f9830006df0).stop 2020-11-20T12:22:23.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 -- 172.21.15.116:0/845447728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9834156060 msgr2=0x7f98340c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9834156060 0x7f98340c2e80 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f98340b97a0 tx=0x7f982c016000).stop 2020-11-20T12:22:23.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 -- 172.21.15.116:0/845447728 shutdown_connections 2020-11-20T12:22:23.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f980c0557c0 0x7f980c057c80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9834156060 0x7f98340c2e80 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f98340bed20 0x7f98341424c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.539 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 --2- 172.21.15.116:0/845447728 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9834155c80 0x7f98340a87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 -- 172.21.15.116:0/845447728 >> 172.21.15.116:0/845447728 conn(0x7f9834068350 msgr2=0x7f9834066f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:23.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 -- 172.21.15.116:0/845447728 shutdown_connections 2020-11-20T12:22:23.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.531+0000 7f983aa64700 1 -- 172.21.15.116:0/845447728 wait complete. 2020-11-20T12:22:23.562 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:22:23.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.731+0000 7f6fe34dc700 1 Processor -- start 2020-11-20T12:22:23.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.731+0000 7f6fe34dc700 1 -- start start 2020-11-20T12:22:23.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.731+0000 7f6fe34dc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc156440 0x7f6fdc0aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:23.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.731+0000 7f6fe34dc700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc0ace40 con 0x7f6fdc0aa7d0 2020-11-20T12:22:23.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.731+0000 7f6fe34dc700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fdc1526c0 con 0x7f6fdc0a8400 2020-11-20T12:22:23.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.731+0000 7f6fe34dc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc152960 con 0x7f6fdc156440 2020-11-20T12:22:23.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe24da700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6fdc0aa7d0 0x7f6fdc151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33316/0 (socket says 172.21.15.116:33316) 2020-11-20T12:22:23.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe24da700 1 -- 172.21.15.116:0/3036565125 learned_addr learned my addr 172.21.15.116:0/3036565125 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:23.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe2cdb700 1 --2- 172.21.15.116:0/3036565125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc156440 0x7f6fdc0aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:23.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe14d8700 1 -- 172.21.15.116:0/3036565125 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954003068 0 0) 0x7f6fdc0ace40 con 0x7f6fdc0aa7d0 2020-11-20T12:22:23.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe14d8700 1 -- 172.21.15.116:0/3036565125 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc4003240 con 0x7f6fdc0aa7d0 2020-11-20T12:22:23.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe14d8700 1 -- 172.21.15.116:0/3036565125 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716979079 0 0) 0x7f6fdc1526c0 con 0x7f6fdc0a8400 2020-11-20T12:22:23.741 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe2cdb700 1 -- 172.21.15.116:0/3036565125 >> v1:172.21.15.173:6789/0 conn(0x7f6fdc0a8400 legacy=0x7f6fdc0ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe2cdb700 1 -- 172.21.15.116:0/3036565125 >> v1:172.21.15.116:6789/0 conn(0x7f6fdc0aa7d0 legacy=0x7f6fdc151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe2cdb700 1 -- 172.21.15.116:0/3036565125 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc152c00 con 0x7f6fdc156440 2020-11-20T12:22:23.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe2cdb700 1 --2- 172.21.15.116:0/3036565125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc156440 0x7f6fdc0aabb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f6fd4009cc0 tx=0x7f6fd40091d0).ready entity=mon.2 client_cookie=eb572d5c236cf421 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:23.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe14d8700 1 -- 172.21.15.116:0/3036565125 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd80014e0 con 0x7f6fdc156440 2020-11-20T12:22:23.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe14d8700 1 -- 172.21.15.116:0/3036565125 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd400b7c0 con 0x7f6fdc156440 2020-11-20T12:22:23.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.735+0000 7f6fe14d8700 1 -- 172.21.15.116:0/3036565125 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd400bb40 con 0x7f6fdc156440 2020-11-20T12:22:23.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.739+0000 7f6fe34dc700 1 -- 172.21.15.116:0/3036565125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc156440 msgr2=0x7f6fdc0aabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.739+0000 7f6fe34dc700 1 --2- 172.21.15.116:0/3036565125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc156440 0x7f6fdc0aabb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f6fd4009cc0 tx=0x7f6fd40091d0).stop 2020-11-20T12:22:23.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.739+0000 7f6fe34dc700 1 -- 172.21.15.116:0/3036565125 shutdown_connections 2020-11-20T12:22:23.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.739+0000 7f6fe34dc700 1 --2- 172.21.15.116:0/3036565125 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc156440 0x7f6fdc0aabb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.739+0000 7f6fe34dc700 1 -- 172.21.15.116:0/3036565125 >> 172.21.15.116:0/3036565125 conn(0x7f6fdc0678a0 msgr2=0x7f6fdc067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:23.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.739+0000 7f6fe34dc700 1 -- 172.21.15.116:0/3036565125 shutdown_connections 2020-11-20T12:22:23.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.739+0000 7f6fe34dc700 1 -- 172.21.15.116:0/3036565125 wait complete. 2020-11-20T12:22:23.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.739+0000 7f6fe34dc700 1 Processor -- start 2020-11-20T12:22:23.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.739+0000 7f6fe34dc700 1 -- start start 2020-11-20T12:22:23.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe34dc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fdc0a8400 0x7f6fdc14dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:23.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe34dc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc0aa7d0 0x7f6fdc14c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:23.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe34dc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fdc156440 0x7f6fdc14a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:23.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe34dc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc141b40 con 0x7f6fdc0a8400 2020-11-20T12:22:23.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe34dc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc14e100 con 0x7f6fdc156440 2020-11-20T12:22:23.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe34dc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc14ca60 con 0x7f6fdc0aa7d0 2020-11-20T12:22:23.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe24da700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fdc0a8400 0x7f6fdc14dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:23.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe24da700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fdc0a8400 0x7f6fdc14dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52120/0 (socket says 172.21.15.116:52120) 2020-11-20T12:22:23.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe24da700 1 -- 172.21.15.116:0/280495088 learned_addr learned my addr 172.21.15.116:0/280495088 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:23.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe2cdb700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fdc156440 0x7f6fdc14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:23.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe1cd9700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc0aa7d0 0x7f6fdc14c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:23.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe24da700 1 -- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc0aa7d0 msgr2=0x7f6fdc14c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe24da700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc0aa7d0 0x7f6fdc14c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe24da700 1 -- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fdc156440 msgr2=0x7f6fdc14a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe24da700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fdc156440 0x7f6fdc14a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe24da700 1 -- 172.21.15.116:0/280495088 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc14ccf0 con 0x7f6fdc0a8400 2020-11-20T12:22:23.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe1cd9700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc0aa7d0 0x7f6fdc14c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:23.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe24da700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fdc0a8400 0x7f6fdc14dc10 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f6fd800cb40 tx=0x7f6fd8001c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:23.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fca7fc700 1 -- 172.21.15.116:0/280495088 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8007170 con 0x7f6fdc0a8400 2020-11-20T12:22:23.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fca7fc700 1 -- 172.21.15.116:0/280495088 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd80072d0 con 0x7f6fdc0a8400 2020-11-20T12:22:23.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe34dc700 1 -- 172.21.15.116:0/280495088 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc14cf90 con 0x7f6fdc0a8400 2020-11-20T12:22:23.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fca7fc700 1 -- 172.21.15.116:0/280495088 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd800ab90 con 0x7f6fdc0a8400 2020-11-20T12:22:23.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.743+0000 7f6fe34dc700 1 -- 172.21.15.116:0/280495088 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc14d4d0 con 0x7f6fdc0a8400 2020-11-20T12:22:23.754 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.747+0000 7f6fe34dc700 1 -- 172.21.15.116:0/280495088 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fdc0c5bd0 con 0x7f6fdc0a8400 2020-11-20T12:22:23.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.747+0000 7f6fca7fc700 1 -- 172.21.15.116:0/280495088 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6fd8010070 con 0x7f6fdc0a8400 2020-11-20T12:22:23.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.747+0000 7f6fca7fc700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb80557f0 0x7f6fb8057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:23.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.747+0000 7f6fca7fc700 1 -- 172.21.15.116:0/280495088 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6fd806e640 con 0x7f6fdc0a8400 2020-11-20T12:22:23.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.747+0000 7f6fe1cd9700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb80557f0 0x7f6fb8057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:23.756 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.751+0000 7f6fe1cd9700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb80557f0 0x7f6fb8057cb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f6fdc149e30 tx=0x7f6fcc009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:23.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.755+0000 7f6fca7fc700 1 -- 172.21.15.116:0/280495088 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6fd80444e0 con 0x7f6fdc0a8400 2020-11-20T12:22:23.919 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.911+0000 7f6fe34dc700 1 -- 172.21.15.116:0/280495088 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6fdc05cf30 con 0x7f6fdc0a8400 2020-11-20T12:22:23.920 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.911+0000 7f6fca7fc700 1 -- 172.21.15.116:0/280495088 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) v1 ==== 104+0+10 (secure 0 0 0) 0x7f6fd8047b00 con 0x7f6fdc0a8400 2020-11-20T12:22:23.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 -- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb80557f0 msgr2=0x7f6fb8057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb80557f0 0x7f6fb8057cb0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7f6fdc149e30 tx=0x7f6fcc009040).stop 2020-11-20T12:22:23.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 -- 172.21.15.116:0/280495088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fdc0a8400 msgr2=0x7f6fdc14dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:23.921 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fdc0a8400 0x7f6fdc14dc10 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f6fd800cb40 tx=0x7f6fd8001c80).stop 2020-11-20T12:22:23.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 -- 172.21.15.116:0/280495088 shutdown_connections 2020-11-20T12:22:23.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6fb80557f0 0x7f6fb8057cb0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6fdc156440 0x7f6fdc14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6fdc0a8400 0x7f6fdc14dc10 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.922 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 --2- 172.21.15.116:0/280495088 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6fdc0aa7d0 0x7f6fdc14c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:23.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 -- 172.21.15.116:0/280495088 >> 172.21.15.116:0/280495088 conn(0x7f6fdc0678a0 msgr2=0x7f6fdc0618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:23.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 -- 172.21.15.116:0/280495088 shutdown_connections 2020-11-20T12:22:23.923 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:23.915+0000 7f6fe34dc700 1 -- 172.21.15.116:0/280495088 wait complete. 2020-11-20T12:22:23.923 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:23.943 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:22:24.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.119+0000 7fba1f59e700 1 Processor -- start 2020-11-20T12:22:24.126 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.119+0000 7fba1f59e700 1 -- start start 2020-11-20T12:22:24.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1f59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fba200aa7d0 0x7fba200a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200ad540 0x7fba200ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.128 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1f59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba200ace40 con 0x7fba200aabb0 2020-11-20T12:22:24.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1f59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba201526c0 con 0x7fba200aa7d0 2020-11-20T12:22:24.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba20152960 con 0x7fba200ad540 2020-11-20T12:22:24.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1ed9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200ad540 0x7fba200ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1ed9d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200ad540 0x7fba200ad9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:32846/0 (socket says 172.21.15.116:32846) 2020-11-20T12:22:24.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1ed9d700 1 -- 172.21.15.116:0/3877950770 learned_addr learned my addr 172.21.15.116:0/3877950770 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:24.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1e59c700 1 --2- 172.21.15.116:0/3877950770 >> v2:172.21.15.173:3300/0 conn(0x7fba200aa7d0 0x7fba200a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1ed9d700 1 -- 172.21.15.116:0/3877950770 >> v2:172.21.15.173:3300/0 conn(0x7fba200aa7d0 msgr2=0x7fba200a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:24.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1ed9d700 1 --2- 172.21.15.116:0/3877950770 >> v2:172.21.15.173:3300/0 conn(0x7fba200aa7d0 0x7fba200a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1ed9d700 1 -- 172.21.15.116:0/3877950770 >> v1:172.21.15.173:6789/0 conn(0x7fba200aabb0 legacy=0x7fba20151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.131 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1ed9d700 1 -- 172.21.15.116:0/3877950770 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba20152c00 con 0x7fba200ad540 2020-11-20T12:22:24.132 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.123+0000 7fba1ed9d700 1 --2- 172.21.15.116:0/3877950770 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200ad540 0x7fba200ad9d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fba200acfd0 tx=0x7fba18009290).ready entity=mon.2 client_cookie=ef49beace7e76c98 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:24.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.127+0000 7fba1d59a700 1 -- 172.21.15.116:0/3877950770 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1800bbc0 con 0x7fba200ad540 2020-11-20T12:22:24.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.127+0000 7fba1d59a700 1 -- 172.21.15.116:0/3877950770 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1800bd20 con 0x7fba200ad540 2020-11-20T12:22:24.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.127+0000 7fba1d59a700 1 -- 172.21.15.116:0/3877950770 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18011610 con 0x7fba200ad540 2020-11-20T12:22:24.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.127+0000 7fba1f59e700 1 -- 172.21.15.116:0/3877950770 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200ad540 msgr2=0x7fba200ad9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.127+0000 7fba1f59e700 1 --2- 172.21.15.116:0/3877950770 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200ad540 0x7fba200ad9d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fba200acfd0 tx=0x7fba18009290).stop 2020-11-20T12:22:24.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.127+0000 7fba1f59e700 1 -- 172.21.15.116:0/3877950770 shutdown_connections 2020-11-20T12:22:24.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.127+0000 7fba1f59e700 1 --2- 172.21.15.116:0/3877950770 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200ad540 0x7fba200ad9d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.127+0000 7fba1f59e700 1 --2- 172.21.15.116:0/3877950770 >> v2:172.21.15.173:3300/0 conn(0x7fba200aa7d0 0x7fba200a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.127+0000 7fba1f59e700 1 -- 172.21.15.116:0/3877950770 >> 172.21.15.116:0/3877950770 conn(0x7fba200678a0 msgr2=0x7fba20067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:24.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.127+0000 7fba1f59e700 1 -- 172.21.15.116:0/3877950770 shutdown_connections 2020-11-20T12:22:24.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.127+0000 7fba1f59e700 1 -- 172.21.15.116:0/3877950770 wait complete. 2020-11-20T12:22:24.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1f59e700 1 Processor -- start 2020-11-20T12:22:24.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1f59e700 1 -- start start 2020-11-20T12:22:24.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1f59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200aa7d0 0x7fba2014c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1f59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba200aabb0 0x7fba201510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1f59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba200ad540 0x7fba20160400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1f59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba20156390 con 0x7fba200aabb0 2020-11-20T12:22:24.137 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1f59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba20156630 con 0x7fba200ad540 2020-11-20T12:22:24.138 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1f59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba201568d0 con 0x7fba200aa7d0 2020-11-20T12:22:24.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1dd9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba200aabb0 0x7fba201510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1dd9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba200aabb0 0x7fba201510c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52134/0 (socket says 172.21.15.116:52134) 2020-11-20T12:22:24.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1dd9b700 1 -- 172.21.15.116:0/1574298668 learned_addr learned my addr 172.21.15.116:0/1574298668 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:24.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1ed9d700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba200ad540 0x7fba20160400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.139 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1e59c700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200aa7d0 0x7fba2014c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1ed9d700 1 -- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200aa7d0 msgr2=0x7fba2014c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1ed9d700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200aa7d0 0x7fba2014c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1ed9d700 1 -- 172.21.15.116:0/1574298668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba200aabb0 msgr2=0x7fba201510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1ed9d700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba200aabb0 0x7fba201510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.140 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1ed9d700 1 -- 172.21.15.116:0/1574298668 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba20156b70 con 0x7fba200ad540 2020-11-20T12:22:24.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.131+0000 7fba1dd9b700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba200aabb0 0x7fba201510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:24.141 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.135+0000 7fba1ed9d700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba200ad540 0x7fba20160400 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fba200acfd0 tx=0x7fba18016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:24.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.135+0000 7fba0e7fc700 1 -- 172.21.15.116:0/1574298668 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba18020070 con 0x7fba200ad540 2020-11-20T12:22:24.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.135+0000 7fba0e7fc700 1 -- 172.21.15.116:0/1574298668 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba1800be90 con 0x7fba200ad540 2020-11-20T12:22:24.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.135+0000 7fba1f59e700 1 -- 172.21.15.116:0/1574298668 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba20156e10 con 0x7fba200ad540 2020-11-20T12:22:24.142 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.135+0000 7fba0e7fc700 1 -- 172.21.15.116:0/1574298668 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba1800bbc0 con 0x7fba200ad540 2020-11-20T12:22:24.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.135+0000 7fba1f59e700 1 -- 172.21.15.116:0/1574298668 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba201515b0 con 0x7fba200ad540 2020-11-20T12:22:24.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.135+0000 7fba1f59e700 1 -- 172.21.15.116:0/1574298668 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba20152de0 con 0x7fba200ad540 2020-11-20T12:22:24.143 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.139+0000 7fba0e7fc700 1 -- 172.21.15.116:0/1574298668 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fba18017070 con 0x7fba200ad540 2020-11-20T12:22:24.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.139+0000 7fba0e7fc700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9f40557c0 0x7fb9f4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.144 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.139+0000 7fba0e7fc700 1 -- 172.21.15.116:0/1574298668 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fba18070260 con 0x7fba200ad540 2020-11-20T12:22:24.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.139+0000 7fba1e59c700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9f40557c0 0x7fb9f4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.145 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.139+0000 7fba1e59c700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9f40557c0 0x7fb9f4057c80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fba1000da10 tx=0x7fba10008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:24.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.147+0000 7fba0e7fc700 1 -- 172.21.15.116:0/1574298668 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fba18045ff0 con 0x7fba200ad540 2020-11-20T12:22:24.317 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.311+0000 7fba1f59e700 1 -- 172.21.15.116:0/1574298668 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba2005cf30 con 0x7fba200ad540 2020-11-20T12:22:24.318 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.311+0000 7fba0e7fc700 1 -- 172.21.15.116:0/1574298668 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) v1 ==== 100+0+10 (secure 0 0 0) 0x7fba18015020 con 0x7fba200ad540 2020-11-20T12:22:24.319 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:24.320 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 -- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9f40557c0 msgr2=0x7fb9f4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9f40557c0 0x7fb9f4057c80 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fba1000da10 tx=0x7fba10008040).stop 2020-11-20T12:22:24.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 -- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba200ad540 msgr2=0x7fba20160400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba200ad540 0x7fba20160400 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7fba200acfd0 tx=0x7fba18016000).stop 2020-11-20T12:22:24.321 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 -- 172.21.15.116:0/1574298668 shutdown_connections 2020-11-20T12:22:24.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fb9f40557c0 0x7fb9f4057c80 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fba200ad540 0x7fba20160400 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fba200aa7d0 0x7fba2014c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.322 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 --2- 172.21.15.116:0/1574298668 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fba200aabb0 0x7fba201510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 -- 172.21.15.116:0/1574298668 >> 172.21.15.116:0/1574298668 conn(0x7fba200678a0 msgr2=0x7fba201545a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:24.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 -- 172.21.15.116:0/1574298668 shutdown_connections 2020-11-20T12:22:24.323 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.315+0000 7fba1f59e700 1 -- 172.21.15.116:0/1574298668 wait complete. 2020-11-20T12:22:24.349 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:24.522 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.515+0000 7f8ff497f700 1 Processor -- start 2020-11-20T12:22:24.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.515+0000 7f8ff497f700 1 -- start start 2020-11-20T12:22:24.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8ff497f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff0062c60 0x7f8ff0065d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8ff497f700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8ff00bfe20 0x7f8ff01556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8ff497f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff00c02b0 con 0x7f8ff0062880 2020-11-20T12:22:24.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8ff497f700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff00c0530 con 0x7f8ff00bfe20 2020-11-20T12:22:24.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8ff497f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff00c07d0 con 0x7f8ff0062c60 2020-11-20T12:22:24.949 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8fe7fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff0062c60 0x7f8ff0065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8fef59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8ff00bfe20 0x7f8ff01556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8fef59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f8ff00bfe20 0x7f8ff01556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:51196/0 (socket says 172.21.15.116:51196) 2020-11-20T12:22:24.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8fef59e700 1 -- 172.21.15.116:0/651073557 learned_addr learned my addr 172.21.15.116:0/651073557 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:24.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8fe7fff700 1 -- 172.21.15.116:0/651073557 >> v2:172.21.15.173:3300/0 conn(0x7f8ff00bfe20 msgr2=0x7f8ff01556e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:24.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8fe7fff700 1 --2- 172.21.15.116:0/651073557 >> v2:172.21.15.173:3300/0 conn(0x7f8ff00bfe20 0x7f8ff01556e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8fe7fff700 1 -- 172.21.15.116:0/651073557 >> v1:172.21.15.173:6789/0 conn(0x7f8ff0062880 legacy=0x7f8ff00677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.519+0000 7f8fe7fff700 1 -- 172.21.15.116:0/651073557 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff00c0a70 con 0x7f8ff0062c60 2020-11-20T12:22:24.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8fe7fff700 1 --2- 172.21.15.116:0/651073557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff0062c60 0x7f8ff0065d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f8fd800b9a0 tx=0x7f8fd8006df0).ready entity=mon.2 client_cookie=c64d557f651e0f9b server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:24.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8fee59c700 1 -- 172.21.15.116:0/651073557 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd80097b0 con 0x7f8ff0062c60 2020-11-20T12:22:24.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8fee59c700 1 -- 172.21.15.116:0/651073557 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd8009910 con 0x7f8ff0062c60 2020-11-20T12:22:24.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8fee59c700 1 -- 172.21.15.116:0/651073557 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8009c90 con 0x7f8ff0062c60 2020-11-20T12:22:24.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8ff497f700 1 -- 172.21.15.116:0/651073557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff0062c60 msgr2=0x7f8ff0065d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8ff497f700 1 --2- 172.21.15.116:0/651073557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff0062c60 0x7f8ff0065d40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f8fd800b9a0 tx=0x7f8fd8006df0).stop 2020-11-20T12:22:24.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8ff497f700 1 -- 172.21.15.116:0/651073557 shutdown_connections 2020-11-20T12:22:24.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8ff497f700 1 --2- 172.21.15.116:0/651073557 >> v2:172.21.15.173:3300/0 conn(0x7f8ff00bfe20 0x7f8ff01556e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8ff497f700 1 --2- 172.21.15.116:0/651073557 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff0062c60 0x7f8ff0065d40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8ff497f700 1 -- 172.21.15.116:0/651073557 >> 172.21.15.116:0/651073557 conn(0x7f8ff0060c40 msgr2=0x7f8ff00c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:24.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8ff497f700 1 -- 172.21.15.116:0/651073557 shutdown_connections 2020-11-20T12:22:24.953 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.523+0000 7f8ff497f700 1 -- 172.21.15.116:0/651073557 wait complete. 2020-11-20T12:22:24.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.527+0000 7f8ff497f700 1 Processor -- start 2020-11-20T12:22:24.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.527+0000 7f8ff497f700 1 -- start start 2020-11-20T12:22:24.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.527+0000 7f8ff497f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ff0062880 0x7f8ff01554b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.527+0000 7f8ff497f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff0062c60 0x7f8ff0168c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.954 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8ff497f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff00bfe20 0x7f8ff01521d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8ff497f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01559a0 con 0x7f8ff0062c60 2020-11-20T12:22:24.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8ff497f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff015b710 con 0x7f8ff0062880 2020-11-20T12:22:24.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8ff497f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff015b9b0 con 0x7f8ff00bfe20 2020-11-20T12:22:24.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff0062c60 0x7f8ff0168c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe7fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff0062c60 0x7f8ff0168c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52150/0 (socket says 172.21.15.116:52150) 2020-11-20T12:22:24.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe7fff700 1 -- 172.21.15.116:0/254264360 learned_addr learned my addr 172.21.15.116:0/254264360 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:24.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8feed9d700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ff0062880 0x7f8ff01554b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fef59e700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff00bfe20 0x7f8ff01521d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe7fff700 1 -- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff00bfe20 msgr2=0x7f8ff01521d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe7fff700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff00bfe20 0x7f8ff01521d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe7fff700 1 -- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ff0062880 msgr2=0x7f8ff01554b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe7fff700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ff0062880 0x7f8ff01554b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe7fff700 1 -- 172.21.15.116:0/254264360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff015bc50 con 0x7f8ff0062c60 2020-11-20T12:22:24.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fef59e700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff00bfe20 0x7f8ff01521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:24.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe7fff700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff0062c60 0x7f8ff0168c10 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f8ff00c0910 tx=0x7f8fd8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:24.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe6ffd700 1 -- 172.21.15.116:0/254264360 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd8011de0 con 0x7f8ff0062c60 2020-11-20T12:22:24.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe6ffd700 1 -- 172.21.15.116:0/254264360 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fd801e460 con 0x7f8ff0062c60 2020-11-20T12:22:24.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8ff497f700 1 -- 172.21.15.116:0/254264360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff015bef0 con 0x7f8ff0062c60 2020-11-20T12:22:24.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8fe6ffd700 1 -- 172.21.15.116:0/254264360 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fd801e800 con 0x7f8ff0062c60 2020-11-20T12:22:24.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.531+0000 7f8ff497f700 1 -- 172.21.15.116:0/254264360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff00a3d70 con 0x7f8ff0062c60 2020-11-20T12:22:24.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.535+0000 7f8ff497f700 1 -- 172.21.15.116:0/254264360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff00a3fc0 con 0x7f8ff0062c60 2020-11-20T12:22:24.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.535+0000 7f8fe6ffd700 1 -- 172.21.15.116:0/254264360 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8fd8025030 con 0x7f8ff0062c60 2020-11-20T12:22:24.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.535+0000 7f8fe6ffd700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fcc0557c0 0x7f8fcc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:24.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.535+0000 7f8fe6ffd700 1 -- 172.21.15.116:0/254264360 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8fd806fdb0 con 0x7f8ff0062c60 2020-11-20T12:22:24.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.535+0000 7f8feed9d700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fcc0557c0 0x7f8fcc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:24.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.539+0000 7f8feed9d700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fcc0557c0 0x7f8fcc057c80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f8fe0003610 tx=0x7f8fe0000b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:24.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.539+0000 7f8fe6ffd700 1 -- 172.21.15.116:0/254264360 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8fd8045d00 con 0x7f8ff0062c60 2020-11-20T12:22:24.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.747+0000 7f8ff497f700 1 -- 172.21.15.116:0/254264360 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ff005cf30 con 0x7f8ff0062c60 2020-11-20T12:22:24.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.747+0000 7f8fe6ffd700 1 -- 172.21.15.116:0/254264360 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 68 v68) v1 ==== 94+0+2854 (secure 0 0 0) 0x7f8fd8023070 con 0x7f8ff0062c60 2020-11-20T12:22:24.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 -- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fcc0557c0 msgr2=0x7f8fcc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fcc0557c0 0x7f8fcc057c80 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f8fe0003610 tx=0x7f8fe0000b80).stop 2020-11-20T12:22:24.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 -- 172.21.15.116:0/254264360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff0062c60 msgr2=0x7f8ff0168c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:24.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff0062c60 0x7f8ff0168c10 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7f8ff00c0910 tx=0x7f8fd8015000).stop 2020-11-20T12:22:24.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 -- 172.21.15.116:0/254264360 shutdown_connections 2020-11-20T12:22:24.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fcc0557c0 0x7f8fcc057c80 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8ff00bfe20 0x7f8ff01521d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8ff0062880 0x7f8ff01554b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 --2- 172.21.15.116:0/254264360 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8ff0062c60 0x7f8ff0168c10 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:24.963 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 -- 172.21.15.116:0/254264360 >> 172.21.15.116:0/254264360 conn(0x7f8ff0060c40 msgr2=0x7f8ff00a85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:24.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 -- 172.21.15.116:0/254264360 shutdown_connections 2020-11-20T12:22:24.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:24.751+0000 7f8ff497f700 1 -- 172.21.15.116:0/254264360 wait complete. 2020-11-20T12:22:24.964 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 68 2020-11-20T12:22:24.964 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:24.964 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":68,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":6893,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6835/1612795957","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6834","nonce":1612795957},{"type":"v1","addr":"172.21.15.116:6835","nonce":1612795957}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8011,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6835/3255680781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":3255680781},{"type":"v1","addr":"172.21.15.173:6835","nonce":3255680781}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":63},{"gid":8017,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/4058046293","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":4058046293},{"type":"v1","addr":"172.21.15.173:6837","nonce":4058046293}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":64}],"filesystems":[{"mdsmap":{"epoch":68,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:21:56.570302+0000","modified":"2020-11-20T12:21:58.589498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8148},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8148":{"gid":8148,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":5,"addr":"172.21.15.116:6833/2775705840","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":2775705840},{"type":"v1","addr":"172.21.15.116:6833","nonce":2775705840}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2020-11-20T12:22:24.965 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:25.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.147+0000 7f0305285700 1 Processor -- start 2020-11-20T12:22:25.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.147+0000 7f0305285700 1 -- start start 2020-11-20T12:22:25.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f0305285700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f030013b380 0x7f03000acd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:25.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f0305285700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f03000ad220 0x7f03000a8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:25.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f0305285700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03000ada00 con 0x7f030013afa0 2020-11-20T12:22:25.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f0305285700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03000adca0 con 0x7f03000ad220 2020-11-20T12:22:25.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f0305285700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03000adf40 con 0x7f030013b380 2020-11-20T12:22:25.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f02ff7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f030013b380 0x7f03000acd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:25.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f02ff7fe700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f030013b380 0x7f03000acd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:32874/0 (socket says 172.21.15.116:32874) 2020-11-20T12:22:25.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f02ff7fe700 1 -- 172.21.15.116:0/4269759768 learned_addr learned my addr 172.21.15.116:0/4269759768 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:25.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f0304a84700 1 --2- 172.21.15.116:0/4269759768 >> v2:172.21.15.173:3300/0 conn(0x7f03000ad220 0x7f03000a8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:25.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f02ff7fe700 1 -- 172.21.15.116:0/4269759768 >> v2:172.21.15.173:3300/0 conn(0x7f03000ad220 msgr2=0x7f03000a8790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:25.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f02ff7fe700 1 --2- 172.21.15.116:0/4269759768 >> v2:172.21.15.173:3300/0 conn(0x7f03000ad220 0x7f03000a8790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f02ff7fe700 1 -- 172.21.15.116:0/4269759768 >> v1:172.21.15.173:6789/0 conn(0x7f030013afa0 legacy=0x7f03000aa5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:25.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.151+0000 7f02ff7fe700 1 -- 172.21.15.116:0/4269759768 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03000ae1e0 con 0x7f030013b380 2020-11-20T12:22:25.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f02ff7fe700 1 --2- 172.21.15.116:0/4269759768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f030013b380 0x7f03000acd30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f02f4006fd0 tx=0x7f02f4006eb0).ready entity=mon.2 client_cookie=898150c16bb6b506 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:25.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f02feffd700 1 -- 172.21.15.116:0/4269759768 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f40097d0 con 0x7f030013b380 2020-11-20T12:22:25.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f02feffd700 1 -- 172.21.15.116:0/4269759768 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f4009930 con 0x7f030013b380 2020-11-20T12:22:25.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f02feffd700 1 -- 172.21.15.116:0/4269759768 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4009cb0 con 0x7f030013b380 2020-11-20T12:22:25.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f0305285700 1 -- 172.21.15.116:0/4269759768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f030013b380 msgr2=0x7f03000acd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:25.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f0305285700 1 --2- 172.21.15.116:0/4269759768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f030013b380 0x7f03000acd30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f02f4006fd0 tx=0x7f02f4006eb0).stop 2020-11-20T12:22:25.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f0305285700 1 -- 172.21.15.116:0/4269759768 shutdown_connections 2020-11-20T12:22:25.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f0305285700 1 --2- 172.21.15.116:0/4269759768 >> v2:172.21.15.173:3300/0 conn(0x7f03000ad220 0x7f03000a8790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f0305285700 1 --2- 172.21.15.116:0/4269759768 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f030013b380 0x7f03000acd30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f0305285700 1 -- 172.21.15.116:0/4269759768 >> 172.21.15.116:0/4269759768 conn(0x7f03000678a0 msgr2=0x7f0300067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:25.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f0305285700 1 -- 172.21.15.116:0/4269759768 shutdown_connections 2020-11-20T12:22:25.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.155+0000 7f0305285700 1 -- 172.21.15.116:0/4269759768 wait complete. 2020-11-20T12:22:25.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0305285700 1 Processor -- start 2020-11-20T12:22:25.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0305285700 1 -- start start 2020-11-20T12:22:25.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0305285700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03000ad220 0x7f0300169140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:25.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0305285700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f030013afa0 0x7f0300067010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:25.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0305285700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f030013b380 0x7f0300151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:25.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0305285700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0300152fe0 con 0x7f030013b380 2020-11-20T12:22:25.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0305285700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0300153210 con 0x7f030013afa0 2020-11-20T12:22:25.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0305285700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03001534b0 con 0x7f03000ad220 2020-11-20T12:22:25.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0304a84700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f030013b380 0x7f0300151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:25.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0304a84700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f030013b380 0x7f0300151260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52166/0 (socket says 172.21.15.116:52166) 2020-11-20T12:22:25.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0304a84700 1 -- 172.21.15.116:0/4092659462 learned_addr learned my addr 172.21.15.116:0/4092659462 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:25.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f02ff7fe700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f030013afa0 0x7f0300067010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:25.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f02fffff700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03000ad220 0x7f0300169140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:25.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0304a84700 1 -- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03000ad220 msgr2=0x7f0300169140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:25.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0304a84700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03000ad220 0x7f0300169140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0304a84700 1 -- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f030013afa0 msgr2=0x7f0300067010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:25.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0304a84700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f030013afa0 0x7f0300067010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f0304a84700 1 -- 172.21.15.116:0/4092659462 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0300168a00 con 0x7f030013b380 2020-11-20T12:22:25.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.159+0000 7f02ff7fe700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f030013afa0 0x7f0300067010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:25.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.163+0000 7f0304a84700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f030013b380 0x7f0300151260 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f02f800ef50 tx=0x7f02f80091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:25.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.163+0000 7f02dffff700 1 -- 172.21.15.116:0/4092659462 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f800cb40 con 0x7f030013b380 2020-11-20T12:22:25.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.163+0000 7f02dffff700 1 -- 172.21.15.116:0/4092659462 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f800cca0 con 0x7f030013b380 2020-11-20T12:22:25.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.163+0000 7f0305285700 1 -- 172.21.15.116:0/4092659462 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0300168ca0 con 0x7f030013b380 2020-11-20T12:22:25.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.163+0000 7f02dffff700 1 -- 172.21.15.116:0/4092659462 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f80123f0 con 0x7f030013b380 2020-11-20T12:22:25.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.163+0000 7f0305285700 1 -- 172.21.15.116:0/4092659462 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0300169630 con 0x7f030013b380 2020-11-20T12:22:25.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.163+0000 7f0305285700 1 -- 172.21.15.116:0/4092659462 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0300168e30 con 0x7f030013b380 2020-11-20T12:22:25.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.163+0000 7f02dffff700 1 -- 172.21.15.116:0/4092659462 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f02f8022070 con 0x7f030013b380 2020-11-20T12:22:25.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.167+0000 7f02dffff700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f02d80557c0 0x7f02d8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:25.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.167+0000 7f02dffff700 1 -- 172.21.15.116:0/4092659462 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f02f806ef10 con 0x7f030013b380 2020-11-20T12:22:25.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.167+0000 7f02fffff700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f02d80557c0 0x7f02d8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:25.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.167+0000 7f02fffff700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f02d80557c0 0x7f02d8057c80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f02f00036c0 tx=0x7f02f0007b70).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:25.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.171+0000 7f02dffff700 1 -- 172.21.15.116:0/4092659462 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f02f8044db0 con 0x7f030013b380 2020-11-20T12:22:25.336 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.331+0000 7f0305285700 1 -- 172.21.15.116:0/4092659462 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0300153670 con 0x7f030013b380 2020-11-20T12:22:25.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.331+0000 7f02dffff700 1 -- 172.21.15.116:0/4092659462 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f02f8016030 con 0x7f030013b380 2020-11-20T12:22:25.338 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:25.339 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":118,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:21:56.569994+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:21:53.361384+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2020-11-20T12:21:53.918177+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"118","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:25.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 -- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f02d80557c0 msgr2=0x7f02d8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:25.340 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f02d80557c0 0x7f02d8057c80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f02f00036c0 tx=0x7f02f0007b70).stop 2020-11-20T12:22:25.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 -- 172.21.15.116:0/4092659462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f030013b380 msgr2=0x7f0300151260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:25.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f030013b380 0x7f0300151260 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7f02f800ef50 tx=0x7f02f80091d0).stop 2020-11-20T12:22:25.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 -- 172.21.15.116:0/4092659462 shutdown_connections 2020-11-20T12:22:25.341 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f02d80557c0 0x7f02d8057c80 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f030013b380 0x7f0300151260 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f03000ad220 0x7f0300169140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 --2- 172.21.15.116:0/4092659462 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f030013afa0 0x7f0300067010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.342 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 -- 172.21.15.116:0/4092659462 >> 172.21.15.116:0/4092659462 conn(0x7f03000678a0 msgr2=0x7f03000a9b40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:25.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 -- 172.21.15.116:0/4092659462 shutdown_connections 2020-11-20T12:22:25.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.335+0000 7f0305285700 1 -- 172.21.15.116:0/4092659462 wait complete. 2020-11-20T12:22:25.363 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2020-11-20T12:22:25.364 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2020-11-20T12:22:25.533 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.527+0000 7f29efaa1700 1 Processor -- start 2020-11-20T12:22:25.534 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.527+0000 7f29efaa1700 1 -- start start 2020-11-20T12:22:25.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29efaa1700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f29e80a6c20 0x7f29e814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:25.535 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29efaa1700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e8150340 con 0x7f29e80a6840 2020-11-20T12:22:25.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29efaa1700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29e80677e0 con 0x7f29e8066cb0 2020-11-20T12:22:25.536 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29efaa1700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29e8067a80 con 0x7f29e80a6c20 2020-11-20T12:22:25.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29ed03c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f29e80a6c20 0x7f29e814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:25.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29ed83d700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f29e80a6840 0x7f29e80bff50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33374/0 (socket says 172.21.15.116:33374) 2020-11-20T12:22:25.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29ed83d700 1 -- 172.21.15.116:0/3005620166 learned_addr learned my addr 172.21.15.116:0/3005620166 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:25.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29ed03c700 1 -- 172.21.15.116:0/3005620166 >> v1:172.21.15.173:6789/0 conn(0x7f29e8066cb0 legacy=0x7f29e8153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:25.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29ed03c700 1 -- 172.21.15.116:0/3005620166 >> v1:172.21.15.116:6789/0 conn(0x7f29e80a6840 legacy=0x7f29e80bff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:25.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29ec83b700 1 -- 172.21.15.116:0/3005620166 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1545965138 0 0) 0x7f29e8150340 con 0x7f29e80a6840 2020-11-20T12:22:25.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29ed03c700 1 -- 172.21.15.116:0/3005620166 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e8067d20 con 0x7f29e80a6c20 2020-11-20T12:22:25.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29ed03c700 1 --2- 172.21.15.116:0/3005620166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29e80a6c20 0x7f29e814fe50 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f29d800b9a0 tx=0x7f29d8006df0).ready entity=mon.0 client_cookie=14ba5c3cdb12378c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:25.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29ec83b700 1 -- 172.21.15.116:0/3005620166 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8009770 con 0x7f29e80a6c20 2020-11-20T12:22:25.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29ec83b700 1 -- 172.21.15.116:0/3005620166 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29d80098d0 con 0x7f29e80a6c20 2020-11-20T12:22:25.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.531+0000 7f29ec83b700 1 -- 172.21.15.116:0/3005620166 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29d8009c50 con 0x7f29e80a6c20 2020-11-20T12:22:25.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.535+0000 7f29efaa1700 1 -- 172.21.15.116:0/3005620166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29e80a6c20 msgr2=0x7f29e814fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:25.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.535+0000 7f29efaa1700 1 --2- 172.21.15.116:0/3005620166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29e80a6c20 0x7f29e814fe50 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7f29d800b9a0 tx=0x7f29d8006df0).stop 2020-11-20T12:22:25.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.535+0000 7f29efaa1700 1 -- 172.21.15.116:0/3005620166 shutdown_connections 2020-11-20T12:22:25.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.535+0000 7f29efaa1700 1 --2- 172.21.15.116:0/3005620166 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29e80a6c20 0x7f29e814fe50 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.535+0000 7f29efaa1700 1 -- 172.21.15.116:0/3005620166 >> 172.21.15.116:0/3005620166 conn(0x7f29e8060170 msgr2=0x7f29e8060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:25.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.535+0000 7f29efaa1700 1 -- 172.21.15.116:0/3005620166 shutdown_connections 2020-11-20T12:22:25.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.535+0000 7f29efaa1700 1 -- 172.21.15.116:0/3005620166 wait complete. 2020-11-20T12:22:25.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.535+0000 7f29efaa1700 1 Processor -- start 2020-11-20T12:22:25.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.535+0000 7f29efaa1700 1 -- start start 2020-11-20T12:22:25.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29efaa1700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29e8066cb0 0x7f29e81625f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:25.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29efaa1700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e80a6840 0x7f29e8090fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:25.544 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29efaa1700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29e80a6c20 0x7f29e80c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:25.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29efaa1700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e8091490 con 0x7f29e80a6c20 2020-11-20T12:22:25.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29efaa1700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29e8162c20 con 0x7f29e80a6840 2020-11-20T12:22:25.545 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29efaa1700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29e8162ec0 con 0x7f29e8066cb0 2020-11-20T12:22:25.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29ee03e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29e80a6c20 0x7f29e80c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:25.546 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29ed03c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e80a6840 0x7f29e8090fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29ee03e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29e80a6c20 0x7f29e80c11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52180/0 (socket says 172.21.15.116:52180) 2020-11-20T12:22:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29ed83d700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29e8066cb0 0x7f29e81625f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29ee03e700 1 -- 172.21.15.116:0/3563756172 learned_addr learned my addr 172.21.15.116:0/3563756172 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29ee03e700 1 -- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29e8066cb0 msgr2=0x7f29e81625f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:25.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29ee03e700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29e8066cb0 0x7f29e81625f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29ee03e700 1 -- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e80a6840 msgr2=0x7f29e8090fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:25.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29ee03e700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e80a6840 0x7f29e8090fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:25.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29ee03e700 1 -- 172.21.15.116:0/3563756172 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29e8163160 con 0x7f29e80a6c20 2020-11-20T12:22:25.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.539+0000 7f29ee03e700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29e80a6c20 0x7f29e80c11d0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f29e000bae0 tx=0x7f29e00090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:25.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.543+0000 7f29d5ffb700 1 -- 172.21.15.116:0/3563756172 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e000c700 con 0x7f29e80a6c20 2020-11-20T12:22:25.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.543+0000 7f29d5ffb700 1 -- 172.21.15.116:0/3563756172 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e000c860 con 0x7f29e80a6c20 2020-11-20T12:22:25.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.543+0000 7f29d5ffb700 1 -- 172.21.15.116:0/3563756172 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e000ca10 con 0x7f29e80a6c20 2020-11-20T12:22:25.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.543+0000 7f29efaa1700 1 -- 172.21.15.116:0/3563756172 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29e815f130 con 0x7f29e80a6c20 2020-11-20T12:22:25.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.543+0000 7f29efaa1700 1 -- 172.21.15.116:0/3563756172 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29e80c16c0 con 0x7f29e80a6c20 2020-11-20T12:22:25.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.543+0000 7f29efaa1700 1 -- 172.21.15.116:0/3563756172 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29e805d2a0 con 0x7f29e80a6c20 2020-11-20T12:22:25.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.543+0000 7f29d5ffb700 1 -- 172.21.15.116:0/3563756172 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f29e001c070 con 0x7f29e80a6c20 2020-11-20T12:22:25.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.547+0000 7f29d5ffb700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f29c00557c0 0x7f29c0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:25.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.547+0000 7f29d5ffb700 1 -- 172.21.15.116:0/3563756172 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f29e006dbb0 con 0x7f29e80a6c20 2020-11-20T12:22:25.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.547+0000 7f29ed83d700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f29c00557c0 0x7f29c0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:25.553 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.547+0000 7f29ed83d700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f29c00557c0 0x7f29c0057c80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f29e8159640 tx=0x7f29e4007a00).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:25.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.551+0000 7f29d5ffb700 1 -- 172.21.15.116:0/3563756172 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f29e0074080 con 0x7f29e80a6c20 2020-11-20T12:22:25.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:25.759+0000 7f29efaa1700 1 -- 172.21.15.116:0/3563756172 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f29e80c3260 con 0x7f29e80a6c20 2020-11-20T12:22:26.455 INFO:tasks.ceph.mon.a.smithi116.stderr:2020-11-20T12:22:26.431+0000 7fc111d1b700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2020-11-20T12:22:26.465 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.459+0000 7f29d5ffb700 1 -- 172.21.15.116:0/3563756172 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v69) v1 ==== 161+0+0 (secure 0 0 0) 0x7f29e001a030 con 0x7f29e80a6c20 2020-11-20T12:22:26.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 -- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f29c00557c0 msgr2=0x7f29c0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:26.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f29c00557c0 0x7f29c0057c80 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f29e8159640 tx=0x7f29e4007a00).stop 2020-11-20T12:22:26.471 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 -- 172.21.15.116:0/3563756172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29e80a6c20 msgr2=0x7f29e80c11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:26.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29e80a6c20 0x7f29e80c11d0 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f29e000bae0 tx=0x7f29e00090f0).stop 2020-11-20T12:22:26.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 -- 172.21.15.116:0/3563756172 shutdown_connections 2020-11-20T12:22:26.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f29c00557c0 0x7f29c0057c80 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.472 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f29e80a6c20 0x7f29e80c11d0 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f29e8066cb0 0x7f29e81625f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 --2- 172.21.15.116:0/3563756172 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f29e80a6840 0x7f29e8090fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 -- 172.21.15.116:0/3563756172 >> 172.21.15.116:0/3563756172 conn(0x7f29e8060170 msgr2=0x7f29e80c34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 -- 172.21.15.116:0/3563756172 shutdown_connections 2020-11-20T12:22:26.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.463+0000 7f29efaa1700 1 -- 172.21.15.116:0/3563756172 wait complete. 2020-11-20T12:22:26.474 INFO:tasks.ceph.mds.a.smithi116.stderr: -1504> 2020-11-20T12:21:58.384+0000 7f01f5c20700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:22:26.474 INFO:teuthology.orchestra.run.smithi116.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2020-11-20T12:22:26.494 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2020-11-20T12:22:26.516 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:22:26.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.655+0000 7fe21e50a700 1 Processor -- start 2020-11-20T12:22:26.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.655+0000 7fe21e50a700 1 -- start start 2020-11-20T12:22:26.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe21e50a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2180bd8e0 0x7fe2180c09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe21e50a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe2180bdcc0 0x7fe2181534f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe21e50a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2181539e0 con 0x7fe218062880 2020-11-20T12:22:26.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe21e50a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe218153c70 con 0x7fe2180bd8e0 2020-11-20T12:22:26.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe21e50a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe218153f10 con 0x7fe2180bdcc0 2020-11-20T12:22:26.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe217fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2180bd8e0 0x7fe2180c09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:26.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe217fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2180bd8e0 0x7fe2180c09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52196/0 (socket says 172.21.15.116:52196) 2020-11-20T12:22:26.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe217fff700 1 -- 172.21.15.116:0/1720131599 learned_addr learned my addr 172.21.15.116:0/1720131599 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:26.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe2177fe700 1 --2- 172.21.15.116:0/1720131599 >> v2:172.21.15.173:3300/0 conn(0x7fe2180bdcc0 0x7fe2181534f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:26.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe217fff700 1 -- 172.21.15.116:0/1720131599 >> v2:172.21.15.173:3300/0 conn(0x7fe2180bdcc0 msgr2=0x7fe2181534f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:26.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe217fff700 1 --2- 172.21.15.116:0/1720131599 >> v2:172.21.15.173:3300/0 conn(0x7fe2180bdcc0 0x7fe2181534f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe217fff700 1 -- 172.21.15.116:0/1720131599 >> v1:172.21.15.173:6789/0 conn(0x7fe218062880 legacy=0x7fe2181505b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:26.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe217fff700 1 -- 172.21.15.116:0/1720131599 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2181541b0 con 0x7fe2180bd8e0 2020-11-20T12:22:26.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe2177fe700 1 --2- 172.21.15.116:0/1720131599 >> v2:172.21.15.173:3300/0 conn(0x7fe2180bdcc0 0x7fe2181534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:26.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.659+0000 7fe217fff700 1 --2- 172.21.15.116:0/1720131599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2180bd8e0 0x7fe2180c09a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7fe218064c50 tx=0x7fe208006eb0).ready entity=mon.0 client_cookie=81b7554b4fa65947 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:26.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe216ffd700 1 -- 172.21.15.116:0/1720131599 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2080098d0 con 0x7fe2180bd8e0 2020-11-20T12:22:26.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe216ffd700 1 -- 172.21.15.116:0/1720131599 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208009a30 con 0x7fe2180bd8e0 2020-11-20T12:22:26.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe216ffd700 1 -- 172.21.15.116:0/1720131599 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208009db0 con 0x7fe2180bd8e0 2020-11-20T12:22:26.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe21e50a700 1 -- 172.21.15.116:0/1720131599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2180bd8e0 msgr2=0x7fe2180c09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:26.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe21e50a700 1 --2- 172.21.15.116:0/1720131599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2180bd8e0 0x7fe2180c09a0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7fe218064c50 tx=0x7fe208006eb0).stop 2020-11-20T12:22:26.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe21e50a700 1 -- 172.21.15.116:0/1720131599 shutdown_connections 2020-11-20T12:22:26.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe21e50a700 1 --2- 172.21.15.116:0/1720131599 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2180bd8e0 0x7fe2180c09a0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe21e50a700 1 --2- 172.21.15.116:0/1720131599 >> v2:172.21.15.173:3300/0 conn(0x7fe2180bdcc0 0x7fe2181534f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe21e50a700 1 -- 172.21.15.116:0/1720131599 >> 172.21.15.116:0/1720131599 conn(0x7fe21805ba80 msgr2=0x7fe2180c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:26.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe21e50a700 1 -- 172.21.15.116:0/1720131599 shutdown_connections 2020-11-20T12:22:26.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe21e50a700 1 -- 172.21.15.116:0/1720131599 wait complete. 2020-11-20T12:22:26.682 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe21e50a700 1 Processor -- start 2020-11-20T12:22:26.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.663+0000 7fe21e50a700 1 -- start start 2020-11-20T12:22:26.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe21e50a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe218062880 0x7fe2180ba7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe21e50a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2180bdcc0 0x7fe2180bbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe21e50a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2180b9400 0x7fe21814e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe21e50a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180b98a0 con 0x7fe218062880 2020-11-20T12:22:26.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe21e50a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180b9b40 con 0x7fe2180b9400 2020-11-20T12:22:26.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe21e50a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2180b9de0 con 0x7fe2180bdcc0 2020-11-20T12:22:26.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe217fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe218062880 0x7fe2180ba7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:26.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe217fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe218062880 0x7fe2180ba7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52204/0 (socket says 172.21.15.116:52204) 2020-11-20T12:22:26.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe217fff700 1 -- 172.21.15.116:0/1602394025 learned_addr learned my addr 172.21.15.116:0/1602394025 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:26.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe2177fe700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2180bdcc0 0x7fe2180bbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:26.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe21caa7700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2180b9400 0x7fe21814e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:26.706 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe217fff700 1 -- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2180bdcc0 msgr2=0x7fe2180bbff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:26.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe217fff700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2180bdcc0 0x7fe2180bbff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe217fff700 1 -- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2180b9400 msgr2=0x7fe21814e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:26.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe217fff700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2180b9400 0x7fe21814e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:26.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe217fff700 1 -- 172.21.15.116:0/1602394025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2180ba080 con 0x7fe218062880 2020-11-20T12:22:26.707 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe217fff700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe218062880 0x7fe2180ba7c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7fe208006fd0 tx=0x7fe208015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:26.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe1f7fff700 1 -- 172.21.15.116:0/1602394025 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe208011c80 con 0x7fe218062880 2020-11-20T12:22:26.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe1f7fff700 1 -- 172.21.15.116:0/1602394025 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe208011de0 con 0x7fe218062880 2020-11-20T12:22:26.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe21e50a700 1 -- 172.21.15.116:0/1602394025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2180ba320 con 0x7fe218062880 2020-11-20T12:22:26.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe1f7fff700 1 -- 172.21.15.116:0/1602394025 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe20801e6b0 con 0x7fe218062880 2020-11-20T12:22:26.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe21e50a700 1 -- 172.21.15.116:0/1602394025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2180bc4e0 con 0x7fe218062880 2020-11-20T12:22:26.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.667+0000 7fe21e50a700 1 -- 172.21.15.116:0/1602394025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21805d2a0 con 0x7fe218062880 2020-11-20T12:22:26.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.671+0000 7fe1f7fff700 1 -- 172.21.15.116:0/1602394025 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe208025030 con 0x7fe218062880 2020-11-20T12:22:26.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.671+0000 7fe1f7fff700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1f00557c0 0x7fe1f0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:26.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.671+0000 7fe1f7fff700 1 -- 172.21.15.116:0/1602394025 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe20806fce0 con 0x7fe218062880 2020-11-20T12:22:26.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.671+0000 7fe2177fe700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1f00557c0 0x7fe1f0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:26.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.671+0000 7fe2177fe700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1f00557c0 0x7fe1f0057c80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fe218152c40 tx=0x7fe20c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:26.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.675+0000 7fe1f7fff700 1 -- 172.21.15.116:0/1602394025 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe208045b30 con 0x7fe218062880 2020-11-20T12:22:26.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:26.883+0000 7fe21e50a700 1 -- 172.21.15.116:0/1602394025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe218061380 con 0x7fe218062880 2020-11-20T12:22:27.469 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.463+0000 7fe1f7fff700 1 -- 172.21.15.116:0/1602394025 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v70) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe208023020 con 0x7fe218062880 2020-11-20T12:22:27.473 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 -- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1f00557c0 msgr2=0x7fe1f0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:27.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1f00557c0 0x7fe1f0057c80 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7fe218152c40 tx=0x7fe20c008040).stop 2020-11-20T12:22:27.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 -- 172.21.15.116:0/1602394025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe218062880 msgr2=0x7fe2180ba7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:27.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe218062880 0x7fe2180ba7c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7fe208006fd0 tx=0x7fe208015000).stop 2020-11-20T12:22:27.474 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 -- 172.21.15.116:0/1602394025 shutdown_connections 2020-11-20T12:22:27.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe1f00557c0 0x7fe1f0057c80 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2180b9400 0x7fe21814e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe218062880 0x7fe2180ba7c0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 --2- 172.21.15.116:0/1602394025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2180bdcc0 0x7fe2180bbff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.475 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 -- 172.21.15.116:0/1602394025 >> 172.21.15.116:0/1602394025 conn(0x7fe21805ba80 msgr2=0x7fe21814ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:27.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 -- 172.21.15.116:0/1602394025 shutdown_connections 2020-11-20T12:22:27.476 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.467+0000 7fe21e50a700 1 -- 172.21.15.116:0/1602394025 wait complete. 2020-11-20T12:22:27.499 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2020-11-20T12:22:27.694 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.687+0000 7f2ac6f46700 1 Processor -- start 2020-11-20T12:22:27.696 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.687+0000 7f2ac6f46700 1 -- start start 2020-11-20T12:22:27.697 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.691+0000 7f2ac6f46700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 0x7f2ac0155f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:27.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.691+0000 7f2ac6f46700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2ac014c050 0x7f2ac00c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:27.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.691+0000 7f2ac6f46700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ac00c3d10 con 0x7f2ac00c5260 2020-11-20T12:22:27.698 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.691+0000 7f2ac6f46700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ac00a81b0 con 0x7f2ac014c050 2020-11-20T12:22:27.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.691+0000 7f2ac6f46700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00c3fa0 con 0x7f2ac014bc70 2020-11-20T12:22:27.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2ac4ce2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 0x7f2ac0155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:27.699 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2abffff700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f2ac014c050 0x7f2ac00c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:27.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2ac54e3700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f2ac00c5260 0x7f2ac0061570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:57880/0 (socket says 172.21.15.116:57880) 2020-11-20T12:22:27.700 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2ac54e3700 1 -- 172.21.15.116:0/2316231852 learned_addr learned my addr 172.21.15.116:0/2316231852 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:27.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2abf7fe700 1 -- 172.21.15.116:0/2316231852 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 162655944 0 0) 0x7f2ac00c3d10 con 0x7f2ac00c5260 2020-11-20T12:22:27.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2ac4ce2700 1 -- 172.21.15.116:0/2316231852 >> v2:172.21.15.173:3300/0 conn(0x7f2ac014c050 msgr2=0x7f2ac00c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:27.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2ac4ce2700 1 --2- 172.21.15.116:0/2316231852 >> v2:172.21.15.173:3300/0 conn(0x7f2ac014c050 0x7f2ac00c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.701 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2ac4ce2700 1 -- 172.21.15.116:0/2316231852 >> v1:172.21.15.173:6789/0 conn(0x7f2ac00c5260 legacy=0x7f2ac0061570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:27.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2ac4ce2700 1 -- 172.21.15.116:0/2316231852 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac0151fb0 con 0x7f2ac014bc70 2020-11-20T12:22:27.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2ac4ce2700 1 --2- 172.21.15.116:0/2316231852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 0x7f2ac0155f60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f2ab400b9a0 tx=0x7f2ab4006df0).ready entity=mon.2 client_cookie=e70172c50429bb09 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:27.702 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2abf7fe700 1 -- 172.21.15.116:0/2316231852 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab40097b0 con 0x7f2ac014bc70 2020-11-20T12:22:27.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2abf7fe700 1 -- 172.21.15.116:0/2316231852 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab4009910 con 0x7f2ac014bc70 2020-11-20T12:22:27.703 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.695+0000 7f2abf7fe700 1 -- 172.21.15.116:0/2316231852 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4009c90 con 0x7f2ac014bc70 2020-11-20T12:22:27.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.699+0000 7f2ac6f46700 1 -- 172.21.15.116:0/2316231852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 msgr2=0x7f2ac0155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:27.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.699+0000 7f2ac6f46700 1 --2- 172.21.15.116:0/2316231852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 0x7f2ac0155f60 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f2ab400b9a0 tx=0x7f2ab4006df0).stop 2020-11-20T12:22:27.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.699+0000 7f2ac6f46700 1 -- 172.21.15.116:0/2316231852 shutdown_connections 2020-11-20T12:22:27.704 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.699+0000 7f2ac6f46700 1 --2- 172.21.15.116:0/2316231852 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 0x7f2ac0155f60 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.699+0000 7f2ac6f46700 1 --2- 172.21.15.116:0/2316231852 >> v2:172.21.15.173:3300/0 conn(0x7f2ac014c050 0x7f2ac00c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.699+0000 7f2ac6f46700 1 -- 172.21.15.116:0/2316231852 >> 172.21.15.116:0/2316231852 conn(0x7f2ac00678a0 msgr2=0x7f2ac0067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:27.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.699+0000 7f2ac6f46700 1 -- 172.21.15.116:0/2316231852 shutdown_connections 2020-11-20T12:22:27.705 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.699+0000 7f2ac6f46700 1 -- 172.21.15.116:0/2316231852 wait complete. 2020-11-20T12:22:27.708 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac6f46700 1 Processor -- start 2020-11-20T12:22:27.709 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac6f46700 1 -- start start 2020-11-20T12:22:27.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac6f46700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac00c5260 0x7f2ac00c12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:27.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac6f46700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 0x7f2ac0147950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:27.710 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac6f46700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac014c050 0x7f2ac014e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:27.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac6f46700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac014ece0 con 0x7f2ac00c5260 2020-11-20T12:22:27.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac6f46700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00c0620 con 0x7f2ac014c050 2020-11-20T12:22:27.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac6f46700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac00c08c0 con 0x7f2ac014bc70 2020-11-20T12:22:27.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac4ce2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac00c5260 0x7f2ac00c12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:27.711 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac4ce2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac00c5260 0x7f2ac00c12a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52218/0 (socket says 172.21.15.116:52218) 2020-11-20T12:22:27.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac4ce2700 1 -- 172.21.15.116:0/1115878177 learned_addr learned my addr 172.21.15.116:0/1115878177 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:27.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2abffff700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 0x7f2ac0147950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:27.712 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac54e3700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac014c050 0x7f2ac014e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:27.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac4ce2700 1 -- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 msgr2=0x7f2ac0147950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:27.713 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac4ce2700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 0x7f2ac0147950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.703+0000 7f2ac4ce2700 1 -- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac014c050 msgr2=0x7f2ac014e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:27.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2ac4ce2700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac014c050 0x7f2ac014e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:27.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2ac4ce2700 1 -- 172.21.15.116:0/1115878177 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac00c0b60 con 0x7f2ac00c5260 2020-11-20T12:22:27.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2abffff700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 0x7f2ac0147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:27.714 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2ac4ce2700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac00c5260 0x7f2ac00c12a0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f2ab4006e30 tx=0x7f2ab4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:27.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2abcff9700 1 -- 172.21.15.116:0/1115878177 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab4011e50 con 0x7f2ac00c5260 2020-11-20T12:22:27.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2abcff9700 1 -- 172.21.15.116:0/1115878177 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ab401e460 con 0x7f2ac00c5260 2020-11-20T12:22:27.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2abcff9700 1 -- 172.21.15.116:0/1115878177 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ab401e7e0 con 0x7f2ac00c5260 2020-11-20T12:22:27.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2ac6f46700 1 -- 172.21.15.116:0/1115878177 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac00c0e00 con 0x7f2ac00c5260 2020-11-20T12:22:27.715 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2ac6f46700 1 -- 172.21.15.116:0/1115878177 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac0147e40 con 0x7f2ac00c5260 2020-11-20T12:22:27.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2ac6f46700 1 -- 172.21.15.116:0/1115878177 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac00a7c90 con 0x7f2ac00c5260 2020-11-20T12:22:27.716 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2abcff9700 1 -- 172.21.15.116:0/1115878177 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f2ab4025030 con 0x7f2ac00c5260 2020-11-20T12:22:27.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2abcff9700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a980557c0 0x7f2a98057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:27.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.707+0000 7f2abcff9700 1 -- 172.21.15.116:0/1115878177 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2ab406feb0 con 0x7f2ac00c5260 2020-11-20T12:22:27.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.711+0000 7f2abffff700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a980557c0 0x7f2a98057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:27.717 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.711+0000 7f2abffff700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a980557c0 0x7f2a98057c80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f2ac014e0c0 tx=0x7f2aac008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:27.721 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.715+0000 7f2abcff9700 1 -- 172.21.15.116:0/1115878177 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f2ac00a7c90 con 0x7f2ac00c5260 2020-11-20T12:22:27.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:27.875+0000 7f2ac6f46700 1 -- 172.21.15.116:0/1115878177 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2ac00a7c90 con 0x7f2ac00c5260 2020-11-20T12:22:28.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.551+0000 7f2abcff9700 1 -- 172.21.15.116:0/1115878177 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v120) v1 ==== 181+0+0 (secure 0 0 0) 0x7f2ac00a7c90 con 0x7f2ac00c5260 2020-11-20T12:22:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 -- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a980557c0 msgr2=0x7f2a98057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a980557c0 0x7f2a98057c80 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f2ac014e0c0 tx=0x7f2aac008040).stop 2020-11-20T12:22:28.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 -- 172.21.15.116:0/1115878177 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac00c5260 msgr2=0x7f2ac00c12a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac00c5260 0x7f2ac00c12a0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f2ab4006e30 tx=0x7f2ab4015000).stop 2020-11-20T12:22:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 -- 172.21.15.116:0/1115878177 shutdown_connections 2020-11-20T12:22:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f2a980557c0 0x7f2a98057c80 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f2ac014c050 0x7f2ac014e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:28.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f2ac00c5260 0x7f2ac00c12a0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 --2- 172.21.15.116:0/1115878177 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f2ac014bc70 0x7f2ac0147950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 -- 172.21.15.116:0/1115878177 >> 172.21.15.116:0/1115878177 conn(0x7f2ac00678a0 msgr2=0x7f2ac0148780 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:28.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 -- 172.21.15.116:0/1115878177 shutdown_connections 2020-11-20T12:22:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.555+0000 7f2ac6f46700 1 -- 172.21.15.116:0/1115878177 wait complete. 2020-11-20T12:22:28.562 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' removed 2020-11-20T12:22:28.585 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2020-11-20T12:22:28.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.755+0000 7fe2c5622700 1 Processor -- start 2020-11-20T12:22:28.762 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.755+0000 7fe2c5622700 1 -- start start 2020-11-20T12:22:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.759+0000 7fe2c5622700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe2c014f270 0x7fe2c00aab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.759+0000 7fe2c5622700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe2c0155db0 0x7fe2c00acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.759+0000 7fe2c5622700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c0156210 con 0x7fe2c0152ac0 2020-11-20T12:22:28.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.759+0000 7fe2c5622700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c00ab150 con 0x7fe2c014f270 2020-11-20T12:22:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.759+0000 7fe2c5622700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c01520e0 con 0x7fe2c0155db0 2020-11-20T12:22:28.766 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.759+0000 7fe2b7fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe2c0152ac0 0x7fe2c00a9e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33430/0 (socket says 172.21.15.116:33430) 2020-11-20T12:22:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.759+0000 7fe2b7fff700 1 -- 172.21.15.116:0/633713691 learned_addr learned my addr 172.21.15.116:0/633713691 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.759+0000 7fe2bffff700 1 --2- 172.21.15.116:0/633713691 >> v2:172.21.15.116:3300/0 conn(0x7fe2c014f270 0x7fe2c00aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.759+0000 7fe2c4e21700 1 --2- 172.21.15.116:0/633713691 >> v2:172.21.15.173:3300/0 conn(0x7fe2c0155db0 0x7fe2c00acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:28.767 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.759+0000 7fe2bf7fe700 1 -- 172.21.15.116:0/633713691 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 723424059 0 0) 0x7fe2c0156210 con 0x7fe2c0152ac0 2020-11-20T12:22:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.759+0000 7fe2bf7fe700 1 -- 172.21.15.116:0/633713691 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a0003240 con 0x7fe2c0152ac0 2020-11-20T12:22:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2bffff700 1 -- 172.21.15.116:0/633713691 >> v2:172.21.15.173:3300/0 conn(0x7fe2c0155db0 msgr2=0x7fe2c00acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2bffff700 1 --2- 172.21.15.116:0/633713691 >> v2:172.21.15.173:3300/0 conn(0x7fe2c0155db0 0x7fe2c00acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:28.768 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2bffff700 1 -- 172.21.15.116:0/633713691 >> v1:172.21.15.116:6789/0 conn(0x7fe2c0152ac0 legacy=0x7fe2c00a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:28.769 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2bffff700 1 -- 172.21.15.116:0/633713691 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c0152380 con 0x7fe2c014f270 2020-11-20T12:22:28.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2bffff700 1 --2- 172.21.15.116:0/633713691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c014f270 0x7fe2c00aab20 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7fe2c00a9ba0 tx=0x7fe2b0006df0).ready entity=mon.0 client_cookie=532a80f44a47c015 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:28.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2bf7fe700 1 -- 172.21.15.116:0/633713691 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0009640 con 0x7fe2c014f270 2020-11-20T12:22:28.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2bf7fe700 1 -- 172.21.15.116:0/633713691 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b00097a0 con 0x7fe2c014f270 2020-11-20T12:22:28.770 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2bf7fe700 1 -- 172.21.15.116:0/633713691 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0009b20 con 0x7fe2c014f270 2020-11-20T12:22:28.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2c5622700 1 -- 172.21.15.116:0/633713691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c014f270 msgr2=0x7fe2c00aab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:28.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2c5622700 1 --2- 172.21.15.116:0/633713691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c014f270 0x7fe2c00aab20 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7fe2c00a9ba0 tx=0x7fe2b0006df0).stop 2020-11-20T12:22:28.771 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2c5622700 1 -- 172.21.15.116:0/633713691 shutdown_connections 2020-11-20T12:22:28.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2c5622700 1 --2- 172.21.15.116:0/633713691 >> v2:172.21.15.173:3300/0 conn(0x7fe2c0155db0 0x7fe2c00acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:28.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2c5622700 1 --2- 172.21.15.116:0/633713691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c014f270 0x7fe2c00aab20 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:28.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2c5622700 1 -- 172.21.15.116:0/633713691 >> 172.21.15.116:0/633713691 conn(0x7fe2c00678a0 msgr2=0x7fe2c0067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:28.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2c5622700 1 -- 172.21.15.116:0/633713691 shutdown_connections 2020-11-20T12:22:28.772 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.763+0000 7fe2c5622700 1 -- 172.21.15.116:0/633713691 wait complete. 2020-11-20T12:22:28.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2c5622700 1 Processor -- start 2020-11-20T12:22:28.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2c5622700 1 -- start start 2020-11-20T12:22:28.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2c5622700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c014f270 0x7fe2c014ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:28.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2c5622700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2c0152ac0 0x7fe2c0164e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:28.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2c5622700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c0155db0 0x7fe2c014f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:28.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2c5622700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c014ffd0 con 0x7fe2c014f270 2020-11-20T12:22:28.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2c5622700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c0150270 con 0x7fe2c0155db0 2020-11-20T12:22:28.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2c5622700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c0164700 con 0x7fe2c0152ac0 2020-11-20T12:22:28.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c014f270 0x7fe2c014ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:28.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c014f270 0x7fe2c014ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52232/0 (socket says 172.21.15.116:52232) 2020-11-20T12:22:28.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bffff700 1 -- 172.21.15.116:0/287390868 learned_addr learned my addr 172.21.15.116:0/287390868 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:28.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2c4e21700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c0155db0 0x7fe2c014f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:28.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2b7fff700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2c0152ac0 0x7fe2c0164e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:28.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bffff700 1 -- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2c0152ac0 msgr2=0x7fe2c0164e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:28.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bffff700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2c0152ac0 0x7fe2c0164e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:28.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bffff700 1 -- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c0155db0 msgr2=0x7fe2c014f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:28.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bffff700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c0155db0 0x7fe2c014f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:28.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bffff700 1 -- 172.21.15.116:0/287390868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c01649a0 con 0x7fe2c014f270 2020-11-20T12:22:28.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2c4e21700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c0155db0 0x7fe2c014f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:28.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2b7fff700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2c0152ac0 0x7fe2c0164e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:28.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bffff700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c014f270 0x7fe2c014ab40 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7fe2b0009340 tx=0x7fe2b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:28.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bcff9700 1 -- 172.21.15.116:0/287390868 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0011d70 con 0x7fe2c014f270 2020-11-20T12:22:28.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bcff9700 1 -- 172.21.15.116:0/287390868 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b001e460 con 0x7fe2c014f270 2020-11-20T12:22:28.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2c5622700 1 -- 172.21.15.116:0/287390868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c014b030 con 0x7fe2c014f270 2020-11-20T12:22:28.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.767+0000 7fe2bcff9700 1 -- 172.21.15.116:0/287390868 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b001e7e0 con 0x7fe2c014f270 2020-11-20T12:22:28.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.771+0000 7fe2c5622700 1 -- 172.21.15.116:0/287390868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c01654d0 con 0x7fe2c014f270 2020-11-20T12:22:28.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.771+0000 7fe2c5622700 1 -- 172.21.15.116:0/287390868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c005cf30 con 0x7fe2c014f270 2020-11-20T12:22:28.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.771+0000 7fe2bcff9700 1 -- 172.21.15.116:0/287390868 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe2b0025030 con 0x7fe2c014f270 2020-11-20T12:22:28.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.771+0000 7fe2bcff9700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe29c0557c0 0x7fe29c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:28.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.771+0000 7fe2bcff9700 1 -- 172.21.15.116:0/287390868 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5721+0+0 (secure 0 0 0) 0x7fe2b006fd00 con 0x7fe2c014f270 2020-11-20T12:22:28.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.771+0000 7fe2b7fff700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe29c0557c0 0x7fe29c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:28.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.775+0000 7fe2b7fff700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe29c0557c0 0x7fe29c057c80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fe2c00a98f0 tx=0x7fe2a8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:28.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.779+0000 7fe2bcff9700 1 -- 172.21.15.116:0/287390868 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe2b0073030 con 0x7fe2c014f270 2020-11-20T12:22:28.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:28.935+0000 7fe2c5622700 1 -- 172.21.15.116:0/287390868 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2c0060f00 con 0x7fe2c014f270 2020-11-20T12:22:29.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.567+0000 7fe2bcff9700 1 -- 172.21.15.116:0/287390868 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v121) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe2b0023070 con 0x7fe2c014f270 2020-11-20T12:22:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 -- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe29c0557c0 msgr2=0x7fe29c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:29.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe29c0557c0 0x7fe29c057c80 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fe2c00a98f0 tx=0x7fe2a8009040).stop 2020-11-20T12:22:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 -- 172.21.15.116:0/287390868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c014f270 msgr2=0x7fe2c014ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c014f270 0x7fe2c014ab40 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7fe2b0009340 tx=0x7fe2b0015000).stop 2020-11-20T12:22:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 -- 172.21.15.116:0/287390868 shutdown_connections 2020-11-20T12:22:29.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe29c0557c0 0x7fe29c057c80 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe2c0155db0 0x7fe2c014f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe2c014f270 0x7fe2c014ab40 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 --2- 172.21.15.116:0/287390868 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe2c0152ac0 0x7fe2c0164e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 -- 172.21.15.116:0/287390868 >> 172.21.15.116:0/287390868 conn(0x7fe2c00678a0 msgr2=0x7fe2c00a9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:29.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 -- 172.21.15.116:0/287390868 shutdown_connections 2020-11-20T12:22:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.571+0000 7fe2c5622700 1 -- 172.21.15.116:0/287390868 wait complete. 2020-11-20T12:22:29.579 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' removed 2020-11-20T12:22:29.602 INFO:tasks.ceph.mds.a:Restarting daemon 2020-11-20T12:22:29.602 INFO:tasks.ceph.mds.a:Stopping old one... 2020-11-20T12:22:29.603 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2020-11-20T12:22:29.603 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:22:29.604 INFO:tasks.ceph.mds.c:Restarting daemon 2020-11-20T12:22:29.604 INFO:tasks.ceph.mds.c:Stopping old one... 2020-11-20T12:22:29.604 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2020-11-20T12:22:29.605 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:22:29.605 INFO:tasks.ceph.mds.b:Restarting daemon 2020-11-20T12:22:29.606 INFO:tasks.ceph.mds.b:Stopping old one... 2020-11-20T12:22:29.606 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2020-11-20T12:22:29.606 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:22:29.607 INFO:tasks.ceph.mds.d:Restarting daemon 2020-11-20T12:22:29.607 INFO:tasks.ceph.mds.d:Stopping old one... 2020-11-20T12:22:29.608 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2020-11-20T12:22:29.608 INFO:teuthology.orchestra.run:waiting for 300 2020-11-20T12:22:29.627 INFO:tasks.ceph.mds.a:Stopped 2020-11-20T12:22:29.629 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2020-11-20T12:22:29.632 INFO:tasks.ceph.mds.c:Stopped 2020-11-20T12:22:29.632 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2020-11-20T12:22:29.634 INFO:tasks.ceph.mds.d:Stopped 2020-11-20T12:22:29.635 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2020-11-20T12:22:29.636 INFO:tasks.ceph.mds.b:Stopped 2020-11-20T12:22:29.637 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2020-11-20T12:22:29.642 INFO:tasks.ceph.mds.d:Started 2020-11-20T12:22:29.643 DEBUG:teuthology.parallel:result is None 2020-11-20T12:22:29.645 INFO:tasks.ceph.mds.b:Started 2020-11-20T12:22:29.646 DEBUG:teuthology.parallel:result is None 2020-11-20T12:22:29.683 INFO:tasks.ceph.mds.a:Started 2020-11-20T12:22:29.684 DEBUG:teuthology.parallel:result is None 2020-11-20T12:22:29.688 INFO:tasks.ceph.mds.c:Started 2020-11-20T12:22:29.688 DEBUG:teuthology.parallel:result is None 2020-11-20T12:22:29.689 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2020-11-20T12:22:29.747 INFO:tasks.ceph.mds.d.smithi173.stdout:starting mds.d at 2020-11-20T12:22:29.751 INFO:tasks.ceph.mds.b.smithi173.stdout:starting mds.b at 2020-11-20T12:22:29.797 INFO:tasks.ceph.mds.a.smithi116.stdout:starting mds.a at 2020-11-20T12:22:29.798 INFO:tasks.ceph.mds.c.smithi116.stdout:starting mds.c at 2020-11-20T12:22:29.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.863+0000 7f72eaf59700 1 Processor -- start 2020-11-20T12:22:29.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.863+0000 7f72eaf59700 1 -- start start 2020-11-20T12:22:29.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72eaf59700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f72e4151d70 0x7f72e4066e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:29.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72eaf59700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e4068670 con 0x7f72e4151990 2020-11-20T12:22:29.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72eaf59700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e40683d0 con 0x7f72e40677e0 2020-11-20T12:22:29.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72eaf59700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e4066480 con 0x7f72e4151d70 2020-11-20T12:22:29.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e3fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f72e4151d70 0x7f72e4066e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:29.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e3fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f72e4151d70 0x7f72e4066e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52270/0 (socket says 172.21.15.116:52270) 2020-11-20T12:22:29.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e3fff700 1 -- 172.21.15.116:0/459963112 learned_addr learned my addr 172.21.15.116:0/459963112 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:29.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e8cf5700 1 --1- 172.21.15.116:0/459963112 >> v1:172.21.15.116:6789/0 conn(0x7f72e4151990 0x7f72e40bedc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33472/0 (socket says 172.21.15.116:33472) 2020-11-20T12:22:29.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e3fff700 1 -- 172.21.15.116:0/459963112 >> v1:172.21.15.173:6789/0 conn(0x7f72e40677e0 legacy=0x7f72e41556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:29.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e3fff700 1 -- 172.21.15.116:0/459963112 >> v1:172.21.15.116:6789/0 conn(0x7f72e4151990 legacy=0x7f72e40bedc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:29.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e3fff700 1 -- 172.21.15.116:0/459963112 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4066720 con 0x7f72e4151d70 2020-11-20T12:22:29.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e37fe700 1 -- 172.21.15.116:0/459963112 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635846342 0 0) 0x7f72e4068670 con 0x7f72e4151990 2020-11-20T12:22:29.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e3fff700 1 --2- 172.21.15.116:0/459963112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72e4151d70 0x7f72e4066e60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f72d0006fd0 tx=0x7f72d0006eb0).ready entity=mon.0 client_cookie=a91ff88cbf7078c8 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:29.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e37fe700 1 -- 172.21.15.116:0/459963112 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d00098d0 con 0x7f72e4151d70 2020-11-20T12:22:29.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e37fe700 1 -- 172.21.15.116:0/459963112 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0009a30 con 0x7f72e4151d70 2020-11-20T12:22:29.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72e37fe700 1 -- 172.21.15.116:0/459963112 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0009db0 con 0x7f72e4151d70 2020-11-20T12:22:29.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72eaf59700 1 -- 172.21.15.116:0/459963112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72e4151d70 msgr2=0x7f72e4066e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:29.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72eaf59700 1 --2- 172.21.15.116:0/459963112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72e4151d70 0x7f72e4066e60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f72d0006fd0 tx=0x7f72d0006eb0).stop 2020-11-20T12:22:29.876 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72eaf59700 1 -- 172.21.15.116:0/459963112 shutdown_connections 2020-11-20T12:22:29.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72eaf59700 1 --2- 172.21.15.116:0/459963112 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72e4151d70 0x7f72e4066e60 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:29.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72eaf59700 1 -- 172.21.15.116:0/459963112 >> 172.21.15.116:0/459963112 conn(0x7f72e4060170 msgr2=0x7f72e4060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:29.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72eaf59700 1 -- 172.21.15.116:0/459963112 shutdown_connections 2020-11-20T12:22:29.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.867+0000 7f72eaf59700 1 -- 172.21.15.116:0/459963112 wait complete. 2020-11-20T12:22:29.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72eaf59700 1 Processor -- start 2020-11-20T12:22:29.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72eaf59700 1 -- start start 2020-11-20T12:22:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72eaf59700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72e40677e0 0x7f72e4150be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72eaf59700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e4151990 0x7f72e4154440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72eaf59700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72e4151d70 0x7f72e4091a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72eaf59700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e414f1a0 con 0x7f72e40677e0 2020-11-20T12:22:29.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72eaf59700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e414e280 con 0x7f72e4151990 2020-11-20T12:22:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72eaf59700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e414e520 con 0x7f72e4151d70 2020-11-20T12:22:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72e8cf5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72e40677e0 0x7f72e4150be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72e8cf5700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72e40677e0 0x7f72e4150be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52276/0 (socket says 172.21.15.116:52276) 2020-11-20T12:22:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72e8cf5700 1 -- 172.21.15.116:0/3998862900 learned_addr learned my addr 172.21.15.116:0/3998862900 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:29.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72e3fff700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e4151990 0x7f72e4154440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72e94f6700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72e4151d70 0x7f72e4091a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72e8cf5700 1 -- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72e4151d70 msgr2=0x7f72e4091a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72e8cf5700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72e4151d70 0x7f72e4091a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72e8cf5700 1 -- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e4151990 msgr2=0x7f72e4154440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:29.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72e8cf5700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e4151990 0x7f72e4154440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.871+0000 7f72e8cf5700 1 -- 172.21.15.116:0/3998862900 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e414e7c0 con 0x7f72e40677e0 2020-11-20T12:22:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.875+0000 7f72e8cf5700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72e40677e0 0x7f72e4150be0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f72d800c710 tx=0x7f72d8007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.875+0000 7f72e0ff9700 1 -- 172.21.15.116:0/3998862900 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d8007320 con 0x7f72e40677e0 2020-11-20T12:22:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.875+0000 7f72e0ff9700 1 -- 172.21.15.116:0/3998862900 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d8005530 con 0x7f72e40677e0 2020-11-20T12:22:29.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.875+0000 7f72eaf59700 1 -- 172.21.15.116:0/3998862900 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e414ea60 con 0x7f72e40677e0 2020-11-20T12:22:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.875+0000 7f72e0ff9700 1 -- 172.21.15.116:0/3998862900 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d80058b0 con 0x7f72e40677e0 2020-11-20T12:22:29.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.875+0000 7f72eaf59700 1 -- 172.21.15.116:0/3998862900 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e4150740 con 0x7f72e40677e0 2020-11-20T12:22:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.875+0000 7f72eaf59700 1 -- 172.21.15.116:0/3998862900 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e405d2a0 con 0x7f72e40677e0 2020-11-20T12:22:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.875+0000 7f72e0ff9700 1 -- 172.21.15.116:0/3998862900 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f72d8010030 con 0x7f72e40677e0 2020-11-20T12:22:29.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.879+0000 7f72e0ff9700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72bc0557c0 0x7f72bc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:29.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.879+0000 7f72e0ff9700 1 -- 172.21.15.116:0/3998862900 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5339+0+0 (secure 0 0 0) 0x7f72d806e220 con 0x7f72e40677e0 2020-11-20T12:22:29.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.879+0000 7f72e3fff700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72bc0557c0 0x7f72bc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:29.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.879+0000 7f72e3fff700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72bc0557c0 0x7f72bc057c80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f72e41502e0 tx=0x7f72d0015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:29.888 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:29.883+0000 7f72e0ff9700 1 -- 172.21.15.116:0/3998862900 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f72d8044230 con 0x7f72e40677e0 2020-11-20T12:22:30.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.055+0000 7f72eaf59700 1 -- 172.21.15.116:0/3998862900 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f72e40612a0 con 0x7f72e40677e0 2020-11-20T12:22:30.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.799+0000 7f72e0ff9700 1 -- 172.21.15.116:0/3998862900 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v123) v1 ==== 97+0+0 (secure 0 0 0) 0x7f72d8047850 con 0x7f72e40677e0 2020-11-20T12:22:30.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 -- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72bc0557c0 msgr2=0x7f72bc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:30.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72bc0557c0 0x7f72bc057c80 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f72e41502e0 tx=0x7f72d0015000).stop 2020-11-20T12:22:30.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 -- 172.21.15.116:0/3998862900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72e40677e0 msgr2=0x7f72e4150be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:30.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72e40677e0 0x7f72e4150be0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f72d800c710 tx=0x7f72d8007ae0).stop 2020-11-20T12:22:30.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 -- 172.21.15.116:0/3998862900 shutdown_connections 2020-11-20T12:22:30.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72bc0557c0 0x7f72bc057c80 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:30.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f72e4151d70 0x7f72e4091a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:30.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f72e40677e0 0x7f72e4150be0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:30.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 --2- 172.21.15.116:0/3998862900 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f72e4151990 0x7f72e4154440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:30.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 -- 172.21.15.116:0/3998862900 >> 172.21.15.116:0/3998862900 conn(0x7f72e4060170 msgr2=0x7f72e40c24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:30.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 -- 172.21.15.116:0/3998862900 shutdown_connections 2020-11-20T12:22:30.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:30.807+0000 7f72eaf59700 1 -- 172.21.15.116:0/3998862900 wait complete. 2020-11-20T12:22:30.854 INFO:teuthology.orchestra.run.smithi116.stderr: removed all blocklist entries 2020-11-20T12:22:30.856 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2020-11-20T12:22:31.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.043+0000 7fe0c9d49700 1 Processor -- start 2020-11-20T12:22:31.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.047+0000 7fe0c9d49700 1 -- start start 2020-11-20T12:22:31.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.047+0000 7fe0c9d49700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c40a4b30 0x7fe0c40c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.047+0000 7fe0c9d49700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c40c53c0 con 0x7fe0c40c6090 2020-11-20T12:22:31.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.047+0000 7fe0c9d49700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0c40c42c0 con 0x7fe0c40a4750 2020-11-20T12:22:31.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.047+0000 7fe0c9d49700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40c4560 con 0x7fe0c40a4b30 2020-11-20T12:22:31.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.047+0000 7fe0c2ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe0c40c6090 0x7fe0c40ba8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33488/0 (socket says 172.21.15.116:33488) 2020-11-20T12:22:31.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.047+0000 7fe0c2ffd700 1 -- 172.21.15.116:0/2882526446 learned_addr learned my addr 172.21.15.116:0/2882526446 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:31.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.047+0000 7fe0c3fff700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c40a4b30 0x7fe0c40c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.047+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 668324597 0 0) 0x7fe0c40c53c0 con 0x7fe0c40c6090 2020-11-20T12:22:31.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.047+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0b0003240 con 0x7fe0c40c6090 2020-11-20T12:22:31.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3314459424 0 0) 0x7fe0b0003240 con 0x7fe0c40c6090 2020-11-20T12:22:31.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c40a4b30 msgr2=0x7fe0c40c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:31.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c40a4b30 0x7fe0c40c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 >> v1:172.21.15.173:6789/0 conn(0x7fe0c40a4750 legacy=0x7fe0c4060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c40c4800 con 0x7fe0c40c6090 2020-11-20T12:22:31.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fe0b80014e0 con 0x7fe0c40c6090 2020-11-20T12:22:31.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 >> v1:172.21.15.116:6789/0 conn(0x7fe0c40c6090 legacy=0x7fe0c40ba8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c40a4b30 0x7fe0b0003760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0b0003c50 0x7fe0b00044e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0b00049d0 0x7fe0b0004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40c4560 con 0x7fe0b00049d0 2020-11-20T12:22:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0004170 con 0x7fe0b0003c50 2020-11-20T12:22:31.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b0004330 con 0x7fe0c40a4b30 2020-11-20T12:22:31.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c3fff700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0b00049d0 0x7fe0b0004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c37fe700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c40a4b30 0x7fe0b0003760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c2ffd700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0b0003c50 0x7fe0b00044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c3fff700 1 -- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c40a4b30 msgr2=0x7fe0b0003760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:31.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c3fff700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c40a4b30 0x7fe0b0003760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c3fff700 1 -- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0b0003c50 msgr2=0x7fe0b00044e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:31.070 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c3fff700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0b0003c50 0x7fe0b00044e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c3fff700 1 -- 172.21.15.116:0/2882526446 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b0003240 con 0x7fe0b00049d0 2020-11-20T12:22:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c2ffd700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0b0003c50 0x7fe0b00044e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c3fff700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0b00049d0 0x7fe0b0004e50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7fe0bc009ea0 tx=0x7fe0bc0091d0).ready entity=mon.0 client_cookie=74d89ca91ec983f3 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc0095c0 con 0x7fe0b00049d0 2020-11-20T12:22:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0bc00ed40 con 0x7fe0b00049d0 2020-11-20T12:22:31.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c27fc700 1 -- 172.21.15.116:0/2882526446 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0bc012600 con 0x7fe0b00049d0 2020-11-20T12:22:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c9d49700 1 -- 172.21.15.116:0/2882526446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0b00049d0 msgr2=0x7fe0b0004e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c9d49700 1 --2- 172.21.15.116:0/2882526446 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0b00049d0 0x7fe0b0004e50 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7fe0bc009ea0 tx=0x7fe0bc0091d0).stop 2020-11-20T12:22:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c3fff700 1 -- 172.21.15.116:0/2882526446 reap_dead start 2020-11-20T12:22:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c9d49700 1 -- 172.21.15.116:0/2882526446 shutdown_connections 2020-11-20T12:22:31.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.051+0000 7fe0c9d49700 1 -- 172.21.15.116:0/2882526446 >> 172.21.15.116:0/2882526446 conn(0x7fe0c4061a60 msgr2=0x7fe0c40c55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:31.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.055+0000 7fe0c9d49700 1 -- 172.21.15.116:0/2882526446 shutdown_connections 2020-11-20T12:22:31.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.055+0000 7fe0c9d49700 1 -- 172.21.15.116:0/2882526446 wait complete. 2020-11-20T12:22:31.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.055+0000 7fe0c9d49700 1 Processor -- start 2020-11-20T12:22:31.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.055+0000 7fe0c9d49700 1 -- start start 2020-11-20T12:22:31.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c9d49700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c414f8f0 0x7fe0c4154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c9d49700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c414fcd0 0x7fe0c40a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c9d49700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0c40a7bf0 0x7fe0c4152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c9d49700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c41537d0 con 0x7fe0c414f8f0 2020-11-20T12:22:31.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c9d49700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40a67e0 con 0x7fe0c40a7bf0 2020-11-20T12:22:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c9d49700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c40a6a80 con 0x7fe0c414fcd0 2020-11-20T12:22:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c37fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c414f8f0 0x7fe0c4154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c37fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c414f8f0 0x7fe0c4154b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52296/0 (socket says 172.21.15.116:52296) 2020-11-20T12:22:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c37fe700 1 -- 172.21.15.116:0/3503521248 learned_addr learned my addr 172.21.15.116:0/3503521248 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:31.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c2ffd700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c414fcd0 0x7fe0c40a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c3fff700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0c40a7bf0 0x7fe0c4152960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c37fe700 1 -- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c414fcd0 msgr2=0x7fe0c40a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c37fe700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c414fcd0 0x7fe0c40a7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c37fe700 1 -- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0c40a7bf0 msgr2=0x7fe0c4152960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c37fe700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0c40a7bf0 0x7fe0c4152960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c37fe700 1 -- 172.21.15.116:0/3503521248 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0c40a6d20 con 0x7fe0c414f8f0 2020-11-20T12:22:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c2ffd700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c414fcd0 0x7fe0c40a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c3fff700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0c40a7bf0 0x7fe0c4152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c37fe700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c414f8f0 0x7fe0c4154b00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7fe0b40010f0 tx=0x7fe0b4009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:31.080 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0a37fe700 1 -- 172.21.15.116:0/3503521248 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4010550 con 0x7fe0c414f8f0 2020-11-20T12:22:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0a37fe700 1 -- 172.21.15.116:0/3503521248 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b4010040 con 0x7fe0c414f8f0 2020-11-20T12:22:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c9d49700 1 -- 172.21.15.116:0/3503521248 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c40a6fc0 con 0x7fe0c414f8f0 2020-11-20T12:22:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0a37fe700 1 -- 172.21.15.116:0/3503521248 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b40108c0 con 0x7fe0c414f8f0 2020-11-20T12:22:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.059+0000 7fe0c9d49700 1 -- 172.21.15.116:0/3503521248 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c40a63c0 con 0x7fe0c414f8f0 2020-11-20T12:22:31.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.063+0000 7fe0c9d49700 1 -- 172.21.15.116:0/3503521248 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c40a7460 con 0x7fe0c414f8f0 2020-11-20T12:22:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.063+0000 7fe0a37fe700 1 -- 172.21.15.116:0/3503521248 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fe0b4010c70 con 0x7fe0c414f8f0 2020-11-20T12:22:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.063+0000 7fe0a37fe700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe09c0557c0 0x7fe09c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.063+0000 7fe0a37fe700 1 -- 172.21.15.116:0/3503521248 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fe0b406e650 con 0x7fe0c414f8f0 2020-11-20T12:22:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.063+0000 7fe0c2ffd700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe09c0557c0 0x7fe09c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.063+0000 7fe0c2ffd700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe09c0557c0 0x7fe09c057c80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fe0b8001e90 tx=0x7fe0b8009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:31.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.071+0000 7fe0a37fe700 1 -- 172.21.15.116:0/3503521248 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fe0c40a7460 con 0x7fe0c414f8f0 2020-11-20T12:22:31.288 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.283+0000 7fe0c9d49700 1 -- 172.21.15.116:0/3503521248 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe0c40a7460 con 0x7fe0c414f8f0 2020-11-20T12:22:31.289 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.283+0000 7fe0a37fe700 1 -- 172.21.15.116:0/3503521248 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v34) v1 ==== 80+0+5399 (secure 0 0 0) 0x7fe0c40a7460 con 0x7fe0c414f8f0 2020-11-20T12:22:31.290 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:31.290 INFO:teuthology.orchestra.run.smithi116.stdout:{ 2020-11-20T12:22:31.291 INFO:teuthology.orchestra.run.smithi116.stdout: "auth_dump": [ 2020-11-20T12:22:31.291 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.292 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.a", 2020-11-20T12:22:31.292 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfSZFZGhAAXikxkjclpO3y7Q72PXYljw==", 2020-11-20T12:22:31.292 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.293 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:22:31.293 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.293 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.294 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.294 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.294 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.295 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.296 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.b", 2020-11-20T12:22:31.296 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df8Uo9KxAAQC7tCv9i6UW1fHDLTOObbQ==", 2020-11-20T12:22:31.296 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.296 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:22:31.297 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.297 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.297 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.297 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.297 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.298 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.298 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.c", 2020-11-20T12:22:31.298 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfk2nvIxAAW+LhxX+jbejIBvLjHw+3ag==", 2020-11-20T12:22:31.298 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.298 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:22:31.299 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.299 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.299 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.299 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.299 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.300 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.300 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mds.d", 2020-11-20T12:22:31.300 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfGnxcMhAAkDKLU+5JanZnYF3vE56kBQ==", 2020-11-20T12:22:31.300 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.301 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:22:31.301 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.301 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.301 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.301 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.302 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.302 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.302 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.0", 2020-11-20T12:22:31.302 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBHs7dfrAl8OxAAOohGk6gSW1R1pQv/NZanuQ==", 2020-11-20T12:22:31.302 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.303 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.303 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.303 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.303 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.303 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.304 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.304 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.1", 2020-11-20T12:22:31.304 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBLs7df7iVOCxAATjPCx6ElkzQ8UQtWir+V0Q==", 2020-11-20T12:22:31.304 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.304 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.305 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.305 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.305 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.305 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.309 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.309 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.2", 2020-11-20T12:22:31.309 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBOs7dfbA4AFRAAfrez1683gMQgpcL4RGNZbw==", 2020-11-20T12:22:31.309 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.310 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.310 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.310 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.310 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.310 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.311 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.311 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.3", 2020-11-20T12:22:31.311 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBRs7dfpW82HhAAYkW9vSEbR3GMPfKTSxbE8w==", 2020-11-20T12:22:31.311 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.311 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.312 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.312 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.312 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.312 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.313 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.313 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.4", 2020-11-20T12:22:31.314 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBXs7dfLYQCGBAAJIfx7QjW2XCXGN1AwXZgKA==", 2020-11-20T12:22:31.314 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.315 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.315 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.315 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.316 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.316 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.316 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.317 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.5", 2020-11-20T12:22:31.318 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBas7df7SoIIhAAtXT5oxWqG8t+clkPQaVcAQ==", 2020-11-20T12:22:31.318 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.318 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.319 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.319 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.319 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.320 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.320 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.320 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.6", 2020-11-20T12:22:31.321 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBes7dfdjE+OBAA3prtRRXRjHbFJuQgUrDxSg==", 2020-11-20T12:22:31.321 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.321 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.322 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.322 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.322 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.323 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.323 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.323 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "osd.7", 2020-11-20T12:22:31.323 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBis7dfcSdkBhAAJTkJDv9JzYfbUYiRelXNNg==", 2020-11-20T12:22:31.324 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.324 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.324 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.325 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.325 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.325 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.326 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.326 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.0", 2020-11-20T12:22:31.327 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfqupsOBAA7i4JdA6Yzee7aM8ecEB0YA==", 2020-11-20T12:22:31.327 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.328 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:22:31.328 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow r", 2020-11-20T12:22:31.328 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw pool=cephfs_data" 2020-11-20T12:22:31.329 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.329 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.329 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.330 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.1", 2020-11-20T12:22:31.330 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfGnnRBhAAdXMPscjW9kg+WsdFIravow==", 2020-11-20T12:22:31.330 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.331 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:22:31.332 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:22:31.333 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:22:31.333 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:22:31.333 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.334 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.336 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.336 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.2", 2020-11-20T12:22:31.337 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dftmqLARAAfEOX6/TgjgzFB+mnxmOHIQ==", 2020-11-20T12:22:31.337 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.337 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:22:31.337 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:22:31.337 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:22:31.338 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:22:31.338 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.338 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.338 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.339 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.3", 2020-11-20T12:22:31.339 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBFs7dfHe2rCxAAa9EV2Pygf6vjCBMAs832AQ==", 2020-11-20T12:22:31.339 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.339 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow", 2020-11-20T12:22:31.339 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow r", 2020-11-20T12:22:31.340 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow rw", 2020-11-20T12:22:31.340 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rwx" 2020-11-20T12:22:31.340 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.340 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.340 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.341 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.admin", 2020-11-20T12:22:31.341 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBDs7dfh+KKOBAAZvcPXWly9/mxRE7NWuR85A==", 2020-11-20T12:22:31.341 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.341 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:22:31.341 INFO:teuthology.orchestra.run.smithi116.stdout: "mgr": "allow *", 2020-11-20T12:22:31.342 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.342 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.342 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.342 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.342 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.343 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "client.manila", 2020-11-20T12:22:31.343 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQD6tLdfuyhFBxAAL3B+R6VAZsxb6g9xsF8+wA==", 2020-11-20T12:22:31.343 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.343 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:22:31.343 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow *", 2020-11-20T12:22:31.344 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow rw" 2020-11-20T12:22:31.344 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.344 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.344 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.344 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.x", 2020-11-20T12:22:31.345 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7df/tKtFBAAfqk3phZmn6mkLLzRm4zQ8A==", 2020-11-20T12:22:31.345 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.345 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:22:31.345 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:22:31.346 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.346 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.346 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:31.346 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:31.346 INFO:teuthology.orchestra.run.smithi116.stdout: "entity": "mgr.y", 2020-11-20T12:22:31.347 INFO:teuthology.orchestra.run.smithi116.stdout: "key": "AQBEs7dfP2FRDxAA0xruM4xw7OvbduVlJ5NsoA==", 2020-11-20T12:22:31.347 INFO:teuthology.orchestra.run.smithi116.stdout: "caps": { 2020-11-20T12:22:31.347 INFO:teuthology.orchestra.run.smithi116.stdout: "mds": "allow *", 2020-11-20T12:22:31.349 INFO:teuthology.orchestra.run.smithi116.stdout: "mon": "allow profile mgr", 2020-11-20T12:22:31.349 INFO:teuthology.orchestra.run.smithi116.stdout: "osd": "allow *" 2020-11-20T12:22:31.349 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.350 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:31.350 INFO:teuthology.orchestra.run.smithi116.stdout: ] 2020-11-20T12:22:31.350 INFO:teuthology.orchestra.run.smithi116.stdout:} 2020-11-20T12:22:31.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 -- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe09c0557c0 msgr2=0x7fe09c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe09c0557c0 0x7fe09c057c80 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7fe0b8001e90 tx=0x7fe0b8009040).stop 2020-11-20T12:22:31.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 -- 172.21.15.116:0/3503521248 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c414f8f0 msgr2=0x7fe0c4154b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c414f8f0 0x7fe0c4154b00 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7fe0b40010f0 tx=0x7fe0b4009040).stop 2020-11-20T12:22:31.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 -- 172.21.15.116:0/3503521248 shutdown_connections 2020-11-20T12:22:31.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fe09c0557c0 0x7fe09c057c80 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0c40a7bf0 0x7fe0c4152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0c414f8f0 0x7fe0c4154b00 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 --2- 172.21.15.116:0/3503521248 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe0c414fcd0 0x7fe0c40a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 -- 172.21.15.116:0/3503521248 >> 172.21.15.116:0/3503521248 conn(0x7fe0c4061a60 msgr2=0x7fe0c40c2040 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:31.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 -- 172.21.15.116:0/3503521248 shutdown_connections 2020-11-20T12:22:31.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.287+0000 7fe0c9d49700 1 -- 172.21.15.116:0/3503521248 wait complete. 2020-11-20T12:22:31.357 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.manila 2020-11-20T12:22:31.420 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:22:31.428 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:22:31.540 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.535+0000 7fdf91c33700 1 Processor -- start 2020-11-20T12:22:31.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.535+0000 7fdf91c33700 1 -- start start 2020-11-20T12:22:31.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.535+0000 7fdf91c33700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdf8c0c8a50 0x7fdf8c0adcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.535+0000 7fdf91c33700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c0c8e30 0x7fdf8c0aa8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.547 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.535+0000 7fdf91c33700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf8c0ad0e0 con 0x7fdf8c0aada0 2020-11-20T12:22:31.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.535+0000 7fdf91c33700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8c0ad310 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.535+0000 7fdf91c33700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c0ad5b0 con 0x7fdf8c0c8e30 2020-11-20T12:22:31.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.535+0000 7fdf90c31700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdf8c0c8a50 0x7fdf8c0adcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.548 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.535+0000 7fdf90c31700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fdf8c0c8a50 0x7fdf8c0adcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52304/0 (socket says 172.21.15.116:52304) 2020-11-20T12:22:31.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.535+0000 7fdf90c31700 1 -- 172.21.15.116:0/1134090554 learned_addr learned my addr 172.21.15.116:0/1134090554 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:31.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf83fff700 1 --2- 172.21.15.116:0/1134090554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c0c8e30 0x7fdf8c0aa8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf90c31700 1 -- 172.21.15.116:0/1134090554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c0c8e30 msgr2=0x7fdf8c0aa8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:31.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf90c31700 1 --2- 172.21.15.116:0/1134090554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c0c8e30 0x7fdf8c0aa8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.549 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf90c31700 1 -- 172.21.15.116:0/1134090554 >> v1:172.21.15.173:6789/0 conn(0x7fdf8c0aada0 legacy=0x7fdf8c152010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf90c31700 1 -- 172.21.15.116:0/1134090554 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8c0ad850 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf90c31700 1 --2- 172.21.15.116:0/1134090554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf8c0c8a50 0x7fdf8c0adcf0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7fdf88006fd0 tx=0x7fdf88006eb0).ready entity=mon.0 client_cookie=3a9c995713bc97a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:31.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf837fe700 1 -- 172.21.15.116:0/1134090554 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf880098d0 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf837fe700 1 -- 172.21.15.116:0/1134090554 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf88009a30 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf837fe700 1 -- 172.21.15.116:0/1134090554 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf88009db0 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf91c33700 1 -- 172.21.15.116:0/1134090554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf8c0c8a50 msgr2=0x7fdf8c0adcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf91c33700 1 --2- 172.21.15.116:0/1134090554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf8c0c8a50 0x7fdf8c0adcf0 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7fdf88006fd0 tx=0x7fdf88006eb0).stop 2020-11-20T12:22:31.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf91c33700 1 -- 172.21.15.116:0/1134090554 shutdown_connections 2020-11-20T12:22:31.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf91c33700 1 --2- 172.21.15.116:0/1134090554 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf8c0c8a50 0x7fdf8c0adcf0 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf91c33700 1 --2- 172.21.15.116:0/1134090554 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c0c8e30 0x7fdf8c0aa8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf91c33700 1 -- 172.21.15.116:0/1134090554 >> 172.21.15.116:0/1134090554 conn(0x7fdf8c0615f0 msgr2=0x7fdf8c0619d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:31.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf91c33700 1 -- 172.21.15.116:0/1134090554 shutdown_connections 2020-11-20T12:22:31.552 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.539+0000 7fdf91c33700 1 -- 172.21.15.116:0/1134090554 wait complete. 2020-11-20T12:22:31.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.543+0000 7fdf91c33700 1 Processor -- start 2020-11-20T12:22:31.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.543+0000 7fdf91c33700 1 -- start start 2020-11-20T12:22:31.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.543+0000 7fdf91c33700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c0aada0 0x7fdf8c14db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.543+0000 7fdf91c33700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf8c0c8a50 0x7fdf8c157930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.555 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.543+0000 7fdf91c33700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf8c0c8e30 0x7fdf8c14c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.543+0000 7fdf91c33700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c157e20 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.543+0000 7fdf91c33700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c14cc40 con 0x7fdf8c0c8e30 2020-11-20T12:22:31.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.543+0000 7fdf91c33700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c14ead0 con 0x7fdf8c0aada0 2020-11-20T12:22:31.556 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.543+0000 7fdf83fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf8c0c8a50 0x7fdf8c157930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.543+0000 7fdf83fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf8c0c8a50 0x7fdf8c157930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52314/0 (socket says 172.21.15.116:52314) 2020-11-20T12:22:31.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.543+0000 7fdf83fff700 1 -- 172.21.15.116:0/3088742819 learned_addr learned my addr 172.21.15.116:0/3088742819 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:31.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf91432700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf8c0c8e30 0x7fdf8c14c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.557 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf90c31700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c0aada0 0x7fdf8c14db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf83fff700 1 -- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c0aada0 msgr2=0x7fdf8c14db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf83fff700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c0aada0 0x7fdf8c14db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf83fff700 1 -- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf8c0c8e30 msgr2=0x7fdf8c14c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf83fff700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf8c0c8e30 0x7fdf8c14c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.558 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf83fff700 1 -- 172.21.15.116:0/3088742819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8c14ed70 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf90c31700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c0aada0 0x7fdf8c14db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:31.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf83fff700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf8c0c8a50 0x7fdf8c157930 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7fdf7c00d840 tx=0x7fdf7c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:31.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf80ff9700 1 -- 172.21.15.116:0/3088742819 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c0125f0 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf80ff9700 1 -- 172.21.15.116:0/3088742819 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c012750 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf80ff9700 1 -- 172.21.15.116:0/3088742819 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c012ad0 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf91c33700 1 -- 172.21.15.116:0/3088742819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8c14f010 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf91c33700 1 -- 172.21.15.116:0/3088742819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8c14f750 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.547+0000 7fdf91c33700 1 -- 172.21.15.116:0/3088742819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8c14f9d0 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.551+0000 7fdf80ff9700 1 -- 172.21.15.116:0/3088742819 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fdf7c013480 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.551+0000 7fdf80ff9700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf640557f0 0x7fdf64057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:31.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.551+0000 7fdf80ff9700 1 -- 172.21.15.116:0/3088742819 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fdf7c06e5b0 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.551+0000 7fdf90c31700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf640557f0 0x7fdf64057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:31.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.551+0000 7fdf90c31700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf640557f0 0x7fdf64057cb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fdf8800b1b0 tx=0x7fdf88015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:31.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.555+0000 7fdf80ff9700 1 -- 172.21.15.116:0/3088742819 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fdf7c044720 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.775+0000 7fdf91c33700 1 -- 172.21.15.116:0/3088742819 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.manila"} v 0) v1 -- 0x7fdf8c05cd40 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.783+0000 7fdf80ff9700 1 -- 172.21.15.116:0/3088742819 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.manila"}]=0 updated v35) v1 ==== 90+0+0 (secure 0 0 0) 0x7fdf7c047d40 con 0x7fdf8c0c8a50 2020-11-20T12:22:31.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 -- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf640557f0 msgr2=0x7fdf64057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf640557f0 0x7fdf64057cb0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fdf8800b1b0 tx=0x7fdf88015000).stop 2020-11-20T12:22:31.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 -- 172.21.15.116:0/3088742819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf8c0c8a50 msgr2=0x7fdf8c157930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:31.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf8c0c8a50 0x7fdf8c157930 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7fdf7c00d840 tx=0x7fdf7c008040).stop 2020-11-20T12:22:31.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 -- 172.21.15.116:0/3088742819 shutdown_connections 2020-11-20T12:22:31.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdf640557f0 0x7fdf64057cb0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fdf8c0c8e30 0x7fdf8c14c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fdf8c0aada0 0x7fdf8c14db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 --2- 172.21.15.116:0/3088742819 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fdf8c0c8a50 0x7fdf8c157930 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:31.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 -- 172.21.15.116:0/3088742819 >> 172.21.15.116:0/3088742819 conn(0x7fdf8c0615f0 msgr2=0x7fdf8c152d80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:31.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 -- 172.21.15.116:0/3088742819 shutdown_connections 2020-11-20T12:22:31.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.787+0000 7fdf91c33700 1 -- 172.21.15.116:0/3088742819 wait complete. 2020-11-20T12:22:31.796 INFO:teuthology.orchestra.run.smithi116.stderr:updated 2020-11-20T12:22:31.819 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:31.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.991+0000 7f56248be700 1 Processor -- start 2020-11-20T12:22:31.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.991+0000 7f56248be700 1 -- start start 2020-11-20T12:22:32.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f56248be700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56200bfa80 0x7f56200bfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f56248be700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56200bac00 0x7f562014db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f56248be700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56200b9d20 con 0x7f56200bf100 2020-11-20T12:22:32.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f56248be700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56200b9f80 con 0x7f56200bfa80 2020-11-20T12:22:32.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f56248be700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56200ba220 con 0x7f56200bac00 2020-11-20T12:22:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f561e59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f56200bfa80 0x7f56200bfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f561dd9b700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f56200bf100 0x7f5620066e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33522/0 (socket says 172.21.15.116:33522) 2020-11-20T12:22:32.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f561dd9b700 1 -- 172.21.15.116:0/1980220093 learned_addr learned my addr 172.21.15.116:0/1980220093 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:32.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f561ed9d700 1 --2- 172.21.15.116:0/1980220093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56200bac00 0x7f562014db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f561e59c700 1 -- 172.21.15.116:0/1980220093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56200bac00 msgr2=0x7f562014db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:32.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f561e59c700 1 --2- 172.21.15.116:0/1980220093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56200bac00 0x7f562014db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f561e59c700 1 -- 172.21.15.116:0/1980220093 >> v1:172.21.15.116:6789/0 conn(0x7f56200bf100 legacy=0x7f5620066e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.995+0000 7f561e59c700 1 -- 172.21.15.116:0/1980220093 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200ba4c0 con 0x7f56200bfa80 2020-11-20T12:22:32.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f561e59c700 1 --2- 172.21.15.116:0/1980220093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56200bfa80 0x7f56200bfe60 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f561000b9a0 tx=0x7f5610006df0).ready entity=mon.0 client_cookie=d61ab4ae8e30f20c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:32.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f561d59a700 1 -- 172.21.15.116:0/1980220093 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56100097b0 con 0x7f56200bfa80 2020-11-20T12:22:32.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f561d59a700 1 -- 172.21.15.116:0/1980220093 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5610009910 con 0x7f56200bfa80 2020-11-20T12:22:32.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f561d59a700 1 -- 172.21.15.116:0/1980220093 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5610009c90 con 0x7f56200bfa80 2020-11-20T12:22:32.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f56248be700 1 -- 172.21.15.116:0/1980220093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56200bfa80 msgr2=0x7f56200bfe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f56248be700 1 --2- 172.21.15.116:0/1980220093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56200bfa80 0x7f56200bfe60 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f561000b9a0 tx=0x7f5610006df0).stop 2020-11-20T12:22:32.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f56248be700 1 -- 172.21.15.116:0/1980220093 shutdown_connections 2020-11-20T12:22:32.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f56248be700 1 --2- 172.21.15.116:0/1980220093 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56200bfa80 0x7f56200bfe60 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f56248be700 1 --2- 172.21.15.116:0/1980220093 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56200bac00 0x7f562014db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f56248be700 1 -- 172.21.15.116:0/1980220093 >> 172.21.15.116:0/1980220093 conn(0x7f56200c4f80 msgr2=0x7f56200c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:32.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f56248be700 1 -- 172.21.15.116:0/1980220093 shutdown_connections 2020-11-20T12:22:32.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f56248be700 1 -- 172.21.15.116:0/1980220093 wait complete. 2020-11-20T12:22:32.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:31.999+0000 7f56248be700 1 Processor -- start 2020-11-20T12:22:32.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f56248be700 1 -- start start 2020-11-20T12:22:32.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f56248be700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56200bac00 0x7f5620160b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f56248be700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56200bf100 0x7f56200bb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f56248be700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56200bfa80 0x7f56200bd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f56248be700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200bc430 con 0x7f56200bf100 2020-11-20T12:22:32.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f56248be700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56200a6420 con 0x7f56200bac00 2020-11-20T12:22:32.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f56248be700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56200a66c0 con 0x7f56200bfa80 2020-11-20T12:22:32.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561dd9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56200bf100 0x7f56200bb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561dd9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56200bf100 0x7f56200bb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52326/0 (socket says 172.21.15.116:52326) 2020-11-20T12:22:32.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561dd9b700 1 -- 172.21.15.116:0/3919056443 learned_addr learned my addr 172.21.15.116:0/3919056443 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:32.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561e59c700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56200bac00 0x7f5620160b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561ed9d700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56200bfa80 0x7f56200bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561dd9b700 1 -- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56200bfa80 msgr2=0x7f56200bd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561dd9b700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56200bfa80 0x7f56200bd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561dd9b700 1 -- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56200bac00 msgr2=0x7f5620160b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561dd9b700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56200bac00 0x7f5620160b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561dd9b700 1 -- 172.21.15.116:0/3919056443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200a6960 con 0x7f56200bf100 2020-11-20T12:22:32.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561e59c700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56200bac00 0x7f5620160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:32.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f561dd9b700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56200bf100 0x7f56200bb5f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f561400c710 tx=0x7f5614007be0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:32.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f560e7fc700 1 -- 172.21.15.116:0/3919056443 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56140072e0 con 0x7f56200bf100 2020-11-20T12:22:32.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f560e7fc700 1 -- 172.21.15.116:0/3919056443 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5614007440 con 0x7f56200bf100 2020-11-20T12:22:32.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f560e7fc700 1 -- 172.21.15.116:0/3919056443 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5614005800 con 0x7f56200bf100 2020-11-20T12:22:32.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f56248be700 1 -- 172.21.15.116:0/3919056443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5620161050 con 0x7f56200bf100 2020-11-20T12:22:32.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.003+0000 7f56248be700 1 -- 172.21.15.116:0/3919056443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56200a6e00 con 0x7f56200bf100 2020-11-20T12:22:32.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.007+0000 7f56248be700 1 -- 172.21.15.116:0/3919056443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56200a6270 con 0x7f56200bf100 2020-11-20T12:22:32.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.007+0000 7f560e7fc700 1 -- 172.21.15.116:0/3919056443 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5614010070 con 0x7f56200bf100 2020-11-20T12:22:32.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.007+0000 7f560e7fc700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55f40557c0 0x7f55f4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.017 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.007+0000 7f560e7fc700 1 -- 172.21.15.116:0/3919056443 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f561406da60 con 0x7f56200bf100 2020-11-20T12:22:32.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.011+0000 7f561e59c700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55f40557c0 0x7f55f4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.011+0000 7f561e59c700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55f40557c0 0x7f55f4057c80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f561000b150 tx=0x7f5610015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:32.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.015+0000 7f560e7fc700 1 -- 172.21.15.116:0/3919056443 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5614043c30 con 0x7f56200bf100 2020-11-20T12:22:32.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.175+0000 7f56248be700 1 -- 172.21.15.116:0/3919056443 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f562005d2a0 con 0x7f56200bf100 2020-11-20T12:22:32.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.175+0000 7f560e7fc700 1 -- 172.21.15.116:0/3919056443 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+13733 (secure 0 0 0) 0x7f5614047250 con 0x7f56200bf100 2020-11-20T12:22:32.182 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:32.183 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":123,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:22:30.794702+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":116,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":114,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:32.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 -- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55f40557c0 msgr2=0x7f55f4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55f40557c0 0x7f55f4057c80 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f561000b150 tx=0x7f5610015000).stop 2020-11-20T12:22:32.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 -- 172.21.15.116:0/3919056443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56200bf100 msgr2=0x7f56200bb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56200bf100 0x7f56200bb5f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f561400c710 tx=0x7f5614007be0).stop 2020-11-20T12:22:32.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 -- 172.21.15.116:0/3919056443 shutdown_connections 2020-11-20T12:22:32.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f55f40557c0 0x7f55f4057c80 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f56200bfa80 0x7f56200bd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f56200bac00 0x7f5620160b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 --2- 172.21.15.116:0/3919056443 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f56200bf100 0x7f56200bb5f0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 -- 172.21.15.116:0/3919056443 >> 172.21.15.116:0/3919056443 conn(0x7f56200c4f80 msgr2=0x7f562014cdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:32.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 -- 172.21.15.116:0/3919056443 shutdown_connections 2020-11-20T12:22:32.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.179+0000 7f56248be700 1 -- 172.21.15.116:0/3919056443 wait complete. 2020-11-20T12:22:32.210 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2020-11-20T12:22:32.211 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:22:32.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.387+0000 7f5493ec9700 1 Processor -- start 2020-11-20T12:22:32.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.387+0000 7f5493ec9700 1 -- start start 2020-11-20T12:22:32.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.391+0000 7f5493ec9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f548c14bc90 0x7f548c14ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.391+0000 7f5493ec9700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c063740 con 0x7f548c0601e0 2020-11-20T12:22:32.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.391+0000 7f5493ec9700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f548c0639e0 con 0x7f548c14b8b0 2020-11-20T12:22:32.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.391+0000 7f5493ec9700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f548c14f420 con 0x7f548c14bc90 2020-11-20T12:22:32.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.391+0000 7f5491464700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f548c14bc90 0x7f548c14ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.391+0000 7f5491464700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f548c14bc90 0x7f548c14ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52334/0 (socket says 172.21.15.116:52334) 2020-11-20T12:22:32.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.391+0000 7f5491464700 1 -- 172.21.15.116:0/3448308269 learned_addr learned my addr 172.21.15.116:0/3448308269 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:32.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5490c63700 1 -- 172.21.15.116:0/3448308269 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559303354 0 0) 0x7f548c0639e0 con 0x7f548c14b8b0 2020-11-20T12:22:32.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5491464700 1 -- 172.21.15.116:0/3448308269 >> v1:172.21.15.173:6789/0 conn(0x7f548c14b8b0 legacy=0x7f548c0a56f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5491464700 1 -- 172.21.15.116:0/3448308269 >> v1:172.21.15.116:6789/0 conn(0x7f548c0601e0 legacy=0x7f548c060690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5491464700 1 -- 172.21.15.116:0/3448308269 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c063c80 con 0x7f548c14bc90 2020-11-20T12:22:32.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5491464700 1 --2- 172.21.15.116:0/3448308269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f548c14bc90 0x7f548c14ef30 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f547c006fd0 tx=0x7f547c006eb0).ready entity=mon.0 client_cookie=f5a187d0a935c88 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:32.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5490c63700 1 -- 172.21.15.116:0/3448308269 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c0098d0 con 0x7f548c14bc90 2020-11-20T12:22:32.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5490c63700 1 -- 172.21.15.116:0/3448308269 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c009a30 con 0x7f548c14bc90 2020-11-20T12:22:32.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5490c63700 1 -- 172.21.15.116:0/3448308269 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c009db0 con 0x7f548c14bc90 2020-11-20T12:22:32.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5493ec9700 1 -- 172.21.15.116:0/3448308269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f548c14bc90 msgr2=0x7f548c14ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5493ec9700 1 --2- 172.21.15.116:0/3448308269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f548c14bc90 0x7f548c14ef30 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f547c006fd0 tx=0x7f547c006eb0).stop 2020-11-20T12:22:32.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5493ec9700 1 -- 172.21.15.116:0/3448308269 shutdown_connections 2020-11-20T12:22:32.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5493ec9700 1 --2- 172.21.15.116:0/3448308269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f548c14bc90 0x7f548c14ef30 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5493ec9700 1 -- 172.21.15.116:0/3448308269 >> 172.21.15.116:0/3448308269 conn(0x7f548c0c17c0 msgr2=0x7f548c0c1ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:32.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5493ec9700 1 -- 172.21.15.116:0/3448308269 shutdown_connections 2020-11-20T12:22:32.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.395+0000 7f5493ec9700 1 -- 172.21.15.116:0/3448308269 wait complete. 2020-11-20T12:22:32.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.399+0000 7f5493ec9700 1 Processor -- start 2020-11-20T12:22:32.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.399+0000 7f5493ec9700 1 -- start start 2020-11-20T12:22:32.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5493ec9700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c0601e0 0x7f548c0a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5493ec9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f548c14b8b0 0x7f548c13d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5493ec9700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f548c0a5d20 0x7f548c13da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5493ec9700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0a8190 con 0x7f548c14b8b0 2020-11-20T12:22:32.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5493ec9700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0a8430 con 0x7f548c0601e0 2020-11-20T12:22:32.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5493ec9700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f548c0a86d0 con 0x7f548c0a5d20 2020-11-20T12:22:32.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5491464700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f548c14b8b0 0x7f548c13d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5491464700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f548c14b8b0 0x7f548c13d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52340/0 (socket says 172.21.15.116:52340) 2020-11-20T12:22:32.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5491464700 1 -- 172.21.15.116:0/674060251 learned_addr learned my addr 172.21.15.116:0/674060251 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:32.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5492466700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f548c0a5d20 0x7f548c13da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5491c65700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c0601e0 0x7f548c0a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5491464700 1 -- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f548c0a5d20 msgr2=0x7f548c13da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5491464700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f548c0a5d20 0x7f548c13da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5491464700 1 -- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c0601e0 msgr2=0x7f548c0a5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5491464700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c0601e0 0x7f548c0a5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5491464700 1 -- 172.21.15.116:0/674060251 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c0a8970 con 0x7f548c14b8b0 2020-11-20T12:22:32.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5491464700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f548c14b8b0 0x7f548c13d560 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f548c061bf0 tx=0x7f547c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:32.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5479ffb700 1 -- 172.21.15.116:0/674060251 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c006760 con 0x7f548c14b8b0 2020-11-20T12:22:32.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5479ffb700 1 -- 172.21.15.116:0/674060251 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f547c01e460 con 0x7f548c14b8b0 2020-11-20T12:22:32.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5493ec9700 1 -- 172.21.15.116:0/674060251 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548c13df40 con 0x7f548c14b8b0 2020-11-20T12:22:32.418 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5479ffb700 1 -- 172.21.15.116:0/674060251 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547c01e820 con 0x7f548c14b8b0 2020-11-20T12:22:32.419 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.403+0000 7f5493ec9700 1 -- 172.21.15.116:0/674060251 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c15eec0 con 0x7f548c14b8b0 2020-11-20T12:22:32.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.407+0000 7f5493ec9700 1 -- 172.21.15.116:0/674060251 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c05d2a0 con 0x7f548c14b8b0 2020-11-20T12:22:32.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.407+0000 7f5479ffb700 1 -- 172.21.15.116:0/674060251 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f547c025030 con 0x7f548c14b8b0 2020-11-20T12:22:32.420 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.407+0000 7f5479ffb700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54640557c0 0x7f5464057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.407+0000 7f5479ffb700 1 -- 172.21.15.116:0/674060251 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f547c06fc50 con 0x7f548c14b8b0 2020-11-20T12:22:32.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.407+0000 7f5491c65700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54640557c0 0x7f5464057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.421 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.411+0000 7f5491c65700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54640557c0 0x7f5464057c80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f5488003460 tx=0x7f5488009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:32.422 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.415+0000 7f5479ffb700 1 -- 172.21.15.116:0/674060251 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f547c045d10 con 0x7f548c14b8b0 2020-11-20T12:22:32.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.571+0000 7f5493ec9700 1 -- 172.21.15.116:0/674060251 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f548c0c15a0 con 0x7f548c14b8b0 2020-11-20T12:22:32.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5479ffb700 1 -- 172.21.15.116:0/674060251 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v123) v1 ==== 110+0+10 (secure 0 0 0) 0x7f547c023070 con 0x7f548c14b8b0 2020-11-20T12:22:32.581 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:22:32.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 -- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54640557c0 msgr2=0x7f5464057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54640557c0 0x7f5464057c80 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f5488003460 tx=0x7f5488009040).stop 2020-11-20T12:22:32.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 -- 172.21.15.116:0/674060251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f548c14b8b0 msgr2=0x7f548c13d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f548c14b8b0 0x7f548c13d560 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f548c061bf0 tx=0x7f547c015000).stop 2020-11-20T12:22:32.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 -- 172.21.15.116:0/674060251 shutdown_connections 2020-11-20T12:22:32.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f54640557c0 0x7f5464057c80 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f548c0a5d20 0x7f548c13da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f548c0601e0 0x7f548c0a5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 --2- 172.21.15.116:0/674060251 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f548c14b8b0 0x7f548c13d560 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 -- 172.21.15.116:0/674060251 >> 172.21.15.116:0/674060251 conn(0x7f548c0c17c0 msgr2=0x7f548c14cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:32.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 -- 172.21.15.116:0/674060251 shutdown_connections 2020-11-20T12:22:32.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.575+0000 7f5493ec9700 1 -- 172.21.15.116:0/674060251 wait complete. 2020-11-20T12:22:32.606 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:22:32.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.783+0000 7fd25b59e700 1 Processor -- start 2020-11-20T12:22:32.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.783+0000 7fd25b59e700 1 -- start start 2020-11-20T12:22:32.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.787+0000 7fd25b59e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd25c0aa7d0 0x7fd25c0a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.787+0000 7fd25b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd25c0ad540 0x7fd25c0ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.787+0000 7fd25b59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd25c0ace40 con 0x7fd25c0aabb0 2020-11-20T12:22:32.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.787+0000 7fd25b59e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd25c1526c0 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.787+0000 7fd25b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c152960 con 0x7fd25c0ad540 2020-11-20T12:22:32.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25a59c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fd25c0aa7d0 0x7fd25c0a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd259d9b700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7fd25c0aabb0 0x7fd25c151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:58012/0 (socket says 172.21.15.116:58012) 2020-11-20T12:22:32.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd259d9b700 1 -- 172.21.15.116:0/4048235798 learned_addr learned my addr 172.21.15.116:0/4048235798 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:32.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25ad9d700 1 --2- 172.21.15.116:0/4048235798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd25c0ad540 0x7fd25c0ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25a59c700 1 -- 172.21.15.116:0/4048235798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd25c0ad540 msgr2=0x7fd25c0ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:32.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25959a700 1 -- 172.21.15.116:0/4048235798 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487710517 0 0) 0x7fd25c0ace40 con 0x7fd25c0aabb0 2020-11-20T12:22:32.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25a59c700 1 --2- 172.21.15.116:0/4048235798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd25c0ad540 0x7fd25c0ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25a59c700 1 -- 172.21.15.116:0/4048235798 >> v1:172.21.15.173:6789/0 conn(0x7fd25c0aabb0 legacy=0x7fd25c151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25a59c700 1 -- 172.21.15.116:0/4048235798 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c152c00 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25a59c700 1 --2- 172.21.15.116:0/4048235798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd25c0aa7d0 0x7fd25c0a8400 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fd25c0acfd0 tx=0x7fd24c006df0).ready entity=mon.1 client_cookie=4e7efa78aae16772 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:32.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25959a700 1 -- 172.21.15.116:0/4048235798 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c0097b0 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25959a700 1 -- 172.21.15.116:0/4048235798 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c009910 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25959a700 1 -- 172.21.15.116:0/4048235798 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c009c90 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25b59e700 1 -- 172.21.15.116:0/4048235798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd25c0aa7d0 msgr2=0x7fd25c0a8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.791+0000 7fd25b59e700 1 --2- 172.21.15.116:0/4048235798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd25c0aa7d0 0x7fd25c0a8400 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fd25c0acfd0 tx=0x7fd24c006df0).stop 2020-11-20T12:22:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.795+0000 7fd25b59e700 1 -- 172.21.15.116:0/4048235798 shutdown_connections 2020-11-20T12:22:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.795+0000 7fd25b59e700 1 --2- 172.21.15.116:0/4048235798 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd25c0ad540 0x7fd25c0ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.795+0000 7fd25b59e700 1 --2- 172.21.15.116:0/4048235798 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd25c0aa7d0 0x7fd25c0a8400 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.795+0000 7fd25b59e700 1 -- 172.21.15.116:0/4048235798 >> 172.21.15.116:0/4048235798 conn(0x7fd25c0678a0 msgr2=0x7fd25c067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:32.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.795+0000 7fd25b59e700 1 -- 172.21.15.116:0/4048235798 shutdown_connections 2020-11-20T12:22:32.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.795+0000 7fd25b59e700 1 -- 172.21.15.116:0/4048235798 wait complete. 2020-11-20T12:22:32.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.795+0000 7fd25b59e700 1 Processor -- start 2020-11-20T12:22:32.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.795+0000 7fd25b59e700 1 -- start start 2020-11-20T12:22:32.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd25c0aa7d0 0x7fd25c164660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd25c0aabb0 0x7fd25c0bd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd25c0ad540 0x7fd25c1358c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c14aa00 con 0x7fd25c0aabb0 2020-11-20T12:22:32.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0a9080 con 0x7fd25c0ad540 2020-11-20T12:22:32.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd25c0a9320 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd259d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd25c0aabb0 0x7fd25c0bd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25a59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd25c0aa7d0 0x7fd25c164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd259d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd25c0aabb0 0x7fd25c0bd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52354/0 (socket says 172.21.15.116:52354) 2020-11-20T12:22:32.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd259d9b700 1 -- 172.21.15.116:0/2534074078 learned_addr learned my addr 172.21.15.116:0/2534074078 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:32.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25ad9d700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd25c0ad540 0x7fd25c1358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25a59c700 1 -- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd25c0ad540 msgr2=0x7fd25c1358c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25a59c700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd25c0ad540 0x7fd25c1358c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25a59c700 1 -- 172.21.15.116:0/2534074078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd25c0aabb0 msgr2=0x7fd25c0bd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25a59c700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd25c0aabb0 0x7fd25c0bd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25a59c700 1 -- 172.21.15.116:0/2534074078 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd25c0a95c0 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.799+0000 7fd25a59c700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd25c0aa7d0 0x7fd25c164660 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fd25c155390 tx=0x7fd24c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:32.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.803+0000 7fd24a7fc700 1 -- 172.21.15.116:0/2534074078 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c011c80 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.803+0000 7fd24a7fc700 1 -- 172.21.15.116:0/2534074078 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd24c011de0 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.803+0000 7fd24a7fc700 1 -- 172.21.15.116:0/2534074078 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c01e670 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.803+0000 7fd25b59e700 1 -- 172.21.15.116:0/2534074078 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25c0a9860 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.803+0000 7fd25b59e700 1 -- 172.21.15.116:0/2534074078 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c0a8d70 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.803+0000 7fd25b59e700 1 -- 172.21.15.116:0/2534074078 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd25c05cf30 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.803+0000 7fd24a7fc700 1 -- 172.21.15.116:0/2534074078 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fd24c01a040 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.803+0000 7fd24a7fc700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2300557c0 0x7fd230057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.803+0000 7fd24a7fc700 1 -- 172.21.15.116:0/2534074078 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fd24c06f450 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.807+0000 7fd259d9b700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2300557c0 0x7fd230057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:32.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.807+0000 7fd259d9b700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2300557c0 0x7fd230057c80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fd25c0c5670 tx=0x7fd250007ac0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:32.818 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.811+0000 7fd24a7fc700 1 -- 172.21.15.116:0/2534074078 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fd24c0456d0 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.971+0000 7fd25b59e700 1 -- 172.21.15.116:0/2534074078 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd25c0601b0 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.989 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd24a7fc700 1 -- 172.21.15.116:0/2534074078 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v123) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd24c023020 con 0x7fd25c0aa7d0 2020-11-20T12:22:32.990 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 -- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2300557c0 msgr2=0x7fd230057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2300557c0 0x7fd230057c80 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7fd25c0c5670 tx=0x7fd250007ac0).stop 2020-11-20T12:22:32.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 -- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd25c0aa7d0 msgr2=0x7fd25c164660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd25c0aa7d0 0x7fd25c164660 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7fd25c155390 tx=0x7fd24c015000).stop 2020-11-20T12:22:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 -- 172.21.15.116:0/2534074078 shutdown_connections 2020-11-20T12:22:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fd2300557c0 0x7fd230057c80 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fd25c0ad540 0x7fd25c1358c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fd25c0aa7d0 0x7fd25c164660 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 --2- 172.21.15.116:0/2534074078 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fd25c0aabb0 0x7fd25c0bd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:32.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 -- 172.21.15.116:0/2534074078 >> 172.21.15.116:0/2534074078 conn(0x7fd25c0678a0 msgr2=0x7fd25c14ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:32.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 -- 172.21.15.116:0/2534074078 shutdown_connections 2020-11-20T12:22:32.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:32.975+0000 7fd25b59e700 1 -- 172.21.15.116:0/2534074078 wait complete. 2020-11-20T12:22:32.995 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:33.009 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2020-11-20T12:22:33.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.170+0000 7fa20d72d700 1 Processor -- start 2020-11-20T12:22:33.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.170+0000 7fa20d72d700 1 -- start start 2020-11-20T12:22:33.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20d72d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa208062c60 0x7fa208065d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20d72d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa2080bfe20 0x7fa2081556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20d72d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2080c02b0 con 0x7fa208062880 2020-11-20T12:22:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20d72d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2080c0530 con 0x7fa2080bfe20 2020-11-20T12:22:33.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20d72d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2080c07d0 con 0x7fa208062c60 2020-11-20T12:22:33.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20cf2c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa2080bfe20 0x7fa2081556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20cf2c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa2080bfe20 0x7fa2081556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52364/0 (socket says 172.21.15.116:52364) 2020-11-20T12:22:33.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20cf2c700 1 -- 172.21.15.116:0/178050963 learned_addr learned my addr 172.21.15.116:0/178050963 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:33.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa2077fe700 1 --2- 172.21.15.116:0/178050963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa208062c60 0x7fa208065d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20cf2c700 1 -- 172.21.15.116:0/178050963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa208062c60 msgr2=0x7fa208065d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:33.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20cf2c700 1 --2- 172.21.15.116:0/178050963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa208062c60 0x7fa208065d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20cf2c700 1 -- 172.21.15.116:0/178050963 >> v1:172.21.15.116:6789/0 conn(0x7fa208062880 legacy=0x7fa2080677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20cf2c700 1 -- 172.21.15.116:0/178050963 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2080c0a70 con 0x7fa2080bfe20 2020-11-20T12:22:33.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa2077fe700 1 --2- 172.21.15.116:0/178050963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa208062c60 0x7fa208065d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:33.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.174+0000 7fa20cf2c700 1 --2- 172.21.15.116:0/178050963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2080bfe20 0x7fa2081556e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7fa200009d50 tx=0x7fa200009290).ready entity=mon.0 client_cookie=a2478a5384796a49 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.178+0000 7fa206ffd700 1 -- 172.21.15.116:0/178050963 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa20000bc80 con 0x7fa2080bfe20 2020-11-20T12:22:33.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.178+0000 7fa206ffd700 1 -- 172.21.15.116:0/178050963 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa20000bde0 con 0x7fa2080bfe20 2020-11-20T12:22:33.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.178+0000 7fa206ffd700 1 -- 172.21.15.116:0/178050963 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa200011610 con 0x7fa2080bfe20 2020-11-20T12:22:33.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.178+0000 7fa20d72d700 1 -- 172.21.15.116:0/178050963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2080bfe20 msgr2=0x7fa2081556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.178+0000 7fa20d72d700 1 --2- 172.21.15.116:0/178050963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2080bfe20 0x7fa2081556e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7fa200009d50 tx=0x7fa200009290).stop 2020-11-20T12:22:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.178+0000 7fa20d72d700 1 -- 172.21.15.116:0/178050963 shutdown_connections 2020-11-20T12:22:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.178+0000 7fa20d72d700 1 --2- 172.21.15.116:0/178050963 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa2080bfe20 0x7fa2081556e0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.178+0000 7fa20d72d700 1 --2- 172.21.15.116:0/178050963 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa208062c60 0x7fa208065d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.178+0000 7fa20d72d700 1 -- 172.21.15.116:0/178050963 >> 172.21.15.116:0/178050963 conn(0x7fa208060c40 msgr2=0x7fa2080c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:33.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.178+0000 7fa20d72d700 1 -- 172.21.15.116:0/178050963 shutdown_connections 2020-11-20T12:22:33.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.178+0000 7fa20d72d700 1 -- 172.21.15.116:0/178050963 wait complete. 2020-11-20T12:22:33.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa20d72d700 1 Processor -- start 2020-11-20T12:22:33.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa20d72d700 1 -- start start 2020-11-20T12:22:33.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa20d72d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa208062880 0x7fa208151d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa20d72d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa208062c60 0x7fa208156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa20d72d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2080bfe20 0x7fa208156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa20d72d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2080a48a0 con 0x7fa208062880 2020-11-20T12:22:33.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa20d72d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa208156e30 con 0x7fa2080bfe20 2020-11-20T12:22:33.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa20d72d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2080a4ad0 con 0x7fa208062c60 2020-11-20T12:22:33.191 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa207fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa208062880 0x7fa208151d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa207fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa208062880 0x7fa208151d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52366/0 (socket says 172.21.15.116:52366) 2020-11-20T12:22:33.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa207fff700 1 -- 172.21.15.116:0/3669400410 learned_addr learned my addr 172.21.15.116:0/3669400410 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:33.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa2077fe700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa208062c60 0x7fa208156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa20cf2c700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2080bfe20 0x7fa208156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa207fff700 1 -- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa208062c60 msgr2=0x7fa208156390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa207fff700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa208062c60 0x7fa208156390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa207fff700 1 -- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2080bfe20 msgr2=0x7fa208156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa207fff700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2080bfe20 0x7fa208156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa207fff700 1 -- 172.21.15.116:0/3669400410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2080a4d70 con 0x7fa208062880 2020-11-20T12:22:33.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.182+0000 7fa207fff700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa208062880 0x7fa208151d10 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7fa1f800c710 tx=0x7fa1f8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.186+0000 7fa1e7fff700 1 -- 172.21.15.116:0/3669400410 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800e560 con 0x7fa208062880 2020-11-20T12:22:33.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.186+0000 7fa1e7fff700 1 -- 172.21.15.116:0/3669400410 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f800e6c0 con 0x7fa208062880 2020-11-20T12:22:33.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.186+0000 7fa20d72d700 1 -- 172.21.15.116:0/3669400410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2080a5010 con 0x7fa208062880 2020-11-20T12:22:33.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.186+0000 7fa1e7fff700 1 -- 172.21.15.116:0/3669400410 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f800ea60 con 0x7fa208062880 2020-11-20T12:22:33.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.186+0000 7fa20d72d700 1 -- 172.21.15.116:0/3669400410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa208152200 con 0x7fa208062880 2020-11-20T12:22:33.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.186+0000 7fa20d72d700 1 -- 172.21.15.116:0/3669400410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa20805cf30 con 0x7fa208062880 2020-11-20T12:22:33.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.186+0000 7fa1e7fff700 1 -- 172.21.15.116:0/3669400410 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa1f8019020 con 0x7fa208062880 2020-11-20T12:22:33.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.186+0000 7fa1e7fff700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1e00557c0 0x7fa1e0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.186+0000 7fa1e7fff700 1 -- 172.21.15.116:0/3669400410 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5253+0+0 (secure 0 0 0) 0x7fa1f806dc60 con 0x7fa208062880 2020-11-20T12:22:33.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.190+0000 7fa2077fe700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1e00557c0 0x7fa1e0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.197 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.190+0000 7fa2077fe700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1e00557c0 0x7fa1e0057c80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fa2080a83b0 tx=0x7fa1fc008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.194+0000 7fa1e7fff700 1 -- 172.21.15.116:0/3669400410 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa1f8043e30 con 0x7fa208062880 2020-11-20T12:22:33.359 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.354+0000 7fa20d72d700 1 -- 172.21.15.116:0/3669400410 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa208061230 con 0x7fa208062880 2020-11-20T12:22:33.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.354+0000 7fa1e7fff700 1 -- 172.21.15.116:0/3669400410 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v123) v1 ==== 84+0+143 (secure 0 0 0) 0x7fa1f8047450 con 0x7fa208062880 2020-11-20T12:22:33.360 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:33.361 INFO:teuthology.orchestra.run.smithi116.stdout:[ 2020-11-20T12:22:33.361 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:33.361 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 1, 2020-11-20T12:22:33.361 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "device_health_metrics" 2020-11-20T12:22:33.362 INFO:teuthology.orchestra.run.smithi116.stdout: }, 2020-11-20T12:22:33.362 INFO:teuthology.orchestra.run.smithi116.stdout: { 2020-11-20T12:22:33.362 INFO:teuthology.orchestra.run.smithi116.stdout: "poolnum": 2, 2020-11-20T12:22:33.362 INFO:teuthology.orchestra.run.smithi116.stdout: "poolname": "rbd" 2020-11-20T12:22:33.363 INFO:teuthology.orchestra.run.smithi116.stdout: } 2020-11-20T12:22:33.363 INFO:teuthology.orchestra.run.smithi116.stdout:] 2020-11-20T12:22:33.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 -- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1e00557c0 msgr2=0x7fa1e0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1e00557c0 0x7fa1e0057c80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fa2080a83b0 tx=0x7fa1fc008040).stop 2020-11-20T12:22:33.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 -- 172.21.15.116:0/3669400410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa208062880 msgr2=0x7fa208151d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa208062880 0x7fa208151d10 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7fa1f800c710 tx=0x7fa1f8007c10).stop 2020-11-20T12:22:33.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 -- 172.21.15.116:0/3669400410 shutdown_connections 2020-11-20T12:22:33.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa1e00557c0 0x7fa1e0057c80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa2080bfe20 0x7fa208156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa208062880 0x7fa208151d10 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 --2- 172.21.15.116:0/3669400410 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa208062c60 0x7fa208156390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 -- 172.21.15.116:0/3669400410 >> 172.21.15.116:0/3669400410 conn(0x7fa208060c40 msgr2=0x7fa2080c0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:33.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 -- 172.21.15.116:0/3669400410 shutdown_connections 2020-11-20T12:22:33.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.358+0000 7fa20d72d700 1 -- 172.21.15.116:0/3669400410 wait complete. 2020-11-20T12:22:33.389 INFO:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2020-11-20T12:22:33.390 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 8 2020-11-20T12:22:33.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.570+0000 7f85a759e700 1 Processor -- start 2020-11-20T12:22:33.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.570+0000 7f85a759e700 1 -- start start 2020-11-20T12:22:33.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a759e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f85a80aa7d0 0x7f85a80a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a759e700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f85a80ad540 0x7f85a80ad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a759e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a80ace40 con 0x7f85a80aabb0 2020-11-20T12:22:33.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a759e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a81526c0 con 0x7f85a80aa7d0 2020-11-20T12:22:33.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a759e700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a8152960 con 0x7f85a80ad540 2020-11-20T12:22:33.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a659c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f85a80aa7d0 0x7f85a80a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a659c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f85a80aa7d0 0x7f85a80a8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52376/0 (socket says 172.21.15.116:52376) 2020-11-20T12:22:33.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a659c700 1 -- 172.21.15.116:0/2757080014 learned_addr learned my addr 172.21.15.116:0/2757080014 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:33.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a6d9d700 1 --2- 172.21.15.116:0/2757080014 >> v2:172.21.15.173:3300/0 conn(0x7f85a80ad540 0x7f85a80ad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a659c700 1 -- 172.21.15.116:0/2757080014 >> v2:172.21.15.173:3300/0 conn(0x7f85a80ad540 msgr2=0x7f85a80ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:33.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a659c700 1 --2- 172.21.15.116:0/2757080014 >> v2:172.21.15.173:3300/0 conn(0x7f85a80ad540 0x7f85a80ad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a659c700 1 -- 172.21.15.116:0/2757080014 >> v1:172.21.15.173:6789/0 conn(0x7f85a80aabb0 legacy=0x7f85a8151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a659c700 1 -- 172.21.15.116:0/2757080014 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a8152c00 con 0x7f85a80aa7d0 2020-11-20T12:22:33.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a659c700 1 --2- 172.21.15.116:0/2757080014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a80aa7d0 0x7f85a80a8400 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f85a80c6320 tx=0x7f8598006eb0).ready entity=mon.0 client_cookie=4c1fb550baf3ef73 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.574+0000 7f85a559a700 1 -- 172.21.15.116:0/2757080014 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598009810 con 0x7f85a80aa7d0 2020-11-20T12:22:33.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a559a700 1 -- 172.21.15.116:0/2757080014 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8598009970 con 0x7f85a80aa7d0 2020-11-20T12:22:33.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a559a700 1 -- 172.21.15.116:0/2757080014 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598009cf0 con 0x7f85a80aa7d0 2020-11-20T12:22:33.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a759e700 1 -- 172.21.15.116:0/2757080014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a80aa7d0 msgr2=0x7f85a80a8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a759e700 1 --2- 172.21.15.116:0/2757080014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a80aa7d0 0x7f85a80a8400 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f85a80c6320 tx=0x7f8598006eb0).stop 2020-11-20T12:22:33.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a759e700 1 -- 172.21.15.116:0/2757080014 shutdown_connections 2020-11-20T12:22:33.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a759e700 1 --2- 172.21.15.116:0/2757080014 >> v2:172.21.15.173:3300/0 conn(0x7f85a80ad540 0x7f85a80ad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a759e700 1 --2- 172.21.15.116:0/2757080014 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a80aa7d0 0x7f85a80a8400 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a759e700 1 -- 172.21.15.116:0/2757080014 >> 172.21.15.116:0/2757080014 conn(0x7f85a80678a0 msgr2=0x7f85a8067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:33.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a759e700 1 -- 172.21.15.116:0/2757080014 shutdown_connections 2020-11-20T12:22:33.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a759e700 1 -- 172.21.15.116:0/2757080014 wait complete. 2020-11-20T12:22:33.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a759e700 1 Processor -- start 2020-11-20T12:22:33.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.578+0000 7f85a759e700 1 -- start start 2020-11-20T12:22:33.599 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a759e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85a80aa7d0 0x7f85a814a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a80aabb0 0x7f85a80be110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a759e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85a80ad540 0x7f85a80613a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a759e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a81613d0 con 0x7f85a80aabb0 2020-11-20T12:22:33.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a759e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a8157750 con 0x7f85a80aa7d0 2020-11-20T12:22:33.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a759e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a81579f0 con 0x7f85a80ad540 2020-11-20T12:22:33.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a5d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a80aabb0 0x7f85a80be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a5d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a80aabb0 0x7f85a80be110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52382/0 (socket says 172.21.15.116:52382) 2020-11-20T12:22:33.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a5d9b700 1 -- 172.21.15.116:0/1419086988 learned_addr learned my addr 172.21.15.116:0/1419086988 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:33.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a659c700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85a80aa7d0 0x7f85a814a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a6d9d700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85a80ad540 0x7f85a80613a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a5d9b700 1 -- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85a80ad540 msgr2=0x7f85a80613a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a5d9b700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85a80ad540 0x7f85a80613a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a5d9b700 1 -- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85a80aa7d0 msgr2=0x7f85a814a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a5d9b700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85a80aa7d0 0x7f85a814a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a5d9b700 1 -- 172.21.15.116:0/1419086988 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a8157c90 con 0x7f85a80aabb0 2020-11-20T12:22:33.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a659c700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85a80aa7d0 0x7f85a814a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:33.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a5d9b700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a80aabb0 0x7f85a80be110 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f859c001e70 tx=0x7f859c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85967fc700 1 -- 172.21.15.116:0/1419086988 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c007020 con 0x7f85a80aabb0 2020-11-20T12:22:33.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85967fc700 1 -- 172.21.15.116:0/1419086988 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c007180 con 0x7f85a80aabb0 2020-11-20T12:22:33.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a759e700 1 -- 172.21.15.116:0/1419086988 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a8157f30 con 0x7f85a80aabb0 2020-11-20T12:22:33.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85967fc700 1 -- 172.21.15.116:0/1419086988 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00e560 con 0x7f85a80aabb0 2020-11-20T12:22:33.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.582+0000 7f85a759e700 1 -- 172.21.15.116:0/1419086988 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a814a9f0 con 0x7f85a80aabb0 2020-11-20T12:22:33.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.586+0000 7f85a759e700 1 -- 172.21.15.116:0/1419086988 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a805cf30 con 0x7f85a80aabb0 2020-11-20T12:22:33.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.586+0000 7f85967fc700 1 -- 172.21.15.116:0/1419086988 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f859c018020 con 0x7f85a80aabb0 2020-11-20T12:22:33.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.586+0000 7f85967fc700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f857c0557c0 0x7f857c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:33.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.586+0000 7f85967fc700 1 -- 172.21.15.116:0/1419086988 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5253+0+0 (secure 0 0 0) 0x7f859c06eaf0 con 0x7f85a80aabb0 2020-11-20T12:22:33.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.586+0000 7f85a659c700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f857c0557c0 0x7f857c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:33.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.590+0000 7f85a659c700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f857c0557c0 0x7f857c057c80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f859800b1b0 tx=0x7f8598015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:33.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.594+0000 7f85967fc700 1 -- 172.21.15.116:0/1419086988 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f859c044ce0 con 0x7f85a80aabb0 2020-11-20T12:22:33.760 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.754+0000 7f85a759e700 1 -- 172.21.15.116:0/1419086988 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8} v 0) v1 -- 0x7f85a80601b0 con 0x7f85a80aabb0 2020-11-20T12:22:33.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.846+0000 7f85967fc700 1 -- 172.21.15.116:0/1419086988 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 8}]=0 pool 'cephfs_metadata' created v124) v1 ==== 133+0+0 (secure 0 0 0) 0x7f859c016030 con 0x7f85a80aabb0 2020-11-20T12:22:33.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 -- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f857c0557c0 msgr2=0x7f857c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.857 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f857c0557c0 0x7f857c057c80 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f859800b1b0 tx=0x7f8598015000).stop 2020-11-20T12:22:33.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 -- 172.21.15.116:0/1419086988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a80aabb0 msgr2=0x7f85a80be110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:33.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a80aabb0 0x7f85a80be110 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f859c001e70 tx=0x7f859c007c50).stop 2020-11-20T12:22:33.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 -- 172.21.15.116:0/1419086988 shutdown_connections 2020-11-20T12:22:33.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f857c0557c0 0x7f857c057c80 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.860 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f85a80ad540 0x7f85a80613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f85a80aa7d0 0x7f85a814a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 --2- 172.21.15.116:0/1419086988 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f85a80aabb0 0x7f85a80be110 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:33.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 -- 172.21.15.116:0/1419086988 >> 172.21.15.116:0/1419086988 conn(0x7f85a80678a0 msgr2=0x7f85a80c87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:33.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 -- 172.21.15.116:0/1419086988 shutdown_connections 2020-11-20T12:22:33.864 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:33.850+0000 7f85a759e700 1 -- 172.21.15.116:0/1419086988 wait complete. 2020-11-20T12:22:33.866 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_metadata' created 2020-11-20T12:22:33.880 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 8 2020-11-20T12:22:34.049 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.042+0000 7f9aa277a700 1 Processor -- start 2020-11-20T12:22:34.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.042+0000 7f9aa277a700 1 -- start start 2020-11-20T12:22:34.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa277a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9a9c155c80 0x7f9a9c062ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:34.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa277a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9a9c156060 0x7f9a9c0c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:34.051 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa277a700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a9c0b93c0 con 0x7f9a9c0c12c0 2020-11-20T12:22:34.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa277a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9c0ba580 con 0x7f9a9c156060 2020-11-20T12:22:34.052 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa277a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a9c0b9660 con 0x7f9a9c155c80 2020-11-20T12:22:34.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa0f77700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9a9c156060 0x7f9a9c0c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:34.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa1f79700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f9a9c0c12c0 0x7f9a9c060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33592/0 (socket says 172.21.15.116:33592) 2020-11-20T12:22:34.053 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa1f79700 1 -- 172.21.15.116:0/1864687271 learned_addr learned my addr 172.21.15.116:0/1864687271 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:34.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa1778700 1 --2- 172.21.15.116:0/1864687271 >> v2:172.21.15.173:3300/0 conn(0x7f9a9c155c80 0x7f9a9c062ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:34.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa0f77700 1 -- 172.21.15.116:0/1864687271 >> v2:172.21.15.173:3300/0 conn(0x7f9a9c155c80 msgr2=0x7f9a9c062ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:34.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa0f77700 1 --2- 172.21.15.116:0/1864687271 >> v2:172.21.15.173:3300/0 conn(0x7f9a9c155c80 0x7f9a9c062ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:34.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9a8bfff700 1 -- 172.21.15.116:0/1864687271 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487998367 0 0) 0x7f9a9c0b93c0 con 0x7f9a9c0c12c0 2020-11-20T12:22:34.054 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa0f77700 1 -- 172.21.15.116:0/1864687271 >> v1:172.21.15.116:6789/0 conn(0x7f9a9c0c12c0 legacy=0x7f9a9c060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:34.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa0f77700 1 -- 172.21.15.116:0/1864687271 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c0b9900 con 0x7f9a9c156060 2020-11-20T12:22:34.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9aa0f77700 1 --2- 172.21.15.116:0/1864687271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a9c156060 0x7f9a9c0c0dd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f9a9c065ab0 tx=0x7f9a8c006df0).ready entity=mon.0 client_cookie=e33d7cfaa6be0b23 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:34.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9a8bfff700 1 -- 172.21.15.116:0/1864687271 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c0097b0 con 0x7f9a9c156060 2020-11-20T12:22:34.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9a8bfff700 1 -- 172.21.15.116:0/1864687271 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a8c009910 con 0x7f9a9c156060 2020-11-20T12:22:34.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.046+0000 7f9a8bfff700 1 -- 172.21.15.116:0/1864687271 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a8c009c90 con 0x7f9a9c156060 2020-11-20T12:22:34.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.050+0000 7f9aa277a700 1 -- 172.21.15.116:0/1864687271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a9c156060 msgr2=0x7f9a9c0c0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:34.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.050+0000 7f9aa277a700 1 --2- 172.21.15.116:0/1864687271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a9c156060 0x7f9a9c0c0dd0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7f9a9c065ab0 tx=0x7f9a8c006df0).stop 2020-11-20T12:22:34.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.050+0000 7f9aa277a700 1 -- 172.21.15.116:0/1864687271 shutdown_connections 2020-11-20T12:22:34.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.050+0000 7f9aa277a700 1 --2- 172.21.15.116:0/1864687271 >> v2:172.21.15.173:3300/0 conn(0x7f9a9c155c80 0x7f9a9c062ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:34.057 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.050+0000 7f9aa277a700 1 --2- 172.21.15.116:0/1864687271 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a9c156060 0x7f9a9c0c0dd0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:34.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.050+0000 7f9aa277a700 1 -- 172.21.15.116:0/1864687271 >> 172.21.15.116:0/1864687271 conn(0x7f9a9c068350 msgr2=0x7f9a9c0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:34.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.050+0000 7f9aa277a700 1 -- 172.21.15.116:0/1864687271 shutdown_connections 2020-11-20T12:22:34.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.050+0000 7f9aa277a700 1 -- 172.21.15.116:0/1864687271 wait complete. 2020-11-20T12:22:34.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa277a700 1 Processor -- start 2020-11-20T12:22:34.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa277a700 1 -- start start 2020-11-20T12:22:34.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa277a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a9c0c12c0 0x7f9a9c158950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:34.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa277a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a9c155c80 0x7f9a9c0a9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:34.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa277a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a9c0bb6b0 0x7f9a9c0bbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:34.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa277a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c0be440 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa277a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c1515b0 con 0x7f9a9c155c80 2020-11-20T12:22:34.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa277a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a9c0be1a0 con 0x7f9a9c0c12c0 2020-11-20T12:22:34.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa1f79700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a9c0bb6b0 0x7f9a9c0bbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:34.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa1f79700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a9c0bb6b0 0x7f9a9c0bbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52398/0 (socket says 172.21.15.116:52398) 2020-11-20T12:22:34.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa1f79700 1 -- 172.21.15.116:0/1328127861 learned_addr learned my addr 172.21.15.116:0/1328127861 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:34.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.054+0000 7f9aa1778700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a9c0c12c0 0x7f9a9c158950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:34.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9aa0f77700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a9c155c80 0x7f9a9c0a9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:34.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9aa1f79700 1 -- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a9c0c12c0 msgr2=0x7f9a9c158950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:34.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9aa1f79700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a9c0c12c0 0x7f9a9c158950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:34.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9aa1f79700 1 -- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a9c155c80 msgr2=0x7f9a9c0a9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:34.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9aa1f79700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a9c155c80 0x7f9a9c0a9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:34.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9aa1f79700 1 -- 172.21.15.116:0/1328127861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c157f70 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9aa1f79700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a9c0bb6b0 0x7f9a9c0bbb30 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7f9a9400cab0 tx=0x7f9a94007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:34.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9a897fa700 1 -- 172.21.15.116:0/1328127861 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94005b80 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9a897fa700 1 -- 172.21.15.116:0/1328127861 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a94005ce0 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9a897fa700 1 -- 172.21.15.116:0/1328127861 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a94007360 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9aa277a700 1 -- 172.21.15.116:0/1328127861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a9c158210 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9aa277a700 1 -- 172.21.15.116:0/1328127861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a9c0bde90 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.058+0000 7f9aa277a700 1 -- 172.21.15.116:0/1328127861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a9c05cf30 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.062+0000 7f9a897fa700 1 -- 172.21.15.116:0/1328127861 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9a94010030 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.068 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.062+0000 7f9a897fa700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9a740557c0 0x7f9a74057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:34.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.062+0000 7f9a897fa700 1 -- 172.21.15.116:0/1328127861 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5607+0+0 (secure 0 0 0) 0x7f9a9406e340 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.062+0000 7f9aa1778700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9a740557c0 0x7f9a74057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:34.069 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.062+0000 7f9aa1778700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9a740557c0 0x7f9a74057c80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f9a9c168c60 tx=0x7f9a98006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:34.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.066+0000 7f9a897fa700 1 -- 172.21.15.116:0/1328127861 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9a940443b0 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.226+0000 7f9aa277a700 1 -- 172.21.15.116:0/1328127861 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8} v 0) v1 -- 0x7f9a9c0639c0 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.874+0000 7f9a897fa700 1 -- 172.21.15.116:0/1328127861 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 8}]=0 pool 'cephfs_data' created v127) v1 ==== 125+0+0 (secure 0 0 0) 0x7f9a940479d0 con 0x7f9a9c0bb6b0 2020-11-20T12:22:34.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.878+0000 7f9aa277a700 1 -- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9a740557c0 msgr2=0x7f9a74057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:34.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.878+0000 7f9aa277a700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9a740557c0 0x7f9a74057c80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f9a9c168c60 tx=0x7f9a98006df0).stop 2020-11-20T12:22:34.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.878+0000 7f9aa277a700 1 -- 172.21.15.116:0/1328127861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a9c0bb6b0 msgr2=0x7f9a9c0bbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:34.884 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.878+0000 7f9aa277a700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a9c0bb6b0 0x7f9a9c0bbb30 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7f9a9400cab0 tx=0x7f9a94007d80).stop 2020-11-20T12:22:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.878+0000 7f9aa277a700 1 -- 172.21.15.116:0/1328127861 shutdown_connections 2020-11-20T12:22:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.878+0000 7f9aa277a700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9a740557c0 0x7f9a74057c80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:34.885 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.878+0000 7f9aa277a700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9a9c0bb6b0 0x7f9a9c0bbb30 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.878+0000 7f9aa277a700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9a9c0c12c0 0x7f9a9c158950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.878+0000 7f9aa277a700 1 --2- 172.21.15.116:0/1328127861 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9a9c155c80 0x7f9a9c0a9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:34.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.878+0000 7f9aa277a700 1 -- 172.21.15.116:0/1328127861 >> 172.21.15.116:0/1328127861 conn(0x7f9a9c068350 msgr2=0x7f9a9c066150 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:34.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.882+0000 7f9aa277a700 1 -- 172.21.15.116:0/1328127861 shutdown_connections 2020-11-20T12:22:34.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:34.882+0000 7f9aa277a700 1 -- 172.21.15.116:0/1328127861 wait complete. 2020-11-20T12:22:34.893 INFO:teuthology.orchestra.run.smithi116.stderr:pool 'cephfs_data' created 2020-11-20T12:22:34.917 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data --force 2020-11-20T12:22:35.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.090+0000 7efc52459700 1 Processor -- start 2020-11-20T12:22:35.099 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.090+0000 7efc52459700 1 -- start start 2020-11-20T12:22:35.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.094+0000 7efc52459700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7efc4c14c050 0x7efc4c155f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.094+0000 7efc52459700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc4c0c3820 0x7efc4c15b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.109 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.094+0000 7efc52459700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc4c0c3fa0 con 0x7efc4c14bc70 2020-11-20T12:22:35.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.094+0000 7efc52459700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc4c0a81b0 con 0x7efc4c14c050 2020-11-20T12:22:35.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.094+0000 7efc52459700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c151fb0 con 0x7efc4c0c3820 2020-11-20T12:22:35.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc4bfff700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7efc4c14bc70 0x7efc4c0c5260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:58070/0 (socket says 172.21.15.116:58070) 2020-11-20T12:22:35.110 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc4bfff700 1 -- 172.21.15.116:0/1180689227 learned_addr learned my addr 172.21.15.116:0/1180689227 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:35.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc509f6700 1 --2- 172.21.15.116:0/1180689227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc4c0c3820 0x7efc4c15b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc43fff700 1 --2- 172.21.15.116:0/1180689227 >> v2:172.21.15.173:3300/0 conn(0x7efc4c14c050 0x7efc4c155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc4b7fe700 1 -- 172.21.15.116:0/1180689227 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125579815 0 0) 0x7efc4c0c3fa0 con 0x7efc4c14bc70 2020-11-20T12:22:35.111 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc4b7fe700 1 -- 172.21.15.116:0/1180689227 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc30003240 con 0x7efc4c14bc70 2020-11-20T12:22:35.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc43fff700 1 -- 172.21.15.116:0/1180689227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc4c0c3820 msgr2=0x7efc4c15b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:35.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc43fff700 1 --2- 172.21.15.116:0/1180689227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc4c0c3820 0x7efc4c15b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc43fff700 1 -- 172.21.15.116:0/1180689227 >> v1:172.21.15.173:6789/0 conn(0x7efc4c14bc70 legacy=0x7efc4c0c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc43fff700 1 -- 172.21.15.116:0/1180689227 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c152250 con 0x7efc4c14c050 2020-11-20T12:22:35.112 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc43fff700 1 --2- 172.21.15.116:0/1180689227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc4c14c050 0x7efc4c155f60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7efc3400b9a0 tx=0x7efc34006df0).ready entity=mon.1 client_cookie=45117bf3e97cbc54 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:35.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc4b7fe700 1 -- 172.21.15.116:0/1180689227 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34009680 con 0x7efc4c14c050 2020-11-20T12:22:35.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc4b7fe700 1 -- 172.21.15.116:0/1180689227 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc340097e0 con 0x7efc4c14c050 2020-11-20T12:22:35.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.098+0000 7efc4b7fe700 1 -- 172.21.15.116:0/1180689227 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc34009b60 con 0x7efc4c14c050 2020-11-20T12:22:35.113 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.102+0000 7efc52459700 1 -- 172.21.15.116:0/1180689227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc4c14c050 msgr2=0x7efc4c155f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.102+0000 7efc52459700 1 --2- 172.21.15.116:0/1180689227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc4c14c050 0x7efc4c155f60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7efc3400b9a0 tx=0x7efc34006df0).stop 2020-11-20T12:22:35.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.102+0000 7efc52459700 1 -- 172.21.15.116:0/1180689227 shutdown_connections 2020-11-20T12:22:35.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.102+0000 7efc52459700 1 --2- 172.21.15.116:0/1180689227 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc4c0c3820 0x7efc4c15b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.102+0000 7efc52459700 1 --2- 172.21.15.116:0/1180689227 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc4c14c050 0x7efc4c155f60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.114 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.102+0000 7efc52459700 1 -- 172.21.15.116:0/1180689227 >> 172.21.15.116:0/1180689227 conn(0x7efc4c0678a0 msgr2=0x7efc4c067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:35.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.102+0000 7efc52459700 1 -- 172.21.15.116:0/1180689227 shutdown_connections 2020-11-20T12:22:35.115 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.102+0000 7efc52459700 1 -- 172.21.15.116:0/1180689227 wait complete. 2020-11-20T12:22:35.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.102+0000 7efc52459700 1 Processor -- start 2020-11-20T12:22:35.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.102+0000 7efc52459700 1 -- start start 2020-11-20T12:22:35.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc52459700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc4c0c3820 0x7efc4c0a9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.117 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc52459700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc4c14bc70 0x7efc4c147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc52459700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc4c14c050 0x7efc4c147a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc52459700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0c8a80 con 0x7efc4c14c050 2020-11-20T12:22:35.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc52459700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0c87e0 con 0x7efc4c0c3820 2020-11-20T12:22:35.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc52459700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc4c0bf9e0 con 0x7efc4c14bc70 2020-11-20T12:22:35.118 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc509f6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc4c14c050 0x7efc4c147a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc4bfff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc4c0c3820 0x7efc4c0a9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc43fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc4c14bc70 0x7efc4c147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc509f6700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc4c14c050 0x7efc4c147a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52416/0 (socket says 172.21.15.116:52416) 2020-11-20T12:22:35.119 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc509f6700 1 -- 172.21.15.116:0/479933623 learned_addr learned my addr 172.21.15.116:0/479933623 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:35.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc4bfff700 1 -- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc4c14bc70 msgr2=0x7efc4c147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc4bfff700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc4c14bc70 0x7efc4c147560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc4bfff700 1 -- 172.21.15.116:0/479933623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc4c14c050 msgr2=0x7efc4c147a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc4bfff700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc4c14c050 0x7efc4c147a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.120 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc4bfff700 1 -- 172.21.15.116:0/479933623 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc4c0bfc80 con 0x7efc4c0c3820 2020-11-20T12:22:35.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc509f6700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc4c14c050 0x7efc4c147a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:35.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.106+0000 7efc4bfff700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc4c0c3820 0x7efc4c0a9bc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7efc3c00c6e0 tx=0x7efc3c007a20).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:35.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.110+0000 7efc48ff9700 1 -- 172.21.15.116:0/479933623 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c0072e0 con 0x7efc4c0c3820 2020-11-20T12:22:35.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.110+0000 7efc48ff9700 1 -- 172.21.15.116:0/479933623 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc3c007440 con 0x7efc4c0c3820 2020-11-20T12:22:35.121 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.110+0000 7efc52459700 1 -- 172.21.15.116:0/479933623 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc4c0bff20 con 0x7efc4c0c3820 2020-11-20T12:22:35.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.110+0000 7efc48ff9700 1 -- 172.21.15.116:0/479933623 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc3c005820 con 0x7efc4c0c3820 2020-11-20T12:22:35.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.110+0000 7efc52459700 1 -- 172.21.15.116:0/479933623 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc4c0aa2d0 con 0x7efc4c0c3820 2020-11-20T12:22:35.122 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.110+0000 7efc52459700 1 -- 172.21.15.116:0/479933623 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc4c0c00b0 con 0x7efc4c0c3820 2020-11-20T12:22:35.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.110+0000 7efc48ff9700 1 -- 172.21.15.116:0/479933623 <== mon.1 v2:172.21.15.173:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7efc3c01a060 con 0x7efc4c0c3820 2020-11-20T12:22:35.123 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.114+0000 7efc48ff9700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc1c0557c0 0x7efc1c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.114+0000 7efc48ff9700 1 -- 172.21.15.116:0/479933623 <== mon.1 v2:172.21.15.173:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5957+0+0 (secure 0 0 0) 0x7efc3c06ddf0 con 0x7efc4c0c3820 2020-11-20T12:22:35.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.114+0000 7efc43fff700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc1c0557c0 0x7efc1c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.124 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.114+0000 7efc43fff700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc1c0557c0 0x7efc1c057c80 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7efc3400c010 tx=0x7efc34015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:35.127 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.122+0000 7efc48ff9700 1 -- 172.21.15.116:0/479933623 <== mon.1 v2:172.21.15.173:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7efc3c043bf0 con 0x7efc4c0c3820 2020-11-20T12:22:35.338 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.330+0000 7efc52459700 1 -- 172.21.15.116:0/479933623 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true} v 0) v1 -- 0x7efc4c069950 con 0x7efc4c0c3820 2020-11-20T12:22:35.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.346+0000 7efc48ff9700 1 -- 172.21.15.116:0/479933623 <== mon.1 v2:172.21.15.173:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data", "force": true}]=0 new fs with metadata pool 23 and data pool 24 v73) v1 ==== 189+0+0 (secure 0 0 0) 0x7efc3c018030 con 0x7efc4c0c3820 2020-11-20T12:22:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.354+0000 7efc52459700 1 -- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc1c0557c0 msgr2=0x7efc1c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.354+0000 7efc52459700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc1c0557c0 0x7efc1c057c80 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7efc3400c010 tx=0x7efc34015000).stop 2020-11-20T12:22:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.354+0000 7efc52459700 1 -- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc4c0c3820 msgr2=0x7efc4c0a9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.354+0000 7efc52459700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc4c0c3820 0x7efc4c0a9bc0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7efc3c00c6e0 tx=0x7efc3c007a20).stop 2020-11-20T12:22:35.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.358+0000 7efc52459700 1 -- 172.21.15.116:0/479933623 shutdown_connections 2020-11-20T12:22:35.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.358+0000 7efc52459700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7efc1c0557c0 0x7efc1c057c80 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.358+0000 7efc52459700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7efc4c14c050 0x7efc4c147a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.358+0000 7efc52459700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7efc4c0c3820 0x7efc4c0a9bc0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.358+0000 7efc52459700 1 --2- 172.21.15.116:0/479933623 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7efc4c14bc70 0x7efc4c147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.358+0000 7efc52459700 1 -- 172.21.15.116:0/479933623 >> 172.21.15.116:0/479933623 conn(0x7efc4c0678a0 msgr2=0x7efc4c0bda50 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.366+0000 7efc52459700 1 -- 172.21.15.116:0/479933623 shutdown_connections 2020-11-20T12:22:35.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.366+0000 7efc52459700 1 -- 172.21.15.116:0/479933623 wait complete. 2020-11-20T12:22:35.377 INFO:teuthology.orchestra.run.smithi116.stderr:new fs with metadata pool 23 and data pool 24 2020-11-20T12:22:35.399 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:35.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.562+0000 7f6932e6a700 1 Processor -- start 2020-11-20T12:22:35.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.562+0000 7f6932e6a700 1 -- start start 2020-11-20T12:22:35.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6932e6a700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f692c14bcf0 0x7f692c0c37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6932e6a700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f692c0c3ca0 0x7f692c0a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6932e6a700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692c153190 con 0x7f692c152c50 2020-11-20T12:22:35.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6932e6a700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692c153430 con 0x7f692c14bcf0 2020-11-20T12:22:35.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6932e6a700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692c1536d0 con 0x7f692c0c3ca0 2020-11-20T12:22:35.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6930c06700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f692c14bcf0 0x7f692c0c37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6930c06700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f692c14bcf0 0x7f692c0c37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52430/0 (socket says 172.21.15.116:52430) 2020-11-20T12:22:35.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6930c06700 1 -- 172.21.15.116:0/1523853890 learned_addr learned my addr 172.21.15.116:0/1523853890 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:35.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6931407700 1 --2- 172.21.15.116:0/1523853890 >> v2:172.21.15.173:3300/0 conn(0x7f692c0c3ca0 0x7f692c0a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6930c06700 1 -- 172.21.15.116:0/1523853890 >> v2:172.21.15.173:3300/0 conn(0x7f692c0c3ca0 msgr2=0x7f692c0a74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:35.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6930c06700 1 --2- 172.21.15.116:0/1523853890 >> v2:172.21.15.173:3300/0 conn(0x7f692c0c3ca0 0x7f692c0a74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6930c06700 1 -- 172.21.15.116:0/1523853890 >> v1:172.21.15.173:6789/0 conn(0x7f692c152c50 legacy=0x7f692c061530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6930c06700 1 -- 172.21.15.116:0/1523853890 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c153970 con 0x7f692c14bcf0 2020-11-20T12:22:35.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6931407700 1 --2- 172.21.15.116:0/1523853890 >> v2:172.21.15.173:3300/0 conn(0x7f692c0c3ca0 0x7f692c0a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:35.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f6930c06700 1 --2- 172.21.15.116:0/1523853890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c14bcf0 0x7f692c0c37b0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f692c153080 tx=0x7f6920006eb0).ready entity=mon.0 client_cookie=c88bac2ff17fb413 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:35.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f692b7fe700 1 -- 172.21.15.116:0/1523853890 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920009810 con 0x7f692c14bcf0 2020-11-20T12:22:35.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f692b7fe700 1 -- 172.21.15.116:0/1523853890 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6920009970 con 0x7f692c14bcf0 2020-11-20T12:22:35.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.566+0000 7f692b7fe700 1 -- 172.21.15.116:0/1523853890 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920009cf0 con 0x7f692c14bcf0 2020-11-20T12:22:35.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.570+0000 7f6932e6a700 1 -- 172.21.15.116:0/1523853890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c14bcf0 msgr2=0x7f692c0c37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.570+0000 7f6932e6a700 1 --2- 172.21.15.116:0/1523853890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c14bcf0 0x7f692c0c37b0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f692c153080 tx=0x7f6920006eb0).stop 2020-11-20T12:22:35.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.570+0000 7f6932e6a700 1 -- 172.21.15.116:0/1523853890 shutdown_connections 2020-11-20T12:22:35.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.570+0000 7f6932e6a700 1 --2- 172.21.15.116:0/1523853890 >> v2:172.21.15.173:3300/0 conn(0x7f692c0c3ca0 0x7f692c0a74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.570+0000 7f6932e6a700 1 --2- 172.21.15.116:0/1523853890 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c14bcf0 0x7f692c0c37b0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.570+0000 7f6932e6a700 1 -- 172.21.15.116:0/1523853890 >> 172.21.15.116:0/1523853890 conn(0x7f692c068370 msgr2=0x7f692c0c88d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:35.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.570+0000 7f6932e6a700 1 -- 172.21.15.116:0/1523853890 shutdown_connections 2020-11-20T12:22:35.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.570+0000 7f6932e6a700 1 -- 172.21.15.116:0/1523853890 wait complete. 2020-11-20T12:22:35.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.570+0000 7f6932e6a700 1 Processor -- start 2020-11-20T12:22:35.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6932e6a700 1 -- start start 2020-11-20T12:22:35.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6932e6a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c0c3ca0 0x7f692c0a92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6932e6a700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f692c14bcf0 0x7f692c0be9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6932e6a700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f692c152c50 0x7f692c14b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6932e6a700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c161fc0 con 0x7f692c0c3ca0 2020-11-20T12:22:35.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6932e6a700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0a88c0 con 0x7f692c14bcf0 2020-11-20T12:22:35.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6932e6a700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0a8b60 con 0x7f692c152c50 2020-11-20T12:22:35.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6930c06700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c0c3ca0 0x7f692c0a92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6930c06700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c0c3ca0 0x7f692c0a92a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52434/0 (socket says 172.21.15.116:52434) 2020-11-20T12:22:35.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6930c06700 1 -- 172.21.15.116:0/2191279597 learned_addr learned my addr 172.21.15.116:0/2191279597 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:35.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6931407700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f692c152c50 0x7f692c14b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f692bfff700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f692c14bcf0 0x7f692c0be9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6930c06700 1 -- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f692c152c50 msgr2=0x7f692c14b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.589 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6930c06700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f692c152c50 0x7f692c14b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6930c06700 1 -- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f692c14bcf0 msgr2=0x7f692c0be9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6930c06700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f692c14bcf0 0x7f692c0be9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6930c06700 1 -- 172.21.15.116:0/2191279597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c0a8e00 con 0x7f692c0c3ca0 2020-11-20T12:22:35.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6931407700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f692c152c50 0x7f692c14b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:35.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f692bfff700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f692c14bcf0 0x7f692c0be9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:35.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6930c06700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c0c3ca0 0x7f692c0a92a0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f692000b1b0 tx=0x7f6920015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:35.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6928ff9700 1 -- 172.21.15.116:0/2191279597 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692001e460 con 0x7f692c0c3ca0 2020-11-20T12:22:35.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6928ff9700 1 -- 172.21.15.116:0/2191279597 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692001e5c0 con 0x7f692c0c3ca0 2020-11-20T12:22:35.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.574+0000 7f6932e6a700 1 -- 172.21.15.116:0/2191279597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c0a9790 con 0x7f692c0c3ca0 2020-11-20T12:22:35.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.578+0000 7f6928ff9700 1 -- 172.21.15.116:0/2191279597 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692001e980 con 0x7f692c0c3ca0 2020-11-20T12:22:35.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.578+0000 7f6932e6a700 1 -- 172.21.15.116:0/2191279597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c0bf0b0 con 0x7f692c0c3ca0 2020-11-20T12:22:35.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.578+0000 7f6932e6a700 1 -- 172.21.15.116:0/2191279597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692c0a8f90 con 0x7f692c0c3ca0 2020-11-20T12:22:35.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.578+0000 7f6928ff9700 1 -- 172.21.15.116:0/2191279597 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6920025030 con 0x7f692c0c3ca0 2020-11-20T12:22:35.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.578+0000 7f6928ff9700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69040557c0 0x7f6904057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.578+0000 7f6928ff9700 1 -- 172.21.15.116:0/2191279597 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f692006ff60 con 0x7f692c0c3ca0 2020-11-20T12:22:35.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.582+0000 7f692bfff700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69040557c0 0x7f6904057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.582+0000 7f692bfff700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69040557c0 0x7f6904057c80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f692c0bf850 tx=0x7f6918009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:35.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.586+0000 7f6928ff9700 1 -- 172.21.15.116:0/2191279597 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6920045e00 con 0x7f692c0c3ca0 2020-11-20T12:22:35.757 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.750+0000 7f6932e6a700 1 -- 172.21.15.116:0/2191279597 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f692c069950 con 0x7f692c0c3ca0 2020-11-20T12:22:35.761 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.754+0000 7f6928ff9700 1 -- 172.21.15.116:0/2191279597 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16901 (secure 0 0 0) 0x7f6920023070 con 0x7f692c0c3ca0 2020-11-20T12:22:35.761 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:35.762 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":128,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:22:35.338133+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:22:33.838237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2020-11-20T12:22:34.312791+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:35.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.754+0000 7f6932e6a700 1 -- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69040557c0 msgr2=0x7f6904057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.763 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.754+0000 7f6932e6a700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69040557c0 0x7f6904057c80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7f692c0bf850 tx=0x7f6918009040).stop 2020-11-20T12:22:35.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.754+0000 7f6932e6a700 1 -- 172.21.15.116:0/2191279597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c0c3ca0 msgr2=0x7f692c0a92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.754+0000 7f6932e6a700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c0c3ca0 0x7f692c0a92a0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f692000b1b0 tx=0x7f6920015000).stop 2020-11-20T12:22:35.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.754+0000 7f6932e6a700 1 -- 172.21.15.116:0/2191279597 shutdown_connections 2020-11-20T12:22:35.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.754+0000 7f6932e6a700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69040557c0 0x7f6904057c80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.764 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.754+0000 7f6932e6a700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f692c152c50 0x7f692c14b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.754+0000 7f6932e6a700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f692c0c3ca0 0x7f692c0a92a0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.754+0000 7f6932e6a700 1 --2- 172.21.15.116:0/2191279597 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f692c14bcf0 0x7f692c0be9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.754+0000 7f6932e6a700 1 -- 172.21.15.116:0/2191279597 >> 172.21.15.116:0/2191279597 conn(0x7f692c068370 msgr2=0x7f692c0aa890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:35.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.758+0000 7f6932e6a700 1 -- 172.21.15.116:0/2191279597 shutdown_connections 2020-11-20T12:22:35.765 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.758+0000 7f6932e6a700 1 -- 172.21.15.116:0/2191279597 wait complete. 2020-11-20T12:22:35.795 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:35.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.962+0000 7fa9bf53f700 1 Processor -- start 2020-11-20T12:22:35.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.962+0000 7fa9bf53f700 1 -- start start 2020-11-20T12:22:35.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.962+0000 7fa9bf53f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa9b80663a0 0x7fa9b8066800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bf53f700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b80c5010 con 0x7fa9b80a6480 2020-11-20T12:22:35.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bf53f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b8066e00 con 0x7fa9b814e480 2020-11-20T12:22:35.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bf53f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b81530f0 con 0x7fa9b80663a0 2020-11-20T12:22:35.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bdd3c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa9b80663a0 0x7fa9b8066800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bdd3c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa9b80663a0 0x7fa9b8066800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52444/0 (socket says 172.21.15.116:52444) 2020-11-20T12:22:35.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bdd3c700 1 -- 172.21.15.116:0/3418987967 learned_addr learned my addr 172.21.15.116:0/3418987967 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:35.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bdd3c700 1 -- 172.21.15.116:0/3418987967 >> v1:172.21.15.173:6789/0 conn(0x7fa9b814e480 legacy=0x7fa9b81529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bdd3c700 1 -- 172.21.15.116:0/3418987967 >> v1:172.21.15.116:6789/0 conn(0x7fa9b80a6480 legacy=0x7fa9b80677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bdd3c700 1 -- 172.21.15.116:0/3418987967 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b80670a0 con 0x7fa9b80663a0 2020-11-20T12:22:35.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bdd3c700 1 --2- 172.21.15.116:0/3418987967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b80663a0 0x7fa9b8066800 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fa9a8006fd0 tx=0x7fa9a8006eb0).ready entity=mon.0 client_cookie=f3ee9ab01c249339 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:35.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bd53b700 1 -- 172.21.15.116:0/3418987967 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a80098d0 con 0x7fa9b80663a0 2020-11-20T12:22:35.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bd53b700 1 -- 172.21.15.116:0/3418987967 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a8009a30 con 0x7fa9b80663a0 2020-11-20T12:22:35.975 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bd53b700 1 -- 172.21.15.116:0/3418987967 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a8009db0 con 0x7fa9b80663a0 2020-11-20T12:22:35.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bf53f700 1 -- 172.21.15.116:0/3418987967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b80663a0 msgr2=0x7fa9b8066800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.966+0000 7fa9bf53f700 1 --2- 172.21.15.116:0/3418987967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b80663a0 0x7fa9b8066800 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fa9a8006fd0 tx=0x7fa9a8006eb0).stop 2020-11-20T12:22:35.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.970+0000 7fa9bf53f700 1 -- 172.21.15.116:0/3418987967 shutdown_connections 2020-11-20T12:22:35.976 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.970+0000 7fa9bf53f700 1 --2- 172.21.15.116:0/3418987967 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b80663a0 0x7fa9b8066800 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.970+0000 7fa9bf53f700 1 -- 172.21.15.116:0/3418987967 >> 172.21.15.116:0/3418987967 conn(0x7fa9b8060c40 msgr2=0x7fa9b80c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:35.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.970+0000 7fa9bf53f700 1 -- 172.21.15.116:0/3418987967 shutdown_connections 2020-11-20T12:22:35.977 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.970+0000 7fa9bf53f700 1 -- 172.21.15.116:0/3418987967 wait complete. 2020-11-20T12:22:35.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.970+0000 7fa9bf53f700 1 Processor -- start 2020-11-20T12:22:35.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.970+0000 7fa9bf53f700 1 -- start start 2020-11-20T12:22:35.978 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bf53f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b80663a0 0x7fa9b8062880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.979 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bf53f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9b80a6480 0x7fa9b80bcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bf53f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b814e480 0x7fa9b80bd260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bf53f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b8151c90 con 0x7fa9b814e480 2020-11-20T12:22:35.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bf53f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80bd880 con 0x7fa9b80a6480 2020-11-20T12:22:35.980 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bf53f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80a7320 con 0x7fa9b80663a0 2020-11-20T12:22:35.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bed3e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b814e480 0x7fa9b80bd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bed3e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b814e480 0x7fa9b80bd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52452/0 (socket says 172.21.15.116:52452) 2020-11-20T12:22:35.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bed3e700 1 -- 172.21.15.116:0/111447526 learned_addr learned my addr 172.21.15.116:0/111447526 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:35.981 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bdd3c700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9b80a6480 0x7fa9b80bcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.982 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9be53d700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b80663a0 0x7fa9b8062880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bed3e700 1 -- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b80663a0 msgr2=0x7fa9b8062880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bed3e700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b80663a0 0x7fa9b8062880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bed3e700 1 -- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9b80a6480 msgr2=0x7fa9b80bcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:35.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bed3e700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9b80a6480 0x7fa9b80bcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:35.983 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bed3e700 1 -- 172.21.15.116:0/111447526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b80a75c0 con 0x7fa9b814e480 2020-11-20T12:22:35.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bed3e700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b814e480 0x7fa9b80bd260 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7fa9b000ca90 tx=0x7fa9b0001360).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:35.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9a67fc700 1 -- 172.21.15.116:0/111447526 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b000a730 con 0x7fa9b814e480 2020-11-20T12:22:35.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9a67fc700 1 -- 172.21.15.116:0/111447526 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9b000a890 con 0x7fa9b814e480 2020-11-20T12:22:35.984 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bf53f700 1 -- 172.21.15.116:0/111447526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b80a7860 con 0x7fa9b814e480 2020-11-20T12:22:35.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9a67fc700 1 -- 172.21.15.116:0/111447526 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b000ac30 con 0x7fa9b814e480 2020-11-20T12:22:35.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.974+0000 7fa9bf53f700 1 -- 172.21.15.116:0/111447526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b8062d70 con 0x7fa9b814e480 2020-11-20T12:22:35.985 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.978+0000 7fa9bf53f700 1 -- 172.21.15.116:0/111447526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b80a7d00 con 0x7fa9b814e480 2020-11-20T12:22:35.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.978+0000 7fa9a67fc700 1 -- 172.21.15.116:0/111447526 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa9b0019030 con 0x7fa9b814e480 2020-11-20T12:22:35.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.978+0000 7fa9a67fc700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 0x7fa990057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:35.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.978+0000 7fa9a67fc700 1 -- 172.21.15.116:0/111447526 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa9b006ddf0 con 0x7fa9b814e480 2020-11-20T12:22:35.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.982+0000 7fa9be53d700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 0x7fa990057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:35.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.982+0000 7fa9be53d700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 0x7fa990057cb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fa9b40034f0 tx=0x7fa9b4009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:35.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:35.986+0000 7fa9a67fc700 1 -- 172.21.15.116:0/111447526 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa9b80a7d00 con 0x7fa9b814e480 2020-11-20T12:22:36.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.150+0000 7fa9bf53f700 1 -- 172.21.15.116:0/111447526 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9b80a7d00 con 0x7fa9b814e480 2020-11-20T12:22:36.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.150+0000 7fa9a67fc700 1 -- 172.21.15.116:0/111447526 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v128) v1 ==== 74+0+16901 (secure 0 0 0) 0x7fa9b0025220 con 0x7fa9b814e480 2020-11-20T12:22:36.158 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:36.158 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":128,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:22:35.338133+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:22:33.838237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2020-11-20T12:22:34.312791+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 -- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 msgr2=0x7fa990057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:36.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 0x7fa990057cb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fa9b40034f0 tx=0x7fa9b4009040).stop 2020-11-20T12:22:36.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 -- 172.21.15.116:0/111447526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b814e480 msgr2=0x7fa9b80bd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:36.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b814e480 0x7fa9b80bd260 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7fa9b000ca90 tx=0x7fa9b0001360).stop 2020-11-20T12:22:36.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 -- 172.21.15.116:0/111447526 shutdown_connections 2020-11-20T12:22:36.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa9900557f0 0x7fa990057cb0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:36.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa9b814e480 0x7fa9b80bd260 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:36.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa9b80663a0 0x7fa9b8062880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:36.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 --2- 172.21.15.116:0/111447526 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa9b80a6480 0x7fa9b80bcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:36.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 -- 172.21.15.116:0/111447526 >> 172.21.15.116:0/111447526 conn(0x7fa9b8060c40 msgr2=0x7fa9b80c45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:36.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 -- 172.21.15.116:0/111447526 shutdown_connections 2020-11-20T12:22:36.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.154+0000 7fa9bf53f700 1 -- 172.21.15.116:0/111447526 wait complete. 2020-11-20T12:22:36.185 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2020-11-20T12:22:36.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.354+0000 7f81fd774700 1 Processor -- start 2020-11-20T12:22:36.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.354+0000 7f81fd774700 1 -- start start 2020-11-20T12:22:36.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81f8062ce0 0x7f81f813a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:36.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f81f8155480 0x7f81f8152630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:36.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81f8152b20 0x7f81f80c2f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:36.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f813ae90 con 0x7f81f8062ce0 2020-11-20T12:22:36.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f8155c70 con 0x7f81f8155480 2020-11-20T12:22:36.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f8155f10 con 0x7f81f8152b20 2020-11-20T12:22:36.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f7fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81f8062ce0 0x7f81f813a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:36.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f7fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f81f8062ce0 0x7f81f813a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52460/0 (socket says 172.21.15.116:52460) 2020-11-20T12:22:36.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f77fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f81f8155480 0x7f81f8152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:36.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f7fff700 1 -- 172.21.15.116:0/2482594741 learned_addr learned my addr 172.21.15.116:0/2482594741 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:36.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fcf73700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81f8152b20 0x7f81f80c2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:36.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f7fff700 1 -- 172.21.15.116:0/2482594741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81f8152b20 msgr2=0x7f81f80c2f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:36.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f7fff700 1 --2- 172.21.15.116:0/2482594741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81f8152b20 0x7f81f80c2f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:36.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f7fff700 1 -- 172.21.15.116:0/2482594741 >> v2:172.21.15.173:3300/0 conn(0x7f81f8155480 msgr2=0x7f81f8152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:36.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f7fff700 1 --2- 172.21.15.116:0/2482594741 >> v2:172.21.15.173:3300/0 conn(0x7f81f8155480 0x7f81f8152630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:36.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f7fff700 1 -- 172.21.15.116:0/2482594741 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f81561b0 con 0x7f81f8062ce0 2020-11-20T12:22:36.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f77fe700 1 --2- 172.21.15.116:0/2482594741 >> v2:172.21.15.173:3300/0 conn(0x7f81f8155480 0x7f81f8152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:36.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f7fff700 1 --2- 172.21.15.116:0/2482594741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81f8062ce0 0x7f81f813a9a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f81f8068950 tx=0x7f81e8006eb0).ready entity=mon.0 client_cookie=42dc53e0129d29ee server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:36.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f6ffd700 1 -- 172.21.15.116:0/2482594741 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8009720 con 0x7f81f8062ce0 2020-11-20T12:22:36.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f6ffd700 1 -- 172.21.15.116:0/2482594741 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e8009880 con 0x7f81f8062ce0 2020-11-20T12:22:36.369 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81f6ffd700 1 -- 172.21.15.116:0/2482594741 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8009c00 con 0x7f81f8062ce0 2020-11-20T12:22:36.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 -- 172.21.15.116:0/2482594741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81f8062ce0 msgr2=0x7f81f813a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:36.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 --2- 172.21.15.116:0/2482594741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81f8062ce0 0x7f81f813a9a0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7f81f8068950 tx=0x7f81e8006eb0).stop 2020-11-20T12:22:36.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 -- 172.21.15.116:0/2482594741 shutdown_connections 2020-11-20T12:22:36.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 --2- 172.21.15.116:0/2482594741 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81f8152b20 0x7f81f80c2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:36.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 --2- 172.21.15.116:0/2482594741 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81f8062ce0 0x7f81f813a9a0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:36.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 --2- 172.21.15.116:0/2482594741 >> v2:172.21.15.173:3300/0 conn(0x7f81f8155480 0x7f81f8152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:36.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 -- 172.21.15.116:0/2482594741 >> 172.21.15.116:0/2482594741 conn(0x7f81f807ff70 msgr2=0x7f81f8080380 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:36.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.358+0000 7f81fd774700 1 -- 172.21.15.116:0/2482594741 shutdown_connections 2020-11-20T12:22:36.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.362+0000 7f81fd774700 1 -- 172.21.15.116:0/2482594741 wait complete. 2020-11-20T12:22:36.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.362+0000 7f81fd774700 1 Processor -- start 2020-11-20T12:22:36.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.362+0000 7f81fd774700 1 -- start start 2020-11-20T12:22:36.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fd774700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81f8152b20 0x7f81f81547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:36.373 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fd774700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81f8155480 0x7f81f8135b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:36.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fd774700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81f80bfda0 0x7f81f81382f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:36.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fd774700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f80c5680 con 0x7f81f80bfda0 2020-11-20T12:22:36.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fd774700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f80c05c0 con 0x7f81f8155480 2020-11-20T12:22:36.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fd774700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f8153e20 con 0x7f81f8152b20 2020-11-20T12:22:36.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fcf73700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81f80bfda0 0x7f81f81382f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:36.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fcf73700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81f80bfda0 0x7f81f81382f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52466/0 (socket says 172.21.15.116:52466) 2020-11-20T12:22:36.376 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fcf73700 1 -- 172.21.15.116:0/3674377351 learned_addr learned my addr 172.21.15.116:0/3674377351 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:36.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81f7fff700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81f8152b20 0x7f81f81547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:36.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81f77fe700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81f8155480 0x7f81f8135b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:36.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fcf73700 1 -- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81f8152b20 msgr2=0x7f81f81547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:36.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fcf73700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81f8152b20 0x7f81f81547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:36.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fcf73700 1 -- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81f8155480 msgr2=0x7f81f8135b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:36.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fcf73700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81f8155480 0x7f81f8135b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:36.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fcf73700 1 -- 172.21.15.116:0/3674377351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f8161280 con 0x7f81f80bfda0 2020-11-20T12:22:36.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fcf73700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81f80bfda0 0x7f81f81382f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f81f000ef50 tx=0x7f81f00091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:36.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81d7fff700 1 -- 172.21.15.116:0/3674377351 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f000cb40 con 0x7f81f80bfda0 2020-11-20T12:22:36.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81d7fff700 1 -- 172.21.15.116:0/3674377351 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f000cca0 con 0x7f81f80bfda0 2020-11-20T12:22:36.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fd774700 1 -- 172.21.15.116:0/3674377351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f80c0320 con 0x7f81f80bfda0 2020-11-20T12:22:36.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81d7fff700 1 -- 172.21.15.116:0/3674377351 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f00123f0 con 0x7f81f80bfda0 2020-11-20T12:22:36.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.366+0000 7f81fd774700 1 -- 172.21.15.116:0/3674377351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f8136050 con 0x7f81f80bfda0 2020-11-20T12:22:36.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.370+0000 7f81fd774700 1 -- 172.21.15.116:0/3674377351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f805cf30 con 0x7f81f80bfda0 2020-11-20T12:22:36.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.370+0000 7f81d7fff700 1 -- 172.21.15.116:0/3674377351 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f81f0022070 con 0x7f81f80bfda0 2020-11-20T12:22:36.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.370+0000 7f81d7fff700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81d00557c0 0x7f81d0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:36.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.370+0000 7f81d7fff700 1 -- 172.21.15.116:0/3674377351 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f81f006ef40 con 0x7f81f80bfda0 2020-11-20T12:22:36.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.370+0000 7f81f7fff700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81d00557c0 0x7f81d0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:36.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.374+0000 7f81f7fff700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81d00557c0 0x7f81d0057c80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f81f80bd6a0 tx=0x7f81e8015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:36.383 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.378+0000 7f81d7fff700 1 -- 172.21.15.116:0/3674377351 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f81f0044de0 con 0x7f81f80bfda0 2020-11-20T12:22:36.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:36.586+0000 7f81fd774700 1 -- 172.21.15.116:0/3674377351 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f81f80c3f30 con 0x7f81f80bfda0 2020-11-20T12:22:37.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.350+0000 7f81d7fff700 1 -- 172.21.15.116:0/3674377351 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) v1 ==== 118+0+0 (secure 0 0 0) 0x7f81f0016030 con 0x7f81f80bfda0 2020-11-20T12:22:37.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 -- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81d00557c0 msgr2=0x7f81d0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:37.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81d00557c0 0x7f81d0057c80 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f81f80bd6a0 tx=0x7f81e8015000).stop 2020-11-20T12:22:37.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 -- 172.21.15.116:0/3674377351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81f80bfda0 msgr2=0x7f81f81382f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:37.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81f80bfda0 0x7f81f81382f0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f81f000ef50 tx=0x7f81f00091d0).stop 2020-11-20T12:22:37.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 -- 172.21.15.116:0/3674377351 shutdown_connections 2020-11-20T12:22:37.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f81d00557c0 0x7f81d0057c80 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f81f80bfda0 0x7f81f81382f0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f81f8152b20 0x7f81f81547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 --2- 172.21.15.116:0/3674377351 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f81f8155480 0x7f81f8135b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 -- 172.21.15.116:0/3674377351 >> 172.21.15.116:0/3674377351 conn(0x7f81f807ff70 msgr2=0x7f81f8137a30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:37.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 -- 172.21.15.116:0/3674377351 shutdown_connections 2020-11-20T12:22:37.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.354+0000 7f81fd774700 1 -- 172.21.15.116:0/3674377351 wait complete. 2020-11-20T12:22:37.388 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:37.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f9148700 1 Processor -- start 2020-11-20T12:22:37.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f9148700 1 -- start start 2020-11-20T12:22:37.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f9148700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f60f4062880 0x7f60f4062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:37.561 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f9148700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60f40677a0 0x7f60f40b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:37.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f9148700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60f41543a0 con 0x7f60f4153450 2020-11-20T12:22:37.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f9148700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f4153920 con 0x7f60f40677a0 2020-11-20T12:22:37.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f9148700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60f4153bc0 con 0x7f60f4062880 2020-11-20T12:22:37.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f8947700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60f40677a0 0x7f60f40b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:37.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f8947700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f60f40677a0 0x7f60f40b9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52472/0 (socket says 172.21.15.116:52472) 2020-11-20T12:22:37.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f8947700 1 -- 172.21.15.116:0/2887531783 learned_addr learned my addr 172.21.15.116:0/2887531783 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:37.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60ebfff700 1 --1- 172.21.15.116:0/2887531783 >> v1:172.21.15.116:6789/0 conn(0x7f60f4153450 0x7f60f40c4c10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33676/0 (socket says 172.21.15.116:33676) 2020-11-20T12:22:37.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f3fff700 1 --2- 172.21.15.116:0/2887531783 >> v2:172.21.15.173:3300/0 conn(0x7f60f4062880 0x7f60f4062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:37.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f8947700 1 -- 172.21.15.116:0/2887531783 >> v2:172.21.15.173:3300/0 conn(0x7f60f4062880 msgr2=0x7f60f4062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:37.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f8947700 1 --2- 172.21.15.116:0/2887531783 >> v2:172.21.15.173:3300/0 conn(0x7f60f4062880 0x7f60f4062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f8947700 1 -- 172.21.15.116:0/2887531783 >> v1:172.21.15.116:6789/0 conn(0x7f60f4153450 legacy=0x7f60f40c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:37.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.554+0000 7f60f8947700 1 -- 172.21.15.116:0/2887531783 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f4067c30 con 0x7f60f40677a0 2020-11-20T12:22:37.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f8947700 1 --2- 172.21.15.116:0/2887531783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60f40677a0 0x7f60f40b9710 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f60f40c1e20 tx=0x7f60ec009290).ready entity=mon.0 client_cookie=8c499d9a38f1004 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:37.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f37fe700 1 -- 172.21.15.116:0/2887531783 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00bbc0 con 0x7f60f40677a0 2020-11-20T12:22:37.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f37fe700 1 -- 172.21.15.116:0/2887531783 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec00bd20 con 0x7f60f40677a0 2020-11-20T12:22:37.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f37fe700 1 -- 172.21.15.116:0/2887531783 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec011610 con 0x7f60f40677a0 2020-11-20T12:22:37.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f9148700 1 -- 172.21.15.116:0/2887531783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60f40677a0 msgr2=0x7f60f40b9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:37.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f9148700 1 --2- 172.21.15.116:0/2887531783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60f40677a0 0x7f60f40b9710 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7f60f40c1e20 tx=0x7f60ec009290).stop 2020-11-20T12:22:37.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f9148700 1 -- 172.21.15.116:0/2887531783 shutdown_connections 2020-11-20T12:22:37.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f9148700 1 --2- 172.21.15.116:0/2887531783 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60f40677a0 0x7f60f40b9710 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f9148700 1 --2- 172.21.15.116:0/2887531783 >> v2:172.21.15.173:3300/0 conn(0x7f60f4062880 0x7f60f4062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f9148700 1 -- 172.21.15.116:0/2887531783 >> 172.21.15.116:0/2887531783 conn(0x7f60f4060c40 msgr2=0x7f60f40c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:37.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f9148700 1 -- 172.21.15.116:0/2887531783 shutdown_connections 2020-11-20T12:22:37.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.558+0000 7f60f9148700 1 -- 172.21.15.116:0/2887531783 wait complete. 2020-11-20T12:22:37.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f9148700 1 Processor -- start 2020-11-20T12:22:37.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f9148700 1 -- start start 2020-11-20T12:22:37.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f9148700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f4062880 0x7f60f4161ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:37.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f9148700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60f40677a0 0x7f60f40bc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:37.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f9148700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60f4153450 0x7f60f414a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:37.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f9148700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40bd6b0 con 0x7f60f40677a0 2020-11-20T12:22:37.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f9148700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60f4162090 con 0x7f60f4153450 2020-11-20T12:22:37.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f9148700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60f40bc9a0 con 0x7f60f4062880 2020-11-20T12:22:37.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60ebfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60f40677a0 0x7f60f40bc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:37.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60ebfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60f40677a0 0x7f60f40bc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52478/0 (socket says 172.21.15.116:52478) 2020-11-20T12:22:37.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60ebfff700 1 -- 172.21.15.116:0/3524177269 learned_addr learned my addr 172.21.15.116:0/3524177269 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:37.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f3fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f4062880 0x7f60f4161ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:37.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f8947700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60f4153450 0x7f60f414a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:37.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60ebfff700 1 -- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f4062880 msgr2=0x7f60f4161ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:37.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60ebfff700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f4062880 0x7f60f4161ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60ebfff700 1 -- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60f4153450 msgr2=0x7f60f414a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:37.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60ebfff700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60f4153450 0x7f60f414a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60ebfff700 1 -- 172.21.15.116:0/3524177269 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60f40bcc30 con 0x7f60f40677a0 2020-11-20T12:22:37.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60ebfff700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60f40677a0 0x7f60f40bc4b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f60dc00c6e0 tx=0x7f60dc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:37.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f0ff9700 1 -- 172.21.15.116:0/3524177269 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0072e0 con 0x7f60f40677a0 2020-11-20T12:22:37.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f0ff9700 1 -- 172.21.15.116:0/3524177269 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60dc005530 con 0x7f60f40677a0 2020-11-20T12:22:37.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f9148700 1 -- 172.21.15.116:0/3524177269 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60f40bced0 con 0x7f60f40677a0 2020-11-20T12:22:37.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f0ff9700 1 -- 172.21.15.116:0/3524177269 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60dc0058b0 con 0x7f60f40677a0 2020-11-20T12:22:37.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.562+0000 7f60f9148700 1 -- 172.21.15.116:0/3524177269 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60f414a8c0 con 0x7f60f40677a0 2020-11-20T12:22:37.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.566+0000 7f60f9148700 1 -- 172.21.15.116:0/3524177269 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60f405cf30 con 0x7f60f40677a0 2020-11-20T12:22:37.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.566+0000 7f60f0ff9700 1 -- 172.21.15.116:0/3524177269 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f60dc01a060 con 0x7f60f40677a0 2020-11-20T12:22:37.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.566+0000 7f60f0ff9700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60d00557c0 0x7f60d0057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:37.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.566+0000 7f60f0ff9700 1 -- 172.21.15.116:0/3524177269 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f60dc06ddb0 con 0x7f60f40677a0 2020-11-20T12:22:37.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.566+0000 7f60f3fff700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60d00557c0 0x7f60d0057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:37.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.570+0000 7f60f3fff700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60d00557c0 0x7f60d0057c80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f60e400ba00 tx=0x7f60e4008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:37.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.574+0000 7f60f0ff9700 1 -- 172.21.15.116:0/3524177269 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f60dc072070 con 0x7f60f40677a0 2020-11-20T12:22:37.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.778+0000 7f60f9148700 1 -- 172.21.15.116:0/3524177269 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60f40bd490 con 0x7f60f40677a0 2020-11-20T12:22:37.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.778+0000 7f60f0ff9700 1 -- 172.21.15.116:0/3524177269 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2853 (secure 0 0 0) 0x7f60dc018030 con 0x7f60f40677a0 2020-11-20T12:22:37.786 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:37.787 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":75,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/363707569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":363707569},{"type":"v1","addr":"172.21.15.173:6835","nonce":363707569}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":71},{"gid":7202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/1854325940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1854325940},{"type":"v1","addr":"172.21.15.116:6834","nonce":1854325940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72},{"gid":8437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3920088127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3920088127},{"type":"v1","addr":"172.21.15.173:6837","nonce":3920088127}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:22:35.338520+0000","modified":"2020-11-20T12:22:37.346491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8589":{"gid":8589,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1101967997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1101967997},{"type":"v1","addr":"172.21.15.116:6835","nonce":1101967997}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-11-20T12:22:37.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 -- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60d00557c0 msgr2=0x7f60d0057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:37.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60d00557c0 0x7f60d0057c80 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f60e400ba00 tx=0x7f60e4008040).stop 2020-11-20T12:22:37.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 -- 172.21.15.116:0/3524177269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60f40677a0 msgr2=0x7f60f40bc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:37.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60f40677a0 0x7f60f40bc4b0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7f60dc00c6e0 tx=0x7f60dc007c10).stop 2020-11-20T12:22:37.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 -- 172.21.15.116:0/3524177269 shutdown_connections 2020-11-20T12:22:37.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f60d00557c0 0x7f60d0057c80 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f60f4153450 0x7f60f414a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f60f4062880 0x7f60f4161ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 --2- 172.21.15.116:0/3524177269 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f60f40677a0 0x7f60f40bc4b0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:37.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 -- 172.21.15.116:0/3524177269 >> 172.21.15.116:0/3524177269 conn(0x7f60f4060c40 msgr2=0x7f60f40c0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:37.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 -- 172.21.15.116:0/3524177269 shutdown_connections 2020-11-20T12:22:37.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.782+0000 7f60f9148700 1 -- 172.21.15.116:0/3524177269 wait complete. 2020-11-20T12:22:37.791 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 75 2020-11-20T12:22:37.814 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:37.986 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.978+0000 7f3d1d0bb700 1 Processor -- start 2020-11-20T12:22:37.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.978+0000 7f3d1d0bb700 1 -- start start 2020-11-20T12:22:37.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d1d0bb700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3d180c3780 0x7f3d180c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d1d0bb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d180c1970 0x7f3d18150560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:37.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d1d0bb700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d180a6950 con 0x7f3d180a6280 2020-11-20T12:22:37.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d1d0bb700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d1814fed0 con 0x7f3d180c3780 2020-11-20T12:22:37.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d1d0bb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180a6730 con 0x7f3d180c1970 2020-11-20T12:22:37.993 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d1659c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f3d180a6280 0x7f3d180c22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33686/0 (socket says 172.21.15.116:33686) 2020-11-20T12:22:37.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d1659c700 1 -- 172.21.15.116:0/2204348173 learned_addr learned my addr 172.21.15.116:0/2204348173 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:37.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d1759e700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d180c1970 0x7f3d18150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:37.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d16d9d700 1 --2- 172.21.15.116:0/2204348173 >> v2:172.21.15.173:3300/0 conn(0x7f3d180c3780 0x7f3d180c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:37.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593110488 0 0) 0x7f3d180a6950 con 0x7f3d180a6280 2020-11-20T12:22:37.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d04003240 con 0x7f3d180a6280 2020-11-20T12:22:37.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2389395610 0 0) 0x7f3d04003240 con 0x7f3d180a6280 2020-11-20T12:22:37.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d180c1970 msgr2=0x7f3d18150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:37.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d180c1970 0x7f3d18150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.000 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 >> v2:172.21.15.173:3300/0 conn(0x7f3d180c3780 msgr2=0x7f3d180c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:38.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 --2- 172.21.15.116:0/2204348173 >> v2:172.21.15.173:3300/0 conn(0x7f3d180c3780 0x7f3d180c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.001 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d180a6bf0 con 0x7f3d180a6280 2020-11-20T12:22:38.002 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f3d0c0014e0 con 0x7f3d180a6280 2020-11-20T12:22:38.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 >> v1:172.21.15.116:6789/0 conn(0x7f3d180a6280 legacy=0x7f3d180c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d04003760 0x7f3d04003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d040044e0 0x7f3d04004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d180c1970 0x7f3d04004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180a6730 con 0x7f3d04003760 2020-11-20T12:22:38.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d04005390 con 0x7f3d040044e0 2020-11-20T12:22:38.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.982+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d04004320 con 0x7f3d180c1970 2020-11-20T12:22:38.021 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d16d9d700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d04003760 0x7f3d04003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d1659c700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d040044e0 0x7f3d04004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.022 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d1759e700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d180c1970 0x7f3d04004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d16d9d700 1 -- 172.21.15.116:0/2204348173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d180c1970 msgr2=0x7f3d04004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:38.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d16d9d700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d180c1970 0x7f3d04004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d16d9d700 1 -- 172.21.15.116:0/2204348173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d040044e0 msgr2=0x7f3d04004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:38.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d16d9d700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d040044e0 0x7f3d04004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d16d9d700 1 -- 172.21.15.116:0/2204348173 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d04003240 con 0x7f3d04003760 2020-11-20T12:22:38.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d16d9d700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d04003760 0x7f3d04003b80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f3d0800bc00 tx=0x7f3d08006df0).ready entity=mon.0 client_cookie=5b1f5bd3e30f6b59 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0800b310 con 0x7f3d04003760 2020-11-20T12:22:38.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0800ea90 con 0x7f3d04003760 2020-11-20T12:22:38.028 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d15d9b700 1 -- 172.21.15.116:0/2204348173 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0800ee10 con 0x7f3d04003760 2020-11-20T12:22:38.029 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/2204348173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d04003760 msgr2=0x7f3d04003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d1d0bb700 1 --2- 172.21.15.116:0/2204348173 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d04003760 0x7f3d04003b80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7f3d0800bc00 tx=0x7f3d08006df0).stop 2020-11-20T12:22:38.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d1759e700 1 -- 172.21.15.116:0/2204348173 reap_dead start 2020-11-20T12:22:38.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/2204348173 shutdown_connections 2020-11-20T12:22:38.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/2204348173 >> 172.21.15.116:0/2204348173 conn(0x7f3d180c4f80 msgr2=0x7f3d180c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:38.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/2204348173 shutdown_connections 2020-11-20T12:22:38.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.986+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/2204348173 wait complete. 2020-11-20T12:22:38.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.990+0000 7f3d1d0bb700 1 Processor -- start 2020-11-20T12:22:38.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.990+0000 7f3d1d0bb700 1 -- start start 2020-11-20T12:22:38.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1d0bb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d04003760 0x7f3d180628c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1d0bb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d18153220 0x7f3d1813aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1d0bb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d181536a0 0x7f3d1813b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1d0bb700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180a7470 con 0x7f3d181536a0 2020-11-20T12:22:38.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1d0bb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180a7660 con 0x7f3d04003760 2020-11-20T12:22:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1d0bb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d180a7900 con 0x7f3d18153220 2020-11-20T12:22:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d181536a0 0x7f3d1813b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1759e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d181536a0 0x7f3d1813b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52500/0 (socket says 172.21.15.116:52500) 2020-11-20T12:22:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1759e700 1 -- 172.21.15.116:0/4104320862 learned_addr learned my addr 172.21.15.116:0/4104320862 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:38.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1659c700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d18153220 0x7f3d1813aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d16d9d700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d04003760 0x7f3d180628c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1759e700 1 -- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d18153220 msgr2=0x7f3d1813aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1759e700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d18153220 0x7f3d1813aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1759e700 1 -- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d04003760 msgr2=0x7f3d180628c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1759e700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d04003760 0x7f3d180628c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1759e700 1 -- 172.21.15.116:0/4104320862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d180a7ba0 con 0x7f3d181536a0 2020-11-20T12:22:38.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1659c700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d18153220 0x7f3d1813aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:38.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1759e700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d181536a0 0x7f3d1813b090 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7f3d18065540 tx=0x7f3d10008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:38.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3cfeffd700 1 -- 172.21.15.116:0/4104320862 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1000e9d0 con 0x7f3d181536a0 2020-11-20T12:22:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3cfeffd700 1 -- 172.21.15.116:0/4104320862 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1000eb30 con 0x7f3d181536a0 2020-11-20T12:22:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/4104320862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d180a7e40 con 0x7f3d181536a0 2020-11-20T12:22:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3cfeffd700 1 -- 172.21.15.116:0/4104320862 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10014430 con 0x7f3d181536a0 2020-11-20T12:22:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.994+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/4104320862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d1813b580 con 0x7f3d181536a0 2020-11-20T12:22:38.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.998+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/4104320862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d18067660 con 0x7f3d181536a0 2020-11-20T12:22:38.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.998+0000 7f3cfeffd700 1 -- 172.21.15.116:0/4104320862 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3d10014820 con 0x7f3d181536a0 2020-11-20T12:22:38.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.998+0000 7f3cfeffd700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3cf40557c0 0x7f3cf4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.998+0000 7f3cfeffd700 1 -- 172.21.15.116:0/4104320862 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f3d1006ed60 con 0x7f3d181536a0 2020-11-20T12:22:38.039 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:37.998+0000 7f3d16d9d700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3cf40557c0 0x7f3cf4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.002+0000 7f3d16d9d700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3cf40557c0 0x7f3cf4057c80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f3d0800b4d0 tx=0x7f3d08019000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:38.040 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.006+0000 7f3cfeffd700 1 -- 172.21.15.116:0/4104320862 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3d18067660 con 0x7f3d181536a0 2020-11-20T12:22:38.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.162+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/4104320862 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3d18067660 con 0x7f3d181536a0 2020-11-20T12:22:38.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3cfeffd700 1 -- 172.21.15.116:0/4104320862 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f3d18067660 con 0x7f3d181536a0 2020-11-20T12:22:38.172 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:38.173 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":129,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:22:36.342598+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:22:33.838237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2020-11-20T12:22:34.312791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:38.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3cf40557c0 msgr2=0x7f3cf4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3cf40557c0 0x7f3cf4057c80 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f3d0800b4d0 tx=0x7f3d08019000).stop 2020-11-20T12:22:38.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/4104320862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d181536a0 msgr2=0x7f3d1813b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d181536a0 0x7f3d1813b090 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7f3d18065540 tx=0x7f3d10008040).stop 2020-11-20T12:22:38.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/4104320862 shutdown_connections 2020-11-20T12:22:38.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f3cf40557c0 0x7f3cf4057c80 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3d181536a0 0x7f3d1813b090 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3d04003760 0x7f3d180628c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 --2- 172.21.15.116:0/4104320862 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f3d18153220 0x7f3d1813aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/4104320862 >> 172.21.15.116:0/4104320862 conn(0x7f3d180c4f80 msgr2=0x7f3d1814daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:38.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/4104320862 shutdown_connections 2020-11-20T12:22:38.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.166+0000 7f3d1d0bb700 1 -- 172.21.15.116:0/4104320862 wait complete. 2020-11-20T12:22:38.200 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2020-11-20T12:22:38.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.386+0000 7f9b98960700 1 Processor -- start 2020-11-20T12:22:38.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.386+0000 7f9b98960700 1 -- start start 2020-11-20T12:22:38.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.386+0000 7f9b98960700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f9b940a9750 0x7f9b940c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.386+0000 7f9b98960700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9b9414a1d0 0x7f9b940be240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.386+0000 7f9b98960700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b940a9c20 con 0x7f9b94149df0 2020-11-20T12:22:38.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.386+0000 7f9b98960700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b940a9ec0 con 0x7f9b9414a1d0 2020-11-20T12:22:38.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.386+0000 7f9b98960700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b94153670 con 0x7f9b940a9750 2020-11-20T12:22:38.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.386+0000 7f9b92d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9b9414a1d0 0x7f9b940be240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.386+0000 7f9b92d9d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f9b9414a1d0 0x7f9b940be240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52506/0 (socket says 172.21.15.116:52506) 2020-11-20T12:22:38.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.386+0000 7f9b92d9d700 1 -- 172.21.15.116:0/2195146733 learned_addr learned my addr 172.21.15.116:0/2195146733 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:38.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b91d9b700 1 --2- 172.21.15.116:0/2195146733 >> v2:172.21.15.173:3300/0 conn(0x7f9b940a9750 0x7f9b940c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b92d9d700 1 -- 172.21.15.116:0/2195146733 >> v2:172.21.15.173:3300/0 conn(0x7f9b940a9750 msgr2=0x7f9b940c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:38.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b92d9d700 1 --2- 172.21.15.116:0/2195146733 >> v2:172.21.15.173:3300/0 conn(0x7f9b940a9750 0x7f9b940c0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b92d9d700 1 -- 172.21.15.116:0/2195146733 >> v1:172.21.15.173:6789/0 conn(0x7f9b94149df0 legacy=0x7f9b94153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:38.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b92d9d700 1 -- 172.21.15.116:0/2195146733 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b94153910 con 0x7f9b9414a1d0 2020-11-20T12:22:38.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b92d9d700 1 --2- 172.21.15.116:0/2195146733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b9414a1d0 0x7f9b940be240 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f9b940c4dc0 tx=0x7f9b8c009290).ready entity=mon.0 client_cookie=7f6e64c8537d866a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:38.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b9159a700 1 -- 172.21.15.116:0/2195146733 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00bb80 con 0x7f9b9414a1d0 2020-11-20T12:22:38.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b9159a700 1 -- 172.21.15.116:0/2195146733 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c00bce0 con 0x7f9b9414a1d0 2020-11-20T12:22:38.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b9159a700 1 -- 172.21.15.116:0/2195146733 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c011610 con 0x7f9b9414a1d0 2020-11-20T12:22:38.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b98960700 1 -- 172.21.15.116:0/2195146733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b9414a1d0 msgr2=0x7f9b940be240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b98960700 1 --2- 172.21.15.116:0/2195146733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b9414a1d0 0x7f9b940be240 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f9b940c4dc0 tx=0x7f9b8c009290).stop 2020-11-20T12:22:38.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b98960700 1 -- 172.21.15.116:0/2195146733 shutdown_connections 2020-11-20T12:22:38.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b98960700 1 --2- 172.21.15.116:0/2195146733 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b9414a1d0 0x7f9b940be240 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b98960700 1 --2- 172.21.15.116:0/2195146733 >> v2:172.21.15.173:3300/0 conn(0x7f9b940a9750 0x7f9b940c0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b98960700 1 -- 172.21.15.116:0/2195146733 >> 172.21.15.116:0/2195146733 conn(0x7f9b9405ba80 msgr2=0x7f9b940c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:38.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b98960700 1 -- 172.21.15.116:0/2195146733 shutdown_connections 2020-11-20T12:22:38.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.390+0000 7f9b98960700 1 -- 172.21.15.116:0/2195146733 wait complete. 2020-11-20T12:22:38.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.394+0000 7f9b98960700 1 Processor -- start 2020-11-20T12:22:38.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.394+0000 7f9b98960700 1 -- start start 2020-11-20T12:22:38.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b98960700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b940a9750 0x7f9b94139e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b98960700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b94149df0 0x7f9b940ba300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b98960700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b9414a1d0 0x7f9b940bceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b98960700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940bb7b0 con 0x7f9b9414a1d0 2020-11-20T12:22:38.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b98960700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940bad30 con 0x7f9b940a9750 2020-11-20T12:22:38.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b98960700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b940bafd0 con 0x7f9b94149df0 2020-11-20T12:22:38.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b92d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b9414a1d0 0x7f9b940bceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b92d9d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b9414a1d0 0x7f9b940bceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52514/0 (socket says 172.21.15.116:52514) 2020-11-20T12:22:38.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b92d9d700 1 -- 172.21.15.116:0/401630160 learned_addr learned my addr 172.21.15.116:0/401630160 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:38.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b91d9b700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b94149df0 0x7f9b940ba300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b9259c700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b940a9750 0x7f9b94139e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b92d9d700 1 -- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b94149df0 msgr2=0x7f9b940ba300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b92d9d700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b94149df0 0x7f9b940ba300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b92d9d700 1 -- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b940a9750 msgr2=0x7f9b94139e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b92d9d700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b940a9750 0x7f9b94139e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b92d9d700 1 -- 172.21.15.116:0/401630160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b940bb270 con 0x7f9b9414a1d0 2020-11-20T12:22:38.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b92d9d700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b9414a1d0 0x7f9b940bceb0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f9b940611d0 tx=0x7f9b8c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:38.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b827fc700 1 -- 172.21.15.116:0/401630160 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c00bb80 con 0x7f9b9414a1d0 2020-11-20T12:22:38.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b827fc700 1 -- 172.21.15.116:0/401630160 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b8c00be50 con 0x7f9b9414a1d0 2020-11-20T12:22:38.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b827fc700 1 -- 172.21.15.116:0/401630160 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b8c010e10 con 0x7f9b9414a1d0 2020-11-20T12:22:38.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b98960700 1 -- 172.21.15.116:0/401630160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b940bb510 con 0x7f9b9414a1d0 2020-11-20T12:22:38.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b98960700 1 -- 172.21.15.116:0/401630160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9413a3c0 con 0x7f9b9414a1d0 2020-11-20T12:22:38.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.398+0000 7f9b98960700 1 -- 172.21.15.116:0/401630160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b940bcc10 con 0x7f9b9414a1d0 2020-11-20T12:22:38.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.402+0000 7f9b827fc700 1 -- 172.21.15.116:0/401630160 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f9b8c017070 con 0x7f9b9414a1d0 2020-11-20T12:22:38.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.402+0000 7f9b827fc700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9b680557f0 0x7f9b68057cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.402+0000 7f9b827fc700 1 -- 172.21.15.116:0/401630160 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9b8c0707a0 con 0x7f9b9414a1d0 2020-11-20T12:22:38.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.402+0000 7f9b9259c700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9b680557f0 0x7f9b68057cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.402+0000 7f9b9259c700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9b680557f0 0x7f9b68057cb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f9b84003500 tx=0x7f9b84009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:38.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.406+0000 7f9b827fc700 1 -- 172.21.15.116:0/401630160 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f9b940bcc10 con 0x7f9b9414a1d0 2020-11-20T12:22:38.629 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.622+0000 7f9b98960700 1 -- 172.21.15.116:0/401630160 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f9b940bcc10 con 0x7f9b9414a1d0 2020-11-20T12:22:38.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.630+0000 7f9b827fc700 1 -- 172.21.15.116:0/401630160 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v36) v1 ==== 184+0+0 (secure 0 0 0) 0x7f9b940bcc10 con 0x7f9b9414a1d0 2020-11-20T12:22:38.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 -- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9b680557f0 msgr2=0x7f9b68057cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9b680557f0 0x7f9b68057cb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f9b84003500 tx=0x7f9b84009040).stop 2020-11-20T12:22:38.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 -- 172.21.15.116:0/401630160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b9414a1d0 msgr2=0x7f9b940bceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b9414a1d0 0x7f9b940bceb0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f9b940611d0 tx=0x7f9b8c016000).stop 2020-11-20T12:22:38.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 -- 172.21.15.116:0/401630160 shutdown_connections 2020-11-20T12:22:38.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9b680557f0 0x7f9b68057cb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f9b9414a1d0 0x7f9b940bceb0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f9b940a9750 0x7f9b94139e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 --2- 172.21.15.116:0/401630160 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f9b94149df0 0x7f9b940ba300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 -- 172.21.15.116:0/401630160 >> 172.21.15.116:0/401630160 conn(0x7f9b9405ba80 msgr2=0x7f9b94066ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:38.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 -- 172.21.15.116:0/401630160 shutdown_connections 2020-11-20T12:22:38.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.634+0000 7f9b98960700 1 -- 172.21.15.116:0/401630160 wait complete. 2020-11-20T12:22:38.644 INFO:teuthology.orchestra.run.smithi116.stderr:updated caps for client.0 2020-11-20T12:22:38.667 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:38.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.838+0000 7f6ed0811700 1 Processor -- start 2020-11-20T12:22:38.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ed0811700 1 -- start start 2020-11-20T12:22:38.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ed0811700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f6ecc14e480 0x7f6ecc1529e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ed0811700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc066800 0x7f6ecc0677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ed0811700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ecc0c5010 con 0x7f6ecc0663a0 2020-11-20T12:22:38.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ed0811700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ecc067340 con 0x7f6ecc14e480 2020-11-20T12:22:38.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ed0811700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc066e00 con 0x7f6ecc066800 2020-11-20T12:22:38.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ec3fff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f6ecc0663a0 0x7f6ecc0a6480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33722/0 (socket says 172.21.15.116:33722) 2020-11-20T12:22:38.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ec3fff700 1 -- 172.21.15.116:0/2266232826 learned_addr learned my addr 172.21.15.116:0/2266232826 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:38.862 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ecb7fe700 1 --2- 172.21.15.116:0/2266232826 >> v2:172.21.15.173:3300/0 conn(0x7f6ecc14e480 0x7f6ecc1529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ecbfff700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc066800 0x7f6ecc0677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.863 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3490440077 0 0) 0x7f6ecc0c5010 con 0x7f6ecc0663a0 2020-11-20T12:22:38.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.842+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6eac003240 con 0x7f6ecc0663a0 2020-11-20T12:22:38.865 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3580989725 0 0) 0x7f6eac003240 con 0x7f6ecc0663a0 2020-11-20T12:22:38.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc066800 msgr2=0x7f6ecc0677a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:38.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc066800 0x7f6ecc0677a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.867 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 >> v2:172.21.15.173:3300/0 conn(0x7f6ecc14e480 msgr2=0x7f6ecc1529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:38.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 --2- 172.21.15.116:0/2266232826 >> v2:172.21.15.173:3300/0 conn(0x7f6ecc14e480 0x7f6ecc1529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.868 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc0670a0 con 0x7f6ecc0663a0 2020-11-20T12:22:38.869 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7f6eb40014e0 con 0x7f6ecc0663a0 2020-11-20T12:22:38.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 >> v1:172.21.15.116:6789/0 conn(0x7f6ecc0663a0 legacy=0x7f6ecc0a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.870 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eac003730 0x7f6eac003b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eac0044e0 0x7f6eac004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.871 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc066800 0x7f6eac004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.872 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc066e00 con 0x7f6eac0044e0 2020-11-20T12:22:38.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac005390 con 0x7f6eac003730 2020-11-20T12:22:38.873 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0042f0 con 0x7f6ecc066800 2020-11-20T12:22:38.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ec3fff700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eac0044e0 0x7f6eac004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.874 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecb7fe700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eac003730 0x7f6eac003b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecbfff700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc066800 0x7f6eac004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ec3fff700 1 -- 172.21.15.116:0/2266232826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc066800 msgr2=0x7f6eac004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:38.875 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ec3fff700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc066800 0x7f6eac004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ec3fff700 1 -- 172.21.15.116:0/2266232826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eac003730 msgr2=0x7f6eac003b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:38.877 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ec3fff700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6eac003730 0x7f6eac003b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ec3fff700 1 -- 172.21.15.116:0/2266232826 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac003240 con 0x7f6eac0044e0 2020-11-20T12:22:38.878 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ec3fff700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eac0044e0 0x7f6eac004960 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f6eb400c490 tx=0x7f6eb4007750).ready entity=mon.0 client_cookie=d7d3ad3eb4dabf16 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:38.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb40070d0 con 0x7f6eac0044e0 2020-11-20T12:22:38.879 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb4007250 con 0x7f6eac0044e0 2020-11-20T12:22:38.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.846+0000 7f6ecaffd700 1 -- 172.21.15.116:0/2266232826 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb40055f0 con 0x7f6eac0044e0 2020-11-20T12:22:38.880 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.850+0000 7f6ed0811700 1 -- 172.21.15.116:0/2266232826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eac0044e0 msgr2=0x7f6eac004960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.850+0000 7f6ed0811700 1 --2- 172.21.15.116:0/2266232826 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6eac0044e0 0x7f6eac004960 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f6eb400c490 tx=0x7f6eb4007750).stop 2020-11-20T12:22:38.881 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.850+0000 7f6ecbfff700 1 -- 172.21.15.116:0/2266232826 reap_dead start 2020-11-20T12:22:38.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.850+0000 7f6ed0811700 1 -- 172.21.15.116:0/2266232826 shutdown_connections 2020-11-20T12:22:38.882 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.850+0000 7f6ed0811700 1 -- 172.21.15.116:0/2266232826 >> 172.21.15.116:0/2266232826 conn(0x7f6ecc060c40 msgr2=0x7f6ecc0c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:38.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.850+0000 7f6ed0811700 1 -- 172.21.15.116:0/2266232826 shutdown_connections 2020-11-20T12:22:38.883 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.850+0000 7f6ed0811700 1 -- 172.21.15.116:0/2266232826 wait complete. 2020-11-20T12:22:38.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.850+0000 7f6ed0811700 1 Processor -- start 2020-11-20T12:22:38.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.850+0000 7f6ed0811700 1 -- start start 2020-11-20T12:22:38.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ed0811700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ecc0a65c0 0x7f6ecc0a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ed0811700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ecc0a7320 0x7f6ecc0a77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ed0811700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc0a9260 0x7f6ecc0a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ed0811700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0a9bb0 con 0x7f6ecc0a7320 2020-11-20T12:22:38.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ed0811700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0a7e30 con 0x7f6ecc0a65c0 2020-11-20T12:22:38.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ed0811700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ecc0a80d0 con 0x7f6ecc0a9260 2020-11-20T12:22:38.893 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ecc0a7320 0x7f6ecc0a77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec3fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ecc0a7320 0x7f6ecc0a77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52532/0 (socket says 172.21.15.116:52532) 2020-11-20T12:22:38.894 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ecb7fe700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ecc0a65c0 0x7f6ecc0a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec3fff700 1 -- 172.21.15.116:0/2319746551 learned_addr learned my addr 172.21.15.116:0/2319746551 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:38.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ecbfff700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc0a9260 0x7f6ecc0a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.895 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec3fff700 1 -- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc0a9260 msgr2=0x7f6ecc0a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec3fff700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc0a9260 0x7f6ecc0a96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.896 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec3fff700 1 -- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ecc0a65c0 msgr2=0x7f6ecc0a69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:38.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec3fff700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ecc0a65c0 0x7f6ecc0a69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:38.897 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec3fff700 1 -- 172.21.15.116:0/2319746551 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc0a8370 con 0x7f6ecc0a7320 2020-11-20T12:22:38.898 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec3fff700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ecc0a7320 0x7f6ecc0a77a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f6eb400c330 tx=0x7f6eb4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:38.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec37fe700 1 -- 172.21.15.116:0/2319746551 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb4000f10 con 0x7f6ecc0a7320 2020-11-20T12:22:38.899 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec37fe700 1 -- 172.21.15.116:0/2319746551 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6eb40013e0 con 0x7f6ecc0a7320 2020-11-20T12:22:38.900 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ec37fe700 1 -- 172.21.15.116:0/2319746551 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6eb40136a0 con 0x7f6ecc0a7320 2020-11-20T12:22:38.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ed0811700 1 -- 172.21.15.116:0/2319746551 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ecc0a8610 con 0x7f6ecc0a7320 2020-11-20T12:22:38.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.854+0000 7f6ed0811700 1 -- 172.21.15.116:0/2319746551 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ecc151070 con 0x7f6ecc0a7320 2020-11-20T12:22:38.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.858+0000 7f6ed0811700 1 -- 172.21.15.116:0/2319746551 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ecc0ba380 con 0x7f6ecc0a7320 2020-11-20T12:22:38.901 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.858+0000 7f6ec37fe700 1 -- 172.21.15.116:0/2319746551 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6eb4026070 con 0x7f6ecc0a7320 2020-11-20T12:22:38.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.858+0000 7f6ec37fe700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ea80557c0 0x7f6ea8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:38.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.858+0000 7f6ec37fe700 1 -- 172.21.15.116:0/2319746551 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f6eb406f240 con 0x7f6ecc0a7320 2020-11-20T12:22:38.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.858+0000 7f6ecb7fe700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ea80557c0 0x7f6ea8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:38.902 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.862+0000 7f6ecb7fe700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ea80557c0 0x7f6ea8057c80 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f6ebc006fd0 tx=0x7f6ebc006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:38.903 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:38.866+0000 7f6ec37fe700 1 -- 172.21.15.116:0/2319746551 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6eb40450e0 con 0x7f6ecc0a7320 2020-11-20T12:22:39.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.070+0000 7f6ed0811700 1 -- 172.21.15.116:0/2319746551 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ecc147850 con 0x7f6ecc0a7320 2020-11-20T12:22:39.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.070+0000 7f6ec37fe700 1 -- 172.21.15.116:0/2319746551 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2853 (secure 0 0 0) 0x7f6eb4048700 con 0x7f6ecc0a7320 2020-11-20T12:22:39.079 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:39.079 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":75,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/363707569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":363707569},{"type":"v1","addr":"172.21.15.173:6835","nonce":363707569}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":71},{"gid":7202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/1854325940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1854325940},{"type":"v1","addr":"172.21.15.116:6834","nonce":1854325940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72},{"gid":8437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3920088127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3920088127},{"type":"v1","addr":"172.21.15.173:6837","nonce":3920088127}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:22:35.338520+0000","modified":"2020-11-20T12:22:37.346491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8589":{"gid":8589,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1101967997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1101967997},{"type":"v1","addr":"172.21.15.116:6835","nonce":1101967997}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-11-20T12:22:39.081 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 -- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ea80557c0 msgr2=0x7f6ea8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:39.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ea80557c0 0x7f6ea8057c80 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f6ebc006fd0 tx=0x7f6ebc006df0).stop 2020-11-20T12:22:39.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 -- 172.21.15.116:0/2319746551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ecc0a7320 msgr2=0x7f6ecc0a77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:39.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ecc0a7320 0x7f6ecc0a77a0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f6eb400c330 tx=0x7f6eb4015000).stop 2020-11-20T12:22:39.082 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 -- 172.21.15.116:0/2319746551 shutdown_connections 2020-11-20T12:22:39.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f6ea80557c0 0x7f6ea8057c80 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:39.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f6ecc0a9260 0x7f6ecc0a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:39.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f6ecc0a65c0 0x7f6ecc0a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:39.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 --2- 172.21.15.116:0/2319746551 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f6ecc0a7320 0x7f6ecc0a77a0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:39.083 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 -- 172.21.15.116:0/2319746551 >> 172.21.15.116:0/2319746551 conn(0x7f6ecc060c40 msgr2=0x7f6ecc0c0670 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:39.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 -- 172.21.15.116:0/2319746551 shutdown_connections 2020-11-20T12:22:39.084 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:39.074+0000 7f6ed0811700 1 -- 172.21.15.116:0/2319746551 wait complete. 2020-11-20T12:22:39.085 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 75 2020-11-20T12:22:39.508 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:22:35.338520+0000', 'modified': '2020-11-20T12:22:37.346491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8589': {'gid': 8589, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/1101967997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 1101967997}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 1101967997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:22:39.509 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:22:39.510 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:22:39.627 INFO:tasks.ceph.mds.c.smithi116.stderr:2020-11-20T12:22:39.622+0000 7fbc16b46700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2020-11-20T12:22:39.706 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":8589,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":4.3421188409999996,"mdsmap_epoch":75,"osdmap_epoch":129,"osdmap_epoch_barrier":129,"uptime":9.8886388919999995} 2020-11-20T12:22:39.707 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 8589, "mdsmap_epoch": 75, "osdmap_epoch": 129, "osdmap_epoch_barrier": 129, "rank_uptime": 4.342118841, "state": "up:active", "uptime": 9.888638892, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:22:39.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:22:39.708 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:22:39.708 INFO:tasks.cephfs.mount:self.client.name = client.0 2020-11-20T12:22:39.709 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2020-11-20T12:22:39.709 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:22:39.709 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:22:39.710 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:22:39.710 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:22:39.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:39.711 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:22:39.727 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:22:39.727 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:22:39.727 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:22:39.728 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:39.728 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:22:39.728 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:39.728 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:39.729 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:39.729 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:39.729 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:39.729 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:39.730 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:39.730 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:22:39.730 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:39.730 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:22:39.730 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:39.731 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:22:39.731 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:39.731 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:22:39.731 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:39.732 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:22:39.732 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:39.732 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:22:39.732 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:39.732 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:22:39.733 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:39.733 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:22:39.733 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:39.733 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:39.734 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:22:39.734 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:22:39.734 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:39.734 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:39.734 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:22:39.735 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:22:39.735 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:39.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:39.736 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:22:39.807 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:22:39.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:39.811 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:22:39.824 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:22:39.825 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:22:39.825 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:22:39.825 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:22:39.826 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:22:39.826 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:22:39.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:39.828 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:22:39.828 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:22:39.828 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:22:39.829 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:22:39.829 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:22:39.830 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:22:39.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:39.930 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:22:39.946 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:22:39.947 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:22:39.948 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2020-11-20T12:22:39.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:39.949 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.0 up 2020-11-20T12:22:40.020 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:40.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc4167fc700 1 Processor -- start 2020-11-20T12:22:40.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc4167fc700 1 -- start start 2020-11-20T12:22:40.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc4167fc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc410062880 0x7fc410062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc4167fc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc410153450 0x7fc4100677a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc4167fc700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4101543a0 con 0x7fc4100c4c10 2020-11-20T12:22:40.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc4167fc700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc410153920 con 0x7fc410153450 2020-11-20T12:22:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc4167fc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc410153bc0 con 0x7fc410062880 2020-11-20T12:22:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc414ff9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc410153450 0x7fc4100677a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc415ffb700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fc4100c4c10 0x7fc4100b9710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33744/0 (socket says 172.21.15.116:33744) 2020-11-20T12:22:40.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc415ffb700 1 -- 172.21.15.116:0/1825874759 learned_addr learned my addr 172.21.15.116:0/1825874759 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:40.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc4157fa700 1 --2- 172.21.15.116:0/1825874759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc410062880 0x7fc410062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc414ff9700 1 -- 172.21.15.116:0/1825874759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc410062880 msgr2=0x7fc410062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:40.198 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc414ff9700 1 --2- 172.21.15.116:0/1825874759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc410062880 0x7fc410062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc414ff9700 1 -- 172.21.15.116:0/1825874759 >> v1:172.21.15.116:6789/0 conn(0x7fc4100c4c10 legacy=0x7fc4100b9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.186+0000 7fc414ff9700 1 -- 172.21.15.116:0/1825874759 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410153e60 con 0x7fc410153450 2020-11-20T12:22:40.199 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc414ff9700 1 --2- 172.21.15.116:0/1825874759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc410153450 0x7fc4100677a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7fc40000b9a0 tx=0x7fc400006df0).ready entity=mon.0 client_cookie=ea53e3d1a9d7d18e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:40.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc3fffff700 1 -- 172.21.15.116:0/1825874759 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4000097b0 con 0x7fc410153450 2020-11-20T12:22:40.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc3fffff700 1 -- 172.21.15.116:0/1825874759 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc400009910 con 0x7fc410153450 2020-11-20T12:22:40.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc3fffff700 1 -- 172.21.15.116:0/1825874759 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400009c90 con 0x7fc410153450 2020-11-20T12:22:40.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc4167fc700 1 -- 172.21.15.116:0/1825874759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc410153450 msgr2=0x7fc4100677a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc4167fc700 1 --2- 172.21.15.116:0/1825874759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc410153450 0x7fc4100677a0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7fc40000b9a0 tx=0x7fc400006df0).stop 2020-11-20T12:22:40.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc4167fc700 1 -- 172.21.15.116:0/1825874759 shutdown_connections 2020-11-20T12:22:40.201 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc4167fc700 1 --2- 172.21.15.116:0/1825874759 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc410062880 0x7fc410062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc4167fc700 1 --2- 172.21.15.116:0/1825874759 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc410153450 0x7fc4100677a0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc4167fc700 1 -- 172.21.15.116:0/1825874759 >> 172.21.15.116:0/1825874759 conn(0x7fc410060c40 msgr2=0x7fc4100c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:40.202 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc4167fc700 1 -- 172.21.15.116:0/1825874759 shutdown_connections 2020-11-20T12:22:40.203 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.190+0000 7fc4167fc700 1 -- 172.21.15.116:0/1825874759 wait complete. 2020-11-20T12:22:40.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.194+0000 7fc4167fc700 1 Processor -- start 2020-11-20T12:22:40.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.194+0000 7fc4167fc700 1 -- start start 2020-11-20T12:22:40.205 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.194+0000 7fc4167fc700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc410062880 0x7fc4100c2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.194+0000 7fc4167fc700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4100c4c10 0x7fc410152410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.194+0000 7fc4167fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc410153450 0x7fc410165cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.194+0000 7fc4167fc700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100c3600 con 0x7fc410153450 2020-11-20T12:22:40.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.194+0000 7fc4167fc700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100c38a0 con 0x7fc410062880 2020-11-20T12:22:40.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.194+0000 7fc4167fc700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100c3b40 con 0x7fc4100c4c10 2020-11-20T12:22:40.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc415ffb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc410153450 0x7fc410165cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc415ffb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc410153450 0x7fc410165cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52548/0 (socket says 172.21.15.116:52548) 2020-11-20T12:22:40.208 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc415ffb700 1 -- 172.21.15.116:0/2112048322 learned_addr learned my addr 172.21.15.116:0/2112048322 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:40.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc4157fa700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc410062880 0x7fc4100c2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc414ff9700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4100c4c10 0x7fc410152410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc415ffb700 1 -- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4100c4c10 msgr2=0x7fc410152410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc415ffb700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4100c4c10 0x7fc410152410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc415ffb700 1 -- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc410062880 msgr2=0x7fc4100c2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc415ffb700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc410062880 0x7fc4100c2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc415ffb700 1 -- 172.21.15.116:0/2112048322 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc410165820 con 0x7fc410153450 2020-11-20T12:22:40.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc415ffb700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc410153450 0x7fc410165cc0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fc40800ca60 tx=0x7fc408007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:40.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc3fd7fa700 1 -- 172.21.15.116:0/2112048322 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408005b80 con 0x7fc410153450 2020-11-20T12:22:40.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc4167fc700 1 -- 172.21.15.116:0/2112048322 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc410165540 con 0x7fc410153450 2020-11-20T12:22:40.212 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc3fd7fa700 1 -- 172.21.15.116:0/2112048322 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc408005ce0 con 0x7fc410153450 2020-11-20T12:22:40.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc3fd7fa700 1 -- 172.21.15.116:0/2112048322 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408007320 con 0x7fc410153450 2020-11-20T12:22:40.213 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc4167fc700 1 -- 172.21.15.116:0/2112048322 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4100c3360 con 0x7fc410153450 2020-11-20T12:22:40.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.198+0000 7fc4167fc700 1 -- 172.21.15.116:0/2112048322 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4101659b0 con 0x7fc410153450 2020-11-20T12:22:40.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.202+0000 7fc3fd7fa700 1 -- 172.21.15.116:0/2112048322 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc408010030 con 0x7fc410153450 2020-11-20T12:22:40.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.202+0000 7fc3fd7fa700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc3e80557c0 0x7fc3e8057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.215 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.202+0000 7fc3fd7fa700 1 -- 172.21.15.116:0/2112048322 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc40806e7c0 con 0x7fc410153450 2020-11-20T12:22:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.202+0000 7fc4157fa700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc3e80557c0 0x7fc3e8057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.202+0000 7fc4157fa700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc3e80557c0 0x7fc3e8057c80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7fc40c00ba00 tx=0x7fc40c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:40.216 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.206+0000 7fc3fd7fa700 1 -- 172.21.15.116:0/2112048322 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc408044550 con 0x7fc410153450 2020-11-20T12:22:40.372 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.366+0000 7fc4167fc700 1 -- 172.21.15.116:0/2112048322 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc41005cf30 con 0x7fc410153450 2020-11-20T12:22:40.374 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.366+0000 7fc3fd7fa700 1 -- 172.21.15.116:0/2112048322 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16888 (secure 0 0 0) 0x7fc408047b70 con 0x7fc410153450 2020-11-20T12:22:40.375 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:40.375 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":129,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:22:36.342598+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:22:33.838237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2020-11-20T12:22:34.312791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:40.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 -- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc3e80557c0 msgr2=0x7fc3e8057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc3e80557c0 0x7fc3e8057c80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7fc40c00ba00 tx=0x7fc40c008040).stop 2020-11-20T12:22:40.377 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 -- 172.21.15.116:0/2112048322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc410153450 msgr2=0x7fc410165cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc410153450 0x7fc410165cc0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fc40800ca60 tx=0x7fc408007f40).stop 2020-11-20T12:22:40.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 -- 172.21.15.116:0/2112048322 shutdown_connections 2020-11-20T12:22:40.378 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc3e80557c0 0x7fc3e8057c80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc410153450 0x7fc410165cc0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc410062880 0x7fc4100c2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 --2- 172.21.15.116:0/2112048322 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc4100c4c10 0x7fc410152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.379 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 -- 172.21.15.116:0/2112048322 >> 172.21.15.116:0/2112048322 conn(0x7fc410060c40 msgr2=0x7fc410066bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:40.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 -- 172.21.15.116:0/2112048322 shutdown_connections 2020-11-20T12:22:40.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.370+0000 7fc4167fc700 1 -- 172.21.15.116:0/2112048322 wait complete. 2020-11-20T12:22:40.401 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:22:33.838237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:22:34.312791+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:22:40.402 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:22:40.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.578+0000 7f1c32b74700 1 Processor -- start 2020-11-20T12:22:40.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.582+0000 7f1c32b74700 1 -- start start 2020-11-20T12:22:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.582+0000 7f1c32b74700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1c2c0aa7d0 0x7f1c2c0a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.582+0000 7f1c32b74700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f1c2c0aabb0 0x7f1c2c151e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.590 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.582+0000 7f1c32b74700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c2c0ace40 con 0x7f1c2c0ad540 2020-11-20T12:22:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.582+0000 7f1c32b74700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2c1526c0 con 0x7f1c2c0aa7d0 2020-11-20T12:22:40.591 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.582+0000 7f1c32b74700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c2c152960 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.582+0000 7f1c31b72700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1c2c0aa7d0 0x7f1c2c0a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c31b72700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f1c2c0aa7d0 0x7f1c2c0a8400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52552/0 (socket says 172.21.15.116:52552) 2020-11-20T12:22:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c31b72700 1 -- 172.21.15.116:0/262266481 learned_addr learned my addr 172.21.15.116:0/262266481 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c31371700 1 --2- 172.21.15.116:0/262266481 >> v2:172.21.15.173:3300/0 conn(0x7f1c2c0aabb0 0x7f1c2c151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.592 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c31b72700 1 -- 172.21.15.116:0/262266481 >> v2:172.21.15.173:3300/0 conn(0x7f1c2c0aabb0 msgr2=0x7f1c2c151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c31b72700 1 --2- 172.21.15.116:0/262266481 >> v2:172.21.15.173:3300/0 conn(0x7f1c2c0aabb0 0x7f1c2c151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.593 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c31b72700 1 -- 172.21.15.116:0/262266481 >> v1:172.21.15.173:6789/0 conn(0x7f1c2c0ad540 legacy=0x7f1c2c0ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c31b72700 1 -- 172.21.15.116:0/262266481 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c152c00 con 0x7f1c2c0aa7d0 2020-11-20T12:22:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c31b72700 1 --2- 172.21.15.116:0/262266481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c2c0aa7d0 0x7f1c2c0a8400 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f1c2c155390 tx=0x7f1c28006eb0).ready entity=mon.0 client_cookie=c56a1bf65d00e2a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c30b70700 1 -- 172.21.15.116:0/262266481 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28009810 con 0x7f1c2c0aa7d0 2020-11-20T12:22:40.594 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c30b70700 1 -- 172.21.15.116:0/262266481 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c28009970 con 0x7f1c2c0aa7d0 2020-11-20T12:22:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c30b70700 1 -- 172.21.15.116:0/262266481 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28009cf0 con 0x7f1c2c0aa7d0 2020-11-20T12:22:40.595 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c32b74700 1 -- 172.21.15.116:0/262266481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c2c0aa7d0 msgr2=0x7f1c2c0a8400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c32b74700 1 --2- 172.21.15.116:0/262266481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c2c0aa7d0 0x7f1c2c0a8400 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f1c2c155390 tx=0x7f1c28006eb0).stop 2020-11-20T12:22:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c32b74700 1 -- 172.21.15.116:0/262266481 shutdown_connections 2020-11-20T12:22:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c32b74700 1 --2- 172.21.15.116:0/262266481 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c2c0aa7d0 0x7f1c2c0a8400 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c32b74700 1 --2- 172.21.15.116:0/262266481 >> v2:172.21.15.173:3300/0 conn(0x7f1c2c0aabb0 0x7f1c2c151e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.596 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.586+0000 7f1c32b74700 1 -- 172.21.15.116:0/262266481 >> 172.21.15.116:0/262266481 conn(0x7f1c2c0678a0 msgr2=0x7f1c2c067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:40.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.590+0000 7f1c32b74700 1 -- 172.21.15.116:0/262266481 shutdown_connections 2020-11-20T12:22:40.597 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.590+0000 7f1c32b74700 1 -- 172.21.15.116:0/262266481 wait complete. 2020-11-20T12:22:40.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.590+0000 7f1c32b74700 1 Processor -- start 2020-11-20T12:22:40.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.590+0000 7f1c32b74700 1 -- start start 2020-11-20T12:22:40.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c32b74700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c2c0aabb0 0x7f1c2c0a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c32b74700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1c2c0ad540 0x7f1c2c14f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.600 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c32b74700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c2c14fb50 0x7f1c2c1569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c32b74700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c0a0570 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c32b74700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c14ff90 con 0x7f1c2c14fb50 2020-11-20T12:22:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c32b74700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c2c14ae00 con 0x7f1c2c0ad540 2020-11-20T12:22:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c31b72700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c2c0aabb0 0x7f1c2c0a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c31b72700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c2c0aabb0 0x7f1c2c0a0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52560/0 (socket says 172.21.15.116:52560) 2020-11-20T12:22:40.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c31b72700 1 -- 172.21.15.116:0/260612689 learned_addr learned my addr 172.21.15.116:0/260612689 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:40.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c31371700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1c2c0ad540 0x7f1c2c14f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c32373700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c2c14fb50 0x7f1c2c1569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c31b72700 1 -- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1c2c0ad540 msgr2=0x7f1c2c14f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c31b72700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1c2c0ad540 0x7f1c2c14f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c31b72700 1 -- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c2c14fb50 msgr2=0x7f1c2c1569a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c31b72700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c2c14fb50 0x7f1c2c1569a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c31b72700 1 -- 172.21.15.116:0/260612689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c09f690 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c31b72700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c2c0aabb0 0x7f1c2c0a0080 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f1c28006fd0 tx=0x7f1c28015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:40.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c19ffb700 1 -- 172.21.15.116:0/260612689 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c28011b70 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c19ffb700 1 -- 172.21.15.116:0/260612689 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c28011cd0 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c32b74700 1 -- 172.21.15.116:0/260612689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c2c157520 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c19ffb700 1 -- 172.21.15.116:0/260612689 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2801e6b0 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.594+0000 7f1c32b74700 1 -- 172.21.15.116:0/260612689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c2c157a60 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.598+0000 7f1c32b74700 1 -- 172.21.15.116:0/260612689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c2c05cf30 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.598+0000 7f1c19ffb700 1 -- 172.21.15.116:0/260612689 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f1c28025030 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.598+0000 7f1c19ffb700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1c040557c0 0x7f1c04057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.598+0000 7f1c19ffb700 1 -- 172.21.15.116:0/260612689 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f1c2806fc50 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.598+0000 7f1c31371700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1c040557c0 0x7f1c04057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.602+0000 7f1c31371700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1c040557c0 0x7f1c04057c80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f1c1c006fd0 tx=0x7f1c1c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:40.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.606+0000 7f1c19ffb700 1 -- 172.21.15.116:0/260612689 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f1c28045ba0 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 -- 172.21.15.116:0/260612689 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f1c2c0601b0 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c19ffb700 1 -- 172.21.15.116:0/260612689 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v129) v1 ==== 110+0+10 (secure 0 0 0) 0x7f1c28023030 con 0x7f1c2c0aabb0 2020-11-20T12:22:40.776 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:22:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 -- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1c040557c0 msgr2=0x7f1c04057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1c040557c0 0x7f1c04057c80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f1c1c006fd0 tx=0x7f1c1c008040).stop 2020-11-20T12:22:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 -- 172.21.15.116:0/260612689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c2c0aabb0 msgr2=0x7f1c2c0a0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c2c0aabb0 0x7f1c2c0a0080 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f1c28006fd0 tx=0x7f1c28015000).stop 2020-11-20T12:22:40.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 -- 172.21.15.116:0/260612689 shutdown_connections 2020-11-20T12:22:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f1c040557c0 0x7f1c04057c80 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f1c2c14fb50 0x7f1c2c1569a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f1c2c0aabb0 0x7f1c2c0a0080 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 --2- 172.21.15.116:0/260612689 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f1c2c0ad540 0x7f1c2c14f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 -- 172.21.15.116:0/260612689 >> 172.21.15.116:0/260612689 conn(0x7f1c2c0678a0 msgr2=0x7f1c2c0a8430 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 -- 172.21.15.116:0/260612689 shutdown_connections 2020-11-20T12:22:40.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.770+0000 7f1c32b74700 1 -- 172.21.15.116:0/260612689 wait complete. 2020-11-20T12:22:40.802 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:22:40.987 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.982+0000 7f4435326700 1 Processor -- start 2020-11-20T12:22:40.988 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.982+0000 7f4435326700 1 -- start start 2020-11-20T12:22:40.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.982+0000 7f4435326700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f44300aa7d0 0x7f44300a8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.991 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.982+0000 7f4435326700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f44300aabb0 0x7f4430151e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:40.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f4435326700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44300ace40 con 0x7f44300ad540 2020-11-20T12:22:40.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f4435326700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44301526c0 con 0x7f44300aabb0 2020-11-20T12:22:40.992 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f4435326700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4430152960 con 0x7f44300aa7d0 2020-11-20T12:22:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f442f7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f44300aabb0 0x7f4430151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.994 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f4434b25700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f44300ad540 0x7f44300ad980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33772/0 (socket says 172.21.15.116:33772) 2020-11-20T12:22:40.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f4434b25700 1 -- 172.21.15.116:0/1507610927 learned_addr learned my addr 172.21.15.116:0/1507610927 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:40.995 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f442ffff700 1 --2- 172.21.15.116:0/1507610927 >> v2:172.21.15.173:3300/0 conn(0x7f44300aa7d0 0x7f44300a8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:40.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f442f7fe700 1 -- 172.21.15.116:0/1507610927 >> v2:172.21.15.173:3300/0 conn(0x7f44300aa7d0 msgr2=0x7f44300a8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:40.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f442f7fe700 1 --2- 172.21.15.116:0/1507610927 >> v2:172.21.15.173:3300/0 conn(0x7f44300aa7d0 0x7f44300a8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:40.996 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f442f7fe700 1 -- 172.21.15.116:0/1507610927 >> v1:172.21.15.116:6789/0 conn(0x7f44300ad540 legacy=0x7f44300ad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:40.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f442f7fe700 1 -- 172.21.15.116:0/1507610927 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4430152c00 con 0x7f44300aabb0 2020-11-20T12:22:40.997 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f442effd700 1 -- 172.21.15.116:0/1507610927 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2630296059 0 0) 0x7f44300ace40 con 0x7f44300ad540 2020-11-20T12:22:40.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f442f7fe700 1 --2- 172.21.15.116:0/1507610927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44300aabb0 0x7f4430151e00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f4430081950 tx=0x7f4424006df0).ready entity=mon.0 client_cookie=75c9ba30469af52 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:40.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f442effd700 1 -- 172.21.15.116:0/1507610927 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44240097b0 con 0x7f44300aabb0 2020-11-20T12:22:40.998 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f442effd700 1 -- 172.21.15.116:0/1507610927 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4424009910 con 0x7f44300aabb0 2020-11-20T12:22:40.999 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.986+0000 7f442effd700 1 -- 172.21.15.116:0/1507610927 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4424009c90 con 0x7f44300aabb0 2020-11-20T12:22:41.003 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 -- 172.21.15.116:0/1507610927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44300aabb0 msgr2=0x7f4430151e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 --2- 172.21.15.116:0/1507610927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44300aabb0 0x7f4430151e00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f4430081950 tx=0x7f4424006df0).stop 2020-11-20T12:22:41.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 -- 172.21.15.116:0/1507610927 shutdown_connections 2020-11-20T12:22:41.004 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 --2- 172.21.15.116:0/1507610927 >> v2:172.21.15.173:3300/0 conn(0x7f44300aa7d0 0x7f44300a8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 --2- 172.21.15.116:0/1507610927 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44300aabb0 0x7f4430151e00 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.005 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 -- 172.21.15.116:0/1507610927 >> 172.21.15.116:0/1507610927 conn(0x7f44300678a0 msgr2=0x7f4430067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:41.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 -- 172.21.15.116:0/1507610927 shutdown_connections 2020-11-20T12:22:41.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 -- 172.21.15.116:0/1507610927 wait complete. 2020-11-20T12:22:41.006 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 Processor -- start 2020-11-20T12:22:41.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 -- start start 2020-11-20T12:22:41.007 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44300aa7d0 0x7f4430063e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44300ad540 0x7f4430149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4430168ae0 0x7f4430168f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.008 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44300661b0 con 0x7f44300aa7d0 2020-11-20T12:22:41.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4430169430 con 0x7f4430168ae0 2020-11-20T12:22:41.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.990+0000 7f4435326700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44300674c0 con 0x7f44300ad540 2020-11-20T12:22:41.009 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f442ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44300aa7d0 0x7f4430063e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f442ffff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44300aa7d0 0x7f4430063e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52576/0 (socket says 172.21.15.116:52576) 2020-11-20T12:22:41.010 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f442ffff700 1 -- 172.21.15.116:0/1457456263 learned_addr learned my addr 172.21.15.116:0/1457456263 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:41.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f4434b25700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4430168ae0 0x7f4430168f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f442f7fe700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44300ad540 0x7f4430149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.011 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f442ffff700 1 -- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44300ad540 msgr2=0x7f4430149550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f442ffff700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44300ad540 0x7f4430149550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.012 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f442ffff700 1 -- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4430168ae0 msgr2=0x7f4430168f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f442ffff700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4430168ae0 0x7f4430168f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f442ffff700 1 -- 172.21.15.116:0/1457456263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4430066390 con 0x7f44300aa7d0 2020-11-20T12:22:41.013 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f442f7fe700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44300ad540 0x7f4430149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:41.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f442ffff700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44300aa7d0 0x7f4430063e80 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f442000eba0 tx=0x7f4420006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:41.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f440ffff700 1 -- 172.21.15.116:0/1457456263 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442000c9b0 con 0x7f44300aa7d0 2020-11-20T12:22:41.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f440ffff700 1 -- 172.21.15.116:0/1457456263 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442000cb10 con 0x7f44300aa7d0 2020-11-20T12:22:41.014 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f4435326700 1 -- 172.21.15.116:0/1457456263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4430066630 con 0x7f44300aa7d0 2020-11-20T12:22:41.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f440ffff700 1 -- 172.21.15.116:0/1457456263 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442000ce90 con 0x7f44300aa7d0 2020-11-20T12:22:41.015 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f4435326700 1 -- 172.21.15.116:0/1457456263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4430064370 con 0x7f44300aa7d0 2020-11-20T12:22:41.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.994+0000 7f4435326700 1 -- 172.21.15.116:0/1457456263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443005cf30 con 0x7f44300aa7d0 2020-11-20T12:22:41.016 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.998+0000 7f440ffff700 1 -- 172.21.15.116:0/1457456263 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4420012600 con 0x7f44300aa7d0 2020-11-20T12:22:41.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.998+0000 7f440ffff700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44080557c0 0x7f4408057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.998+0000 7f440ffff700 1 -- 172.21.15.116:0/1457456263 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f442006eb40 con 0x7f44300aa7d0 2020-11-20T12:22:41.018 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:40.998+0000 7f442f7fe700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44080557c0 0x7f4408057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.002+0000 7f442f7fe700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44080557c0 0x7f4408057c80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f4430068770 tx=0x7f4424015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:41.019 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.006+0000 7f440ffff700 1 -- 172.21.15.116:0/1457456263 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4420044870 con 0x7f44300aa7d0 2020-11-20T12:22:41.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.166+0000 7f4435326700 1 -- 172.21.15.116:0/1457456263 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f44300601b0 con 0x7f44300aa7d0 2020-11-20T12:22:41.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.166+0000 7f440ffff700 1 -- 172.21.15.116:0/1457456263 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7f4420047e90 con 0x7f44300aa7d0 2020-11-20T12:22:41.173 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:41.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.166+0000 7f4435326700 1 -- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44080557c0 msgr2=0x7f4408057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.166+0000 7f4435326700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44080557c0 0x7f4408057c80 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f4430068770 tx=0x7f4424015000).stop 2020-11-20T12:22:41.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.170+0000 7f4435326700 1 -- 172.21.15.116:0/1457456263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44300aa7d0 msgr2=0x7f4430063e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.170+0000 7f4435326700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44300aa7d0 0x7f4430063e80 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f442000eba0 tx=0x7f4420006df0).stop 2020-11-20T12:22:41.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.170+0000 7f4435326700 1 -- 172.21.15.116:0/1457456263 shutdown_connections 2020-11-20T12:22:41.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.170+0000 7f4435326700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f44080557c0 0x7f4408057c80 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.170+0000 7f4435326700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4430168ae0 0x7f4430168f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.170+0000 7f4435326700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f44300aa7d0 0x7f4430063e80 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.170+0000 7f4435326700 1 --2- 172.21.15.116:0/1457456263 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f44300ad540 0x7f4430149550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.170+0000 7f4435326700 1 -- 172.21.15.116:0/1457456263 >> 172.21.15.116:0/1457456263 conn(0x7f44300678a0 msgr2=0x7f44300c7b30 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:41.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.170+0000 7f4435326700 1 -- 172.21.15.116:0/1457456263 shutdown_connections 2020-11-20T12:22:41.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.170+0000 7f4435326700 1 -- 172.21.15.116:0/1457456263 wait complete. 2020-11-20T12:22:41.198 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:22:41.380 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.374+0000 7fa56c9df700 1 Processor -- start 2020-11-20T12:22:41.381 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.374+0000 7fa56c9df700 1 -- start start 2020-11-20T12:22:41.384 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.374+0000 7fa56c9df700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa5681518c0 0x7fa568151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.374+0000 7fa56c9df700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 0x7fa568061550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.374+0000 7fa56c9df700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa568061a40 0x7fa5680ad010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.374+0000 7fa56c9df700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa568061ea0 con 0x7fa5681518c0 2020-11-20T12:22:41.385 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.374+0000 7fa56c9df700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5680c9450 con 0x7fa568061a40 2020-11-20T12:22:41.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.374+0000 7fa56c9df700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5680a92f0 con 0x7fa5680c9010 2020-11-20T12:22:41.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa565d9b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 0x7fa568061550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56659c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa5681518c0 0x7fa568151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.386 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa565d9b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 0x7fa568061550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:33298/0 (socket says 172.21.15.116:33298) 2020-11-20T12:22:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa565d9b700 1 -- 172.21.15.116:0/2304302979 learned_addr learned my addr 172.21.15.116:0/2304302979 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa566d9d700 1 --2- 172.21.15.116:0/2304302979 >> v2:172.21.15.173:3300/0 conn(0x7fa568061a40 0x7fa5680ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa565d9b700 1 -- 172.21.15.116:0/2304302979 >> v2:172.21.15.173:3300/0 conn(0x7fa568061a40 msgr2=0x7fa5680ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa565d9b700 1 --2- 172.21.15.116:0/2304302979 >> v2:172.21.15.173:3300/0 conn(0x7fa568061a40 0x7fa5680ad010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.387 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa565d9b700 1 -- 172.21.15.116:0/2304302979 >> v2:172.21.15.116:3300/0 conn(0x7fa5681518c0 msgr2=0x7fa568151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa565d9b700 1 --2- 172.21.15.116:0/2304302979 >> v2:172.21.15.116:3300/0 conn(0x7fa5681518c0 0x7fa568151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa565d9b700 1 -- 172.21.15.116:0/2304302979 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5680a9590 con 0x7fa5680c9010 2020-11-20T12:22:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa566d9d700 1 --2- 172.21.15.116:0/2304302979 >> v2:172.21.15.173:3300/0 conn(0x7fa568061a40 0x7fa5680ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa565d9b700 1 --2- 172.21.15.116:0/2304302979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 0x7fa568061550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fa5680c5370 tx=0x7fa55c006eb0).ready entity=mon.2 client_cookie=8c7296b5bc40cc92 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:41.388 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56559a700 1 -- 172.21.15.116:0/2304302979 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0097d0 con 0x7fa5680c9010 2020-11-20T12:22:41.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56559a700 1 -- 172.21.15.116:0/2304302979 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c009930 con 0x7fa5680c9010 2020-11-20T12:22:41.389 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56559a700 1 -- 172.21.15.116:0/2304302979 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c009cb0 con 0x7fa5680c9010 2020-11-20T12:22:41.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56c9df700 1 -- 172.21.15.116:0/2304302979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 msgr2=0x7fa568061550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.390 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56c9df700 1 --2- 172.21.15.116:0/2304302979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 0x7fa568061550 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7fa5680c5370 tx=0x7fa55c006eb0).stop 2020-11-20T12:22:41.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56c9df700 1 -- 172.21.15.116:0/2304302979 shutdown_connections 2020-11-20T12:22:41.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56c9df700 1 --2- 172.21.15.116:0/2304302979 >> v2:172.21.15.173:3300/0 conn(0x7fa568061a40 0x7fa5680ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.391 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56c9df700 1 --2- 172.21.15.116:0/2304302979 >> v2:172.21.15.116:3300/0 conn(0x7fa5681518c0 0x7fa568151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56c9df700 1 --2- 172.21.15.116:0/2304302979 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 0x7fa568061550 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56c9df700 1 -- 172.21.15.116:0/2304302979 >> 172.21.15.116:0/2304302979 conn(0x7fa568068370 msgr2=0x7fa5680c8900 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:41.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56c9df700 1 -- 172.21.15.116:0/2304302979 shutdown_connections 2020-11-20T12:22:41.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.378+0000 7fa56c9df700 1 -- 172.21.15.116:0/2304302979 wait complete. 2020-11-20T12:22:41.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.382+0000 7fa56c9df700 1 Processor -- start 2020-11-20T12:22:41.394 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.382+0000 7fa56c9df700 1 -- start start 2020-11-20T12:22:41.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56c9df700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa568061a40 0x7fa568161d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56c9df700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 0x7fa5680bd860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56c9df700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5681518c0 0x7fa5680bee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56c9df700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5680c3490 con 0x7fa568061a40 2020-11-20T12:22:41.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56c9df700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5680be7c0 con 0x7fa5681518c0 2020-11-20T12:22:41.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56c9df700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5680bdd50 con 0x7fa5680c9010 2020-11-20T12:22:41.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa568061a40 0x7fa568161d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56659c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa568061a40 0x7fa568161d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52586/0 (socket says 172.21.15.116:52586) 2020-11-20T12:22:41.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56659c700 1 -- 172.21.15.116:0/796843656 learned_addr learned my addr 172.21.15.116:0/796843656 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:41.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa565d9b700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 0x7fa5680bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa566d9d700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5681518c0 0x7fa5680bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56659c700 1 -- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 msgr2=0x7fa5680bd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56659c700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 0x7fa5680bd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56659c700 1 -- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5681518c0 msgr2=0x7fa5680bee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56659c700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5681518c0 0x7fa5680bee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56659c700 1 -- 172.21.15.116:0/796843656 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5680bdfe0 con 0x7fa568061a40 2020-11-20T12:22:41.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56659c700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa568061a40 0x7fa568161d60 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7fa55800eba0 tx=0x7fa558008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:41.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa5567fc700 1 -- 172.21.15.116:0/796843656 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55800c820 con 0x7fa568061a40 2020-11-20T12:22:41.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa5567fc700 1 -- 172.21.15.116:0/796843656 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55800c980 con 0x7fa568061a40 2020-11-20T12:22:41.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56c9df700 1 -- 172.21.15.116:0/796843656 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5680be280 con 0x7fa568061a40 2020-11-20T12:22:41.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa5567fc700 1 -- 172.21.15.116:0/796843656 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55800cd00 con 0x7fa568061a40 2020-11-20T12:22:41.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.386+0000 7fa56c9df700 1 -- 172.21.15.116:0/796843656 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5680bf360 con 0x7fa568061a40 2020-11-20T12:22:41.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.390+0000 7fa56c9df700 1 -- 172.21.15.116:0/796843656 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa568069950 con 0x7fa568061a40 2020-11-20T12:22:41.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.390+0000 7fa5567fc700 1 -- 172.21.15.116:0/796843656 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa558022070 con 0x7fa568061a40 2020-11-20T12:22:41.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.390+0000 7fa5567fc700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa53c0557c0 0x7fa53c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.390+0000 7fa5567fc700 1 -- 172.21.15.116:0/796843656 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa55806eaf0 con 0x7fa568061a40 2020-11-20T12:22:41.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.394+0000 7fa565d9b700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa53c0557c0 0x7fa53c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.394+0000 7fa565d9b700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa53c0557c0 0x7fa53c057c80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fa55c00b1b0 tx=0x7fa55c015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:41.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.398+0000 7fa5567fc700 1 -- 172.21.15.116:0/796843656 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa558044880 con 0x7fa568061a40 2020-11-20T12:22:41.559 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 -- 172.21.15.116:0/796843656 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5680608c0 con 0x7fa568061a40 2020-11-20T12:22:41.560 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa5567fc700 1 -- 172.21.15.116:0/796843656 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa558016070 con 0x7fa568061a40 2020-11-20T12:22:41.560 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:41.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 -- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa53c0557c0 msgr2=0x7fa53c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.562 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa53c0557c0 0x7fa53c057c80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7fa55c00b1b0 tx=0x7fa55c015000).stop 2020-11-20T12:22:41.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 -- 172.21.15.116:0/796843656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa568061a40 msgr2=0x7fa568161d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa568061a40 0x7fa568161d60 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7fa55800eba0 tx=0x7fa558008040).stop 2020-11-20T12:22:41.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 -- 172.21.15.116:0/796843656 shutdown_connections 2020-11-20T12:22:41.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa53c0557c0 0x7fa53c057c80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.563 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa5681518c0 0x7fa5680bee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa568061a40 0x7fa568161d60 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.564 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 --2- 172.21.15.116:0/796843656 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa5680c9010 0x7fa5680bd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 -- 172.21.15.116:0/796843656 >> 172.21.15.116:0/796843656 conn(0x7fa568068370 msgr2=0x7fa5680c7000 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:41.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.554+0000 7fa56c9df700 1 -- 172.21.15.116:0/796843656 shutdown_connections 2020-11-20T12:22:41.565 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.558+0000 7fa56c9df700 1 -- 172.21.15.116:0/796843656 wait complete. 2020-11-20T12:22:41.586 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:22:41.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.770+0000 7f8b7692f700 1 Processor -- start 2020-11-20T12:22:41.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.774+0000 7f8b7692f700 1 -- start start 2020-11-20T12:22:41.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.774+0000 7f8b7692f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b701558f0 0x7f8b70151e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.774+0000 7f8b7692f700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8b700a8910 0x7f8b700aa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.774+0000 7f8b7692f700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b701563e0 con 0x7f8b700a84b0 2020-11-20T12:22:41.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.774+0000 7f8b7692f700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b701527c0 con 0x7f8b700a8910 2020-11-20T12:22:41.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.774+0000 7f8b7692f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b70152a60 con 0x7f8b701558f0 2020-11-20T12:22:41.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.774+0000 7f8b7612e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f8b700a8910 0x7f8b700aa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.774+0000 7f8b7512c700 1 --1- >> v1:172.21.15.173:6789/0 conn(0x7f8b700a84b0 0x7f8b70155cd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:58260/0 (socket says 172.21.15.116:58260) 2020-11-20T12:22:41.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7512c700 1 -- 172.21.15.116:0/143308982 learned_addr learned my addr 172.21.15.116:0/143308982 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:41.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7592d700 1 --2- 172.21.15.116:0/143308982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b701558f0 0x7f8b70151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7612e700 1 -- 172.21.15.116:0/143308982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b701558f0 msgr2=0x7f8b70151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:41.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7612e700 1 --2- 172.21.15.116:0/143308982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b701558f0 0x7f8b70151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7612e700 1 -- 172.21.15.116:0/143308982 >> v1:172.21.15.173:6789/0 conn(0x7f8b700a84b0 legacy=0x7f8b70155cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7612e700 1 -- 172.21.15.116:0/143308982 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b70152d00 con 0x7f8b700a8910 2020-11-20T12:22:41.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7612e700 1 --2- 172.21.15.116:0/143308982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b700a8910 0x7f8b700aa820 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f8b68009c90 tx=0x7f8b680091d0).ready entity=mon.0 client_cookie=d474035347401b0d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:41.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7492b700 1 -- 172.21.15.116:0/143308982 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800ba70 con 0x7f8b700a8910 2020-11-20T12:22:41.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7492b700 1 -- 172.21.15.116:0/143308982 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6800bbd0 con 0x7f8b700a8910 2020-11-20T12:22:41.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7492b700 1 -- 172.21.15.116:0/143308982 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68011400 con 0x7f8b700a8910 2020-11-20T12:22:41.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7692f700 1 -- 172.21.15.116:0/143308982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b700a8910 msgr2=0x7f8b700aa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7692f700 1 --2- 172.21.15.116:0/143308982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b700a8910 0x7f8b700aa820 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f8b68009c90 tx=0x7f8b680091d0).stop 2020-11-20T12:22:41.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7692f700 1 -- 172.21.15.116:0/143308982 shutdown_connections 2020-11-20T12:22:41.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7692f700 1 --2- 172.21.15.116:0/143308982 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b700a8910 0x7f8b700aa820 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7692f700 1 --2- 172.21.15.116:0/143308982 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b701558f0 0x7f8b70151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7692f700 1 -- 172.21.15.116:0/143308982 >> 172.21.15.116:0/143308982 conn(0x7f8b700678a0 msgr2=0x7f8b70067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:41.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7692f700 1 -- 172.21.15.116:0/143308982 shutdown_connections 2020-11-20T12:22:41.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.778+0000 7f8b7692f700 1 -- 172.21.15.116:0/143308982 wait complete. 2020-11-20T12:22:41.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.782+0000 7f8b7692f700 1 Processor -- start 2020-11-20T12:22:41.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.782+0000 7f8b7692f700 1 -- start start 2020-11-20T12:22:41.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7692f700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b700a84b0 0x7f8b70164700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7692f700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b700a8910 0x7f8b70164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.796 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7692f700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b701558f0 0x7f8b700c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7692f700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c86c0 con 0x7f8b700a84b0 2020-11-20T12:22:41.797 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7692f700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c8960 con 0x7f8b700a8910 2020-11-20T12:22:41.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7692f700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b700c8c00 con 0x7f8b701558f0 2020-11-20T12:22:41.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7592d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b700a84b0 0x7f8b70164700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7592d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b700a84b0 0x7f8b70164700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52600/0 (socket says 172.21.15.116:52600) 2020-11-20T12:22:41.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7592d700 1 -- 172.21.15.116:0/2161102086 learned_addr learned my addr 172.21.15.116:0/2161102086 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:41.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7512c700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b700a8910 0x7f8b70164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7612e700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b701558f0 0x7f8b700c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7592d700 1 -- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b701558f0 msgr2=0x7f8b700c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7592d700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b701558f0 0x7f8b700c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7592d700 1 -- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b700a8910 msgr2=0x7f8b70164bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.800 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7592d700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b700a8910 0x7f8b70164bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7592d700 1 -- 172.21.15.116:0/2161102086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b70162300 con 0x7f8b700a84b0 2020-11-20T12:22:41.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7592d700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b700a84b0 0x7f8b70164700 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f8b6c00ebd0 tx=0x7f8b6c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:41.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b5dffb700 1 -- 172.21.15.116:0/2161102086 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c00b810 con 0x7f8b700a84b0 2020-11-20T12:22:41.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b5dffb700 1 -- 172.21.15.116:0/2161102086 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6c00b970 con 0x7f8b700a84b0 2020-11-20T12:22:41.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b5dffb700 1 -- 172.21.15.116:0/2161102086 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6c00bcf0 con 0x7f8b700a84b0 2020-11-20T12:22:41.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7692f700 1 -- 172.21.15.116:0/2161102086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b701625a0 con 0x7f8b700a84b0 2020-11-20T12:22:41.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.786+0000 7f8b7692f700 1 -- 172.21.15.116:0/2161102086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b70165510 con 0x7f8b700a84b0 2020-11-20T12:22:41.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.790+0000 7f8b7692f700 1 -- 172.21.15.116:0/2161102086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b7005cd40 con 0x7f8b700a84b0 2020-11-20T12:22:41.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.790+0000 7f8b5dffb700 1 -- 172.21.15.116:0/2161102086 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8b6c012600 con 0x7f8b700a84b0 2020-11-20T12:22:41.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.790+0000 7f8b5dffb700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b480557c0 0x7f8b48057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:41.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.790+0000 7f8b5dffb700 1 -- 172.21.15.116:0/2161102086 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8b6c06e850 con 0x7f8b700a84b0 2020-11-20T12:22:41.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.794+0000 7f8b7512c700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b480557c0 0x7f8b48057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:41.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.794+0000 7f8b7512c700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b480557c0 0x7f8b48057c80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f8b700c9270 tx=0x7f8b60007b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:41.805 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.798+0000 7f8b5dffb700 1 -- 172.21.15.116:0/2161102086 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8b6c044580 con 0x7f8b700a84b0 2020-11-20T12:22:41.966 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.958+0000 7f8b7692f700 1 -- 172.21.15.116:0/2161102086 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8b70165510 con 0x7f8b700a84b0 2020-11-20T12:22:41.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b5dffb700 1 -- 172.21.15.116:0/2161102086 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+10 (secure 0 0 0) 0x7f8b70165510 con 0x7f8b700a84b0 2020-11-20T12:22:41.968 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:41.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 -- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b480557c0 msgr2=0x7f8b48057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b480557c0 0x7f8b48057c80 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f8b700c9270 tx=0x7f8b60007b80).stop 2020-11-20T12:22:41.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 -- 172.21.15.116:0/2161102086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b700a84b0 msgr2=0x7f8b70164700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:41.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b700a84b0 0x7f8b70164700 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f8b6c00ebd0 tx=0x7f8b6c006df0).stop 2020-11-20T12:22:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 -- 172.21.15.116:0/2161102086 shutdown_connections 2020-11-20T12:22:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8b480557c0 0x7f8b48057c80 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f8b701558f0 0x7f8b700c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f8b700a84b0 0x7f8b70164700 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 --2- 172.21.15.116:0/2161102086 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f8b700a8910 0x7f8b70164bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:41.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 -- 172.21.15.116:0/2161102086 >> 172.21.15.116:0/2161102086 conn(0x7f8b700678a0 msgr2=0x7f8b700a8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 -- 172.21.15.116:0/2161102086 shutdown_connections 2020-11-20T12:22:41.974 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:41.962+0000 7f8b7692f700 1 -- 172.21.15.116:0/2161102086 wait complete. 2020-11-20T12:22:41.994 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:22:41.995 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:42.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.158+0000 7fad725a8700 1 Processor -- start 2020-11-20T12:22:42.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.158+0000 7fad725a8700 1 -- start start 2020-11-20T12:22:42.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.162+0000 7fad725a8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad6c0c3b60 0x7fad6c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:42.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.162+0000 7fad725a8700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6c0a6950 con 0x7fad6c0c3780 2020-11-20T12:22:42.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.162+0000 7fad725a8700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad6c0a6bf0 con 0x7fad6c0c1970 2020-11-20T12:22:42.169 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.162+0000 7fad725a8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0a6770 con 0x7fad6c0c3b60 2020-11-20T12:22:42.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.162+0000 7fad6bfff700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fad6c0c3780 0x7fad6c0c22f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33810/0 (socket says 172.21.15.116:33810) 2020-11-20T12:22:42.170 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.162+0000 7fad6bfff700 1 -- 172.21.15.116:0/3044809969 learned_addr learned my addr 172.21.15.116:0/3044809969 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:42.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.162+0000 7fad63fff700 1 --2- 172.21.15.116:0/3044809969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad6c0c3b60 0x7fad6c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:42.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.162+0000 7fad6b7fe700 1 -- 172.21.15.116:0/3044809969 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309906702 0 0) 0x7fad6c0a6950 con 0x7fad6c0c3780 2020-11-20T12:22:42.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.162+0000 7fad6b7fe700 1 -- 172.21.15.116:0/3044809969 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad4c003240 con 0x7fad6c0c3780 2020-11-20T12:22:42.171 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.162+0000 7fad6b7fe700 1 -- 172.21.15.116:0/3044809969 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3715422689 0 0) 0x7fad6c0a6bf0 con 0x7fad6c0c1970 2020-11-20T12:22:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.162+0000 7fad6b7fe700 1 -- 172.21.15.116:0/3044809969 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad6c0a6950 con 0x7fad6c0c1970 2020-11-20T12:22:42.172 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad63fff700 1 -- 172.21.15.116:0/3044809969 >> v1:172.21.15.173:6789/0 conn(0x7fad6c0c1970 legacy=0x7fad6c150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad63fff700 1 -- 172.21.15.116:0/3044809969 >> v1:172.21.15.116:6789/0 conn(0x7fad6c0c3780 legacy=0x7fad6c0c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad63fff700 1 -- 172.21.15.116:0/3044809969 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c0a6e90 con 0x7fad6c0c3b60 2020-11-20T12:22:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad63fff700 1 --2- 172.21.15.116:0/3044809969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad6c0c3b60 0x7fad6c0a6280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fad5400b9d0 tx=0x7fad54006df0).ready entity=mon.2 client_cookie=c0146c98f97cc318 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad6b7fe700 1 -- 172.21.15.116:0/3044809969 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54009680 con 0x7fad6c0c3b60 2020-11-20T12:22:42.173 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad6b7fe700 1 -- 172.21.15.116:0/3044809969 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad540097e0 con 0x7fad6c0c3b60 2020-11-20T12:22:42.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad6b7fe700 1 -- 172.21.15.116:0/3044809969 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad54009b60 con 0x7fad6c0c3b60 2020-11-20T12:22:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad725a8700 1 -- 172.21.15.116:0/3044809969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad6c0c3b60 msgr2=0x7fad6c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad725a8700 1 --2- 172.21.15.116:0/3044809969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad6c0c3b60 0x7fad6c0a6280 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7fad5400b9d0 tx=0x7fad54006df0).stop 2020-11-20T12:22:42.175 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad725a8700 1 -- 172.21.15.116:0/3044809969 shutdown_connections 2020-11-20T12:22:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad725a8700 1 --2- 172.21.15.116:0/3044809969 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad6c0c3b60 0x7fad6c0a6280 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad725a8700 1 -- 172.21.15.116:0/3044809969 >> 172.21.15.116:0/3044809969 conn(0x7fad6c0c4f80 msgr2=0x7fad6c0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad725a8700 1 -- 172.21.15.116:0/3044809969 shutdown_connections 2020-11-20T12:22:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.166+0000 7fad725a8700 1 -- 172.21.15.116:0/3044809969 wait complete. 2020-11-20T12:22:42.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad725a8700 1 Processor -- start 2020-11-20T12:22:42.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad725a8700 1 -- start start 2020-11-20T12:22:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad725a8700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad6c0c1970 0x7fad6c152ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad725a8700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad6c0c3780 0x7fad6c13ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:42.178 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad725a8700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad6c0c3b60 0x7fad6c0c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad725a8700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0a77f0 con 0x7fad6c0c3b60 2020-11-20T12:22:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad725a8700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0a7a90 con 0x7fad6c0c3780 2020-11-20T12:22:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad725a8700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad6c0a7d30 con 0x7fad6c0c1970 2020-11-20T12:22:42.179 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad6bfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad6c0c1970 0x7fad6c152ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad63fff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad6c0c3780 0x7fad6c13ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad70b45700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad6c0c3b60 0x7fad6c0c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad63fff700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad6c0c3780 0x7fad6c13ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:51666/0 (socket says 172.21.15.116:51666) 2020-11-20T12:22:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad63fff700 1 -- 172.21.15.116:0/2871691565 learned_addr learned my addr 172.21.15.116:0/2871691565 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:42.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad70b45700 1 -- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad6c0c1970 msgr2=0x7fad6c152ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad70b45700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad6c0c1970 0x7fad6c152ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad70b45700 1 -- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad6c0c3780 msgr2=0x7fad6c13ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad70b45700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad6c0c3780 0x7fad6c13ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:42.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.170+0000 7fad70b45700 1 -- 172.21.15.116:0/2871691565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad6c0a7fd0 con 0x7fad6c0c3b60 2020-11-20T12:22:42.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.174+0000 7fad70b45700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad6c0c3b60 0x7fad6c0c0e50 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fad6400ca60 tx=0x7fad64007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:42.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.174+0000 7fad68ff9700 1 -- 172.21.15.116:0/2871691565 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6400a520 con 0x7fad6c0c3b60 2020-11-20T12:22:42.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.174+0000 7fad68ff9700 1 -- 172.21.15.116:0/2871691565 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad6400a680 con 0x7fad6c0c3b60 2020-11-20T12:22:42.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.174+0000 7fad725a8700 1 -- 172.21.15.116:0/2871691565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6c0a8270 con 0x7fad6c0c3b60 2020-11-20T12:22:42.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.174+0000 7fad68ff9700 1 -- 172.21.15.116:0/2871691565 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad6400aa00 con 0x7fad6c0c3b60 2020-11-20T12:22:42.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.174+0000 7fad725a8700 1 -- 172.21.15.116:0/2871691565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad6c164f30 con 0x7fad6c0c3b60 2020-11-20T12:22:42.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.174+0000 7fad725a8700 1 -- 172.21.15.116:0/2871691565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6c05d2a0 con 0x7fad6c0c3b60 2020-11-20T12:22:42.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.174+0000 7fad68ff9700 1 -- 172.21.15.116:0/2871691565 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fad64010070 con 0x7fad6c0c3b60 2020-11-20T12:22:42.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.174+0000 7fad68ff9700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad480557c0 0x7fad48057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:42.184 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.174+0000 7fad68ff9700 1 -- 172.21.15.116:0/2871691565 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fad6406e1f0 con 0x7fad6c0c3b60 2020-11-20T12:22:42.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.178+0000 7fad6bfff700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad480557c0 0x7fad48057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:42.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.178+0000 7fad6bfff700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad480557c0 0x7fad48057c80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fad5c0055a0 tx=0x7fad5c009040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:42.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.182+0000 7fad68ff9700 1 -- 172.21.15.116:0/2871691565 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fad64044090 con 0x7fad6c0c3b60 2020-11-20T12:22:42.397 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.386+0000 7fad725a8700 1 -- 172.21.15.116:0/2871691565 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad6c0a8400 con 0x7fad6c0c3b60 2020-11-20T12:22:42.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.386+0000 7fad68ff9700 1 -- 172.21.15.116:0/2871691565 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2853 (secure 0 0 0) 0x7fad640476b0 con 0x7fad6c0c3b60 2020-11-20T12:22:42.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 -- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad480557c0 msgr2=0x7fad48057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:42.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad480557c0 0x7fad48057c80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fad5c0055a0 tx=0x7fad5c009040).stop 2020-11-20T12:22:42.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 -- 172.21.15.116:0/2871691565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad6c0c3b60 msgr2=0x7fad6c0c0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:42.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad6c0c3b60 0x7fad6c0c0e50 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fad6400ca60 tx=0x7fad64007d40).stop 2020-11-20T12:22:42.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 -- 172.21.15.116:0/2871691565 shutdown_connections 2020-11-20T12:22:42.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fad480557c0 0x7fad48057c80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:42.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fad6c0c3b60 0x7fad6c0c0e50 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:42.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fad6c0c1970 0x7fad6c152ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:42.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 --2- 172.21.15.116:0/2871691565 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fad6c0c3780 0x7fad6c13ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:42.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 -- 172.21.15.116:0/2871691565 >> 172.21.15.116:0/2871691565 conn(0x7fad6c0c4f80 msgr2=0x7fad6c153af0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:42.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 -- 172.21.15.116:0/2871691565 shutdown_connections 2020-11-20T12:22:42.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:42.390+0000 7fad725a8700 1 -- 172.21.15.116:0/2871691565 wait complete. 2020-11-20T12:22:42.401 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:42.401 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":75,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/363707569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":363707569},{"type":"v1","addr":"172.21.15.173:6835","nonce":363707569}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":71},{"gid":7202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/1854325940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1854325940},{"type":"v1","addr":"172.21.15.116:6834","nonce":1854325940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72},{"gid":8437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3920088127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3920088127},{"type":"v1","addr":"172.21.15.173:6837","nonce":3920088127}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:22:35.338520+0000","modified":"2020-11-20T12:22:37.346491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8589":{"gid":8589,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1101967997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1101967997},{"type":"v1","addr":"172.21.15.116:6835","nonce":1101967997}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-11-20T12:22:42.401 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 75 2020-11-20T12:22:42.824 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:22:35.338520+0000', 'modified': '2020-11-20T12:22:37.346491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8589': {'gid': 8589, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/1101967997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 1101967997}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 1101967997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:22:42.862 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:22:42.863 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:22:43.064 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":8589,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":7.7014625360000002,"mdsmap_epoch":75,"osdmap_epoch":129,"osdmap_epoch_barrier":129,"uptime":13.24796804} 2020-11-20T12:22:43.065 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 8589, "mdsmap_epoch": 75, "osdmap_epoch": 129, "osdmap_epoch_barrier": 129, "rank_uptime": 7.701462536, "state": "up:active", "uptime": 13.24796804, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:22:43.066 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:22:43.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:43.067 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.0 2020-11-20T12:22:43.084 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:22:43.085 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:43.199 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:43.199 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:22:43.224 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:43.225 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2020-11-20T12:22:43.230 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:22:43.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:43.230 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2020-11-20T12:22:43.259 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:22:43.260 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:22:43.260 INFO:tasks.cephfs.mount:self.client.name = client.1 2020-11-20T12:22:43.260 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2020-11-20T12:22:43.261 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:22:43.261 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:22:43.261 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:22:43.262 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:22:43.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:43.263 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:22:43.281 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:22:43.282 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:22:43.282 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:22:43.282 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:43.282 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:22:43.282 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:43.283 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:43.283 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:43.283 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:43.283 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:43.284 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:43.284 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:43.284 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:22:43.284 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:43.284 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:22:43.285 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:43.285 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:22:43.285 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:43.285 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:22:43.286 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:43.286 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:22:43.286 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:43.286 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:22:43.286 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:43.287 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:22:43.287 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:43.287 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:22:43.287 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:43.288 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:43.288 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:22:43.288 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:22:43.288 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:43.288 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:43.289 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:22:43.289 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:22:43.289 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:43.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:43.290 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:22:43.324 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:22:43.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:43.327 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:22:43.387 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:22:43.387 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:22:43.388 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:22:43.388 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:22:43.388 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:22:43.389 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:22:43.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:43.390 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:22:43.391 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:22:43.392 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:22:43.393 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:22:43.393 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:22:43.394 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:22:43.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:43.502 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:22:43.537 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:22:43.537 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:22:43.538 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2020-11-20T12:22:43.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:43.539 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.0 up 2020-11-20T12:22:43.573 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:43.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc44c986700 1 Processor -- start 2020-11-20T12:22:43.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc44c986700 1 -- start start 2020-11-20T12:22:43.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc44c986700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc44814baa0 0x7fc44814be80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:43.773 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc44c986700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fc448065a20 0x7fc448151d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:43.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc44c986700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc448065e80 0x7fc448152230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:43.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc44c986700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc448152810 con 0x7fc448065e80 2020-11-20T12:22:43.774 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc44c986700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc448152ab0 con 0x7fc448065a20 2020-11-20T12:22:43.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc44c986700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480652e0 con 0x7fc44814baa0 2020-11-20T12:22:43.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc447fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc448065e80 0x7fc448152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:43.775 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc447fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fc448065e80 0x7fc448152230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52630/0 (socket says 172.21.15.116:52630) 2020-11-20T12:22:43.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc447fff700 1 -- 172.21.15.116:0/3312926900 learned_addr learned my addr 172.21.15.116:0/3312926900 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:43.776 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.754+0000 7fc43effd700 1 --2- 172.21.15.116:0/3312926900 >> v2:172.21.15.173:3300/0 conn(0x7fc448065a20 0x7fc448151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc4477fe700 1 --2- 172.21.15.116:0/3312926900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc44814baa0 0x7fc44814be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc447fff700 1 -- 172.21.15.116:0/3312926900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc44814baa0 msgr2=0x7fc44814be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:43.777 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc447fff700 1 --2- 172.21.15.116:0/3312926900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc44814baa0 0x7fc44814be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc447fff700 1 -- 172.21.15.116:0/3312926900 >> v2:172.21.15.173:3300/0 conn(0x7fc448065a20 msgr2=0x7fc448151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:43.778 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc447fff700 1 --2- 172.21.15.116:0/3312926900 >> v2:172.21.15.173:3300/0 conn(0x7fc448065a20 0x7fc448151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc447fff700 1 -- 172.21.15.116:0/3312926900 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc448065580 con 0x7fc448065e80 2020-11-20T12:22:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc447fff700 1 --2- 172.21.15.116:0/3312926900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc448065e80 0x7fc448152230 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fc448066470 tx=0x7fc440009290).ready entity=mon.0 client_cookie=6c0707c2d302be6c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:43.779 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc446ffd700 1 -- 172.21.15.116:0/3312926900 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44000bbc0 con 0x7fc448065e80 2020-11-20T12:22:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc446ffd700 1 -- 172.21.15.116:0/3312926900 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44000bd20 con 0x7fc448065e80 2020-11-20T12:22:43.780 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc446ffd700 1 -- 172.21.15.116:0/3312926900 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc440011610 con 0x7fc448065e80 2020-11-20T12:22:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc44c986700 1 -- 172.21.15.116:0/3312926900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc448065e80 msgr2=0x7fc448152230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc44c986700 1 --2- 172.21.15.116:0/3312926900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc448065e80 0x7fc448152230 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fc448066470 tx=0x7fc440009290).stop 2020-11-20T12:22:43.781 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc44c986700 1 -- 172.21.15.116:0/3312926900 shutdown_connections 2020-11-20T12:22:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc44c986700 1 --2- 172.21.15.116:0/3312926900 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc448065e80 0x7fc448152230 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:43.782 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc44c986700 1 --2- 172.21.15.116:0/3312926900 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc44814baa0 0x7fc44814be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc44c986700 1 --2- 172.21.15.116:0/3312926900 >> v2:172.21.15.173:3300/0 conn(0x7fc448065a20 0x7fc448151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc44c986700 1 -- 172.21.15.116:0/3312926900 >> 172.21.15.116:0/3312926900 conn(0x7fc448061280 msgr2=0x7fc448061660 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc44c986700 1 -- 172.21.15.116:0/3312926900 shutdown_connections 2020-11-20T12:22:43.783 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.758+0000 7fc44c986700 1 -- 172.21.15.116:0/3312926900 wait complete. 2020-11-20T12:22:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc44c986700 1 Processor -- start 2020-11-20T12:22:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc44c986700 1 -- start start 2020-11-20T12:22:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc44c986700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc448065a20 0x7fc448149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:43.784 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc44c986700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc448065e80 0x7fc448164640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc44c986700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc44814baa0 0x7fc448164b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:43.785 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc44c986700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480a93a0 con 0x7fc448065a20 2020-11-20T12:22:43.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc44c986700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480a9640 con 0x7fc44814baa0 2020-11-20T12:22:43.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc44c986700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4480a98e0 con 0x7fc448065e80 2020-11-20T12:22:43.786 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc4477fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc448065a20 0x7fc448149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:43.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc4477fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc448065a20 0x7fc448149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52634/0 (socket says 172.21.15.116:52634) 2020-11-20T12:22:43.787 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc4477fe700 1 -- 172.21.15.116:0/1142657029 learned_addr learned my addr 172.21.15.116:0/1142657029 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:43.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc43effd700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc448065e80 0x7fc448164640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:43.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc447fff700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc44814baa0 0x7fc448164b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:43.788 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc4477fe700 1 -- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc448065e80 msgr2=0x7fc448164640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:43.789 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc4477fe700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc448065e80 0x7fc448164640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:43.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc4477fe700 1 -- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc44814baa0 msgr2=0x7fc448164b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:43.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc4477fe700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc44814baa0 0x7fc448164b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:43.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc4477fe700 1 -- 172.21.15.116:0/1142657029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4480a9b80 con 0x7fc448065a20 2020-11-20T12:22:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.762+0000 7fc447fff700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc44814baa0 0x7fc448164b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.766+0000 7fc4477fe700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc448065a20 0x7fc448149f60 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7fc43800d8e0 tx=0x7fc438008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.766+0000 7fc43ffff700 1 -- 172.21.15.116:0/1142657029 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438009820 con 0x7fc448065a20 2020-11-20T12:22:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.766+0000 7fc43ffff700 1 -- 172.21.15.116:0/1142657029 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc438009980 con 0x7fc448065a20 2020-11-20T12:22:43.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.766+0000 7fc43ffff700 1 -- 172.21.15.116:0/1142657029 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438009d20 con 0x7fc448065a20 2020-11-20T12:22:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.766+0000 7fc44c986700 1 -- 172.21.15.116:0/1142657029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4480a9e20 con 0x7fc448065a20 2020-11-20T12:22:43.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.766+0000 7fc44c986700 1 -- 172.21.15.116:0/1142657029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4480aa2c0 con 0x7fc448065a20 2020-11-20T12:22:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.766+0000 7fc44c986700 1 -- 172.21.15.116:0/1142657029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44805cf30 con 0x7fc448065a20 2020-11-20T12:22:43.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.766+0000 7fc43ffff700 1 -- 172.21.15.116:0/1142657029 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fc438013690 con 0x7fc448065a20 2020-11-20T12:22:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.770+0000 7fc43ffff700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4240557c0 0x7fc424057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.770+0000 7fc43ffff700 1 -- 172.21.15.116:0/1142657029 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fc43806ea10 con 0x7fc448065a20 2020-11-20T12:22:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.770+0000 7fc43effd700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4240557c0 0x7fc424057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.770+0000 7fc43effd700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4240557c0 0x7fc424057c80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fc4481645a0 tx=0x7fc430008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:43.794 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.774+0000 7fc43ffff700 1 -- 172.21.15.116:0/1142657029 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fc4380448b0 con 0x7fc448065a20 2020-11-20T12:22:43.935 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc44c986700 1 -- 172.21.15.116:0/1142657029 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4480a5460 con 0x7fc448065a20 2020-11-20T12:22:43.937 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc43ffff700 1 -- 172.21.15.116:0/1142657029 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16888 (secure 0 0 0) 0x7fc438047ed0 con 0x7fc448065a20 2020-11-20T12:22:43.938 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:43.938 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":129,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:22:36.342598+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:22:33.838237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2020-11-20T12:22:34.312791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:43.940 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc44c986700 1 -- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4240557c0 msgr2=0x7fc424057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:43.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc44c986700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4240557c0 0x7fc424057c80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fc4481645a0 tx=0x7fc430008040).stop 2020-11-20T12:22:43.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc44c986700 1 -- 172.21.15.116:0/1142657029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc448065a20 msgr2=0x7fc448149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:43.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc44c986700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc448065a20 0x7fc448149f60 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7fc43800d8e0 tx=0x7fc438008040).stop 2020-11-20T12:22:43.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc44c986700 1 -- 172.21.15.116:0/1142657029 shutdown_connections 2020-11-20T12:22:43.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc44c986700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fc4240557c0 0x7fc424057c80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:43.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc44c986700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fc44814baa0 0x7fc448164b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:43.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc44c986700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fc448065a20 0x7fc448149f60 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:43.943 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc44c986700 1 --2- 172.21.15.116:0/1142657029 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fc448065e80 0x7fc448164640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:43.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.930+0000 7fc44c986700 1 -- 172.21.15.116:0/1142657029 >> 172.21.15.116:0/1142657029 conn(0x7fc448061280 msgr2=0x7fc4480c5b90 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:43.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.934+0000 7fc44c986700 1 -- 172.21.15.116:0/1142657029 shutdown_connections 2020-11-20T12:22:43.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:43.934+0000 7fc44c986700 1 -- 172.21.15.116:0/1142657029 wait complete. 2020-11-20T12:22:43.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:22:33.838237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:22:34.312791+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:22:43.963 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:22:44.147 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.142+0000 7fa67b59e700 1 Processor -- start 2020-11-20T12:22:44.148 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.142+0000 7fa67b59e700 1 -- start start 2020-11-20T12:22:44.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67b59e700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa67c0aa1a0 0x7fa67c0c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.151 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0c2de0 0x7fa67c0a8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.152 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67b59e700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa67c151cf0 con 0x7fa67c0aa580 2020-11-20T12:22:44.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67b59e700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa67c0c7e00 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c0c80a0 con 0x7fa67c0c2de0 2020-11-20T12:22:44.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa67c0aa1a0 0x7fa67c0c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.153 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67a59c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fa67c0aa1a0 0x7fa67c0c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52640/0 (socket says 172.21.15.116:52640) 2020-11-20T12:22:44.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67a59c700 1 -- 172.21.15.116:0/975101691 learned_addr learned my addr 172.21.15.116:0/975101691 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:44.154 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67ad9d700 1 --2- 172.21.15.116:0/975101691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0c2de0 0x7fa67c0a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67a59c700 1 -- 172.21.15.116:0/975101691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0c2de0 msgr2=0x7fa67c0a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:44.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67a59c700 1 --2- 172.21.15.116:0/975101691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0c2de0 0x7fa67c0a8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67a59c700 1 -- 172.21.15.116:0/975101691 >> v1:172.21.15.173:6789/0 conn(0x7fa67c0aa580 legacy=0x7fa67c061460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.155 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67a59c700 1 -- 172.21.15.116:0/975101691 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c0c34f0 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67a59c700 1 --2- 172.21.15.116:0/975101691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67c0aa1a0 0x7fa67c0c37d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7fa66c006fd0 tx=0x7fa66c006eb0).ready entity=mon.0 client_cookie=ced8f7aed1798a40 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:44.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67959a700 1 -- 172.21.15.116:0/975101691 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c009810 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67959a700 1 -- 172.21.15.116:0/975101691 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c009970 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.156 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.146+0000 7fa67959a700 1 -- 172.21.15.116:0/975101691 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c009cf0 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.150+0000 7fa67b59e700 1 -- 172.21.15.116:0/975101691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67c0aa1a0 msgr2=0x7fa67c0c37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.150+0000 7fa67b59e700 1 --2- 172.21.15.116:0/975101691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67c0aa1a0 0x7fa67c0c37d0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7fa66c006fd0 tx=0x7fa66c006eb0).stop 2020-11-20T12:22:44.157 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.150+0000 7fa67b59e700 1 -- 172.21.15.116:0/975101691 shutdown_connections 2020-11-20T12:22:44.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.150+0000 7fa67b59e700 1 --2- 172.21.15.116:0/975101691 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0c2de0 0x7fa67c0a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.150+0000 7fa67b59e700 1 --2- 172.21.15.116:0/975101691 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67c0aa1a0 0x7fa67c0c37d0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.150+0000 7fa67b59e700 1 -- 172.21.15.116:0/975101691 >> 172.21.15.116:0/975101691 conn(0x7fa67c068370 msgr2=0x7fa67c0c87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:44.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.150+0000 7fa67b59e700 1 -- 172.21.15.116:0/975101691 shutdown_connections 2020-11-20T12:22:44.158 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.150+0000 7fa67b59e700 1 -- 172.21.15.116:0/975101691 wait complete. 2020-11-20T12:22:44.159 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.150+0000 7fa67b59e700 1 Processor -- start 2020-11-20T12:22:44.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.150+0000 7fa67b59e700 1 -- start start 2020-11-20T12:22:44.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67b59e700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0aa1a0 0x7fa67c161e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67b59e700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67c0aa580 0x7fa67c0c6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.160 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67b59e700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa67c0c2de0 0x7fa67c0c2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67b59e700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c1655a0 con 0x7fa67c0aa580 2020-11-20T12:22:44.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67b59e700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c161440 con 0x7fa67c0c2de0 2020-11-20T12:22:44.161 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67b59e700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa67c1616e0 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.162 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa679d9b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67c0aa580 0x7fa67c0c6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67a59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0aa1a0 0x7fa67c161e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67a59c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0aa1a0 0x7fa67c161e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:33364/0 (socket says 172.21.15.116:33364) 2020-11-20T12:22:44.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67a59c700 1 -- 172.21.15.116:0/1862608777 learned_addr learned my addr 172.21.15.116:0/1862608777 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:44.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67ad9d700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa67c0c2de0 0x7fa67c0c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.163 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67a59c700 1 -- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa67c0c2de0 msgr2=0x7fa67c0c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67a59c700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa67c0c2de0 0x7fa67c0c2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67a59c700 1 -- 172.21.15.116:0/1862608777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67c0aa580 msgr2=0x7fa67c0c6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67a59c700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67c0aa580 0x7fa67c0c6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67a59c700 1 -- 172.21.15.116:0/1862608777 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa67c161980 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.164 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67a59c700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0aa1a0 0x7fa67c161e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa66c009e10 tx=0x7fa66c015000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:44.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa66a7fc700 1 -- 172.21.15.116:0/1862608777 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c011c80 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa66a7fc700 1 -- 172.21.15.116:0/1862608777 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66c011de0 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67b59e700 1 -- 172.21.15.116:0/1862608777 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa67c161140 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.165 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa66a7fc700 1 -- 172.21.15.116:0/1862608777 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66c01e690 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.154+0000 7fa67b59e700 1 -- 172.21.15.116:0/1862608777 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa67c0c6910 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.158+0000 7fa67b59e700 1 -- 172.21.15.116:0/1862608777 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa67c05cf30 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.166 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.158+0000 7fa66a7fc700 1 -- 172.21.15.116:0/1862608777 <== mon.2 v2:172.21.15.173:3301/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa66c025030 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.158+0000 7fa66a7fc700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa650055810 0x7fa650057cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.167 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.158+0000 7fa66a7fc700 1 -- 172.21.15.116:0/1862608777 <== mon.2 v2:172.21.15.173:3301/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa66c06fd90 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.162+0000 7fa679d9b700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa650055810 0x7fa650057cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.168 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.162+0000 7fa679d9b700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa650055810 0x7fa650057cd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fa67c0c6b20 tx=0x7fa670007b90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:44.174 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.166+0000 7fa66a7fc700 1 -- 172.21.15.116:0/1862608777 <== mon.2 v2:172.21.15.173:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa66c045ba0 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.326+0000 7fa67b59e700 1 -- 172.21.15.116:0/1862608777 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fa67c060f40 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.337 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.330+0000 7fa66a7fc700 1 -- 172.21.15.116:0/1862608777 <== mon.2 v2:172.21.15.173:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v129) v1 ==== 110+0+10 (secure 0 0 0) 0x7fa66c023030 con 0x7fa67c0aa1a0 2020-11-20T12:22:44.337 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:22:44.343 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 -- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa650055810 msgr2=0x7fa650057cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.344 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa650055810 0x7fa650057cd0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7fa67c0c6b20 tx=0x7fa670007b90).stop 2020-11-20T12:22:44.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 -- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0aa1a0 msgr2=0x7fa67c161e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.345 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0aa1a0 0x7fa67c161e20 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fa66c009e10 tx=0x7fa66c015000).stop 2020-11-20T12:22:44.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 -- 172.21.15.116:0/1862608777 shutdown_connections 2020-11-20T12:22:44.346 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa650055810 0x7fa650057cd0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa67c0c2de0 0x7fa67c0c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa67c0aa1a0 0x7fa67c161e20 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 --2- 172.21.15.116:0/1862608777 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa67c0aa580 0x7fa67c0c6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 -- 172.21.15.116:0/1862608777 >> 172.21.15.116:0/1862608777 conn(0x7fa67c068370 msgr2=0x7fa67c0ad7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:44.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 -- 172.21.15.116:0/1862608777 shutdown_connections 2020-11-20T12:22:44.349 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.334+0000 7fa67b59e700 1 -- 172.21.15.116:0/1862608777 wait complete. 2020-11-20T12:22:44.364 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:22:44.550 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.542+0000 7fa029b53700 1 Processor -- start 2020-11-20T12:22:44.551 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.542+0000 7fa029b53700 1 -- start start 2020-11-20T12:22:44.566 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa029b53700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0240b9db0 0x7fa0240c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.567 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa029b53700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0240a8930 con 0x7fa02414f880 2020-11-20T12:22:44.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa029b53700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0240a8690 con 0x7fa0240a81f0 2020-11-20T12:22:44.568 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa029b53700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240b94d0 con 0x7fa0240b9db0 2020-11-20T12:22:44.569 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa028b51700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fa02414f880 0x7fa024060170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33856/0 (socket says 172.21.15.116:33856) 2020-11-20T12:22:44.570 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa028b51700 1 -- 172.21.15.116:0/3775074985 learned_addr learned my addr 172.21.15.116:0/3775074985 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:44.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa029352700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0240b9db0 0x7fa0240c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.571 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3933717712 0 0) 0x7fa0240a8930 con 0x7fa02414f880 2020-11-20T12:22:44.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa008003240 con 0x7fa02414f880 2020-11-20T12:22:44.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1961156222 0 0) 0x7fa008003240 con 0x7fa02414f880 2020-11-20T12:22:44.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0240b9db0 msgr2=0x7fa0240c15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:44.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa0237fe700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0240b9db0 0x7fa0240c15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.572 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 >> v1:172.21.15.173:6789/0 conn(0x7fa0240a81f0 legacy=0x7fa0240c0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 --> v1:172.21.15.116:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240b9770 con 0x7fa02414f880 2020-11-20T12:22:44.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.546+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 <== mon.0 v1:172.21.15.116:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2382240062 0 0) 0x7fa0180014e0 con 0x7fa02414f880 2020-11-20T12:22:44.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 >> v1:172.21.15.116:6789/0 conn(0x7fa02414f880 legacy=0x7fa024060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.573 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa0237fe700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa008003760 0x7fa008003b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa0237fe700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0080044e0 0x7fa008004960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa0237fe700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0240b9db0 0x7fa008004e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240b94d0 con 0x7fa008003760 2020-11-20T12:22:44.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa008005340 con 0x7fa0080044e0 2020-11-20T12:22:44.574 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa00800b730 con 0x7fa0240b9db0 2020-11-20T12:22:44.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa028b51700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa008003760 0x7fa008003b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa029352700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0240b9db0 0x7fa008004e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa023fff700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0080044e0 0x7fa008004960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.575 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa028b51700 1 -- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0240b9db0 msgr2=0x7fa008004e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:44.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa028b51700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0240b9db0 0x7fa008004e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa028b51700 1 -- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0080044e0 msgr2=0x7fa008004960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:44.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa028b51700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa0080044e0 0x7fa008004960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa028b51700 1 -- 172.21.15.116:0/3775074985 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa008003240 con 0x7fa008003760 2020-11-20T12:22:44.576 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa029352700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa0240b9db0 0x7fa008004e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:44.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa028b51700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa008003760 0x7fa008003b80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7fa01800c410 tx=0x7fa0180077d0).ready entity=mon.0 client_cookie=6bcdf5298dfed152 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:44.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01800ce80 con 0x7fa008003760 2020-11-20T12:22:44.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0180072e0 con 0x7fa008003760 2020-11-20T12:22:44.577 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa0237fe700 1 -- 172.21.15.116:0/3775074985 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018005820 con 0x7fa008003760 2020-11-20T12:22:44.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa029b53700 1 -- 172.21.15.116:0/3775074985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa008003760 msgr2=0x7fa008003b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa029b53700 1 --2- 172.21.15.116:0/3775074985 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa008003760 0x7fa008003b80 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7fa01800c410 tx=0x7fa0180077d0).stop 2020-11-20T12:22:44.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa029352700 1 -- 172.21.15.116:0/3775074985 reap_dead start 2020-11-20T12:22:44.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa029b53700 1 -- 172.21.15.116:0/3775074985 shutdown_connections 2020-11-20T12:22:44.578 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa029b53700 1 -- 172.21.15.116:0/3775074985 >> 172.21.15.116:0/3775074985 conn(0x7fa024068350 msgr2=0x7fa0240627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:44.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa029b53700 1 -- 172.21.15.116:0/3775074985 shutdown_connections 2020-11-20T12:22:44.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.550+0000 7fa029b53700 1 -- 172.21.15.116:0/3775074985 wait complete. 2020-11-20T12:22:44.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa029b53700 1 Processor -- start 2020-11-20T12:22:44.579 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa029b53700 1 -- start start 2020-11-20T12:22:44.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa029b53700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa02414d1e0 0x7fa0240bfe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa029b53700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02414d5c0 0x7fa0240c0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa029b53700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa024164630 0x7fa024164ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa029b53700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240c0360 con 0x7fa02414d5c0 2020-11-20T12:22:44.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa029b53700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa024153bf0 con 0x7fa02414d1e0 2020-11-20T12:22:44.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa029b53700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa024153e90 con 0x7fa024164630 2020-11-20T12:22:44.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa023fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02414d5c0 0x7fa0240c0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa023fff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02414d5c0 0x7fa0240c0db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52668/0 (socket says 172.21.15.116:52668) 2020-11-20T12:22:44.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa023fff700 1 -- 172.21.15.116:0/2805671606 learned_addr learned my addr 172.21.15.116:0/2805671606 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:44.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa028b51700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa02414d1e0 0x7fa0240bfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa029352700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa024164630 0x7fa024164ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.554+0000 7fa023fff700 1 -- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa024164630 msgr2=0x7fa024164ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa023fff700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa024164630 0x7fa024164ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.582 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa023fff700 1 -- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa02414d1e0 msgr2=0x7fa0240bfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa023fff700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa02414d1e0 0x7fa0240bfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa023fff700 1 -- 172.21.15.116:0/2805671606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240bf730 con 0x7fa02414d5c0 2020-11-20T12:22:44.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa023fff700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02414d5c0 0x7fa0240c0db0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7fa01000e8e0 tx=0x7fa010009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:44.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa020ff9700 1 -- 172.21.15.116:0/2805671606 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01000eee0 con 0x7fa02414d5c0 2020-11-20T12:22:44.583 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa020ff9700 1 -- 172.21.15.116:0/2805671606 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01000c7b0 con 0x7fa02414d5c0 2020-11-20T12:22:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa029b53700 1 -- 172.21.15.116:0/2805671606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0240bf9d0 con 0x7fa02414d5c0 2020-11-20T12:22:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa020ff9700 1 -- 172.21.15.116:0/2805671606 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01000cb30 con 0x7fa02414d5c0 2020-11-20T12:22:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa029b53700 1 -- 172.21.15.116:0/2805671606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa024153870 con 0x7fa02414d5c0 2020-11-20T12:22:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa029b53700 1 -- 172.21.15.116:0/2805671606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02405cf30 con 0x7fa02414d5c0 2020-11-20T12:22:44.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.558+0000 7fa020ff9700 1 -- 172.21.15.116:0/2805671606 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa01001b030 con 0x7fa02414d5c0 2020-11-20T12:22:44.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.562+0000 7fa020ff9700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ffc0557c0 0x7f9ffc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.562+0000 7fa020ff9700 1 -- 172.21.15.116:0/2805671606 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa01006e350 con 0x7fa02414d5c0 2020-11-20T12:22:44.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.562+0000 7fa028b51700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ffc0557c0 0x7f9ffc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.562+0000 7fa028b51700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ffc0557c0 0x7f9ffc057c80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fa02405f2f0 tx=0x7fa018015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:44.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.566+0000 7fa020ff9700 1 -- 172.21.15.116:0/2805671606 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa010044220 con 0x7fa02414d5c0 2020-11-20T12:22:44.731 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.726+0000 7fa029b53700 1 -- 172.21.15.116:0/2805671606 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa0240639c0 con 0x7fa02414d5c0 2020-11-20T12:22:44.732 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.726+0000 7fa020ff9700 1 -- 172.21.15.116:0/2805671606 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa010047840 con 0x7fa02414d5c0 2020-11-20T12:22:44.732 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:44.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.726+0000 7fa029b53700 1 -- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ffc0557c0 msgr2=0x7f9ffc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.726+0000 7fa029b53700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ffc0557c0 0x7f9ffc057c80 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7fa02405f2f0 tx=0x7fa018015000).stop 2020-11-20T12:22:44.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.726+0000 7fa029b53700 1 -- 172.21.15.116:0/2805671606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02414d5c0 msgr2=0x7fa0240c0db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.735 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.726+0000 7fa029b53700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02414d5c0 0x7fa0240c0db0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7fa01000e8e0 tx=0x7fa010009040).stop 2020-11-20T12:22:44.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.730+0000 7fa029b53700 1 -- 172.21.15.116:0/2805671606 shutdown_connections 2020-11-20T12:22:44.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.730+0000 7fa029b53700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f9ffc0557c0 0x7f9ffc057c80 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.736 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.730+0000 7fa029b53700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa024164630 0x7fa024164ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.730+0000 7fa029b53700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa02414d1e0 0x7fa0240bfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.730+0000 7fa029b53700 1 --2- 172.21.15.116:0/2805671606 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa02414d5c0 0x7fa0240c0db0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.730+0000 7fa029b53700 1 -- 172.21.15.116:0/2805671606 >> 172.21.15.116:0/2805671606 conn(0x7fa024068350 msgr2=0x7fa0240bd770 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:44.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.730+0000 7fa029b53700 1 -- 172.21.15.116:0/2805671606 shutdown_connections 2020-11-20T12:22:44.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.730+0000 7fa029b53700 1 -- 172.21.15.116:0/2805671606 wait complete. 2020-11-20T12:22:44.761 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:22:44.941 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.934+0000 7f90f19dd700 1 Processor -- start 2020-11-20T12:22:44.942 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.934+0000 7f90f19dd700 1 -- start start 2020-11-20T12:22:44.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f19dd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90ec0a8400 0x7f90ec0aabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.944 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f19dd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f90ec0ad540 0x7f90ec0ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f19dd700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90ec0ace40 con 0x7f90ec0aa7d0 2020-11-20T12:22:44.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f19dd700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90ec1526c0 con 0x7f90ec0ad540 2020-11-20T12:22:44.945 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f19dd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec152960 con 0x7f90ec0a8400 2020-11-20T12:22:44.946 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f09db700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f90ec0aa7d0 0x7f90ec151e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33876/0 (socket says 172.21.15.116:33876) 2020-11-20T12:22:44.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f09db700 1 -- 172.21.15.116:0/42318006 learned_addr learned my addr 172.21.15.116:0/42318006 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:44.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f11dc700 1 --2- 172.21.15.116:0/42318006 >> v2:172.21.15.116:3300/0 conn(0x7f90ec0ad540 0x7f90ec0ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90ebfff700 1 --2- 172.21.15.116:0/42318006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90ec0a8400 0x7f90ec0aabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90eb7fe700 1 -- 172.21.15.116:0/42318006 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838993932 0 0) 0x7f90ec0ace40 con 0x7f90ec0aa7d0 2020-11-20T12:22:44.947 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90eb7fe700 1 -- 172.21.15.116:0/42318006 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90d0003240 con 0x7f90ec0aa7d0 2020-11-20T12:22:44.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90eb7fe700 1 -- 172.21.15.116:0/42318006 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 265193141 0 0) 0x7f90d0003240 con 0x7f90ec0aa7d0 2020-11-20T12:22:44.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f11dc700 1 -- 172.21.15.116:0/42318006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90ec0a8400 msgr2=0x7f90ec0aabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:44.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f11dc700 1 --2- 172.21.15.116:0/42318006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90ec0a8400 0x7f90ec0aabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f11dc700 1 -- 172.21.15.116:0/42318006 >> v1:172.21.15.116:6789/0 conn(0x7f90ec0aa7d0 legacy=0x7f90ec151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.948 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.938+0000 7f90f11dc700 1 -- 172.21.15.116:0/42318006 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec152c00 con 0x7f90ec0ad540 2020-11-20T12:22:44.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90f11dc700 1 --2- 172.21.15.116:0/42318006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90ec0ad540 0x7f90ec0ad9f0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f90e4009c90 tx=0x7f90e40091d0).ready entity=mon.0 client_cookie=d73746dcff410021 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:44.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90eb7fe700 1 -- 172.21.15.116:0/42318006 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e00014e0 con 0x7f90ec0ad540 2020-11-20T12:22:44.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90eb7fe700 1 -- 172.21.15.116:0/42318006 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e400b6b0 con 0x7f90ec0ad540 2020-11-20T12:22:44.950 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90eb7fe700 1 -- 172.21.15.116:0/42318006 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e400ba30 con 0x7f90ec0ad540 2020-11-20T12:22:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90f19dd700 1 -- 172.21.15.116:0/42318006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90ec0ad540 msgr2=0x7f90ec0ad9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90f19dd700 1 --2- 172.21.15.116:0/42318006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90ec0ad540 0x7f90ec0ad9f0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 rx=0x7f90e4009c90 tx=0x7f90e40091d0).stop 2020-11-20T12:22:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90f19dd700 1 -- 172.21.15.116:0/42318006 shutdown_connections 2020-11-20T12:22:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90f19dd700 1 --2- 172.21.15.116:0/42318006 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90ec0ad540 0x7f90ec0ad9f0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.951 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90f19dd700 1 --2- 172.21.15.116:0/42318006 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90ec0a8400 0x7f90ec0aabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90f19dd700 1 -- 172.21.15.116:0/42318006 >> 172.21.15.116:0/42318006 conn(0x7f90ec0678a0 msgr2=0x7f90ec067c80 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:44.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90f19dd700 1 -- 172.21.15.116:0/42318006 shutdown_connections 2020-11-20T12:22:44.952 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90f19dd700 1 -- 172.21.15.116:0/42318006 wait complete. 2020-11-20T12:22:44.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.942+0000 7f90f19dd700 1 Processor -- start 2020-11-20T12:22:44.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f19dd700 1 -- start start 2020-11-20T12:22:44.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f19dd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90ec0a8400 0x7f90ec13da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.955 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f19dd700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90ec0aa7d0 0x7f90ec1643d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f19dd700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90ec0ad540 0x7f90ec0a9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f19dd700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec13df00 con 0x7f90ec0a8400 2020-11-20T12:22:44.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f19dd700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec164ce0 con 0x7f90ec0ad540 2020-11-20T12:22:44.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f19dd700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90ec164f80 con 0x7f90ec0aa7d0 2020-11-20T12:22:44.956 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f09db700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90ec0a8400 0x7f90ec13da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f09db700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90ec0a8400 0x7f90ec13da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52682/0 (socket says 172.21.15.116:52682) 2020-11-20T12:22:44.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f09db700 1 -- 172.21.15.116:0/3871217301 learned_addr learned my addr 172.21.15.116:0/3871217301 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:44.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f11dc700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90ec0ad540 0x7f90ec0a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.957 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90ebfff700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90ec0aa7d0 0x7f90ec1643d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f09db700 1 -- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90ec0aa7d0 msgr2=0x7f90ec1643d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f09db700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90ec0aa7d0 0x7f90ec1643d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f09db700 1 -- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90ec0ad540 msgr2=0x7f90ec0a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:44.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f09db700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90ec0ad540 0x7f90ec0a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:44.958 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f09db700 1 -- 172.21.15.116:0/3871217301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90ec165220 con 0x7f90ec0a8400 2020-11-20T12:22:44.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90ebfff700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90ec0aa7d0 0x7f90ec1643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:44.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f09db700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90ec0a8400 0x7f90ec13da10 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f90e000cae0 tx=0x7f90e0001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:44.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90e8ff9700 1 -- 172.21.15.116:0/3871217301 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e0007170 con 0x7f90ec0a8400 2020-11-20T12:22:44.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90e8ff9700 1 -- 172.21.15.116:0/3871217301 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e00072d0 con 0x7f90ec0a8400 2020-11-20T12:22:44.959 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90f19dd700 1 -- 172.21.15.116:0/3871217301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90ec09f430 con 0x7f90ec0a8400 2020-11-20T12:22:44.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.946+0000 7f90e8ff9700 1 -- 172.21.15.116:0/3871217301 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e000a9f0 con 0x7f90ec0a8400 2020-11-20T12:22:44.960 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.950+0000 7f90f19dd700 1 -- 172.21.15.116:0/3871217301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90ec1648c0 con 0x7f90ec0a8400 2020-11-20T12:22:44.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.950+0000 7f90f19dd700 1 -- 172.21.15.116:0/3871217301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90ec05cf30 con 0x7f90ec0a8400 2020-11-20T12:22:44.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.950+0000 7f90e8ff9700 1 -- 172.21.15.116:0/3871217301 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f90e001b060 con 0x7f90ec0a8400 2020-11-20T12:22:44.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.950+0000 7f90e8ff9700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90c40557c0 0x7f90c4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:44.961 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.950+0000 7f90e8ff9700 1 -- 172.21.15.116:0/3871217301 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f90e006e3b0 con 0x7f90ec0a8400 2020-11-20T12:22:44.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.954+0000 7f90ebfff700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90c40557c0 0x7f90c4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:44.962 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.954+0000 7f90ebfff700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90c40557c0 0x7f90c4057c80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f90ec14b750 tx=0x7f90d8008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:44.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:44.958+0000 7f90e8ff9700 1 -- 172.21.15.116:0/3871217301 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f90e0044140 con 0x7f90ec0a8400 2020-11-20T12:22:45.129 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.122+0000 7f90f19dd700 1 -- 172.21.15.116:0/3871217301 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f90ec0601b0 con 0x7f90ec0a8400 2020-11-20T12:22:45.130 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.122+0000 7f90e8ff9700 1 -- 172.21.15.116:0/3871217301 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+10 (secure 0 0 0) 0x7f90e0019020 con 0x7f90ec0a8400 2020-11-20T12:22:45.130 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:45.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 -- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90c40557c0 msgr2=0x7f90c4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.133 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90c40557c0 0x7f90c4057c80 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f90ec14b750 tx=0x7f90d8008040).stop 2020-11-20T12:22:45.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 -- 172.21.15.116:0/3871217301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90ec0a8400 msgr2=0x7f90ec13da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90ec0a8400 0x7f90ec13da10 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7f90e000cae0 tx=0x7f90e0001e50).stop 2020-11-20T12:22:45.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 -- 172.21.15.116:0/3871217301 shutdown_connections 2020-11-20T12:22:45.134 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f90c40557c0 0x7f90c4057c80 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f90ec0ad540 0x7f90ec0a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f90ec0a8400 0x7f90ec13da10 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 --2- 172.21.15.116:0/3871217301 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f90ec0aa7d0 0x7f90ec1643d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.135 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 -- 172.21.15.116:0/3871217301 >> 172.21.15.116:0/3871217301 conn(0x7f90ec0678a0 msgr2=0x7f90ec0613a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:45.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 -- 172.21.15.116:0/3871217301 shutdown_connections 2020-11-20T12:22:45.136 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.126+0000 7f90f19dd700 1 -- 172.21.15.116:0/3871217301 wait complete. 2020-11-20T12:22:45.159 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:22:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.342+0000 7f69357a4700 1 Processor -- start 2020-11-20T12:22:45.348 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.342+0000 7f69357a4700 1 -- start start 2020-11-20T12:22:45.350 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.342+0000 7f69357a4700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f693014f880 0x7f69300b9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.342+0000 7f69357a4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69300c0e90 0x7f69300c1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.342+0000 7f69357a4700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69300a8930 con 0x7f69300a81f0 2020-11-20T12:22:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.342+0000 7f69357a4700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69300ba2a0 con 0x7f693014f880 2020-11-20T12:22:45.351 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.342+0000 7f69357a4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69300a8690 con 0x7f69300c0e90 2020-11-20T12:22:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f692ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f693014f880 0x7f69300b9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:45.352 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f692ffff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f693014f880 0x7f69300b9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52688/0 (socket says 172.21.15.116:52688) 2020-11-20T12:22:45.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f692ffff700 1 -- 172.21.15.116:0/2322006667 learned_addr learned my addr 172.21.15.116:0/2322006667 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:45.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f6934fa3700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69300c0e90 0x7f69300c1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:45.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f692ffff700 1 -- 172.21.15.116:0/2322006667 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69300c0e90 msgr2=0x7f69300c1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:45.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f692ffff700 1 --2- 172.21.15.116:0/2322006667 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69300c0e90 0x7f69300c1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.353 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f692ffff700 1 -- 172.21.15.116:0/2322006667 >> v1:172.21.15.173:6789/0 conn(0x7f69300a81f0 legacy=0x7f6930060170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.354 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f692ffff700 1 -- 172.21.15.116:0/2322006667 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69300b94d0 con 0x7f693014f880 2020-11-20T12:22:45.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f692ffff700 1 --2- 172.21.15.116:0/2322006667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693014f880 0x7f69300b9db0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7f6920006fd0 tx=0x7f6920006eb0).ready entity=mon.0 client_cookie=1f7244c9903ba24d server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:45.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f692effd700 1 -- 172.21.15.116:0/2322006667 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920009810 con 0x7f693014f880 2020-11-20T12:22:45.355 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f692effd700 1 -- 172.21.15.116:0/2322006667 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6920009970 con 0x7f693014f880 2020-11-20T12:22:45.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f692effd700 1 -- 172.21.15.116:0/2322006667 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6920009cf0 con 0x7f693014f880 2020-11-20T12:22:45.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f69357a4700 1 -- 172.21.15.116:0/2322006667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693014f880 msgr2=0x7f69300b9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f69357a4700 1 --2- 172.21.15.116:0/2322006667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693014f880 0x7f69300b9db0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7f6920006fd0 tx=0x7f6920006eb0).stop 2020-11-20T12:22:45.356 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f69357a4700 1 -- 172.21.15.116:0/2322006667 shutdown_connections 2020-11-20T12:22:45.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f69357a4700 1 --2- 172.21.15.116:0/2322006667 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69300c0e90 0x7f69300c1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f69357a4700 1 --2- 172.21.15.116:0/2322006667 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693014f880 0x7f69300b9db0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.346+0000 7f69357a4700 1 -- 172.21.15.116:0/2322006667 >> 172.21.15.116:0/2322006667 conn(0x7f6930068350 msgr2=0x7f69300627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:45.357 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.350+0000 7f69357a4700 1 -- 172.21.15.116:0/2322006667 shutdown_connections 2020-11-20T12:22:45.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.350+0000 7f69357a4700 1 -- 172.21.15.116:0/2322006667 wait complete. 2020-11-20T12:22:45.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.350+0000 7f69357a4700 1 Processor -- start 2020-11-20T12:22:45.358 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.350+0000 7f69357a4700 1 -- start start 2020-11-20T12:22:45.360 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f69357a4700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69300a81f0 0x7f69300a56e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:45.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f69357a4700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69300c0e90 0x7f6930145ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:45.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f69357a4700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693014f880 0x7f69300c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:45.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f69357a4700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69300a5da0 con 0x7f693014f880 2020-11-20T12:22:45.361 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f69357a4700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69300a6040 con 0x7f69300c0e90 2020-11-20T12:22:45.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f69357a4700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69300a62e0 con 0x7f69300a81f0 2020-11-20T12:22:45.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f6934fa3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693014f880 0x7f69300c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:45.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f6934fa3700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693014f880 0x7f69300c4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52698/0 (socket says 172.21.15.116:52698) 2020-11-20T12:22:45.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f6934fa3700 1 -- 172.21.15.116:0/2676493880 learned_addr learned my addr 172.21.15.116:0/2676493880 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:45.362 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f692ffff700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69300a81f0 0x7f69300a56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:45.363 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f692f7fe700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69300c0e90 0x7f6930145ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f6934fa3700 1 -- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69300a81f0 msgr2=0x7f69300a56e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f6934fa3700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69300a81f0 0x7f69300a56e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.364 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f6934fa3700 1 -- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69300c0e90 msgr2=0x7f6930145ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f6934fa3700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69300c0e90 0x7f6930145ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f6934fa3700 1 -- 172.21.15.116:0/2676493880 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69300a6580 con 0x7f693014f880 2020-11-20T12:22:45.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f692f7fe700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69300c0e90 0x7f6930145ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:45.365 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f6934fa3700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693014f880 0x7f69300c4460 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f692800ed20 tx=0x7f69280091d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:45.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f690ffff700 1 -- 172.21.15.116:0/2676493880 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f692800eee0 con 0x7f693014f880 2020-11-20T12:22:45.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f690ffff700 1 -- 172.21.15.116:0/2676493880 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f692800cbb0 con 0x7f693014f880 2020-11-20T12:22:45.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f690ffff700 1 -- 172.21.15.116:0/2676493880 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69280123f0 con 0x7f693014f880 2020-11-20T12:22:45.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f69357a4700 1 -- 172.21.15.116:0/2676493880 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69300a6820 con 0x7f693014f880 2020-11-20T12:22:45.366 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.354+0000 7f69357a4700 1 -- 172.21.15.116:0/2676493880 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69301463b0 con 0x7f693014f880 2020-11-20T12:22:45.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.358+0000 7f69357a4700 1 -- 172.21.15.116:0/2676493880 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69300a69b0 con 0x7f693014f880 2020-11-20T12:22:45.367 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.358+0000 7f690ffff700 1 -- 172.21.15.116:0/2676493880 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f6928022070 con 0x7f693014f880 2020-11-20T12:22:45.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.358+0000 7f690ffff700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69080557c0 0x7f6908057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:45.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.358+0000 7f690ffff700 1 -- 172.21.15.116:0/2676493880 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f692806eeb0 con 0x7f693014f880 2020-11-20T12:22:45.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.358+0000 7f692ffff700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69080557c0 0x7f6908057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:45.368 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.362+0000 7f692ffff700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69080557c0 0x7f6908057c80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f692000b1b0 tx=0x7f6920015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:45.371 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.362+0000 7f690ffff700 1 -- 172.21.15.116:0/2676493880 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f6928044c40 con 0x7f693014f880 2020-11-20T12:22:45.537 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.530+0000 7f69357a4700 1 -- 172.21.15.116:0/2676493880 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f693005cf30 con 0x7f693014f880 2020-11-20T12:22:45.538 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.530+0000 7f690ffff700 1 -- 172.21.15.116:0/2676493880 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+10 (secure 0 0 0) 0x7f6928016030 con 0x7f693014f880 2020-11-20T12:22:45.538 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:45.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 -- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69080557c0 msgr2=0x7f6908057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69080557c0 0x7f6908057c80 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f692000b1b0 tx=0x7f6920015000).stop 2020-11-20T12:22:45.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 -- 172.21.15.116:0/2676493880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693014f880 msgr2=0x7f69300c4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.541 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693014f880 0x7f69300c4460 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7f692800ed20 tx=0x7f69280091d0).stop 2020-11-20T12:22:45.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 -- 172.21.15.116:0/2676493880 shutdown_connections 2020-11-20T12:22:45.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f69080557c0 0x7f6908057c80 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f693014f880 0x7f69300c4460 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.542 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f69300a81f0 0x7f69300a56e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 --2- 172.21.15.116:0/2676493880 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f69300c0e90 0x7f6930145ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 -- 172.21.15.116:0/2676493880 >> 172.21.15.116:0/2676493880 conn(0x7f6930068350 msgr2=0x7f69300bc270 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:45.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 -- 172.21.15.116:0/2676493880 shutdown_connections 2020-11-20T12:22:45.543 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.534+0000 7f69357a4700 1 -- 172.21.15.116:0/2676493880 wait complete. 2020-11-20T12:22:45.564 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:22:45.564 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:45.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.726+0000 7faa982ee700 1 Processor -- start 2020-11-20T12:22:45.734 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.726+0000 7faa982ee700 1 -- start start 2020-11-20T12:22:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.730+0000 7faa982ee700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa900a6280 0x7faa900c1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.730+0000 7faa982ee700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa900c1e60 con 0x7faa900c3780 2020-11-20T12:22:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.730+0000 7faa982ee700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa900a66f0 con 0x7faa900c3b60 2020-11-20T12:22:45.737 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.730+0000 7faa982ee700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa9014fed0 con 0x7faa900a6280 2020-11-20T12:22:45.738 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.730+0000 7faa9688b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa900a6280 0x7faa900c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.730+0000 7faa9688b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7faa900a6280 0x7faa900c1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52706/0 (socket says 172.21.15.116:52706) 2020-11-20T12:22:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.730+0000 7faa9688b700 1 -- 172.21.15.116:0/875485127 learned_addr learned my addr 172.21.15.116:0/875485127 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.730+0000 7faa9688b700 1 -- 172.21.15.116:0/875485127 >> v1:172.21.15.173:6789/0 conn(0x7faa900c3b60 legacy=0x7faa90150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.730+0000 7faa9688b700 1 -- 172.21.15.116:0/875485127 >> v1:172.21.15.116:6789/0 conn(0x7faa900c3780 legacy=0x7faa900c22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.739 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.730+0000 7faa9688b700 1 -- 172.21.15.116:0/875485127 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa900a6950 con 0x7faa900a6280 2020-11-20T12:22:45.740 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.730+0000 7faa9688b700 1 --2- 172.21.15.116:0/875485127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa900a6280 0x7faa900c1970 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7faa88009d50 tx=0x7faa88009290).ready entity=mon.0 client_cookie=ee0b4b1816b4c33f server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa95088700 1 -- 172.21.15.116:0/875485127 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8800bc80 con 0x7faa900a6280 2020-11-20T12:22:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa95088700 1 -- 172.21.15.116:0/875485127 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8800bde0 con 0x7faa900a6280 2020-11-20T12:22:45.742 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa95088700 1 -- 172.21.15.116:0/875485127 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88011610 con 0x7faa900a6280 2020-11-20T12:22:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa982ee700 1 -- 172.21.15.116:0/875485127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa900a6280 msgr2=0x7faa900c1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa982ee700 1 --2- 172.21.15.116:0/875485127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa900a6280 0x7faa900c1970 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 rx=0x7faa88009d50 tx=0x7faa88009290).stop 2020-11-20T12:22:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa982ee700 1 -- 172.21.15.116:0/875485127 shutdown_connections 2020-11-20T12:22:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa982ee700 1 --2- 172.21.15.116:0/875485127 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa900a6280 0x7faa900c1970 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.743 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa982ee700 1 -- 172.21.15.116:0/875485127 >> 172.21.15.116:0/875485127 conn(0x7faa900c4f80 msgr2=0x7faa900c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa982ee700 1 -- 172.21.15.116:0/875485127 shutdown_connections 2020-11-20T12:22:45.744 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa982ee700 1 -- 172.21.15.116:0/875485127 wait complete. 2020-11-20T12:22:45.745 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa982ee700 1 Processor -- start 2020-11-20T12:22:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.734+0000 7faa982ee700 1 -- start start 2020-11-20T12:22:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa982ee700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa900a6280 0x7faa90164d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa982ee700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa900c3780 0x7faa900bb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:45.746 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa982ee700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa900c3b60 0x7faa900a4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa982ee700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa900bb760 con 0x7faa900c3b60 2020-11-20T12:22:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa982ee700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa900a4560 con 0x7faa900a6280 2020-11-20T12:22:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa982ee700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa90165430 con 0x7faa900c3780 2020-11-20T12:22:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa9688b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa900c3b60 0x7faa900a4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:45.747 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa9688b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa900c3b60 0x7faa900a4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52710/0 (socket says 172.21.15.116:52710) 2020-11-20T12:22:45.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa9688b700 1 -- 172.21.15.116:0/2045658434 learned_addr learned my addr 172.21.15.116:0/2045658434 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:45.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa9608a700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa900a6280 0x7faa90164d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:45.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa95889700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa900c3780 0x7faa900bb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:45.748 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa9688b700 1 -- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa900c3780 msgr2=0x7faa900bb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa9688b700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa900c3780 0x7faa900bb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa9688b700 1 -- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa900a6280 msgr2=0x7faa90164d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa9688b700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa900a6280 0x7faa90164d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa9688b700 1 -- 172.21.15.116:0/2045658434 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa901656d0 con 0x7faa900c3b60 2020-11-20T12:22:45.749 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa9688b700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa900c3b60 0x7faa900a4070 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7faa880092d0 tx=0x7faa88016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:45.750 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa7e7fc700 1 -- 172.21.15.116:0/2045658434 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8800bc80 con 0x7faa900c3b60 2020-11-20T12:22:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa7e7fc700 1 -- 172.21.15.116:0/2045658434 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa88009e90 con 0x7faa900c3b60 2020-11-20T12:22:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa7e7fc700 1 -- 172.21.15.116:0/2045658434 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88010cf0 con 0x7faa900c3b60 2020-11-20T12:22:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa982ee700 1 -- 172.21.15.116:0/2045658434 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa90165970 con 0x7faa900c3b60 2020-11-20T12:22:45.751 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.738+0000 7faa982ee700 1 -- 172.21.15.116:0/2045658434 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa900bcab0 con 0x7faa900c3b60 2020-11-20T12:22:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.742+0000 7faa982ee700 1 -- 172.21.15.116:0/2045658434 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa900bcd30 con 0x7faa900c3b60 2020-11-20T12:22:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.742+0000 7faa7e7fc700 1 -- 172.21.15.116:0/2045658434 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7faa88017070 con 0x7faa900c3b60 2020-11-20T12:22:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.742+0000 7faa7e7fc700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa680557c0 0x7faa68057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.742+0000 7faa7e7fc700 1 -- 172.21.15.116:0/2045658434 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7faa880702b0 con 0x7faa900c3b60 2020-11-20T12:22:45.752 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.746+0000 7faa9608a700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa680557c0 0x7faa68057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:45.753 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.746+0000 7faa9608a700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa680557c0 0x7faa68057c80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7faa8c003670 tx=0x7faa8c007b90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:45.755 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.750+0000 7faa7e7fc700 1 -- 172.21.15.116:0/2045658434 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7faa88046040 con 0x7faa900c3b60 2020-11-20T12:22:45.964 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.958+0000 7faa982ee700 1 -- 172.21.15.116:0/2045658434 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa9005d2a0 con 0x7faa900c3b60 2020-11-20T12:22:45.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.958+0000 7faa7e7fc700 1 -- 172.21.15.116:0/2045658434 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2853 (secure 0 0 0) 0x7faa88015020 con 0x7faa900c3b60 2020-11-20T12:22:45.966 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:45.966 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":75,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/363707569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":363707569},{"type":"v1","addr":"172.21.15.173:6835","nonce":363707569}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":71},{"gid":7202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/1854325940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1854325940},{"type":"v1","addr":"172.21.15.116:6834","nonce":1854325940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72},{"gid":8437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3920088127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3920088127},{"type":"v1","addr":"172.21.15.173:6837","nonce":3920088127}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:22:35.338520+0000","modified":"2020-11-20T12:22:37.346491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8589":{"gid":8589,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1101967997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1101967997},{"type":"v1","addr":"172.21.15.116:6835","nonce":1101967997}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-11-20T12:22:45.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.962+0000 7faa982ee700 1 -- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa680557c0 msgr2=0x7faa68057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.962+0000 7faa982ee700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa680557c0 0x7faa68057c80 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7faa8c003670 tx=0x7faa8c007b90).stop 2020-11-20T12:22:45.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.962+0000 7faa982ee700 1 -- 172.21.15.116:0/2045658434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa900c3b60 msgr2=0x7faa900a4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:45.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.962+0000 7faa982ee700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa900c3b60 0x7faa900a4070 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 rx=0x7faa880092d0 tx=0x7faa88016000).stop 2020-11-20T12:22:45.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.962+0000 7faa982ee700 1 -- 172.21.15.116:0/2045658434 shutdown_connections 2020-11-20T12:22:45.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.962+0000 7faa982ee700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7faa680557c0 0x7faa68057c80 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.962+0000 7faa982ee700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7faa900c3b60 0x7faa900a4070 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.962+0000 7faa982ee700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7faa900a6280 0x7faa90164d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.972 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.962+0000 7faa982ee700 1 --2- 172.21.15.116:0/2045658434 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7faa900c3780 0x7faa900bb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:45.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.966+0000 7faa982ee700 1 -- 172.21.15.116:0/2045658434 >> 172.21.15.116:0/2045658434 conn(0x7faa900c4f80 msgr2=0x7faa9014e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:45.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.966+0000 7faa982ee700 1 -- 172.21.15.116:0/2045658434 shutdown_connections 2020-11-20T12:22:45.973 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:45.966+0000 7faa982ee700 1 -- 172.21.15.116:0/2045658434 wait complete. 2020-11-20T12:22:45.974 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 75 2020-11-20T12:22:46.397 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:22:35.338520+0000', 'modified': '2020-11-20T12:22:37.346491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8589': {'gid': 8589, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/1101967997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 1101967997}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 1101967997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:22:46.397 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:22:46.398 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:22:46.597 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":8589,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":11.234613258,"mdsmap_epoch":75,"osdmap_epoch":129,"osdmap_epoch_barrier":129,"uptime":16.781127412} 2020-11-20T12:22:46.598 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 8589, "mdsmap_epoch": 75, "osdmap_epoch": 129, "osdmap_epoch_barrier": 129, "rank_uptime": 11.234613258, "state": "up:active", "uptime": 16.781127412, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:22:46.598 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:22:46.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:46.599 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.1 2020-11-20T12:22:46.618 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:22:46.620 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:46.689 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:46.689 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:22:46.712 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:46.713 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2020-11-20T12:22:46.717 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:22:46.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:46.718 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2020-11-20T12:22:46.748 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:22:46.749 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi116.front.sepia.ceph.com 2020-11-20T12:22:46.749 INFO:tasks.cephfs.mount:self.client.name = client.2 2020-11-20T12:22:46.749 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2020-11-20T12:22:46.750 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:22:46.750 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:22:46.750 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:22:46.751 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:22:46.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:46.752 INFO:teuthology.orchestra.run.smithi116:> ip addr 2020-11-20T12:22:46.769 INFO:teuthology.orchestra.run.smithi116.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:22:46.769 INFO:teuthology.orchestra.run.smithi116.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:22:46.769 INFO:teuthology.orchestra.run.smithi116.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:22:46.769 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:46.770 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 ::1/128 scope host 2020-11-20T12:22:46.770 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:46.770 INFO:teuthology.orchestra.run.smithi116.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:46.770 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:90 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:46.771 INFO:teuthology.orchestra.run.smithi116.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:46.771 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:d8:cb:91 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:46.771 INFO:teuthology.orchestra.run.smithi116.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:46.771 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:66 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:46.771 INFO:teuthology.orchestra.run.smithi116.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:22:46.772 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 0c:c4:7a:8f:3f:67 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:46.772 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.21.15.116/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:22:46.772 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:46.772 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::ec4:7aff:fe8f:3f67/64 scope link 2020-11-20T12:22:46.772 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:46.773 INFO:teuthology.orchestra.run.smithi116.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:22:46.773 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 02:42:a1:ae:37:db brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:46.773 INFO:teuthology.orchestra.run.smithi116.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:22:46.773 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:46.773 INFO:teuthology.orchestra.run.smithi116.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:22:46.774 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:46.774 INFO:teuthology.orchestra.run.smithi116.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:22:46.774 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:46.774 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::dcb1:beff:fe16:40a6/64 scope link 2020-11-20T12:22:46.775 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:46.775 INFO:teuthology.orchestra.run.smithi116.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:46.775 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 4e:a8:74:28:ba:c2 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:22:46.775 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::4ca8:74ff:fe28:bac2/64 scope link 2020-11-20T12:22:46.776 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:46.776 INFO:teuthology.orchestra.run.smithi116.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:46.776 INFO:teuthology.orchestra.run.smithi116.stdout: link/ether 2e:16:30:1f:ca:b7 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:22:46.776 INFO:teuthology.orchestra.run.smithi116.stdout: inet6 fe80::2c16:30ff:fe1f:cab7/64 scope link 2020-11-20T12:22:46.776 INFO:teuthology.orchestra.run.smithi116.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:46.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:46.778 INFO:teuthology.orchestra.run.smithi116:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:22:46.812 INFO:teuthology.orchestra.run.smithi116.stdout:1 2020-11-20T12:22:46.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:46.816 INFO:teuthology.orchestra.run.smithi116:> route 2020-11-20T12:22:46.876 INFO:teuthology.orchestra.run.smithi116.stdout:Kernel IP routing table 2020-11-20T12:22:46.877 INFO:teuthology.orchestra.run.smithi116.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:22:46.877 INFO:teuthology.orchestra.run.smithi116.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:22:46.877 INFO:teuthology.orchestra.run.smithi116.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:22:46.877 INFO:teuthology.orchestra.run.smithi116.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:22:46.878 INFO:teuthology.orchestra.run.smithi116.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:22:46.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:46.879 INFO:teuthology.orchestra.run.smithi116:> (cd / && exec sudo bash -c ' 2020-11-20T12:22:46.879 INFO:teuthology.orchestra.run.smithi116:> set -e 2020-11-20T12:22:46.880 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:22:46.880 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:22:46.881 INFO:teuthology.orchestra.run.smithi116:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:22:46.881 INFO:teuthology.orchestra.run.smithi116:> ') 2020-11-20T12:22:46.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:46.976 INFO:teuthology.orchestra.run.smithi116:> ip netns list 2020-11-20T12:22:46.991 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2020-11-20T12:22:46.992 INFO:teuthology.orchestra.run.smithi116.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2020-11-20T12:22:46.993 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.2' netns for '/home/ubuntu/cephtest/mnt.2' 2020-11-20T12:22:46.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:46.994 INFO:teuthology.orchestra.run.smithi116:> sudo ip link set brx.1 up 2020-11-20T12:22:47.025 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:47.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.238+0000 7f72818e2700 1 Processor -- start 2020-11-20T12:22:47.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.238+0000 7f72818e2700 1 -- start start 2020-11-20T12:22:47.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f72818e2700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f727c0a5b60 0x7f727c0c3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:47.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f72818e2700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727c0c11d0 con 0x7f727c1554d0 2020-11-20T12:22:47.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f72818e2700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727c155a50 con 0x7f727c0c30a0 2020-11-20T12:22:47.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f72818e2700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727c0bffb0 con 0x7f727c0a5b60 2020-11-20T12:22:47.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f727b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f727c0a5b60 0x7f727c0c3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:47.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f727b7fe700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f727c0a5b60 0x7f727c0c3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52720/0 (socket says 172.21.15.116:52720) 2020-11-20T12:22:47.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f727a7fc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f727c1554d0 0x7f727c0a5450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33918/0 (socket says 172.21.15.116:33918) 2020-11-20T12:22:47.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f727b7fe700 1 -- 172.21.15.116:0/3892636729 learned_addr learned my addr 172.21.15.116:0/3892636729 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:47.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f727b7fe700 1 -- 172.21.15.116:0/3892636729 >> v1:172.21.15.173:6789/0 conn(0x7f727c0c30a0 legacy=0x7f727c0bf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f727b7fe700 1 -- 172.21.15.116:0/3892636729 >> v1:172.21.15.116:6789/0 conn(0x7f727c1554d0 legacy=0x7f727c0a5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f727b7fe700 1 -- 172.21.15.116:0/3892636729 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c155cf0 con 0x7f727c0a5b60 2020-11-20T12:22:47.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f727b7fe700 1 --2- 172.21.15.116:0/3892636729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f727c0a5b60 0x7f727c0c3480 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7f7274009d50 tx=0x7f7274009290).ready entity=mon.0 client_cookie=19e17a955ce84bea server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:47.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f7279ffb700 1 -- 172.21.15.116:0/3892636729 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727400bc80 con 0x7f727c0a5b60 2020-11-20T12:22:47.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f7279ffb700 1 -- 172.21.15.116:0/3892636729 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727400bde0 con 0x7f727c0a5b60 2020-11-20T12:22:47.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f7279ffb700 1 -- 172.21.15.116:0/3892636729 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274011610 con 0x7f727c0a5b60 2020-11-20T12:22:47.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f72818e2700 1 -- 172.21.15.116:0/3892636729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f727c0a5b60 msgr2=0x7f727c0c3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.242+0000 7f72818e2700 1 --2- 172.21.15.116:0/3892636729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f727c0a5b60 0x7f727c0c3480 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 rx=0x7f7274009d50 tx=0x7f7274009290).stop 2020-11-20T12:22:47.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.246+0000 7f72818e2700 1 -- 172.21.15.116:0/3892636729 shutdown_connections 2020-11-20T12:22:47.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.246+0000 7f72818e2700 1 --2- 172.21.15.116:0/3892636729 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f727c0a5b60 0x7f727c0c3480 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.246+0000 7f72818e2700 1 -- 172.21.15.116:0/3892636729 >> 172.21.15.116:0/3892636729 conn(0x7f727c060170 msgr2=0x7f727c060550 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:47.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.246+0000 7f72818e2700 1 -- 172.21.15.116:0/3892636729 shutdown_connections 2020-11-20T12:22:47.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.246+0000 7f72818e2700 1 -- 172.21.15.116:0/3892636729 wait complete. 2020-11-20T12:22:47.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.250+0000 7f72818e2700 1 Processor -- start 2020-11-20T12:22:47.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.250+0000 7f72818e2700 1 -- start start 2020-11-20T12:22:47.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.250+0000 7f72818e2700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f727c0a5b60 0x7f727c0a7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:47.266 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.250+0000 7f72818e2700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f727c0c30a0 0x7f727c0a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:47.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.250+0000 7f72818e2700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f727c1554d0 0x7f727c161430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:47.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.250+0000 7f72818e2700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c0aaf90 con 0x7f727c0a5b60 2020-11-20T12:22:47.267 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.250+0000 7f72818e2700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c162400 con 0x7f727c1554d0 2020-11-20T12:22:47.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.250+0000 7f72818e2700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727c161c20 con 0x7f727c0c30a0 2020-11-20T12:22:47.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.250+0000 7f727affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f727c0a5b60 0x7f727c0a7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:47.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.250+0000 7f727affd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f727c0a5b60 0x7f727c0a7cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52722/0 (socket says 172.21.15.116:52722) 2020-11-20T12:22:47.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.250+0000 7f727affd700 1 -- 172.21.15.116:0/3217001832 learned_addr learned my addr 172.21.15.116:0/3217001832 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:47.269 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f727a7fc700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f727c0c30a0 0x7f727c0a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:47.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f727b7fe700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f727c1554d0 0x7f727c161430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:47.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f727affd700 1 -- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f727c0c30a0 msgr2=0x7f727c0a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.270 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f727affd700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f727c0c30a0 0x7f727c0a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f727affd700 1 -- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f727c1554d0 msgr2=0x7f727c161430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f727affd700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f727c1554d0 0x7f727c161430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.271 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f727affd700 1 -- 172.21.15.116:0/3217001832 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c161ec0 con 0x7f727c0a5b60 2020-11-20T12:22:47.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f727affd700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f727c0a5b60 0x7f727c0a7cd0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7f726c00c6e0 tx=0x7f726c007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:47.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f7262ffd700 1 -- 172.21.15.116:0/3217001832 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c007020 con 0x7f727c0a5b60 2020-11-20T12:22:47.272 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f7262ffd700 1 -- 172.21.15.116:0/3217001832 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c007180 con 0x7f727c0a5b60 2020-11-20T12:22:47.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f7262ffd700 1 -- 172.21.15.116:0/3217001832 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c0064f0 con 0x7f727c0a5b60 2020-11-20T12:22:47.273 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f72818e2700 1 -- 172.21.15.116:0/3217001832 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727c162160 con 0x7f727c0a5b60 2020-11-20T12:22:47.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f72818e2700 1 -- 172.21.15.116:0/3217001832 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727c0a6430 con 0x7f727c0a5b60 2020-11-20T12:22:47.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.254+0000 7f72818e2700 1 -- 172.21.15.116:0/3217001832 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727c0ab380 con 0x7f727c0a5b60 2020-11-20T12:22:47.274 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.258+0000 7f7262ffd700 1 -- 172.21.15.116:0/3217001832 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f726c010070 con 0x7f727c0a5b60 2020-11-20T12:22:47.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.258+0000 7f7262ffd700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72580557c0 0x7f7258057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:47.275 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.258+0000 7f7262ffd700 1 -- 172.21.15.116:0/3217001832 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f726c06da60 con 0x7f727c0a5b60 2020-11-20T12:22:47.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.258+0000 7f727a7fc700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72580557c0 0x7f7258057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:47.276 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.262+0000 7f727a7fc700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72580557c0 0x7f7258057c80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f727c1625b0 tx=0x7f7270007b80).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:47.277 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.266+0000 7f7262ffd700 1 -- 172.21.15.116:0/3217001832 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f726c042fd0 con 0x7f727c0a5b60 2020-11-20T12:22:47.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.430+0000 7f72818e2700 1 -- 172.21.15.116:0/3217001832 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f727c05cf30 con 0x7f727c0a5b60 2020-11-20T12:22:47.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.430+0000 7f7262ffd700 1 -- 172.21.15.116:0/3217001832 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f726c024210 con 0x7f727c0a5b60 2020-11-20T12:22:47.439 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:47.439 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":129,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:22:36.342598+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:22:33.838237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2020-11-20T12:22:34.312791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:47.442 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 -- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72580557c0 msgr2=0x7f7258057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.443 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72580557c0 0x7f7258057c80 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7f727c1625b0 tx=0x7f7270007b80).stop 2020-11-20T12:22:47.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 -- 172.21.15.116:0/3217001832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f727c0a5b60 msgr2=0x7f727c0a7cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.444 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f727c0a5b60 0x7f727c0a7cd0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 rx=0x7f726c00c6e0 tx=0x7f726c007c50).stop 2020-11-20T12:22:47.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 -- 172.21.15.116:0/3217001832 shutdown_connections 2020-11-20T12:22:47.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f72580557c0 0x7f7258057c80 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.445 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f727c1554d0 0x7f727c161430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f727c0a5b60 0x7f727c0a7cd0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 --2- 172.21.15.116:0/3217001832 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f727c0c30a0 0x7f727c0a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.446 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 -- 172.21.15.116:0/3217001832 >> 172.21.15.116:0/3217001832 conn(0x7f727c060170 msgr2=0x7f727c13ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 -- 172.21.15.116:0/3217001832 shutdown_connections 2020-11-20T12:22:47.447 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.434+0000 7f72818e2700 1 -- 172.21.15.116:0/3217001832 wait complete. 2020-11-20T12:22:47.464 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:22:33.838237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:22:34.312791+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:22:47.466 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:22:47.660 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.654+0000 7f35a4390700 1 Processor -- start 2020-11-20T12:22:47.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.654+0000 7f35a4390700 1 -- start start 2020-11-20T12:22:47.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.654+0000 7f35a4390700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f359c14bee0 0x7f359c152990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:47.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a4390700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f359c0c8fa0 0x7f359c0adba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:47.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a4390700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c0abc10 con 0x7f359c0ae090 2020-11-20T12:22:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a4390700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359c0acf60 con 0x7f359c14bee0 2020-11-20T12:22:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a4390700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c152e80 con 0x7f359c0c8fa0 2020-11-20T12:22:47.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a192b700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f359c0c8fa0 0x7f359c0adba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:47.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a212c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f359c14bee0 0x7f359c152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:47.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a212c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f359c14bee0 0x7f359c152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:51782/0 (socket says 172.21.15.116:51782) 2020-11-20T12:22:47.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a212c700 1 -- 172.21.15.116:0/3576601193 learned_addr learned my addr 172.21.15.116:0/3576601193 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:47.665 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a292d700 1 --1- 172.21.15.116:0/3576601193 >> v1:172.21.15.173:6789/0 conn(0x7f359c0ae090 0x7f359c0616b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.173:6789/0 says I am v1:172.21.15.116:58398/0 (socket says 172.21.15.116:58398) 2020-11-20T12:22:47.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a212c700 1 -- 172.21.15.116:0/3576601193 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f359c0c8fa0 msgr2=0x7f359c0adba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:47.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a212c700 1 --2- 172.21.15.116:0/3576601193 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f359c0c8fa0 0x7f359c0adba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a212c700 1 -- 172.21.15.116:0/3576601193 >> v1:172.21.15.173:6789/0 conn(0x7f359c0ae090 legacy=0x7f359c0616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a112a700 1 -- 172.21.15.116:0/3576601193 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3066234180 0 0) 0x7f359c0abc10 con 0x7f359c0ae090 2020-11-20T12:22:47.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a212c700 1 -- 172.21.15.116:0/3576601193 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0c9520 con 0x7f359c14bee0 2020-11-20T12:22:47.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a212c700 1 --2- 172.21.15.116:0/3576601193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f359c14bee0 0x7f359c152990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f3598006fd0 tx=0x7f3598006eb0).ready entity=mon.1 client_cookie=4ec592905f1da6ab server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:47.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a112a700 1 -- 172.21.15.116:0/3576601193 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598009810 con 0x7f359c14bee0 2020-11-20T12:22:47.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a112a700 1 -- 172.21.15.116:0/3576601193 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3598009970 con 0x7f359c14bee0 2020-11-20T12:22:47.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.658+0000 7f35a112a700 1 -- 172.21.15.116:0/3576601193 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3598009cf0 con 0x7f359c14bee0 2020-11-20T12:22:47.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.662+0000 7f35a4390700 1 -- 172.21.15.116:0/3576601193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f359c14bee0 msgr2=0x7f359c152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.662+0000 7f35a4390700 1 --2- 172.21.15.116:0/3576601193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f359c14bee0 0x7f359c152990 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7f3598006fd0 tx=0x7f3598006eb0).stop 2020-11-20T12:22:47.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.662+0000 7f35a4390700 1 -- 172.21.15.116:0/3576601193 shutdown_connections 2020-11-20T12:22:47.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.662+0000 7f35a4390700 1 --2- 172.21.15.116:0/3576601193 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f359c14bee0 0x7f359c152990 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.662+0000 7f35a4390700 1 --2- 172.21.15.116:0/3576601193 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f359c0c8fa0 0x7f359c0adba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.662+0000 7f35a4390700 1 -- 172.21.15.116:0/3576601193 >> 172.21.15.116:0/3576601193 conn(0x7f359c068370 msgr2=0x7f359c0c8890 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:47.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.662+0000 7f35a4390700 1 -- 172.21.15.116:0/3576601193 shutdown_connections 2020-11-20T12:22:47.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.662+0000 7f35a4390700 1 -- 172.21.15.116:0/3576601193 wait complete. 2020-11-20T12:22:47.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a4390700 1 Processor -- start 2020-11-20T12:22:47.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a4390700 1 -- start start 2020-11-20T12:22:47.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a4390700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f359c0ae090 0x7f359c0a9830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:47.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a4390700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c0c8fa0 0x7f359c0aadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:47.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a4390700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f359c14bee0 0x7f359c1474f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:47.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a4390700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0aac40 con 0x7f359c0c8fa0 2020-11-20T12:22:47.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a4390700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0aa9a0 con 0x7f359c14bee0 2020-11-20T12:22:47.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a4390700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c0a8e50 con 0x7f359c0ae090 2020-11-20T12:22:47.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a192b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c0c8fa0 0x7f359c0aadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:47.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a192b700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c0c8fa0 0x7f359c0aadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52738/0 (socket says 172.21.15.116:52738) 2020-11-20T12:22:47.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a192b700 1 -- 172.21.15.116:0/571817529 learned_addr learned my addr 172.21.15.116:0/571817529 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:47.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a292d700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f359c14bee0 0x7f359c1474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:47.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a212c700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f359c0ae090 0x7f359c0a9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:47.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a192b700 1 -- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f359c0ae090 msgr2=0x7f359c0a9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a192b700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f359c0ae090 0x7f359c0a9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a192b700 1 -- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f359c14bee0 msgr2=0x7f359c1474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a192b700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f359c14bee0 0x7f359c1474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a192b700 1 -- 172.21.15.116:0/571817529 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0a90f0 con 0x7f359c0c8fa0 2020-11-20T12:22:47.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.666+0000 7f35a192b700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c0c8fa0 0x7f359c0aadf0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7f358c00eba0 tx=0x7f358c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:47.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.670+0000 7f358a7fc700 1 -- 172.21.15.116:0/571817529 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c00b810 con 0x7f359c0c8fa0 2020-11-20T12:22:47.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.670+0000 7f358a7fc700 1 -- 172.21.15.116:0/571817529 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c00b970 con 0x7f359c0c8fa0 2020-11-20T12:22:47.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.670+0000 7f35a4390700 1 -- 172.21.15.116:0/571817529 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c0a9390 con 0x7f359c0c8fa0 2020-11-20T12:22:47.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.670+0000 7f358a7fc700 1 -- 172.21.15.116:0/571817529 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c00bd10 con 0x7f359c0c8fa0 2020-11-20T12:22:47.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.670+0000 7f35a4390700 1 -- 172.21.15.116:0/571817529 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c1479e0 con 0x7f359c0c8fa0 2020-11-20T12:22:47.679 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.670+0000 7f35a4390700 1 -- 172.21.15.116:0/571817529 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359c0a9520 con 0x7f359c0c8fa0 2020-11-20T12:22:47.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.670+0000 7f358a7fc700 1 -- 172.21.15.116:0/571817529 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f358c012600 con 0x7f359c0c8fa0 2020-11-20T12:22:47.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.674+0000 7f358a7fc700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35740557c0 0x7f3574057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:47.680 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.674+0000 7f358a7fc700 1 -- 172.21.15.116:0/571817529 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f358c06e900 con 0x7f359c0c8fa0 2020-11-20T12:22:47.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.674+0000 7f35a212c700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35740557c0 0x7f3574057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:47.681 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.674+0000 7f35a212c700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35740557c0 0x7f3574057c80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f3598002fc0 tx=0x7f3598015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:47.686 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.678+0000 7f358a7fc700 1 -- 172.21.15.116:0/571817529 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f358c0447a0 con 0x7f359c0c8fa0 2020-11-20T12:22:47.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.838+0000 7f35a4390700 1 -- 172.21.15.116:0/571817529 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f359c05d2a0 con 0x7f359c0c8fa0 2020-11-20T12:22:47.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.838+0000 7f358a7fc700 1 -- 172.21.15.116:0/571817529 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v129) v1 ==== 110+0+10 (secure 0 0 0) 0x7f358c047dc0 con 0x7f359c0c8fa0 2020-11-20T12:22:47.846 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:22:47.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 -- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35740557c0 msgr2=0x7f3574057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35740557c0 0x7f3574057c80 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f3598002fc0 tx=0x7f3598015000).stop 2020-11-20T12:22:47.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 -- 172.21.15.116:0/571817529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c0c8fa0 msgr2=0x7f359c0aadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:47.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c0c8fa0 0x7f359c0aadf0 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 rx=0x7f358c00eba0 tx=0x7f358c006df0).stop 2020-11-20T12:22:47.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 -- 172.21.15.116:0/571817529 shutdown_connections 2020-11-20T12:22:47.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f35740557c0 0x7f3574057c80 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f359c14bee0 0x7f359c1474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f359c0ae090 0x7f359c0a9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 --2- 172.21.15.116:0/571817529 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f359c0c8fa0 0x7f359c0aadf0 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:47.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 -- 172.21.15.116:0/571817529 >> 172.21.15.116:0/571817529 conn(0x7f359c068370 msgr2=0x7f359c153050 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:47.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 -- 172.21.15.116:0/571817529 shutdown_connections 2020-11-20T12:22:47.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:47.842+0000 7f35a4390700 1 -- 172.21.15.116:0/571817529 wait complete. 2020-11-20T12:22:47.875 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:22:48.055 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.050+0000 7fadd0b79700 1 Processor -- start 2020-11-20T12:22:48.056 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.050+0000 7fadd0b79700 1 -- start start 2020-11-20T12:22:48.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.050+0000 7fadd0b79700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fadcc0606f0 0x7fadcc0bf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:48.058 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.050+0000 7fadd0b79700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc066b90 con 0x7fadcc060310 2020-11-20T12:22:48.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.050+0000 7fadd0b79700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadcc066e00 con 0x7fadcc0bfaa0 2020-11-20T12:22:48.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.050+0000 7fadd0b79700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadcc0670a0 con 0x7fadcc0606f0 2020-11-20T12:22:48.059 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadcb7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fadcc060310 0x7fadcc066480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33946/0 (socket says 172.21.15.116:33946) 2020-11-20T12:22:48.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadcb7fe700 1 -- 172.21.15.116:0/3326057422 learned_addr learned my addr 172.21.15.116:0/3326057422 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:48.060 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadcaffd700 1 --2- 172.21.15.116:0/3326057422 >> v2:172.21.15.173:3300/0 conn(0x7fadcc0606f0 0x7fadcc0bf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:48.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadca7fc700 1 -- 172.21.15.116:0/3326057422 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953322602 0 0) 0x7fadcc066b90 con 0x7fadcc060310 2020-11-20T12:22:48.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadca7fc700 1 -- 172.21.15.116:0/3326057422 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadb8003240 con 0x7fadcc060310 2020-11-20T12:22:48.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadca7fc700 1 -- 172.21.15.116:0/3326057422 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280039868 0 0) 0x7fadcc066e00 con 0x7fadcc0bfaa0 2020-11-20T12:22:48.061 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadca7fc700 1 -- 172.21.15.116:0/3326057422 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadcc066b90 con 0x7fadcc0bfaa0 2020-11-20T12:22:48.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadcaffd700 1 -- 172.21.15.116:0/3326057422 >> v1:172.21.15.173:6789/0 conn(0x7fadcc0bfaa0 legacy=0x7fadcc0c1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:48.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadcaffd700 1 -- 172.21.15.116:0/3326057422 >> v1:172.21.15.116:6789/0 conn(0x7fadcc060310 legacy=0x7fadcc066480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:48.062 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadca7fc700 1 -- 172.21.15.116:0/3326057422 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 3539525463 0 0) 0x7fadb8003240 con 0x7fadcc060310 2020-11-20T12:22:48.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadcaffd700 1 -- 172.21.15.116:0/3326057422 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc067340 con 0x7fadcc0606f0 2020-11-20T12:22:48.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.054+0000 7fadcaffd700 1 --2- 172.21.15.116:0/3326057422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadcc0606f0 0x7fadcc0bf5b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fadc000b9a0 tx=0x7fadc0006df0).ready entity=mon.1 client_cookie=8c2cb775e6d0c570 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:48.063 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadca7fc700 1 -- 172.21.15.116:0/3326057422 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc0014e0 con 0x7fadcc0606f0 2020-11-20T12:22:48.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadca7fc700 1 -- 172.21.15.116:0/3326057422 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc0011400 con 0x7fadcc0606f0 2020-11-20T12:22:48.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadca7fc700 1 -- 172.21.15.116:0/3326057422 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc0011780 con 0x7fadcc0606f0 2020-11-20T12:22:48.064 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadd0b79700 1 -- 172.21.15.116:0/3326057422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadcc0606f0 msgr2=0x7fadcc0bf5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:48.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadd0b79700 1 --2- 172.21.15.116:0/3326057422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadcc0606f0 0x7fadcc0bf5b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fadc000b9a0 tx=0x7fadc0006df0).stop 2020-11-20T12:22:48.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadd0b79700 1 -- 172.21.15.116:0/3326057422 shutdown_connections 2020-11-20T12:22:48.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadd0b79700 1 --2- 172.21.15.116:0/3326057422 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadcc0606f0 0x7fadcc0bf5b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.065 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadd0b79700 1 -- 172.21.15.116:0/3326057422 >> 172.21.15.116:0/3326057422 conn(0x7fadcc068350 msgr2=0x7fadcc0627c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:48.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadd0b79700 1 -- 172.21.15.116:0/3326057422 shutdown_connections 2020-11-20T12:22:48.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadd0b79700 1 -- 172.21.15.116:0/3326057422 wait complete. 2020-11-20T12:22:48.066 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadd0b79700 1 Processor -- start 2020-11-20T12:22:48.067 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.058+0000 7fadd0b79700 1 -- start start 2020-11-20T12:22:48.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadd0b79700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadcc060310 0x7fadcc14fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:48.071 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadd0b79700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadcc0bfaa0 0x7fadcc161370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:48.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadd0b79700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadcc165390 0x7fadcc151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:48.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadd0b79700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc14e850 con 0x7fadcc060310 2020-11-20T12:22:48.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadd0b79700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc1619a0 con 0x7fadcc0bfaa0 2020-11-20T12:22:48.072 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadd0b79700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadcc161c40 con 0x7fadcc165390 2020-11-20T12:22:48.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadcb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadcc060310 0x7fadcc14fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:48.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadcb7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadcc060310 0x7fadcc14fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52750/0 (socket says 172.21.15.116:52750) 2020-11-20T12:22:48.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadcb7fe700 1 -- 172.21.15.116:0/3865416163 learned_addr learned my addr 172.21.15.116:0/3865416163 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:48.073 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadcaffd700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadcc0bfaa0 0x7fadcc161370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:48.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadcbfff700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadcc165390 0x7fadcc151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:48.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadcb7fe700 1 -- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadcc165390 msgr2=0x7fadcc151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:48.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadcb7fe700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadcc165390 0x7fadcc151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.074 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadcb7fe700 1 -- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadcc0bfaa0 msgr2=0x7fadcc161370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:48.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadcb7fe700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadcc0bfaa0 0x7fadcc161370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadcb7fe700 1 -- 172.21.15.116:0/3865416163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadcc161ee0 con 0x7fadcc060310 2020-11-20T12:22:48.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadcb7fe700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadcc060310 0x7fadcc14fc30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7fadbc00cae0 tx=0x7fadbc001ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:48.075 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadab7fe700 1 -- 172.21.15.116:0/3865416163 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc007170 con 0x7fadcc060310 2020-11-20T12:22:48.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadab7fe700 1 -- 172.21.15.116:0/3865416163 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadbc0072d0 con 0x7fadcc060310 2020-11-20T12:22:48.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadd0b79700 1 -- 172.21.15.116:0/3865416163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadcc162180 con 0x7fadcc060310 2020-11-20T12:22:48.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadab7fe700 1 -- 172.21.15.116:0/3865416163 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadbc00aa30 con 0x7fadcc060310 2020-11-20T12:22:48.076 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.062+0000 7fadd0b79700 1 -- 172.21.15.116:0/3865416163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadcc1523d0 con 0x7fadcc060310 2020-11-20T12:22:48.077 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.066+0000 7fadd0b79700 1 -- 172.21.15.116:0/3865416163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc162310 con 0x7fadcc060310 2020-11-20T12:22:48.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.066+0000 7fadab7fe700 1 -- 172.21.15.116:0/3865416163 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fadbc010070 con 0x7fadcc060310 2020-11-20T12:22:48.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.066+0000 7fadab7fe700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fada40557c0 0x7fada4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:48.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.066+0000 7fadab7fe700 1 -- 172.21.15.116:0/3865416163 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fadbc06e3b0 con 0x7fadcc060310 2020-11-20T12:22:48.078 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.066+0000 7fadcaffd700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fada40557c0 0x7fada4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:48.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.070+0000 7fadcaffd700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fada40557c0 0x7fada4057c80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fadcc060ad0 tx=0x7fadc0013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:48.079 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.070+0000 7fadab7fe700 1 -- 172.21.15.116:0/3865416163 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fadbc044140 con 0x7fadcc060310 2020-11-20T12:22:48.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 -- 172.21.15.116:0/3865416163 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fadcc05cf30 con 0x7fadcc060310 2020-11-20T12:22:48.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadab7fe700 1 -- 172.21.15.116:0/3865416163 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7fadbc047760 con 0x7fadcc060310 2020-11-20T12:22:48.240 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:48.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 -- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fada40557c0 msgr2=0x7fada4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:48.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fada40557c0 0x7fada4057c80 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7fadcc060ad0 tx=0x7fadc0013000).stop 2020-11-20T12:22:48.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 -- 172.21.15.116:0/3865416163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadcc060310 msgr2=0x7fadcc14fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:48.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadcc060310 0x7fadcc14fc30 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7fadbc00cae0 tx=0x7fadbc001ac0).stop 2020-11-20T12:22:48.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 -- 172.21.15.116:0/3865416163 shutdown_connections 2020-11-20T12:22:48.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fada40557c0 0x7fada4057c80 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fadcc165390 0x7fadcc151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fadcc060310 0x7fadcc14fc30 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 --2- 172.21.15.116:0/3865416163 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fadcc0bfaa0 0x7fadcc161370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 -- 172.21.15.116:0/3865416163 >> 172.21.15.116:0/3865416163 conn(0x7fadcc068350 msgr2=0x7fadcc0bd530 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:48.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 -- 172.21.15.116:0/3865416163 shutdown_connections 2020-11-20T12:22:48.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.234+0000 7fadd0b79700 1 -- 172.21.15.116:0/3865416163 wait complete. 2020-11-20T12:22:48.265 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:22:48.833 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.826+0000 7fa234052700 1 Processor -- start 2020-11-20T12:22:48.834 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.826+0000 7fa234052700 1 -- start start 2020-11-20T12:22:48.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa234052700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa22c151b60 0x7fa22c0a5670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:48.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa234052700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa22c0c9490 0x7fa22c0c38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:48.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa234052700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22c0c6840 con 0x7fa22c0c3dd0 2020-11-20T12:22:48.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa234052700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22c0c6a70 con 0x7fa22c0c9490 2020-11-20T12:22:48.838 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa234052700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c0c6d10 con 0x7fa22c151b60 2020-11-20T12:22:48.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa2315ed700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa22c0c9490 0x7fa22c0c38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:48.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa2315ed700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fa22c0c9490 0x7fa22c0c38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:51812/0 (socket says 172.21.15.116:51812) 2020-11-20T12:22:48.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa2315ed700 1 -- 172.21.15.116:0/2561372773 learned_addr learned my addr 172.21.15.116:0/2561372773 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:48.839 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa231dee700 1 --2- 172.21.15.116:0/2561372773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa22c151b60 0x7fa22c0a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:48.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa2315ed700 1 -- 172.21.15.116:0/2561372773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa22c151b60 msgr2=0x7fa22c0a5670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:48.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa2315ed700 1 --2- 172.21.15.116:0/2561372773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa22c151b60 0x7fa22c0a5670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa2315ed700 1 -- 172.21.15.116:0/2561372773 >> v1:172.21.15.173:6789/0 conn(0x7fa22c0c3dd0 legacy=0x7fa22c0c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:48.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.830+0000 7fa2315ed700 1 -- 172.21.15.116:0/2561372773 --> v2:172.21.15.173:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c0c6fb0 con 0x7fa22c0c9490 2020-11-20T12:22:48.840 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa2315ed700 1 --2- 172.21.15.116:0/2561372773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa22c0c9490 0x7fa22c0c38e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fa21c006fd0 tx=0x7fa21c006eb0).ready entity=mon.1 client_cookie=7312600baa77641a server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:48.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa230dec700 1 -- 172.21.15.116:0/2561372773 <== mon.1 v2:172.21.15.173:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c0098d0 con 0x7fa22c0c9490 2020-11-20T12:22:48.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa230dec700 1 -- 172.21.15.116:0/2561372773 <== mon.1 v2:172.21.15.173:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c009a30 con 0x7fa22c0c9490 2020-11-20T12:22:48.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa230dec700 1 -- 172.21.15.116:0/2561372773 <== mon.1 v2:172.21.15.173:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c009db0 con 0x7fa22c0c9490 2020-11-20T12:22:48.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa234052700 1 -- 172.21.15.116:0/2561372773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa22c0c9490 msgr2=0x7fa22c0c38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:48.842 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa234052700 1 --2- 172.21.15.116:0/2561372773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa22c0c9490 0x7fa22c0c38e0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7fa21c006fd0 tx=0x7fa21c006eb0).stop 2020-11-20T12:22:48.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa234052700 1 -- 172.21.15.116:0/2561372773 shutdown_connections 2020-11-20T12:22:48.843 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa234052700 1 --2- 172.21.15.116:0/2561372773 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa22c151b60 0x7fa22c0a5670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa234052700 1 --2- 172.21.15.116:0/2561372773 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa22c0c9490 0x7fa22c0c38e0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa234052700 1 -- 172.21.15.116:0/2561372773 >> 172.21.15.116:0/2561372773 conn(0x7fa22c068080 msgr2=0x7fa22c0c8940 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:48.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa234052700 1 -- 172.21.15.116:0/2561372773 shutdown_connections 2020-11-20T12:22:48.844 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.834+0000 7fa234052700 1 -- 172.21.15.116:0/2561372773 wait complete. 2020-11-20T12:22:48.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.838+0000 7fa234052700 1 Processor -- start 2020-11-20T12:22:48.845 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.838+0000 7fa234052700 1 -- start start 2020-11-20T12:22:48.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa234052700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa22c0c3dd0 0x7fa22c164a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:48.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa234052700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c0c9490 0x7fa22c161be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:48.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa234052700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa22c151b60 0x7fa22c0adf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:48.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa234052700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c165260 con 0x7fa22c0c9490 2020-11-20T12:22:48.848 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa234052700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c165500 con 0x7fa22c0c3dd0 2020-11-20T12:22:48.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa234052700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22c1657a0 con 0x7fa22c151b60 2020-11-20T12:22:48.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa2315ed700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c0c9490 0x7fa22c161be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:48.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa2315ed700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c0c9490 0x7fa22c161be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52766/0 (socket says 172.21.15.116:52766) 2020-11-20T12:22:48.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa2315ed700 1 -- 172.21.15.116:0/2442622512 learned_addr learned my addr 172.21.15.116:0/2442622512 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:48.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa231dee700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa22c0c3dd0 0x7fa22c164a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:48.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa2325ef700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa22c151b60 0x7fa22c0adf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:48.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa2315ed700 1 -- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa22c151b60 msgr2=0x7fa22c0adf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:48.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa2315ed700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa22c151b60 0x7fa22c0adf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa2315ed700 1 -- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa22c0c3dd0 msgr2=0x7fa22c164a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:48.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa2315ed700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa22c0c3dd0 0x7fa22c164a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:48.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa2315ed700 1 -- 172.21.15.116:0/2442622512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22c161740 con 0x7fa22c0c9490 2020-11-20T12:22:48.853 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa2315ed700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c0c9490 0x7fa22c161be0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7fa21c00b1b0 tx=0x7fa21c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:48.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa219ffb700 1 -- 172.21.15.116:0/2442622512 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c011e50 con 0x7fa22c0c9490 2020-11-20T12:22:48.854 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa219ffb700 1 -- 172.21.15.116:0/2442622512 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa21c01e460 con 0x7fa22c0c9490 2020-11-20T12:22:48.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa234052700 1 -- 172.21.15.116:0/2442622512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa22c161400 con 0x7fa22c0c9490 2020-11-20T12:22:48.855 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.842+0000 7fa219ffb700 1 -- 172.21.15.116:0/2442622512 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa21c01e820 con 0x7fa22c0c9490 2020-11-20T12:22:48.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.846+0000 7fa234052700 1 -- 172.21.15.116:0/2442622512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22c1620d0 con 0x7fa22c0c9490 2020-11-20T12:22:48.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.846+0000 7fa234052700 1 -- 172.21.15.116:0/2442622512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa22c162380 con 0x7fa22c0c9490 2020-11-20T12:22:48.856 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.846+0000 7fa219ffb700 1 -- 172.21.15.116:0/2442622512 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fa21c025030 con 0x7fa22c0c9490 2020-11-20T12:22:48.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.850+0000 7fa219ffb700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2040557c0 0x7fa204057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:48.858 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.850+0000 7fa219ffb700 1 -- 172.21.15.116:0/2442622512 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fa21c06fdb0 con 0x7fa22c0c9490 2020-11-20T12:22:48.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.850+0000 7fa231dee700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2040557c0 0x7fa204057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:48.859 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.850+0000 7fa231dee700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2040557c0 0x7fa204057c80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fa22800da50 tx=0x7fa228006f90).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:48.861 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:48.854+0000 7fa219ffb700 1 -- 172.21.15.116:0/2442622512 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fa21c045c50 con 0x7fa22c0c9490 2020-11-20T12:22:49.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.014+0000 7fa234052700 1 -- 172.21.15.116:0/2442622512 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa22c161590 con 0x7fa22c0c9490 2020-11-20T12:22:49.020 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.014+0000 7fa219ffb700 1 -- 172.21.15.116:0/2442622512 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+10 (secure 0 0 0) 0x7fa21c023020 con 0x7fa22c0c9490 2020-11-20T12:22:49.021 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:49.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.014+0000 7fa234052700 1 -- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2040557c0 msgr2=0x7fa204057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.014+0000 7fa234052700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2040557c0 0x7fa204057c80 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7fa22800da50 tx=0x7fa228006f90).stop 2020-11-20T12:22:49.023 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.014+0000 7fa234052700 1 -- 172.21.15.116:0/2442622512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c0c9490 msgr2=0x7fa22c161be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.014+0000 7fa234052700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c0c9490 0x7fa22c161be0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7fa21c00b1b0 tx=0x7fa21c015000).stop 2020-11-20T12:22:49.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.018+0000 7fa234052700 1 -- 172.21.15.116:0/2442622512 shutdown_connections 2020-11-20T12:22:49.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.018+0000 7fa234052700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fa2040557c0 0x7fa204057c80 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.024 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.018+0000 7fa234052700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fa22c151b60 0x7fa22c0adf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.018+0000 7fa234052700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fa22c0c3dd0 0x7fa22c164a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.018+0000 7fa234052700 1 --2- 172.21.15.116:0/2442622512 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fa22c0c9490 0x7fa22c161be0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.018+0000 7fa234052700 1 -- 172.21.15.116:0/2442622512 >> 172.21.15.116:0/2442622512 conn(0x7fa22c068080 msgr2=0x7fa22c0c7a70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:49.025 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.018+0000 7fa234052700 1 -- 172.21.15.116:0/2442622512 shutdown_connections 2020-11-20T12:22:49.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.018+0000 7fa234052700 1 -- 172.21.15.116:0/2442622512 wait complete. 2020-11-20T12:22:49.048 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:22:49.229 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.222+0000 7f4c8d659700 1 Processor -- start 2020-11-20T12:22:49.230 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.222+0000 7f4c8d659700 1 -- start start 2020-11-20T12:22:49.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c8d659700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4c88151770 0x7f4c88151b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c8d659700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c88060530 0x7f4c880609a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c8d659700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c880c7640 con 0x7f4c880c4380 2020-11-20T12:22:49.232 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c8d659700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c880bed80 con 0x7f4c88151770 2020-11-20T12:22:49.233 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c8d659700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c880bf020 con 0x7f4c88060530 2020-11-20T12:22:49.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c86ffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4c88151770 0x7f4c88151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c87fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c88060530 0x7f4c880609a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.234 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c867fc700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f4c880c4380 0x7f4c880653e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:33974/0 (socket says 172.21.15.116:33974) 2020-11-20T12:22:49.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c86ffd700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f4c88151770 0x7f4c88151b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52772/0 (socket says 172.21.15.116:52772) 2020-11-20T12:22:49.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c867fc700 1 -- 172.21.15.116:0/3516800941 learned_addr learned my addr 172.21.15.116:0/3516800941 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:49.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c85ffb700 1 -- 172.21.15.116:0/3516800941 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360788293 0 0) 0x7f4c880c7640 con 0x7f4c880c4380 2020-11-20T12:22:49.235 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c85ffb700 1 -- 172.21.15.116:0/3516800941 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c74003240 con 0x7f4c880c4380 2020-11-20T12:22:49.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c86ffd700 1 -- 172.21.15.116:0/3516800941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c88060530 msgr2=0x7f4c880609a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:49.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c86ffd700 1 --2- 172.21.15.116:0/3516800941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c88060530 0x7f4c880609a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.237 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c86ffd700 1 -- 172.21.15.116:0/3516800941 >> v1:172.21.15.116:6789/0 conn(0x7f4c880c4380 legacy=0x7f4c880653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c85ffb700 1 -- 172.21.15.116:0/3516800941 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 1543080773 0 0) 0x7f4c74003240 con 0x7f4c880c4380 2020-11-20T12:22:49.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.226+0000 7f4c86ffd700 1 -- 172.21.15.116:0/3516800941 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c880bf2c0 con 0x7f4c88151770 2020-11-20T12:22:49.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c86ffd700 1 --2- 172.21.15.116:0/3516800941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c88151770 0x7f4c88151b50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f4c7800b9a0 tx=0x7f4c78006df0).ready entity=mon.0 client_cookie=2134bd4d161dc49e server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:49.238 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c85ffb700 1 -- 172.21.15.116:0/3516800941 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c0014e0 con 0x7f4c88151770 2020-11-20T12:22:49.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c85ffb700 1 -- 172.21.15.116:0/3516800941 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c78011400 con 0x7f4c88151770 2020-11-20T12:22:49.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c85ffb700 1 -- 172.21.15.116:0/3516800941 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c78011780 con 0x7f4c88151770 2020-11-20T12:22:49.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c8d659700 1 -- 172.21.15.116:0/3516800941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c88151770 msgr2=0x7f4c88151b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.239 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c8d659700 1 --2- 172.21.15.116:0/3516800941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c88151770 0x7f4c88151b50 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f4c7800b9a0 tx=0x7f4c78006df0).stop 2020-11-20T12:22:49.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c8d659700 1 -- 172.21.15.116:0/3516800941 shutdown_connections 2020-11-20T12:22:49.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c8d659700 1 --2- 172.21.15.116:0/3516800941 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c88151770 0x7f4c88151b50 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c8d659700 1 --2- 172.21.15.116:0/3516800941 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c88060530 0x7f4c880609a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.240 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c8d659700 1 -- 172.21.15.116:0/3516800941 >> 172.21.15.116:0/3516800941 conn(0x7f4c880ca810 msgr2=0x7f4c880c6820 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:49.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c8d659700 1 -- 172.21.15.116:0/3516800941 shutdown_connections 2020-11-20T12:22:49.241 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.230+0000 7f4c8d659700 1 -- 172.21.15.116:0/3516800941 wait complete. 2020-11-20T12:22:49.242 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c8d659700 1 Processor -- start 2020-11-20T12:22:49.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c8d659700 1 -- start start 2020-11-20T12:22:49.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c8d659700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c88060530 0x7f4c88161d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.243 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c8d659700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c880c4380 0x7f4c880a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.244 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c8d659700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c880abc60 0x7f4c881497c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c8d659700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88164940 con 0x7f4c880c4380 2020-11-20T12:22:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c8d659700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88164be0 con 0x7f4c88060530 2020-11-20T12:22:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c8d659700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c88164e80 con 0x7f4c880abc60 2020-11-20T12:22:49.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c867fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c880c4380 0x7f4c880a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c867fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c880c4380 0x7f4c880a9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52780/0 (socket says 172.21.15.116:52780) 2020-11-20T12:22:49.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c867fc700 1 -- 172.21.15.116:0/2720400962 learned_addr learned my addr 172.21.15.116:0/2720400962 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:49.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c87fff700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c880abc60 0x7f4c881497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.246 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.234+0000 7f4c86ffd700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c88060530 0x7f4c88161d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c867fc700 1 -- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c880abc60 msgr2=0x7f4c881497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c867fc700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c880abc60 0x7f4c881497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c867fc700 1 -- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c88060530 msgr2=0x7f4c88161d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c867fc700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c88060530 0x7f4c88161d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c867fc700 1 -- 172.21.15.116:0/2720400962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c88165120 con 0x7f4c880c4380 2020-11-20T12:22:49.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c86ffd700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c88060530 0x7f4c88161d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:49.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c867fc700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c880c4380 0x7f4c880a9ad0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f4c7c00cae0 tx=0x7f4c7c0019b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:49.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c6effd700 1 -- 172.21.15.116:0/2720400962 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c007170 con 0x7f4c880c4380 2020-11-20T12:22:49.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c6effd700 1 -- 172.21.15.116:0/2720400962 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7c0072f0 con 0x7f4c880c4380 2020-11-20T12:22:49.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c8d659700 1 -- 172.21.15.116:0/2720400962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c880a9fc0 con 0x7f4c880c4380 2020-11-20T12:22:49.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c6effd700 1 -- 172.21.15.116:0/2720400962 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00a980 con 0x7f4c880c4380 2020-11-20T12:22:49.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c8d659700 1 -- 172.21.15.116:0/2720400962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c880ac0e0 con 0x7f4c880c4380 2020-11-20T12:22:49.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.238+0000 7f4c8d659700 1 -- 172.21.15.116:0/2720400962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c880ac3c0 con 0x7f4c880c4380 2020-11-20T12:22:49.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.242+0000 7f4c6effd700 1 -- 172.21.15.116:0/2720400962 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f4c7c010070 con 0x7f4c880c4380 2020-11-20T12:22:49.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.242+0000 7f4c6effd700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c640557c0 0x7f4c64057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.242+0000 7f4c6effd700 1 -- 172.21.15.116:0/2720400962 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f4c7c06e550 con 0x7f4c880c4380 2020-11-20T12:22:49.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.242+0000 7f4c86ffd700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c640557c0 0x7f4c64057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.242+0000 7f4c86ffd700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c640557c0 0x7f4c64057c80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f4c88066f00 tx=0x7f4c78013000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:49.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.246+0000 7f4c6effd700 1 -- 172.21.15.116:0/2720400962 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f4c880ac3c0 con 0x7f4c880c4380 2020-11-20T12:22:49.411 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.402+0000 7f4c8d659700 1 -- 172.21.15.116:0/2720400962 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c880ac3c0 con 0x7f4c880c4380 2020-11-20T12:22:49.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c6effd700 1 -- 172.21.15.116:0/2720400962 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+10 (secure 0 0 0) 0x7f4c880ac3c0 con 0x7f4c880c4380 2020-11-20T12:22:49.412 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:49.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 -- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c640557c0 msgr2=0x7f4c64057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c640557c0 0x7f4c64057c80 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f4c88066f00 tx=0x7f4c78013000).stop 2020-11-20T12:22:49.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 -- 172.21.15.116:0/2720400962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c880c4380 msgr2=0x7f4c880a9ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c880c4380 0x7f4c880a9ad0 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 rx=0x7f4c7c00cae0 tx=0x7f4c7c0019b0).stop 2020-11-20T12:22:49.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 -- 172.21.15.116:0/2720400962 shutdown_connections 2020-11-20T12:22:49.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f4c640557c0 0x7f4c64057c80 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f4c880abc60 0x7f4c881497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f4c88060530 0x7f4c88161d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 --2- 172.21.15.116:0/2720400962 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f4c880c4380 0x7f4c880a9ad0 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 -- 172.21.15.116:0/2720400962 >> 172.21.15.116:0/2720400962 conn(0x7f4c880ca810 msgr2=0x7f4c88063d10 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:49.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 -- 172.21.15.116:0/2720400962 shutdown_connections 2020-11-20T12:22:49.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.406+0000 7f4c8d659700 1 -- 172.21.15.116:0/2720400962 wait complete. 2020-11-20T12:22:49.437 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:22:49.437 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:49.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.646+0000 7fec92536700 1 Processor -- start 2020-11-20T12:22:49.655 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.646+0000 7fec92536700 1 -- start start 2020-11-20T12:22:49.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.650+0000 7fec92536700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c3780 0x7fec8c0c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.650+0000 7fec92536700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fec8c0a6280 0x7fec8c0c1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.650+0000 7fec92536700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec8c0c1e60 con 0x7fec8c0c22f0 2020-11-20T12:22:49.661 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.650+0000 7fec92536700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec8c14fed0 con 0x7fec8c0a6280 2020-11-20T12:22:49.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.650+0000 7fec92536700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0a6700 con 0x7fec8c0c3780 2020-11-20T12:22:49.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.650+0000 7fec8bfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c3780 0x7fec8c0c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.650+0000 7fec8b7fe700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fec8c0a6280 0x7fec8c0c1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.662 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.650+0000 7fec8bfff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c3780 0x7fec8c0c3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:33504/0 (socket says 172.21.15.116:33504) 2020-11-20T12:22:49.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.650+0000 7fec8bfff700 1 -- 172.21.15.116:0/3856926405 learned_addr learned my addr 172.21.15.116:0/3856926405 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:49.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec8bfff700 1 -- 172.21.15.116:0/3856926405 >> v2:172.21.15.173:3300/0 conn(0x7fec8c0a6280 msgr2=0x7fec8c0c1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:49.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec8bfff700 1 --2- 172.21.15.116:0/3856926405 >> v2:172.21.15.173:3300/0 conn(0x7fec8c0a6280 0x7fec8c0c1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec8bfff700 1 -- 172.21.15.116:0/3856926405 >> v1:172.21.15.173:6789/0 conn(0x7fec8c0c22f0 legacy=0x7fec8c150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.663 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec8bfff700 1 -- 172.21.15.116:0/3856926405 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c0a6950 con 0x7fec8c0c3780 2020-11-20T12:22:49.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec8b7fe700 1 --2- 172.21.15.116:0/3856926405 >> v2:172.21.15.173:3300/0 conn(0x7fec8c0a6280 0x7fec8c0c1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:49.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec8bfff700 1 --2- 172.21.15.116:0/3856926405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c3780 0x7fec8c0c3b60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fec7c00ba20 tx=0x7fec7c006eb0).ready entity=mon.2 client_cookie=fb36bd7d20cdd299 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:49.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec8affd700 1 -- 172.21.15.116:0/3856926405 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c009710 con 0x7fec8c0c3780 2020-11-20T12:22:49.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec8affd700 1 -- 172.21.15.116:0/3856926405 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c009870 con 0x7fec8c0c3780 2020-11-20T12:22:49.664 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec8affd700 1 -- 172.21.15.116:0/3856926405 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c009bf0 con 0x7fec8c0c3780 2020-11-20T12:22:49.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec92536700 1 -- 172.21.15.116:0/3856926405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c3780 msgr2=0x7fec8c0c3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec92536700 1 --2- 172.21.15.116:0/3856926405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c3780 0x7fec8c0c3b60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fec7c00ba20 tx=0x7fec7c006eb0).stop 2020-11-20T12:22:49.666 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec92536700 1 -- 172.21.15.116:0/3856926405 shutdown_connections 2020-11-20T12:22:49.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec92536700 1 --2- 172.21.15.116:0/3856926405 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c3780 0x7fec8c0c3b60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec92536700 1 --2- 172.21.15.116:0/3856926405 >> v2:172.21.15.173:3300/0 conn(0x7fec8c0a6280 0x7fec8c0c1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec92536700 1 -- 172.21.15.116:0/3856926405 >> 172.21.15.116:0/3856926405 conn(0x7fec8c0c4f80 msgr2=0x7fec8c0c0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:49.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec92536700 1 -- 172.21.15.116:0/3856926405 shutdown_connections 2020-11-20T12:22:49.667 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.654+0000 7fec92536700 1 -- 172.21.15.116:0/3856926405 wait complete. 2020-11-20T12:22:49.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.658+0000 7fec92536700 1 Processor -- start 2020-11-20T12:22:49.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.658+0000 7fec92536700 1 -- start start 2020-11-20T12:22:49.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.658+0000 7fec92536700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec8c0a6280 0x7fec8c0a4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.668 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.658+0000 7fec92536700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c22f0 0x7fec8c14c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.658+0000 7fec92536700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec8c0c3780 0x7fec8c0c0840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.658+0000 7fec92536700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0a50c0 con 0x7fec8c0a6280 2020-11-20T12:22:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.658+0000 7fec92536700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c1669e0 con 0x7fec8c0c3780 2020-11-20T12:22:49.669 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.658+0000 7fec92536700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec8c0a41f0 con 0x7fec8c0c22f0 2020-11-20T12:22:49.670 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.658+0000 7fec8bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec8c0a6280 0x7fec8c0a4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.658+0000 7fec8bfff700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec8c0a6280 0x7fec8c0a4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52792/0 (socket says 172.21.15.116:52792) 2020-11-20T12:22:49.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec8bfff700 1 -- 172.21.15.116:0/3577149956 learned_addr learned my addr 172.21.15.116:0/3577149956 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:49.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec90ad3700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec8c0c3780 0x7fec8c0c0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.671 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec8b7fe700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c22f0 0x7fec8c14c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec8bfff700 1 -- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c22f0 msgr2=0x7fec8c14c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec8bfff700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c22f0 0x7fec8c14c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec8bfff700 1 -- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec8c0c3780 msgr2=0x7fec8c0c0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.672 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec8bfff700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec8c0c3780 0x7fec8c0c0840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.673 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec8bfff700 1 -- 172.21.15.116:0/3577149956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec8c0a4490 con 0x7fec8c0a6280 2020-11-20T12:22:49.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec8b7fe700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c22f0 0x7fec8c14c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:49.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec8bfff700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec8c0a6280 0x7fec8c0a4bd0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fec7c00c010 tx=0x7fec7c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:49.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec6bfff700 1 -- 172.21.15.116:0/3577149956 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c011c40 con 0x7fec8c0a6280 2020-11-20T12:22:49.674 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec6bfff700 1 -- 172.21.15.116:0/3577149956 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7c011da0 con 0x7fec8c0a6280 2020-11-20T12:22:49.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec6bfff700 1 -- 172.21.15.116:0/3577149956 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7c01e6b0 con 0x7fec8c0a6280 2020-11-20T12:22:49.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec92536700 1 -- 172.21.15.116:0/3577149956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec8c0a4730 con 0x7fec8c0a6280 2020-11-20T12:22:49.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec92536700 1 -- 172.21.15.116:0/3577149956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec8c0c0d30 con 0x7fec8c0a6280 2020-11-20T12:22:49.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.662+0000 7fec92536700 1 -- 172.21.15.116:0/3577149956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec8c0aa600 con 0x7fec8c0a6280 2020-11-20T12:22:49.675 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.666+0000 7fec6bfff700 1 -- 172.21.15.116:0/3577149956 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fec7c025030 con 0x7fec8c0a6280 2020-11-20T12:22:49.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.666+0000 7fec6bfff700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec640557c0 0x7fec64057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:49.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.666+0000 7fec6bfff700 1 -- 172.21.15.116:0/3577149956 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fec7c06f950 con 0x7fec8c0a6280 2020-11-20T12:22:49.676 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.666+0000 7fec8b7fe700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec640557c0 0x7fec64057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:49.677 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.666+0000 7fec8b7fe700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec640557c0 0x7fec64057c80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7fec8c0c13c0 tx=0x7fec80008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:49.678 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.670+0000 7fec6bfff700 1 -- 172.21.15.116:0/3577149956 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fec8c0aa600 con 0x7fec8c0a6280 2020-11-20T12:22:49.886 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.878+0000 7fec92536700 1 -- 172.21.15.116:0/3577149956 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec8c0aa600 con 0x7fec8c0a6280 2020-11-20T12:22:49.887 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec6bfff700 1 -- 172.21.15.116:0/3577149956 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2853 (secure 0 0 0) 0x7fec8c0aa600 con 0x7fec8c0a6280 2020-11-20T12:22:49.887 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:49.888 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":75,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/363707569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":363707569},{"type":"v1","addr":"172.21.15.173:6835","nonce":363707569}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":71},{"gid":7202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/1854325940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1854325940},{"type":"v1","addr":"172.21.15.116:6834","nonce":1854325940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72},{"gid":8437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3920088127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3920088127},{"type":"v1","addr":"172.21.15.173:6837","nonce":3920088127}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:22:35.338520+0000","modified":"2020-11-20T12:22:37.346491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8589":{"gid":8589,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1101967997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1101967997},{"type":"v1","addr":"172.21.15.116:6835","nonce":1101967997}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-11-20T12:22:49.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 -- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec640557c0 msgr2=0x7fec64057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec640557c0 0x7fec64057c80 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7fec8c0c13c0 tx=0x7fec80008040).stop 2020-11-20T12:22:49.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 -- 172.21.15.116:0/3577149956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec8c0a6280 msgr2=0x7fec8c0a4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:49.889 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec8c0a6280 0x7fec8c0a4bd0 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fec7c00c010 tx=0x7fec7c015000).stop 2020-11-20T12:22:49.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 -- 172.21.15.116:0/3577149956 shutdown_connections 2020-11-20T12:22:49.890 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fec640557c0 0x7fec64057c80 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fec8c0c3780 0x7fec8c0c0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fec8c0a6280 0x7fec8c0a4bd0 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 --2- 172.21.15.116:0/3577149956 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fec8c0c22f0 0x7fec8c14c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:49.891 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 -- 172.21.15.116:0/3577149956 >> 172.21.15.116:0/3577149956 conn(0x7fec8c0c4f80 msgr2=0x7fec8c0ba640 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:49.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 -- 172.21.15.116:0/3577149956 shutdown_connections 2020-11-20T12:22:49.892 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:49.882+0000 7fec92536700 1 -- 172.21.15.116:0/3577149956 wait complete. 2020-11-20T12:22:49.892 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 75 2020-11-20T12:22:50.320 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:22:35.338520+0000', 'modified': '2020-11-20T12:22:37.346491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8589': {'gid': 8589, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/1101967997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 1101967997}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 1101967997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:22:50.321 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:22:50.322 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:22:50.516 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":8589,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.153699877999999,"mdsmap_epoch":75,"osdmap_epoch":129,"osdmap_epoch_barrier":129,"uptime":20.700214431999999} 2020-11-20T12:22:50.517 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 8589, "mdsmap_epoch": 75, "osdmap_epoch": 129, "osdmap_epoch_barrier": 129, "rank_uptime": 15.153699878, "state": "up:active", "uptime": 20.700214432, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:22:50.517 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:22:50.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:50.518 INFO:teuthology.orchestra.run.smithi116:> mkdir -p /home/ubuntu/cephtest/mnt.2 2020-11-20T12:22:50.540 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:22:50.541 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:50.613 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:50.614 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:22:50.641 INFO:teuthology.orchestra.run.smithi116.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:50.641 INFO:teuthology.orchestra.run.smithi116.stdout:mount.ceph: options "norequire_active_mds,name=2,norbytes" will pass to kernel. 2020-11-20T12:22:50.646 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:22:50.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:50.648 INFO:teuthology.orchestra.run.smithi116:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2020-11-20T12:22:50.681 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2020-11-20T12:22:50.682 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi173.front.sepia.ceph.com 2020-11-20T12:22:50.683 INFO:tasks.cephfs.mount:self.client.name = client.3 2020-11-20T12:22:50.684 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2020-11-20T12:22:50.684 INFO:tasks.cephfs.mount:self.cephfs_name = None 2020-11-20T12:22:50.685 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2020-11-20T12:22:50.685 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2020-11-20T12:22:50.686 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:22:50.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:50.688 INFO:teuthology.orchestra.run.smithi173:> ip addr 2020-11-20T12:22:50.708 INFO:teuthology.orchestra.run.smithi173.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2020-11-20T12:22:50.709 INFO:teuthology.orchestra.run.smithi173.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2020-11-20T12:22:50.709 INFO:teuthology.orchestra.run.smithi173.stdout: inet 127.0.0.1/8 scope host lo 2020-11-20T12:22:50.710 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:50.710 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 ::1/128 scope host 2020-11-20T12:22:50.711 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:50.713 INFO:teuthology.orchestra.run.smithi173.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:50.713 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6a brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:50.714 INFO:teuthology.orchestra.run.smithi173.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:50.714 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:d9:38:6b brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:50.714 INFO:teuthology.orchestra.run.smithi173.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2020-11-20T12:22:50.714 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:58 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:50.715 INFO:teuthology.orchestra.run.smithi173.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2020-11-20T12:22:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 0c:c4:7a:88:72:59 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.21.15.173/20 brd 172.21.15.255 scope global enp3s0f1 2020-11-20T12:22:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:50.715 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::ec4:7aff:fe88:7259/64 scope link 2020-11-20T12:22:50.716 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:50.716 INFO:teuthology.orchestra.run.smithi173.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2020-11-20T12:22:50.716 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 02:42:76:a1:05:87 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:50.716 INFO:teuthology.orchestra.run.smithi173.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2020-11-20T12:22:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:50.717 INFO:teuthology.orchestra.run.smithi173.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2020-11-20T12:22:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff 2020-11-20T12:22:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2020-11-20T12:22:50.717 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:50.718 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::649a:64ff:fe62:78d6/64 scope link 2020-11-20T12:22:50.718 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:50.718 INFO:teuthology.orchestra.run.smithi173.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:50.719 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether ce:a9:2b:51:0b:5a brd ff:ff:ff:ff:ff:ff link-netnsid 0 2020-11-20T12:22:50.720 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::cca9:2bff:fe51:b5a/64 scope link 2020-11-20T12:22:50.720 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:50.720 INFO:teuthology.orchestra.run.smithi173.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2020-11-20T12:22:50.721 INFO:teuthology.orchestra.run.smithi173.stdout: link/ether 7e:b8:82:cf:a0:20 brd ff:ff:ff:ff:ff:ff link-netnsid 1 2020-11-20T12:22:50.721 INFO:teuthology.orchestra.run.smithi173.stdout: inet6 fe80::7cb8:82ff:fecf:a020/64 scope link 2020-11-20T12:22:50.721 INFO:teuthology.orchestra.run.smithi173.stdout: valid_lft forever preferred_lft forever 2020-11-20T12:22:50.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:50.723 INFO:teuthology.orchestra.run.smithi173:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2020-11-20T12:22:50.787 INFO:teuthology.orchestra.run.smithi173.stdout:1 2020-11-20T12:22:50.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:50.795 INFO:teuthology.orchestra.run.smithi173:> route 2020-11-20T12:22:50.819 INFO:teuthology.orchestra.run.smithi173.stdout:Kernel IP routing table 2020-11-20T12:22:50.820 INFO:teuthology.orchestra.run.smithi173.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2020-11-20T12:22:50.820 INFO:teuthology.orchestra.run.smithi173.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2020-11-20T12:22:50.820 INFO:teuthology.orchestra.run.smithi173.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2020-11-20T12:22:50.820 INFO:teuthology.orchestra.run.smithi173.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 enp3s0f1 2020-11-20T12:22:50.821 INFO:teuthology.orchestra.run.smithi173.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2020-11-20T12:22:50.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:50.822 INFO:teuthology.orchestra.run.smithi173:> (cd / && exec sudo bash -c ' 2020-11-20T12:22:50.822 INFO:teuthology.orchestra.run.smithi173:> set -e 2020-11-20T12:22:50.823 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2020-11-20T12:22:50.823 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2020-11-20T12:22:50.823 INFO:teuthology.orchestra.run.smithi173:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2020-11-20T12:22:50.824 INFO:teuthology.orchestra.run.smithi173:> ') 2020-11-20T12:22:50.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:50.970 INFO:teuthology.orchestra.run.smithi173:> ip netns list 2020-11-20T12:22:50.990 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2020-11-20T12:22:50.990 INFO:teuthology.orchestra.run.smithi173.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2020-11-20T12:22:50.990 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.3' netns for '/home/ubuntu/cephtest/mnt.3' 2020-11-20T12:22:50.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:50.991 INFO:teuthology.orchestra.run.smithi173:> sudo ip link set brx.1 up 2020-11-20T12:22:51.070 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2020-11-20T12:22:51.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.238+0000 7f3189d84700 1 Processor -- start 2020-11-20T12:22:51.245 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.238+0000 7f3189d84700 1 -- start start 2020-11-20T12:22:51.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.238+0000 7f3189d84700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f3184153450 0x7f3184062c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.238+0000 7f3189d84700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 0x7f31840b9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.248 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.238+0000 7f3189d84700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31841543a0 con 0x7f3184062880 2020-11-20T12:22:51.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.238+0000 7f3189d84700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3184067c00 con 0x7f3184153450 2020-11-20T12:22:51.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.238+0000 7f3189d84700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3184153920 con 0x7f31840677a0 2020-11-20T12:22:51.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f3189583700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 0x7f31840b9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.249 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f3189583700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 0x7f31840b9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:33526/0 (socket says 172.21.15.116:33526) 2020-11-20T12:22:51.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f3189583700 1 -- 172.21.15.116:0/1783576072 learned_addr learned my addr 172.21.15.116:0/1783576072 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:51.250 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f3183fff700 1 --2- 172.21.15.116:0/1783576072 >> v2:172.21.15.173:3300/0 conn(0x7f3184153450 0x7f3184062c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f3189583700 1 -- 172.21.15.116:0/1783576072 >> v2:172.21.15.173:3300/0 conn(0x7f3184153450 msgr2=0x7f3184062c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:51.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f3189583700 1 --2- 172.21.15.116:0/1783576072 >> v2:172.21.15.173:3300/0 conn(0x7f3184153450 0x7f3184062c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.251 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f3189583700 1 -- 172.21.15.116:0/1783576072 >> v1:172.21.15.173:6789/0 conn(0x7f3184062880 legacy=0x7f31840c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:51.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f3189583700 1 -- 172.21.15.116:0/1783576072 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184153bc0 con 0x7f31840677a0 2020-11-20T12:22:51.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f3189583700 1 --2- 172.21.15.116:0/1783576072 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 0x7f31840b9710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f317c009d50 tx=0x7f317c009290).ready entity=mon.2 client_cookie=ca4602124b4f2f2c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:51.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f31837fe700 1 -- 172.21.15.116:0/1783576072 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c00bbc0 con 0x7f31840677a0 2020-11-20T12:22:51.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f31837fe700 1 -- 172.21.15.116:0/1783576072 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f317c00bd20 con 0x7f31840677a0 2020-11-20T12:22:51.252 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.242+0000 7f31837fe700 1 -- 172.21.15.116:0/1783576072 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f317c011610 con 0x7f31840677a0 2020-11-20T12:22:51.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.246+0000 7f3189d84700 1 -- 172.21.15.116:0/1783576072 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 msgr2=0x7f31840b9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:51.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.246+0000 7f3189d84700 1 --2- 172.21.15.116:0/1783576072 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 0x7f31840b9710 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f317c009d50 tx=0x7f317c009290).stop 2020-11-20T12:22:51.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.246+0000 7f3189d84700 1 -- 172.21.15.116:0/1783576072 shutdown_connections 2020-11-20T12:22:51.253 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.246+0000 7f3189d84700 1 --2- 172.21.15.116:0/1783576072 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 0x7f31840b9710 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.246+0000 7f3189d84700 1 --2- 172.21.15.116:0/1783576072 >> v2:172.21.15.173:3300/0 conn(0x7f3184153450 0x7f3184062c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.246+0000 7f3189d84700 1 -- 172.21.15.116:0/1783576072 >> 172.21.15.116:0/1783576072 conn(0x7f3184060c40 msgr2=0x7f31840c4370 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:51.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.246+0000 7f3189d84700 1 -- 172.21.15.116:0/1783576072 shutdown_connections 2020-11-20T12:22:51.254 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.246+0000 7f3189d84700 1 -- 172.21.15.116:0/1783576072 wait complete. 2020-11-20T12:22:51.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3189d84700 1 Processor -- start 2020-11-20T12:22:51.255 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3189d84700 1 -- start start 2020-11-20T12:22:51.256 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3189d84700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184062880 0x7f31840a7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3189d84700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 0x7f3184125f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3189d84700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3184153450 0x7f31841269c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3189d84700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3184151c80 con 0x7f3184062880 2020-11-20T12:22:51.257 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3189d84700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3184151f20 con 0x7f3184153450 2020-11-20T12:22:51.258 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3189d84700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31841521c0 con 0x7f31840677a0 2020-11-20T12:22:51.259 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3188d82700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184062880 0x7f31840a7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3183fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 0x7f3184125f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3189583700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3184153450 0x7f31841269c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3188d82700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184062880 0x7f31840a7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52810/0 (socket says 172.21.15.116:52810) 2020-11-20T12:22:51.260 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3189583700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3184153450 0x7f31841269c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3300/0 says I am v2:172.21.15.116:51862/0 (socket says 172.21.15.116:51862) 2020-11-20T12:22:51.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3183fff700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 0x7f3184125f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.173:3301/0 says I am v2:172.21.15.116:33532/0 (socket says 172.21.15.116:33532) 2020-11-20T12:22:51.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3188d82700 1 -- 172.21.15.116:0/19928327 learned_addr learned my addr 172.21.15.116:0/19928327 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:51.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3188d82700 1 -- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 msgr2=0x7f3184125f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:51.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3188d82700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 0x7f3184125f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.261 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3188d82700 1 -- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3184153450 msgr2=0x7f31841269c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:51.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3188d82700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3184153450 0x7f31841269c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.250+0000 7f3188d82700 1 -- 172.21.15.116:0/19928327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3184152460 con 0x7f3184062880 2020-11-20T12:22:51.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.254+0000 7f3188d82700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184062880 0x7f31840a7320 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f3178001e70 tx=0x7f3178007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:51.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.254+0000 7f3180ff9700 1 -- 172.21.15.116:0/19928327 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3178007020 con 0x7f3184062880 2020-11-20T12:22:51.262 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.254+0000 7f3180ff9700 1 -- 172.21.15.116:0/19928327 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3178007180 con 0x7f3184062880 2020-11-20T12:22:51.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.254+0000 7f3180ff9700 1 -- 172.21.15.116:0/19928327 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31780064f0 con 0x7f3184062880 2020-11-20T12:22:51.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.254+0000 7f3189d84700 1 -- 172.21.15.116:0/19928327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3184152700 con 0x7f3184062880 2020-11-20T12:22:51.263 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.254+0000 7f3189d84700 1 -- 172.21.15.116:0/19928327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31840aa1c0 con 0x7f3184062880 2020-11-20T12:22:51.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.254+0000 7f3189d84700 1 -- 172.21.15.116:0/19928327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f318405cf30 con 0x7f3184062880 2020-11-20T12:22:51.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.254+0000 7f3180ff9700 1 -- 172.21.15.116:0/19928327 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f3178010070 con 0x7f3184062880 2020-11-20T12:22:51.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.258+0000 7f3180ff9700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f315c0557c0 0x7f315c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.264 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.258+0000 7f3180ff9700 1 -- 172.21.15.116:0/19928327 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f317806d960 con 0x7f3184062880 2020-11-20T12:22:51.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.258+0000 7f3183fff700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f315c0557c0 0x7f315c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.265 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.258+0000 7f3183fff700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f315c0557c0 0x7f315c057c80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f31840673c0 tx=0x7f3170006df0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:51.268 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.262+0000 7f3180ff9700 1 -- 172.21.15.116:0/19928327 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f3178042fd0 con 0x7f3184062880 2020-11-20T12:22:51.430 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.422+0000 7f3189d84700 1 -- 172.21.15.116:0/19928327 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3184152890 con 0x7f3184062880 2020-11-20T12:22:51.433 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.426+0000 7f3180ff9700 1 -- 172.21.15.116:0/19928327 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v129) v1 ==== 74+0+16888 (secure 0 0 0) 0x7f3178024210 con 0x7f3184062880 2020-11-20T12:22:51.434 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:51.434 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":129,"fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","created":"2020-11-20T12:15:41.360400+0000","modified":"2020-11-20T12:22:36.342598+0000","last_up_change":"2020-11-20T12:15:50.549644+0000","last_in_change":"2020-11-20T12:15:50.549644+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2020-11-20T12:15:46.607758+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2020-11-20T12:15:55.044020+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2020-11-20T12:22:33.838237+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"128","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2020-11-20T12:22:34.312791+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"129","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"b46f3e7a-a64d-4724-8c3e-ea252ab40ce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6800","nonce":14053},{"type":"v1","addr":"172.21.15.116:6801","nonce":14053}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6803","nonce":14053},{"type":"v1","addr":"172.21.15.116:6805","nonce":14053}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6810","nonce":14053},{"type":"v1","addr":"172.21.15.116:6812","nonce":14053}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6806","nonce":14053},{"type":"v1","addr":"172.21.15.116:6808","nonce":14053}]},"public_addr":"172.21.15.116:6801/14053","cluster_addr":"172.21.15.116:6805/14053","heartbeat_back_addr":"172.21.15.116:6812/14053","heartbeat_front_addr":"172.21.15.116:6808/14053","state":["exists","up"]},{"osd":1,"uuid":"90b5585c-eff0-40f1-8574-01d58ad5d7d0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6824","nonce":14069},{"type":"v1","addr":"172.21.15.116:6825","nonce":14069}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6826","nonce":14069},{"type":"v1","addr":"172.21.15.116:6827","nonce":14069}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6830","nonce":14069},{"type":"v1","addr":"172.21.15.116:6831","nonce":14069}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6828","nonce":14069},{"type":"v1","addr":"172.21.15.116:6829","nonce":14069}]},"public_addr":"172.21.15.116:6825/14069","cluster_addr":"172.21.15.116:6827/14069","heartbeat_back_addr":"172.21.15.116:6831/14069","heartbeat_front_addr":"172.21.15.116:6829/14069","state":["exists","up"]},{"osd":2,"uuid":"063ebbab-aa94-46f2-a8c3-774fd7d8c39c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6814","nonce":14054},{"type":"v1","addr":"172.21.15.116:6816","nonce":14054}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6818","nonce":14054},{"type":"v1","addr":"172.21.15.116:6819","nonce":14054}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6822","nonce":14054},{"type":"v1","addr":"172.21.15.116:6823","nonce":14054}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6820","nonce":14054},{"type":"v1","addr":"172.21.15.116:6821","nonce":14054}]},"public_addr":"172.21.15.116:6816/14054","cluster_addr":"172.21.15.116:6819/14054","heartbeat_back_addr":"172.21.15.116:6823/14054","heartbeat_front_addr":"172.21.15.116:6821/14054","state":["exists","up"]},{"osd":3,"uuid":"5a9021b2-8618-45a1-a93d-d000a17ef73d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6802","nonce":14055},{"type":"v1","addr":"172.21.15.116:6804","nonce":14055}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6807","nonce":14055},{"type":"v1","addr":"172.21.15.116:6809","nonce":14055}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6815","nonce":14055},{"type":"v1","addr":"172.21.15.116:6817","nonce":14055}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6811","nonce":14055},{"type":"v1","addr":"172.21.15.116:6813","nonce":14055}]},"public_addr":"172.21.15.116:6804/14055","cluster_addr":"172.21.15.116:6809/14055","heartbeat_back_addr":"172.21.15.116:6817/14055","heartbeat_front_addr":"172.21.15.116:6813/14055","state":["exists","up"]},{"osd":4,"uuid":"b3238680-8ed8-4a81-aa6a-20548bb44de2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6802","nonce":14206},{"type":"v1","addr":"172.21.15.173:6803","nonce":14206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6804","nonce":14206},{"type":"v1","addr":"172.21.15.173:6805","nonce":14206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6808","nonce":14206},{"type":"v1","addr":"172.21.15.173:6809","nonce":14206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6806","nonce":14206},{"type":"v1","addr":"172.21.15.173:6807","nonce":14206}]},"public_addr":"172.21.15.173:6803/14206","cluster_addr":"172.21.15.173:6805/14206","heartbeat_back_addr":"172.21.15.173:6809/14206","heartbeat_front_addr":"172.21.15.173:6807/14206","state":["exists","up"]},{"osd":5,"uuid":"38264535-e8e9-409b-bbc5-46f51fcfcd75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6818","nonce":14215},{"type":"v1","addr":"172.21.15.173:6821","nonce":14215}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6824","nonce":14215},{"type":"v1","addr":"172.21.15.173:6826","nonce":14215}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6831","nonce":14215},{"type":"v1","addr":"172.21.15.173:6833","nonce":14215}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6827","nonce":14215},{"type":"v1","addr":"172.21.15.173:6829","nonce":14215}]},"public_addr":"172.21.15.173:6821/14215","cluster_addr":"172.21.15.173:6826/14215","heartbeat_back_addr":"172.21.15.173:6833/14215","heartbeat_front_addr":"172.21.15.173:6829/14215","state":["exists","up"]},{"osd":6,"uuid":"be530a7d-5f7c-4107-9171-15f4a6a4c468","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6810","nonce":14207},{"type":"v1","addr":"172.21.15.173:6811","nonce":14207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6812","nonce":14207},{"type":"v1","addr":"172.21.15.173:6813","nonce":14207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6820","nonce":14207},{"type":"v1","addr":"172.21.15.173:6823","nonce":14207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6814","nonce":14207},{"type":"v1","addr":"172.21.15.173:6816","nonce":14207}]},"public_addr":"172.21.15.173:6811/14207","cluster_addr":"172.21.15.173:6813/14207","heartbeat_back_addr":"172.21.15.173:6823/14207","heartbeat_front_addr":"172.21.15.173:6816/14207","state":["exists","up"]},{"osd":7,"uuid":"949fed52-858c-459b-8fcc-317b259ed305","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":127,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6815","nonce":14231},{"type":"v1","addr":"172.21.15.173:6817","nonce":14231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6819","nonce":14231},{"type":"v1","addr":"172.21.15.173:6822","nonce":14231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6830","nonce":14231},{"type":"v1","addr":"172.21.15.173:6832","nonce":14231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6825","nonce":14231},{"type":"v1","addr":"172.21.15.173:6828","nonce":14231}]},"public_addr":"172.21.15.173:6817/14231","cluster_addr":"172.21.15.173:6822/14231","heartbeat_back_addr":"172.21.15.173:6832/14231","heartbeat_front_addr":"172.21.15.173:6828/14231","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.290082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.346769+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.335366+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.261673+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.289965+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.300182+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.376325+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2020-11-20T12:15:49.431267+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2020-11-20T12:22:51.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.426+0000 7f3189d84700 1 -- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f315c0557c0 msgr2=0x7f315c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:51.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.426+0000 7f3189d84700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f315c0557c0 0x7f315c057c80 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f31840673c0 tx=0x7f3170006df0).stop 2020-11-20T12:22:51.436 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.426+0000 7f3189d84700 1 -- 172.21.15.116:0/19928327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184062880 msgr2=0x7f31840a7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:51.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.426+0000 7f3189d84700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184062880 0x7f31840a7320 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7f3178001e70 tx=0x7f3178007c50).stop 2020-11-20T12:22:51.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.426+0000 7f3189d84700 1 -- 172.21.15.116:0/19928327 shutdown_connections 2020-11-20T12:22:51.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.426+0000 7f3189d84700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f315c0557c0 0x7f315c057c80 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.437 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.426+0000 7f3189d84700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f3184153450 0x7f31841269c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.426+0000 7f3189d84700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f3184062880 0x7f31840a7320 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.426+0000 7f3189d84700 1 --2- 172.21.15.116:0/19928327 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f31840677a0 0x7f3184125f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.426+0000 7f3189d84700 1 -- 172.21.15.116:0/19928327 >> 172.21.15.116:0/19928327 conn(0x7f3184060c40 msgr2=0x7f318414f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:51.438 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.430+0000 7f3189d84700 1 -- 172.21.15.116:0/19928327 shutdown_connections 2020-11-20T12:22:51.439 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.430+0000 7f3189d84700 1 -- 172.21.15.116:0/19928327 wait complete. 2020-11-20T12:22:51.460 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2020-11-20T12:15:46.607758+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2020-11-20T12:15:55.044020+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2020-11-20T12:22:33.838237+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '128', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2020-11-20T12:22:34.312791+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '129', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2020-11-20T12:22:51.461 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2020-11-20T12:22:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.626+0000 7f775599c700 1 Processor -- start 2020-11-20T12:22:51.635 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.626+0000 7f775599c700 1 -- start start 2020-11-20T12:22:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f775599c700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77500a9430 0x7f7750065b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f775599c700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f77500ca810 0x7f77501557a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f775599c700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7750155eb0 con 0x7f7750065770 2020-11-20T12:22:51.636 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f775599c700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77500cacc0 con 0x7f77500a9430 2020-11-20T12:22:51.637 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f775599c700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7750155c90 con 0x7f77500ca810 2020-11-20T12:22:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f7747fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77500a9430 0x7f7750065b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.638 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f7747fff700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f77500a9430 0x7f7750065b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52818/0 (socket says 172.21.15.116:52818) 2020-11-20T12:22:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f7747fff700 1 -- 172.21.15.116:0/1741696943 learned_addr learned my addr 172.21.15.116:0/1741696943 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f7747fff700 1 -- 172.21.15.116:0/1741696943 >> v2:172.21.15.173:3300/0 conn(0x7f77500ca810 msgr2=0x7f77501557a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f774ffff700 1 --2- 172.21.15.116:0/1741696943 >> v2:172.21.15.173:3300/0 conn(0x7f77500ca810 0x7f77501557a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f7747fff700 1 --2- 172.21.15.116:0/1741696943 >> v2:172.21.15.173:3300/0 conn(0x7f77500ca810 0x7f77501557a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.639 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f7747fff700 1 -- 172.21.15.116:0/1741696943 >> v1:172.21.15.173:6789/0 conn(0x7f7750065770 legacy=0x7f77500a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f7747fff700 1 -- 172.21.15.116:0/1741696943 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7750156150 con 0x7f77500a9430 2020-11-20T12:22:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f774ffff700 1 --2- 172.21.15.116:0/1741696943 >> v2:172.21.15.173:3300/0 conn(0x7f77500ca810 0x7f77501557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f7747fff700 1 --2- 172.21.15.116:0/1741696943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77500a9430 0x7f7750065b50 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f7750066040 tx=0x7f7738006eb0).ready entity=mon.0 client_cookie=c66a84da5cc95107 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f774e7fc700 1 -- 172.21.15.116:0/1741696943 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738009850 con 0x7f77500a9430 2020-11-20T12:22:51.640 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f774e7fc700 1 -- 172.21.15.116:0/1741696943 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77380099b0 con 0x7f77500a9430 2020-11-20T12:22:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.630+0000 7f774e7fc700 1 -- 172.21.15.116:0/1741696943 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738009d30 con 0x7f77500a9430 2020-11-20T12:22:51.641 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.634+0000 7f775599c700 1 -- 172.21.15.116:0/1741696943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77500a9430 msgr2=0x7f7750065b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.634+0000 7f775599c700 1 --2- 172.21.15.116:0/1741696943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77500a9430 0x7f7750065b50 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 rx=0x7f7750066040 tx=0x7f7738006eb0).stop 2020-11-20T12:22:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.634+0000 7f775599c700 1 -- 172.21.15.116:0/1741696943 shutdown_connections 2020-11-20T12:22:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.634+0000 7f775599c700 1 --2- 172.21.15.116:0/1741696943 >> v2:172.21.15.173:3300/0 conn(0x7f77500ca810 0x7f77501557a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.642 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.634+0000 7f775599c700 1 --2- 172.21.15.116:0/1741696943 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f77500a9430 0x7f7750065b50 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.634+0000 7f775599c700 1 -- 172.21.15.116:0/1741696943 >> 172.21.15.116:0/1741696943 conn(0x7f77500647f0 msgr2=0x7f77500c9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.634+0000 7f775599c700 1 -- 172.21.15.116:0/1741696943 shutdown_connections 2020-11-20T12:22:51.643 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.634+0000 7f775599c700 1 -- 172.21.15.116:0/1741696943 wait complete. 2020-11-20T12:22:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.638+0000 7f775599c700 1 Processor -- start 2020-11-20T12:22:51.644 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.638+0000 7f775599c700 1 -- start start 2020-11-20T12:22:51.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.638+0000 7f775599c700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7750065770 0x7f7750165510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.646 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.638+0000 7f775599c700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77500a9430 0x7f775013f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.638+0000 7f775599c700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77500ca810 0x7f77501616a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.638+0000 7f775599c700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77500af530 con 0x7f7750065770 2020-11-20T12:22:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.638+0000 7f775599c700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7750162170 con 0x7f77500ca810 2020-11-20T12:22:51.647 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.638+0000 7f775599c700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7750162410 con 0x7f77500a9430 2020-11-20T12:22:51.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.638+0000 7f774effd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7750065770 0x7f7750165510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.638+0000 7f774effd700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7750065770 0x7f7750165510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52824/0 (socket says 172.21.15.116:52824) 2020-11-20T12:22:51.648 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.638+0000 7f774effd700 1 -- 172.21.15.116:0/532549089 learned_addr learned my addr 172.21.15.116:0/532549089 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:51.649 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f7747fff700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77500a9430 0x7f775013f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f774ffff700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77500ca810 0x7f77501616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f774effd700 1 -- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77500a9430 msgr2=0x7f775013f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:51.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f774effd700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77500a9430 0x7f775013f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.650 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f774effd700 1 -- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77500ca810 msgr2=0x7f77501616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:51.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f774effd700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77500ca810 0x7f77501616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f774effd700 1 -- 172.21.15.116:0/532549089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7750164dd0 con 0x7f7750065770 2020-11-20T12:22:51.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f774effd700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7750065770 0x7f7750165510 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7f774000c730 tx=0x7f7740007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:51.651 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f7746ffd700 1 -- 172.21.15.116:0/532549089 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774000e560 con 0x7f7750065770 2020-11-20T12:22:51.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f7746ffd700 1 -- 172.21.15.116:0/532549089 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f774000e6e0 con 0x7f7750065770 2020-11-20T12:22:51.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f775599c700 1 -- 172.21.15.116:0/532549089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7750165070 con 0x7f7750065770 2020-11-20T12:22:51.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f7746ffd700 1 -- 172.21.15.116:0/532549089 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774000eaa0 con 0x7f7750065770 2020-11-20T12:22:51.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f775599c700 1 -- 172.21.15.116:0/532549089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7750165a00 con 0x7f7750065770 2020-11-20T12:22:51.652 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f775599c700 1 -- 172.21.15.116:0/532549089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f775005d2a0 con 0x7f7750065770 2020-11-20T12:22:51.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.642+0000 7f7746ffd700 1 -- 172.21.15.116:0/532549089 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f7740019030 con 0x7f7750065770 2020-11-20T12:22:51.653 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.646+0000 7f7746ffd700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f772c0557c0 0x7f772c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:51.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.646+0000 7f7746ffd700 1 -- 172.21.15.116:0/532549089 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f774006dce0 con 0x7f7750065770 2020-11-20T12:22:51.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.646+0000 7f7747fff700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f772c0557c0 0x7f772c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:51.654 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.646+0000 7f7747fff700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f772c0557c0 0x7f772c057c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f77500ca4c0 tx=0x7f7738015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:51.658 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.650+0000 7f7746ffd700 1 -- 172.21.15.116:0/532549089 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f7740043b20 con 0x7f7750065770 2020-11-20T12:22:51.816 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.810+0000 7f775599c700 1 -- 172.21.15.116:0/532549089 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f77500603a0 con 0x7f7750065770 2020-11-20T12:22:51.817 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.810+0000 7f7746ffd700 1 -- 172.21.15.116:0/532549089 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v129) v1 ==== 110+0+10 (secure 0 0 0) 0x7f7740047140 con 0x7f7750065770 2020-11-20T12:22:51.817 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 1 2020-11-20T12:22:51.819 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 -- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f772c0557c0 msgr2=0x7f772c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:51.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f772c0557c0 0x7f772c057c80 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f77500ca4c0 tx=0x7f7738015000).stop 2020-11-20T12:22:51.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 -- 172.21.15.116:0/532549089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7750065770 msgr2=0x7f7750165510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:51.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7750065770 0x7f7750165510 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7f774000c730 tx=0x7f7740007c30).stop 2020-11-20T12:22:51.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 -- 172.21.15.116:0/532549089 shutdown_connections 2020-11-20T12:22:51.820 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f772c0557c0 0x7f772c057c80 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f77500ca810 0x7f77501616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f7750065770 0x7f7750165510 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 --2- 172.21.15.116:0/532549089 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f77500a9430 0x7f775013f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:51.821 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 -- 172.21.15.116:0/532549089 >> 172.21.15.116:0/532549089 conn(0x7f77500647f0 msgr2=0x7f775014f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:51.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 -- 172.21.15.116:0/532549089 shutdown_connections 2020-11-20T12:22:51.822 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:51.814+0000 7f775599c700 1 -- 172.21.15.116:0/532549089 wait complete. 2020-11-20T12:22:51.842 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2020-11-20T12:22:52.026 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.018+0000 7f5a9d3eb700 1 Processor -- start 2020-11-20T12:22:52.027 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.018+0000 7f5a9d3eb700 1 -- start start 2020-11-20T12:22:52.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.022+0000 7f5a9d3eb700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f5a980a6f40 0x7f5a98061410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.022+0000 7f5a9d3eb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a98061900 0x7f5a98061d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.030 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.022+0000 7f5a9d3eb700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a980a8ae0 con 0x7f5a9814e7b0 2020-11-20T12:22:52.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.022+0000 7f5a9d3eb700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a980604f0 con 0x7f5a980a6f40 2020-11-20T12:22:52.031 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.022+0000 7f5a9d3eb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98060790 con 0x7f5a98061900 2020-11-20T12:22:52.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.022+0000 7f5a96ffd700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f5a9814e7b0 0x7f5a981523b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34034/0 (socket says 172.21.15.116:34034) 2020-11-20T12:22:52.032 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.022+0000 7f5a96ffd700 1 -- 172.21.15.116:0/3873849484 learned_addr learned my addr 172.21.15.116:0/3873849484 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:52.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a97fff700 1 --2- 172.21.15.116:0/3873849484 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a98061900 0x7f5a98061d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a967fc700 1 --2- 172.21.15.116:0/3873849484 >> v2:172.21.15.173:3300/0 conn(0x7f5a980a6f40 0x7f5a98061410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a95ffb700 1 -- 172.21.15.116:0/3873849484 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243352581 0 0) 0x7f5a980a8ae0 con 0x7f5a9814e7b0 2020-11-20T12:22:52.033 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a95ffb700 1 -- 172.21.15.116:0/3873849484 --> v1:172.21.15.116:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a84003240 con 0x7f5a9814e7b0 2020-11-20T12:22:52.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a95ffb700 1 -- 172.21.15.116:0/3873849484 <== mon.0 v1:172.21.15.116:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 390+0+0 (unknown 2982937637 0 0) 0x7f5a84003240 con 0x7f5a9814e7b0 2020-11-20T12:22:52.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a97fff700 1 -- 172.21.15.116:0/3873849484 >> v2:172.21.15.173:3300/0 conn(0x7f5a980a6f40 msgr2=0x7f5a98061410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:52.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a97fff700 1 --2- 172.21.15.116:0/3873849484 >> v2:172.21.15.173:3300/0 conn(0x7f5a980a6f40 0x7f5a98061410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.034 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a97fff700 1 -- 172.21.15.116:0/3873849484 >> v1:172.21.15.116:6789/0 conn(0x7f5a9814e7b0 legacy=0x7f5a981523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a97fff700 1 -- 172.21.15.116:0/3873849484 --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98060a30 con 0x7f5a98061900 2020-11-20T12:22:52.035 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a967fc700 1 --2- 172.21.15.116:0/3873849484 >> v2:172.21.15.173:3300/0 conn(0x7f5a980a6f40 0x7f5a98061410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:52.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a97fff700 1 --2- 172.21.15.116:0/3873849484 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a98061900 0x7f5a98061d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f5a90009c90 tx=0x7f5a900091d0).ready entity=mon.2 client_cookie=5b6a9334f62410b0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:52.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a95ffb700 1 -- 172.21.15.116:0/3873849484 <== mon.2 v2:172.21.15.173:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a880014e0 con 0x7f5a98061900 2020-11-20T12:22:52.036 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a95ffb700 1 -- 172.21.15.116:0/3873849484 <== mon.2 v2:172.21.15.173:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a9000b6b0 con 0x7f5a98061900 2020-11-20T12:22:52.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a95ffb700 1 -- 172.21.15.116:0/3873849484 <== mon.2 v2:172.21.15.173:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a9000ba30 con 0x7f5a98061900 2020-11-20T12:22:52.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/3873849484 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a98061900 msgr2=0x7f5a98061d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a9d3eb700 1 --2- 172.21.15.116:0/3873849484 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a98061900 0x7f5a98061d90 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f5a90009c90 tx=0x7f5a900091d0).stop 2020-11-20T12:22:52.037 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/3873849484 shutdown_connections 2020-11-20T12:22:52.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a9d3eb700 1 --2- 172.21.15.116:0/3873849484 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a98061900 0x7f5a98061d90 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a9d3eb700 1 --2- 172.21.15.116:0/3873849484 >> v2:172.21.15.173:3300/0 conn(0x7f5a980a6f40 0x7f5a98061410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/3873849484 >> 172.21.15.116:0/3873849484 conn(0x7f5a980c6480 msgr2=0x7f5a980c2450 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:52.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/3873849484 shutdown_connections 2020-11-20T12:22:52.038 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.026+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/3873849484 wait complete. 2020-11-20T12:22:52.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a9d3eb700 1 Processor -- start 2020-11-20T12:22:52.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a9d3eb700 1 -- start start 2020-11-20T12:22:52.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a9d3eb700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a98061900 0x7f5a98162550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a9d3eb700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a980a6f40 0x7f5a980a4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.041 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a9d3eb700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a9814e7b0 0x7f5a98150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a9d3eb700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a980a4750 con 0x7f5a980a6f40 2020-11-20T12:22:52.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a9d3eb700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98162bf0 con 0x7f5a9814e7b0 2020-11-20T12:22:52.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a9d3eb700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a98162e90 con 0x7f5a98061900 2020-11-20T12:22:52.042 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a967fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a980a6f40 0x7f5a980a4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a967fc700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a980a6f40 0x7f5a980a4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52842/0 (socket says 172.21.15.116:52842) 2020-11-20T12:22:52.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a967fc700 1 -- 172.21.15.116:0/389177025 learned_addr learned my addr 172.21.15.116:0/389177025 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:52.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a96ffd700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a98061900 0x7f5a98162550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.030+0000 7f5a97fff700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a9814e7b0 0x7f5a98150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.043 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a967fc700 1 -- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a98061900 msgr2=0x7f5a98162550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a967fc700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a98061900 0x7f5a98162550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a967fc700 1 -- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a9814e7b0 msgr2=0x7f5a98150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a967fc700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a9814e7b0 0x7f5a98150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a967fc700 1 -- 172.21.15.116:0/389177025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98163130 con 0x7f5a980a6f40 2020-11-20T12:22:52.044 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a97fff700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a9814e7b0 0x7f5a98150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:52.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a967fc700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a980a6f40 0x7f5a980a4260 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f5a8c00ef50 tx=0x7f5a8c006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:52.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a7effd700 1 -- 172.21.15.116:0/389177025 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00cb30 con 0x7f5a980a6f40 2020-11-20T12:22:52.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a7effd700 1 -- 172.21.15.116:0/389177025 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c00cc90 con 0x7f5a980a6f40 2020-11-20T12:22:52.045 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/389177025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a981633d0 con 0x7f5a980a6f40 2020-11-20T12:22:52.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a7effd700 1 -- 172.21.15.116:0/389177025 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c012600 con 0x7f5a980a6f40 2020-11-20T12:22:52.046 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/389177025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a98151230 con 0x7f5a980a6f40 2020-11-20T12:22:52.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.034+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/389177025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a981514b0 con 0x7f5a980a6f40 2020-11-20T12:22:52.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.038+0000 7f5a7effd700 1 -- 172.21.15.116:0/389177025 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f5a8c012970 con 0x7f5a980a6f40 2020-11-20T12:22:52.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.038+0000 7f5a7effd700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5a740557c0 0x7f5a74057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.047 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.038+0000 7f5a7effd700 1 -- 172.21.15.116:0/389177025 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f5a8c06ece0 con 0x7f5a980a6f40 2020-11-20T12:22:52.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.038+0000 7f5a96ffd700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5a740557c0 0x7f5a74057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.048 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.038+0000 7f5a96ffd700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5a740557c0 0x7f5a74057c80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f5a880013f0 tx=0x7f5a880019a0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:52.050 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.042+0000 7f5a7effd700 1 -- 172.21.15.116:0/389177025 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f5a8c044b80 con 0x7f5a980a6f40 2020-11-20T12:22:52.206 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.198+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/389177025 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5a9805d2a0 con 0x7f5a980a6f40 2020-11-20T12:22:52.207 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a7effd700 1 -- 172.21.15.116:0/389177025 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v129) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5a8c0481a0 con 0x7f5a980a6f40 2020-11-20T12:22:52.207 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:52.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5a740557c0 msgr2=0x7f5a74057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5a740557c0 0x7f5a74057c80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f5a880013f0 tx=0x7f5a880019a0).stop 2020-11-20T12:22:52.209 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/389177025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a980a6f40 msgr2=0x7f5a980a4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a980a6f40 0x7f5a980a4260 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7f5a8c00ef50 tx=0x7f5a8c006df0).stop 2020-11-20T12:22:52.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/389177025 shutdown_connections 2020-11-20T12:22:52.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f5a740557c0 0x7f5a74057c80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.210 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f5a9814e7b0 0x7f5a98150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f5a98061900 0x7f5a98162550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 --2- 172.21.15.116:0/389177025 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f5a980a6f40 0x7f5a980a4260 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/389177025 >> 172.21.15.116:0/389177025 conn(0x7f5a980c6480 msgr2=0x7f5a980c4590 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:52.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/389177025 shutdown_connections 2020-11-20T12:22:52.211 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.202+0000 7f5a9d3eb700 1 -- 172.21.15.116:0/389177025 wait complete. 2020-11-20T12:22:52.232 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2020-11-20T12:22:52.392 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.386+0000 7ff7a6c4d700 1 Processor -- start 2020-11-20T12:22:52.393 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.386+0000 7ff7a6c4d700 1 -- start start 2020-11-20T12:22:52.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.386+0000 7ff7a6c4d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7a0149df0 0x7ff7a0153db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.386+0000 7ff7a6c4d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7a00a9750 0x7ff7a00c0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.395 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.386+0000 7ff7a6c4d700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7a00a9c20 con 0x7ff7a014a1d0 2020-11-20T12:22:52.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.386+0000 7ff7a6c4d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7a00a9ec0 con 0x7ff7a0149df0 2020-11-20T12:22:52.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.386+0000 7ff7a6c4d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a0153670 con 0x7ff7a00a9750 2020-11-20T12:22:52.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a49e9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7a0149df0 0x7ff7a0153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a49e9700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7ff7a0149df0 0x7ff7a0153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52846/0 (socket says 172.21.15.116:52846) 2020-11-20T12:22:52.396 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a49e9700 1 -- 172.21.15.116:0/3659211051 learned_addr learned my addr 172.21.15.116:0/3659211051 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:52.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a49e9700 1 -- 172.21.15.116:0/3659211051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7a00a9750 msgr2=0x7ff7a00c0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:52.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a49e9700 1 --2- 172.21.15.116:0/3659211051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7a00a9750 0x7ff7a00c0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:52.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a49e9700 1 -- 172.21.15.116:0/3659211051 >> v1:172.21.15.173:6789/0 conn(0x7ff7a014a1d0 legacy=0x7ff7a00be240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:52.398 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a49e9700 1 -- 172.21.15.116:0/3659211051 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a0153910 con 0x7ff7a0149df0 2020-11-20T12:22:52.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a49e9700 1 --2- 172.21.15.116:0/3659211051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7a0149df0 0x7ff7a0153db0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7ff794006fd0 tx=0x7ff794006eb0).ready entity=mon.0 client_cookie=1d3dc5b79008aed6 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:52.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff79f7fe700 1 -- 172.21.15.116:0/3659211051 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794009890 con 0x7ff7a0149df0 2020-11-20T12:22:52.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff79f7fe700 1 -- 172.21.15.116:0/3659211051 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7940099f0 con 0x7ff7a0149df0 2020-11-20T12:22:52.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff79f7fe700 1 -- 172.21.15.116:0/3659211051 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794009d70 con 0x7ff7a0149df0 2020-11-20T12:22:52.399 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3659211051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7a0149df0 msgr2=0x7ff7a0153db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a6c4d700 1 --2- 172.21.15.116:0/3659211051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7a0149df0 0x7ff7a0153db0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 rx=0x7ff794006fd0 tx=0x7ff794006eb0).stop 2020-11-20T12:22:52.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3659211051 shutdown_connections 2020-11-20T12:22:52.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a6c4d700 1 --2- 172.21.15.116:0/3659211051 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7a0149df0 0x7ff7a0153db0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a6c4d700 1 --2- 172.21.15.116:0/3659211051 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7a00a9750 0x7ff7a00c0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2020-11-20T12:22:52.400 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3659211051 >> 172.21.15.116:0/3659211051 conn(0x7ff7a005ba80 msgr2=0x7ff7a00c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:52.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3659211051 shutdown_connections 2020-11-20T12:22:52.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.390+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3659211051 wait complete. 2020-11-20T12:22:52.401 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.394+0000 7ff7a6c4d700 1 Processor -- start 2020-11-20T12:22:52.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.394+0000 7ff7a6c4d700 1 -- start start 2020-11-20T12:22:52.402 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.394+0000 7ff7a6c4d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7a00a9750 0x7ff7a015fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.394+0000 7ff7a6c4d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7a0149df0 0x7ff7a0147da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.394+0000 7ff7a6c4d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7a014a1d0 0x7ff7a014b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.394+0000 7ff7a6c4d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a014bbc0 con 0x7ff7a00a9750 2020-11-20T12:22:52.403 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.394+0000 7ff7a6c4d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a014be60 con 0x7ff7a0149df0 2020-11-20T12:22:52.404 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.394+0000 7ff7a6c4d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a014c100 con 0x7ff7a014a1d0 2020-11-20T12:22:52.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a49e9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7a00a9750 0x7ff7a015fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a49e9700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7a00a9750 0x7ff7a015fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52852/0 (socket says 172.21.15.116:52852) 2020-11-20T12:22:52.405 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a49e9700 1 -- 172.21.15.116:0/3053030012 learned_addr learned my addr 172.21.15.116:0/3053030012 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:52.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a51ea700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7a014a1d0 0x7ff7a014b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff79ffff700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7a0149df0 0x7ff7a0147da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a49e9700 1 -- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7a014a1d0 msgr2=0x7ff7a014b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.406 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a49e9700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7a014a1d0 0x7ff7a014b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a49e9700 1 -- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7a0149df0 msgr2=0x7ff7a0147da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a49e9700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7a0149df0 0x7ff7a0147da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a49e9700 1 -- 172.21.15.116:0/3053030012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a014c3a0 con 0x7ff7a00a9750 2020-11-20T12:22:52.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a49e9700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7a00a9750 0x7ff7a015fcd0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7ff79400c010 tx=0x7ff794015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:52.407 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff79cff9700 1 -- 172.21.15.116:0/3053030012 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff794011e50 con 0x7ff7a00a9750 2020-11-20T12:22:52.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff79cff9700 1 -- 172.21.15.116:0/3053030012 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff79401e460 con 0x7ff7a00a9750 2020-11-20T12:22:52.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff79cff9700 1 -- 172.21.15.116:0/3053030012 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff79401e7e0 con 0x7ff7a00a9750 2020-11-20T12:22:52.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3053030012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a014c640 con 0x7ff7a00a9750 2020-11-20T12:22:52.408 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3053030012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a014b820 con 0x7ff7a00a9750 2020-11-20T12:22:52.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3053030012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a005d2a0 con 0x7ff7a00a9750 2020-11-20T12:22:52.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.398+0000 7ff79cff9700 1 -- 172.21.15.116:0/3053030012 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7ff794025030 con 0x7ff7a00a9750 2020-11-20T12:22:52.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.402+0000 7ff79cff9700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7780557c0 0x7ff778057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.402+0000 7ff79cff9700 1 -- 172.21.15.116:0/3053030012 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7ff79406feb0 con 0x7ff7a00a9750 2020-11-20T12:22:52.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.402+0000 7ff79ffff700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7780557c0 0x7ff778057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.402+0000 7ff79ffff700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7780557c0 0x7ff778057c80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7ff7a00c3ae0 tx=0x7ff78c008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:52.414 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.406+0000 7ff79cff9700 1 -- 172.21.15.116:0/3053030012 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7ff794045e00 con 0x7ff7a00a9750 2020-11-20T12:22:52.580 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.574+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3053030012 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff7a0061380 con 0x7ff7a00a9750 2020-11-20T12:22:52.581 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.574+0000 7ff79cff9700 1 -- 172.21.15.116:0/3053030012 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v129) v1 ==== 104+0+10 (secure 0 0 0) 0x7ff794023070 con 0x7ff7a00a9750 2020-11-20T12:22:52.581 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:52.584 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.574+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7780557c0 msgr2=0x7ff778057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.574+0000 7ff7a6c4d700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7780557c0 0x7ff778057c80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7ff7a00c3ae0 tx=0x7ff78c008040).stop 2020-11-20T12:22:52.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.574+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3053030012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7a00a9750 msgr2=0x7ff7a015fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.585 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.574+0000 7ff7a6c4d700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7a00a9750 0x7ff7a015fcd0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7ff79400c010 tx=0x7ff794015000).stop 2020-11-20T12:22:52.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.574+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3053030012 shutdown_connections 2020-11-20T12:22:52.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.578+0000 7ff7a6c4d700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7ff7780557c0 0x7ff778057c80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.586 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.578+0000 7ff7a6c4d700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7ff7a014a1d0 0x7ff7a014b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.578+0000 7ff7a6c4d700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7ff7a00a9750 0x7ff7a015fcd0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.578+0000 7ff7a6c4d700 1 --2- 172.21.15.116:0/3053030012 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7ff7a0149df0 0x7ff7a0147da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.587 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.578+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3053030012 >> 172.21.15.116:0/3053030012 conn(0x7ff7a005ba80 msgr2=0x7ff7a00c0f40 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:52.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.578+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3053030012 shutdown_connections 2020-11-20T12:22:52.588 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.578+0000 7ff7a6c4d700 1 -- 172.21.15.116:0/3053030012 wait complete. 2020-11-20T12:22:52.606 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2020-11-20T12:22:52.790 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.782+0000 7f0f5a769700 1 Processor -- start 2020-11-20T12:22:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.782+0000 7f0f5a769700 1 -- start start 2020-11-20T12:22:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.782+0000 7f0f5a769700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f54148cf0 0x7f0f540a8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.791 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.782+0000 7f0f5a769700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0f540c5030 0x7f0f5414f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.782+0000 7f0f5a769700 1 -- --> v1:172.21.15.173:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f540a8fa0 con 0x7f0f540c4bb0 2020-11-20T12:22:52.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.782+0000 7f0f5a769700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f54152710 con 0x7f0f540c5030 2020-11-20T12:22:52.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.782+0000 7f0f5a769700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f541529b0 con 0x7f0f54148cf0 2020-11-20T12:22:52.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.782+0000 7f0f58d06700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0f540c5030 0x7f0f5414f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.792 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.782+0000 7f0f58d06700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f0f540c5030 0x7f0f5414f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52864/0 (socket says 172.21.15.116:52864) 2020-11-20T12:22:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.782+0000 7f0f58d06700 1 -- 172.21.15.116:0/2854225310 learned_addr learned my addr 172.21.15.116:0/2854225310 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:52.793 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.782+0000 7f0f53fff700 1 --2- 172.21.15.116:0/2854225310 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f54148cf0 0x7f0f540a8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.795 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f52ffd700 1 -- 172.21.15.116:0/2854225310 <== mon.1 v1:172.21.15.173:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429083394 0 0) 0x7f0f480010d0 con 0x7f0f540c4bb0 2020-11-20T12:22:52.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f52ffd700 1 -- 172.21.15.116:0/2854225310 --> v1:172.21.15.173:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f40003240 con 0x7f0f540c4bb0 2020-11-20T12:22:52.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f58d06700 1 -- 172.21.15.116:0/2854225310 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f54148cf0 msgr2=0x7f0f540a8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:52.798 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f58d06700 1 --2- 172.21.15.116:0/2854225310 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f54148cf0 0x7f0f540a8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f58d06700 1 -- 172.21.15.116:0/2854225310 >> v1:172.21.15.173:6789/0 conn(0x7f0f540c4bb0 legacy=0x7f0f540a66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.799 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f58d06700 1 -- 172.21.15.116:0/2854225310 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54152c50 con 0x7f0f540c5030 2020-11-20T12:22:52.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f53fff700 1 --2- 172.21.15.116:0/2854225310 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f54148cf0 0x7f0f540a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2020-11-20T12:22:52.801 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f58d06700 1 --2- 172.21.15.116:0/2854225310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f540c5030 0x7f0f5414f030 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f0f4c009d50 tx=0x7f0f4c009290).ready entity=mon.0 client_cookie=2dad515464b87214 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:52.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f52ffd700 1 -- 172.21.15.116:0/2854225310 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c00bb10 con 0x7f0f540c5030 2020-11-20T12:22:52.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f52ffd700 1 -- 172.21.15.116:0/2854225310 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c00bc70 con 0x7f0f540c5030 2020-11-20T12:22:52.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f52ffd700 1 -- 172.21.15.116:0/2854225310 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c011400 con 0x7f0f540c5030 2020-11-20T12:22:52.802 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f5a769700 1 -- 172.21.15.116:0/2854225310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f540c5030 msgr2=0x7f0f5414f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f5a769700 1 --2- 172.21.15.116:0/2854225310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f540c5030 0x7f0f5414f030 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f0f4c009d50 tx=0x7f0f4c009290).stop 2020-11-20T12:22:52.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f5a769700 1 -- 172.21.15.116:0/2854225310 shutdown_connections 2020-11-20T12:22:52.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f5a769700 1 --2- 172.21.15.116:0/2854225310 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f540c5030 0x7f0f5414f030 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.803 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f5a769700 1 --2- 172.21.15.116:0/2854225310 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f54148cf0 0x7f0f540a8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f5a769700 1 -- 172.21.15.116:0/2854225310 >> 172.21.15.116:0/2854225310 conn(0x7f0f5405ba80 msgr2=0x7f0f540c44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:52.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f5a769700 1 -- 172.21.15.116:0/2854225310 shutdown_connections 2020-11-20T12:22:52.804 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.786+0000 7f0f5a769700 1 -- 172.21.15.116:0/2854225310 wait complete. 2020-11-20T12:22:52.806 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.790+0000 7f0f5a769700 1 Processor -- start 2020-11-20T12:22:52.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.790+0000 7f0f5a769700 1 -- start start 2020-11-20T12:22:52.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f5a769700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f540c4bb0 0x7f0f54161c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f5a769700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f540c5030 0x7f0f54145e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.807 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f5a769700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f54148cf0 0x7f0f541490d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f5a769700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f54146380 con 0x7f0f540c5030 2020-11-20T12:22:52.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f5a769700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f54162230 con 0x7f0f54148cf0 2020-11-20T12:22:52.808 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f5a769700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f54149b60 con 0x7f0f540c4bb0 2020-11-20T12:22:52.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f537fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f540c5030 0x7f0f54145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f537fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f540c5030 0x7f0f54145e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52868/0 (socket says 172.21.15.116:52868) 2020-11-20T12:22:52.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f537fe700 1 -- 172.21.15.116:0/1531054121 learned_addr learned my addr 172.21.15.116:0/1531054121 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:52.809 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f53fff700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f540c4bb0 0x7f0f54161c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f58d06700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f54148cf0 0x7f0f541490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f537fe700 1 -- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f540c4bb0 msgr2=0x7f0f54161c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f537fe700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f540c4bb0 0x7f0f54161c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.810 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f537fe700 1 -- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f54148cf0 msgr2=0x7f0f541490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f537fe700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f54148cf0 0x7f0f541490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f537fe700 1 -- 172.21.15.116:0/1531054121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f54149e00 con 0x7f0f540c5030 2020-11-20T12:22:52.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f537fe700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f540c5030 0x7f0f54145e90 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f0f4800c6f0 tx=0x7f0f48002800).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:52.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f33fff700 1 -- 172.21.15.116:0/1531054121 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800e560 con 0x7f0f540c5030 2020-11-20T12:22:52.811 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f33fff700 1 -- 172.21.15.116:0/1531054121 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4800e6c0 con 0x7f0f540c5030 2020-11-20T12:22:52.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f33fff700 1 -- 172.21.15.116:0/1531054121 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800ea60 con 0x7f0f540c5030 2020-11-20T12:22:52.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f5a769700 1 -- 172.21.15.116:0/1531054121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f54144890 con 0x7f0f540c5030 2020-11-20T12:22:52.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.794+0000 7f0f5a769700 1 -- 172.21.15.116:0/1531054121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f54143e10 con 0x7f0f540c5030 2020-11-20T12:22:52.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.798+0000 7f0f33fff700 1 -- 172.21.15.116:0/1531054121 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f0f48019030 con 0x7f0f540c5030 2020-11-20T12:22:52.812 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.798+0000 7f0f5a769700 1 -- 172.21.15.116:0/1531054121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f540c1db0 con 0x7f0f540c5030 2020-11-20T12:22:52.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.798+0000 7f0f33fff700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0f2c0557c0 0x7f0f2c057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:52.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.798+0000 7f0f33fff700 1 -- 172.21.15.116:0/1531054121 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f0f4806daf0 con 0x7f0f540c5030 2020-11-20T12:22:52.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.798+0000 7f0f53fff700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0f2c0557c0 0x7f0f2c057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:52.813 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.798+0000 7f0f53fff700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0f2c0557c0 0x7f0f2c057c80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f0f44006fd0 tx=0x7f0f44008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:52.814 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.806+0000 7f0f33fff700 1 -- 172.21.15.116:0/1531054121 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f0f540c1db0 con 0x7f0f540c5030 2020-11-20T12:22:52.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.958+0000 7f0f5a769700 1 -- 172.21.15.116:0/1531054121 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0f540a8fa0 con 0x7f0f540c5030 2020-11-20T12:22:52.965 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.958+0000 7f0f33fff700 1 -- 172.21.15.116:0/1531054121 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v129) v1 ==== 100+0+10 (secure 0 0 0) 0x7f0f540a8fa0 con 0x7f0f540c5030 2020-11-20T12:22:52.966 INFO:teuthology.orchestra.run.smithi116.stdout:pg_num: 8 2020-11-20T12:22:52.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 -- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0f2c0557c0 msgr2=0x7f0f2c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.967 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0f2c0557c0 0x7f0f2c057c80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f0f44006fd0 tx=0x7f0f44008040).stop 2020-11-20T12:22:52.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 -- 172.21.15.116:0/1531054121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f540c5030 msgr2=0x7f0f54145e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:52.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f540c5030 0x7f0f54145e90 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 rx=0x7f0f4800c6f0 tx=0x7f0f48002800).stop 2020-11-20T12:22:52.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 -- 172.21.15.116:0/1531054121 shutdown_connections 2020-11-20T12:22:52.969 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f0f2c0557c0 0x7f0f2c057c80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f0f54148cf0 0x7f0f541490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f0f540c4bb0 0x7f0f54161c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 --2- 172.21.15.116:0/1531054121 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f0f540c5030 0x7f0f54145e90 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 -- 172.21.15.116:0/1531054121 >> 172.21.15.116:0/1531054121 conn(0x7f0f5405ba80 msgr2=0x7f0f540bf2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:52.970 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 -- 172.21.15.116:0/1531054121 shutdown_connections 2020-11-20T12:22:52.971 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:52.962+0000 7f0f5a769700 1 -- 172.21.15.116:0/1531054121 wait complete. 2020-11-20T12:22:52.994 INFO:tasks.cephfs.mount:Wait for MDS to reach steady state... 2020-11-20T12:22:52.994 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2020-11-20T12:22:53.176 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.170+0000 7f901228d700 1 Processor -- start 2020-11-20T12:22:53.177 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.170+0000 7f901228d700 1 -- start start 2020-11-20T12:22:53.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f901228d700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f900c13afa0 0x7f900c0aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:53.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f901228d700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7f900c13b380 0x7f900c0aaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:53.180 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f901228d700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c0ada00 con 0x7f900c0acd30 2020-11-20T12:22:53.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f901228d700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c0adca0 con 0x7f900c13afa0 2020-11-20T12:22:53.181 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f901228d700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c0adf40 con 0x7f900c13b380 2020-11-20T12:22:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f901128b700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7f900c13afa0 0x7f900c0aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f9011a8c700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7f900c0acd30 0x7f900c0ad1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34080/0 (socket says 172.21.15.116:34080) 2020-11-20T12:22:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f9011a8c700 1 -- 172.21.15.116:0/2483050564 learned_addr learned my addr 172.21.15.116:0/2483050564 (peer_addr_for_me v1:172.21.15.116:0/0) 2020-11-20T12:22:53.182 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f9010a8a700 1 --2- 172.21.15.116:0/2483050564 >> v2:172.21.15.173:3300/0 conn(0x7f900c13b380 0x7f900c0aaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:53.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f901128b700 1 -- 172.21.15.116:0/2483050564 >> v2:172.21.15.173:3300/0 conn(0x7f900c13b380 msgr2=0x7f900c0aaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:53.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f901128b700 1 --2- 172.21.15.116:0/2483050564 >> v2:172.21.15.173:3300/0 conn(0x7f900c13b380 0x7f900c0aaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:53.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f901128b700 1 -- 172.21.15.116:0/2483050564 >> v1:172.21.15.116:6789/0 conn(0x7f900c0acd30 legacy=0x7f900c0ad1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:53.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f8ffbfff700 1 -- 172.21.15.116:0/2483050564 <== mon.0 v1:172.21.15.116:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2616483482 0 0) 0x7f900c0ada00 con 0x7f900c0acd30 2020-11-20T12:22:53.183 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f901128b700 1 -- 172.21.15.116:0/2483050564 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c0ae1e0 con 0x7f900c13afa0 2020-11-20T12:22:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f901128b700 1 --2- 172.21.15.116:0/2483050564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c13afa0 0x7f900c0aa5b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f900c0625b0 tx=0x7f9008006df0).ready entity=mon.0 client_cookie=39b6dbac414583dd server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f8ffbfff700 1 -- 172.21.15.116:0/2483050564 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90080097b0 con 0x7f900c13afa0 2020-11-20T12:22:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.174+0000 7f8ffbfff700 1 -- 172.21.15.116:0/2483050564 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9008009910 con 0x7f900c13afa0 2020-11-20T12:22:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.178+0000 7f8ffbfff700 1 -- 172.21.15.116:0/2483050564 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008009c90 con 0x7f900c13afa0 2020-11-20T12:22:53.185 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.178+0000 7f901228d700 1 -- 172.21.15.116:0/2483050564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c13afa0 msgr2=0x7f900c0aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.178+0000 7f901228d700 1 --2- 172.21.15.116:0/2483050564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c13afa0 0x7f900c0aa5b0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f900c0625b0 tx=0x7f9008006df0).stop 2020-11-20T12:22:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.178+0000 7f901228d700 1 -- 172.21.15.116:0/2483050564 shutdown_connections 2020-11-20T12:22:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.178+0000 7f901228d700 1 --2- 172.21.15.116:0/2483050564 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c13afa0 0x7f900c0aa5b0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:53.186 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.178+0000 7f901228d700 1 --2- 172.21.15.116:0/2483050564 >> v2:172.21.15.173:3300/0 conn(0x7f900c13b380 0x7f900c0aaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:53.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.178+0000 7f901228d700 1 -- 172.21.15.116:0/2483050564 >> 172.21.15.116:0/2483050564 conn(0x7f900c0678a0 msgr2=0x7f900c067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:53.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.178+0000 7f901228d700 1 -- 172.21.15.116:0/2483050564 shutdown_connections 2020-11-20T12:22:53.187 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.178+0000 7f901228d700 1 -- 172.21.15.116:0/2483050564 wait complete. 2020-11-20T12:22:53.188 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.178+0000 7f901228d700 1 Processor -- start 2020-11-20T12:22:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.178+0000 7f901228d700 1 -- start start 2020-11-20T12:22:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f901228d700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f900c0acd30 0x7f900c13a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f901228d700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c13afa0 0x7f900c161960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f901228d700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f900c13b380 0x7f900c164670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:53.189 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f901228d700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1650d0 con 0x7f900c13afa0 2020-11-20T12:22:53.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f901228d700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c165370 con 0x7f900c13b380 2020-11-20T12:22:53.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f901228d700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c161220 con 0x7f900c0acd30 2020-11-20T12:22:53.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f9010a8a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c13afa0 0x7f900c161960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:53.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f9010a8a700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c13afa0 0x7f900c161960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52880/0 (socket says 172.21.15.116:52880) 2020-11-20T12:22:53.190 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f9010a8a700 1 -- 172.21.15.116:0/1893491644 learned_addr learned my addr 172.21.15.116:0/1893491644 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:53.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f901128b700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f900c0acd30 0x7f900c13a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:53.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f9011a8c700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f900c13b380 0x7f900c164670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:53.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f9010a8a700 1 -- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f900c0acd30 msgr2=0x7f900c13a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:53.192 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f9010a8a700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f900c0acd30 0x7f900c13a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:53.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f9010a8a700 1 -- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f900c13b380 msgr2=0x7f900c164670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:53.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f9010a8a700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f900c13b380 0x7f900c164670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:53.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f9010a8a700 1 -- 172.21.15.116:0/1893491644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c1614c0 con 0x7f900c13afa0 2020-11-20T12:22:53.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f9010a8a700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c13afa0 0x7f900c161960 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f8ffc00eba0 tx=0x7f8ffc006df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:53.193 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f8ff97fa700 1 -- 172.21.15.116:0/1893491644 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00b9a0 con 0x7f900c13afa0 2020-11-20T12:22:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f8ff97fa700 1 -- 172.21.15.116:0/1893491644 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc00bb00 con 0x7f900c13afa0 2020-11-20T12:22:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f901228d700 1 -- 172.21.15.116:0/1893491644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c13a8c0 con 0x7f900c13afa0 2020-11-20T12:22:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f8ff97fa700 1 -- 172.21.15.116:0/1893491644 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00be80 con 0x7f900c13afa0 2020-11-20T12:22:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.182+0000 7f901228d700 1 -- 172.21.15.116:0/1893491644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c161e50 con 0x7f900c13afa0 2020-11-20T12:22:53.194 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.186+0000 7f901228d700 1 -- 172.21.15.116:0/1893491644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c165530 con 0x7f900c13afa0 2020-11-20T12:22:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.186+0000 7f8ff97fa700 1 -- 172.21.15.116:0/1893491644 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7f8ffc012600 con 0x7f900c13afa0 2020-11-20T12:22:53.195 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.186+0000 7f8ff97fa700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fe40557c0 0x7f8fe4057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:53.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.186+0000 7f8ff97fa700 1 -- 172.21.15.116:0/1893491644 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f8ffc06ea10 con 0x7f900c13afa0 2020-11-20T12:22:53.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.186+0000 7f901128b700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fe40557c0 0x7f8fe4057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:53.196 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.190+0000 7f901128b700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fe40557c0 0x7f8fe4057c80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f9008006e30 tx=0x7f9008015000).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:53.200 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.194+0000 7f8ff97fa700 1 -- 172.21.15.116:0/1893491644 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7f8ffc044850 con 0x7f900c13afa0 2020-11-20T12:22:53.409 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.402+0000 7f901228d700 1 -- 172.21.15.116:0/1893491644 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f900c05cf30 con 0x7f900c13afa0 2020-11-20T12:22:53.410 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.402+0000 7f8ff97fa700 1 -- 172.21.15.116:0/1893491644 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 75 v75) v1 ==== 94+0+2853 (secure 0 0 0) 0x7f8ffc047e70 con 0x7f900c13afa0 2020-11-20T12:22:53.410 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:53.411 INFO:teuthology.orchestra.run.smithi116.stdout:{"epoch":75,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":false,"ever_enabled_multiple":false},"standbys":[{"gid":7190,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.173:6835/363707569","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6834","nonce":363707569},{"type":"v1","addr":"172.21.15.173:6835","nonce":363707569}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":71},{"gid":7202,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.116:6834/1854325940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6832","nonce":1854325940},{"type":"v1","addr":"172.21.15.116:6834","nonce":1854325940}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72},{"gid":8437,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.173:6837/3920088127","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.173:6836","nonce":3920088127},{"type":"v1","addr":"172.21.15.173:6837","nonce":3920088127}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":72}],"filesystems":[{"mdsmap":{"epoch":75,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2020-11-20T12:22:35.338520+0000","modified":"2020-11-20T12:22:37.346491+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8589},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8589":{"gid":8589,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.116:6835/1101967997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.116:6833","nonce":1101967997},{"type":"v1","addr":"172.21.15.116:6835","nonce":1101967997}]},"join_fscid":-1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2020-11-20T12:22:53.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 -- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fe40557c0 msgr2=0x7f8fe4057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:53.412 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fe40557c0 0x7f8fe4057c80 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f9008006e30 tx=0x7f9008015000).stop 2020-11-20T12:22:53.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 -- 172.21.15.116:0/1893491644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c13afa0 msgr2=0x7f900c161960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:53.413 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c13afa0 0x7f900c161960 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 rx=0x7f8ffc00eba0 tx=0x7f8ffc006df0).stop 2020-11-20T12:22:53.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 -- 172.21.15.116:0/1893491644 shutdown_connections 2020-11-20T12:22:53.415 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7f8fe40557c0 0x7f8fe4057c80 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:53.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7f900c13b380 0x7f900c164670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:53.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7f900c0acd30 0x7f900c13a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:53.416 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 --2- 172.21.15.116:0/1893491644 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7f900c13afa0 0x7f900c161960 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:53.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 -- 172.21.15.116:0/1893491644 >> 172.21.15.116:0/1893491644 conn(0x7f900c0678a0 msgr2=0x7f900c0c8130 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:53.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 -- 172.21.15.116:0/1893491644 shutdown_connections 2020-11-20T12:22:53.417 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:53.406+0000 7f901228d700 1 -- 172.21.15.116:0/1893491644 wait complete. 2020-11-20T12:22:53.418 INFO:teuthology.orchestra.run.smithi116.stderr:dumped fsmap epoch 75 2020-11-20T12:22:53.840 INFO:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 75, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2020-11-20T12:22:35.338520+0000', 'modified': '2020-11-20T12:22:37.346491+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8589}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8589': {'gid': 8589, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.116:6835/1101967997', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.116:6833', 'nonce': 1101967997}, {'type': 'v1', 'addr': '172.21.15.116:6835', 'nonce': 1101967997}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138297136906239, 'flags': 0}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2020-11-20T12:22:53.840 INFO:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2020-11-20T12:22:53.841 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2020-11-20T12:22:54.055 INFO:teuthology.orchestra.run.smithi116.stdout:{"cluster_fsid":"5ca3454c-e55c-402a-9a47-9fb04981c9e5","whoami":0,"id":8589,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":18.692102634000001,"mdsmap_epoch":75,"osdmap_epoch":129,"osdmap_epoch_barrier":129,"uptime":24.238625535000001} 2020-11-20T12:22:54.056 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "5ca3454c-e55c-402a-9a47-9fb04981c9e5", "fs_name": "cephfs", "id": 8589, "mdsmap_epoch": 75, "osdmap_epoch": 129, "osdmap_epoch_barrier": 129, "rank_uptime": 18.692102634, "state": "up:active", "uptime": 24.238625535, "want_state": "up:active", "whoami": 0 } 2020-11-20T12:22:54.056 INFO:tasks.cephfs.mount:Ready to start KernelMount... 2020-11-20T12:22:54.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:54.057 INFO:teuthology.orchestra.run.smithi173:> mkdir -p /home/ubuntu/cephtest/mnt.3 2020-11-20T12:22:54.082 INFO:teuthology.orchestra.run:Running command with timeout 1800 2020-11-20T12:22:54.082 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:54.205 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:54.206 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:22:54.227 INFO:teuthology.orchestra.run.smithi173.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,norbytes 2020-11-20T12:22:54.228 INFO:teuthology.orchestra.run.smithi173.stdout:mount.ceph: options "norequire_active_mds,name=3,norbytes" will pass to kernel. 2020-11-20T12:22:54.235 INFO:tasks.cephfs.kernel_mount:mount command passed 2020-11-20T12:22:54.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:54.236 INFO:teuthology.orchestra.run.smithi173:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2020-11-20T12:22:54.266 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2020-11-20T12:22:54.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2020-11-20T12:22:54.267 INFO:teuthology.orchestra.run.smithi173:> sudo umount /home/ubuntu/cephtest/mnt.1 2020-11-20T12:22:54.388 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi173.front.sepia.ceph.com 2020-11-20T12:22:54.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2020-11-20T12:22:54.389 INFO:teuthology.orchestra.run.smithi173:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2020-11-20T12:22:54.405 INFO:teuthology.orchestra.run.smithi116:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get-or-create client.manila mds 'allow *' osd 'allow rw' mon 'allow *' 2020-11-20T12:22:54.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.590+0000 7fe0057e7700 1 Processor -- start 2020-11-20T12:22:54.598 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.590+0000 7fe0057e7700 1 -- start start 2020-11-20T12:22:54.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.594+0000 7fe0057e7700 1 --2- >> v2:172.21.15.173:3300/0 conn(0x7fe000151630 0x7fe000062170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.601 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.594+0000 7fe0057e7700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0000ae250 0x7fe000151a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.602 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.594+0000 7fe0057e7700 1 -- --> v1:172.21.15.116:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0000abbf0 con 0x7fe000152640 2020-11-20T12:22:54.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.594+0000 7fe0057e7700 1 -- --> v2:172.21.15.116:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0000aac10 con 0x7fe0000ae250 2020-11-20T12:22:54.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.594+0000 7fe0057e7700 1 -- --> v2:172.21.15.173:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0000aaeb0 con 0x7fe000151630 2020-11-20T12:22:54.603 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.594+0000 7fdffe7fc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0000ae250 0x7fe000151a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:54.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.594+0000 7fdffe7fc700 1 --2- >> v2:172.21.15.116:3300/0 conn(0x7fe0000ae250 0x7fe000151a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52888/0 (socket says 172.21.15.116:52888) 2020-11-20T12:22:54.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.594+0000 7fdfff7fe700 1 --1- >> v1:172.21.15.116:6789/0 conn(0x7fe000152640 0x7fe000061490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.116:6789/0 says I am v1:172.21.15.116:34094/0 (socket says 172.21.15.116:34094) 2020-11-20T12:22:54.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.594+0000 7fdffe7fc700 1 -- 172.21.15.116:0/2245019817 learned_addr learned my addr 172.21.15.116:0/2245019817 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:54.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.594+0000 7fdffeffd700 1 --2- 172.21.15.116:0/2245019817 >> v2:172.21.15.173:3300/0 conn(0x7fe000151630 0x7fe000062170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:54.604 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.594+0000 7fdffe7fc700 1 -- 172.21.15.116:0/2245019817 >> v2:172.21.15.173:3300/0 conn(0x7fe000151630 msgr2=0x7fe000062170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2020-11-20T12:22:54.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fdffe7fc700 1 --2- 172.21.15.116:0/2245019817 >> v2:172.21.15.173:3300/0 conn(0x7fe000151630 0x7fe000062170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fdffe7fc700 1 -- 172.21.15.116:0/2245019817 >> v1:172.21.15.116:6789/0 conn(0x7fe000152640 legacy=0x7fe000061490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:54.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fdffe7fc700 1 -- 172.21.15.116:0/2245019817 --> v2:172.21.15.116:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0000ab150 con 0x7fe0000ae250 2020-11-20T12:22:54.605 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fdffeffd700 1 --2- 172.21.15.116:0/2245019817 >> v2:172.21.15.173:3300/0 conn(0x7fe000151630 0x7fe000062170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2020-11-20T12:22:54.606 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fdffe7fc700 1 --2- 172.21.15.116:0/2245019817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0000ae250 0x7fe000151a10 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7fe00005c5c0 tx=0x7fdff4006e50).ready entity=mon.0 client_cookie=b245174162e2017c server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:54.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fdffdffb700 1 -- 172.21.15.116:0/2245019817 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4006030 con 0x7fe0000ae250 2020-11-20T12:22:54.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fdffdffb700 1 -- 172.21.15.116:0/2245019817 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff4006190 con 0x7fe0000ae250 2020-11-20T12:22:54.607 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fdffdffb700 1 -- 172.21.15.116:0/2245019817 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff4006510 con 0x7fe0000ae250 2020-11-20T12:22:54.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fe0057e7700 1 -- 172.21.15.116:0/2245019817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0000ae250 msgr2=0x7fe000151a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:54.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fe0057e7700 1 --2- 172.21.15.116:0/2245019817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0000ae250 0x7fe000151a10 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7fe00005c5c0 tx=0x7fdff4006e50).stop 2020-11-20T12:22:54.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fe0057e7700 1 -- 172.21.15.116:0/2245019817 shutdown_connections 2020-11-20T12:22:54.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fe0057e7700 1 --2- 172.21.15.116:0/2245019817 >> v2:172.21.15.173:3300/0 conn(0x7fe000151630 0x7fe000062170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.608 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fe0057e7700 1 --2- 172.21.15.116:0/2245019817 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe0000ae250 0x7fe000151a10 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fe0057e7700 1 -- 172.21.15.116:0/2245019817 >> 172.21.15.116:0/2245019817 conn(0x7fe000068370 msgr2=0x7fe0000c8760 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:54.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fe0057e7700 1 -- 172.21.15.116:0/2245019817 shutdown_connections 2020-11-20T12:22:54.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.598+0000 7fe0057e7700 1 -- 172.21.15.116:0/2245019817 wait complete. 2020-11-20T12:22:54.609 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fe0057e7700 1 Processor -- start 2020-11-20T12:22:54.610 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fe0057e7700 1 -- start start 2020-11-20T12:22:54.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fe0057e7700 1 --2- >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0000ae250 0x7fe00014f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fe0057e7700 1 --2- >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000151630 0x7fe0000bf730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fe0057e7700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000152640 0x7fe0000c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fe0057e7700 1 -- --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000161c60 con 0x7fe000152640 2020-11-20T12:22:54.612 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fe0057e7700 1 -- --> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe00014e980 con 0x7fe0000ae250 2020-11-20T12:22:54.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fe0057e7700 1 -- --> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe00014ec20 con 0x7fe000151630 2020-11-20T12:22:54.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdfff7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000152640 0x7fe0000c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:54.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdfff7fe700 1 --2- >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000152640 0x7fe0000c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.116:3300/0 says I am v2:172.21.15.116:52898/0 (socket says 172.21.15.116:52898) 2020-11-20T12:22:54.613 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdfff7fe700 1 -- 172.21.15.116:0/3966103786 learned_addr learned my addr 172.21.15.116:0/3966103786 (peer_addr_for_me v2:172.21.15.116:0/0) 2020-11-20T12:22:54.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdffe7fc700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000151630 0x7fe0000bf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:54.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdffeffd700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0000ae250 0x7fe00014f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:54.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdfff7fe700 1 -- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000151630 msgr2=0x7fe0000bf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:54.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdfff7fe700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000151630 0x7fe0000bf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.614 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdfff7fe700 1 -- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0000ae250 msgr2=0x7fe00014f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:54.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdfff7fe700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0000ae250 0x7fe00014f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdfff7fe700 1 -- 172.21.15.116:0/3966103786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00014eec0 con 0x7fe000152640 2020-11-20T12:22:54.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdffe7fc700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000151630 0x7fe0000bf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:54.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdffeffd700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0000ae250 0x7fe00014f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2020-11-20T12:22:54.615 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.602+0000 7fdfff7fe700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000152640 0x7fe0000c4c40 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7fdff800ca60 tx=0x7fdff8007f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.606+0000 7fdfe6ffd700 1 -- 172.21.15.116:0/3966103786 <== mon.0 v2:172.21.15.116:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8005ac0 con 0x7fe000152640 2020-11-20T12:22:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.606+0000 7fdfe6ffd700 1 -- 172.21.15.116:0/3966103786 <== mon.0 v2:172.21.15.116:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff8005c20 con 0x7fe000152640 2020-11-20T12:22:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.606+0000 7fe0057e7700 1 -- 172.21.15.116:0/3966103786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe00014f850 con 0x7fe000152640 2020-11-20T12:22:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.606+0000 7fdfe6ffd700 1 -- 172.21.15.116:0/3966103786 <== mon.0 v2:172.21.15.116:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8007110 con 0x7fe000152640 2020-11-20T12:22:54.616 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.606+0000 7fe0057e7700 1 -- 172.21.15.116:0/3966103786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0000c5130 con 0x7fe000152640 2020-11-20T12:22:54.617 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.606+0000 7fe0057e7700 1 -- 172.21.15.116:0/3966103786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe00005cf30 con 0x7fe000152640 2020-11-20T12:22:54.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.606+0000 7fdfe6ffd700 1 -- 172.21.15.116:0/3966103786 <== mon.0 v2:172.21.15.116:3300/0 4 ==== mgrmap(e 5) v1 ==== 66655+0+0 (secure 0 0 0) 0x7fdff8010030 con 0x7fe000152640 2020-11-20T12:22:54.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.610+0000 7fdfe6ffd700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdfdc0557c0 0x7fdfdc057c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2020-11-20T12:22:54.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.610+0000 7fdfe6ffd700 1 -- 172.21.15.116:0/3966103786 <== mon.0 v2:172.21.15.116:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6073+0+0 (secure 0 0 0) 0x7fdff806e550 con 0x7fe000152640 2020-11-20T12:22:54.618 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.610+0000 7fdffeffd700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdfdc0557c0 0x7fdfdc057c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2020-11-20T12:22:54.619 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.610+0000 7fdffeffd700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdfdc0557c0 0x7fdfdc057c80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fdff0006fd0 tx=0x7fdff0008040).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2020-11-20T12:22:54.620 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.614+0000 7fdfe6ffd700 1 -- 172.21.15.116:0/3966103786 <== mon.0 v2:172.21.15.116:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+143281 (secure 0 0 0) 0x7fdff80443f0 con 0x7fe000152640 2020-11-20T12:22:54.837 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.830+0000 7fe0057e7700 1 -- 172.21.15.116:0/3966103786 --> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]} v 0) v1 -- 0x7fe0000abbf0 con 0x7fe000152640 2020-11-20T12:22:54.847 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.838+0000 7fdfe6ffd700 1 -- 172.21.15.116:0/3966103786 <== mon.0 v2:172.21.15.116:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.manila", "caps": ["mds", "allow *", "osd", "allow rw", "mon", "allow *"]}]=0 v37) v1 ==== 158+0+64 (secure 0 0 0) 0x7fe0000abbf0 con 0x7fe000152640 2020-11-20T12:22:54.847 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:22:54.847 INFO:teuthology.orchestra.run.smithi116.stdout: key = AQAetbdfbGjZMRAAykPGllFl22pSstR/9bbcZQ== 2020-11-20T12:22:54.849 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 -- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdfdc0557c0 msgr2=0x7fdfdc057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:54.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdfdc0557c0 0x7fdfdc057c80 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7fdff0006fd0 tx=0x7fdff0008040).stop 2020-11-20T12:22:54.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 -- 172.21.15.116:0/3966103786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000152640 msgr2=0x7fe0000c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2020-11-20T12:22:54.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000152640 0x7fe0000c4c40 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7fdff800ca60 tx=0x7fdff8007f40).stop 2020-11-20T12:22:54.850 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 -- 172.21.15.116:0/3966103786 shutdown_connections 2020-11-20T12:22:54.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:6800/13860,v1:172.21.15.173:6801/13860] conn(0x7fdfdc0557c0 0x7fdfdc057c80 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.116:3300/0,v1:172.21.15.116:6789/0] conn(0x7fe000152640 0x7fe0000c4c40 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:3300/0,v1:172.21.15.173:6789/0] conn(0x7fe0000ae250 0x7fe00014f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 --2- 172.21.15.116:0/3966103786 >> [v2:172.21.15.173:3301/0,v1:172.21.15.173:6790/0] conn(0x7fe000151630 0x7fe0000bf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2020-11-20T12:22:54.851 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 -- 172.21.15.116:0/3966103786 >> 172.21.15.116:0/3966103786 conn(0x7fe000068370 msgr2=0x7fe000154ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2020-11-20T12:22:54.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 -- 172.21.15.116:0/3966103786 shutdown_connections 2020-11-20T12:22:54.852 INFO:teuthology.orchestra.run.smithi116.stderr:2020-11-20T12:22:54.842+0000 7fe0057e7700 1 -- 172.21.15.116:0/3966103786 wait complete. 2020-11-20T12:22:54.876 INFO:teuthology.orchestra.run.smithi173:> set -ex 2020-11-20T12:22:54.876 INFO:teuthology.orchestra.run.smithi173:> sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:22:54.891 INFO:teuthology.orchestra.run.smithi173.stderr:+ sudo dd of=/etc/ceph/ceph.client.manila.keyring 2020-11-20T12:22:54.904 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records in 2020-11-20T12:22:54.904 INFO:teuthology.orchestra.run.smithi173.stderr:0+1 records out 2020-11-20T12:22:54.904 INFO:teuthology.orchestra.run.smithi173.stderr:64 bytes copied, 0.000180176 s, 355 kB/s 2020-11-20T12:22:54.908 INFO:teuthology.orchestra.run.smithi116:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:22:54.928 INFO:teuthology.orchestra.run.smithi173:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2020-11-20T12:22:54.988 INFO:teuthology.orchestra.run.smithi116.stdout:[global] 2020-11-20T12:22:54.989 INFO:teuthology.orchestra.run.smithi116.stdout: chdir = "" 2020-11-20T12:22:54.989 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:22:54.989 INFO:teuthology.orchestra.run.smithi116.stdout: auth supported = cephx 2020-11-20T12:22:54.989 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.990 INFO:teuthology.orchestra.run.smithi116.stdout: filestore xattr use omap = true 2020-11-20T12:22:54.990 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.990 INFO:teuthology.orchestra.run.smithi116.stdout: mon clock drift allowed = 1.000 2020-11-20T12:22:54.990 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.990 INFO:teuthology.orchestra.run.smithi116.stdout: osd crush chooseleaf type = 0 2020-11-20T12:22:54.991 INFO:teuthology.orchestra.run.smithi116.stdout: auth debug = true 2020-11-20T12:22:54.991 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.991 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on old message = true 2020-11-20T12:22:54.991 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on bug = true 2020-11-20T12:22:54.991 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.992 INFO:teuthology.orchestra.run.smithi116.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:22:54.992 INFO:teuthology.orchestra.run.smithi116.stdout: mon pg warn max object skew = 0 2020-11-20T12:22:54.992 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.992 INFO:teuthology.orchestra.run.smithi116.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:22:54.993 INFO:teuthology.orchestra.run.smithi116.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:22:54.993 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.993 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default size = 2 2020-11-20T12:22:54.993 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.993 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow primary affinity = true 2020-11-20T12:22:54.994 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd allow pg remap = true 2020-11-20T12:22:54.994 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:22:54.994 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:22:54.994 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on no sortbitwise = false 2020-11-20T12:22:54.994 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:22:54.995 INFO:teuthology.orchestra.run.smithi116.stdout: mon warn on too few osds = false 2020-11-20T12:22:54.995 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:22:54.995 INFO:teuthology.orchestra.run.smithi116.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:22:54.995 INFO:teuthology.orchestra.run.smithi116.stdout: mon_allow_pool_size_one = true 2020-11-20T12:22:54.995 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.996 INFO:teuthology.orchestra.run.smithi116.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:22:54.996 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.996 INFO:teuthology.orchestra.run.smithi116.stdout: osd default data pool replay window = 5 2020-11-20T12:22:54.996 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.996 INFO:teuthology.orchestra.run.smithi116.stdout: mon allow pool delete = true 2020-11-20T12:22:54.997 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.997 INFO:teuthology.orchestra.run.smithi116.stdout: mon cluster log file level = debug 2020-11-20T12:22:54.997 INFO:teuthology.orchestra.run.smithi116.stdout: debug asserts on shutdown = true 2020-11-20T12:22:54.997 INFO:teuthology.orchestra.run.smithi116.stdout: mon health detail to clog = false 2020-11-20T12:22:54.997 INFO:teuthology.orchestra.run.smithi116.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:22:54.998 INFO:teuthology.orchestra.run.smithi116.stdout: ms die on skipped message = False 2020-11-20T12:22:54.998 INFO:teuthology.orchestra.run.smithi116.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:22:54.998 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.998 INFO:teuthology.orchestra.run.smithi116.stdout:[osd] 2020-11-20T12:22:54.998 INFO:teuthology.orchestra.run.smithi116.stdout: osd journal size = 100 2020-11-20T12:22:54.999 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:54.999 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub load threshold = 5.0 2020-11-20T12:22:54.999 INFO:teuthology.orchestra.run.smithi116.stdout: osd scrub max interval = 600 2020-11-20T12:22:54.999 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:55.000 INFO:teuthology.orchestra.run.smithi116.stdout: osd recover clone overlap = true 2020-11-20T12:22:55.000 INFO:teuthology.orchestra.run.smithi116.stdout: osd recovery max chunk = 1048576 2020-11-20T12:22:55.000 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:55.001 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug shutdown = true 2020-11-20T12:22:55.001 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug op order = true 2020-11-20T12:22:55.001 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug verify stray on activate = true 2020-11-20T12:22:55.002 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:55.002 INFO:teuthology.orchestra.run.smithi116.stdout: osd open classes on start = true 2020-11-20T12:22:55.002 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug pg log writeout = true 2020-11-20T12:22:55.002 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:22:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: osd map max advance = 10 2020-11-20T12:22:55.003 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:55.004 INFO:teuthology.orchestra.run.smithi116.stdout: journal zero on create = true 2020-11-20T12:22:55.004 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:55.004 INFO:teuthology.orchestra.run.smithi116.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:22:55.004 INFO:teuthology.orchestra.run.smithi116.stdout: filestore apply finisher threads = 3 2020-11-20T12:22:55.004 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: bdev debug aio = true 2020-11-20T12:22:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: osd debug misdirected ops = true 2020-11-20T12:22:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: bdev async discard = True 2020-11-20T12:22:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: bdev enable discard = True 2020-11-20T12:22:55.005 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore allocator = bitmap 2020-11-20T12:22:55.006 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore block size = 96636764160 2020-11-20T12:22:55.006 INFO:teuthology.orchestra.run.smithi116.stdout: bluestore fsck on mount = True 2020-11-20T12:22:55.006 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluefs = 20 2020-11-20T12:22:55.006 INFO:teuthology.orchestra.run.smithi116.stdout: debug bluestore = 20 2020-11-20T12:22:55.006 INFO:teuthology.orchestra.run.smithi116.stdout: debug filestore = 20 2020-11-20T12:22:55.007 INFO:teuthology.orchestra.run.smithi116.stdout: debug journal = 20 2020-11-20T12:22:55.007 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 20 2020-11-20T12:22:55.007 INFO:teuthology.orchestra.run.smithi116.stdout: debug osd = 25 2020-11-20T12:22:55.007 INFO:teuthology.orchestra.run.smithi116.stdout: debug rocksdb = 10 2020-11-20T12:22:55.007 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:22:55.008 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd full ratio = 0.9 2020-11-20T12:22:55.008 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:22:55.008 INFO:teuthology.orchestra.run.smithi116.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:22:55.008 INFO:teuthology.orchestra.run.smithi116.stdout: osd objectstore = bluestore 2020-11-20T12:22:55.008 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:22:55.009 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:55.009 INFO:teuthology.orchestra.run.smithi116.stdout:[mgr] 2020-11-20T12:22:55.009 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:55.009 INFO:teuthology.orchestra.run.smithi116.stdout: debug mgr = 20 2020-11-20T12:22:55.009 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:22:55.010 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:22:55.010 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:55.010 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:55.010 INFO:teuthology.orchestra.run.smithi116.stdout: mgr/telemetry/nag = false 2020-11-20T12:22:55.010 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:55.011 INFO:teuthology.orchestra.run.smithi116.stdout:[mon] 2020-11-20T12:22:55.011 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:55.011 INFO:teuthology.orchestra.run.smithi116.stdout: debug mon = 20 2020-11-20T12:22:55.011 INFO:teuthology.orchestra.run.smithi116.stdout: debug paxos = 20 2020-11-20T12:22:55.011 INFO:teuthology.orchestra.run.smithi116.stdout: debug auth = 20 2020-11-20T12:22:55.012 INFO:teuthology.orchestra.run.smithi116.stdout: mon data avail warn = 5 2020-11-20T12:22:55.012 INFO:teuthology.orchestra.run.smithi116.stdout: mon mgr mkfs grace = 240 2020-11-20T12:22:55.012 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:55.012 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd reporter subtree level = osd 2020-11-20T12:22:55.012 INFO:teuthology.orchestra.run.smithi116.stdout: mon osd prime pg temp = true 2020-11-20T12:22:55.013 INFO:teuthology.orchestra.run.smithi116.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:55.013 INFO:teuthology.orchestra.run.smithi116.stdout: mon op complaint time = 120 2020-11-20T12:22:55.013 INFO:teuthology.orchestra.run.smithi116.stdout: 2020-11-20T12:22:55.013 INFO:teuthology.orchestra.run.smithi116.stdout:[client] 2020-11-20T12:22:55.014 INFO:teuthology.orchestra.run.smithi116.stdout: rgw cache enabled = true 2020-11-20T12:22:55.014 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable ops log = true 2020-11-20T12:22:55.014 INFO:teuthology.orchestra.run.smithi116.stdout: rgw enable usage log = true 2020-11-20T12:22:55.014 INFO:teuthology.orchestra.run.smithi116.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:22:55.014 INFO:teuthology.orchestra.run.smithi116.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:22:55.015 INFO:teuthology.orchestra.run.smithi116.stdout: client mount timeout = 600 2020-11-20T12:22:55.015 INFO:teuthology.orchestra.run.smithi116.stdout: debug client = 20 2020-11-20T12:22:55.015 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:55.015 INFO:teuthology.orchestra.run.smithi116.stdout: pid file = "" 2020-11-20T12:22:55.015 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:22:55.016 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:22:55.016 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.a] 2020-11-20T12:22:55.016 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.b] 2020-11-20T12:22:55.016 INFO:teuthology.orchestra.run.smithi116.stdout:[mon.c] 2020-11-20T12:22:55.016 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.a] 2020-11-20T12:22:55.017 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.c] 2020-11-20T12:22:55.017 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.b] 2020-11-20T12:22:55.017 INFO:teuthology.orchestra.run.smithi116.stdout:[mds.d] 2020-11-20T12:22:55.017 INFO:teuthology.orchestra.run.smithi116.stdout:[mds] 2020-11-20T12:22:55.017 INFO:teuthology.orchestra.run.smithi116.stdout: debug mds = 20 2020-11-20T12:22:55.018 INFO:teuthology.orchestra.run.smithi116.stdout: debug ms = 1 2020-11-20T12:22:55.018 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal frag = True 2020-11-20T12:22:55.018 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal fragment size max = 10000 2020-11-20T12:22:55.018 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal merge size = 5 2020-11-20T12:22:55.018 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split bits = 3 2020-11-20T12:22:55.019 INFO:teuthology.orchestra.run.smithi116.stdout: mds bal split size = 100 2020-11-20T12:22:55.019 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug frag = True 2020-11-20T12:22:55.019 INFO:teuthology.orchestra.run.smithi116.stdout: mds debug scatterstat = True 2020-11-20T12:22:55.019 INFO:teuthology.orchestra.run.smithi116.stdout: mds op complaint time = 180 2020-11-20T12:22:55.019 INFO:teuthology.orchestra.run.smithi116.stdout: mds verify scatter = True 2020-11-20T12:22:55.020 INFO:teuthology.orchestra.run.smithi116.stdout: osd op complaint time = 180 2020-11-20T12:22:55.020 INFO:teuthology.orchestra.run.smithi116.stdout: rados mon op timeout = 15m 2020-11-20T12:22:55.020 INFO:teuthology.orchestra.run.smithi116.stdout: rados osd op timeout = 15m 2020-11-20T12:22:55.020 INFO:teuthology.orchestra.run.smithi116.stdout:[client.manila] 2020-11-20T12:22:55.020 INFO:teuthology.orchestra.run.smithi116.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:22:55.021 INFO:teuthology.orchestra.run.smithi116.stdout:[client.21501] 2020-11-20T12:22:55.044 INFO:teuthology.orchestra.run.smithi173.stdout:[global] 2020-11-20T12:22:55.044 INFO:teuthology.orchestra.run.smithi173.stdout: chdir = "" 2020-11-20T12:22:55.045 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2020-11-20T12:22:55.045 INFO:teuthology.orchestra.run.smithi173.stdout: auth supported = cephx 2020-11-20T12:22:55.045 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.045 INFO:teuthology.orchestra.run.smithi173.stdout: filestore xattr use omap = true 2020-11-20T12:22:55.046 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.046 INFO:teuthology.orchestra.run.smithi173.stdout: mon clock drift allowed = 1.000 2020-11-20T12:22:55.046 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.046 INFO:teuthology.orchestra.run.smithi173.stdout: osd crush chooseleaf type = 0 2020-11-20T12:22:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: auth debug = true 2020-11-20T12:22:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on old message = true 2020-11-20T12:22:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on bug = true 2020-11-20T12:22:55.047 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: mon max pg per osd = 10000 # >= luminous 2020-11-20T12:22:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: mon pg warn max object skew = 0 2020-11-20T12:22:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: # disable pg_autoscaler by default for new pools 2020-11-20T12:22:55.048 INFO:teuthology.orchestra.run.smithi173.stdout: osd_pool_default_pg_autoscale_mode = off 2020-11-20T12:22:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default size = 2 2020-11-20T12:22:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow primary affinity = true 2020-11-20T12:22:55.049 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd allow pg remap = true 2020-11-20T12:22:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on legacy crush tunables = false 2020-11-20T12:22:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on crush straw calc version zero = false 2020-11-20T12:22:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on no sortbitwise = false 2020-11-20T12:22:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on osd down out interval zero = false 2020-11-20T12:22:55.050 INFO:teuthology.orchestra.run.smithi173.stdout: mon warn on too few osds = false 2020-11-20T12:22:55.051 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2020-11-20T12:22:55.051 INFO:teuthology.orchestra.run.smithi173.stdout: mon_warn_on_pool_no_redundancy = false 2020-11-20T12:22:55.051 INFO:teuthology.orchestra.run.smithi173.stdout: mon_allow_pool_size_one = true 2020-11-20T12:22:55.051 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.051 INFO:teuthology.orchestra.run.smithi173.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2020-11-20T12:22:55.052 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.052 INFO:teuthology.orchestra.run.smithi173.stdout: osd default data pool replay window = 5 2020-11-20T12:22:55.052 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.052 INFO:teuthology.orchestra.run.smithi173.stdout: mon allow pool delete = true 2020-11-20T12:22:55.052 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.053 INFO:teuthology.orchestra.run.smithi173.stdout: mon cluster log file level = debug 2020-11-20T12:22:55.053 INFO:teuthology.orchestra.run.smithi173.stdout: debug asserts on shutdown = true 2020-11-20T12:22:55.053 INFO:teuthology.orchestra.run.smithi173.stdout: mon health detail to clog = false 2020-11-20T12:22:55.053 INFO:teuthology.orchestra.run.smithi173.stdout: mon host = "172.21.15.116,172.21.15.173,[v2:172.21.15.173:3301,v1:172.21.15.173:6790]" 2020-11-20T12:22:55.054 INFO:teuthology.orchestra.run.smithi173.stdout: ms die on skipped message = False 2020-11-20T12:22:55.054 INFO:teuthology.orchestra.run.smithi173.stdout: fsid = 5ca3454c-e55c-402a-9a47-9fb04981c9e5 2020-11-20T12:22:55.054 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.054 INFO:teuthology.orchestra.run.smithi173.stdout:[osd] 2020-11-20T12:22:55.054 INFO:teuthology.orchestra.run.smithi173.stdout: osd journal size = 100 2020-11-20T12:22:55.055 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.055 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub load threshold = 5.0 2020-11-20T12:22:55.055 INFO:teuthology.orchestra.run.smithi173.stdout: osd scrub max interval = 600 2020-11-20T12:22:55.055 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.055 INFO:teuthology.orchestra.run.smithi173.stdout: osd recover clone overlap = true 2020-11-20T12:22:55.056 INFO:teuthology.orchestra.run.smithi173.stdout: osd recovery max chunk = 1048576 2020-11-20T12:22:55.056 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.056 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug shutdown = true 2020-11-20T12:22:55.056 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug op order = true 2020-11-20T12:22:55.056 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug verify stray on activate = true 2020-11-20T12:22:55.057 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.057 INFO:teuthology.orchestra.run.smithi173.stdout: osd open classes on start = true 2020-11-20T12:22:55.057 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug pg log writeout = true 2020-11-20T12:22:55.057 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.057 INFO:teuthology.orchestra.run.smithi173.stdout: osd deep scrub update digest min age = 30 2020-11-20T12:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout: osd map max advance = 10 2020-11-20T12:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout: journal zero on create = true 2020-11-20T12:22:55.058 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: filestore ondisk finisher threads = 3 2020-11-20T12:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: filestore apply finisher threads = 3 2020-11-20T12:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: bdev debug aio = true 2020-11-20T12:22:55.059 INFO:teuthology.orchestra.run.smithi173.stdout: osd debug misdirected ops = true 2020-11-20T12:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: bdev async discard = True 2020-11-20T12:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: bdev enable discard = True 2020-11-20T12:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore allocator = bitmap 2020-11-20T12:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore block size = 96636764160 2020-11-20T12:22:55.060 INFO:teuthology.orchestra.run.smithi173.stdout: bluestore fsck on mount = True 2020-11-20T12:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluefs = 20 2020-11-20T12:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: debug bluestore = 20 2020-11-20T12:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: debug filestore = 20 2020-11-20T12:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: debug journal = 20 2020-11-20T12:22:55.061 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 20 2020-11-20T12:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: debug osd = 25 2020-11-20T12:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: debug rocksdb = 10 2020-11-20T12:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd backfillfull_ratio = 0.85 2020-11-20T12:22:55.062 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd full ratio = 0.9 2020-11-20T12:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd nearfull ratio = 0.8 2020-11-20T12:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: osd failsafe full ratio = 0.95 2020-11-20T12:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: osd objectstore = bluestore 2020-11-20T12:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:22:55.063 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout:[mgr] 2020-11-20T12:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: debug mgr = 20 2020-11-20T12:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:22:55.064 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:22:55.065 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:55.065 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:55.065 INFO:teuthology.orchestra.run.smithi173.stdout: mgr/telemetry/nag = false 2020-11-20T12:22:55.065 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.065 INFO:teuthology.orchestra.run.smithi173.stdout:[mon] 2020-11-20T12:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: debug mon = 20 2020-11-20T12:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: debug paxos = 20 2020-11-20T12:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: debug auth = 20 2020-11-20T12:22:55.066 INFO:teuthology.orchestra.run.smithi173.stdout: mon data avail warn = 5 2020-11-20T12:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: mon mgr mkfs grace = 240 2020-11-20T12:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min pgs per osd = 4 2020-11-20T12:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd reporter subtree level = osd 2020-11-20T12:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: mon osd prime pg temp = true 2020-11-20T12:22:55.067 INFO:teuthology.orchestra.run.smithi173.stdout: mon reweight min bytes per osd = 10 2020-11-20T12:22:55.068 INFO:teuthology.orchestra.run.smithi173.stdout: mon op complaint time = 120 2020-11-20T12:22:55.068 INFO:teuthology.orchestra.run.smithi173.stdout: 2020-11-20T12:22:55.068 INFO:teuthology.orchestra.run.smithi173.stdout:[client] 2020-11-20T12:22:55.068 INFO:teuthology.orchestra.run.smithi173.stdout: rgw cache enabled = true 2020-11-20T12:22:55.068 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable ops log = true 2020-11-20T12:22:55.069 INFO:teuthology.orchestra.run.smithi173.stdout: rgw enable usage log = true 2020-11-20T12:22:55.069 INFO:teuthology.orchestra.run.smithi173.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2020-11-20T12:22:55.069 INFO:teuthology.orchestra.run.smithi173.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2020-11-20T12:22:55.069 INFO:teuthology.orchestra.run.smithi173.stdout: client mount timeout = 600 2020-11-20T12:22:55.069 INFO:teuthology.orchestra.run.smithi173.stdout: debug client = 20 2020-11-20T12:22:55.070 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:55.070 INFO:teuthology.orchestra.run.smithi173.stdout: pid file = "" 2020-11-20T12:22:55.070 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:22:55.070 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:22:55.070 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.a] 2020-11-20T12:22:55.071 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.b] 2020-11-20T12:22:55.071 INFO:teuthology.orchestra.run.smithi173.stdout:[mon.c] 2020-11-20T12:22:55.071 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.a] 2020-11-20T12:22:55.071 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.c] 2020-11-20T12:22:55.071 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.b] 2020-11-20T12:22:55.072 INFO:teuthology.orchestra.run.smithi173.stdout:[mds.d] 2020-11-20T12:22:55.072 INFO:teuthology.orchestra.run.smithi173.stdout:[mds] 2020-11-20T12:22:55.072 INFO:teuthology.orchestra.run.smithi173.stdout: debug mds = 20 2020-11-20T12:22:55.072 INFO:teuthology.orchestra.run.smithi173.stdout: debug ms = 1 2020-11-20T12:22:55.072 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal frag = True 2020-11-20T12:22:55.073 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal fragment size max = 10000 2020-11-20T12:22:55.073 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal merge size = 5 2020-11-20T12:22:55.073 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split bits = 3 2020-11-20T12:22:55.073 INFO:teuthology.orchestra.run.smithi173.stdout: mds bal split size = 100 2020-11-20T12:22:55.073 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug frag = True 2020-11-20T12:22:55.074 INFO:teuthology.orchestra.run.smithi173.stdout: mds debug scatterstat = True 2020-11-20T12:22:55.074 INFO:teuthology.orchestra.run.smithi173.stdout: mds op complaint time = 180 2020-11-20T12:22:55.074 INFO:teuthology.orchestra.run.smithi173.stdout: mds verify scatter = True 2020-11-20T12:22:55.075 INFO:teuthology.orchestra.run.smithi173.stdout: osd op complaint time = 180 2020-11-20T12:22:55.075 INFO:teuthology.orchestra.run.smithi173.stdout: rados mon op timeout = 15m 2020-11-20T12:22:55.075 INFO:teuthology.orchestra.run.smithi173.stdout: rados osd op timeout = 15m 2020-11-20T12:22:55.075 INFO:teuthology.orchestra.run.smithi173.stdout:[client.manila] 2020-11-20T12:22:55.075 INFO:teuthology.orchestra.run.smithi173.stdout: keyring = /etc/ceph/ceph.client.manila.keyring 2020-11-20T12:22:55.076 INFO:teuthology.orchestra.run.smithi173.stdout:[client.21501] 2020-11-20T12:22:55.078 INFO:teuthology.orchestra.run.smithi173:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2020-11-20T12:22:55.078 INFO:teuthology.orchestra.run.smithi173:> from __future__ import print_function 2020-11-20T12:22:55.078 INFO:teuthology.orchestra.run.smithi173:> from ceph_volume_client import CephFSVolumeClient, VolumePath 2020-11-20T12:22:55.079 INFO:teuthology.orchestra.run.smithi173:> from sys import version_info as sys_version_info 2020-11-20T12:22:55.079 INFO:teuthology.orchestra.run.smithi173:> from rados import OSError as rados_OSError 2020-11-20T12:22:55.080 INFO:teuthology.orchestra.run.smithi173:> import logging 2020-11-20T12:22:55.080 INFO:teuthology.orchestra.run.smithi173:> log = logging.getLogger("ceph_volume_client") 2020-11-20T12:22:55.080 INFO:teuthology.orchestra.run.smithi173:> log.addHandler(logging.StreamHandler()) 2020-11-20T12:22:55.081 INFO:teuthology.orchestra.run.smithi173:> log.setLevel(logging.DEBUG) 2020-11-20T12:22:55.081 INFO:teuthology.orchestra.run.smithi173:> vc = CephFSVolumeClient("manila", "/etc/ceph/ceph.conf", "ceph", None, None) 2020-11-20T12:22:55.081 INFO:teuthology.orchestra.run.smithi173:> vc.connect() 2020-11-20T12:22:55.082 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:22:55.082 INFO:teuthology.orchestra.run.smithi173:> vp = VolumePath("grpid", "volid") 2020-11-20T12:22:55.082 INFO:teuthology.orchestra.run.smithi173:> vc.create_volume(vp, 10) 2020-11-20T12:22:55.083 INFO:teuthology.orchestra.run.smithi173:> vc.create_volume(vp, 10) 2020-11-20T12:22:55.083 INFO:teuthology.orchestra.run.smithi173:> vc.authorize(vp, "guest") 2020-11-20T12:22:55.084 INFO:teuthology.orchestra.run.smithi173:> vc.authorize(vp, "guest") 2020-11-20T12:22:55.084 INFO:teuthology.orchestra.run.smithi173:> vc.deauthorize(vp, "guest") 2020-11-20T12:22:55.085 INFO:teuthology.orchestra.run.smithi173:> vc.deauthorize(vp, "guest") 2020-11-20T12:22:55.085 INFO:teuthology.orchestra.run.smithi173:> vc.delete_volume(vp) 2020-11-20T12:22:55.086 INFO:teuthology.orchestra.run.smithi173:> vc.delete_volume(vp) 2020-11-20T12:22:55.086 INFO:teuthology.orchestra.run.smithi173:> vc.purge_volume(vp) 2020-11-20T12:22:55.086 INFO:teuthology.orchestra.run.smithi173:> vc.purge_volume(vp) 2020-11-20T12:22:55.087 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:22:55.087 INFO:teuthology.orchestra.run.smithi173:> vc.create_volume(vp, 10, data_isolated=True) 2020-11-20T12:22:55.087 INFO:teuthology.orchestra.run.smithi173:> vc.create_volume(vp, 10, data_isolated=True) 2020-11-20T12:22:55.088 INFO:teuthology.orchestra.run.smithi173:> vc.authorize(vp, "guest") 2020-11-20T12:22:55.088 INFO:teuthology.orchestra.run.smithi173:> vc.authorize(vp, "guest") 2020-11-20T12:22:55.088 INFO:teuthology.orchestra.run.smithi173:> vc.deauthorize(vp, "guest") 2020-11-20T12:22:55.089 INFO:teuthology.orchestra.run.smithi173:> vc.deauthorize(vp, "guest") 2020-11-20T12:22:55.089 INFO:teuthology.orchestra.run.smithi173:> vc.evict("guest") 2020-11-20T12:22:55.090 INFO:teuthology.orchestra.run.smithi173:> vc.evict("guest") 2020-11-20T12:22:55.090 INFO:teuthology.orchestra.run.smithi173:> vc.delete_volume(vp, data_isolated=True) 2020-11-20T12:22:55.090 INFO:teuthology.orchestra.run.smithi173:> vc.delete_volume(vp, data_isolated=True) 2020-11-20T12:22:55.091 INFO:teuthology.orchestra.run.smithi173:> vc.purge_volume(vp, data_isolated=True) 2020-11-20T12:22:55.091 INFO:teuthology.orchestra.run.smithi173:> vc.purge_volume(vp, data_isolated=True) 2020-11-20T12:22:55.091 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:22:55.092 INFO:teuthology.orchestra.run.smithi173:> vc.create_volume(vp, 10, namespace_isolated=False) 2020-11-20T12:22:55.092 INFO:teuthology.orchestra.run.smithi173:> vc.create_volume(vp, 10, namespace_isolated=False) 2020-11-20T12:22:55.092 INFO:teuthology.orchestra.run.smithi173:> vc.authorize(vp, "guest") 2020-11-20T12:22:55.093 INFO:teuthology.orchestra.run.smithi173:> vc.authorize(vp, "guest") 2020-11-20T12:22:55.093 INFO:teuthology.orchestra.run.smithi173:> vc.deauthorize(vp, "guest") 2020-11-20T12:22:55.094 INFO:teuthology.orchestra.run.smithi173:> vc.deauthorize(vp, "guest") 2020-11-20T12:22:55.095 INFO:teuthology.orchestra.run.smithi173:> vc.evict("guest") 2020-11-20T12:22:55.095 INFO:teuthology.orchestra.run.smithi173:> vc.evict("guest") 2020-11-20T12:22:55.095 INFO:teuthology.orchestra.run.smithi173:> vc.delete_volume(vp) 2020-11-20T12:22:55.096 INFO:teuthology.orchestra.run.smithi173:> vc.delete_volume(vp) 2020-11-20T12:22:55.096 INFO:teuthology.orchestra.run.smithi173:> vc.purge_volume(vp) 2020-11-20T12:22:55.096 INFO:teuthology.orchestra.run.smithi173:> vc.purge_volume(vp) 2020-11-20T12:22:55.097 INFO:teuthology.orchestra.run.smithi173:> 2020-11-20T12:22:55.097 INFO:teuthology.orchestra.run.smithi173:> vc.disconnect() 2020-11-20T12:22:55.098 INFO:teuthology.orchestra.run.smithi173:> ' 2020-11-20T12:22:55.299 INFO:teuthology.orchestra.run.smithi173.stderr:Configuring to RADOS with config /etc/ceph/ceph.conf... 2020-11-20T12:22:55.305 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to RADOS... 2020-11-20T12:22:55.315 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to RADOS complete 2020-11-20T12:22:55.315 INFO:teuthology.orchestra.run.smithi173.stderr:Connecting to cephfs... 2020-11-20T12:22:55.316 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS initializing... 2020-11-20T12:22:55.323 INFO:teuthology.orchestra.run.smithi173.stderr:CephFS mounting... 2020-11-20T12:22:55.334 INFO:teuthology.orchestra.run.smithi173.stderr:Connection to cephfs complete 2020-11-20T12:22:55.334 INFO:teuthology.orchestra.run.smithi173.stderr:Recovering from partial auth updates (if any)... 2020-11-20T12:22:55.335 INFO:teuthology.orchestra.run.smithi173.stderr:Nothing to recover. No auth meta files. 2020-11-20T12:22:55.335 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: /volumes/grpid/volid 2020-11-20T12:22:55.355 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: grpid/volid, using rados namespace fsvolumens_volid to isolate data. 2020-11-20T12:22:55.364 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: /volumes/grpid/volid 2020-11-20T12:22:55.365 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: grpid/volid, using rados namespace fsvolumens_volid to isolate data. 2020-11-20T12:22:55.384 INFO:teuthology.orchestra.run.smithi173.stderr:Creating meta for ID guest with tenant None 2020-11-20T12:22:55.385 INFO:teuthology.orchestra.run.smithi173.stderr:Authorize: no existing meta 2020-11-20T12:22:55.403 INFO:teuthology.orchestra.run.smithi173.stderr:Authorizing Ceph id 'guest' for path '/volumes/grpid/volid' 2020-11-20T12:22:55.451 INFO:teuthology.orchestra.run.smithi173.stderr:Authorize: existing tenant None 2020-11-20T12:22:55.489 INFO:teuthology.orchestra.run.smithi173.stderr:Authorizing Ceph id 'guest' for path '/volumes/grpid/volid' 2020-11-20T12:22:55.614 INFO:teuthology.orchestra.run.smithi173.stderr:deauthorized called for already-removed authID 'guest' for volume ID 'volid' 2020-11-20T12:22:55.617 INFO:teuthology.orchestra.run.smithi173.stderr:delete_volume: /volumes/grpid/volid 2020-11-20T12:22:55.627 INFO:teuthology.orchestra.run.smithi173.stderr:delete_volume: /volumes/grpid/volid 2020-11-20T12:22:55.627 INFO:teuthology.orchestra.run.smithi173.stderr:Trying to delete volume '/volumes/grpid/volid' but it's already gone 2020-11-20T12:22:55.627 INFO:teuthology.orchestra.run.smithi173.stderr:rmtree /volumes/_deleting/volid 2020-11-20T12:22:55.645 INFO:teuthology.orchestra.run.smithi173.stderr:Trying to purge volume '/volumes/_deleting/volid' but it's already been purged 2020-11-20T12:22:55.647 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: /volumes/grpid/volid 2020-11-20T12:22:55.655 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: grpid/volid, create pool fsvolume_volid as data_isolated =True. 2020-11-20T12:23:01.501 INFO:teuthology.orchestra.run.smithi116:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:23:01.518 INFO:teuthology.orchestra.run.smithi173:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2020-11-20T12:23:01.636 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: grpid/volid, using rados namespace fsvolumens_volid to isolate data. 2020-11-20T12:23:01.645 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: /volumes/grpid/volid 2020-11-20T12:23:01.647 INFO:teuthology.orchestra.run.smithi173.stderr:create_volume: grpid/volid, create pool fsvolume_volid as data_isolated =True. 2020-11-20T12:23:01.651 INFO:teuthology.orchestra.run.smithi173.stderr:Pool fsvolume_volid already exists